Performance Considerations

This page is a comprehensive guide to getting the most performance out of PlanetPress Connect as well as a rough guideline to indicate when it's best to upgrade.

Performance Analysis Details

In order to get the most out of PlanetPress Connect, it is important to determine how best to maximize performance. The following guidelines will be helpful in extracting the best performance from PlanetPress Connect before looking into hardware upgrades or extra PlanetPress Connect performance packs.

  • Job Sizes and Speed: In terms of pure output speed, it's important to first determine what job size is expected, and adjust Scheduling Preferences accordingly. The basic rules are:
    • If processing a small number of very large records (when each individual record is composed of a large number of pages), more instances with an equal amount of speed units is better. For hardware, RAM and Hard Drive speeds are most important, since the smallest divisible part (the record) cannot be split on multiple machines or even cores.
    • If creating a very large number of small records (hundreds of thousands of 2-3 page individual records, for instance), a smaller number of instances with a large number of speed packs would be better. As for hardware, then the number of cores becomes critical, whereas RAM and hard drive are secondary. Performance Packs, as well as the MySQL instance being separate, would be helpful if your most powerful machine starts struggling.
    • Mix and match. For example, one instance prioritized for large jobs and the rest for smaller, quicker jobs. Or the contrary. Or, whatever you want, really.
  • RAM Configuration: By default, each instance of the Merge Engine and Weaver Engine is set to use 640MB of RAM. This means that regardless of speed units, if not enough memory is available, output speed might not be as expected. Assuming that the machine itself is not running any other software, the rule of thumb is the following: The total number of used memory in the machine should be pretty much the maximum available (around 95%).

    For each engine, it's necessary to modify the .ini file that controls its JAVA arguments. Edit as follows:

    • For the Merge Engine: see C:\Program Files\Objectif Lune\OL Connect\MergeEngine\Mergeengine.ini
    • For the Weaver Engine: see C:\Program Files\Objectif Lune\OL Connect\weaverengine\Weaverengine.ini
    • The parameters are -Xms640m for the minimum RAM size, -Xmx640m for the maximum RAM size. Explaining Java arguments is beyond the scope of this document. Please read references here, here and here for more details (fair warning: these can get pretty technical!).
  • Template and data mapping optimization: Some functionality offered by theDataMapper and Designer modules are very useful, and sometimes downright awesome, but can cause the generation of records and of contents items to slow down due to their nature. Here are some of them:
    • Preprocessor and Postprocessor scripts: manipulating data using a script may cause delays before and after the data mapping action has actually taken place, especially file conversion and data enrichment from other sources.
    • Loading external and network resources: In Designer, using images, javascript or css resources located on a slow network or on a slow internet connection will obviously lead to a loss of speed. While we do our best for caching, a document with 100,000 records which queries a page that takes 1 second to return a different image each time will, naturally, slow output generation down by up to 27 hours.
    • External JavaScript Libraries: While loading a single JavaScript library from the web is generally very fast (and only done once for the record set), actually running a script on each generated page can take some time. Because yes, JavaScript will run for each record, and often take the same time for each record.
    • Inefficient Selectors: Using very precise ID selectors in script wizards can be much faster than using a text selector, especially on very large documents. (more details on this in another upcoming page).
    • Complex Scripts: Custom scripts with large, complex or non-optimized loops can lead to slowing down content creation. While it is sometimes difficult to troubleshoot, there are many resources online to help learn about JavaScript performance and coding mistakes. Here, here, and here are a few. Note that most resources on the web are about JavaScript in the browser, but the greatest majority of the tips do, indeed, apply to scripts in general, wherever they are used.

High-Performance Hardware

The following is suggested when processing speed is important. Before looking into a Performance Packs to enhance performance, ensure that the below requirements are met.

  • A physical, non-virtualized server. VMWare servers are great for reducing the numbers of physical machines in your IT space, but they must share the hardware between each other. While you can create a virtual machine that seems as powerful as a physical, it will still be sharing hardware with any other virtual machines, and this will adversely affect performance.
  • MySQL Database on a separate machine. MySQL's main possible bottleneck is file I/O, and as such a high-performance setup will require this server to be on a separate machine, ideally with a high-performance, low-latency hard drive. A Solid State Drive (SSD) would be recommended.
  • High-Quality 16+ GB Ram.This is especially true when working with many server instances ("speed units") running in parallel. The more parallel processing, the more RAM is recommended.
  • 4 or 8 physical cores. We're not talking Hyper-Threading here, but physical cores. Hyper-Threading is great with small applications, but the overhead of "switching" between the virtual cores, and the fact that, well, they're virtual, means the performance is much lesser on high-power applications such as OL Connect. In short, a dual-core processor with Hyper-Threading enabled is not equivalent to a quad-core processor.
 
  • Last Topic Update: 24/01/2017 09:31
  • Last Published: 7/6/2017 : 9:48 AM