Connect: a peek under the hood

Connect consists of visible and invisible parts.
The visible parts are the tools that you use to create templates, data mapping configurations, and Print Presets (the Designer/DataMapper), and to create Workflow configurations (the Workflow configuration tool).
The invisible parts process the Connect job to provide the actual output. This topic introduces you to those parts.

Here's a simplified, graphical representation of the architecture of PlanetPress Connect. The components described below are all located in the 'Server' part.

The Workflow server

The Workflow server (also referred to as the 'Watch service') executes processes independently, after a Workflow configuration has been uploaded and the services have been started. The Workflow server can run only one configuration at a time.

There are a number of services related to Workflow. The Messenger service, for example, receives the files sent to Workflow from the Designer and the Workflow configuration tool.
The Workflow Service Console lets you start and stop the different services, except the Connect server, and see their log files (see Workflow Service Console).

Note that Workflow isn't limited to Connect functionality. It was originally developed as part of the PlanetPress Suite. Many of the plugins in the Workflow configuration tool are older than Connect. They were left in for compatibility reasons, even though they aren't all useful or usable within Connect. However, the Connect plugins cannot be used with the PlanetPress Suite software.

The Connect server

As opposed to the Workflow server, the Connect server was designed to be used only with Connect. The Connect server performs several different tasks, all of which are related to Connect content and content management:

It communicates with the Workflow service (with the Connect plugins, specifically) and with the Designer when output is generated from the Designer. It creates records (by extracting data from a data source using a data mapping configuration), and jobs. It communicates with the engines (see below) in order to make them create content items and output (spool) files.

The Connect server is one of the components that has to be installed with Connect (see Installation Wizard).

In the Workflow Configuration Tool preferences you have to set the OL Connect server settings to enable Workflow to communicate with the server (see Workflow Preferences).

The Connect Server Configuration tool lets you change the settings for the Connect server, the engines and the service that cleans up the database and the file store. These settings can also be made in the preferences of the Designer.

The Connect database

The Connect database is the database back-end used by Connect itself when processing jobs. It can be either the MySQL instance provided by the Connect installer, or a pre-existing (external) instance (see Database Considerations).

All generated items (records, content items etc.) are stored in this database , for the next task in the process, as well as for future use, making it possible to commingle Print jobs, for example.

A clean-up of the database is performed at regular intervals in accordance with the settings (see Clean-up Service preferences).

The File Store

Connect has its own File Store which it uses for transient files.
The Clean-up service takes care of removing obsolete files when those files are not marked as permanent (see Clean-up Service preferences).

The engines

DataMapper engine /s . A DataMapper engine extracts data from a data file. The number of DataMapper engines is configurable (DataMapper Engine Scheduling).

Merge engine/s. A merge engine merges data with a template using the scripts in the template, in order to create (Print,Email or Web) content items.
The number of merge engines is configurable (see Merge engine scheduling): it can be increased depending on the capacity of the machine that runs the solution (see Performance considerations).

Weaver engine s. The Weaver engines create Print output from Print content items. It takes the settings made in Print Presets or in the Print Wizard into account.
The number of Weaver engines is configurable as well (see Weaver engine scheduling).

Speed units

The number of 'speed units' is the maximum number of Merge engines or Weaver engines that are allowed to work in parallel. The output speed of all engines together is limited to a certain number of output items (web pages, emails, or printed pages) per minute. How many speed units you have and what the maximum total output speed will be is determined by your licence and any additional Performance Packs you might have.

There is one important twist: when generating Print output, the limit imposed by the number of speed units, only applies to the Weaver engines; when creating Email or Web output, the limit applies to the Merge engines only (the Weaver engine is not involved). Nevertheless, in situations where Print and Email or Web output are being created at the same time, all Merge engines and Weaver engines count towards the maximum number of speed units. (DataMapper engines are not taken into account.)

Each Merge engine and each Weaver engine needs at least one speed unit. However, since the number of engines is configurable, and since small, medium and large jobs may run concurrently, the number of engines in use may not match the number of available speed units. When there are more speed units than there are engines in use, the Connect server distributes the speed units and the maximum output speed to the engines proportionally.

For advice on engine settings and performance, see Engine configuration and Performance considerations.

The REST API

The Connect server receives REST commands (see The Connect REST API CookBook), normally either via the Workflow service or from the Designer. This design allows the Connect functionality to be used by other applications. The server forwards the commands to the appropriate engine and returns the results to the caller. The results are the id's of the items (records, content items, job etc.) that are stored in the Connect database (see below). All Connect tasks except the Create Web Content task integrate the results in the Metadata in Workflow.

The figure below shows the communication between Connect tasks and the Connect server in a Print process.

Printing and emailing from the Designer

To print or send email from within the Designer, the Connect service has to be running. The service is started automatically when the Designer starts, but it may not be running if the Connect Server and the Designer are installed on different computers. The Connect service can be found on the Services tab in the Task Manager.
For a proof print the Connect server is not used. Proof printing is always done locally, by the Designer.