i have a large amount of single, multi-page pdf files in a local folder. In a design template these are loaded as variable background pdf depending on a data field (control script).
In the output a datamatrix code is used as additional content and a splitting is done every 100 sheets.
For jobs up to 1600 records (total output pages = 16000 pages) the output works. For jobs with more than 2000 records (total output pages > 28000 pages), no output takes place.
The MergeEngine processes in the TaskManager stop at a CPU utilization of about 13 percent and a RAM utilization of about 1.2 GB (CPU and RAM values still move, but only minimal fluctuations up and down). The processes do not really hang and there is no error message in any log. However, the switch to the server and then to the WeaverEnigine process just does not happen.
I have already set all RAM values significantly high (both in the Connect Server Configuration and in the INI files).
Does anyone have any ideas what else I can adjust to get the “big” jobs processed?
Hello,
you can try to add more Merge engines available for production. This can be done in Connect Server Configuration.
Pay attention to have enough virtual cores of the CPU available for those engines to run.
Just as a test, you could try setting the Merge Engines to 1 only (thereby ensuring the entire job is processed with a single engine) and assigning a ludicrous amount of memory to that engine (8GBs or more). Monitor the RAM as the job is being processed, this might give us some additional clues as to what’s happening.
I have never tried to set only 1 MergeEngine, but because we have more than enough RAM I tried to set 3 MergeEngines with 6 GB per Engine. The single MergeEngines never use more than 1,6 GB RAM (according to TaskManager).
We could never monitor an overload.
But yesterday we tried to split the input xml to three smaller ones (each xml had 1000 records = 8000 pages in output). The job (one xml file with 1000 records) finished within 8 minutes.
After 3 minutes the MergeEngines stopped moving and after another 3 minutes the Server and then the WeaverEngine came up in TaskManager to finish the job correctly.
We have PlanetPress Connect in version 2020.1 in use (currently an update is not an option, because of the high risk of a faulty update with subsequent CleanUninstall). The time between the MergeEngines and the server/WeaverEngine was approximately the calculated output speed time (8000 pages / 3000 pages per minute = 2.7 minutes). But that can also be a coincidence.
To my knowledge, the time (output speed limit) should only come into play in the WeaverEngine or has this been changed as of version 2020.1?