I have a user with a process that at one point calls the Create PDF plugin. This process is self-replicating. Is the Create PDF process a bottleneck? Does it use the ppAlambic service? And is that thread-safe?
It uses the PPAlambic Service and it is thread safe.
It could become a bottleneck if there are more jobs being processed simultaneously than there are Alambic instances allocated in the Messenger preferences. When that happens, jobs are queued and the queued jobs are processed one by one as soon as an instance gets freed.
1 Like
It should also be noted that Alambic maximum number of instance should be set in Workflow->Preferences->Plug-in->Messenger.
If not, then the system will try to add as much as it can, depending of how many memory is still available. That can cause the CPU to reach 100% and that it never a good idea as it could create weird unreproducible errors with Alamibc.
I usually calculate those as follow:
(Number of cores available on the VM - 1) * 1.5
This minus 1 core is for the OS and sometime more if any other program run on there that is CPU intensive.
This formula is not an exact science but it does help preventing undesirable Alambic error.
1 Like
Great information, thank you both. This is a client that receives PDFs, generates and manipulates metadata to group and sort the PDF, then Create PDF to generate the final version for mailing.
The test now is to see if doing this all in Connect via Job and Output presets is any faster.
So it’s threads & Alambic instances VS. Connect Engines.
Slight clarification, the internal algorithm for determining the number of Alambic engines to run is twice the number of available cores, with a minimum of 3 and a maximum of 8. This has been determined (admittedly, a long time ago) to be the optimal amount of engine to benefit from CPU scheduling without becoming I/O-bound.
Does that mean that if we setup a number of Alambic in Workflow greater than 8, even if we have the needed core, it won’t go over 8?
Non, that’s the default. If the user sets a number than that number overrides the algorithm.