I have a process, in production for several months, that largely works just fine. I don’t see any flaws with my design of the process. It uses PlanetPress Image to output TIFF files, which are then, outside of PlanetPress, zipped and sent to our client agency for their archival system. That’s initiated with a call to a batch file:
Digital Imaging in Passthrough processes a PS file generated by Connect:
Intermittently, two or three documents are “missing” from the output. No TIFF is generated. But when we test, we get all the output.
My question is, how do I troubleshoot this? What log files should I look at?
My only working theory is that, the branch that has the Digital Imaging plugin will finish, and move onto the batch file that zips the TIFFs, but in actuality, the Image service isn’t REALLY finished (it runs asynchronously to the Workflow Process that calls it), so the batch file is called BEFORE all the TIFFs are output.
Is that what’s happening? If so, how to resolve?
Thanks in advance, PlanetPress Cloud Brain.
The Digital Action plugin only reason to be is that it is synchronous to Workflow.
I would concentrate on the why some TIFF do not get created.
It would be better to open a ticket for that.
I understand, but don’t want to open a ticket for a problem that I can’t reproduce. Thus my question, what would OL want to see in order to work on this issue?
I would add another folder to keep a copy of the TIFF. This way we can check if they were created or not or something else went wrong during the zipping process.
That’s done… the issue is that in a random group of documents passed through to Digital Imaging, from time to time, a few files are simply not produced.
Then next step, since you cannot reproduce the issue when resent (both resent LIVE and in debug?), would be to look at the source file of those who did produce a problem to pinpoint if they have something in common. Maybe they call a very big static images.
I also would look at the number of Alambic instance max you currently have setup as to ensure that the CPU doesn’t reach 100%, which can cause unreproducible problems.
To do so:
Preferences->Plug-in->Messenger
Have the Let me set how many instances to run checked
Set the value following this formula (not an exact science):
((Nb virtual cores) - 1) * 1.5
The -1 is for the OS. You can increase it to 2 if you have other CPU consuming software running on your server like a really demanding AV.
As Jean-Cedric mentioned in passing, the Digital Imaging plugin does not send jobs to the PlanetPress Image service, but rather does everything synchronously. In other words, the process stops and waits until the conversion is finished before moving on to the next task. In your process, that would be the Delete task at the end of the branch, and then the Run External Program on the trunk.
Because the Digital Imaging plugin’s work happens within the main Workflow service, everything it does should be logged in the Workflow log file.
Well, that’s almost true… The actual conversion from PostScript/PDF to TIFF is done by a pool of PlanetPress Alambic engines (ppalmbic.exe), which run as external helper processes managed by PlanetPress Messenger.
So, to have a complete overview of this process, you have to look at the Workflow log for everything that happens in the process, but also at the Messenger log to see the result of the image conversion. So please verify both logs at the time of the missing file(s) to see whether an error occurred at that time.
Sorry for the late reply. The upgrade to 2021.1 has me swamped with issues. Today again, in a job that should produce about 2000 TIFF files, two were not produced. This is from the Workflow log:
Starting at line 48350:
INFO : 14:49:22.536 [0026] Initializing PlanetPress Alambic
INFO : 14:49:22.536 [0026] Ripping file C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\dig24449063.ps to C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\BEN-21191-514689094-000631-0001%d.tif
ERROR: 14:49:22.567 [0026] W3645 : Error while converting to raster: Error converting file: undefined
ERROR: 14:49:22.567 [0026] Digital Action: W1603 : Plugin failed - 2:49:22 PM (elapsed time: 00:00:00.047)
Starting at line 85151:
INFO : 15:09:49.732 [0026] Initializing PlanetPress Alambic
INFO : 15:09:49.732 [0026] Ripping file C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\dig24574A26.ps to C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\BEN-21191-607145782-000136-0002%d.tif
ERROR: 15:09:49.764 [0026] W3645 : Error while converting to raster: Error converting file: undefined
ERROR: 15:09:49.764 [0026] Digital Action: W1603 : Plugin failed - 3:09:49 PM (elapsed time: 00:00:00.047)
And what about the settings on the number of PPalambic instances?
Was it setup?
Did you changed it?
No. The only recent change was the upgrade from 2020.1 to 2021.1. The problems began immediately after. And there are no differences in terms of static images between documents that were produced and documents that were not.
Then can you set it up? The number of Alambic instances I mean.
I did, I set the number of instances to 4. The errors persisted, and the number of ppalambic services shown in Task Manager stayed at 1, with no CPU spikes. However, the log files show that the Output Creation step, which creates the PS passed into Digital Imaging, failed. I’m opening a support ticket.
NFO : 07:53:39.370 [0021] File captured : C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\job010AKL57XKBYN4527EBFFC3.dat, size: 12475 bytes
INFO : 07:53:39.370 [0021] Plugin Folder Capture completed successfully - 07:53:39 (elapsed time: 00:00:00.003)
INFO : 07:53:39.374 [0022] Sending file data source...
INFO : 07:53:39.379 [0022] Sending data mapping command...
INFO : 07:53:39.479 [0022] Data successfully created from : PDF_to_TIFF.OL-datamapper
INFO : 07:53:39.479 [0022] Plugin Execute Data Mapping completed successfully - 07:53:39 (elapsed time: 00:00:00.109)
INFO : 07:53:39.479 [0023] Source data is MetaData
ERROR: 07:53:40.197 [0023] W3001 : Error while executing plugin: HTTP/1.1 500 There was an error running the content creation process caused by ApplicationException: IPC communication error (CC1000009)
InputStream.read() returned 0 characters when trying to read 8000 bytes (SRV000022)
ERROR: 07:53:40.198 [0023] Create Print Content: W1603 : Plugin failed - 07:53:40 (elapsed time: 00:00:00.719)
ERROR: 07:53:40.200 [0024] W3001 : Error while executing plugin: W4203 - Metadata does not contain valid Content Item Collection
ERROR: 07:53:40.200 [0024] Create Job: W1603 : Plugin failed - 07:53:40 (elapsed time: 00:00:00.002)
ERROR: 07:53:40.201 [0025] W3001 : Error while executing plugin: W4230 - Metadata does not contain a valid Job Item Collection
ERROR: 07:53:40.201 [0025] Create Output: W1603 : Plugin failed - 07:53:40 (elapsed time: 00:00:00.001)
INFO : 07:53:40.220 [0026] Initializing PlanetPress Alambic
INFO : 07:53:40.220 [0026] Ripping file C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\dig27EE6A89.ps to C:\ProgramData\Objectif Lune\PlanetPress Workflow 8\PlanetPress Watch\Spool\32.tmp\BEN-21205-145668258-000172-0002%d.tif
ERROR: 07:53:40.250 [0026] W3645 : Error while converting to raster: Error converting file: undefined
ERROR: 07:53:40.250 [0026] Digital Action: W1603 : Plugin failed - 07:53:40 (elapsed time: 00:00:00.049)
INFO : 07:53:40.250 [0027] Deleting file : job010AKL57XKBYN4527EBFFC3.dat, size: 12475 bytes
INFO : 07:53:40.250 [0027] Plugin Delete completed successfully - 07:53:40 (elapsed time: 00:00:00.000)
Increasing the memory assigned to the Merge Engine actually resulted in more failures last night.
