I have a Workflow process running multiple ‘folder captures’ and ‘send to folders’. I’m using them to split data into groups based on metadata fields. Problem is that the folder capture does not process all files in its folder because it moves to the second folder capture that is reading off its send to folder.
The documentation reads “will return to the input task once the current file has finished processing. Each time it returns to the input task, it again only captures one single file. It does this until there are no more files in the folder”.
Workflow as follows:
(1)Folder Capture (C:\Spool\DataIn)
Execute Data Mapping
Branched - Meta/DataSplit/JobInfo etc.
Branched - Send to folder (__C:\Spool\DataInTemp__)
(2)Folder Capture (C:\Spool\DataInTemp)
More branches etc…
From what I see is that because the second Folder Capture (2) contains a file, sent from the Send to folder, it immediately executes the second Folder Capture and ignores the files in Folder Capture 1.
If 1) and 2) are in the same process then this is what Workflow does:
Get file from DataIn (lets call it INPUT1)
Processes the file (which ends with Send to DataInTemp)
Get file from DataInTemp ?(lets call it INPUT2)
Processes the file
Goes back to DataInTemp for any new file
Processes any other file or if none, returns to main process
?Goes back to DataIn
Processes any other file or if none, exits process
There are a few reasons why your process may not behave as expected:
?The process is set to self replicate: this works fine as far as INPUT1 is concerned, but if several clones are created, at some point you are likely to end up having several INPUT2 tasks attempting to capture files in the same folder simultaneously, which is always a no-no.
There is an error at some point and the process stops once it encounters it. Check the logs and see if anything stands out.
From what I think I understand of your processes, it seems you might want all files from INPUT1 to have been processed before INPUT2 kicks in. If that’s the case, you’ll want to move INPUT2 in its own process and trigger its launch by having your first process deposit a dummy “marker” file in DataInTemp to signal that all files have been stored there. Your INPUT2 would then monitor the folder for that specific dummy file to determine when it should launch.
Another way to process all files from Input1 before Input2 is processed is to put the Folder capture Input1 into the branch. This will make the branch to loop until there is nothing more in Input1, and then, Input2 will be executed.