Hi @Erik,
How would I display the progress message of a running job when the job is in the “All in One” segment of the node-red?
Are you looking for information like: data mapping 0% > data mapping 12% > data mapping 83% > paginated content 24% etc
That is correct. So for example, see below. How would I implemet it in the sample
Summary
[ { "id": "fcebd73895ea1597", "type": "group", "z": "f6f2187d.f17ca8", "style": { "stroke": "#999999", "stroke-opacity": "1", "fill": "none", "fill-opacity": "1", "label": true, "label-position": "nw", "color": "#a4a4a4" }, "nodes": [ "55011c954b2dac52", "a17e2e7b89038c9b", "ad6193a482583e6f", "1b9cd204c2e83970", "21f0f32b154c4ad2", "881d91c2eb232baa", "01a0447e528c42e0", "b041a51f9b67a7e1", "22715caaa93ea788", "e1b52554b842201e", "4716677716c759a4", "da9157abef734ca1" ], "x": 354, "y": 2059, "w": 1792, "h": 242 }, { "id": "55011c954b2dac52", "type": "comment", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "localhost:1880/pdf-compress-a4", "info": "POST to /pdf-compress-a4\n\n- Compress PDF A4", "x": 510, "y": 2100, "wires": [] }, { "id": "a17e2e7b89038c9b", "type": "http in", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "POST /pdf-compress-a4", "url": "/pdf-compress-a4", "method": "post", "upload": true, "swaggerDoc": "", "x": 500, "y": 2260, "wires": [ [ "1b9cd204c2e83970", "22715caaa93ea788" ] ] }, { "id": "ad6193a482583e6f", "type": "http response", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "", "statusCode": "", "headers": {}, "x": 2070, "y": 2260, "wires": [] }, { "id": "1b9cd204c2e83970", "type": "change", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "", "rules": [ { "t": "move", "p": "req.files[0].buffer", "pt": "msg", "to": "payload", "tot": "msg" }, { "t": "move", "p": "req.files[0].originalname", "pt": "msg", "to": "basename", "tot": "msg" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 820, "y": 2260, "wires": [ [ "b041a51f9b67a7e1", "e1b52554b842201e" ] ] }, { "id": "21f0f32b154c4ad2", "type": "debug", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "Output File name", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "payload", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 1780, "y": 2180, "wires": [] }, { "id": "881d91c2eb232baa", "type": "debug", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "Output file ID", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "managedFileId", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 1770, "y": 2120, "wires": [] }, { "id": "01a0447e528c42e0", "type": "debug", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "managedFileId", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 1370, "y": 2180, "wires": [] }, { "id": "b041a51f9b67a7e1", "type": "debug", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "Print input file name", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "basename", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 1070, "y": 2180, "wires": [] }, { "id": "22715caaa93ea788", "type": "debug", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "name": "Data file properties", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "req.files[0]", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 790, "y": 2160, "wires": [] }, { "id": "e1b52554b842201e", "type": "olcnr-file-upload", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "connect_server": "", "filename": "", "fileDuration": "temporary", "name": "file upload", "x": 1140, "y": 2260, "wires": [ [ "4716677716c759a4", "01a0447e528c42e0" ] ] }, { "id": "4716677716c759a4", "type": "olcnr-all-in-one", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "connect_server": "", "dmConfiguration": "pdf-compress-a4.OL-datamapper", "template": "pdf-compress-a4.OL-template", "jobPreset": "", "outputPreset": "Generic PDF.OL-outputpreset", "dataStorage": "database", "name": "All in One", "x": 1480, "y": 2260, "wires": [ [ "da9157abef734ca1", "21f0f32b154c4ad2", "881d91c2eb232baa" ] ] }, { "id": "da9157abef734ca1", "type": "olcnr-file-download", "z": "f6f2187d.f17ca8", "g": "fcebd73895ea1597", "connect_server": "", "managedFileId": "", "name": "download file", "outputType": "buffer", "x": 1830, "y": 2260, "wires": [ [ "ad6193a482583e6f" ] ] } ]
This level of detail is currently not available for the all in one node. Basically requires a change to the way the underlying end point in the OL Connect Server work.
Hi @Erik,
I’ve been trying to play around with this.
I am wondering if it’s possible to create a sample flow similar to a workflow process. So, what I mean with that is watch folder for file (folder catcher) > take file > process file (all in one) > then drop file in another folder when complete. Is this feasable with the current version on the automation?
I can see the nodes “folder catcher”, “all in one” etc… I don’t know of a way to do this but please if you can guide me to make an example.
Thank you in advance.
James.
Hello James,
Certainly, which version of our nodes are you currently using?
In addition to the OL Connect nodes, we’ve also developed our own folder capture and folder listing nodes, which are included in our @objectif-lune/node-red-contrib-files-and-folders package.
We include a variety of samples in our OL Connect package. To access them, go to the Node-RED menu in the upper right corner and select Import. Then, click on Examples and navigate to the node-red-contrib-connect package. You’ll find a range of samples, including ones for an all-in-one setup. Simply choose the relevant sample and click Import. We have plans to expand this list with other commonly used flows.
For further guidance, you can refer to the documentation available in the Read me nodes (comment nodes).
Hope this helps,
Erik
Hi @Erik,
I have installed and updated yours node version to 0.9.9. Thank you for the examples 
Works perfectly, one other question is I’m trying to output a PDF file as the output but rather comes out as a zip file. Even thought I untick the “store resulting zip in a msg.payload”. Is there a way to get the generated file as a PDF result? I Cannot seem to make it work.
There are various ways to output the individual files. One could unzip the file in the flow and write the individual files or download the files using the ‘file store download’ node. In the end these files sit in the OL Connect File Store.
In the following setup the all in one I disabled the option to download the result in zip format. Instead it returns an array with the names of the output files in msg.payload. In addition it returns the ID to which these files belong msg.managedFileId. The msg.managedFileId can be used to download the zip using the file store download node. When combining this information with the name the of the file it can be used to download individual files.
In the example the split node is used to iterate over the file names. This generates a payload with the file name of the file (for example: olacme-invoice_0006.pdf).
I used the change node to append this information to msg.managedFileId. In addition it specifies the target location for that file. The concatenated managedFileId could look something like: 382018/olacme-invoice_0006.pdf. The subsequent file store download nodes downloads the respective file and the write file node writes the file to disk (similar to the original example).
Bit of a long read but hope it helps,
Erik
[{"id":"538ad5e5207b2729","type":"inject","z":"52482c780f912d79","name":"10 sample recs","props":[{"p":"basepath","v":"\\workspace\\all in one\\","vt":"str"},{"p":"file.name","v":"invoices-10","vt":"str"},{"p":"filename","v":"msg.basepath & msg.file.name & \".xml\"","vt":"jsonata"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":140,"y":1420,"wires":[["6fa8de9be96f3aec"]]},{"id":"0f2fafae2a0a8b02","type":"olfnf-folder-capture-in","z":"52482c780f912d79","folder":"\\workspace\\all in one\\in dynamic","folder_type":"str","subfolder":false,"ignoreempty":true,"masks":"*.xml","masksregex":false,"name":"","x":130,"y":1360,"wires":[["6fa8de9be96f3aec"]]},{"id":"6fa8de9be96f3aec","type":"olcnr-all-in-one","z":"52482c780f912d79","connect_server":"591cbd3c704b942c","dmConfiguration":"olacme-invoice.OL-datamapper","template":"olacme-invoice.OL-template","jobPreset":"","outputPreset":"olacme-pdf.OL-outputpreset","downloadResult":false,"dataStorage":"none","datafile":"filename","datafile_type":"msg","name":"","x":360,"y":1360,"wires":[["347ad9d3da0d6e0f"]]},{"id":"1e778184cdf80507","type":"file","z":"52482c780f912d79","name":"write output","filename":"outputFilename","filenameType":"msg","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"none","x":830,"y":1460,"wires":[["a71c355136b23d37"]]},{"id":"a71c355136b23d37","type":"debug","z":"52482c780f912d79","name":"debug outputFilename","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"outputFilename","targetType":"msg","statusVal":"","statusType":"auto","x":1020,"y":1460,"wires":[]},{"id":"f607e53bf71a56be","type":"inject","z":"52482c780f912d79","name":"100 sample recs","props":[{"p":"basepath","v":"\\workspace\\all in one\\","vt":"str"},{"p":"file.name","v":"invoices-100","vt":"str"},{"p":"filename","v":"msg.basepath & msg.file.name & \".xml\"","vt":"jsonata"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":140,"y":1460,"wires":[["6fa8de9be96f3aec"]]},{"id":"2ed217c2a287d010","type":"comment","z":"52482c780f912d79","name":"Read me: individual output files","info":"The **all in one** node returns the information of the files in an array stored in msg.payload (instead of the zip).\nThe **split** node iterates over the array, the subsequent **change** constucts the path for the 'current'file\n in the OL Connect file store and constructes the target location for the file. The **file store download** retrieves \n the file from the OL Connect File Store and the **file write** writes the file to the target location.\n","x":190,"y":1320,"wires":[]},{"id":"347ad9d3da0d6e0f","type":"split","z":"52482c780f912d79","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":490,"y":1360,"wires":[["b5f0233199138c94"]]},{"id":"b5f0233199138c94","type":"change","z":"52482c780f912d79","name":"","rules":[{"t":"set","p":"contentName","pt":"msg","to":"payload","tot":"msg"},{"t":"set","p":"managedFileId","pt":"msg","to":"managedFileId & \"/\" & contentName","tot":"jsonata"},{"t":"set","p":"outputFilename","pt":"msg","to":"\"\\\\workspace\\\\all in one\\\\out\\\\\" & contentName","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":1460,"wires":[["6d3b9835443cc150"]]},{"id":"6d3b9835443cc150","type":"olcnr-file-download","z":"52482c780f912d79","connect_server":"591cbd3c704b942c","managedFileId":"","name":"","outputType":"buffer","x":650,"y":1460,"wires":[["1e778184cdf80507"]]},{"id":"591cbd3c704b942c","type":"OL Connect Server","url":"http://localhost:9340","ignorecer":false,"name":"OL Connect Server","disableProgress":false}]
Hi @Erik,
Thank you a lot
Now just to throw another question, how would I go about renaming the output file generated apending the _0001.pdf at the end of the filename. As it seems the output filename is named according to the PP connect template used.
Glad it helped! On the name, do you want to replace the template name part with something like a customer number?
Yes with the input data file name without extension but retain the nr bit, so “input.txt” > “input_0001.pdf”.
Ah I see, all roads to rome. One way to do it is by replacing the template name in that string in the change node. In this example the name of the data file is stored in msg.file.name.
I’ve added Change option to the change as illustrated below. It performs a search and replace on the property holding the target file name. So the Search for field holds the name of the template and the Replace with field refers to msg.file.name.
Perhaps a bit blund it does the job. See the images below (my data file is called ‘invoices-10’)
Erik
Hi @Erik
A bit complex for me that is It is a bit different when you try to look at it like the workflow way.
That example did not work very what with what I wanted, so I did this. Maybe might need twicking a little to suite ones needs.
[{"id":"74b1f2ac0ef8cf83","type":"olfnf-folder-capture-in","z":"1ba36da3d4e220e6","folder":"\\workdspace\\in","folder_type":"str","subfolder":false,"ignoreempty":true,"masks":"*.txt","masksregex":false,"name":"","x":250,"y":1360,"wires":[["eee42d3ddf9a121f"]]},{"id":"fbd3c5f5ef112442","type":"file","z":"1ba36da3d4e220e6","name":"write output","filename":"outputFilename","filenameType":"msg","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"none","x":1490,"y":1360,"wires":[["91db3c422ecb278f"]]},{"id":"91db3c422ecb278f","type":"debug","z":"1ba36da3d4e220e6","name":"debug filename","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"filename","targetType":"msg","statusVal":"","statusType":"auto","x":1640,"y":1180,"wires":[]},{"id":"7055f77c8a119059","type":"olcnr-all-in-one","z":"1ba36da3d4e220e6","connect_server":"323a7dd3e19674d6","dmConfiguration":"Sample1.OL-datamapper","template":"Sample1.OL-template","jobPreset":"Sample1.OL-jobpreset","outputPreset":"JSample1.OL-outputpreset","downloadResult":false,"dataStorage":"database","datafile":"filename","datafile_type":"msg","name":"","x":660,"y":1360,"wires":[["850674531e47e44a"]]},{"id":"593160c08e011257","type":"comment","z":"1ba36da3d4e220e6","name":"Example: Merge","info":"Merge","x":260,"y":1240,"wires":[]},{"id":"dce84e41868d844c","type":"olfnf-file-operations","z":"1ba36da3d4e220e6","operation":"delete","source_file":"filename","source_file_type":"msg","target_file":"","target_file_type":"msg","create_directory":true,"name":"delete data file","x":1520,"y":1480,"wires":[[]]},{"id":"de6e4d093e83250d","type":"catch","z":"1ba36da3d4e220e6","name":"catch errors","scope":[],"uncaught":false,"x":1190,"y":1500,"wires":[["dce84e41868d844c"]]},{"id":"40969daa44304430","type":"olcnr-file-download","z":"1ba36da3d4e220e6","connect_server":"323a7dd3e19674d6","managedFileId":"","name":"","outputType":"buffer","x":1210,"y":1360,"wires":[["fbd3c5f5ef112442","dce84e41868d844c","ad45b8d7be5d05fa"]]},{"id":"850674531e47e44a","type":"split","z":"1ba36da3d4e220e6","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":810,"y":1440,"wires":[["e6e2540126987d0d"]]},{"id":"e6e2540126987d0d","type":"change","z":"1ba36da3d4e220e6","name":"","rules":[{"t":"set","p":"contentName","pt":"msg","to":"payload","tot":"msg"},{"t":"set","p":"managedFileId","pt":"msg","to":"managedFileId & \"/\" & contentName","tot":"jsonata"},{"t":"set","p":"outputFilename","pt":"msg","to":"\"\\\\workspace\\\\out\\\\\" & outFile_name & \".pdf\"","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":960,"y":1440,"wires":[["40969daa44304430"]]},{"id":"eee42d3ddf9a121f","type":"change","z":"1ba36da3d4e220e6","name":"","rules":[{"t":"set","p":"outFile_name","pt":"msg","to":"file.name","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":1440,"wires":[["7055f77c8a119059"]]},{"id":"ad45b8d7be5d05fa","type":"debug","z":"1ba36da3d4e220e6","name":"debug outFile_name","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"outFile_name","targetType":"msg","statusVal":"","statusType":"auto","x":1390,"y":1180,"wires":[]},{"id":"323a7dd3e19674d6","type":"OL Connect Server","url":"http://xxx.xxx.xxx.xxx:xxxx","ignorecer":true,"name":"PPConnectServer","disableProgress":false}]
Again thank for the guidance
Think you are close. Open the change node after the split node and do the following:
Add a new Rule, set this Change, set the first option to msg.outputFilename. Set Search for field to the name of your template and set Replace with to msg.outFile_name. This bascially performs a simple search & replace where it replaces the name of the template with the value you stored in msg.outFile_name.
Make sure to change the path of outputFIlename to contentName (without the .pdf)
Erik
I’m really trying to follow along here, but it’s losing me. I have a similar issue trying to create my first Flow. I cannot get PDF output, where I want it, and named how I want it. I’m ending up with a ZIP file in the correct folder and with the correct filename (using flow variables)… but it’s a ZIP file, not a PDF!
If I choose the “Managed by Output Preset” Output Option, I cannot use the flow variables to specify the path and filename (if there’s a way to do this, let me know).
If I choose the other options, zip or “file list”… then I’m baffled.
After my “all in one” with the output set to “zip”, I have a Change Node where I set “msg.outputfile” to the value of “flow.PDFoutput”, hoping I could use “msg.outputfile” in the Write File node.
And indeed, I get a file with that name, in the proper folder. But it’s a ZIP file, not the PDF file. Inside the ZIP file is the PDF file, but with the name as spec’d in the Ouput Preset!
How do I get the output of the “all in one” in its native format (PDF), to a path+filename I’ve stored in a flow variable?
Here is what I ended up doing, and would respectfully ask for a “sanity check” on this approach:
[
{
"id": "7615da8e6333db8f",
"type": "tab",
"label": "Flow 1",
"disabled": false,
"info": "",
"env": []
},
{
"id": "7b35f5d7165612f7",
"type": "olfnf-folder-capture-in",
"z": "7615da8e6333db8f",
"folder": "E:\\nodeRED_hotfolder",
"folder_type": "str",
"create_directory": true,
"subfolder": false,
"ignoreempty": true,
"masks": "*.*",
"masksregex": false,
"watchMethod": "polling",
"name": "",
"x": 110,
"y": 100,
"wires": [
[
"c49425b246af79fb"
]
]
},
{
"id": "c49425b246af79fb",
"type": "function",
"z": "7615da8e6333db8f",
"name": "GenerateCommonVariables",
"func": "var inputData = msg.filename;\nvar baseFileName = inputData.split(\"\\\\\").pop();\nvar archivePath = \"E:\\\\pbCCMData\\\\Guild\\\\nodeRed\\\\Archive\\\\input\\\\\"\nvar printBatchPath = \"E:\\\\pbCCMData\\\\Guild\\\\nodeRed\\\\PrintBackup\\\\\"\nvar printBatchName;\n\nvar timestamp = new Date();\nconst year = timestamp.getFullYear();\nconst month = String(timestamp.getMonth() + 1).padStart(2, '0');\nconst day = String(timestamp.getDate()).padStart(2, '0');\nconst hours = String(timestamp.getHours()).padStart(2, '0');\nconst minutes = String(timestamp.getMinutes()).padStart(2, '0');\nconst seconds = String(timestamp.getSeconds()).padStart(2, '0');\ntimestamp = `${year}${month}${day}_${hours}${minutes}${seconds}`;\n\nmsg.payload = timestamp;\nflow.set('timestamp', timestamp.toString());\n\nprintBatchName = \"GMCOUT_\" + timestamp + \"_BK07.pdf\";\n\nflow.set('PDFoutput', printBatchPath + printBatchName);\n\nmsg.payload = archivePath + \n timestamp + \"_\" + \n baseFileName;;\n\nreturn msg;\n",
"outputs": 1,
"timeout": 0,
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 240,
"y": 40,
"wires": [
[
"b2da6a59b36003e8"
]
]
},
{
"id": "b2da6a59b36003e8",
"type": "olfnf-file-operations",
"z": "7615da8e6333db8f",
"operation": "move",
"source_file": "filename",
"source_file_type": "msg",
"target_file": "payload",
"target_file_type": "msg",
"create_directory": true,
"name": "",
"x": 360,
"y": 100,
"wires": [
[
"e905211d5f79d11c"
]
]
},
{
"id": "e905211d5f79d11c",
"type": "olcnr-all-in-one",
"z": "7615da8e6333db8f",
"connect_server": "567749e0.a988b620",
"dmConfiguration": "GUILDst9.OL-datamapper",
"template": "GUILDst9.OL-template",
"jobPreset": "",
"outputPreset": "GUILDst9.OL-outputpreset",
"outputOptions": "fileList",
"outputFolder": "E:\\pbCCMData\\GUILD\\nodeRed\\PrintBackups",
"outputFolder_type": "str",
"dataStorage": "none",
"datafile": "payload",
"deletefile": false,
"datafile_type": "msg",
"name": "",
"x": 540,
"y": 100,
"wires": [
[
"4d87e82f32ae169f"
]
]
},
{
"id": "c4e9e2190d6b3bf6",
"type": "file",
"z": "7615da8e6333db8f",
"name": "write output",
"filename": "outputFilename",
"filenameType": "msg",
"appendNewline": false,
"createDir": true,
"overwriteFile": "true",
"encoding": "none",
"x": 1270,
"y": 100,
"wires": [
[]
]
},
{
"id": "4d87e82f32ae169f",
"type": "split",
"z": "7615da8e6333db8f",
"name": "",
"splt": "\\n",
"spltType": "str",
"arraySplt": 1,
"arraySpltType": "len",
"stream": false,
"addname": "",
"property": "payload",
"x": 690,
"y": 100,
"wires": [
[
"1b0579754278aba0"
]
]
},
{
"id": "1b0579754278aba0",
"type": "change",
"z": "7615da8e6333db8f",
"name": "",
"rules": [
{
"t": "set",
"p": "contentName",
"pt": "msg",
"to": "payload",
"tot": "msg"
},
{
"t": "set",
"p": "managedFileId",
"pt": "msg",
"to": "managedFileId & \"/\" & contentName",
"tot": "jsonata"
},
{
"t": "set",
"p": "outputFilename",
"pt": "msg",
"to": "PDFoutput",
"tot": "flow"
}
],
"action": "",
"property": "",
"from": "",
"to": "",
"reg": false,
"x": 860,
"y": 100,
"wires": [
[
"7e9aee1550eb42b3"
]
]
},
{
"id": "7e9aee1550eb42b3",
"type": "olcnr-file-download",
"z": "7615da8e6333db8f",
"connect_server": "567749e0.a988b620",
"file": "managedFileId",
"file_type": "msg",
"name": "",
"outputType": "buffer",
"x": 1070,
"y": 100,
"wires": [
[
"c4e9e2190d6b3bf6"
]
]
},
{
"id": "567749e0.a988b620",
"type": "OL Connect Server",
"url": "http://localhost:9340",
"ignorecer": false,
"name": "OL Connect Server",
"disableProgress": false
}
]
We have a tutorial outlining the various techniques, see Writing output to disk in OL Connect Automate
Topics covered:
- Handle output through OL Connect’s output engine (also explains how to set the output folder location from the all in one and paginated output nodes)
- Write the default zip output
- Download individual files from OL Connect’s File Store
- Unzipping the default zip archive and write individual files
Erik