In datamapper, how can I access the record count?

Hello,

I need to identify the last record in my data for a script I’m creating that assembles the values needed for a barcode string. I didn’t really think it would be a problem to access this value, but I don’t see in the datamapper API where I can get this value, so is the only way doing a loop and counting records in a script?

thanks!
Lou

The only way to get the total number of record from the Datamapper can be found here.

Then, you can create an empty field named BarcodeData in you Datamodel which you will update after the Execute Datamapping step of your Workflow.

You could export the data as XML or JSON (whichever is easier to use for you) and use it to build your barcode data, from a script.

Then, simply update the BarcodeData field, in the records, in the Create Print Content step.

Are there any plans to add this information to the data mapper API? The data mapper clearly knows what the record count is, so why not have this value in the API as “record.length” or data.count?
What happens if the end user is a Print Shop Mail Connect user and there is no automation?

We are considering it but implementing it will have an unavoidable impact on the data mapping process. When the DM starts processing a file, it doesn’t know how many records it will find. Its main thread applies boundaries, up to a certain number of records. Once it has reached that number of records, it hands them off to a second thread. Meanwhile, the first thread computes the boundaries for another batch of records that is then handed off to another thread… and so on.
This allows the entire process to be parallelized, making it much more efficient.

So something seemingly as simple as a Record Count feature would either prevent the system from running on multiple threads or it would require the DataMapper to pre-process all boundaries before it does anything else. Either way, it would have an impact on overall performance.

OK understood. Thank you for the quick reply!

Can a pre-processor script be used to count the records? The help file says this step runs before any boundaries are created, so wouldn’t that work?

Well if no boundaries have been set yet, you can’t know how many records there are!

Maybe you could add your barcode using Additional Content in Output Preset? Then you could use variables like ${segment.count.documents} in the barcode.

https://help.objectiflune.com/en/planetpress-connect-user-guide/2019.2/#designer/Output/Print/Variables_available_in_the_Output.htm

This is the best solution. Update it from metadata in the Print Content step once you know what the value is - i.e. when the DataMapper has finished executing.

Unless there’s a specific reason you need it at the DM stage though I can’t imagine why? You could even shove it in ExtraData as a means of transport back to the document for presentation.

Modifying Workflow is off limits in my case, the end user has a customized version from another vendor.

Hello

I had a similar need to get the record count as a field in each record and this seems to work:

  1. Add a JavaScript field to the extract step - say RECORD_COUNT - of type Integer and give it a suitable value

  2. Add a postprocessor script:

    let countRecords = data.records.length;
    for (let i = 0; i < countRecords; i++) {
    data.records[i].fields.RECORD_COUNT = countRecords;
    data.records[i].apply();
    }
    

In the designer the value will only ever be 1 when you apply the script, but the real value should be available when run through the workflow (or with a proof print).

I haven’t tested it extensively but it seems to work and I assume that the postprocessor scripts must be run in one thread as they potentially need to access all records, so this wouldn’t impact the parallelisation of the data extraction. With the record count available in every record I imagine you could use this in any way you want in the template scripts (obviously as it’s done the postprocessor you couldn’t use the field in another field in record).

For example to identify the last record you could use

if (record.fields["RECORD_COUNT"] == record.index) .....