Calculating Multi Paged Records

Sharne · February 27, 2017, 1:49am

Hi,

Anyone have a way of calculating how many multi paged records there are in Connect designer. (or Workflow). My data has mixed single and multi paged records. I’m using job creation to separate single paged records from multi paged records in Workflow. What I need is a count of both. (not pages but records.) Any way using metadata to achieve this? or something else?

Regards,

S

Phil · February 27, 2017, 7:15am

In Workflow, after either the Execute Data Mapping task or the Create Print Content task, you can use the count of documents in the job ( GetMeta(SelectedDocumentCount[0], 11, Job) )

Sharne · February 27, 2017, 7:34am

Hi Phil,

That wont work but gave it a try anyway. It reveals the number of records in the entire database.

What I need is the total of all the multi paged records that the one branch puts into its own PDF and then the total number of singles that the rest of the branch creates in its own PDF. In the multi paged record PDF I don’t want the total pages in the PDF but rather the number of records. A record can be from 2 to 6 pages.

As mentioned above the line printer data has single and multi paged records mixed together and I’m using Job presets to filter them out. Document Length >=2.

Hope I’m making sense.

Regards,

S

Phil · February 27, 2017, 9:50am

Sorry, I misread your initial post.

There is currently no simple way of doing this (we are working on implementing that functionality in a future release).

A brute force way of doing it would be to split the output on every single document, then merge all the PDF back together after you’ve computed the count of individual files.

Sharne · February 28, 2017, 2:11am

That is kind of a deal breaker there. This should have been addressed long time ago. I have many many jobs that I need to generate a log containing these totals and to break down PDF’s to get the total is not a viable solution when dealing with hundreds of thousands of records/PDF’s.

Is there another way? Alambic API script perhaps? Can support come up with something for me?

Regards,

S

Phil · February 28, 2017, 6:21pm

You can use the Connect REST API to obtain the exact count of items in each job from Workflow. Let’s say your process has two branches, each calling a different job preset. After the Job Creation task is run in a branch, you can use a script that reads the JobID from the metadata and uses it to query the server for its contents.

The following script does this, storing the number of items for the job in JobInfo[9]. Note that it uses the default username/password for accessing the Connect server, you must change that to reflect your own settings. The code may appear complex at first, but once you realize most of it is for the btoa() utility method, which is only required for authentication purposes, then it becomes much less daunting.

var baseURL = "http://localhost:9340";

var xhr = new ActiveXObject("Microsoft.XMLHTTP");
var uN = "ol-admin";
var uP = "secret";

var authToken = getToken(uN, uP);
var jobID = Watch.ExpandString("GetMeta(_vger_job_id[0], 10, Job.Group[0])");

var response = jsonGetRequest("/rest/serverengine/entity/jobs/"+jobID+"/contents",authToken);
if (response) {
  var contentItems = JSON.parse(response);
  Watch.Log("Items in job: " + contentItems.identifiers.length,2);
  Watch.SetJobInfo(9,contentItems.identifiers.length);
}

/////////////////////////////
// Utility methods are below
/////////////////////////////

function jsonGetRequest(url, token){
    var res=null;
    var xhr = new ActiveXObject("Microsoft.XMLHTTP");
    xhr.open("GET", baseURL + url, false);
    xhr.setRequestHeader("auth_token", token);
    xhr.send();
    if(xhr.status==200) {
      res = xhr.responseText;
    }
    xhr = null;
    return res;
}

function getToken(name, password){
    var res=null;
    var xhr = new ActiveXObject("Microsoft.XMLHTTP");
    var url = baseURL + "/rest/serverengine/authentication/login";
    xhr.open("POST",url,false);
    xhr.setRequestHeader("Authorization", "Basic " + btoa(name + ":" + password));
    xhr.send();
    if(xhr.status==200) {
      res = xhr.responseText;
    }
    xhr = null;
    return res;
}

function btoa(input) {
  var TABLE = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
  var REGEX_SPACE_CHARACTERS = /[\t\n\f\r ]/g;

    input = String(input);
    if (/[^\0-\xFF]/.test(input)) {
      Watch.Log('String contains characters outside of the Latin1 range.',2);
    }
    var padding = input.length % 3;
    var output = '';
    var position = -1;
    var a;
    var b;
    var c;
    var d;
    var buffer;

var length = input.length - padding;

while (++position < length) {

      a = input.charCodeAt(position) &lt;&lt; 16;
      b = input.charCodeAt(++position) &lt;&lt; 8;
      c = input.charCodeAt(++position);
      buffer = a + b + c;

      output += (
        TABLE.charAt(buffer &gt;&gt; 18 &amp; 0x3F) +
        TABLE.charAt(buffer &gt;&gt; 12 &amp; 0x3F) +
        TABLE.charAt(buffer &gt;&gt; 6 &amp; 0x3F) +
        TABLE.charAt(buffer &amp; 0x3F)
      );
    }

    if (padding == 2) {
      a = input.charCodeAt(position) &lt;&lt; 8;
      b = input.charCodeAt(++position);
      buffer = a + b;
      output += (
        TABLE.charAt(buffer &gt;&gt; 10) +
        TABLE.charAt((buffer &gt;&gt; 4) &amp; 0x3F) +
        TABLE.charAt((buffer &lt;&lt; 2) &amp; 0x3F) +
        '='
      );
    } else if (padding == 1) {
      buffer = input.charCodeAt(position);
      output += (
        TABLE.charAt(buffer &gt;&gt; 2) +
        TABLE.charAt((buffer &lt;&lt; 4) &amp; 0x3F) +
        '=='
      );
    }

    return output;
  };

Sharne · March 1, 2017, 1:54am

Thanks Phil, I knew you guys would have something up your sleeves. However I’m getting an error.

W3602: Error 0 on line 42, column 5: msxml3.dll: Access denied.

Line 42 on my side reads xhr.send();

I added the script after the Create Job Plugin and did change the uN & uP variables using my user and pass. The user account on the Windows 10 machine is also admin.

My process is as follows:

Folder Capture - All In One (data map & content creation only) - branch - Create Job - Your Script - Create Output - Send To Folder.

Thanks for the help.

Regards,

S

Phil · March 1, 2017, 6:41am

Line 42 is in the getToken() function, which tells me the UserName/Password combination you provided is incorrect.

Those values are not your user credentials, but rather the Server Connection Settings found in the Workflow Preferences:

Sharne · March 1, 2017, 7:24am

I did use those credentials and still get the error. (I changed mine to what my PC login is when I first installed Workflow.)

https://learn.objectiflune.com/qa-blobs/4350059730409620854.png

Phil · March 1, 2017, 7:30am

Perhaps your tasks are using different credentials? Check any of your Connect-related tasks in your process (say, the DataMap task for instance) and see if the OL Connect Proxy tab is empty (which it should be if you have default settings in your Workflow Preferences). If it isn’t empty, use the credentials from that Tab in your script.

Sharne · March 1, 2017, 8:01am

They are all empty. Not sure what is wrong. By the way I have always disabled server security settings in the Connect Server Configuration.

EDIT: And in Configure Services in Workflow I’m not using local system account option.

Phil · March 1, 2017, 8:34am

>> By the way I have always disabled server security settings in the Connect Server Configuration

Aaaaaahhhhh… well that explains it: the script is trying to authenticate itself, but server security is disabled!

In the script, replace the following line:

   var authToken = getToken(uN, uP);

with this one:

   var authToken="";

and replace the following line:

   xhr.setRequestHeader("auth_token", token);

with this one:

   // xhr.setRequestHeader("auth_token", token);

This disables authentication and should do the trick. If ever you re-enable server security, you can then change the script back to what you had until now.

Sharne · March 2, 2017, 1:06am

Thanks Phil. Works like a charm. Much appreciated, however I do look forward to a simpler solution.

Regards,

S

Phil · March 2, 2017, 7:11am

The feature was already in our development backlog, so I nudged it a bit higher in the priority list…

TDGreer · October 16, 2019, 11:57pm

This counts the number of Documents, but not the number of Document Sets.

I need to know the number of Document Sets in a Job.

jbeal84 · March 31, 2022, 8:50am

Has this been resolved now in the latest version?

TGREER · June 22, 2023, 4:57pm

Phil,

I tried this script, after Create Job. The Job contains 2 4-page documents. I’d expect the script to return “2” (number of Documents) or “8” (number of total Pages), but instead it returns “4”.

For a Job that produces 10 documents, this is the response:

[0024]
{“identifiers”:
[
{“item”:736501,“record”:616501},{“item”:736501,“record”:616501},
{“item”:736502,“record”:616502},{“item”:736502,“record”:616502},
{“item”:736503,“record”:616503},{“item”:736503,“record”:616503},
{“item”:736504,“record”:616504},{“item”:736504,“record”:616504},
{“item”:736505,“record”:616505},{“item”:736505,“record”:616505},
{“item”:736506,“record”:616506},{“item”:736506,“record”:616506},
{“item”:736507,“record”:616507},{“item”:736507,“record”:616507},
{“item”:736508,“record”:616508},{“item”:736508,“record”:616508},
{“item”:736509,“record”:616509},{“item”:736509,“record”:616509},
{“item”:736510,“record”:616510},{“item”:736510,“record”:616510}
]
}

And “contentItems.identifiers.length” returns 20 when it seems it should return 10.