Is the number of print sections in Connect limited?

Hi,

We have a project which involves more than 100 different templates varying between 1 up to 5 pages where most of the pages will require personalisation from a single data source. Is there any limitations in Connect that we would have to keep in mind before starting and what would be the best way to approach this. At the moment we are considering 2 different options.

First would be to treat each template as separate section - is there a limit of how many sections we can have in connect?

Second to have each template as separate html file stored outside and use control script and loadhtml function to decide what is being used for each record. I’m hoping this approach would allow us to use some sort of version control and it would be easier to manage all the templates. (I’ve seen it mentioned in a different post that github can be controlled by workflow would there be any more info available about this? Did somebody tried to do it?)

Which approach would have better performance? And is there any other way of doing it?

I know this is a broad question, but unfortunately that’s all of the info I have at the moment.

Regards

Darek

Hi Darek,

I’m not personally aware of any limitation for the number of sections in a template (and there shouldn’t be any as far as I know), but I don’t believe this has been really tested either. YMMV on that one.

In terms of what’s best for you, it will depend on a few factors of course, which come down to performance, maintenance, and design capabilities. So let’s explore this a little more into details - the advantage/inconvenients of each possible method.

First, what about one template file per design? You can still have a single data mapping configuration, but use separate template files. This method is best if you’re not sharing any design elements or resources - if none of your images are shared, if there’s no similar contents, no shared stylesheets, etc. The advantage here is that when producing output you’re only loading a single template that’s smaller and simpler, thus performance is better. It also means if you want to version things, you have access to the history of each template separately, not just one huge file with lots of contents.

Second, you have the idea of one section per design. This is useful if you have different requirements for Master Pages and Medias. Each section’s Sheet Configuration needs to be set for First/middle/Last/Single sheets with identical or different Master Pages and/or Media configurations. Margins and page sizes can also be different. You don’t necessarily need one section per design though - you could just as easily have specific configurations (like one section for letter documents with single pages, another that has Legal page size with multiple pages, etc) that you call upon dynamically using Control Scripts. Then the contents of the section can either be conditional text, or snippets, depending on your needs.

And last you have the idea of one snippet per design. Great if only the contents changes, but the design surrounding it (aka, the page size, margins, headers/footers, etc) is identical for each of your outputs. Snippets are the only method by which you would be able to load your contents dynamically from an external location, since Sections are internal resources only. However, it can also be the slowest. Accessing contents externally using loadhtml() is certainly not as fast as Control Scripts simply showing a Section. And if those snippets are external then you have to consider disk I/O (and network I/O, if on a network share) as part of the overhead.

So in short, it depends on the design needs, as well as performance. Unfortunately we don’t really have any benchmarks on either of these methods, so what I’m telling you here is pretty much an educated guess. In my personal opinion, having 100 snippets in a single template would be much harder to manage - only one person could update the document at a time, the performance is relatively worse than one template per design, etc.

And lastly, speaking of versioning: There is no integration to any CVS in our software. However, you could easily use git, svn, mercurial, or any system you choose, on the files used by Connect (templates, data mappings, etc). For the demo files produced by the training department we generally use bitbucket, since you can have a free private repository there (unless you have your own server, that is).

~Evie

Hi Evie,

Thank you very much for your answer and pointing out pros and cons of each solution. This has been very helpful and gave us some good grounds for choosing the right path.

Just to summarize:

Option 1 - 1 template per design - assuming that we are not sharing any design elements. So in this case we would execute 1 data mapper for all templates but how we could define which connect file should be used for each record the connect plugin doesn’t give us an option to do it dynamically based on the field value apart from using %o but we have just one data file. So am I’m getting this right that in this case we have to sort and split the data and then process it that way having 1 data file per template so %o works? Or is there some other way around this?

What if we would have common elements for example an insert with some personalization and would be required for let say only 20 templates. We could include it in each file but then any change would require editing all 20 templates or have a it as a completely separate connect file to produce it and then merge with documents based on metadata or just include it as a pdf or html from external source if the personalization and design concerns allows. What would be best in this case, or is there any other way of doing this. Depends on the specification and how many common elements we have this might be deciding factor.

Option 2 - So after your comments I have to completely agree that managing of this kind of setup could become impossible if more than 1 person would have to work on it at the time especially at the early stage unless we create and edit the snippets outside connect and than paste them in.

Option 3 - Seems like to be the slowest option available.

Best Regards
Darek

Let me blow your mind for a second: click on that %o in the Dynamic Description entry in the template selection… you can change that to what you want! You can use local variables and data selections in here, too! (for example on our demo we have this as learn%{templateID}process.OL-template )

However, note that this is not dynamic per record. It will apply to the whole job, all the records would use the same template. If your data contains many records and they have to use different designs, you would either have to split for each file (which adds overhead!) or use the Metadata Tools to first sort by required template (if some trigger is available in your data, this should be trivial using the Metadata Sorter task), create a group for each template (Metadata Level Creation task) and then split the job per template (Metadata Sequencer, 1 group per sequence). The Create Print Content task will use the modified metadata and only process that sequence (group) one after the other.

If you have some templates that share resources you could of course have “less than 100 templates”. If 20 documents share an insert and 10 others share an image (but nothing else) then you’re already down to 72 templates. There’s no perfect science here, it really depends on your exact implementation details.

Or you could use the loadhtml() function to grab specific contents as necessary. If you have an insert shared between 20 templates, using loadhtml() only to load that insert will be less of an overhead than one template loading 100 html files dynamically.

Also one last point: The more data you have in your template (images, sections, snippets, stylesheets, etc), the more overhead since the template is loaded into memory (and disk space!) on the server. This might be an important consideration for you.

~Evie

Hi Evie,

Woow, that’s brilliant we’ve been trying to double click on it and nothing was happening.
We’ve got it working and run some tests with 100 templates where each had a header with 10 different personalised fields loaded through load.html and the results were really good as it is producing about 100 records in 1 minute.
We are waiting for more info about the job but at the moment it looks promising.

Thank you very much for all your comments and help with choosing the right approach.

Have a great weekend.
Darek