Hi All,
Can data mapper handle multi input files?
Recently I got a job that has three input CSV files.
One of them includes all mailing information
and the other two are detail tables for different transactional tables.
I have tried to create the data mapper by using mailing info file but been stuck by creating details tables from another file
Any ideas?
Many thanks
HI,
You can read in two objects your details and transactions,
and the when you extract you do search into yours objects the related datas to the main.
HH
Hi Herve,
Can you provide an example?
This is an interesting topic to be followed.
Cheers,
fsh22
Hi CainZ,
Unfortunately this is a feature that really misses… This would a really great improvement among many others, being able to handle multiple input data through DM.
We are facing this kind of challenges on a daily basis. For now, we are “cooking” data out of PreS and Workflow in order to get jobs done.
Let’s hope someone will have a solution for your/our problem.
Cheers,
fsh22
I have opened an improvement request as i think that it is a great idea. The more agree, the more it would influence our decision makers.
To reference this ticket, please refer to TS-6376.
Hi fsh22
Thanks for your reply.
Yeah, we did the same thing, however we are seeking a fully automative way to do the job
Regards
To be honest, it is unlikely that we can add that functionality natively to the DataMapper.
You have to remember that each DataMapper configuration can only handle a single type of data source (CSV, XML, PDF, etc.). So if you were to send multiple files of multiple formats, the DM wouldn’t know how to handle them because each data type is handled by a distinct internal module.
Assuming we implement a restriction such that all files must be of the same type (let’s say XLS). Then we’d have to be able to tell which Tab of which XLS to use when extracting, looping or creating conditions in the DM config. And we’d have to set up each additional file so that we’d know the boundaries for each record in that file.
So for instance, the JavaScript code for extracting a single field (currently something like data.extract('FirstName',0);) would have to be modified to something like data.extract('FirstName',0,'MyOtherFile','MyFirstTab'), but that still wouldn’t work either because the 0 represents the offset from the current position in the current record… but what is the current position in that other file? And what is the current record?
Now imagine having to implement the same type of lookup for all data formats, including Text files and PDFs!
I think a more realistic approach would be for us to implement a native method for looking up values in certain specific types of external data sources. So for instance, we could lookup inside an Excel file using something like
data.lookup("MyOtherFile.xsl","Tab1.column['FirstName']","John", "Tab1.column['LastName'])
which would search for the first John in column FirstName and retrieve the value of the LastName column on the same row. (By the way, that is essentially what the Workflow plugin does).
However, we’d have to craft it in such a way that performing the lookup doesn’t slow down the DM Engine to a crawl: we wouldn’t want the lookup to have to open, fetch and then close the external file on each call of the lookup function!.
So hopefully, this explains why the feature - although it may seem like an obvious shoo-in for adding to the DM - is extremely complex to analyze and implement in a way that would actually be useful.
That said, I find this discussion very interesting, because it is prompted by actual use cases where you guys have to resolve actual problems day in and day out with our software. So let me ask you something in return, then: in what kind of data file do you most often find yourself looking for additional values? Would it be external CSV files? Or XML? Something else?
If, for instance, we find that most users are just looking for a quick way to lookup values in an XML file or in an Access database, then we could implement something specifically for that data format.
Hi Phil,
Thank for your thoroughly explanation. Working with PreS for more than one year made me aware of the DM architecture. Of course, I was not referring to multiple files of multiple formats. You’ve anticipated me with the assumption of having files of the same type.
“I think a more realistic approach would be for us to implement a native method for looking up values in certain specific types of external data sources.”
I have to admit, I will agree with any approach in order to make my life easy, setting up clients and jobs, that normally require some pain in the neck.
So to answer your question, most often we are looking for additional values in files such as CSV and EXCEL.
In a more unrealistic approach, maybe a feature offering the ability to link 2 DM to 1 Print Content…
Anyway, I will be happy if this topic will stay active and see replies from other users.
PS: Thumbs up for you and your colleagues for taking time to write in the new BLOG section 
Cheers,
fsh22
For me, csv and Excel.
Regards,
S