I added a text tab delimited file to a new data mapping. When going to the steps tab the preprocessor gives me the following error. While the error is there I get 56 records. The only way to get rid of it is to check the Ignore unparseable lines checkbox in the input data pane. I then get 90 records which is the correct amount. I imported the file into Excel and it is only 8 columns.
Refresh boundaries
Line contains wrong number of columns: 9 8
Preprocessor
Other tab delimited files from the same client work fine. Only the one errors out.
One more quick one regarding missing fields. So you are telling me that If I have 2 CSV files and one has 10 columns of data and the second has 13 columns of data, the data mapper will error out? Rhetorical question because I tried and it does give an error. So, back to why I’m posting, can I in some way have the mapper allow any CSV data that has up to 15 lines of data without errors?
When I used PP7 I would cater for this by adding more Lines To Page in its data selector. Therefore allowing any CSV file that has 1 to 15 columns of data.
There is a fundamental difference between how PlanetPress and Connect process CSV files: the former treated it as a text file and therefore analyzed each line separately. The latter treats it as a database and therefore analyzes it as a whole.
The Connect approach increases the performance and allows an almost unlimited number of rows per record, but it does so at the expense of flexibility.
In designing the application, we considered the user comments (in forums such as this one, for instance) and realized that processing data errors from inside the data-processing operation (or inside the template) could quickly become cumbersome. For instance, designing a template that expects 10 fields but that only receives 5 means that all fields must be subject to a condition that determines if the fields actually exist. This can quickly lead to a very messy template that’s hard to maintain.
We elected to make sure errors (or simple discrepancies) would be trapped as soon as possible.
That said, we are looking at implementing variable-record CSV files, but probably as a completely separate mode. This will eventually be added to Connect (no timeline yet).
Thanks for the info Phil. I’m going to write a preprocessor script that will count the commas in a line/record and add more to the end of each line making all files the same amount of columns.
Working great.