I have a CSV with customer names and addresses. There can be up to 10 fields per record, however if there are fewer than 10 populated fields there are no trailing commas in the record. The Connect datamapper doesn’t like this and can’t parse records that don’t have the expected number of fields.
If I select “ignore unparseable lines” the offending records are skipped but this is not what I need. I need to be able to work with however many data fields there are in each record.
Planet Press Suite handles the same file the way I require and adds blank fields after the final populated field (or at least ignores “missing” fields). How do I get Connect to behave the same way?
I have to second this. While PlanetPress7 may have been able to handle this fine, it’s just tip-toeing around the fact that this is not standard CSV. A well formatted CSV file should always encode all of the columns, even when a field is empty.
This could be corrected in the preprocessor or in the workflow simply enough, but it will have to be corrected before Connect is going to work with it as a CSV file. It would be possible, of course, to work around this again by treating it as a plain text file and building some complicated datamapping logic to parse each whole line, splitting on delimiters. But that’s probably going to be more work than to just fix the data.
Agreed too. I used a script in the preprocessor for this exact reason a few days after purchasing Connect. I just make all CSV default to 20 columns since most of the data I receive did not exceed that.