Hi
I have a pipe delimited CSV file which contains a Header record (with 20 fields) and the rest of the data records has 465 fields, one record per line.
I still need to extract the header record details as it contains information needed for processing (for example stock names, Colour/BW, file type)
I’m guessing I need to do something in the PreProcesser section but I’m unsure of how to go about this
Thanks Andy
Have a look at this old post. I posted a pre-processing script that adds a field to each line of a CSV.
You can modify the script (well, just the loop, really) to add 445 commas to the first line only (leaving the others untouched). That way, all lines will have the same number of fields.
To add 445 commas at once, you can use :
",".repeat(445)
I think Andy meant there is a line at the top of his data that is needed as fields in the mapper. Then his second line is the actual header that he wants the mapper to use as field names. So normally you would set the mapper to ignore that first line but in his case needs it for mapping purposes. I have this where the statement date is only in that first line.
So if I’m right, he would set fields from that first line into properties which he can then use as fields.
var inputFile = openTextReader(data.filename);
var tmpFile = createTmpFile();
var outputFile = openTextWriter(tmpFile.getPath());
var line = "";
var newLine;
while((line = inputFile.readLine())!== null){
newLine = "";
if (line.includes("Column1")){
var arrFields = line.split(";");
data.properties.column1 = arrFields[0];
data.properties.column2 = arrFields[1];
data.properties.column3 = arrFields[2];
data.properties.column4 = arrFields[3];
data.properties.column5 = arrFields[4];
}else{
newLine = line.concat("\n");
}
outputFile.write(newLine);
}
outputFile.close();
inputFile.close();
deleteFile(data.filename);
tmpFile.move(data.filename);Data sample I used: (csv)
Column1;Column2;Column3;Column4;Column5;Column6
Field1;Field2;Field3;Field4;Field5;Field6;Field7;Field8;Field9;Field10;Field11;Field12;Field13;Field14;Field15;Field16;Field17;Field18;Field19
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data
data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;data;dataIF… I’m right 
Regards,
S
