I can confirm that. In one project I constantly have to convert from utf-8 to latin-1 and back. And if the data has to be generated as CSV for Excel in UTF-8, care has to be taken to add the BOM and for certain workflow conversions the BOM has to be removed again.
Meanwhile so many projects are connected with web page requests (Ajax requests). It would be nice if the workflow could take over this encoding.
The worst case is when you don’t know reliably in which character encoding the input file will come. Is there any way to determine the character encoding?
Except for XML, which defines an encoding attribute, the other text-based data formats have no way of natively storing the encoding of characters. Even the BOM that some software prepend to UTF-8 files is not actually part of the UTF-8 standard, so you can’t rely on its presence/absence to identify the encoding.