Strings with Emoji dropped when executing standalone datamapping

Hello,

I am running a datamapping process to generate postcards with user entered text which can include emoji’s. In other documents I am doing this with, I am doing it in an all in one and not storing the data in the database and it seems to work without much issue.

However for this document, since I am storing the data in metadata and then pulling it out of the database to run through the print content / output creation, some of the strings which contain Emoji’s are blank when they are viewed from view metadata in debug and then do not appear on the output. Though its worth noting it does not appear to be all strings with emoji’s. I have at least 2 examples where it went through without a problem. Lastly, its not just the emoji that is missing, but the entire string I’m trying to extract is blank

If I open the template/datamapper directly, load the data, and print, it outputs the document as expected.

Hi sam

what version of the product are you using?

thanks
Alex

I am on 2021.1.1.6675

Hello,

There are two distinct limitations which may explain this situation.

The first is that extended unicode characters, such as emoji or uncommon Chinese characters, are not yet fully supported, i.e. some work, some don’t. The circumstances under which they are or are not supported is rather vague and implementation-specific, but the pattern seems to be that it fails whenever the characters are written to the database. This seems to fit the behavior you describe, where it works in the all-in-one task but not the other.

The second limitation is that Workflow is not fully Unicode-aware. You don’t explicitly mention Workflow being involved in your original post, but your mention of “debug” and “metadata” suggests usage of Workflow.

I would suggest that you open a support ticket with us and submit your process so that we can add it as a use case when this item is scheduled to be worked on.

In the mean time, there is probably a way to make it work with a mix of “no database”, scripting, direct Server REST API calls, etc., at the cost of additional complexity. This something that Support may be able to help with, or perhaps Professional Services for a fee.

Guess it may also depend on the notation of the emoji’s in the data. Admittedly I didn’t test this with meta data but the following is what I used in the past. Perhaps it is of some help.

<xml>
<record>
<field1>I will display: &#128540;</field1>
<field2>I will display: &#x1F61C;</field2>
<field3>I will display: \xF0\x9F\x98\x81</field3>
<field4>\xf0\x9f\x8d\x80</field4>
</record>
</xml>

emoji-sample