Mapping xml data

OL Connect Enterprise DataMapper
1

I am working on mapping my xml file, please see a basic structure with no data below:

h /xnl:NameElement> 2 /xnl:NameElement> /nswec:AECLGWardDescription> 
	<EventIdentifier IdNumber="">
		<EventName></EventName>
		<EventQualifier IdNumber=""/>
	</EventIdentifier>
	<Messages>
		<Message Type="DeliveryChannel"></Message>
		<Message Type="BatchNumber"></Message>
		<Message Type="RollRequestID"></Message>
		<Message Type="EventGroupUniqueID"></Message>
		<Message Type="PollingDate"></Message>
		<Message Type="Disclaimer"></Message>
		<Message Type="AreaText"></Message>
	</Messages>
</OutgoingGenericCommunication>

I need each ‘Voter’ to be it’s own record. To do this I have set my input data path to the below and get the desired results with each voter as it’s own record. My issue is I need the data that sits below my voter data (Event Identifier and Messages) to be displayed in every record. This data will be the same for every record and is only present once at the end of my xml file. The way i have it set up now the Event Identifier and Messages data isn’t even being displayed in the xml viewer. Is there a way to map these values and have it repeat for every record?

Thanks in advance!

2

Hi @bonsuth, and welcome to the community!

The XML snippet you posted is incomplete, but I think it’s safe to assume it looks something like this:

<?xml version="1.0" encoding="UTF-8"?>
<OutgoingGenericCommunication>
	<EventIdentifier IdNumber="">
		<EventName></EventName>
		<EventQualifier IdNumber="1"/>
	</EventIdentifier>
	<Messages>
		<Message Type="DeliveryChannel"></Message>
		<Message Type="BatchNumber"></Message>
		<Message Type="RollRequestID"></Message>
		<Message Type="EventGroupUniqueID"></Message>
		<Message Type="PollingDate"></Message>
		<Message Type="Disclaimer"></Message>
		<Message Type="AreaText"></Message>
	</Messages>
	<EventIdentifier IdNumber="">
		<EventName></EventName>
		<EventQualifier IdNumber="2"/>
	</EventIdentifier>
	<Messages>
		<Message Type="DeliveryChannel"></Message>
		<Message Type="BatchNumber"></Message>
		<Message Type="RollRequestID"></Message>
		<Message Type="EventGroupUniqueID"></Message>
		<Message Type="PollingDate"></Message>
		<Message Type="Disclaimer"></Message>
		<Message Type="AreaText"></Message>
	</Messages>
</OutgoingGenericCommunication>

From what I understand, each <EventIdentifier> element has an associated <Messages> element.

It would be much easier if you XML structure grouped those two elements inside a common parent element, such as :

<?xml version="1.0" encoding="UTF-8"?>
<OutgoingGenericCommunication>
    <Voter>
	   <EventIdentifier IdNumber="">
	    	<EventName></EventName>
	    	<EventQualifier IdNumber="1"/>
	   </EventIdentifier>
	   <Messages>
	    	<Message Type="DeliveryChannel"></Message>
	    	<Message Type="BatchNumber"></Message>
	    	<Message Type="RollRequestID"></Message>
	    	<Message Type="EventGroupUniqueID"></Message>
	    	<Message Type="PollingDate"></Message>
	    	<Message Type="Disclaimer"></Message>
	    	<Message Type="AreaText"></Message>
	   </Messages>
    </Voter>
    <Voter>
	   <EventIdentifier IdNumber="">
	    	<EventName></EventName>
	    	<EventQualifier IdNumber="2"/>
	   </EventIdentifier>
	   <Messages>
	    	<Message Type="DeliveryChannel"></Message>
	    	<Message Type="BatchNumber"></Message>
	     	<Message Type="RollRequestID"></Message>
	    	<Message Type="EventGroupUniqueID"></Message>
	    	<Message Type="PollingDate"></Message>
	    	<Message Type="Disclaimer"></Message>
	    	<Message Type="AreaText"></Message>
	   </Messages>
    </Voter>
</OutgoingGenericCommunication>

This way you could split your file based on the <Voter> element. Do you have any control over how the XML file is generated?

3

Hey Phil,

Apologies for the incomplete snippet, unfortunately we don’t have any control over how the xml is generated.

4

@bonsuth. I happen to have a very similar problem to yours. I have a data source that is structured similarly to yours and I also have no control over the data creation.

Did you ever find a solution?

5

Hey @joshf, I ended up adding a step into my workflow process, when i run my xml through it adds the bottom level data (EventIdentifier and Messages) into each element so it was mapped at each repeat meaning it is always present in my datamodel for each record. This updated xml is what is than automatically used for the rest of the process.

6

Thank you, @bonsuth. Was that workflow step that you added an “Open XSLT” step?

I was able to create an Open XSLT step that works for my application on a small sample of my production data, but when I throw the production file at it, the output is untouched. It seems to be failing silently. Probably due to the size of the xml file I’m trying to process (About 700mb)

7

Hello @joshf ,

Due to its internal configuration and the library used to read XML file, saif XML file can reach up to 10 times their size in Worklow memory which, as a reminder, is a 32 bit application, limited in memory to 1.8 gb.

Should you be able to get another type of input file, like a JSON file or anything else, that would be better.

You could also try to extract the XML needed be repeated from the XML file and pass its content as a runtime paramter to the Datamapper, allowing its access to all records.

You could even run a different Datamapper only to extract the XML needed to all record, store it in a variable in Workflow and pass it as a runtime parameter to the Datamapper needing it.