Q&A: Complex Data Integration in Centerprise

Q: I have hundreds of XMLs located on a server, I need to be able to extract all available XMLs from that directory and create a source location of all conjoined XML files to begin my dataflow.

Sounds like you have a bunch of XML files and you need to create a single target based on those XMLs. You can do this in Centerprise by creating a workflow.

In Sources, you’ll find the File System Entry Source.

In File System Entry Source you can point in your directory to your .XML files, click OK, and now you can use an XML dataflow to take your input as the XML file source. Select the full path and drag it to the workflow and you are done. This will now loop through the system and process each XML file individually.

Another way to do this would be to use one of the Centerprise functions in the dataflow. Under Function Transformations you’ll find file functions and you would use the Read File Text function. You can take the full file path as the output path, which will be a bunch of text. Then you can send that text to the Centerprise text parser, give it your schema of the XML file, and process as shown in the previous example.

Q: Do you support HIPAA transactions?

This is in the context of EDI. Yes we do support HIPAA. In the source you can see this is an X12 850, we support all X12 messages including HIPAA transactions. You can find out more technical detail on our website under our Technology Solutions.

Q: Can you process messages in real time?

Yes. Everything I demonstrated today was done in batch processing, but when you are talking about real time processing, especially in the concept of web services, such as if you need to monitor or write to a message queue or send a message to a web service, these activities are typically done in real time and involve components combining dataflow and workflows.

The hierarchical processing does not change in the two modes, the only difference between batch and real time is that when you are dealing with transactions such as writing to relational tables, each individual record will be its own transaction as opposed to writing everything in one transaction. There is no difference between how you are doing the mapping, in dealing with the hierarchical information it is all exactly the same.

Q: Do we need to use XSLT to perform these transformations?

No, and that is hopefully it has been made clear in this webinar is that you do not need to do that with Centerprise. In a lot of other ETL tools if you are presented with XML data you have to run some sort of XSLT to process it in order to get it into a flat format, in which case you are looking for individual nodes or maybe you have some other script you have to run ahead of time.

With Centerprise you don’t have to do any of that, you can just point to the file, point to your structure and simply start mapping to it, mapping from it, or creating a structure. This is something very unique in Centerprise that is both powerful and at the same time easy to use.