X12 837 message handling with Camel PoC

The purpose of this PoC is to demonstrate reading and writing X12 837 EDI message in Camel Route with using Apache Camel DFDL Dataformat support, as well as letting Kaoto DataMapper participate in.

The most important piece in this PoC is the DFDL (Data Format Definition Language) schema file which describes the ruleset for transforming between EDI data and XML. Since DFDL is just an annotated XML schema, once it's defined appropriately, it can be directly attached as a Kaoto DataMapper Document schema, and perform visual data mapping from that point.

In the future, we can explore to integrate the marshal/unmarshalstep with the DFDL DataFormat into Kaoto DataMapper step. For example, when you attach a DFDL schema in DataMapper, it automatically creates a marshal/unmarshal sub-step inside the DataMapper step so that user can immediately start working on data mappings with EDI formats.

In this PoC, we only created an example schema for X12 837P. We hope this PoC help you to start creating a real world solution for your X12 837P use case, handle other X12 transaction sets and even other standards such as EDIFACT in a similar way.

Prerequisites

Kaoto Visual Studio Code extension

• Download Extension Pack (Marketplace)
• Official Site

Kaoto extension is used to visually authoring Camel routes and configure the DataMapper step.

Apache Daffodil Extension for Visual Studio Code

• Download (Marketplace)
• Documentation

This extension is a powerful tool when authoring a DFDL schema file. Especially the Visual Debugger provides the intermediate view of the transformed data model - infoset - along with the corresponding line in the DFDL schema file, and you can step forward line by line and see which line of the DFDL schema makes what data into the outcome.

Camel JBang

• Official Documentation

This PoC uses Camel JBang to run the Camel routes.

Contents

Here we have 3 separate camel routes:

Transform X12 837P to XML

This route flows as following:

1. Every 3 seconds, execute this route
2. Reads X12 837P text input into the body
3. Transform X12 837P input to the XML using DFDL data format (unmarshal)
4. Prints the outcome

Note that the DFDL schema file is specified on unmarshal step for the transformation. This DFDL schema file is the most important piece in this PoC, and also the most time taking part to create this PoC. The Daffodil Visual Studio Code extension helped a lot to debug the file and make it work.

To run this route using Camel JBang (assuming the CWD is 02.x12-837):

camel run X12-837P.dfdl.xsd 837-to-xml*

The expected output is same as the input XML file for xml-to-837 route

Transform XML to X12 837P

This route flows as following:

1. Every 3 seconds, execute this route
2. Reads XML input into the body
3. Transform XML input to the X12 837P text using DFDL data format (marshal)
4. Prints the outcome

Note that on marshal step, the same DFDL schema file is specified with the previous 837-to-xml route. Once the DFDL schema is defined appropriately, it works for both direction.

To run this route using Camel JBang (assuming the CWD is 02.x12-837):

camel run X12-837P.dfdl.xsd xml-to-837*

The expected output is same as the X12 837P text input file

X12 837P > XML > Kaoto DataMapper

This route flows as following:

1. Every 3 seconds, execute this route
2. Reads X12 837P text input into the body
3. Transforms X12 837P input to the XML using DFDL data format
4. Perform data mappings with Kaoto DataMapper and create a Message837 XML out of X12 837P XML
5. Prints the outcome

This route is almost same with the previous 837-to-xml route. The only difference is the step 4, Kaoto DataMapper. The XML structure from DFDL unmarshal output is described by the DFDL schema file. We can attach the DFDL schema file directly as a Source Document schema in Kaoto DataMapper UI. Also we created a Message837 example XML schema to describe the output XML structure, and attached as a Target Document schema. This data mappings create the final Message837 XML out of X12 837P XML.

We created an example data mappings as following:

To run this route using Camel JBang (assuming the CWD is 02.x12-837):

camel run  X12-837P.dfdl.xsd \
    x12-837-datamapper.camel.yaml \
    837-to-xml-input-message.edi \
    kaoto-datamapper-2df1aa0e.xsl

Here is the expected XML output

<?xml version="1.0" encoding="UTF-8"?>
<Message837>
   <From>PREMIER BILLING SERVICE</From>
   <To>KEY INSURANCE COMPANY</To>
   <Date>20061015</Date>
   <Time>1023</Time>
   <HealthClaim>
      <SubmitterId>26463774</SubmitterId>
      <MonetaryAmount>100</MonetaryAmount>
      <HealthCareServiceLocationInfo>
         <FacilityCodeValue>11</FacilityCodeValue>
         <FacilityCodeQualifier>B</FacilityCodeQualifier>
         <ClaimFrequencyTypeCode>1</ClaimFrequencyTypeCode>
      </HealthCareServiceLocationInfo>
      <YesNoConditionOrResponseCodeFile>Y</YesNoConditionOrResponseCodeFile>
      <ProviderAcceptAssignmentCode>A</ProviderAcceptAssignmentCode>
      <YesNoConditionOrResponseCodeBenefits>Y</YesNoConditionOrResponseCodeBenefits>
      <ReleaseOfInformationCode>I</ReleaseOfInformationCode>
   </HealthClaim>
</Message837>

References

• Example: X12 837 - Health Care Claim: Professional - Commercial Health Insurance The example input message used in this PoC was created from this official example, wrapped with dummy ISA, GS, GE and IEA segments.
• 01.camel-smooks-edi-xml-example Just a reference. it was copied from Smooks official example and wrapped into camel route. It is using EDIFACT common object. This was a starting point of this PoC.
• Camel DFDL data format The Camel data format to use DFDL
• Camel DFDL component The Camel component to use DFDL
• Apache Daffodil The DFDL (Data Format Description Language) implementation. Smooks DFDL cartridge leverages Apache Daffodil underneath.
• 03.X12-837-smooks The old PoC using camel-smooks component. This was done before camel-dfdl component and data format were added.
• Camel Smooks The Camel component to use Smooks

TODO

• Take Smooks edi-to-xml and xml-to-edi examples in to and wrap into camel routes
• Verify both edi-to-xml and xml-to-edi works
• Copy Smooks example files into working dir and rename them to represent X12 837 usecase
• Start making hands dirty - replace the example input payloads with X12 837 (input for 837-to-xml, output for xml-to-837)
• Create DFDL schema file to convert 837 to XML
• Execute 837-to-xml and save generated XML as xml-to-837-input-message.xml
• Make sure it runs and prints the log as expected. In theory the created DFDL schema should work for the other way as well, i.e. xml-to-837
• Add DataMapper, so not only just convert between X12 837 and XML, but also demonstrate to read/write from/to a different data shape
• Verify it runs and prints the log as expected
• Update to use camel-dfdl component