Parsing EDIFACT in Python with Syntax Trees: A Practical Look at Edison

An EDIFACT message arrives as a run of segments ended by apostrophes. The obvious way to read one is to split on the terminator and loop over the pieces. That works until the same tag means two different things depending on where it sits. Take a message where REF appears once at header level as a customer reference, then again inside each item group as a list of item references. A flat loop sees two REF segments and cannot tell them apart without hand-written state: which group am I in, have I seen the group opener yet, has the group closed.

That bookkeeping is where EDIFACT parsers rot. Each new message type adds flags, and each new version of a message adds branches. Validation turns into a separate pile of ad hoc checks that slowly drifts away from the parsing code. Edison, an open-source Python library from Amadeus IT Group, works the other way round. You declare the message structure once as a syntax tree, and that one declaration drives parsing, validation and pretty-printing.

From a segment list to a syntax tree

In Edison a segment can define a group of child segments, and those children can define groups of their own, recursively. The project README describes the core idea plainly: rather than treating a message as a flat list of segments, it models the message as a tree. When the parser class is defined, Edison converts that tree into a state machine of the possible transitions between segments.

The practical consequence is that a segment's meaning comes from its position in the tree, not from its tag alone. A REF directly under the message root and a REF inside a PIZ group are different nodes. You attach callbacks to nodes, so a handler written for the nested REF never fires on the header one. The disambiguation you would otherwise code by hand with flags is now a property of the schema.

Groups can be conditional, mandatory or repeatable. Those three properties cover most of what an EDIFACT message implementation guide expresses at segment level, and Edison checks them against the same structure it uses to dispatch callbacks.

Installing and writing a first parser

Edison requires Python 3.12 or newer and installs from PyPI under a name that differs from the import name:

pip install edison-parser

A parser is a subclass of BaseParser with a schemas class attribute and ordinary methods that act as callbacks. This is the README's own example, a pizza order message with the two kinds of REF:

from edison import BaseParser

class PIZZAQParser(BaseParser):
    schemas = {
        (0, 0): [
            ["UNH"],
            ["CVR"],
            ["REF"],  # one type of REF segment
            [
                "PIZ",
                [
                    ["TYP", [], "parse_typ"],
                    ["REF", [], "parse_ref"],  # another type of REF
                ],
            ],
            ["UNT"],
        ],
    }

    def __init__(self):
        self.summary = {}
        self.pizza = None

    def parse_typ(self, segment: str):
        self.pizza = segment[4:]

    def parse_ref(self, segment: str):
        # Only runs for the REF inside PIZ
        refs = segment[4:].split("*")
        self.summary[self.pizza] = len(refs)

Feeding it a message with a header REF+LUIS' and two PIZ groups produces {"MARGARITA": 2, "PESTO": 1} in parser.summary. The header reference is matched by the tree and passes through without reaching parse_ref. The class also exposes PIZZAQParser.validate(edi), which returns a boolean, and PIZZAQParser.show(edi), which prints the message laid out according to the tree.

The schema node format

A schema is a dictionary keyed by the minimum message version it supports, written as a tuple such as (0, 0). The value is a list of segment nodes. Each node is a positional list:

["TAG", children, action, repetition, flag]

• TAG: the three-letter segment tag, such as UNH or QTY.
• children: a list of child nodes, which makes this node a group opener. An empty list means a leaf.
• action: the name of the callback method as a string, or None when you want the structure enforced but nothing run.
• repetition: 1 for a single occurrence, anything greater for repeatable.
• flag: "M" marks the node as mandatory.

Trailing fields can be left off, which is why the simplest nodes are just ["UNH"]. Because the fields are positional, you have to pass a placeholder None for the action when you only want to set repetition and the flag. The order example below does exactly that on its group node. Callback names are strings resolved against the class, so a typo surfaces as a missing method rather than a syntax error. It is worth exercising every callback in a test.

Repetition and mandatory segments

The rules here are precise and slightly different from what a reader of a message implementation guide might expect. A repetition of 1 means the node can be used once in a given sequence. Any value above 1 makes it repeatable. The transition graph only distinguishes 'once' from 'repeatable': it does not count occurrences, so a limit of 2 and a limit of 99999 behave identically.

Mandatory nodes constrain the valid paths. Once a mandatory node is next in a sequence, the parser cannot skip it to reach a later sibling, the parent level, or a child that would bypass it. A mandatory child does not make its parent mandatory. If the whole group is required, the group node needs its own "M":

class OrderParser(BaseParser):
    schemas = {
        (0, 0): [
            ["UNH"],
            [
                "ITM",
                [
                    ["DSC"],
                    ["QTY", [], "parse_quantity", 1, "M"],
                    ["PRI"],
                ],
                None, 99999, "M",
            ],
            ["UNT"],
        ],
    }

    def __init__(self):
        self.quantities = []

    def parse_quantity(self, segment: str):
        self.quantities.append(segment[4:])

Against this schema, UNH'ITM'DSC'QTY+2'PRI'ITM'QTY+1'UNT' validates: two item groups, the second skipping the optional DSC and PRI. UNH'UNT' fails because ITM is mandatory. UNH'ITM'PRI'UNT' fails because it jumps over the mandatory QTY. UNH'ITM'QTY+2'QTY+3'UNT' fails because QTY has a repetition of 1. Those four cases are a good template for the first tests you write against your own schema.

Handling several versions of the same message

EDIFACT messages change shape over time, and a production parser usually has to accept more than one layout at once. Edison handles this by holding several trees in the same schemas dictionary. The callbacks can stay the same:

class RouteParser(BaseParser):
    schemas = {
        (0, 0): [
            ["UNH"],
            ["ORG", [], "parse_origin"],
            ["DST", [], "parse_destination"],
            ["UNT"],
        ],
        (2, 0): [
            ["UNH"],
            [
                "LEG",
                [
                    ["ORG", [], "parse_origin"],
                    ["DST", [], "parse_destination"],
                ],
            ],
            ["UNT"],
        ],
    }

Calling parse(..., version=(1, 0)) selects the highest schema version less than or equal to the argument, here (0, 0), so the flat ORG/DST layout is expected. With version=(2, 0) the segments must sit inside a LEG group. Both runs fill self.route with the same origin and destination, so downstream code never sees the structural change. validate() takes the same version argument.

Two edge cases matter in practice. When the version is omitted, incomplete, or newer than every known schema, Edison falls back to the latest one. That is convenient, but it means an unexpected new version is parsed as the newest layout you know about rather than rejected. When the requested version is older than the first schema, find_transitions() raises ValueError. That exception needs handling wherever versions come from external input.

Shared envelopes and per-parse state

Most message families share envelope segments, and repeating UNH and UNT in every schema invites inconsistencies. Edison lets you declare header and footer lists on the class. They are prepended and appended to every schema when the class is compiled:

class EnvelopeParser(BaseParser):
    header = [["UNH", [], "on_segment"]]
    footer = [["UNT", [], "on_segment"]]
    schemas = {(0, 0): [["BGM", [], "on_segment"]]}

    def __init__(self):
        self.segments = []

    def set_up(self):
        self.segments.clear()

    def on_segment(self, segment: str):
        self.segments.append(segment)

    def tear_down(self):
        self.count = len(self.segments)

The set_up() and tear_down() hooks run before and after each parse. They answer a question the earlier examples sidestep: those examples create a fresh parser per message, so callback state cannot leak between parses. If you reuse one instance across a batch of messages, anything accumulated in attributes carries over unless set_up() resets it. The clear() call above is the line that prevents one message's segments from being counted in the next. tear_down() is the natural place to derive totals or hand the result to whatever consumes it.

Under the hood: the transition graph

When the parser class is created, Edison builds a directed graph from the tree. Each node is a specific segment at a specific position, and each edge is a valid transition to the next segment. Parsing then becomes a walk through that graph. Every incoming segment must match an outgoing edge from the current node, and the node it lands on determines which callback runs. Validation is the same walk without side effects. Because the graph is built once per class rather than per message, the structural work is paid at class definition.

The graph can be exported with pydot. That is the fastest way to debug a schema that rejects a message you believe is valid, because you can see whether a mandatory node is blocking the path or a group was never marked repeatable. Below the graph sits a tokenizer that does more than split on apostrophes. It handles escaped EDIFACT separators and EBCDIC control characters, and it correctly tokenizes segments with a specified length, such as BLB, where a naive split would cut binary content at the first stray terminator.

Where this approach does not fit, and what to watch

The hard requirement is Python 3.12 or newer. Services pinned to an older interpreter cannot adopt Edison without an upgrade. Beyond that, the limits follow from the design:

• Occurrence counts are not enforced. A guide that says a segment may repeat at most 9 times cannot be expressed in the graph, which only knows 'once' or 'repeatable'. If maximum counts matter, check them in callbacks or tear_down().
• The schema is segment-level. The five node fields describe tags, nesting, callbacks, repetition and presence. Callbacks receive the raw segment as a string, and the README examples slice it with segment[4:] and split elements themselves. Element and component parsing, code-list checks and data-type validation are your code.
• Validation shown in the README is boolean. validate() returns True or False. If your pipeline must tell a trading partner which segment was wrong, confirm what diagnostics the library exposes before relying on it for rejection messages.
• Version fallback is permissive. An unknown future version silently uses the latest schema. If that should be an error, check the version yourself before calling parse().
• Schemas are written by hand. For a handful of message types this is a strength: the tree is readable Python and lives next to its callbacks. For a large directory of messages, transcribing every structure is real work, so budget for it or generate the lists from your own specifications.

Where Edison clearly earns its place is the case that started this article: messages whose meaning depends on nesting, which evolve across versions, and where the code that extracts data must agree with the code that decides whether a message is valid. Declaring the tree once removes the flag-tracking state machine you would otherwise maintain by hand, and the pydot export makes the resulting state machine something you can actually inspect.

This article draws on What does the "yield" keyword do in Python? and answers to it (1, 2), on the Stack Exchange network, licensed under CC BY-SA 4.0.