Flat File Schemas

Here are the different types of components that make up a flat file schema, going from the most elementary to the more complex. Elements and Segments are always required, while Composites, Groups, and Structures might be needed depending on the top-level structure of your document.

The top-level definition in an FFD document starts with the form of the schema. In this case it must always be "FLATFILE", "COPYBOOK", or "FIXEDWIDTH". The differences between these forms are minor and mostly relate to how they are handled in Studio. The rest depends on the form of definitions present in the file, with the following alternatives:

Element definitions are the basic building blocks of application data, consisting of basic key-value pairs for standard characteristics. Flat file schemas generally use inline element definitions, where each element is defined at the point it is used within a segment or composite structure, but you can also define elements separately and reference them as needed. Here are several element definitions defined for use by reference:

The supplied id value is used as the idRef value when referencing one of these definitions as part of a segment or composite. Note that if you are defining elements inline within a segment definition (as opposed to defining them at the end of the document and referencing them), the id field is not required.

Element definitions might have the following attributes, classified by Form as applying to inline definitions, referenced definitions (as in the above example), or references to definitions:

The allowed types for defining an element are:

Value types support a range of format options that affect the text form of the values. These are the main options, along with the types to which they apply:

Composites serve to reference a list of elements that are typically presented together. For example, firstName and lastName can be bundled together into a single composite because they are likely to be referred to as a group. Grouping elements into a composite also allows the list to be repeated.

Composite definitions are very similar to segment definitions, composed of some key-value pairs for standard characteristics along with lists of values. Composites might include both references to elements or other nested composites and inlined definitions. This is a example of a composite definition:

Composite definitions might have the following attributes:

The values list takes the same form as the values list in a segment definition.

A segment describes a type of record in your data. Segments are mainly composed of references or direct definitions of elements and composites, together with some key-value pairs that describe the segment. In a somewhat complex schema, you might have a structure that contains two different segments, where one of these describes the fields that go in the single header of a bill of materials (such as date and person), while the other segment describes the recurring fields that go into each of the actual items in the bill of materials.

This is a sample segment definition that includes one simple element and a composite with two elements within:

Segment definitions might include the following attributes:

Structure definitions are composed of a list of references to segments and group definitions, as well as a set of key-value pairs for standard characteristics. Segments may be further organized into groups consisting of a potentially repeated sequence of segments.

Here’s a sample structure definition again:

This example includes references to two segments at the top level (RQH and RQF), as well as a group definition Batch that includes references to other segments (BCH, TDR and BCF). Note that for this structure to work, each of the referenced segments needs to be defined. See Referenced versus Inlined Definitions to understand how segments are referenced in this example.

A structure definition can contain the following attributes:

Each item in a segment list is either a segment reference (or inline definition) or a group definition (always inline).

Besides the choice of top-level form, you also have choices when it comes to representing the components of a structure, segment, or composite. You can define the component segments, composites, and elements inline at the point of use, or you can define them in a table and reference them from anywhere. Inlining definitions is simpler and more compact, but the table form allows definitions to be reused. Table form examples must include an id value, and each reference to that definition uses an idRef. This example shows how this applies to the segments making up a structure:

In the above example, the BatchReq structure references segments in the data definition section. The segments are each then defined in the segments section at the top level of the schema, and these in turn reference elements that are later defined in the elements section.

An inlined definition of the same structure looks like this:

This example contains a single structure named 'BatchReq' with 5 components segments, using a doubly-nested structure of file and batch data for the segments. Each batch contains repeating detail records. All element definitions are in-lined.

The BatchReq structure definition requires that the data will consist of:

For this example every record starts with a three-character Record Type field with a specified tagValue. In the case of the batch records, the record type is further specified by a Batch Function tagValue.

This is a sample of data matching the schema example:

The lines in the example match the defined structure as listed below:

tagValue fields provide a lot of flexibility. The above example shows using a single tagValue for some record types, while adding a second tagValue for others, but you can also use completely different fields (or even disjoint sets of fields) for a tagValue, as long as you provide enough details for the parser to distinguish between the different types of records.

Note that older versions of the documentation showed a different way of distinguishing records based on tag values, using tagStart and tagLength values for the structure and tag values for the segments. This method of distinguishing segments is much more limited than the tagValue approach, and is now deprecated.

Supported Data Formats