...
Most implementations of KFS are going to integrate with other enterprise systems in a variety of ways. The picking up and parsing of batch files is an extremely common method for external applications to push data to KFS. By using KFS's flat file parser, an implementing institution can configure the parse of pushed flat files in Spring configuration, greatly faciliating the parsing process for flat files.
Configuring a Flat File Parser
Imagine, for instance, a file which told KFS to create documents of certain types with accounting lines. Here is our imaginary file format:
...
There are four levels to our configuration: FlatFileParser, which deals with the parse process as a whole; FixedWidthFlatFileSpecification, which handles the parsing of the whole file into an object graph; FlatFileObjectSpecification, which handles the parsing of a single line into an object; and FixedWidthFlatFilePropertySpecification, which populates a business object with data from substrings of a given line. Let's take a look at each of these in turn.
FlatFileParser configuration
There's three major pieces of FlatFileParser configuration: the normal BatchInputFileType properties, the parse specification, and the processor injection.
...
The processor injection occurs on line 40. It's assumed that lurking somewhere else in spring-sys.xml, there is a bean named DocumentHolderFlatFileProcessor which implements the interface org.kuali.kfs.sys.batch.FlatFileDataHandler. This interface has two methods on it: validate and process, which correspond to BatchInputFileType's validate and process directly. When BatchInputFileType#validate is called, FlatFileParserBase will attempt to hand validation off to whatever its injected processor property is - in this case, DocumentHolderFlatFileProcessor. In our case, we would likely use the process method to create the documents specified in the file and attempt to save them.
FixedWidthFlatFileSpecification configuration
The next level is the file level. Here we need to make two choices: first, how do we know what elements in which lines should populate properties in our business objects? and secondly, what prefixes are associated wich what objects?
...
Note, finally, that we need not specify a defaultBusinessObjectClass if EVERY line has a prefix. In that case, we would have no need to generate a list of insignificantPrefixes either - if no defaultBusinessObjectClass is specified, then the parser will ignore lines with prefixes it cannot match. Instead, just have FlatFileObjectSpecification do all the class determinacy: as we're about to see.
FlatFileObjectSpecification configuration
Now, we're getting down to the object level and our parsing is really taking off. We're now at the point where a line gets turned into an object. We've got three points of configuration in our example above, at lines 6, 16, and 26, and from those we can see that there's a number of elements each configuration has in common.
...
Again, there's no theoretical depth limit on how many parents and children you can have (only the constraints of phyiscal memory). An object can have as many types of children objects as needed to parse the file, and it can be the child of some other parent as well. The FlatFileParser - specifically, the ParseTracker implementation - will attempt to correctly build the object graph based on this parent/child hierarchy.
FixedWidthFlatFilePropertySpecification configuration
We've got our line, we've got the object that it needs to be parsed into. Now the final step: finding the significant substrings in the line and populating the object based on those. Let's take a look at the first of these in our sample configuration above, line 9:
...
And with that, we've parsed our first flat file.
Delimited Files
Time for our next challenge. Let's say we had a file where every line looked like this:
...
Delimited files can use prefixes just like fixed width files.
Messages for logical files
Oftentimes, a flat file will be made up of a few major header objects with many children. It makes sense in these situations to organize errors and informational messages which are recording during validation around these header objects rather than the file as a whole. The FlatFileParser also includes support for that.
...
org.kuali.kfs.sys.batch.FlatFileTransactionInformation is simply a holder for errors and info messages. With this convenient holding, e-mails can be sent out to users responsible for each "transaction" object. For instance, our example above, we could e-mail the intended initiator of the document to tell them that validation is off.
Thoughts for improvements
Just some off the cuff thoughts for improvements here...
...