Flat File Parser
Ingest CSV, TSV, and fixed-width flat files from laboratory instruments — including legacy analyzers and LIS exports that don't speak HL7 — through template-based column-to-HL7 mapping. Each row converts into a valid HL7 ORU^R01 result message ready for auto-verification, routing, and delivery.
How It Works
- Upload a flat file in CSV, TSV, or fixed-width format through the Interface Analyzer in the dashboard
- Vorian parses the file structure — header detection, delimiter identification, and column boundary inference — and surfaces a preview of the detected table layout
- An interface engineer creates or selects a mapping template that defines three things: how to parse the file (delimiter, skip rows, footer rows, fixed-width column positions), default HL7 segment values (sending application, facility), and a column-to-HL7 field map (which CSV column maps to PID.3, OBX.3, OBX.5, OBX.6, etc.)
- The parser reads each data row and transforms it into a complete HL7 ORU^R01 message with MSH, PID, OBR, and OBX segments
- Value types are auto-detected: numeric columns produce
NM, date columns produceDT, text columns produceST - Each generated HL7 message is persisted to the message log and flows through the standard auto-verification, routing, and delivery pipeline
- Ingestion runs are logged with row counts, success/error rates, and per-message traceability
What's Included
- CSV parsing — comma-delimited files with optional quoted fields, configurable skip rows and footer rows, automatic header detection
- TSV parsing — tab-delimited files using the same configurable parsing pipeline as CSV
- Fixed-width parsing — column positions defined as character offsets (start, end pairs). Fixed-width files don't carry headers; the template alone defines the column layout
- Mapping templates — reusable, per-instrument-model configurations stored in the database. Each template captures: file format instructions (delimiter, skip/footer rows, column positions), HL7 segment defaults (sending application in MSH.3, patient identifiers in PID fields), and column-to-HL7 mappings (column name or index → HL7 field path like OBX.3 or OBX.5)
- Template lifecycle — full CRUD for templates (create, read, update, delete) scoped to the tenant. Templates are ordered by vendor and instrument model for quick lookup
- Ingestion logs — every file ingestion records the template used, filename, row count, success and error counts, and status (processing, completed, failed)
- Interface Analyzer integration — pasted CSV, TSV, and fixed-width samples in the Interface Analyzer surface produce the same classification, proposal staging, and review workflow as HL7 and ASTM samples. Column boundaries are inferred from spacing for fixed-width preview
Where It Fits
The Flat File Parser lives in the Interface Analyzer workflow alongside the HL7 parser and ASTM parser. It targets instruments and LIS exports that produce tabular data instead of standard HL7 — common with legacy hematology analyzers, flow cytometers, microbiology systems, and regional reference lab exports.
The Interface Analyzer auto-detects the format when you paste a sample: CSV flat file, TSV flat file, or Fixed-width flat file. Non-HL7 samples go through the same review posture as HL7 samples — staged as mapping proposals with test code, condition, and reference range candidates — before a reviewer approves them for production use.
After a template is created and validated, batch ingestion runs the full pipeline: template lookup → file parse → HL7 transform → database persist → downstream delivery. The same auto-verification rules, routing policies, and destination delivery that apply to native HL7 traffic also apply to flat-file-generated messages.
Supported Formats
- CSV — comma-separated values with quoted field support and lazy quoting for vendor variants
- TSV — tab-separated values, same parsing pipeline as CSV
- Fixed-width — character-position-based columns defined by
(start, end)byte offset pairs
Template Configuration
Each mapping template has three configuration layers:
- File config — delimiter, skip rows, footer rows, and fixed-width column positions. These instructions tell the parser how to read the file
- HL7 config — default values for MSH and PID segment fields that aren't derived from the flat file (e.g. sending application
MSH.3, patient namePID.5) - Field maps — an ordered list of column-to-HL7 mappings. Each entry specifies a column name or index and the target HL7 field (PID.3 for patient ID, OBR.4 for test code, OBX.3 for observation identifier, OBX.5 for result value, OBX.6 for units, OBX.8 for abnormal flags)
Templates are reusable — once validated for an instrument model, the same template ingests any file from that vendor and instrument. Updates to a template take effect on the next ingestion without affecting previously stored messages.