THE CHALLENGE

Harmonizing Legacy Lab Data for Research

The University Hospital Cologne (UKK) Medical Data Integration Centre (MeDIC) required a high-performance solution to migrate legacy laboratory results into the HL7 FHIR® standard. The objective was to transform massive volumes of proprietary CSV exports into high-fidelity FHIR Observation resources compliant with the German Medical Informatics Initiative (MII) Core Data Set and securely persist them into a production Firely server. While the project validated the workflow with an initial dataset, the architecture was designed to scale to the millions of diagnostic records required for a live production environment.

THE SOLUTION

Configuring a High-Performance Ignifyr Pipeline

We utilized Ignifyr as the central transformation engine, moving beyond simple script-based conversion to a managed, production-grade ETL architecture. By deploying Ignifyr within the hospital’s secure infrastructure, the team establishsed a “plug-and-play” environment where data transformation is decoupled from the underlying software code.

Intelligent Integration Workflow

By configuring specialized mapping jobs within the Ignifyr environment, we addressed the hospital’s specific infrastructure and security requirements:

• Automated Folder Watchers: We utilized Ignifyr’s native file-system watchers to trigger transformations automatically the moment new CSV exports were detected in the hospital’s data directories.
• Streaming ETL via Apache Spark: Utilizing Ignifyr’s underlying Spark-powered engine, we enabled the processing of massive clinical datasets via streaming, ensuring consistent high throughput and system stability even as data volumes scaled into the millions.
• Secured Connectivity & Identity Mapping: We developed mapping jobs within Ignifyr that utilized OAuth 2.0 for secure communication. Furthermore, we utilized Ignifyr’s “Identity Service” mode to automatically link new Observations to existing Patient and Encounter resources, ensuring clinical continuity.

Mastering Semantic Translation

We utilized Ignifyr’s advanced terminology and transformation features to handle the complex, non-standardized reality of hospital data:

• LOINC & UCUM Mapping: We configured Ignifyr’s integrated terminology service to map approximately 800 local laboratory codes to international LOINC standards. Simultaneously, hospital-specific units (such as RE/ml) were normalized to UCUM using Ignifyr’s built-in unit conversion functions to ensure profile compatibility.
• Standardizing Qualitative Results: For non-numeric results (e.g., “+”, “positive”), we utilized Ignifyr to implement specialized concept maps that parse text and output standardized SNOMED CT codes.
• Data Quality Guardrails: Leveraging Ignifyr’s flexible logic, the pipeline was configured to identify and handle invalid rows or missing units. This ensures that individual data errors flag notifications for the IT team rather than halting the entire ingestion process.

Operational “Quality of Life” Configurations

To ensure the system remains maintainable for the UKK IT team, we utilized several enterprise-ready features within Ignifyr:

• Intelligent Backlog Management: By configuring Ignifyr settings such as maxFileAge and latestFirst triggers, the hospital can prioritize the most recent clinical data while simultaneously processing massive historical backlogs in the background.
• Automated Data Lifecycle: We enabled Ignifyr’s archiveMode to automatically move or delete processed files, preventing data duplication and maintaining optimal storage performance.
• Containerized Deployment: The entire configuration—including mapping definitions and environment settings—was packaged as a containerized microservice for seamless deployment within the hospital’s in-house servers.

THE ımpact

A Foundation for Scalable Clinical Research

The UKK MeDIC deployment established a robust, automated FHIR pipeline that successfully transitioned from a pilot phase to a production-scale system. By externalizing the mapping logic, the hospital can now update the mapping scripts, medical terminologies or MII profiles independently, without modifying the core software.

• Engineering Rigor: Replaced ad-hoc transformation scripts with a versioned, monitored, and fully automated pipeline.
• Clinical Fidelity: Achieved 100% data retention by utilizing Ignifyr to preserve local codes whenever direct international matches were unavailable.
• Production Scalability: Established a technical blueprint that has successfully scaled to process millions of laboratory records, providing a high-availability data source for the national research network.

SEE OUR PACKAGES

Do you want to transform your health data into FHIR?

Discover our implementation packages designed to support healthcare organizations in building scalable FHIR pipelines. Or contact us to discuss your data integration challenges and project requirements.