Zero Tolerance for Failure: Rebuilding a Seven-Hospital ETL Pipeline Mid-Crisis

The situation

The data team at a seven-hospital network had been fighting the same war every morning for eleven weeks. The Caboodle ETL — the overnight pipeline that loads Epic clinical and operational data into the enterprise analytics environment — was failing. Not occasionally. Routinely.

When the ETL failed, it did not fail quietly. Downstream reporting jobs that depended on fresh data simply did not run. Quality dashboards serving seven hospitals went dark. Revenue cycle reports that should have been in leadership inboxes by 7:00 AM were not there. Clinical operations teams were making decisions on day-old data. In a seven-hospital system where executives rely on near-real-time analytics for capacity management and throughput decisions, day-old data is not a minor inconvenience. It is an operational problem.

The organization’s internal team had applied three rounds of fixes. None held. They had opened a high-priority case with Epic; the support ticket was active but moving slowly. They needed someone who could walk in, diagnose the root cause rather than the symptoms, and rebuild the pipeline to a standard it had not had before.

Challenge and approach

The broader clinical impact

Data infrastructure failures are rarely just IT problems. When the Caboodle ETL failed, the downstream effects touched clinical operations directly: capacity managers were flying blind, throughput decisions were based on yesterday’s census, and quality reporting gaps created risk for an organization under continuous regulatory scrutiny.

The stabilization APSIS delivered was not just a technical fix. It restored the analytical foundation the health system’s clinical and operational leadership depends on to run seven hospitals. That is a different kind of outcome — and it is the kind that only comes when the team solving the problem understands both the technical architecture and what the data is actually used for.

Results

The night the rebuilt pipeline ran for the first time, every job completed successfully. The following morning, quality dashboards were populated, revenue cycle reports were in leadership inboxes before 7:00 AM, and the data team — which had been on morning crisis calls for eleven weeks straight — had a normal day.

The monitoring infrastructure APSIS implemented changed the operational posture entirely. Instead of discovering failures after clinical staff called in, the data team now receives alerts within 60 seconds of a job anomaly. Trend data on runtime allows them to anticipate degradation before it becomes failure.

The seven-hospital network retained APSIS for ongoing Caboodle managed services support following the engagement — a direct result of the confidence built during the stabilization work.

APSIS capabilities demonstrated