At Abbott, I served as Technical Program Manager, Data & ML (contract), on a $12 million data modernization program. This case covers its planned migration of historical diagnostic instrument data from an on-premises Oracle analytics warehouse to an AWS data lake. The work had to preserve completeness, schema integrity, customer linkages, and validation evidence while the legacy warehouse and ongoing feeds remained available.
Planning a diagnostic data migration from Oracle to AWS
- AWS · Historical data migration · Computerized system validation · Cross-team delivery
- Technical program manager$12M program budget16 months
Challenge
The future AWS feeds did not contain the complete historical record held in two Oracle schemas. The program therefore needed a defensible historical foundation without interrupting live operations or mixing unvalidated data into production.
- Terabyte-scale historical data
- Separate migration, quality-assurance, and production environments
- Validation for completeness, schema, keys, transformations, and customer linkages
Historical recordRecentOngoing
Oracle schema 1
Oracle schema 2
Future AWS feeds
Gap: the historical record exists only in Oracle. It must be migrated and validated before the legacy warehouse retires.
Terabyte-scale historyMigration → QA → Production environmentsLegacy warehouse stays live
Action
I structured the program through explicit gates: scope and mapping, extraction, normalization, enrichment, quality assurance, promotion, and operational handoff. Engineering owned the code; I coordinated the sequence, dependencies, approvers, evidence, risk controls, and excluded downstream scope.
The engineering path used controlled Oracle extracts, intermediate files, Spark and Scala normalization into partitioned Parquet, customer linkage, and governed movement through AWS environments.
| Gate | Evidence required | Owner | Approver | Exit criterion |
|---|---|---|---|---|
| 01 · Scope & mapping | Source-to-target mapping, excluded downstream scope | Engineering | Data owner | Mapping signed off |
| 02 · Extraction | Controlled Oracle extracts, row counts | Engineering | Program lead | Counts reconcile to source |
| 03 · Normalization | Spark / Scala to partitioned Parquet | Engineering | Architecture | Schema conforms |
| 04 · Enrichment | Customer linkage results | Engineering | Data owner | Linkage exceptions resolved |
| 05 · Quality assurance | Completeness, schema, keys, transformations, linkages | QA | Quality | All checks pass in QA |
| 06 · Promotion | Movement between AWS environments | Platform | Change board | Approved for promotion |
| 07 · Operational handoff | Runbook, support model | Operations | Service owner | Support accepted |
Result
The documented outcome was a detailed migration and validation plan with revised sequencing after a logged defect. Two instrument workstreams moved by approximately four weeks, making the dependency visible instead of allowing affected work to advance without updated validation dates.
Across the wider program, which supported approximately 27,000 Architect and Alinity instruments, data retrieval became 65% more efficient, one delivery cycle shortened from about six months to three weeks, and an FDA inspection was completed with zero audit findings.
For this migration, the available evidence does not establish a final production cutover, so this case makes no such claim. It demonstrates the controlled delivery path and validation plan needed to make a future production decision defensible.
W1W2W3W4W5W6W7W8W9W10W11W12
Instrument workstream 1
Instrument workstream 2
Other workstreams
Logged defect
Logged
Original planRevised planValidation dates moved with the work; no production cutover claimed.
Reference architecture · Amazon Web Services
Diagnostic data migration from Oracle to AWS
A validated, phased move of on-premises Oracle schemas and instrument data to managed AWS services. Schema conversion, change data capture, and reconciliation run as separate, evidenced steps so cutover is a controlled decision rather than a single event. See all four reference architectures.
AWS account · Amazon VPC (private subnets, multiple Availability Zones)
On-premises Oracle
Schema 1Operational
Schema 2Reference
PL/SQL & jobs
VPN / DX
Lab & instrument data
Diagnostic instruments
Instrument file shares
Lab middleware
DataSync
Partners & reference
Reference data
Service partners
SFTP
Convert & migrate
DMS Schema ConversionSchema & code
AWS DMSFull load + CDC
AWS DataSyncFile transfer
AWS Transfer FamilySFTP
Operational store & data lake
Aurora PostgreSQLMulti-AZ target
Amazon S3
- Raw
- Validated
- Curated
Glue Data Catalog
Lake FormationAccess grants
Validate & reconcile
DMS data validationRow-level compare
Glue Data QualityRules
Step FunctionsReconciliation runs
AWS LambdaChecks & alerts
Load
Verify
Applications
Diagnostic appsAmazon ECS
Route 53Cutover switch
Serve
Lab & service teams
Analytics & reporting
Amazon Athena
Amazon QuickSight
Serve
Quality & regulatory
Cutover evidence
Validation reportsS3 Object Lock
Runbook & sign-off
Serve
Migration & QA leads
Security & operations
AWS IAM
AWS KMS
Secrets Manager
AWS CloudTrail
Amazon CloudWatch
AWS Config
Amazon GuardDuty
AWS Backup
100%
Drag to pan · Ctrl/⌘ + scroll to zoomDrag to pan · Pinch to zoom
Convert & migrate
DMS Schema Conversion translates Oracle schemas and PL/SQL; DMS runs a full load followed by change data capture. DataSync and Transfer Family bring instrument and partner files.
Operational store & data lake
Aurora PostgreSQL becomes the system of record. Files move through raw, validated, and curated S3 zones, cataloged in Glue and governed by Lake Formation.
Validate & reconcile
DMS validation compares source and target row by row; Glue Data Quality applies business rules; Step Functions runs repeatable reconciliation and logs each defect.
Serve & cut over
Applications repoint to Aurora behind a Route 53 switch. Validation reports are kept immutable in S3 as the evidence for go and rollback decisions.
Security & operations
IAM, KMS encryption, and Secrets Manager protect data in motion and at rest; CloudTrail, Config, GuardDuty, and CloudWatch provide audit and monitoring.
