Data Engineer Lead

Back Data Engineer Lead American IT Systems Lisle, IL, United States 1 day ago Discuss this with your agent Open in Claude Open in ChatGPT Copy prompt Apply on the employer's site Prepare application Draft this with your agent Open in Claude Open in ChatGPT Copy prompt Role details Contract type Permanent contract Employment type Full-time (> 32 hours) Experience level Expert Experience required 1 year minimum Working hours Regular working hours Job source the employer's site Tech stack Application Programming Interfaces (APIs) Artificial Intelligence Data Analysis Business Logic Spreadsheets Information Systems Continuous Integration Data Deduplication Data Discovery Information Engineering Data Governance Data Integration +29 more Data Systems IBM DB2 Decision Support Systems Distributed Computing Environment Event-Driven Programming Apache Hadoop Python (Programming Language) Operational Data Store Operational Databases Reference Data Reverse Engineering SAP (Applications) Enterprise Data Management Pega Cloud Platform System Azure Data Factory Sql Optimization Palantir Foundry Change Data Capture Information Technology Data Lineage Drilldown Data Management Domain Driven Design Software Version Control Data Pipelines Legacy Systems Servicenow Databricks Job description Build trusted data products and pipelines. Design, develop and maintain reliable, reusable data products and pipelines that support service product creation, sales, contract management, billing, claims, downstream execution and Performance Management. Discover and translate business processes into data requirements. Profile unfamiliar datasets; map workflows and system handoffs; identify grain, keys, relationships and temporal behavior; clarify business rules; investigate exceptions; and define testable requirements and acceptance criteria with product and business partners. Establish domain and performance data models. Define consistent entities, relationships, identifiers and lifecycle states for service products, policies, eligibility, pricing, customers, vehicles, orders, contracts, coverage, billing, claims and maintenance plans. Align definitions and authoritative sources across applications and enterprise data platforms. Engineer preventive data quality controls. Partner with application teams to implement validation for product eligibility, required customer and vehicle data, effective dates and downstream setup. Place controls at the earliest appropriate point in the process. Establish trusted and reconciled data products. Reverse-engineer current operational and financial reporting from source systems through legacy DB2/Hadoop transformations and downstream spreadsheets. Build automated reconciliation between legacy and target state data, explain material variances, and establish trusted datasets with Finance, Pricing and business data owners. Operationalize exception management. Create monitoring, alerts and actionable exception queues with clear reason codes, ownership, ageing and resolution status. Connect recurring failures to root-cause analysis and product backlog improvements. Support business data stewardship. Work with named business and domain owners, and stewards to maintain definitions, quality rules, reference data and issue-resolution processes. Make business decisions and exceptions explicit, documented and traceable. Deliver actionable operational and performance information. Develop reusable data products and reporting that help teams understand service sales performance, transaction completeness, quality trends, billing and claims outcomes, exceptions and service readiness, with drill-down to the underlying records and source lineage. Integrate data across current and target-state systems. Connect transactional applications, APIs, events and enterprise data platforms across CAPM, SAP, legacy DB2 and other sources. Apply appropriate approaches to matching, deduplication, normalization, reference-data management and historical continuity. Support data-platform modernization. Map current-state data lineage and business logic across DB2, Hadoop and business-managed reporting to target-state sources including CAPM, SAP and remaining legacy systems. Identify data gaps, undocumented transformations and business rules that must be preserved, deliberately changed or retired during migration. Own production reliability and traceability. Establish testing, observability, lineage, documentation, version control and CI/CD practices. Ensure data can be traced from business output back through transformations to authoritative source records, and support production incidents and the technical lifecycle of the data solutions you build. Lead technical delivery. Influence solution architecture and engineering standards, mentor other practitioners, and coordinate delivery across Product, Engineering, Data and business teams. Measure data and business improvement. Define and monitor outcomes such as Finance-approved reconciliation, dat