Senior Data Engineer

Job Description

The Senior Data Engineer will build and operate reliable data pipelines and curated data assets The role focuses on agency servicing and distribution use cases by enabling high quality governed data products across ingestion transformation validation and deployment stages working closely with Data Product Owners, Data Modelers, Business Analysts, QA, MLOps, CICD LBU technology teams and implementation partners

Key Responsibilities
Data pipeline delivery Build and maintain scalable batch and near real time data pipelines from source systems into curated data layers ensuring reliability auditability and operational side.
Data product implementation Implement data assets required for Agency Data Products including mapping source to target fields applying transformation logic creating reusable datasets and downstream analytics AI consumption
Data quality and controls Embed data profiling validation checks reconciliation logic exception handling and monitoring for reliable data products
Security privacy and compliance Apply required access controls data masking PII handling retention and governance standards in collaboration with Data Governance and Architecture teams
Engineering standards Follow engineering best practices for version control modular code review automated CICD deployment readiness and production handover
Squad collaboration Translate user stories and data requirements into technical and delivery tasks clarify dependencies with business technology data modelers and QA teams
Defect resolution and Investigate data defects performance bottlenecks and production issues provide root cause analysis and permanent fixes
Documentation Maintain technical data lineage data dictionary source mapping runbooks deployment notes and operational documentation
Mentorship Guide mid-level engineers on coding standards review pull requests and unblock technical issues within the squad

Job Description - Grade Specific

1 Data ingestion and transformation pipelines aligned to approved data product backlog
Enable Agency data products for servicing distribution use cases
2 Source to target mapping transformation rules and data lineage documentation
Provide traceability from source systems to curated layer and data product consumption
3 Data quality rules validation reports and reconciliation outputs
Prove completeness accuracy and readiness for downstream test and adoption
4 Unit tested peer reviewed code with deployment and rollback instructions
Help controlled release management and engineering quality
5 Operational runbooks monitoring approach and production handover notes
Enable sustainable production operations after delivery