Descripción de la oferta
Job description_About the Role_Join our client and help bring AI-driven quality control to the data pipelines that power their business. You’ll analyze how ETL processes are validated today and design an AI-powered validation platform on AWS that detects anomalies, reconciles source and target data, and explains issues before they reach downstream consumers. This role blends data engineering, machine learning / anomaly detection, and generative AI (Amazon Bedrock), giving you the opportunity to shape both the data quality and governance strategy and the roadmap for AI-driven validation across additional workloads.A day in this role_Analyze existing ETL validation processes and identify automation opportunities.Develop AI-driven validation frameworks for data pipelines.Implement anomaly detection for continuous data quality monitoring.Compare source data against ETL outputs to identify and track discrepancies.Automate record count, completeness, and consistency checks across pipelines.Generate AI-assisted validation reports that explain detected issues in clear, actionable terms.Integrate AI validation capabilities into AWS data platforms and workflows (Glue, Lambda, Redshift, Athena, S3).Evaluate how AI can be used to deliver in-depth QA analysis of deliverables.Collaborate with data engineering, QA, and business teams to define data quality rules, thresholds, and SLAs.AI-powered data validation platform.Automated ETL quality control framework.Anomaly detection models and monitoring dashboards.Validation reports and root-cause analysis capabilities.Data quality KPIs and governance recommendations.Roadmap for expanding AI-driven validation across additional workloads.The expertise requested_Bachelor’s degree in Computer Science, Data Engineering, Software Engineering, or related field (or equivalent experience).5+ years of experience in data engineering, ETL development, or data quality roles.Strong programming skills in Python and advanced SQL.Hands-on experience with Amazon Bedrock and building generative AI solutions (prompt design, LLM-based analysis and reporting).Hands-on experience with AWS Lambda, AWS Glue, Amazon Redshift, Amazon Athena, and Amazon S3.Practical experience with machine learning and anomaly detection techniques applied to data (statistical, time-series, or ML-based).Solid understanding of data quality and data governance principles (completeness, accuracy, consistency, lineage).Experience designing source-to-target reconciliation and ETL testing strategies.Solid understanding of Agile, SDLC, and modern engineering best practices.Desired or nice to have expertise_Experience with data quality frameworks such as AWS Glue Data Quality, Great Expectations, or Deequ.Experience building monitoring dashboards (Amazon QuickSight, CloudWatch, or similar).Familiarity with Amazon SageMaker or other ML platforms for model training and deployment.Exposure to agentic AI workflows and AI-assisted development tools (e.g., GitHub Copilot, Claude Code, Cursor).AWS certifications (e.g., Data Engineer – Associate, Machine Learning – Specialty)Our Core values are focus to inclusion and diversity, all qualified applicants will be considered for employment and will go thru a fair recruitment process regardless of their race, religion, gender identity, sexual orientation, national origin or disability status.Your journey with us begins here!!
