Databricks with Python and Cloud

Your Role

• 7+ years of experience in data engineering, data platforms & analytics, 10+ years of consulting experience
• Minimum 6-8+ projects delivered with hands-on experience in development on Databricks
• Strong hands-on experience in Python programming. Expertise in:
• Pandas (data manipulation, transformation, analysis)
• NumPy (array operations, numerical computing)
• Design, develop, and optimize scalable data pipelines on Databricks using Apache Spark (PySpark/Scala), ensuring high performance and reliability for batch and real-time processing.
• Implement data engineering best practices including Delta Lake, data modeling, partitioning, and performance tuning; manage data workflows using Databricks Workflows or orchestration tools.
• Collaborate with data scientists and analysts to build and deploy machine learning models and analytics solutions, leveraging Databricks notebooks, MLflow, and Unity Catalog for governance.

Your Profile

• Ensure data quality, security, and compliance by applying monitoring, logging, access controls, and CI/CD pipelines (Azure DevOps/Git), supporting end-to-end data lifecycle management
• Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one
• Deep experience with distributed computing with Spark with knowledge of Spark runtime internals
• Familiarity with CI/CD for production deployments
• Working knowledge of MLOps
• Current knowledge across the breadth of Databricks product and platform features
• Familiarity with optimizations for performance and scalability
• Databricks certification is an added advantage

What will you love working at Capgemini

• You will have the opportunity to learn on one of the industry's largest digital learning platforms, with access to 250,000+ courses and numerous certifications.
• We’re committed to ensure that people of all backgrounds feel encouraged and have a sense of belonging at Capgemini. You are valued for who you are, and you can bring your original self to work.
• At Capgemini, you can work on cutting-edge projects in tech and engineering with industry leaders or create solutions to overcome societal and environmental challenges. 
• Capgemini office campuses in India are green and run on 100% renewable electricity. We have installed Solar plants across India locations and ‘Battery Energy Storage Solution’ (BESS) in the Noida and Mumbai campuses. You will have chance to make a difference everyday.