Data Engineer
At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world’s most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and engineering services across all industries. Join us for a career full of opportunities. Where you can make a difference. Where no two days are the same.
Your Role
- 5+ yrs of total IT experience and 4+ yrs of relevant experience as Azure Data Engineer and having good hands-on experience on Azure Data Factory, Azure Databricks, Python, Azure Synapse, Azure Data Lake, PySpark, SQL Server and Power BI.
- Knowledge in design of Extract, Transform and Load environment using Data bricks, Spark Azure Data Factory (ADF).
- Setting Up Microsoft Azure with Data bricks, Data bricks Workspace for Business Analytics, Manage Clusters in Data bricks.
- Hands on experience Data extraction (extract, Schemas, corrupt record handling and parallelized code), transformations and loads (user defined functions, join optimizations) and Production (optimize and automate Extract, Transform and Load).
- Copy of data from on-premises data (SQL Server) to Azure Data Lake Store (ADLS) using Azure Data Factory (ADF V2).
- Data migrated using Azure Data Factory and created new pipelines, data flows.
- Hands on experience on Migrating SQL database to Azure Data Lake, Azure datalake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access.
- Experience in working with Data bricks notebooks for developing and executing transformation logic and data processing tasks. Extensive experience in designing ETL pipelines using Data bricks, Apache Spark, and Azure Data Factory (ADF).
Your Profile
- Proficient in data extraction, schema management, and handling corrupt records with parallelized processing.
- Expertise in transformations and data loads, including optimization of joins and implementing user-defined functions (UDFs).
- Created Data bricks notebooks for data transformation, using various transformation processes such as join, merge, lookup, filter, remove duplicates, and aggregation.
- Designed ADF pipelines to execute Databricks notebooks and carry out complex transformations.
- Carryout transformations and creating notebooks using various transformation processes like Join, Merge, Lookup, remove Duplicates, Filter and Aggregator.
- Utilized spark transformations in Databricks for data cleaning, filter in and transformation tasks.
- Experience in troubleshooting SQL issues by using SQL Tools, execution plans, Trace, statistics, index tuning wizard.
What will you love working at Capgemini
- You will have the opportunity to learn on one of the industry's largest digital learning platforms, with access to 250,000+ courses and numerous certifications.
- We’re committed to ensure that people of all backgrounds feel encouraged and have a sense of belonging at Capgemini. You are valued for who you are, and you can bring your original self to work.
- At Capgemini, you can work on cutting-edge projects in tech and engineering with industry leaders or create solutions to overcome societal and environmental challenges.
- Capgemini office campuses in India are green and run on 100% renewable electricity. We have installed Solar plants across India locations and ‘Battery Energy Storage Solution’ (BESS) in the Noida and Mumbai campuses. You will have chance to make a difference everyday.
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.
Bangalore, IN