Role : Azure Data Engineer - Lead
Exp: 8+ yrs
Loc: Bangalore / Noida (Hybrid)
Job Description :
- Data Engineering with Databricks:
- Design, implement, and optimize data pipelines in Databricks to process large-scale structured and unstructured data.
- Develop ETL workflows for data ingestion, transformation, and integration using Databricks and other cloud-based tools.
- Implement Delta Lake for efficient data management and real-time data processing.
- Collaborate with data architects and stakeholders to build scalable and reusable data solutions.
- Leverage Databricks' MLflow for managing and deploying machine learning workflows, if required.
- Ensure data quality, integrity, and compliance with organizational standards and governance policies.
Required Skills & Qualifications:
- Education: Bachelor’s or master’s degree in computer science, Data Analytics, Information Technology, or a related field.
- Technical Proficiency:
- Hands-on experience with Databricks: Spark, Pyspark, Scala, or SQL
- Strong understanding of database concepts and working experience with SQL/NoSQL databases.
- Analytical Skills: Strong ability to analyse, interpret, and visualize large datasets.
- Problem Solving: Demonstrated ability to troubleshoot complex data workflows and reporting challenges.
- Soft Skills: Strong written and verbal communication, collaboration, and stakeholder management.