Staff Data Engineer

Job Summary:

We are seeking a highly experienced Staff Data Engineer to lead the design, development, and optimisation of our data architecture, pipelines, and workflows. This role will serve as a technical lead within the organisation, setting best practices, mentoring team members, and solving complex data challenges to enable data-driven decision-making at scale.

As a Staff Data Engineer, you will collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to design systems that transform raw data into actionable insights while ensuring scalability, security, and reliability.

Key Responsibilities:

Technical Leadership

• Design, and implement scalable and reliable data pipelines, ensuring the processing of large volumes of structured and unstructured data.
• Define and enforce data engineering best practices, coding standards, and architectural principles across teams.
• Conduct code reviews and provide mentorship to junior and senior data engineers.

Data Pipeline Development

• Build and maintain batch and real-time data pipelines using tools such as Apache Spark, Kinesis, and AWS services.
• Works with multiple teams to coordinate the event-driven architecture, managing inter-dependencies and promoting consistency.
• Ensure data quality, governance, and security by implementing monitoring, validation, and compliance tools.

Collaboration & Cross-Functional Engagement

• Partner with product, analytics, and data science teams to understand business requirements and translate them into technical solutions.
• Work closely with DevOps and software engineering teams to deploy and maintain production-ready data infrastructure.

Innovation & Scalability

• Evaluate and recommend emerging technologies and frameworks to ensure the data platform remains future-proof.
• Drive initiatives to improve the performance, scalability, and efficiency of existing systems.

Required Skills & Experience

• 12+ years of experience in data engineering field, with at least 2 years in a senior or staff-level role.
• Expertise in designing and implementing scalable data architectures for big data platforms.
• Strong programming skills in Python, Scala.
• Deep experience with distributed data processing systems such as Apache Spark, Databricks, Delta Lake.
• Proficiency with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (Dynamo).
• Strong understanding of ETL/ELT workflows, data warehousing concepts, and modern data lake architectures.
• Employ the established Data Governance model to sustain Data Quality for the data objects and implement the necessary operating mechanisms to ensure compliance
• Knowledge of CI/CD practices.
• Excellent problem-solving skills and the ability to design creative, efficient solutions for complex data challenges.
• Background in AI, machine learning pipelines is a plus
• Proactive, self-driven, and detail-oriented with a strong sense of ownership.