This role involves managing petabytes of data and designing and implementing new frameworks to build scalable and efficient data processing workflows. The successful candidate will be responsible for ensuring the completeness of all data ingestion and full metadata enrichment covering data classification annotations, dataset descriptions and all essential required tagging, while optimizing for performance and scalability. You will also be responsible for monitoring the performance of the system, optimizing it for cost and efficiency, and solving any issues that arise. This is an exciting opportunity to work on cutting-edge technology and collaborate with cross-functional teams to deliver high-quality software solutions. The ideal candidate should have a strong background in software development, experience with public cloud platforms, and familiarity with distributed databases.