Software Engineer, Gemma

Snapshot

Gemma is a family of open weights large language models (LLM), built and released for free by Google DeepMind. The Gemma team is looking for experienced senior software engineers to help us build the training, data, and analysis infrastructure for newer, better open LLMs. 

Experience in machine learning is not required.

About Us

Artificial Intelligence could be one of humanity’s most useful inventions. At Google DeepMind, we’re a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority.

The Role

As a software engineer in Gemma, you will be a part of the pretraining team, which is responsible for training our foundation models. You will be exposed to all parts of the training process, from data ingestion to testing our final checkpoints.

Key responsibilities:

• Building data quality filtering pipelines
• Improving our knowledge distillation infrastructure 
• Understanding and solving large-scale TPU communication issues
• Deep diving into XLA compiler optimizations and their impact 
• Setting a standard for code health, testing, etc.
• Prototyping model training ideas alongside researchers
• Adding new evals to our evaluation suite
• Designing and building novel data analysis tools
• Work on our OSS infrastructure and tooling

About You

In order to set you up for success as a Software Engineer at Google DeepMind,  we look for the following skills and experience:

• building and maintaining large scale software systems
• distributed systems
• profiling & optimization
• experience with both Python and statically-typed programming languages (we mostly program in Python)
• writing design documents and code review
• a collaborative attitude
• willingness to adapt to a research environment
• strong communication skills needed for a distributed team

In addition, the following would be an advantage: 

• experience with machine learning or LLMs (note that is not a requirement)
• solving complex algorithmic problems
• working in an academic or research environment
• mentorship
• working on OSS

Application deadline - 14th March

At Google DeepMind, we value diversity of experience, knowledge, backgrounds and perspectives and harness these qualities to create extraordinary impact. We are committed to equal employment opportunity regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, pregnancy, or related condition (including breastfeeding) or any other basis as protected by applicable law. If you have a disability or additional need that requires accommodation, please do not hesitate to let us know.