Senior Software Engineer

Microsoft’s bold vision of Azure Machine Learning (ML) is to democratize ML and make it available to every enterprise, developer and data scientist. Do you want to join a team that builds the platform to run all internal and external ML workloads? We are already serving billions of requests per day on the most cutting-edge scenarios and models across the industry.

You will be joining the Inference team that builds the model-serving platform for large models including the OpenAI generative models. We host models at the scale of Bing and Office and solve exciting problems on the intersection of AI and Cloud. As part of the Inference team, you'll be working with the Front Door team, which oversees the Data plane layer that handles inference requests post model creation and deployment. Front Door is a multi-tenant ingress service, offering a range of admission control, routing customization, and observability options.

We are looking for a strong Software Engineer who is passionate about designing and building highly reliable, available platform to support model inferencing at scale. You will also be expected to work on high throughput/low latency scenarios, drive performance optimization capabilities, bringing parity between OAI and AOAI routing capabilities.