Senior Data Scientist

Share job

Overview:

Are you passionate about building a platform that leverages crowd? Do you love working with an energetic and cross-functional group of people? Does pioneering innovative ways to understand and interact with LLMs sound exciting to you? Are you passionate about quality of the products and want to contribute to product improvement driven by insights from crowd and LLM? If so, Crowd Intelligence Platform team might be an excellent place for you to grow your career.

The Team:

The Crowd Intelligence Platform team within Bing Fundamentals is the central team that evaluates the quality of products within Microsoft AI leveraging crowdsourcing and LLM’s. We have an ambition to scale to all products in Microsoft, providing necessary infrastructure to schedule & distribute a large number of product scenarios for validation. This system leverages the power of crowdsourcing to auto generate a user perceived quality score representative of the health of functionalities offered by the product, to enable teams to be in control of their feature quality and take pride in their product.

The service is expected to run 10s of thousands of scenarios every day for both webpages and apps starting with deployment of the right version of operating system/hosting app with appropriate settings and connecting with relevant judges across multiple crowdsourced platforms. In addition, the service will include capabilities to capture the validation process for future debuggability, built in intelligence to target high fidelity judges, ability to dynamically update the pricing for these tasks and support real time routing across platforms to optimize for cost. To name a few products this service is expected to support - Bing, Maps, Edge, Windows Feeds.

The Role:

We are seeking a Senior Data Scientist to join our Crowd Intelligence Platform team in Microsoft AI. The ideal candidate will have hands-on experience in Deep Learning, Machine Learning and Generative Large Language Models. In this role you will be responsible for developing a platform that will optimize and produce SLMs based on the labels from LLMs and human annotations. To optimize costs and quality, we expect many teams within Bing and Microsoft AI to go through this process of producing SLMs and we want to provide a platform that enables this easily. You will be responsible for developing prompts, introducing new features / capabilities, and enabling groundbreaking scenarios. Come play at the absolute cutting edge of artificial intelligence. You will find engineering challenges around big data and will have the opportunity to work with world class data scientists. You will work across our stack, from our public-facing React-based UX, to our Azure-hosted backend, analyzing data with the latest available tools. You will impact the latest products across Microsoft AI, used by billions of people.

Required Qualifications:

• Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
  • OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
  • OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
  • OR equivalent experience.
• Proven experience with training, deploying, evaluating large scale models, preferably for consumer products.
• Experience in programming languages such as C++, C#, Java, or Python

Preferred Qualifications:

• Familiarity with machine learning frameworks and libraries (e.g., TensorFlow, PyTorch) Knowledge of model deployment and integration in production environments.
• Familiarity with cloud platforms and services (e.g., Azure, AWS).
• Analytical and problem-solving skills.
• Ability to work collaboratively in a team environment.

Data Science IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

Microsoft will accept applications for the role until December 22, 2024.

• Evaluate the performance of new LLM and SLM models through rigorous testing and analysis.

• Validate the accuracy, efficiency, and reliability of these models in various applications.

• Collaborate with cross-functional teams to integrate validated models into production systems.

• Analyze model outputs and provide detailed reports on performance metrics.

• Stay up to date with the latest advancements in AI, including reinforcement learning, and data analytics.

• Incorporate best practices into the evaluation process.

• Troubleshoot and resolve issues related to model performance and integration.