Operationalize Machine Learning and Generative AI Solutions (AI-300T00)

Course Duration

4 Days

Audience

Employees of federal, state and local governments; and businesses working with the government.

Prerequisites

Experience with Python, a foundational understanding of machine learning concepts, and basic familiarity with DevOps practices such as source control, CI/CD, and command-line tools.

Course Description

This course prepares learners to design, implement, and operate Machine Learning Operations (MLOps) and Generative AI Operations (GenAIOps) solutions on Azure. It covers building secure and scalable AI infrastructure, managing the full lifecycle of traditional machine learning models with Azure Machine Learning, and deploying, evaluating, monitoring, and optimizing generative AI applications and agents using Microsoft Foundry. Learners gain hands-on knowledge of automation, continuous integration and delivery, infrastructure as code, and observability using tools such as GitHub Actions, Azure CLI, and Bicep.

Learning Objectives

• Design and implement MLOps pipelines for machine learning models on Azure
• Use Azure Machine Learning to train, track, and deploy models with MLflow
• Automate model training and deployment using GitHub Actions
• Build and manage GenAIOps workflows using Microsoft Foundry
• Apply version control and CI/CD practices to generative AI prompts and agents
• Evaluate and optimize AI agents using structured experiments and automated evaluations
• Monitor generative AI applications using Azure observability tools
• Debug and trace generative AI application behavior in production
• Implement responsible AI practices across the ML and generative AI lifecycle

Course Outline

Module 1: Experiment with Azure Machine Learning

• Preprocess data and configure featurization
• Run an automated machine learning experiment
• Evaluate and compare models
• Configure MLflow for model tracking in notebooks
• Train and track models in notebooks
• Evaluate models with the Responsible AI dashboard

Module 2: Perform Hyperparameter Tuning with Azure Machine Learning

• Define a search space
• Configure a sampling method
• Configure early termination
• Use a sweep job for hyperparameter tuning

Module 3: Run Pipelines in Azure Machine Learning

• Create components
• Create a pipeline
• Run a pipeline job

Module 4: Trigger Azure Machine Learning Jobs with GitHub Actions

• Explore the solution architecture
• Use GitHub Actions for model training

Module 5: Trigger GitHub Actions with Feature-Based Development

• Trigger a workflow

Module 6: Work with Environments in GitHub Actions

• Set up environments

Module 7: Deploy a Model with GitHub Actions

• Model deployment

Module 8: Plan and Prepare a GenAIOps Solution

• Explore use cases for GenAIOps
• Select the right generative AI model
• Understand the development lifecycle of a language model application
• Explore available tools and frameworks to implement GenAIOps

Module 9: Manage Prompts for Agents in Microsoft Foundry with GitHub

• Apply version control to prompts
• Organize prompts in GitHub repositories
• Develop safe prompt deployment workflows

Module 10: Evaluate and Optimize AI Agents Through Structured Experiments

• Design evaluation experiments
• Apply Git-based workflows to optimization experiments
• Apply evaluation rubrics for consistent scoring

Module 11: Automate AI Evaluations with Microsoft Foundry and GitHub Actions

• Create evaluation datasets
• Implement batch evaluations with Python
• Integrate evaluations into GitHub Actions

Module 12: Monitor Your Generative AI Application

• Understand key metrics to monitor
• Integrate monitoring into your app
• Interpret monitoring results

Module 13: Analyze and Debug Your Generative AI App with Tracing

• Implement tracing in generative AI applications
• Debug complex workflows with advanced tracing patterns
• Make informed decisions with trace data analysis

Frequently Asked Questions