As organizations move beyond experimentation to production deployment of GenAI applications, Kubernetes has emerged as a preferred platform for managing inference workloads at scale. In this hands-on workshop, you'll learn how to build and scale production-ready Generative AI deployments on Amazon EKS using NVIDIA GPUs.
Â
You'll gain practical experience with EKS cluster setup optimized for GPU workloads, efficient model serving and scaling using vLLM, and distributed inference architecture implementation with Ray. Master comprehensive monitoring and observability using Prometheus and Grafana, while learning best practices for production GenAI deployments on Kubernetes from AWS experts who have helped organizations successfully deploy and manage large-scale GenAI infrastructure
This workshop is designed for developers, DevOps engineers, solution/systems architects, and ML engineers looking to deploy and scale GenAI applications in production. Whether you're deploying your first language model or scaling existing GenAI workloads, this workshop will help you leverage AWS best practices for inference workloads on Kubernetes. Basic knowledge of Kubernetes is required.
Â
