18 min read

This tutorial explains how to implement shadow traffic techniques for safely rolling out new machine learning models in e-commerce applications. Prerequisites include knowledge of machine learning deployment and A/B testing.

20 min read

This tutorial focuses on assessing the reliability of tool-calling mechanisms in IT support systems during peak loads. Prerequisites include familiarity with load testing and IT support workflows.

15 min read

This tutorial covers best practices for managing Personally Identifiable Information (PII) in retrieval pipelines within healthcare settings, ensuring compliance with regulations like HIPAA. Prerequisites include understanding of data privacy laws and basic retrieval pipeline concepts.

18 min read

This tutorial provides a framework for assessing the reliability of tool-calling in RAG systems under high load conditions, specifically for IT support applications. It requires knowledge of system performance metrics and load testing methodologies.

16 min read

This tutorial focuses on best practices for handling Personally Identifiable Information (PII) in RAG pipelines within legal firms. It requires knowledge of legal compliance and data protection standards.

14 min read

This tutorial explores the cost implications of batching versus streaming token usage in RAG systems for financial services. It requires familiarity with RAG tokenization and financial data processing.

12 min read

This tutorial examines the trade-offs between structured outputs and JSON mode in RAG systems tailored for e-commerce applications. It requires a basic understanding of RAG and JSON data formats.

15 min read

This tutorial covers the design of golden sets for ensuring RAG (Retrieval-Augmented Generation) faithfulness in healthcare applications. It requires an understanding of RAG principles and access to domain-specific datasets.

10 min read

Using multimodal prompts can improve document question answering (QA) in legal contexts. Prerequisites include access to relevant legal documents and a model capable of processing multimodal inputs.

11 min read

Implementing citation constraints can significantly reduce hallucinations in research-oriented models. Prerequisites include a robust database of citations and a model capable of handling constraints.

9 min read

Establishing latency budgets can enhance the user experience in customer support chat applications. Prerequisites include understanding user expectations and system capabilities.

12 min read

Monitoring embedding drift is crucial for financial services to ensure model accuracy over time. Prerequisites include a data pipeline that captures embeddings and a monitoring framework.

10 min read

Implementing shadow traffic allows e-commerce platforms to test new models against live traffic without affecting user experience. Prerequisites include a robust logging mechanism and a dual model setup.

10 min read

This tutorial explores the differences between offline and online evaluation methods for machine learning models, focusing on their respective benefits and drawbacks. Prerequisites include a basic understanding of machine learning evaluation metrics and experience with model deployment.

20 min read

Learn how to tune pgvector indexes using HNSW and IVF algorithms for optimal performance. Prerequisites include familiarity with PostgreSQL and vector databases.

11 min read

Explore how to create effective multimodal prompts for document question answering (QA) systems. Prerequisites include understanding of multimodal models and QA frameworks.

18 min read

Establishing Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for generative endpoints is crucial for maintaining quality and reliability. This tutorial outlines how to define and implement SLIs/SLOs effectively.

12 min read

This tutorial discusses the trade-offs between offline and online evaluation frequencies for machine learning models, focusing on their impact on model performance and user experience.

18 min read

This tutorial explains the planner-executor loop in AI systems and how to implement effective failure recovery strategies. Prerequisites include knowledge of AI planning algorithms and system design.

20 min read

This tutorial compares scratchpad memory and vector store memory in AI agents, focusing on their use cases and performance characteristics. Prerequisites include a basic understanding of AI memory architectures.

15 min read

This tutorial outlines how to create effective runbooks to address overnight quality regressions in software systems. Prerequisites include familiarity with incident management and basic scripting skills.

16 min read

Utilizing canary prompts to detect regressions in language models. Prerequisites include familiarity with regression testing and LLM evaluation metrics.

22 min read

Developing strategies to protect multi-tenant applications from prompt injection attacks. Prerequisites include understanding of security vulnerabilities and multi-tenant architecture.

18 min read

Implementing observability practices to trace interactions between large language models (LLMs) and external tools. Prerequisites include knowledge of observability tools and LLM architectures.