sagemaker-ai

name: sagemaker-ai description: > Comprehensive SOP for Amazon SageMaker AI — SDK v3 code generation, serverless model customization (SFT, DPO, RLVR, RLAIF), LLM inference endpoints, model training (classical ML + LLM fine-tuning), HyperPod clusters (training + inference), Model Monitor, and AutoML with AutoGluon. Generates correct, working code every time. Triggers: "sagemaker", "training job", "deploy model", "endpoint", "llm inference", "vllm", "model customization", "djl", "fine-tune", "qlora", "lora", "rlvr", "rlaif", "dpo", "sft", "reward function", "hyperpod", "model monitor", "data quality", "bias", "autogluon", "jumpstart", "deploy on hyperpod", "training script", "inference script". argument-hint: "[topic: train|deploy|hyperpod|monitor|autogluon] [framework: xgboost|sklearn|pytorch|llm]"

allowed-tools: Read, Write, Bash, Glob, Grep

SageMaker AI — Standard Operating Procedure

You are now operating as a SageMaker AI expert. Your job is to generate correct, working SageMaker Python SDK v3 code — never v2 estimators, never wrong imports, never wrong parameter names.

Read the relevant reference file for the user's task before generating any code.

1. Guardrails (Always-On)

These rules apply to EVERY piece of SageMaker code. Check each one before output.

Container Decision Tree

1. Standard text LLM (Llama, Mistral, Qwen) → DJL LMI with vLLM backend
2. Multimodal/Vision model (Idefics3, LLaVA, Qwen-VL) → DJL LMI with vLLM backend
3. Simple HF pipeline model (classification, NER) → HuggingFace Inference DLC
4. Custom model with custom handler → HuggingFace Inference DLC + custom inference.py
5. Classical ML (XGBoost, SKLearn, PyTorch) → Framework-specific DLC

CUDA Compatibility

2. Intent Detection + Routing

If ambiguous, ask ONE clarifying question.

3. Quick Reference

Reference Repositories

• Inference hosting examples: https://github.com/aws-samples/sagemaker-genai-hosting-examples
• Model customization recipes: https://github.com/aws-samples/amazon-sagemaker-generativeai/tree/main/0_model_customization_recipes

DLC Images

Always check: https://aws.github.io/deep-learning-containers/reference/available_images/

Key SDK v3 Imports

# Session + role
from sagemaker.core.helper.session_helper import Session, get_execution_role

# Training
from sagemaker.train import ModelTrainer
from sagemaker.core.training.configs import Compute, SourceCode, OutputDataConfig

# LLM Deployment (Core API)
from sagemaker.core.resources import Model, EndpointConfig, Endpoint
from sagemaker.core.shapes.shapes import ContainerDefinition, ProductionVariant

# Simple Deployment (ModelBuilder)
from sagemaker.serve import ModelBuilder
from sagemaker.serve.builder.schema_builder import SchemaBuilder

# Image URIs
from sagemaker.core import image_uris

# Model Monitor
from sagemaker.core.model_monitor import DefaultModelMonitor, ModelQualityMonitor, DataCaptureConfig

# Model Customization (Serverless)
from sagemaker.modules.train.sft_trainer import SFTTrainer
from sagemaker.modules.train.dpo_trainer import DPOTrainer
from sagemaker.modules.train.rlvr_trainer import RLVRTrainer
from sagemaker.modules.train.rlaif_trainer import RLAIFTrainer
from sagemaker.assets import DataSet
from sagemaker.train.evaluate import BenchMarkEvaluator, LLMAsJudgeEvaluator

# HyperPod
from sagemaker.hyperpod.inference.hp_jumpstart_endpoint import HPJumpStartEndpoint
from sagemaker.hyperpod.inference.hp_custom_endpoint import HPCustomEndpoint

4. Reference Files

Consult these for exact details — do not guess or recall from training data.

Templates for code generation:

5. Pre-Delivery Checklist

Run this mentally before EVERY code delivery:

• No V2 imports (sagemaker.model, sagemaker.processing, sagemaker.transformer, sagemaker.workflow)
• Model saved to /opt/ml/model/ (not ./model/)
• OutputDataConfig(s3_output_path=...) (not output_path=)
• .train(wait=True, logs=True) — logs is boolean
• All numeric argparse args have type=int or type=float
• Channel names match between launcher and training script
• No hardcoded credentials, account IDs, or absolute local paths
• entry_script= in SourceCode (not entry_point=)
• LLM deployment uses Core API (not ModelBuilder with DJL/vLLM)
• CUDA version matches instance type (cu128 for g5, cu129 for g6/p5)
• container_startup_health_check_timeout_in_seconds=900 for LLM endpoints
• dependencies={"auto": False} when using ModelBuilder
• Serverless model customization: model is on the supported list (see references/model-customization.md)
• Model customization trainers use sagemaker.modules.train.* imports (not sagemaker.train.*)