aws-config-validation-exercise - SKILL.md Agent Skill

name: aws-config-validation-exercise description: Interactive guided exercise teaching engineers how to prompt AI to create AWS configuration validation schemas for Python/boto3 projects. Use when someone wants to practice AI-assisted AWS config validation, learn the AI Flower framework's "Config Schema & Validation" exercise, or understand how to get reliable config validation artifacts from AI. This teaches prompting, validation, and iteration - not manual implementation. 20-30 min session.

AWS Config Validation Exercise

Guide an engineer through practicing how to prompt AI to create AWS configuration validation schemas for Python/boto3 projects.

Core Concept

This exercise teaches:

Structure effective prompts (inputs + constraints)
Validate AI outputs against quality criteria
Iterate with follow-up prompts

The AI does the actual work. The user learns to direct AI effectively.

Exercise Flow (6 Phases)

Phase 0: Setup Practice Environment (2 min)

Create practice files with broken AWS code:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Exercise: Config Schema & Validation (AWS Python)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Let me set up practice files with broken AWS code.

Where should I create these? [Suggest: practice-aws-config/]

After getting location, copy files from assets/practice-files/ to the chosen directory:

src/services/email.py (broken boto3 code)
src/handlers/signup.py (uses email service)
src/db/users.py (DynamoDB operations)
.env.example (config template)
README.md (context)

Confirm setup:

✅ Practice environment created!

📂 Files: src/services/email.py, src/handlers/signup.py, src/db/users.py, .env.example, README.md

👀 Next: Open src/services/email.py in VS Code and look at the problems.

Ready? (yes/no)

Phase 1: Explore the Broken Code (3 min)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
STEP 1: Explore the Code
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Open the files in VS Code and look for:
✗ Hardcoded values (regions, emails, table names)
✗ No validation or error handling
✗ boto3 client management issues
✗ Python anti-patterns (print, Exception, no types)

Found problems? (yes/no)

After confirmation, list key problems:

Line 6: Hardcoded region 'us-east-1'
Line 9: Hardcoded email 'noreply@example.com'
Line 19: Generic Exception instead of ClientError
Line 21: Using print() instead of logging
Line 4: boto3 client per-request (performance issue)
Missing type hints and SES verification

Phase 2: Teach Prompt Structure (5 min)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
STEP 2: Craft Your AI Prompt
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Effective prompt = 4 parts:

1. INPUTS (files to analyze)
2. OUTPUT STRUCTURE (what to produce)
3. CONSTRAINTS (prevent hallucinations)
4. FORMAT (language-specific patterns)

Here's the prompt to use:

Generate this prompt:

Analyze these AWS Python files and create a Config Schema & Validation Contract.

FILES: src/services/email.py, src/handlers/signup.py, src/db/users.py
CONTEXT: boto3, Python 3.9+, multi-account (dev/staging/prod), multi-region

OUTPUT REQUIRED:

1. Config Schema (TypedDict)
   - AWS region (validated list), account ID (12-digit), SES source email (verified)
   - SES config set (optional), timeout config, DynamoDB table name
   - For each: name, type, required/optional, validation rules

2. Validation Rules
   - Fail-fast at startup (raise ValueError)
   - Verify SES email with boto3 (show get_identity_verification_attributes)
   - Test IAM permissions (show test send)
   - Check DynamoDB table exists (show describe_table)

3. Loading Strategy
   - Where to create boto3 clients (once at startup, factory pattern)
   - How to pass config (dependency injection)
   - Session vs client distinction

4. Anti-Patterns Found
   - Cite file:line for each (e.g., "src/services/email.py:6")
   - What's wrong and why it's a problem

CONSTRAINTS:
❗ Do not guess. Cite file:line for every claim.
❗ If not in code, mark "unknown" and state needed evidence.
❗ Use Python type hints (TypedDict, -> Type)
❗ Show actual boto3 API calls (not pseudocode)
❗ Use logging module (not print), ClientError (not Exception)

Instruct user:

Copy this prompt and send it to your AI tool (ChatGPT, Claude, Copilot).
When you get AI's response, paste it here for validation.

Phase 3: User Prompts AI (External)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
YOUR TURN: Prompt AI
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Send the prompt to your AI tool.

TIPS:
- VS Code: Use @workspace in Copilot Chat
- ChatGPT/Claude web: Copy/paste file contents
- Claude Code: I can read files directly

Paste AI's output here when ready.

Phase 4: Validate AI Output (5 min)

Read references/north-star.md and score AI's output (0-10):

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⭐ NORTH STAR VALIDATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Checking AI's output against 10 criteria...

SCORE: X/10

[Show detailed scoring per criterion]

✅ Strong: Config schema structure, type hints
❌ Gaps: Missing boto3 validation code, no client factory, anti-patterns lack line numbers

[If score >= 8] → Phase 6 (Success)
[If score < 8] → Phase 5 (Iteration)

Phase 5: Teach Iteration (5 min)

If score < 8, generate follow-up prompt based on gaps:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
ITERATION: Improve AI's Output
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Score: 6/10. Let's improve it.

FOLLOW-UP PROMPT:

Improve the validation section:

1. SES Email Verification - Use get_identity_verification_attributes(),
   check status == 'Success', show complete function with error handling

2. boto3 Client Factory - Show AWSClients class that creates session once,
   creates clients with timeout config, reuses via @property

3. Anti-Patterns - Cite exact line numbers (src/services/email.py:LINE)

Provide complete Python code with imports.

Copy this and send as follow-up. Paste improved output when ready.

Validate again. Repeat until score >= 8 or user stops.

Phase 6: Success & Save Artifacts (3 min)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ EXERCISE COMPLETE - QUALITY BAR MET!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

FINAL SCORE: 9/10 🎉

Save artifacts? (yes/no)

[If yes, create files from references/artifacts.md]

Show reflection:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
💭 KEY LEARNINGS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Prompting Skills:
✓ Structure: Inputs + Output + Constraints + Format
✓ "Do not guess" + "Cite line numbers" prevents hallucinations
✓ First draft rarely perfect (6/10 → 9/10)
✓ Specific follow-ups improve quality systematically

AWS/Python Patterns:
✓ Validate at startup (fail-fast), check actual AWS state
✓ boto3 clients once (factory pattern), TypedDict for config
✓ ClientError > Exception, logging > print

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 AI FLOWER PROGRESS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Exercise 2: Config Schema & Validation ✅ COMPLETE
Maturity: Level 1 (Exploration)

Next: Try this on your real AWS project, or try another exercise
(Feature Flags, Error Handling, Idempotency)

Want another exercise? (yes/no)

Teaching Principles

1. Realistic Practice - Create actual files users open in VS Code, not code in chat

2. Focus on Prompting - User learns to structure prompts, validate outputs, iterate. NOT learning to write config code themselves.

3. "Do Not Guess" + "Cite Line Numbers" - Two constraints that prevent hallucinations

4. Iteration is Expected - First draft: 6-7/10 → After iteration: 9-10/10

5. Systematic Validation - Use North Star criteria (10 specific, measurable points)

6. Transferable Pattern - This prompting approach works for all AI Flower exercises

Reference Files

Load as needed during exercise:

references/scenario.md - Practice scenario details
references/north-star.md - 10 validation criteria with scoring
references/artifacts.md - Example outputs to save

Session Management

Time: 20-30 minutes
Can pause between phases
User can skip setup with their own code
Adapt to user's AI experience level

When NOT to Use

User wants actual AWS config help (not practice)
User wants to write code manually (not AI-assisted)
User asks about AWS concepts without wanting prompting practice