starter-execution - SKILL.md Agent Skill

name: starter-execution description: Orchestrates complete Azure AI Foundry deployment to Azure DevOps. Coordinates repository-setup, service-connection-setup, environment-setup, pipeline-setup, and deployment-validation skills. Use when deploying the complete Azure AI Foundry starter template end-to-end. license: Apache-2.0

Azure AI Foundry Starter Execution Orchestrator

Orchestrates the complete deployment of the Azure AI Foundry starter template to Azure DevOps by coordinating five specialized skills.

Overview

End-to-end orchestration skill that coordinates five specialized skills to deploy the Azure AI Foundry starter template from template-app/ to Azure DevOps.

Prerequisites

Before using this orchestrator:

✅ configuration-management - Configuration must be set up FIRST
✅ resource-creation - Azure resources and Service Principal must exist
✅ environment-validation - Environment prerequisites validated
✅ Bearer token valid - 30+ minutes remaining for deployment
✅ Azure DevOps permissions - Contributor access to target project

What Gets Created

1 Repository: azure-ai-foundry-app with template application code
3 Service Connections: dev/test/prod with Workload Identity Federation (no secrets!)
3 Variable Groups: {projectName}-{env}-vars with environment-specific configuration (projectName from config)
3 Environments: dev, test, production with approval gates
3+ Pipelines: Agent creation, evaluation, and red team testing

Orchestration Flow

This orchestrator executes five specialized skills in sequence:

repository-setup - Create Azure DevOps repository and push template code
service-connection-setup - Configure service connections with federated credentials
environment-setup - Create variable groups and environments
pipeline-setup - Create CI/CD pipelines from YAML templates
deployment-validation - Validate complete deployment

Usage

# Complete deployment (all phases)
cd .github/skills/starter-execution
# Follow Phase 1-6 commands below

# Or execute individual skills as needed
cd .github/skills/repository-setup
# See individual skill documentation

Deployment Workflow

Execute these six phases in sequence for successful deployment:

Phase 1: Authentication & Configuration

# Load configuration
. ./.github/skills/configuration-management/config-functions.ps1
$config = Get-StarterConfig

# Extract values
$org = $config.azureDevOps.organizationUrl
$project = $config.azureDevOps.projectName

# Login to Azure
az login

# Set subscription
az account set --subscription $config.azure.subscriptionId

# Get bearer token (valid for ~1 hour)
$env:ADO_TOKEN = az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query "accessToken" -o tsv
$env:AZURE_DEVOPS_EXT_PAT = $env:ADO_TOKEN

# Configure Azure DevOps CLI
az devops configure --defaults organization=$org project=$project

Write-Host "✅ Authentication complete"

Phase 2: Repository Setup

Skill: repository-setup

Purpose: Create Azure DevOps repository and push template application code

Key Actions:

Create repository azure-ai-foundry-app
Initialize git and push template code from template-app/
Create .env file from sample.env

Direct Usage:

cd .github/skills/repository-setup
# Follow skill documentation for detailed steps

Quick Validation:

az repos show --repository "azure-ai-foundry-app" -o table

Troubleshooting: See repository-setup/SKILL.md

Phase 3: Service Connection Setup

Skill: service-connection-setup

Purpose: Create service connections with Workload Identity Federation (passwordless, no secrets!)

Key Actions:

Create 3 service connections: azure-foundry-dev, azure-foundry-test, azure-foundry-prod
Configure federated credentials on Service Principal
Authorize connections for all pipelines
Verify RBAC roles (Contributor + Cognitive Services User)

Critical: Federated credential issuer/subject format must match Azure DevOps exactly!

Direct Usage:

cd .github/skills/service-connection-setup
# Follow skill documentation for detailed steps

Quick Validation:

az devops service-endpoint list -o table
az ad app federated-credential list --id $config.servicePrincipal.appId -o table

Troubleshooting: See service-connection-setup/SKILL.md

Phase 4: Environment Setup

Skill: environment-setup

Purpose: Create variable groups and environments for all deployment stages

Key Actions:

Create 3 variable groups: {projectName}-dev-vars, {projectName}-test-vars, {projectName}-prod-vars (using config.naming.projectName)
Configure environment-specific variables (endpoints, model names, connection strings)
Create 3 environments: dev, test, production
Authorize variable groups for pipeline access

Critical: Variable group names must match pipeline YAML exactly!

Direct Usage:

cd .github/skills/environment-setup
# Follow skill documentation for detailed steps

Quick Validation:

az pipelines variable-group list -o table
az pipelines environment list -o table

Troubleshooting: See environment-setup/SKILL.md

Phase 5: Pipeline Setup

Skill: pipeline-setup

Purpose: Create CI/CD pipelines from template YAML files

Key Actions:

Update pipeline YAML files - Automatically replaces REPLACE_WITH_YOUR_PROJECTNAME with config.naming.projectName
Commit and push changes - Updates YAML files in repository with correct variable group names
Create pipeline: Azure AI Foundry - Create Agent
Create pipeline: Azure AI Foundry - Agent Evaluation
Create pipeline: Azure AI Foundry - Red Team
Link pipelines to repository branch
Configure with --skip-first-run flag

Critical: Automated script updates YAML files to match variable group names created in Phase 4!

Direct Usage:

cd .github/skills/pipeline-setup
./scripts/create-pipelines.ps1 -UseConfig

Follow skill documentation for detailed steps


**Quick Validation**:
```powershell
az pipelines list -o table

Troubleshooting: See pipeline-setup/SKILL.md

Phase 6: Deployment Validation

Skill: deployment-validation

Purpose: Validate complete deployment and readiness for pipeline execution

Key Actions:

Validate repository exists and has code
Validate service connections and federated credentials
Validate variable groups and environments
Validate pipelines are configured correctly
Verify RBAC permissions (Contributor + Cognitive Services User)
(Optional) Execute first pipeline run

Direct Usage:

cd .github/skills/deployment-validation
# Follow skill documentation for detailed steps

Quick Validation:

# Comprehensive validation
az repos list -o table
az devops service-endpoint list -o table
az pipelines variable-group list -o table
az pipelines environment list -o table
az pipelines list -o table

First Pipeline Run:

# Run Create Agent pipeline
$pipelineId = az pipelines list --query "[?name=='Azure AI Foundry - Create Agent'].id" -o tsv
az pipelines run --id $pipelineId

# Monitor at: $config.azureDevOps.organizationUrl/$config.azureDevOps.projectName/_build

Troubleshooting: See deployment-validation/SKILL.md

Specialized Skills

This orchestrator delegates to five specialized skills:

1. Repository Setup Skill

Location: ../repository-setup
Purpose: Create Azure DevOps repository and push template application code
When to Use: Initial deployment or repository recreation

2. Service Connection Setup Skill

Location: ../service-connection-setup
Purpose: Configure service connections with Workload Identity Federation
When to Use: Initial deployment, fix federated credentials, add new environments

3. Environment Setup Skill

Location: ../environment-setup
Purpose: Create variable groups and environments
When to Use: Initial deployment, update variables, add new environments

4. Pipeline Setup Skill

Location: ../pipeline-setup
Purpose: Create CI/CD pipelines from YAML templates
When to Use: Initial deployment, add new pipelines, fix pipeline configuration

5. Deployment Validation Skill

Location: ../deployment-validation
Purpose: Validate complete deployment readiness
When to Use: After each phase, troubleshooting, pre-deployment verification

Troubleshooting

For detailed troubleshooting, refer to the specific skill documentation:

Repository Issues → repository-setup/SKILL.md
Service Connection Issues → service-connection-setup/SKILL.md
Variable/Environment Issues → environment-setup/SKILL.md
Pipeline Issues → pipeline-setup/SKILL.md
Validation Issues → deployment-validation/SKILL.md

Common Issues

Symptom: AADSTS70021: No matching federated identity record found
Cause: Federated credential issuer/subject mismatch
Solution: See service-connection-setup troubleshooting

Symptom: The pipeline is not valid. Could not find service connection
Cause: Service connection not authorized for pipeline use
Solution: See service-connection-setup troubleshooting

Symptom: Variable group name contains invalid characters
Cause: Invalid characters in variable group name
Solution: See environment-setup troubleshooting

Symptom: 401 Unauthorized errors
Cause: Bearer token expired (valid for ~1 hour)
Solution:

$env:ADO_TOKEN = az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query "accessToken" -o tsv
$env:AZURE_DEVOPS_EXT_PAT = $env:ADO_TOKEN

Complete Troubleshooting Guide: docs/troubleshooting.md

Best Practices

Execution Strategy

Follow the phase order: Always execute phases 1-6 in sequence
One environment at a time: Test dev thoroughly before test/prod
Use modular skills: Run individual skills for easier troubleshooting
Validate early and often: Use deployment-validation after each phase

Security

Workload Identity Federation: Zero secrets stored - use federated credentials
RBAC principle: Contributor + Cognitive Services User roles only (least privilege)
Configuration management: Store starter-config.json securely (add to .gitignore)
Token management: Refresh bearer token every 30-45 minutes

Configuration

Use starter-config.json: Centralized configuration for consistency
Variable group naming: Must match pipeline YAML exactly
Federated credential format: Must match Azure DevOps issuer/subject precisely
Document customizations: Track any template modifications

Deployment

Skip first run: Use --skip-first-run when creating pipelines for control
Test incrementally: Validate each phase before proceeding
Monitor first execution: Watch logs for the initial Create Agent pipeline run
Use feedback mechanism: Report issues via template-app/FEEDBACK.md

Related Skills

Prerequisite Skills (Run Before This Orchestrator)

configuration-management - Set up centralized configuration (REQUIRED FIRST)
resource-creation - Create Azure resources and Service Principal (REQUIRED)
environment-validation - Validate prerequisites (RECOMMENDED)

Orchestrated Skills (Called By This Orchestrator)

repository-setup - Create repository and push template code
service-connection-setup - Configure service connections with federated credentials
environment-setup - Create variable groups and environments
pipeline-setup - Create CI/CD pipelines
deployment-validation - Validate complete deployment

Documentation

docs/starter-guide.md - Complete deployment guide with screenshots
docs/quick-start.md - Fast track deployment guide
docs/execution-guide.md - GitHub Copilot usage patterns
docs/troubleshooting.md - All 12 critical lessons learned
docs/az-devops-cli-reference.md - Azure DevOps CLI command reference
template-app/README.md - Template application documentation
template-app/FEEDBACK.md - Submit feedback and issues