fluxloop-scenario

name: fluxloop-scenario description: | Use for creating test scenarios, contracts, and agent wrapper setup. Frequency: when test objectives change or new scenarios are needed. Reuses existing scenarios otherwise. Keywords: scenario, create scenario, init, contract, wrapper, agent setup

Auto-activates on requests like: - "create a scenario", "init scenario" - "set up a test", "configure the wrapper"

FluxLoop Scenario Skill

Scenario-First: Agent profile check → Scenario init → Contract creation → Wrapper setup → Test ready

Output Format

📎 All user-facing output must follow: read skills/_shared/OUTPUT_FORMAT.md

Context Protocol

1. fluxloop context show → confirm project is set up and scenario state
2. .fluxloop/test-memory/ check:
   • Load agent-profile.md → stale detection (compare git_commit vs git rev-parse --short HEAD)
   • Load learnings.md → incorporate previous insights (if exists)
   • Missing files → proceed (first run)
3. Dual Write:
   • Server: fluxloop scenarios create/refine
   • Local: save to .fluxloop/test-memory/test-strategy.md
4. On completion: verify test-strategy.md is current for the test skill

📎 Stale detection: read skills/_shared/CONTEXT_PROTOCOL.md 📎 Collection procedure (for inline refresh): read skills/_shared/CONTEXT_COLLECTION.md

Prerequisite

Run fluxloop context show first:

• ✅ Project selected + .fluxloop/test-memory/agent-profile.md exists → proceed
• ❌ No project (setup missing) → Prerequisite Resolution (📎 read skills/_shared/PREREQUISITE_RESOLUTION.md):
  • "Both project setup and agent analysis are required. Proceed with setup → context in order?"
  • Approved:
    1. 📎 Run skills/setup/SKILL.md inline → "✅ Setup complete."
    2. 📎 Run skills/context/SKILL.md inline → "✅ Context complete. Continuing with scenario." → return to Step 1
  • Denied: stop
• ❌ Project selected but no agent-profile.md (context missing) → Prerequisite Resolution:
  • "Agent analysis is required. Would you like to run context first?"
  • Approved: 📎 Run skills/context/SKILL.md inline → on completion return to Step 1
  • Denied: stop

Workflow

Step 1: Context Load + Stale Detection

• Read .fluxloop/test-memory/agent-profile.md:
  • Extract git_commit from metadata comment
  • Compare with git rev-parse --short HEAD
  • If different → "Code has changed since the last profile was created (previous: {old_commit} → current: {new_commit}). Would you like to update the profile?"
    • Yes → follow _shared/CONTEXT_COLLECTION.md procedure inline
    • No → continue with existing profile
  • If git_commit is no-git → continue without warning (stale detection unavailable)
• Read .fluxloop/test-memory/learnings.md (if exists):
  • Incorporate previous insights into scenario design
  • Display: "Previous learnings found: {summary}"
• Understand agent characteristics → use in Step 3 for scenario recommendations

Step 2: Scenario Initialization

fluxloop init scenario <name>

• Naming rules:
  • Folder name: English kebab-case only (e.g., order-bot)
  • Suggest 3 name candidates based on agent characteristics, allow custom input
• ⚠️ Common mistake: "Run from workspace root, not home directory. Check pwd and fluxloop context show."

Step 3: Scenario Recommendation

Suggest 3 scenarios based on agent-profile.md:

💡 Always explain each type to the user:

• Happy Path: Verifies the agent's core functionality works correctly. E.g., "Scenario where {agent} performs {key function} without issues"
• Edge Cases: Verifies the ability to handle exceptions and boundary conditions. E.g., "Behavior with invalid inputs, empty values, special conditions"
• Advanced: Verifies multi-turn conversations or domain-specific scenarios. E.g., "Complex requests, consecutive conversations requiring context retention"

Tailor explanations with specific examples based on agent characteristics from agent-profile.md.

• Present 3 options + "Custom input" → user selects
• If learnings.md insights exist, reflect them in recommendations
  • Example: "Previous tests showed weak edge case handling" → prioritize Edge Cases
• After selection: specify display name (any language allowed)

Step 4: Language Selection

• Select language for scenario generation
• If project-level language is already set → suggest as default, allow override
• Apply to fluxloop scenarios create --name "..." --goal "..." command

Step 5: API Key Setup

• Check .fluxloop/.env → if exists, skip
• If missing: fluxloop apikeys create
• API key file location: .fluxloop/.env (shared across scenarios)
• Manual addition guide:
  • OpenAI: OPENAI_API_KEY=sk-xxx
  • Anthropic: ANTHROPIC_API_KEY=sk-ant-xxx

Step 6: Contract Creation + Strategy Save (Dual Write)

💡 What is a Contract? Rules (YAML) that define the expected behavior an agent must follow in each scenario. E.g., "Must include the amount when confirming an order." The server auto-generates them, and users can edit them in the web app.

(Server):

fluxloop scenarios create --name "Order Accuracy Test" --goal "..."
fluxloop scenarios refine --scenario-id <id>
fluxloop sync pull --scenario <name>

After scenarios refine — must include scenario URL in output:

✅ Contracts → N generated 🔗 https://alpha.app.fluxloop.ai/simulate/scenarios/{scenario_id}?project={project_id}
📋 You can review and edit contracts at the link above.

(Local): Save to .fluxloop/test-memory/test-strategy.md (format: test-memory-template/test-strategy.md)

Fields to populate:

• Active Scenarios: scenario name, ID, goal, status=active
• Test Objectives: extracted from scenario goal
• Contract Summary: contract count, key coverage
• Evaluation Criteria: set based on scenario characteristics (default: accuracy, completeness, relevance)
• Test Configuration: turn mode (TBD), input count (TBD), wrapper path

Required link output: After scenario creation, extract scenario_id and project_id from CLI output to construct the URL. URL pattern: https://alpha.app.fluxloop.ai/simulate/scenarios/{scenario_id}?project={project_id}

Optional: Ground Truth Data Binding

After scenario creation, the user may want to bind validation data:

"Do you have Ground Truth validation data (e.g., Q&A pairs with expected answers)? (enter path / skip)"

• Path entered → upload as GT:

  fluxloop data push <gt-file> --usage ground-truth --scenario <scenario_id> --label-column <col>
  

  • Output: ✅ Data (GT) → {filename} bound to scenario, {N} GT contracts generated
  • Check status if needed: fluxloop data gt status --scenario <scenario_id>
• skip → proceed to Step 7

📎 For detailed GT classification guidance: read skills/context/SKILL.md Step 5-1

Step 7: Wrapper Setup

Basic flow:

1. Determine if wrapper is needed (based on agent type)
2. If needed → create wrapper.py + update simulation.yaml
3. Debug test: python -c "from agents.wrapper import run; print(run('test'))"

📎 Wrapper setup detail: read skills/scenario/references/wrapper-guide.md

Interactive Checkpoints

Max 1 required user response (scenario selection).

Error Handling

Next Steps

Scenario ready. Available next actions:

• Run tests against the scenario (test skill)
• Bind Ground Truth validation data (if not done): fluxloop data push <file> --usage ground-truth --scenario <id>

Quick Reference

📎 Full CLI reference: read skills/_shared/QUICK_REFERENCE.md

Key Rules

1. Always read agent-profile.md first — use agent characteristics for scenario recommendations
2. Always check stale detection on agent-profile.md before proceeding
3. Read learnings.md (if exists) to incorporate previous insights into scenario design
4. Use the template from test-memory-template/test-strategy.md for output format
5. Scenario folder names: English kebab-case only (order-bot)
6. Display names: any language allowed
7. Suggest 3 naming candidates, allow custom input
8. Run fluxloop init scenario from workspace root (NOT home directory)
9. Dual Write: server (scenarios create/refine) and local (test-strategy.md) at the same time
10. On update: overwrite test-strategy.md entirely (not append)