mcp-headless-test

name: mcp-headless-test description: Run live tests of the Ariadne MCP server using Claude Code in headless mode. Tests tool discovery, invocation, and filtering. Use when validating MCP server changes or checking integration with Claude Code. allowed-tools: Bash(./packages/mcp/tests/claude-headless-test.sh:, npm run build:), Read

MCP Headless Test Pipeline

Purpose

Validate the Ariadne MCP server by running it with Claude Code in headless mode (claude -p). This tests the real user experience rather than just programmatic MCP client behavior.

Prerequisites

Before running tests, ensure:

1. Claude Code CLI is installed and configured
2. API key is set (either ANTHROPIC_API_KEY env var or Claude's configured key)
3. MCP package is built:

npm run build -w packages/mcp

Running the Tests

Quick Run

cd packages/mcp
./tests/claude-headless-test.sh

Verbose Mode (for debugging)

./tests/claude-headless-test.sh --verbose

Test Scenarios

The script runs 3 tests:

Interpreting Results

Success Output

========================================
 Ariadne MCP - Claude Headless Tests
========================================

[INFO] Checking prerequisites...
[INFO] All prerequisites met.

[INFO] Test 1: Tool Discovery
[INFO] Checking if Claude can see the list_entrypoints tool...
[INFO] PASS: Tool 'list_entrypoints' discovered

[INFO] Test 2: Tool Invocation
[INFO] Running list_entrypoints on fixture code...
[INFO]   - Found 'entry point' in output
[INFO] PASS: Tool invocation succeeded

[INFO] Test 3: Filtered Analysis
[INFO] Testing file/folder filtering parameters...
[INFO] PASS: Filtered analysis completed

========================================
 Test Summary
========================================
Total:  3
Passed: 3
Failed: 0

[INFO] All tests passed!

Failure Indicators

Verifying Correctness

Test 1: Tool Discovery

Expected behavior: Claude should list list_entrypoints when asked about available tools.

Manual verification:

claude -p "What MCP tools do you have?" \
  --mcp-config ./packages/mcp/tests/mcp-test-config.json \
  --output-format json

Look for output containing:

• Tool name: list_entrypoints
• Description mentioning "entry point" and "call tree"

Test 2: Tool Invocation

Expected behavior: Claude should successfully invoke the tool and return analysis results.

Manual verification:

claude -p "Use list_entrypoints to analyze the codebase" \
  --mcp-config ./packages/mcp/tests/mcp-test-config.json

Valid output should contain:

• "Entry Points" header
• Function signatures (e.g., function_name(...): return_type)
• Tree size indicators (e.g., -- N functions)
• File locations (e.g., Location: ...)
• Total count (e.g., Total: X entry points)

Test 3: Filtered Analysis

Expected behavior: Tool accepts folders parameter and analyzes only specified directories.

Manual verification:

claude -p "Use list_entrypoints with folders=['functions']" \
  --mcp-config ./packages/mcp/tests/mcp-test-config.json

Valid output: Should show entry points only from the functions/ subdirectory of the fixtures.

Test Configuration

The tests use packages/mcp/tests/mcp-test-config.json:

{
  "mcpServers": {
    "ariadne": {
      "type": "stdio",
      "command": "node",
      "args": ["./packages/mcp/dist/server.js"],
      "env": {
        "PROJECT_PATH": "./packages/core/tests/fixtures/typescript/code"
      }
    }
  }
}

This points to TypeScript fixtures in packages/core/tests/fixtures/typescript/code/ for deterministic, reproducible tests.

Server Lifecycle Notes

Important: The MCP server process stays alive across tool calls within a session, BUT each tool call creates a fresh Project instance and re-indexes the codebase. This is intentional to support scoped analysis (file/folder filtering).

Performance implication: Large codebases may take 2+ seconds per tool call due to re-indexing.

Troubleshooting

Claude hangs or times out

• Check ANTHROPIC_API_KEY is valid
• Try with --verbose flag to see debug output
• Ensure network connectivity to Anthropic API

Tool not discovered

1. Verify MCP config path is correct

2. Check server.js exists: ls packages/mcp/dist/server.js

3. Test server directly:

   echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | \
     PROJECT_PATH=. node packages/mcp/dist/server.js
   

Unexpected tool output

1. Check fixtures exist: ls packages/core/tests/fixtures/typescript/code/
2. Test with E2E tests first: npm test -w packages/mcp
3. Review server stderr for errors (use --verbose)

Related Files