primus-projection

name: primus-projection description: Opinionated guide for using Primus Projection to choose parallelism (TP/PP/EP/CP/DP) and pipeline schedules, validate memory fit on target nodes, reason about communication collectives, and explore optimization trade-offs with minimal compute. Use when the user asks how to pick a parallelism strategy or pipeline schedule, whether a model fits in memory, which optimizations such as DeepEP, SyncFree, zero-bubble, FP8, recomputation, or FSDP2 matter most, or how to run primus projection commands. Read-only planning guidance; no large multi-node training runs required.

Primus Projection Skill

Purpose

Provide a quick, opinionated guide for using Primus Projection to:

• Choose parallelism and pipeline schedules
• Validate memory fit on target nodes
• Understand communication collectives/algorithms
• Explore optimization trade-offs with minimal compute
• Run the right commands fast

Scope

• Docs: docs/tech_blogs/projection/projection.md, docs/projection.md
• Projection code: primus/core/projection/
• Projection performance entrypoints: primus/core/projection/performance_projection/
• Projection memory entrypoints: primus/core/projection/memory_projection/
• Pipeline schedulers: primus/core/pipeline_parallel/scheduler/
• Example configs: examples/megatron/configs/
• Hardware configs: examples/hardware_configs/

Projection Advisor (What users want to know)

1. Best parallelism strategy

• Evaluate TP/PP/EP/CP/DP for the target cluster using performance projection.
• Guidance:
  • Prefer keeping TP intra-node (for faster AR); scale EP and DP across nodes.
  • For MoE, use EP that avoids inter-node A2A when possible; otherwise enable DeepEP.
  • Use CP only when sequence length forces it; in MoE, CP may be folded into EP.
  • Increase DP last (weak scaling) once a minimal viable PP×EP×TP is established.

2. Best pipeline schedule

• Compare (for the chosen PP/VPP):
  • 1F1B
  • Interleaved 1F1B (VPP > 1)
  • Zero-Bubble (B/W split, VPP = 1)
  • ZBV Formatted (VPP = 2)
  • ZBV Greedy (VPP = 2, memory modes: min/half)
  • Megatron ILP (VPP = 1, where available)
• Recommendation pattern:
  • Dense models: interleaved or ZB depending on VPP feasibility and memory.
  • MoE (e.g., Mixtral): VPP=2 with ZBV Formatted often wins, especially with DeepEP ON.

3. Memory fit at target nodes

• Run memory projection first. Review:
  • Parameters + optimizer (sharded by DP)
  • Activations (attention vs MoE MLP; MoE often dominates)
  • Pipeline scaling: peak activations scale with PP and VPP schedule
• If not fitting:
  • Enable recomputation (full or partial)
  • Adjust microbatch size and PP/VPP
  • Reduce top-k or expert sizes for MoE if acceptable

4. Communication collectives and algorithms

• In use (typical):
  • AllReduce: TP and DP gradients (best-of Ring/Hypercube/Bruck/Single-shot)
  • All-to-All: EP token dispatch/combine (pairwise, topology-aware)
  • Reduce-Scatter / All-Gather: optimizer/activation sharding (if configured)
  • P2P Send/Recv: PP stage activations/gradients
• Distinguish intra-node (xGMI/NVLink/UALink) vs inter-node (IB/RoCE). Provide --hardware-config to model your network accurately.

5. Sample command lines

# Memory projection (check fit)
bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection memory \
  --config <model_config.yaml>

# Performance projection (benchmark mode)
bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --target-nodes <N>

# Performance projection (simulation-only, no GPU)
bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --profiling-mode simulate \
  --gpu-arch <mi300x|mi325x|mi355x> \
  --target-nodes <N>

# Sub-node benchmarking (e.g., on 1 GPU), project to many nodes
bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --benchmark-gpus 1 \
  --target-nodes <N>

# Compare benchmark vs simulation (accuracy check)
bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --profiling-mode both \
  --target-nodes <N>

# Override parallelism from environment (optional)
export PRIMUS_TP=1
export PRIMUS_PP=3
export PRIMUS_EP=8

bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --target-nodes <N>

# Provide cluster network topology
bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --hardware-config examples/hardware_configs/mi355x.yml \
  --target-nodes <N>

Optimization Space Exploration

Primus can project throughput and memory across dozens of optimization combinations from a single-node benchmark (8 GPUs), eliminating the need for expensive multi-node runs. This was validated on DeepSeek V2 (236B MoE) across 17 configurations on 4-node MI355X — the projection tracked measured results across all optimization combinations and correctly identified the highest-impact optimizations.

Category 1: MoE Communication Optimizations (CLI flags)

Category 2: Pipeline Schedule & Layout (CLI flags + config)

Category 3: Memory Optimizations (config)

Category 4: Parallelism Configuration (config + env)

Category 5: Precision & Attention (config)

Category 6: Batch & Gradient Configuration (CLI + config)

Category 7: Network / Hardware Tuning (hardware config YAML)

Optimization Exploration Guidelines

Step 1: Establish baseline. Run projection with no optimization flags to get the baseline throughput and memory.

bash runner/primus-cli direct --script primus/cli/main.py -- \
  projection performance \
  --config <model_config.yaml> \
  --target-nodes <N> \
  --hardware-config examples/hardware_configs/mi355x.yml

Step 2: Parallelism search. Sweep TP/PP/EP/CP combinations:

• Keep TP intra-node (TP ≤ 8 for 8-GPU nodes)
• For MoE: try EP=8 (intra-node) vs EP=16+ (inter-node + DeepEP)
• Use CP only when sequence length requires it
• Compare PP=1 (no bubble) vs PP=2/4 (less memory, more bubble)

Step 3: Pipeline schedule comparison. Use --pipeline-schedule all to compare every algorithm in a single run:

... --pipeline-schedule all

Try uneven pipeline layouts when first/last stages have embedding overhead:

# In config:
decoder_first_pipeline_num_layers: 6   # fewer layers on first stage (has embedding)
decoder_last_pipeline_num_layers: 6    # fewer layers on last stage (has output layer)

Step 4: Sweep MoE optimizations (for MoE models). Recommended order:

1. DeepEP — single largest gain: --enable-deepep
2. SyncFree stages — progressive fusion: --sync-free-stage 2, then 3
3. EP size — compare: --target-ep-size 4 vs --target-ep-size 8

Step 5: Precision and compute optimizations.

# In config — compare BF16 vs FP8:
fp8: hybrid          # ~2× compute speedup for linear layers
fp8: null            # BF16 baseline

# Loss fusion (MoE + large vocab):
cross_entropy_loss_fusion: true

# SwiGLU (if model supports it):
swiglu: true

Step 6: Memory-side sweep. Find the best memory/throughput trade-off:

# Full recomputation — minimum memory, maximum recompute overhead:
recompute_granularity: full
recompute_num_layers: 80   # all layers

# Selective recomputation — partial savings, less overhead:
recompute_granularity: selective

# No recomputation — maximum throughput, highest memory:
recompute_granularity: null

# FSDP2 — shard optimizer + gradients across DP ranks:
use_torch_fsdp2: true

# Distributed optimizer — shard optimizer state only:
use_distributed_optimizer: true

Compare micro-batch sizes to find the sweet spot:

... --micro-batch-size 1    # low memory, low GPU util
... --micro-batch-size 2    # balanced
... --micro-batch-size 4    # high GPU util, high memory

Step 7: Network tuning. If projections show large comm overhead, tune the hardware config to match your actual cluster:

hardware_config:
  node_bw: 896.0          # Measured intra-node BW (GB/s)
  pod_bw: 48.0            # Measured per-NIC BW (GB/s)
  bw_eff: 0.88            # Measured efficiency
  nics_per_node: 8
  a2a_peer_lat: 0.5       # Tune for your fabric

Step 8: Combine best options. Once you know which optimizations help most individually, combine them and project the final configuration.

What the Projection Captures vs. Doesn't

Key insight from validation: The projection correctly identifies the relative ranking of optimizations — which matters most for decision-making. Even when absolute error is 10–18%, the ordering (DeepEP >> SyncFree > VPP

LF) is preserved, so you can confidently narrow the search space on the simulator before running expensive full-scale validation.

Quick Interpretation Tips

• If tokens/s improves most when DP increases: you're compute-bound; check comm overlap.
• If comm breakdown shows large EP A2A: enable DeepEP or reduce inter-node EP links.
• If pipeline bubble ratio is high: increase VPP or switch to ZB/ZBV schedules.
• If memory is tight: recomputation + smaller microbatch + redistribute PP.
• If DeepEP shows big gains: try --sync-free-stage 2 or 3 for additional overlap.
• If MoE A2A dominates comm: check whether EP fits intra-node; if not, DeepEP is essential.
• If absolute error seems high: check measurement variance first (re-run measured baseline).

References

• Tech blog: docs/tech_blogs/projection/projection.md
• User docs: docs/projection.md
• Schedulers: primus/core/pipeline_parallel/scheduler/
• Simulation backends: Origami (GEMM), SDPA simulator (attention)