name: ocas-rally description: 'Rally: governed portfolio research, scoring, constrained allocation, auto-executed trade planning, and pre-market / pre-close portfolio health checks for public markets. Maintains durable pending-action queue so missed cron runs are recovered automatically. Use for universe screening, signal scoring, rebalance planning, automatic trade execution within risk limits, stop-loss enforcement, and auditable investment decisions. Trigger phrases: ''portfolio'', ''allocation'', ''rebalance'', ''trade plan'', ''research stocks'', ''daily report'', ''health check'', ''market signals'', ''update rally''. Do NOT use for meme-stock speculation, margin trading, or budget planning. ' license: MIT source: https://github.com/indigokarasu/rally includes:
- references/**
- scripts/** metadata: author: Indigo Karasu (indigokarasu) version: 3.12.0 triggers:
- portfolio research
- stock scoring
- trade planning
- market research
- investment allocation
Rally
Rally turns public-market research into explainable, risk-bounded allocation and trade plans. It screens a universe of candidates, computes composite signals from price, fundamentals, regime, congressional flow, and optional news/social sentiment, solves for constrained allocations that respect hard limits (long-only, max position size, max drawdown, sector caps), and, when execution is enabled, submits trades automatically once every clause of the execution safety gate has passed.
Four scheduled jobs carry the full cycle: overnight research and plan generation, a pre-market-open health check that re-validates and submits queued orders, a pre-close health check that runs stop enforcement and writes the EOD-proxy report, and a weekly self-update. A durable pending-action queue means a single missed or errored cron run does not lose a planned rebalance: the next scheduled job picks it up, re-validates against fresh data, and fires if still relevant.
Interactive Menu
When invoked interactively (via / command), present a two-level menu using the clarify tool so the user can pick which function to run.
Level 1 — Category selection (max 4 choices):
result = clarify(
question="What would you like to do?",
choices=[
"Research — run research, ranking, and universe refresh",
"Plans & Trades — allocation plans, trade plans, execution",
"Portfolio & Reports — import positions, pending actions, daily reports",
"Health & Status — health checks and system status",
]
)
Level 2 — Action selection based on Level 1 choice:
- Research → clarify with choices: "research — Run full research sequence", "healthcheck — Run pre-open/pre-close health check", "ingest.portfolio — Import portfolio positions", "universe.refresh — Refresh investable universe"
- Plans & Trades → clarify with choices: "candidates.rank — Rank investment candidates", "plan.allocation — Generate allocation plan", "plan.trade — Generate trade plan", "execute.trades — Execute planned trades"
- Portfolio & Reports → clarify with choices: "pending.list — List pending trade actions", "report.daily — Generate daily report"
- Health & Status → run "healthcheck — Run pre-open/pre-close health check" or "status — Show system status" directly (single action — no sub-menu needed)
After the user selects an action, execute it following the relevant procedure in this skill. Loop back to the menu after each action completes, until the user chooses to exit or sends /stop.
Response parsing
Match the user's response against the full choice string. Extract the action key by splitting on " — " and taking the first segment. If the response doesn't match any known choice (user typed free-form via "Other"), match key prefixes case-insensitively. Re-present the current menu level on no match.
Platform adaptation
On CLI, choices are navigable with arrow keys. On messaging platforms, choices render as a numbered list. The two-level hierarchy ensures no more than 4 options appear at any level on any platform.
Reference files
This skill uses progressive disclosure. SKILL.md contains the routing, command surface, and invariant summaries. The files in references/ contain the full operational procedures. Read each reference file when its trigger condition applies; do not try to execute a workflow from SKILL.md alone when a referenced procedure exists.
Gotchas
append_jsonlinrally_utils.pyDESTROYS JSONL files. The function writes to a.tmpfile and renames it to the target, OVERWRITING all existing content instead of appending. Fixed (2026-06-20) by addingshutil.copy2(path, tmp)before the append. Always verify JSONL file line counts after any append operation — a drop from thousands to 1 is the signature of this bug. If you find truncated JSONL files, checkindigo-repo/commons/data/ocas-rally/for a backup copy ofsignals.jsonl.- Dossiers are machine-readable first. The allocation solver, rationale generator, and factor IC computation consume JSON dossiers. Keep them compact, structured, and queryable. No lengthy prose. The
.mdfiles are thin human summaries — the.jsonfiles are the canonical format. - Finnhub profile2 API returns flat JSON, not nested. The endpoint
/api/v1/stock/profile2?symbol=Xreturns{"ticker":"X","name":"...","finnhubIndustry":"Real Estate",...}— NOT{"profile":{"sector":"..."}}. The sector field isfinnhubIndustry, notsector. Scripts that readdata["profile"]["sector"]silently getNone. Usedata.get("finnhubIndustry", "")directly. - Never say data doesn't exist when it's a transient issue. Finnhub 429s, empty responses, and "no data" are almost always transient — retry with backoff, cycle to another source, or fall back to price-proxy. Only mark a field as unavailable after 3+ retries AND a fallback attempt. The user has explicitly called this out as a persistent pattern to fix. Every "no data" claim in this session turned out to be wrong — the data existed but the script had a bug, hit a rate limit, or didn't try the right endpoint.
- Snapshot, duplicate, modify, confirm, replace. When editing production files (scripts, SKILL.md, config), always: (1) snapshot current state, (2) write new version to temp, (3) verify temp works, (4) replace original. Never edit in-place without a rollback path. The user has explicitly required this workflow.
- Cycle sources on failure, don't just retry the same one. When Finnhub returns 429, wait and retry. When it returns empty/null, try price-proxy. When Massive returns 404, skip it. Build fallback chains, not retry loops. The source chain for fundamentals is: Finnhub metrics → price-proxy (computed from price history). For insider data: Finnhub only (Massive SEC filings are dead). For news/earnings: Finnhub only.
- Python 3.14 json boolean parsing bug in loops. When reading many JSON files in a loop with
json.load(fh), boolean values in nested dicts (likedata_sources.fundamentals) silently returnNoneinstead ofTrue/False. Workaround: use text search ("\"fundamentals\": true" in f.read_text()) for field presence checks, or read files individually outside loops. This affectsfill-gapsmode and any batch validation of dossier fields. - Massive SEC filings endpoint is dead. All form-4/form-3 URL patterns on
api.massive.comreturn 404 as of 2026-06-17. Insider flow data comes from Finnhub's/stock/insider-transactionsonly. Do not waste time trying Massive filings endpoints. - Backup must cover commons/ wholesale. The backup script must back up entire
commons/directories, not individual subdirectories. Manually listing subdirs guarantees something gets missed when new subdirs are added. Use[ -d /root/.hermes/commons ] && FILES="$FILES /root/.hermes/commons"not[ -d /root/.hermes/commons/data ] && .... - Finnhub API coverage gaps. Some symbols (especially foreign, small-cap, or ETFs) don't have coverage for recommendations, news, or earnings endpoints. The dossier records
data_sourcesaccurately — missing fields are genuine API gaps, not script bugs. HERMES_HOMEmay include profile path. WhenHERMES_HOMEis set to/root/.hermes/profiles/indigo, the script detects this and avoids double-appendingprofiles/indigo.rally_ingest_alpaca.pyhas known bugs — The quick portfolio snapshot script has C1–C4 data corruption bugs. Never use it for canonical refresh; use the full pre-open/pre-close scripts instead.- Execution safety gate has 10 clauses — A trade plan is submitted only when ALL 10 clauses pass. Any single failure keeps the action
stagedfor the next wake. Emergency sells bypass only clause 6 (turnover budget) and minimum rebalance interval. - Market holidays are detected at runtime — Jobs detect market closure via the broker clock endpoint and exit early. Pending actions remain queued and are processed on the next trading day.
- Reference files are authoritative over SKILL.md — If a concept is described in both SKILL.md and a reference file, the reference file wins. Always read the relevant reference before executing a workflow.
- Post-trade constraint verification required — When deriving trade plans or re-evaluating pending actions, verify that the post-trade state (after sells and proposed buys) satisfies ALL constraints. A plan that fixes one violation (e.g., position cap) may worsen another (e.g., top-3 concentration, sector cap). Supersede stale plans that fail this check, don't just gate-block them — deadlock risks accumulate in the queue.
- Cash must be deployed after sells-only plans — If a constraint-fix sell plan executes and cash ends up >10pp above the regime-appropriate cash target, the NEXT
rally.researchrun MUST generate a full buys-inclusive allocation. Never leave elevated cash unaddressed. Seereferences/operating-model.md"Cash deployment after sells-only plans" and cash drift trigger (trigger 5). - Do not queue plans with constraint violations — The solver must never produce a plan where
risk_checksshowsmax_sector_ok: Falseortop3_concentration_ok: False. If the solver cannot satisfy constraints, it must adjust trades until all pass, or produce a sells-only plan (with all constraints passing in the post-trade state). A plan with violations is a solver bug, not a gate issue. - Config read path must use absolute paths — The sweep script must read
config.jsonfrom{agent_root}/commons/data/ocas-rally/config.jsonusing an absolute path. If the script cannot find the config, it defaults toexecution.enabled=falsewhich silently blocks all trade execution. Log the resolved config path and gate config values in every sweep. - Execution must be explicitly enabled for learning — The default config has
execution.enabled: false. Without live/paper fills, the learning loop (allocation → trade → fill → P&L attribution → factor IC update → next allocation) never closes. As of 2026-06-17: only 1 execution attempt ever (rejected), 0 filled trades,factor_ic.jsonlempty. Enable execution (paper first) before expecting any model improvement. Seereferences/attribution-backtest.mdfor the replay/attribution design. - Verify data before concluding — When asked about system state (e.g., "are trades happening?"), always query the actual data sources (Alpaca API, JSONL files) before reporting. Do not assume from config alone. The config said
execution: falsebut Alpaca had 48 fills; thefactor_ic.jsonlwas empty butallocation_plans.jsonlhad 20 plans. Cross-reference at least two independent data sources before rendering a verdict. Always query the broker API directly for trade history — do not rely on local JSONL files which may be stale or written to a different path. Usecurlto Alpaca's/v2/ordersendpoint as the source of truth for recent trades. Search methodology: when looking for rally data files, always resolveHERMES_HOMEfirst (echo $HERMES_HOME) and use that path. Do NOT usefind /root -path "*ocas-rally*"and take the first hit — the indigo-repo backup copy will appear first alphabetically and is always stale. The correct path is$HERMES_HOME/commons/data/ocas-rally/. Seereferences/execution-pipeline-diagnosis-2026-06-17.mdfor the full execution pipeline diagnosis. - Yahoo Finance quoteSummary API returns 404 from this IP — Do NOT use
query1.finance.yahoo.com/v11/finance/quoteSummary/orquery2for fundamentals. Use Finnhub metrics instead. The chart API (/v8/finance/chart/) works fine. - Alpaca data API works —
https://data.alpaca.markets/v2/stocks/{symbol}/barsworks from this IP for historical bars (used by backtest). The trading API (paper-api.alpaca.markets) also works. Use data API for backtest price history, trading API for orders/positions. - No fee model in P&L — Rally currently tracks P&L without accounting for trading costs. SEC fees (
$22.90 per $1M sold), FINRA TAF ($0.000119/share), and spread/slippage (~10 bps) are NOT deducted from reported P&L. For accurate backtest results, apply fee model fromreferences/fee-model.md. - Russell 2000 GitHub CSV is dead —
raw.githubusercontent.com/datasets/russell-2000-stocks/main/data/constituents.csvreturns 404 as of 2026-06-14. Use a curated list of mid-cap names or source from Wikipedia. - Data directory is profile-specific — Rally data lives at
{HERMES_HOME}/commons/data/ocas-rally/which resolves to the active profile's commons (e.g.,/root/.hermes/profiles/indigo/commons/data/ocas-rally/), NOT/root/.hermes/commons/data/ocas-rally/or/root/indigo-repo/commons/data/ocas-rally/. Therally_data_sources.pymodule usesHERMES_HOMEenv var for this. Always verify the resolved path at runtime withget_rally_data_dir(). When checking for recent trades or system state, always query the broker API directly — local JSONL files may be stale, out of sync, or written to a different path than expected. The broker API (Alpaca/v2/orders) is the source of truth. Seereferences/data-paths.mdfor the full path map. - HERMES_HOME may already include the profile path — When
HERMES_HOMEis/root/.hermes/profiles/indigo, do NOT appendprofiles/indigoagain. Check ifprofilesis already in the path before constructingRALLY_DATA. Double-pathing causes all file lookups to fail silently. ALPACA_BASE_URLmay lack/v2suffix — The env varALPACA_BASE_URLcan be set tohttps://paper-api.alpaca.markets(no/v2). The script's fallbackor 'https://paper-api.alpaca.markets/v2'does NOT kick in when the env var is set but missing/v2. Result: all Alpaca calls construct URLs likehttps://paper-api.alpaca.markets/positions→ HTTP 404. Fix: normalize at read time withbase_url = raw_base if raw_base.rstrip('/').endswith('/v2') else f'{raw_base.rstrip("/")}/v2'. Applied torally_pre_close.pyline 35 (2026-06-19). Check all other scripts that readALPACA_BASE_URLfor the same pattern.- Env must load before module-level key variables — In Rally scripts,
load_env()must be called BEFOREFINNHUB_KEY = os.environ.get(...)at module level. Module-level variables are evaluated at import time; ifload_env()runs later inmain(), the keys will be empty. Callload_env()right after the function definition, before reading any keys. - Dossiers are JSON-first, markdown-second — Research dossiers are stored as
research_dossiers/{SYMBOL}.json(primary, machine-readable) +.md(compact human summary). The JSON dossier is the canonical format consumed by the allocation solver, rationale generator, and factor IC computation. The markdown file is a thin summary for quick human inspection only. Seereferences/research-memory.mdfor the JSON schema. - Weekend research script mapping —
rally_weekend_research.pyand the individual step scripts (rally_step1_universe.pythroughrally_step4_journal.py) do NOT exist. Run the pipeline viarally_deep_research.py --fill-gaps,rally_rebuild_universe.py,rally_score_new.py,rally_batch_dossiers.py,rally_allocate.pyin sequence. Seereferences/weekend-research.mdandreferences/cron-research-procedure.md. - NXP price anomaly pattern — YF chart API can return prices for the wrong instrument (ADR, renamed ticker, foreign listing). NXP showed $14.29 in signals vs ~$300 on NYSE. Run Finnhub
/quotecross-check after scoring; flag >50% discrepancies. Allocation solver should use Finnhub live price for order sizing, notsignals.jsonlprice. Seereferences/price-anomaly-patterns.md. - Cash deployment scope in allocation solver — When deploying excess cash, scale new weights to
excess_cash_pct(current cash % − target cash %), NOT toinvestable(1 − target cash %). Usinginvestabledeploys 100% of portfolio value in new buys, driving cash negative. Compute:excess_cash_pct = max(0, current_cash_pct − cash_target_pct), then scale new position weights so their sum equalsexcess_cash_pct. Current holdings weights are untouched; only new buys are scaled. - Yahoo Finance chart API missing sector for mid-caps — The YF v8 chart endpoint (
/v8/finance/chart/{symbol}) often returns nosectorfield for mid-cap and small-cap symbols. When profiling new symbols from Wikipedia or other sources, do NOT rely on YF for sector data. Use Finnhub/stock/profile2for sector, or maintain a manual mapping. Unknown sectors cause the allocation solver's sector cap check to be meaningless (all "Unknown" symbols aggregate into one pseudo-sector). - Python urllib can hang on slow responses — Even with
timeoutparameter,urlopenmay hang on the read phase. Usecurlviasubprocess.runwith both--connect-timeoutand--max-timefor reliable timeout behavior. Seereferences/api-quirks.mdfor details. - Congressional flow env var mismatch — The
.envfile stores the key asMASSIVE_API_KEY, but the v3 config default saysAINVEST_API_KEY. This causes congressional flow to silently fail. Config defaults for v6+ useMASSIVE_API_KEY. If you see "congressional_flow" insignals_unavailable, check the env var name first. - Post-trade fill drift is real — After a sell plan fills, always recompute actual weights from fill prices before marking the action
complete. Projected weights from the allocation plan will differ from actual weights. A marginal breach (within 0.5pp) should be flagged asmarginal_breachin the daily report, not assumed resolved. - Cron step scripts (
rally_step1_universe.pythroughrally_step4_journal.py) do NOT exist. Therally:researchcron job prompt references these scripts but they were never created. The weekend research pipeline must be run manually via inline Python or separate scripts. Do NOT waste time looking for them — seereferences/weekend-research.mdfor the manual procedure. - Inline Python via
python3 -cis fragile with complex scripts. Nested quotes, backslash escapes, and f-strings with curly braces cause shell quoting failures that are hard to debug. For any script longer than ~10 lines or containing nested quotes: write to a.pyfile and execute it instead of using-c. This also makes the script reusable and testable. execute_codeis blocked in cron jobs — Cron jobs run without a user present to approve arbitrary Python. Useterminal()with inlinepython3 -c "..."or heredoc scripts instead. Never callexecute_codefrom a cron context.- API keys are in
{DATA_DIR}/.env, not~/.hermes/.env— Rally's.envlives at{agent_root}/commons/data/ocas-rally/.env(e.g.,/root/.hermes/profiles/indigo/commons/data/ocas-rally/.env). Alwayssourcethis file before making API calls. The keys areFINNHUB_API_KEY,MASSIVE_API_KEY,FRED_API_KEY,FMP_API_KEY. Alpaca keys are in~/.hermes/.env. - Stale queue deadlock pattern — When a rebalance plan fails (e.g., insufficient shares held for sells), subsequent research runs keep generating new plans that also fail because the underlying positions haven't changed. This creates a growing queue of
stagedactions that all hit the same error. Before generating a new plan, checkpending_actions.jsonlfor repeated failures on the same constraint. If the same action_id prefix fails 3+ times with the same error, the queue is deadlocked — fix the root cause (e.g., cancel conflicting orders, adjust share counts) rather than queuing another plan. - Cash deployment must not be gated on staged actions — The sweep script's cash deployment check (
cash_pct > target + 10pp) must run independently of whether staged actions exist. A staged action with a missing or stale trade plan will otherwise block cash deployment forever. The correct flow: (1) check cash drift and generate cash deployment orders, (2) process staged actions, (3) if a staged action can't generate trades but cash deployment orders exist, supersede the action, (4) submit cash deployment orders. Seereferences/sweep-script-pitfalls.mdfor the exact guard logic. - Allocation plan field name:
target_weightnottarget_pct— List-format allocation plans store target weights astarget_weight, nottarget_pct. The dict-format usestarget_pct. The sweep script'sgenerate_cash_deployment_trades()must checktarget_weightfirst, then fall back totarget_pct. If this is wrong, all cash deployment trades silently fail with "No allocation targets available." - Sweep must check Alpaca open orders before deploying —
generate_cash_deployment_trades()must queryGET /v2/orders?status=open&side=buyto account for already-placed orders. Without this, consecutive sweep runs will over-deploy and drive cash negative. Seereferences/sweep-script-pitfalls.mdfor the exact guard logic. - Sweep must fetch prices for new symbols —
generate_cash_deployment_trades()generates buy orders for symbols not in current holdings. Thepricesdict only covers held symbols. The sweep must callfetch_yf_price()for any target symbol with price=0, or all new-position buys are silently skipped. - Research cron must execute scripts, not summarize — The
rally:researchcron job must runscripts/rally_weekend_research.py(or equivalent) to produce actual allocation plans, trade plans, and pending actions. An LLM-only summary approach will not write the data files that the sweep needs to execute. The cron prompt must explicitly say "Execute the Python scripts directly — do NOT just summarize." - Pre-open script must deploy cash —
rally_pre_open.pymust callrally_sweep_and_execute.pyafter the health check if cash is >10pp above target. Without this, cash deployment only happens during research runs (every 24h) instead of every pre-open (every trading day). - Dossier structure for scoring — When reading fundamentals from dossiers in batch: fundamentals are at
dossier['metrics']['metric']['roeTTM'](double-nested), NOTdossier['metrics']['roeTTM']. Sector is atdossier['meta']['sector']ordossier['industry'], NOTdossier['sector']. Reading from wrong path silently returns None → all scores default to 0.500. Always verify with a sample dossier first. - Scoring formula pitfalls — When writing inline scoring scripts: (1) RSI must use a proper 14-period calculation, not a 5-day proxy that always returns 1.0. (2) Insider transaction data is in
data['data']— watch for typos likeinsandersvsinsiders. (3) Safety scores must be clamped to [0,1] — the PE formula1.0 - (pe - 10) / 40can go negative for high-PE stocks withoutmax(0, min(1.0, ...)). (4) Always verify post-trade constraints: a plan that fixes one violation (position cap) may worsen another (top-3 concentration). - Paper-to-live transition — When switching from paper to real money: (1) Change
config.execution.broker_modefrom"paper"to"live". (2) UpdateALPACA_BASE_URLtohttps://api.alpaca.markets/v2in~/.hermes/.env. (3) Replace paper API keys (PK- prefix) with live keys (AK- prefix). (4) Liquidate paper positions first — the allocation solver generates plans for the current portfolio, so old paper positions will produce incorrect trade plans at the new capital level. (5) After config change, runrally.researchto generate a fresh allocation plan sized for the new capital. (6) The sweep script, pre_open, and pre_close scripts all read config at runtime — no code changes needed beyond the config values. Seereferences/sweep-script-pitfalls.mdfor structural bugs fixed during the 2026-06-17 transition audit.
Platform notes
Rally uses the memory tool (3 references) for portfolio state. On platforms without memory, write pending actions and scores to references/portfolio-state.md via write_file/read_file. Market data API calls are platform-independent.
Support File Map
| File | When to read |
|---|---|
references/operating-model.md |
At the start of every scheduled wake — contains Portfolio State Refresh Procedure, safety gate, regime filter, position sizing |
references/api-field-map.md |
Finnhub & Yahoo Finance API field paths, JSON structure quirks, and common data pitfalls (2026-06-20) |
references/api-quirks.md |
Before any data source or network call — Yahoo Finance, Finnhub, Russell 2000 availability, Python urllib timeout workaround |
references/data-model.md |
Before writing any JSONL record — canonical schemas and field invariants |
references/research-and-scoring.md |
Before scoring candidates — universe filters, signal categories, sentiment chain |
references/dossier-generation.md |
Dossier generation commands, output format, quality levels, known issues (Python 3.14 json bug, HTTP 429, Massive SEC dead) |
references/research-memory.md |
Before reading/writing dossiers or emitting to Elephas |
references/config-defaults.md |
During rally.init or config validation |
references/journal.md |
At end of every run for rally.journal |
references/spec-ocas-recovery.md |
When modifying recovery behavior |
references/market-data-sources.md |
Before fetching market data |
references/alpaca-pitfalls.md |
Before Alpaca API code or debugging |
references/constrained-allocation-solver.md |
Before running the allocation solver |
references/plans/portfolio-rebalance.plan.md |
When running the portfolio rebalance workflow via Mentor |
references/storage-layout.md |
Before writing any file to disk — canonical directory structure and file paths |
references/data-paths.md |
Active vs stale data directories, path resolution, key file map |
references/okrs.md |
During performance review or scorecard generation |
references/self-update.md |
Before running rally.update or troubleshooting update failures |
references/execution-safety-gate.md |
Before submitting any trade — quick reference for all 10 gate clauses |
references/data-source-quirks.md |
Before fetching price bars — YF v7/v8 endpoint quirks, NVDA fallback, batch patterns |
references/weekend-research.md |
Before running rally_weekend_research.py — procedure, invariants, output artifacts |
references/cron-research-procedure.md |
When running rally:research from a cron job — step-by-step procedure, API key paths, scoring pitfalls, known endpoint status |
references/sweep-script-pitfalls.md |
When modifying sweep/pre_open/pre_close scripts — sector lookup, cash drift re-validation, trigger field location |
references/transition-audit-2026-06-17.md |
|
references/fee-model.md |
|
references/price-anomaly-patterns.md |
Price discrepancy patterns (NXP $14 vs $300), dual-listed/ADR ticker detection, Finnhub cross-check procedure |
references/backtest/README.md |
|
references/attribution-backtest.md |
|
references/backtest/model_proposal.json |
|
references/backtest/backtest_results.json |
|
references/backtest/factor_ic_backfill.json |
If a concept is described in both SKILL.md and a reference file, the reference file is authoritative.
Dossier format
See references/dossier-schema.md for the canonical JSON dossier schema, quality levels, risk flag format, projection model details, and generation commands.
Analysis Style
When producing market assessments, probability estimates, or bet recommendations:
- Research the data first (price history, market signals, fundamentals)
- Produce a probability estimate mechanically from the data
- Compare to market price to identify edge
- Recommend the bet mechanically — what to buy/sell, how much, and at what price
- Never ask the user for their opinion on the assessment. The model outputs the recommendation.
- Be concise: what/how much/link, not edge theory
Dossier format preference: Machine-readable JSON first, compact markdown summary second. Dossiers are consumed by the allocation solver and factor IC computation — not human readers. No lengthy prose. Keep structured, queryable, and dense.
When to Use
- Run overnight research and produce an allocation plan
- Pre-open health check: update stops, execute queued orders
- Pre-close health check: enforce stops, log daily report
- Validate portfolio state against risk constraints
- Produce daily or monthly portfolio reports
- Ad-hoc signal scoring or candidate ranking
- Inspect the pending-action queue
When NOT to Use
- Meme-stock speculation or hype-driven picks
- Margin, shorting, or leveraged products (unless explicitly enabled)
- Budget or personal finance planning
- Generic financial news summarization (use Sift)
Responsibility boundary
Rally owns governed portfolio research, scoring, allocation, automated trade execution within configured risk limits, stop enforcement, and the pending-action queue.
Rally does not own: general web research (Sift), knowledge graph (Elephas), communications (Dispatch), pattern analysis (Corvus). Rally depends on Sift for sentiment enrichment; if Sift is absent, sentiment scoring is skipped.
Data source priority
scripts/rally_data_sources.py provides a unified interface with automatic graceful degradation. Every source is optional — a missing source never crashes a run. See references/market-data-sources.md for the full priority table.
Weekend research: Use scripts/rally_weekend_research.py for the full research cycle (universe refresh → scoring → dossiers → allocation → journal). The individual step scripts (rally_step1_universe.py through rally_step4_journal.py) are thin wrappers that delegate to this script.
| Signal | Weight | Primary sources | Reach supplement |
|---|---|---|---|
| Quality | 40% | Finnhub+Massive+FRED+YF (roeTTM, margins, debt/equity, FCF) | sec_edgar (raw filings), bls (sector labor data) |
| Safety | 25% | Finnhub+Massive+FRED+YF (PE, revenue growth, EPS consistency) | federal_register (regulatory risk), courtlistener (litigation) |
| Momentum | 15% | YF price data (6mo/12mo returns, SPY-relative strength) | — |
| Reversion | 10% | YF price data (5-day RSI) | — |
| Congressional Flow | 5% | Massive Form 3/4 + Finnhub insider transactions | congress_gov (legislative risk), campaign finance via Reach |
| Sentiment | 5% | Finnhub news + analyst recommendations | gdelt (global news events) |
| Regime | — | FRED HY OAS + VIX, YF SPY SMA | bls (employment/CPI), world_bank (global macro) |
Degradation: Finnhub → FMP → price-based proxy. Massive + Finnhub → either alone → null. Reach degradation: per-source fallthrough. If Reach is unavailable entirely, Rally falls back to its existing API-only chain without regression.
Ontology mapping
- Thing — securities (stocks, ETFs, funds). Emitted to Elephas after each research run.
- Concept/Event — market events (earnings, dividends, rebalancing, emergency sells). Recorded in
research_events.jsonlanddecisions.jsonl. Material events emitted to Elephas.
State lives in {agent_root}/commons/data/ocas-rally/. Chronicle is the long-term knowledge substrate; Rally's dossiers are short-term working memory (see references/research-memory.md).
Daily reports at {agent_root}/commons/data/ocas-rally/reports/YYYY-MM-DD-daily.json conform to PortfolioOutcomeRecord schema, read by Vesper.
Commands
Every scheduled command runs the Portfolio State Refresh Procedure from references/operating-model.md before any other work. Do not skip this step; six bug classes are prevented by running the procedure and its step-6 validation.
Critical: read references BEFORE executing. At the start of every
rally.researchorrally.healthcheckrun, readreferences/operating-model.mdviaskill_viewbefore writing any data. Field-level constraints prevent data corruption bugs (C1–C4, M1).
rally.init— one-time initialization; runs automatically on first invocation.rally.research— full overnight sequence.rally.healthcheck pre_open— pre-open health check.rally.healthcheck pre_close— pre-close health check. Reference:scripts/rally_pre_close.py.rally.ingest.portfolio— ad-hoc portfolio refresh.rally.universe.refresh— rebuild investable universe.rally.research.signals— compute signals and composite rankings.rally.research.sentiment— compute sentiment signals via Sift.rally.research.dossier <ticker>— print or refresh a research dossier. Dossiers are stored as JSON (canonical) + markdown (summary) inresearch_dossiers/.rally.candidates.rank— ranked candidate list.rally.plan.allocation— constrained allocation plan.rally.plan.trade— trade plan from current-vs-target deltas; writes to pending-action queue.rally.plan.deploy <pct>— deploy excess cash into existing target allocation.pctis the portion of excess cash to deploy (0.0-1.0, default 1.0 = all excess). Uses the most recent AllocationPlan targets for weight ratios among buys. This is the command the cash drift trigger and sells-only follow-up invoke — it generates a buy-only trade plan that moves cash towardcash_target_pctwithout changing target weights.rally.execute.trades— pending-action sweep and broker submission. Subject to execution safety gate.rally.pending.list— list all pending actions.rally.pending.inspect <action_id>— show condition snapshot and re-validation history.rally.pending.cancel <action_id>— manually cancel a pending action.rally.report.daily— daily portfolio report.rally.report.monthly— monthly performance attribution.rally.validate— risk and constraint validation.rally.status— portfolio summary, active plan, pending actions, risk/cron status.rally.journal— write journal for the current run; called at end of every run.rally.update— pull latest from source repository; preserves journals and data.
Scripts
| Script | Purpose |
|---|---|
scripts/rally_deep_research.py |
Deep research dossier generator — fetches Finnhub profile/metrics/recommendations/insider/news/earnings + Yahoo Finance price data per symbol, produces standardized JSON dossiers with signal history, fundamentals, projection models, risk flags. Run with --top N for top N symbols, --fill-gaps to re-fetch missing fields, --skip-existing to skip already-done symbols. See references/dossier-generation.md for usage. |
scripts/rally_batch_dossiers.py |
Batch dossier generation for multiple tickers — wraps rally_deep_research.py --tickers X for sequential execution. Use when --tickers only accepts one symbol at a time. |
scripts/rally_rebuild_universe.py |
Rebuild investable_universe.jsonl from Wikipedia S&P 400/500 scrape + YF chart API. Use when universe file is missing or needs expansion. |
scripts/rally_score_new.py |
Score new universe symbols not yet in signals.jsonl via price-proxy (YF v8 chart). Handles delisted/missing symbols gracefully. |
scripts/rally_allocate.py |
Constrained allocation solver — reads signals + universe sectors + Alpaca portfolio state, produces allocation plan. |
scripts/rally_enrich_sectors.py |
Enrich universe symbols with Finnhub finnhubIndustry sector data. |
scripts/rally_pre_open.py |
Pre-open healthcheck — portfolio refresh, pending-action sweep, dual stop check, emergency sells. |
scripts/rally_pre_close.py |
Pre-close healthcheck — portfolio refresh, pending-action sweep, dual stop check, regime, congressional flow, daily report. |
scripts/rally_sweep_and_execute.py |
Pending-action sweep and trade execution — validates, runs safety gate, submits to Alpaca. |
scripts/rally_ingest_alpaca.py |
Quick portfolio snapshot for Vesper. Do NOT use for canonical refresh (has C1–C4 bugs). |
scripts/rally_congressional_flow.py |
Congressional/insider flow — SEC Form 3/4 via Massive API. |
scripts/rally_fundamental_data.py |
Fundamental data — Finnhub metrics for Quality/Safety signals. |
scripts/self_update.py |
Weekly self-update from source repository. |
scripts/rally_weekend_research.py |
Full weekend research cycle — universe refresh → scoring → dossiers → allocation → journal. Does NOT execute trades. |
scripts/rally_deep_research.py |
Deep research dossier generator — fetches Finnhub profile/metrics/recommendations/insider/news/earnings + Yahoo Finance price data per symbol, produces standardized JSON dossiers with signal history, fundamentals, projection models, risk flags. Run with --top N for top N symbols, --fill-gaps to re-fetch missing fields, --skip-existing to skip already-done symbols. See references/dossier-generation.md for usage. |
scripts/extract_trades.py |
Extract Alpaca fills + signals into unified backtest dataset. |
scripts/backtest.py |
Compute round-trip P&L, factor attribution, win/loss analysis. |
scripts/analyze.py |
Deep analysis: IC per factor, bucket analysis, model proposal. |
scripts/backfill_ic.py |
Backfill factor IC from historical plans + Alpaca bars. |
scripts/fetch_forward_returns.py |
Fetch Alpaca historical bars for forward return computation. |
Recovery
Rally implements the full recovery model defined in spec-ocas-recovery.md. See references/spec-ocas-recovery.md for the detailed spec mapping.
Core mechanisms:
- Durable Intent Queue (
pending_actions.jsonl): state machine:staged → in_progress → complete | superseded | stale | cancelled. - Execution Evidence Log (
decisions.jsonl): every run writes a decision entry, including no-ops. - Schedule Gap Detection: gaps >28h (research) or 8-32h (healthchecks) trigger remedial passes.
- Self-Repair: gate failures keep actions
stagedfor retry. Stale eviction at ≥10 attempts AND ≥72h old. - Idempotency: fresh-data re-validation before every sweep.
execution_log.jsonlprevents duplicate submissions.
Queue processing: Every job starts with a pending-action sweep. Re-evaluate each staged/in_progress action against fresh data; submit if gate passes, otherwise increment attempts. New plans supersede old staged plans. Emergency sells never supersede each other.
Execution safety gate
A trade plan is submitted to the broker only when every clause is satisfied. Any failure keeps the action staged for the next wake. See references/execution-safety-gate.md for the full clause list and references/operating-model.md for detailed clause logic.
Emergency sells bypass clause 6 (turnover budget) and minimum rebalance interval, but still require clauses 1, 2, 8, 10.
Hard boundaries
- Long-only unless explicitly configured otherwise
- No margin, shorting, leverage, or derivatives unless enabled
- No assumed external deposits; growth from returns and reallocation only
- Gate failures keep actions in the pending queue (never discarded)
- Every target has rationale and evidence references
Storage layout
See references/storage-layout.md for the full directory tree and file paths.
OKRs
See references/okrs.md for the current OKR definitions and targets.
Optional skill cooperation
- Sift — sentiment enrichment, news ingestion, ad-hoc lookups. If absent, 5% weight redistributed to Momentum.
- Vesper — cooperative read of daily reports during briefing generation.
- Elephas — Thing and Event signals emitted after each research run. Journals ingested by Elephas.
- Reach — live world-data query engine for ground-truth external data. Rally invokes Reach during research for: SEC filings (sec_edgar), congressional legislation (congress_gov), federal regulations (federal_register), macroeconomic indicators (fred, bls), litigation risk (courtlistener), government contracts (usaspending), and global news events (gdelt). Reach is the authoritative source for material non-market data; Rally's own API calls handle price, fundamentals, and flow data. If Reach is unavailable, Rally falls back to its existing per-source degradation chain for each category.
- Corvus — pattern analysis engine. Rally emits cross-skill Thing/Event signals to Chronicle; Corvus may surface patterns from trading history, market events, or behaviors that inform future allocation decisions. Corvus is read-only for Rally — it never modifies portfolio state.
Journal outputs
- Observation Journal — research, signals, sentiment, ranking, allocation/trade plans, reports, validate, status, pending list/inspect
- Action Journal — healthchecks, trade execution, update, pending cancel, any order submission or queue mutation
Path: {agent_root}/commons/journals/ocas-rally/YYYY-MM-DD/{run_id}.json. Written atomically (.tmp then rename). Never edited after write.
Decision payloads: events_observed, things_observed, orders_submitted, gate_failures, pending_action_changes, schedule_gap, sift_used, data_freshness.
Initialization
On first invocation, rally.init:
- Create data, reports, and dossier directories
- Write default
config.jsonif absent - Create empty JSONL files (portfolio_state, research_events, signals, decisions, allocation_plans, trade_plans, pending_actions, execution_log, wash_sale_exclusions, factor_ic, congressional_flow_cache, news_cache, sentiment_cache)
- Create journals directory
- Copy rebalance plan to Mentor if installed (never overwrite)
- Register cron jobs from frontmatter if absent (idempotent)
- Verify credentials; log warnings for missing optional ones
- Log initialization as a DecisionRecord
Background tasks
Four scheduled jobs. All timestamps America/Los_Angeles.
| Job name | Schedule (PT) | Purpose |
|---|---|---|
rally:research |
0 2 * * 1-5 (02:00 weekdays) |
Pending-action sweep, research, sentiment, rebalance, plan generation |
rally:healthcheck-pre-open |
30 5 * * 1-5 (05:30 weekdays) |
Sweep with fresh-data re-validation, ATR/trailing_high refresh, stops, submit orders |
rally:healthcheck-pre-close |
0 12 * * 1-5 (12:00 weekdays) |
Sweep, stops, emergency sells, daily PortfolioOutcomeRecord |
rally:update |
0 3 * * 0 (03:00 Sunday) |
Weekly self-update |
Registration during rally.init: query scheduler for each job name; register from frontmatter if absent.
Market holidays: jobs detect closure via broker clock endpoint, log decision_type: "market_closed", exit early. Pending actions remain queued.
Self-update
rally.update pulls the latest package from the source: URL in frontmatter. See references/self-update.md for the full procedure.
Visibility
public