By name: brave-headless
description:
Headless web search and content extraction via the Brave Search API.
Features exponential-backoff retry, circuit breaker fault isolation, bounded-concurrency
parallel page fetching, structured leveled logging, and smart paragraph-boundary truncation.
No browser required. Use for web research, documentation lookup, URL content extraction,
and any workflow requiring scriptable, non-interactive web search.
version: 0.2.0
license: MIT
compatibility: Requires Node.js >= 20 and npm. Works on macOS and Linux.
metadata:
author: kelexine
homepage: https://github.com/kelexine/brave-headless
openclaw:
requires:
env:
- BRAVE_API_KEY
bins:
- node
- npm
primaryEnv: BRAVE_API_KEY
emoji: "🔍"
homepage: https://github.com/kelexine/brave-headless
os:
- macos
- linux
install:
- kind: node
package: "@mozilla/readability"
bins: []
- kind: node
package: jsdom
bins: []
- kind: node
package: turndown
bins: []
- kind: node
package: turndown-plugin-gfm
bins: []
brave-search
Headless web search and content extraction via the Brave Search API.
Setup
Run once before first use:
cd <skill-root>
npm ci
Required environment variable:
export BRAVE_API_KEY="your-key-here"
Get a free API key at brave.com/search/api.
Usage
Search
node scripts/search.js "query" # Basic (5 results)
node scripts/search.js "query" -n 10 # Up to 20 results
node scripts/search.js "query" --content # Include page content
node scripts/search.js "query" -n 3 --content # Combined
node scripts/search.js "query" --json # Newline-delimited JSON
node scripts/search.js --help # Full options + env vars
Extract page content
node scripts/content.js https://example.com/article
node scripts/content.js https://example.com/article --json
node scripts/content.js https://example.com/article --max-length 8000
Output format (plain text)
--- Result 1 ---
Title: Page Title
URL: https://example.com/page
Snippet: Description from Brave Search
Content:
# Page Title
Extracted markdown content...
--- Result 2 ---
...
Pass --json to get one JSON object per line instead, suitable for piping.
Exit codes
| Code | Meaning |
|------|--------------------------------------------------|
| 0 | Success |
| 1 | Invalid input or configuration error |
| 2 | Page had no extractable content (content.js) |
| 130| Interrupted (SIGINT) |
Configuration (environment variables)
All behaviour is configurable without touching code:
| Variable | Default | Description |
|------------------------|----------|------------------------------------------------|
| BRAVE_API_KEY | — | Required. Brave Search subscription token |
| LOG_LEVEL | info | debug · info · warn · error · silent |
| LOG_JSON | false | Emit logs as newline-delimited JSON to stderr |
| FETCH_TIMEOUT_MS | 15000 | Per-page fetch timeout |
| SEARCH_TIMEOUT_MS | 10000 | Brave API call timeout |
| MAX_CONTENT_LENGTH | 5000 | Max chars of extracted content |
| MAX_RETRY_ATTEMPTS | 3 | Retry attempts on transient errors |
| RETRY_BASE_DELAY_MS | 500 | Base delay for exponential backoff |
| RETRY_MAX_DELAY_MS | 30000 | Backoff delay cap |
| CONCURRENCY_LIMIT | 3 | Parallel page fetches when --content is set |
| CB_FAILURE_THRESHOLD | 5 | Consecutive failures before circuit opens |
| CB_RESET_TIMEOUT_MS | 60000 | Circuit breaker reset window |
All variables are validated at startup — misconfigured runs fail immediately with a descriptive
list of every bad value rather than crashing mid-execution.
Architecture
See references/ARCHITECTURE.md for a full module breakdown.
scripts/
├── search.js ← Search CLI entry point
├── content.js ← Content extraction CLI entry point
├── content-fetcher.js ← HTTP fetch + Readability + DOM fallback
├── config.js ← Schema-validated env config
├── circuit-breaker.js ← Fault isolation (CLOSED → OPEN → HALF_OPEN)
├── retry.js ← Exponential backoff with full jitter
├── concurrency.js ← Bounded parallel execution pool
├── utils.js ← htmlToMarkdown, smartTruncate, parseURL
├── logger.js ← Structured leveled logger → stderr
└── errors.js ← Typed error hierarchy