fact-checker

name: fact-checker description: "Use when verifying claims in a journalism-investigation KB. Evaluates unverified claims, searches for corroborating evidence, updates claim status and confidence, and creates evidence entries linking claims to sources."

Fact-Checker Skill

Claim verification methodology for journalism-investigation Pyrite knowledge bases. Produces a verification report with traceable evidence chains and confidence assessments.

Announce at start: "I'm using the fact-checker skill."

Methodology: Identify → Gather → Document → Evaluate → Cross-Check

The non-negotiable rule: Never change a claim's status without creating at least one evidence entry first. The evidence chain must be traceable: claim → evidence → source. No shortcuts.

Phase 1: Identify Claims to Verify

Find unverified claims and select which to process.

Using MCP tools (preferred when available):

Use the investigation_claims MCP tool to list claims filtered by status.

Using CLI:

# List all unverified claims
pyrite investigation claims --status unverified -k <kb>

# See all claims to understand the landscape
pyrite investigation claims -k <kb>

Prioritization rules:

1. High-importance claims first (claims tagged priority:high or linked to active investigation threads)
2. Among equal importance, oldest unverified claims first
3. Claims with partial evidence already collected come before cold claims

Present to user:

UNVERIFIED CLAIMS FOUND: <count>

Recommended next:
1. [[claim-id-1]] — "<assertion>" (importance: high, age: 14 days)
2. [[claim-id-2]] — "<assertion>" (importance: medium, age: 30 days)
3. [[claim-id-3]] — "<assertion>" (importance: medium, age: 7 days)

Select a claim to verify, or I'll start with #1.

If the user does not specify, begin with the highest-priority claim.

Phase 2: Gather Evidence

Collect everything relevant to the claim under review.

2a: Read the claim

pyrite get <claim-id> -k <kb>

Extract from the claim entry:

• The assertion text (the specific factual statement)
• Named entities (people, organizations, places, dates)
• Any existing evidence_refs already linked
• The source that originated this claim

2b: Check existing evidence chain

Use the investigation_evidence_chain MCP tool to see what evidence already exists for this claim.

pyrite investigation evidence-chain <claim-id> -k <kb>

If evidence entries already exist, read each one. Do not duplicate work.

2c: Search for corroborating evidence

Search broadly, then narrow. Use multiple strategies:

# Search the assertion text directly
pyrite search "<claim assertion>" -k <kb>

# Search for key entities mentioned in the claim
pyrite search "<person name>" -k <kb>
pyrite search "<organization>" -k <kb>

# Semantic search for conceptually related content
pyrite search "<claim assertion>" -k <kb> --mode semantic

# Cross-KB correlation search
pyrite investigation search --correlate -k <kb>

# Find source documents that might corroborate
pyrite investigation sources -k <kb>

# Look up entities mentioned in the claim
pyrite investigation entities -k <kb>

# Search across all investigation KBs
pyrite investigation search-all "<key phrase>"

2d: Search for contradicting evidence

This is equally important. Actively look for information that challenges the claim:

# Search for the opposite or alternative
pyrite search "<negation or alternative>" -k <kb>
pyrite search "<entity> NOT <claimed relationship>" -k <kb> --mode semantic

2e: Follow the source chain

For each piece of evidence found:

1. Read the full entry: pyrite get <entry-id> -k <kb>
2. Check what source document backs it: follow source_document refs
3. Assess the source tier:

4. Check if the source is independent (not derived from the same origin as another source)

Gather completeness check:

- [ ] Read the claim entry fully
- [ ] Checked existing evidence chain
- [ ] Searched claim assertion text (keyword + semantic)
- [ ] Searched each named entity in the claim
- [ ] Used cross-KB correlation search
- [ ] Looked for contradicting evidence
- [ ] Assessed source tier for each piece of evidence
- [ ] Verified source independence (no circular sourcing)

Phase 3: Create Evidence Entries

For every piece of evidence found -- corroborating or contradicting -- create an evidence entry. This is mandatory. The evidence chain must be documented before any status change.

pyrite create -k <kb> -t evidence --title "<concise description of what this evidence shows>" \
  -b "<detailed description of the evidence, what it says, where it comes from>" \
  -f evidence_type=document \
  -f source_document="[[source-id]]" \
  -f reliability=<high|medium|low> \
  --tags "evidence,<claim-topic>"

Evidence types:

Reliability assessment:

After creating evidence entries, link them to the claim by updating the claim's evidence_refs:

pyrite update <claim-id> -k <kb> -f evidence_refs="[[evidence-id-1]],[[evidence-id-2]]"

Phase 4: Evaluate and Update Claim

Apply verification rules based on the evidence collected. These rules are not suggestions -- follow them consistently.

Verification decision matrix

Update the claim

pyrite update <claim-id> -k <kb> \
  -f claim_status=<new-status> \
  -f confidence=<high|medium|low>

Record the reasoning

Add a verification note to the claim body explaining why this status was assigned. Include:

• How many independent sources were found
• What tiers those sources represent
• Whether any contradictions exist
• What gaps remain

Phase 5: Cross-Claim Contradiction Detection

After verifying a claim, check whether it contradicts other claims in the KB. This step catches inconsistencies that single-claim verification misses.

# Search for other claims about the same entities
pyrite investigation claims -k <kb>
pyrite search "<entity from verified claim>" -k <kb> --mode semantic

# Check for claims about the same event or relationship
pyrite search "<event or relationship>" -k <kb>

For each potential contradiction found:

1. Read both claims fully
2. Determine if they genuinely contradict (not just different aspects of the same truth)
3. If contradicted, update both claims:

# Add disputed_by reference to each claim
pyrite update <claim-id-1> -k <kb> -f disputed_by="[[claim-id-2]]"
pyrite update <claim-id-2> -k <kb> -f disputed_by="[[claim-id-1]]"

4. Consider whether the contradiction changes either claim's status

Verification Report

After completing all phases, produce a verification report. This is the primary output.

## Claim: <title>
ID: [[claim-id]]
Assertion: <assertion text>
Previous status: <old-status> → New status: <new-status>

### Evidence found:
1. [[evidence-id-1]] — <description> (source: [[source-id]], tier: 1, reliability: high)
2. [[evidence-id-2]] — <description> (source: [[source-id]], tier: 2, reliability: medium)

### Confidence: <high|medium|low>
Reason: <why this confidence level — cite number of sources, tiers, independence>

### Contradictions: <none found | list of contradicting claims>

### Gaps:
- <what evidence would strengthen or weaken this claim>
- <what sources were searched but yielded nothing>

Key MCP Tools Reference

These are the JI plugin MCP tools available for claim verification:

Anti-Patterns