Bullshit Detector
Bullshit Detector is an open-source collection of portable agent skills that fact-check online content. It turns any URL—video, article, tweet, or PDF—into a structured report where every claim is individually searched, sourced, and scored from 0 to 10. The tool is implemented as plain Markdown instructions plus self-contained Python scripts, requiring no API keys or paid endpoints. It works with any AI agent harness that supports web search, including Claude Code, Codex, and OpenCode. Installation is simple via npx or as a plugin. The workflow consists of four stages: fetch (extracting clean text and metadata), extract (isolating discrete claims and classifying them by importance), verify (performing a separate web search per claim and requiring sources for every verdict), and score (assigning a BS rating, hype-signal scan, and incentive analysis). Verdicts are categorized as Confirmed, Plausible, Misleading, False, or Unverifiable, each backed by evidence links. A built-in recount step checks the report’s arithmetic. The project includes seven separate skills: the main fact-checker, a content fetcher, a coverage checker (counting independent origins), a summarizer, an explainer, a report-card renderer, and a social media share tool. This modular design allows users to compose custom verification pipelines. Bullshit Detector is honest about its limits: it checks factual premises but not logical reasoning, can only cite what web crawlers can reach, and lacks a formal evaluation harness. Despite these caveats, it serves as a transparent, privacy-respecting alternative to black-box fact-checking services.
- Category
- Content
- Pricing
- Free
Problem solved
Misinformation and unsupported claims spread rapidly across social media, videos, and articles, but verifying them is time-consuming and often requires manual searching. Bullshit Detector automates claim-by-claim fact-checking with web-sourced evidence, giving users a quick, transparent credibility score and exposing hype without relying on proprietary APIs or black-box models.
Use cases
- Fact-check viral tweets or social media posts
- Verify claims in YouTube videos or TikTok captions
- Audit articles or blog posts before sharing
- Pre-publish self-check for content creators
- Analyze PDF reports for unsupported assertions
- Teach media literacy by demonstrating how to source-check claims
- Compare news stories to count original sources vs. rewrites
Audience
- Developers
- ML Engineers
- Data Scientists
- Researchers
- Content Creators
Pros
- Free and open source (MIT license) with no API keys needed
- Installs quickly via npx or Claude Code plugin
- Provides sourced verdicts for every claim, not just a summary
- Modular skills allow custom fact-checking workflows
- Transparent about limitations and failure modes
Cons
- Cannot evaluate reasoning validity—only checks factual premises
- Restricted to sources reachable by web crawlers; paywalled or blocked sites are excluded
- No formal evaluation harness; accuracy metrics are not yet available
- Requires technical comfort with CLI and agent environments
- May produce false negatives if search results are outdated or low-quality
Official website: https://korniienko.dev/bullshit-detector
Canonical LambdaBase page: https://www.lambdabase.com/tools/bullshit-detector