Multi-Agent Misinformation Analysis System
DSC 180B Capstone · Winter 2026
HOME

A Multi-Agent System for Detecting Misinformation and Explaining How Online Text Misleads

Distinguishing fact from fiction has become increasingly difficult with the rise of artificial intelligence and an overwhelming amount of online information. As major tech companies scale back their content verification programs, the burden of credibility assessment has fallen entirely on individual users. Our project addresses this critical gap by presenting a hybrid machine learning and multi-agent system for factuality evaluation of online text. By evaluating distinct factuality dimensions, this work empowers users to independently verify the reliability of online content and contributes toward a more informed digital ecosystem.

Overview

What we built

We developed an interactive factuality evaluation web app that evaluates the credibility of news articles. Powered by a multi-agent architecture and four predictive machine learning models, the system scores text across multiple factuality dimensions and provides users with detailed reasoning to help them identify misleading rhetoric.

View GitHub repository Jump to key results

Problem & Motivation

Over the last decade, the infrastructure that once helped maintain a fact-based web has been rapidly weakening. Major platforms have scaled back fact-checking programs due to political pressure, regulatory conflicts, and declining trust in content moderation. At the same time, advances in artificial intelligence and algorithmic recommendation systems have made misinformation easier to produce and faster to spread. Traditional solutions like manual fact-checking cannot keep pace with the scale and speed of modern online content.

When misinformation spreads faster than it can be verified, people are left to judge credibility on their own without reliable tools. This project explores a building multi-agent system that evaluates factuality across multiple factors including frequency heuristic, malicious account, and naive realism to determine it's truthfulness. By breaking credibility analysis into specialized agents, the system provides transparent and interpretable assessments that help online news consumers better evaluate the reliability of online information.

Approach

Multi-Agent Architecture
We build our system in three layers using Google’s Agent Development Kit (ADK): (1) a claim extraction and verification pipeline augmented with web search, (2) a parallel factuality analysis layer with four specialized LLM agents equipped with machine learning models, and (3) a sequential aggregation layer that merges scores into an overall credibility assessment and explanation.
See: Methods Multi-agent ADK
Machine Learning Models
Using the LIAR-PLUS dataset, we trained four machine learning models to score frequency heuristic, sensationalism, malicious accounts, and naive realism based on linguistic features and metadata. These models are exposed to the agents as callable agent tools.
See: Data ML as tools
Prompt Design
We ran a sequence of prompting experiments: zero-shot baselines, Chain-of-Thought, In-Context Learning tool calling, and four iterations of Fractal Chain-of-Thought (FCoT).
See: Experiments Prompt engineering
Evaluation
On our 40-article ground truth set, more structured prompts consistently beat naive baselines, especially for Malicious Account detection. However, Naive Realism remains challenging.
See: Results Limitations

So what?

Our system is not a binary “true / false” classifier. Instead, it analyzes multiple interpretable signals about how a piece of text may mislead and explains why it assigned each score. This is intended to:

  • Help readers spot rhetorical tricks like repetition-as-evidence and emotionally charged framing.
  • Give journalists and researchers a diagnostic lens for comparing outlets, articles, or time periods.