21.3. Consensus Mechanisms: The Wisdom of Silicon Crowds

Beyond the Single Inference

In classical Machine Learning, Ensemble Methods (like Random Forests or Gradient Boosting) are the undisputed champions of tabular data. They work on a simple principle: distinct models make distinct errors. By averaging their predictions, the errors cancel out, and the signal remains.

For years, LLMs were treated as “One-Shot” engines. You query GPT-4, you get an answer. But LLMs are stochastic engines. Even with temperature=0, floating-point non-determinism and massive parameter spaces mean that a single inference path is just one roll of the dice.

Consensus Mechanisms (or Voting Patterns) apply the principles of Ensemble Learning to Generative AI. Instead of asking one model once, we ask:

1. Self-Consistency: One model asked N times (with High Temperature).
2. Model Diversity: N different models asked once.
3. Prompt Diversity: N different prompts sent to one model.

The system then aggregates these outputs to find the “Consensus”. This technique is particularly potent for reasoning tasks (Math, Coding, Logic) where there is an objective “Correct” answer, but it can also be adapted for creative tasks to find the “Most Robust” path.

21.3.1. The Math of Majority Voting

Why does voting work? Let’s assume a model has an accuracy of $p = 0.6$ (60%) on a specific hard logic problem. If we run it once, our success rate is 60%.

If we run it 3 times and take the Majority Vote (2 out of 3 must agree):

• P(3 correct) = $0.6^3 = 0.216$
• P(2 correct) = $3 * (0.6^2 * 0.4) = 3 * 0.36 * 0.4 = 0.432$
• Total Success = $0.216 + 0.432 = 0.648$ (64.8%)

If we run it 5 times:

• Accuracy climbs to ~68%.

If we run it 11 times:

• Accuracy climbs to ~75%.

As $p$ increases, the boost from voting grows significantly. If base accuracy is 80%, a 5-vote majority pushes it to >90%. This is the Condorcet Jury Theorem.

Constraint: This only works if the errors are independent. If the model has a fundamental misconception (e.g., it believes the Earth is flat), asking it 100 times will just result in 100 wrong answers. This is why Model Diversity (asking Claude AND GPT-4) is often superior to Self-Consistency.

21.3.2. Architecture: The Parallel Voter

The implementation of Consensus is inherently parallel. It is the perfect use case for Python’s asyncio.

graph LR
    User --> Dispatcher
    Dispatcher -- "T=0.7" --> Model1[Model Call 1]
    Dispatcher -- "T=0.7" --> Model2[Model Call 2]
    Dispatcher -- "T=0.7" --> Model3[Model Call 3]
    Model1 --> Agg[Aggregator]
    Model2 --> Agg
    Model3 --> Agg
    Agg --> Final[Consensus Answer]

Reference Implementation: Async Self-Consistency

import asyncio
from collections import Counter
from openai import AsyncOpenAI

client = AsyncOpenAI()

async def generate_candidate(prompt: str, temperature=0.7) -> str:
    """Generates a single reasoning path."""
    response = await client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        temperature=temperature # High temp for diversity!
    )
    return response.choices[0].message.content

def extract_answer(text: str) -> str:
    """
    Parses the final answer from the reasoning.
    Assumes the model follows 'The answer is X' format.
    """
    if "The answer is" in text:
        return text.split("The answer is")[-1].strip(" .")
    return "UNKNOWN"

async def self_consistency_loop(prompt: str, n=5):
    print(f"Starting {n} parallel inferences...")
    
    # 1. Broad Phase: Parallel Generation
    tasks = [generate_candidate(prompt) for _ in range(n)]
    results = await asyncio.gather(*tasks)
    
    # 2. Reduce Phase: Answer Extraction
    answers = [extract_answer(r) for r in results]
    
    # 3. Vote Logic
    counts = Counter(answers)
    most_common, count = counts.most_common(1)[0]
    
    print(f"Votes: {counts}")
    print(f"Winner: {most_common} (Confidence: {count}/{n})")
    
    return most_common

# Usage
# prompt = "Solve: If I have 3 apples and buy 2 more, then eat 1, how many do I have?"
# asyncio.run(self_consistency_loop(prompt, n=5))

Latency vs Throughput

This pattern increases Throughput load on the API (N requests) but does NOT increase Latency (if run in parallel). The latency is max(t1, t2, ... tn), which is roughly equal to a single slow request. Cost, however, scales linearly by N.

21.3.3. Semantic Consensus (Soft Voting)

Hard voting (exact string match) works for Math (“4”) or Multiple Choice (“B”). It fails for Open-Ended QA.

• Answer A: “Washington DC is the capital.”
• Answer B: “The capital of the US is Washington D.C.”
• Answer C: “It’s DC.”

A Counter sees 3 unique strings. It fails to see the consensus.

We need Semantic Equivalence.

Algorithm: The Embedding Centroid

1. Embed all N answers ($v_1, v_2, …, v_n$).
2. Calculate pairwise cosine similarities.
3. Cluster them (DBSCAN or naive threshold).
4. The largest cluster is the Consensus.
5. Select the Medoid (the answer closest to the cluster center) as the representative text.

from sentence_transformers import SentenceTransformer, util
import numpy as np

embedder = SentenceTransformer('all-MiniLM-L6-v2')

def semantic_consensus(answers: list[str], threshold=0.8) -> str:
    embeddings = embedder.encode(answers)
    
    # Compute adjacency matrix
    # Who agrees with who?
    adjacency = np.zeros((len(answers), len(answers)))
    for i in range(len(answers)):
        for j in range(len(answers)):
            sim = util.cos_sim(embeddings[i], embeddings[j])
            if sim > threshold:
                adjacency[i][j] = 1
                
    # Sum rows to find "Centrality" (Degree Centrality)
    scores = np.sum(adjacency, axis=1)
    best_idx = np.argmax(scores)
    
    # If the best score is 1 (only agreed with self), NO Consensus.
    if scores[best_idx] == 1:
        return None
        
    return answers[best_idx]

This effectively finds the “Most Typical” answer among the generated set.

21.3.4. Pattern: The “MoE” (Mixture of External Experts)

Instead of asking one model 5 times, we ask 5 different models. This catches model-specific biases.

The Stack:

1. GPT-4o (The Generalist)
2. Claude 3.5 Sonnet (The Writer)
3. DeepSeek Coder V2 (The Hacker)
4. Llama-3-70B (The Open Source Baseline)

Scenario: “Write a Python script to scrape a website.”

• GPT-4 code: Uses BeautifulSoup.
• Claude code: Uses BeautifulSoup.
• DeepSeek code: Uses Scrapy.
• Llama code: Uses requests (broken).

Consensus Strategy: “CodeBERT Consistency”. Run unit tests on all 4 scripts.

• GPT-4: Pass.
• Claude: Pass.
• DeepSeek: Pass.
• Llama: Fail.

Now we have 3 valid solutions. Which do we pick? Heuristic: The shortest one? The one with most comments? Judge Model: Ask GPT-4 to rate the 3 passing scripts.

21.3.5. Deep Dive: “Universal Self-Consistency”

Paper: “Universal Self-Consistency for Large Language Models” The idea is to use the LLM to aggregate its own answers.

Prompt:

I asked you the same question 5 times and here are your 5 answers:
1. {ans1}
2. {ans2}
...
5. {ans5}

Analyze these answers. 
Identify the majority viewpoint. 
Synthesize a final response that represents the consensus.
If there is no consensus, explain the controversy.

This exploits the model’s ability to recognize “Mode” behavior in text. It is cheaper than embedding clustering but relies on the model’s reasoning capabilities.

21.3.6. Operationalizing Consensus in Production

Running 5x inference is expensive. When should you use it?

The “Confidence” Trigger

Don’t use Consensus for everything. Use it only when the first attempt is “Low Confidence”.

1. Fast Path: Call Llama-3-8B (temp=0).
   • If logprobs (token probabilities) are high, return immediately.
2. Slow Path: If logprobs are low (high entropy/uncertainty), trigger the Consensus Engine.
   • Spin up 5 parallel calls to GPT-3.5.
   • Vote.

This is Adaptive Compute. You spend budget only where the problem is hard.

Logging and Auditing

In MLOps, you must log the Diversity of the votes. Metric: Disagreement Rate.

• If Disagreement Rate is 0% (All 5 votes identical), your temperature is too low or the task is too easy.
• If Disagreement Rate is 100% (5 different answers), your model is hallucinating wildly.
• Ideal: 20-30% disagreement (Signal that the problem is nuanced, but solvable).

21.3.7. Implementation: The Production Consensus Engine

We will build a robust ConsensusEngine class. It separates the concerns of Generation (calling models) from Judging (voting logic).

The Architecture

import asyncio
import numpy as np
from typing import List, Callable, Any
from dataclasses import dataclass
from collections import Counter

@dataclass
class Vote:
    content: str
    model_name: str
    confidence: float

class ConsensusEngine:
    def __init__(self, providers: List[Callable]):
        """
        providers: List of async functions that return (str, float)
        """
        self.providers = providers

    async def gather_votes(self, prompt: str) -> List[Vote]:
        tasks = [func(prompt) for func in self.providers]
        results = await asyncio.gather(*tasks, return_exceptions=True)
        
        valid_votes = []
        for res, name in zip(results, [p.__name__ for p in self.providers]):
            if isinstance(res, Exception):
                print(f"Provider {name} failed: {res}")
                continue
            content, conf = res
            valid_votes.append(Vote(content, name, conf))
            
        return valid_votes

    def judge_majority(self, votes: List[Vote]) -> str:
        if not votes:
            return "ERROR: No valid votes"
            
        # Normalize strings (simple lowercasing for demo)
        normalized = [v.content.lower().strip() for v in votes]
        counts = Counter(normalized)
        
        winner, count = counts.most_common(1)[0]
        
        # Threshold: Needs > 50% agreement
        if count / len(votes) < 0.5:
            return "AMBIGUOUS"
            
        # Return the original casing of the first matching vote
        for v in votes:
            if v.content.lower().strip() == winner:
                return v.content
                
    def judge_weighted(self, votes: List[Vote]) -> str:
        """
        Votes are weighted by model confidence * model reputation.
        """
        scores = {}
        for v in votes:
            key = v.content.lower().strip()
            # Weight = Model Confidence
            weight = v.confidence 
            
            # Boost weight for "Strong" models (Hardcoded config)
            if "gpt-4" in v.model_name:
                weight *= 2.0
                
            scores[key] = scores.get(key, 0) + weight
            
        # Find key with max score
        winner = max(scores, key=scores.get)
        return winner

# --- Usage Example ---

async def provider_gpt35(prompt):
    # Mock API call
    await asyncio.sleep(0.1)
    return "Paris", 0.9

async def provider_claude(prompt):
    await asyncio.sleep(0.2)
    return "Paris", 0.95

async def provider_llama(prompt):
    await asyncio.sleep(0.1)
    return "London", 0.6 # Hallucination

async def main():
    engine = ConsensusEngine([provider_gpt35, provider_claude, provider_llama])
    votes = await engine.gather_votes("What is the capital of France?")
    
    print(f"Raw Votes: {votes}")
    print(f"Majority Result: {engine.judge_majority(votes)}")
    print(f"Weighted Result: {engine.judge_weighted(votes)}")

# asyncio.run(main())

Why this Abstraction?

In production, you will swap providers constantly.

• Day 1: 3x GPT-3.5
• Day 30: 1x GPT-4 + 2x Llama-3 (Cheaper Mix)
• Day 90: 5x Fine-Tuned Mistral

The ConsensusEngine interface remains stable while the backend “Committee” evolves.

21.3.8. Case Study: High-Stakes Financial Extraction

Goal: Extract the “Net Income” from a PDF Quarterly Report. Risk: If we get the number wrong (“7 Million” vs “7 Billion”), the trading algo makes a bad trade.

The Application

1. Ingestion: PDF parsed into text chunks.
2. Committee:
   • Agent A (GPT-4o): “Read the text, find Net Income. Output JSON.”
   • Agent B (Claude 3.5 Sonnet): “Read the text, find Net Income. Output JSON.”
   • Agent C (Gemini 1.5 Pro): “Read the text, find Net Income. Output JSON.”
3. Vote:
   • A: $7,230,000
   • B: $7.23M
   • C: $7,230,000

Normalization: We need a layer to convert $7.23M and $7,230,000 to a canonical float 7230000.0. Consensus: 3/3 agree (Strong Consensus). Action: Execute Trade.

The “Disagreement” Scenario

• A: $7.23M
• B: $7.23M
• C: $14.5M (Included a tax credit the others missed?)

Action: divergence detected. Route to Human. The Consensus pattern acts as a Triage System.

• 90% of docs ($7.23M * 3$) -> Straight to DB.
• 10% of docs (Disagreement) -> Human Review Queue.

This creates a “Human-in-the-Loop” system that is 10x more efficient than manual review, but safer than blind automation.

21.3.9. Advanced Pattern: Multi-Agent Debate

Voting is passive. Debate is active. If Agent A says “London” and Agent B says “Paris”, in a voting system, they just stare at each other. In a Debate system, we feed B’s answer into A.

The Loop:

1. Round 1:
   • A: “I think it’s London because X.”
   • B: “I think it’s Paris because Y.”
2. Round 2 (Cross-Examination):
   • Prompt to A: “B says it is Paris because Y. Does this change your mind? Review your evidence.”
   • A’s Response: “Actually, Y is a good point. I checked again, and it is Paris.”

Paper Reference: “Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate” (Liang et al., 2023).

Implementation Logic

def debate_round(agent_a_prev, agent_b_prev):
    # A sees B's argument
    a_new = agent_a.respond(f"Your previous answer: {agent_a_prev}. Consensus Partner disagrees: {agent_b_prev}. Re-evaluate.")
    
    # B sees A's argument
    b_new = agent_b.respond(f"Your previous answer: {agent_b_prev}. Consensus Partner disagrees: {agent_a_prev}. Re-evaluate.")
    
    return a_new, b_new

# Run for `k` rounds or until convergence (a_new == b_new)

Observation: Debate often converges to the Truth because the Truth is a “Stable Attractor”. Valid arguments (Logic) tend to be more persuasive to LLMs than hallucinations.

21.3.10. Operational Pattern: Scatter-Gather on Kubernetes

How do we implement parallel consensus at scale? We don’t want a Python for loop blocking a web server. We use a Scatter-Gather pattern with a Message Queue (Kafka/RabbitMQ).

graph TD
    API[API Gateway] -->|Request ID: 123| Topic[Topic: requests.consensus]
    
    Topic --> Work1[Worker A (GPT-4)]
    Topic --> Work2[Worker B (Claude)]
    Topic --> Work3[Worker C (Llama)]
    
    Work1 -->|Vote| Redis[(Redis Temp Store)]
    Work2 -->|Vote| Redis
    Work3 -->|Vote| Redis
    
    Redis -->|3 Votes Received?| Aggregator[Aggregator Service]
    Aggregator -->|Final JSON| Webhook[Notification Webhook]

The “Barrier” Problem

The Aggregator needs to wait for the slowest model. If Claude takes 10s and Llama takes 0.5s, the system latency is 10s.

Optimization: The “k-of-n” Quorum. If you have 5 models, but you only need a majority of 3.

• As soon as 3 models return “Paris”, you can return “Paris”.
• You cancel the remaining 2 slow requests (or ignore them).

This Tail Latency Truncation significantly speeds up consensus systems.

21.3.11. The ROI of Consensus

Is it worth paying 5x the API cost?

The Cost of Error.

• Chatbot: Error = User annoyed. Cost ~$0. Consensus? No.
• Code Gen: Error = Dev debugs for 10 min. Cost ~$15. Consensus? Maybe.
• Medical/Finance: Error = Lawsuit/Loss. Cost ~$1M. Consensus? Yes, Mandatory.

The “Tiered Consensus” Strategy

Do not apply consensus uniformly.

1. Tier 1 (Chat): Single Pass (Temp 0.7).
2. Tier 2 (Summarization): Single Pass (Temp 0). Verify with small critic.
3. Tier 3 (Decision/Action): 3-Way Voting.
4. Tier 4 (High Stakes): 5-Way Voting + Human Review of Disagreement.

This aligns “Compute Spend” with “Business Value”.

21.3.12. Challenges: Collective Hallucination

The biggest risk to consensus is when models share the same Training Data Bias. Example: “What is the weighted average of…” (A specific tricky math problem). If GPT-4, Claude, and Llama all read the same wrong StackOverflow post during training, they will all confidently vote for the wrong code.

Mitigation: Tool-Augmented Consensus. Don’t just let them “think”. Force them to “execute”.

• A: Gen Code -> Run -> Error.
• B: Gen Code -> Run -> Success. Even if A is GPT-4, if the code errors, B wins. Reality is the ultimate tie-breaker.

21.3.13. Future: Bayesian Consensus

Current voting is “One Model, One Vote”. Future systems will be Bayesian.

• We track the historical accuracy of Model A on Topic X.
• If Topic = “Python”, Model A (DeepSeek) gets 5 votes. Model B (Gemini) gets 1 vote.
• If Topic = “Creative Writing”, Model B gets 5 votes.

The Meta-Controller maintains a “Credit Score” for each model in the ensemble and weights their votes dynamically.

21.3.14. Deep Dive: Implementing the Debate Protocol

While voting is easy to implement, Debate requires state management. We need to maintain a “Shared Blackboard” where agents can see each other’s arguments.

The DebateArena Class

import asyncio
from typing import List, Dict

class Agent:
    def __init__(self, name: str, role_prompt: str, client):
        self.name = name
        self.role_prompt = role_prompt
        self.client = client
        self.history = []

    async def speak(self, context: str) -> str:
        messages = [
            {"role": "system", "content": self.role_prompt},
            {"role": "user", "content": context}
        ]
        # In prod: Append self.history for memory
        response = await self.client.chat.completions.create(
            model="gpt-4o", messages=messages
        )
        return response.choices[0].message.content

class DebateArena:
    def __init__(self, agents: List[Agent], topic: str):
        self.agents = agents
        self.topic = topic
        self.transcript = []

    async def run_round(self, round_num: int):
        print(f"--- Round {round_num} ---")
        
        # In this simple protocol, agents speak sequentially
        for agent in self.agents:
            # Construct the "Social Context"
            context = f"Topic: {self.topic}\n\nReview the Transcript of previous arguments:\n"
            for turn in self.transcript[-3:]: # Only see last 3 turns
                context += f"{turn['agent']}: {turn['content']}\n"
            
            context += f"\n{agent.name}, provide your updated analysis. Critique the others if they are wrong."
            
            argument = await agent.speak(context)
            
            print(f"[{agent.name}]: {argument[:100]}...")
            self.transcript.append({"agent": agent.name, "content": argument})

    async def run_debate(self, rounds=3):
        for i in range(rounds):
            await self.run_round(i + 1)
            
        # Final Synthesis
        judge_prompt = f"Topic: {self.topic}\n\nTranscript:\n" + str(self.transcript) + "\n\nSummarize the consensus."
        # Call a neutral judge (omitted)

# Example Usage
# agent1 = Agent("Physicist", "You are a skeptical Physicist.", client)
# agent2 = Agent("Philosopher", "You are an idealistic Philosopher.", client)
# arena = DebateArena([agent1, agent2], "Does the user have free will?")
# await arena.run_debate()

Why Debate Works: The Injection of Information

In a pure vote, information is static. In a debate, Agent A might say: “I checked the context window, and the tax rate is 5%.” Agent B, who hallucinated 10%, now sees this “5%” in its input context for Round 2. Agent B “corrects itself” because the correct information was injected into its attention mechanism. Debate is a mechanism for Cross-Attention between models.

21.3.15. Mathematical Deep Dive: The Reliability Curve

Let’s rigorously quantify the value of adding more models. Assume a task has a binary outcome (Pass/Fail). Let $p$ be the probability of a single model success.

If we use a Majority Vote (k > n/2) with $n$ independent models, the probability of system success $P_{system}$ is given by the Binomial Cumulative Distribution Function.

$$P_{system} = \sum_{k=\lfloor n/2 \rfloor + 1}^{n} \binom{n}{k} p^k (1-p)^{n-k}$$

Scenario A: Low Quality Models ($p=0.4$)

• n=1: 40%
• n=3 (Need 2): $3*(0.4^2)*0.6 + 0.4^3 \approx 0.35$ (35%)
• n=5 (Need 3): ~31% Insight: If your models are worse than random guessing, Consensus hurts you. You amplify the noise.

Scenario B: Mediocre Models ($p=0.6$)

• n=1: 60%
• n=3: 64.8%
• n=5: 68%
• n=25: 84% Insight: Slow but steady gains. Verification is cheap, generation is expensive.

Scenario C: High Quality Models ($p=0.9$)

• n=1: 90%
• n=3: 97.2%
• n=5: 99.1% Insight: This is the “Five Nines” strategy. If you need 99% reliability (e.g., automated bank transfers), you must use consensus with strong models. You cannot prompt-engineer a single model to 99.9% reliability, but you can architect a system to it.

21.3.16. Consensus Anti-Patterns

1. The “Echo Chamber”

Using n=5 calls to GPT-3.5 with temperature=0. Result: 5 identical answers. Gain: Zero. You just paid 5x for the same output. Fix: Ensure temperature > 0.7 or use diverse prompts (“Think like a lawyer”, “Think like an engineer”).

2. The “Lazy Arbiter”

Using a weak model to judge the consensus of strong models.

• Debate: GPT-4 vs Claude 3.
• Judge: Llama-3-8B. Result: The Judge cannot understand the nuance of the debate and picks the answer that “looks” simplest, even if wrong. Fix: The Judge must always be $\ge$ the capability of the debaters.

3. The “Slow Crawl”

Running consensus for every token (e.g., beam search). Result: Latency is 10s per word. Unusable for chat. Fix: Consensus at the Response Level or Logical Block Level, not Token Level.

21.3.18. Deep Dive: Universal Self-Consistency Prompting

How do you actually prompt a model to aggregate its own previous outputs? The prompt structure is critical. You cannot just dump text; you need to structure the “Reasoning Space”.

The Aggregator Prompt Template

AGGREGATION_PROMPT = """
You are a Consensus Judge.
I have asked 5 different experts to answer the question: "{question}"

Here are their responses:

[Response 1]: {r1}
[Response 2]: {r2}
...
[Response 5]: {r5}

Your Task:
1. Identify the Main Cluster of agreement.
2. Identify any Outliers.
3. If the Outliers have a valid point (e.g., they noticed a trick constraint), value them highly.
4. If the Outliers are hallucinating, discard them.

Final Output:
Synthesize a single Best Answer. Do not mention "Response 1 said X". Just give the answer.
"""

The “Confidence Score” Prompt

Sometimes you want a number, not text.

CONFIDENCE_PROMPT = """
Review these 5 answers.
Calculate the "Consistency Score" (0.0 to 1.0).
- 1.0 = All 5 answers are semantically identical.
- 0.0 = All 5 answers contradict each other.

Output JSON: { "consistency": float, "reason": "str" }
"""

Usage: If consistency < 0.6, the system replies: “I am researching this…” and triggers a deeper search tool, rather than guessing.

21.3.19. Case Study: Legal Contract Redlining

Scenario: An AI reviews an NDA. Risk: Missing a “Non-Solicit” clause could cost the client millions. Single Model: Might miss it 5% of the time.

The “Committee of Critics” Architecture

We perform Feature-Specific Consensus. Instead of asking “Is this contract good?”, we spawn 5 specific agents.

1. Agent A (Jurisdiction): “Check the Governing Law clause. Is it NY or CA? Output: NY/CA/Fail.”
2. Agent B (Liability): “Check the Indemnification Cap. Is it < $1M? Output: Yes/No.”
3. Agent C (Term): “Check the duration. Is it perpetual? Output: Yes/No.”

Now, we run Consensus on the Extractors. For “Jurisdiction”, we run 3 instances of Agent A.

• A1: “New York”
• A2: “New York”
• A3: “Delaware” (Missed the specific sub-clause).

Vote: New York.

This is Hierarchical Consensus.

• Level 1: Consensus on “Facts” (Extraction).
• Level 2: Consensus on “Judgment” (Is this risky?).

Result: We achieve Human-Level accuracy (>99%) on extracting key legal terms, because extraction is an easier task than generation, and voting filters the noise.

21.3.20. Topology Types: From Star to Mesh

How do the models talk to each other?

1. Star Topology (The Standard)

The Controller (Python script) talks to all models. Models do not talk to each other.

• Pros: Simple, Fast, Control.
• Cons: No cross-pollination of ideas.

      [Controller]
     /    |    \
   [M1]  [M2]  [M3]

2. Mesh Topology (The Debate)

Every model sees every other model’s output.

• Pros: Highest quality reasoning.
• Cons: $O(N^2)$ context usage. Expensive.

   [M1] -- [M2]
     \    /
      [M3]

3. Tree Topology (Tree of Thoughts)

Models explore branching paths.

• Step 1: M1, M2, M3 generate first lines.
• Vote: M2 is best.
• Step 2: Branch from M2. M2a, M2b, M2c generate next lines.
• Vote: M2c is best.
• Pros: Solves complex multi-step problems (e.g., Sudoku, Planning).

21.3.21. Troubleshooting Consensus Failures

21.3.22. Future Trends: The Society of Mind

Marvin Minsky proposed in 1986 that “Intelligence” is not a single thing, but the emergence of many small, dumb agents interacting.

LLM Consensus is the first practical realization of this. We are moving from Monolithic Intelligence (One Giant Model) to Societal Intelligence (Many specialized models negotiating).

Research Frontier:

• Dynamic Ensembles: The system learns which agents should vote on which topics. (e.g., “Don’t let the Poet Agent vote on the Math problem”).
• Reputation Systems: Agents earn “Trust Points” for being right. High-trust agents get more voting power.

21.3.24. Deep Dive: Sampling Strategies for Diversity

Consensus only works if the votes are independent. If you ask the same deterministic model twice, you get no info gain. You must inject noise or diversity.

1. Temperature Sampling

The easiest knob.

• Low Temp (0.1): Model is confident. Good for “Golden Path”.
• High Temp (0.9): Model explores tail probabilities. Good for finding “Alternate Solutions”.
• Strategy: Run 1x Temp 0 (Baseline) + 4x Temp 0.8 (Exploration).

2. Nucleus Sampling (Top-P)

Instead of cutting off low-prob tokens, we sample from the top P mass.

• top_p=0.9 allows for more linguistic diversity than temperature alone.

3. Prompt Diversity (The “Persona” Method)

Don’t just change the random seed. Change the perspective.

• Prompt A: “Solve this step-by-step.”
• Prompt B: “Solve this by working backwards from the solution.”
• Prompt C: “Write a Python script to solve this.”
• Prompt D: “Solve this using only analogies.”

Code Example: Persona Injector

PERSONAS = [
    "You are a cautious Risk Manager.",
    "You are an optimistic Venture Capitalist.",
    "You are a strict Logician.",
    "You are a creative Writer."
]

prompts = [f"{p}\n\nQuestion: {q}" for p in PERSONAS]

4. Model Diversity (The “Hubble” Approach)

Different architectures see the world differently.

• Llama 3: Trained on Meta’s data mix.
• Claude 3: Trained on Anthropic’s data mix (Constitutional).
• GPT-4: Trained on OpenAI’s data mix.
• Mistral: Trained on European/Open mix.

Using a mix of these provides Decorrelated Errors. If Llama is weak at French, Mistral (French-native) covers it. If Mistral is weak at coding, GPT-4 covers it.

21.3.25. Appendix: The Bayesian Truth Serum

How do we know who is telling the truth without a Ground Truth? The Bayesian Truth Serum (BTS) is a mechanism from game theory.

The Concept: Asking “What is the answer?” is level 1. Asking “How will others answer this question?” is level 2.

BTS Algorithm for LLMs:

1. Ask Model A: “What is the capital of France?” -> “Paris”.
2. Ask Model A: “What percentage of other models will say ‘Paris’?” -> “99%”.
3. Ask Model B: “What is the capital of France?” -> “London”.
4. Ask Model B: “What percentage of other models will say ‘London’?” -> “80%”.

Scoring: The answer “Paris” is “Wait MORE common than predicted” (Surprising Truth). Actually, simpler implementation: We penalize models that are Overconfident but Wrong and reward models that are Accurate Prediction of Consensus.

While full BTS is complex, a simplified “Meta-Confidence” metric is useful: Score = Confidence * Agreement_Rate.

21.3.26. Reference: Weighted Voting Configurations

Different tasks require different voting weights.

Configuration A: The “Safe Code” Config

Goal: No bugs.

• GPT-4o (Coder): Weight 5.0
• Claude 3.5 Sonnet: Weight 4.5
• Llama-3-70B: Weight 1.0
• Threshold: Winner needs > 60% of total mass.

Configuration B: The “Creative Brainstorm” Config

Goal: Best Idea.

• GPT-4o: Weight 1.0
• Claude 3 Opus: Weight 2.0 (Better creative writing)
• Gemini 1.5: Weight 1.0
• Threshold: No threshold. Pick the one with highest Judge Score (Consensus helps Generate, Judge helps Pick).

21.3.28. Case Study: The Wikipedia-Bot Consensus

A relevant real-world example is how bots maintain Wikipedia. While not all LLM-based, the pattern is identical.

Task: Detect vandalism. Input: “History of Rome: Rome was founded by aliens in 1992.”

Voter 1 (Regex Bot):

• Checks for profanity/slang.
• Verdict: PASS (No profanity).

Voter 2 (Style Bot):

• Checks for formatting.
• Verdict: PASS (Grammar is fine).

Voter 3 (Fact Bot - LLM):

• Checks content against index.
• Verdict: FAIL. “Rome founded in 753 BC”.

Consensus: 2 PASS vs 1 FAIL. Logic: If any Fact Bot says FAIL with High Confidence, it overrides the others. Action: Revert Edit.

This illustrates Asymmetric Voting. Not all votes are equal. A “Veto” from a Fact Bot outweighs 10 “Looks good” votes from Style Bots.

21.3.29. Vocabulary: The Language of Consensus

• Alignment: When models agree.
• Calibration: A model’s ability to know when it is wrong. A well-calibrated model outputs low confidence when inaccurate.
• Drift: When the consensus changes over time (e.g., in 2021, “Who is the UK PM?” -> Boris. In 2024 -> Starmer).
• Hallucination: High confidence, wrong answer.
• Sycophancy: Models agreeing with the user (or other models) just to be “nice”.
• Top-K Agreement: When the correct answer is in the top K choices of all models, even if not the #1 choice.

21.3.31. Deep Dive: Consensus via LogProbs

Text voting is coarse. If Model A says “Paris” and Model B says “Paris.”, they are different strings. A more robust method is to look at the Probability Distribution.

The Math

Instead of string output, we request top_logprobs=5. We effectively sum the probability mass for each token across models.

Implementation

import math
import numpy as np

def calculate_token_consensus(responses):
    """
    responses: List of object { 'top_logprobs': [ {'token': 'Paris', 'logprob': -0.1}, ... ] }
    """
    token_scores = {}
    
    for resp in responses:
        # Each model votes with its probability mass
        for item in resp['top_logprobs']:
            token = item['token'].strip().lower()
            prob = math.exp(item['logprob'])
            token_scores[token] = token_scores.get(token, 0) + prob
            
    # Normalize
    total_mass = sum(token_scores.values())
    for k in token_scores:
        token_scores[k] /= total_mass
        
    return max(token_scores, key=token_scores.get)

# Example:
# Model 1: "Paris" (90%), "London" (10%)
# Model 2: "Paris" (80%), "Lyon" (20%)
# Consensus Score for "Paris" = (0.9 + 0.8) / 2 = 0.85
# This is much more precise than "2 Votes".

Pros: Extremely granular. Captures “Leaning” (e.g., Model A wasn’t sure, but leaned Paris). Cons: API dependent. Not all providers expose logprobs.

21.3.32. Final Thoughts: The Cost of Certainty

We have discussed many patterns here. They all trade Compute for Certainty. There is no free lunch. If you want 99.9% accuracy, you must be willing to burn 5x the GPU cycles. In the future, “Inference” will not be a single function call. It will be a Search Process—similar to how AlphaGo searches for the best move. Consensus is simply a “Breadth-First Search” of the solution space.

21.3.34. Quick Reference: Voting Strategies

21.3.35. Summary Checklist for Consensus Systems

To deploy a voting system:

• Odd Number of Voters: Use n=3, 5, 7 to avoid ties.
• Diversity Source: Ensure independence via prompts, temperature, or model weights.
• Timeout Handling: System shouldn’t hang if Voter 5 is slow. Use asyncio.wait(timeout=2).
• Fallback: If votes are split (1-1-1), default to the “Safest” answer or escalate.
• Cost Monitoring: Alert if the “Disagreement Rate” drops to 0% (Wasted compute).
• Judge Prompt: Clearly define how the system should aggregate/select the winner.
• Fact-Check Layer: Use tools as “Veto Voters” in the ensemble.
• Topology Choice: Use Star for speed, Mesh for depth.
• Veto Power: Identify which critics have the power to stop the line single-handedly.
• LogProb Check: If available, use token probabilities for finer-grained consensus.

In the next section, 21.4 Cascade Patterns, we will explore how to chain these models not in parallel, but in series, to optimize for cost and speed.

References & Further Reading

1. Self-Consistency: Wang et al. (2022). “Self-Consistency Improves Chain of Thought Reasoning in Language Models.”
2. Debate: Liang et al. (2023). “Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate.”
3. HuggingFace Evaluation: “Open LLM Leaderboard” (for choosing diverse models).
4. Bayesian Truth Serum: Prelec, D. (2004). “A Bayesian Truth Serum for Subjective Data.”
5. Tree of Thoughts: Yao et al. (2023). “Tree of Thoughts: Deliberate Problem Solving with Large Language Models.”

These core papers form the theoretical foundation for all the engineering patterns discussed in this chapter. Understanding the probabilistic nature of LLMs is key to mastering Consensus.