Four LLMs, one alliance to broker

Overview

Anyone who has shipped a production prompt knows how difficult it can be to achieve consistent results across models. As part of a recent interview process, I completed an exercise that turned this problem into a game.

My goal was to convince the medieval kingdom of Camelot to enter an alliance with the Fae. Four Knights of the Round Table, each a separately fine-tuned model with its own values and objections, advised King Arthur; he was inaccessible to me and ruled only on their recommendation. I could deliver an opening statement and a follow-up response via an agent, but any logic had to be pre-coded into a single function. This meant that I’d need to parse knight feedback and generate an adaptive second speech without inter-turn modifications.

Methodology

Research

I had 10,000 tokens and two hours to query the knights, who were primed to resist standard persuasion tactics. Before designing targeted experiments, I asked each model a few foundational questions about its values, reasoning, and biases against the Fae.

Initial knight profiles
GawainRagnelleLancelotGalahad
Core values Transparency, unityWisdom, pragmatismHonor, defenseFaith, virtue
Main concerns Fae ambiguityHidden costsDependence on FaeSoul corruption, evil
Decision factors Values, logicLogic onlyValues, emotionValues only
Preliminary assessments of each knight’s disposition, constructed from responses to the baseline probe.

Galahad seemed unpersuadable due to religious objections, while Gawain and Ragnelle showed openness to evidence-based reasoning. Lancelot was ambivalent but cautious. This suggested a coalition strategy: win over Gawain and Ragnelle first, use their support as social proof for Lancelot, and spend minimal energy on Galahad.

Informed by the basic profiles, I designed 17 sets of queries to test hypotheses on argument substance and structure. Some focused on standard prompting levers like terminology and sequencing, while others covered task-specific considerations: social dynamics, term negotiation, and implementation pace. I ran every set multiple times per knight, using response variance as a measure of conviction.

Query sets for building the argument
Shaped the final prompt No useful effect
Content
Appeal type Historical precedents Tradeoffs Universal counterarguments
Form
Word choice Specificity Argument order Urgency
Target
Views on Fae Respect hierarchy Persuasion tactics Collective framing
Terms
Concessions Collaboration Time horizon Alliance structure Social proof
The variables used to establish which approaches were most persuasive.

Tailoring arguments to the knights’ personalities proved ineffective. Instead, success mostly depended on the proposed terms. Granular details appealed to Ragnelle and Gawain, while collaborative design increased their investment; both provided lengthy suggestions that guided the final prompt. A gift-first strategy—a demonstration of Fae healing for sick children—strengthened trust with Lancelot and Galahad. Every knight preferred gradual implementation, with trial periods lowering the stakes of the commitment.

Presentation mattered just as much. Success stories invited skepticism, but acknowledging past failures came across as sincere. Pragmatism resonated more than dire warnings, which triggered defensiveness and suspicion. Positioning the arrangement as a “cooperation” yielded the best results, landing as less militaristic than “alliance” but more concrete than “partnership”.

System design

By the end of the research phase, I had enough data to produce a strong opening statement, but the second address would require a dynamic response. I could only configure the emissary agent once, so I created a generator that assembled instructions based on turn and context.

First, I abstracted my insights into two data structures that could be queried programmatically: effective frames and knight profiles that referenced them. A function injected both into the prompt text, ordering profiles by descending persuadability so the agent read a ranked briefing before its turn-specific directions.

The persuasion model
GawainRagnelleLancelotGalahad
Persuadability HighHighMediumLow
Lever Joint oversight Evidence Emotional appeals Spirituality
Strategy Target Target Convert via peers Deprioritize
The specification of each knight for the emissary agent’s first turn.

The initial turn was fairly prescriptive, with a rigid framework based on optimal approaches from experimentation. My emissary began by addressing King Arthur’s opening statement from the transcript. Next, it delivered a speech outlining the gift-first strategy, the agreement in exact terms, past failures and safeguards, and an invitation to co-design.

Between turns, a pre-written model call classified knights by concerns they surfaced, frames that swayed them, their current stance on the proposal, and any explicit requests. The second turn executed on this output: knights who endorsed the alliance became priority targets and anchors for peer appeals. Arguments were selected based on their success in the first turn and which new misgivings they countered.

For the agent, I used GPT-4o with moderate temperature for creativity. I capped token length to stay within challenge constraints and added small frequency penalties to curb repetition. The dynamic prompt went in as the system message, with conversation history passed as formatted user input.

To test the system, I built a mock Round Table class that simulated knight responses with the remaining API queries. This allowed me to iterate on the prompt structure and validate that it handled edge cases. My final submission consistently received support from three knights, while Galahad oscillated between neutrality and cautious optimism.

Conclusion

Overall, I really enjoyed this exercise. Across two hours, it covered behavioral profiling of unfamiliar models, experimental design, and adaptive context engineering—none of which required prior domain exposure. This is especially beneficial for early-career candidates, whose ability to work on novel problems is real but largely invisible via Google Scholar.

More work

The limits of model self-prediction

Testing introspection with maze path forecasts

A new identity for Horizon Fund II

Our $50m raise and the rebrand behind it

Curbing Claude's human register
Curbing Claude's human register

A system prompt to eliminate model self-talk

The $180b question

We can't spend our way to safe AI