task_prompt = '''Three people check into a hotel room that costs $30. They each contribute $10.
Later, the manager realizes the room only costs $25 and gives $5 to the bellboy to return.
The bellboy keeps $2 and gives $1 back to each person.
So each person paid $9 (total $27), plus the bellboy has $2, which equals $29.
Where did the extra dollar go?'''RSA - Recursive Self-Aggregation
A general-purpose LLM aggregation algorithm using litellm
RSA implements Recursive Self-Aggregation, a technique for improving LLM responses by generating multiple candidate answers and iteratively aggregating them. The algorithm samples k candidates from a pool of M responses, asks the LLM to synthesize an improved answer, and repeats this process across multiple loops to converge on higher-quality outputs.
Developer Guide
If you are new to using nbdev here are some useful pointers to get you started.
Install in Development mode
# make sure package is installed in development mode
$ pip install -e .
# make changes under nbs/ directory
# ...
# compile to have changes apply to
$ nbdev_prepareUsage
Installation
Install latest from the GitHub [repository][repo]:
$ pip install git+https://github.com//.gitor from [conda][conda]
$ conda install -c or from [pypi][pypi]
$ pip install [repo]: [docs]: https://.github.io// [pypi]: https://pypi.org/project// [conda]: https://anaconda.org//
Documentation
Documentation can be found hosted on this GitHub [repository][repo]’s [pages][docs]. Additionally you can find package manager specific guidelines on [conda][conda] and [pypi][pypi] respectively.
[repo]: [docs]: https://.github.io// [pypi]: https://pypi.org/project// [conda]: https://anaconda.org//
How to use
Basic Usage
Create an RSA instance with your task prompt and call it to run the aggregation:
agg_prompt = """Below is a reasoning problem followed by several candidate solutions.
Your job is to:
1. Carefully analyze each candidate's reasoning step-by-step
2. Identify which candidates make logical errors or arithmetic mistakes
3. Note which approaches lead to correct reasoning
4. Synthesize the best reasoning into a single, clear, correct solution
Show your work step-by-step, then state your final answer clearly."""from RSA.core import RSA
# Create RSA instance with a reasoning task
rsa = RSA(
task_prompt=task_prompt,
agg_prompt=agg_prompt,
M=4,
k=2,
loops=3
)
# Run the aggregation
results = rsa()
print(f"Generated {len(rsa.history)} total candidates across {rsa.loops} loops")
print('llm response: \n', results[-1].response)Generated 12 total candidates across 3 loops
llm response:
### Analysis of Candidate Reasoning
Both Candidate 1 and Candidate 2 correctly identify the logical fallacy presented in the "missing dollar" riddle. They both recognize that:
1. **The Math is Arbitrary:** Adding the $2 tip to the $27 total spent is a "category error."
2. **The Actual Components:** The $27 paid by the guests is not a separate figure from the $2 bellboy tip; rather, the $27 *already includes* the $2 tip.
3. **The Correct Sums:** To reach $30, one must add the money the guests **kept** ($3) to the money they **spent** ($27). Alternatively, to account for where the $30 went, one must add the hotel's revenue ($25), the tip ($2), and the refund ($3).
Both candidates provided excellent, logically sound explanations. Candidate 1’s breakdown is particularly helpful for its clear "initial state vs. final state" comparison.
---
### Step-by-Step Correct Reasoning
To solve the riddle, we must look at the cash flow from two distinct, valid accounting perspectives to see where the "missing dollar" logic fails.
#### 1. The Audit Perspective (Where did the $30 go?)
If we track the original $30 contributed by the guests, we can see exactly where every dollar ended up:
* **$25** is in the hotel's cash register (the actual price of the room).
* **$2** is in the bellboy's pocket (the tip he kept).
* **$3** is in the guests' pockets ($1 returned to each).
* **Total:** $25 + $2 + $3 = **$30**.
* *Conclusion: No money is missing.*
#### 2. The Net Expenditure Perspective (What did the guests spend?)
The guests paid $30 and got $3 back. Therefore, their net expenditure is $27. We can break down what that $27 paid for:
* **$25** went toward the room cost.
* **$2** went to the bellboy as a tip.
* **Total:** $25 + $2 = **$27**.
* *Conclusion: The $2 tip is a subset of the $27, not an addition to it.*
#### 3. Identifying the Fallacy
The riddle says: *"Each person paid $9 (total $27), plus the bellboy has $2, which equals $29."*
The error is in the word **"plus."** The $27 includes the $2 tip. Adding them together ($27 + $2) is mathematically incorrect because you are counting the bellboy's tip twice. It is like saying, "I bought a $27 shirt, and $2 of that price was tax, so I spent $27 + $2 = $29."
To reach the original $30 using the $27 figure, you must add the **$3 refund** that the guests kept, not the tip they already paid.
### Final Answer
The dollar is not missing. The total of $27 paid by the guests already includes the $2 tip kept by the bellboy ($25 for the room + $2 for the bellboy). The riddle creates a fallacy by adding the tip to the total cost instead of adding the $3 refund to the total cost ($27 spent + $3 refund = $30).from litellm import completion
# Single direct call (baseline)
response = completion(
model='openrouter/google/gemini-3-flash-preview',
messages=[{"role": "user", "content": task_prompt}],
temperature=1.0
)
baseline_answer = response.choices[0].message.content
print("=== BASELINE (single call) ===")
print(baseline_answer)=== BASELINE (single call) ===
This is a classic riddle that relies on a **false premis**—it tricks you into adding numbers that shouldn't be added.
The "missing dollar" is a result of a math error in the way the story is summarized. Here is the breakdown of where the money actually is:
### 1. Follow the Actual Money
At the end of the story, there is exactly **$30** accounted for:
* The Hotel Manager has **$25**.
* The Bellboy has **$2**.
* The three guests have **$3** ($1 each).
* **Total: $25 + $2 + $3 = $30.**
### 2. The Logical Fallacy
The riddle tricks you with this sentence:
> *"So each person paid $9 (total $27), plus the bellboy has $2, which equals $29."*
This is incorrect because **the $2 held by the bellboy is already included in the $27 paid by the guests.**
Think of it this way:
* The guests paid **$27**.
* Where did that $27 go? **$25** went to the register and **$2** went into the bellboy's pocket.
To find the original $30, you should **add** the $3 refund to the $27 paid. Instead, the riddle incorrectly **adds** the bellboy’s tip to the payment, when it should be **subtracted** to find the room's actual cost ($27 - $2 = $25).
**The extra dollar doesn't exist; the riddle simply adds two numbers that don't belong together.**Configuration Options
| Parameter | Default | Description |
|---|---|---|
task_prompt |
(required) | The main task/question to solve |
model |
'openrouter/google/gemini-3-flash-preview' |
LLM model to use (any litellm-compatible model) |
M |
8 | Number of candidates generated per loop |
k |
4 | Number of candidates sampled for each aggregation |
loops |
3 | Number of aggregation iterations |
temperature |
1.0 | LLM sampling temperature |
n_workers |
4 | Parallel workers for LLM calls |
agg_prompt |
(auto) | Custom aggregation prompt (optional) |
How RSA Works
- Loop 0: Generate M independent responses to the task prompt
- Loop 1+: For each of M new candidates, randomly sample k previous candidates and ask the LLM to aggregate them into an improved answer
- Repeat for the specified number of loops
- Return the final pool of aggregated candidates
The history attribute stores all candidates across all loops, allowing you to trace the aggregation process.