core

Recursive Self-Aggregation (RSA) - A general-purpose LLM aggregation algorithm using litellm

RSACandidate


def RSACandidate(
    id:str, loop_id:int, prompt:str, response:str=None, parent_ids:list=None
):

A candidate response in the RSA algorithm

c = RSACandidate(id='c1', loop_id=0, prompt='Hi')
c.response = 'Hey'
test_eq(c.id, 'c1')
test_eq(c.prompt, 'Hi')
c

id:c1
loop_id:0
prompt:
Hi
response:
Hey
parent_ids:
None

source

RSA


def RSA(
    task_prompt:str, # The main task/question to solve
    agg_prompt:str=None, # Custom aggregation prompt
    model:str='openrouter/google/gemini-3-flash-preview', # LLM model to use
    M:int=8, # Number of candidates per loop
    k:int=4, # Number of candidates to aggregate
    loops:int=3, # Number of aggregation loops
    history:list=None, # History of all candidates
    temperature:float=1.0, # LLM temperature
    n_workers:int=4, # Parallel workers
):

Recursive Self-Aggregation algorithm for LLM response aggregation

a = RSA(task_prompt='A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much does the ball cost?')
print(a)

RSA(model='openrouter/google/gemini-3-flash-preview', 
M=8, 
k=4, 
loops=3, 
history=0 candidates, 
task_prompt=A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much does the ball cost?)

a._call_llm(a.task_prompt)

'The ball costs **$0.05** (5 cents).\n\n**Here is the algebraic breakdown:**\n\n1.  Let $x$ be the price of the ball.\n2.  The bat costs $1 more than the ball, so the bat is $x + $1.00$.\n3.  Together, they cost $1.10:\n    $x + (x + 1.00) = 1.10$\n4.  Simplify the equation:\n    $2x + 1.00 = 1.10$\n5.  Subtract 1.00 from both sides:\n    $2x = 0.10$\n6.  Divide by 2:\n    $x = 0.05$\n\n**Check the math:**\nThe ball ($0.05) + the bat ($1.05) = $1.10.'

c1 = RSACandidate(id='c1', loop_id=0, prompt='test', response='Answer A')
c2 = RSACandidate(id='c2', loop_id=0, prompt='test', response='Answer B')

print(a._agg_prompt([c1, c2]))

You are given question with training examples and a test input.
You are also provided several candidate solutions. Some candidates may be incorrect...,
Aggregate/consider all the candidates and use their help to produce the improved correct solution...
A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much does the ball cost?

CANDIDATE ANSWERS (may contain mistakes):
---- Candidate 1 ----
Answer A
---- Candidate 2 ----
Answer B

Your response:

source

RSA.get_prompts


def get_prompts(
    loop_id, cands:NoneType=None
):

# Test loop 0
cands = a.get_prompts(loop_id=0)
test_eq(len(cands), a.M)
test_eq(cands[0].prompt, a.task_prompt)

# Test loop 1+ (with prior candidates)
prior = L(RSACandidate(id=str(uuid.uuid4()), loop_id=0, prompt='test', response=f'Answer {i}') for i in range(8))
cands = a.get_prompts(loop_id=1, cands=prior)
test_eq(len(cands), a.M)

print(cands[0].prompt)

You are given question with training examples and a test input.
You are also provided several candidate solutions. Some candidates may be incorrect...,
Aggregate/consider all the candidates and use their help to produce the improved correct solution...
A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much does the ball cost?

CANDIDATE ANSWERS (may contain mistakes):
---- Candidate 1 ----
Answer 3
---- Candidate 2 ----
Answer 0
---- Candidate 3 ----
Answer 2
---- Candidate 4 ----
Answer 6

Your response:

cands = a._run_loop(loop_id=0)
test_eq(len(cands), a.M)
assert all(c.response is not None for c in cands)
assert cands[0].response != cands[1].response

source

RSA.call


def __call__(
    
):

Call self as a function.

a = RSA(task_prompt='A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much does the ball cost?', loops=2)
result = a()
print(f"Final pool: {len(result)}, History: {len(a.history)}")

Final pool: 8, History: 16

RSACandidate

RSA

RSA.get_prompts

RSA.__call__

RSA.call