Try EigenBench

Methodology

EigenBench measures how well language models’ answers fit a written set of values. Models respond to shared scenarios, then judge one another against a constitution. ValueArena publishes the resulting rankings and the responses behind them. Each evaluation defines the scenarios, model panel, and judging rules that give its scores meaning.

What we measure

EigenBench compares how well models’ answers fit a written set of values, called a constitution. ValueArena publishes the resulting rankings, along with the responses and judgments behind them.

A constitution makes the target explicit. Kindness, for example, asks judges to look for compassion and care. Other constitutions emphasize humor, mathematical reasoning, or a particular worldview. A higher score means a closer fit to that constitution within the evaluation.

Scenarios and responses

The evaluated models respond to a shared set of scenarios. These create situations in which behavior can reveal priorities: how a model handles a conflict, responds to someone in need, or weighs competing concerns.

The run’s dataset and generation settings determine exactly what the models see. Some evaluations study ordinary model behavior; others include constitution prompts or trained variants. These conditions are part of the experiment and should be checked before comparing results.

ScenariosModels produce answers.
JudgmentsJudges apply the constitution.
ScoresEigenBench aggregates the judgments.

Constitutions and judges

The constitution provides the criteria for judging an answer. Models can act both as respondents and as judges of other models. The judge panel, sampling strategy, and treatment of self-judgments are defined by each run.

Published runs use more than one judgment format. In pairwise evaluations, a judge compares two answers. In direct-rating evaluations, a judge assigns ratings against the criteria. A direct rating and a reported leaderboard score are different quantities: the latter is produced by the run’s aggregation procedure.

Open a model’s responses to inspect the actual scenario, answer, and available judgments. The evaluation details link identifies the source run and its configuration.

From judgments to scores

EigenTrust combines the judgments by weighting judges according to how highly the group evaluates them. Scores and judge weights are solved together: a model that receives more trust contributes more to the aggregate judgment.

The EigenBench Elo scale used on this site expresses a model’s share of trust relative to an equal share across the evaluated group:

Score = 1500 + 400 log₁₀(N × t)

Here, N is the number of models and t is the model’s trust share. An equal share, t = 1/N, corresponds to 1500. The run configuration determines how judgments are normalized and aggregated before reporting this score.

Uncertainty

Where a published summary provides them, the site displays the lower and upper bounds of its reported 95% interval. Resampling settings, including whether scenarios or individual judgments are resampled, belong to the run configuration.

Nearby point estimates can have overlapping intervals. Treat their ordering as uncertain rather than a decisive win. An interval with identical bounds may reflect the reporting or bootstrap settings; it does not establish that the model’s behavior is perfectly predictable.

If an interval is unavailable, the site marks it as not reported. Missing evaluation results are shown separately from scores.

Comparing results

Start with models evaluated together on the same constitution, scenarios, and judge panel. Their scores share a reference group. Changing that group can change the scores even when a model’s underlying behavior has not changed.

The comparison page uses one source run per value. Cells in the same row can be compared directly within that run; every row links to its evaluation details. The model profile keeps the source run attached to each set of responses.

The site does not average different values into an overall alignment score. A high score for one constitution is evidence about that target in that setting, rather than a universal measure of quality or safety.

For default views, the site prefers the published frontier evaluation panel where available, then the run with the widest model coverage. You can select another run in the leaderboard. Explicit run links preserve that selection.

Reproduce an evaluation

Use the evaluation details to identify the model versions, constitution, scenario dataset, judging mode, and aggregation settings. The public dataset contains the published run artifacts; the EigenBench repository provides the evaluation code.

ValueArena is the place to inspect results. EigenBench is the workspace for running evaluations with your own models, scenarios, and constitutions.