Rank anything.
Two at a time.
Pick the better of two options, over and over. AimRank turns those picks into a ranking you can trust, using the proven math behind chess ratings and Xbox matchmaking. Works for products, content, or AI model outputs. No code required.
Turn preferences into eval rankings and RLHF data you can trust.
Pointwise eval (1-5 stars) is noisy. Pairwise is rigorous, but nobody ships the tooling. AimRank does: collect human or AI judgments, get a calibrated ranking from 6 proven algorithms, validate it with per-annotator Cohen-κ, and export an RLHF-ready dataset with an audit-citeable data card.
- → RLHF-ready export (CSV / JSONL)
- → κ + held-out validation, Wilson CIs
- → Private rankings, API + MCP access
- → Annex-IV-citeable data card
Three steps to a calibrated ranking
Compare. Rank. Act.
Compare
Show two options side by side. Pick the better one. Works for text, images, audio, video, any media type. Judges can be humans, AI models, or both.
Rank
Six calibrated rating engines (Glicko-2, Bradley-Terry, Elo, TrueSkill, OpenSkill, Elo-MMR) turn your votes into statistically rigorous leaderboards, with Plackett-Luce and CrowdBT run as post-hoc audits that cross-check the result.
Act
Read the leaderboard, run a bracket, export as JSON/YAML, feed into your pipeline. 56 MCP tools let AI agents drive the whole loop autonomously.
What you can rank
One engine. Every domain.
The same math that powers chess ratings, Xbox matchmaking, and LLM arenas, applied to whatever you care about.
AI outputs
Prompts, models, configs
Compare LLM responses, image generations, audio clips side by side. Bayesian optimization finds your best config automatically.
Sports & esports
Athletes, teams, fighters
Glicko-2 tracks confidence and activity decay. Inactive players get wider error bars, not stale ratings.
Products & tools
Features, frameworks, gear
Bradley-Terry gives you calibrated probabilities: "Product A has a 72% chance of being preferred over B."
Ideas & decisions
Roadmaps, priorities, designs
Turn "what should we build next?" into a ranked list your whole team contributed to. No more loudest-voice-wins.
Content & media
Movies, music, restaurants
Run bracket tournaments, collect community votes, embed live leaderboards. Share every result on social media.
Anything else
If you can compare two, we can rank all
Text, images, audio, video: any media type. Six rating algorithms adapt to your domain, and the math works on anything.
Built for developers
REST API. MCP server. Or just click.
56 MCP tools for AI agents. Full REST API for programmatic access. Or use the web UI. No code required.
# Create a domain, add items, run a competition (via MCP or REST).
# 1. Create a domain (entity pool with shared ratings)
domain = await mcp.aimrank_create_domain(
name="Best Coffee Shops NYC",
entities=["Blue Bottle", "Stumptown", "La Colombe", "Devocion",
"Sey Coffee", "Abraço", "Birch Coffee", "Gimme! Coffee"],
default_algorithm="elo", # fast convergence for casual voting
)
# 2. Start a bracket tournament
comp = await mcp.aimrank_create_competition(
domain_id=domain["domain_id"],
name="March Madness Bracket",
mode="bracket",
bracket_size=8,
)
# 3. Vote on matchups (human, AI judge, or crowd)
matchup = await mcp.aimrank_get_matchup(ranking_id=comp["competition_id"])
await mcp.aimrank_submit_vote(
ranking_id=comp["competition_id"],
entity_a_id=matchup["entity_a"]["id"],
entity_b_id=matchup["entity_b"]["id"],
winner_id=matchup["entity_a"]["id"], # Blue Bottle wins!
)
# 4. Check the domain leaderboard (shared ratings across all competitions)
pool = await mcp.aimrank_list_domain_entities(domain_id=domain["domain_id"])
for e in pool["entities"]:
print(f" {e['name']}: {e['rating']} (±{e['confidence']})")
# → Blue Bottle: 1532 (±320), Stumptown: 1488 (±325), ...Built for builders
Two ways to go deeper.
Run viral bracket tournaments or embed live leaderboards on your site. The engine adapts to how your audience engages.
Start ranking in 30 seconds.
Free tier covers 1,000 votes/month. No credit card. Create a ranking, add items, vote, see results.