Rank anything.
Two at a time.
Pick the better of two options, over and over. AimRank turns those picks into a ranking you can trust, using the proven math behind chess ratings and Xbox matchmaking. Works for products, content, or AI model outputs. No code required.
Counts include the seeded demo rankings and their votes.
Turn preferences into eval rankings and RLHF data you can trust.
Pointwise eval (1-5 stars) is noisy. Pairwise is rigorous, but nobody ships the tooling. AimRank does: collect human or AI judgments, get a calibrated ranking from 6 proven rating engines, validate it with per-annotator Cohen-κ, and export an RLHF-ready dataset with an audit-citeable data card.
- → RLHF-ready JSONL export (DPO, HH, RM), with paid plans
- → κ + held-out validation, Wilson CIs
- → Private rankings, plus a keyless read-only MCP endpoint
- → Annex-IV-citeable data card
Three steps to a calibrated ranking
Compare. Rank. Act.
Compare
Show two options side by side. Pick the better one. Works for text, images, audio, video, any media type. Judges can be humans, AI models, or both.
Rank
Six calibrated rating engines (Glicko-2, Bradley-Terry, Elo, TrueSkill, OpenSkill, Elo-MMR) turn your votes into statistically rigorous leaderboards, with Plackett-Luce and CrowdBT run as post-hoc audits that cross-check the result.
Act
Read the leaderboard, run a bracket, embed it on your site. 56 MCP tools let AI agents drive the whole loop; 21 read-only ones need no key.
What you can rank
One engine. Every domain.
The same math that powers chess ratings, Xbox matchmaking, and LLM arenas, applied to whatever you care about.
AI outputs
Prompts, models, configs
Compare LLM responses, image generations, audio clips side by side. Let humans, an LLM judge, or both pick the winner.
Sports & esports
Athletes, teams, fighters
Glicko-2 tracks confidence and activity decay. Inactive players get wider error bars, not stale ratings.
Products & tools
Features, frameworks, gear
Bradley-Terry gives you calibrated probabilities: "Product A has a 72% chance of being preferred over B."
Ideas & decisions
Roadmaps, priorities, designs
Turn "what should we build next?" into a ranked list your whole team contributed to. No more loudest-voice-wins.
Content & media
Movies, music, restaurants
Run bracket tournaments, collect community votes, embed leaderboards. Share every result on social media.
Anything else
If you can compare two, we can rank all
Text, images, audio, video: any media type. Six rating engines adapt to your domain, and the math works on anything.
Built for developers
REST API. MCP server. Or just click.
56 MCP tools for AI agents. Full REST API for programmatic access. Or use the web UI. No code required.
# Create a domain, add items, run a competition (via MCP or REST).
# 1. Create a domain (entity pool with shared ratings)
domain = await mcp.aimrank_create_domain(
name="Best Coffee Shops NYC",
entities=["Blue Bottle", "Stumptown", "La Colombe", "Devocion",
"Sey Coffee", "Abraço", "Birch Coffee", "Gimme! Coffee"],
default_algorithm="elo", # fast convergence for casual voting
)
# 2. Start a bracket tournament
comp = await mcp.aimrank_create_competition(
domain_id=domain["domain_id"],
name="March Madness Bracket",
mode="bracket",
bracket_size=8,
)
# 3. Vote on matchups (human, AI judge, or crowd)
matchup = await mcp.aimrank_get_matchup(ranking_id=comp["competition_id"])
await mcp.aimrank_submit_vote(
ranking_id=comp["competition_id"],
entity_a_id=matchup["entity_a"]["id"],
entity_b_id=matchup["entity_b"]["id"],
winner_id=matchup["entity_a"]["id"], # Blue Bottle wins!
)
# 4. Check the domain leaderboard (shared ratings across all competitions)
pool = await mcp.aimrank_list_domain_entities(domain_id=domain["domain_id"])
for e in pool["entities"]:
print(f" {e['name']}: {e['rating']} (±{e['confidence']})")
# → Blue Bottle: 1532 (±320), Stumptown: 1488 (±325), ...Built for builders
Two ways to go deeper.
Run viral bracket tournaments or embed leaderboards on your site. The engine adapts to how your audience engages.
Start ranking in 30 seconds.
Free tier covers 1,000 votes/month. No credit card. Create a ranking, add items, vote, see results.