このMCPでできること
Provides nonogram puzzles and benchmark data for comparing language-model accuracy, cost, latency, token use, and solution outcomes.
機能
ツール
11
ツール
check_solution
Check solution
Check a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['id', 'grid'], 'properties': {'id': {'type': 'string', 'description': 'Puzzle id from list_puzzles'}, 'grid': {'type': 'string', 'description': 'Row-major string of 0 (empty) and 1 (filled), width Ã\x97 height characters'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['correct', 'rowViolations', 'columnViolations'], 'properties': {'error': {'type': 'string'}, 'correct': {'type': 'boolean'}, 'rowViolations': {'type': 'array', 'items': {}}, 'columnViolations': {'type': 'array', 'items': {}}}, 'additionalProperties': {}}
compare_models
Compare models
Side-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['models'], 'properties': {'models': {'type': 'array', 'items': {'type': 'string'}, 'maxItems': 20, 'minItems': 2, 'description': '2 to 20 model variant ids or family names, e.g. claude-opus-5.5 or gpt-6-astra-xhigh'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['models'], 'properties': {'models': {'type': 'array', 'items': {'type': 'object', 'required': ['model', 'displayName', 'family', 'effort', 'provider', 'reasoning', 'accuracy', 'correct', 'total', 'failedRuns', 'bySize'], 'properties': {'model': {'type': 'string', 'description': 'Model variant id, e.g. claude-opus-5.5-high'}, 'total': {'type': 'number'}, 'bySize': {'type': 'array', 'items': {'type': 'object', 'required': ['size'], 'properties': {'size': {'type': 'string'}}, 'additionalProperties': {}}}, 'effort': {'type': ['string', 'null']}, 'family': {'type': 'string'}, 'correct': {'type': 'number'}, 'accuracy': {'type': 'number', 'description': 'Percentage of puzzles solved'}, 'provider': {'type': ['string', 'null']}, 'reasoning': {'type': 'boolean'}, 'failedRuns': {'type': 'number'}, 'displayName': {'type': 'string'}}, 'additionalProperties': {}}}}, 'additionalProperties': False}
get_leaderboard
Get leaderboard
Models ranked by accuracy; defaults to all effort levels for compatibility.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'properties': {'size': {'enum': ['5x5', '10x10', '15x15', '20x20'], 'type': 'string', 'description': 'Grid size to filter on'}, 'effort': {'type': 'string', 'description': 'best, all (default), or one effort level; empty means all'}, 'family': {'type': 'string', 'description': 'Comma-separated family ids; empty means no filter'}, 'version': {'type': 'string', 'description': 'Comma-separated benchmark versions: 1.0, 1.1, 1.2; empty means all'}, 'provider': {'type': 'string', 'description': 'Comma-separated provider ids; empty means no filter'}, 'reasoning': {'type': 'boolean', 'description': 'Only reasoning (true) or non-reasoning (false) variants'}, 'min_correct': {'type': 'integer', 'maximum': 9007199254740991, 'minimum': 0, 'description': 'Minimum puzzles solved in the selected tier; default 0 includes unsolved variants'}, 'open_weights': {'type': 'boolean', 'description': 'Only open-weight (true) or closed (false) models'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['updatedAt', 'size', 'models'], 'properties': {'size': {'type': 'string'}, 'models': {'type': 'array', 'items': {'type': 'object', 'required': ['model', 'displayName', 'family', 'effort', 'provider', 'reasoning', 'accuracy', 'rank', 'correct', 'total', 'totalCostUsd'], 'properties': {'rank': {'type': 'number'}, 'model': {'type': 'string', 'description': 'Model variant id, e.g. claude-opus-5.5-high'}, 'total': {'type': 'number'}, 'effort': {'type': ['string', 'null']}, 'family': {'type': 'string'}, 'correct': {'type': 'number'}, 'accuracy': {'type': 'number', 'description': 'Percentage of puzzles solved'}, 'provider': {'type': ['string', 'null']}, 'reasoning': {'type': 'boolean'}, 'displayName': {'type': 'string'}, 'totalCostUsd': {'type': 'number'}}, 'additionalProperties': {}}}, 'updatedAt': {'type': 'string'}}, 'additionalProperties': False}
get_model_puzzles
Get model puzzles
Which puzzles one model solved, missed, timed out on, or has not run.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['model'], 'properties': {'model': {'type': 'string', 'description': 'Model variant id as listed on the leaderboard, e.g. claude-opus-5.5-high'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['model', 'displayName', 'solved', 'attempted', 'puzzles'], 'properties': {'model': {'type': 'string'}, 'solved': {'type': 'number'}, 'puzzles': {'type': 'array', 'items': {'type': 'object', 'required': ['id', 'index', 'size', 'state'], 'properties': {'id': {'type': 'string'}, 'size': {'type': 'string'}, 'index': {'type': 'number'}, 'state': {'enum': ['solved', 'wrong', 'cut-off', 'not-run'], 'type': 'string'}}, 'additionalProperties': {}}}, 'attempted': {'type': 'number'}, 'displayName': {'type': 'string'}}, 'additionalProperties': False}
get_model_results
Get model results
Accuracy, cost, latency and token use for one model, broken down by grid size.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['model'], 'properties': {'model': {'type': 'string', 'description': 'Model name as listed on the leaderboard, e.g. gpt-5.4-xhigh'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['model', 'displayName', 'family', 'effort', 'provider', 'reasoning', 'accuracy', 'correct', 'total', 'failedRuns', 'bySize'], 'properties': {'model': {'type': 'string', 'description': 'Model variant id, e.g. claude-opus-5.5-high'}, 'total': {'type': 'number'}, 'bySize': {'type': 'array', 'items': {'type': 'object', 'required': ['size'], 'properties': {'size': {'type': 'string'}}, 'additionalProperties': {}}}, 'effort': {'type': ['string', 'null']}, 'family': {'type': 'string'}, 'correct': {'type': 'number'}, 'accuracy': {'type': 'number', 'description': 'Percentage of puzzles solved'}, 'provider': {'type': ['string', 'null']}, 'reasoning': {'type': 'boolean'}, 'failedRuns': {'type': 'number'}, 'displayName': {'type': 'string'}}, 'additionalProperties': {}}
get_puzzle
Get puzzle
One puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['id'], 'properties': {'id': {'type': 'string', 'description': 'Puzzle id from list_puzzles'}, 'include_solution': {'type': 'boolean', 'description': 'Include a reference solution'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['id', 'index', 'size', 'width', 'height', 'rowClues', 'columnClues', 'url', 'prompt'], 'properties': {'id': {'type': 'string'}, 'url': {'type': 'string'}, 'size': {'type': 'string'}, 'index': {'type': 'number'}, 'width': {'type': 'number'}, 'height': {'type': 'number'}, 'prompt': {'type': 'string', 'description': 'Clue text as models received it'}, 'rowClues': {'type': 'array', 'items': {'type': 'array', 'items': {'type': 'number'}}}, 'columnClues': {'type': 'array', 'items': {'type': 'array', 'items': {'type': 'number'}}}, 'referenceSolution': {'type': 'string'}}, 'additionalProperties': {}}
get_puzzle_results
Get puzzle results
Per-model outcomes for one puzzle. Answers are omitted unless requested.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['id'], 'properties': {'id': {'type': 'string', 'description': 'Puzzle id from list_puzzles'}, 'effort': {'type': 'string', 'description': 'best, all (default), or one effort level'}, 'family': {'type': 'string', 'description': 'Comma-separated family ids; empty means no filter'}, 'provider': {'type': 'string', 'description': 'Comma-separated provider ids; empty means no filter'}, 'reasoning': {'type': 'boolean', 'description': 'Only reasoning (true) or non-reasoning (false) variants'}, 'open_weights': {'type': 'boolean', 'description': 'Only open-weight (true) or closed (false) models'}, 'include_answers': {'type': 'boolean', 'description': "Include each model's answer grid"}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['updatedAt', 'puzzleId', 'index', 'size', 'attempts', 'solved', 'runs'], 'properties': {'runs': {'type': 'array', 'items': {'type': 'object', 'required': ['model', 'correct', 'status', 'displayName'], 'properties': {'model': {'type': 'string'}, 'answer': {'type': ['string', 'null']}, 'status': {'type': 'string'}, 'correct': {'type': 'boolean'}, 'displayName': {'type': 'string'}}, 'additionalProperties': {}}}, 'size': {'type': 'string'}, 'index': {'type': 'number'}, 'solved': {'type': 'number'}, 'attempts': {'type': 'number'}, 'puzzleId': {'type': 'string'}, 'updatedAt': {'type': 'string'}}, 'additionalProperties': False}
list_families
List families
Model families, available efforts and best variants.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'properties': {}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['families'], 'properties': {'families': {'type': 'array', 'items': {'type': 'object', 'required': ['family', 'displayName', 'provider', 'bestVariant', 'efforts'], 'properties': {'family': {'type': 'string'}, 'efforts': {'type': 'array', 'items': {'type': 'string'}}, 'provider': {'type': ['string', 'null']}, 'bestVariant': {'type': 'string'}, 'displayName': {'type': 'string'}}, 'additionalProperties': {}}}}, 'additionalProperties': False}
list_providers
List providers
Provider ids, names, families and variant counts.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'properties': {}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['providers'], 'properties': {'providers': {'type': 'array', 'items': {'type': 'object', 'required': ['id', 'name', 'variantCount', 'families'], 'properties': {'id': {'type': 'string'}, 'name': {'type': 'string'}, 'families': {'type': 'array', 'items': {'type': 'string'}}, 'variantCount': {'type': 'number'}}, 'additionalProperties': {}}}}, 'additionalProperties': False}
list_puzzles
List puzzles
The benchmark puzzles with their ids and row/column clues.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'properties': {'size': {'enum': ['5x5', '10x10', '15x15', '20x20'], 'type': 'string', 'description': 'Grid size to filter on'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['puzzles'], 'properties': {'puzzles': {'type': 'array', 'items': {'type': 'object', 'required': ['id', 'index', 'size', 'width', 'height', 'rowClues', 'columnClues', 'url'], 'properties': {'id': {'type': 'string'}, 'url': {'type': 'string'}, 'size': {'type': 'string'}, 'index': {'type': 'number'}, 'width': {'type': 'number'}, 'height': {'type': 'number'}, 'rowClues': {'type': 'array', 'items': {'type': 'array', 'items': {'type': 'number'}}}, 'columnClues': {'type': 'array', 'items': {'type': 'array', 'items': {'type': 'number'}}}}, 'additionalProperties': {}}}}, 'additionalProperties': False}
list_runs
List runs
Individual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.
読み取り専用
入力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'properties': {'size': {'enum': ['5x5', '10x10', '15x15', '20x20'], 'type': 'string', 'description': 'Grid size to filter on'}, 'limit': {'type': 'integer', 'maximum': 500, 'minimum': 1, 'description': 'Default 100'}, 'model': {'type': 'string', 'description': 'Only runs of this model variant id'}, 'offset': {'type': 'integer', 'maximum': 9007199254740991, 'minimum': 0, 'description': 'Runs to skip, for paging; default 0'}, 'puzzle_id': {'type': 'string', 'description': 'Only runs on this puzzle id from list_puzzles'}, 'include_output': {'type': 'boolean', 'description': 'Include raw prompt and model output (large)'}}}
出力スキーマ
{'type': 'object', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'required': ['total', 'limit', 'offset', 'runs'], 'properties': {'runs': {'type': 'array', 'items': {'type': 'object', 'required': ['model', 'puzzleId', 'size', 'correct', 'status'], 'properties': {'size': {'type': 'string'}, 'model': {'type': 'string'}, 'status': {'type': 'string'}, 'correct': {'type': 'boolean'}, 'puzzleId': {'type': 'string'}}, 'additionalProperties': {}}}, 'limit': {'type': 'number'}, 'total': {'type': 'number'}, 'offset': {'type': 'number'}}, 'additionalProperties': False}
履歴
最近のツール変更
追加
list_runs
2026年10月2日2:40
追加
get_model_puzzles
2026年10月2日2:40
追加
get_puzzle_results
2026年10月2日2:40
追加
check_solution
2026年10月2日2:40
追加
get_puzzle
2026年10月2日2:40
追加
list_puzzles
2026年10月2日2:40
追加
get_model_results
2026年10月2日2:40
追加
compare_models
2026年10月2日2:40
追加
list_families
2026年10月2日2:40
追加
list_providers
2026年10月2日2:40
追加
get_leaderboard
2026年10月2日2:40
関連MCP
類似のMCPサーバー
Huggingface
io.github.pipeworx-io/huggingface
Provides access to Hugging Face model, dataset, and Space metadata, alongside broader structured research and data-routing tools.
Ai Model Experiments
io.github.pipeworx-io/ai-model-experiments
Runs prompts across multiple AI models and compares their outputs, costs, latency, token usage, and errors.
branchly
io.branchly/branchly
Manages an AI application’s knowledge-base content, prompts, tools, data sources, retrieval context, sessions, and usage analytic…
myriade
ai.myriade/myriade
Lets users explore and query a data warehouse through an AI data analyst agent.
HuggingFace New Dataset Release Tracker (hfdatasets)
com.a2awire/data-new-dataset-release-huggingface-tracker
Tracks newly released Hugging Face datasets, including updates, tags, and licenses.
aifu Agent Market
asia.aifu/agent-market
Discovers and hires agents or approved humans through a job ledger, verifies agent performance, and provides crypto, equity, sect…
SigRank — AI Operator Benchmarking
com.signalaf/sigrank
Benchmarks AI operator token usage, calculates cascade metrics, compares leaderboard performance, diagnoses efficiency, and simul…
Automan
io.github.i02202/automan
Offers paid AI skills for data analysis, classification, extraction, reporting, summarization, translation, text editing, copywri…