Servidor MCP

robots-txt-checker

io.github.ryuamarines/robots-txt-checker
Herramientas para desarrolladores Requiere autenticación MCP 2025-11-25

Qué hace este MCP

Checks whether a URL path is permitted by a website's robots.txt rules.

robots_txt_checker
robots_txt_checker
Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).
Esquema de entrada
{'type': 'object', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['checks'], 'properties': {'checks': {'type': 'array', 'items': {'type': 'string'}, 'description': 'JSON-encoded strings, each an object like {"url":...,"path":...,"userAgent":...}'}, 'payment': {'type': 'string', 'description': 'x402 payment payload (base64 JSON), signed EIP-712 exact/transferWithAuthorization. Omit on the first call â\x80\x94 the tool will return payment requirements (accepts[]) as an error result.'}}, 'additionalProperties': False}
Añadido
robots_txt_checker
17 de September de 2026 a las 12:49