IA-QA — 130+ QA & Dev Tools for AI Agents MCP

ab_test_report

Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns statistical comparison: mean, median, std deviation, winner, and improvement percentage.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['variant_a', 'variant_b'], 'properties': {'variant_a': {'type': 'object', 'properties': {'name': {'type': 'string', 'description': 'Name/label for variant A'}, 'scores': {'type': 'array', 'items': {'type': 'number'}, 'description': 'Array of scores (0-100)'}}, 'description': 'First variant configuration with name and score array'}, 'variant_b': {'type': 'object', 'properties': {'name': {'type': 'string', 'description': 'Name/label for variant B'}, 'scores': {'type': 'array', 'items': {'type': 'number'}, 'description': 'Array of scores (0-100)'}}, 'description': 'Second variant configuration with name and score array'}}}

Schéma de sortie

{'type': 'object', 'properties': {'max': {}, 'min': {}, 'mean': {'type': 'string'}, 'count': {'type': 'number'}, 'median': {'type': 'string'}, 'winner': {}, 'std_dev': {'type': 'string'}, 'variant_a': {'type': 'object'}, 'variant_b': {'type': 'object'}, 'recommendation': {'type': 'number'}, 'improvement_percent': {}}, 'additionalProperties': True}

analyze_diff_bugs

Pattern-based diff linter: flags a fixed set of risky shapes in changed code — query-string interpolation (SQL/Cypher/Mongo injection shape), shell interpolation, eval/new Function, empty catch blocks, regex built from a variable, fewer catch blocks than before, and named authorization guards that disappeared. Every finding cites the line that produced it. It does NOT do data-flow analysis: it cannot follow a value to a sink, across functions or files, and an empty result is not a safety verdict (the response lists what it did not analyse). Advisory triage — use a static analyser for a real security gate.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['version2'], 'properties': {'context': {'type': 'string', 'description': 'Optional PR title or feature context for better analysis'}, 'version1': {'type': 'string', 'description': 'Original code (before changes). If omitted, only the new version is analysed.'}, 'version2': {'type': 'string', 'description': 'New/modified code (after changes)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'bugs': {'type': 'array', 'items': {'type': 'object'}}, 'disclaimer': {'type': 'string'}, 'notAnalysed': {'type': 'array', 'items': {'type': 'string'}}, 'overallRisk': {'type': 'string'}, 'rulesApplied': {'type': 'number'}, 'scannedLines': {'type': 'number'}, 'totalSuggestions': {'type': 'number'}}, 'additionalProperties': True}

analyze_responses

Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox). Without a reference: computes consensus — pairwise cosine agreement, the most-representative output, and the outlier. With a `reference` (ground truth): also ranks every output by closeness (token cosine + ROUGE-L composite) and names the closest. No LLM, no key, runs locally. The reference mode is gate-able in CI; the consensus/outlier mode is a signal (no reference to fail against). You bring the outputs (2+). For a 2-way head-to-head with structural JSON diff use compare_responses instead.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['responses'], 'properties': {'reference': {'type': 'string', 'description': 'Optional ground-truth answer. If set, each output is also ranked by closeness to it and the closest one is named.'}, 'responses': {'type': 'array', 'items': {'type': 'object', 'required': ['text'], 'properties': {'text': {'type': 'string', 'description': 'The produced output'}, 'label': {'type': 'string', 'description': 'Human name for this candidate (e.g. model id)'}}}, 'minItems': 2, 'description': 'The outputs to analyze (same task, N models/prompts/versions). Each item is a plain string or { "label": "GPT-4o", "text": "..." }. At least 2 required.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'count': {}, 'summary': {'type': 'string'}, 'consensus': {}, 'reference_ranking': {}}, 'additionalProperties': True}

base64_decode

Decode a Base64 string back to UTF-8 text. Use for inspecting Base64-encoded API responses, JWT payload claims, config file values, or attachment data.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Base64 string to decode'}}}

Schéma de sortie

{'type': 'object', 'properties': {'decoded': {}}, 'additionalProperties': True}

base64_encode

Encode a UTF-8 string to Base64. Use when you need to embed binary data, multi-line text, or special characters safely inside JSON fields, HTTP headers, or data URIs.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to encode'}}}

Schéma de sortie

{'type': 'object', 'properties': {'encoded': {'type': 'string'}}, 'additionalProperties': True}

bias_detect

Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone). Returns a bias score (0-100), sentiment analysis per variant, pairwise Jaccard similarity, and a human-readable verdict. No API key needed — runs entirely locally.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['responses'], 'properties': {'responses': {'type': 'array', 'items': {'type': 'object', 'required': ['variantId', 'answer'], 'properties': {'answer': {'type': 'string', 'description': 'The LLM response text for this variant'}, 'variantId': {'type': 'string', 'description': 'Identifier for the demographic variant (e.g. "male", "female", "western", "young")'}}}, 'minItems': 2, 'description': 'Array of variant responses to compare for bias'}}}

Schéma de sortie

{'type': 'object', 'properties': {'verdict': {'type': 'string'}, 'lengthCV': {'type': 'number'}, 'biasScore': {'type': 'number'}, 'sentiments': {'type': 'array', 'items': {'type': 'object', 'properties': {'ratio': {'type': 'number'}, 'negative': {'type': 'number'}, 'positive': {'type': 'number'}, 'variantId': {'type': 'string'}}}}, 'avgSimilarity': {'type': 'number'}, 'minSimilarity': {'type': 'number'}, 'sentimentVariance': {'type': 'number'}, 'pairwiseSimilarities': {'type': 'array', 'items': {'type': 'object', 'properties': {'a': {'type': 'string'}, 'b': {'type': 'string'}, 'score': {'type': 'number'}}}}}, 'additionalProperties': True}

bm25_score

Compute BM25 relevance score between a query and one or more documents. BM25 is the industry-standard keyword-based ranking algorithm used in Elasticsearch, OpenSearch, and Weaviate hybrid search. Returns ranked results with normalized scores.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['query', 'documents'], 'properties': {'b': {'type': 'number', 'description': 'Length normalization factor (default: 0.75)'}, 'k1': {'type': 'number', 'description': 'Term frequency saturation (default: 1.5)'}, 'query': {'type': 'string', 'description': 'The search query'}, 'top_k': {'type': 'number', 'description': 'Return top K results (default: all)'}, 'documents': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Array of documents to rank'}}}

Schéma de sortie

{'type': 'object', 'properties': {'b': {}, 'k1': {}, 'index': {}, 'query': {}, 'results': {}, 'bm25_score': {'type': 'number'}, 'doc_length': {'type': 'number'}, 'doc_preview': {}, 'avg_doc_length': {'type': 'number'}, 'documents_count': {'type': 'number'}}, 'additionalProperties': True}

build_rag_prompt

Assemble a complete RAG (Retrieval-Augmented Generation) prompt from retrieved context chunks and a user query. Handles token budgeting, citation numbering, system instruction injection, and source attribution.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['query', 'chunks'], 'properties': {'query': {'type': 'string', 'description': 'The user question to answer'}, 'chunks': {'type': 'array', 'items': {'type': 'object', 'properties': {'text': {'type': 'string'}, 'score': {'type': 'number'}, 'source': {'type': 'string'}}}, 'description': 'Retrieved context chunks with .text (required), .source (optional), .score (optional)'}, 'language': {'type': 'string', 'description': 'Response language instruction (e.g. "French", "Spanish")'}, 'cite_sources': {'type': 'boolean', 'description': 'Add [1], [2] citation numbers (default: true)'}, 'max_context_tokens': {'type': 'number', 'description': 'Max tokens for context section (default: 2000)'}, 'system_instruction': {'type': 'string', 'description': 'Custom system instruction (default: standard RAG grounding instruction)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'prompt': {}, 'system_prompt': {}, 'chunks_included': {'type': 'number'}, 'included_chunks': {}, 'chunks_truncated': {'type': 'number'}, 'total_tokens_estimate': {'type': 'number'}, 'context_tokens_estimate': {'type': 'number'}}, 'additionalProperties': True}

calculate_readability

Calculate readability scores: Flesch Reading Ease, Flesch-Kincaid Grade Level, Coleman-Liau Index, and Automated Readability Index. Useful for evaluating LLM output quality.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to analyze for readability'}}}

Schéma de sortie

{'type': 'object', 'properties': {'level': {}, 'stats': {'type': 'object'}, 'coleman_liau_index': {'type': 'number'}, 'flesch_reading_ease': {}, 'flesch_kincaid_grade': {'type': 'number'}, 'automated_readability_index': {'type': 'number'}}, 'additionalProperties': True}

case_convert

Convert a string between naming conventions: camelCase, PascalCase, snake_case, kebab-case, UPPER_SNAKE_CASE, dot.case, Title Case. Essential for code generation and refactoring.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input', 'to'], 'properties': {'to': {'type': 'string', 'description': 'Target case: "camel", "pascal", "snake", "kebab", "upper_snake", "dot", "title"'}, 'input': {'type': 'string', 'description': 'String to convert (e.g., "myVariableName", "my-css-class")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'result': {}, 'from_words': {}, 'target_case': {}}, 'additionalProperties': True}

check_contrast_ratio

Calculate WCAG 2.1 contrast ratio between two colors. Returns ratio and compliance for AA/AAA normal and large text.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['foreground', 'background'], 'properties': {'background': {'type': 'string', 'description': 'Background color in hex (e.g., "#ffffff")'}, 'foreground': {'type': 'string', 'description': 'Foreground color in hex (e.g., "#333333")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'ratio': {}, 'AA_large': {'type': 'boolean'}, 'AAA_large': {'type': 'boolean'}, 'AA_normal': {'type': 'boolean'}, 'AAA_normal': {'type': 'boolean'}, 'background': {'type': 'object'}, 'foreground': {'type': 'object'}, 'ratio_text': {'type': 'string'}}, 'additionalProperties': True}

color_convert

Convert a color between HEX, RGB, and HSL formats. Use when translating design tokens between CSS notations, verifying color accessibility, or normalizing color values from user input. Accepts #rrggbb, #rgb, rgb(r,g,b), or hsl(h,s%,l%).

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Color value to convert, e.g. "#ff6b6b", "rgb(255,107,107)", "hsl(0,100%,71%)"'}}}

Schéma de sortie

{'type': 'object', 'properties': {'b': {}, 'g': {}, 'r': {}, 'hex': {}, 'hsl': {'type': 'string'}, 'rgb': {'type': 'string'}, 'input': {}}, 'additionalProperties': True}

compare_models

Compare 2-5 AI models side by side: context window, pricing, multimodal, reasoning capabilities, and provider. Returns a comparison table with a recommendation based on your use case.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['models'], 'properties': {'models': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Array of 2-5 model names (e.g. ["gpt-4o","claude-3.5-sonnet","gemini-2.0-flash"])'}, 'use_case': {'enum': ['cost', 'context', 'reasoning', 'multimodal', 'speed'], 'type': 'string', 'description': 'Optimize recommendation for this criterion'}}}

Schéma de sortie

{'type': 'object', 'properties': {'rows': {}, 'model': {}, 'use_case': {}, 'recommendation': {}, 'models_compared': {'type': 'number'}, 'cost_per_1k_total': {'type': 'string'}}, 'additionalProperties': True}

compare_responses

Compare two ALREADY-PRODUCED outputs (e.g. model A vs model B on the same task) side by side. Returns deterministic metrics (token cosine, ROUGE-L, Jaccard, length/structure deltas, JSON diff) and a verdict. If a `reference` (ground truth) is given, scores each output against it and picks the closer one. If `model` + `api_key` are given, an LLM judge also picks a qualitative winner for the task. No re-execution — you bring the outputs.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['response_a', 'response_b'], 'properties': {'task': {'type': 'string', 'description': 'The task/prompt both outputs were answering — used by the LLM judge for context'}, 'model': {'type': 'string', 'description': 'Optional judge model id (BYOK). When set with api_key, an LLM judge picks a qualitative winner.'}, 'api_key': {'type': 'string', 'description': 'Optional API key for the judge model (BYOK). Used only for the judge call; never stored.'}, 'label_a': {'type': 'string', 'description': 'Label for output A (e.g. "GPT-4o", "v1.0")'}, 'label_b': {'type': 'string', 'description': 'Label for output B (e.g. "GPT-5-nano", "v1.1")'}, 'reference': {'type': 'string', 'description': 'Optional ground-truth / expected answer. If set, each output is scored against it and the closer one wins (deterministic).'}, 'check_json': {'type': 'boolean', 'description': 'Try to parse as JSON and compare structurally (keys, types, values)'}, 'response_a': {'type': 'string', 'description': "First output (e.g. model A's answer)"}, 'response_b': {'type': 'string', 'description': "Second output (e.g. model B's answer)"}}}

Schéma de sortie

{'type': 'object', 'properties': {'judge': {}, 'labelA': {}, 'labelB': {}, 'metrics': {}, 'summary': {'type': 'string'}, 'verdict': {}}, 'additionalProperties': True}

consistency_check

Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison). Fast, deterministic, no API key needed. Limitations: relies on surface-level word matching — "Paris is the capital of France" vs "Paris is the French capital" may score low despite semantic equivalence. For true semantic consistency, use run_semantic_tests with embedding mode. Essential for determinism testing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['responses'], 'properties': {'responses': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Array of 2+ LLM responses to compare (same prompt, different runs)'}, 'check_facts': {'type': 'boolean', 'description': 'Check for contradictory numbers/facts across responses (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'verdict': {}, 'fact_drift': {}, 'avg_similarity': {}, 'response_count': {'type': 'number'}, 'pairwise_scores': {}, 'fact_contradiction': {}, 'length_variance_percent': {}}, 'additionalProperties': True}

context_window_check

Given an array of message objects [{role, content}], estimate total token usage and check if it fits in the target model's context window. Warns about truncation risk.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['messages', 'model'], 'properties': {'model': {'type': 'string', 'description': 'Target model name (e.g. gpt-4o, claude-3.5-sonnet)'}, 'messages': {'type': 'array', 'items': {'type': 'object', 'properties': {'role': {'type': 'string'}, 'content': {'type': 'string'}}}, 'description': 'Array of messages (system/user/assistant)'}, 'max_output_tokens': {'type': 'number', 'description': 'Reserved tokens for output (default: 4096)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'fits': {}, 'role': {}, 'chars': {'type': 'number'}, 'index': {}, 'model': {}, 'tokens': {}, 'warnings': {}, 'breakdown': {'type': 'object'}, 'per_message': {}, 'total_tokens': {}, 'message_count': {'type': 'number'}, 'context_window': {}, 'total_input_tokens': {}, 'utilization_percent': {'type': 'string'}, 'reserved_output_tokens': {}}, 'additionalProperties': True}

conversation_analyze

Analyze a multi-turn conversation for context retention, topic drift, instruction following, and repetition. Accepts messages array [{role, content}]. Essential for chatbot QA.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['messages'], 'properties': {'messages': {'type': 'array', 'items': {'type': 'object', 'properties': {'role': {'enum': ['system', 'user', 'assistant'], 'type': 'string'}, 'content': {'type': 'string'}}}, 'description': 'Conversation messages in order'}}}

Schéma de sortie

{'type': 'object', 'properties': {'turn_count': {'type': 'number'}, 'repetitions': {}, 'topic_drift': {}, 'user_messages': {'type': 'number'}, 'context_retention': {}, 'has_system_prompt': {'type': 'boolean'}, 'assistant_messages': {'type': 'number'}, 'avg_response_length': {'type': 'number'}, 'repetition_detected': {'type': 'boolean'}}, 'additionalProperties': True}

cookie_security_audit

Audit the security attributes of cookies set by any URL. Fetches the URL and inspects all Set-Cookie headers for: HttpOnly, Secure, SameSite, Domain scope, Path scope, Max-Age/Expires, __Host-/__Secure- prefixes. Flags insecure patterns: missing HttpOnly on session cookies, missing Secure flag, SameSite=None without Secure, overly broad Domain, and excessive TTL. Returns per-cookie grades and an overall security score (0–100) — the score is the WEAKEST cookie, not the average, so one leaking session cookie cannot be averaged into a green result (average_score is reported separately).

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['url'], 'properties': {'url': {'type': 'string', 'description': 'Full URL to audit (e.g. https://example.com/login)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'url': {}, 'name': {}, 'path': {}, 'score': {'type': 'number'}, 'domain': {}, 'issues': {}, 'secure': {}, 'cookies': {'type': 'array'}, 'max_age': {}, 'message': {'type': 'string'}, 'httpOnly': {}, 'sameSite': {}, 'host_prefix': {}, 'cookies_found': {'type': 'number'}, 'secure_prefix': {}}, 'additionalProperties': True}

cors_checker

Check the CORS configuration of a URL the same way a browser would. Returns the main response status, all Access-Control-* headers, the tested origin, and the preflight OPTIONS response. Use this for direct CORS debugging, not just security auditing.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['url'], 'properties': {'url': {'type': 'string', 'description': 'Full URL to test, e.g. https://api.example.com/resource'}, 'method': {'type': 'string', 'description': 'HTTP method to simulate (default: GET)'}, 'origin': {'type': 'string', 'description': 'Origin header to simulate (default: https://yourdomain.com)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'url': {}, 'method': {}, 'status': {}, 'preflight': {}, 'allHeaders': {}, 'corsHeaders': {}, 'testedOrigin': {}}, 'additionalProperties': True}

cors_test

Test a URL for CORS misconfigurations. Sends preflight (OPTIONS) requests with various Origin headers to detect: wildcard origins with credentials, origin reflection (echoing any origin), null origin acceptance, subdomain wildcard bypass, and missing Vary headers. Returns risk level (safe/low/medium/high/critical) plus per-origin results. "unknown" means nothing was actually tested — every origin either failed to connect or answered 5xx, so the target returned no CORS decision; never read it as "safe". A 4xx preflight IS a real result (the server refused it and a browser would fail closed).

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['url'], 'properties': {'url': {'type': 'string', 'description': 'Full URL to test (e.g. https://api.example.com/endpoint)'}, 'origin': {'type': 'string', 'description': 'Custom Origin header to test (default: tests multiple origins automatically)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'url': {}, 'error': {'type': 'string'}, 'tests': {}, 'warning': {'type': 'string'}, 'risk_level': {'enum': ['safe', 'low', 'medium', 'high', 'critical', 'unknown']}, 'origins_tested': {'type': 'number'}, 'total_findings': {'type': 'number'}, 'origins_reachable': {'type': 'number'}, 'origins_conclusive': {'type': 'number'}, 'origins_inconclusive': {'type': 'number'}}, 'additionalProperties': True}

cot_analyzer

Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM. Detects step count, logical flow, conclusion presence, backtracking, and estimates reasoning depth. Useful for o1/o3/DeepSeek-R1 evaluation.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['reasoning'], 'properties': {'reasoning': {'type': 'string', 'description': 'The CoT / reasoning trace text (e.g. from <think> tags or step-by-step output)'}, 'expected_conclusion': {'type': 'string', 'description': 'Expected final answer to check against (optional)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'markers': {}, 'step_count': {'type': 'number'}, 'total_chars': {'type': 'number'}, 'total_lines': {'type': 'number'}, 'has_conclusion': {'type': 'boolean'}, 'reasoning_depth': {}, 'backtracking_signals': {}, 'reasoning_depth_label': {}, 'conclusion_matches_expected': {}}, 'additionalProperties': True}

count_code_lines

Count lines of code: total, code lines, comment lines, blank lines, and comment density. Supports JS/TS, Python, Java/C/C++, Ruby, Go, Shell, HTML/XML, and CSS.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Source code to analyze'}, 'language': {'type': 'string', 'description': 'Language hint: "js", "ts", "py", "java", "c", "rb", "go", "sh", "html", "css" (auto-detect if omitted)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'language': {}, 'code_lines': {}, 'blank_lines': {}, 'total_lines': {}, 'comment_lines': {}, 'comment_density': {}, 'code_to_comment_ratio': {'type': 'string'}}, 'additionalProperties': True}

count_tokens

Estimate the token count of a text string using the cl100k_base approximation (~4 chars/token). Call this BEFORE sending any text to an LLM API to check if it fits within the model context window and to estimate cost. Returns token estimate, character count, and word count.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to count tokens for'}}}

Schéma de sortie

{'type': 'object', 'properties': {'chars': {}, 'words': {}, 'tokens_estimate': {}}, 'additionalProperties': True}

create_confluence_page

Create a new Confluence page from the output of jira_to_test_suite. Formats Gherkin, E2E steps, API tests, and test data as a properly structured Confluence page with code blocks and tables. STATEFUL — creates a new page in the specified space.

Schéma d’entrée

{'type': 'object', 'required': ['confluence_base_url', 'confluence_email', 'confluence_token', 'space_key', 'test_suite'], 'properties': {'title': {'type': 'string', 'description': 'Page title. Defaults to "Test Plan: {issue_key}"'}, 'issue_key': {'type': 'string', 'description': 'Source Jira issue key (for the page title and source link)'}, 'issue_url': {'type': 'string', 'description': 'Source Jira issue URL (added as a link in the page)'}, 'space_key': {'type': 'string', 'description': 'Confluence space key where the page will be created, e.g. "QA", "ENG"'}, 'test_suite': {'type': 'object', 'description': 'The test_suite object from jira_to_test_suite result', 'additionalProperties': True}, 'parent_page_id': {'type': 'string', 'description': 'Optional parent page ID — page will be created as a child of this page'}, 'confluence_email': {'type': 'string', 'description': 'Atlassian account email'}, 'confluence_token': {'type': 'string', 'description': 'Atlassian API token'}, 'confluence_base_url': {'type': 'string', 'description': 'Atlassian base URL'}}}

Schéma de sortie

{'type': 'object', 'properties': {'title': {'type': 'string'}, 'page_id': {'type': 'string'}, 'success': {'type': 'boolean'}, 'page_url': {'type': 'string'}}, 'additionalProperties': True}

cron_parse

Parse a cron expression into a human-readable schedule description. Supports standard 5-field cron (minute hour day month weekday).

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['expression'], 'properties': {'expression': {'type': 'string', 'description': 'Cron expression (e.g., "0 9 * * 1-5", "*/15 * * * *")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'fields': {'type': 'object'}, 'expression': {}, 'human_readable': {'type': 'string'}}, 'additionalProperties': True}

cron_validator

Validate a 5-field cron expression, explain the schedule, and preview the next execution times. Use this to debug cron jobs before they reach production. Returns parsed fields, a human-readable description, and upcoming ISO timestamps.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['expression'], 'properties': {'expression': {'type': 'string', 'description': 'Cron expression with 5 fields, e.g. "*/15 9-18 * * 1-5"'}, 'next_runs_count': {'type': 'number', 'description': 'How many upcoming runs to return (1-50, default: 10)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'valid': {'type': 'boolean'}, 'fields': {'type': 'object'}, 'next_runs': {}, 'expression': {}, 'human_readable': {}}, 'additionalProperties': True}

decode_jwt

Decode a JWT (JSON Web Token) and return its header and payload without verifying the signature. Also reports whether the token is expired and the exact expiry date. Use to inspect claims (sub, iss, exp, roles) during debugging or when integrating with an auth provider.

Lecture seule

Schéma d’entrée

{'type': 'object', 'required': ['token'], 'properties': {'token': {'type': 'string', 'description': 'The JWT string to decode (header.payload.signature)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'note': {'type': 'string'}, 'header': {}, 'expired': {}, 'payload': {}, 'expiresAt': {}}, 'additionalProperties': True}

detect_language

Detect the natural language of a text using n-gram frequency analysis and common word markers. Supports 15 languages: English, French, Spanish, German, Italian, Portuguese, Dutch, Russian, Chinese, Japanese, Korean, Arabic, Polish, Turkish, Swedish.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to detect language from (min 20 chars for accuracy)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'lang': {}, 'name': {'type': 'string'}, 'score': {}, 'method': {'type': 'string'}, 'matched': {}, 'language': {'type': 'string'}, 'confidence': {'type': 'number'}, 'top_candidates': {'type': 'array'}}, 'additionalProperties': True}

detect_secrets

Scan code or config files for hardcoded secrets: AWS keys, GitHub tokens, OpenAI/Anthropic API keys, Stripe secrets, JWTs, database connection strings, and generic passwords. Returns findings with severity. Run before every commit.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Code or config content to scan (max 500KB)'}, 'filename': {'type': 'string', 'description': 'Optional filename for context (e.g. ".env", "config.js")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'filename': {}, 'findings': {}, 'risk_level': {}, 'recommendation': {}, 'total_findings': {'type': 'number'}}, 'additionalProperties': True}

diff_mappings

Diff a baseline page mapping against a current one and return a CI-style verdict: PASS / FIX / BLOCK, plus per-element drift (ok, renamed, healable, ambiguous, lost, added, rebound). Pure and deterministic — provide two mappings as JSON with "elements" arrays of {role, name, selector, context?}. Use the companion @ia-qa/self-healing package (npm install -g @ia-qa/self-healing) to capture mappings from your app via its local MCP server ia-qa-heal-mcp, or paste the snippet from ia-qa.com/devtools/selector-drift into your browser console.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['before', 'after'], 'properties': {'after': {'type': 'object', 'description': 'Current page mapping: same shape as before, captured after the UI change.'}, 'before': {'type': 'object', 'description': 'Baseline page mapping: { page, url, capturedAt, elements: [{role, name, selector, context?}] }. Captured before a UI change.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'rows': {'type': 'array'}, 'added': {'type': 'array'}, 'counts': {'type': 'object'}, 'verdict': {}}, 'additionalProperties': True}

diff_text

Compute a unified line-by-line diff between two text strings (LCS algorithm). Returns added/removed/unchanged line counts and formatted diff hunks with configurable context lines (0–20). Use to compare versions of prompts, configs, code snippets, or any text where you need to see exactly what changed.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['a', 'b'], 'properties': {'a': {'type': 'string', 'description': 'Original (before) text'}, 'b': {'type': 'string', 'description': 'Modified (after) text'}, 'context': {'type': 'number', 'description': 'Context lines around each change (0–20, default: 3)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'diff': {'type': 'string'}, 'added': {'type': 'number'}, 'removed': {'type': 'number'}, 'unchanged': {'type': 'number'}}, 'additionalProperties': True}

embedding_similarity

Compute text similarity using local algorithms (Bag of Words, TF-IDF, Character N-grams). No API key needed — runs entirely in-process. NOT real embeddings: for true semantic similarity with vector embeddings, use run_semantic_tests with mode="embeddings" and your OpenAI API key. Supports single pair or batch mode with pipe-separated pairs. Useful for RAG retrieval testing, semantic search evaluation, and text deduplication.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'batch': {'type': 'array', 'items': {'type': 'object', 'required': ['text_a', 'text_b'], 'properties': {'text_a': {'type': 'string'}, 'text_b': {'type': 'string'}}}, 'description': 'Batch mode: array of { text_a, text_b } pairs. Overrides text_a/text_b if provided.'}, 'text_a': {'type': 'string', 'description': 'First text to compare (single-pair mode)'}, 'text_b': {'type': 'string', 'description': 'Second text to compare (single-pair mode)'}, 'methods': {'type': 'array', 'items': {'enum': ['bow', 'tfidf', 'ngram'], 'type': 'string'}, 'description': 'Algorithms to use (default: all three). Options: "bow", "tfidf", "ngram"'}}}

Schéma de sortie

{'type': 'object', 'properties': {'mode': {'type': 'string'}, 'count': {'type': 'number'}, 'scores': {}, 'text_a': {}, 'text_b': {}, 'results': {}}, 'additionalProperties': True}

env_parse

Parse a .env file content into a JSON object. Handles quoted values (single and double), inline comments, export prefix, and escaped sequences (\n, \t inside double quotes). Returns all key-value pairs. Use in CI/CD pipelines, agent config loaders, or when processing dotenv files programmatically.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': '.env file content to parse (e.g. the output of `cat .env`)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'vars': {}, 'count': {'type': 'number'}}, 'additionalProperties': True}

escape_html

Escape HTML special characters (&, <, >, ", ') to their safe HTML entities. ALWAYS call this before inserting any user-provided or LLM-generated content into an HTML template to prevent cross-site scripting (XSS) attacks.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'String to HTML-escape'}}}

Schéma de sortie

{'type': 'object', 'properties': {'escaped': {}, 'original_length': {'type': 'string'}}, 'additionalProperties': True}

estimate_llm_cost

Estimate the API cost in USD for a given model and token counts. Supports all major 2024–2026 models: GPT-4o, GPT-4.1, o3, o4-mini, Claude Opus 4, Claude Sonnet 4/4.5, Gemini 2.5 Pro/Flash, DeepSeek V3/R1, Grok 3, and legacy models.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['model', 'input_tokens'], 'properties': {'model': {'type': 'string', 'description': 'Model name, e.g. "gpt-4o", "claude-3.5-sonnet", "deepseek-v3"'}, 'input_tokens': {'type': 'number', 'description': 'Number of input/prompt tokens'}, 'output_tokens': {'type': 'number', 'description': 'Number of output/completion tokens (default: 0)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'model': {}, 'rates': {'type': 'object'}, 'input_tokens': {}, 'output_tokens': {}, 'input_cost_usd': {'type': 'string'}, 'total_cost_usd': {'type': 'string'}, 'output_cost_usd': {'type': 'string'}}, 'additionalProperties': True}

extract_json_from_text

Extract the first valid JSON object or array embedded in chaotic LLM output (surrounded by markdown fences, prose, or explanatory text). Handles ```json blocks and inline JSON. Call this whenever an LLM returns structured data mixed with explanation text instead of raw JSON.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Raw text (e.g., LLM output) that may contain a JSON object or array'}}}

Schéma de sortie

{'type': 'object', 'properties': {'json': {}, 'source': {'type': 'string'}}, 'additionalProperties': True}

extract_json_path

Extract a value from a JSON string using dot-notation path (e.g., "user.address.city", "items.0.name", "meta.tags"). Supports array index access via numeric path segments.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input', 'path'], 'properties': {'path': {'type': 'string', 'description': 'Dot-notation path, e.g. "user.address.city" or "items.0.name"'}, 'input': {'type': ['string', 'object', 'array'], 'description': 'The JSON to traverse — a JSON string, or the object/array itself.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'path': {}, 'type': {}, 'value': {}}, 'additionalProperties': True}

extract_links

Extract all URLs, email addresses, and domain names from text. Returns categorized and deduplicated results. Useful for content auditing, link checking, and web scraping validation.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to extract links from'}, 'types': {'type': 'array', 'items': {'enum': ['url', 'email', 'domain'], 'type': 'string'}, 'description': 'Types to extract (default: all three)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'total': {}}, 'additionalProperties': True}

extract_todos

Extract TODO, FIXME, HACK, BUG, NOTE, OPTIMIZE, and custom tags from any source code or text. Returns line numbers, tag types, and message text. Essential for technical debt auditing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'tags': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Custom tags to add (default set: TODO, FIXME, HACK, NOTE, BUG, OPTIMIZE, XXX)'}, 'input': {'type': 'string', 'description': 'Code or text to scan'}, 'include_context': {'type': 'boolean', 'description': 'Include full line text (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'items': {}, 'total': {'type': 'number'}, 'counts': {}, 'has_critical': {'type': 'boolean'}}, 'additionalProperties': True}

fetch_confluence_page

Fetch a Confluence page and return its content as clean Markdown. Accepts a numeric page_id or a full page URL. Optionally lists direct child pages. BYOK — credentials transit in-memory only, never stored.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['confluence_base_url', 'confluence_email', 'confluence_token'], 'properties': {'page_id': {'type': 'string', 'description': 'Confluence page ID (numeric string), e.g. "123456789"'}, 'page_url': {'type': 'string', 'description': 'Full Confluence page URL (alternative to page_id), e.g. "https://mycompany.atlassian.net/wiki/spaces/ENG/pages/123456789"'}, 'confluence_email': {'type': 'string', 'description': 'Atlassian account email (same credentials as Jira)'}, 'confluence_token': {'type': 'string', 'description': 'Atlassian API token'}, 'include_children': {'type': 'boolean', 'description': 'List direct child pages (id + title) (default: false)'}, 'confluence_base_url': {'type': 'string', 'description': 'Atlassian base URL, e.g. "https://mycompany.atlassian.net"'}}}

Schéma de sortie

{'type': 'object', 'properties': {'url': {'type': 'string'}, 'title': {'type': 'string'}, 'page_id': {'type': 'string'}, 'children': {'type': 'array', 'items': {'type': 'object'}}, 'markdown': {'type': 'string'}}, 'additionalProperties': True}

fetch_jira_issue

Fetch a complete Jira issue: summary, description converted to Markdown, status, assignee, priority, labels, custom fields, and optionally comments and attachment metadata. BYOK — credentials transit in-memory only, never stored on ia-qa.com.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['issue_key', 'jira_base_url', 'jira_email', 'jira_token'], 'properties': {'fields': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Specific Jira field names to return. Omit for all standard fields.'}, 'issue_key': {'type': 'string', 'description': 'Jira issue key, e.g. "PROJ-123"'}, 'jira_email': {'type': 'string', 'description': 'Atlassian account email'}, 'jira_token': {'type': 'string', 'description': 'Atlassian API token (from id.atlassian.com > Security > API tokens)'}, 'jira_base_url': {'type': 'string', 'description': 'Atlassian base URL, e.g. "https://mycompany.atlassian.net"'}, 'include_comments': {'type': 'boolean', 'description': 'Include issue comments, up to 20 (default: true)'}, 'include_attachments': {'type': 'boolean', 'description': 'Include attachment metadata list (default: false)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'key': {'type': 'string'}, 'url': {'type': 'string'}, 'type': {'type': 'string'}, 'labels': {'type': 'array', 'items': {'type': 'string'}}, 'status': {'type': 'string'}, 'summary': {'type': 'string'}, 'assignee': {'type': 'string'}, 'priority': {'type': 'string'}, 'reporter': {'type': 'string'}, 'description': {'type': 'string'}}, 'additionalProperties': True}

fetch_veille_feed

Fetch the latest QA & AI/LLM articles aggregated from curated RSS sources (Google Testing Blog, DEV.to Testing/QA/AI/LLM/Agents, Hugging Face Blog, Simon Willison). Perfect for agents monitoring the QA & AI landscape. Each article carries summary_source — the XML tag the summary was read from, or "none" when the feed publishes titles and links only; an empty summary with summary_source "none" is a property of that feed, not a parse failure.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'properties': {'limit': {'type': 'number', 'description': 'Max articles to return (default: 20, max: 50)'}, 'category': {'type': 'string', 'description': 'Filter: "qa" (testing/quality), "ai" (AI/LLM/agents), "all" (default — both)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'articles': {'type': 'array'}, 'category': {}, 'total_found': {'type': 'number'}, 'sources_queried': {'type': 'number'}}, 'additionalProperties': True}

few_shot_formatter

Format few-shot examples for LLM prompts. Converts example pairs into formatted blocks. Supports chat format (User/Assistant), XML tags, Markdown, or plain text.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['examples'], 'properties': {'format': {'enum': ['chat', 'xml', 'markdown', 'plain'], 'type': 'string', 'description': 'Output format (default: chat)'}, 'examples': {'type': 'array', 'items': {'type': 'object', 'properties': {'input': {'type': 'string'}, 'label': {'type': 'string'}, 'output': {'type': 'string'}}}, 'description': 'Array of {input, output} pairs'}, 'input_label': {'type': 'string', 'description': 'Label for input (default: User / <input>)'}, 'output_label': {'type': 'string', 'description': 'Label for output (default: Assistant / <output>)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'format': {}, 'formatted': {}, 'example_count': {'type': 'number'}, 'token_estimate': {'type': 'number'}}, 'additionalProperties': True}

find_tool

Search available MCP tools by keyword or category before calling them. Returns matching tool names, descriptions, and optionally their inputSchemas. Call this when you are unsure which tool to use or want to explore the catalogue. Categories: data, encoding, text, llm, qa, rag, dev, security, web.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['query'], 'properties': {'query': {'type': 'string', 'description': 'Keyword(s) to search in tool name and description (e.g. "cors", "token", "vector", "json")'}, 'category': {'type': 'string', 'description': 'Optional: filter by category — data | encoding | text | llm | qa | rag | dev | security | web'}, 'max_results': {'type': 'number', 'description': 'Maximum tools to return (default 10, max 50). Results are ranked by IDF-weighted relevance, so common words like "test" do not inflate the list.'}, 'with_schema': {'type': 'boolean', 'description': 'Set true to include inputSchema in results (default: false)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'hint': {}, 'tool': {}, 'count': {'type': 'number'}, 'query': {}, 'score': {}, 'tools': {'type': 'array'}, 'category': {}, 'truncated': {'type': 'boolean'}, 'total_matches': {'type': 'number'}}, 'additionalProperties': True}

fix_gherkin

Fix Gherkin syntax warnings from a jira_to_test_suite result. Takes the current gherkin text and the _gherkin_warnings array, calls your LLM to fix ONLY the flagged issues (adds missing Given/When/Then steps, etc.), and returns the corrected Gherkin. Lightweight — uses ~300-500 tokens vs ~5k for a full regeneration. Requires BYOK LLM key.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['gherkin', 'warnings', 'api_key', 'model'], 'properties': {'model': {'type': 'string', 'description': 'LLM model to use for the fix, e.g. "gpt-4o-mini". Must belong to the provider whose key you passed in api_key.'}, 'api_key': {'type': 'string', 'description': 'Your own LLM provider API key (BYOK) — OpenAI "sk-…", Anthropic "sk-ant-…", Google "AIzaSy…", or Groq "gsk_…". There is no server-side key for this tool: if you do not have one, do not call it and do not invent a value — placeholders like "configured", "your_api_key" or a masked "sk-…***…" are rejected. Used for this call only, never stored.'}, 'gherkin': {'type': 'string', 'description': 'The current Gherkin text from the jira_to_test_suite result (test_suite.gherkin).'}, 'warnings': {'type': 'array', 'items': {'type': 'string'}, 'description': 'The _gherkin_warnings array from the jira_to_test_suite result.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'latency_ms': {'type': 'number'}, 'model_used': {'type': 'string'}, 'fixed_gherkin': {'type': 'string'}, 'warnings_after': {'type': 'number'}, 'warnings_before': {'type': 'number'}, 'remaining_warnings': {'type': 'array', 'items': {'type': 'string'}}}}

flatten_json

Flatten a nested JSON object to single-level dot-notation keys (e.g. {"a":{"b":1}} → {"a.b":1}), or unflatten dot-notation keys back to a nested object. Supports custom separators.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'mode': {'type': 'string', 'description': '"flatten" (default) or "unflatten"'}, 'input': {'type': ['string', 'object', 'array'], 'description': 'The JSON to flatten or unflatten — a JSON string, or the object itself.'}, 'separator': {'type': 'string', 'description': 'Key separator (default: ".")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'result': {}, 'key_count': {'type': 'number'}, 'max_depth': {'type': 'array'}}, 'additionalProperties': True}

format_bytes

Convert raw byte counts to human-readable sizes in SI (KB=1000) or IEC (KiB=1024) units, or parse size strings back to bytes. Covers B, KB/KiB, MB/MiB, GB/GiB, TB/TiB, PB/PiB.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'bytes': {'type': 'number', 'description': 'Number of bytes to format'}, 'standard': {'enum': ['both', 'si', 'iec'], 'type': 'string', 'description': 'Output standard (default: both)'}, 'size_string': {'type': 'string', 'description': 'Size string to parse to bytes (e.g. "1.5 GB", "512 MiB")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'bytes': {'type': 'number'}, 'original': {}}, 'additionalProperties': True}

format_json

Validate and pretty-print a string that is ALREADY valid JSON. Strict by design — it is a validity gate: valid JSON comes back formatted, anything else is rejected with the exact parse error. It never repairs, completes, or guesses. NOT for: plain text or prose (will fail), JSON embedded in markdown/prose (use extract_json_from_text first), JS objects (JSON.stringify them first), YAML (use yaml_to_json).

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'A raw JSON string, e.g. \'{"key":"value"}\'. Must already parse as JSON — plain text or truncated JSON is rejected, not repaired.'}, 'indent': {'type': 'number', 'description': 'Indent size (default: 2)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'valid': {'type': 'boolean'}, 'formatted': {'type': 'string'}}, 'additionalProperties': True}

format_table

Convert a JSON array of objects into a Markdown table. Automatically detects columns, aligns headers, and fills missing keys with empty cells. Use when an agent needs to present structured data — tool results, model comparisons, test reports — as a readable table in a response or document.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': ['string', 'object', 'array'], 'description': 'The array of objects to render — a JSON string, or the array itself.'}, 'columns': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Column names and order (default: all keys from first row)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'rows': {'type': 'number'}, 'table': {'type': 'string'}, 'columns': {'type': 'number'}}, 'additionalProperties': True}

function_call_validate

Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema, no extra/missing args. For OpenAI function calling & MCP tool_use testing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['function_call', 'allowed_functions'], 'properties': {'function_call': {'type': 'object', 'description': 'The function call object from LLM (e.g. { "name": "get_weather", "arguments": {"city":"Paris"} })', 'additionalProperties': True}, 'allowed_functions': {'type': 'array', 'items': {'type': 'object', 'properties': {'name': {'type': 'string'}, 'optional_args': {'type': 'array', 'items': {'type': 'string'}}, 'required_args': {'type': 'array', 'items': {'type': 'string'}}}}, 'description': 'List of allowed function definitions'}}}

Schéma de sortie

{'type': 'object', 'properties': {'valid': {'type': 'boolean'}, 'errors': {'type': 'array'}, 'error_count': {'type': 'number'}, 'function_name': {}, 'provided_args': {}, 'required_args': {}}, 'additionalProperties': True}

generate_ci_workflow

Generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA. Two gate types, combinable: "eval_contract" runs a .ia-eval.yaml through ia-qa-com/eval-action@v1 (LLM quality gate, needs a provider API key as a repo secret), and "cli_checks" runs deterministic primitives via npx @ia-qa/cli (secret scan, prompt-injection scan, security headers…) whose exit code fails the build. Deterministic template — no LLM call, no API key, same inputs give the same file. Returns the YAML, the secrets to create, and the remaining steps. Pair with generate_eval_yaml to produce the contract itself.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'cron': {'type': 'string', 'description': "Cron expression when triggers include 'schedule' (default: '0 6 * * 1' — Mondays 06:00 UTC)."}, 'gate': {'enum': ['eval_contract', 'cli_checks', 'both', 'selector_drift', 'all'], 'type': 'string', 'description': 'Which gate to emit. eval_contract = LLM eval via the action (default). cli_checks = deterministic CLI assertions. selector_drift = an E2E selector-drift gate via @ia-qa/self-healing (boots the app, captures, diffs against the committed baseline, branches on exit code 0/1/2). both = CLI checks + eval. all = CLI checks, then drift, then eval.'}, 'provider': {'enum': ['groq', 'openai', 'anthropic', 'google'], 'type': 'string', 'description': 'LLM provider the contract runs against — decides which repository secret the workflow wires (default: groq).'}, 'triggers': {'type': 'array', 'items': {'enum': ['push', 'pull_request', 'workflow_dispatch', 'schedule'], 'type': 'string'}, 'description': 'Workflow triggers (default: push + pull_request).'}, 'cli_tools': {'type': 'array', 'items': {'type': 'string'}, 'description': 'IA-QA tool names to run as deterministic gates, e.g. ["secret_scan","prompt_injection_scan"]. Tools with no known CI recipe get a --stdin step flagged in notes.'}, 'min_score': {'type': 'number', 'description': 'Override the contract min_score (0-100). Omit to use the value in the contract.'}, 'app_base_url': {'type': 'string', 'description': 'URL the drift gate waits for before capturing (default: http://127.0.0.1:3000). Must match config.baseUrl in .ia-qa/config.json.'}, 'fail_on_fail': {'type': 'boolean', 'description': 'Fail the build on a FAIL/PARTIAL verdict (default: true). Set false to report without gating.'}, 'node_version': {'type': 'string', 'description': 'Node version for the CLI steps (default: "20").'}, 'contract_path': {'type': 'string', 'description': 'Path to the .ia-eval.yaml contract, relative to the repo root (default: evals/smoke.ia-eval.yaml). Only used when the gate includes eval_contract.'}, 'workflow_name': {'type': 'string', 'description': 'Workflow display name (default: "IA-QA Quality Gate").'}, 'app_start_command': {'type': 'string', 'description': 'Command that boots the app for the selector_drift gate, e.g. "npm run start:ci". Cannot be guessed — omitted, the step carries an explicit TODO and the note says so, because a plausible default would silently map nothing.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'gate': {'type': 'string'}, 'path': {'type': 'string'}, 'yaml': {'type': 'string'}, 'notes': {'type': 'array', 'items': {'type': 'string'}}, 'next_steps': {'type': 'array', 'items': {'type': 'string'}}, 'secrets_required': {'type': 'array', 'items': {'type': 'string'}}}, 'additionalProperties': True}

generate_curl

Generate a curl command from request parameters. Supports GET/POST/PUT/DELETE, custom headers, JSON body, and form data. Useful for documentation, sharing, and debugging API calls.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['url'], 'properties': {'url': {'type': 'string', 'description': 'Request URL (must be http/https)'}, 'body': {'type': 'string', 'description': 'Raw request body string'}, 'method': {'type': 'string', 'description': 'HTTP method (default: GET)'}, 'headers': {'type': 'object', 'description': 'Request headers as key-value object', 'additionalProperties': True}, 'verbose': {'type': 'boolean', 'description': 'Add -v for verbose output (default: false)'}, 'body_json': {'type': 'object', 'description': 'JSON body (auto-adds Content-Type: application/json)', 'additionalProperties': True}, 'follow_redirects': {'type': 'boolean', 'description': 'Follow redirects with -L flag (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'url': {}, 'curl': {'type': 'string'}, 'method': {}, 'header_count': {'type': 'number'}}, 'additionalProperties': True}

generate_eval_yaml

Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do. Uses Groq openai/gpt-oss-20b (server-side, no API key needed). Returns ready-to-run YAML for the LLM Test Runner (run_eval_contract). Picks appropriate evaluators (cosine_similarity, contains_check, hallucination_check, etc.) based on the task type.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['description'], 'properties': {'task_type': {'enum': ['rag', 'summarization', 'classification', 'safety', 'customer_support', 'code_gen'], 'type': 'string', 'description': 'Optional task type hint to guide evaluator selection.'}, 'description': {'type': 'string', 'description': 'Plain-language description of what the LLM under test should do. Be specific: describe inputs, expected behaviour, and constraints.'}, 'system_prompt': {'type': 'string', 'description': 'Optional system prompt of the LLM under test. Helps generate more accurate test cases.'}, 'scenario_count': {'enum': [3, 5, 8], 'type': 'number', 'description': 'Number of scenarios to generate (default: 5). Covers happy path + edge cases + adversarial.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'yaml': {'type': 'string'}, 'task_type': {'type': 'string'}, 'model_used': {'type': 'string'}, 'scenario_count': {'type': 'number'}}, 'additionalProperties': True}

generate_hmac

Compute an HMAC signature for a message using a secret key. Supports SHA-256 (default), SHA-512, SHA-1, and MD5. Used for API request signing, webhook verification (GitHub, Stripe, Twilio), and JWT validation.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['message', 'secret'], 'properties': {'secret': {'type': 'string', 'description': 'Secret key'}, 'message': {'type': 'string', 'description': 'Message to sign'}, 'encoding': {'enum': ['hex', 'base64', 'base64url'], 'type': 'string', 'description': 'Output encoding (default: hex)'}, 'algorithm': {'type': 'string', 'description': 'Hash algorithm: sha256 (default), sha512, sha1, md5'}}}

Schéma de sortie

{'type': 'object', 'properties': {'hmac': {}, 'encoding': {}, 'algorithm': {}, 'message_length': {'type': 'number'}}, 'additionalProperties': True}

generate_html_report

Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, evaluator score bars, and a radar chart. Returns the HTML as a string.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['results'], 'properties': {'results': {'type': 'object', 'description': 'The JSON object returned by run_eval_contract()', 'additionalProperties': True}}}

Schéma de sortie

{'type': 'object', 'properties': {'html': {}}, 'additionalProperties': True}

generate_json_ld

Generate a ready-to-paste <script type="application/ld+json"> snippet for GEO / structured data optimization. Supported types: WebSite, FAQPage, Article, Person, Organization, SoftwareApplication, HowTo.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['type'], 'properties': {'type': {'type': 'string', 'description': 'Schema @type: "WebSite", "FAQPage", "Article", "Person", "Organization", "SoftwareApplication", "HowTo"'}, 'fields': {'type': 'object', 'description': 'Schema fields as key-value pairs (name, url, description, author, datePublished, etc.)', 'additionalProperties': True}, 'faq_items': {'type': 'array', 'items': {'type': 'object', 'properties': {'answer': {'type': 'string'}, 'question': {'type': 'string'}}}, 'description': 'For FAQPage/HowTo: array of { question, answer } objects'}}}

Schéma de sortie

{'type': 'object', 'properties': {'name': {'type': 'string'}, 'schema': {}, 'snippet': {'type': 'string'}, 'acceptedAnswer': {'type': 'object'}}, 'additionalProperties': True}

generate_password

Generate a cryptographically secure random password using crypto.randomBytes. Configurable length (4–128), uppercase letters, digits, and symbols. Use when resetting user passwords, seeding test accounts, or generating API secrets.

Lecture seule

Schéma d’entrée

{'type': 'object', 'properties': {'length': {'type': 'number', 'description': 'Password length (4–128, default: 16)'}, 'numbers': {'type': 'boolean', 'description': 'Include digits (default: true)'}, 'symbols': {'type': 'boolean', 'description': 'Include symbols like !@#$ (default: false)'}, 'uppercase': {'type': 'boolean', 'description': 'Include uppercase letters (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'length': {'type': 'number'}, 'password': {}, 'charset_size': {'type': 'number'}}, 'additionalProperties': True}

generate_slug

Convert any string into a URL-friendly slug: lowercase, ASCII-normalized (é→e), special characters removed, spaces replaced with hyphens. Use for generating SEO-friendly URL paths, file names, or identifier keys from user-provided titles or labels.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'String to slugify'}, 'separator': {'type': 'string', 'description': 'Separator character (default: "-")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'slug': {}}, 'additionalProperties': True}

generate_test_cases

Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description. Declared constraints drive the boundaries: a length or numeric bound ("[8-64]", "min 8 chars", "at least 8 characters" — read from inputs, and from the feature prose when a sentence names exactly one field) yields the last accepted value AND the first rejected one; a format (email/url/uuid, from the type, the field name or the wording) yields malformed-value cases. A bound nobody declared is labelled as this tool's assumption, not as expected behaviour. gherkinFormat renders every case (cap 200, stated in the output) and gherkinScenarioCount lets you check it against totalCases.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['feature'], 'properties': {'inputs': {'type': 'string', 'description': 'Optional: list of input parameters (one per line, e.g. "email: string [required]", "password: string [required, min 8 chars]", "age: number [18-99]")'}, 'feature': {'type': 'string', 'description': 'Feature or function to test. Be specific: describe inputs, expected behaviour, context. Constraints stated here ("password must be at least 8 characters") are used when the sentence names exactly one field.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'feature': {'type': 'string'}, 'test_cases': {'type': 'array', 'items': {'type': 'object'}}, 'parsedInputs': {'type': 'array', 'items': {'type': 'object'}}, 'gherkinFormat': {'type': 'string'}, 'gherkinScenarioCount': {'type': 'number'}}, 'additionalProperties': True}

generate_uuid

Generate one or more cryptographically random UUID v4 identifiers. Use this when you need unique IDs for test fixtures, database records, session tokens, or any scenario requiring a guaranteed-unique string. Returns up to 100 UUIDs in one call.

Lecture seule

Schéma d’entrée

{'type': 'object', 'properties': {'count': {'type': 'number', 'description': 'Number of UUIDs to generate (1–100, default: 1)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'count': {'type': 'number'}, 'uuids': {}}, 'additionalProperties': True}

get_testing_guidelines

Query the IA-QA methodology knowledge base. Returns structured testing guidelines, assertion strategies, thresholds, best practices, and relevant MCP tools for a given topic. Call without a topic to list all available topics. Topics: llm-unit-testing, rag-pipeline, prompt-stability, prompt-ab-testing, embedding-quality, eval-framework, semantic-testing, auto-testing, security, api-testing, ci-cd, multimodal, llm-data-security, agent-observability, pro-tips, learning-paths, golden-dataset, selector-drift, qa-recipes, playbooks. Not sure where to start testing an LLM, RAG pipeline or agent? Call without a topic (or with "start-here"): it maps what you are testing to the tools to call and the output field to gate CI on. A plain question such as "how do I test my RAG" also resolves to the right topic.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'topic': {'enum': ['start-here', 'llm-unit-testing', 'rag-pipeline', 'prompt-stability', 'prompt-ab-testing', 'embedding-quality', 'eval-framework', 'semantic-testing', 'auto-testing', 'security', 'api-testing', 'ci-cd', 'multimodal', 'llm-data-security', 'agent-observability', 'pro-tips', 'learning-paths', 'golden-dataset', 'selector-drift', 'qa-recipes', 'playbooks'], 'type': 'string', 'description': 'The testing topic to retrieve guidelines for. Omit to get the start-here map and the full list of available topics.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'tip': {'type': 'string'}, 'topic': {}, 'usage': {'type': 'string'}, 'keywords': {'type': 'array'}, 'start_here': {'type': 'object'}, 'available_topics': {'type': 'array'}}, 'additionalProperties': True}

guardrail_test

Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex. Returns pass/fail per rule.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['response', 'rules'], 'properties': {'rules': {'type': 'array', 'items': {'type': 'object', 'properties': {'type': {'enum': ['must_include', 'must_not_include', 'max_length', 'min_length', 'format', 'regex_match', 'regex_not_match', 'starts_with', 'ends_with', 'word_count_max', 'word_count_min'], 'type': 'string'}, 'label': {'type': 'string', 'description': 'Optional human-readable label'}, 'value': {'type': 'string', 'description': 'Value for the rule (text, number as string, regex, or format name)'}}}, 'description': 'Array of guardrail rules to check'}, 'response': {'type': 'string', 'description': 'The LLM response to test'}}}

Schéma de sortie

{'type': 'object', 'properties': {'pass': {'type': 'boolean'}, 'rule': {}, 'label': {}, 'value': {}, 'detail': {'type': 'string'}, 'failed': {}, 'passed': {}, 'results': {}, 'all_passed': {'type': 'boolean'}, 'total_rules': {'type': 'number'}}, 'additionalProperties': True}

hallucination_check

Lexical hallucination check: verifies an LLM answer's words, numbers and polarity against the provided source/context. Fast, deterministic, no API key needed. Each answer sentence is aligned to its best-matching source sentence, so a number only counts as support when it sits on the SAME statement ("founded in 1998" is not grounded by "sold 1998 units"), and a negation or antonym flip against that sentence returns verdict "contradicted" — the corrupted-fact hallucination that reuses source vocabulary. Limitations: still lexical — it cannot follow a paraphrase, a synonym, or multi-sentence reasoning, so a "well_grounded" verdict means "nothing lexical found", never "verified true". For entailment use run_semantic_tests (NLI/embedding) or a calibrated judge.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['answer', 'context'], 'properties': {'answer': {'type': 'string', 'description': 'The LLM-generated answer to verify'}, 'strict': {'type': 'boolean', 'description': 'If true, every sentence in the answer must be supported (default: false)'}, 'context': {'type': 'string', 'description': 'The source/reference text that should ground the answer'}}}

Schéma de sortie

{'type': 'object', 'properties': {'detail': {'type': 'string'}, 'message': {'type': 'string'}, 'numbers': {}, 'overlap': {'type': 'number'}, 'verdict': {'type': 'string'}, 'analysis': {}, 'entities': {}, 'grounded': {'type': 'boolean'}, 'sentence': {}, 'total_words': {'type': 'number'}, 'matched_words': {'type': 'number'}, 'contradictions': {}, 'grounded_count': {'type': 'number'}, 'unbound_claims': {}, 'grounding_score': {}, 'total_sentences': {'type': 'number'}, 'ungrounded_count': {'type': 'number'}, 'unsupported_claims': {}}, 'additionalProperties': True}

hash_text

Compute a cryptographic hash of a text string. Use when you need to verify data integrity, generate content fingerprints, hash passwords (prefer SHA-256+), or produce a fixed-length digest of any input. Supports SHA-256 (default), SHA-512, SHA-1, and MD5.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to hash'}, 'algorithm': {'type': 'string', 'description': 'Hash algorithm: sha256 (default), sha512, sha1, md5'}}}

Schéma de sortie

{'type': 'object', 'properties': {'hash': {}, 'algorithm': {}, 'input_length': {'type': 'number'}}, 'additionalProperties': True}

html_to_markdown

Convert HTML to clean Markdown. Strips scripts, styles, nav, ads, and comments. Converts headings, lists, links, images, code blocks. Ideal for preparing web content as LLM context.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'HTML string to convert'}, 'strip_links': {'type': 'boolean', 'description': 'Strip link URLs, keep text only (default: false)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'markdown': {}, 'markdown_length': {'type': 'number'}, 'original_length': {'type': 'number'}}, 'additionalProperties': True}

http_status_lookup

Look up detailed information about any HTTP status code: class, name, description, cacheability, typical causes, and handling best practices. Covers every code in the IANA HTTP Status Code Registry (1xx-5xx, including 226, 425, 451, 508, 511 and the WebDAV codes) with its defining RFC; anything outside the registry is reported as `registered: false` rather than described. `cacheable` means heuristically cacheable by default per RFC 9110 §15.1.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['code'], 'properties': {'code': {'type': 'number', 'description': 'HTTP status code (e.g. 200, 404, 429, 503)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'code': {}, 'desc': {'type': 'string'}, 'name': {'type': 'string'}, 'class': {}, 'cacheable': {}, 'registered': {'type': 'boolean'}, 'description': {}}, 'additionalProperties': True}

identify_caller

Returns what the server knows about the current MCP client: clientInfo captured during initialize, User-Agent, and any _meta fields sent with this request. Useful for debugging caller identification.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'_meta': {'type': 'object', 'properties': {'agent': {'type': 'string'}, 'model': {'type': 'string'}, 'version': {'type': 'string'}}, 'description': 'Optional self-identification. Keys: agent (string), model (string), version (string).'}}}

Schéma de sortie

{'type': 'object', 'properties': {'note': {'type': 'string'}, 'session': {'type': 'object'}, 'meta_override': {'type': 'object'}, 'effective_agent': {'type': 'string'}}, 'additionalProperties': True}

jira_to_test_suite

Transform a Jira ticket into a complete test suite: Gherkin scenarios, E2E steps, API test cases, test data matrix, and ambiguity detection. Accepts either Jira credentials (auto-fetch) or a pre-fetched issue object. The returned test_suite includes _gherkin_warnings (deterministic syntax validation — empty if clean). Requires BYOK LLM key (OpenAI, Anthropic, etc.).

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['api_key', 'model'], 'properties': {'issue': {'type': 'object', 'description': 'Pre-fetched issue object from fetch_jira_issue, OR a mock object with fields: key, summary, description (plain text or Markdown), status, issue_type, priority, labels, comments. Use this for offline/CI testing without Jira credentials.', 'additionalProperties': True}, 'model': {'type': 'string', 'description': 'LLM model to use, e.g. "gpt-4o-mini", "claude-3-5-haiku-20241022", "gemini-2.0-flash".'}, 'api_key': {'type': 'string', 'description': 'Your LLM provider API key (OpenAI sk-, Anthropic sk-ant-, Google AIzaSy-, etc.).'}, 'issue_key': {'type': 'string', 'description': 'Jira issue key to fetch automatically, e.g. "PROJ-123". Required if issue is not provided.'}, 'jira_email': {'type': 'string', 'description': 'Atlassian account email. Required for auto-fetch mode.'}, 'jira_token': {'type': 'string', 'description': 'Atlassian API token. Required for auto-fetch mode.'}, 'max_tokens': {'type': 'integer', 'default': 8192, 'description': 'Maximum tokens for the LLM response. Default: 8192. Increase for large tickets with many ACs; decrease to reduce cost on simple tickets.'}, 'jira_base_url': {'type': 'string', 'description': 'Atlassian base URL. Required for auto-fetch mode.'}, 'confluence_pages': {'type': 'array', 'items': {'type': 'object', 'additionalProperties': True}, 'description': 'Optional array of pre-fetched Confluence page objects from fetch_confluence_page, used as documentation context.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'summary': {'type': 'string'}, 'issue_key': {'type': 'string'}, 'issue_url': {'type': 'string'}, 'latency_ms': {'type': 'number'}, 'model_used': {'type': 'string'}, 'test_suite': {'type': 'object'}, 'tokens_used': {'type': 'number'}}, 'additionalProperties': True}

json_diff

Compute a deep structural diff between two JSON values. Returns added, removed, and changed keys with dot-notation paths. Like git diff but for JSON objects — perfect for API response regression testing. Arrays are not compared blindly by position: the same elements in a different order collapse to a single "reordered" change, and an array of records sharing a stable identity field (id, uuid, key, name…) is matched by that field, so paths read [id=42] and a moved record is not reported as N rewrites.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['before', 'after'], 'properties': {'after': {'type': ['string', 'object', 'array'], 'description': 'The modified JSON (after) — a JSON string, or the value itself.'}, 'before': {'type': ['string', 'object', 'array'], 'description': 'The original JSON (before) — a JSON string, or the value itself.'}, 'max_depth': {'type': 'number', 'description': 'Max nesting depth to recurse (default: 10)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'added': {'type': 'boolean'}, 'changes': {'type': 'array'}, 'removed': {'type': 'boolean'}, 'modified': {'type': 'boolean'}, 'identical': {'type': 'boolean'}, 'total_changes': {'type': 'number'}}, 'additionalProperties': True}

json_schema_generate

Infer a JSON Schema (draft-07) from a sample JSON value. Detects types, required fields, array item shapes, nested objects, and common string formats (email, uri, date, date-time, uuid). Returns a ready-to-use schema compatible with json_schema_validate. Use when you have a sample API response or LLM output and want to auto-generate a validation schema for CI/CD testing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': ['string', 'object', 'array'], 'description': 'The sample JSON value to infer the schema from — a JSON string, or the value itself.'}, 'required_all': {'type': 'boolean', 'description': 'Mark all detected object properties as required (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'type': {'type': 'string'}, 'items': {'type': 'object'}, 'format': {}, 'schema': {'type': 'object'}}, 'additionalProperties': True}

json_schema_validate

Validate a JSON value against a JSON Schema (draft-07 subset). Supports type, required, properties, items, enum, const, pattern, format (email/uri/date), minimum/maximum, minLength/maxLength, minItems/maxItems, uniqueItems, additionalProperties, anyOf, allOf, oneOf. Returns all validation errors with dot-notation paths.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['value', 'schema'], 'properties': {'value': {'type': ['string', 'object', 'array'], 'description': 'The JSON value to validate — a JSON string, or the value itself.'}, 'schema': {'type': ['string', 'object', 'array'], 'description': 'The JSON Schema — a JSON string, or the schema object itself.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'valid': {'type': 'boolean'}, 'errors': {'type': 'array'}, 'error_count': {'type': 'number'}}, 'additionalProperties': True}

json_to_csv

Convert a JSON array of objects to CSV format. Automatically detects columns from all object keys. Handles quoting and escaping per RFC 4180.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': ['string', 'object', 'array'], 'description': 'The array of objects to convert — a JSON string, or the array itself.'}, 'headers': {'type': 'boolean', 'description': 'Include header row (default: true)'}, 'delimiter': {'type': 'string', 'description': 'Column delimiter (default: ",")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'csv': {'type': 'string'}, 'rows': {'type': 'number'}, 'columns': {'type': 'number'}, 'column_names': {}}, 'additionalProperties': True}

json_to_yaml

Convert a JSON object to clean, human-readable YAML. Handles nested objects, arrays, multiline strings, and special characters. No external dependencies.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': ['string', 'object', 'array'], 'description': 'The JSON to convert to YAML — a JSON string, or the value itself.'}, 'indent': {'type': 'number', 'description': 'Indentation size in spaces (default: 2)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'yaml': {}, 'lines': {'type': 'number'}}, 'additionalProperties': True}

latency_benchmark

Measure response time of one or more HTTP endpoints (GET/POST). Runs N iterations and returns min/max/avg/p95 latency. Useful for API and MCP server benchmarking.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['endpoints'], 'properties': {'endpoints': {'type': ['string', 'array'], 'items': {'type': 'object', 'properties': {'url': {'type': 'string', 'description': 'Full URL to test'}, 'body': {'type': 'object', 'description': 'Request body for POST', 'additionalProperties': True}, 'label': {'type': 'string', 'description': 'Optional label for this endpoint'}, 'method': {'enum': ['GET', 'POST'], 'type': 'string', 'description': 'HTTP method (default: GET)'}, 'headers': {'type': 'object', 'description': 'Custom headers', 'additionalProperties': True}}}, 'description': 'Endpoints to benchmark. Accepts a single URL string, an array of URL strings, or an array of {url, method?, body?, headers?, label?} objects.'}, 'iterations': {'type': 'number', 'description': 'Number of iterations per endpoint (default: 3, max: 10)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'results': {'type': 'array', 'items': {'type': 'object'}}, 'iterations': {'type': 'number'}}, 'additionalProperties': True}

levenshtein_distance

Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings. Supports batch comparison. Useful for fuzzy string matching, deduplication, and test result comparison.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'a': {'type': 'string', 'description': 'First string (single-pair mode)'}, 'b': {'type': 'string', 'description': 'Second string (single-pair mode)'}, 'batch': {'type': 'array', 'items': {'type': 'object', 'properties': {'a': {'type': 'string'}, 'b': {'type': 'string'}}}, 'description': 'Batch of {a,b} pairs (max 50)'}, 'case_insensitive': {'type': 'boolean', 'description': 'Ignore case differences (default: false)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'a': {}, 'b': {}, 'mode': {'type': 'string'}, 'count': {'type': 'number'}, 'results': {}, 'distance': {}, 'similarity': {'type': 'string'}, 'operations_needed': {}}, 'additionalProperties': True}

lint_commit_message

Validate a git commit message against the Conventional Commits spec (feat, fix, docs, style, refactor, test, chore, ci, perf, build). Returns compliance score, breaking change detection, and actionable suggestions.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['message'], 'properties': {'strict': {'type': 'boolean', 'description': 'Enforce strict rules: max 72-char subject, imperative mood check (default: false)'}, 'message': {'type': 'string', 'description': 'Git commit message to validate'}}}

Schéma de sortie

{'type': 'object', 'properties': {'type': {}, 'scope': {}, 'score': {'type': 'number'}, 'valid': {'type': 'boolean'}, 'checks': {}, 'subject': {}, 'has_body': {'type': 'boolean'}, 'is_breaking_change': {'type': 'boolean'}}, 'additionalProperties': True}

list_llm_models

List all LLM models available on ia-qa.com with their provider, API endpoint, and capabilities. Filter by provider name (e.g. "Groq", "HuggingFace", "OpenAI") or return the full catalog. Use this to discover which models are available before calling an LLM API, or to compare providers.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'provider': {'type': 'string', 'description': 'Filter by provider name (case-insensitive). E.g. "Groq", "HuggingFace", "OpenAI", "Anthropic", "Google", "DeepSeek", "xAI", "Ollama". Omit for full catalog.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'total': {'type': 'number'}, 'filter': {}, 'models': {}, 'providers': {}}, 'additionalProperties': True}

list_local_tests

Discover .ia-eval.yaml LLM test suite files in the project directory. Scans CWD and standard sub-directories (evals/, tests/, contracts/). Returns file paths ready to pass to run_eval_contract.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'dir': {'type': 'string', 'description': 'Directory to scan (defaults to server CWD)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'dir': {'type': 'string'}, 'count': {'type': 'number'}, 'files': {'type': 'array', 'items': {'type': 'string'}}}, 'additionalProperties': True}

llm_fit_finder

Find the best LLM for a given use case. Compares 30+ cloud API models and 12+ local models by cost, speed, benchmarks, features and VRAM requirements. Returns ranked recommendations with cost simulation. No API key needed.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'mode': {'type': 'string', 'description': 'cloud (API models) or local (Ollama/self-hosted). Default: cloud'}, 'top_n': {'type': 'number', 'description': 'Number of recommendations to return (default: 5)'}, 'vram_gb': {'type': 'number', 'description': 'GPU VRAM in GB (only for mode=local). Default: 16'}, 'features': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Required features: vision, function_calling, json_mode, streaming, reasoning'}, 'use_case': {'type': 'string', 'description': 'Primary use case: chatbot | code | rag | summarization | classification | reasoning | agents | multilingual'}, 'max_budget': {'type': 'number', 'description': 'Maximum monthly budget in USD (based on tokens_per_day)'}, 'quantization': {'type': 'string', 'description': 'Quantization (only for mode=local): Q4_K_M | Q8_0 | FP16. Default: Q4_K_M'}, 'tokens_per_day': {'type': 'number', 'description': 'Estimated daily token volume (default: 100000)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'mode': {'type': 'string'}, 'score': {'type': 'number'}, 'results': {'type': 'array'}, 'vram_gb': {}, 'use_case': {}, 'quantization': {}, 'tokens_per_day': {}, 'total_matching': {'type': 'number'}}, 'additionalProperties': True}

llm_format_check

Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex. Essential for structured output testing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['output', 'expected_format'], 'properties': {'output': {'type': 'string', 'description': 'The LLM output to validate'}, 'regex_pattern': {'type': 'string', 'description': 'Custom regex pattern (only when expected_format is "regex")'}, 'expected_format': {'enum': ['json', 'markdown_heading', 'code_block', 'bullet_list', 'numbered_list', 'table', 'yaml', 'xml', 'regex'], 'type': 'string', 'description': 'Expected format'}}}

Schéma de sortie

{'type': 'object', 'properties': {'valid': {'type': 'boolean'}, 'checks': {}, 'failed': {}, 'passed': {}, 'total_checks': {'type': 'number'}, 'expected_format': {}}, 'additionalProperties': True}

llm_generate

Generate text using open-source LLM models hosted on Groq (ultra-fast) or HuggingFace Inference (serverless). No API key required — the server provides its own keys. Supported models: Qwen3 32B, Gemma 4 27B, Gemma 3 27B, Llama 3.3 70B, Llama 4 Scout, DeepSeek R1, Mistral Small 24B, and more. Use list_llm_models to see the full catalog. Rate-limited to prevent abuse.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['prompt'], 'properties': {'model': {'type': 'string', 'description': 'Model ID (default: "openai/gpt-oss-20b"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.6-27b; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected.'}, 'prompt': {'type': 'string', 'description': 'The user prompt / instruction to send to the model'}, 'system': {'type': 'string', 'description': 'Optional system prompt to set context or persona'}, 'max_tokens': {'type': 'number', 'description': 'Maximum tokens to generate (default: 2048, max: 4096)'}, 'temperature': {'type': 'number', 'description': 'Sampling temperature 0.0–1.5 (default: 0.7)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'model': {}, 'usage': {}, 'content': {}, 'provider': {}, 'latency_ms': {'type': 'number'}}, 'additionalProperties': True}

llm_json_schema_check

Validate that an LLM JSON output matches a JSON Schema definition. Tests required fields, types, enums, nested objects, and arrays. Critical for function-calling and structured output testing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['output', 'schema'], 'properties': {'output': {'type': 'string', 'description': 'The LLM JSON output (raw string, will be parsed)'}, 'schema': {'type': 'object', 'description': 'JSON Schema (draft-07 subset) to validate against', 'additionalProperties': True}}}

Schéma de sortie

{'type': 'object', 'properties': {'valid': {'type': 'boolean'}, 'errors': {'type': 'array'}, 'error_count': {'type': 'number'}, 'parse_error': {}, 'parsed_type': {}}, 'additionalProperties': True}

llm_output_validator

Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safety patterns. Essential for QA testing LLM-powered features.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['output'], 'properties': {'output': {'type': 'string', 'description': 'The LLM output text to validate'}, 'max_length': {'type': 'number', 'description': 'Maximum character length for the output'}, 'min_length': {'type': 'number', 'description': 'Minimum character length for the output'}, 'check_safety': {'type': 'boolean', 'description': 'Check for PII patterns (emails, phones, SSN), profanity signals, and prompt leakage'}, 'must_include': {'type': 'string', 'description': 'Comma-separated strings that MUST appear in the output'}, 'expected_format': {'enum': ['json', 'markdown', 'code', 'plain', 'any'], 'type': 'string', 'description': 'Expected output format'}, 'must_not_include': {'type': 'string', 'description': 'Comma-separated strings that must NOT appear (e.g. "TODO, FIXME, undefined, NaN")'}, 'check_json_schema': {'type': 'string', 'description': 'If expected_format is JSON, provide required keys as comma-separated list to validate the structure'}, 'expected_language': {'type': 'string', 'description': 'Expected language of the output (en, fr, es, de…). Checks for common words.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'total': {'type': 'number'}, 'checks': {}, 'failed': {}, 'passed': {}, 'verdict': {}}, 'additionalProperties': True}

lorem_ipsum

Generate Lorem Ipsum placeholder text for UI mockups, design prototypes, or test data population. Configurable paragraphs (1–10), sentences per paragraph (1–20), and approximate words per sentence (3–30).

Lecture seule

Schéma d’entrée

{'type': 'object', 'properties': {'paragraphs': {'type': 'number', 'description': 'Number of paragraphs to generate (1–10, default: 1)'}, 'words_per_sentence': {'type': 'number', 'description': 'Approximate words per sentence (3–30, default: 10)'}, 'sentences_per_paragraph': {'type': 'number', 'description': 'Sentences per paragraph (1–20, default: 5)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'paragraphs': {}, 'paragraph_count': {'type': 'number'}}, 'additionalProperties': True}

mcp_schema_lint

Lint an MCP tool definition for best practices: naming conventions, description quality, schema completeness, required fields consistency, description length. Returns actionable warnings.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['tool_definition'], 'properties': {'tool_definition': {'type': 'object', 'description': 'MCP tool definition object with name, description, inputSchema', 'additionalProperties': True}}}

Schéma de sortie

{'type': 'object', 'properties': {'grade': {}, 'errors': {}, 'warnings': {}, 'error_count': {'type': 'number'}, 'quality_score': {}, 'warning_count': {'type': 'number'}}, 'additionalProperties': True}

mcp_server_evaluate

Run a full compliance evaluation against a live MCP server URL. Tests: server reachability (ping), manifest discovery (GET /mcp), schema quality (snake_case names, descriptions, inputSchema), JSON-RPC 2.0 test call, and P50/P95 latency. Returns a PASS/FIX/BLOCK verdict with a 0-100 score and per-check details.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['url'], 'properties': {'url': {'type': 'string', 'description': 'Base URL of the MCP server (e.g. https://www.ia-qa.com or http://localhost:3001)'}, 'test_tool_name': {'type': 'string', 'description': 'Specific tool name to use in the JSON-RPC test call (defaults to the first tool in the manifest)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'url': {'type': 'string'}, 'score': {'type': 'number'}, 'checks': {'type': 'object'}, 'latency': {'type': 'object'}, 'verdict': {'type': 'string'}}, 'additionalProperties': True}

mcp_server_health_check

Generate a health check report for an MCP server's tool manifest. Validates tool definitions, schema quality, naming conventions, and documentation completeness. Paste the server manifest JSON to audit.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['manifest'], 'properties': {'strict': {'type': 'boolean', 'description': 'Enable strict mode: also check for optional best practices (examples, default values, descriptions > 20 chars)'}, 'manifest': {'type': 'string', 'description': 'MCP server manifest JSON (the response from GET /mcp or tools/list)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'stats': {'type': 'object'}, 'total': {'type': 'number'}, 'checks': {'type': 'array'}, 'failed': {'type': 'number'}, 'passed': {'type': 'number'}, 'verdict': {'type': 'string'}, 'toolIssues': {}}, 'additionalProperties': True}

merge_json

Deep merge two JSON objects. Supports three array strategies: replace (default), concat, or unique (dedup concat). Nested objects are recursively merged — override takes precedence for primitives.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['base', 'override'], 'properties': {'base': {'type': ['string', 'object', 'array'], 'description': 'The base JSON object (merged into) — a JSON string, or the object itself.'}, 'override': {'type': ['string', 'object', 'array'], 'description': 'The override JSON object (takes precedence) — a JSON string, or the object itself.'}, 'array_strategy': {'enum': ['replace', 'concat', 'unique'], 'type': 'string', 'description': 'Array merge strategy: replace (default), concat, or unique'}}}

Schéma de sortie

{'type': 'object', 'properties': {'merged': {}, 'new_keys': {'type': 'array'}, 'total_keys': {'type': 'number'}, 'overridden_keys': {'type': 'array'}}, 'additionalProperties': True}

metamorphic_check

Reference-free stability primitive: instead of comparing an answer to a ground truth, it checks that an assistant's answer stays INVARIANT when the QUESTION is transformed (typo, casing, paraphrase, reordering, translation). Catches the failure class no reference answer can expose — an assistant that handles one phrasing well and a trivial variant of it badly. You bring the outputs (no model is called), so it is deterministic and free in tfidf mode. Relations: case (θ .95), typo (.90), paraphrase (.80), reorder (.80), translation (.75, embeddings only), specialization (.60, ADVISORY — directional, never gated). Returns PASS / FAIL / INVALID, where INVALID means the BASE answer was a refusal or too short so invariance was never measurable — an assistant that refuses every variant would otherwise score a perfect 1.0. Use run_semantic_tests alongside it: invariance without a correctness floor is a green light for a broken assistant.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['base', 'variants'], 'properties': {'base': {'type': 'object', 'properties': {'output': {'type': 'string', 'description': 'Required — the answer your system produced for the original question.'}, 'question': {'type': 'string', 'description': 'Optional — the original question, echoed back in the output for readability.'}}, 'description': 'The reference run: the original question and the answer your system produced for it.'}, 'mode': {'enum': ['tfidf', 'embeddings'], 'type': 'string', 'description': 'tfidf (default): free, lexical, deterministic — but a genuine paraphrase rarely reaches 0.80, so gate on case/typo and treat paraphrase as a trend. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key. translation requires this mode.'}, 'api_key': {'type': 'string', 'description': 'OpenAI API key — required only when mode is embeddings.'}, 'variants': {'type': 'array', 'items': {'type': 'object', 'required': ['relation', 'output'], 'properties': {'id': {'type': 'string', 'description': 'Optional identifier (default: variant_<n>).'}, 'output': {'type': 'string', 'description': 'Required — the answer your system produced for the transformed question.'}, 'question': {'type': 'string', 'description': 'Optional — the transformed question, echoed back for readability.'}, 'relation': {'enum': ['case', 'typo', 'paraphrase', 'reorder', 'translation', 'specialization'], 'type': 'string', 'description': 'Transformation applied to the QUESTION. Decides the metric and the default threshold.'}}}, 'maxItems': 20, 'description': 'Answers produced for transformed versions of the same question, each tagged with the relation that was applied.'}, 'thresholds': {'type': 'object', 'properties': {'case': {'type': 'number', 'maximum': 1, 'minimum': 0}, 'typo': {'type': 'number', 'maximum': 1, 'minimum': 0}, 'reorder': {'type': 'number', 'maximum': 1, 'minimum': 0}, 'paraphrase': {'type': 'number', 'maximum': 1, 'minimum': 0}, 'translation': {'type': 'number', 'maximum': 1, 'minimum': 0}, 'specialization': {'type': 'number', 'maximum': 1, 'minimum': 0}}, 'description': 'Per-relation threshold overrides. Calibrate on your own corpus before gating — the defaults are starting points, not measurements.'}, 'require_all': {'type': 'boolean', 'description': 'If true (default), every gated variant must pass. KEEP THE DEFAULT for any run you gate on. Setting it false is not a tolerance dial but an off switch: relations have asymmetric pass rates (a typo variant usually scores ~1.0 because the answer really is identical), so one trivial row is enough to hold the whole run at PASS while a paraphrase fails. When that happens the result carries an explicit warning naming the failed rows.'}, 'baseline_guard': {'type': 'object', 'properties': {'min_length': {'type': 'number', 'description': 'Minimum base answer length in chars (default 40).'}, 'must_not_match': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Refusal patterns, matched case-insensitively in the FIRST 200 CHARS of the base answer (refusals lead; matching anywhere would flag a long correct answer that merely mentions one). Replaces the FR+EN default list, never merges — pass [] to disable.'}}, 'description': 'Correctness floor applied to the BASE answer before anything is scored. Failing it returns INVALID, not FAIL.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'mode': {'type': 'string'}, 'summary': {'type': 'object'}, 'verdict': {'type': 'string'}, 'weakest': {'type': 'object'}, 'baseline': {'type': 'object'}, 'variants': {'type': 'array', 'items': {'type': 'object'}}, 'warnings': {'type': 'array', 'items': {'type': 'string'}}, 'thresholds': {'type': 'object'}}, 'additionalProperties': True}

minify_js

Minify a JavaScript snippet, function, class, or module up to 50 KB using Terser. Returns minified code and byte savings. Use when embedding scripts in HTML templates, report payloads, or injecting inline code programmatically.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['code'], 'properties': {'code': {'type': 'string', 'description': 'JavaScript code to minify (max 50kb)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'minified': {}}, 'additionalProperties': True}

mock_from_schema

Generate realistic mock data from a JSON Schema. Supports all common types (string, number, integer, boolean, array, object, null), format hints (email, date, date-time, uri, uuid), enum, const, and nested schemas. Perfect for testing MCP tools with realistic data.

Lecture seule

Schéma d’entrée

{'type': 'object', 'required': ['schema'], 'properties': {'seed': {'type': 'string', 'description': 'Optional seed string for deterministic output (uses first char codes)'}, 'count': {'type': 'number', 'description': 'Number of mock objects to generate (default: 1, max: 20)'}, 'schema': {'type': ['string', 'object', 'array'], 'description': 'The JSON Schema to generate from — a JSON string, or the schema object itself.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'count': {'type': 'number'}, 'results': {}}, 'additionalProperties': True}

model_info

Get detailed specs for an AI model: context window, pricing per 1K tokens, knowledge cutoff, provider, multimodal support, reasoning capabilities, and feature list. Covers 30+ models from OpenAI, Anthropic, Google, DeepSeek, Meta, Mistral, Cohere, xAI.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['model'], 'properties': {'model': {'type': 'string', 'description': 'Model name (e.g. "gpt-4o", "claude-3.5-sonnet", "gemini-2.5-pro")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'model': {}, 'pricing_per_1k': {'type': 'object'}}, 'additionalProperties': True}

multimodal_eval_guide

Unified tool for multimodal AI evaluation: set action=guide for reference thresholds/interpretation (CLIP, FID, VQA), or set action=clip_score / fid_score / vqa_accuracy / pipeline to compute real metrics via HuggingFace Inference API and VLM BYOK calls. One tool for both reference and computation.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'fid': {'type': 'object', 'description': '[pipeline] {real_images, generated_images} for FID.', 'additionalProperties': True}, 'vqa': {'type': 'object', 'description': '[pipeline] VQA config object (same inputs as vqa_accuracy).', 'additionalProperties': True}, 'clip': {'type': 'object', 'description': '[pipeline] {image_url, text} for CLIP.', 'additionalProperties': True}, 'text': {'type': 'string', 'description': '[clip_score only] Text description to compare against the image.'}, 'model': {'type': 'string', 'description': '[vqa_accuracy] VLM model ID (default: gpt-4o).'}, 'score': {'type': 'number', 'description': '[guide only] Optional score value to interpret.'}, 'action': {'enum': ['guide', 'clip_score', 'fid_score', 'vqa_accuracy', 'pipeline'], 'type': 'string', 'description': 'guide (default) = reference thresholds/interpretation. clip_score/fid_score/vqa_accuracy = compute that metric. pipeline = run all three.'}, 'metric': {'enum': ['clip_score', 'fid', 'vqa_accuracy', 'all'], 'type': 'string', 'description': '[guide only] Metric to explain.'}, 'api_key': {'type': 'string', 'description': '[vqa_accuracy] Your API key for the provider (BYOK).'}, 'image_url': {'type': 'string', 'description': '[clip_score/vqa_accuracy] Public URL of the image.'}, 'test_cases': {'type': 'array', 'items': {'type': 'object', 'properties': {'question': {'type': 'string'}, 'accepted_answers': {'type': 'array', 'items': {'type': 'string'}}}}, 'description': '[vqa_accuracy] Array of {question, accepted_answers} objects.'}, 'real_images': {'type': 'array', 'items': {'type': 'string'}, 'description': '[fid_score] Array of real image URLs.'}, 'image_base64': {'type': 'string', 'description': '[clip_score/vqa_accuracy] Base64-encoded image data.'}, 'system_prompt': {'type': 'string', 'description': '[vqa_accuracy] Optional system prompt.'}, 'image_mime_type': {'type': 'string', 'description': '[clip_score/vqa_accuracy] MIME type for base64 image.'}, 'generated_images': {'type': 'array', 'items': {'type': 'string'}, 'description': '[fid_score] Array of generated image URLs.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'errors': {}, 'metrics': {}, 'results': {}, 'web_tool': {'type': 'string'}, 'best_practices': {'type': 'array'}, 'comparison_table': {'type': 'array'}, 'score_interpretation': {}}, 'additionalProperties': True}

needle_haystack_generate

Generate a "needle in a haystack" test: embeds a target fact into a large block of filler text at a specified position. Use this to test LLM context window retrieval accuracy. Returns the full haystack, the question to ask, and metadata. No API key needed.

Lecture seule

Schéma d’entrée

{'type': 'object', 'required': ['needle', 'question'], 'properties': {'needle': {'type': 'string', 'description': 'The fact to hide (e.g. "The secret code is ALPHA-42")'}, 'tokens': {'type': 'integer', 'default': 5000, 'description': 'Target haystack size in tokens (default: 5000, max: 100000)'}, 'position': {'enum': ['start', 'middle', 'end', 'random'], 'type': 'string', 'default': 'middle', 'description': 'Where to insert the needle: "start", "middle", "end", "random" (default: "middle")'}, 'question': {'type': 'string', 'description': 'The question to ask the LLM (e.g. "What is the secret code?")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'needle': {}, 'haystack': {}, 'position': {}, 'question': {}, 'insert_block': {}, 'total_blocks': {'type': 'number'}, 'estimated_tokens': {}}, 'additionalProperties': True}

normalize_vector

L2-normalize a float vector (produce a unit vector with norm=1). Required by many vector DBs (Pinecone, Qdrant cosine). Supports batch normalization of up to 1000 vectors.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'batch': {'type': 'array', 'items': {'type': 'array', 'items': {'type': 'number'}}, 'description': 'Batch of vectors to normalize (overrides vector)'}, 'vector': {'type': 'array', 'items': {'type': 'number'}, 'description': 'Single vector to normalize'}}}

Schéma de sortie

{'type': 'object', 'properties': {'mode': {'type': 'string'}, 'norm': {'type': 'number'}, 'count': {'type': 'number'}, 'index': {}, 'vector': {}, 'results': {}, 'dimension': {'type': 'number'}, 'norm_after': {'type': 'number'}, 'normalized': {}, 'norm_before': {'type': 'number'}}, 'additionalProperties': True}

normalize_whitespace

Normalize whitespace: trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces. Useful for cleaning code, configs, and text before processing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to normalize'}, 'trim_file': {'type': 'boolean', 'description': 'Trim leading/trailing blank lines (default: true)'}, 'trim_lines': {'type': 'boolean', 'description': 'Trim trailing whitespace from each line (default: true)'}, 'line_ending': {'type': 'string', 'description': '"lf" (default), "crlf", or "cr"'}, 'tab_to_spaces': {'type': 'number', 'description': 'Convert tabs to N spaces (omit to keep tabs)'}, 'collapse_blanks': {'type': 'boolean', 'description': 'Collapse runs of blank lines down to max_blank_lines (default: true)'}, 'max_blank_lines': {'type': 'number', 'description': 'Blank lines to keep when collapsing, 0-10 (default: 2)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'result': {}, 'line_ending': {}, 'original_length': {'type': 'number'}, 'normalized_length': {'type': 'number'}}, 'additionalProperties': True}

number_base_convert

Convert numbers between bases: decimal, binary, octal, hexadecimal, or any base 2–36. Auto-detects 0x, 0b, 0o prefixes.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Number to convert (e.g., "255", "0xFF", "0b1010", "0o77")'}, 'to_base': {'type': 'number', 'description': 'Target base 2–36 (omit to get all common bases)'}, 'from_base': {'type': 'number', 'description': 'Source base 2–36 (auto-detects prefix if omitted)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'octal': {'type': 'string'}, 'binary': {'type': 'string'}, 'result': {'type': 'string'}, 'decimal': {}, 'to_base': {}, 'from_base': {}, 'hexadecimal': {'type': 'string'}}, 'additionalProperties': True}

openapi_validate

Validate the structure of an OpenAPI 3.x specification (JSON or YAML). Checks required top-level fields (openapi, info.title, info.version, paths), validates each operation (responses, operationId uniqueness), detects undeclared $ref components, and flags missing 2xx responses. Returns a PASS/FAIL verdict, a 0–100 compliance score, and a list of errors and warnings with JSON-pointer locations. Use before publishing an API spec or generating SDK code.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': ['string', 'object', 'array'], 'description': 'The OpenAPI 3.x spec — a JSON string, a YAML string, or the already-parsed spec object.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'score': {'type': 'number'}, 'stats': {'type': 'object'}, 'errors': {}, 'verdict': {}, 'warnings': {}}, 'additionalProperties': True}

optimize_prompt_tokens

Compress an LLM prompt by removing filler words, verbose phrases, duplicate sentences, and unnecessary whitespace. Returns optimized text with token savings breakdown. 100% deterministic, no API key needed.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['text'], 'properties': {'text': {'type': 'string', 'description': 'The prompt text to optimize'}, 'options': {'type': 'object', 'properties': {'fillers': {'type': 'boolean', 'default': True}, 'duplicates': {'type': 'boolean', 'default': True}, 'whitespace': {'type': 'boolean', 'default': True}, 'instructions': {'type': 'boolean', 'default': True}}, 'description': 'Toggle optimization steps (all true by default)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'steps': {}, 'optimized': {}, 'tokens_after': {}, 'tokens_saved': {}, 'percent_saved': {'type': 'string'}, 'tokens_before': {}}, 'additionalProperties': True}

parse_csv

Parse a CSV string into a JSON array of objects (or raw arrays). Full RFC 4180: quoted fields may contain the delimiter, embedded newlines (the Excel/Sheets multi-line cell), and doubled quotes. Custom delimiters supported. An unterminated quote is rejected with its position rather than parsed into corrupted rows. Use when processing spreadsheet exports, data imports, or structured text pipelines where the source is CSV. Supports up to 200 KB.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'CSV content to parse'}, 'header': {'type': 'boolean', 'description': 'Treat the first row as headers (default: true)'}, 'delimiter': {'type': 'string', 'description': 'Field delimiter character (default: ",")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'rows': {'type': 'array'}, 'columns': {'type': 'number'}, 'headers': {'type': 'array'}, 'row_count': {'type': 'number'}}, 'additionalProperties': True}

parse_http_headers

Parse a raw HTTP headers block into a structured JSON object. Detects multi-value headers, masks Authorization values, and optionally audits for missing security headers (HSTS, CSP, X-Frame-Options, etc.).

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['headers'], 'properties': {'headers': {'type': 'string', 'description': 'Raw HTTP headers (one "Name: Value" per line)'}, 'analyze_security': {'type': 'boolean', 'description': 'Audit for missing security headers (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'parsed': {'type': 'object'}, 'security': {'type': 'object'}, 'header_count': {'type': 'number'}}, 'additionalProperties': True}

post_jira_comment

Post the output of jira_to_test_suite as a formatted comment on the source Jira ticket. Converts Gherkin, E2E steps, API tests, and ambiguities into Atlassian Document Format (ADF). STATEFUL — creates a comment on the issue.

Schéma d’entrée

{'type': 'object', 'required': ['issue_key', 'jira_base_url', 'jira_email', 'jira_token', 'test_suite'], 'properties': {'issue_key': {'type': 'string', 'description': 'Jira issue key, e.g. "PROJ-123"'}, 'jira_email': {'type': 'string', 'description': 'Atlassian account email'}, 'jira_token': {'type': 'string', 'description': 'Atlassian API token'}, 'test_suite': {'type': 'object', 'description': 'The test_suite object from jira_to_test_suite result', 'additionalProperties': True}, 'jira_base_url': {'type': 'string', 'description': 'Atlassian base URL'}}}

Schéma de sortie

{'type': 'object', 'properties': {'success': {'type': 'boolean'}, 'comment_id': {'type': 'string'}, 'comment_url': {'type': 'string'}}, 'additionalProperties': True}

pr_gatekeeper

Compound quality gate for pull requests. Runs three sequential checks: (1) secret detection — scans diff for API keys, tokens, passwords matching 16 regex patterns; (2) bug analysis — heuristic scan for eval(), innerHTML, empty catch, console.log, TODO/FIXME; (3) commit message linting against Conventional Commits spec. Returns gate verdict (PASS/WARN/BLOCK), blockers, and actionable warnings. Use before merging any code change.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['diff', 'commit_message'], 'properties': {'diff': {'type': 'string', 'description': 'Unified git diff (output of `git diff HEAD`)'}, 'context': {'type': 'string', 'description': 'Optional: PR title or description for richer bug analysis'}, 'commit_message': {'type': 'string', 'description': 'The commit message to lint (e.g. "feat(auth): add OAuth2 login")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'flags': {'type': 'array', 'items': {'type': 'string'}}, 'score': {'type': 'number'}, 'checks': {'type': 'object'}, 'verdict': {'type': 'string'}}, 'additionalProperties': True}

prompt_injection_scan

Scan user input or prompts for common prompt injection patterns. Detects system prompt overrides, jailbreak attempts, role manipulation, encoding tricks, delimiter attacks (chat-template tags `<|im_start|>`/`[INST]`/`<<SYS>>` AND fake role headers imitating markdown or chat separators: "### System:", "--- SYSTEM ---", "---BEGIN SYSTEM OVERRIDE---", "--- SYSTEM:"), template/interpolation injection ({{...}}, ${...}), and context-exfiltration attempts ("repeat everything above"). A match inside quoted or fenced text (documentation citing a payload) is reported one severity level lower and marked `quoted` — never suppressed, since an LLM reading the document as data can still follow a quoted instruction. A quote preceded by a live imperative ("output the following: ...") keeps its full severity.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'The user input or prompt to scan for injection patterns'}, 'sensitivity': {'enum': ['low', 'medium', 'high'], 'type': 'string', 'description': 'Detection sensitivity (default: medium)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'notes': {}, 'detections': {}, 'risk_level': {}, 'sensitivity': {}, 'input_length': {'type': 'number'}, 'detections_count': {'type': 'number'}, 'quoted_detections': {'type': 'number'}, 'injection_detected': {'type': 'boolean'}}, 'additionalProperties': True}

prompt_template_fill

Fill a prompt template with variables. Supports {{variable}} syntax and {{#if key}}...{{/if}} conditional blocks. Returns the filled prompt and lists unfilled variables.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['template'], 'properties': {'strict': {'type': 'boolean', 'description': 'Throw error if any variable is not provided (default: false)'}, 'template': {'type': 'string', 'description': 'Prompt template with {{variable}} placeholders'}, 'variables': {'type': 'object', 'description': 'Key-value pairs to fill (e.g. {"name":"Alice","role":"engineer"})', 'additionalProperties': True}}}

Schéma de sortie

{'type': 'object', 'properties': {'result': {}, 'total_vars': {}, 'filled_variables': {}, 'unfilled_variables': {}}, 'additionalProperties': True}

prompt_test_suite

Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria. Returns a test plan with scored rubric — use this as input for manual or automated LLM evaluation.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['system_prompt', 'user_prompt'], 'properties': {'max_tokens': {'type': 'number', 'description': 'Max token budget for the test'}, 'temperature': {'type': 'number', 'description': 'Temperature to use'}, 'user_prompt': {'type': 'string', 'description': 'The user prompt to send'}, 'check_safety': {'type': 'boolean', 'description': 'Include safety/PII checks in the rubric'}, 'must_include': {'type': 'string', 'description': 'Required content (comma-separated)'}, 'system_prompt': {'type': 'string', 'description': 'The system prompt under test'}, 'expected_format': {'enum': ['json', 'markdown', 'code', 'plain', 'any'], 'type': 'string', 'description': 'Expected output format'}, 'must_not_include': {'type': 'string', 'description': 'Forbidden content (comma-separated)'}, 'expected_behavior': {'type': 'string', 'description': 'Description of what the LLM should do (free text)'}, 'adversarial_prompts': {'type': 'boolean', 'description': 'Auto-generate adversarial test variants (jailbreak, injection, edge cases)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'rubric': {}, 'categories': {'type': 'array'}, 'total_tests': {'type': 'number'}, 'instructions': {'type': 'string'}, 'test_suite_name': {'type': 'string'}}, 'additionalProperties': True}

rag_relevance_rank

Rank an array of text chunks by relevance to a query using TF-IDF scoring. Simulates retrieval ranking for RAG testing without needing embeddings or an API.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['query', 'chunks'], 'properties': {'query': {'type': 'string', 'description': 'The user query'}, 'top_k': {'type': 'number', 'description': 'Return top K results (default: all)'}, 'chunks': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Array of text chunks to rank'}}}

Schéma de sortie

{'type': 'object', 'properties': {'rank': {'type': 'number'}, 'index': {}, 'query': {}, 'score': {'type': 'string'}, 'results': {}, 'returned': {'type': 'number'}, 'total_chunks': {'type': 'number'}, 'chunk_preview': {}, 'keyword_overlap': {}}, 'additionalProperties': True}

rate_tool

Give honest usage feedback on an IA-QA MCP tool. Provide a score (1-5) and a comment. Rate low (1-2) if the tool was wrong, irrelevant, or a poor fit; rate high (4-5) only if it genuinely solved your need. Ratings are aggregated on a public dashboard at /devtools/mcp-ratings. Skip rating routine successes — we want signal, not praise. Example: rate_tool({ tool_name: "format_json", score: 2, comment: "Tried to pretty-print a JSON5 file, it rejected trailing commas — not usable for my case." })

Schéma d’entrée

{'type': 'object', 'required': ['tool_name', 'score'], 'properties': {'score': {'type': 'number', 'maximum': 5, 'minimum': 1, 'description': 'Rating from 1 (poor) to 5 (excellent)'}, 'comment': {'type': 'string', 'description': 'Strongly encouraged — explain what you were trying to do and whether the tool got you there. Be specific about what was missing, wrong, or a poor fit. This is the most valuable part of the rating. Up to 2000 chars are stored; go over and the response says so (truncated: true) — send the remainder as a second call rather than assuming it landed.'}, 'tool_name': {'type': 'string', 'description': 'Name of the MCP tool to rate (e.g. "format_json", "shield_analyze")'}}}

Schéma de sortie

{'type': 'object', 'properties': {'ok': {'type': 'boolean'}, 'score': {}, 'comment': {}, 'message': {'type': 'string'}, 'rated_at': {'type': 'string'}, 'tool_name': {}, 'truncated': {'type': 'boolean'}, 'stored_chars': {'type': 'number'}}, 'additionalProperties': True}

redact_pii

Automatically detect and redact Personally Identifiable Information (PII) from text. Replaces emails, phone numbers, SSNs, credit cards, IP addresses, and JWT tokens with [REDACTED_TYPE] placeholders. Safe to use before logging or sending to an LLM.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to redact PII from'}, 'types': {'type': 'string', 'description': 'Comma-separated types to redact (default: all). Options: email, phone, ssn, credit_card, ip_address, jwt'}, 'marker': {'type': 'string', 'description': 'Custom replacement marker (default: "REDACTED"). Result: [REDACTED_EMAIL]'}}}

Schéma de sortie

{'type': 'object', 'properties': {'clean': {'type': 'boolean'}, 'pii_found': {}, 'replacements': {}, 'redacted_text': {}, 'total_redactions': {}}, 'additionalProperties': True}

regex_test

Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups. Use for validating user inputs, extracting structured data from text, or debugging regex patterns. Supports flags g, i, m, s, u, y. The match runs in an isolated thread with a 500 ms budget: a pattern that blows up (catastrophic backtracking, e.g. "(a+)+$") comes back as redos_detected:true — a real ReDoS verdict on your pattern — instead of hanging.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['pattern', 'input'], 'properties': {'flags': {'type': 'string', 'description': 'Regex flags: g (global), i (case-insensitive), m (multiline), s (dotAll) — default: ""'}, 'input': {'type': 'string', 'description': 'The string to test against (max 50 KB)'}, 'pattern': {'type': 'string', 'description': 'Regular expression pattern (without delimiters)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'note': {}, 'flags': {}, 'matched': {}, 'matches': {}, 'pattern': {}, 'verdict': {}, 'elapsed_ms': {'type': 'number'}, 'match_count': {'type': 'number'}, 'redos_detected': {'type': 'boolean'}}, 'additionalProperties': True}

rerank_evaluate

Evaluate RAG retrieval quality: rank passages against a query and compute Precision@k / Recall@k plus a PASS/FAIL CI verdict from ground-truth relevance labels. Three modes, all keyless except the last. (1) BYO scores — give each passage the `score` your own reranker produced (Cohere, Jina, a self-hosted NIM, a cross-encoder): deterministic, offline, and it evaluates YOUR reranker rather than someone else's. This is the mode to gate CI on. (2) Default, no scores and no key — ranks with local BM25, a lexical keyword baseline: it answers "does a keyword floor already surface my relevant passages?", never "is my neural reranker good". (3) Live NVIDIA reranker — supply `api_key` for an NVIDIA account that still has reranking entitlement; NVIDIA retired its hosted reranking endpoints on 2026-05-18, so this one is for accounts that were grandfathered in.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['query', 'passages'], 'properties': {'query': {'type': 'string', 'description': 'The search query or question to rank against'}, 'top_k': {'type': 'integer', 'maximum': 10, 'minimum': 1, 'description': 'k for Precision@k evaluation (default 3)'}, 'api_key': {'type': 'string', 'description': 'Your NVIDIA API key (BYOK), used only when no passage carries a score. Transits RAM for the single call, never stored.'}, 'passages': {'type': 'array', 'items': {'type': 'object', 'required': ['text'], 'properties': {'id': {'type': 'string'}, 'text': {'type': 'string'}, 'score': {'type': 'number', 'description': 'Relevance score from YOUR reranker. Present on every passage → ranking is done from these (offline, no key). Higher = more relevant. Score all passages or none.'}, 'relevant': {'type': 'boolean', 'description': 'Ground truth: is this passage relevant?'}}}, 'description': 'Array of passage objects to rank (min 2, max 20)'}, 'threshold': {'type': 'number', 'maximum': 1, 'minimum': 0, 'description': 'Minimum Precision@k to PASS (0-1, default 0.5)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'k': {'type': 'number'}, 'mode': {'type': 'string'}, 'model': {'type': 'string'}, 'ranked': {'type': 'array', 'items': {'type': 'object'}}, 'verdict': {'type': 'string'}, 'threshold': {'type': 'number'}, 'latency_ms': {'type': 'number'}, 'recall_at_k': {'type': 'number'}, 'precision_at_k': {'type': 'number'}, 'total_passages': {'type': 'number'}, 'has_ground_truth': {'type': 'boolean'}}, 'additionalProperties': True}

response_quality_score

Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length. Returns a 0-100 score over the criteria actually measured — or total_score null with the reason when none is given, because relevance and correctness cannot be read off word overlap (a correct one-word answer shares no words with its question). For meaning against a reference answer, use run_semantic_tests. Also returns unscored signals: question-term overlap, average sentence length, markdown.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['question', 'response'], 'properties': {'question': {'type': 'string', 'description': 'The original question/prompt'}, 'response': {'type': 'string', 'description': 'The LLM response to score'}, 'max_length': {'type': 'number', 'description': 'Maximum character length. Scored as a proportional penalty beyond it.'}, 'expected_keywords': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Keywords a correct answer must contain (case-insensitive). Scored as coverage.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'grade': {}, 'stats': {'type': 'object'}, 'reason': {'type': 'string'}, 'signals': {'type': 'object'}, 'breakdown': {'type': 'object'}, 'max_score': {'type': 'number'}, 'total_score': {}, 'not_measured': {'type': 'array'}}, 'additionalProperties': True}

run_eval_contract

Parse a .ia-eval.yaml LLM test suite, call the specified LLM model for each scenario, run all configured scorers, and return a structured JSON report with per-scenario Pass/Fail verdicts and a Markdown summary. Use list_local_tests to discover available test files.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'properties': {'api_keys': {'type': 'object', 'properties': {'hf': {'type': 'string'}, 'groq': {'type': 'string'}, 'google': {'type': 'string'}, 'openai': {'type': 'string'}, 'anthropic': {'type': 'string'}}, 'description': 'API keys to use for LLM generation (all optional — falls back to server env vars)'}, 'overrides': {'type': 'object', 'properties': {'model': {'type': 'string'}, 'provider': {'type': 'string'}, 'temperature': {'type': 'number'}, 'system_prompt': {'type': 'string'}}, 'description': 'Override contract defaults'}, 'contract_path': {'type': 'string', 'description': 'Absolute or relative path to a .ia-eval.yaml file (required unless inline_contract is provided)'}, 'inline_contract': {'type': 'object', 'description': 'Raw contract object (alternative to contract_path). Must contain top-level "metadata" ({name, version, model?, provider?}), "expectations" ({min_score?}), and "scenarios" ([{id, input, ground_truth?}]) — scenarios alone are rejected. Use generate_eval_yaml to scaffold one.', 'additionalProperties': True}}}

Schéma de sortie

{'type': 'object', 'properties': {'summary': {'type': 'object'}, 'metadata': {'type': 'object'}, 'warnings': {}, 'contract_path': {}, 'scenario_results': {}}, 'additionalProperties': True}

run_pr_gate_pipeline

Review triage for a pull request. Takes a unified git diff (`git diff HEAD`) and returns: diff-lint findings with the lines that produced them, regression impact areas, a risk score 0–100 with the factors that built it (churn, files touched, sensitive paths, whether any test file changed, lint severities, impacted risk areas), generated test cases, and a PASS / CONDITIONAL / BLOCK recommendation. Advisory: the score measures properties of the diff, not the correctness of the change — it does not read the code semantically and does not replace a reviewer or a static analyser. See notAnalysed in the response.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['git_diff'], 'properties': {'context': {'type': 'string', 'description': 'Optional PR title or description for richer analysis'}, 'git_diff': {'type': 'string', 'description': 'Unified git diff (output of `git diff HEAD` or copied from GitHub diff view)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'sla': {}, 'high': {}, 'topBugs': {'type': 'array'}, 'critical': {}, 'bugsFound': {'type': 'number'}, 'riskLevel': {}, 'riskScore': {}, 'disclaimer': {'type': 'string'}, 'impactAreas': {'type': 'array'}, 'inputFormat': {'type': 'string'}, 'notAnalysed': {'type': 'array'}, 'riskFactors': {'type': 'array'}, 'changedFiles': {}, 'severityLevel': {}, 'testCasesGenerated': {}, 'mergeRecommendation': {}}, 'additionalProperties': True}

run_semantic_tests

Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L. Two modes: tfidf (default, free, no API key) or embeddings (OpenAI text-embedding-3-small, BYOK, true semantic similarity). Returns per-case PASS/FAIL verdicts and an overall verdict. CI-ready: pipe the JSON verdict field to gate a build.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['cases'], 'properties': {'mode': {'enum': ['tfidf', 'embeddings'], 'type': 'string', 'description': 'tfidf (default): fast, free, lexical. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key.'}, 'cases': {'type': 'array', 'items': {'type': 'object', 'required': ['actual', 'expected'], 'properties': {'id': {'type': 'string', 'description': 'Optional identifier for this case.'}, 'actual': {'type': 'string', 'description': 'The text produced by your LLM/system.'}, 'expected': {'type': 'string', 'description': 'The reference/ground-truth text.'}}}, 'maxItems': 50, 'description': 'Array of (actual, expected) pairs to evaluate.'}, 'api_key': {'type': 'string', 'description': 'OpenAI API key — required only when mode is embeddings.'}, 'thresholds': {'type': 'object', 'properties': {'cosine': {'type': 'number', 'maximum': 1, 'minimum': 0, 'description': 'Minimum cosine similarity to pass (default: 0.75).'}, 'rouge_l': {'type': 'number', 'maximum': 1, 'minimum': 0, 'description': 'Minimum ROUGE-L F1 to pass (default: 0.5).'}}, 'description': 'Pass/fail thresholds (defaults: cosine 0.75, rouge_l 0.5).'}, 'require_all': {'type': 'boolean', 'description': 'If true (default), all cases must pass for overall PASS. If false, at least one case passing returns PASS.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'mode': {'type': 'string'}, 'total': {'type': 'number'}, 'failed': {'type': 'number'}, 'passed': {'type': 'number'}, 'results': {'type': 'array', 'items': {'type': 'object'}}, 'verdict': {'type': 'string'}}, 'additionalProperties': True}

run_vlm_test_suite

Run a test suite against a Vision-Language Model (VLM) — send an image (URL or base64) + N test cases (each with a question + assertion) to GPT-4o, Claude 3.5, or Gemini. Returns per-case PASS/FAIL verdicts, a pass rate, an overall PASS/WARNING/FAIL verdict (customizable threshold), and latency stats. Assertion types: contains, not_contains, json_format, min_length, max_length, semantic_contains (TF-IDF cosine similarity ≥ 0.4). BYOK: requires your own API key for the target provider.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['test_cases', 'model', 'api_key'], 'properties': {'model': {'enum': ['gpt-4o', 'gpt-4o-mini', 'claude-3-5-sonnet-20241022', 'claude-3-5-haiku-20241022', 'gemini-1.5-flash', 'gemini-2.0-flash'], 'type': 'string', 'description': 'VLM model to use.'}, 'api_key': {'type': 'string', 'description': 'API key for the model provider (OpenAI sk-, Anthropic sk-ant-, or Google AIzaSy...).'}, 'image_url': {'type': 'string', 'description': 'Public URL of the image to evaluate (required unless image_base64 is provided).'}, 'threshold': {'type': 'number', 'description': 'Pass rate threshold for overall verdict (default: 80, 0–100).'}, 'test_cases': {'type': 'array', 'items': {'type': 'object', 'required': ['question'], 'properties': {'id': {'type': 'string', 'description': 'Optional identifier for this case.'}, 'question': {'type': 'string', 'description': 'Question to ask the VLM about the image.'}, 'assertion_type': {'enum': ['contains', 'not_contains', 'json_format', 'min_length', 'max_length', 'semantic_contains'], 'type': 'string', 'description': 'Assertion to run on the VLM response. semantic_contains uses TF-IDF cosine similarity ≥ 0.4.'}, 'assertion_value': {'type': 'string', 'description': 'Expected value for the assertion (not needed for json_format).'}}}, 'maxItems': 10, 'description': 'Array of test cases to run.'}, 'image_base64': {'type': 'string', 'description': 'Base64-encoded image data (required unless image_url is provided).'}, 'system_prompt': {'type': 'string', 'description': 'Optional system prompt sent to the VLM.'}, 'image_mime_type': {'type': 'string', 'description': 'MIME type of the image if using image_base64 (default: image/jpeg).'}}}

Schéma de sortie

{'type': 'object', 'properties': {'model': {'type': 'string'}, 'total': {'type': 'number'}, 'failed': {'type': 'number'}, 'passed': {'type': 'number'}, 'results': {'type': 'array', 'items': {'type': 'object'}}, 'verdict': {'type': 'string'}}, 'additionalProperties': True}

run_vlm_test_suite_batch

Compare multiple VLMs on the same test suite in parallel — send an image (URL or base64) + N test cases to all models simultaneously. Returns per-model PASS/FAIL verdicts, pass rates, latency stats, and a comparison table. Assertion types: contains, not_contains, json_format, min_length, max_length, semantic_contains. BYOK: requires API keys for each provider.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['test_cases', 'models', 'api_keys'], 'properties': {'models': {'type': 'array', 'items': {'enum': ['gpt-4o', 'gpt-4o-mini', 'claude-3-5-sonnet-20241022', 'claude-3-5-haiku-20241022', 'gemini-1.5-flash', 'gemini-2.0-flash'], 'type': 'string'}, 'maxItems': 6, 'minItems': 1, 'description': 'Array of model IDs to compare (runs in parallel).'}, 'api_keys': {'type': 'object', 'description': 'Map of model ID → API key. Example: { "gpt-4o": "sk-...", "claude-3-5-sonnet-20241022": "sk-ant-..." }', 'additionalProperties': {'type': 'string'}}, 'image_url': {'type': 'string', 'description': 'Public URL of the image to evaluate (required unless image_base64 is provided).'}, 'threshold': {'type': 'number', 'description': 'Pass rate threshold for overall verdict (default: 80, 0–100).'}, 'test_cases': {'type': 'array', 'items': {'type': 'object', 'required': ['question'], 'properties': {'id': {'type': 'string', 'description': 'Optional identifier for this case.'}, 'question': {'type': 'string', 'description': 'Question to ask the VLM about the image.'}, 'assertion_type': {'enum': ['contains', 'not_contains', 'json_format', 'min_length', 'max_length', 'semantic_contains'], 'type': 'string', 'description': 'Assertion to run on the VLM response.'}, 'assertion_value': {'type': 'string', 'description': 'Expected value for the assertion (not needed for json_format).'}}}, 'maxItems': 10, 'description': 'Array of test cases to run against every model.'}, 'image_base64': {'type': 'string', 'description': 'Base64-encoded image data (required unless image_url is provided).'}, 'system_prompt': {'type': 'string', 'description': 'Optional system prompt sent to every VLM.'}, 'image_mime_type': {'type': 'string', 'description': 'MIME type of the image if using image_base64 (default: image/jpeg).'}}}

Schéma de sortie

{'type': 'object', 'properties': {'suites': {'type': 'array', 'items': {'type': 'object'}}, 'verdict': {'type': 'string'}, 'total_failed': {'type': 'number'}, 'total_passed': {'type': 'number'}}, 'additionalProperties': True}

sandbox_scenario

Get a ready-made selector-drift test case with a known-correct answer, for testing this MCP server or an agent workflow end to end. Each scenario is a real DOM capture of a deliberately breakable app, taken before and after one specific UI change (a renamed label, two swapped buttons, a duplicated locator, an element moved behind a menu…), plus the verdict those two contracts MUST produce. Call with no arguments to list the scenarios; call with a scenario id to get "baseline" and "current" mappings. THE LOOP: pass baseline and current to diff_mappings, then compare its "verdict" and "counts" to this tool's "expected" — they must match exactly. A mismatch means this server's diff engine has drifted, not that your inputs are wrong. Deterministic and offline: the captures are committed fixtures, identical on every call. Try it live at https://www.ia-qa.com/devtools/sandbox

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'scenario': {'type': 'string', 'description': 'Scenario id. Omit to list every available scenario with its expected verdict. Ids: no-change, swap-label, add-testid, duplicate-role-name, remove-element, insert-sibling, rename-label, counter-label, move-behind-menu, add-element'}, 'include_html': {'type': 'boolean', 'description': 'Include the generated HTML of the mutated page (default false). Only useful if you want to render or re-capture it yourself; the loop does not need it.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'html': {}, 'page': {}, 'blurb': {}, 'count': {'type': 'number'}, 'title': {}, 'current': {'type': 'object'}, 'teaches': {}, 'baseline': {'type': 'object'}, 'expected': {'type': 'object'}, 'scenario': {}, 'mutations': {'type': 'array'}, 'scenarios': {'type': 'array'}, 'how_to_run_the_loop': {}}, 'additionalProperties': True}

score_geo_signals

Analyze a webpage <head> HTML (or full HTML) for GEO (Generative Engine Optimization) signals. Returns a score /60 with per-check results and improvement tips. GEO = optimizing pages for AI-powered search engines (ChatGPT Search, Perplexity, etc.).

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['head_html'], 'properties': {'head_html': {'type': 'string', 'description': 'Raw HTML of the <head> section (or full page HTML) to analyze'}}}

Schéma de sortie

{'type': 'object', 'properties': {'grade': {}, 'score': {}, 'checks': {}, 'passed': {'type': 'number'}, 'max_score': {'type': 'number'}, 'total_checks': {'type': 'number'}}, 'additionalProperties': True}

search_jira_issues

Search Jira using JQL (Jira Query Language). Returns matching issues with key fields. Ideal for finding open bugs, sprint tickets, or issues by label/assignee/component. BYOK — credentials transit in-memory only, never stored.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['jql', 'jira_base_url', 'jira_email', 'jira_token'], 'properties': {'jql': {'type': 'string', 'description': 'JQL query string, e.g. "project = PROJ AND status = Open AND assignee = currentUser() ORDER BY priority DESC"'}, 'fields': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Fields per issue. Default: summary, status, assignee, priority, issuetype, labels, created, updated'}, 'jira_email': {'type': 'string', 'description': 'Atlassian account email'}, 'jira_token': {'type': 'string', 'description': 'Atlassian API token'}, 'max_results': {'type': 'number', 'description': 'Max issues to return (default: 10, max: 50)'}, 'jira_base_url': {'type': 'string', 'description': 'Atlassian base URL, e.g. "https://mycompany.atlassian.net"'}}}

Schéma de sortie

{'type': 'object', 'properties': {'jql': {'type': 'string'}, 'total': {'type': 'number'}, 'issues': {'type': 'array', 'items': {'type': 'object'}}, 'returned': {'type': 'number'}}, 'additionalProperties': True}

secret_scan

Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), private keys (RSA/EC/PGP), JWTs, database connection strings, Bearer tokens, and Basic auth headers. Returns a list of findings with type, severity, line number, and a redacted preview. Use before committing code, sharing logs, or sending text to an LLM. 100% regex-based, zero network calls.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text or code to scan for secrets'}, 'types': {'type': 'string', 'description': 'Comma-separated families to scan (default: all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, twilio, sendgrid, jwt, private_key, connection_string, bearer, basic_auth, generic. Individual pattern names (e.g. "aws_access_key", "github_fine") are also accepted. An unknown value is rejected with an error — a scoped scan never silently returns "clean".'}}}

Schéma de sortie

{'type': 'object', 'properties': {'summary': {'type': 'string'}, 'findings': {}, 'risk_level': {}, 'input_lines': {'type': 'number'}, 'scanned_types': {}, 'secrets_found': {}, 'findings_count': {'type': 'number'}}, 'additionalProperties': True}

security_headers_check

Analyse the HTTP security headers of a public URL OR of raw response headers you paste in. Grades each header (A–F) for: Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, X-XSS-Protection, Cross-Origin-Opener-Policy, Cross-Origin-Resource-Policy, and Cross-Origin-Embedder-Policy. Returns an overall score (0–100), per-header grades, missing headers, and fix snippets for Express, Nginx, and Apache. For localhost/private targets the remote server cannot reach, pass the `headers` parameter instead of `url`.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'properties': {'url': {'type': 'string', 'description': 'Optional. Full public URL to check (e.g. https://example.com). Omit it entirely when using `headers`. The server cannot reach localhost/private IPs.'}, 'headers': {'description': 'Optional, and sufficient on its own (no url needed). The response headers to grade, either as an object {"strict-transport-security": "max-age=...", ...} or as the raw header block pasted as a string (e.g. `curl -sI` output). Use this to audit a local server the remote MCP cannot reach.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'fix': {}, 'key': {}, 'url': {}, 'weak': {}, 'grade': {}, 'score': {}, 'value': {}, 'header': {}, 'source': {}, 'weight': {}, 'details': {}, 'missing': {}, 'weak_count': {'type': 'number'}, 'missing_count': {'type': 'number'}, 'overall_grade': {}, 'headers_checked': {'type': 'number'}}, 'additionalProperties': True}

shield_analyze

Run a comprehensive AI guardrail analysis on an LLM response. Orchestrates 7 deterministic safety checks plus an optional LLM-powered deep analysis in parallel: hallucination detection (grounding score), prompt injection scan on BOTH the response and the original prompt (reported separately as checks.injection / checks.injection_prompt, scored once at the worse of the two), toxicity scan, output validation (PII/safety), guardrail rules, response quality scoring, and AI verdict (via Qwen, Gemma, Llama, etc.). Returns a unified PASS/FIX/BLOCK verdict with a 0-100 safety score, per-check results, and actionable fix recommendations. PII gates the verdict: an SSN or credit card in the response forces BLOCK, an email forces FIX, phone/IP matches are flagged only (their regexes also match dates and version strings). Use this as a single-call safety gate before surfacing any LLM output to users.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['response'], 'properties': {'model': {'type': 'string', 'description': 'LLM model for AI-powered deep analysis (default: "openai/gpt-oss-20b"). Set to "none" to skip LLM check. Supports any model from list_llm_models.'}, 'rules': {'type': 'array', 'items': {'type': 'object', 'properties': {'type': {'type': 'string'}, 'label': {'type': 'string'}, 'value': {'type': 'string'}}}, 'description': 'Optional guardrail rules array (same format as guardrail_test tool)'}, 'prompt': {'type': 'string', 'description': 'Optional original prompt. Used for quality scoring AND scanned for prompt injection in its own right (checks.injection_prompt) — pass it whenever you have it, it is where the attack actually lands.'}, 'source': {'type': 'string', 'description': 'Optional reference/source text for hallucination grounding check'}, 'response': {'type': 'string', 'description': 'The LLM-generated response to analyze'}}}

Schéma de sortie

{'type': 'object', 'properties': {'flags': {}, 'grade': {}, 'score': {}, 'checks': {'type': 'object'}, 'verdict': {}}, 'additionalProperties': True}

similarity_score

Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'properties': {'batch': {'type': 'array', 'items': {'type': 'object', 'required': ['reference', 'hypothesis'], 'properties': {'reference': {'type': 'string'}, 'hypothesis': {'type': 'string'}}}, 'description': 'Batch mode: array of {reference, hypothesis} pairs.'}, 'metrics': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Metrics to compute (default: all). Options: "cosine_bow", "cosine_tfidf", "jaccard", "rouge1", "rouge2", "rougeL", "bleu"'}, 'reference': {'type': 'string', 'description': 'Reference / expected text (ground truth)'}, 'threshold': {'type': 'number', 'description': 'Optional pass/fail threshold (0-1). Applies to ROUGE-L F1 score.'}, 'hypothesis': {'type': 'string', 'description': 'Hypothesis / actual text (LLM output)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'f1': {'type': 'number'}, 'mode': {'type': 'string'}, 'count': {'type': 'number'}, 'recall': {'type': 'number'}, 'results': {}, 'precision': {'type': 'number'}}, 'additionalProperties': True}

sort_lines

Sort, deduplicate, reverse, or filter lines of text. Useful for cleaning import lists, dependencies, log files, and config entries.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'trim': {'type': 'boolean', 'description': 'Trim whitespace from each line (default: true)'}, 'input': {'type': 'string', 'description': 'Multi-line text to process'}, 'filter': {'type': 'string', 'description': 'For "filter": keep lines containing this substring (case-insensitive)'}, 'operation': {'type': 'string', 'description': '"sort" (default), "sort_desc", "reverse", "deduplicate", "unique_sort", "filter"'}, 'remove_empty': {'type': 'boolean', 'description': 'Remove empty lines (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'result': {'type': 'string'}, 'removed': {'type': 'number'}, 'line_count': {'type': 'number'}, 'original_count': {'type': 'number'}}, 'additionalProperties': True}

split_chunks

Split text into chunks of at most N tokens (cl100k_base: ~4 chars/token) with optional overlap. Designed for RAG ingestion pipelines.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input', 'chunk_tokens'], 'properties': {'input': {'type': 'string', 'description': 'Text to split into chunks'}, 'overlap': {'type': 'number', 'description': 'Token overlap between consecutive chunks (default: 0)'}, 'chunk_tokens': {'type': 'number', 'description': 'Maximum tokens per chunk (10–8000)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'chunks': {'type': 'array'}, 'chunk_count': {'type': 'number'}, 'overlap_tokens': {}, 'tokens_per_chunk': {}}, 'additionalProperties': True}

ssl_certificate_check

Analyse the SSL/TLS certificate of any HTTPS host. Returns certificate subject, issuer, validity dates, days until expiry, protocol version, cipher suite, key exchange info, and an overall grade (A+, A, B, C, F). Detects expired, self-signed, and weak certificates. Use this to audit TLS posture before production deployment or during security reviews.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['host'], 'properties': {'host': {'type': 'string', 'description': 'Hostname to check (e.g. example.com). Do not include https:// prefix.'}, 'port': {'type': 'number', 'description': 'Port number (default: 443)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'host': {'type': 'string'}, 'grade': {'type': 'string'}, 'cipher': {'type': 'object'}, 'issuer': {'type': 'object'}, 'issues': {'type': 'array', 'items': {'type': 'string'}}, 'subject': {'type': 'object'}, 'protocol': {'type': 'string'}, 'valid_to': {'type': 'string'}, 'is_expired': {'type': 'boolean'}, 'valid_from': {'type': 'string'}, 'is_self_signed': {'type': 'boolean'}, 'days_until_expiry': {'type': 'number'}}, 'additionalProperties': True}

strip_markdown

Strip all Markdown formatting (headers, bold, italic, code fences, links, lists) from text and return clean plain text. Run this before injecting scraped documentation, README files, or user content into an LLM prompt to eliminate redundant markup tokens and reduce cost.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Markdown text to convert to plain text'}}}

Schéma de sortie

{'type': 'object', 'properties': {'text': {}, 'original_length': {'type': 'number'}, 'stripped_length': {'type': 'number'}}, 'additionalProperties': True}

system_prompt_builder

Build a structured system prompt from components: role, task, constraints, output format, tone, language, and examples. Generates a production-ready system prompt with token estimate.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['role'], 'properties': {'role': {'type': 'string', 'description': 'Role/persona (e.g. "Senior QA Engineer", "JSON extraction assistant")'}, 'task': {'type': 'string', 'description': 'Main task or objective'}, 'tone': {'enum': ['professional', 'friendly', 'concise', 'technical', 'educational'], 'type': 'string', 'description': 'Communication tone'}, 'examples': {'type': 'string', 'description': 'Brief examples to include'}, 'language': {'type': 'string', 'description': 'Response language (e.g. "French")'}, 'constraints': {'type': 'array', 'items': {'type': 'string'}, 'description': 'Rules and constraints to follow'}, 'output_format': {'type': 'string', 'description': 'Expected output format description'}}}

Schéma de sortie

{'type': 'object', 'properties': {'sections': {'type': 'object'}, 'system_prompt': {}, 'token_estimate': {'type': 'number'}}, 'additionalProperties': True}

test_skill

Validate a SKILL.md definition (Cursor / GitHub Copilot / Windsurf) by auto-generating trigger-positive and trigger-negative scenarios, running each through the model with the skill injected as a system prompt, and scoring trigger accuracy + step adherence. Returns a PASS/FIX/BLOCK verdict with per-scenario breakdown. Uses Groq llama-3.3-70b by default (server key, no api_key needed). Pass api_key + model to use your own provider.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['skill_md'], 'properties': {'model': {'type': 'string', 'description': 'LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to openai/gpt-oss-20b (Groq, server key).'}, 'api_key': {'type': 'string', 'description': 'API key for the chosen model provider. Not required when using the default Groq model.'}, 'skill_md': {'type': 'string', 'description': 'Full content of the SKILL.md file to test. Must include a name, a "Use when:" trigger description, and at least one step.'}, 'scenario_count': {'enum': [4, 6, 8, 10], 'type': 'number', 'description': 'Number of test scenarios to generate: half trigger-positive, half trigger-negative. Default: 6.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'score': {'type': 'number'}, 'verdict': {'type': 'string'}, 'scenarios': {'type': 'array', 'items': {'type': 'object'}}, 'step_adherence': {'type': 'number'}, 'trigger_accuracy': {'type': 'number'}}, 'additionalProperties': True}

text_stats

Compute comprehensive statistics for any text: character count (with and without spaces), word count, line count, sentence count, paragraph count, and estimated reading time in minutes. Sentence counting is abbreviation-aware — titles (Mr., Dr.), acronyms (U.S., i.e., p.m.), initials, decimals, URLs and emails do not end a sentence, and a text with no terminal punctuation still counts as one. Use for validating form field lengths, evaluating LLM output verbosity, or content auditing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'The text to analyse'}}}

Schéma de sortie

{'type': 'object', 'properties': {'chars': {}, 'lines': {}, 'words': {}, 'sentences': {}, 'paragraphs': {}, 'chars_no_space': {}, 'reading_time_minutes': {}}, 'additionalProperties': True}

timestamp_convert

Convert between Unix timestamps (seconds or milliseconds) and ISO-8601 / UTC date strings. Auto-detects epoch vs. millisecond format. Omit input to get the current time. Returns iso, unix_s, unix_ms, utc, date, and time fields.

Lecture seule

Schéma d’entrée

{'type': 'object', 'properties': {'input': {'description': 'Unix timestamp (number, seconds or ms) or ISO date string. Omit to get the current time.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'iso': {'type': 'string'}, 'utc': {'type': 'string'}, 'date': {'type': 'string'}, 'time': {'type': 'string'}, 'unix_s': {'type': 'number'}, 'unix_ms': {'type': 'number'}}, 'additionalProperties': True}

token_budget_calculator

Plan token allocation across system prompt, user input, context/RAG chunks, and expected output. Warns if budget exceeds model context window. Supports 25+ models.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['model'], 'properties': {'model': {'type': 'string', 'description': 'Model name (e.g. gpt-4o, claude-3.5-sonnet, gemini-2.0-flash)'}, 'context': {'type': 'string', 'description': 'Actual context text (will estimate tokens)'}, 'user_input': {'type': 'string', 'description': 'Actual user input text (will estimate tokens)'}, 'system_prompt': {'type': 'string', 'description': 'Actual system prompt text (will estimate tokens)'}, 'context_tokens': {'type': 'number', 'description': 'Token count for RAG context / documents'}, 'user_input_tokens': {'type': 'number', 'description': 'Token count for user message'}, 'system_prompt_tokens': {'type': 'number', 'description': 'Token count for system prompt'}, 'expected_output_tokens': {'type': 'number', 'description': 'Expected max output tokens'}}}

Schéma de sortie

{'type': 'object', 'properties': {'model': {}, 'warnings': {}, 'breakdown': {'type': 'object'}, 'context_window': {}, 'fits_in_window': {}, 'remaining_tokens': {}, 'utilization_percent': {}}, 'additionalProperties': True}

toxicity_scan

Scan text for toxic language, hate speech, bias/stereotype framing, violence, sexual and self-harm content. Lexical + structural pattern matching (identity term + predicate), not a semantic classifier — returns per-category risk plus the named rules that fired, so every finding can be checked. Useful for LLM safety guardrail testing and triage; signal-only, not a calibrated CI gate.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['text'], 'properties': {'text': {'type': 'string', 'description': 'Text to scan'}, 'categories': {'type': 'array', 'items': {'enum': ['profanity', 'hate_speech', 'violence', 'sexual', 'self_harm', 'bias'], 'type': 'string'}, 'description': 'Categories to check (default: all)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'method': {'type': 'string'}, 'results': {}, 'text_length': {'type': 'number'}, 'overall_risk': {}, 'categories_checked': {'type': 'number'}}, 'additionalProperties': True}

transform_json_array

Transform a JSON array using common operations: pluck (extract specific fields), filter (by field value), sort_by (field), group_by (field), count_by (field), uniq_by (field). Useful for processing MCP tool results and LLM structured outputs.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input', 'operation'], 'properties': {'n': {'type': 'number', 'description': 'For first_n / last_n: number of items'}, 'path': {'type': 'string', 'description': 'Optional dot-notation path to the array within the JSON object (e.g. "data.items")'}, 'field': {'type': 'string', 'description': 'Field to operate on (for sort_by, group_by, count_by, uniq_by, filter)'}, 'input': {'type': ['string', 'object', 'array'], 'description': 'The JSON containing an array (or an object with an array at `path`) — a JSON string, or the value itself.'}, 'fields': {'type': 'string', 'description': 'Comma-separated field list for "pluck" (e.g. "id,name,email")'}, 'filter_op': {'type': 'string', 'description': 'For "filter": "==" | "!=" | ">" | ">=" | "<" | "<=" | "contains" | "exists" | "!exists"'}, 'operation': {'type': 'string', 'description': 'Operation: "pluck", "filter", "sort_by", "group_by", "count_by", "uniq_by", "reverse", "first_n", "last_n", "flatten"'}, 'sort_order': {'type': 'string', 'description': 'For sort_by: "asc" (default) or "desc"'}, 'filter_value': {'type': 'string', 'description': 'For "filter": value to compare against'}}}

Schéma de sortie

{'type': 'object', 'properties': {'count': {'type': 'number'}, 'field': {}, 'order': {}, 'total': {'type': 'number'}, 'fields': {}, 'result': {}, 'removed': {'type': 'number'}, 'operation': {'type': 'string'}, 'group_count': {'type': 'number'}, 'unique_values': {'type': 'number'}, 'removed_duplicates': {'type': 'number'}}, 'additionalProperties': True}

truncate_to_tokens

Truncate text to at most N tokens (cl100k_base: ~4 chars/token) to avoid exceeding an LLM context window. Optionally keeps the end of the text instead of the start (useful for keeping recent conversation history). Reports whether truncation occurred and the estimated token count.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input', 'max_tokens'], 'properties': {'input': {'type': 'string', 'description': 'Text to truncate'}, 'from_end': {'type': 'boolean', 'description': 'Keep the end of the text instead of the start (default: false)'}, 'max_tokens': {'type': 'number', 'description': 'Maximum number of tokens to keep'}}}

Schéma de sortie

{'type': 'object', 'properties': {'text': {}, 'truncated': {'type': 'boolean'}, 'tokens_estimate': {}, 'original_tokens_estimate': {}}, 'additionalProperties': True}

unescape_html

Convert HTML entities (&, <, >, ", ', and numeric &#NNN;) back to plain characters. Use when processing HTML-encoded text from APIs, email content, or legacy database fields before passing to an LLM or displaying to users.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'HTML-encoded string to unescape'}}}

Schéma de sortie

{'type': 'object', 'properties': {'unescaped': {}}, 'additionalProperties': True}

url_decode

Decode a percent-encoded URL string back to plain text. Use when parsing query parameters from raw URLs or when displaying encoded values to users.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'URL-encoded string to decode'}}}

Schéma de sortie

{'type': 'object', 'properties': {'decoded': {}}, 'additionalProperties': True}

url_encode

Percent-encode a string for safe use in URLs. Call this before programmatically building query strings, path segments, or form-encoded bodies to prevent injection and malformed URLs.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'mode': {'type': 'string', 'description': '"component" (default) or "full" for encodeURI behavior'}, 'input': {'type': 'string', 'description': 'String to URL-encode'}}}

Schéma de sortie

{'type': 'object', 'properties': {'encoded': {}}, 'additionalProperties': True}

validate_agent_trajectory

Run declarative assertions on an agent trace (OpenAI tool-call messages, Anthropic tool_use/tool_result blocks, LangChain run trees, or plain text ReAct logs). No LLM call — deterministic. Assertion types: order (tool A before B), must_call, must_not_call, max_calls, min_calls, no_error, recovery (agent completes a successful step after its last error). A step counts as errored when the trace says so — is_error/isError, status/state in {error,failed,exception,…}, an error field, a JSON body with error/success:false — or when its text payload STARTS with an error marker (Error:, Traceback, TypeError:). Model prose is never scanned for keywords, and every errored step reports error_signal naming what flagged it. Returns per-assertion PASS/FAIL, parsed steps, warnings (a trace parsing to 0 steps is flagged — assertions passing on emptiness prove nothing), and an overall verdict. Use this to gate CI/CD on agent behavior correctness.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['trace', 'assertions'], 'properties': {'trace': {'oneOf': [{'type': 'string'}, {'type': 'object', 'additionalProperties': True}, {'type': 'array', 'items': {}}], 'description': 'Agent execution trace as JSON (OpenAI messages array, LangChain run tree) or plain text log (Thought/Action/Observation format).'}, 'format': {'enum': ['auto', 'openai', 'langchain'], 'type': 'string', 'description': 'Trace format. auto (default) detects automatically.'}, 'assertions': {'type': 'array', 'items': {'type': 'object', 'required': ['type'], 'properties': {'id': {'type': 'string', 'description': 'Optional assertion identifier.'}, 'max': {'type': 'number', 'description': '[max_calls] Maximum number of allowed calls.'}, 'min': {'type': 'number', 'description': '[min_calls] Minimum number of required calls.'}, 'tool': {'type': 'string', 'description': 'Tool name to check (for must_call, must_not_call, max_calls, min_calls).'}, 'type': {'enum': ['order', 'must_call', 'must_not_call', 'max_calls', 'min_calls', 'no_error', 'recovery'], 'type': 'string', 'description': 'Assertion type.'}, 'after': {'type': 'string', 'description': '[order] Tool that must be called after.'}, 'before': {'type': 'string', 'description': '[order] Tool that must be called first.'}}}, 'maxItems': 30, 'description': 'List of assertions to validate against the trace.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'steps': {'type': 'array', 'items': {'type': 'object'}}, 'results': {'type': 'array', 'items': {'type': 'object'}}, 'summary': {'type': 'object', 'additionalProperties': True}, 'verdict': {'type': 'string'}, 'warnings': {'type': 'array', 'items': {'type': 'string'}}}, 'additionalProperties': True}

validate_email

Validate an email address against RFC 5322 syntax before storing it, sending a transactional email, or adding it to a mailing list. Returns { valid, email } — use this to avoid bounces and malformed data.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['email'], 'properties': {'email': {'type': 'string', 'description': 'Email address to validate'}}}

Schéma de sortie

{'type': 'object', 'properties': {'email': {}, 'valid': {'type': 'boolean'}, 'reason': {'type': 'string'}}, 'additionalProperties': True}

validate_mcp_response

Validate that an MCP tool response conforms to expected format, schema, and content rules. Use this to QA-test any MCP server tool. Supply the tool's actual JSON result and a set of checks to perform.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['response'], 'properties': {'response': {'type': 'string', 'description': 'The MCP tool result as a JSON string to validate'}, 'min_items': {'type': 'number', 'description': 'If response is an array, minimum number of items expected'}, 'expected_type': {'enum': ['object', 'array', 'string', 'number'], 'type': 'string', 'description': 'Expected top-level type: "object", "array", "string", "number"'}, 'required_keys': {'type': 'string', 'description': 'Comma-separated list of keys that MUST exist in the response (dot-notation for nested: "data.id, data.name")'}, 'actual_latency': {'type': 'number', 'description': 'Actual measured latency in ms (from the call)'}, 'forbidden_keys': {'type': 'string', 'description': 'Comma-separated list of keys that MUST NOT exist (e.g. "password, secret, token")'}, 'max_size_bytes': {'type': 'number', 'description': 'Maximum acceptable response size in bytes'}, 'max_response_ms': {'type': 'number', 'description': 'Maximum acceptable latency in ms (will be compared if provided)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'total': {'type': 'number'}, 'checks': {}, 'failed': {'type': 'number'}, 'passed': {'type': 'number'}, 'verdict': {'type': 'string'}}, 'additionalProperties': True}

validate_url

Parse and validate a URL. Returns decomposed components: protocol, hostname, port, path, query parameters, hash, and origin.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'URL to validate and parse'}}}

Schéma de sortie

{'type': 'object', 'properties': {'full': {}, 'hash': {}, 'port': {}, 'valid': {'type': 'boolean'}, 'origin': {}, 'search': {}, 'hostname': {}, 'pathname': {}, 'protocol': {}, 'query_params': {}}, 'additionalProperties': True}

vector_quantize

Simulate int8 or int4 quantization of float32 embedding vectors. Reduces storage by 4x (int8) or 8x (int4). Returns quantized values, scale factor, and precision loss (MSE). Useful for understanding vector DB compression trade-offs.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['vector'], 'properties': {'bits': {'type': 'number', 'description': 'Quantization bits: 8 (int8, default) or 4 (int4)'}, 'vector': {'type': 'array', 'items': {'type': 'number'}, 'description': 'Float32 vector to quantize'}}}

Schéma de sortie

{'type': 'object', 'properties': {'mse': {'type': 'number'}, 'bits': {}, 'offset': {'type': 'number'}, 'dimension': {'type': 'number'}, 'quantized': {}, 'scale_factor': {'type': 'number'}, 'compression_ratio': {'type': 'string'}, 'storage_bytes_float32': {}, 'storage_bytes_quantized': {'type': 'number'}}, 'additionalProperties': True}

vector_similarity

Compute similarity/distance between two float vectors: cosine similarity, dot product, Euclidean and Manhattan distance. Essential for vector DB relevance scoring, embedding evaluation, and nearest-neighbor testing.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['vector_a', 'vector_b'], 'properties': {'metric': {'enum': ['cosine', 'dot_product', 'euclidean', 'manhattan', 'all'], 'type': 'string', 'description': 'Distance metric (default: all)'}, 'vector_a': {'type': 'array', 'items': {'type': 'number'}, 'description': 'First vector as array of floats'}, 'vector_b': {'type': 'array', 'items': {'type': 'number'}, 'description': 'Second vector as array of floats'}}}

Schéma de sortie

{'type': 'object', 'properties': {'norm_a': {}, 'norm_b': {}, 'dimension': {}, 'dot_product': {}, 'interpretation': {}, 'cosine_distance': {}, 'cosine_similarity': {}, 'euclidean_distance': {}, 'manhattan_distance': {}}, 'additionalProperties': True}

vector_stats

Compute statistics for a float vector or matrix of vectors: mean, std, L2 norm, min, max, sparsity, top-K indices. Useful for debugging embedding quality and analyzing vector distributions in a vector DB.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'anyOf': [{'required': ['vector']}, {'required': ['matrix']}], 'properties': {'top_k': {'type': 'number', 'description': 'Return indices of top K absolute values (default: 5)'}, 'matrix': {'type': 'array', 'items': {'type': 'array', 'items': {'type': 'number'}}, 'description': 'Matrix of vectors (overrides vector). Returns per-vector + matrix-level stats. Required unless `vector` is given.'}, 'vector': {'type': 'array', 'items': {'type': 'number'}, 'description': 'Single vector to analyze. Required unless `matrix` is given.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'max': {'type': 'number'}, 'min': {'type': 'number'}, 'std': {'type': 'number'}, 'mean': {'type': 'number'}, 'l2_norm': {'type': 'number'}, 'sparsity': {'type': 'number'}, 'dimension': {}, 'per_vector': {}, 'matrix_shape': {'type': 'array'}, 'matrix_stats': {'type': 'object'}, 'top_k_indices': {}}, 'additionalProperties': True}

webhook_endpoint_create

Create a temporary webhook endpoint that captures incoming HTTP requests for one hour. Returns the webhook id, public URL, expiration timestamp, and current request count. Use together with webhook_endpoint_requests to inspect captured payloads.

Schéma d’entrée

{'type': 'object', 'properties': {'base_url': {'type': 'string', 'description': 'Optional public base URL. Default: https://www.ia-qa.com/mcp/webhook (the apex ia-qa.com answers 301 and a redirected POST loses its body, so an apex base_url is normalized to www)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'id': {}, 'url': {'type': 'string'}, 'expires_at': {'type': 'string'}, 'request_count': {'type': 'number'}, 'retention_minutes': {'type': 'number'}}, 'additionalProperties': True}

webhook_endpoint_requests

Fetch the requests captured by a webhook created with webhook_endpoint_create. Returns the newest requests first with method, headers, query params, body payload, and timestamps.

Lecture seule

Schéma d’entrée

{'type': 'object', 'required': ['id'], 'properties': {'id': {'type': 'string', 'description': 'Webhook id returned by webhook_endpoint_create'}, 'limit': {'type': 'number', 'description': 'Maximum number of requests to return (1-100, default: 20)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'id': {'type': 'string'}, 'requests': {'type': 'array'}, 'expires_at': {'type': 'string'}, 'request_count': {'type': 'number'}}, 'additionalProperties': True}

web_security_audit

Run a comprehensive web security audit combining headers, SSL, CORS, and cookies checks — then use an LLM to produce a prioritised remediation plan. Orchestrates security_headers_check + ssl_certificate_check + cors_test + cookie_security_audit in parallel, merges all findings, then asks an AI model to: (1) rank vulnerabilities by real-world exploitability, (2) generate a remediation roadmap, (3) produce fix code snippets for the detected stack. Returns both raw audit data and the AI analysis. Use this as a one-click security posture assessment.

Lecture seule Accès externe

Schéma d’entrée

{'type': 'object', 'required': ['url'], 'properties': {'url': {'type': 'string', 'description': 'Full URL to audit (e.g. https://example.com)'}, 'model': {'type': 'string', 'description': 'LLM model for AI analysis (default: "openai/gpt-oss-20b"). Set to "none" to skip AI analysis.'}, 'api_key': {'type': 'string', 'description': 'Your Groq or HuggingFace API key. Required to enable AI analysis.'}}}

Schéma de sortie

{'type': 'object', 'properties': {'fix': {}, 'key': {}, 'url': {}, 'name': {}, 'weak': {}, 'grade': {}, 'score': {}, 'tests': {}, 'value': {}, 'header': {}, 'issues': {}, 'secure': {}, 'weight': {}, 'cookies': {'type': 'array'}, 'details': {}, 'message': {'type': 'string'}, 'missing': {}, 'httpOnly': {}, 'sameSite': {}, 'risk_level': {}, 'weak_count': {'type': 'number'}, 'cookies_found': {'type': 'number'}, 'missing_count': {'type': 'number'}, 'overall_grade': {}, 'origins_tested': {'type': 'number'}, 'total_findings': {'type': 'number'}, 'headers_checked': {'type': 'number'}}, 'additionalProperties': True}

word_frequency

Analyze word frequency in text. Returns top N words with counts and percentages. Supports English stopword filtering. Useful for content analysis, keyword extraction, and LLM output analysis.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'Text to analyze'}, 'top_n': {'type': 'number', 'description': 'Return top N words (default: 20, max: 200)'}, 'min_length': {'type': 'number', 'description': 'Minimum word length to include (default: 3)'}, 'remove_stopwords': {'type': 'boolean', 'description': 'Remove common English stopwords (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'top_words': {'type': 'array'}, 'total_words': {}, 'unique_words': {'type': 'number'}, 'stopwords_removed': {}}, 'additionalProperties': True}

xml_to_json

Convert an XML string to a JSON object. Supports attributes, nested elements, arrays, CDATA, and namespaces. Options: parse numbers, parse booleans, ignore attributes.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'XML string to convert'}, 'attr_prefix': {'type': 'string', 'description': 'Prefix for attribute keys (default: "@_")'}, 'ignore_attrs': {'type': 'boolean', 'description': 'Ignore XML attributes (default: false)'}, 'parse_values': {'type': 'boolean', 'description': 'Auto-parse numbers and booleans (default: true)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'result': {}, 'key_count': {'type': 'number'}}, 'additionalProperties': True}

yaml_to_json

Parse a YAML string and return the equivalent JSON value. The reverse of json_to_yaml. Supports nested objects, arrays, anchors, aliases, multi-document streams, and all scalar types. Use when processing config files, CI/CD pipeline definitions, or OpenAPI specs authored in YAML.

Lecture seule Idempotent

Schéma d’entrée

{'type': 'object', 'required': ['input'], 'properties': {'input': {'type': 'string', 'description': 'YAML string to parse'}, 'multi': {'type': 'boolean', 'description': 'If true, parse all documents in a multi-document stream and return an array (default: false)'}}}

Schéma de sortie

{'type': 'object', 'properties': {'json': {}, 'count': {'type': 'number'}, 'documents': {}}, 'additionalProperties': True}

IA-QA — 130+ QA & Dev Tools for AI Agents

Ce que fait ce MCP

Outils

Modifications récentes des outils

Serveurs MCP similaires

invinoveritas

hyperion

Andreax

Vee3

GoCreative Agent API

validoria-mcp

HubVibe: Pay-per-Call Tools for AI Agents: Web Search, Email Verify, KYC, Stocks, Crypto, News, Data

developer-tools