How to Use the ContextQA MCP in LlamaIndex

Q: How does LlamaIndex store ContextQA test executions?

Your agent calls listexecutions to retrieve raw run logs. LlamaIndex then parses these logs and indexes them into your vector database, making your test history fully searchable.

Q: Can I use LlamaIndex to query specific ContextQA projects?

Yes. Your agent can run getproject to retrieve explicit project UUIDs and mapping data. This data is indexed so your RAG pipeline can instantly locate test execution spaces.

Q: How do I index ContextQA test cases in LlamaIndex?

You use listcases to extract the routing limits and structure of your test tree. LlamaIndex indexes this structural data, allowing your agent to search and retrieve specific test cases semantically.

Q: Can my agent trigger a test run based on indexed data?

Yes, your agent can call triggerrun to dispatch a live testing command. It can decide to do this after querying your index and finding that a specific test suite hasn't run recently.

Index ContextQA test runs and search execution histories using LlamaIndex RAG pipelines.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect ContextQA MCP to LlamaIndex

Create your Vinkius account to connect ContextQA to LlamaIndex and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup ContextQA with LlamaIndex

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Index ContextQA run results in LlamaIndex

This MCP Server lets you turn test execution history into a searchable knowledge base. Your LlamaIndex agent calls `list_executions` to pull deep interaction logs and tracks global run chunks directly. Instead of reading raw logs, you index these execution details into a vector store. When a test fails, your agent queries this index to find historical patterns and solutions based on previous AI-healing runs.

Search ContextQA API tests with LlamaIndex

The ContextQA MCP Server exposes your native REST and OpenAPI configurations to your indexers. Your agent runs `list_api_tests` to pull these configurations and index them alongside your codebase documentation. This setup lets your RAG pipeline search for mismatches between your API specs and actual test cases. It uses `get_case` to validate data science object extractions against the indexed API schemas.

Query project test suites using LlamaIndex

Mapping test suites becomes a semantic search task when you connect this MCP Server. Your agent runs `list_suites` to extract structural payloads from your GUI test suites and adds them to your local index. By combining this with `list_projects`, your agent can instantly answer which test suites cover specific project UUIDs. You don't have to hunt through dashboards to find the right test suite anymore.

Setup guide

Set up ContextQA MCP in LlamaIndex

Prerequisites

Python 3.10+ installed
llama-index-tools-mcp package
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Run pip install llama-index-tools-mcp llama-index-llms-openai. The MCP tools package provides BasicMCPClient and McpToolSpec.
2

Connect with BasicMCPClient
Point BasicMCPClient to your Vinkius endpoint URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. Supports SSE and Streamable HTTP transports.
3

Convert to LlamaIndex tools
Call mcp_tool_spec.to_tool_list_async() to convert all ContextQA MCP tools into native FunctionTool objects that any LlamaIndex agent can use.
4

Run with any LLM
Create a FunctionAgent with the tools and your preferred LLM. Swap OpenAI for Anthropic, Gemini, or any LlamaIndex-supported provider.

agent.py

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

# Connect to the MCP
mcp_client = BasicMCPClient(
    "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)
mcp_tool_spec = McpToolSpec(client=mcp_client)

# Convert MCP tools to LlamaIndex tools
tools = await mcp_tool_spec.to_tool_list_async()

# Create and run the agent
agent = FunctionAgent(
    tools=tools,
    llm=OpenAI(model="gpt-4o"),
    system_prompt="You have access to ContextQA tools.",
)
response = await agent.run("List recent ContextQA data")

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by ContextQA. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect ContextQA now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about ContextQA MCP in LlamaIndex

Your agent calls `list_executions` to retrieve raw run logs. LlamaIndex then parses these logs and indexes them into your vector database, making your test history fully searchable.

Yes. Your agent can run `get_project` to retrieve explicit project UUIDs and mapping data. This data is indexed so your RAG pipeline can instantly locate test execution spaces.

You use `list_cases` to extract the routing limits and structure of your test tree. LlamaIndex indexes this structural data, allowing your agent to search and retrieve specific test cases semantically.

Yes, your agent can call `trigger_run` to dispatch a live testing command. It can decide to do this after querying your index and finding that a specific test suite hasn't run recently.

All test configurations, environment details, and execution histories are processed locally. The Vinkius MCP Server uses ephemeral V8 sandboxes, ensuring your proprietary testing data is never cached or used for training outside your network.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript