How to Use the Internet Archive MCP in LlamaIndex

Q: How does the Internet Archive toolset integrate with LlamaIndex vector stores?

Your agent runs search or searchbycollection to find relevant historical items using the MCP endpoint. It then fetches the raw text files via getitemfiles and indexes those documents directly into your vector store for semantic search.

Q: Can I use LlamaIndex to query past Wayback Machine snapshots of the Internet Archive?

Yes, you can use waybackavailability within your LlamaIndex pipeline to find the closest archived snapshot of a URL. The retrieved historical page can then be parsed and stored as a node in your index.

Q: Can my index filter items by creator or author?

Yes, the searchbycreator tool allows your agent to find works by specific authors or organizations. This is perfect for building specialized research indexes centered around specific historical figures.

Index 40 million public books, videos, and Wayback snapshots directly into your LlamaIndex vector stores.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Internet Archive MCP to LlamaIndex

Create your Vinkius account to connect Internet Archive to LlamaIndex and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Internet Archive with LlamaIndex

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Ground your LlamaIndex RAG in historical truth

By running the `get_item_metadata` tool, you can ground your LlamaIndex RAG pipelines in historical truth and eliminate hallucinations. This MCP Server lets your pipelines query the world's largest open library to pull down verified texts, documents, and code. Instead of relying on static files, your query engine can run real-time lookups using `search_by_collection`. It pulls in specific archival sets like Project Gutenberg or NASA records, immediately indexing them to answer niche user queries with absolute accuracy.

Build dynamic web archives inside your index

By executing the `wayback_availability` tool, your LlamaIndex agents can locate historical versions of target URLs. It brings stability to your knowledge graphs by ensuring your references point to permanent, archived snapshots via this MCP integration. Your agent can crawl a list of links, check their historical availability, and pull the archived content directly into your index. This turns your LlamaIndex setup into a living, self-healing knowledge base that preserves web history.

Filter and ingest media by popularity metrics

Using the `get_views_stats` tool allows your LlamaIndex ingest pipelines to evaluate the popularity of any archived item before processing it. This ensures only high-value, highly-viewed documents make it into your vector store. Your agent can execute `search_by_mediatype` to isolate text documents, check their download counts, and fetch the files with `get_item_files`. It creates a highly selective ingestion pipeline that saves you tokens and storage space.

Setup guide

Set up Internet Archive MCP in LlamaIndex

Prerequisites

Python 3.10+ installed
llama-index-tools-mcp package
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Run pip install llama-index-tools-mcp llama-index-llms-openai. The MCP tools package provides BasicMCPClient and McpToolSpec.
2

Connect with BasicMCPClient
Point BasicMCPClient to your Vinkius endpoint URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. Supports SSE and Streamable HTTP transports.
3

Convert to LlamaIndex tools
Call mcp_tool_spec.to_tool_list_async() to convert all Internet Archive MCP tools into native FunctionTool objects that any LlamaIndex agent can use.
4

Run with any LLM
Create a FunctionAgent with the tools and your preferred LLM. Swap OpenAI for Anthropic, Gemini, or any LlamaIndex-supported provider.

agent.py

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

# Connect to the MCP
mcp_client = BasicMCPClient(
    "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)
mcp_tool_spec = McpToolSpec(client=mcp_client)

# Convert MCP tools to LlamaIndex tools
tools = await mcp_tool_spec.to_tool_list_async()

# Create and run the agent
agent = FunctionAgent(
    tools=tools,
    llm=OpenAI(model="gpt-4o"),
    system_prompt="You have access to Internet Archive tools.",
)
response = await agent.run("List recent Internet Archive data")

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Internet Archive. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Internet Archive now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Internet Archive MCP in LlamaIndex

Your agent runs `search` or `search_by_collection` to find relevant historical items using the MCP endpoint. It then fetches the raw text files via `get_item_files` and indexes those documents directly into your vector store for semantic search.

Yes, you can use `wayback_availability` within your LlamaIndex pipeline to find the closest archived snapshot of a URL. The retrieved historical page can then be parsed and stored as a node in your index.

Install the MCP adapter for LlamaIndex and initialize the client pointing to our secure endpoint. Wrap the client in an `McpToolSpec` to expose the search and metadata tools directly to your agent.

Yes, the `search_by_creator` tool allows your agent to find works by specific authors or organizations. This is perfect for building specialized research indexes centered around specific historical figures.

Yes, only the search queries and item identifiers you send to the Archive's public API are processed. Our ephemeral V8 sandbox isolates all requests, meaning your local index configurations and private vector embeddings are never exposed to the MCP Server host.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript