Robots.txt Generator MCP. Control exactly what web crawlers see on your site.

Open your terminal

Ensure you have Claude Code installed and initialized in your current directory or globally.

Add the MCP Server

Run the following command to add the Vinkius endpoint to your configuration:

Replace [YOUR_TOKEN_HERE] with your Vinkius token. This securely stores the connection via your CLI.

Start coding

Launch Claude Code. The Robots.txt Generator capabilities will be automatically discovered and ready to use in your CLI workspace.

One-Click Install (Recommended)

In your Vinkius Dashboard, simply click the Add to Cursor button for this server. We'll automatically configure your local workspace.

Open Cursor Settings

Open the Cursor editor, go to Settings (the gear icon), and navigate to Features → MCP Servers.

Add New Server

Click + Add New MCP Server. Set the type to SSE, enter Robots.txt Generator as the name, and paste your Vinkius endpoint:

Remember to replace [YOUR_TOKEN_HERE] with your secure token from cloud.vinkius.com.

Use in Composer

Save the configuration. You can now use the Robots.txt Generator tools directly within Cursor Composer and Chat.

Configure Agent Environment

Open your Antigravity agent's workspace configuration or mcp-servers.json file.

Bind the Endpoint

Add the Vinkius endpoint URL to your agent's MCP connections list:

"mcp_servers": {
  "robotstxt-generator": {
    "serverUrl": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
  }
}

Provide your secure token in place of [YOUR_TOKEN_HERE] to ensure your agent requests are authenticated.

Execute

Start your Antigravity session. The agent will autonomously discover and utilize the Robots.txt Generator tools with full Vinkius guardrails applied.

One-Click Install (Recommended)

In your Vinkius Dashboard, simply click the Add to VS Code button for this server. We'll automatically configure your local workspace.

Open MCP Settings

Open VS Code, press Ctrl/Cmd + Shift + P, and search for GitHub Copilot: MCP Servers.

Add Server Config

Add the Vinkius endpoint configuration to your mcp-servers.json file:

"robotstxt-generator": {
  "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
}

Ensure you replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com.

One-Click Install (Recommended)

In your Vinkius Dashboard, simply click the Add to Windsurf button for this server. We'll automatically configure your local workspace.

Open Windsurf Settings

Open the Windsurf editor, navigate to your MCP Settings or open the global mcp_settings.json.

Add Server Endpoint

Provide the Vinkius SSE endpoint in your configuration:

Replace [YOUR_TOKEN_HERE] with your secure token from cloud.vinkius.com.

Install Dependencies

Install the LangChain MCP adapters for your environment:

Connect the Server

Use the SSEClient in LangChain to connect to the Vinkius managed endpoint:

from langchain_mcp_adapters.client import SSEClient

# Connect to Vinkius
client = SSEClient(url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp")
tools = client.get_tools()

Define the Tool

Load the Vinkius MCP tools into your CrewAI agents:

from crewai import Agent
from mcp_crewai import MCPTool

# Connect securely to Vinkius
vinkius_tools = MCPTool(url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp")

# Assign to Agent
researcher = Agent(
    role='Data Researcher',
    tools=vinkius_tools.get_all()
)

Execute Task

Run your CrewAI process. The agent will autonomously route tasks to the Vinkius managed server.