How to Use the Anyscale MCP in Vercel AI SDK

Q: How do I handle Anyscale API authentication inside Vercel AI SDK?

You pass your Vinkius endpoint token using the authProvider option in createMCPClient. This routes your Vercel AI SDK requests securely to Anyscale without exposing raw API keys on the client side.

Q: Can I stream Llama-2 outputs from Anyscale using Vercel AI SDK?

Yes, you feed the chatcompletion tool into streamText and request meta-llama/Llama-2-70b-chat-hf. The MCP Server handles the connection, sending tokens directly to your React components.

Q: How do I clean up connections after generating text?

Always invoke mcpClient.close() once your streamText promise resolves. This prevents dangling SSE or HTTP connections between your Vercel deployment and the Anyscale infrastructure.

Stream live Anyscale completions and cluster job statuses directly into your Next.js frontend using the Vercel AI SDK.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Anyscale MCP to Vercel AI SDK

Create your Vinkius account to connect Anyscale to Vercel AI SDK and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Anyscale with Vercel AI SDK

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Stream Anyscale LLM Outputs directly to Vercel AI SDK UI

The `chat_completion` tool lets your React frontend render Llama model tokens the millisecond they leave Anyscale endpoints. You don't make users stare at loading spinners while waiting for text to compile. By feeding this tool into `streamText`, the token stream flows directly through your Next.js Edge Functions. This MCP setup cuts out middleman servers to keep latency under 100 milliseconds.

Generate Embeddings on the Edge

The `generate_embeddings` tool converts raw text input into dense semantic vectors directly during the user's session. Your Vercel AI SDK client handles the vector creation on the fly, allowing instant similarity matching in your database. You call this tool inside `generateText` to process user queries before they hit your vector database. This keeps all vectorization logic contained inside your TypeScript edge runtime without external MCP wrappers.

Expose Anyscale Job Statuses to Admin Dashboards

The `list_jobs` tool pulls active batch and training runs from your cluster and pushes them straight to your React dashboard. Your admin UI displays real-time job progress without polling databases. Integrating this with your Vercel AI SDK setup means your support agents can ask the chatbot about active training runs. The agent calls this tool to retrieve the live cluster state and formats it into a neat markdown table.

Setup guide

Set up Anyscale MCP in Vercel AI SDK

Prerequisites

Node.js 18+ and a TypeScript project
ai + @modelcontextprotocol/sdk packages
Active Vinkius subscription with a valid endpoint token

1

Install dependencies

Run npm install ai @modelcontextprotocol/sdk plus your preferred model provider (e.g. @ai-sdk/openai).
2

Create the Streamable HTTP transport

Use StreamableHTTPClientTransport with your Vinkius endpoint URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com.
3

Discover and use tools

Call mcpClient.tools() to auto-discover all Anyscale tools. Pass them directly to generateText() or streamText() — no manual schema definitions needed.
4

Works with any model provider

Swap openai("gpt-4o") for any AI SDK provider — Anthropic, Google, Mistral. The MCP tools work identically across all supported models.

index.ts

import { experimental_createMCPClient as createMCPClient } from "ai";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

const transport = new StreamableHTTPClientTransport(
  new URL("https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp")
);

const mcpClient = await createMCPClient({ transport });
const tools = await mcpClient.tools();

const { text } = await generateText({
  model: openai("gpt-4o"),
  tools,
  prompt: "List recent Anyscale transactions",
});

console.log(text);
await mcpClient.close();

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Anyscale. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Anyscale now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Anyscale MCP in Vercel AI SDK

You pass your Vinkius endpoint token using the `authProvider` option in `createMCPClient`. This routes your Vercel AI SDK requests securely to Anyscale without exposing raw API keys on the client side.

Yes, you feed the `chat_completion` tool into `streamText` and request `meta-llama/Llama-2-70b-chat-hf`. The MCP Server handles the connection, sending tokens directly to your React components.

Yes, the server runs in a V8 sandbox, and the SDK setup works inside Edge runtimes. You initialize the client, run your query, and call `mcpClient.close()` to prevent memory leaks in short-lived edge instances.

Always invoke `mcpClient.close()` once your `streamText` promise resolves. This prevents dangling SSE or HTTP connections between your Vercel deployment and the Anyscale infrastructure.

Your LLM prompt strings and embedding vectors never touch permanent storage on our end. Vinkius runs the server in an ephemeral, zero-trust V8 isolate that immediately discards your text payloads once the API call completes.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript