• SPEED
  • MODELS
  • WORKFLOW
  • HUMAN

How to run open source models with your AI

Connect SambaNova to your AI client to access high-throughput inference. Once connected, your agent uses specialized hardware to run open source models.

Ask AI about this page

Short answer

How can I run open source models through my AI?

You connect SambaNova to your AI client to access high-throughput inference for models like Llama 3 and DeepSeek. Your agent handles the model execution through a single interface. You get direct access to these weights without switching apps.

The outcomes

What you get from this connection

Connecting SambaNova gives your agent new ways to process information.

SPEED

High-throughput inference

Your agent uses SambaNova's hardware to run large models. This handles heavy reasoning tasks without the usual lag.

MODELS

Open source access

You can run models like Llama 3 or DeepSeek. Your agent picks the right model for the job automatically.

WORKFLOW

Single interface use

Everything stays in your current chat window. You don't need to manage separate API endpoints or web consoles.

HUMAN

Manual oversight

The AI generates the text, but you review the output. You decide which model responses are useful for your work.

The workflow

What your AI does when a request arrives.

Your agent uses the connection to bridge your prompts to high-performance hardware.

  1. Receive prompt

    Your agent takes your instruction and prepares the request for the inference engine.

    create_chat_completion

  2. Route to model

    The agent selects the specific open source model you requested.

    create_chat_completion

  3. Execute inference

    SambaNova runs the model on specialized hardware to generate the response.

    create_chat_completion

  4. Deliver text

    The completed response is sent back to your chat interface.

    create_chat_completion

Try it

Copy these to start.

Paste these into your chat to test the connection.

Starting points

These are opening lines. Swap the model names or the specific tasks to suit your needs.

SambaNova Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_z52tsMdkFaoCTCBeg6u7bbxNhhnjcT0r5KXU6hjj/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — SambaNova capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "sambanova-ai-inference-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_z52tsMdkFaoCTCBeg6u7bbxNhhnjcT0r5KXU6hjj/mcp"
    }
  }
}
Copy into chat04

  • Run a complex reasoning task using Llama 3.

  • Use DeepSeek to analyze this Python script for bugs.

  • Generate a technical summary using the open source models.

  • Compare how Llama 3 handles this logic puzzle.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect SambaNova once, then ask.

One link connects the MCP to your client. Your login stays encrypted on our side, and you can find the full details on the Connector page.

Connect SambaNova to your AI

FAQ

How this task behaves

  • 01

    Can I use this to run my own local models?

    No. This MCP connects you to SambaNova's hosted open source models, not your local machine.

  • 02

    Does the AI have access to my files?

    No. This MCP only provides inference capabilities. It cannot read or write files on your computer.

  • 03

    Can I choose which model to use?

    Yes. You can specify models like Llama 3 or DeepSeek in your prompts.

  • 04

    Can the AI delete my chat history?

    No. This MCP has no tools to manage or delete your chat history or account data.

  • 05

    Do I need to manage API keys?

    No. Vinkius manages the connection so you don't have to handle individual API keys for the inference engine.

  • More questions about SambaNova? The Connector page answers them. See everything the SambaNova Connector can do

Connect SambaNova to Claude, Cursor, ChatGPT & more