• TEXT
  • CHAT
  • MODELS
  • HUMAN

How to run model inference with your AI client

Connect Together AI to your client once. From then on, you can trigger model runs and chat sessions using simple text commands.

Ask AI about this page

Short answer

How can I run model inference with my AI?

You use Together AI to send text completion or chat requests to open-source models. Your agent handles the heavy lifting by calling the right tools to get the job done. You'll see the results right in your chat window.

The outcomes.

Where your requests land.

Every command you send results in one of these four states.

TEXT

Raw completions

The AI generates text based on your specific prompts. You get the raw output directly in your interface.

CHAT

Interactive sessions

Your agent engages in back-and-forth dialogue. This allows for multi-turn conversations using open-source models.

MODELS

Model selection

You can see which models are ready to use. This helps you pick the right tool for the specific task at hand.

HUMAN

Manual review

The AI provides the output, but you decide if it's good. You remain in control of the final decision.

The workflow

What your AI does when a command arrives.

Your agent acts as the bridge between your chat window and the Together AI infrastructure.

  1. Check availability

    Your agent looks up the current list of supported open-source models to ensure the requested one is online.

    list_available_models

  2. Generate text

    For single-turn tasks, the agent sends your prompt to complete a specific string or block of text.

    text_completion

  3. Start a chat

    For conversational tasks, the agent manages the context and history to maintain a dialogue.

    chat_completion

  4. Deliver response

    The agent receives the model output and presents it to you as a standard message.

    chat_completiontext_completion

Try it

Copy these to start.

Use these prompts to test the connection immediately.

Starting points

These are basic ideas. Swap out the bracketed text with your actual content.

Together AI Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_uwxFbBTrWtpKdtNP0Ee38Zmxxir6fie6FBKXtn0o/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Together AI capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "together-ai-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_uwxFbBTrWtpKdtNP0Ee38Zmxxir6fie6FBKXtn0o/mcp"
    }
  }
}
Copy into chat04

  • List all the models available through Together AI.

  • Complete this sentence for me: The future of open source is

  • Start a chat session to help me brainstorm a new project.

  • Use text completion to summarize this paragraph: [paste text here]

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Together AI once, then ask.

Click the link to connect. Your credentials stay encrypted, and you can start prompting immediately in your preferred client.

Connect Together AI to your AI

FAQ

How the inference task behaves.

  • 01

    Can I use my own private models?

    No. You can only use the open-source models provided through the Together AI interface.

  • 02

    Can the AI delete my files?

    No. This MCP only has tools for text completion and chat; it cannot access or delete your local files.

  • 03

    Does the AI handle the billing?

    No. You manage your Together AI usage and billing directly through their platform.

  • 04

    Can I see which models are online?

    Yes. Your agent can check the available models before you start a task.

  • 05

    Will the AI remember my previous chats?

    The AI handles the chat context during a session, but it doesn't have permanent memory of your past interactions unless you provide that context.

  • More questions about Together AI? The Connector page answers them. See everything the Together AI Connector can do

Connect Together AI to Claude, Cursor, ChatGPT & more