• INFERENCE
  • INVENTORY
  • VALIDATION
  • HUMAN

How to run model predictions with your AI

Connect your AI to Baseten to trigger inference on hosted weights. You can send payloads and get results back in the same conversation.

Ask AI about this page

Short answer

How can I run model predictions with my AI?

You send data payloads directly to your deployed models on Baseten. Your AI uses the connection to trigger inference and returns the model's response to your chat interface. This allows you to test weights and verify outputs without leaving your client.

The results of your requests

Where your data lands

What happens after your AI interacts with Baseten.

INFERENCE

Live model runs

Your AI sends data to the GPU. You get the model's prediction back as text or structured data.

INVENTORY

Model visibility

Your agent identifies which models are currently active and ready for use. This keeps your prompts accurate.

VALIDATION

Output checking

You can immediately ask your AI to evaluate if the model's response meets your specific requirements.

HUMAN

Manual oversight

You decide if the prediction is good enough to use in your application or if you need to tweak the input.

The workflow

What your AI does when the request arrives.

The AI handles the communication between your prompt and the GPU hardware.

  1. Identify available models

    The AI checks which specific model endpoints are currently deployed and available for inference.

    list_models

  2. Format the payload

    Your agent turns your natural language request into the specific JSON format the model expects.

    predict

  3. Execute inference

    The AI sends the formatted data to the Baseten endpoint to trigger the GPU workload.

    predict

  4. Deliver the result

    The AI receives the raw output from the model and presents it to you in the chat.

    predict

Try it

Copy these to start.

Use these prompts to begin interacting with your models.

Starting points

Swap out the model names and data types for your specific use case.

Baseten Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_Iok4x74580t8DfXgX0kQJnXTan6vttePrqZ5XRGU/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Baseten capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "baseten-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_Iok4x74580t8DfXgX0kQJnXTan6vttePrqZ5XRGU/mcp"
    }
  }
}
Copy into chat04

  • List my deployed models so I know what I can use.

  • Run a prediction on my image-gen model using this description: a sunset over a neon city.

  • Check my sentiment-analysis model with the text: 'The service was okay, but the food was cold.'

  • What models do I have available right now?

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Baseten once, then ask.

You only need to link your account once. Your credentials stay encrypted, and you can start running predictions immediately.

Connect Baseten to your AI

FAQ

Common questions

  • 01

    Can my AI deploy new models to Baseten?

    No. The AI can only interact with models that are already deployed and listed in your account.

  • 02

    Can the AI change my GPU instance types?

    No. The AI can only trigger predictions on existing endpoints; it cannot modify your Baseten infrastructure or hardware settings.

  • 03

    How does the AI know which model to use?

    It uses the list of available models to identify the correct endpoint for the task you described.

  • 04

    Does the AI see my raw data?

    Yes, the AI must process your input to format the payload for the model prediction.

  • 05

    Can I use this to monitor model costs?

    No. This connector is for running predictions and listing models, not for accessing billing or usage metrics.

  • More questions about Baseten? The Connector page answers them. See everything the Baseten Connector can do

Connect Baseten to Claude, Cursor, ChatGPT & more