• ACTIVE
  • ROUTING
  • CONTROL
  • HUMAN

How to deploy model inference with your AI

Connect CoreWeave to your AI client to automate GPU resource management. You can spin up new model versions and set up routing rules through a single chat interface.

Ask AI about this page

Short answer

How do I deploy model inference with my AI?

You tell your AI to create or update deployments on CoreWeave to host your models on GPUs. It also handles the gateway configuration so your traffic hits the right endpoints. You get full control over your inference infrastructure without touching a dashboard.

Deployment outcomes

Infrastructure results

What happens when your AI interacts with CoreWeave.

ACTIVE

Live model endpoints

The AI sets up the necessary GPU resources. You get running inference endpoints ready for traffic.

ROUTING

Traffic management

Gateways are configured to point to your specific model versions. This ensures requests land on the right target.

CONTROL

Version updates

You can swap out model versions or adjust existing deployments. The AI handles the configuration changes for you.

HUMAN

Manual oversight

You decide when to scale or kill a deployment. The AI executes your specific commands but doesn't make autonomous scaling decisions.

The workflow

What your AI does when <the thing arrives>.

Your AI acts as the interface between your natural language commands and the CoreWeave API.

  1. Check current state

    The AI looks at what is currently running on your CoreWeave account to understand your existing setup.

    list_deploymentslist_gateways

  2. Provision resources

    The AI sends the command to spin up new GPU instances for your specific model requirements.

    create_deployment

  3. Update versions

    When you need to roll out a new model, the AI modifies the existing deployment settings.

    update_deployment

  4. Route traffic

    The AI sets up or modifies gateways to ensure incoming requests reach the correct inference endpoints.

    create_gatewayupdate_gateway

Try it

Copy these to start.

Use these prompts to begin managing your CoreWeave infrastructure.

Starting points

Swap out the bracketed text with your actual model names or deployment IDs.

CoreWeave Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_a151JnxoNH7y2muaVwaqzr9Q5YKz8wmiZJSE1HZv/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — CoreWeave capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "coreweave-ai-gpu-cloud-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_a151JnxoNH7y2muaVwaqzr9Q5YKz8wmiZJSE1HZv/mcp"
    }
  }
}
Copy into chat04

  • List all my current CoreWeave deployments and gateways.

  • Create a new deployment for my 'llama-3-70b' model.

  • Update the deployment named 'prod-inference' to use the latest model version.

  • Create a new gateway to route traffic to my 'vision-model' endpoint.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect CoreWeave once, then ask.

Link your CoreWeave account to Vinkius. Your credentials stay encrypted, and you can immediately start managing deployments from your favorite AI client.

Connect CoreWeave to your AI

FAQ

How this task behaves

  • 01

    Can the AI delete my deployments?

    No. There is no tool provided to delete deployments or gateways, so the AI cannot remove your resources.

  • 02

    Will the AI automatically scale my GPUs?

    No. The AI only performs the specific actions you request, such as creating or updating a deployment. It does not monitor load or scale independently.

  • 03

    How do I see what is currently running?

    You can ask your AI to list your deployments or gateways to see your active infrastructure.

  • 04

    Can I change the model version of an existing deployment?

    Yes. You can ask the AI to update an existing deployment with a new model configuration.

  • 05

    Does the AI handle the billing for CoreWeave?

    No. The AI only manages the technical deployment and gateway configuration. All billing is handled directly through CoreWeave.

  • More questions about CoreWeave? The Connector page answers them. See everything the CoreWeave Connector can do

Connect CoreWeave to Claude, Cursor, ChatGPT & more