# Ollama MCP for AI Agents MCP > Ollama lets you run local LLMs through your AI agent. You can generate text, chat with multimodal models, and create embeddings for RAG pipelines. It connects your local model library to clients like Claude or Cursor, giving you full control over which models your agent uses without sending data to the cloud. ## Overview - **Category:** developer-tools - **Price:** Free - **Endpoint:** https://edge.vinkius.com/vk_preview_hYpu67YLXAzJ2w17TRhdjXKZcyynLB0AY9L4fYBJ/mcp - **Tags:** llm, ollama, gemma, gpt-oss, qwen, embeddings, openai-compatible, multimodal ## Description You want to use your own models instead of relying on a single cloud provider. This MCP acts as the bridge between your local Ollama setup and your favorite AI client. Once you're connected, you can ask your agent to run specific models like Llama, Qwen, or Gemma to handle your tasks. It handles everything from standard chat to complex vision-based queries. If you're building a RAG system, you can use it to turn your data into vector embeddings without jumping between different tools. You can also peek under the hood to see which models are currently in your VRAM or check the specific architecture of a model before you commit to it. It even supports OpenAI-compatible endpoints, so if you have existing workflows, they'll just work. It's one of the most practical ways to bring your own models into the fold, and it's easy to manage through the Vinkius catalog. You don't have to worry about your data leaving your machine or paying per-token fees for every request. You get the speed of local execution with the ease of a chat interface that feels like a natural extension of your current setup. Whether you're testing a new quantization or just trying to keep your private documents private, this connection gives you the freedom to choose your own weights. You can also use it to check your local version for troubleshooting when things don't behave as expected. ## Tools ### chat Send multimodal messages and get structured JSON responses from your models. It handles images and tool calling in one go. ### generate_embeddings Generate vector embeddings from single or multiple text inputs. This is perfect for building your own RAG pipelines. ### generate Generate text completions with full control over parameters like seed and temperature. It supports structured JSON outputs too. ### get_version Retrieve the version of your local environment. Use this to ensure compatibility with your current tools. ### list_models Get a full list of all models you have downloaded. Use this to see what's ready to run. ### list_running_models See which models are currently loaded in your VRAM. It shows the model size and when they'll be unloaded. ### openai_chat_completions Generate chat completions using an OpenAI-compatible endpoint. It supports vision and tool calling. ### openai_completions Use the OpenAI-compatible completion endpoint for your existing workflows. It supports various stop sequences. ### openai_embeddings Generate embeddings via an OpenAI-compatible endpoint. It's a drop-in replacement for standard embedding calls. ### openai_list_models Get model IDs and ownership info using the OpenAI-compatible list endpoint. This is great for existing SDK users. ### openai_responses Use the OpenAI-compatible Responses API to generate full responses. It works with your existing OpenAI-style logic. ### show_model_details Get the deep specs for a specific model. You can check parameters, quantization, and tokenizer settings here. ## Prompt Examples **Prompt:** ``` Can you list my local models and tell me which ones can see images? ``` **Response:** ``` I've checked your local library. You have 12 models available. Here are the ones that support vision: | Model Name | Parameters | Capabilities | | :--- | :--- | :--- | | **gemma4** | 8B | Completion + Vision | | **qwen3-vl:8b** | 8B | Completion + Vision + Tools | Would you like to see the specific architecture for any of these? ``` **Prompt:** ``` Create a vector embedding for this paragraph: 'The quick brown fox jumps over the lazy dog'. ``` **Response:** ``` I've generated the vector embedding for that text using the default model. **Result:** - **Dimensions:** 768 - **Status:** Success - **Tokens Processed:** 8 I can also do this for a batch of text if you have a list ready. ``` **Prompt:** ``` Check my VRAM and tell me if I have room to load a larger model. ``` **Response:** ``` Here is your current memory usage: - **Loaded Models:** 2 - **Total VRAM Usage:** 12GB / 16GB - **Next Unload Time:** 4 hours You have about 4GB of VRAM free. You might be able to fit a smaller 7B model, but a 30B model would likely exceed your limit. ``` ## Capabilities ### Run local LLMs Use models like Llama or Qwen directly through your agent for private inference. ### Generate vector embeddings Convert text into vectors for use in similarity matching or RAG pipelines. ### Chat with vision models Send images to multimodal models for analysis and description. ### Monitor VRAM usage See which models are loaded in your memory and how much space they're taking. ### Inspect model specs Get detailed information on parameters, quantization, and tokenizer settings. ### Access OpenAI endpoints Use chat and completion endpoints that mirror the standard OpenAI format. ## Use Cases ### Building a private RAG You need to index internal docs. Use generate_embeddings to create vectors and store them in your private database. ### Testing local weights You want to see if Qwen handles your specific task better than Llama. Use show_model_details to check specs first. ### Vision-based analysis You need to describe images in a batch. Send the files to a multimodal model using the chat tool. ### Managing hardware limits Your agent is slow. Use list_running_models to see if your VRAM is maxed out and unload old models. ## Benefits - Run models locally to keep your data private and off the cloud by using your own weights. - Build RAG systems without extra middleware by using the generate_embeddings tool for your data. - Manage your hardware limits effectively by checking VRAM usage with list_running_models. - Get multimodal capabilities by sending base64 images to vision models through the chat tool. - Maintain your existing code logic by using the various openai_compatible endpoints. - Control your output format precisely using structured JSON schemas in the generate tool. ## How It Works The bottom line is you get a direct line to your local model library without any extra coding. 1. Subscribe to this MCP on Vinkius. 2. Enter your Ollama API key from your account settings. 3. Start calling models directly from Claude, Cursor, or Windsurf. ## Frequently Asked Questions **How do I get my Ollama API key?** Visit the [Ollama API keys page](https://ollama.com/settings/keys) to create or manage your API key. Once created, copy it and paste it into the API key field in the setup wizard. The key is used as a Bearer token in the Authorization header for all requests to https://ollama.com/api. API keys don't currently expire, but you can revoke them at any time. **Which models are available through this MCP?** All models available on ollama.com are accessible, including Gemma, GPT-OSS, Qwen, Llama, and many others. Use `list_models` to see all available models, and `show_model_details` to inspect a model's capabilities (completion, vision, tools), parameter size, quantization level, and architecture details. **Can my AI use vision models to analyze images?** Yes! The `generate` and `chat` tools support image inputs as base64-encoded strings. Use a vision-capable model (check with `show_model_details` — look for 'vision' in capabilities) and pass images in the request. The OpenAI-compatible `openai_chat_completions` tool also supports image_url content parts for vision models. **What is the difference between native and OpenAI-compatible endpoints?** The native Ollama endpoints (`generate`, `chat`, `embed`) offer the full feature set including thinking mode, keep_alive control, and detailed response metadata (token counts, durations). The OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, `/v1/responses`) provide drop-in compatibility for existing applications built with the OpenAI SDK — they use the same request/response format as OpenAI's API. **Does this MCP support tool calling (function calling)?** Yes. The `chat` tool supports tool calling — pass a JSON array of tool definitions in the `tools_json` parameter. The model will return tool calls in the response when it determines a function should be invoked. The OpenAI-compatible `openai_chat_completions` tool also supports tools via the `tools_json` parameter. **Can I generate structured JSON output?** Yes. Both `generate` and `chat` tools support structured outputs via the `format_json` parameter. You can pass the string 'json' for basic JSON mode, or a full JSON schema object to enforce a specific output structure. The OpenAI-compatible endpoints also support `response_format` for structured output. **What is the base URL for API calls?** All API calls are sent to `https://ollama.com/api` with the `Authorization: Bearer YOUR_API_KEY` header. The engine automatically appends the correct path for each endpoint (e.g. `/generate`, `/chat`, `/v1/chat/completions`). This MCP targets the Ollama cloud API — for local Ollama instances, the base URL would be `http://localhost:11434/api` (no API key required).