Together AI MCP. Run open-source LLMs and ML pipelines directly in your agent.

Look, you've got an agent running locally, and it needs muscle that doesn't cost a fortune or tie you down to some closed system. This MCP server connects your setup directly to dozens of open-source models and ML services from the Together AI network. It gives you high-speed inference for big language models like Llama 3 and Mixtral.

You can run everything—from simple prompts to full custom model training runs—all through one API endpoint, right inside your IDE.

When you need to figure out what's available, start with the list_available_models tool. It checks the entire Together AI network and spits back a comprehensive list of every open-source LLM and diffusion model they support. This lets you know exactly which engine—whether it's for natural language processing or image generation—you need to tackle your current task.

For basic text tasks, you've got two ways to go. If you just need a quick answer based on a single prompt, use text_completion. You just send over the specific model ID and the prompt, and it spits out the requested text. But if you’re building a chat interface or running a complex dialogue that requires remembering context, you'll want to run a multi-turn conversation using chat_completion.

This tool handles the entire message history—you pass in the model ID along with an array of previous messages—so your agent doesn't forget what was said two turns ago.

If your goal is building a Retrieval Augmented Generation (RAG) pipeline, you gotta deal with embeddings. Use generate_embeddings to convert any list of raw input strings into numerical vector embeddings. You just specify the embedding model ID, and it handles turning that plain text into vectors ready for database indexing.

This is how you make your documents searchable.

Need some visual flair? If you're working on anything graphical, generate_image uses external diffusion models to create image files. All you gotta do is send over a detailed descriptive text prompt—the more specific you are about what you want the picture to look like, the better it turns out.

For custom AI development, you have two tools managing the entire lifecycle of fine-tuning. First, when your open-source model isn't quite hitting the mark for your niche use case, you kick off a new training run using create_finetune_job. This tool takes two key inputs: the base model ID and the specific dataset you want it to train on.

That starts the whole process.

Once that job is running in the background—and it will take time—you need to know if it's stuck or done. Use list_finetune_jobs to retrieve a list of all your submitted fine-tuning jobs. This lets you check the current status of every single job, giving you visibility into whether they're queued, running, or finished.

It covers everything from checking existing runs to listing them for an audit.