# Together AI MCP for AI Agents AI Agent Connect > Together AI MCP lets you run open-source models like Llama and Mixtral directly through your AI agent. You can use it to generate text, create embeddings for vector databases, fine-tune models on your own datasets, and even generate images from text prompts. It connects your agent to high-speed inference infrastructure so you do not have to manage the hardware yourself. ## Overview - **Category:** ai-frontier - **Price:** Free - **Endpoint:** https://edge.vinkius.com/vk_preview_uwxFbBTrWtpKdtNP0Ee38Zmxxir6fie6FBKXtn0o/ai-agent-connect - **Tags:** llm, model-inference, fine-tuning, open-source-ai, machine-learning, api-deployment ## Description This Connector connects your AI agent to the Together AI infrastructure so you can run high-performance open-source models without managing your own hardware. Instead of jumping between different websites or manual CLI commands to test models, you can do it all from your chat interface. You can query Llama or Mixtral for complex reasoning, generate rich embeddings for your search systems, or start a custom fine-tuning job just by describing the dataset. It is a way to bring production-grade inference into your local workflow. If you are looking for a reliable way to manage these tasks, Vinkius makes it easy to plug this into your existing setup. You get sub-second responses and the ability to switch between different models instantly based on the task at hand, whether you are building a chatbot or preparing data for a RAG pipeline. ## Tools ### chat_completion Get a conversational response from a model like Mixtral or Llama. It handles multi-turn chat logic easily. ### text_completion Generate a direct text response based on a single prompt. This is useful for simple completion tasks. ### create_finetune_job Start a new training run using a specific base model and dataset. It automates the provisioning of training resources. ### generate_embeddings Turn a list of text strings into vector embeddings for your database. This is perfect for building search tools. ### generate_image Create an image from a text prompt using diffusion models. It lets your agent create visual content on the fly. ### list_finetune_jobs See all your active and finished fine-tuning jobs. This helps you keep track of your training history. ### list_available_models See every model currently hosted on the Together AI network. Use this to find the right engine for your task. ## Prompt Examples **Prompt:** ``` What models can I use for a fast chat bot? ``` **Response:** ``` I've checked the current offerings and here are some of the best models for chat tasks: | Model | Best For | | :--- | :--- | | **Llama 3** | General purpose reasoning and chat | | **Mixtral** | High-performance complex tasks | | **Gemma** | Fast, efficient responses | Would you like me to run a test prompt through one of these? ``` **Prompt:** ``` Generate embeddings for these three sentences: 'The sky is blue', 'The grass is green', 'The sun is hot'. ``` **Response:** ``` I've generated the vector embeddings for those sentences. Here is a sample of the first result: **Sentence:** The sky is blue **Vector (first 5 dims):** `[-0.0124, 0.0411, 0.0812, -0.005, 0.021]` I can process a larger list of strings for you if you provide them in a JSON array. ``` **Prompt:** ``` Check my current fine-tuning jobs. ``` **Response:** ``` Here is the current status of your training jobs: * **Job ID: 8821** | Status: **Completed** | Model: Llama-3-8B * **Job ID: 9942** | Status: **Running** | Model: Mixtral-8x7B * **Job ID: 1023** | Status: **Queued** | Model: Llama-3-70B Let me know if you want more details on a specific job. ``` ## Capabilities ### Run chat completions Get conversational responses from models like Llama and Mixtral directly in your chat. ### Generate vector embeddings Turn text strings into numerical vectors for use in your search or recommendation systems. ### Start custom fine-tuning Provision new training runs on specific base models using your own dataset files. ### Create images from text Generate visual content by providing a descriptive prompt to diffusion models. ### Browse available models See the full list of open-source models currently hosted on the Together AI network. ### Track training progress Check the status of your active and completed fine-tuning jobs. ## Use Cases ### Comparing model performance A developer asks their agent to run the same prompt through Llama 3 and Mixtral to see which model handles complex reasoning better. ### Building a RAG system An engineer asks the agent to turn 50 product descriptions into embeddings for a vector store using the generate_embeddings tool. ### Training a niche model An ML specialist uses create_finetune_job to train a model on internal company documents for a private knowledge base. ### Quick image generation A content creator asks the agent to make a 3D render style image for a website header using a diffusion model. ## Benefits - Run Llama and Mixtral models instantly without needing to manage your own GPU hardware or drivers. - Create embeddings for your vector database using the generate_embeddings tool to power your search features. - Launch custom training runs for your specific data with create_finetune_job to build specialized models. - Test multiple open-source models side-by-side to find the best fit for your application's needs. - Generate images from text descriptions directly through your agent's chat for quick visual content creation. - Monitor your training progress in real-time using the list_finetune_jobs tool without leaving your workspace. ## How It Works The bottom line is you get high-speed inference for open-source models without needing to manage any GPU infrastructure. 1. Get your developer API key from your Together AI dashboard. 2. Add the key to your AI agent configuration in your preferred client. 3. Ask your agent to run a model, generate embeddings, or start a training job. ## Frequently Asked Questions **How do I run Llama 3 models with the Together AI MCP?** You can run Llama 3 by simply asking your agent to perform a chat or text completion task. The Connector handles the connection to the high-speed inference hardware for you. **Can I use Together AI to generate images?** Yes, the Connector includes a tool to generate images from text descriptions. You just need to provide a descriptive prompt to your agent. **How do I create custom embeddings for my database?** You can ask your agent to generate embeddings for a list of strings. It will return the vector data you need to populate your search or recommendation database. **Can I fine-tune models using the Together AI MCP?** Yes, you can start new fine-tuning jobs by providing a base model and a dataset file through your chat interface. You can also check the status of these jobs. **Is Together AI good for high-speed inference?** Together AI is known for its high-speed inference, providing sub-second responses for models like Llama and Mixtral. **How do I see what models are available on Together AI?** You can ask your agent to list the available models. It will fetch the current list of open-source models hosted on the network. **Where do I obtain my Together AI API Key?** Log in to the developer portal via `api.together.xyz/settings/api-keys`. If you do not have an existing key, click **Create API Key**. This token enables the execution of remote inferences spanning their hosted clusters securely. **Do I have to pay to use Together models through the agent?** Yes. This connector simply routes your instructions to Together AI. Any tokens consumed during chat completion, embeddings, images generation, or fine-tuning workloads are billed directly to your registered Together AI account balance according to their official compute pricing models. **Can I access free models on Together AI?** Yes! Together AI frequently offers free tiers for certain open-source models intended for experimentation and research. You can query these directly from your agent without depleting your account balance, though specific free-tier rate limits will apply.