# Portkey MCP

> Portkey helps you manage and audit all your LLM calls in one place. It lets agents monitor costs, track usage across different providers, and enforce budget rules instantly. Instead of jumping between OpenAI dashboards, Anthropic consoles, and cost reports, Portkey centralizes everything. You can ask it to find out who spent too much or export logs for compliance review.

## Overview
- **Category:** ai-frontier
- **Price:** Free
- **Tags:** llm-gateway, ai-observability, token-tracking, cost-management, model-routing, prompt-engineering

## Description

Listen up. You run an AI stack; you don't just connect it—you put it through the Portkey gateway first. That’s your central control point for every token spent and every dollar billed across all your models. Forget jumping between OpenAI dashboards, Anthropic consoles, or whatever billing report gives you a headache. Your agent handles everything here.

### Cost Guardrails and Policy Management

You gotta make sure nobody blows the budget, right? You set up guardrails using `create_policy` to enforce specific usage limits or hard spending caps for entire projects or user groups. If someone starts chewing through tokens too fast, that policy stops them dead in their tracks.

If a project wraps up and those guardrails aren't needed anymore, you use `delete_policy` to clean up the system. You can review every active budget rule or usage ceiling with `list_policies`, seeing exactly who’s hit what consumption rate and which projects are affected by current limits.

### Tracking Usage and Performance Logs

When things go sideways, you need data, not guesses. The agent pulls a summary of recent AI calls using `list_logs`. This gives you IDs, timestamps, how many tokens were used, the latency number, and the cost code for every single interaction.

Need to debug one specific failure? You don't waste time sifting through thousands of entries; you just pull deep info on that exact gateway log entry using `get_log_details`. For compliance or a big-picture review, you generate detailed, filterable log files with `export_logs`, letting you narrow down activity by date range or model type. You can also submit user feedback—a simple like or dislike recorded via `submit_feedback` against any specific response log to track and improve your model quality.

### Auditing the System Infrastructure

You gotta know what’s running under the hood, too. To see which LLM models are actually available through your gateway, you run `list_models`; this tells you if it's chat-enabled or just for embeddings, and what provider handles it.

When you need to audit how requests are being routed—like checking retry rules or fallbacks—you list all active gateway configurations using `list_configs`. For the money side of things, you check out which virtual API keys are in use via `get_virtual_keys`, seeing exactly what provider key is running, its limits, and how much it's consumed.

### Policy Management Details

To get a complete picture of your system’s financial exposure, the agent also lets you list all defined policies using `list_policies`. This shows you the name, the current consumption rate against the cap, and which users are tied to that rule. You can manage these rules by setting them up with `create_policy` or removing them completely with `delete_policy`.

## Tools

### create_policy
Sets up a new budget or usage limit for AI gateway access on specific projects or user groups.

### delete_policy
Removes an existing cost or usage policy when a project is finished and no longer needs guardrails.

### export_logs
Generates a downloadable, filterable log file for compliance reviews or offline cost analysis of AI activity.

### get_log_details
Pulls deep information about one specific gateway log entry when you need to debug a failed LLM interaction.

### get_virtual_keys
Lists all virtual API keys used by the system, detailing their associated provider, usage limits, and current consumption status.

### list_configs
Retrieves a list of all active gateway configurations, allowing you to audit how LLM requests are currently being routed or handled.

### list_logs
Gets a paginated summary of recent AI calls, showing IDs, timestamps, token usage, latency, and the final cost code.

### list_models
Checks which LLM models are currently supported by your gateway, including their provider (e.g., OpenAI) and capabilities (chat, embeddings).

### list_policies
Lists all active budget rules and usage policies, showing the name, current consumption rate, and affected users.

### submit_feedback
Records user feedback (like/dislike) against a specific AI response log to help train or monitor model quality.

## Prompt Examples

**Prompt:** 
```
Show me the most expensive LLM calls from the last 24 hours
```

**Response:** 
```
I'll retrieve recent gateway logs and sort them by cost to identify the top spenders.
```

**Prompt:** 
```
Create a budget policy limiting the Marketing team to $500/month on LLM usage
```

**Response:** 
```
I'll create a policy with a $500 monthly budget target for the Marketing virtual keys.
```

**Prompt:** 
```
Export all logs from last week for our compliance audit
```

**Response:** 
```
I'll trigger a log export for the last 7 days in JSON format for your records.
```

## Capabilities

### Enforce Spending Policies
You create, review, and delete budget guardrails using tools like `create_policy` and `list_policies`, ensuring no team exceeds its allotted spending or token cap.

### Audit API Key Usage
The agent lists all virtual keys (`get_virtual_keys`), showing exactly which provider key is being used, how much it's consumed, and what limits are in place.

### Track Call Performance and Cost
You pull recent log data using `list_logs`, getting timestamps, model names, token usage, latency metrics, and the associated cost for every single call.

### Generate Audit Reports
The system exports detailed log files (`export_logs`) filtered by date range or model, generating structured data ready for compliance checks.

### Manage Gateway Settings
You can list and review all active gateway configurations (`list_configs`) to check how requests are being routed—like retry rules or fallbacks.

## Use Cases

### The unexpected cost spike
A team runs a new feature, and suddenly the bill jumps 30%. Instead of panicking and calling Ops, your agent checks `list_logs` to pinpoint the exact model and user responsible. Then, it uses `create_policy` to cap that spending immediately.

### The quarterly compliance audit
Compliance needs a full log of all LLM interactions from Q2. The agent runs `export_logs`, filtering by date and user ID, creating a single, auditable JSON file instantly. Done in minutes, not days.

### Debugging a failed feature rollout
A key workflow fails randomly. You use `list_logs` to find the approximate time of failure. Then you feed that ID into `get_log_details`. The output tells you if it was a rate limit error or a bad configuration, saving hours of guesswork.

### Onboarding a new department
The Sales team needs access but shouldn't spend more than $1k/month. Your agent uses `create_policy` to apply that budget limit specifically to the virtual keys associated with their project, keeping costs controlled from day one.

## Benefits

- **Total Visibility:** Use `list_logs` to see a consolidated feed of every call—latency, tokens, cost, status code. You don't have to check 5 different provider dashboards to know what happened.
- **Cost Guardrails:** Set and review budgets instantly. If the Marketing team overspends, your agent can enforce it using `create_policy`, saving you from unexpected bills.
- **Compliance Ready:** When auditors ask for proof of usage, run `export_logs`. You get structured data (JSON) that's ready to submit, not just a screen capture.
- **Key Management Audit:** Never worry about forgotten keys again. Run `get_virtual_keys` to see every active API key and its remaining quota at a glance.
- **Performance Debugging:** If an LLM call is slow or failing, use `list_logs` then grab the ID and run `get_log_details`. You instantly get the stack trace or error code.

## How It Works

The bottom line is that natural language commands replace clicking through ten different vendor dashboards to manage LLM usage.

1. First, your agent connects using the Portkey API key. You don't touch any dashboards.
2. Next, you tell your agent what to check—like 'Show me all policies for Marketing.'
3. Finally, the agent calls the specific tool (e.g., `list_policies`) and returns a clean, actionable summary.

## Frequently Asked Questions

**How do I use create_policy with Portkey?**
You tell your agent to create a policy, specifying the budget limit (USD or tokens) and which virtual keys or users should be restricted. It returns the full details of the new guardrail.

**Can I check all my LLM usage with list_logs?**
Yes. `list_logs` retrieves a summary of recent calls from *all* connected models. You get the log ID, timestamp, token count, latency, and cost code for each entry.

**What is the best tool to check model support?**
Use `list_models`. This checks which LLM names are currently routable through Portkey and what endpoints (like chat or embeddings) they support right now.

**How do I audit API key usage with get_virtual_keys?**
The agent runs `get_virtual_keys`. This lists every virtual key, showing its underlying provider, current usage against the limit, and what policies are attached.

**Do I need to use export_logs for compliance?**
If you need an official record—something that needs to be filed or analyzed offline—yes. `export_logs` generates a structured, downloadable log file (like JSON) ready for audit.

**When an LLM call fails, how can I use get_log_details to debug the issue?**
It provides deep visibility into a single log entry. You input the specific Log ID and retrieve the full request/response payload, including detailed error codes or latency bottlenecks.

**What does list_configs show me about LLM routing reliability?**
This tool displays current gateway configurations. You can review active retry policies, fallback models, and cache settings that determine how requests are routed when a primary model fails or slows down.

**How do I use submit_feedback to improve model quality?**
You must provide the specific Log ID along with your rating (LIKE/DISLIKE). This action collects data points that help refine and measure user satisfaction for future model training cycles.

**Which LLM providers does Portkey support?**
Portkey supports 1,600+ LLMs including OpenAI, Anthropic, Google, Mistral, Azure OpenAI, AWS Bedrock, Cohere, Hugging Face, and many more. Use the list_models tool to see the full catalog available via your gateway.

**How does Portkey help control AI costs?**
Portkey provides granular visibility into token usage, latency, and costs per model, team, or virtual key. You can create budget policies with hard limits to prevent runaway spending. The gateway also supports caching to reduce duplicate calls and fallbacks to cheaper models when appropriate.

**Can I track feedback on AI responses?**
Yes! Portkey allows you to submit Like/Dislike feedback for any logged LLM call. This data helps improve model selection, evaluate agent performance, and build RLHF datasets for fine-tuning.