# Speculative Decoding Speedup Calculator AI Agent Connect

> Calculate efficiency gains and throughput improvements for speculative decoding strategies.

## Overview
- **Category:** optimization
- **Price:** Free
- **Endpoint:** https://edge.vinkius.com/vk_preview_R6zM4edJsq4MqH1NEytgBARQUwtouiYeWtp5FpDR/ai-agent-connect
- **Tags:** speculative-decoding, llm, inference-optimization, throughput, cost-analysis

## Description

This MCP server provides a deterministic optimization engine to evaluate the performance of speculative decoding in LLM inference. It calculates critical metrics such as speedup ratio, effective throughput, and optimal draft length. Use `calculate_speculative_metrics` to evaluate efficiency, `find_optimal_draft_length` to maximize throughput, and `calculate_economic_impact` to translate performance gains into monetary savings.

## Tools

### calculate_economic_impact
Translates performance gains into monetary savings

### calculate_speculative_metrics
Calculates primary performance indicators (speedup, throughput, and token distribution) based on a specific configuration

### find_optimal_draft_length
Determines the most efficient speculative window size to maximize token throughput

## Prompt Examples

**Prompt:** 
```
Calculate the metrics for a draft model at 50 tokens/s, target at 10 tokens/s, with a 0.7 acceptance rate, draft length of 5, and 20ms overhead.
```

**Response:** 
```
The speedup ratio is 3.5, with 3.5 accepted tokens and 1.5 rejected tokens per window. The effective throughput is 145.2 tokens per second.
```

**Prompt:** 
```
What is the optimal draft length if my draft model is 100 tokens/s and target is 10 tokens/s with 0.6 acceptance rate and 5ms overhead?
```

**Response:** 
```
The optimal draft length for these parameters is 8 tokens.
```

**Prompt:** 
```
How much money do I save if I save 3600 seconds with a speedup and my cost is 0.01 per second?
```

**Response:** 
```
The total cost savings is $36.00.
```

## Frequently Asked Questions

**What is speculative decoding?**
Speculative decoding is an acceleration technique where a small draft model predicts tokens that a larger target model verifies in parallel, increasing overall throughput.

**How do I find the best draft length?**
You can use the `find_optimal_draft_length` tool to automatically test window sizes from 1 to 20 to find the one that maximizes effective throughput.

**Can I calculate the cost savings of this optimization?**
Yes, the `calculate_economic_impact` tool allows you to convert time saved during inference into monetary savings based on your operational costs.