# Duplicate Line Detector AI Agent Connect

> Identify redundant lines in text and calculate token savings.

## Overview
- **Category:** productivity
- **Price:** Free
- **Endpoint:** https://edge.vinkius.com/vk_preview_yDAB8XEDyIoWhlpd7GkvaYAhjoQ8W5fjH1Z8xePT/ai-agent-connect
- **Tags:** deduplication, text-analysis, token-optimization, redundancy, efficiency

## Description

This MCP server provides tools to analyze multi-line text for redundancy. Use `analyze_text_redundancy` to get a full redundancy profile, including duplicate counts and estimated token savings. Use `get_line_frequency_map` to see which specific lines are repeated, or `summarize_deduplication_impact` to calculate the efficiency gain from removing duplicates.

## Tools

### get_line_frequency_map
Maps each unique line to its frequency of occurrence

### summarize_deduplication_impact
Summarizes the efficiency gain and redundancy percentage

### analyze_text_redundancy
Analyzes the redundancy profile of a multi-line text block

## Prompt Examples

**Prompt:** 
```
Analyze this text for redundancy: 'Hello\nWorld\nHello\nAgain\nWorld'
```

**Response:** 
```
The text contains 5 total lines, with 3 unique lines and 2 duplicate lines. The duplicate ratio is 0.4, and you would save approximately 2 tokens if deduplicated.
```

**Prompt:** 
```
Which lines are repeated in: 'apple\nbanana\napple\ncherry\nbanana\napple'
```

**Response:** 
```
The line 'apple' appears 3 times and the line 'banana' appears 2 times.
```

**Prompt:** 
```
What is the efficiency gain for a text with 100 lines and 25 unique lines?
```

**Response:** 
```
The efficiency gain is 0.25.
```

## Frequently Asked Questions

**How does the tool calculate token savings?**
The tool estimates token savings by multiplying the number of duplicate lines by the average number of tokens per line, using a standard of four characters per token.

**Does the tool ignore whitespace?**
Yes, all lines are trimmed of leading and trailing whitespace before being evaluated for uniqueness.

**What is the difference between the redundancy profile and the frequency map?**
The redundancy profile provides high-level metrics like duplicate ratios, while the frequency map provides a specific breakdown of every repeated line and its count.