MinHash Text Deduplicator Connector for AI agents.
3 live capabilities
Find and group near-duplicate text in large datasets
Waiting for input…
Why people use MinHash Text Deduplicator
MinHash Text Deduplicator for cleaning messy text datasets
This MCP changes the workflow by moving the heavy lifting to your agent. Instead of manual checking, you just tell your agent to find the clusters or check for a duplicate. You get structured, mathematical results back instantly, turning hours of scanning into seconds of processing.
What Vinkius changes
You get a mathematical way to find redundant text without writing a single line of Python.
Use it from Claude, ChatGPT, Cursor or another AI client you already have.
One account · 6,100+ Connectors
- Real-world use case 01
Cleaning scraped web data
A data engineer has a massive list of scraped news articles.
- Real-world use case 02
Preventing SEO cannibalization
A content manager wants to ensure their blog isn't competing with itself.
- Real-world use case 03
Deduplicating customer feedback
A researcher has thousands of survey responses.
Complete set · 3capabilities
The complete MinHash Text Deduplicator capability set.
These are the exact actions your AI can choose when you ask it to work with MinHash Text Deduplicator.
01—03
3 capabilities in this set.
Part of 3 available through MinHash Text Deduplicator.
- 01 Capability
Identify duplicate clusters
Groups similar texts into sets based on a similarity threshold. It's perfect for organizing messy data into logical groups.
- 02 Capability
Check is duplicate
Checks if a specific text is a near-duplicate of anything in your existing library. Use this to prevent redundant entries.
- 03 Capability
Compute similarity matrix
Generates a full comparison of how all provided texts relate to one another. This gives you a complete view of your dataset's overlap.
Set up in minutes
One URL. Then ask MinHash Text Deduplicator to work.
Claude and ChatGPT only need the Connector URL. Copy it once, add it in settings, and use MinHash Text Deduplicator from the conversation.
Choose your client
Live previewAdvanced clients IDE · CLI
Claude · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp - Step 01
Open Connectors
In Claude Web or Claude Desktop, open Settings and choose Connectors.
- Step 02
Add the URL
Choose Add custom connector, name it MinHash Text Deduplicator, and paste the URL above.
- Step 03
Turn it on in chat
Select +, open Connectors, and enable MinHash Text Deduplicator for the conversation.
ChatGPT · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp - Step 01
Open MCP settings
On desktop, open Settings and MCP servers. On web, open your workspace app or connector settings.
- Step 02
Add the URL
Choose Add server with Streamable HTTP, or create a custom MCP app, then paste the MinHash Text Deduplicator URL.
- Step 03
Save and start
Save the connection and enable MinHash Text Deduplicator in your conversation. Desktop may ask you to restart once.
Cursor · IDE configuration
Advanced setup
{
"mcpServers": {
"minhash-text-deduplicator": {
"url": "https://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp"
}
}
}- Step 01
Open MCP Settings
Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows/Linux) → search "MCP Settings"
- Step 02
Add the server config
Paste the JSON configuration above into the mcp.json file that opens
- Step 03
Save the file
Cursor will automatically detect the new Connector
- Step 04
Start using MinHash Text Deduplicator
Open Agent mode in chat and ask: "Using MinHash Text Deduplicator, help me...". 3 tools available
VS Code Copilot · IDE configuration
Advanced setup
{
"mcpServers": {
"minhash-text-deduplicator": {
"url": "https://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp"
}
}
}- Step 01
Create MCP config
Create a .vscode/mcp.json file in your project root
- Step 02
Add the server config
Paste the JSON configuration above
- Step 03
Enable Agent mode
Open GitHub Copilot Chat and switch to Agent mode using the dropdown
- Step 04
Start using MinHash Text Deduplicator
Ask Copilot: "Using MinHash Text Deduplicator, help me...". 3 tools available
Windsurf · IDE configuration
Advanced setup
{
"mcpServers": {
"minhash-text-deduplicator": {
"url": "https://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp"
}
}
}- Step 01
Open MCP Settings
Go to Settings → MCP Configuration or press Cmd+Shift+P and search "MCP"
- Step 02
Add the server
Paste the JSON configuration above into mcp_config.json
- Step 03
Save and reload
Windsurf will detect the new server automatically
- Step 04
Start using MinHash Text Deduplicator
Open Cascade and ask: "Using MinHash Text Deduplicator, help me...". 3 tools available
Cline · IDE configuration
Advanced setup
{
"mcpServers": {
"minhash-text-deduplicator": {
"url": "https://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp"
}
}
}- Step 01
Open Cline MCP Settings
Click the Connectors icon in the Cline sidebar panel
- Step 02
Add remote server
Click "Add Connector" and paste the configuration above
- Step 03
Enable the server
Toggle the server switch to ON
- Step 04
Start using MinHash Text Deduplicator
Ask Cline: "Using MinHash Text Deduplicator, help me...". 3 tools available
Claude Code · Terminal command
Advanced setup
claude mcp add minhash-text-deduplicator --transport http "https://edge.vinkius.com/vk_preview_Hrwd2eQSW1CKLVQMLfqo9vh7UJ78eowNZ6AnF9OF/mcp"- Step 01
Install Claude Code
Run npm install -g @anthropic-ai/claude-code if not already installed
- Step 02
Add the Connector
Run the command above in your terminal
- Step 03
Verify the connection
Run claude mcp to list connected servers, or type /mcp inside a session
- Step 04
Start using MinHash Text Deduplicator
Ask Claude: "Using MinHash Text Deduplicator, show me...". 3 tools are ready
Where the request belongs
Work MinHash Text Deduplicator can move forward.
Data engineers and content managers who are tired of dealing with messy, repetitive datasets and need a way to clean them up quickly.
Data Engineer
Cleaning up scraped web data or deduplicating large text corpora for machine learning training.
Content Strategist
Auditing massive content libraries to find repetitive articles or SEO cannibalization issues.
Research Analyst
Filtering through large sets of survey responses or academic papers to remove redundant entries.
Build the capability set
Add more capabilities.
Each Connector adds new actions and data without changing how you work.
Browse Connectors
Regex from Examples
Generate ranked regular expression candidates from input and extraction pairs.
Phonetic Algorithm Matcher
Generate precise phonetic codes and compare string similarity using Soundex, Metaphone, and Double Metaphone.
Tokenization Normalizer
Resolves tokenization drift by normalizing text to match specific LLM tokenizer profiles.
OBV Calculator
Deterministic On-Balance Volume (OBV) analysis engine.
X Ads (Twitter)
Connect your X Ads account to any AI agent. audit campaigns, analyze line item performance, and pull engagement reports through natural conversation.
Box
Store, share, and collaborate on files securely with enterprise-grade cloud content management and governance controls.
Bring your own AI
Change the model, client or framework. Keep MinHash Text Deduplicator connected.
-
Claude -
ChatGPT -
Gemini -
Cursor -
VS Code -
Windsurf -
ZCode -
Cline -
Zed -
Continue -
Kiro -
Roo Code -
Zencoder -
Goose -
Void -
Augment Code -
Amp -
Qodo -
Tabnine -
Pieces -
Sourcegraph Cody -
JetBrains -
Warp -
Amazon Q -
Antigravity -
BoltAI -
Raycast -
Jan -
LM Studio -
AnythingLLM -
Open WebUI -
Msty -
Cherry Studio -
LibreChat -
TypingMind -
Chorus -
5ire -
n8n -
LangChain -
LlamaIndex -
CrewAI -
Vercel AI SDK
Before you connect
Questions about MinHash Text Deduplicator.
The practical details behind the request, access and result.
How does the MinHash Text Deduplicator find near-duplicates?
It uses a mathematical approach called MinHash and shingling to estimate how much two pieces of text overlap. This allows it to find similarities even when the text isn't an exact match.
Can I use MinHash Text Deduplicator to clean my training data?
Yes. It is highly effective for removing redundant or highly similar text entries from large datasets used for machine learning.
Is MinHash Text Deduplicator better than exact match searching?
Yes, if you are looking for 'fuzzy' matches. Exact matching only finds identical strings, while this MCP finds text that is structurally similar.
How do I group similar documents using MinHash Text Deduplicator?
You can instruct your agent to group items into clusters based on a similarity threshold you define.
Can I check a single sentence against a large list with MinHash Text Deduplicator?
Yes, you can perform a binary check to see if a specific piece of text is already represented in your existing collection.
How does the similarity estimation work?
It uses MinHash signatures to estimate the Jaccard similarity between sets of shingles, providing a score between 0.0 and 1.0.
What is the purpose of the `ngramSize` parameter?
The ngramSize determines the length of the shingles. A larger size requires more exact sequence matches to trigger a duplicate detection.
Can I use this to clean up my vector database?
Yes, you can use identify_duplicate_clusters to find redundant entries and remove them to prevent bloat in your vector stores.
One connection away
Give your agent a direct line to MinHash Text Deduplicator.
Connect MinHash Text Deduplicator once. Keep it beside 6,100+ managed Connectors when the next task needs more.
Explore every Connector No credit card required · Free tier available