• PENDING
  • RUNNING
  • COMPLETED
  • FAILED

How to process large document batches with your AI

Connect Voyage AI to your agent to handle massive vectorization tasks in the background. You trigger the work and let the API run the heavy lifting.

Ask AI about this page

Short answer

How can I vectorize huge datasets without hitting rate limits?

You use background batch jobs to process files in bulk. Your AI triggers the job, monitors the status, and pulls the results once the API finishes the work.

Where the data goes.

The outcomes of batch processing.

Once the job finishes, your data lands in one of these states.

PENDING

Job in queue

The API has accepted your request. Your AI waits for the batch to move from the queue to active processing.

RUNNING

Active processing

The batch is currently being vectorized. Your AI can check the status periodically to see how much progress has been made.

COMPLETED

Vectors ready

The job finished successfully. Your AI can now access the completed embeddings for your downstream tasks.

FAILED

Human intervention

Something went wrong with the batch. You'll need to check the error logs and decide how to fix the source files.

The workflow

What your AI does when the data arrives.

The AI manages the lifecycle of the batch job so you don't have to manually poll the API.

  1. Prepare the data

    The AI sends your raw documents to the Voyage AI storage to prepare them for embedding.

    upload_file

  2. Start the job

    The AI tells the API to begin processing the uploaded files as a single large batch.

    create_batch

  3. Track progress

    The AI checks the status of the job to see if it is still running or if it has finished.

    get_batch

  4. Review history

    The AI looks through previous jobs to find specific batch IDs or check recent activity.

    list_batches

Try it

Copy these to start.

Use these prompts to kick off your first batch job.

Starting points

Swap out the bracketed text for your actual file names or batch IDs.

Voyage AI Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_x7lwFbbAdz5StvxoOu2Tkh9eDMu9Uh5i3GKObLMP/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Voyage AI capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "voyage-ai-ai-embeddings-api-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_x7lwFbbAdz5StvxoOu2Tkh9eDMu9Uh5i3GKObLMP/mcp"
    }
  }
}
Copy into chat04

  • Upload this PDF and start a new batch job for it.

  • Check the status of my last batch job.

  • List all the batch jobs I've run recently.

  • Tell me if batch job [ID] is finished yet.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Voyage AI once, then ask.

Just link your account once. Your credentials stay encrypted, and you can immediately start managing batches through your AI client.

Connect Voyage AI to your AI

FAQ

How this task behaves.

  • 01

    Can the AI delete my uploaded files?

    No. The current tools only allow for uploading and managing batch jobs.

  • 02

    Does the AI wait for the batch to finish before responding?

    It depends on how you ask. You can ask it to check the status, or you can ask it to monitor the job until it's done.

  • 03

    Can I change the embedding model mid-batch?

    No. Once a batch job is created, the parameters are set for that specific job.

  • 04

    Can the AI see my other API activity?

    No. The AI only has access to the specific actions provided by the Voyage AI tools.

  • 05

    How does the AI know when a job fails?

    The AI uses the batch status tools to check for error states and will report them to you.

  • More questions about Voyage AI? The Connector page answers them. See everything the Voyage AI Connector can do

Connect Voyage AI to Claude, Cursor, ChatGPT & more