• AUDIO
  • LISTENING
  • COMMUNICATION
  • HUMAN

How to build realtime voice agents with your AI

Connect Inworld AI to your agent to enable spoken dialogue. Your agent moves from text-only responses to fluid, multimodal conversations.

Ask AI about this page

Short answer

How do I make my AI talk in real time?

You link Inworld AI to your agent to bridge the gap between text processing and spoken audio. This setup allows your agent to listen to audio, turn it into text, and respond with synthesized speech through a live call. You get a complete loop for verbal interaction.

The outcomes

What your agent achieves

Connecting Inworld AI enables these specific interaction modes.

AUDIO

Spoken responses

The agent uses speech synthesis to turn text into natural audio. You get a voice that sounds human rather than robotic.

LISTENING

Voice recognition

The agent processes incoming audio to understand what you said. It converts speech to text so it can reason about the input.

COMMUNICATION

Live voice calls

The agent establishes a direct audio link. This enables a continuous conversation instead of a turn-based text exchange.

HUMAN

Manual control

The AI handles the audio processing and speech generation. You stay in charge of the high-level logic and personality.

The workflow

What your AI does when the voice request arrives.

The agent handles the heavy lifting of audio processing so you can focus on the conversation logic.

  1. Listen

    The agent captures the incoming audio stream and converts it into text that it can understand.

    transcribe_audio

  2. Connect

    The agent opens a dedicated channel for the verbal interaction to ensure the connection stays active.

    create_realtime_call

  3. Speak

    The agent generates a continuous stream of audio based on its text response.

    synthesize_speech_stream

  4. Respond

    The agent uses the transcribed text to formulate a logical reply before turning it back into speech.

    transcribe_audiosynthesize_speech_stream

Try it

Copy these to start.

Use these prompts to trigger the voice capabilities.

Starting points

These are starting points. Swap the bracketed text for your specific character or scenario.

Inworld AI Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_3lkEwqqakpRf4XlQXHb3G8jJBe7cyrieiM1PCdg3/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Inworld AI capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "inworld-ai-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_3lkEwqqakpRf4XlQXHb3G8jJBe7cyrieiM1PCdg3/mcp"
    }
  }
}
Copy into chat04

  • Start a realtime call with a character acting as a [persona].

  • Listen to my audio input and respond using speech synthesis.

  • Create a realtime call to talk to the [character name] persona.

  • Transcribe my audio so we can have a spoken conversation.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Inworld AI once, then ask.

You only need to link your account once. Your credentials stay encrypted, and you can start talking to your agent immediately.

Connect Inworld AI to your AI

FAQ

How the voice connection behaves

  • 01

    Can the AI hear background noise?

    The agent uses the transcription tool to process audio, but it is designed to focus on the primary speech input.

  • 02

    Does the AI decide how it sounds?

    The agent uses the synthesis tool to generate audio, but the specific voice profile is determined by your Inworld AI settings.

  • 03

    Can the AI take over my microphone?

    No. The AI uses the tools to process audio streams, but it cannot control your hardware or system settings.

  • 04

    Will the AI delete my previous voice recordings?

    No. The tools only process the audio for transcription and synthesis; they do not have the ability to delete files.

  • 05

    Can the AI initiate a call without me?

    No. You must trigger the call creation through your agent or client.

  • More questions about Inworld AI? The Connector page answers them. See everything the Inworld AI Connector can do

Connect Inworld AI to Claude, Cursor, ChatGPT & more