Ukti Voice API

Speech to text (Prakrit) and text to speech (Birbal) for Indian languages, served from Azure Central India at https://api.uktiai.com. The wire shapes follow the Sarvam Saaras and Bulbul conventions, so existing Sarvam integrations move over with a base-URL change.

Overview

Languages: hi-IN, en-IN, bn-IN, mr-IN, ta-IN, te-IN; STT also accepts unknown (REST) or auto (realtime) to detect the language. Hinglish is returned in Devanagari with English terms in Roman script, following the transcription convention used in training.

Authentication

Every request carries an API key issued by your admin from the console. Keys look like uk_live_… (billable) or uk_test_…. Send it one of two ways:

Authorization: Bearer uk_live_…          # REST
api-subscription-key: uk_live_…          # REST header alternative, and the query parameter on WebSockets
wss://api.uktiai.com/v1/speech-to-text/realtime?…&api-subscription-key=uk_live_…

A key belongs to one account (your tenant) and one tier (starter, pro, business). The account decides which vocabulary the gateway may use and which rate limits apply; a key can never read another account's data. Revoked keys stop working within ten seconds on every replica.

Quickstart

Transcribe a file

curl -sS https://api.uktiai.com/v1/speech-to-text \
  -H "Authorization: Bearer $UKTI_API_KEY" \
  -F "file=@clip.wav" -F language_code=hi-IN

import requests, os
r = requests.post("https://api.uktiai.com/v1/speech-to-text",
    headers={"Authorization": f"Bearer {os.environ['UKTI_API_KEY']}"},
    files={"file": open("clip.wav", "rb")}, data={"language_code": "hi-IN"})
r.raise_for_status(); j = r.json()
print(j["transcript"], j["confidence"], j["engine"])

const fd = new FormData(); fd.append("file", blob, "clip.wav"); fd.append("language_code", "hi-IN");
const r = await fetch("https://api.uktiai.com/v1/speech-to-text", { method: "POST", headers: { Authorization: `Bearer ${key}` }, body: fd });
const j = await r.json(); console.log(j.transcript, j.confidence, j.engine);

Response (abridged):

{"request_id": "req_01…", "transcript": "मेरा नाम अंकुर झावेरी है", "raw_transcript": "…", "language_code": "hi-IN",
 "confidence": 0.93, "words": [{"text": "मेरा", "start": 0.12, "end": 0.38, "confidence": 0.98}, …],
 "engine": "ukti:prakrit-v0", "vocabulary_used": 0, "corrections": []}

Speak a sentence

curl -sS https://api.uktiai.com/v1/text-to-speech \
  -H "Authorization: Bearer $UKTI_API_KEY" -H "content-type: application/json" \
  -d '{"text":"आपकी SIP की अगली किस्त पंद्रह तारीख को है।","language_code":"hi-IN","output_audio_codec":"wav","speech_sample_rate":16000}' \
  | python3 -c 'import sys,json,base64; open("out.wav","wb").write(base64.b64decode(json.load(sys.stdin)["audios"][0]))'

REST transcription

POST /v1/speech-to-text, multipart/form-data.

engine names the model that produced the text (ukti:prakrit-v0) or vendor:sarvam when a language is still routed to the fallback. reason appears only when a transcript is empty on purpose (no_speech, low_confidence, blocklisted, foreign_script, degenerate).

Realtime streaming

Open wss://api.uktiai.com/v1/speech-to-text/realtime with query parameters, send audio as JSON events, receive events. The gateway runs voice activity detection: each utterance yields transcript.partial events while you speak and one transcript.final when it ends. Full schema: WebSocket API.

const ws = new WebSocket(`wss://api.uktiai.com/v1/speech-to-text/realtime?language_code=hi-IN&encoding=linear16&sample_rate=16000&call_id=call_123&api-subscription-key=${key}`);
ws.onmessage = e => { const ev = JSON.parse(e.data);
  if (ev.event === "transcript.partial") render(ev.text, /*interim*/ true);
  if (ev.event === "transcript.final")   render(ev.text, false);          // ev.confidence, ev.words, ev.engine, ev.corrections
};
// every 100 ms: 1,600 samples of 16-bit PCM, base64
ws.send(JSON.stringify({ event: "audio_input", audio: base64Pcm16 }));
// the agent learns the caller's name mid-call:
ws.send(JSON.stringify({ event: "config.update", context_entities: [{ entity_id: "c_456", canonical_text: "अंकुर झावेरी", aliases: ["Ankur Jhaveri"] }] }));
ws.send(JSON.stringify({ event: "end" }));                                  // flushes the last utterance, then session.end

Server events: session.begin (echoes the config and vocabulary: {call, account, terms} counts), vad.speech_start, transcript.partial, vad.speech_end, transcript.final, config.updated, pong, session.end, error (is_fatal tells you whether the socket closes). Client events: audio_input, speech_start / speech_end (manual endpointing), flush, config.update (language_code, prompt, mode, stream_type, context, context_entities), ping, end.

Recognition settings

Live transcription runs a neural voice activity detector (Silero) in front of the model. Four numbers decide what counts as speech and where a sentence ends; they can be set per session in the WebSocket URL, or stored per account so every session of that account starts from them. Precedence: URL parameter, then account setting, then the built-in default. session.begin.config echoes the effective values and settings_applied says where each came from.

curl -sS https://api.uktiai.com/v1/settings -H "Authorization: Bearer $UKTI_API_KEY"              # this key's account
curl -sS -X PUT https://api.uktiai.com/v1/settings -H "Authorization: Bearer $UKTI_API_KEY"   -H "content-type: application/json" -d '{"vad_threshold":0.6,"min_speech_duration_ms":400,"narrator_lock":true}'
# response: {"account":"bachatt-agent","settings":{…effective…},"sources":{"vad_threshold":"account",…},"updated_at":…,"updated_by":"key:…"}

Admins can read and write any account's settings in the console. A session can still override a stored value for itself: …/realtime?language_code=hi-IN&threshold=0.7&silence_duration_ms=800.

Speaker modes

The realtime session takes speakers, in the URL or as an account setting:

Labels are consistent within a session only: S1 is whoever spoke first, up to ten speakers are tracked, and utterances shorter than 1.5 s carry speaker: null. Overlapping speech within one utterance is not separated. session.end reports speakers. config.update {"speakers": "multi"} switches mid-call and resets the speaker memory.

Narrator lock (speakers=single)

With narrator_lock=true the session keeps one voice and drops the others. The first utterance of at least 1.5 s starts a voice profile; a second long utterance from the same voice confirms it (nothing is dropped before confirmation). From then on every utterance's voice is compared with the profile: a match at or above narrator_similarity is delivered and refines the profile; a clear mismatch (0.40 or below) is not delivered, and the session emits instead:

{"event": "transcript.dropped", "utterance_idx": 7, "reason": "other_speaker", "similarity": 0.31, "duration_s": 2.4}

Scores between the two limits are delivered without touching the profile. Utterances too short to judge are delivered. config.update accepts narrator_lock to switch it during a call and narrator_reset: true to forget the profile and enrol whoever speaks next (a transfer to another agent, for example). session.end reports dropped_utterances.

Names and spelling

Prakrit hears Indian names well and can still spell them differently from your records. Three mechanisms, from lightest to strongest:

1. Biasing terms (context): plain strings the decoder is nudged towards. Good for a handful of product names.
2. Call-scoped entities (context_entities): structured records with the canonical spelling, likely alternative spellings and the spoken form. The decoder is biased towards them, the resolver snaps near-matches in the transcript to the canonical text and reports them in corrections, and Birbal pronounces the spoken form. This is the mechanism for customer names.
3. Account vocabulary: tenant-wide terminology (your company, products, funds) stored once by your admin and applied to every request of the account.

REST synthesis

To hear every voice before choosing one, open the console's voice preview: one editable sample sentence per language, a play button per voice, latency and engine shown for each.

POST /v1/text-to-speech with a JSON body; the response carries base64 audio in audios. /v1/text-to-speech/stream returns the raw audio bytes as they are produced.

Hindi and English are synthesised by Birbal; the other four languages fall back to the vendor voice until Birbal speaks them. voice_fallback in the response names the voice actually used when the requested one had no match.

Streaming synthesis

wss://api.uktiai.com/v1/text-to-speech/ws: send a config event (language, voice, codec, sample rate, optional context_entities), then text events as your LLM produces clauses, then flush; audio arrives as binary frames with event messages marking clause boundaries. See the protocol reference.

Call-scoped entities

Bachatt identifies the caller before or during a call and knows the relevant names: the customer and household, the advisor, the funds and banks on the account. Send exactly that shortlist, not your whole customer base. Two ways:

Inline, per request

-F 'context_entities=[{"entity_id":"customer_456","canonical_text":"अंकुर झावेरी","aliases":["Ankur Jhaveri","Ankur Javeri"],"spoken_forms":["अंकुर झावेरी"],"priority":1000}]'

Stored for the call (two hours by default)

curl -sS -X PUT https://api.uktiai.com/v1/calls/call_123/entities \
  -H "Authorization: Bearer $UKTI_API_KEY" -H "content-type: application/json" \
  -d '{"entities":[{"entity_id":"customer_456","type":"person","canonical_text":"अंकुर झावेरी","aliases":["Ankur Jhaveri"],"spoken_forms":["अंकुर झावेरी"],"priority":1000},
                   {"entity_id":"fund_ppfcf","type":"fund","canonical_text":"Parag Parikh Flexi Cap Fund","spoken_forms":["पराग पारिख फ्लेक्सी कैप फंड"]}],
       "ttl_s": 7200}'
# then every STT, realtime and TTS request that carries call_id=call_123 uses the same shortlist

The inline value has three states: absent (use the stored list for call_id), an empty list (use no call entities, for example after a privacy opt-out), a list (replace for this request or, on a realtime session, for the rest of the session). GET and DELETE on the same path read and remove the stored list. Entries expire on their own.

Account vocabulary

Tenant-wide terminology (company names, products, schemes) that applies to every request of the account. Admins manage it in the console or through the admin API. Keep it small: at most ten account entities join a request's shortlist, after the call-scoped ones. It is not the place for customer names.

Vocabulary limits

GPU on demand

The owned engines run on a GPU node that is switched off after 30 minutes without any STT or TTS request and switched back on by the first request that needs it. This keeps the cost of an idle deployment near zero; the price is a cold start.

GET  /v1/gpu        -> {"state":"ready","desired":{"stt":1,"tts":1},"ready":{"stt":true,"tts":true},
                        "last_activity_at":"2026-09-27T05:10:02Z","idle_minutes":30,
                        "idle_shutdown_at":"2026-09-27T05:40:02Z","eta_s":null,"node_present":true}
POST /v1/gpu/wake   -> 202 {"state":"starting","woke":true,"eta_s":600, ...}   # idempotent

While the GPU is off or starting, every STT and TTS request answers 503 gpu_starting with Retry-After: 30 and eta_s in the body; the first such request is what wakes the node, so you never have to call the wake route explicitly. A realtime or streaming-synthesis socket receives {"event":"error","code":"gpu_starting","is_fatal":true,"eta_s":…} and is closed with 1013. The console shows the same state as a pill in the top bar and offers a Wake GPU button.

Languages the rollout flags route to the vendor engine (today Bengali, Marathi, Tamil and Telugu) are served while the GPU is off: they never wake it and never see gpu_starting. Only Hindi and English, served by the owned engines, do.

Two integration patterns: warm up ahead of time by calling POST /v1/gpu/wake when a user opens the screen that will need speech (a call queue, a dictation page), and poll GET /v1/gpu until state is ready; or keep it warm by sending real traffic at least every 30 minutes. Every request counts as activity, including ones refused for other reasons. Administrators can change the idle window (5 to 720 minutes) or force a shutdown from the console.

Engines and routing

GET /v1/models lists prakrit:v1, prakrit:v1-realtime and birbal:v1 with the engine actually serving each: today ukti:prakrit-v0 (whisper-large-v3-turbo adapted on Bachatt's own corpus) and ukti:birbal-v0. Hindi and English are served by the owned engines; Bengali, Marathi, Tamil and Telugu are routed to the Sarvam fallback while the owned models learn them, with the owned STT decoding the same audio in shadow for comparison. The engine field in every response tells you which one answered. GET /readyz reports each dependency, including the ephemeral and durable stores.

Rate limits

A refusal is 429 rate_limit_exceeded_error with x-ratelimit-limit, x-ratelimit-remaining and x-ratelimit-reset (seconds). Limits are per account and shared across all gateway replicas.

Errors

{"error": {"code": "invalid_parameter", "message": "language_code must be one of …", "request_id": "req_01…"}}

On WebSockets the same codes arrive as {"event":"error","code":…,"is_fatal":…}; fatal errors close the socket with 1008 (auth), 4000 (parameters), 1013 (busy) or 1011 (server).

SDKs and compatibility

Request and response shapes mirror Sarvam's Saaras (STT) and Bulbul (TTS) APIs: the multipart field names, the audios base64 array, the header api-subscription-key. An existing Sarvam client usually works after pointing it at https://api.uktiai.com; the additive fields (engine, confidence, words, vocabulary_used, corrections, lexicon_applied) are ignored by clients that do not know them. The machine-readable contracts are the OpenAPI document and the AsyncAPI document; generated SDKs are built from them.

Data handling

• Audio is processed in memory on the owned workers; the vendor fallback receives audio only for the languages routed to it.
• Call-scoped entities are stored under your account and call_id in a durable, TLS-protected store and expire automatically. Entity text is never written to logs; audit records carry ids and counts.
• Metering records account, request id, seconds of audio and characters; transcripts are not retained by the gateway beyond shadow-comparison logs, which follow the retention policy in the operator runbook.

Ukti Voice by Bachatt · console · status