Voice API

Join voice channels, receive and send audio, record and play

dc = DiscordClient()
gid = '1493461895615873044'
gld = await dc.guild(gid)
chs = await gld.channels()
intents = (1 << 0) | (1 << 7) | (1 << 9) | (1 << 15) # GUILDS | VOICE_EVENTS | GUILD_MESSAGES | MESSAGE_CONTENT
gc = GatewayClient(intents, dc)

await gc.start(debug=True)
vch = next(c for c in chs if c.id == '1493461896139903029'); vch

Channel(id=1327046393453613080, name='General', type=2)

DEBUG: Received Opcode: 10
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 0
DEBUG: Received Opcode: 11
DEBUG: Received Opcode: 0

Voice requires three simultaneous connections:

Main Gateway WebSocket (gc) — to request joining a voice channel and receive voice server info
Voice Gateway WebSocket — a separate WebSocket to a dedicated voice server for session coordination
Voice UDP — a UDP (datagram) connection for actual audio data. UDP is used over TCP because real-time audio needs low latency more than guaranteed delivery

VoiceClient manages the voice gateway WebSocket and UDP connections for a single voice channel, coordinated through the main GatewayClient.

source

VoiceClient

def VoiceClient(
    gc, gid, ch
):

Initialize self. See help(type(self)) for accurate signature.

vc = VoiceClient(gc, gid, vch); vc

VoiceClient(self.ch=Channel(id=1327046393453613080, name='General', type=2))

Voice requires additional Op opcodes beyond the main gateway’s identify/heartbeat. These handle the voice-specific protocol: requesting to join a channel (voice_state), authenticating with the voice server (voice_identify), negotiating encryption (select_protocol), signaling audio intent (speaking), and keeping the voice connection alive (voice_heartbeat).

The voice heartbeat uses a different format from the main gateway — v8 requires seq_ack tracking the last received sequence number. It must start immediately after receiving Hello, before UDP setup, or the voice WebSocket will time out (close code 4006).

source

Op.voice_resume

def voice_resume(
    server_id, session_id, token, seq_ack:int=-1
):

Call self as a function.

source

Op.voice_heartbeat

def voice_heartbeat(
    seq_ack:int=-1
):

Call self as a function.

source

Op.speaking

def speaking(
    ssrc, speaking:int=0
):

Call self as a function.

source

Op.select_protocol

def select_protocol(
    ip, port, mode:str='aead_xchacha20_poly1305_rtpsize'
):

Call self as a function.

source

Op.voice_identify

def voice_identify(
    server_id, user_id, session_id, token
):

Call self as a function.

source

Op.voice_state

def voice_state(
    guild_id, channel_id
):

Call self as a function.

# await vc._join()

Op 2 - Ready Event

IP Discovery: Discord needs your external IP/port (what the internet sees after NAT), not your local one. We send a 74-byte UDP request containing our SSRC, and Discord responds with our external address filled in:

Field	Size	Description
Type	2 bytes	1=request, 2=response
Length	2 bytes	70 (size of remaining fields)
SSRC	4 bytes	Our audio stream identifier
Address	64 bytes	Blank in request; our external IP in response
Port	2 bytes	Blank in request; our external port in response

Note: Discord requires a Speaking event (even with speaking=0) before it will send audio packets to you.

source

VoiceUDP

def VoiceUDP():

Interface for datagram protocol.

One more thing the UDP path needs: while a bot only listens (like a recorder), it never sends a packet after IP discovery, so NAT/conntrack mappings expire after a few minutes and inbound audio silently stops while the websocket stays healthy. A tiny 8-byte counter ping every 5s (the same trick discord.js uses) keeps the mapping alive; the voice server ignores unknown packets.

Op 4 - Session Description

source

VoiceClient.execute_transition

def execute_transition(
    tid
):

Call self as a function.

Op 13 (CLIENT_DISCONNECT) tells us a user left the channel — without dropping their decoder and SSRC mapping, a rejoining user could inherit stale jitter state (and the maps grow forever in long sessions).

fvc = object.__new__(VoiceClient)
fvc.decoders,fvc.ssrc_to_user,fvc.last_seq = {5:'dec'},{111:5, 222:9},{111:1, 222:2}
await fvc._handle_trans(13, dict(user_id='5'))
assert fvc.decoders == {} and fvc.ssrc_to_user == {222:9} and fvc.last_seq == {222:2}

The voice gateway runs on two cooperating loops. _hb sends heartbeats at the interval Discord specifies and watches for ACK responses; a missed ACK closes with code 4000, which preserves the session for a resume attempt. _reconnect uses one path for both flavors: a resume keeps session_id, token, endpoint, UDP, and DAVE state intact, while a fresh reconnect resets the audio session and re-runs _join to get new server info from the main gateway. Because _listen schedules _reconnect as its own task, _reconnect must never cancel the task it is running in (that was a bug that killed reconnects mid-flight), so it only cancels _listen_task when called from elsewhere. Failed connection attempts retry with exponential backoff (capped at 30s), and the counter resets once a session is established or resumed. Recording state is preserved in either path, so per-user files keep accumulating across reconnects with only a silence gap during the swap.

source

VoiceClient.reset_audio

def reset_audio():

Call self as a function.

The listener is where the voice gateway’s state machine actually runs. HELLO (op 8) branches on self.resuming to choose between voice_resume and voice_identify, which is how a single reconnect path supports both flavors. Opcode 9 confirms resume at the gateway level, but media encryption is a separate handshake; DAVE has to catch up before outgoing audio can be encrypted again, so we wait on _wait_dave_ready before setting resumed.

When the socket closes, the close code decides the recovery per the docs: the auth/session family (4003–4006, 4009, 4011) means this session’s credentials are dead, and since voice tokens are minted per-session by the gateway, the right move is a full rejoin with fresh ones — resuming just loops (live testing showed a bad resume earns 4003 Not authenticated, not the 4006 you might expect). 4014 means we were kicked/moved (or the voice server is being swapped) so we must not touch this endpoint again and instead wait for gateway events. Anything else (like 4015, voice server crashed) is transient so we resume. Recovery runs as a separate task so the dying listener never cancels its own reconnect. Handler errors are logged and skipped rather than silently killing the listener.

source

VoiceClient.join

async def join(
    debug:bool=False
):

Call self as a function.

source

VoiceClient.leave

async def leave():

Notify the main gateway that we are leaving the voice channel, then tear down the connection

source

VoiceClient.disconnect

async def disconnect():

Tear down voice ws/UDP without notifying the main gateway (used when Discord already removed us)

leave is the opposite of join: it tells the main gateway we are gone, then tears down. disconnect is the teardown alone, for when Discord has already removed us (kick, channel delete).

await vc.join(debug=True)

voice json 15 {'any': 100}
voice json 11 {'user_ids': ['346450717025894400']}
voice json 18 {'user_id': '346450717025894400', 'flags': 2}
voice json 20 {'user_id': '346450717025894400', 'platform': 0}

With the session up we can watch the UDP keepalive doing its job on the real socket — wrapping (not replacing) sendto to observe the 8-byte pings that keep NAT mappings alive during listen-only sessions.

await asyncio.wait_for(vc.sess.wait(), 15)
assert vc._ka_task and not vc._ka_task.done()
pings = []
_orig_sendto = vc.trans.sendto
vc.trans.sendto = lambda d, addr=None: (pings.append(d), _orig_sendto(d))[1]
await asyncio.sleep(5.5)
del vc.trans.sendto
assert pings and all(len(p) == 8 for p in pings)

Audio arrives as UDP packets using the RTP (Real-time Transport Protocol) format. Each packet contains: - RTP header (12+ bytes) — version, sequence number, timestamp, SSRC (identifies which user is speaking) - Encrypted payload — Opus audio wrapped first by DAVE, then by Discord’s voice transport encryption using the secret_key - Nonce (4 bytes at end) — used for the transport decrypt

Decryption is two-stage: - First, decrypt the Discord transport layer with aead_xchacha20_poly1305_rtpsize - The cipher expects a 24-byte nonce, but only 4 bytes are transmitted — we pad with 20 zero bytes - The unencrypted RTP header is used as AAD (Additional Authenticated Data) — it’s verified but not encrypted - For rtpsize mode, the AAD includes the 12-byte base header, any CSRC entries (4 bytes each, usually 0), and only the 4-byte extension preamble — the extension data is part of the encrypted payload - After transport decryption, skip past the RTP extension data to get the DAVE-protected audio frame — its length comes from the extension preamble (a count of 32-bit words), not a fixed size, and packets without the extension bit have nothing to skip - Then decrypt that frame with DAVE: self.dave.decrypt(uid, davey.MediaType.audio, data), where uid comes from the SSRC → user mapping learned from voice SPEAKING events - The result is the Opus payload, ready for Opus decoding

Packets with byte[1] == 0x78 are RTP voice data. Other packets (like 0xC9) are RTCP control packets used for connection quality reporting — we skip those.

source

VoiceClient.decrypt

def decrypt(
    pkt, uid
):

Call self as a function.

A hardcoded 8-byte skip here used to corrupt audio whenever the extension wasn’t exactly two words (and ate real Opus data on extension-less packets). Discord won’t produce a specific extension size on demand, so we build real RTP packets ourselves and round-trip them through the real cipher — varying extension sizes, no extension, and no DAVE session (where the transport plaintext is the Opus frame).

_key = bytes(range(32))
def _mk_pkt(payload, ext_words=2, has_ext=True):
    hdr = struct.pack('>BBHII', 0x80 | (0x10 if has_ext else 0), 0x78, 1, 960, 1234)
    if has_ext: hdr += struct.pack('>HH', 0xBEDE, ext_words)
    n4 = (7).to_bytes(4, 'big')
    enc = xchacha_encrypt((bytes(4 * ext_words) if has_ext else b'') + payload, hdr, n4 + b'\0'*20, _key)
    return hdr + enc + n4

fvc = object.__new__(VoiceClient)
fvc.secret_key = _key
for kw in [dict(ext_words=0), dict(ext_words=2), dict(ext_words=3), dict(has_ext=False)]:
    assert fvc.decrypt(_mk_pkt(b'opus!', **kw), 0) == b'opus!', kw

decode turns incoming RTP voice packets into PCM while keeping each speaker’s stream ordered without adding a jitter buffer. RTP sequence numbers are compared modulo 16 bits, so normal wraparound from 65535 to 0 is accepted; duplicates and packets that arrive behind the last accepted sequence are dropped. A forward sequence gap represents genuine packet loss: small gaps use Opus packet-loss concealment and larger gaps use raw silence.

Packets are decrypted and decoded before their sequence and timestamp become accepted state. A corrupt packet therefore cannot move the stream backwards or poison the next gap calculation. The tradeoff is intentional: a reordered packet is discarded instead of briefly buffering for it, which keeps recording simple at the cost of an occasional 20 ms loss.

source

VoiceClient.decode

def decode(
    pkt
):

Call self as a function.

source

silence

def silence(
    n_smpls:int, # 2 bytes per sample (s16le)
):

Call self as a function.

Recording is split per speaker by opening one ffmpeg process per user and writing that user’s decoded PCM to its stdin. Each file is the session timeline: _written counts the samples on disk per user, and before each write the track is padded with silence up to the current wall-clock position, so audio lands at the moment it was spoken — whether the user joined late, paused mid-call, or left and rejoined. Padding only kicks in when a track falls more than 200ms behind: while someone is speaking, each packet appends its 20ms and the track keeps pace on its own, so arrival jitter never inserts silence mid-sentence.

start_recording drains the UDP queue first so stale packets from before the recording do not get written into the new files.

stop_recording pads every track with silence out to the stop time, so all files are exactly session length and stay aligned, then closes each ffmpeg stdin and waits for the process. That finalizes the containers; otherwise the files can look valid but be missing buffered audio at the end.

source

VoiceClient.stop_recording

async def stop_recording(
    mix:bool=True, mix_path:NoneType=None, timeout:int=10
):

Call self as a function.

source

VoiceClient.mix_recording

async def mix_recording(
    mix_path:NoneType=None, timeout:NoneType=None, **out_kw
):

Mix the per-speaker files (all session-length, so already aligned) into one recording at mix_path; out_kw passes ffmpeg output options

source

VoiceClient.start_recording

def start_recording(
    path:str='/tmp/recording.mp3'
):

Call self as a function.

mix_recording is separate from stop_recording so a bot can leave the channel before the (potentially slow) mix: call stop_recording(mix=False), leave(), then mix_recording(). Output options pass through to ffmpeg, e.g. ar=16000, ac=1 for speech-to-text.

vc.start_recording()

'/tmp/recording.mp3'

out = await vc.stop_recording()
out

{'speakers': {}, 'mixed': None}

Outgoing audio is the receive path in reverse: encode PCM to Opus, wrap it with DAVE only when the E2EE session is actually ready, then encrypt it with Discord’s voice transport key and send it as an RTP packet. The readiness check matters: a bot alone in a channel negotiates a DAVE protocol version but no MLS group exists until the epoch starts, so clients send plain Opus until then (davey raises NotReady if asked to encrypt earlier) — the same applies briefly during epoch transitions.

The RTP sequence, timestamp, and nonce are advanced manually for each 20 ms frame. If those drift or repeat, Discord will hear either broken audio or nothing at all.

source

VoiceClient.send_pkt

def send_pkt(
    payload
):

Call self as a function.

source

VoiceClient.encode

def encode(
    fr
):

Call self as a function.

We can check the readiness gating on the live session: DAVE-wrapped frames end with the 0xFAFA magic marker, so whether the encoder output carries it must always agree with dave.ready — a bot alone in the channel has a negotiated version but no epoch, so it sends plain Opus.

assert vc.encode(silence(spf)).endswith(b'\xfa\xfa') == vc.dave.ready

We must notify discord when our bot is speaking and stop speaking. When a transmission truly ends, the docs require five frames of Opus silence (0xF8 0xFF 0xFE) so receivers don’t interpolate the tail of our audio into the next utterance — skipping them is audible as a garbled blip at the end of playback. Streamed audio that arrives in chunks (e.g. a realtime TTS feed) should pass end=False for all but the last chunk so the transmission stays continuous. The finally keeps Discord from leaving the bot marked as speaking if playback errors halfway through.

source

VoiceClient.send_frames

async def send_frames(
    frames, end:bool=True
):

Send 20ms Opus frames; unless end=False, finish with 5 silence frames and clear the speaking flag.

source

VoiceClient.send_frame

def send_frame(
    fr
):

Call self as a function.

source

VoiceClient.speaking

async def speaking(
    on:bool=True
):

Call self as a function.

Discord voice expects 20 ms Opus frames. Padding the final partial frame preserves the tail of short clips instead of silently dropping it.

source

VoiceClient.send_pcm

async def send_pcm(
    pcm, end:bool=True
):

Call self as a function.

source

pcm2frames

def pcm2frames(
    pcm
):

Call self as a function.

t = np.arange(sr) / sr
mono = (0.08 * np.sin(2*np.pi*440*t) * 32767).astype(np.int16)
pcm = np.column_stack([mono, mono]).ravel().tobytes()
len(pcm)

await vc.send_pcm(pcm)

Spying on the live send path (wrapping send_pkt so packets still go out for real) shows the transmission shape on the wire: every send_pcm ends with exactly five extra packets — the silence frames — while end=False leaves the transmission open with no tail.

n_frames = len(list(pcm2frames(pcm)))
sent = []
_orig_send_pkt = vc.send_pkt
vc.send_pkt = lambda p: (sent.append(p), _orig_send_pkt(p))[1]
await vc.send_pcm(pcm)
assert len(sent) == n_frames + 5      # real transmission ends with 5 opus silence frames

sent.clear()
await vc.send_pcm(pcm, end=False)     # mid-stream chunk: no tail, transmission stays open
del vc.send_pkt
assert len(sent) == n_frames
await vc.speaking(False)

await vc._reconnect(resume=True)
await asyncio.wait_for(vc.resumed.wait(), 5)
await vc.send_pcm(pcm)

_reconnect was called directly above; real drops arrive as ConnectionClosed in the listener, which classifies the close code per the docs and schedules recovery as a separate task (a listener that ran the reconnect inline could cancel its own recovery — a bug this design retired). Killing the live socket with a transient code exercises the resume path end-to-end.

vc.resumed.clear()
await vc.ws.close(code=4000)                  # a real dropped connection
await asyncio.wait_for(vc.resumed.wait(), 20) # listener resumes unaided
await vc.send_pcm(pcm)

The auth/session close family (4003–4006, 4009, 4011) means this session’s credentials are dead and resuming would loop forever — the client must rejoin from scratch through the main gateway, which mints fresh ones. We can earn one for real by resuming with a corrupted token (Discord answers 4003 Not authenticated — a fake test here would have guessed 4006 and encoded the wrong policy): the client should come back with a fresh token and a working session, without us doing anything.

vc.token = 'not-a-real-token'
vc.resumed.clear()
await vc.ws.close(code=4000)   # resume attempt with the bad token -> 4003 -> automatic full rejoin
for _ in range(60):
    if vc.sess.is_set() and vc.token != 'not-a-real-token': break
    await asyncio.sleep(0.5)
assert vc.token != 'not-a-real-token' and vc.sess.is_set()
await vc.send_pcm(pcm)

To play audio back from a file, we’ve created a little helper using ffmpeg:

source

VoiceClient.play_file

async def play_file(
    path
):

Call self as a function.

source

file2pcm

def file2pcm(
    path
):

Call self as a function.

await vc.play_file("/tmp/recording_mixed.mp3")

Finally, the hostile ending: an admin disconnecting the bot. Discord sends our own VOICE_STATE_UPDATE with a null channel (and closes the voice socket with 4014) — the client must tear down cleanly, finalize any in-flight recording, and not fight its way back in. We can do this to ourselves over REST.

vc.start_recording('/tmp/kick_test.mp3')
await asyncio.sleep(1)
await vch('PATCH', f'/guilds/{gld.id}/members/{gc.user_id}', channel_id=None)  # kick ourselves
for _ in range(30):
    if not vc.running: break
    await asyncio.sleep(0.5)
assert not vc.running and not vc._recording   # clean teardown, recording finalized

Recording can continue across reconnect types.

vc.start_recording()
await asyncio.sleep(2)
procs_before, start_before = dict(vc._rec_procs), vc._rec_start
await vc._reconnect(resume=False)
await asyncio.sleep(3)
await vc.stop_recording()

await vc.play_file("/tmp/recording_mixed.mp3")

await vc.leave()