AV Call Protocol¶

freeq voice and video calls are built from two independent layers:

• Signaling rides ordinary IRC. Participants announce calls with
  TAGMSGs; the server echoes call state back to the channel. This is
  the control plane — who is in the call, and what its id is.
• Media rides a separate transport: MoQ (Media over QUIC) through an
  SFU. This is the data plane — the actual Opus audio and H.264 video.

Keeping them apart means a call is just metadata on a normal channel:
any IRC client sees the TAGMSGs, and a client with no AV support is
unaffected. This page is the reference for both layers. To build an
agent on top of them, see Build a Voice & Video Agent.

1. Signaling: the +freeq.at/av-* tags¶

All call signaling is IRCv3 TAGMSGs sent to a channel. A client opens,
joins, or leaves a call by sending message tags; the server validates the
action and broadcasts an av-state TAGMSG to every channel member.

Client → server¶

Server → channel¶

The server answers every accepted action with one av-state TAGMSG:

av-state=ended fires when the last participant leaves.

In the Rust SDK¶

freeq_sdk::av builds these tag maps and parses the replies, so an agent
never hand-assembles a HashMap:

use freeq_sdk::av::{self, AvAction, parse_av_state};

// Sending: ClientHandle has the three convenience methods.
handle.av_start("#standup", &av::new_av_instance(), Some("Daily standup")).await?;
handle.av_join("#standup", &session_id, &instance).await?;
handle.av_leave("#standup", &session_id, &instance).await?;

// Receiving: apply parse_av_state to every TAGMSG you get.
if let Some(state) = parse_av_state(&tags) {
    match state.action {
        AvAction::Started => { /* a call opened — decide whether to join */ }
        AvAction::Joined  => { /* someone joined state.session_id */ }
        AvAction::Left    => {}
        AvAction::Ended   => { /* tear down */ }
    }
}

parse_av_state returns None for any TAGMSG that isn't an
av-state broadcast, so it is safe to call on every incoming tag event.

2. Session lifecycle¶

  Alice                     Server                      Bob
    │  TAGMSG av-start ───────▶│                          │
    │                          │── av-state=started ─────▶│   (broadcast
    │◀──── av-state=started ───│                          │    to channel)
    │                          │                          │
    │                          │◀──── TAGMSG av-join ─────│
    │◀──── av-state=joined ────│──── av-state=joined ────▶│
    │                          │                          │
    │   ══ media flows over MoQ between Alice and Bob ══   │
    │                          │                          │
    │                          │◀──── TAGMSG av-leave ────│
    │◀──── av-state=left ──────│──── av-state=left ──────▶│
    │  TAGMSG av-leave ───────▶│                          │
    │◀──── av-state=ended ─────│── av-state=ended ───────▶│

1. Start. A client sends av-start with a fresh instance id. The
   server creates a session, assigns it a session id, and echoes
   av-state=started carrying av-id=<session id>. The starter is the
   call's first participant — it does not also send av-join.
2. Join. Other clients send av-join with that av-id and their own
   instance id. The server echoes av-state=joined.
3. Media. Each participant publishes a MoQ broadcast and subscribes
   to the others (§3, §4).
4. Leave. av-leave produces av-state=left; the last leave
   produces av-state=ended.

Discover-or-start¶

A blind av-start is rejected when the channel already has a live call.
An agent that wants a call running should probe first:

GET /api/v1/channels/{channel}/sessions
→ { "active": { "id": "<session id>", "state": "Active", ... } }

If active is non-null and Active, av-join it; otherwise av-start.
This avoids the race where two clients both try to open the same call.

3. Broadcast addressing¶

Every participant publishes exactly one MoQ broadcast. Its path is:

{session_id}/{nick}~{instance}

• session_id — the call, from av-id. Shared by every broadcast in
  the call, so a subscriber filters the SFU's announce stream by the
  "{session_id}/" prefix and ignores stale broadcasts from other calls.
• nick — the participant's display name.
• instance — a per-device id: 8 lowercase hex characters
  (freeq_sdk::av::new_av_instance()). The same identity joining from
  two devices gets two instance ids, so the two broadcast paths don't
  collide. An agent skipping its own broadcast (to avoid transcribing
  its own voice) matches on nick, not the full path.

freeq-av has helpers for both directions:

use freeq_av::{broadcast_path, path_nick};

let path = broadcast_path("01HXYZ", "eliza", "0a1b2c3d"); // "01HXYZ/eliza~0a1b2c3d"
let nick = path_nick(&path);                              // "eliza"

4. Media transport¶

Media rides MoQ — Media over QUIC — through an SFU (selective
forwarding unit). The SFU endpoint is /av/moq on the freeq server host;
QUIC is the low-latency path, with a WebSocket fallback for environments
where QUIC can't establish.

• Audio — Opus, 48 kHz mono.
• Video — H.264.
• Each participant publishes its own broadcast and subscribes to every
  other broadcast in the same session. The SFU forwards; it does not mix.

A subscriber decodes each remote broadcast to PCM locally. Publishing a
continuous audio stream (silence included) keeps subscribers attached,
so there is no join latency when a participant actually starts talking.

The freeq-av crate packages this whole plane — connecting, publishing,
watching the announce stream, and decoding every participant — behind one
AvSession. See the agent tutorial.

5. Compatibility¶

• A client that ignores +freeq.at/av-* tags sees a normal channel.
• A call adds no channel modes and no special channel state beyond the
  TAGMSG history.
• The signaling and media layers are independent: signaling works with
  no media support, and the media transport carries no identity of its
  own — it trusts the IRC-side session_id.

See also¶

• Build a Voice & Video Agent — the tutorial.
• Bot Quickstart — a text-only bot in 10 minutes.
• Protocol Notes — the SASL ATPROTO-CHALLENGE
  mechanism and the rest of the IRC extensions.