Share Realtime Media (Camera, Mic, Canvas)

Problem

You want participants to send and receive live video/audio — a webcam, a microphone, a shared screen, or a canvas.captureStream() visual — to the whole session, to a specific subset of peers, or to whoever is watching a named topic. You don't want to hand-manage transceivers, renegotiation, or offer/answer glare.

Solution

Use the realtime media plane, client.media. You share() a MediaStream with a declarative scope and the SDK realizes it over the WebRTC mesh — adding the tracks to the right peers, renegotiating, and tagging inbound streams with their origin. Media mirrors the data addressing model exactly:

Data plane	Media plane	Reaches
broadcast(payload)	media.share(stream)	all peers in the session
send(to, payload)	media.share(stream, { to })	a specific peer / subset
publish(topic, …) + subscribe(topic)	media.share(stream, { topic }) + media.subscribe(topic)	topic subscribers

Browser-only, mesh-only today

The media plane needs MediaStream/WebRTC, so it is TypeScript (browser) only — Python and Swift clients have no media API yet. It runs over a peer-to-peer mesh: sharing one stream to K peers means K encoded uploads from the publisher. Subset and small-group sharing is the well-suited case; a media topic with dozens of subscribers is a mesh foot-gun. The API is deliberately topology-neutral so an SFU can be dropped in later (server capability + SDK strategy) with no change to application code — but no SFU exists yet.

client.media requires RTC to be configured on the client (the rtc option); the getter throws if RTC is not enabled. Media flows over the same peer connections you open with connectRTC().

Code

import { StarfishClient } from "@starfish/sdk";

const client = new StarfishClient({
  server: "ws://localhost:4000",
  rtc: {
    factory: RTCPeerConnection, // browser built-in
    iceServers: [{ urls: "stun:stun.l.google.com:19302" }],
  },
});

await client.connect();
await client.join("jam-room");

// Media rides the WebRTC mesh — open peer connections to the people you share with.
client.peers$.subscribe((peers) => {
  for (const peer of peers) {
    client.connectRTC(peer.id).catch(() => {
      /* peer without RTC support — nothing to share to */
    });
  }
});

// --- Obtain a stream (browser APIs) ---
const cam = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });

// --- Share (pick one scope) ---
const pub = client.media.share(cam);                       // session (default)
// const pub = client.media.share(cam, { to: [peerA, peerB] }); // subset
// const pub = client.media.share(stageCam, { topic: "stage-cam" }); // topic

// --- Manage the publication ---
pub.setEnabled(false);              // mute: disables every track
await pub.replaceTrack(screenTrack); // swap camera → screenshare, no re-share
pub.stop();                          // stop sharing everywhere and stop the tracks

// --- Receive ---
client.media.on("stream", ({ peerId, topic, stream }) => {
  mountVideo(peerId, topic, stream);
});
client.media.on("streamended", ({ peerId, topic }) => {
  unmountVideo(peerId, topic);
});

share() returns a Publication synchronously — the renegotiation happens in the background as tracks are added to each peer. It automatically extends to peers that connect after you call share(), so you can share once up front and let new arrivals pick it up.

Explanation

Scopes

share(stream, opts?) takes an optional ShareOptions:

interface ShareOptions {
  to?: string[];      // audience scope — share to this subset of peers
  topic?: string;     // topic scope — share to whoever subscribed to this topic
  simulcast?: unknown; // reserved for the future SFU; a no-op in mesh
}

• Omit both (or pass nothing) → session scope: every peer in the session.
• to → audience scope: only the listed peers.
• topic → topic scope: whoever has called media.subscribe(topic).

Only peers you have an RTC connection to actually receive the tracks; the publisher skips peers that aren't connected and adds them if they connect later.

Audience enforcement is client-side

In the mesh, audience and topic membership are enforced by the publisher simply not adding the track for non-audience peers. It is only as strong as the publishing client. A future SFU would enforce this server-side.

The Publication handle

share() returns a handle to the local stream you're sharing:

interface Publication {
  readonly id: string;
  setEnabled(enabled: boolean): void;          // mute/unmute all tracks
  replaceTrack(track: MediaStreamTrack): Promise<void>; // swap without re-sharing
  stop(): void;                                // remove from all peers, stop tracks
}

• setEnabled(false) toggles enabled on every track — the connection stays up, frames just stop flowing. Use it for mute, not teardown.
• replaceTrack(track) swaps the outgoing track in place (camera → screenshare) without re-sharing or a visible renegotiation on the receiver.
• stop() removes the tracks from every peer (renegotiating) and stops them.

Receiving streams

Two ways to consume inbound media, both delivering a RemoteStream:

interface RemoteStream {
  peerId: string;         // who is sending it
  topic: string | null;   // the media topic, or null for session/audience scope
  stream: MediaStream;
}

• Events — media.on("stream", cb) fires when a remote stream arrives and media.on("streamended", cb) when it goes away. Both return an unsubscribe function. Good for imperative mount/unmount.
• media.remote$ — an Observable<RemoteStream[]> holding the current set of inbound streams. Good for reactive rendering (re-render your video grid on every change):

client.media.remote$.subscribe((streams) => {
  renderVideoGrid(streams); // streams: RemoteStream[]
});

Media topics

To receive topic-scoped media you must declare interest, exactly like data topics:

await client.media.subscribe("stage-cam");   // start receiving the topic
// ... inbound streams for "stage-cam" arrive with topic === "stage-cam"
await client.media.unsubscribe("stage-cam"); // stop

Media subscriptions reuse the data-topic subscription authority, so the server's topic/peers bookkeeping tells publishers who to send to.

Escape hatch: the raw peer connection

For power users who need transceiver-level control, stats, or bandwidth tuning, grab the live RTCPeerConnection for a peer:

const pc = client.getPeerConnection(peerId); // RTCPeerConnection | null

It returns null if there is no connection to that peer.

Variations

Share a canvas as video

Any MediaStream works — you're not limited to camera/mic. Capture a <canvas> (great for generative visuals) and share it like any other stream:

const canvas = document.querySelector("canvas");
const visual = canvas.captureStream(30); // 30 fps
const pub = client.media.share(visual, { topic: "visuals" });

Switch from camera to screenshare

Keep the same publication; just replace the video track:

const screen = await navigator.mediaDevices.getDisplayMedia({ video: true });
const [screenTrack] = screen.getVideoTracks();
await pub.replaceTrack(screenTrack); // receivers keep rendering, source swaps

Mute without dropping the connection

micButton.onclick = () => pub.setEnabled(!muted);

Related

• Realtime Media API reference — the full client.media surface, types, and protocol frames.
• Use WebRTC Data Channels — the low-latency data transport media shares its peer connections with.
• Publish/Subscribe to a Topic — the data-plane addressing model media mirrors.