livehal

Architecture and build guide for the libcamera / V4L2 / RDK X5 native capture-and-encode pipeline.

livehal is the Hardware Abstraction Layer crate that wraps the native-pipeline C++ pipeline. It exposes a safe Rust API (NativePipeline) to liveion while keeping all FFI details crate-private.

Architecture

┌──────────────────────────────────────────────────────────┐
│ liveion (RTP / WHEP / source manager)                    │
│                                                          │
│  NativeSource  (unified thin wrapper)                    │
│         │                                                │
│         ▼                                                │
│  NativeEncodedSource                                     │
│    webrtc-rs H264Payloader / Packetizer                  │
│    → MediaPacket::RtpPacket(Arc<Packet>)                 │
│    → inject_rtp (no marshal/unmarshal roundtrip)         │
│         │                                                │
│  RTCP PLI → request_keyframe()                           │
│         │                                                │
│─────────│─ optional dep boundary ────────────────────────│
│         ▼                                                │
│ livehal (native backend crate)                           │
│                                                          │
│  NativePipeline  (safe Rust wrapper)                     │
│         │                                                │
│         ▼  crate-private FFI                             │
│  native_ffi.rs  →  source_pipeline_ffi.h                 │
│         │                                                │
│         ▼                                                │
│ C++ (native-pipeline)                                    │
│                                                          │
│  SourcePipeline                                          │
│    ├─ CaptureBackend  →  RawFrame  (C++ internal)        │
│    └─ EncoderBackend  →  EncodedPacket                   │
│                              │                           │
│                              ▼  FFI callback             │
│  on_encoded_packet()  ← EncodedPacketFFI                 │
│         │                                                │
│         ▼  data copied immediately → mpsc channel        │
│  EncodedPacket → liveion NativeEncodedSource             │
└──────────────────────────────────────────────────────────┘

• RawFrame is C++-internal and never crosses the FFI boundary.
• EncodedPacket crosses FFI via a pure-C callback inside livehal; data is copied immediately and sent through an mpsc channel to liveion.
• All FFI details are crate-private in livehal; liveion only sees EncodedPacket through the channel.
• RTP path for native sources: EncodedPacket → webrtc-rs H264Payloader / Packetizer → MediaPacket::RtpPacket(Arc<Packet>) → track.inject_rtp. This avoids the Packet → bytes → Packet::unmarshal roundtrip that other sources use.
• MediaPacket::Rtp { data } bytes path is still used by rtp_listener / rtsp_source / sdp_source.
• DMA-BUF zero-copy: implemented for generic V4L2 → RKMPP and for Raspberry Pi (libcamera / generic V4L2 → V4L2 M2M). With prefer_dmabuf = true on both capture and encoder, the capture hands each frame's DMA-BUF to the encoder and the encoder queues it directly — no CPU copy of the frame. Buffer ownership is explicit: the capture buffer is owned by the encoder between dequeue and the frame's encode completion, and is requeued only after the hardware has finished reading it (a deferred-requeue release contract, so the camera can never overwrite a buffer mid-encode). If export or import fails (or the frame layout does not match what the encoder can import) the pipeline falls back to the CPU-copy path. The RDK backend does not implement DMA-BUF import yet (encoder_rdk.cpp rejects BufferKind::DmaBuf); there everything is still copied through the CPU path.

Config

Sources are configured under per-stream [[stream.<name>.sources]] blocks in conf/live777.toml. The stream name is the key under [stream]; each stream can have an optional DataChannel <-> UDP channel. Only one source per stream runs at a time — the source registry is keyed by stream name, so configure a single source per stream.

Provisioned streams and on-demand sources

Every [stream.<name>] entry is provisioned: the stream is registered at startup, always appears in the API and Dashboard (even while idle), is exempt from the automatic teardown strategies (orphan reaper, auto_delete_whip / auto_delete_whep), and cannot be created or deleted through the admin API (POST / DELETE /api/streams/<name> return 409).

A provisioned stream is permanent, but its media plane still follows publisher lifecycles:

• When a WHIP publisher leaves gracefully, the stream is reset to standby: its subscribers are disconnected (WHEP cannot renegotiate tracks, so viewers must re-subscribe) and a stream-deleted + stream-created hook pair with reason reset fires. Always-on sources are restarted; on-demand sources stay stopped until the next subscriber.
• A WHIP publisher cannot attach while the stream's configured source is running (and vice versa) — that would mix two publishers' tracks; the publish request fails with 409.

By default a stream's sources start unconditionally at server startup. Set on_demand = true to run them only while someone is watching — the camera / encoder / RTSP pull stays off until the first subscriber (WHEP, cascade push, or RTSP pull) arrives, and stops again after the last one leaves:

[stream.cam1]
on_demand = true
# Grace period after the last subscriber leaves before sources stop (default 10000)
on_demand_close_after_ms = 10000
# How long the first subscriber waits for the source to become ready before
# its subscribe request fails (default 10000)
on_demand_start_timeout_ms = 10000

[[stream.cam1.sources]]
url = "rtsp://192.168.1.100:554/stream"

Subscribers that arrive while a source start is still in flight wait for it (and for each other) instead of receiving a track-less answer; if the source does not become ready within on_demand_start_timeout_ms the subscribe request fails and the client can retry. Source start/stop emits PublishStarted / PublishStopped with the session id virtual-source, which also drives recording (recorder.auto_streams) and on_publish_started / on_publish_stopped hooks.

For WHEP URL sources, the effective source readiness wait is at least 35000ms, even if on_demand_start_timeout_ms is lower. This lets a cold upstream WHEP/on-demand source complete its HTTP setup timeout and deliver the first media packet.

On-demand streams show a standby badge in the Dashboard while idle and on-demand while their sources are running; other provisioned streams are marked config.

URL-based (non-native: RTSP / SDP / RTP)

[stream.rtsp-cam]
[[stream.rtsp-cam.sources]]
url = "rtsp://192.168.1.100:554/stream"

An SDP file source whose connection address is a multicast group (for example c=IN IP4 230.1.1.1) joins that group on each media port instead of only binding it, so always-on multicast senders (drone/robot cameras, encoders) can feed live777 directly — combined with on_demand = true the group membership only exists while someone is watching:

[stream.robot-cam]
on_demand = true

[[stream.robot-cam.sources]]
url = "/etc/live777/robot-cam.sdp"   # c=IN IP4 230.1.1.1, m=video 1720 RTP/AVP 96
# Interface used to join the group: an IPv4 address for IPv4 groups, an
# interface index or name for IPv6 groups (a `%zone` on the SDP address,
# e.g. `c=IN IP6 ff12::1%eth0`, is honored when this is unset). Unset
# lets the kernel choose, which only receives traffic arriving on the
# default-route interface — link-local IPv6 groups (ff02::/16, ff12::/16)
# always need an explicit interface.
multicast_interface = "192.168.123.11"

RFC 4566 suffixes on the connection address (c=IN IP4 230.1.1.1/64, c=IN IP6 ff15::1/3) are accepted. The receiver sockets set SO_REUSEADDR, so several receivers of the same group:port (another stream, a monitoring tool) can coexist on one host. A failed bind/join is retried with backoff before the source start fails — for always-on (non-on_demand) multicast streams, still make sure live777 starts after the interface address is assigned (systemd: After=network-online.target), since a long DHCP delay outlasts the retries.

WARNING

Multicast senders keep the kernel's default loopback (IP_MULTICAST_LOOP), so this source also receives whatever a same-host sender — including live777's own rtp:// output target — sends to the joined group:port. Pointing a source and a target at the same group:port on one host (directly, or via a liveman cascade looping back) makes live777 ingest its own output in a feedback loop; use distinct groups or ports for same-host demos.

Structured native (libcamera / V4L2 / RDK)

[stream.pi-cam]
[[stream.pi-cam.sources]]

[stream.pi-cam.sources.capture]
backend = "libcamera"
device = "0"
width = 640
height = 480
fps = 30
pixel_format = "yuv420"
# hflip = true           # mirror left-right; libcamera only (ISP, zero cost)
# vflip = true           # flip top-bottom; hflip + vflip = 180° rotation

[stream.pi-cam.sources.encoder]
backend = "v4l2-m2m"
codec = "h264"
bitrate = 1_000_000
profile = "baseline"     # or a 6-digit hex profile-level-id such as "42001f"
level = "3.1"            # required when profile is a profile name
gop = 60                 # keyframe interval in frames; 0 = IDR only on
                         # request (PLI/FIR, subscriber join) — v4l2-m2m only
# bitrate_mode = "vbr"   # v4l2-m2m only: "vbr" (driver default) or "cbr";
                         # note the bcm2835-codec firmware rejects CBR

[stream.pi-cam.sources.output]
payload_type = 96
clock_rate = 90000

pixel_format and codec values are validated at startup (unknown values error early). capture + encoder are mutually exclusive with url; the source type is derived from capture.backend (device is a camera ID for libcamera or a path for v4l2).

capture.hflip / capture.vflip (both default false) flip the image — mirror left-right and top-bottom respectively; setting both gives a 180° rotation. They are libcamera-only: the flip is done by the ISP, so it costs no CPU. The v4l2 backend rejects them at startup — its video node (unicam) only DMA-moves frames with no image-processing capability, and the flip controls on the sensor sub-device cannot be used safely outside libcamera (they would desync the ISP's Bayer order). 90°/270° rotation is not offered: the Raspberry Pi ISP cannot transpose an image in hardware.

conf/live777.toml ships with commented-out Pi / RDK examples. Copy them into your own config to enable a camera source.

Adaptive bitrate (experimental)

Adaptive bitrate is on by default for native encoder sources — the running encoder's target bitrate is driven by WHEP subscriber RTCP feedback (issue #409): TWCC loss (falling back to Receiver Report fraction_lost) is sampled once per second, and an AIMD controller retunes the encoder at runtime. No config key is needed; the two optional forms are an opt-out and a custom floor:

[stream.pi-cam.sources.encoder]
bitrate = 4_000_000           # ceiling — the controller only lowers from here
# adaptive = false            # strictly fixed rate (no AIMD)
# adaptive = { min_bitrate = 500_000 }
                              # custom floor (default: lowest tier when
                              # tiers are declared, else
                              # max(bitrate / 8, 300_000))

Semantics: loss above 5 % for two consecutive windows cuts the bitrate by 15 %; a sustained 10 s clean window adds 5 % of the target back. With several subscribers the worst link wins — the shared encoder is lowered for everyone, so a single weak subscriber penalizes the whole stream (simulcast would be the real fix; if that is unacceptable for your stream, use adaptive = false). Subscribers younger than 5 s are ignored while their decoder primes. Runtime retuning currently works on the rkmpp and v4l2-m2m backends; other backends log a warning and the controller disables itself.

Bitrate telemetry & quality tiers

GET /api/sources/:streamId/bitrate reports how the source's encoder bitrate is currently driven — adaptive (the AIMD controller above) or fixed — plus the current rate (see the HTTP API guide). Control goes through quality tiers: named presets of source parameters, applied with POST /api/sources/:streamId/tier. A tier is a partial overlay on the source's own structure — optional capture and encoder blocks holding the parameters to change; anything left out keeps the configured value:

[stream.pi-cam.sources.encoder]
bitrate = 4_000_000        # the AIMD's ceiling at boot
adaptive = { min_bitrate = 300_000 }  # optional; the AIMD is on by default

# Encoder-only tier: retunes the running encoder, seamless.
[[stream.pi-cam.sources.tiers]]
name = "mid"
encoder = { bitrate = 2_000_000 }

# Ladder rung: capture geometry + encoder budget, applied by rebuilding
# the capture+encoder pipeline (sub-second frame gap, subscribers stay
# connected).  encoder.bitrate omitted here — derived from 640x480x15
# at ~0.07 bpp (~645 kbps).
[[stream.pi-cam.sources.tiers]]
name = "low"
capture = { width = 640, height = 480, fps = 15 }

Tiers sit next to capture/encoder (not inside the encoder block) because a rung spans both blocks: the encoder overlay retunes the encoder, while the capture overlay reconfigures the capture (livehal has no scaler stage — capture size is encoder input size — and framerate is a capture-side property). An encoder-only tier switches seamlessly in place; a tier touching capture geometry rebuilds the pipeline underneath the stream — the RTP session survives, subscribers just see a brief freeze and an in-band SPS/PPS change. capture.width/height must be set together and may be any size the sensor supports — including above the boot profile, so the ladder top need not be the boot config (a 972p30 boot with a 1080p20 max-quality rung); capture.fps may likewise be any rate the sensor supports (a low-latency rung can trade resolution for framerate, e.g. a 480p60 tier on an OV5647 — unsupported modes are rejected at apply time and rolled back). encoder.bitrate may likewise exceed the boot encoder.bitrate (only the encoder's signed-32-bit control range bounds it), so the ladder top can carry more bits than the boot rung — and it is optional: when omitted it is derived from the tier's effective geometry at ~0.07 bits per pixel (floored at 50 kbps), which is where hand-tuned ladders usually land anyway; set it explicitly for content that needs more or fewer bits, or to offer premium/economy rungs at the same geometry. An empty tier (nothing set) is rejected.

Applying a tier does not suspend the adaptive controller: the tier's bitrate becomes the AIMD's new ceiling, so it keeps following network conditions within the rung. A failed rebuild (e.g. the camera rejects the new size) rolls back to the previous configuration, and the tier's fields are always overlaid on the configured capture: unset fields restore the configured values, so a mixed ladder (bitrate-only and geometry tiers side by side) is well-defined. The same mechanism can later grow to switch source types entirely (e.g. from the camera to an RTSP URL) — tiers describe source configuration, not encoder internals.

Backend naming

Layer	Value
capture.backend	"libcamera", "v4l2"
encoder.backend	"v4l2-m2m", "rdk", "rkmpp"

pixel_format values

TOML string	RawPixelFormat	Numeric
yuyv, yuyv422	Yuyv422	0
nv12	Nv12	1
yuv420, yuv420p	Yuv420p	2
rgb888, rgb	Rgb888	4
uyvy, uyvy422	Uyvy422	5

codec values

TOML string	VideoCodec	Numeric
h264	H264	100
h265, hevc	H265	101
av1	Av1	102
vp8	Vp8	103
vp9	Vp9	104

Feature flags

Features are split into capture backends, encoder backends, and convenience presets. All backend features imply native-source, which in turn enables source (autostart) and dep:livehal.

Capture backends

Feature	Backend
capture-libcamera	libcamera (Raspberry Pi CSI cameras)
capture-v4l2	V4L2 video capture (USB cameras, generic Linux)

Encoder backends

Feature	Backend
encoder-v4l2-m2m	V4L2 Memory-to-Memory hardware encoder
encoder-rdk	Horizon RDK X5 hardware encoder
encoder-rkmpp	Rockchip MPP hardware encoder (aarch64 only)

Platform presets

Preset	Expands to
native-rpi	capture-libcamera, capture-v4l2, encoder-v4l2-m2m
native-generic-v4l2	capture-v4l2, encoder-v4l2-m2m
native-rdk	capture-v4l2, encoder-rdk
native-rkmpp	capture-v4l2, encoder-rkmpp

No additional --features source is needed — presets include autostart.

# Raspberry Pi CSI
 cargo build --bin live777 --release \
  --target aarch64-unknown-linux-gnu \
  --no-default-features --features native-rpi,webui

# Generic Linux V4L2
cargo build --bin live777 --release \
  --no-default-features --features native-generic-v4l2,webui

# RDK X5
cargo build --bin live777 --release \
  --target aarch64-unknown-linux-gnu \
  --no-default-features --features native-rdk,webui

# Rockchip RKMPP (RK3588, RV1126B)
cargo build --bin live777 --release \
  --target aarch64-unknown-linux-gnu \
  --no-default-features --features native-rkmpp,webui

Build

Prebuilt binaries

You usually do not need to compile anything: releases from v0.10.0 onward publish native builds (live777-<version>-aarch64-unknown-linux-gnu-rpi.tar.gz for Raspberry Pi, -rkmpp.tar.gz for RK3588/RV1126B). See Installation and the Raspberry Pi deployment guide. Build from source only for unreleased changes or RDK X5 (no prebuilt asset; requires RDK_SYSROOT).

Prerequisites

• CMake ≥ 3.16
• A C++17 compiler (gcc or clang)
• Platform SDK as needed (libcamera, RDK sysroot)

Raspberry Pi (libcamera)

cargo build --bin live777 --release \
  --target aarch64-unknown-linux-gnu \
  --no-default-features --features native-rpi,webui

For cross builds, just rpi-cross-build uses the GHCR cross image with a Raspberry Pi libcamera sysroot baked in. Set RPI_SYSROOT only when overriding that with a sysroot copied from a device.

Generic Linux V4L2

cargo build --bin live777 --release \
  --no-default-features --features native-generic-v4l2,webui

RDK X5

cargo build --bin live777 --release \
  --target aarch64-unknown-linux-gnu \
  --no-default-features --features native-rdk,webui

Requires the RDK sysroot with hb_media_codec libraries. RDK_SYSROOT must be set to the sysroot path; there is no default.

Note: The DMA-BUF zero-copy encode path is not yet implemented. See the DMA-BUF notes in the Architecture section above.

Rockchip RKMPP (RK3588, RV1126B)

cargo build --bin live777 --release \
  --target aarch64-unknown-linux-gnu \
  --no-default-features --features native-rkmpp,webui

Requires the Rockchip MPP library (librockchip_mpp) and headers. Set RKMPP_SYSROOT to a sysroot containing them when cross-compiling. The rkmpp encoder accepts NV12 input only; pair it with pixel_format = "nv12" capture.

Note: rkmpp profile configuration is stricter than the other backends. For H.264, profile must be a 6-character profile-level-id hex string (e.g. "420028" = Baseline Level 4.0, "640028" = High Level 4.0) and a separate level field is not allowed — the level is encoded in the hex. For H.265, use profile = "main" with an optional level ("3.0"–"5.1", default "4.0"); only Main profile / Main tier are supported.

The Cross Images workflow publishes ghcr.io/binbat/crossbuilder-aarch64-rkmpp:latest, an aarch64 cross image with the MPP sysroot baked in at /opt/rkmpp-sysroot (built from docker/Dockerfile.cross-aarch64-rkmpp). Point cross at it via CROSS_TARGET_AARCH64_UNKNOWN_LINUX_GNU_IMAGE, or use just rkmpp-pack-size; RKMPP_SYSROOT overrides the baked sysroot with one pulled from a device.

macOS (development / check only)

cargo check --no-default-features
cargo check --features native-rpi,webui

Note: On macOS and Windows, native backend features are silently skipped by the build script; CMake is not invoked and no native symbols are linked. You can run cargo check with native features for linting, but the resulting binary cannot use native sources on those platforms.

Environment variables

Variable	Purpose
RPI_SYSROOT	Optional Raspberry Pi sysroot with libcamera-dev; overrides the sysroot baked into the RPi cross image.
RDK_SYSROOT	Path to the Horizon RDK X5 SDK sysroot. Required when building encoder-rdk / native-rdk for aarch64.
RKMPP_SYSROOT	Path to a sysroot containing the Rockchip MPP library and headers. Used when building encoder-rkmpp / native-rkmpp for aarch64.
LIVEHAL_CXX_STDLIB	Override the C++ standard library to link (stdc++, c++, etc.). Useful for cross-compilation toolchains.
LIVEHAL_RDK_ALLOW_UNDEFINED	Set to 1 to allow unresolved symbols in RDK shared libraries during cross-compilation with an incomplete sysroot.

Backend selection (build-time)

The CMake backend is inferred from the enabled capture/encoder features:

Enabled feature(s)	Selected backend	CMake defines (ON)
capture-libcamera	rpi	ENABLE_BACKEND_PI, ENABLE_CAPTURE_LIBCAMERA, ENABLE_CAPTURE_V4L2, ENABLE_ENCODER_V4L2_M2M
encoder-rdk on aarch64	rdk-x5	ENABLE_BACKEND_RDK_X5, ENABLE_CAPTURE_V4L2, ENABLE_ENCODER_RDK_X5
encoder-rkmpp on aarch64	rkmpp	ENABLE_CAPTURE_V4L2, ENABLE_ENCODER_RKMPP
capture-v4l2 / encoder-v4l2-m2m	generic-v4l2	ENABLE_CAPTURE_V4L2, ENABLE_ENCODER_V4L2_M2M

When no capture-* feature is enabled, CMake is skipped entirely. Encoder-only features do not trigger a CMake build — the SourcePipeline requires a capture backend.

capture-libcamera is mutually exclusive with encoder-rdk and encoder-rkmpp. If enabled together, the encoder feature is ignored with a build warning and the rpi (libcamera) backend is selected.

Config examples

Raspberry Pi CSI

[stream.pi-cam]
[[stream.pi-cam.sources]]

[stream.pi-cam.sources.capture]
backend = "libcamera"
device = "0"
width = 640
height = 480
fps = 30
pixel_format = "yuv420"

[stream.pi-cam.sources.encoder]
backend = "v4l2-m2m"
codec = "h264"
bitrate = 1_000_000
profile = "baseline"
level = "3.1"
gop = 60

[stream.pi-cam.sources.output]
payload_type = 96
clock_rate = 90000

Raspberry Pi USB V4L2

Packed UYVY capture devices can use pixel_format = "uyvy" (uyvy422 is an alias). The generic V4L2 path preserves the capture stride and passes UYVY unchanged to V4L2 M2M, so the encoder device must advertise UYVY input.

[stream.usb-cam]
[[stream.usb-cam.sources]]

[stream.usb-cam.sources.capture]
backend = "v4l2"
device = "/dev/video2"
width = 640
height = 480
fps = 30
pixel_format = "yuyv"

[stream.usb-cam.sources.encoder]
backend = "v4l2-m2m"
codec = "h264"
bitrate = 1_000_000
profile = "42001f"
gop = 60

[stream.usb-cam.sources.output]
payload_type = 96
clock_rate = 90000

RDK X5

[stream.rdk-cam]
[[stream.rdk-cam.sources]]

[stream.rdk-cam.sources.capture]
backend = "v4l2"
device = "/dev/video0"
width = 640
height = 480
fps = 30
pixel_format = "yuyv"

[stream.rdk-cam.sources.encoder]
backend = "rdk"
codec = "h264"
bitrate = 1_000_000
profile = "42001f"
gop = 60

[stream.rdk-cam.sources.output]
payload_type = 96
clock_rate = 90000

Rockchip RKMPP (RK3588 / RV1126B)

[stream.rk-cam]
[[stream.rk-cam.sources]]

[stream.rk-cam.sources.capture]
backend = "v4l2"
device = "/dev/video0"
width = 1280
height = 720
fps = 60
pixel_format = "nv12"
prefer_dmabuf = true      # DMA-BUF zero-copy (capture side)

[stream.rk-cam.sources.encoder]
backend = "rkmpp"
codec = "h264"
bitrate = 4_000_000
profile = "420028"
gop = 120
prefer_dmabuf = true      # DMA-BUF zero-copy (encoder side)

[stream.rk-cam.sources.output]
payload_type = 96
clock_rate = 90000

prefer_dmabuf must be set on both sides for the zero-copy path — see Zero-copy (DMA-BUF) for the semantics and platform support. Input must be NV12.

Zero-copy (DMA-BUF)

With zero-copy enabled, the capture backend hands each frame's dma-buf straight to the encoder — the frame never touches the CPU. Without it, every frame is copied twice on the way in: once into a CPU staging buffer at capture, once into the encoder's input buffer.

Buffer ownership is explicit: a captured buffer is requeued only after the encoder hardware has finished reading it (a deferred-requeue release contract), so the camera can never overwrite a frame mid-encode. If a buffer cannot be exported or imported, or its layout does not match the encoder's (e.g. stride padding), the pipeline falls back per frame to a single copy — the pre-zero-copy cost — and logs it once.

Enabling: prefer_dmabuf on both sides

Zero-copy is an ownership contract between two independent ends, so both must opt in:

Flag	Allows that side to ...
capture.prefer_dmabuf	hold buffers off the capture queue and hand their ownership to the consumer (deferred requeue)
encoder.prefer_dmabuf	import foreign dma-bufs and hold them until the hardware has finished reading

With either flag unset the pipeline still runs — it simply uses the CPU-copy path (never an error). The same two flags configure zero-copy on every supported platform, so a Rockchip RK3588 / RV1126B config (the RKMPP sample above) has exactly the same shape as the Raspberry Pi one:

[stream.cam.sources.capture]
backend = "libcamera"
device = "0"
width = 1280
height = 720
fps = 30
pixel_format = "yuv420"
prefer_dmabuf = true      # DMA-BUF zero-copy (capture side)

[stream.cam.sources.encoder]
backend = "v4l2-m2m"
codec = "h264"
bitrate = 2_000_000
prefer_dmabuf = true      # DMA-BUF zero-copy (encoder side)

Confirm it engaged from the startup log — [V4l2M2mEncoder] zero-copy input enabled; a later frame layout not dmabuf-importable line means the copy fallback is in use (and tells you which layout parameter mismatched).

Platform support

Platform	Capture backend	Encoder backend	Status
Raspberry Pi	libcamera	v4l2-m2m	supported
Raspberry Pi / generic Linux	v4l2 (single-plane NV12 or YUV420)	v4l2-m2m	supported
Rockchip RK3588 / RV1126B	v4l2 (NV12)	rkmpp	supported
RDK X5	—	rdk	not implemented (always copies)

libcamera vs v4l2 capture zero-copy

	libcamera	v4l2
Buffer origin	libcamera-allocated dma-bufs (already a single fd)	driver MMAP buffers exported with VIDIOC_EXPBUF (needs driver support)
Armed layouts	YUV420P — 3 planes at offsets in one fd	single-plane contiguous NV12 or YUV420; multi-fd NV12M / YUV420M stay on the CPU path
Buffer count	12 (up to 8 held by the encoder + ISP headroom)	16
Pixel conversion	none — ISP output used as-is, stride padding exposed	none for the armed formats

In both cases the encoder imports the dma-buf only when the capture stride and plane offsets exactly match its own single-plane layout; anything else takes the one-copy fallback through a dma_heap pool. On Raspberry Pi both the ISP and the bcm2835-codec round strides to 64 bytes, so the usual widths (320 / 640 / 1280 / 1296 / 1920) all import cleanly.

Benchmarks (Raspberry Pi Zero 2 W)

Measured with this build, Debian 13, OV5647, libcamera → v4l2-m2m H.264 baseline 4.0, no subscribers attached; steady-state % of one CPU core:

Stream	copy path	zero-copy	saving
1296x972@30	~55%	~12%	~4.5x
1920x1080@30	~65%	~12%	~5x
640x480@60	~40%	~15%	~2.7x
1280x720@10	~17%	~4.5%	~3.7x
320x240@10	~5.5%	~3%	~1.8x

What remains on the zero-copy path is per-frame fixed cost (driver ioctls, RTP packetization), not per-pixel cost — which is why high-fps and small-frame streams benefit least in relative terms, while 1080p@30 (hard to sustain next to anything else on the copy path) becomes cheap.

Raspberry Pi notes

Verified on a Raspberry Pi Zero 2 W (Debian 13) with the OV5647 (v1) camera.

Sensor frame-rate limits

fps in the capture config is a request: libcamera clamps it to what the selected sensor mode allows. Measured per-mode ceilings (OV5647; the selected mode's limit is also printed by rpicam-hello at startup):

Mode	Max fps
640x480	~62.5
1296x972	~46
1920x1080	~33
2592x1944	~15.6

• 60 fps is only reachable at 640x480 on this sensor; asking for more simply runs at the ceiling. 120 fps is not possible.
• Rough CPU cost on a Zero 2 W (libcamera → v4l2-m2m H.264): ~20% of one core at 640x480@30, ~40% at 640x480@60, ~65% at 1296x972@30.
• Zero-copy removes most of that cost — see Zero-copy (DMA-BUF) for enablement, the libcamera vs v4l2 comparison, and per-resolution benchmark data from this board.

Low-latency streaming

Glass-to-glass latency is dominated by frame cadence, not encoder speed: every stage (sensor readout, capture queue, encoder input, network, player jitter buffer) can hold a frame for up to one frame period. Doubling the frame rate halves the per-stage worst case — 30→60 fps (33.3 → 16.7 ms) is typically worth 30–50 ms end-to-end.

The trade-off is exposure: the per-frame exposure budget is 1/fps (16.7 ms at 60 fps, 8.3 ms at 120 fps). In low light the picture gets darker and noisier. On Raspberry Pi the pipeline sets FrameDurationLimits from fps, so the frame rate stays locked — the AE can only shorten exposure and raise gain (a darker/noisier image, but latency is preserved).

Latency budget on a LAN (Raspberry Pi)

No glass-to-glass number is published for the Pi yet — measure yours with the QR-code timer method (camera films a millisecond timer on a high-refresh display; photograph the WHEP playback next to the timer). The budget at 640x480@60 on a LAN looks like:

• Sensor cadence + ISP + capture: 1–2 frame periods (17–33 ms at 60 fps).
• v4l2-m2m encoder: a few ms per frame; it keeps up with 60 fps at VGA with plenty of headroom on a Zero 2 W (~40% of one core).
• LAN network: a few ms with host-candidate ICE on the same segment.
• Browser player: typically the largest single share — Chromium jitter buffering, decode and render usually account for half or more of the total.

So after frame rate, the remaining lever is the player (decode/render path and jitter buffer), not the server.

60 fps reference config (Raspberry Pi)

[stream.cam]
[[stream.cam.sources]]

[stream.cam.sources.capture]
backend = "libcamera"
device = "0"
width = 640
height = 480
fps = 60            # OV5647 ceiling is ~62.5 at VGA
pixel_format = "yuv420"

[stream.cam.sources.encoder]
backend = "v4l2-m2m"
codec = "h264"
bitrate = 2_000_000
profile = "baseline"
level = "3.1"
gop = 120           # 2 s at 60 fps; a shorter GOP joins/recovers faster

[stream.cam.sources.output]
payload_type = 96
clock_rate = 90000

• Confirm the real rate from the encoder stats line ([V4l2M2mEncoder] stats: injected=…): it advances at the actual capture fps.
• Keep players on the same LAN (host-candidate ICE, ice_udp_addrs = ["auto"]) to avoid relay detours.
• Browser WHEP clients should gather ICE candidates before POSTing (full-SDP offer); trickle-only clients currently fail to connect — see issue #417.