58 lines
1.9 KiB
Markdown
58 lines
1.9 KiB
Markdown
---
|
|
name: linux-multimedia-capture
|
|
description: |
|
|
Guidelines for Linux screen/audio capture, FFmpeg encoding/decoding, and
|
|
hardware acceleration in the screen_cast project.
|
|
disable-model-invocation: false
|
|
---
|
|
|
|
# linux-multimedia-capture
|
|
|
|
Guidelines for Linux screen/audio capture, FFmpeg encoding/decoding, and
|
|
hardware acceleration in the `screen_cast` project.
|
|
|
|
## Scope
|
|
|
|
Use this skill for:
|
|
|
|
- Screen or window capture (PipeWire, xdg-desktop-portal D-Bus interfaces).
|
|
- Audio capture or playback (PipeWire audio nodes).
|
|
- Video encoding/decoding with FFmpeg/libavcodec.
|
|
- Pixel format conversion, scaling (libswscale, libavutil).
|
|
- Hardware acceleration (VAAPI, Vulkan Video, NVENC) via FFmpeg.
|
|
|
|
## Capture
|
|
|
|
- Use PipeWire with the `xdg-desktop-portal` screen-cast portal for Wayland and
|
|
X11 compatibility.
|
|
- Prefer screencast source types that request monitor, window, or virtual.
|
|
- Treat portal session tokens, node IDs, and stream FDs as owned resources;
|
|
close and destroy them on error paths.
|
|
|
|
## Encoding / decoding
|
|
|
|
- Use `AVCodecContext`, `AVFrame`, and `AVPacket` through RAII wrappers.
|
|
- Validate codec capabilities (e.g. supported pixel formats, thread counts).
|
|
- For H.264 baseline path use `libx264` or `h264_vaapi` when available.
|
|
- Parse NAL units carefully before packetizing; reject malformed data.
|
|
|
|
## Hardware acceleration
|
|
|
|
- Probe available encoders with `avcodec_find_encoder_by_name()`.
|
|
- Prefer VAAPI on Intel/AMD, NVENC on NVIDIA, Vulkan Video where supported.
|
|
- Keep a software fallback path for compatibility.
|
|
|
|
## Security / safety
|
|
|
|
- Bounds-check all buffer sizes before copying into packets.
|
|
- Validate resolution, stride, and pixel-format assumptions from the portal.
|
|
- Avoid exposing capture tokens or session handles in logs.
|
|
|
|
## References
|
|
|
|
- PipeWire API docs and `libpipewire-0.3`
|
|
- `libportal` (optional wrapper for the desktop portal)
|
|
- FFmpeg documentation: `doc/APIchanges`, `doc/examples`
|
|
- `docs/ARCHITECTURE.md`
|
|
- `docs/PHASES.md`
|