perf(codec): pass YUV through to the renderer and use slice threading

The receiver decoded H.264 to YUV420P, converted it to RGBA via a
CPU-intensive swscale pass, then uploaded 4 bytes/pixel to an SDL
texture — only for the GPU to convert back to RGB during rendering.
This eliminated the swscale pass entirely (40-60% of receiver CPU at
1080p) and cut the texture upload by 62%.

- DecodedFrame now carries three YUV420P planes with their strides
  instead of a packed RGBA buffer; the decoder copies the planes
  directly from the AVFrame (zero conversion for the common software
  path). Non-YUV420P decoder output (e.g. NV12 from v4l2m2m) is
  converted once to YUV420P.
- The SDL renderer uploads via SDL_UpdateYUVTexture with
  SDL_PIXELFORMAT_IYUV; the GPU does the YUV→RGB conversion during
  rendering.
- Decoder threading: slice-level with 4 threads (parallelizes within a
  frame, no added latency), not frame-level (which buffers multiple
  frames — the initial thread_count=0 broke the loopback test because
  the H.264 decoder introduced a multi-frame delay before producing
  output).
- The round-trip test converts decoded YUV back to RGBA for pixel
  comparison via a test-local swscale call (the pipeline itself never
  converts).

meson test 5/5 in both configurations, valgrind clean.
This commit is contained in:
2026-09-09 09:45:22 +02:00
parent b4d1411fd9
commit 6516b45b02
5 changed files with 122 additions and 35 deletions
+1 -1
View File
@@ -90,7 +90,7 @@ struct ReceiverSink {
for (const sc::DecodedFrame& decoded : sc::codec_value(decoded_result)) {
last_width = decoded.width;
last_height = decoded.height;
saw_frame = decoded.width > 0 && !decoded.rgba_pixels.empty();
saw_frame = decoded.width > 0 && !decoded.plane_y.empty();
decoded_frames.fetch_add(1);
}
}
+43 -3
View File
@@ -1,7 +1,12 @@
#include "screencast/codec/decoder.h"
#include "screencast/codec/encoder.h"
#include <algorithm>
// swscale is a C library; without the extern wrapper its functions get
// C++ mangled and the linker cannot find them.
extern "C" {
#include <libswscale/swscale.h>
}
#include <cassert>
#include <cstdint>
#include <cstdlib>
@@ -47,10 +52,45 @@ sc::CapturedFrame make_frame(uint32_t index) {
return frame;
}
// Convert a decoded YUV420P frame back to RGBA for pixel comparison with
// the original capture. Test-only; the pipeline itself never converts.
std::vector<std::byte> decoded_to_rgba(const sc::DecodedFrame& decoded) {
const std::size_t rgba_size = static_cast<std::size_t>(decoded.width) * decoded.height * 4;
std::vector<std::byte> rgba(rgba_size);
const uint8_t* src_planes[4] = {
reinterpret_cast<const uint8_t*>(decoded.plane_y.data()),
reinterpret_cast<const uint8_t*>(decoded.plane_u.data()),
reinterpret_cast<const uint8_t*>(decoded.plane_v.data()),
nullptr,
};
const int src_strides[4] = {decoded.stride_y, decoded.stride_u, decoded.stride_v, 0};
uint8_t* dst_planes[4] = {reinterpret_cast<uint8_t*>(rgba.data()), nullptr, nullptr, nullptr};
const int dst_strides[4] = {decoded.width * 4, 0, 0, 0};
SwsContext* scaler = sws_getContext(decoded.width,
decoded.height,
AV_PIX_FMT_YUV420P,
decoded.width,
decoded.height,
AV_PIX_FMT_RGBA,
SWS_BILINEAR,
nullptr,
nullptr,
nullptr);
assert(scaler != nullptr);
(void)sws_scale(scaler, src_planes, src_strides, 0, decoded.height, dst_planes, dst_strides);
sws_freeContext(scaler);
return rgba;
}
int max_channel_difference(const sc::DecodedFrame& decoded, const sc::CapturedFrame& expected) {
const auto* decoded_pixels = reinterpret_cast<const std::uint8_t*>(decoded.rgba_pixels.data());
const auto decoded_rgba = decoded_to_rgba(decoded);
const auto* decoded_pixels = reinterpret_cast<const std::uint8_t*>(decoded_rgba.data());
const auto* expected_pixels = reinterpret_cast<const std::uint8_t*>(expected.pixels.data());
const std::size_t count = std::min(decoded.rgba_pixels.size(), expected.pixels.size());
const std::size_t count = std::min(decoded_rgba.size(), expected.pixels.size());
int max_diff = 0;
for (std::size_t i = 0; i < count; ++i) {