Sometimes OpenH264 returns a key frame even though we have not
requested one. However, SVC controller does not know about this
and will not reset its state. Since we are comparing expected tid
from SVC controller with actual tid from OpenH264, and drop frames
if they do not match, that causes a missing frame.
This CL resets the SVC controller state on key frames, ensuring
that it accurately maintains its state and does not drop frames.
Also, changes the message of the error log to be more descriptive.
Now, it will print the expected tid and actual tid.
Bug: webrtc:14877
Change-Id: I6c9e7532b2478773f03e5707bf7a1ca56e4f7b99
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/324001
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40972}
To avoid name collision with Timestamp type,
To avoid confusion with capture time represented as Timestamp
Bug: webrtc:9378
Change-Id: I8438a9cf4316e5f81d98c2af9dc9454c21c78e70
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/320601
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40796}
This uses the field trial introduced is crbug.com/1406331 and
extends the usage to OpenH264. This simplifies experimentation
whether this change improves performance without requiring
multi-slice encoding.
BUG=webrtc:14368
Change-Id: I0031e59059f7113dd5453234869c957d46f311bb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/294340
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#39371}
As part of go/unblocking-vp9-simulcast (Step 1), EncodedImage is being
upgraded to be able to differentiate between what is a simulcast index
and what is a spatial index.
In order not to break existing code assuming that "if codec != VP9,
SpatialIndex() is the simulcast index", SimulcastIndex() has fallback
logic to return the value of spatial_index_ in the event that
SetSimulcastIndex() has not been called. This allows migrating external
code from (Set)SpatialIndex() to (Set)SimulcastIndex(). During this
intermediate time, codec gates are still necessary in some places of
the code, see TODOs added.
In a follow-up CL, after having fixed dependencies, we'll be able to
remove the fallback logic and rely on SimulcastIndex() and
SpatialIndex() actually being the advertised index and "if codec..."
hacks will be a thing of the past!
Bug: webrtc:14884
Change-Id: I70095c091d0ce2336640451150888a3c3841df80
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/293343
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39318}
This is in the webrtc-stats spec at
https://www.w3.org/TR/webrtc-stats/#dom-rtcoutboundrtpstreamstats-scalabilitymode.
This adds the scalability mode to CodecSpecificInfo which is used to
plumb the modes for each simulcast layer.
TBR=orphis@webrtc.org
Tested: Compiled into Chrome and confirmed the scalability mode set for AV1, VP9, VP8 and H264 software encoders in chrome://webrtc-internals.
Bug: webrtc:14730
Change-Id: I71ceba8f6485a4f4a73e0856031b8d5f16f913f2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/285085
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38847}
Passes frame_types to the underlying encoder in bypass mode.
For libvpx this has no effect, for H264 this changes the behavior
to allow generating keyframes on a per-layer basis.
BUG=chromium:1354101
Change-Id: I26fc22d9e2ec4681a57ce591e9eafd0b1ec962b0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/285083
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#38821}
Openh264 switched from api/svc to api/wels as the location for some
codec header files. During the transition it was necessary to
conditionally from either the old or new location, but now that the
switch is completed and has settled for about two weeks the conditionals
can be removed. This finishes the #include transition started by
webrtc-review.googlesource.com/c/280800
Bug: chromium:1218384
Change-Id: Ic0847428d134687908cc26fec1fdec0c612674b2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/281561
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38622}
This should be landed after https://chromium-review.googlesource.com/c/chromium/src/+/3986032
Bug: chromium:1218384
Change-Id: Id4104d2914f811e722a083021f515fd06b69b910
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/280800
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Bruce Dawson <brucedawson@google.com>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38515}
This reverts commit bc8a62b244.
Reason for revert: reverting per
https://bugs.chromium.org/p/webrtc/issues/detail?id=14368#c5
This needs more careful consideration and should be put behind a finch flag or origin trial
Original change's description:
> Enable Multithreaded H264 Encoding For OpenH264
>
> Re-enabled multithreaded encoding using OpenH264, as the issue described in crbug.com/583348 no longer applies.
>
> Bug: webrtc:14368
> Change-Id: I5ae768a6edf3b40d99c13fb4ee4662626c993a66
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/271820
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#37837}
Bug: webrtc:14368
Change-Id: Icebedfe4eb8e3901670b9f90e229379fca95206b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/272600
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37878}
Re-enabled multithreaded encoding using OpenH264, as the issue described in crbug.com/583348 no longer applies.
Bug: webrtc:14368
Change-Id: I5ae768a6edf3b40d99c13fb4ee4662626c993a66
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/271820
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37837}
Add implementation of RTC_DCHECK_NOTREACHED equal to the RTC_NOTREACHED.
The new macros will replace the old one when old one's usage will be
removed. The idea of the renaming to provide a clear signal that this
is debug build only macros and will be stripped in the production build.
Bug: webrtc:9065
Change-Id: I4c35d8b03e74a4b3fd1ae75dba2f9c05643101db
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237802
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35348}
iNumRefFrame specifies total number of reference buffers to allocate.
For N temporal layers we need at least (N - 1) buffers to store last
encoded frames of all reference temporal layers.
There is no API in OpenH254 encoder to specify exact set of references
to be used to prediction of a given frame. Encoder can theoretically
use all available references.
Note that there is logic in OpenH264 which overrides iNumRefFrame to
max(iNumRefFrame, N - 1): https://source.chromium.org/chromium/chromium/src/+/main:third_party/openh264/src/codec/encoder/core/src/au_set.cpp;drc=8e90a2775c5b9448324fe8fef11d177cb65f36cc;l=122.
I.e., this change has no real effect. It only makes setup more clear.
Bug: none
Change-Id: If4b4970007e1cc55d8f052ea05213ab2e89a878f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/225480
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34445}
Bilinear is faster but lesser quality, box is best quality. Our code
base has disagreed about which filter to use for quite some time,
causing aliasing bug reports. In an effort to avoid aliasing artifacts
and make our scaling filters more predictable, we're updating all uses
to kFilterBox.
WebRTC already uses kFilterBox everywhere except for these three
places. The main discrepency was between Chromium and WebRTC but that
has already been fixed. This CL fixes the last remaining bilinears.
This brings the WebRTC kFilterBox use count up from 11 to 14 and the
kFilterBilinear use count down from 3 to 0.
Bug: chromium:1212630
Change-Id: I5fe4aa92b9275d65b91ea97925533055d190d317
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/221372
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34248}
Follow-up CL to VP8 and VP9 encoders taking care of mapping.
Context again:
This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
In this CL, VideoStreamEncoder no longer calls GetMappedFrameBuffer() on
behalf of the encoders, since the encoders are now able to either do the
mapping or performs ToI420() anyway.
- Tests for old VSE behaviors are updated to test the new behavior (i.e.
that native frames are pretty much always forwarded).
- The "having to call ToI420() twice" workaround to Android bug
https://crbug.com/webrtc/12602 is added to H264 and AV1 encoders.
Bug: webrtc:12469
Change-Id: Ibdc2e138d4782a140f433c8330950e61b9829f43
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211940
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33548}
In the old jitter buffer the two VCMVideoProtection modes |kProtectionNone| and |kProtectionFEC| could be set on the jitter buffer for it to not wait for NACK and instead generate incomplete frames. This has not been possible for a long time.
Bug: webrtc:9378, webrtc:7408
Change-Id: I0a2d3ec34d721126c1128306d5fad88314f8d59f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/190680
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32513}
INCREASING_ID, which is the default mode, triggers HW reset in chromium
decoder wrapper. Set eSpsPpsIdStrategy to SPS_LISTING to prevent that.
Note that WebRTC always resets the encoder on resolution change. This
makes all strategies except INCREASING_ID essentially equivalent to
CONSTANT_ID.
Bug: chromium:1111273
Change-Id: I37405c97b3390f812d1dcaa111694b3b1d638035
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/190440
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32505}
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.
This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.
Testing: Tested in Chrome with media::VideoFrame adapters.
Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
Max encoder bitrate in WebRTC and OpenH264 are different settings. In
WebRTC it is a cap for encoder target bitrate whilst in OpenH264 it is
a peak bitrate. I.e. OpenH264 is allowed to produce bitrate up to
iMaxBitrate for short time interval. That is not what WebRTC expects.
https://webrtc.googlesource.com/src/+/5ee6967c4edc667688d736c27db6f2e7be00dd0a
disabled encoders re-initialization on min/max bitrate change. Reinit of
some HW encoders takes hundreds of milliseconds and causes video freeze.
I missed that max bitrate is used by OpenH264. This caused regression
described in webrtc:11543.
This change sets iMaxBitrate=UNSPECIFIED_BIT_RATE (which is the default
value). Settings iMaxBitrate=UNSPECIFIED_BIT_RATE disables the frame
dropping logic based on that parameter. But the encoder still will drop
frames based on buffer fullness, https://source.chromium.org/chromium/chromium/src/+/master:third_party/openh264/src/codec/encoder/core/src/ratectl.cpp;l=806-807
Bug: webrtc:10773, webrtc:11543
Change-Id: I728be49e0df8a0d9a8f4438299e4c7b4c1497a78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174745
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31192}
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format
After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.
This primary benefit of this change is a small reduction in binary size.
Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
This CL adds an optional second encoder factory to SimulcastEncoderAdapter,
that can be used to create software fallback adapter per simulcast layer.
It also adds logic to check if the encoder supports simulcast natively, if so
it only allocates a single instance and delegates the simulcast logic to that
encoder instead. This means we will be able to remove EncoderSimulcastProxy.
Bug: webrtc:11000
Change-Id: Ifd5f029cc281ee2cedf9d18efa5e7e460884d6ff
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/155171
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29364}
We want to evaluate more data in order to make better choices in the
bitrate allocators.
In order to freely update the parameter list without
breaking the API many times for projects customizing them, we'll use a
struct instead.
Bug: webrtc:10126
Change-Id: I443f86781c5134950294cdd1e3197a47447cf973
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141418
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28748}
- Don't reset encoder if max/min bitrate changed.
- Removed min/max bitrate DCHECKs from encoder wrappers.
- Reset encoder if start_bitrate changed. Only do this if encoding
has not yet started.
- Updated ReconfigureBitratesSetsEncoderBitratesCorrectly test.
- Removed EncoderSetupPropagatesCommonEncoderConfigValues test since it
was a subset of ReconfigureBitratesSetsEncoderBitratesCorrectly.
Bug: webrtc:10773
Change-Id: Id9cbb2ea229232fd95967819e2a937b26948de9f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144028
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28446}
This is a reland of 11dfff0878
Now that I am sure that WebRTC code is not calling the obsolete
versions, I will just remove the NOT_REACHED and call the
new version from the old ones, so as not to trip up downstream
projects.
Original change's description:
> Inform VideoEncoder of negotiated capabilities
>
> After this CL lands, an announcement will be made to
> discuss-webrtc about the deprecation of one version
> of InitEncode().
>
> Bug: webrtc:10720
> Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Elad Alon <eladalon@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28224}
TBR=sakal@webrtc.org,kwiberg@webrtc.org,sprang@webrtc.org
Bug: webrtc:10720
Change-Id: I46c69e45c190805c07f7e51acbe277d7eebd1600
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141412
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28236}
This reverts commit 11dfff0878.
Reason for revert: Downstream import failure.
Original change's description:
> Inform VideoEncoder of negotiated capabilities
>
> After this CL lands, an announcement will be made to
> discuss-webrtc about the deprecation of one version
> of InitEncode().
>
> Bug: webrtc:10720
> Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Elad Alon <eladalon@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28224}
TBR=sakal@webrtc.org,kwiberg@webrtc.org,eladalon@webrtc.org,kthelgason@webrtc.org,sprang@webrtc.org
Change-Id: I7f833055c67f1f879b01dd8c156ba7b8840e8747
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10720
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141411
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28225}
After this CL lands, an announcement will be made to
discuss-webrtc about the deprecation of one version
of InitEncode().
Bug: webrtc:10720
Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28224}
As this is handled higher up the pipeline in a single
place for all encoders/decoders
Bug: webrtc:10460
Change-Id: I95b0a69aecaf07283c8776ac0d7e85d097e3576b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139882
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28172}
This CL makes it more flexible and easier to include/exclude H264 code
when using other build systems because it delegates the decision to
remove the code to the preprocessor instead of GN.
This CL should be a noop, and for WebRTC/Chromium the GN param
`rtc_use_h264` will still be the only thing to change in order to
include/exclude H264.
Moving code that requires ffmpeg or h264 out of the #ifdef/#endif
part should break the build since dependencies are only added if
`rtc_use_h264=true`.
Bug: webrtc:9213
Change-Id: Ibc04edc2f6b9e51489ffe638d5be4b32959cdca0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/137430
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28055}
