All valid scalability modes should be supported by the builtin
software decoder/encoder.
Bug: chromium:1187565
Change-Id: If66105d210d5055019f35dae2f80a18ad4a70cdd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/222642
Commit-Queue: Johannes Kron <kron@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34998}
Unlike libvpx, the VideoBitrateAllocation expects that the bitrate
allocation is separate for each temporal layer. In this instance, if the
bitrates are not separated it will fool the SVC controller into thinking
that all temporal layers are always active.
Bug: webrtc:11999
Change-Id: Ibc33ac00b8b7716c011b94e1ec9c640cedb5274e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231693
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Emil Lundmark <lndmrk@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34980}
This cover scenario where target bitrate is changed in a middle of
of group of frame after spatial upswitch.
This change should avoid wasting encoder resources to produce those
frames, reduce number of errors
"Encoder produced a frame for layer that wasn't requested"
Bug: webrtc:11999
Change-Id: I06045259b1cee2c21bfdabbafff3892b57c82a84
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/230543
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34969}
This is a reland of 3097008de0
Patchset 1 is a pure reland. Patchset 2 contains a bugfix plus a test
covering that case.
Bug: webrtc:12354, chromium:1230448
Original change's description:
> Rename vp9::FrameInfo to vp9::UncompressedHeader and add more fields.
>
> These fields will be used for bitstream validation in upcoming CLs.
> A new vp9_constants.h file is also added, containing common constants
> defined by the bitstream spec.
>
> Bug: webrtc:12354
> Change-Id: If04256d83409069c8bee43ad41aed41c3707dfd3
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226060
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Philip Eliasson <philipel@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#34476}
Bug: webrtc:12354
Change-Id: Ibd301eb458a6104b562cefbc0e616c39b54fb38b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/229060
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34789}
This CL completes the removal of assert() and relative headers from
the codebase (excluded
//examples/objc/AppRTCMobile/third_party/SocketRocket which is in a
third_party sub-directory).
Bug: webrtc:6779
Change-Id: I93ed57168d2c0e011626873d66529488c5f484f2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/225546
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34528}
This reverts commit 3097008de0.
Reason for revert: suspected crash
Bug: chromium:1230239
TBR=philipel@webrtc.org
Original change's description:
> Rename vp9::FrameInfo to vp9::UncompressedHeader and add more fields.
>
> These fields will be used for bitstream validation in upcoming CLs.
> A new vp9_constants.h file is also added, containing common constants
> defined by the bitstream spec.
>
> Bug: webrtc:12354
> Change-Id: If04256d83409069c8bee43ad41aed41c3707dfd3
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226060
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Philip Eliasson <philipel@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#34476}
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:12354
Change-Id: Ia4d5180d593c66a053d5747e714a579c62ea2a37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226327
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34496}
These fields will be used for bitstream validation in upcoming CLs.
A new vp9_constants.h file is also added, containing common constants
defined by the bitstream spec.
Bug: webrtc:12354
Change-Id: If04256d83409069c8bee43ad41aed41c3707dfd3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226060
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34476}
CL partially auto-generated with:
git grep -l "\bassert(" | grep "\.[c|h]" | \
xargs sed -i 's/\bassert(/RTC_DCHECK(/g'
And with:
git grep -l "RTC_DCHECK(false)" | \
xargs sed -i 's/RTC_DCHECK(false)/RTC_NOTREACHED()/g'
With some manual changes to include "rtc_base/checks.h" where
needed.
A follow-up CL will remove assert() from Obj-C code as well
and remove the #include of <assert.h>.
The choice to replace with RTC_DCHECK is because assert()
is because RTC_DCHECK has similar behavior as assert()
based on NDEBUG.
This CL also contains manual changes to switch from
basic RTC_DCHECK to other (preferred) versions like
RTC_DCHECK_GT (and similar).
Bug: webrtc:6779
Change-Id: I00bed8886e03d685a2f42324e34aef2c9b7a63b0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/224846
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34442}
In cases where ToI420 fails it should be able to return null.
Bug: webrtc:12877
Change-Id: Ia13859c104d978a29712ae10f8e15acada8406ac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/222613
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#34342}
This is a refactor to simplify a follow-up CL of adding
SdpVideoFormat::IsSameCodec.
The original files media/base/h264_profile_level_id.* and
media/base/vp9_profile.h must be kept until downstream projects
stop using them.
Bug: chroimium:1187565
Change-Id: Ib39eca095a3d61939a914d9bffaf4b891ddd222f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/215236
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33782}
This is a follow-up to the VP9, fixing VP8 this time. Context again:
This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
- To achieve this, WebRTC encoders are updated to map kNative video
buffers so that in a follow-up CL VideoStreamEncoder can stop mapping
intermediate buffer sizes.
Bug: webrtc:12469, chromium:1157072
Change-Id: I026527ae77e36f66d02e149ad6fe304f6a8ccb05
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212600
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33537}
This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
- To achieve this, WebRTC encoders are updated to map kNative video
buffers so that in a follow-up CL VideoStreamEncoder can stop mapping
intermediate buffer sizes.
In this CL LibvpxVp9Encoder is updated to map kNative buffers of pixel
formats it supports and convert ToI420() if the kNative buffer is
something else. A fake native buffer that keeps track of which
resolutions were mapped, MappableNativeBuffer, is added.
Because VP9 is currently an SVC encoder and not a simulcast encoder, it
does not need to invoke CropAndScale.
This CL also fixes MultiplexEncoderAdapter, but because it simply
forwards frames it only cares about the pixel format when
|supports_augmented_data_| is true so this is the only time we map it.
Because this encoder is not used with kNative in practise, we don't care
to make this path optimal.
Bug: webrtc:12469, chromium:1157072
Change-Id: I74edf85b18eccd0d250776bbade7a6444478efce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33526}
Since for such frame SvcController haven't setup how buffer should be
referenced and updated, the frame would likely have unexpected configuration.
Log an error to note resource have been wasted produce it and drop such frame.
Bug: webrtc:11999
Change-Id: I1784403e67b7207092d46016510460738994404e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205140
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33148}
Using WebRTC-VP9-PerformanceFlags and settings a multi-layer config,
and then configuring the codec in non-svc mode would cause us to not
set the cpu speed in libvpx. For some reason, that could trigger a
crash in the encoder.
This CL fixes that, and adds new test coverage for the code affected
byt the trial.
Bug: chromium:1167353, webrtc:11551
Change-Id: Iddb92fe03fc12bac37717908a8b5df4f3d411bf2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202761
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33051}
This adds a new way to poll decoder metadata.
A default implementation still delegates to the old methods.
Root call site is updates to not use the olds methods.
Follow-ups will dismantle usage of the olds methods in wrappers.
Bug: webrtc:12271
Change-Id: Id0fa6863c96ff9e3b849da452d6540e7c5da4512
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196520
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32976}
Also moves the LibvpxVp8Interface from codec/vp8 to codec/interface and
drops vp8 from the name.
Follow-up CLs will wire up actual usage in the new classes through the
interface so that we can unit test them more easily.
Bug: webrtc:12274
Change-Id: I95f66e90245d9320e5fc23cdc845fbeb2648b38b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196522
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32816}
This is a reland of 6e7167456b
Patch set 1 is the original.
Later patch sets fix a parsing bug, and adds a new flag which enables
or disabled the ability to set separate per spatial layer speed
(use_per_layer_speed).
Original change's description:
> Adds experimental libvpx VP9 speed settings.
>
> Using the field trial WebRTC-VP9-PerformanceFlags, this CL allows you to
> configure the libvpx VP9 encoder with a list of flags to affect the
> quality vs speed tradeoff. This CL adds support for:
>
> * Speed (effort), for the temporal base layer frames
> * Speed for higher (non-base) layer frames
> * De-blocking (as part of the loopfilter) enabled for:
> 0 = all frames
> 1 = all but frames from the highest temporal layer
> 2 = no frames
>
> Each entry in the list has a threshold in min number of pixels needed
> for settings in the entry to apply.
>
> Example: Two spatial layers (180p, 360p) with three temporal
> layers are configured. Field trial "WebRTC-VP9-PerformanceFlags" set to:
> "use_per_layer_speed,min_pixel_count:0|129600,base_layer_speed:5|7,high_layer_speed:8|8,deblock_mode:1|2"
> This translates to:
> S0:
> - TL0: Speed 5, deblocked
> - TL1: Speed 8, deblocked
> - TL2: Speed 8, not deblocked
> S1:
> - TL0: Speed 7, not deblocked
> - TL1: Speed 8, not deblocked
> - TL2: Speed 8, not deblocked
>
> Bug: webrtc:11551
> Change-Id: Ieef6816d3e0831ff53348ecc4a90260e2ef10422
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/188461
> Reviewed-by: Michael Horowitz <mhoro@webrtc.org>
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32749}
Bug: webrtc:11551
Change-Id: Ie7c703eb122197235d8ce77cb72db7a347382468
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196345
Reviewed-by: Michael Horowitz <mhoro@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32780}
This reverts commit 6e7167456b.
Reason for revert: Unexpected perf change
Original change's description:
> Adds experimental libvpx VP9 speed settings.
>
> Using the field trial WebRTC-VP9-PerformanceFlags, this CL allows you to
> configure the libvpx VP9 encoder with a list of flags to affect the
> quality vs speed tradeoff. This CL adds support for:
>
> * Speed (effort), for the temporal base layer frames
> * Speed for higher (non-base) layer frames
> * De-blocking (as part of the loopfilter) enabled for:
> 0 = all frames
> 1 = all but frames from the highest temporal layer
> 2 = no frames
>
> Each entry in the list has a threshold in min number of pixels needed
> for settings in the entry to apply.
>
> Example: Two spatial layers (180p, 360p) with three temporal
> layers are configured. Field trial "WebRTC-VP9-PerformanceFlags" set to:
> "min_pixel_count:0|129600,base_layer_speed:5|8,high_layer_speed:7|8,deblock_mode:1|2"
> This translates to:
> S0:
> - TL0: Speed 5, deblocked
> - TL1: Speed 8, deblocked
> - TL2: Speed 8, not deblocked
> S1:
> - TL0: Speed 7, not deblocked
> - TL1: Speed 8, not deblocked
> - TL2: Speed 8, not deblocked
>
> Bug: webrtc:11551
> Change-Id: Ieef6816d3e0831ff53348ecc4a90260e2ef10422
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/188461
> Reviewed-by: Michael Horowitz <mhoro@webrtc.org>
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32749}
TBR=sprang@webrtc.org,ssilkin@webrtc.org,mhoro@webrtc.org
Change-Id: If910963441ac1a0e002aac7066791c7cc7764a1a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:11551
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196344
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32762}
Using the field trial WebRTC-VP9-PerformanceFlags, this CL allows you to
configure the libvpx VP9 encoder with a list of flags to affect the
quality vs speed tradeoff. This CL adds support for:
* Speed (effort), for the temporal base layer frames
* Speed for higher (non-base) layer frames
* De-blocking (as part of the loopfilter) enabled for:
0 = all frames
1 = all but frames from the highest temporal layer
2 = no frames
Each entry in the list has a threshold in min number of pixels needed
for settings in the entry to apply.
Example: Two spatial layers (180p, 360p) with three temporal
layers are configured. Field trial "WebRTC-VP9-PerformanceFlags" set to:
"min_pixel_count:0|129600,base_layer_speed:5|8,high_layer_speed:7|8,deblock_mode:1|2"
This translates to:
S0:
- TL0: Speed 5, deblocked
- TL1: Speed 8, deblocked
- TL2: Speed 8, not deblocked
S1:
- TL0: Speed 7, not deblocked
- TL1: Speed 8, not deblocked
- TL2: Speed 8, not deblocked
Bug: webrtc:11551
Change-Id: Ieef6816d3e0831ff53348ecc4a90260e2ef10422
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/188461
Reviewed-by: Michael Horowitz <mhoro@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32749}
In particular move end_of_picture flag out of vp9 specific information
since VP9 is not the only codec that can use spatial scalability and
thus need to distinguish layer frame and picture (aka temporal unit).
Bug: webrtc:12167
Change-Id: I0d046d8785fbea55281209ad099738c03ea7db96
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/192542
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32588}
In the old jitter buffer the two VCMVideoProtection modes |kProtectionNone| and |kProtectionFEC| could be set on the jitter buffer for it to not wait for NACK and instead generate incomplete frames. This has not been possible for a long time.
Bug: webrtc:9378, webrtc:7408
Change-Id: I0a2d3ec34d721126c1128306d5fad88314f8d59f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/190680
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32513}
This changes the default behavior to use pacing factor of 1.1x instead
of 2.5x, it also sets libvpx rate controler as trusted, turns on the
encoder pushback mechanism and sets spatial hysteresis to 1.2.
The unused "dynamic rate" settings in libvpx is removed.
The new settings matches what has been used in chromium since 2019.
If needed, the legacy behavior can be enabled using the field trial
WebRTC-VideoRateControl.
Bug: webrtc:10155
Change-Id: I8186b491aa5bef61e8f568e96c980ca68f0c208f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186661
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32477}
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.
This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.
Testing: Tested in Chrome with media::VideoFrame adapters.
Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
The trial name WebRTC-VP9-PerLayerSpeed is used to
a) set encoding speed per spatial layer, based on resolution
b) allow explicitly overriding speed per layer, for testing
Additionally, this CL updates the vp9 wrapper in preparation for
injectable trials.
Bug: webrtc:11551, webrtc:11926
Change-Id: I2bb3a664feaef60483ffc241b71070284d3e0172
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186400
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32294}
Otherwise if the pixel format is not I420, the image buffer will
need to be reallocated on each reconfiguration.
Bug: webrtc:11974
Change-Id: Ib13f1865d7dbba4635f57dc09c7bff846e127585
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186340
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32281}
Before this change the allocated buffer and encoder complexity was set
based on the highest resolution configured regardless if that spatial
layer was active or not.
This should reduce memory pressure and improve visual quality when only
a low resolution is requested. In test, increasing the encoder
complexity has paradoxically also resulted in increased decoder speed.
Bug: webrtc:11551
Change-Id: I3ae47a5856de82ff7d40fddfcb160935b12b1d2b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186301
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32280}
NV12 frames can be encoded by libvpx now, and this change allows for
encoding of them with VP9.
VP9 encode/decode tests now run with NV12 as well as I420.
Manually tested using video loopback with VP9 and NV12 generated frames.
out/Default/video_loopback.app/Contents/MacOS/video_loopback --clip=GeneratorNV12 --codec="VP9"
Bug: webrtc:11635, webrtc:11974
Change-Id: Ifc5cbf77d2a27821cd5560c253d5d447c7a7cf53
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185123
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32220}
1) Fix several typos and small mistakes which could lead to crashes
2) Adjust bitrates if leading layers are disabled
3) Wire up webrtc quality scaler
Bug: webrtc:11319
Change-Id: I16e52bdb1c315d64906288e4f2be55fe698d5ceb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177525
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31546}
enable this opt can give 20% performance improvement for video
decoding with 720P video loopback and fake camera on chromebook sarien.
Bug: None
Test: ./modules_tests on chromebook sarien
Change-Id: I8c6487b291b5861e6ba6b6d55b24d7ddb51c341e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177335
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31543}
This is a reland of d592575698
Patchset 2 is a reland of
https://webrtc-review.googlesource.com/c/src/+/177012
Patchset 3 is a fix for a potential crash when InitDecode()is called from
VideoStreamDecoderImpl::GetDecoder(), where the decoder_settings
parameter is a but surprisingly set to nullptr.
Original change's description:
> VP9 decoder: Sets thread count based on resolution, reinit on change.
>
> Previously, number of decoder threads for VP9 were always set to 8 but
> with a cap at number of cores. This was done since we "can't know" the
> resolution that will be used.
>
> With this change, we now intialize the number of threads based on
> resolution given in InitDecode(). If a resolution change happens in
> flight, it requires a keyframe. We therefore parse the header from
> any key frame and if it has a new resolution, we re-initialize the
> decoder.
>
> The number of threads used is based on pixel count. We set one thread
> as target for 1280x720, and scale up lineraly from there. The 8-thread
> cap is gone, but still limit it core count.
>
> This means for instance: 1 <= 720p, 2 for 1080p, 4 for 1440p, 9 for 4K.
>
> Bug: webrtc:11551
> Change-Id: I14c169a6c651c50bd1b870c4b22bc4495c8448fd
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174460
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31507}
Bug: webrtc:11551
Change-Id: I2b4b146d0b8319f07ce1660202d6aa4b374eb015
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177246
Reviewed-by: Johannes Kron <kron@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31527}
Show it can make vp9 tests cleaner too.
Bug: None
Change-Id: I8333a61dec1ef90ade9faffea94e1555ccbfcfaa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177013
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31523}
This reverts commit d592575698.
Reason for revert: May cause crashes.
Original change's description:
> VP9 decoder: Sets thread count based on resolution, reinit on change.
>
> Previously, number of decoder threads for VP9 were always set to 8 but
> with a cap at number of cores. This was done since we "can't know" the
> resolution that will be used.
>
> With this change, we now intialize the number of threads based on
> resolution given in InitDecode(). If a resolution change happens in
> flight, it requires a keyframe. We therefore parse the header from
> any key frame and if it has a new resolution, we re-initialize the
> decoder.
>
> The number of threads used is based on pixel count. We set one thread
> as target for 1280x720, and scale up lineraly from there. The 8-thread
> cap is gone, but still limit it core count.
>
> This means for instance: 1 <= 720p, 2 for 1080p, 4 for 1440p, 9 for 4K.
>
> Bug: webrtc:11551
> Change-Id: I14c169a6c651c50bd1b870c4b22bc4495c8448fd
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174460
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31507}
TBR=ilnik@webrtc.org,sprang@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:11551
Change-Id: Id235c8ded83b3e1fc1d132c8f56c9f20001f6f22
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177242
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31521}
This reverts commit 26e5046951.
Reason for revert: Part of change that may cause crashes.
Original change's description:
> Adjusts allowable thread count for vp9 decoders.
>
> Set 2 thread as target for 1280x720 pixel count, and then scale up
> linearly from there - but cap at physical core count.
> For common resolutions this results in:
> 1 for 360p
> 2 for 720p
> 4 for 1080p
> 8 for 1440p
> 18 for 4K
>
> Bug: webrtc:11551
> Change-Id: I666bd971eccddee096749f20d3b08eb40fe868ad
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177012
> Reviewed-by: Johannes Kron <kron@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31513}
TBR=sprang@webrtc.org,kron@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:11551
Change-Id: I4ea5166efeed3d55255a0243a69deb584a0e19e2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177240
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31517}
Set 2 thread as target for 1280x720 pixel count, and then scale up
linearly from there - but cap at physical core count.
For common resolutions this results in:
1 for 360p
2 for 720p
4 for 1080p
8 for 1440p
18 for 4K
Bug: webrtc:11551
Change-Id: I666bd971eccddee096749f20d3b08eb40fe868ad
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177012
Reviewed-by: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31513}
Previously, number of decoder threads for VP9 were always set to 8 but
with a cap at number of cores. This was done since we "can't know" the
resolution that will be used.
With this change, we now intialize the number of threads based on
resolution given in InitDecode(). If a resolution change happens in
flight, it requires a keyframe. We therefore parse the header from
any key frame and if it has a new resolution, we re-initialize the
decoder.
The number of threads used is based on pixel count. We set one thread
as target for 1280x720, and scale up lineraly from there. The 8-thread
cap is gone, but still limit it core count.
This means for instance: 1 <= 720p, 2 for 1080p, 4 for 1440p, 9 for 4K.
Bug: webrtc:11551
Change-Id: I14c169a6c651c50bd1b870c4b22bc4495c8448fd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174460
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31507}
This is more logical way to remove inactive lower layers.
Current way is to notify the encoder that the layer is inactive,
then renumber layers at the packatization level.
This Cl will allow to simplify libvpx vp9 encoder, svcRateAllocator and
vp9 packetizer.
Bug: webrtc:11319
Change-Id: Idf0bb30b729f5ecc97e31454b32934546b681aa2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173182
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31058}
This CL fixes a few issues where the reported fraction of frames
allocated to various temporal layers could be incorrect:
* In LibvpxVp8Encoder, calling GetEncoderInfo() while not initialized,
or when first configuring with temporal layers and then without,
could trigger incorrect fps allocations.
* In VP9 when different spatial layers have different max framerates,
the layer fps should be compared to the layer with the highest
configured fps, not codec_.maxFramerate which is updated to the
current input fps on SetRates().
* In EncoderBitrateAdjuster, just warn and ignore if a layer has
non-zero bps but zero fps, rather than passing down the chain and
risk weird behavior or divide by zero.
Bug: b/152040235
Change-Id: I548fb3e099b1ec9f536a7b93313fb40c4d32e596
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171516
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30880}
~3-5% speed up on webrtc_perf_tests of vp9 on linux desktop.
Avoid going thru a lot of unnecessary code checks.
Change-Id: I2cb0d794bcf239c5057dfc04cd07a496f89a5016
Bug: None
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167640
Commit-Queue: Jerome Jiang <jianj@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30586}
If e.g. CPU adaptation reduces input video size too much, video pipeline would
reduce the number of used simulcast streams/spatial layers. This may result in
disabled video if some streams are disabled by Rtp encoding parameters API.
Bug: webrtc:11319
Change-Id: Id7f157255599dcb6f494129b83477cda4bea982a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168480
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30498}
This CL uses |width| and |height| in RTPVideoHeaderVP9 to pass information
about enabled layers from encoder to packetizer.
Bug: webrtc:11319
Change-Id: Idc1c337f8dfb3f7631506acb784d2a634b41b955
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167724
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30428}
Chromting is trying vp9 444 to have better color. This fix is needed to decode 444 properly.
Bug: webrtc:11326
Change-Id: I4498930591d8876af9f6b7238a8c9fe450ecbfcc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166220
Commit-Queue: Jerome Jiang <jianj@google.com>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30410}
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format
After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.
This primary benefit of this change is a small reduction in binary size.
Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
Now vp9 screenshare would enable new layers as soon as requested and will force all spatial layers present on the next frame, even if they should be dropped because of frame-rate limiting.
This might cause frame-rate liming to be exceeded if layer is toggling on and off very often, but this situation is bad itself. E.g. in realtime video it will cause too many key-frames.
Now SvcRateAllocator and VP9EncoderImpl are aware that there may be some skipped layers before the first enabled. Key-frames and ss_info triggering logic is also updated.
(This is a reland without changes after updates to downstream projects)
Original-Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153483
Bug: webrtc:10977
Change-Id: I02459c5982da2e0542a837514f5753c5f96401c6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154355
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29330}
This reverts commit 88fe84b7fb.
Reason for revert: Downstream project isn't updated to the latest libvpx roll yet, thus some tests are broken.
Original change's description:
> VP9 encoder: handle disabled layers correctly
>
> Now vp9 screenshare would enable new layers as soon as requested and will
> force all spatial layers present on the next frame, even if they should be
> dropped because of frame-rate limiting.
>
> This might cause frame-rate liming to be exceeded if layer is toggling on
> and off very often, but this situation is bad itself. E.g. in realtime video
> it will cause too many key-frames.
>
> Now SvcRateAllocator and VP9EncoderImpl are aware that there may be some skipped
> layers before the first enabled. Key-frames and ss_info triggering logic is also
> updated.
>
> Bug: webrtc:10977
> Change-Id: Ie2555210c0368a1d3c51ddf6670d0052e6d679de
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153483
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#29296}
TBR=ilnik@webrtc.org,ssilkin@webrtc.org
Change-Id: If33886a5f8a0c3b33168dcadfe45c11a6f4387c1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10977
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154354
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29299}
Now vp9 screenshare would enable new layers as soon as requested and will
force all spatial layers present on the next frame, even if they should be
dropped because of frame-rate limiting.
This might cause frame-rate liming to be exceeded if layer is toggling on
and off very often, but this situation is bad itself. E.g. in realtime video
it will cause too many key-frames.
Now SvcRateAllocator and VP9EncoderImpl are aware that there may be some skipped
layers before the first enabled. Key-frames and ss_info triggering logic is also
updated.
Bug: webrtc:10977
Change-Id: Ie2555210c0368a1d3c51ddf6670d0052e6d679de
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153483
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29296}
Passing an empty arg is working at the moment but it is not
guaranteed to continue to work in the future.
This CL has been generated with:
git grep -l "INSTANTIATE_TEST_SUITE_P(," | xargs sed -i \
"s/INSTANTIATE_TEST_SUITE_P(,/INSTANTIATE_TEST_SUITE_P(All,/g"
Bug: None
Change-Id: Icd2fb9d9d29aed5d692a234124bd990d0f097db4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153890
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29282}
This mode was added by libvpx team specificaly for this usecase: if a
layer is dropped, all lower layers have to be dropped also.
This ensures that higher layers always have higher framerate than the
lower layers and stream is RTP compatible.
This CL also renames full_superframe_drop_ to !layer_buffering, as it
closer reflects the purpose of that flag (in screenshare mode, no
buffering is needed, because the highest layer is always present in the
superframe, yet, it's not a full-superframe dropping mode).
Bug: webrtc:10257
Change-Id: I2589bfd2b9b63de0e410f277a716276234993843
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151764
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29155}
if higher layer is enabled, then disabled, then key-frame is issued, then
the layer is enabled again, the buffer would contain a picture from before
the key-frame and it might have a higher pid than the currently encoded one.
This would trigger the DCHECK. It's safe to remove the DCHECK completely, because
such occasions would cause unsigned overflow and cause the following check for
maximum allowed picture difference to fail and the wrong picture won't
be used as a temporal reference.
This error only caused failures in debug builds and couldn't lead to corruptions
because there're periodical key-frames generated and pid difference can never become so
big that negative value would overflow to something close to 0.
Bug: webrtc:10257
Change-Id: Ie3b3ed0e24421787e3b40a37987ccecb75d04635
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151643
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29099}
