Commit graph

81 commits

Author SHA1 Message Date
Philipp Hancke
b660b7a89c Enable multithreaded OpenH264 encoding behind field trial
This uses the field trial introduced is crbug.com/1406331 and
extends the usage to OpenH264. This simplifies experimentation
whether this change improves performance without requiring
multi-slice encoding.

BUG=webrtc:14368

Change-Id: I0031e59059f7113dd5453234869c957d46f311bb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/294340
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#39371}
2023-02-22 14:26:33 +00:00
Henrik Boström
fbd0ddb32e Introduce WebRTC-VideoEncoderSettings/encoder_thread_limit:X.
As requested by a CEF hosted application (https://crbug.com/1406331)
who want to be able to limit the number of threads in a controlled
environment, this CL adds a flag to control the max limit per encoder.

For plumbing-reasons, this is placed in VideoEncoder::Settings but
with a note that this is considered an experimental API with limited
support. For now only LibvpxVp8Encoder uses it and there are no plans
to roll this out.

I have manually confirmed this is working with printf debugging,
--force-fieldtrials=WebRTC-VideoEncoderSettings/encoder_thread_limit:2
and https://jsfiddle.net/henbos/2bd6m7Lt/

Bug: chromium:1406331
Change-Id: Ib02bd83e2071034874843d3aaa0d3b0adc5bbf46
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/293960
Reviewed-by: Markus Handell <handellm@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39349}
2023-02-20 14:01:32 +00:00
Wan-Teh Chang
bd86684bf3 Make VideoEncoder::GetEncoderInfo() pure virtual
Bug: webrtc:9722
Change-Id: I831a9c460425be86e5da2761769b8eecf231462f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/293386
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Cr-Commit-Position: refs/heads/main@{#39319}
2023-02-15 17:26:07 +00:00
Wan-Teh Chang
0c4c9be436 video_encoder.h: update kFullFramerate in comment
During code review, kFullFramerate was renamed kMaxFramerateFraction but
the uses of kFullFramerate in a comment were not updated. See
https://webrtc-review.googlesource.com/c/src/+/117900/3..4

Change-Id: I6b3c06b4c5b302e8ba40bde4ba722b94aab191eb
Bug: None
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/292801
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39285}
2023-02-09 15:17:50 +00:00
Chunbo Hua
46ad25119c Make requested_resolution_alignment of webrtc::EncoderInfo as uint32_t.
At the same time, proper names of some parameters are refactored in SimulcastEncoderAdapter.

Bug: None
Change-Id: Ia036e3f362d1394e90aa26b79953c1ffe75e2fe0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/284961
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Chunbo Hua <chunbo.hua@intel.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38870}
2022-12-12 06:02:59 +00:00
Niels Möller
6939f63ca1 Update old TODO comments
Bug: None
Change-Id: I96850df6cfa19303043108a59ef60d7b686ec747
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267661
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37436}
2022-07-05 09:59:33 +00:00
Niels Möller
13d163654a Delete support for has_internal_source
Bug: webrtc:12875
Change-Id: I9683e71e1fe5b24802033ffcb32a531ca685fc6f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/179220
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35353}
2021-11-16 11:29:40 +00:00
Artem Titov
cfea2182f8 Use backticks not vertical bars to denote variables in comments
Bug: webrtc:12338
Change-Id: I89c8b3a328d04203177522cbdfd9e606fd4bce4c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/228246
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34696}
2021-08-10 10:40:03 +00:00
Qiu Jianlin
b54cfdebfe Add optional is_qp_trusted property for EncoderInfo.
Some hardware H.264 encoders does not place average QP delta in
slice_qp_delta field. Adding an optional flag in EncoderInfo to notify
quality scaler about this.

Bug: webrtc:12942
Change-Id: I3ee29c5ae9bd7bb34d26eba7e6bede3798ca44b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226921
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34627}
2021-08-02 13:49:21 +00:00
Artem Titov
0e61fdd27c Use backticks not vertical bars to denote variables in comments for /api
Bug: webrtc:12338
Change-Id: Ib97b2c3d64dbd895f261ffa76a2e885bd934a87f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226940
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34554}
2021-07-26 18:27:34 +00:00
Danil Chapovalov
e904161cec Replace RTC_DEPRECATED with ABSL_DEPRECATED
This remove webrtc-specific macro that has no reason to be webrtc specific
ABSL_DEPRECATED takes a message parameter encouraging to write text how class or function is deprecated.

Bug: webrtc:12484
Change-Id: I89f1398f91dacadc37f7db469dcd985e3724e444
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208282
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33314}
2021-02-22 12:53:23 +00:00
Evan Shrubsole
b556b08668 Allow encoders to receive preferred pixel formats from native buffers
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.

This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.

Testing: Tested in Chrome with media::VideoFrame adapters.

Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
2020-10-08 13:47:50 +00:00
Per Kjellander
d0a8f51ef7 Reland Refactor reporting of VideoBitrateAllocation
Original description
 Move reporting of target bitrate to just after the encoder has been
 updated. Originall submitted as refs/heads/master@{#32275}

Patch 1 contains the original cl
,patch 2 the fix to send rtcp even if BWE does not change.

Bug: webrtc:12000
Change-Id: I16766e08229fe1f6f65f449e0e074bed03338693
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186948
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32340}
2020-10-07 11:06:56 +00:00
Per Kjellander
b8781796d4 Revert "Refactor reporting of VideoBitrateAllocation"
This reverts commit 39a31afb77.

Reason for revert: Will cause RTCP Target bitrate messages to not be sent unless BWE changes. 

Original change's description:
> Refactor reporting of VideoBitrateAllocation
>
> Move reporting of target bitrate to just after the encoder has been
> updated.
>
> Bug: webrtc:12000
> Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32275}

TBR=sprang@webrtc.org,perkj@webrtc.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:12000
Change-Id: Icf21e6ae28dc17c61b9243c037ffef9b623894ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186945
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32337}
2020-10-07 08:14:21 +00:00
Evan Shrubsole
895556e19c Avoid converting frames to I420 in VideoStreamEncoder
This needs to be done still for kNative frames, but all other frame types
can be passed in.

I have checked all VideoEncoder implementations in Chromium and confirmed they either convert the frame to their preferred pixel format, or just
forward the frame to a delegate encoder.

Tested:
- video_loopback with NV12 generated frames for VP9, the only
codec supporting NV12, as well as VP8 which only accepts I420 frames.
- internal_tests tryrun

Bug: webrtc:11976,webrtc:11635
Change-Id: If39a815fb0c5636fceb1040c8946c3db2fb350a1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185803
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32306}
2020-10-05 08:33:46 +00:00
Per Kjellander
39a31afb77 Refactor reporting of VideoBitrateAllocation
Move reporting of target bitrate to just after the encoder has been
updated.

Bug: webrtc:12000
Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32275}
2020-10-01 14:10:09 +00:00
Åsa Persson
c5a74ffba4 Add support so requested resolution alignment also apply to scaled layers.
Bug: webrtc:11872
Change-Id: I7f904e2765330ee93270b66b0102ce57f336f9a0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181883
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32146}
2020-09-21 09:23:22 +00:00
Danil Chapovalov
70b2cf8b36 Delete deprecated version of EncodedImageCallback::OnEncodedImage
Bug: webrtc:6471
Change-Id: I173cd3b3b9f4badaf7c17574adf1d09a926a9b9b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182380
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Commit-Queue: Kári Helgason <kthelgason@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31982}
2020-08-24 11:00:19 +00:00
Danil Chapovalov
0490c37126 Reland "Add EncodedImageCallback::OnEncodedImage without RTPFragmentationHeader"
This is a reland of fdd41ecf6b

Original change's description:
> Add EncodedImageCallback::OnEncodedImage without RTPFragmentationHeader
> 
> RTPFragmentationHeader is no longer used by any webrtc code.
> 
> Bug: webrtc:6471
> Change-Id: I1362d58465c550f5c8e0a5b292dc0abcd6251431
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180120
> Reviewed-by: Sebastian Jansson <srte@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Reviewed-by: Niels Moller <nisse@webrtc.org>
> Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31833}

Bug: webrtc:6471
Change-Id: I7f551110a68704bfe12897772083c3616acd173c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180881
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31848}
2020-08-04 12:17:06 +00:00
Marina Ciocea
65e3044b40 Revert "Add EncodedImageCallback::OnEncodedImage without RTPFragmentationHeader"
This reverts commit fdd41ecf6b.

Reason for revert: Breaks WebRTC roll into Chrome: 
https://crrev.com/c/2335265

Sample failure:
https://ci.chromium.org/p/chromium/builders/try/linux_chromium_compile_dbg_ng/688153?


Original change's description:
> Add EncodedImageCallback::OnEncodedImage without RTPFragmentationHeader
> 
> RTPFragmentationHeader is no longer used by any webrtc code.
> 
> Bug: webrtc:6471
> Change-Id: I1362d58465c550f5c8e0a5b292dc0abcd6251431
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180120
> Reviewed-by: Sebastian Jansson <srte@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Reviewed-by: Niels Moller <nisse@webrtc.org>
> Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31833}

TBR=danilchap@webrtc.org,brandtr@webrtc.org,nisse@webrtc.org,srte@webrtc.org

Change-Id: I6bd021637b0c140f6625612caebdb5829decdfb0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:6471
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180821
Reviewed-by: Marina Ciocea <marinaciocea@webrtc.org>
Commit-Queue: Marina Ciocea <marinaciocea@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31839}
2020-08-03 18:31:07 +00:00
Danil Chapovalov
fdd41ecf6b Add EncodedImageCallback::OnEncodedImage without RTPFragmentationHeader
RTPFragmentationHeader is no longer used by any webrtc code.

Bug: webrtc:6471
Change-Id: I1362d58465c550f5c8e0a5b292dc0abcd6251431
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180120
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31833}
2020-08-03 13:58:17 +00:00
Henrik Boström
b0f2e0ced4 [Overuse] Make VideoStreamAdapter responsible for executing adaptation.
This CL moves GetAdaptUpTarget(), GetAdaptDownTarget() and
ApplyAdaptationTarget() - and related code - to the VideoStreamAdapter.

This includes pieces related to calculating how to adapt, including:
- DegradationPreference
- BalancedDegradationPreference
- AdaptationRequest and last_adaptation_request_
- CanAdaptUpResolution()

The VideoStreamAdapter's interface has changed: VideoSourceRestrictor
methods are now hidden in favor of methods exposing AdaptationTarget.

This CL also does some misc moves:
- GetEncoderBitrateLimits is moved and renamed to
  VideoEncoder::EncoderInfo::GetEncoderBitrateLimitsForResolution.
- EncoderSettings moved to a separate file.

// For api/video_codecs/video_encoder.[cc/h] changes, which is the
// moving of a function.
TBR=sprang@webrtc.org

Bug: webrtc:11393
Change-Id: Ie6bd8ef644ce927d7eca6ab90a0a7bcace682f3c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169842
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30708}
2020-03-06 13:35:20 +00:00
Henrik Boström
4bab2fcf6b [Overuse] Setting encoder configurations through the interface.
This squashes together several input signals that were spread out
through several calls into a single method and calling place:
SetEncoderSettings(), invoked from ReconfigureEncoder(). This is added
to the abstract interface.

This makes the following methods obsolete which are removed:
- SetEncoder(): The VideoEncoder was only used for GetEncoderInfo();
  the VideoEncoder::EncoderInfo is now part of the EncoderSettings.
- SetEncoderConfig(): The VideoEncoderConfig is part of
  EncoderSettings. The config is used for its codec_type and
  content_type enums.
- SetCodecMaxFrameRate(): The max frame rate was the same as
  VideoCodec::maxFramerate. VideoCodec is now part of EncoderSettings.

There may be some overlap in information between EncoderConfig and
VideoCodec, but that is outside the scope of this CL, which only makes
sure to bundle encoder settings-like information into one input signal.

Bug: webrtc:11222
Change-Id: I67c49c49c0a859cb7d5051939a461593c695a789
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166602
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30332}
2020-01-21 11:48:11 +00:00
Rasmus Brandt
5cad55b240 Signal requested resolution alignment requirements from sinks to sources.
Bug: webrtc:11218
Change-Id: I593b0515ea389bece472234a3c4082ccc5321ea5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162400
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30113}
2019-12-19 10:39:04 +00:00
Hirokazu Honda
1154915024 video_encoder: Add RTC_EXPORT ScalingSettings
ScalingSettings has some public constructors. These should be
able to be called from exteranl code. However, a linker fails
on windows because ScalingSettings doesn't have RTC_EXPORT.

Bug: chromium:1031965
Test: build crrev.com/c/1949841
Change-Id: Iddaea77f87c52edbe8f77551322d7aa198bc0aeb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161860
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30063}
2019-12-11 15:14:56 +00:00
Erik Språng
7968530418 Removes caching SimulcastEncoderAdapter::GetEncoderInfo()
There are edge cases where the caching of encoder info will cause
issues. For instance if a sub-encoder fails en Encode call and falls
back to some other implementation, or if the fps targets shift due to
SetRates() triggering new layers to be enabled.

This CL forces a complete rebuild on every call to GetEncoderInfo().

It also adds new logging of when the info changes, as debugging issues
can be very time consuming if we can't tell that happened.

Bug: webrtc:11000
Change-Id: I7ec7962a589ccba0e188e60a11f851c9de874fab
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160960
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29938}
2019-11-27 17:06:54 +00:00
Mirko Bonadei
8fa616f3b0 Add 2 more RTC_EXPORT.
The component build test failed close to the end of compilation because
of these two missing symbols, see [1].

[1] - https://ci.chromium.org/p/chromium/builders/try/win_chromium_compile_dbg_ng/435732

Bug: webrtc:9419
Change-Id: Ic46acf1acbf3bc04e7410f8d9858785739ca98d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/159683
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29766}
2019-11-12 09:44:29 +00:00
Mirko Bonadei
54875d05f1 Add missing RTC_EXPORT for the Chromium Windows build.
After fixing the issue with crbug.com/1018579, lld-link complained
that some symbols need to be exported, see [1].

[1] - https://ci.chromium.org/p/chromium/builders/try/win_chromium_compile_dbg_ng/432025

Bug: webrtc:9419
Change-Id: I9107a9c76361f4c66463a9af2e81a3991ae14df5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/159007
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29721}
2019-11-07 09:32:24 +00:00
Erik Språng
f4e0c29ed1 SimulcastEncoderAdapter: support per layer fallback and single encoder proxying
This CL adds an optional second encoder factory to SimulcastEncoderAdapter,
that can be used to create software fallback adapter per simulcast layer.

It also adds logic to check if the encoder supports simulcast natively, if so
it only allocates a single instance and delegates the simulcast logic to that
encoder instead. This means we will be able to remove EncoderSimulcastProxy.

Bug: webrtc:11000
Change-Id: Ifd5f029cc281ee2cedf9d18efa5e7e460884d6ff
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/155171
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29364}
2019-10-01 17:31:44 +00:00
Evan Shrubsole
7c079f650d Reland "Fix minor regression caused by a8336d3"
This is a reland of 809198edff

A fix was made in https://webrtc-review.googlesource.com/c/src/+/154343
which fixed the regression issues caused by the original patch.

Original change's description:
> Fix minor regression caused by a8336d3
>
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
>
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}

Bug: webrtc:10126
Change-Id: Iecc3ab6a5cd1193a1fa8e824dcf4f0b8165f9bf8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154359
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29356}
2019-10-01 11:49:38 +00:00
Evan Shrubsole
b6a45dda4c Revert "Fix minor regression caused by a8336d3"
This reverts commit 809198edff.

Reason for revert: Performance regressions that need to be addressed.

Original change's description:
> Fix minor regression caused by a8336d3
> 
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
> 
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}

TBR=sprang@webrtc.org,eshr@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:10126
Change-Id: I133cbe5d8cb894ed944ae8a2d0f63a78bbed72ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153484
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29221}
2019-09-18 14:38:15 +00:00
Evan Shrubsole
809198edff Fix minor regression caused by a8336d3
VideoEncoder::SetRates was being called unnessesarily when the fields
appended to RateControlParameters were changed. Since SetRates only
cares about RateControlParameters, it should have only been called if
the RateControlParameters themselves were actually changed.

Bug: webrtc:10126
Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#29208}
2019-09-17 13:34:18 +00:00
Sergey Silkin
3d642f8442 Rename ..BitrateThresholds to ..BitrateLimits.
Bug: webrtc:10798
Change-Id: I1975206323a520b557652760d1d54c01c26a7405
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144540
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28473}
2019-07-03 14:50:46 +00:00
Elad Alon
8f01c4e1b6 Define FecControllerOverride and plumb it down to VideoEncoder
The purpose of this interface is to allow VideoEncoder to override
the bandwidth allocation set by FecController in RtpVideoSender.

This CL defines the interface and sends it down to VideoSender.
Two upcoming CLs will:
1. Make LibvpxVp8Encoder pass it on to the (injectable)
   FrameBufferController, where it might be put to good use.
2. Modify RtpVideoSender to respond to the message sent to it
   via this API.

TBR=kwiberg@webrtc.org

Bug: webrtc:10769
Change-Id: I2ef82f0ddcde7fd078e32d8aabf6efe43e0f7f8a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143962
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28416}
2019-06-28 15:57:22 +00:00
Sergey Silkin
be0adee768 Add resolution bitrate thresholds to EncoderInfo.
When provided, these thresholds will be used instead of WebRTC default
limits specified in DropDueToSize() and GetMaxDefaultVideoBitrateKbps().

Bug: none
Change-Id: Ida45ea832041963b8b8475d69114b5c60a172fb7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/142170
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Alex Glaznev <glaznev@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28390}
2019-06-26 13:51:09 +00:00
Elad Alon
370f93a34a Reland "Inform VideoEncoder of negotiated capabilities"
This is a reland of 11dfff0878

Now that I am sure that WebRTC code is not calling the obsolete
versions, I will just remove the NOT_REACHED and call the
new version from the old ones, so as not to trip up downstream
projects.

Original change's description:
> Inform VideoEncoder of negotiated capabilities
>
> After this CL lands, an announcement will be made to
> discuss-webrtc about the deprecation of one version
> of InitEncode().
>
> Bug: webrtc:10720
> Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Elad Alon <eladalon@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28224}

TBR=sakal@webrtc.org,kwiberg@webrtc.org,sprang@webrtc.org

Bug: webrtc:10720
Change-Id: I46c69e45c190805c07f7e51acbe277d7eebd1600
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141412
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28236}
2019-06-11 14:49:37 +00:00
Philip Eliasson
49d661a7d3 Revert "Inform VideoEncoder of negotiated capabilities"
This reverts commit 11dfff0878.

Reason for revert: Downstream import failure.

Original change's description:
> Inform VideoEncoder of negotiated capabilities
> 
> After this CL lands, an announcement will be made to
> discuss-webrtc about the deprecation of one version
> of InitEncode().
> 
> Bug: webrtc:10720
> Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Elad Alon <eladalon@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28224}

TBR=sakal@webrtc.org,kwiberg@webrtc.org,eladalon@webrtc.org,kthelgason@webrtc.org,sprang@webrtc.org

Change-Id: I7f833055c67f1f879b01dd8c156ba7b8840e8747
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10720
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141411
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28225}
2019-06-11 11:56:04 +00:00
Elad Alon
11dfff0878 Inform VideoEncoder of negotiated capabilities
After this CL lands, an announcement will be made to
discuss-webrtc about the deprecation of one version
of InitEncode().

Bug: webrtc:10720
Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28224}
2019-06-11 11:32:13 +00:00
Erik Språng
157b7814b9 Remove deprecated SetRates/SetRateAllocation from VideoEncoder.
This CL removes two deprecated methods from the VideoEncoder interface:
* int32_t SetRates(uint32_t, uint32_t);
* int32_t SetRateAllocation(const VideoBitrateAllocation&, uint32_t);

These are no longer used, instead the new version must be implemented:
  void SetRates(const RateControlParameters&) = 0;

Migrating is straight forward. For the old SetRates, simple replace:
  int32_t MyEncoder::SetRates(uint32_t bitrate, uint32_t framerate) {
with
  void MyEncoder::SetRates(const RateControlParameters& parameters) {
    uint32_t bitrate = parameters.bitrate.get_sum_kbps();
    uint32_t framerate =
      static_cast<uint32_t>(parameters.framerate_fps + 0.5);

For SetRateAllocation, replace:
  int32_t MyEncoder::SetRateAllocation(
      const VideoBitrateAllocation& allocation,
      uint32_t framerate) {
with
  void MyEncoder::SetRates(const RateControlParameters& parameters) {
    const VideoBitrateAllocation& allocation = parameters.bitrate;
    uint32_t framerate =
      static_cast<uint32_t>(parameters.framerate_fps + 0.5);

Two more things to note:
1. The new method is void. Previously the only use of the return value
   in production code was to log a more or less generic error message.
   Instead, log the actual error from the encoder when it happens,
   then just return.

2. The new method is pure virtual; it must be overriden even in test.

This CL is intended to be landed two weeks after creation, on Thursday
May 9th 2019.

Bug: webrtc:10481
Change-Id: I61349571a280bd40cd100ca9f93c4aa7748ed30d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134214
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27926}
2019-05-13 11:42:59 +00:00
Erik Språng
16cb8f5d74 Reland "Replace usage of old SetRates/SetRateAllocation methods"
This is a reland of 7ac0d5f348

Original change's description:
> Replace usage of old SetRates/SetRateAllocation methods
>
> This rather large CL replaces all relevant usage of the old
> VideoEncoder::SetRates()/SetRateAllocation() methods in WebRTC.
> API is unchanged to allow downstream projects to update without
> breakage.
>
> Bug: webrtc:10481
> Change-Id: Iab8f292ce6be6c3f5056a239d26361962b14bb38
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131949
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Niels Moller <nisse@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27554}

TBR=brandtr@webrtc.org,sakal@webrtc.org,perkj@webrtc.org

Bug: webrtc:10481
Change-Id: I2978d5c527a18e885b7845c4e53a2424e8ad5b4b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/132551
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27593}
2019-04-12 13:37:32 +00:00
Minyue Li
7ddef1af88 Revert "Replace usage of old SetRates/SetRateAllocation methods"
This reverts commit 7ac0d5f348.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> Replace usage of old SetRates/SetRateAllocation methods
> 
> This rather large CL replaces all relevant usage of the old
> VideoEncoder::SetRates()/SetRateAllocation() methods in WebRTC.
> API is unchanged to allow downstream projects to update without
> breakage.
> 
> Bug: webrtc:10481
> Change-Id: Iab8f292ce6be6c3f5056a239d26361962b14bb38
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131949
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Niels Moller <nisse@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27554}

TBR=brandtr@webrtc.org,sakal@webrtc.org,nisse@webrtc.org,sprang@webrtc.org,perkj@webrtc.org

Change-Id: I576760b584e3f258013b0279c0c173c895bbb37e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10481
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/132561
Reviewed-by: Minyue Li <minyue@webrtc.org>
Commit-Queue: Minyue Li <minyue@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27559}
2019-04-11 10:50:29 +00:00
Erik Språng
7ac0d5f348 Replace usage of old SetRates/SetRateAllocation methods
This rather large CL replaces all relevant usage of the old
VideoEncoder::SetRates()/SetRateAllocation() methods in WebRTC.
API is unchanged to allow downstream projects to update without
breakage.

Bug: webrtc:10481
Change-Id: Iab8f292ce6be6c3f5056a239d26361962b14bb38
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131949
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27554}
2019-04-11 07:46:09 +00:00
Elad Alon
20789e477d Name changes inside VideoEncoder
Change names of two members of VideoEncoder to enhance clarity.

Bug: webrtc:10501
Change-Id: I6bcf64a5936f828bc585e54c429f24676ccfb0ae
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131941
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27514}
2019-04-09 11:27:10 +00:00
Erik Språng
4c6ca30019 Update VideoStreamEncoder to use new VideoEncoder::SetRates() method.
This CL wires up the new SetRates() method of the video encoders, and
refactors a few things in the process:

Most notably, the VideoStreamEncoderInterface is update so that the
|target_headroom| parameter is replaced with |link_allocation|, meaning
that instead of indicating bitrate capacity in excess of the target
bitrate, it indicates to total network capacity allocated for the
stream including the target bitrate. This matches the VideoEncoder API.

The VideoEncoder::RateControlParameters struct gets a few new helper
methods.

In VideoStreamEncoder, instead of adding more fields to the
|last_observed_bitrate*| family, uses an optional struct that
inherits from VideoEncoder::RateControlParameters.

Bug: webrtc:10481
Change-Id: Iee3965531142ae9b964ed86c0d51db59b1cdd61c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131123
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27487}
2019-04-08 14:01:28 +00:00
Elad Alon
6c371ca700 Add OnLossNotification() to VideoEncoder and Vp8FrameBufferController
Bug: webrtc:10501
Change-Id: I33e8bfcf16cf24aadcfdf214d7d9bcd495bf9348
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131021
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27449}
2019-04-04 10:57:02 +00:00
Niels Möller
9d766b91df Delete deprecated variant of VideoEncoder::Encode
Bug: webrtc:10379
Change-Id: I027ceb3323d3fea84f478131dee31dff77e4c0aa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/126228
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27346}
2019-03-28 15:26:23 +00:00
Erik Språng
4d9df38d82 Add new more flexible VideoEncoder::SetRate() method.
This CL changes the API for webrtc::VideoEncoder.

There is a legacy method called SetRates(). This is indicated as being
deprecated, but there seem to be a number of usages still left.

Then there is the new SetRateAllocation() method which takes a
VideoBitrateAllocation instance instead of a single target bitrate.

This CL adds a new version of SetRates() which moves all the existing
parameters in a RateControlParameters struct, and adds a bandwidth
allocation signal. The intent of this signal is to allow the encoder
to know how close to the target it needs to stay. If the encoder rate
is network restricted, it will need to be more aggressive in keep the
rate down and possibly drop frames. If there is some margin for
overshoot, it might do different trade-offs.

Furthermore, the frame rate signal is changes from an integer to a
floating point type in order to get more precise rates at low frame
rates and the return type has been changed to void since the only use
of the previous values to log it and that is better done inside encoder
where the error condition originates.

The intent is to properly deprecate the "old" SetRates() /
SetRateAllocation() methods, send out a PSA and then remove them in two
weeks. Changes required by users should be trivial, as using the new
headroom signal is optional.

Bug: webrtc:10155, webrtc:10481
Change-Id: I4f797b0b0c73086111869ef4ee5f42bf530f5fde
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/129724
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27314}
2019-03-27 14:31:11 +00:00
Elad Alon
cde8ab265e Use single FrameBufferController in VP8, created by a factory.
This CL paves the way to making FrameBufferController injectable.

LibvpxVp8Encoder can manage multiple streams. Prior to this CL,
each stream had its own frame buffer controller, all of them held
in a vector by LibvpxVp8Encoder. This complicated the code and
produced some code duplication (cf. SetupTemporalLayers).

This CL:
1. Replaces CreateVp8TemporalLayers() by a factory. (Later CLs
   will make this factory injectable.)
2. Makes LibvpxVp8Encoder use a single controller. This single
   controller will, in the case of multiple streams, delegate
   its work to multiple controllers, but that fact is not visible
   to LibvpxVp8Encoder.

This CL also squashes CL #126046 (Send notifications of RTT and
PLR changes to Vp8FrameBufferController) into it.

Bug: webrtc:10382
Change-Id: Id9b55734bebb457acc276f34a7a9e52cc19c8eb9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/126483
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27206}
2019-03-20 11:54:02 +00:00
Niels Möller
87e2d785a0 Prepare for splitting FrameType into AudioFrameType and VideoFrameType
This cl deprecates the FrameType enum, and adds aliases AudioFrameType
and VideoFrameType.

After downstream usage is updated, the enums will be separated
and be moved out of common_types.h.

Bug: webrtc:6883
Change-Id: I2aaf660169da45f22574b4cbb16aea8522cc07a6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/123184
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27011}
2019-03-07 10:12:57 +00:00
Niels Möller
c8d2e73ed0 Delete CodecSpecificInfo argument from VideoEncoder::Encode
Bug: webrtc:10379
Change-Id: If9f92eb1e5891df284881082c53f0b1db1c26a38
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/125900
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26992}
2019-03-06 14:01:31 +00:00