Commit graph

31 commits

Author SHA1 Message Date
Danil Chapovalov
abf73de8ea Do not propagate generic descriptor on receiving frame
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.

Bug: webrtc:10342
Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30501}
2020-02-11 16:12:16 +00:00
philipel
b3bb2040a1 Remove unused RtpFrameObject ctor.
Bug: webrtc:10979
Change-Id: I9ab8cbd3da4c753f0fa318c41b6e74ddd9679901
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/155172
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29354}
2019-10-01 11:23:26 +00:00
philipel
85d5c197a8 Added RtpFrameObject ctor with no PacketBuffer pointer.
Bug: webrtc:10979
Change-Id: Ie6a2b56e7374d60d1f74d8c315216b27df22a19b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154426
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29314}
2019-09-26 08:55:00 +00:00
Niels Möller
834a554962 Include module_common_types.h only where needed
Bug: None
Change-Id: I73d493f8f186b429c7be808f4dfac0398f150931
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153891
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29277}
2019-09-24 08:22:38 +00:00
philipel
0cff4fce55 Removed unused frame_size param from RtpFrameObject ctor.
Bug: webrtc:10979
Change-Id: Idde493dc7f5165e3ca173d5a38861b444b5904a4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153668
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29253}
2019-09-20 10:56:01 +00:00
philipel
b5e4785464 RtpFrameObject now takes an EncodedImageBuffer in its ctor.
Bug: webrtc:10979
Change-Id: Ibc8b4a524ca95b5faa8850a41df8f2f0136a2969
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153666
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29251}
2019-09-20 10:15:01 +00:00
philipel
76161f7446 Move the call to GetBitstream out of the RtpFrameObject ctor.
Bug: webrtc:10979
Change-Id: I9159eb04d4a371e8ed8f932a989d6b884faa7be7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153661
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29237}
2019-09-19 12:08:04 +00:00
Johannes Kron
a370556270 Refactor to free up PacketBuffer as soon as possible
The packets belonging to a frame were kept in PacketBuffer
until the frame was decoded. This CL clears the dependencies
of an existing RtpFrameObject to PacketBuffer so that we can
free up PacketBuffer as soon as the RtpFrameObject is created.

Bug: none
Change-Id: Ic939be91815519ae1d1c67ada82006417b2d26a3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/149818
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28977}
2019-08-28 08:07:32 +00:00
Chen Xing
f00bf42d1c Add plumbing of RtpPacketInfos to each VideoFrame as input for SourceTracker.
This change adds the plumbing of RtpPacketInfo from RtpVideoStreamReceiver::OnRtpPacket() to VideoReceiveStream::OnFrame() for video. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.

Bug: webrtc:10668
Change-Id: Ib97d430530c5a8487d3b129936c7c51e118889bd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139891
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Chen Xing <chxg@google.com>
Cr-Commit-Position: refs/heads/master@{#28332}
2019-06-20 10:24:29 +00:00
Niels Möller
4d504c76cb New interface EncodedImageBufferInterface, replacing use of CopyOnWriteBuffer
Bug: webrtc:9378
Change-Id: I62b7adbd9dd539c545b5b1b1520721482a4623c4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/138820
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28317}
2019-06-19 07:02:34 +00:00
Niels Möller
9d8eaac4ee Delete unneeded direct includes of common_types.h
And delete corresponding dependencies on :webrtc_common. After this
change, common_types.h is included directly only from code in the
following directories:

api/
api/video/
api/video_codecs/
common_video/libyuv/include/
media/base/
modules/remote_bitrate_estimator/
modules/rtp_rtcp/source/
modules/video_coding/codecs/vp9/

There remains plenty of indirect dependencies on the types declared in
common_types.h, but the fewer direct dependencies should make it
easier to find the proper place for each type.

Bug: webrtc:5876
Change-Id: I93e8f214025ecb613c19fdec2015bd3f96c59aae
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/130501
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27376}
2019-04-01 07:18:13 +00:00
Niels Möller
87e2d785a0 Prepare for splitting FrameType into AudioFrameType and VideoFrameType
This cl deprecates the FrameType enum, and adds aliases AudioFrameType
and VideoFrameType.

After downstream usage is updated, the enums will be separated
and be moved out of common_types.h.

Bug: webrtc:6883
Change-Id: I2aaf660169da45f22574b4cbb16aea8522cc07a6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/123184
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27011}
2019-03-07 10:12:57 +00:00
Niels Möller
77536a2b81 Rename EncodedImage::_length --> size_, and make private.
Use size() accessor function. Also replace most nearby uses of _buffer
with data().

Bug: webrtc:9378
Change-Id: I1ac3459612f7c6151bd057d05448da1c4e1c6e3d
Reviewed-on: https://webrtc-review.googlesource.com/c/116783
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26273}
2019-01-16 07:40:47 +00:00
Ilya Nikolaevskiy
4348ce240a Calculate min and max receive timestamps for packets in a video frame
Bug: webrtc:10106
Change-Id: I1d3469abb1e7bb7c91a5912d7b781505526abaca
Reviewed-on: https://webrtc-review.googlesource.com/c/113507
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25935}
2018-12-07 16:22:34 +00:00
Benjamin Wright
00765297a2 Add BufferedFrameDecryptor to cleanly deal with receiving encrypted frames.
This change introduces a new class BufferedFrameDecryptor that is responsible
for decrypting received encrypted frames and passing them on to the
RtpReferenceFinder. This decoupling refactoring was triggered by a new
optimization also introduced in this patch to stash a small number of
undecryptable frames if no frames have ever been decrypted. The goal of this
optimization is to prevent re-fectching of key frames on low bandwidth networks
simply because the key to decrypt them had not arrived yet.

The optimization will stash 24 frames (about 1 second of video) in a ring buffer
and will attempt to re-decrypt previously received frames on the first valid
decryption. This allows the decoder to receive the key frame without having
to request due to short key delivery latencies. In testing this is actually hit
quite often and saves an entire RTT which can be up to 200ms on a bad network.

As the scope of frame encryption increases in WebRTC and has more specialized
optimizations that do not apply to the general flow it makes sense to move it
to a more explicit bump in the stack protocol that is decoupled from the WebRTC
main flow, similar to how SRTP is utilized with srtp_protect and srtp_unprotect.

One advantage of this approach is the BufferedFrameDecryptor isn't even
constructed if FrameEncryption is not in use.

I have decided against merging the RtpReferenceFinder and EncryptedFrame stash
because it introduced a lot of complexity around the mixed scenario where some
of the frames in the stash are encrypted and others are not. In this case we
would need to mark certain frames as decrypted which appeared to introduce more
complexity than this simple decoupling.

Bug: webrtc:10022
Change-Id: Iab74f7b7d25ef1cdd15c4a76b5daae1cfa24932c
Reviewed-on: https://webrtc-review.googlesource.com/c/112221
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25865}
2018-12-01 00:55:08 +00:00
Niels Möller
f0eee0087f Move size() method to EncodedImage base class
Deleted from subclass video_coding::EncodedFrame. Also delete Length
and SetLength methods on the intermediate class
video_coding::VCMEncodedFrame.

Bug: webrtc:9378
Change-Id: I3c90b14735f622f50b2f403f79072e22fc025d11
Reviewed-on: https://webrtc-review.googlesource.com/c/112131
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25838}
2018-11-29 13:44:47 +00:00
Niels Möller
648a7cefe1 Delete method EncodedFrame::GetBitstream, part 1
Only caller was the RtpFrameObject constructor, so it's
not needed in the interface.

To be able to delete downstream overrides, add a temporary
default implementation. Method will be completely deleted in part 2.

Bug: webrtc:9378
Change-Id: I9083b6284313b6ebce854c6f2cec4617953331d9
Reviewed-on: https://webrtc-review.googlesource.com/c/112128
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25822}
2018-11-28 14:52:32 +00:00
philipel
5fb245498c Added RtpFrameObject::SetBitstream so that the frame can be updated with the decrypted payload.
Bug: webrtc:9361
Change-Id: I5d61219033f7c3ff7e7691b74322bfa44f49e326
Reviewed-on: https://webrtc-review.googlesource.com/103221
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24934}
2018-10-02 15:56:38 +00:00
philipel
2837edce99 Make RtpGenericFrameDescriptor available for E2EE.
This CL makes the RtpGenericFrameDescriptor available in
RTPSenderVideo::SendVideo for encryption and in
RtpVideoStreamReceiver::OnReceivedFrame for decryption.

Bug: webrtc:9361
Change-Id: I5b6d10138c0874657862f103c8c9a2328e6d4a66
Reviewed-on: https://webrtc-review.googlesource.com/102720
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24929}
2018-10-02 13:35:29 +00:00
Johnny Lee
e0c8b230e7 Frame marking RTP header extension (PART 1: implement extension)
Bug: webrtc:7765
Change-Id: I23896d121afd6be4bce5ff4deaf736149efebcdb
Reviewed-on: https://webrtc-review.googlesource.com/85200
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24695}
2018-09-11 22:35:30 +00:00
philipel
5470f4030f Renamed GetCodecHeader to GetRtpVideoHeader in RtpFrameObject.
Bug: none
Change-Id: I158a19dc85ef12dc86f603ff0f6618b89cb1c242
Reviewed-on: https://webrtc-review.googlesource.com/98863
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24627}
2018-09-07 12:35:12 +00:00
Sami Kalliomäki
9882495eca Use generic video header frame ID as picture ID.
Bug: webrtc:9582
Change-Id: I7b68a66205f704cba3afe1f461cce2c993016056
Reviewed-on: https://webrtc-review.googlesource.com/96501
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24479}
2018-08-29 09:23:14 +00:00
Niels Möller
2377588c82 Add accessor methods for RTP timestamp of EncodedImage.
Intention is to make the member private, but downstream callers
must be updated to use the accessor methods first.

Bug: webrtc:9378
Change-Id: I3495bd8d545b7234fbea10abfd14f082caa420b6
Reviewed-on: https://webrtc-review.googlesource.com/82160
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24352}
2018-08-21 09:15:51 +00:00
Björn Terelius
52f53d5419 Revert "Add Timestamp accessor methods to the EncodedImage class."
This reverts commit f34d467b03.

Reason for revert: Seems to break downstream project.

Original change's description:
> Add Timestamp accessor methods to the EncodedImage class.
> 
> Bug: webrtc:9378
> Change-Id: I59bf14f631f92f0f4e05f60d4af25641a23a53f9
> Reviewed-on: https://webrtc-review.googlesource.com/82100
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Reviewed-by: Philip Eliasson <philipel@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Commit-Queue: Niels Moller <nisse@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#23734}

TBR=brandtr@webrtc.org,nisse@webrtc.org,stefan@webrtc.org,philipel@webrtc.org

Change-Id: I3aa0c0119426886bc583c918aae862eb7f4b6b63
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:9378
Reviewed-on: https://webrtc-review.googlesource.com/85600
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Commit-Queue: Björn Terelius <terelius@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23739}
2018-06-26 11:52:45 +00:00
Niels Möller
f34d467b03 Add Timestamp accessor methods to the EncodedImage class.
Bug: webrtc:9378
Change-Id: I59bf14f631f92f0f4e05f60d4af25641a23a53f9
Reviewed-on: https://webrtc-review.googlesource.com/82100
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23734}
2018-06-26 09:40:18 +00:00
Danil Chapovalov
0040b66ad3 Replace rtc::Optional with absl::optional
This is a no-op change because rtc::Optional is an alias to absl::optional

This CL generated by running script from modules with parameters
'pacing video_coding congestion_controller remote_bitrate_estimator':

find $@ -type f \( -name \*.h -o -name \*.cc \) \
-exec sed -i 's|rtc::Optional|absl::optional|g' {} \+ \
-exec sed -i 's|rtc::nullopt|absl::nullopt|g' {} \+ \
-exec sed -i 's|#include "api/optional.h"|#include "absl/types/optional.h"|' {} \+

find $@ -type f -name BUILD.gn \
-exec sed -r -i 's|"(../)*api:optional"|"//third_party/abseil-cpp/absl/types:optional"|' {} \+;

git cl format

Bug: webrtc:9078
Change-Id: I8ea501d7f1ee36e8d8cd3ed37e6b763c7fe29118
Reviewed-on: https://webrtc-review.googlesource.com/83900
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23640}
2018-06-18 10:24:48 +00:00
philipel
e7c891f953 Renamed FrameObject to EncodedFrame.
The plan is to:
 1. Move FrameObject to api/video.
 2. Rename FrameObject to EncodedFrame.
 3. Move EncodedFrame out of the video_coding namespace.

This is the 2nd CL.

Bug: webrtc:8909
Change-Id: I5e76a0a3b306156b8bc1de67834b4adf14bebef9
Reviewed-on: https://webrtc-review.googlesource.com/56182
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22158}
2018-02-22 16:12:48 +00:00
philipel
d5a272ff51 Create public EncodedFrame interface.
The plan is to:
 1. Move FrameObject to api/video.
 2. Rename FrameObject to EncodedFrame.
 3. Move EncodedFrame out of the video_coding namespace.

This is the 1st CL.

Bug: webrtc:8909
Change-Id: I2e5100eda6c51bcefb32295e03b73cf1f5c213a4
Reviewed-on: https://webrtc-review.googlesource.com/55560
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22135}
2018-02-21 16:24:15 +00:00
Mirko Bonadei
7120742701 Adding NOLINT for typedefs.h and common_types.h
Now that we have moved WebRTC from src/webrtc to src/, common_types.h
and typedefs.h are triggering a cpplint error.

The cpplint complaint is:
Include the directory when naming .h files  [build/include] [4]

This CL disables the error but we have to remove these two headers
from the root directory.

NOPRESUBMIT=true

Bug: webrtc:5876
Change-Id: I08e1b69aadcc4b28ab83bf25e3819d135d41d333
Reviewed-on: https://webrtc-review.googlesource.com/1577
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@google.com>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19859}
2017-09-15 13:03:51 +00:00
Mirko Bonadei
92ea95e34a Fixing WebRTC after moving from src/webrtc to src/
In https://webrtc-review.googlesource.com/c/src/+/1560 we moved WebRTC
from src/webrtc to src/ (in order to preserve an healthy git history).
This CL takes care of fixing header guards, #include paths, etc...

NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org


Bug: chromium:611808
Change-Id: Iea91618212bee0af16aa3f05071eab8f93706578
Reviewed-on: https://webrtc-review.googlesource.com/1561
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19846}
2017-09-15 05:02:56 +00:00
Mirko Bonadei
bb547203bf Moving src/webrtc into src/.
In order to eliminate the WebRTC Subtree mirror in Chromium, 
WebRTC is moving the content of the src/webrtc directory up
to the src/ directory.

NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org

Bug: chromium:611808
Change-Id: Iac59c5b51b950f174119565bac87955a7994bc38
Reviewed-on: https://webrtc-review.googlesource.com/1560
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19845}
2017-09-15 04:25:06 +00:00
Renamed from webrtc/modules/video_coding/frame_object.h (Browse further)