PacketBuffer is not designed to store wide range of the rtp sequence numbers
Bug: webrtc:15508
Change-Id: I62b19ba2896a667d795a41c38a60f55ee3f60566
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/321845
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@google.com>
Cr-Commit-Position: refs/heads/main@{#40839}
With LTR and SVC etc., H.264 should be able to skip lost frames, and continue to play from the new frames. With DependencyDescriptor, it is allowed to reference the previous frames, even there is a gap in the middle. However, we found there is a special logic for H.264 in packet_buffer.cc, which requires no gap for H.264.
We should allow gaps if the packet has GenericDescriptorInfo (either GenericDescriptor or DependencyDescriptor header extension).
Bug: webrtc:14887
Change-Id: Id66726bab33229bd883f257136ff2e8523fb44c0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/294062
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39370}
This updates VideoReceiveStream2::Stop() to symmetrically tear down
state that's built up in VideoReceiveStream2::Start().
Bug: webrtc:11993, webrtc:14486
Change-Id: I41f4feea5584e5baaeed2143432136f8b9761321
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/272537
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38244}
In H264, reordered packets can cause a frame following padding to become stuck in the packet buffer.
A minimal example:
_, P, 1 - padding packet p and frame 1. Frame 1 has not been returned because of missing packet 0
0, P, 1 - when packet 0 arrives, FindFrames will stop incrementing i when it sees padding packet P, and frame 1 will never be returned
Bug: webrtc:14216
Change-Id: I78b76df9709fa8593c5025d647e2b868af3749f2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/266465
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37357}
Historically the PacketBuffer used a callback for assembled frames, and because of that RtpPacketInfos were piped through it even though they didn't have anything to do with the PacketBuffer.
With this CL RtpPacketInfos are stored in the RtpVideoStreamReceiver(2) instead.
Bug: webrtc:12579
Change-Id: Ia6285b59e135910eee7234b89b23425bb0fc0d2b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/215320
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33980}
Before this CL, timestamps of received packets were rounded
to the nearest millisecond and stored as int64_t. Due to the
rounding it sometimes happened that timestamps later in the
pipeline that are not rounded seem to occur even before the
video frame was received.
Change-Id: I92d8f3540b23baae2d4a1dc6a7cb3f58bcdaad18
Bug: webrtc:12722
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/216398
Reviewed-by: Chen Xing <chxg@google.com>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33916}
In RtpVideoStreamReceiver2 it can be protected by the `worker_task_checker_` instead.
Bug: webrtc:12579
Change-Id: I4f7d64f16172139eddc7a3e07d1dbbf338beaf2e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/215224
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33734}
This change introduces a new non-reentrant mutex to WebRTC. It
enables eventual migration to Abseil's mutex.
The mutex types supportable by webrtc::Mutex are
- absl::Mutex
- CriticalSection (Windows only)
- pthread_mutex (POSIX only)
In addition to introducing the mutexes, the CL also changes
PacketBuffer to use the new mutex instead of rtc::CriticalSection.
The method of yielding from critical_section.cc was given a
mini-cleanup and YieldCurrentThread() was added to
rtc_base/synchronization/yield.h/cc.
Additionally, google_benchmark benchmarks for the mutexes were added
(test courtesy of danilchap@), and some results from a pthread/Abseil
shootout were added showing Abseil has the advantage in higher
contention.
Bug: webrtc:11567, webrtc:11634
Change-Id: Iaec324ccb32ec3851bf6db3fd290f5ea5dee4c81
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176230
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31443}
this is a step towards resolving own todo: making AssembleFrame part of
the VideoRtpDepacketizer interface and replacing codec check with a
call to a virtual function.
RtpVideoStreamReceiver has access to the VideoRtpDepacketizers,
PacketBuffer - hasn't.
Bug: None
Change-Id: I83df09975c092bdb71bab270ced356d79a50683d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168056
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30833}
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.
This relands commit abf73de8ea.
with adjustments.
Change-Id: I935977179bef31d8e1023964b967658e9a7db92d
Bug: webrtc:10342
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168489
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30532}
This reverts commit abf73de8ea.
Reason for revert: breaks downstream tests
Original change's description:
> Do not propagate generic descriptor on receiving frame
>
> It was used only for the frame decryptor.
> Decryptor needs only raw representation that it can recreate
> in a way compatible with the new version of the descriptor.
>
> Bug: webrtc:10342
> Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
> Reviewed-by: Philip Eliasson <philipel@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#30501}
TBR=danilchap@webrtc.org,sprang@webrtc.org,philipel@webrtc.org
Change-Id: I6634df06ee75aa8cdfda614994ab11f7a5845c70
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10342
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168488
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30502}
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.
Bug: webrtc:10342
Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30501}
to avoid expensive move of the Packet and prepare PacketBuffer
to return list of packets as a frame.
Bug: None
Change-Id: I19f0452c52238228bbe28284ebb197491eb2bf4e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167063
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30404}
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format
After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.
This primary benefit of this change is a small reduction in binary size.
Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
to return time of the last receieved packet of a key frame rather than
last received first packet of a key frame.
To match VideoReceiveStream expectation and prevent requesting
a new key frame if a large key frame is currently on the way.
Bug: None
Change-Id: I443a60872a3580d324f050080a9868f7b90d71a2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161730
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30084}
Together with RtpDepacketizer refactoring that would reduce
number of memcpy while handling an rtp packet
Bug: webrtc:11152
Change-Id: I6f4e09c93af5e2a9314967a15eac8ced57ec712e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161087
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29985}
it is easier to reduce and eliminate it when it is not bound to legacy video code
Bug: webrtc:10979
Change-Id: I517e298501b3358a914a23ddce40fcb3075d672d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/159707
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29821}
merge two vectors of the same size into single vector
Remove redundant size_ variable.
Remove redundant variables in the StoredPacket internal struct.
Remove frame_created flags since shortly after it is set, used flag is set to false
Bug: webrtc:10979
Change-Id: Ia37944362abda4e2a6c6741f436f95c45e0f7069
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/157174
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29535}
Summary:
There is an issue with WebRTC for handling of certain H.264 bitstreams where the packets forming the H.264 stream has non-zero packets before the packet containing SPS.
Typically a IDR (key frame) will have SPS/PPS (if present) or the IDR slice in the first packet.
But this is not required in all cases, for example when packetization-mode = 0, you can have each NALU in separate packet. And certain NALUs can exist before SPS, for example SEI, AUD.
The way WebRTC associates width/height to encoded frames is by tracking the dependency of IDR slices to SPS/PPS.
RTP packets containing SPS/PPS have correct width/height stored in them during parsing of SPS in RtpDepacketizerH264::ProcessStapAOrSingleNalu
IDR packets refer to SPS using ppsid, spsid and the width/height fields get transferred from packet containing SPS to IDR packet in H264SpsPpsTracker::CopyAndFixBitstream.
When packets are assembled into a single encoded H264 frame in PacketBuffer::FindFrames, the loop goes through all the packets/nalus in backward scan from last RTP packet of IDR to first one.
Hence the order of NALUs during this scan is : Last parts of IDR Slice -> Mid parts of IDR Slice RTP packet -> first IDR slice Packet (this should have correct width / height) -> RTP packet containing SPS/PPS (this should have correct width/height)
start_index points to the first RTP packet of the frame and its passed into RtpFrameObject's constructor. RtpFrameObject will use the width/height stored in first RTP packet.
This works fine as long as the first RTP packet has width/height, which will be the case if first RTP packet is IDR or SPS.
In H.264 first RTP packet may be AUD, SEI in those cases, RtpFrameObject will create IDR with width/height = 0 and this causes problem for Android hardware decoders.
On Android hardware decoders rely on correct width/height to initialize the hardware decoder.
Verified on real scenario that we have.
Simulated on AppRTCMobile on IOS Simulator
Added unit tests : ninja -C out/Default && ./out/Default/modules_unittests --gtest_filter=*FrameResolution*
Bug: webrtc:11025
Change-Id: Ie2273aae5e81fd62497e1add084876a3aa05af4d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156260
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Shyam Sadhwani <shyamsadhwani@fb.com>
Cr-Commit-Position: refs/heads/master@{#29515}
This issue happens for default case sps_pps_idr_is_h264_keyframe_ is false
The way PacketBuffer::FindFrames works for H264 is it keeps on skipping the packets till it finds a packet which has last=1
This is checked here : if (sequence_buffer_[index].frame_end)
Inside this block there is a loop, to go back and scan all the packets till start of the frame.
Since the scan is backwards, the sequence of nalus in this scan is IDR -> PPS -> SPS.
Once IDR is detected if (h264_header->nalus[j].type == H264::NaluType::kIdr) , the code will has_h264_idr = true.
When it scans the previous packets, it skips those as has_h264_idr is true. These packets have the SPS / PPS and hence has_h264_sps / pps flags were never set to true.
This resulted in warning as no SPS/PPS has been found for IDR.
Test plan : verified loopback call on IOS simulator using H264 codec and the warning log "Received H.264-IDR frame..." is not present anymore
Bug: webrtc:11006
Change-Id: Icbe8a393e3679a8d621af6c76e4999fd60db04a0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/155420
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Shyam Sadhwani <shyamsadhwani@fb.com>
Cr-Commit-Position: refs/heads/master@{#29386}
The packets belonging to a frame were kept in PacketBuffer
until the frame was decoded. This CL clears the dependencies
of an existing RtpFrameObject to PacketBuffer so that we can
free up PacketBuffer as soon as the RtpFrameObject is created.
Bug: none
Change-Id: Ic939be91815519ae1d1c67ada82006417b2d26a3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/149818
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28977}
This change adds the plumbing of RtpPacketInfo from RtpVideoStreamReceiver::OnRtpPacket() to VideoReceiveStream::OnFrame() for video. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
Bug: webrtc:10668
Change-Id: Ib97d430530c5a8487d3b129936c7c51e118889bd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139891
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Chen Xing <chxg@google.com>
Cr-Commit-Position: refs/heads/master@{#28332}
The latter is also a member of the former. This cleanup is also
a preparation for dropping WebRtcRTPHeader::frameType (or deleting
WebRtcRTPHeader right away), now that it's a video-specific member.
Tbr: kwiberg@webrtc.org # Comment change in modules/include/
Bug: None
Change-Id: I5c1f3f981f0d750713fc9b9b145278150fe32b5d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133024
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27740}
And delete corresponding dependencies on :webrtc_common. After this
change, common_types.h is included directly only from code in the
following directories:
api/
api/video/
api/video_codecs/
common_video/libyuv/include/
media/base/
modules/remote_bitrate_estimator/
modules/rtp_rtcp/source/
modules/video_coding/codecs/vp9/
There remains plenty of indirect dependencies on the types declared in
common_types.h, but the fewer direct dependencies should make it
easier to find the proper place for each type.
Bug: webrtc:5876
Change-Id: I93e8f214025ecb613c19fdec2015bd3f96c59aae
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/130501
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27376}
The new name fits better.
Bug: None
Change-Id: I1f201ff07915ed6c18efeefb7380e2b286742bb9
Reviewed-on: https://webrtc-review.googlesource.com/c/123800
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26814}
The values are available as part of the RTPVideoHeader member.
Bug: None
Change-Id: I832fffc449929badec3796d7096c9cdc0d43d344
Reviewed-on: https://webrtc-review.googlesource.com/c/123234
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26773}
