Commit graph

63 commits

Author SHA1 Message Date
Jakob Ivarsson
1e5f88c5be Make muted param in GetAudio optional.
It is not necessary for the caller to use it and the mute info can be
found on AudioFrame.muted().

Bug: None
Change-Id: I458f1f2e8489c1d8f8a9078b21f889b2540bdab9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/349940
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42236}
2024-05-06 18:07:34 +00:00
Emil Lundmark
4d598037a8 Remove expired WebRTC-Audio-NetEqFecDelayAdaptation
Bug: webrtc:13322
Change-Id: I50d2ffb16656bd485658cd6c379fa7e834ca1cf8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/345702
Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
Commit-Queue: Emil Lundmark <lndmrk@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42009}
2024-04-06 08:57:52 +00:00
Tomas Lundqvist
aaa123debb Reland "Remove post-decode VAD"
This is a reland of commit 89cf26f1e0

Original change's description:
> Remove post-decode VAD
>
> Bug: webrtc:15806
> Change-Id: I6acf8734a70703085cfc1ccf82a79ee0931f59a4
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/336460
> Reviewed-by: Sam Zackrisson <saza@webrtc.org>
> Commit-Queue: Tomas Lundqvist <tomasl@google.com>
> Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#41653}

Bug: webrtc:15806
Change-Id: I1c2c0ce568c3c1817ff5c65bee91b9f961d46559
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/337442
Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
Commit-Queue: Tomas Lundqvist <tomasl@google.com>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41688}
2024-02-07 16:33:51 +00:00
Jakob Ivarsson
f6ae657b07 Adapt NetEq delay to received FEC (both RED and codec inband).
This is achieved by notifing NetEq controller of all received packets
after splitting, which then does deduping so that only useful packets
are counted.

The goal is to reduce underruns when FEC is used.

The behavior is default enabled with a field trial kill-switch.

Bug: webrtc:13322
Change-Id: I2a1a78ead1a58940ef92da0d43413eda5ba1caf3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/337440
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41665}
2024-02-05 12:33:27 +00:00
Jeremy Leconte
687ef0a136 Revert "Remove post-decode VAD"
This reverts commit 89cf26f1e0.

Reason for revert: breaking upstream projects

Original change's description:
> Remove post-decode VAD
>
> Bug: webrtc:15806
> Change-Id: I6acf8734a70703085cfc1ccf82a79ee0931f59a4
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/336460
> Reviewed-by: Sam Zackrisson <saza@webrtc.org>
> Commit-Queue: Tomas Lundqvist <tomasl@google.com>
> Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#41653}

Bug: webrtc:15806
Change-Id: I20e383a6b6d625d86830ecec1be01b42b22e86a2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/337420
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Jeremy Leconte <jleconte@google.com>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41657}
2024-02-01 15:16:26 +00:00
Tomas Lundqvist
89cf26f1e0 Remove post-decode VAD
Bug: webrtc:15806
Change-Id: I6acf8734a70703085cfc1ccf82a79ee0931f59a4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/336460
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Tomas Lundqvist <tomasl@google.com>
Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41653}
2024-02-01 12:37:23 +00:00
Jakob Ivarsson
526187708d Refactor NetEq insert packet list.
Move some logic from PacketBuffer to NetEqImpl.

Bug: webrtc:13322
Change-Id: I88b1e55c0cd69700730d9ed41be04fcf1effa03f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/328861
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41270}
2023-11-29 09:53:21 +00:00
Jakob Ivarsson
d116350b48 Count type of concealment based on last decoded type.
This is to avoid counting concealed samples after comfort noise as
speech.

Bug: webrtc:13322
Change-Id: I12cf18d720c697d81376c6f6cdc02d7c6bfa49a2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/299300
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39717}
2023-03-29 14:34:57 +00:00
Jakob Ivarsson
664e30ff57 Remove redundant LastDecodedTimestamps.
The same information can be found in `AudioFrame.packet_infos_`.

Bug: none
Change-Id: Ib63bc41ffb896677a445d875afce0a98acea6999
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265161
Reviewed-by: Minyue Li <minyue@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37153}
2022-06-08 13:31:52 +00:00
Byoungchan Lee
604fd2f1ab Remove RTC_DISALLOW_COPY_AND_ASSIGN from modules/
Bug: webrtc:13555, webrtc:13082
Change-Id: I2c2cbcbd918f0cfa970c1a964893220ba11d4b41
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/247960
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: (Daniel.L) Byoungchan Lee <daniel.l@hpcnt.com>
Cr-Commit-Position: refs/heads/main@{#35771}
2022-01-24 11:50:20 +00:00
Jakob Ivarsson
4a97d7281f Remove NetEq extra delay option.
Bug: b/156734419
Change-Id: I787e6961ad283990d633029c0cf296e10b825875
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237403
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35326}
2021-11-09 17:25:46 +00:00
Jakob Ivarsson
7cca016721 Hard code rtx handling option in NetEq.
This allows NetEq to adapt to late reordered packets which are common when using retransmissions.

Remaining cleanup of the plumbing from WebRTC API will be done in a follow-up cl.

Bug: webrtc:10178
Change-Id: Ia9911eaafdabd3b69441dc089116d79e24f1b2b2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231002
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34898}
2021-09-01 18:18:59 +00:00
Artem Titov
d00ce747c7 Use backticks not vertical bars to denote variables in comments for /modules/audio_coding
Bug: webrtc:12338
Change-Id: I02613d9fca45d00e2477f334b7a0416e7912e26b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/227037
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34621}
2021-08-02 10:45:40 +00:00
Tommi
3cc68ec32e Report stats from ChannelReceive::GetAudioFrameWithInfo at 1Hz.
This is a change from the previous 100Hz frequency.
Also changing the  locks slightly in AcmReceiver so that grabbing the
neteq lock right after we've let it go, isn't necessary inside of
AcmReceiver::GetAudio and also to avoid grabbing the neteq lock while
holding the AcmReceiver lock.

Bug: webrtc:12868
Change-Id: If6ee35f3dca20eb5bdbc615123aa099ccecf57c5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/221371
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34258}
2021-06-09 18:41:47 +00:00
Niels Möller
6b4d962947 Fix standard GetStats to not modify NetEq state.
Add a get_and_clear_legacy_stats flag to AudioReceiveStream::GetStats,
to distinguish calls from standard GetStats and legacy GetStats.

Add const method NetEq::CurrentNetworkStatistics to get current
values of stateless NetEq stats. Standard GetStats will then call this
method instead of NetEq::NetworkStatistics.

Bug: webrtc:11622
Change-Id: I3833a246a9e39b18c99657a738da22c6e2bd5f5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/183600
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32092}
2020-09-14 09:51:21 +00:00
Markus Handell
0df0faefd5 Migrate modules/audio_coding, audio_mixer/ and audio_processing/ to webrtc::Mutex.
Bug: webrtc:11567
Change-Id: I03b78bd2e411e9bcca199f85e4457511826cd17e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176745
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31649}
2020-07-07 14:35:58 +00:00
Henrik Lundin
f7cba9f132 Add field trial and test for NetEq extra delay
Adding field trial WebRTC-Audio-NetEqExtraDelay with a parameter value
to set the extra delay in NetEq. This overrides the
extra_output_delay_ms parameter in NetEq::Config.

Bug: b/156734419
Change-Id: Iae7d439fafa3059494249959ac13a02de63d6b7a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176858
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31493}
2020-06-10 17:37:59 +00:00
Henrik Lundin
c49e9c253f Adding a delay line to NetEq's output
This change adds an optional delay to NetEq's output. Note, this is not
equivalent to increasing the jitter buffer with the same extra length.

Bug: b/156734419
Change-Id: I8b70b6b3bffcfd3da296ccf29853864baa03d6bb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/175110
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31343}
2020-05-25 12:03:39 +00:00
Ivo Creusen
3ce44a3540 Move NetEq headers to api/
This CL also introduces NetEqFactory and NetEqControllerFactory
interfaces, as well as several convenience classes for working with
them: DefaultNetEqFactory, DefaultNetEqControllerFactory and
CustomNetEqFactory.

Bug: webrtc:11005
Change-Id: I1e8fc5154636ac2aad1a856828f80a2a758ad392
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156945
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29671}
2019-10-31 15:43:59 +00:00
Ivo Creusen
53a31f7db8 Introduce injectable NetEqController interface.
This interface is implemented by the DecisionLogic class, which now contains the DelayManager and DelayPeakDetector.

Bug: webrtc:11005
Change-Id: I4fb69fa359e60831cf153e41f101d5b623749380
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/155176
Reviewed-by: Minyue Li <minyue@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29613}
2019-10-25 11:36:41 +00:00
Karl Wiberg
4b64411406 NetEqImpl::GetDecoderFormat: Return RTP clockrate, not codec sample rate
Well, in fact we need to return both. But return codec sample rate
separately and let the SdpAudioFormat contain the RTP clockrate,
otherwise we're essentially lying to our callers.

Bug: webrtc:11028
Change-Id: I40f36cb9db6b9824404ade6b0515a8312ff97009
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156307
Commit-Queue: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29444}
2019-10-11 08:34:53 +00:00
Karl Wiberg
45eb135832 Remove the unused receive_timestamp arg to NetEq::InsertPacket
The implementation just ignores the provided timestamp, and gets the
time from the current clock instead.

Bug: webrtc:11028
Change-Id: I7a1fee36bef862c68d8f15fd19ee53b2bbb25892
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156164
Commit-Queue: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29434}
2019-10-10 13:34:30 +00:00
Alex Narest
5b5d97c938 Reland of "Reporting of decoding_codec_plc events""
This is a reland of 0a88ea050c.

The new stat will not be reported unless it is GT 0.

Reporting of decoding_codec_plc events

Bug: webrtc:10838
Change-Id: Ic8585b4eeae9a2643374f15bc2578d1141e59683
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148448
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Alex Narest <alexnarest@google.com>
Cr-Commit-Position: refs/heads/master@{#28797}
2019-08-07 18:41:46 +00:00
Mirko Bonadei
bedb7a8aea Revert "Reporting of decoding_codec_plc events"
This reverts commit 0a88ea050c.

Reason for revert: This CL breaks Chromium's FYI bots (example: https://ci.chromium.org/p/chromium/builders/webrtc.fyi/WebRTC%20Chromium%20FYI%20Linux%20Tester/4033).

Original change's description:
> Reporting of decoding_codec_plc events
> 
> Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f
> 
> Bug: webrtc:10838
> Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147263
> Commit-Queue: Alex Narest <alexnarest@google.com>
> Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28700}

TBR=mflodman@webrtc.org,alexnarest@google.com

Change-Id: I5e5dd29ee375ba422f79932d4b8c3fd028a53db4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10838
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147269
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28707}
2019-07-30 14:39:09 +00:00
Alex Narest
0a88ea050c Reporting of decoding_codec_plc events
Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f

Bug: webrtc:10838
Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147263
Commit-Queue: Alex Narest <alexnarest@google.com>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28700}
2019-07-29 16:40:23 +00:00
Alessio Bazzica
8f319a3472 Reland "Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.""
This reverts commit fab3460a82.

Reason for revert: fix downstream instead

Original change's description:
> Revert "Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.""
> 
> This reverts commit 9973933d2e.
> 
> Reason for revert: breaking downstream projects and not reviewed by direct owners
> 
> Original change's description:
> > Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> > 
> > This reverts commit 24192c267a.
> > 
> > Reason for revert: Analyzed the performance regression in more detail.
> > 
> > Most of the regression comes from the extra RtpPacketInfos-related memory allocations in every `NetEq::GetAudio()` call. Commit 1796a820f6 has removed roughly 2/3rds of the extra allocations from the impacted perf tests. Remaining perf impact is expected to be about "8 microseconds of CPU time per second" on the Linux benchmarking machines and "15 us per second" on Windows/Mac.
> > 
> > There are options to optimize further but they are unlikely worth doing. Note for example that `NetEqPerformanceTest` uses the PCM codec while the real-world use cases would likely use the much heavier Opus codec. The numbers from `OpusSpeedTest` and `NetEqPerformanceTest` suggest that Opus decoding is about 10x as expensive as NetEq overall.
> > 
> > Original change's description:
> > > Revert "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> > >
> > > This reverts commit 3e8ef940fe.
> > >
> > > Reason for revert: This CL causes a performance regression in NetEq, see https://bugs.chromium.org/p/chromium/issues/detail?id=982260.
> > >
> > > Original change's description:
> > > > Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
> > > >
> > > > This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
> > > >
> > > > Bug: webrtc:10668
> > > > Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
> > > > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
> > > > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > > > Reviewed-by: Minyue Li <minyue@webrtc.org>
> > > > Commit-Queue: Chen Xing <chxg@google.com>
> > > > Cr-Commit-Position: refs/heads/master@{#28434}
> > >
> > > TBR=kwiberg@webrtc.org,stefan@webrtc.org,minyue@webrtc.org,chxg@google.com
> > >
> > > Bug: webrtc:10668, chromium:982260
> > > Change-Id: I5e2cfde78c59d1123e21869564d76ed3f6193a5c
> > > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145339
> > > Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> > > Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
> > > Cr-Commit-Position: refs/heads/master@{#28561}
> > 
> > TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com
> > 
> > # Not skipping CQ checks because original CL landed > 1 day ago.
> > 
> > Bug: webrtc:10668, chromium:982260
> > Change-Id: Ie375a0b327ee368317bf3a04b2f1415c3a974470
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146707
> > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > Commit-Queue: Chen Xing <chxg@google.com>
> > Cr-Commit-Position: refs/heads/master@{#28664}
> 
> TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com
> 
> Change-Id: I652cb0814d83b514d3bee34e65ca3bb693099b22
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: webrtc:10668, chromium:982260
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146712
> Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
> Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28671}

TBR=alessiob@webrtc.org,kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com

Change-Id: Id43b7b3da79b4f48004b41767482bae1c1fa1e16
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10668, chromium:982260
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146713
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28672}
2019-07-24 16:47:13 +00:00
Alessio Bazzica
fab3460a82 Revert "Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.""
This reverts commit 9973933d2e.

Reason for revert: breaking downstream projects and not reviewed by direct owners

Original change's description:
> Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> 
> This reverts commit 24192c267a.
> 
> Reason for revert: Analyzed the performance regression in more detail.
> 
> Most of the regression comes from the extra RtpPacketInfos-related memory allocations in every `NetEq::GetAudio()` call. Commit 1796a820f6 has removed roughly 2/3rds of the extra allocations from the impacted perf tests. Remaining perf impact is expected to be about "8 microseconds of CPU time per second" on the Linux benchmarking machines and "15 us per second" on Windows/Mac.
> 
> There are options to optimize further but they are unlikely worth doing. Note for example that `NetEqPerformanceTest` uses the PCM codec while the real-world use cases would likely use the much heavier Opus codec. The numbers from `OpusSpeedTest` and `NetEqPerformanceTest` suggest that Opus decoding is about 10x as expensive as NetEq overall.
> 
> Original change's description:
> > Revert "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> >
> > This reverts commit 3e8ef940fe.
> >
> > Reason for revert: This CL causes a performance regression in NetEq, see https://bugs.chromium.org/p/chromium/issues/detail?id=982260.
> >
> > Original change's description:
> > > Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
> > >
> > > This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
> > >
> > > Bug: webrtc:10668
> > > Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
> > > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
> > > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > > Reviewed-by: Minyue Li <minyue@webrtc.org>
> > > Commit-Queue: Chen Xing <chxg@google.com>
> > > Cr-Commit-Position: refs/heads/master@{#28434}
> >
> > TBR=kwiberg@webrtc.org,stefan@webrtc.org,minyue@webrtc.org,chxg@google.com
> >
> > Bug: webrtc:10668, chromium:982260
> > Change-Id: I5e2cfde78c59d1123e21869564d76ed3f6193a5c
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145339
> > Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> > Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#28561}
> 
> TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com
> 
> # Not skipping CQ checks because original CL landed > 1 day ago.
> 
> Bug: webrtc:10668, chromium:982260
> Change-Id: Ie375a0b327ee368317bf3a04b2f1415c3a974470
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146707
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Commit-Queue: Chen Xing <chxg@google.com>
> Cr-Commit-Position: refs/heads/master@{#28664}

TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com

Change-Id: I652cb0814d83b514d3bee34e65ca3bb693099b22
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10668, chromium:982260
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146712
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28671}
2019-07-24 16:41:13 +00:00
Chen Xing
9973933d2e Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
This reverts commit 24192c267a.

Reason for revert: Analyzed the performance regression in more detail.

Most of the regression comes from the extra RtpPacketInfos-related memory allocations in every `NetEq::GetAudio()` call. Commit 1796a820f6 has removed roughly 2/3rds of the extra allocations from the impacted perf tests. Remaining perf impact is expected to be about "8 microseconds of CPU time per second" on the Linux benchmarking machines and "15 us per second" on Windows/Mac.

There are options to optimize further but they are unlikely worth doing. Note for example that `NetEqPerformanceTest` uses the PCM codec while the real-world use cases would likely use the much heavier Opus codec. The numbers from `OpusSpeedTest` and `NetEqPerformanceTest` suggest that Opus decoding is about 10x as expensive as NetEq overall.

Original change's description:
> Revert "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
>
> This reverts commit 3e8ef940fe.
>
> Reason for revert: This CL causes a performance regression in NetEq, see https://bugs.chromium.org/p/chromium/issues/detail?id=982260.
>
> Original change's description:
> > Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
> >
> > This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
> >
> > Bug: webrtc:10668
> > Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
> > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > Reviewed-by: Minyue Li <minyue@webrtc.org>
> > Commit-Queue: Chen Xing <chxg@google.com>
> > Cr-Commit-Position: refs/heads/master@{#28434}
>
> TBR=kwiberg@webrtc.org,stefan@webrtc.org,minyue@webrtc.org,chxg@google.com
>
> Bug: webrtc:10668, chromium:982260
> Change-Id: I5e2cfde78c59d1123e21869564d76ed3f6193a5c
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145339
> Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28561}

TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:10668, chromium:982260
Change-Id: Ie375a0b327ee368317bf3a04b2f1415c3a974470
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146707
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Chen Xing <chxg@google.com>
Cr-Commit-Position: refs/heads/master@{#28664}
2019-07-24 14:15:28 +00:00
Ivo Creusen
24192c267a Revert "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
This reverts commit 3e8ef940fe.

Reason for revert: This CL causes a performance regression in NetEq, see https://bugs.chromium.org/p/chromium/issues/detail?id=982260.

Original change's description:
> Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
>
> This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
>
> Bug: webrtc:10668
> Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Reviewed-by: Minyue Li <minyue@webrtc.org>
> Commit-Queue: Chen Xing <chxg@google.com>
> Cr-Commit-Position: refs/heads/master@{#28434}

TBR=kwiberg@webrtc.org,stefan@webrtc.org,minyue@webrtc.org,chxg@google.com

Bug: webrtc:10668, chromium:982260
Change-Id: I5e2cfde78c59d1123e21869564d76ed3f6193a5c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145339
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28561}
2019-07-12 16:18:31 +00:00
Jonas Olsson
a4d873786f Format almost everything.
This CL was generated by running

git ls-files | grep -P "(\.h|\.cc)$" | grep -v 'sdk/' | grep -v 'rtc_base/ssl_' | \
grep -v 'fake_rtc_certificate_generator.h' | grep -v 'modules/audio_device/win/' | \
grep -v 'system_wrappers/source/clock.cc' | grep -v 'rtc_base/trace_event.h' | \
grep -v 'modules/audio_coding/codecs/ilbc/' | grep -v 'screen_capturer_mac.h' | \
grep -v 'spl_inl_mips.h' | grep -v 'data_size_unittest.cc' | grep -v 'timestamp_unittest.cc' \
| xargs clang-format -i ; git cl format

Most of these changes are clang-format grouping and reordering includes
differently.

Bug: webrtc:9340
Change-Id: Ic83ddbc169bfacd21883e381b5181c3dd4fe8a63
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144051
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28505}
2019-07-08 13:45:15 +00:00
Chen Xing
3e8ef940fe Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.

Bug: webrtc:10668
Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Commit-Queue: Chen Xing <chxg@google.com>
Cr-Commit-Position: refs/heads/master@{#28434}
2019-07-01 15:56:40 +00:00
Jakob Ivarsson
445070818c Add relative_packet_arrival_delay and jitter_buffer_packets_received statistics.
Bug: webrtc:10333
Change-Id: I415e2286b426cbca940fe3a187957531847272ec
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/124780
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26976}
2019-03-05 18:50:34 +00:00
Ruslan Burakov
9bee67c5c9 Add get/set base min delay to neteq and acm_receiver.
Bug: webrtc:10287
Change-Id: Ia25f11eda1e2ac65e58a060c4f5332189214e189
Reviewed-on: https://webrtc-review.googlesource.com/c/121560
Commit-Queue: Ruslan Burakov <kuddai@google.com>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26547}
2019-02-05 13:27:59 +00:00
Steve Anton
10542f21c8 (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries
Mechanically generated by running this command:

tools_webrtc/do-renames.sh update all-renames.txt && git cl format

Then manually updating:

tools_webrtc/sanitizers/tsan_suppressions_webrtc.cc

Bug: webrtc:10159
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I54824cd91dada8fc3ee3d098f971bc319d477833
Reviewed-on: https://webrtc-review.googlesource.com/c/115653
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26226}
2019-01-11 17:11:39 +00:00
Niels Möller
0554368eed Delete method DecoderDatabase::RegisterPayload(...NetEqDecoder...)
Bug: webrtc:10185
Change-Id: I69ce40b1c7267b039cd1d2237c5d5bbae3a81875
Reviewed-on: https://webrtc-review.googlesource.com/c/116683
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26208}
2019-01-11 07:39:45 +00:00
Niels Möller
f7d636644f Delete method NetEqImpl::CurrentDelayMs, used only by tests
Bug: None
Change-Id: If94695f60ed804f6b43be828dd93f02826269140
Reviewed-on: https://webrtc-review.googlesource.com/c/116687
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26193}
2019-01-10 12:49:12 +00:00
Jakob Ivarsson
39b934ba2e Add NetEq config flag that enables RTX handling.
When enabled, the delay manager is updated with reordered packets. It also makes the peak detector ignore the reordered packets.

Change-Id: I2bdc99764cc76b15e613ed3dc75f83aaf66eee4e
Bug: webrtc:10178
Reviewed-on: https://webrtc-review.googlesource.com/c/116481
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26187}
2019-01-10 10:04:34 +00:00
Niels Möller
9c277dd1dd Delete NetEq::RegisterExternalDecoder.
Bug: webrtc:10080
Change-Id: Ie36b10af6ab22f498636e38f36bef11f28fc7f7e
Reviewed-on: https://webrtc-review.googlesource.com/c/112081
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26174}
2019-01-09 10:38:08 +00:00
Fredrik Solenberg
f693bfae5f Remove CodecInst pt.2
The following APIs on AudioCodingModule are deprecated with this CL:
  static int NumberOfCodecs();
  static int Codec(int, CodecInst*);
  static int Codec(const char*, CodecInst*, int, size_t);
  static int Codec(const char*, int, size_t);
  absl::optional<CodecInst> SendCodec() const;
  bool RegisterReceiveCodec(int, const SdpAudioFormat&);
  int RegisterExternalReceiveCodec(int, AudioDecoder*, int, int, const std::string&);
  int UnregisterReceiveCodec(uint8_t);
  int32_t ReceiveCodec(CodecInst*);
  absl::optional<SdpAudioFormat> ReceiveFormat();

As well as this method on RtpRtcp module:
  int32_t RegisterSendPayload(const CodecInst&);

Bug: webrtc:7626
Change-Id: I1230732136f1fe9048cf74afdeab767ca57ac9ce
Reviewed-on: https://webrtc-review.googlesource.com/c/113816
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26025}
2018-12-17 10:33:55 +00:00
Niels Möller
bb9f4c1252 Delete ssrc book-keeping in NetEq
The ssrc for a given NetEq instance shouldn't change.

Bug: webrtc:7135
Change-Id: Iee0d4cd8bd5d917e819fa2ecf45a40e203c6d9c0
Reviewed-on: https://webrtc-review.googlesource.com/c/111661
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25825}
2018-11-28 15:33:14 +00:00
Niels Möller
6d254bcd5e Delete unused method NetEq::PacketBufferStatistics
Bug: None
Change-Id: I9f87e445e2b5b54f78474489172f694abff38363
Reviewed-on: https://webrtc-review.googlesource.com/c/111660
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25766}
2018-11-23 09:39:32 +00:00
Niels Möller
d51b3553db Delete unused NetEq Rtcp stats.
Bug: webrtc:7135
Change-Id: Ib3ca9e02b051b8b41c2eac4e43a4f1f37999bf75
Reviewed-on: https://webrtc-review.googlesource.com/c/111640
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25743}
2018-11-22 08:00:54 +00:00
Yves Gerey
988cc0870b [Cleanup] Add missing #include. Remove useless ones.
This CL is the result of running include-what-you-use tool on part
of the code base (audio target and dependencies) plus manual fixes.

bug: webrtc:8311
Change-Id: I277d281ce943c3ecc1bd45fd8d83055931743604
Reviewed-on: https://webrtc-review.googlesource.com/c/106280
Commit-Queue: Yves Gerey <yvesg@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25311}
2018-10-23 11:32:56 +00:00
Henrik Lundin
00eb12a20c Let NetEq use the PLC output from a decoder
This change enables NetEq to use the packet concealment audio (aka
PLC) produced by a decoder. The change also includes a new API to the
AudioDecoder interface, which lets the decoder implementation generate
and deliver concealment audio.

Bug: webrtc:9180
Change-Id: Icaacebccf645d4694b0d2d6310f6f2c7132881c4
Reviewed-on: https://webrtc-review.googlesource.com/96340
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24738}
2018-09-14 07:05:20 +00:00
Ivo Creusen
d1c2f78bfe Implement new stats interface on NetEq to monitor the operations and internal state.
Currently we use the NetworkStatistics to monitor these metrics, but because these get reset on every call, this makes it impossible to use them for other purposes.

Bug: webrtc:9667
Change-Id: If648085f04d2d58aae263cff5b9491bcad373a96
Reviewed-on: https://webrtc-review.googlesource.com/99740
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24727}
2018-09-13 14:03:47 +00:00
Ivo Creusen
55de08e7ef Restructure neteq_rtpplay into a library with small executable wrapper.
Most of the code in neteq_rtpplay is moved into a factory class for
NetEqTest. The factory method takes the same argc and argv arguments as
neteq_rtpplay.
This CL also adds a small public API for neteq_test to allow easy
integration into external software.

Bug: webrtc:9667
Change-Id: I5241c1f51736cb6fbe47b0ad25f4bc83dabd727d
Reviewed-on: https://webrtc-review.googlesource.com/96100
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24531}
2018-09-03 10:42:40 +00:00
Niels Möller
d941c09bc0 Delete unimplemented methods from the NetEq interface.
Bug: None
Change-Id: I51949a096c445813acc6649676e32c575732ef40
Reviewed-on: https://webrtc-review.googlesource.com/95643
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24469}
2018-08-28 15:48:26 +00:00
Niels Möller
18f1adc0da Delete AudioCodingModule::LeastRequiredDelayMs and related NetEq code.
Bug: None
Change-Id: I2f68502d19415899b3694f7bf5da523da831b223
Reviewed-on: https://webrtc-review.googlesource.com/95640
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24439}
2018-08-27 09:58:19 +00:00
Niels Möller
a12c42a6b2 Delete root header file typedef.h.
Usage replaced with stdint.h, rtc_base/system/arch.h and
rtc_base/system/unused.h, as appropriate.

Bug: webrtc:6854
Change-Id: I97225465d14b969903d92979e2df3c3c05d35f18
Reviewed-on: https://webrtc-review.googlesource.com/90249
Reviewed-by: Niklas Enbom <niklas.enbom@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24100}
2018-07-25 14:59:26 +00:00
Henrik Lundin
7687ad58b2 Reland "NetEq: Deprecate playout modes Fax, Off and Streaming"
This is a reland of 80c4cca491

Original change's description:
> NetEq: Deprecate playout modes Fax, Off and Streaming
> 
> The playout modes other than Normal have not been reachable for a long
> time, other than through tests. It is time to deprecate them.
> 
> The only meaningful use was that Fax mode was sometimes set from
> tests, in order to avoid time-stretching operations (accelerate and
> pre-emptive expand) from messing with the test results. With this CL,
> a new config is added instead, which lets the user specify exactly
> this: don't do time-stretching.
> 
> As a result of Fax and Off modes being removed, the following code
> clean-up was done:
> - Fold DecisionLogicNormal into DecisionLogic.
> - Remove AudioRepetition and AlternativePlc operations, since they can
>   no longer be reached.
> 
> Bug: webrtc:9421
> Change-Id: I651458e9c1931a99f3b07e242817d303bac119df
> Reviewed-on: https://webrtc-review.googlesource.com/84123
> Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
> Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> Reviewed-by: Minyue Li <minyue@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#23704}

Bug: webrtc:9421
Change-Id: Ice351b635788167f2971b26470f73a5e5fa1a240
Reviewed-on: https://webrtc-review.googlesource.com/86543
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23799}
2018-07-02 10:20:33 +00:00