Commit graph

113 commits

Author SHA1 Message Date
Jakob Ivarsson
d116350b48 Count type of concealment based on last decoded type.
This is to avoid counting concealed samples after comfort noise as
speech.

Bug: webrtc:13322
Change-Id: I12cf18d720c697d81376c6f6cdc02d7c6bfa49a2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/299300
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39717}
2023-03-29 14:34:57 +00:00
Jakob Ivarsson
94b51210f8 Include packet waiting time in concealment decision.
This is to be more robust to packet loss during DTX and paused streams.

Without it, we can wait to decode an available packet when in CNG or
PLC mode until more packets arrive, which for DTX and paused streams
can take a long time.

We already include the waiting time if the last packet in the buffer
is a DTX packet.

Bug: webrtc:13322
Change-Id: Iaf5b3894500140d6f83377ba2cd65b44e0cdac05
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/299009
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39667}
2023-03-24 13:18:58 +00:00
Jakob Ivarsson
63643357b4 Remove CNG state tracking from NetEq decision logic.
It seems like this is legacy and not useful. A comment mentions
transitioning between CNG and DTMF modes, but there is no way this can
happen currently.

Bug: webrtc:13322
Change-Id: I9e4706cb6ee145ee37a9e11e7cab6ea4ff697dc4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/297980
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39590}
2023-03-17 15:00:17 +00:00
Ivo Creusen
1a84b565ac Implement RTCInboundRTPStreamStats.JitterBufferMinimumDelay
This metric was recently added to the standard (see https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-jitterbufferminimumdelay). This CL implements it for audio streams.

Bug: webrtc:14141
Change-Id: I79d918639cd12361ebbc28c2be41549e33fa7e2a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262770
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37567}
2022-07-20 09:14:03 +00:00
Jakob Ivarsson
664e30ff57 Remove redundant LastDecodedTimestamps.
The same information can be found in `AudioFrame.packet_infos_`.

Bug: none
Change-Id: Ib63bc41ffb896677a445d875afce0a98acea6999
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265161
Reviewed-by: Minyue Li <minyue@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37153}
2022-06-08 13:31:52 +00:00
Jakob Ivarsson
098c4ea2ca Add generated comfort noise counter.
Currently only implemented for codec internal CNG (Opus).

Bug: webrtc:13322
Change-Id: I00622f2967f066dba64a792e26081038ae0cb0d9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/259200
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36590}
2022-04-20 14:25:03 +00:00
Ivo Creusen
500d6e7f14 Return kSampleUnderrun if the number of samples does not fit in an AudioFrame.
If the number of samples does not fit in an AudioFrame, we should return
kSampleUnderrun to avoid crashes further downstream.

Bug: chromium:1265806
Change-Id: Ie94e1de53810167fd9b52ade72b3cb669a2a4f06
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238666
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35459}
2021-12-02 11:48:17 +00:00
Ivo Creusen
624fb67bbc Revert "Fix out-of-bounds memory access due to large number of audio channels."
This reverts commit 4cbfe4192c.

Reason for revert: The fix in this CL is ineffective. A better one has been created here: https://webrtc-review.googlesource.com/c/src/+/238666

Original change's description:
> Fix out-of-bounds memory access due to large number of audio channels.
>
> The number of audio channels can be configured in SDP, and can thus be
> set to arbitrary values by an attacker. This CL fixes an out-of-bounds
> memory access that could occur when the number of channels is set to a
> large number.
>
> Bug: chromium:1265806
> Change-Id: Ic88ff6d85b978b8eb99bf03cc52457a4552e8c24
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237808
> Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
> Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#35354}

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:1265806
Change-Id: If695ed92f831c2a9631efdf47f1568f5a15c1841
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238803
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35413}
2021-11-24 11:45:55 +00:00
Ivo Creusen
4cbfe4192c Fix out-of-bounds memory access due to large number of audio channels.
The number of audio channels can be configured in SDP, and can thus be
set to arbitrary values by an attacker. This CL fixes an out-of-bounds
memory access that could occur when the number of channels is set to a
large number.

Bug: chromium:1265806
Change-Id: Ic88ff6d85b978b8eb99bf03cc52457a4552e8c24
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237808
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35354}
2021-11-16 12:32:41 +00:00
Artem Titov
d3251968d1 Prepare to rename RTC_NOTREACHED to RTC_DCHECK_NOTREACHED
Add implementation of RTC_DCHECK_NOTREACHED equal to the RTC_NOTREACHED.
The new macros will replace the old one when old one's usage will be
removed. The idea of the renaming to provide a clear signal that this
is debug build only macros and will be stripped in the production build.

Bug: webrtc:9065
Change-Id: I4c35d8b03e74a4b3fd1ae75dba2f9c05643101db
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237802
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35348}
2021-11-15 21:44:59 +00:00
Jakob Ivarsson
4a97d7281f Remove NetEq extra delay option.
Bug: b/156734419
Change-Id: I787e6961ad283990d633029c0cf296e10b825875
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237403
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35326}
2021-11-09 17:25:46 +00:00
Jakob Ivarsson
fa68ac0c4e Reland "Remove legacy delay manger field trial and update default config."
This is a reland of 93849d4b2a

Original change's description:
> Remove legacy delay manger field trial and update default config.
>
> Bug: webrtc:10333
> Change-Id: I20e55d8d111d93657d1afe556fe3a325337c074c
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/232820
> Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#35321}

Bug: webrtc:10333
Change-Id: I9b3c732309d32640d15c372a4dde37d5d44c95d1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237502
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35325}
2021-11-09 14:49:56 +00:00
Olga Sharonova
46814941f2 Revert "Remove legacy delay manger field trial and update default config."
This reverts commit 93849d4b2a.

Reason for revert: AcmReceiverBitExactnessOldApi tests failing on MacARM64; first failing build https://ci.chromium.org/ui/p/webrtc/builders/ci/MacARM64%20M1%20Release/1038/overview
Example faliure
[ RUN      ] AcmReceiverBitExactnessOldApi.8kHzOutput
...
(rtp_file_reader.cc:165): Failed to read
../../modules/audio_coding/acm2/audio_coding_module_unittest.cc:912: Failure
Expected equality of these values:
  checksum_ref
    Which is: "636efe6d0a148f22c5383f356da3deac"
  checksum_string
    Which is: "6a288942d67e82076b38b17777cdaee4"

Original change's description:
> Remove legacy delay manger field trial and update default config.
>
> Bug: webrtc:10333
> Change-Id: I20e55d8d111d93657d1afe556fe3a325337c074c
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/232820
> Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#35321}

TBR=ivoc@webrtc.org,jakobi@webrtc.org,webrtc-scoped@luci-project-accounts.iam.gserviceaccount.com

Change-Id: I0bd3832aacba8dcd8e836650786cea20b4c083be
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10333
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237441
Reviewed-by: Olga Sharonova <olka@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Olga Sharonova <olka@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35324}
2021-11-09 09:10:39 +00:00
Jakob Ivarsson
93849d4b2a Remove legacy delay manger field trial and update default config.
Bug: webrtc:10333
Change-Id: I20e55d8d111d93657d1afe556fe3a325337c074c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/232820
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35321}
2021-11-08 11:14:09 +00:00
Ivo Creusen
bc503c95b5 Remove support for nack threshold.
The nack threshold feature is unlikely to provide any value, since
reordered packets are rare. This CL also removes the factory method
from the NackTracker class, since it did not add much value.

Bug: webrtc:10178
Change-Id: Ib6bece4a2d9f95bd4298799aaa15627f5c014b61
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231953
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34993}
2021-09-14 16:51:17 +00:00
Ivo Creusen
3a9640a5aa Improve nacking logic
Requesting nacks in more cases allows the delay adaptation to better
predict if it's worth it to increase the jitter buffer delay to wait for
the nacked packets to arrive.

Bug: webrtc:10178
Change-Id: I46adb76c013dbb8df0b99eb3f7a398f8f21c2bf6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231648
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34970}
2021-09-10 14:21:22 +00:00
Jakob Ivarsson
552c83171b Update nack tracker when receiving empty packets.
Bug: webrtc:7611
Change-Id: Ied50829f245ea99f8d737cffdf9a3d646de3fbaf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231641
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34958}
2021-09-09 10:40:35 +00:00
Jakob Ivarsson
7cca016721 Hard code rtx handling option in NetEq.
This allows NetEq to adapt to late reordered packets which are common when using retransmissions.

Remaining cleanup of the plumbing from WebRTC API will be done in a follow-up cl.

Bug: webrtc:10178
Change-Id: Ia9911eaafdabd3b69441dc089116d79e24f1b2b2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231002
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34898}
2021-09-01 18:18:59 +00:00
Artem Titov
d00ce747c7 Use backticks not vertical bars to denote variables in comments for /modules/audio_coding
Bug: webrtc:12338
Change-Id: I02613d9fca45d00e2477f334b7a0416e7912e26b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/227037
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34621}
2021-08-02 10:45:40 +00:00
Mirko Bonadei
190244bb59 Remove all #include <assert.h>/<cassert> and usage in Obj-C code.
This CL completes the removal of assert() and relative headers from
the codebase (excluded
//examples/objc/AppRTCMobile/third_party/SocketRocket which is in a
third_party sub-directory).

Bug: webrtc:6779
Change-Id: I93ed57168d2c0e011626873d66529488c5f484f2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/225546
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34528}
2021-07-22 14:00:26 +00:00
Mirko Bonadei
25ab3228f3 Replace assert() with RTC_DCHECK().
CL partially auto-generated with:

git grep -l "\bassert(" | grep "\.[c|h]" | \
  xargs sed -i 's/\bassert(/RTC_DCHECK(/g'

And with:

git grep -l "RTC_DCHECK(false)" |  \
  xargs sed -i 's/RTC_DCHECK(false)/RTC_NOTREACHED()/g'

With some manual changes to include "rtc_base/checks.h" where
needed.

A follow-up CL will remove assert() from Obj-C code as well
and remove the #include of <assert.h>.

The choice to replace with RTC_DCHECK is because assert()
is because RTC_DCHECK has similar behavior as assert()
based on NDEBUG.

This CL also contains manual changes to switch from
basic RTC_DCHECK to other (preferred) versions like
RTC_DCHECK_GT (and similar).

Bug: webrtc:6779
Change-Id: I00bed8886e03d685a2f42324e34aef2c9b7a63b0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/224846
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34442}
2021-07-09 07:49:43 +00:00
Jared Siskin
f2ed401679 Fix unscaled timestamps passed to nack_tracker
If timestamp_scaler_ is used, then rtp_header.timestamp, passed to UpdateLastDecodedPacket, will advance at a different rate than the scaled timestamp packet->timestamp, passed to UpdateLastDecodedPacket.

NackTracker::EstimateTimestamp uses timestamp_last_received_rtp_, and NackTracker::TimeToPlay uses timestamp_last_decoded_rtp_.

This difference causes TimeToPlay to continuously increase to huge values, so that every missing packet will be returned from GetNackList, even if RTT > real time to play.

Change-Id: Ie6ca347972edea98a202c9cdd26c6ab3f45a73c4
Bug: None
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/222841
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34361}
2021-06-23 08:41:50 +00:00
Tommi
3cc68ec32e Report stats from ChannelReceive::GetAudioFrameWithInfo at 1Hz.
This is a change from the previous 100Hz frequency.
Also changing the  locks slightly in AcmReceiver so that grabbing the
neteq lock right after we've let it go, isn't necessary inside of
AcmReceiver::GetAudio and also to avoid grabbing the neteq lock while
holding the AcmReceiver lock.

Bug: webrtc:12868
Change-Id: If6ee35f3dca20eb5bdbc615123aa099ccecf57c5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/221371
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Minyue Li <minyue@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34258}
2021-06-09 18:41:47 +00:00
Johannes Kron
f7de74c58c Use Timestamp to represent packet receive timestamps
Before this CL, timestamps of received packets were rounded
to the nearest millisecond and stored as int64_t. Due to the
rounding it sometimes happened that timestamps later in the
pipeline that are not rounded seem to occur even before the
video frame was received.

Change-Id: I92d8f3540b23baae2d4a1dc6a7cb3f58bcdaad18
Bug: webrtc:12722
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/216398
Reviewed-by: Chen Xing <chxg@google.com>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33916}
2021-05-04 13:16:54 +00:00
Pablo Barrera González
ff0e01f689 Implement audio_interruption metric for kCodecPlc
Audio interruption metric is not implemented for codecs doing their own PLC.

R=ivoc@webrtc.org, jakobi@webrtc.org

Bug: b/177523033 webrtc:12456
Change-Id: I0aca6fa5c0ff617e76ee1e4ed8d95703c7097223
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/206561
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Pablo Barrera González <barrerap@google.com>
Cr-Commit-Position: refs/heads/master@{#33229}
2021-02-11 09:37:24 +00:00
Ivo Creusen
7b463c5f67 Add a "Smart flushing" feature to NetEq.
Instead of flushing all packets, it makes sense to flush down to the target level instead. This CL also initiates a flush when the packet buffer is a multiple of the target level, instead of waiting until it is completely full.

Bug: webrtc:12201
Change-Id: I8775147624536824eb88752f6e8ffe57ec6199cb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193941
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32701}
2020-11-26 11:20:28 +00:00
Ivo Creusen
5a78eae780 Initialize variables to measure preemptive expansion and acceleration
The variables that are used to track the amount of preemptive expansion
and acceleration are not initialized before being passed to their
respective functions. However, these function can fail in certain cases,
and when they do the uninitialized memory will pollute the NetEq statistics.

Bug: chromium:1140376
Change-Id: I004fbaaf8d24de01dd1997fb73bdf93ca88ceaaf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/191480
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32544}
2020-11-04 08:35:28 +00:00
Ivo Creusen
a2b31c35ff Signal to NetEq Controller if arrived packets are DTX packets.
This CL also puts the arguments in a struct to allow for easier future additions.

Bug: webrtc:11005
Change-Id: I47bf664e7106b724eb1fc42299c42bbf022393ef
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/188385
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32409}
2020-10-15 08:22:03 +00:00
Ivo Creusen
43546869d6 Notify NetEqController during muted state.
During muted state NetEq shortcircuits a large part of the internals to
quickly return a buffer filled with zeros. It can be beneficial for the
controller to be aware that it is in muted state.

Bug: webrtc:11005
Change-Id: I5fe24b4a3704d953cbd68b5a24bbb7ef58b30be0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186760
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32330}
2020-10-06 16:32:04 +00:00
Niels Möller
4461f059d1 Delete unused NetEq stats currentPacketLossRate, currentDiscardRate and addedSamples
Bug: webrtc:11622
Change-Id: I097bb7284d952ada41f4f38dd7adf3536bd040ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/183620
Reviewed-by: Minyue Li <minyue@google.com>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32148}
2020-09-21 12:19:16 +00:00
Niels Möller
6b4d962947 Fix standard GetStats to not modify NetEq state.
Add a get_and_clear_legacy_stats flag to AudioReceiveStream::GetStats,
to distinguish calls from standard GetStats and legacy GetStats.

Add const method NetEq::CurrentNetworkStatistics to get current
values of stateless NetEq stats. Standard GetStats will then call this
method instead of NetEq::NetworkStatistics.

Bug: webrtc:11622
Change-Id: I3833a246a9e39b18c99657a738da22c6e2bd5f5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/183600
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32092}
2020-09-14 09:51:21 +00:00
Markus Handell
0df0faefd5 Migrate modules/audio_coding, audio_mixer/ and audio_processing/ to webrtc::Mutex.
Bug: webrtc:11567
Change-Id: I03b78bd2e411e9bcca199f85e4457511826cd17e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176745
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31649}
2020-07-07 14:35:58 +00:00
Henrik Lundin
f7cba9f132 Add field trial and test for NetEq extra delay
Adding field trial WebRTC-Audio-NetEqExtraDelay with a parameter value
to set the extra delay in NetEq. This overrides the
extra_output_delay_ms parameter in NetEq::Config.

Bug: b/156734419
Change-Id: Iae7d439fafa3059494249959ac13a02de63d6b7a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176858
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31493}
2020-06-10 17:37:59 +00:00
Henrik Lundin
c49e9c253f Adding a delay line to NetEq's output
This change adds an optional delay to NetEq's output. Note, this is not
equivalent to increasing the jitter buffer with the same extra length.

Bug: b/156734419
Change-Id: I8b70b6b3bffcfd3da296ccf29853864baa03d6bb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/175110
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31343}
2020-05-25 12:03:39 +00:00
Artem Titov
e618cc9c1e Add jitterBufferTargetDelay as RTCNonStandardStatsMember to new GetStats API
Bug: webrtc:11381
Change-Id: I7df3450e50da49d178e1e3a5d9f4970672d91aac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169120
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30758}
2020-03-11 12:08:32 +00:00
Ivo Creusen
88636c6dac Improvements for NetEqControllers
When creating a NetEqController it can be useful to have access to a
webrtc::Clock*. Also, NetEqControllers should have access to the
contents of the sync buffer when making decisions.

Bug: webrtc:11005
Change-Id: I7fdba75ce661b2ace52458620a8c1f3c990e5ac2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167208
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30368}
2020-01-24 11:39:52 +00:00
Jonas Olsson
b2b2031457 Concatenate string literals at compile time.
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format

After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.

This primary benefit of this change is a small reduction in binary size.

Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
2020-01-14 14:47:48 +00:00
Alessio Bazzica
2d02c943b2 NetEQ: fix initial decoder frame length.
Bug: webrtc:10548
Change-Id: If020ce0e5bef57f4f783dbc47995fd0c9f7e2137
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161046
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29960}
2019-11-29 13:43:41 +00:00
Henrik Lundin
fe047757d6 Fix a bug in interruption metrics
The reported audio interruption metrics are too high. If GetAudio
calls start before the first packets are arriving, and the sample rate
of the encoded audio is different from the one used to initialize
NetEq (default 16 kHz), the initial silent period of GetAudio calls
will be reported as an interruption.

Modifying a unit test to trigger the bug, and make sure it won't come
back.

Bug: webrtc:11094, b/144567257
Change-Id: Id540422cb7f35d3bef68b9e7c03c6e7c8bdb8d97
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/159980
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29831}
2019-11-19 12:58:50 +00:00
Ivo Creusen
3ce44a3540 Move NetEq headers to api/
This CL also introduces NetEqFactory and NetEqControllerFactory
interfaces, as well as several convenience classes for working with
them: DefaultNetEqFactory, DefaultNetEqControllerFactory and
CustomNetEqFactory.

Bug: webrtc:11005
Change-Id: I1e8fc5154636ac2aad1a856828f80a2a758ad392
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156945
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29671}
2019-10-31 15:43:59 +00:00
Ivo Creusen
53a31f7db8 Introduce injectable NetEqController interface.
This interface is implemented by the DecisionLogic class, which now contains the DelayManager and DelayPeakDetector.

Bug: webrtc:11005
Change-Id: I4fb69fa359e60831cf153e41f101d5b623749380
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/155176
Reviewed-by: Minyue Li <minyue@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29613}
2019-10-25 11:36:41 +00:00
Karl Wiberg
4b64411406 NetEqImpl::GetDecoderFormat: Return RTP clockrate, not codec sample rate
Well, in fact we need to return both. But return codec sample rate
separately and let the SdpAudioFormat contain the RTP clockrate,
otherwise we're essentially lying to our callers.

Bug: webrtc:11028
Change-Id: I40f36cb9db6b9824404ade6b0515a8312ff97009
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156307
Commit-Queue: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29444}
2019-10-11 08:34:53 +00:00
Karl Wiberg
45eb135832 Remove the unused receive_timestamp arg to NetEq::InsertPacket
The implementation just ignores the provided timestamp, and gets the
time from the current clock instead.

Bug: webrtc:11028
Change-Id: I7a1fee36bef862c68d8f15fd19ee53b2bbb25892
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156164
Commit-Queue: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29434}
2019-10-10 13:34:30 +00:00
Niels Möller
ef14f072a9 Delete AudioDecoder method IncomingPacket
Only the ISAC codec had an non-trivial implementation, for its unused
adaptive mode. This cl deletes that implementation, and the call
from NetEq, and the interface method.

Bug: webrtc:10098
Change-Id: Iaf7667e0ae867fc9d64286dff4c01a8ce0b6e2a4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153882
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29279}
2019-09-24 08:30:24 +00:00
Minyue Li
c759f832e9 Avoid copying of vectors in RtpPacketInfos.
Bug: chromium:982260
Change-Id: Ia4dab497b662e825f80c16530cdf615b62f0a5c9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148523
Commit-Queue: Minyue Li <minyue@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Chen Xing <chxg@google.com>
Cr-Commit-Position: refs/heads/master@{#28859}
2019-08-14 15:46:02 +00:00
Alex Narest
5b5d97c938 Reland of "Reporting of decoding_codec_plc events""
This is a reland of 0a88ea050c.

The new stat will not be reported unless it is GT 0.

Reporting of decoding_codec_plc events

Bug: webrtc:10838
Change-Id: Ic8585b4eeae9a2643374f15bc2578d1141e59683
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148448
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Alex Narest <alexnarest@google.com>
Cr-Commit-Position: refs/heads/master@{#28797}
2019-08-07 18:41:46 +00:00
Mirko Bonadei
bedb7a8aea Revert "Reporting of decoding_codec_plc events"
This reverts commit 0a88ea050c.

Reason for revert: This CL breaks Chromium's FYI bots (example: https://ci.chromium.org/p/chromium/builders/webrtc.fyi/WebRTC%20Chromium%20FYI%20Linux%20Tester/4033).

Original change's description:
> Reporting of decoding_codec_plc events
> 
> Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f
> 
> Bug: webrtc:10838
> Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147263
> Commit-Queue: Alex Narest <alexnarest@google.com>
> Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28700}

TBR=mflodman@webrtc.org,alexnarest@google.com

Change-Id: I5e5dd29ee375ba422f79932d4b8c3fd028a53db4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10838
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147269
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28707}
2019-07-30 14:39:09 +00:00
Alex Narest
0a88ea050c Reporting of decoding_codec_plc events
Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f

Bug: webrtc:10838
Change-Id: Id71b37244bc98bffaf25131a519127b3d2b86a8f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147263
Commit-Queue: Alex Narest <alexnarest@google.com>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28700}
2019-07-29 16:40:23 +00:00
Alessio Bazzica
8f319a3472 Reland "Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.""
This reverts commit fab3460a82.

Reason for revert: fix downstream instead

Original change's description:
> Revert "Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.""
> 
> This reverts commit 9973933d2e.
> 
> Reason for revert: breaking downstream projects and not reviewed by direct owners
> 
> Original change's description:
> > Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> > 
> > This reverts commit 24192c267a.
> > 
> > Reason for revert: Analyzed the performance regression in more detail.
> > 
> > Most of the regression comes from the extra RtpPacketInfos-related memory allocations in every `NetEq::GetAudio()` call. Commit 1796a820f6 has removed roughly 2/3rds of the extra allocations from the impacted perf tests. Remaining perf impact is expected to be about "8 microseconds of CPU time per second" on the Linux benchmarking machines and "15 us per second" on Windows/Mac.
> > 
> > There are options to optimize further but they are unlikely worth doing. Note for example that `NetEqPerformanceTest` uses the PCM codec while the real-world use cases would likely use the much heavier Opus codec. The numbers from `OpusSpeedTest` and `NetEqPerformanceTest` suggest that Opus decoding is about 10x as expensive as NetEq overall.
> > 
> > Original change's description:
> > > Revert "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> > >
> > > This reverts commit 3e8ef940fe.
> > >
> > > Reason for revert: This CL causes a performance regression in NetEq, see https://bugs.chromium.org/p/chromium/issues/detail?id=982260.
> > >
> > > Original change's description:
> > > > Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
> > > >
> > > > This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
> > > >
> > > > Bug: webrtc:10668
> > > > Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
> > > > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
> > > > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > > > Reviewed-by: Minyue Li <minyue@webrtc.org>
> > > > Commit-Queue: Chen Xing <chxg@google.com>
> > > > Cr-Commit-Position: refs/heads/master@{#28434}
> > >
> > > TBR=kwiberg@webrtc.org,stefan@webrtc.org,minyue@webrtc.org,chxg@google.com
> > >
> > > Bug: webrtc:10668, chromium:982260
> > > Change-Id: I5e2cfde78c59d1123e21869564d76ed3f6193a5c
> > > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145339
> > > Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> > > Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
> > > Cr-Commit-Position: refs/heads/master@{#28561}
> > 
> > TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com
> > 
> > # Not skipping CQ checks because original CL landed > 1 day ago.
> > 
> > Bug: webrtc:10668, chromium:982260
> > Change-Id: Ie375a0b327ee368317bf3a04b2f1415c3a974470
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146707
> > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > Commit-Queue: Chen Xing <chxg@google.com>
> > Cr-Commit-Position: refs/heads/master@{#28664}
> 
> TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com
> 
> Change-Id: I652cb0814d83b514d3bee34e65ca3bb693099b22
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: webrtc:10668, chromium:982260
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146712
> Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
> Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28671}

TBR=alessiob@webrtc.org,kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com

Change-Id: Id43b7b3da79b4f48004b41767482bae1c1fa1e16
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10668, chromium:982260
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146713
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28672}
2019-07-24 16:47:13 +00:00
Alessio Bazzica
fab3460a82 Revert "Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.""
This reverts commit 9973933d2e.

Reason for revert: breaking downstream projects and not reviewed by direct owners

Original change's description:
> Reland "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> 
> This reverts commit 24192c267a.
> 
> Reason for revert: Analyzed the performance regression in more detail.
> 
> Most of the regression comes from the extra RtpPacketInfos-related memory allocations in every `NetEq::GetAudio()` call. Commit 1796a820f6 has removed roughly 2/3rds of the extra allocations from the impacted perf tests. Remaining perf impact is expected to be about "8 microseconds of CPU time per second" on the Linux benchmarking machines and "15 us per second" on Windows/Mac.
> 
> There are options to optimize further but they are unlikely worth doing. Note for example that `NetEqPerformanceTest` uses the PCM codec while the real-world use cases would likely use the much heavier Opus codec. The numbers from `OpusSpeedTest` and `NetEqPerformanceTest` suggest that Opus decoding is about 10x as expensive as NetEq overall.
> 
> Original change's description:
> > Revert "Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker."
> >
> > This reverts commit 3e8ef940fe.
> >
> > Reason for revert: This CL causes a performance regression in NetEq, see https://bugs.chromium.org/p/chromium/issues/detail?id=982260.
> >
> > Original change's description:
> > > Add plumbing of RtpPacketInfos to each AudioFrame as input for SourceTracker.
> > >
> > > This change adds the plumbing of RtpPacketInfo from ChannelReceive::OnRtpPacket() to ChannelReceive::GetAudioFrameWithInfo() for audio. It is a step towards replacing the non-spec compliant ContributingSources that updates itself at packet-receive time, with the spec-compliant SourceTracker that will update itself at frame-delivery-to-track time.
> > >
> > > Bug: webrtc:10668
> > > Change-Id: I03385d6865bbc7bfbef7634f88de820a934f787a
> > > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/139890
> > > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > > Reviewed-by: Minyue Li <minyue@webrtc.org>
> > > Commit-Queue: Chen Xing <chxg@google.com>
> > > Cr-Commit-Position: refs/heads/master@{#28434}
> >
> > TBR=kwiberg@webrtc.org,stefan@webrtc.org,minyue@webrtc.org,chxg@google.com
> >
> > Bug: webrtc:10668, chromium:982260
> > Change-Id: I5e2cfde78c59d1123e21869564d76ed3f6193a5c
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145339
> > Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
> > Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#28561}
> 
> TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com
> 
> # Not skipping CQ checks because original CL landed > 1 day ago.
> 
> Bug: webrtc:10668, chromium:982260
> Change-Id: Ie375a0b327ee368317bf3a04b2f1415c3a974470
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146707
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Commit-Queue: Chen Xing <chxg@google.com>
> Cr-Commit-Position: refs/heads/master@{#28664}

TBR=kwiberg@webrtc.org,stefan@webrtc.org,ivoc@webrtc.org,minyue@webrtc.org,chxg@google.com

Change-Id: I652cb0814d83b514d3bee34e65ca3bb693099b22
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10668, chromium:982260
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/146712
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28671}
2019-07-24 16:41:13 +00:00