Start introducing ArrayView to AudioFrame and code that flows down
from there. In this first step:
* Add `data_view()` that returns a read-only ArrayView for the
audio buffer. When AudioFrame is not initialized however, data_view()
will return a nullptr whereas the current data() method never returns
nullptr.
* Add `mutable_data()` that requires two arguments for properly setting
the samples per channel and number of channels that's required for
accurately reserving the returned mutable ArrayView.
A notable behavior change is that if the requested number of channels
is larger than supported or the calculated buffer size is too large,
the function will trigger a check.
* Add TODOs for following work.
Bug: chromium:335805780
Change-Id: I2937de800422589ebe6a3840b3caadf3d9ff8b00
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/347982
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42202}
This is to avoid using 0 as a default value.
Also fix a bug in audio_device_buffer where the timestamp aligner used the wrong input timestamp.
Bug: webrtc:13609
Change-Id: I00016e68ab50d052990c2b9f80aa1e2d7e167b93
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291118
Reviewed-by: Olov Brändström <brandstrom@google.com>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39177}
Add timestamps to the function AudioDeviceBuffer::SetRecordedBuffer. This will
be used to store audio timestaps in future changes.
This is a part of the A/V sync metric metric feature for mobile. The metric
have already launched for web clients.
Bug: webrtc:13609
Change-Id: I0031843476ff1b573b262308fca52d587fae30b7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/249085
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Minyue Li <minyue@google.com>
Commit-Queue: Olov Brändström <brandstrom@google.com>
Cr-Commit-Position: refs/heads/main@{#35851}
API to injecting a heavy audio processing operation into WebRTC audio capture pipeline
Bug: webrtc:12003
Change-Id: I9f6f58f468bd84efd0a9d53d703db6229a03959e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165788
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Olga Sharonova <olka@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32291}
This CL adds a build flag to allow building the non-test parts
of WebRTC without the audio processing module.
The CL also ensures that the WebRTC code correctly handles
the case when no APM is available.
Bug: webrtc:5298
Change-Id: I5c8b5d1f7115e5cce2af4c2b5ff701fa1c54e49e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171509
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31133}
This CL moves the implementation of of the AudioFrame
support from the implementation of AudioProcessing
to proxy methods that map the call to the integer
stream interfaces (added in another CL).
The CL also changes the WebRTC code using the AudioFrame
interfaces to instead use the proxy methods.
This CL will be followed by one more CL that removes
the usage of the AudioFrame class from the rest of
APM (apart from the AudioProcessing API).
Bug: webrtc:5298
Change-Id: Iecb72e9fa896ebea3ac30e558489c1bac88f5891
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170110
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30812}
Implements RTCAudioSourceStats members:
- audioLevel
- totalAudioEnergy
- totalSamplesDuration
In this CL description these are collectively referred to as the audio
levels.
The audio levels are removed from sending "track" stats (in Chrome,
these are now reported as undefined instead of 0).
Background:
For sending tracks, audio levels were always reported as 0 in Chrome
(https://crbug.com/736403), while audio levels were correctly reported
for receiving tracks. This problem affected the standard getStats() but
not the legacy getStats(), blocking some people from migrating. This
was likely not a problem in native third_party/webrtc code because the
delivery of audio frames from device to send-stream uses a different
code path outside of chromium.
A recent PR (https://github.com/w3c/webrtc-stats/pull/451) moved the
send-side audio levels to the RTCAudioSourceStats, while keeping the
receive-side audio levels on the "track" stats. This allows an
implementation to report the audio levels even if samples are not sent
onto the network (such as if an ICE connection has not been established
yet), reflecting some of the current implementation.
Changes:
1. Audio levels are added to RTCAudioSourceStats. Send-side audio
"track" stats are left undefined. Receive-side audio "track" stats
are not changed in this CL and continue to work.
2. Audio level computation is moved from the AudioState and
AudioTransportImpl to the AudioSendStream. This is because a) the
AudioTransportImpl::RecordedDataIsAvailable() code path is not
exercised in chromium, and b) audio levels should, per-spec, not be
calculated on a per-call basis, for which the AudioState is defined.
3. The audio level computation is now performed in
AudioSendStream::SendAudioData(), a code path used by both native
and chromium code.
4. Comments are added to document behavior of existing code, such as
AudioLevel and AudioSendStream::SendAudioData().
Note:
In this CL, just like before this CL, audio level is only calculated
after an AudioSendStream has been created. This means that before an
O/A negotiation, audio levels are unavailable.
According to spec, if we have an audio source, we should have audio
levels. An immediate solution to this would have been to calculate the
audio level at pc/rtp_sender.cc. The problem is that the
LocalAudioSinkAdapter::OnData() code path, while exercised in chromium,
is not exercised in native code. The issue of calculating audio levels
on a per-source bases rather than on a per-send stream basis is left to
https://crbug.com/webrtc/10771, an existing "media-source" bug.
This CL can be verified manually in Chrome at:
https://codepen.io/anon/pen/vqRGyq
Bug: chromium:736403, webrtc:10771
Change-Id: I8036cd9984f3b187c3177470a8c0d6670a201a5a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143789
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28480}
These are manual edits please verify there are no typos.
Feel free to auto-submit if there are no issues.
Bug: webrtc:10410
Change-Id: I1b46653b91bce012afabfa0f2d249718e6de2df8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/127626
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27139}
Made path from NetEq to AudioTransport ready for many-channel audio.
If there is one stream, we can handle anything that fits in an
AudioFrame. For many streams, the current limit is 6.
Some multi-channel combinations are not supported: e.g. if we get
stereo audio and attempt to play out 6 channels.
Changes:
* AudioFrameOperations - replaced the MonoTo* and *ToMono methods by
UpmixChannels & DownmixChannels.
* AudioMixer: removed DCHECKs for <= 2 channels and tweaked the mixing
algorithm to handle many channels.
Bug: webrtc:8649
Change-Id: Ib83e16d463694e35658caa09c27849e853d508fb
Reviewed-on: https://webrtc-review.googlesource.com/c/106040
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Commit-Queue: Alex Loiko <aleloi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26446}
Configuration for AudioProcessing::voice_detection() is moving into
AudioProcessing::Config, to get rid of the pointer-to-submodule
interfaces (such as voice_detection()).
Bug: webrtc:9947
Change-Id: Ia64ae996a43d44423aa0d612a3f1185b52a3e534
Reviewed-on: https://webrtc-review.googlesource.com/c/116067
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Commit-Queue: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26216}
This CL is the result of running include-what-you-use tool on part
of the code base (audio target and dependencies) plus manual fixes.
bug: webrtc:8311
Change-Id: I277d281ce943c3ecc1bd45fd8d83055931743604
Reviewed-on: https://webrtc-review.googlesource.com/c/106280
Commit-Queue: Yves Gerey <yvesg@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25311}
Some of the AEC settings in WebRtcVoiceEngine agree with just about everywhere
else and have therefore been set in stone inside AEC2/AECM. A lot of routing
for those settings disappears.
The comfort noise setting is exposed in the API, so the flag for it will be
removed a PSA later.
Bug: webrtc:9535
Change-Id: I53816152415a9a069cea9520cec697b6bcfe0948
Reviewed-on: https://webrtc-review.googlesource.com/101622
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Commit-Queue: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24863}
Running clang-format with chromium's style guide.
The goal is n-fold:
* providing consistency and readability (that's what code guidelines are for)
* preventing noise with presubmit checks and git cl format
* building on the previous point: making it easier to automatically fix format issues
* you name it
Please consider using git-hyper-blame to ignore this commit.
Bug: webrtc:9340
Change-Id: I694567c4cdf8cee2860958cfe82bfaf25848bb87
Reviewed-on: https://webrtc-review.googlesource.com/81185
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23660}
- Move files from voice_engine/ to audio/.
- Rename voice_engine/utility.* to remix_resample.* since there are no other
utilities in those files.
- Move test/mock_voe_channel_proxy.h to audio/.
- Removed voe_channel_id from Audio[Receive|Send]Stream::Config.
- Remove VoiceEngine* from AudioState::Config.
- Fix a few cpplint complaints which showed when moving files.
NOPRESUBMIT=true
Bug: webrtc:4690
Change-Id: Id266c822d956625c358fa5e193e6f4837164aef8
Reviewed-on: https://webrtc-review.googlesource.com/39268
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21657}
AGC APIs have recently been removed from the ADM.
This CL ensures that usage of the analog part of the AGC is removed
as well (since this code is dead today anyhow).
Bug: webrtc:7306, webrtc:8598
Change-Id: I144f01cd545e5ff6900707c9308906081914e413
Reviewed-on: https://webrtc-review.googlesource.com/36120
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21640}
TransmitMixer's functionality is moved into the AudioTransportProxy
owned by AudioState. This removes the need for an AudioTransport
implementation in VoEBaseImpl, which means that the proxy is no longer
a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl.
In the short term, AudioState needs to know which AudioDeviceModule is
used, so it is added in AudioState::Config. AudioTransportImpl needs
to know which AudioSendStream:s are currently enabled to send, so
AudioState maintains a map of them, which is reduced into a simple
vector for AudioTransportImpl.
To encode and transmit audio,
AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame)
is introduced, which is used in both the Chromium and standalone use
cases. This removes the need for two different instances of
voe::Channel::ProcessAndEncodeAudio(), so there is now only one,
taking an AudioFrame as argument. Callers need to allocate their own
AudioFrame:s, which is wasteful but not a regression since this was
already happening in the voe::Channel functions.
Most of the logic changed resides in
AudioTransportImpl::RecordedDataIsAvailable(), where two strange
things were found:
1. The clock drift parameter was ineffective since
apm->echo_cancellation()->enable_drift_compensation(false) is
called during initialization.
2. The output parameter 'new_mic_volume' was never set - instead it
was returned as a result, causing the ADM to never update the
analog mic gain
(https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100).
Besides this, tests are updated, and some dead code is removed which
was found in the process.
Bug: webrtc:4690, webrtc:8591
Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2
Reviewed-on: https://webrtc-review.googlesource.com/26681
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21301}
