API to injecting a heavy audio processing operation into WebRTC audio capture pipeline
Bug: webrtc:12003
Change-Id: I9f6f58f468bd84efd0a9d53d703db6229a03959e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165788
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Olga Sharonova <olka@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32291}
Implements RTCAudioSourceStats members:
- audioLevel
- totalAudioEnergy
- totalSamplesDuration
In this CL description these are collectively referred to as the audio
levels.
The audio levels are removed from sending "track" stats (in Chrome,
these are now reported as undefined instead of 0).
Background:
For sending tracks, audio levels were always reported as 0 in Chrome
(https://crbug.com/736403), while audio levels were correctly reported
for receiving tracks. This problem affected the standard getStats() but
not the legacy getStats(), blocking some people from migrating. This
was likely not a problem in native third_party/webrtc code because the
delivery of audio frames from device to send-stream uses a different
code path outside of chromium.
A recent PR (https://github.com/w3c/webrtc-stats/pull/451) moved the
send-side audio levels to the RTCAudioSourceStats, while keeping the
receive-side audio levels on the "track" stats. This allows an
implementation to report the audio levels even if samples are not sent
onto the network (such as if an ICE connection has not been established
yet), reflecting some of the current implementation.
Changes:
1. Audio levels are added to RTCAudioSourceStats. Send-side audio
"track" stats are left undefined. Receive-side audio "track" stats
are not changed in this CL and continue to work.
2. Audio level computation is moved from the AudioState and
AudioTransportImpl to the AudioSendStream. This is because a) the
AudioTransportImpl::RecordedDataIsAvailable() code path is not
exercised in chromium, and b) audio levels should, per-spec, not be
calculated on a per-call basis, for which the AudioState is defined.
3. The audio level computation is now performed in
AudioSendStream::SendAudioData(), a code path used by both native
and chromium code.
4. Comments are added to document behavior of existing code, such as
AudioLevel and AudioSendStream::SendAudioData().
Note:
In this CL, just like before this CL, audio level is only calculated
after an AudioSendStream has been created. This means that before an
O/A negotiation, audio levels are unavailable.
According to spec, if we have an audio source, we should have audio
levels. An immediate solution to this would have been to calculate the
audio level at pc/rtp_sender.cc. The problem is that the
LocalAudioSinkAdapter::OnData() code path, while exercised in chromium,
is not exercised in native code. The issue of calculating audio levels
on a per-source bases rather than on a per-send stream basis is left to
https://crbug.com/webrtc/10771, an existing "media-source" bug.
This CL can be verified manually in Chrome at:
https://codepen.io/anon/pen/vqRGyq
Bug: chromium:736403, webrtc:10771
Change-Id: I8036cd9984f3b187c3177470a8c0d6670a201a5a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143789
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28480}
The type rtc::scoped_refptr<T> is now part of api/. Please include it from
api/scoped_refptr.h.
More info: See: https://groups.google.com/forum/#!topic/discuss-webrtc/Mme2MSz4z4o.
Bug: webrtc:9887, webrtc:8205
No-Try: True
Change-Id: Ic6c7c81e226e59f12f7933e472f573ae097b55bf
Reviewed-on: https://webrtc-review.googlesource.com/c/119041
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26414}
There are no plans to start using std::shared_ptr in WebRTC.
Bug: webrtc:10198
No-Try: True
Change-Id: I87a6c32b33b30d1b6b98eccda3400ce755a0ae95
Reviewed-on: https://webrtc-review.googlesource.com/c/117362
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26264}
Running clang-format with chromium's style guide.
The goal is n-fold:
* providing consistency and readability (that's what code guidelines are for)
* preventing noise with presubmit checks and git cl format
* building on the previous point: making it easier to automatically fix format issues
* you name it
Please consider using git-hyper-blame to ignore this commit.
Bug: webrtc:9340
Change-Id: I694567c4cdf8cee2860958cfe82bfaf25848bb87
Reviewed-on: https://webrtc-review.googlesource.com/81185
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23660}
- Move files from voice_engine/ to audio/.
- Rename voice_engine/utility.* to remix_resample.* since there are no other
utilities in those files.
- Move test/mock_voe_channel_proxy.h to audio/.
- Removed voe_channel_id from Audio[Receive|Send]Stream::Config.
- Remove VoiceEngine* from AudioState::Config.
- Fix a few cpplint complaints which showed when moving files.
NOPRESUBMIT=true
Bug: webrtc:4690
Change-Id: Id266c822d956625c358fa5e193e6f4837164aef8
Reviewed-on: https://webrtc-review.googlesource.com/39268
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21657}
* VoEBase contains only stub methods (until downstream code is
updated).
* voe::Channel and ChannelProxy classes remain, but are now created
internally to the streams. As a result,
internal::Audio[Receive|Send]Stream can have a ChannelProxy injected
for testing.
* Stream classes share Call::module_process_thread_ for their RtpRtcp
modules, rather than using a separate thread shared only among audio
streams.
* voe::Channel instances use Call::worker_queue_ for encoding packets,
rather than having a separate queue for audio (send) streams.
Bug: webrtc:4690
Change-Id: I8059ef224ad13aa0a6ded2cafc52599c7f64d68d
Reviewed-on: https://webrtc-review.googlesource.com/34640
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21578}
TransmitMixer's functionality is moved into the AudioTransportProxy
owned by AudioState. This removes the need for an AudioTransport
implementation in VoEBaseImpl, which means that the proxy is no longer
a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl.
In the short term, AudioState needs to know which AudioDeviceModule is
used, so it is added in AudioState::Config. AudioTransportImpl needs
to know which AudioSendStream:s are currently enabled to send, so
AudioState maintains a map of them, which is reduced into a simple
vector for AudioTransportImpl.
To encode and transmit audio,
AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame)
is introduced, which is used in both the Chromium and standalone use
cases. This removes the need for two different instances of
voe::Channel::ProcessAndEncodeAudio(), so there is now only one,
taking an AudioFrame as argument. Callers need to allocate their own
AudioFrame:s, which is wasteful but not a regression since this was
already happening in the voe::Channel functions.
Most of the logic changed resides in
AudioTransportImpl::RecordedDataIsAvailable(), where two strange
things were found:
1. The clock drift parameter was ineffective since
apm->echo_cancellation()->enable_drift_compensation(false) is
called during initialization.
2. The output parameter 'new_mic_volume' was never set - instead it
was returned as a result, causing the ADM to never update the
analog mic gain
(https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100).
Besides this, tests are updated, and some dead code is removed which
was found in the process.
Bug: webrtc:4690, webrtc:8591
Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2
Reviewed-on: https://webrtc-review.googlesource.com/26681
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21301}
Second attempt to land https://webrtc-review.googlesource.com/c/src/+/16180
Now removes voice_engine dependency from peerconnection and fixes a minor
const issue in NullAudioPoller.
TBR=solenberg
Bug: webrtc:7313
Change-Id: Ibfddbdc76118581e4a4dc64575203f84c1659e5c
Reviewed-on: https://webrtc-review.googlesource.com/17784
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20526}
This reverts commit 90bace0958.
Reason for revert: The original problem of this CL has been fixed in https://webrtc-review.googlesource.com/17540 but sounds like it is also adding voice_engine as a dependency of pc:peerconnection. We should investigate this because probably we can avoid it.
Original change's description:
> Add SetAudioPlayout and SetAudioRecording methods to the PeerConnection API
>
> (this CL is based on the work by Taylor and Steve in https://webrtc-review.googlesource.com/c/src/+/10201)
>
> This SetAudioPlayout method lets applications disable audio playout while
> still processing incoming audio data and generating statistics on the
> received audio.
>
> This may be useful if the application wants to set up media flows as
> soon as possible, but isn't ready to play audio yet. Currently, native
> applications don't have any API point to control this, unless they
> completely implement their own AudioDeviceModule.
>
> The SetAudioRecording works in a similar fashion but for the recorded
> audio. One difference is that calling SetAudioRecording(false) does not
> keep any audio processing alive.
>
> TBR=solenberg
>
> Bug: webrtc:7313
> Change-Id: I0aa075f6bfef9818f1080f85a8ff7842fb0750aa
> Reviewed-on: https://webrtc-review.googlesource.com/16180
> Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#20499}
TBR=solenberg@webrtc.org,henrika@webrtc.org,kwiberg@webrtc.org
Change-Id: I8431227e21dbffcfed3dd0e6bd7ce26c4ce09394
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:7313
Reviewed-on: https://webrtc-review.googlesource.com/17701
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20512}
(this CL is based on the work by Taylor and Steve in https://webrtc-review.googlesource.com/c/src/+/10201)
This SetAudioPlayout method lets applications disable audio playout while
still processing incoming audio data and generating statistics on the
received audio.
This may be useful if the application wants to set up media flows as
soon as possible, but isn't ready to play audio yet. Currently, native
applications don't have any API point to control this, unless they
completely implement their own AudioDeviceModule.
The SetAudioRecording works in a similar fashion but for the recorded
audio. One difference is that calling SetAudioRecording(false) does not
keep any audio processing alive.
TBR=solenberg
Bug: webrtc:7313
Change-Id: I0aa075f6bfef9818f1080f85a8ff7842fb0750aa
Reviewed-on: https://webrtc-review.googlesource.com/16180
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20499}
In https://webrtc-review.googlesource.com/c/src/+/1560 we moved WebRTC
from src/webrtc to src/ (in order to preserve an healthy git history).
This CL takes care of fixing header guards, #include paths, etc...
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org
Bug: chromium:611808
Change-Id: Iea91618212bee0af16aa3f05071eab8f93706578
Reviewed-on: https://webrtc-review.googlesource.com/1561
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19846}
In order to eliminate the WebRTC Subtree mirror in Chromium,
WebRTC is moving the content of the src/webrtc directory up
to the src/ directory.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org
Bug: chromium:611808
Change-Id: Iac59c5b51b950f174119565bac87955a7994bc38
Reviewed-on: https://webrtc-review.googlesource.com/1560
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19845}
