webrtc/api
Evan Shrubsole b556b08668 Allow encoders to receive preferred pixel formats from native buffers
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.

This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.

Testing: Tested in Chrome with media::VideoFrame adapters.

Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
2020-10-08 13:47:50 +00:00
..
adaptation Add RTC_EXPORT to webrtc::Resource so that it can be used in Chrome. 2020-06-16 18:02:00 +00:00
audio Reland "Reduce the amount of howling reduction in AEC3" 2020-10-03 23:28:33 +00:00
audio_codecs Audio codec factories: Implementations return null on unsupported formats 2020-08-26 18:03:38 +00:00
call Enable congestion window pushback to reduce bitrate by only drop video frames. 2020-02-07 14:14:47 +00:00
crypto frame encryptor: remove note about not being production ready 2020-07-31 06:49:29 +00:00
neteq Notify NetEqController during muted state. 2020-10-06 16:32:04 +00:00
numerics Introduce debug network stats 2020-09-23 09:40:25 +00:00
rtc_event_log Add logging of decoded video frames. 2020-08-06 17:33:24 +00:00
stats Reland "Implement packets_(sent | received) for RTCTransportStats" 2020-07-10 11:50:59 +00:00
task_queue suppress 'Paramaterized test suite never instantiated' warning 2020-06-09 13:31:49 +00:00
test Ensure FakeVp8Encoder::GetEncoderInfo() writes EncoderInfo.fps_allocation: 2020-09-29 19:29:29 +00:00
transport Delete macros RTC_DISALLOW_ASSIGN and RTC_DISALLOW_IMPLICIT_CONSTRUCTORS 2020-09-29 10:19:20 +00:00
units Rename several more tests that use EXPECT_DEATH to *DeathTest. 2020-05-18 16:10:04 +00:00
video Allow encoders to receive preferred pixel formats from native buffers 2020-10-08 13:47:50 +00:00
video_codecs Allow encoders to receive preferred pixel formats from native buffers 2020-10-08 13:47:50 +00:00
voip DTMF Event Sub-API on VoIP API 2020-08-20 17:10:02 +00:00
array_view.h Fix + test copying of fixed-sized ArrayView rvalues 2020-02-28 09:26:11 +00:00
array_view_unittest.cc Rename several more tests that use EXPECT_DEATH to *DeathTest. 2020-05-18 16:10:04 +00:00
async_resolver_factory.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
audio_options.cc Remove deprecated code related to AEC2 2019-08-07 10:09:36 +00:00
audio_options.h Add RtpEncodingParameters.adaptive_ptime. 2020-06-25 14:51:13 +00:00
BUILD.gn Accept NV12 frames into VP9 2020-09-29 07:54:40 +00:00
candidate.cc (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
candidate.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
create_peerconnection_factory.cc Async audio processing API 2020-10-02 12:33:34 +00:00
create_peerconnection_factory.h Async audio processing API 2020-10-02 12:33:34 +00:00
crypto_params.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
data_channel_interface.cc Datachannel: Use absl::optional for maxRetransmits and maxRetransmitTime. 2019-04-09 08:32:43 +00:00
data_channel_interface.h Pass datachannel priority in DC open messages 2020-05-17 10:57:27 +00:00
DEPS Reland "Pass NetworkMonitorFactory through PeerConnectionFactory." 2020-08-01 00:36:27 +00:00
DESIGN.md A threading explanation 2019-06-03 08:15:09 +00:00
dtls_transport_interface.cc Expose TLS version and SRTP cipher to API 2019-10-29 22:20:49 +00:00
dtls_transport_interface.h Expose TLS version and SRTP cipher to API 2019-10-29 22:20:49 +00:00
dtmf_sender_interface.h Allow DTMF delay configurability 2020-01-22 20:46:52 +00:00
fec_controller.h Make VideoFrameType an enum class, and move to separate file and target 2019-03-22 12:44:51 +00:00
fec_controller_override.h Define FecControllerOverride and plumb it down to VideoEncoder 2019-06-28 15:57:22 +00:00
frame_transformer_interface.h Reland "Make TransformableVideoFrameInterface::GetMetadata pure virtual." 2020-05-18 17:53:38 +00:00
function_view.h Move rtc::FunctionView to the public API 2019-03-21 15:23:05 +00:00
function_view_unittest.cc Format almost everything. 2019-07-08 13:45:15 +00:00
ice_transport_factory.cc Use std::make_unique instead of absl::make_unique. 2019-09-17 15:47:29 +00:00
ice_transport_factory.h Make ICE transports injectable. 2019-11-15 21:31:19 +00:00
ice_transport_interface.h Make ICE transports injectable. 2019-11-15 21:31:19 +00:00
jsep.cc Reland "Remove old-style OnFailure callbacks" 2020-02-21 14:07:57 +00:00
jsep.h Reland "Remove old-style OnFailure callbacks" 2020-02-21 14:07:57 +00:00
jsep_ice_candidate.cc Use unique_ptr in JsepCandidateCollection 2019-07-31 01:48:07 +00:00
jsep_ice_candidate.h Use unique_ptr in JsepCandidateCollection 2019-07-31 01:48:07 +00:00
jsep_session_description.h Remove deprecated JsepSession initializer 2019-06-11 17:35:30 +00:00
media_stream_interface.cc New target for api/rtp_parameters.h and api/media_types.h. 2019-08-29 09:04:32 +00:00
media_stream_interface.h VideoTrackSourceInterface: make some newly introduced methods pure virtual. 2020-05-11 12:28:32 +00:00
media_stream_proxy.h Avoid proxy thread hops for reading const properties. 2020-09-10 13:11:44 +00:00
media_stream_track_proxy.h Avoid proxy thread hops for reading const properties. 2020-09-10 13:11:44 +00:00
media_types.cc New target for api/rtp_parameters.h and api/media_types.h. 2019-08-29 09:04:32 +00:00
media_types.h New target for api/rtp_parameters.h and api/media_types.h. 2019-08-29 09:04:32 +00:00
network_state_predictor.h Add base class NetworkPredictor and NetworkPredictorFactory and wire up. 2019-04-10 12:38:58 +00:00
notifier.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
OWNERS Add new people to api/OWNERS 2020-03-24 15:14:09 +00:00
packet_socket_factory.h Prepare to move packet_socket_factory to api/. 2019-09-06 09:09:02 +00:00
peer_connection_factory_proxy.h Delete deprecated version of PeerConnectionFactoryInterface::StartAecDump 2019-06-27 07:33:59 +00:00
peer_connection_interface.cc Reland "Delete PeerConnectionInterface::BitrateParameters" 2020-07-15 07:35:16 +00:00
peer_connection_interface.h Reland "Make cricket::SctpTransportInternalFactory injectable through PeerConnectionFactory Deps" 2020-09-01 12:17:00 +00:00
peer_connection_proxy.h [Perfect Negotiation] Fire onnegotiationneeded when chain is empty. 2020-08-25 09:56:39 +00:00
priority.h Pass datachannel priority in DC open messages 2020-05-17 10:57:27 +00:00
proxy.cc Make MessageHandler cleanup optional. 2020-09-07 12:57:15 +00:00
proxy.h Make MessageHandler cleanup optional. 2020-09-07 12:57:15 +00:00
README.md Add a small README file for api/ 2019-02-01 13:24:47 +00:00
ref_counted_base.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
rtc_error.cc Return an error when datachannel closes due to network error 2019-12-08 17:33:54 +00:00
rtc_error.h Make RTCError::sctp_cause_code const 2020-08-22 09:16:15 +00:00
rtc_error_unittest.cc Format almost everything. 2019-07-08 13:45:15 +00:00
rtc_event_log_output.h Adds functionality to write logs to memory. 2019-01-16 17:36:31 +00:00
rtc_event_log_output_file.cc Format almost everything. 2019-07-08 13:45:15 +00:00
rtc_event_log_output_file.h Format almost everything. 2019-07-08 13:45:15 +00:00
rtc_event_log_output_file_unittest.cc Rename several more tests that use EXPECT_DEATH to *DeathTest. 2020-05-18 16:10:04 +00:00
rtp_headers.cc Remove framemarking RTP extension. 2020-06-15 11:18:00 +00:00
rtp_headers.h Rename PlayoutDelay --> VideoPlayoutDelay, move to api/video/video_timing.h 2020-09-07 08:37:14 +00:00
rtp_packet_info.cc Remove deprecated RtpPacketInfo::RtpPacketInfo. 2020-04-14 10:59:44 +00:00
rtp_packet_info.h Remove deprecated RtpPacketInfo::RtpPacketInfo. 2020-04-14 10:59:44 +00:00
rtp_packet_info_unittest.cc Add AbsoluteCaptureTime to RtpPacketInfo. 2019-08-07 10:12:56 +00:00
rtp_packet_infos.h Add RtpPacketInfo and RtpPacketInfos to RTC_EXPORT 2020-01-21 12:11:41 +00:00
rtp_packet_infos_unittest.cc Add AbsoluteCaptureTime to RtpPacketInfo. 2019-08-07 10:12:56 +00:00
rtp_parameters.cc Remove framemarking RTP extension. 2020-06-15 11:18:00 +00:00
rtp_parameters.h Add RtpEncodingParameters.adaptive_ptime. 2020-06-25 14:51:13 +00:00
rtp_parameters_unittest.cc Format almost everything. 2019-07-08 13:45:15 +00:00
rtp_receiver_interface.cc Insert frame transformer between Depacketizer and Decoder. 2020-03-02 08:33:44 +00:00
rtp_receiver_interface.h Avoid proxy thread hops for reading const properties. 2020-09-10 13:11:44 +00:00
rtp_sender_interface.cc Insert frame transformer between Encoded and Packetizer. 2020-02-28 07:43:13 +00:00
rtp_sender_interface.h Avoid proxy thread hops for reading const properties. 2020-09-10 13:11:44 +00:00
rtp_transceiver_direction.h RtpTransceiverInterface: add header_extensions_to_offer() 2020-03-16 13:16:42 +00:00
rtp_transceiver_interface.cc Reland "Implement transceiver.stop()" 2020-08-11 10:46:23 +00:00
rtp_transceiver_interface.h Deprecate RtpTransceiver.setDirection() 2020-08-17 13:30:30 +00:00
scoped_refptr.h Mark scoped_refptr move and swap operations as noexcept 2019-11-12 10:17:22 +00:00
scoped_refptr_unittest.cc Mark scoped_refptr move and swap operations as noexcept 2019-11-12 10:17:22 +00:00
sctp_transport_interface.cc Format almost everything. 2019-07-08 13:45:15 +00:00
sctp_transport_interface.h Export symbols needed by the Chromium component build (part 8). 2019-04-02 10:13:36 +00:00
set_local_description_observer_interface.h Reland "[Perfect Negotiation] Implement non-racy version of SetLocalDescription." 2020-07-29 11:27:43 +00:00
set_remote_description_observer_interface.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
stats_types.cc Reporting audio device underrun counter 2019-08-16 11:49:55 +00:00
stats_types.h RTC_EXPORT IceServer, StatsReport::{IdBase,Value}. 2019-11-12 12:52:25 +00:00
turn_customizer.h (4) Rename files to snake_case: update BUILD.gn, include paths, header guards, and DEPS entries 2019-01-11 17:11:39 +00:00
uma_metrics.h Remove backwards compatibility names from api/uma_metrics.h. 2019-08-29 13:35:56 +00:00
video_track_source_proxy.h Avoid proxy thread hops for reading const properties. 2020-09-10 13:11:44 +00:00

How to write code in the api/ directory

Mostly, just follow the regular style guide, but:

  • Note that api/ code is not exempt from the “.h and .cc files come in pairs” rule, so if you declare something in api/path/to/foo.h, it should be defined in api/path/to/foo.cc.
  • Headers in api/ should, if possible, not #include headers outside api/. Its not always possible to avoid this, but be aware that it adds to a small mountain of technical debt that were trying to shrink.
  • .cc files in api/, on the other hand, are free to #include headers outside api/.

That is, the preferred way for api/ code to access non-api/ code is to call it from a .cc file, so that users of our API headers wont transitively #include non-public headers.

For headers in api/ that need to refer to non-public types, forward declarations are often a lesser evil than including non-public header files. The usual rules still apply, though.

.cc files in api/ should preferably be kept reasonably small. If a substantial implementation is needed, consider putting it with our non-public code, and just call it from the api/ .cc file.