Commit graph

14 commits

Author SHA1 Message Date
Sergey Silkin
6c60f72a6b Refactor video codec testing stats
This CL introduces VideoCodecStats and VideoCodecStatsImpl which provide baseline functionalities for storing, slicing and aggregation of encoded and/or decoded video frame statistics. To facilitate metrics logging (not implemented yet), SamplesStatsCounter is used for stream parameters.

VideoCodecStats/VideoCodecStatsImpl will replace existing VideoCodecTestStats/VideoCodecTestStatsImpl.

Bug: b/261160916, webrtc:14852
Change-Id: I0f96ce1ed9be3aee2a702804612524676c9882fd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291323
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39248}
2023-02-02 15:56:40 +00:00
Sergey Silkin
2e1a9a4ae0 Add video codec tester.
This tester is an improved version of VideoProcessor and VideoCodecTestFixture and will eventually replace them.

The tester provides better separation between codecs and testing logic. Its knowledge about codecs is limited to frame encode/decode calls and frame ready callbacks. Instantiation and configuration of codecs are the test responsibilities.

Other differences:
- Run encoding and decoding in separate threads
- Run quality analysis in a separate thread
- Reference frame buffering is moved into video source (which re-read frames from the file).
- Make it possible to run decode-only tests

This CL is MVP implementation: it adds only 1 test (video_codec_test.cc, ConstantRate/EncodeDecodeTest) and the test is disabled for now.

Bug: b/261160916
Change-Id: Ida24a2fca1b1496237fa695c812084877c76379f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/283525
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38901}
2022-12-15 14:32:53 +00:00
Sergey Silkin
0b02d637c0 Calculate max/avg encode/decode latency in codec tests
Bug: none
Change-Id: Ie42461dd06b1764c99308393477921ea25319ab4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/251687
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36007}
2022-02-15 18:14:41 +00:00
Sergey Silkin
1fdafaeb21 Calculate bitrate and frame rate mismatches in video codec tests
Bug: webrtc:10812
Change-Id: I3408c0d7adefc37d088a5c6e10fce4f95aa1b668
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/228943
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34782}
2021-08-17 10:33:08 +00:00
Sergey Silkin
706ef1b913 Create name->value text map for frame and video statistics
This is needed to facilitate dumping of stats to CSV in tests.

Bug: none
Change-Id: Ic78a4630f70a9238d26161ac89c205903dfc852f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/225300
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34435}
2021-07-08 08:38:50 +00:00
Sergey Silkin
b72cc6d670 Analyze quality of dropped frames in VideoProcessor.
Calculate quality metrics for dropped frames by comparing original
frame against last decoded one.

This feature makes comparison of encoders which do/don't drop frames
more fair.

The feature is controlled by analyze_quality_of_dropped_frames flag
and is disabled by default.

Bug: none
Change-Id: Ifab8df92d0b76e743ff3193c05d7c8dbd14921c4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/190660
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32518}
2020-10-29 08:23:49 +00:00
Sergey Silkin
44cec0b5bd Handle non-integer frame rates in video codec tests.
Encoder API accepts non-integer frame rate since
https://webrtc-review.googlesource.com/c/src/+/131949.

Bug: webrtc:10812
Change-Id: I5fc9c5dfac4b182b84a735218a2946a95cc2b93c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143483
Reviewed-by: Seth Hampson <shampson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28548}
2019-07-12 07:37:43 +00:00
Jonas Olsson
a4d873786f Format almost everything.
This CL was generated by running

git ls-files | grep -P "(\.h|\.cc)$" | grep -v 'sdk/' | grep -v 'rtc_base/ssl_' | \
grep -v 'fake_rtc_certificate_generator.h' | grep -v 'modules/audio_device/win/' | \
grep -v 'system_wrappers/source/clock.cc' | grep -v 'rtc_base/trace_event.h' | \
grep -v 'modules/audio_coding/codecs/ilbc/' | grep -v 'screen_capturer_mac.h' | \
grep -v 'spl_inl_mips.h' | grep -v 'data_size_unittest.cc' | grep -v 'timestamp_unittest.cc' \
| xargs clang-format -i ; git cl format

Most of these changes are clang-format grouping and reordering includes
differently.

Bug: webrtc:9340
Change-Id: Ic83ddbc169bfacd21883e381b5181c3dd4fe8a63
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144051
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28505}
2019-07-08 13:45:15 +00:00
Rasmus Brandt
7d72d0fb39 Change VideoCodecTestStats API.
- Add GetFrameStatistics API:
  This is useful for downstream test users that want to read frame-level stats.
- Remove other APIs that are not used by downstream tests:
    * AddFrame
    * GetFrame
    * GetFrameWithTimestamp
    * SliceAndCalcAggregatedVideoStatistic
    * PrintFrameStatistics
    * Size
    * Clear
  The implementations, which are used by the fixture implementation, are kept.

Bug: webrtc:10349
Change-Id: Id2f6fa5a36b8341a5ccb365725f71ebe0c0f1570
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/128779
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27306}
2019-03-27 09:50:54 +00:00
Niels Möller
8f7ce222e7 Make VideoFrameType an enum class, and move to separate file and target
Bug: webrtc:5876, webrtc:6883
Change-Id: I1435cfa9e8e54c4ba2978261048ff3fbb993ce0e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/126225
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27239}
2019-03-22 12:44:51 +00:00
Niels Möller
87e2d785a0 Prepare for splitting FrameType into AudioFrameType and VideoFrameType
This cl deprecates the FrameType enum, and adds aliases AudioFrameType
and VideoFrameType.

After downstream usage is updated, the enums will be separated
and be moved out of common_types.h.

Bug: webrtc:6883
Change-Id: I2aaf660169da45f22574b4cbb16aea8522cc07a6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/123184
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27011}
2019-03-07 10:12:57 +00:00
Yves Gerey
3e70781361 [Cleanup] Add missing #include. Remove useless ones. IWYU part 2.
This is a follow-up to
https://webrtc-review.googlesource.com/c/src/+/106280.
This time the whole code base is covered.
Some files may have not been fixed though, whenever the IWYU tool
was breaking the build.

Bug: webrtc:8311
Change-Id: I2c31f552a87e887d33931d46e87b6208b1e483ef
Reviewed-on: https://webrtc-review.googlesource.com/c/111965
Commit-Queue: Yves Gerey <yvesg@google.com>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25830}
2018-11-28 18:25:07 +00:00
Sergey Silkin
02fed02c00 Assign spatial_idx in FrameStatistics ctor.
- Add spatial_idx to FrameStatistics ctor.
- Pass FrameStatistics object to AddFrame.

Bug: none
Change-Id: I9d6de449b45a007438f6fd3317176bf45fb23806
Reviewed-on: https://webrtc-review.googlesource.com/101781
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24856}
2018-09-27 08:35:29 +00:00
Kári Tristan Helgason
169005d8c1 Move VideoCodecTest configuration classes to api/test.
These files are required when implementing tests based on the test fixture,
and should be exposed as part of the test api.

This CL also removes a usage of stringstream and fixes some chromium-style
lint issues.

Bug: webrtc:8982, webrtc:163
Change-Id: I132aea0da79a79587887f21897236fc9802b7574
Reviewed-on: https://webrtc-review.googlesource.com/74586
Commit-Queue: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23346}
2018-05-22 12:14:38 +00:00