Commit graph

25 commits

Author SHA1 Message Date
Kári Tristan Helgason
c3f8c759c4 Add ability for inject custom codec factories in VP integration test.
This makes it easier to add new test cases without modifying the actual test class.

Bug: None
Change-Id: I48e4f14e26cd6610678ffb07ce9fd56e6bc1ac4e
Reviewed-on: https://webrtc-review.googlesource.com/69600
Commit-Queue: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22840}
2018-04-12 15:45:36 +00:00
Rasmus Brandt
d062a3c626 Prepare VideoProcessor for async simulcast support.
* Add support for SimulcastEncoderAdapter wrapping of encoder.
* Store input frame timestamps out-of-band, so we don't need to keep
  a raw VideoFrame around just for it's timestamp.
* Store current frame rate in |framerate_fps_|, instead of in
  codec settings struct.
* Add some comments and reorder some data members.
* Explicitly include VideoBitrateAllocator.
* Change type of |input_frames_|, to avoid one layer of indirection.
* Move VideoProcessor::CalculateFrameQuality to anonymous namespace.

This change should have no functional implications.

Bug: webrtc:8448
Change-Id: I10c140eeda750d9bd37bfb6cb1e8acb401fb91d3
Reviewed-on: https://webrtc-review.googlesource.com/60520
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22346}
2018-03-08 17:41:13 +00:00
Rasmus Brandt
5f7a891257 Minor improvements to TestConfig and VideoProcessor.
* Do not simulate freeze in decoded output file when frames have been dropped.
* Add more DCHECKs and consts.
* Remove unused members |num_encoded_frames_| and |num_decoded_frames_|.
* Move SdpVideoFormat conversion to TestConfig.

Bug: webrtc:8448
Change-Id: Ia879141f36dc23427cd1abcaa66716656fbaac2a
Reviewed-on: https://webrtc-review.googlesource.com/56802
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22239}
2018-03-01 08:42:43 +00:00
Sergey Silkin
06a8f304ef Moved analysis to Stats.
Slicing, aggregation and analysis has been moved to Stats class.
Data of all spatial layers is stored in single Stats object.

Bug: webrtc:8524
Change-Id: Ic9a64859a36a1ccda661942a201cdeeed470686a
Reviewed-on: https://webrtc-review.googlesource.com/50301
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22094}
2018-02-20 09:48:41 +00:00
Rasmus Brandt
17cdcbb57b Access ImplementationName() from task queue.
Accessing this method from the test thread is illegal,
but doesn't always fail.

Bug: webrtc:8524
Change-Id: Ie0e84cc2fb63268fb6d7cbf0c3a58cb35312c16b
Reviewed-on: https://webrtc-review.googlesource.com/49061
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21930}
2018-02-07 12:38:48 +00:00
Rasmus Brandt
f105325a55 Move all Android specific stuff to android_codec_factory_helper.
This helps separate concerns, so that the VideoProcessorIntegrationTest
is almost oblivious to the fact that it needs to connect to the JVM
to get the Android HW codecs.

Bug: webrtc:8448
Change-Id: I4359b31f84be48eaf99d83525bcce6e593e874f8
Reviewed-on: https://webrtc-review.googlesource.com/47384
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21890}
2018-02-05 13:08:26 +00:00
Sergey Silkin
53a2d8a6c7 One decoder/writer per simulcast/spatial layer.
Each simulcast stream requires dedicated decoder for decoding. SVC
can be decoded by single decoder. But in prod each receiver has its
decoder. We want to replicate this and also use one decoder per
spatial layer.

Also we create one frame writer per simulcast/spatial layer to dump
encoded/decoded frames of different layers to separate files.

Note that videoprocessor is still initialized with single
decoder/writer. It will be updated in next CL and start using
separate decoder/writer per layer.

Bug: webrtc:8524
Change-Id: I3bb3de77f97d51138b8b7675dd01bc281a078b2f
Reviewed-on: https://webrtc-review.googlesource.com/43280
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21744}
2018-01-24 13:45:58 +00:00
Sergey Silkin
1723cf9fa2 Get rid of packet loss related stuff from videoprocessor.
This feature is not needed in video codec testing framework. In WebRTC
video codecs never deal with packet loss. Packet loss is handled by
jitter buffer which prevents passing of incomplete frames to decoder.

Bug: webrtc:8768
Change-Id: I211cf51d913bec6a1f935e30691661d428ebd3b6
Reviewed-on: https://webrtc-review.googlesource.com/40740
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21722}
2018-01-22 15:45:58 +00:00
Sergey Silkin
3be2a55e7f Reland "Updated analysis in videoprocessor."
This is a reland of 1880c7162b
Original change's description:
> Updated analysis in videoprocessor.
>
> - Run analysis after all frames are processed. Before part of it was
> done at bitrate change points;
> - Analysis is done for whole stream as well as for each rate update
> interval;
> - Changed units from number of frames to time units for some metrics
> and thresholds. E.g. 'num frames to hit tagret bitrate' is changed to
> 'time to reach target bitrate, sec';
> - Changed data type of FrameStatistic::max_nalu_length (renamed to
> max_nalu_size_bytes) from rtc::Optional to size_t. There it no need to
> use such advanced data type in such low level data structure.
>
> Bug: webrtc:8524
> Change-Id: Ic9f6eab5b15ee12a80324b1f9c101de1bf3c702f
> Reviewed-on: https://webrtc-review.googlesource.com/31901
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#21653}

TBR=brandtr@webrtc.org, stefan@webrtc.org

Bug: webrtc:8524
Change-Id: Ie0ad7790689422ffa61da294967fc492a13b75ae
Reviewed-on: https://webrtc-review.googlesource.com/40202
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21668}
2018-01-18 08:37:27 +00:00
Sergey Silkin
18bc3e19c4 Revert "Updated analysis in videoprocessor."
This reverts commit 1880c7162b.

Reason for revert: breaks internal tests

Original change's description:
> Updated analysis in videoprocessor.
> 
> - Run analysis after all frames are processed. Before part of it was
> done at bitrate change points;
> - Analysis is done for whole stream as well as for each rate update
> interval;
> - Changed units from number of frames to time units for some metrics
> and thresholds. E.g. 'num frames to hit tagret bitrate' is changed to
> 'time to reach target bitrate, sec';
> - Changed data type of FrameStatistic::max_nalu_length (renamed to
> max_nalu_size_bytes) from rtc::Optional to size_t. There it no need to
> use such advanced data type in such low level data structure.
> 
> Bug: webrtc:8524
> Change-Id: Ic9f6eab5b15ee12a80324b1f9c101de1bf3c702f
> Reviewed-on: https://webrtc-review.googlesource.com/31901
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#21653}

TBR=brandtr@webrtc.org,asapersson@webrtc.org,sprang@webrtc.org,stefan@webrtc.org,ssilkin@webrtc.org

Change-Id: Id0b7d387bbba02e71637b229aeed6f6cf012af46
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:8524
Reviewed-on: https://webrtc-review.googlesource.com/40220
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21656}
2018-01-17 13:16:07 +00:00
Sergey Silkin
1880c7162b Updated analysis in videoprocessor.
- Run analysis after all frames are processed. Before part of it was
done at bitrate change points;
- Analysis is done for whole stream as well as for each rate update
interval;
- Changed units from number of frames to time units for some metrics
and thresholds. E.g. 'num frames to hit tagret bitrate' is changed to
'time to reach target bitrate, sec';
- Changed data type of FrameStatistic::max_nalu_length (renamed to
max_nalu_size_bytes) from rtc::Optional to size_t. There it no need to
use such advanced data type in such low level data structure.

Bug: webrtc:8524
Change-Id: Ic9f6eab5b15ee12a80324b1f9c101de1bf3c702f
Reviewed-on: https://webrtc-review.googlesource.com/31901
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21653}
2018-01-17 12:44:06 +00:00
Sergey Silkin
64eaa99cfc On-fly calculation of quality metrics.
Calculation of quality metrics required writing of decoded video
to file. There were two drawbacks with that approach. First, frame
drops significantly affected metrics because comparison was done
against the last decoded frame. Second, simulcast/SVC required
writing of multiple files. This might be too much data to dump.

On-fly metrics calculation is done in frame decoded callback.
Calculation time is excluded from encoding/decoding time. If CPU
usage measurement is enabled metrics calculation is disabled since
it affects CPU usage. The results are reported in Stats::PrintSummary.

Bug: webrtc:8524
Change-Id: Id54fb21f2f95deeb93757afaf46bde7d7ae18dac
Reviewed-on: https://webrtc-review.googlesource.com/22560
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20798}
2017-11-20 16:13:59 +00:00
Oleh Prypin
6581f21f0e Fix some missed stdlib includes
Ran into these when trying a newer libstdc++

Bug: None
Change-Id: Ie3ce0ae1ae1e6da1a15476fbf942b48b37adc9fa
Reviewed-on: https://webrtc-review.googlesource.com/23501
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20701}
2017-11-16 08:50:44 +00:00
Rasmus Brandt
e4c6915b87 Remove verbose setting and reorder some print statements in VideoProcessor.
Always enabling verbose mode means about 100% more text is printed,
but this should not be a problem as the only time that we explicitly
look at the logs is when the bots are failing, or when we want to save
all output for plotting.

BUG=webrtc:8448

Change-Id: Ia5feab5220d047440d15cddb7d3fbca1c5a4aaf5
Reviewed-on: https://webrtc-review.googlesource.com/16140
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20461}
2017-10-27 10:52:14 +00:00
Åsa Persson
f0c44672df Make VideoProcessor::Init/Release methods private and call from constructor/destructor.
TestConfig: Replace Print method with ToString and add test.

Bug: none
Change-Id: I9853cb16875199a51c5731d1cec326159751d001
Reviewed-on: https://webrtc-review.googlesource.com/14320
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20420}
2017-10-25 09:31:12 +00:00
Rasmus Brandt
f7a3558f3e Add VideoProcessor tests verifying that H.264 keyframes contain SPS/PPS/IDR.
This CL adds an EncodedFrameChecker interface which can be used by users
of the VideoProcessor to inject customized per-frame checks to the
encoding/decoding pipeline. This currently has two uses:
- Verifying that the QP parser works correctly for VP8 and VP9, by comparing the
  parsed QP to that produced by libvpx.
- Verifying that our H.264 encoders always produce SPS/PPS/IDR in tandem.

TESTED=Galaxy S8, Pixel 2 XL, iPhone 7.
BUG=webrtc:8423

Change-Id: Ic3e401546e239a9ffaf2ed2907689cebb1127805
Reviewed-on: https://webrtc-review.googlesource.com/14559
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20409}
2017-10-24 13:58:38 +00:00
Åsa Persson
2d27fb5a33 Move TestConfig to separate file.
Move functions Set/PrintCodecSettings, NumberOfTemporalLayers to TestConfig.
Add function NumberOfCores.

Bug: none
Change-Id: Ic33d79681d59d62bf34d9c9ff056a751ed3f8da8
Reviewed-on: https://webrtc-review.googlesource.com/13120
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20358}
2017-10-19 13:56:31 +00:00
Åsa Persson
7173cf20cc Add cpu measurements to VideoProcessorIntegrationTest.
Remove unused method ExcludeFrameTypesToStr.

Bug: webrtc:6634
Change-Id: I2816466ed428b8ce13f3073ca496c2891d5d6368
Reviewed-on: https://webrtc-review.googlesource.com/9400
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20350}
2017-10-19 11:37:51 +00:00
Åsa Persson
dc182a486a VideoProcessorIntegrationTest: remove arrays in RateProfile and use vector of RateProfiles instead.
Move num_frames from RateProfile to TestConfig struct.

Remove methods: SetRateProfile, AddRateControlThresholds.

Bug: none
Change-Id: I14bcafb8c5b3c1d3b6119417dde038fd82381e3f
Reviewed-on: https://webrtc-review.googlesource.com/8540
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20265}
2017-10-13 07:19:56 +00:00
ssilkin
612f858ba0 Adding test for SingleNalUnit mode
Test enables single-nalu mode, sets limit for nalu lenght and verifies
that encoder follows that limit.
I found that QP jumps significantly when the mode is enabled. In result
encoder might produce 4kbyte and 0.4kbyte frames back-to-back. But it
seems that happens only to couple of frames in the beginning. This
caused test to fail with default RC thresholds. To bypass this I
increased frame size mismatch threshold from 20 to 30%. This should be
Ok considering single-nalu mode is rare.

BUG=webrtc:8070

Review-Url: https://codereview.webrtc.org/3014623002
Cr-Commit-Position: refs/heads/master@{#20023}
2017-09-28 16:23:17 +00:00
Rasmus Brandt
638200e1eb Add support for SW fallback decoder in VideoProcessor.
BUG=none

Change-Id: Ib144b377115a48d26ff053e3b4b43f5260aa9f84
Reviewed-on: https://webrtc-review.googlesource.com/3760
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19999}
2017-09-27 12:51:26 +00:00
asapersson
55c7eded94 VideoProcessorIntegrationTest: Group member variables into two structs containing target/actual rates.
- Group member variables into two structs: target rates/actual rates.
- Split verify and print of rate control metrics into separate functions.
- Rename member variables.

BUG=webrtc:6634

Review-Url: https://codereview.webrtc.org/3009423002
Cr-Commit-Position: refs/heads/master@{#19925}
2017-09-22 10:45:15 +00:00
Mirko Bonadei
7120742701 Adding NOLINT for typedefs.h and common_types.h
Now that we have moved WebRTC from src/webrtc to src/, common_types.h
and typedefs.h are triggering a cpplint error.

The cpplint complaint is:
Include the directory when naming .h files  [build/include] [4]

This CL disables the error but we have to remove these two headers
from the root directory.

NOPRESUBMIT=true

Bug: webrtc:5876
Change-Id: I08e1b69aadcc4b28ab83bf25e3819d135d41d333
Reviewed-on: https://webrtc-review.googlesource.com/1577
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@google.com>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19859}
2017-09-15 13:03:51 +00:00
Mirko Bonadei
92ea95e34a Fixing WebRTC after moving from src/webrtc to src/
In https://webrtc-review.googlesource.com/c/src/+/1560 we moved WebRTC
from src/webrtc to src/ (in order to preserve an healthy git history).
This CL takes care of fixing header guards, #include paths, etc...

NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org


Bug: chromium:611808
Change-Id: Iea91618212bee0af16aa3f05071eab8f93706578
Reviewed-on: https://webrtc-review.googlesource.com/1561
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19846}
2017-09-15 05:02:56 +00:00
Mirko Bonadei
bb547203bf Moving src/webrtc into src/.
In order to eliminate the WebRTC Subtree mirror in Chromium, 
WebRTC is moving the content of the src/webrtc directory up
to the src/ directory.

NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org

Bug: chromium:611808
Change-Id: Iac59c5b51b950f174119565bac87955a7994bc38
Reviewed-on: https://webrtc-review.googlesource.com/1560
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19845}
2017-09-15 04:25:06 +00:00
Renamed from webrtc/modules/video_coding/codecs/test/videoprocessor_integrationtest.h (Browse further)