mollyim/webrtc
1
0
Fork 0
mirror of https://github.com/mollyim/webrtc.git synced 2025-05-12 21:30:45 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Tolerate more not completed measurements for CPD uploads

Browse source 
Recently CPD team rolled out upload completion token feature to all users. Pressure on the system increased. Now became more common situations when upload completed, but because of Datastore limitations we can't see confirmation of it for some measurements.

I've checked 6 recent failures. For all of them amount of timeout measurements were less than 3% (less than 15 in absolute numbers, the biggest percent of failures was for 80 measurements, 2 of which timed out).

Bug: b/182111579
Change-Id: Ia5af367870d1cf7d28b9422c4114c6b85c41f865
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/228562
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Andrey Logvin <landrey@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34749}
This commit is contained in:
landrey 2021-08-12 16:27:56 +00:00 committed by WebRTC LUCI CQ
parent dcaf1e79c4
commit 722a8a6875
1 changed files with 8 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
tools_webrtc/perf/catapult_uploader.py
View file

@ -122,8 +122,8 @@ def _WaitForUploadConfirmation(url, upload_token, wait_timeout,
# failed. Check it, so it doesn't increase flakiness of our tests.
# TODO(crbug.com/1145904): Remove check after fixed.
def _CheckFullUploadInfo(url, upload_token,
min_measurements_amount=100,
max_failed_measurements_amount=1):
min_measurements_amount=50,
max_failed_measurements_percent=0.03):
"""Make a HTTP GET requests to the Performance Dashboard to get full info
about upload (including measurements). Checks if upload is correct despite
not having status "COMPLETED".
@ -135,8 +135,8 @@ def _CheckFullUploadInfo(url, upload_token,
for the status check.
min_measurements_amount: minimal amount of measurements that the upload
should have to start tolerating failures in particular measurements.
max_failed_measurements_amount: maximal amount of failured measurements to
tolerate.
max_failed_measurements_percent: maximal percent of failured measurements
to tolerate.
"""
headers = _CreateHeaders(_GenerateOauthToken())
http = httplib2.Http()
@ -160,9 +160,10 @@ def _CheckFullUploadInfo(url, upload_token,
])
if (measurements_cnt >= min_measurements_amount and
not_completed_state_cnt <= max_failed_measurements_amount):
print('Not all measurements were uploaded. Measurements count: %d, '
'failed to upload: %d' %
(not_completed_state_cnt / (measurements_cnt * 1.0) <=
max_failed_measurements_percent)):
print('Not all measurements were confirmed to upload. '
'Measurements count: %d, failed to upload or timed out: %d' %
(measurements_cnt, not_completed_state_cnt))
return True