The expected behavior is to have something similar than python:
https://docs.python.org/dev/library/argparse.html#dest:
"Any internal - characters will be converted to _ characters to make sure the string is a valid attribute name".
This allows to catch chromium arguments like 'isolated-script-test-output' that previously needed some preprocessing done for example in flags_compatibility.py.
This CL also fixes a fuchsia specific issue where the test runner needs a 'isolated-script-test-output' argument but then pass the argument to WebRTC that expects a 'isolated_script_test_output' argument. Thus calling flags_compatibility before the test_runner fails and there is not much room to change the argument in between the test runner and the test.
Change-Id: I48a591743fa50484a0ec584a3f9e97d9e0fd25ef
Bug: webrtc:14694
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/284541
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38707}
* Use the same logdog_butler as Chromium instead of redefining one.
* Use luci-auth to prevent "local auth - HTTP 400" errors.
Change-Id: I2a0d1393f9f0e1e41b2bcc9a9fec2c50c19675f3
Bug: None
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/283520
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#38637}
Also remove tests in //tools_webrtc/mb/mb_unittest.py that are testing exclusively code in //tools/mb/mb.py.
Bug: webrtc:13662
Change-Id: Ifdfbe26c11f7c315e307856b1d3ab06483d57641
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260041
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#36651}
This is because it's the default flag used in the recipes to dump a json output.
This CL also fixes some python3 lint issues in mb.py.
Bug: webrtc:13594
Change-Id: I9275b5da0963f801d3191703c2eb72d90befb5d7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/248142
Reviewed-by: Christoffer Jansson <jansson@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#35986}
Thanks to machenbach@ for the heads up.
WebRTC's MB was generating a Swarming command that was using vpython
(defaulting on python2 on some platforms). This CL switches that to
vpython3 (fixing gtest-parallel-wrapper.py to be python3 compliant).
No-Presubmit: True
Bug: webrtc:13607
Change-Id: Icfa7d23b81e30cebfe8243d4ba65284955593465
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/249604
Reviewed-by: Christoffer Jansson <jansson@google.com>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35821}
* Add a config file for python formatting (.style.yapf).
* Change the default indentation from 4 spaces to 2 spaces.
* Run 'git cl format --python' on a few python files.
Bug: webrtc:13413
Change-Id: Ia71135131276c2c499b00032d57ad16ee5200a5c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238982
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Christoffer Jansson <jansson@google.com>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#35500}
This reverts commit 258ed1a38a.
Reason for revert: This cl breaks the dumping of the perf proto histogram.
Original change's description:
> Use gtest_parallel with 1 worker for webrtc_perf_tests.
>
> This will enable test results to be uploaded to ResultDB.
>
> Bug: b/197492097
> Change-Id: Iec28520c4cd8f35fcff2cbd105a4b851ef41b9fc
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239641
> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Christoffer Jansson <jansson@google.com>
> Commit-Queue: Jeremy Leconte <jleconte@google.com>
> Cr-Commit-Position: refs/heads/main@{#35458}
TBR=mbonadei@webrtc.org,jansson@google.com,webrtc-scoped@luci-project-accounts.iam.gserviceaccount.com,jleconte@webrtc.org
Change-Id: Ic4ab03d0e7f8bc1ce799d30e74420200d2f8f224
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: b/197492097
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239644
Reviewed-by: Jeremy Leconte <jleconte@google.com>
Reviewed-by: Christoffer Jansson <jansson@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#35463}
This will enable test results to be uploaded to ResultDB.
Bug: b/197492097
Change-Id: Iec28520c4cd8f35fcff2cbd105a4b851ef41b9fc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239641
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Christoffer Jansson <jansson@google.com>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#35458}
When wrapping test_env.py command on windows bot, there is an error because the python command is missing.
Adding vpython in the command like it is done on chromium mb.py script fixes the problem.
Bug: b/197492097
Change-Id: I91dbad31549fb29058424ca3b2fb8539c3e8010e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238760
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Christoffer Jansson <jansson@google.com>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#35407}
This is required step in order to upload tests results to ResultDB.
Bug: b/197492097
Change-Id: Ifff898997f6c9a3385659be34e7545217bb08f88
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238000
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Reviewed-by: Christoffer Jansson <jansson@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35385}
This removes every reference but the "--master/-m" cmd-line arg and the
"masters" mb_config.pyl key, which will be removed in a follow-up once
all users of mb.py (ie: recipes) have switched over.
"builder_group" is also the term we're using when replacing "master" in
recipe code: crbug.com/1109276. So we should conform on using that term
going forward.
Bug: chromium:1117773
Change-Id: I1de1b8e68bcf2c9d68b00a05f0f5761cf8b4ef9a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/201382
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#32964}
The K Nexus 5 bot has been upgraded to Android M, this CL fixes the
following error:
MBErr: Builder name "Perf Android32 (M Nexus5)" not found under
masters[client.webrtc.perf]
No-Try: True
Bug: webrtc:11768
Change-Id: Ie45535d622028c4d2dc553205f383b1a920c4ecc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178879
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Jeremy Leconte <jleconte@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31678}
Unfortunately it turns out the Android test runner requires
the isolated script flag to be in its current form, or it
doesn't work. This means we have to keep translating the
flag name.
We can get rid of the isolated_script_test_output flag
at least.
Tbr: mbonadei@webrtc.org
Bug: chromium:1051927
Change-Id: I4fdbff980e65332b757b1c95aa6587328411c0ed
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171809
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30907}
Since we're now calling the shots of what flags get passed in the
recipes, we can just pass the right ones right away and remove all
the flag renaming.
--isolated-script-test-output is no longer passed, so we can just
remove it. The recipe is currently passing
--isolated-script-perf-test-output but I will start passing the
underscore version shortly.
Bug: chromium:1051927
Change-Id: I571090e62f79ea17c793295df7f5abb21f45d207
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171681
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30878}
This is a reland of bd33ce2620
Now it doesn't apply flags_compatibility to Android, because the device runner actually requires it to be dashes (so it can intercept the flag and substitute it with an Android-local file path), but that's OK because the runner also already passes the flag with underscores: https://cs.chromium.org/?q=%22--isolated_script_test_perf_output%22
Original change's description:
> Reland "Add wrapper to normalize flags."
>
> This is a reland of 642a49d1eb
>
> The change has the same effect but is now implemented through mb, rather than specifying a 'script', so that Android's special handling is not skipped.
>
> Original change's description:
> > Add wrapper to normalize flags.
> >
> > Bug: None
> > Change-Id: I9d43602cc66198a29dbc0e7586d948ee76c5ec84
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145204
> > Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> > Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#28532}
>
> Bug: webrtc:10616
> Change-Id: I60ebd4891dbe8de18c653f8af88181ea966307de
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145409
> Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28572}
Bug: webrtc:10616
Change-Id: I56aae5475aed62f069c5cecc01b75d7d6ffcf568
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145920
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28589}
This is a reland of 642a49d1eb
The change has the same effect but is now implemented through mb, rather than specifying a 'script', so that Android's special handling is not skipped.
Original change's description:
> Add wrapper to normalize flags.
>
> Bug: None
> Change-Id: I9d43602cc66198a29dbc0e7586d948ee76c5ec84
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145204
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28532}
Bug: webrtc:10616
Change-Id: I60ebd4891dbe8de18c653f8af88181ea966307de
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/145409
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28572}
Since the linux_memcheck trybot is no more, this CL removes all the
code needed to make it work.
Bug: webrtc:7737, webrtc:8356, webrtc:9570
Change-Id: I09a9467b8bf895146a3384c2c915b54662721af6
Reviewed-on: https://webrtc-review.googlesource.com/90863
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24201}
This is a reland of cd469a4ce5
With vpython specified
Original change's description:
> Add support for launching webcam software for use in recipes
>
> * Copy ensure_webcam_is_running.py from recipes repo
> * Turn it into a wrapper that can launch another script
> (fix_python_path is copied from test_env.py as _ForcePythonInterpreter)
> * Support it in mb.py
> * Add it to video_capture_unittests
>
> No-Try: True
> Bug: chromium:755660
> Change-Id: I376724a77e443620724add7818592e9368d02079
> Reviewed-on: https://webrtc-review.googlesource.com/77320
> Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#23275}
No-Try: True
Bug: chromium:755660
Change-Id: Ibf4fbe3fea4b003a23b12332b3a6078a2db99a8d
Reviewed-on: https://webrtc-review.googlesource.com/77460
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23279}
This reverts commit cd469a4ce5.
Reason for revert: ImportError: No module named psutil
Original change's description:
> Add support for launching webcam software for use in recipes
>
> * Copy ensure_webcam_is_running.py from recipes repo
> * Turn it into a wrapper that can launch another script
> (fix_python_path is copied from test_env.py as _ForcePythonInterpreter)
> * Support it in mb.py
> * Add it to video_capture_unittests
>
> No-Try: True
> Bug: chromium:755660
> Change-Id: I376724a77e443620724add7818592e9368d02079
> Reviewed-on: https://webrtc-review.googlesource.com/77320
> Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#23275}
TBR=phoglund@webrtc.org,oprypin@webrtc.org
Change-Id: I611e49e1c8d8668fe405e462ec4b83e364c3d6e7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:755660
Reviewed-on: https://webrtc-review.googlesource.com/77440
Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23278}
* Copy ensure_webcam_is_running.py from recipes repo
* Turn it into a wrapper that can launch another script
(fix_python_path is copied from test_env.py as _ForcePythonInterpreter)
* Support it in mb.py
* Add it to video_capture_unittests
No-Try: True
Bug: chromium:755660
Change-Id: I376724a77e443620724add7818592e9368d02079
Reviewed-on: https://webrtc-review.googlesource.com/77320
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#23275}
This ensures that .vpython makes it to our swarm bots. I believe this
will solve a problem where psutil is missing when the catapult devil
forwarder tries to import it.
Our .vpython already specifies psutil as being used by catapult, so
I don't think we need to change anything there.
Bug: None
Change-Id: Iee8a08f22d128bca3fd52f9476604d47efe16652
Reviewed-on: https://webrtc-review.googlesource.com/52940
Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#22010}
Adds a new test_type 'raw' to run tests on swarming without wrapping it
on gtest-parallel.
This will be used to run webrtc_perf_tests directly.
Bug: chromium:755660
Change-Id: I8558faadf242d1db1ad3e13083941886c92b1bd9
Reviewed-on: https://webrtc-review.googlesource.com/49360
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Edward Lemur <ehmaldonado@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#21962}
webrtc_perf_tests needs more than 15 min to run.
NOTRY=True
Bug: chromium:755660
Change-Id: Ibabfae3679206105d585c35f80b839f0046f9ccc
Reviewed-on: https://webrtc-review.googlesource.com/4021
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Commit-Queue: Edward Lemur <ehmaldonado@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19996}
