Run Test¶
OpenBox uses pytest
as the test framework.
Tests are automatically run by GitHub Actions when a commit is pushed (to master branch) or a pull request is opened.
Tests can also be run locally.
For pytest¶
See [tool.pytest.ini_options]
in pyproject.toml
.
Options that are not specified in command line are read from this file.
Pytest documentation: https://docs.pytest.org/
Pytest will run all files of the form test_*.py
or *_test.py
in testpath
directories (test/
in this repo). From those files, collect test items:
test
prefixed test functions or methods outside of class.test
prefixed test functions or methods insideTest
prefixed test classes (without an__init__
method). Methods decorated with@staticmethod
and@classmethods
are also considered.
Test discovery rules: https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#test-discovery
Example command:
pytest -rap test
pytest -h # show help
Use -r[chars]
to show extra test summary info as specified by chars:
(f)ailed, (E)rror, (s)kipped, (x)failed, (X)passed,
(p)assed, (P)assed with output, (a)ll except passed
(p/P), or (A)ll. (w)arnings are enabled by default (see
--disable-warnings), 'N' can be used to reset the list.
(default: 'fE').
For GitHub Actions¶
See .github/workflows/test.yml
for the configuration of GitHub Actions.
Documentation: https://docs.github.com/actions
An example repository with GitHub Actions for testing: https://github.com/jhj0411jhj/test_action
Code coverage¶
codecov: https://app.codecov.io
openbox codecov: https://app.codecov.io/gh/PKU-DAIR/open-box
To manage codecov of OpenBox, you need to be a member of PKU-DAIR organization.
Codecov report is automatically generated and uploaded.
TODO: pylint¶
TODO: use pylint (and flake8, …) to check code quality.