Skip to content

automated benchmark

Jump to bottom
Oliver Beckstein edited this page Feb 18, 2022 · 8 revisions

Automated benchmarks run as a cron job on c3potato (see benchmark on c3potato for details for the setup) and nightly update https://www.mdanalysis.org/benchmarks/ .

Benchmarked Python versions

The following Python versions have been benchmarked. The start and stop are git tags.

Python start stop
3.8 release-1.0.1 HEAD
3.6 release-0.17.0 release-2.0.0-beta
2.7 release-0.11.0 release-1.0.0^

"release-1.0.0^" means "commits up to but not including release-1.0.0".

Cron job

Run the script benchmarks/script/cron.sh every night (see #9 for notes on changing from 2.7 to 3.6 and #11 for updating to Python 3.8): It will

  • benchmark the latest NEW commits (all of them)
  • benchmark any missing merge commits since release-1.0.1 under Python 3.8
  • push the results to the benchmarks repo
  • push the updated html pages to be displayed as https://www.mdanalysis.org/benchmarks/

Create the crontab entry (crontab -e) to run 3:30am (AZ time, 10:30am GMT) every day.

# MDAnalysis automated benchmarks
# see https://github.com/MDAnalysis/benchmarks/wiki/automated-benchmark
30 3 * * * $HOME/MDA/benchmarking/benchmarks/scripts/cron.sh

Depending on the amount of new commits/added benchmarks, results on the website https://www.mdanalysis.org/benchmarks/ should update a few minutes or hours later.

Access

In order to push to the benchmark repository, @orbeckst created a personal OAuth token for this purpose; in the following I use 123456...wxyz to symbolize the secret token. The benchmarks repo exists on c3potato:

git remote set-url origin https://orbeckst:[email protected]/MDAnalysis/benchmarks.git

Thus, using git locally in this repo will allow push/pull.

Manual execution

The cron.sh script can also be run manually:

cd $HOME/MDA/benchmarking/benchmarks
./scripts/cron.sh
Clone this wiki locally