Additional test sets

[ottapav]

Currently, there is (well-known) Maros-Meszaros test set.

Does it make sense to add some other established test set? If so, which one?

[ottapav]

For subset of LP, there is a nice benchmark set of challenging problems: https://coap.math.ufl.edu/coap-test-problems/

[ottapav]

There were test QP sequences at http://homes.esat.kuleuven.be/~optec/software/onlineQP for various control problems (crane, diesel, chain80) designed by Hans Joachim Ferreau. But it is not available anymore. Maybe he can be asked for it?

[stephane-caron]

It would be OK to add LP-specific test sets, although several QP solvers require strictly convex problems and would not be able to perform there. That would prompt us to refine a bit the filtering code (currently we only have TestSet.sparse_only to filter dense/sparse solvers), but that's not difficult work.

[stephane-caron]

Yes, PR for new test sets are welcome 👍

[stephane-caron]

The benchmark now has a clear process to integrate new test sets. Each test set comes in its own directory, where it only needs to have a main Python script that we can call as follows:

qpsolvers_benchmark my_test_set/my_test_set.py run

See for instance how this is done in the github_ffa/ or maros_meszaros/ directories.

With this design, test sets can live in their own repositories, i.e. there is no need to merge results back into the qpsolvers_benchmark repository. For example, there could be a "linear model predictive control" test set living in its own repository, which would be interesting for all practitioners of this particular application of quadratic programming. And of course if the scope of a new test set is quadratic programming in general new test sets are welcome in this repository too 👍