We like Travis and it will probably remain part of our arsenal in any event. The reliability issues are being looked into.

Enable Amazon S3 uploads of testing products on Travis. This will help with post-mortem of failures (@mdboom is looking into this now).

We want Mac coverage. The best bet is probably to push Travis to enable it for our project by paying them for a Pro account (since they don't otherwise allow testing on both Linux and Mac).

We want Windows coverage. Shining Panda is an option there.

Investigate finding or building a tool that would collect and synthesize test results from a number of sources and post it to Github using the Github API. This may be of general use to the Scipy community.

For both Windows and Mac, we should document (or better yet, script) the process of setting up the machine for a build, and how to build binaries and installers. This may require getting information from Russel Owen and Christoph Gohlke. This is a necessary step for doing automated builds, but would also be valuable for a number of other reasons.

We should investigate ways to make it take less time

• Eliminating redundant tests, if possible

• General performance improvements to matplotlib will help

We should be covering more things, particularly more backends

We should have more unit tests, fewer integration tests, if possible