« Back to all articles

Fast functional testing with Django and Ghostrunner

Written by Jeroen van Veen on 3rd June 2015

• Django
• Ghostrunner
• testing

About

Ghostrunner is a quick and dirty fork of ghost.py; a QT webkit browser that’s controlled from Python. It was modified to make functional browser tests from Django faster and easier to use. We considered both Ghost.py and Selenium, but favoured Ghost.py’s speed and control over Selenium’s arsenal of available webdrivers.

Segfaults

When I started to experiment with Ghost.py, I experienced some irregular lockups/segfaults. Both PySide and PyQt4 had this issue when running the testsuite with multiple Ghost instances. This is probably caused by Qt/QtWebkit, because both bindings failed for similar reasons. Running multiple instances without lockups is a requirement for running testcases, so I tried whether PyQt5 would fix this issue, which it did. No more lockups!

Performance

Another problem we had was that Django’s LiveServerTestCase was way too sluggish. Regular Django TestCase unit tests are much faster to execute, because they use a transaction rollback between tests instead of a database flush. Django’s LiveServerTestCase uses TransactionTestCase which skips doing transaction rollbacks. The reason for this is that it’s hard to properly share database transactions between the server thread and the main testing thread.

MySQL

Because we use MySQL for this particular project, I first tried to make LiveServerTestCase act like a regular TestCase and use transactions with MySQL. This turned out to be quite difficult. After unsuccessfully tinkering around for a while, I gave Django-shareddb a try. The author seems to have solved this particular issue with MySQL using a database wrapper and a delegate thread to handle the transactions. I got stuck with weird data integrity bugs at some point, so I tried to quick-fix this by adding a flush skip option to the LiveServerTestCase, which made the MySQL tests fast again. Then I realized that this broke factory-data with get_or_create’s, because it left data around between tests. Dirty hack!

Back to Sqlite

Finally I ended up using Sqlite, while modifying the LiveServerTestCase to behave like a regular TestCase. With Sqlite it’s apparently much easier to share transactions between threads. This solved the transaction rollback performance problem. Our functional testcases are now fast and reliable again thanks to Sqlite transaction rollbacks and PyQT5. Check out the Github page of the ghostrunner project in case you want to give it a spin(dle) yourself!