Test automation day

27/06/2016 by Christian Kram | Filed in review, testing | Comments Off on Test automation day

This post is a short review of my experiences at the test automation day in Rotterdam last Thursday. If you are following tester circles on twitters, you will be aware that test automation is one of the hot topics at the moment, especially in the CDT realms, but I will try to stay out of that debate here and focus more on the conference itself.

Test automation day was actually the first big, international conference I attended, so I was quite excited to go there and the first shock at the registration didn’t really ease that. I wasn’t on the attendance list and there was no name badge for me (any “I don’t have a picture for you today” resemblances from that top model reality show are completely coincidental!). But things luckily got sorted out and I got a newly printed name badge and was allowed in. Phew!

Talks were mixed between keynotes and smaller parallel track talks, so nothing unusual here, so here is a small summary of the talks I attended:

What really happens when you deliver software quickly?

Sally Goble, head of quality for the guardian opened the day with this keynote. And what a keynote it was. She described the pains they were having with their automation and a rather radical solution for easing those pains: Getting rid of it! Now that’s quite an opener for a conference named test automation day, isn’t it? Instead of bloating their testing, they went for less testing, more monitoring and quicker releases along a motto called “not long wrong”. So it’s a matter of quick feedback and adaption instead of exaggerated testing. The talk was well presented (you can find the slides here) and I was able to catch up with Sally Goble later to get some information about their approach, which I find highly interesting. I especially liked the idea of canary releases: release something to a small audience and see if it works (well, as cruel as it was, but that’s what canaries where used for in the coalmines of the past…).

Test Automation Health

The second keynote was held by Mark Fewster and dealt with the value side of test automation. When is test automation value, provides a return on invest and more important how can you maintain it after the honeymoon period? The main take aways in my opinion were that automation supports testing (don’t you just automate everything, okay?) and test automation has a death wish. So to keep test automation healthy, don’t just look at your test cases for maintenance but at the automation framework as well and make that a goal as well.

Test Automation Smells

 


I am dead serious about this: I can’t smell! But luckily Benedikt Hauptmann can and he presented a tool his company Qualicen developed to improve test quality in terms of code quality via static code analysis. This is actually an interesting approach I have put some thoughts into recently as well (well, without a tool of its own that is). The bad thing about the tool presented here? It only works with the ranorex test automation framework. So if you are using this one, you should probably have a look.

Monetizing Automation

I was hoping for some more value discussion here. Well, not really. Sezen de Bruijn facilitated the session and it was more or less a presentation on pretty generic answers to the question why we automate with a quick discussion of these whys. In hindsight I probably should have attended a session on Reaching symbiosis of Exploratory and Automation Testing which Mirjana Kolarov gave in a parallel slot. Judging from the tweets flying around that one seemed more interesting.

Model-based testing of probabilistic programs

The third keynote of the day was more scientific. Marielle Stoelinga, who is an Associate Professor at University of Twente presented her research on how probabilistic theory adds to model based testing. My personal take on model based testing has changed a bit over the last years and months and I am not as convinced as I used to be. Don’t get me wrong, I still think it can help testers, but I feel that it is more of a scholarly approach not too applicable in everyday work. Every professor I hear talking about it seems to point out that it will be the way of testing to be done in the future, but I haven’t really heard any practitioners talk on it. Anyway, I still liked the idea of including probability into testing as it goes beyond the pass/fail dichotomy and takes statistics into account.

Automation, Now, Then, Where

The next session was facilitated by Richard Bradshaw, who is a really friendly guy. And I don’t just say that because he sponsored my ticket (thanks a bunch!), but because he is! His session was more interactive which people really seemed to like. He made people talk about the now and then of automation and gave his idea afterwards. And you did notice that it was all about automation and not just test automation, did you? Huge difference! So this one really hit a nerve and resonated well.

Taking responsibility: Why are just the testers testing?

This talk went full circle from the opening keynote. Jon Hare-Winton, also of the Guardian, presented how testing as evolved at the Guardian during their transformation, with a slight focus on setting up their mobile testing. It was an interesting addition to the first keynote and helped to get a nice understanding at what they are doing over there.

Can We Deal with Uncertainty and Faults in a Systematic Way?

The closing keynote by Alexandre Petrenko was a scientific one on model based testing as well. I found the speaker hard to follow and didn’t really agree with all of it and it seems I wasn’t the only one. At least someone raised up in the Q&A part and asked “I don’t really believe you. Could you maybe give us an example?”

Summary

The whole conference was really well organized and you had always time between sessions to get to talk with other people. And for me, that was the best part of it. I got to talk to some of the presenters and quite a few interesting people, which I can’t name all in here (and that’s only maybe because I don’t remember all the names…).  Overall I really enjoyed my time at the conference. The three main themes that I sensed were

  • automation is more than just test automation
  • automation supports testing, but doesn’t replace it
  • there are ideas and approaches beyond automation

 

I haven’t really checked if all presentation are available to the public yet, but some of them are in case you are interested in those.