Berlin Quality Day or is mobile testing really that different?

I was in Berlin last week to attend ASQF’s Quality Day.

After a pleasant trip by coach and a not so pleasant hotel stay (I swear that the street in front of the hotel did NOT have six lanes on Google Maps…) I finally got to Fraunhofer Fokus where the conference took place.

There were eight talks overall. 3 more or less isolated, two about quality assurance in the context of cloud computing and three about the challenges of mobile testing.

The first talk was held by the chair of the Fraunhofer Institute, Prof. Dr. Ina Schieferdecker, and took off where here talk at the Niedersachsen Testing Days more or less ended. The focus (yes, pun intended) was on software testing having to scope more with the hardware side. The premise is that with development as the internet of things, embedded systems will become even more important and therefore the environment of the software as well. The whole idea sounds good to me and I would go even further and say that that usability and ergonomics of the hardware/software combination is something to certainly put more focus on. Context and environmental testing from an anthropological point of view so to say.

Jean Hartmann of Nokia HERE presented how his team tests maps and navigation systems. This was a very neat talk overall, content and presentation wise. Agile development and testing put into service. The interesting thing for me was how they used real-world data from devices like your or my cell phone for their testing and the integration of data warehouses in this process.

My personal highlight of the day was the presentation of Ralf Kruse. I twittered “Wake up talk by @ralfhh interactive and interesting points about knowledge and bottlenecks” during the talk and this still holds true. While the other talks were more reserved he interacted with the attendees and tried to get their input. He talked about his experience in agile projects with distributing knowledge in teams and gave some nice advices on everyday problems. And this is really what made this talk different. The others dealt with more visionary ideas, while this was really down to earth. Everyone could relate and get something out of it. And on top: I had the chance to shortly chat with Ralf afterwards and he seems like a nice guy with sound ideas.

I can’t really say too much about the talk by Prof. Dr. Herbert Werner about quality assurance and his concept of trusted cloud. As the title suggests it was about security issues in the cloud. To be honest, I tuned a bit out during this talk since neither the content nor the presentation really got me. Same goes for Paul Nash’s talk about quality assurance by software analytics. He said that 90% of bugs were of the functional type and 10% of the structural type. And those 10% were the ones he addressed. While I agree that software analytics can be useful, I think that it is rather the 90% that should be dealt with first, since those are the ones bothering the user.

The biggest bunch of talks were about mobile testing. That is an area that I have not really dipped into, so these were interesting and left me a bit puzzled in retrospect. But let’s get to the single talks first.

Ursula Meseberg of the Berlin based company microtool presented their idea of a mobile requirements engineering future. I was looking forward to this talk since I used microtool’s inStep tool for some time and quite liked it. The talk itself was interesting and presented their integration of apps into their tool landscape.

Hannes Lenke’s talked was titled “Mobile Quality provides new challenges”. A broader range of operating systems, different hardware, different screen resolutions and of course data transfer were some of the points he raised. The most important one being maybe that customers giving ratings in the appstore have a direct impact on sales and therefore one should put more emphasis on quality. These are valid points. Just as his conclusions were. While this talk focused more on the functional testing aspect, Henrik Rexed’s splendid talk focused more on the technical side. Especially the network part on latency and bandwidth was interesting for me, since I never really thought about that.

So why was I puzzled and tweeted “On my way home from @yourasqf ‘s Berlin quality day. Lot of info on mobile #testing leaves me wondering if it is really THAT different” after the conference?

While I totally agree that there of course differences in testing, I do also think that most of the solutions presented in those last two mentioned talks also apply to testing software on any other computer (and thus reinforcing what Prof. Schieferdecker said). Let’s get into detail here:

The following points were proposed:

• Know your target audience

I wrote an article (here)on target audience stating that you should know yours, so it is obviously something not just for mobile testing.

• Know the hardware your software is running on

I total agree and there is certainly more diversity in the mobile world. But this reminds me a bit of the PC situation in the 1990s. Standards were not as settled, some software would work on AMD processors, some would not (I had an AMD back then 🙁 ) while it would work on Intel or Cyrix. And if you think of embedded testing, hardware is always something to keep in mind.

• Try to cluster those different hardware types

True as well. For game testing for example, you simply can’t test on every graphic adapter, so take one representative of that group (btw. this has so much linguistic prototype theory in it, I REALLY have to write something about that soon).

• Test on real machines in the real world

The real world part is certainly more important for handheld devices with conditions like travelling on the subway and things like that.

• Use automated tests

Anywhere, anytime where it is appropriate. This one should go without discussion.

• Test on different screen sizes

Now this is an interesting one, since I have had colleagues who laughed when I proposed that we should test on different screen sizes and resolutions in the past. While most screens work on HD nowadays there are still enough notebooks or 4K screens on the verge of becoming mainstream.

I am not trying to downplay all these aspects. On the contrary, but I feel that these are simply not just limited to testing. Maybe I am a bit naive, but good testing is good testing, regardless of conditions und environment.

I really had some nice conversations during the conference. Especially during lunch break where a guided tour through the Fraunhofer facility was offered. While the talks were not 100% congruent with my point of interested, it raised some interesting thoughts for me. All in all ASQF (and Fraunhofer Fokus as the host) did a nice job and I think those small conferences are really worth a visit when they are hosted in your neighborhood.