Extreme Testing

We hired a new QA manager a few weeks ago. He’s a nice guy named Eric and he sits in the cubicle next to mine.

As I was heading out the door the other day he stopped to ask my opinion on a testing approach for a large project his team will begin testing in December. The idea he described is essentially pair testing, along the lines of pair programming, and he mentioned it was a combination of something he did at a previous job and something suggested by our Director of Development, who used to work with Scrum.

Since the project consists of a bunch of applications working together in one ecosystem, it’s going to take a lot of time to run tests on it, and a comparable amount of time just to figure out if the tests are succeeding or failing. The idea is to cut down on QA troubleshooting and data-churning time by having a developer and a tester sit at the same computer and run through tests together.

I’ve read a lot about pair programming but have only done it a handful of times. The concept of pair testing is intriguing because I know how much time our testers spend working on non-testing issues that a developer could likely clear up if they were sitting at the same desk, and I know our code will be more thoroughly tested because of it. That being said, I’m the development lead on the project and I can’t spare a developer for even a few hours right now due to the massive amount of work to do between now and D-day.

It’s quite the conundrum.

The end result: I plan to support the approach on a limited basis until we can measure the benefits and sacrifices that both teams forego to make it work. Obviously we have to get the software written before they can test it, so it will do us no good to pull a developer from a half-finished application to sit with the QA team. But assuming we can eek out the time, something in my gut tells me this is the best way to thoroughly test this beast.

Have you heard of or participated in pair testing? What’s your opinion on whether or not it will benefit both teams (development and QA)?

[ Bookmark this post with del.icio.us (What is del.icio.us?) ]
[ Submit to Slashdot ]

Start Small, Get Big
Growth Secrets for Self-Funded Startups. It'll Change Your Life.
What you get for signing up:
  • A 170-page ebook collecting my best startup articles from the past 5 years
  • Previously unpublished startup-related screencasts
  • Exclusive revenue-growing techniques I don't publish on this blog
"The ideas and information Rob provides should be required reading for anyone that wants to create a successful business on the web." ~ Jeff Lewis
Startups for the Rest of Us...
If you're trying to grow your startup you've come to the right place. I'm a serial web entrepreneur here to share what I've learned in my 11 years as a self-funded startup founder. Luckily several thousand people have decided to stick around and join the conversation.

For more on why you should read this blog, go here.


#1 Adnan Masood on 10.24.05 at 3:49 pm

As a fan of Test driven development, I’d recommend pair testing on modular level, not on code complete. Why? Agile MSF says so! Ok, not really, but if one sees testing as “a way to measure progress with emphasis on cost and repeatable standards”, pair testing would definitely be the direction. IMHO, testing tenants are being Predictable, repeatable and planned. It’s a measure of progress rather than a mere QC process. I understand it can be frustrating at times or practically difficult to test tightly coupled, interconnecting pieces without all of them being finished but this doesn’t stand a chance on merits which Parallel thinking mindset brings to the table. Reminding that pair programming and pair testing are two separate entities, pair testing would definitely bring agility, effectiveness and a certain degree of intercalation in the code which will help you and fellow developers coping with errors. As a general postulate of software risk process [Pressman], if z comprises of x and y and the probability and exposure of bugs in any of them reaches convergence, it will impact the resultant module (z). Therefore, satisfying test cases individually, by spending a regular hour with tester will help developer practice test driven development.

#2 Ben Curtis on 10.25.05 at 3:49 am

I’ve done something very similar to this on my development team. The tester and developer didn’t share the same desk or the same computer, but they did sit side by side and communicated constantly as the product was being developed. This allowed the tester to be very familiar with the pieces to be tested as they came off the line, and it also allowed the developer almost instant feedback. Of course issues were still logged in the tracking system, but the conversations about the issues allowed the bug-fixing/re-testing cycle to be sped up quite a bit. A bonus was that the developer who was working along with the tester began to anticipate the issues that would arise and was able to reduce the bug counts by solving problems before they got to the tester. Based on my experience with it, where both the turn-around time was shortened and the skills of the developer improved, I would definitely recommend experimenting with pairing testers and developers.

#3 http:// on 10.26.05 at 7:10 pm

My idea behind the approach is to get a less buggy product out of development before it ever goes into the regular QA process. Because the QA process as it is normally done is so expensive, if we can find a way to mitigate this overhead and fix or prevent problems earlier, then we are in better shape overall. So it’s not a specific benefit geared toward the QA group or development group, but to the project group overall. An example I discussed with Rob was a particular rollout we did that required a large amount of very minor changes (mainly text changes) across a number of applications. This left a lot of stuff to verify, and lots of room for error. So when the changes hit QA, the tester begin to produce a huge volume of bugs for these changes. To me this is an ideal situation where pairing a developer and tester will reduce the time spent on the project for both the developer and the tester. If both had sat together in this situation and reviewed the changes the developer was making, a lot of these minor issues would have been caught before they had hit QA, and would’ve saved quite a bit of time. Another area I see the benefits in this is communication. I’ve seen a lot of time wasted because of a lack of communication between the developers and testers. By pairing them up, the communication hopefully should be improved. Also, as Ben states above, the developers and testers have something to learn from each other. I do however think that a normal QA process is needed, but I also think there is plenty of room for techniques such as this to help improve our quality and possibly save us some time on the overall project. Of course, I personally have never tried this before (always wanted to), so there’s always a first time for everything:). There will be some problems to work out, and some things might work and others won’t (it will be an iterative process to work out the bugs), but ultimately I am positive there are benefits to trying this.