You Say Test I Say Check. On We Go.

This discussion has been going on for a very long time, and it won’t ever end, it won’t end. I repeat that because it’s reality, such a discussion will never end. But that’s OK.

I consider myself a student of testing, over the last ten plus years I’ve formulated a mass of thoughts on testing. One of those happens to be the believe that there is a huge difference between what a human can do and what a computer can do, in the context of testing. One of those distinctions is labelled by some as checking and testing. The distinction being that when we automate a check, it’s a fixed algorithm, unable to observe anything other than what’s it’s instructed to. Unable to change its behaviour or flow other than the flows that have been coded. Therefore, it’s checking the flow of the steps and some outcome/behaviour at the end. However, when we test, due to the fact we’re humans and have the best tool out there, the brain, we can observe all, we can change our focus hundreds of times a minute. We can observe in real time and adapt accordingly. Now sure, for the AI/ML fans, sure computers are showing some signs of this now, but they won’t be in the regular development teams office any time soon. But that’s not the point of this post.

I believe this distinction is universally agreed by any student of software development, and especially testing students. So, we end up at our choice of words and their importance. They do carry some importance. Others have written far better pieces on that topic then I can, so we won't go in to this.

So, let’s look at this from my angle:

I don’t care what words you use.
I actually don’t. I do not care. I’ll never correct you. I’ll never reply to your blog post going ‘I think you mean checking’, ‘excellent piece, but by the way, automation cannot test, that’s checking!’. I won’t do that.

What I will do, is frame my responses to match my own views of testing, because in my testing world there is testing and checking. Those words are congruent with my view of testing. Therefore, when I write or talk about testing, I use those words, they fit my current beliefs of testing. They fit my models of testing. My models. They may not be in your beliefs or models of testing, and that is OK. In fact, it’s more than OK, it’s awesome, it’s awesome that you’re aware of your own models and sharing those with a wider audience.

You will hear me say ‘automated checking, you will hear me say ‘automation in testing’ and you will hear me say ‘automated check. You will also hear me say, ‘I’m going to use my language for testing during this talk, so if you hear me say the word check and you’re unsure what I mean, replace it with test and say it back to yourself’. I make a similar statement when I say tester these days, ‘if I say tester during this talk, I mean anyone who does some testing, not the role’, but I digress. You will also hear me say the words above without making such a statement, but hey-ho, on we go.

You will hear and read people saying automated testing, you will hear and read people saying automated checking, but allow yourself to process them against your models, let you brain change checking to testing, or check to test, or vice versa and continue the conversation, continue the learning.

Now I do tailor my language for who I’m talking to. If I’m talking to a fellow student and I know their models are similar to mine, I’ll choose my words accordingly. This allows us to get straight to the heart of the discussion. If I’m talking to a client, I’ll use their language, as this will make them more comfortable and allow me to truly understand their problem and needs. Overtime I may introduce some of the words more congruent with me, but only if it brings value and helps us solve the problem(s).

So on we go, I say check, you say test, I’ll say test sometimes, you’ll say check sometime. Be congruent with your own beliefs and models, but at the same time, bend a little bit, flex here and there, I believe such an approach will improve all of us, and in turn, testing and software development.

What is LegoAutomation?


TL; DR. Solid, valuable, reliable automation requires far more important core set of skills than coding. We can and should practice these skills. LegoAutomation is designed to introduce you to these skills, in a highly unique interactive class. The class has something to offer all roles involved with software development. We do not write any code during the class or use laptops; broadening the scope and application of knowledge gained. We use Lego Duplo, the room, a dose of creativity and your desire to learn. Public classes are available now, read further to find out more. Private classes are available internally upon request. 

This is a highly interactive class, completely unique, doesn’t involve a laptop, uses Lego to model theory and is entirely focused on automation in the context of testing.

I’ve spent years teaching automation, designed and ran several technical workshops. All included writing code and using some specific frameworks. These are great for people interested in those frameworks and wanting to learn those programming languages. That’s what those classes do, they teach people how use a specific framework with a specific language using some predesigned scenarios. There is nothing wrong with these courses, I still teach them myself, but what about a more holistic approach to automation? Where can we go to learn about this? Where does someone go to improve their automation skills beyond the code itself? Because, believe me when I say, there is so much more to automation than code! 

These are the reasons I designed Lego Automation. I wanted a class that focused on automation, but didn’t involve laptops and code. For several reasons:
  • It doesn’t alienate anyone; anyone can attend the class and no previous programming experience is required. I’ve had Testers, Developers, Managers, ScrumMasters and even a CTO attend the class.
  • We can dive straight into active, experiential learning, no laptops to try and tame, no dev environments to configure. This allows us to make the most of our time. 
  • Reassure learners that they have far more automation skills then they may realise, a lot tend to just focus on the programming languages and tools they know. 
  • Introduce people to some terminology to allow them to talk about their automation, not just the code.
The class consists of 4-7 rounds, depending on the time allocated. Each round is designed to build upon the last, allowing us to build upon our knowledge gained from earlier rounds. We use Lego Dulpo and the room to design the product we’ll be automating. I don’t want to share too much about the specifics, but you can explore #LegoAutomation on Twitter to get an insight into the class. 

I do want to spend some time talking about my aims for the class, and what I hope attendees leave with.

The class is completely experiential, there is a loose structure in place (the rounds) to try and guide the learning, but importantly there are no right answers in this class! There is no correct way to write the scripts you create, or to solve the problems, there is just the way you take. 

One of the key aspects of the class is getting people talking about their work. We’re working closer and closer together these days, in terms of interacting with team members. Which means we need to be comfortable talking about our work. Which in the context of automation needs to be more than just explaining the code and why X framework was used over Y. We need to be able to walk people through our problem solving, our check design, and our final solution. 

We also spend some time exploring assumptions, specifically how our automation is full of them, but often unspoken or undocumented. We look at the importance of them, and how they influence our design. Talking about influence, we also explore biases in the context of automation and of course Lego!

We’ll explore some heuristics which are invaluable when designing checks and code. We’ll also introduce you to some automation specific terminology. I have too often seen competent ‘automators’ struggle to convey their thinking and rationale to colleagues and this section of the course will help overcome this common problem.

There is much more, but at the very core of this course is what using Lego gives us. It gives us the ability to quickly put theory into action and to get feedback there and then in the class! Giving you a personal and active learning experience, whilst broadening and deepening your automation skills. Sure, we’re using Lego and none of you create your automation using Lego in your workplace, but the thinking and processes are the same. This gives attendees a real foundation for their automation efforts, completely applicable to any language or framework.

Here are some takeaways for different roles:
  • Testers
    • The ability to talk congruently about automation.
    • Realisation that ‘non-technical’ testers have a huge part to play in an automation effort, without writing any code!
    • Modelling is a crucial skill regarding automation.
  • Developers
    • Appreciate that we can’t just create automated checks, they require thought, a testing mind-set and developers can achieve this.
    • The code is important and we should follow good practices. They really can help testers/automators with this, they know a lot about coding!
    • Appreciation for well targeted automation and how the testability of the products is a significant part of this method. Testability is something developers can improve.
  • Managers
    • Queue sweeping generalisations! Appreciation that automation requires constant support, we can’t just write and forget.
    • An understanding of the steps and effort involved in creating highly maintainable automation, both from a code and check perspective.
    • We can’t just hire someone with the best coding skills, we need automators who understand testing. 
  • CTOs
    • I doubt another one will attend, but you never know! I hope they take away that that every team member has a part to play in testing, especially automation, and they should encourage this to be the standard. 
So, with all that being said, if you’re looking to move into automation or improve your universal automation skills, I am certain that this is the class for you! This really is a class like no other! I’ve been running this class for over 2 years now and I’m yet to receive any complaints… perhaps everyone is just too polite. But genuinely, I have received many compliments and positive feedback from attendees from all backgrounds, skill level and role. Especially based around the finding that using Lego really works! It’s not just a gimmick, it truly does support the theory, class and learning. 

I regularly teach the class publicly, the next public classes are:
The class is also available internally, please contact me via my blog for more information on that.