You Say Test I Say Check. On We Go.

This discussion has been going on for a very long time, and it won’t ever end, it won’t end. I repeat that because it’s reality, such a discussion will never end. But that’s OK.

I consider myself a student of testing, over the last ten plus years I’ve formulated a mass of thoughts on testing. One of those happens to be the believe that there is a huge difference between what a human can do and what a computer can do, in the context of testing. One of those distinctions is labelled by some as checking and testing. The distinction being that when we automate a check, it’s a fixed algorithm, unable to observe anything other than what’s it’s instructed to. Unable to change its behaviour or flow other than the flows that have been coded. Therefore, it’s checking the flow of the steps and some outcome/behaviour at the end. However, when we test, due to the fact we’re humans and have the best tool out there, the brain, we can observe all, we can change our focus hundreds of times a minute. We can observe in real time and adapt accordingly. Now sure, for the AI/ML fans, sure computers are showing some signs of this now, but they won’t be in the regular development teams office any time soon. But that’s not the point of this post.

I believe this distinction is universally agreed by any student of software development, and especially testing students. So, we end up at our choice of words and their importance. They do carry some importance. Others have written far better pieces on that topic then I can, so we won't go in to this.

So, let’s look at this from my angle:

I don’t care what words you use.
I actually don’t. I do not care. I’ll never correct you. I’ll never reply to your blog post going ‘I think you mean checking’, ‘excellent piece, but by the way, automation cannot test, that’s checking!’. I won’t do that.

What I will do, is frame my responses to match my own views of testing, because in my testing world there is testing and checking. Those words are congruent with my view of testing. Therefore, when I write or talk about testing, I use those words, they fit my current beliefs of testing. They fit my models of testing. My models. They may not be in your beliefs or models of testing, and that is OK. In fact, it’s more than OK, it’s awesome, it’s awesome that you’re aware of your own models and sharing those with a wider audience.

You will hear me say ‘automated checking, you will hear me say ‘automation in testing’ and you will hear me say ‘automated check. You will also hear me say, ‘I’m going to use my language for testing during this talk, so if you hear me say the word check and you’re unsure what I mean, replace it with test and say it back to yourself’. I make a similar statement when I say tester these days, ‘if I say tester during this talk, I mean anyone who does some testing, not the role’, but I digress. You will also hear me say the words above without making such a statement, but hey-ho, on we go.

You will hear and read people saying automated testing, you will hear and read people saying automated checking, but allow yourself to process them against your models, let you brain change checking to testing, or check to test, or vice versa and continue the conversation, continue the learning.

Now I do tailor my language for who I’m talking to. If I’m talking to a fellow student and I know their models are similar to mine, I’ll choose my words accordingly. This allows us to get straight to the heart of the discussion. If I’m talking to a client, I’ll use their language, as this will make them more comfortable and allow me to truly understand their problem and needs. Overtime I may introduce some of the words more congruent with me, but only if it brings value and helps us solve the problem(s).

So on we go, I say check, you say test, I’ll say test sometimes, you’ll say check sometime. Be congruent with your own beliefs and models, but at the same time, bend a little bit, flex here and there, I believe such an approach will improve all of us, and in turn, testing and software development.

What is LegoAutomation?


TL; DR. Solid, valuable, reliable automation requires far more important core set of skills than coding. We can and should practice these skills. LegoAutomation is designed to introduce you to these skills, in a highly unique interactive class. The class has something to offer all roles involved with software development. We do not write any code during the class or use laptops; broadening the scope and application of knowledge gained. We use Lego Duplo, the room, a dose of creativity and your desire to learn. Public classes are available now, read further to find out more. Private classes are available internally upon request. 

This is a highly interactive class, completely unique, doesn’t involve a laptop, uses Lego to model theory and is entirely focused on automation in the context of testing.

I’ve spent years teaching automation, designed and ran several technical workshops. All included writing code and using some specific frameworks. These are great for people interested in those frameworks and wanting to learn those programming languages. That’s what those classes do, they teach people how use a specific framework with a specific language using some predesigned scenarios. There is nothing wrong with these courses, I still teach them myself, but what about a more holistic approach to automation? Where can we go to learn about this? Where does someone go to improve their automation skills beyond the code itself? Because, believe me when I say, there is so much more to automation than code! 

These are the reasons I designed Lego Automation. I wanted a class that focused on automation, but didn’t involve laptops and code. For several reasons:
  • It doesn’t alienate anyone; anyone can attend the class and no previous programming experience is required. I’ve had Testers, Developers, Managers, ScrumMasters and even a CTO attend the class.
  • We can dive straight into active, experiential learning, no laptops to try and tame, no dev environments to configure. This allows us to make the most of our time. 
  • Reassure learners that they have far more automation skills then they may realise, a lot tend to just focus on the programming languages and tools they know. 
  • Introduce people to some terminology to allow them to talk about their automation, not just the code.
The class consists of 4-7 rounds, depending on the time allocated. Each round is designed to build upon the last, allowing us to build upon our knowledge gained from earlier rounds. We use Lego Dulpo and the room to design the product we’ll be automating. I don’t want to share too much about the specifics, but you can explore #LegoAutomation on Twitter to get an insight into the class. 

I do want to spend some time talking about my aims for the class, and what I hope attendees leave with.

The class is completely experiential, there is a loose structure in place (the rounds) to try and guide the learning, but importantly there are no right answers in this class! There is no correct way to write the scripts you create, or to solve the problems, there is just the way you take. 

One of the key aspects of the class is getting people talking about their work. We’re working closer and closer together these days, in terms of interacting with team members. Which means we need to be comfortable talking about our work. Which in the context of automation needs to be more than just explaining the code and why X framework was used over Y. We need to be able to walk people through our problem solving, our check design, and our final solution. 

We also spend some time exploring assumptions, specifically how our automation is full of them, but often unspoken or undocumented. We look at the importance of them, and how they influence our design. Talking about influence, we also explore biases in the context of automation and of course Lego!

We’ll explore some heuristics which are invaluable when designing checks and code. We’ll also introduce you to some automation specific terminology. I have too often seen competent ‘automators’ struggle to convey their thinking and rationale to colleagues and this section of the course will help overcome this common problem.

There is much more, but at the very core of this course is what using Lego gives us. It gives us the ability to quickly put theory into action and to get feedback there and then in the class! Giving you a personal and active learning experience, whilst broadening and deepening your automation skills. Sure, we’re using Lego and none of you create your automation using Lego in your workplace, but the thinking and processes are the same. This gives attendees a real foundation for their automation efforts, completely applicable to any language or framework.

Here are some takeaways for different roles:
  • Testers
    • The ability to talk congruently about automation.
    • Realisation that ‘non-technical’ testers have a huge part to play in an automation effort, without writing any code!
    • Modelling is a crucial skill regarding automation.
  • Developers
    • Appreciate that we can’t just create automated checks, they require thought, a testing mind-set and developers can achieve this.
    • The code is important and we should follow good practices. They really can help testers/automators with this, they know a lot about coding!
    • Appreciation for well targeted automation and how the testability of the products is a significant part of this method. Testability is something developers can improve.
  • Managers
    • Queue sweeping generalisations! Appreciation that automation requires constant support, we can’t just write and forget.
    • An understanding of the steps and effort involved in creating highly maintainable automation, both from a code and check perspective.
    • We can’t just hire someone with the best coding skills, we need automators who understand testing. 
  • CTOs
    • I doubt another one will attend, but you never know! I hope they take away that that every team member has a part to play in testing, especially automation, and they should encourage this to be the standard. 
So, with all that being said, if you’re looking to move into automation or improve your universal automation skills, I am certain that this is the class for you! This really is a class like no other! I’ve been running this class for over 2 years now and I’m yet to receive any complaints… perhaps everyone is just too polite. But genuinely, I have received many compliments and positive feedback from attendees from all backgrounds, skill level and role. Especially based around the finding that using Lego really works! It’s not just a gimmick, it truly does support the theory, class and learning. 

I regularly teach the class publicly, the next public classes are:
The class is also available internally, please contact me via my blog for more information on that.

How I Get Technical Workshops Up and Running


A few days a go I saw this tweet from Maaret. At the time of reading it, I felt I was at 100% success rate with technical workshops. I had the formula.
I stated on Twitter that I would write about the formula. Then last Tuesday happened.

Last Tuesday, I visited a client to do some internal WebDriver training, and well lets just say, it didn't go to plan. We adapted the plan until lunch, then throw all the plans out after lunch and did something completely different. I'm fortunate to have alternatives at my disposal and be comfortable enough with my knowledge to deal with it, others may not feel the same.

So I'm going to write about some of the approaches I've taken in order to get technical workshops up and running faster, allowing us more time to actually learn.

Virtual Machines (VM)

I haven't used this approach for a while. In fact, I only ever tried it once. However I know several people who swear by this approach. Simple concept, create a VM with all the dependencies on it. Have the attendees download a specific VM client, distribute the VM image, set-up the image and off we go!

My class didn't go to plan. Peoples machines were not powerful enough to run a VM. They got the VM up and running, but interacting with it was painful. Like using IE on 56k modem painful, or watching AOL connect back in the day painful.

Now, other issues I've seen with such approaches, is distributing the image. These images can be big files, some into the GB's depending on the operating system. So downloading them in advance is an option, but could be a slow, data consuming process for some. If they don't download them in advance, it's time to pray to the conference wifi while someone, or several people, download this image. Or, we distribute the image on USB pens, an option not all would be comfortable with, given the security concerns of flash pens.

My final issues with a VM is that in some context, the actual process of setting up the environment is important, especially if a student intends to take their newly found skills forward and use them at work. They will no doubt have to set up an environment, and by using a VM, they won't know how to do this, or not all of it. Easy for the trainer to mitigate by providing a environment setup guide, but I've only had one person do this.

Docker

It wouldn't be right to talk about VMs and not include Docker. Now, let's be clear I'm not stating they are the same thing, they are not. However one use of Docker I've tried on a course I teach with Mark Winteringham is to dockerise the test application. So we ask attendees to install Docker, then we distribute the container. This is because the application is not our focus of the training, it's just the vehicle. We took this approach, as the application has several dependencies, rather poor instructions to deploy, but more importantly, was quick to get up and running. We'd also taken the approach to dockerise the application for our own deployment to AWS, so was a bonus really. 

So I would recommend containers for a test application. But they don't really work for tools that the students are going to interact with, because you cannot get in side them as easy as a VM. 

Going to move away from tools now and talk about approaches I've taken, prior and during the class.

Advanced Prerequisites

Send the students a list of prerequisites as far in advance as you can. Not always easy, especially if using certain tools, as new versions could be released during that window. So be aware of that.

These prerequisites should be more than just a list of tools. Where possible include installation guides or at least link to existing ones. Give the student as much support as you can. Consider writing a list of instructions, referencing specific guides for tools if required, giving the student something to follow. 

Checklists. Provide a checklist for the user to run through to ensure they have all they needed. For example, at the end of installing all the prerequisites for a Java WebDriver class, you might instruct a student to execute 'javac -version' in the console, to ensure the JDK was installed successfully. 

I've more recently taken to recording a short video that students can follow. Details where to find the prerequisites, downloading them, installing them, and checking everything is as expected. It might seem like overkill, however it's something I was doing before each class to check my own environment was prepared, so recording it wasn't much more work.

Now, some of you will would have been reading that, with a little voice in the back of your head saying 'but no one ever does them Richard!!!'. True, there will always be a few people who don't. My only advice here is to ensure they get your message. So ask students to respond to tell you they have received your email, allowing you to chase those that don't respond in the run up to the event. Another additional you can take is to give students a way of informing you they have indeed done it. This could be a simple email, or a Google spreadsheet/form where they can tick their name of.

Also, stress the importance of these to the students. They allow the class to get up and running faster, meaning we can all maximise the time we have together.

Finally, make yourself available to help debug students before the class. I've only had to do this a few times, but it really did make the class go smoother on the day. Helped my focus, and importantly started all students on the same page, keeping the flow going.

Pretest with your contact

This advice is more related to internal client visits, and one I received recently from Alan Richardson. Arrange a call/hangout with your contact at the company, and do a pretest on their machines. So instruct them to run the prerequisites, but they have a call to distribute the specific code for the class and ensure they can compile and run it.

This would have saved me recently, as installing prerequisites was all good, however when it came to actually import the project, it turned out the clients internal network blocked all maven calls, meaning none of the dependencies could be downloaded, something I wasn't prepared for.

Start immediately

Both internally and public events, I tend to get to my room 30-45 minutes before I need to. I like to get my machine set up, the desk how I like it, and just get a general feel for the room. This is also the case for some students. In my experience students tend to arrive 15 minutes or so before the official start time of classes. I like to take advantage of this time.

I will ask those who arrive earlier if they have downloaded all the prerequisites and have everything set. If someone says no, I can now try and get them setup in the free time. By free, I mean it isn't going to eat into the class time. This isn't always an option, but it's one I like to take advantage of if it arises.

Class support

When running a class, especially at a conference, I like to gauge the level of the students. As there is usually someone who has experience with the environment, tool or programming language. If I do believe I've found someone, I will approach them to see if they are willing to help me with getting people setup. Then can be incredible valuable to me, the class, and them as it will speed us up in getting to the fun bits!

Backup plan

Have a back up plan. I could end it there, but let me elaborate with some experiences. Wifi Internet... as we all know, it can be very hit and miss. So if your class requires access to a specific site, and the Internet goes down, you're a bit stuck. So if you have control of the application consider ensuring you can run this site locally. This could then allow you to turn your machine in a hot spot and have people connect and use an instance running on your machine, instead of using the www. 

Another option is to have some theory activities ready, which you can get students to work through, while you try and resolve issues. Visualisation tasks work great for this. Or a retrospectives on what's been covered this far.

Have installations, remember to include all operating systems, on a USB pen ready to distribute if the Internet has gone. Of course this comes with the risks mentioned above.

Pairing

If someone is really struggling with their environment / laptop, see if someone in the class is willing to pair with them. This can be a great way to keep the class flowing. Then at a break or lunch, you can then work with the student on the machine to see if you can get it up and running for them.

Pairing also comes with multiple other benefits which lots of people have written about.

Your machine

If pairing isn't an option, I've actually given my machine to a student before. This works well in a show and tell then practise format. In the instance I did this it was a class where we were exploring how to use proxies with your mobile device.

Not always a valid option but one to consider.

If you have multiple laptops, this could also be a possibility. Give the student a spare machine. I personally don't have a spare, but some of you might!

Stay after the class

I still like to work to the principle that every attendee of my classes will leave with working/running code on their machine. So with that in mind I always try and ensure that I'm free immediately after my class. I do this anyhow for questions etc, but even more time to debug a machine if needed.

Post the class, conference

I like to say that my classes come with life time support. So I do. And I mean it. Funny how very few people take me up on this. Anyhow, what I mean is, if we were still unable to get a machine or something up and running, I will give them access to me post the event. This could be my email, Skype or hangout etc.

So the student can work on the issue in the own time with my support. Or, when they have got their environment working they may want to re-run through specifics from the class, I'm always happy to do this.

Videos / Handouts

Another back up plan is to have a video(s) or handout(s) of the activities available for attendees post the class. This is especially useful when talking over code, or explaining how to use a specific tool. Allowing the student to work though the material again in their own time, pausing where they need to. If you 

Summary

Technically workshops are hard. They are more complex by the very fact we have to use laptops. However we can do a lot to make this easier, however a lot of it requires work prior to the event. However as a teacher of technical classes I feel we have to go this extra mile.

If I design a class on a specific tool I want to ensure I can teach about that tool, not spend the first 0.5-2 hours battling admin permissions or other common issues. I want to teach the class.

If you have any advice on the above, or other ways you try to ensure the smooth running on a technical workshop, I would love to hear it.

Thanks for reading.