Winging It

Cucumber Is Not a QA Tool

In my recent travels, and they have been plenty, I have come across some interesting views about Cucumber.

In one conversation someone was looking for "QA people with Cucumber experience." I thought that was pretty cool, they want some testing folks familiar with a rather popular test framework. Then came the second sentence, "I really don't know what cucumber does, but I know that QA people need to do it." (Yes, this person was a recruiter, if you couldn't tell by now.)

Contrast that with conversations with management and leadership types who don't know what cucumber is, but as you explain it you can see the words scrolling across their foreheads, "If we use this, I can lay off half my QA people! Cost savings!!"

Now, I can not totally discount both points, there is a little merit of truth in each one of them. In the first case, a QA group that knows and understands Cucumber can be a great asset to your product team and the long term quality of your product. Additionally, once Cucumber is in place it will free up some QA departments from doing repetitive manual tests and allow them to focus on more exploratory testing.

However, if you consider Cucumber to be only a QA tool, you are missing a good portion of what this and other Acceptance Test Driven Development (ATDD) or Behavior Driven Development (BDD) frameworks can provide.

Note: From this point forward I will use BDD in the discussion. BDD and ATDD are very closely related, almost indistinguishable in practice. I learned the phrase BDD first, so I tend to stick with that. And for writing purposes, it is one character shorter, thus promoting my natural laziness.

Collaboration

The biggest benefit to employing Cucumber, or any BDD framework, is that it tears down a lot of the communication walls to which we have become accustomed. Specifically the Gherkin syntax of Given/When/Then helps bring down those walls by writing acceptance criteria in a plain text format that everybody can agree on.

From the business's standpoint, finally being able to communicate to the development and QA teams in a clear manner has to be beyond welcome. Being able to work with multiple team members and arrive at something for the delivery team that says:

Given the customer has an item in their shopping cart
When the click the checkout button
Then I will have more money in my bank account

No more reams of documentation around a simple feature with wire frames, UML diagrams, feature documents, and other items buried in a wiki somewhere that gets ignored. Plain text is easy, and in my experience the easier something is to use, the better chance you have of it getting used.

Now from the QA person's standpoint, we have actual acceptance criteria that let us know if a feature is complete or not. No more vague language that allows room for interpretation where I can inadvertently add more functionality to a feature because it is what I think the user might want someday. Maybe.

Finally from the developer's standpoint we have features with an end point and a better description than, "Customer checkout feature." And not only do we have more clear, concise feature descriptions, when we are done writing code for the features we have a set of tests that will execute against the plain text features.

Build What We Want

Cucumber benefits all aspects of the business, but possibly its greatest benefit is that it allows us to easily agree to what "done" means for any given feature. The definition of done for a feature won't change over the life of the project, or if it does change we will have a broken Cucumber spec to alert us that we have changed our mind somewhere. But by having that agreement of "done" in place up front it frees up the developers and testers to focus on specifics of implementation and verification of the feature without ambiguity.

Developers of many languages keep a set of unit tests handy that will break if they make a change to existing code. It may or may not be an intentional break, but the point is they have a safety net to catch a logic change. A suite of Cucumber tests has that same watchful eye over your application as a whole, rather than just at the class or method level. If the business changes how a feature behaves and a cucumber test goes red, then the collaboration can kick back in across the whole team and the team can decide how to handle the original intent of the feature with this new feature request.

In short, we have a living record of what "done" has meant to every feature we have developed over the life of the project. By executing tests against those acceptance criteria, we also have a constant check that "done" for existing features remains done.

Cucumber Helps With Quality

So while I don't think Cucumber is a QA specific tool, it helps with overall product quality. By providing a good place for collaboration and by helping the team define "done" early in the feature life cycle, quality has become a first class citizen to your team. Quality is a whole team goal, not just members of the QA team. (By definition, people on the QA team assure quality, they don't build it.)

If you are not using Cucumber or another BDD framework, I would highly recommend looking into using one. However, look into those tools for the benefits they can bring your whole team not simply as a QA tool. And while they will help alleviate some manual work for the QA folks on your team, Cucumber is no replacement for a human being who can dive into the application in ways you never thought of when laying out the acceptance criteria.

After all who would you rather have exploring your app and poking around looking for things to go wrong?

Your QA team?

Or your users?

Kanban on the Factory Floor

Over the last few years I have done a fair amount of work with Kanban on software teams, but recently I got to see Kanban in action in a factory. Knowing that the Lean software movement was born in manufacturing, it was very interesting to see Lean and Kanban in action on a factory floor.

Some quick background details before we get too much further. At the request of my client, I can not share what was being manufactured (or who my client is). What I can say is the items they build are large. Large enough that they need to be separated into pieces to be loaded on to a semi trailer to be shipped. Sometimes multiple semi trailers are needed to ship one item. Steel is the principle material used in the manufacturing, so there is also plenty of cutting, welding, pressing, and painting going on to go from raw steel to completed product being shipped to the customer.

The building itself sits north to south, with the north end being shipping of the finished product. The south end of the building is where the raw materials are received, so the product moves south to north through the building to become a shippable product.

Factory Floor as a Kanban Board

The first thing that jumped out at me upon touring our factory was that the entire building was the kanban board for this particular product. On a software project we will determine our value stream and write it on a whiteboard. These are the steps our features will work through from concept to cash. In a factory, there is no whiteboard with columns on it. Instead there is the factory floor with overhead cranes, welders, and paint booths.

This idea of the factory as a Kanban board is pretty easy to see when you are looking at it. In an attempt to explain it, the south end of the building has stacks of steel with other supplies nearby in a storage area. The factory is run pretty lean, but they still need a supply of welding materials, nuts, and bolts.

The steel begins its journey north through the building by first being cut and shaped. It makes a stop at a plasma cutter, then some of it will move into a huge hydraulic press to be bent into the correct shape. From there it moves into the welding booth where robotic welders make the standard welds. Any custom welding will be done by hand at the next station. Following welding, the product is essentially put together and the finishing touches start by going through a cleaning process then into the paint booth. Once it is painted the only thing left to do is get it loaded on a truck to get shipped to the customer.

That is a pretty abbreviated version of the process. But from a Kanban standpoint once the table under the welding robots is available, a sheet of steel slides in there. Same with the multi-ton hydraulic press. If it has capacity, it has steel being pressed. The steps of the process are not arbitrary, they are physically located to the north of the step that precedes it.

Lean and We Mean It

On the Lean side of things, there is no inventory. Once an item starts getting produced, it has already been sold. There really is no room to keep one or more of these products just sitting around waiting to be sold. It is not a practical solution for this particular factory.

On the other end of the spectrum, there really is not any room for a large amount of surplus steel to be stored, either. If the steel shows up at the south end of the factory, it will go into production pretty quickly in an effort to get it into its customer's hands.

Once again, there physically is no room in the factory to hold excess steel for that "just in case" order that comes in from the field. Nor is there room to hold any extra finished product at the north end of the factory for that "emergency" client.

Work In Progress Limit? Um, Yeah...

On the software side of things we struggle many times trying to enforce our Work In Progress Limits, or our WIP Limits. Many times if we need one more thing put into development or testing, it is just a post-it note...put it in there! Stick it on top of that lesser post-it note if you need the capacity.

Let's take a look at that same situation in our factory. We have a new order that just came in, and it is an urgent one. Yeah, I know we already have 8 tons of steel being welded by the robots, go ahead and put another 8 tons of steel on top of that and get it done first.

Seems a bit silly, doesn't it?

Once again, the physical limitations involved with moving, shaping, and welding tons of steel comes into play. It makes enforcing WIP Limits pretty easy. Only one thing can be cut, welded, or painted at a time. If an "emergency" comes into the queue, it has to wait its turn, there really is no room for it.

On a software team, that limited capacity is just as real, but it is much harder to visualize. It is very easy to see a paint booth full to capacity, not quite as easy to look at two developers and two testers and see that those four people are already running at capacity with the work they have been given.

What We Learned

I have enjoyed working with Kanban in the software industry, but it was definitely worth seeing it working in manufacturing to see the differences. For software teams the idea of limiting work in progress is a great goal, but clearly much more difficult to visualize than it is in manufacturing. My biggest takeaway from seeing Kanban in action in a manufacturing setting was that WIP Limits are easy where there is physically no more space for more work.

The discipline needed in the software implementation of Kanban to enforce WIP Limits and keep queues from building up is more than a number on a white board.

Going Independent

I have made a few career moves over the last 24 months, but I have finally gotten to the point where I can go off on my own. I guess more correctly, I finally got the courage to pull the trigger on going independent. So October 28th will be my last day as a full time employee with Pillar.

I am looking at doing more agile coaching and training as I move forward, but banging out some code will never be far away. Making this move should free up some time to explore some other opportunities that have presented themselves, and opportunities that have as yet come knocking.

It's not going to be a big shock to the system as my first job as an independent will be helping my current team transition on a couple of new team members. So, day one as an independent developer, off on my own in a scary new world will be in the exact same chair in the exact same room that I occupied the Friday before as a full time employee.

Ironical.

I would love to go off and list all the folks I have to thank for getting this far, but I would end up leaving somebody out, and they would get all upset, and it would just get ugly at some conference somewhere. I would like to thank my family, Wendy, Brendan, and Alex for their constant support.

Iterations Are Not a Deadline

I have worked in iteration based systems for years. I have worked in one week and two week iterations. (I have heard tell of people doing iterations measured in months or quarters, which scares the bejeesus out of me.) I have also worked on teams where we have gotten rid of iterations all together because they did not make sense for our situation. Those teams were Kanban teams and we valued continuous flow over the choppy stop, start, "slam QA on Thursday" that iterations can sometimes become.

At a recent conference I overheard a conversation between a colleague and an attendee. The colleague in this case had just given a seminar on Kanban that this particular person had attended and had a few follow up questions. The first few questions were pretty mild, then it happened...

"How do you make Kanban work without the pressure of the iteration to make your developers finish their work?"

That one fired me up. I did not dive in the middle of the conversation red faced and shaking my finger in this way-too-obvious-project-manager's face. Instead, I did the next best thing, I unleashed a strongly worded tweet.

"We need that by, um, Friday"

The first issue here is we have no proof that a deadline forces a team to deliver anything. It might give the illusion that the team beats a deadline, but under the covers somewhere there is an issue. Think back before the salad days of agile when were waterfalling ourselves against massive deadlines, ever move a deadline?

I was on a team doing two week iterations and the second Friday was the end of our iteration. On normal days the team room was emptying out by 4:30 every afternoon, by 5:30 there might be one or two people there. But on every other Thursday it was full until 6:30 or 7 with the last person leaving closer to 9. Why? They had a deadline to beat. Did they get their work in? Usually, yes, but they were hiding the real problem. They were beating their deadline, but they were working hidden OT to do it, taking time away from home, and hiding the real issue from their Product Owner.

I am sure a few people are thinking, "Hey, Tim, they got their work done! Every iteration! They were heros to company!" Those of you thinking that, I say for shame. OK, I do not say for shame, but I do say that they stretched their time box in an effort to appear successful.

An iteration at its core is a time box. It is a fixed amount of time in which we will try to get as much high quality, well tested work done as we can. As a team, this group had agreed to do two week iterations with an iteration day for retrospective and planning. They had agreed to do 72 hours of development and testing, and every other Thursday they were stretching that to 75 or 80 hours. So maybe they did get the work done, but they had broken their agreement - with their Product Owner, and worse, with themselves - to get that work done.

We have all heard of the iron triangle of software development, there are three points of the triangle of time, scope, and quality and only two of those points can be fixed. We like to operate on quality as a fixed point, so pick time or scope and make it variable. An iteration says time is fixed, but all too often it becomes variable because of fixed scope by the end of the iteration.

Use the Time Box

As noted above, the iteration is meant to be a time box. A start and an end to some amount of work. At the end of that time box we hold a retrospective to learn from the previous block of work.

What went well?

What did not go so well?

What went great?

What does the team never want to do again? Ever.

This is the time to learn about your team and make adjustments. If all the work got done with a whole day left in the iteration find out why. Yes be thrilled that all the work got done and your team is awesome, but we still need to learn WHY it happened and WHAT we can do to leverage that in the next iteration.

Now let's say the team did not get their work done a day early. Instead a couple features did not get completed. I am sure we will be asking WHY in this case. Maybe this is the third or fourth iteration in a row some features did not make it to completion. We could say that our team is terrible and they should type faster, but we are seeing a pattern here. Two things jump out at me right away in this situation: Are our stories too large? And are we taking on too many stories per iteration?

If we are taking on too many stories or a few stories that are sized, shall we say, aggressively then start trimming them down a little. Take on less work at the beginning of the iteration in an effort to be successful at the end of the iteration. A team that knocks out its work by the end of the iteration - WITHOUT working until midnight the day before the end of the iteration - will be all smiles at the retrospective.

One bonus to not crushing the team, aside from the obvious good team morale, is you will build some slack in for the team. Any dev team needs a little slack, and not just to shoot nerf darts at each other. Slack allows the team to refactor. Something that was trouble two iterations ago probably needs some attention to keep it malleable for future features.

Slack allows for innovation. Giving the team time to step back and look at features together allows them to find better ways to solve the problem.

Slack allows the team to breath and it keeps them from rushing through problems. The old adage "Haste makes waste" is not more true than in software development. Rush through a solution and you can expect bugs to be opened, leading to more rushing, then bugs, then rushing...wash, rinse, repeat.

I've Read This Somewhere Before...

Agile is about continuously learning, no matter what process you are using. You should be learning more about your team, your product, and your problems every iteration. Every day.

Agile Manifesto line four: "We value responding to change over following a plan."

Iteration deadline: plan or time box?

Katas for Teaching

While recently bringing a team up to speed on Test Driven Development (TDD) in a very short time frame we leveraged code Katas as a teaching tool. The Kata approach is to use a simple code exercise and repeat it a number of times to practice the act of writing code rather than the act of solving the problem. In fact, “getting done” is not usually a requirement as Katas are typically time boxed activities. By repeating the problem you spend less time on the solution and more time on your craft.

We got to spend two weeks going through two different Katas with the team. At the end of those two weeks they had definitely learned some new things and had strengthened their approach to TDD. The team also struggled in some areas at the end of our practice time, some of which was attributable to their teachers. We had more things to work on during these exercises, so we did not get to be full time proctor every day.

Day 0 - Roman Numerals: The other coach and myself did the kata using ping-pong pairing, where one of us would write a test and the other the implementation. We did this on the projector to demonstrate TDD. We stressed doing the simplest possible thing to solve the problem, and that changing code that has already been written is part of the practice.

Day 1 - Roman Numerals: Today the team got to dive into their first kata, and they did group pairing on the projector. Again we stressed the “simplest possible thing” which was illustrated pretty well with a line of return 1 to solve the first step of the Roman Numeral kata where “I” is the input.

Day 2 - Roman Numerals: Again the team did the Roman Numeral kata where they were taking in a string of Roman numerals and returning the correct integer value. Today we switched up the focus a bit and looked much closer at refactoring code rather than getting closer to a solution. We spent enough time on refactoring that we did not solve the first subtraction case of “IV.” The team got a pretty good look at refactoring this way and it also helped us stress that the Katas were not about solving the problem.

Day 3 - Roman Numerals: The third day was essentially a repeat of days one and two. The two of us coaching the team stayed out of the way on this one and let the team direct themselves. They combined the first two days by getting further with the solution, but also doing a couple of refactoring steps to keep the code clean. The biggest change today was we stopped using the projector and group coding, and instead had them split into pairs.

Day 4 - Roman Numerals: Day four saw us throw monkey wrench number one into the works: Flip the Roman Numeral problem and take in an integer and return the correct Roman Numeral string. I had forgotten how much more difficult this approach to the problem can be, and it slowed our teachings quite a bit as we collectively became focused on the problem and not on the practice.

One good thing did come from switching the problem up to something more difficult, it got the team lead to ask the age old question: “But TDD takes longer, why do we do it?” After the team lead got a straight answer of, “Yes it does,” he seemed to really tune in. After explaining that we will happily trade a little time in initial development for a lot of time during product maintenance, the entire team was nodding in agreement.

Day 5 - Roman Numerals: Our last day of doing the Roman numeral kata and we went back to the regular way of doing the kata from days one, two, and three by taking in a string and returning an integer. For today’s fun at the halfway point of the kata we had everybody get up and switch pairing stations. This was a way to show that you will inherit somebody else’s code as a project goes along, and with the team all taking the same TDD approach it will be easy to see where the previous pair had left off and what move your pair could make next.

Day 6 - Game of Life: For our next kata we switched to Conway’s Game of Life (GOL). This is a tried and true kata, and though I thought it was quite a jump in complexity for a simple kata, I was sure we could carve out just enough of it to continue on the path of Red, Green, Refactor that we had started with the Roman numerals in the previous sessions.

The first day of the GOL was definitely focused on solving the problem, but that was understandable as it was a new problem and it needed to be solved once before you could delete it and practice solving it again. The team regressed a little from their solid TDD approach from the previous week. Again, not surprising as teams will fall back to “what they know” when things get a bit more difficult, which in this case was trial and error implementations. It was not a total regression as there were a few unit tests written by each pair.

Day 7 - Game of Life: Day two with the GOL went much better. The problem solving was basically over, so focus went back to TDD and refactoring much more. We had to have a few reminders for “simplest possible thing” again as a couple of pairs were implementing nested loops to simulate the game’s canvas when the instructions were to ignore the canvas and focus on the rules for the cells to be living or dead. Once brought back on task they got back to good TDD practice and implemented the four rules for the cells.

Day 8 - Game of Life: Day three with the GOL and we added the canvas concept to the list of rules and how many cells should be alive on that canvas to start and that the group would progress at least one generation. They were to do the practice from the previous days with the cell rules and expand on them with all the new rules. Yes, we took them from 0 to 60 took quickly...and the exercise showed it.

We had one pair not even implement the rules for a cell, instead just started hammering out loops to build a canvas. Another pair focused completely on getting the output of 0 or 1 correct. A third pair did not throw away the code from the prior day, so they already had the four cell rules implemented. Though that was “cheating” just a little, the bigger issue was that they were not even going to use the already written code.

As the third day of GOL shook out, we got most of the pairs back on track, but it took the white board, some coaches jumping in to pair for a bit, and a lot of “Do you have a test for that?” questions. It was definitely a big jump, but in the end things were again progressing in the right direction.

Some lessons learned from this particular day: 1. We added too many rules too quickly. 2. When you are new to something, and in novice mode in the Dryfuss Learning Model, then you want specific rules to implement. By giving the team new specific rules, they hedged directly towards implementing them and ignored building on the prior days’ code. 3. On the coaching side, we totally missed on building on the previous days’ exercises. 4. Like our change in the Roman Numeral kata on day four, adding these rules changed the exercise once again to a problem solving exercise.

Day 9 - Game of Life: For our last day with this team we lowered the rules back to a more sane level which again allowed them to focus more on the practice and less on the problem. This helped as we saw unit test counts rise with each pair and better focus at completing smaller steps rather than trying to solve the whole thing from the start.

Lessons Learned

Katas are a great way to teach, and the changing pairing stations exercise was a lot of fun. But you have to keep the focus on the practice not on the solution. Keeping the problem simple early will allow more focus on the practice. When switching problems, be sure to take the solution in small steps so the focus can remain on the practice.

Be clear in the rules: Deleting the code before you start. Red, Green, Refactor. Solving the problem is not the main goal. If you have the time and availability, be a full time proctor. Watch for pairs having issues, or the whole group struggling, and try to get them back on task sooner. If a team is new to TDD that will usually mean stressing the simplest possible thing and asking, “Do you have a test for that?” often.

I have been through a couple of code camps where we practiced for a few hours and have done Katas myself at Code and Coffee a number of times. But this was the first time I put them to work as a teaching tool over the span of days and can say I was really happy with the results. Even with the goofs we made, we still got the team a long way with TDD in a very short time.