TDD, do you speak?

Tdd, Kata, Dojo (and polemics)

Context

Thus last month I did some TDD workshops. I prepared thus workshops aggregating all the information I saw, I heard, I experiment, ... and here is the result. This is a subjective point of view shared and discuss with some teams, friends, coworker, ...

What is TDD? What is not TDD?

Two rules, three steps :

• Red (meaning failure in xUnit): you cannot write more test code than needed for the tests to fail.
• Green: you cannot write more production code than needed to make the tests pass.
• Refactor.

TDD is not a way of testing or writing unit tests. It is a process, a way of coding . TDD is made of small cycles, small steps, small victories, ranging from test code to production code. As every other development process, TDD doesn't question the presence of integration testing and manual testing.

What is  refactoring?

Refactoring is the art of modifying code to improve its design without changing its behavior. A refactoring is made of a set of smaller refactorings. When refactoring, one has to ask the following questions:

• Can I remove code smells? (dead code, duplicated code, switch cases, etc.) See: http://www.codinghorror.com/blog/2006/05/code-smells.html
• Can I implement technical design patterns? (pipeline example in FizzBuzz kata)
• Can I implement domain-specific design patterns? A non-developer domain expert must be able to understand it.

Refactoring is a continuous activity that is fully integrated to the development process proposed by TDD.

What are Katas, Dojos?

Historically speaking, katas come from martial arts. Purpose was:

Train, practice, fail, learn...

• Practice gestures, postures... in given situations, in order to get a relaxed but deep training that will allow instictive reflexes in real-world situations.
• Share fundamentals principles.

As a musician who does scales or a martial art practitioner who get trained, we as programmers do katas. There are many katas: http://codingdojo.org/ section Kata catalog (video statement and solution).

A Coding Dojo is a meeting where a bunch of coders get together to work  on a programming challenge. They are there have fun and to engage in deliberate practices in order to improve their skills.

TDD myths and polemics

Here are some of the remarks we often hear around TDD and some response bits:

a.“Tons of useless tests!!!” A verified intrinsic quality 

• A verified application

The all test harness allows to have a verifiable application: on a distributed project, with 20 parallel developments. If  the "red, green, refactor" process is followed up then, any time, everyone can stop coding and within a minute (time to get back to last green state), a version can be delivered (almost … :p)

• A virtuous circle

Did you ever get into that situation? You are reading bad code, you want to refactor it, and ... no, finally: "it stinks, but it works … if I modify my code I’ll never be sure that the behavior will be maintained...". The fear of regression. What does remove that fear? The tests which are a security harness. We refactor to obtain a quality code and we can refactor because we have tests that remove the fear of refactoring. "Red, green, refactor" must become our mantra.

b.« All thus tests! It’s a waste of time ! » I do not have time not to do TDD 

• « Today's productivity is yesterday's quality».[i]

Quality code is possible if code is refactored, code can be refactored if it is tested, non-tested code cannot be refactored: it's a technical debt that will have to be paid (either by writing tests later or by having a high impact on productivity when we add new features).

• Design from the client point of view.

The activity of typing code is not time-consuming. Thinking about the code we want write and designing it is. Writing a test is the formalization of a code design process : a definition of your API [ii]. Writing a test and the associated production code do not take more time than thinking/designing the code and write it. Moreover, it is designing the code from the point of view of the one which will use it and this may be one of the greatest virtue of TDD.

• Less debugging

The biggest advantage of TDD is the time that is not used to debug the software. Tests will influence your design. The code will be naturally modular, functions will be smaller and unit tested. Bug fixing will be fast. The point here is that we are making a fruitful short term investment.

• Written documentation

Tested code is documented code, so there is no time needed to write a technical document explaining how to use the code.

• Test are written

The unit tests are finally done! The (in)famous "write unit tests" post-it is included in the development process.

• No time to NOT do TDD

Finally, when you balance the job done (unit tests are written), the alive code documentation, the fruitful work and the formal design, to "we've got no time to do TDD", I reply "I don't have time NOT to do TDD". :-)

c.« With legacy code, you can't do TDD» With legacy team, you can’t do TDD

This is a real problem and there are real solutions. Many teams have already met and overcome this. Many works (some good, others less) were written on this matter. Some workarounds are validated. Here are some good practices:

•    A code running in production for 6 months is considered stable. So the idea is not to refactor all the existing code and get full test coverage on it.
•    TDD will be introduced when adding a new feature. It will then be mandatory to write tests and refactor.
•    For a bug fix, the very first step will be to write a unit test that highlights the bug.
•    One should focus on a specific code change instead of trying to modify all at once using a legacy backlog for instance.
•    In order to make existing code safe, write a unit test and check it comes out "red" when removing or commenting out the code being tested.
•    Start by writing acceptance test to secure the global behavior.[iii]

Moreover, one mustn't be naive. There are problems when fixing bugs or adding features on legacy code that you can't encounter when developing "from scratch". We may get rid of these by using code isolation techniques in order to make it testable and thus refactorable. "Working effectively with legacy code" illustrates this.

The real problem to be addressed here is more "legacy team" than "legacy code". Teams that still write untested code, mainly because the feedback value still isn't anchored in their minds.

In the end, it all relies on you, the team members, the code surgeons: No one will prevent you from and no one will force you to do it. This is your choice and YES YOU CAN.

d.« Network, file system... With all these external dependencies, I cannot test!» Mock, Fake, … your dependencies

Dependencies are handled by unit tests frameworks. "Mocks", "fakes" or "stubs", which are test patterns, can be used to isolate code that must be unit-tested.
A point of interest is the impact of tests on code design. "Dependency injection" or "factory" patterns required by "mocks" and "fakes" bring low coupling on production code, preventing the spaghetti effect.

Finally what is the added value of test driven development comparing to unit test after coding?

 

Previously, you may have noticed some advantages of TDD over 'After Test Code’. Here are three outstanding:

a. Independence of the test relative to the code[iv]

A test written before production code doesn't check what the code does but what the code should do: by construction, it is not possible to put the obtained result in the expected result.

b. No dead code, no surplus, less bug... less time lost

Incremental construction of the product driven by the unit test involves the creation of code just to meet operational requirements: there will be no surplus or dead code and that means less bug!

c. Intrinsic quality

Refactoring is fully integrated to the development process described by TDD - red, green, refactor - As a true professional, as software craftmanship our goal is to maximize the development return on investment. Building daily quality is the only guarantee of future productivity.

Tdd, how to start?

Implementation is tricky and can become frustrating. We often hear "TDD applied to our daily work... I tried : it's just impossible." Practical implementation should not be made directly on the actual daily work: there is a learning curve. The best solution: every day for 30 minutes chrono do a Kata. Repeat until mastering, then pick another Kata. Repeat until feeling comfortable enough to switch to project code. [v]

So, « modify and pray? Or test and modify? » [vi]

Literature

•    « Test-driven development », by exemple – Kent Beck
•    « Working Effectively with Legacy Code » – Michael C. Feathers
•    « Clean code » – Robert C. Martin
… There are many others books on the topic.

Thanks!!

Alexandre, Colin, Denis, Jean-baptiste, Samuel 

 

Sources

[i] Jean-baptiste Dusseaut, Arpinum, « Le professionnel », blog arpinum, http://www.arpinum.fr/2011/07/19/le-professionnel/
[ii] Colin Garriga, Yaal, www.yaal.fr
[iii] Emmanuel Chenu, Thales Avionics, « Agilité et logiciel critiques », Agile Tour Bordeaux 2010  -http://vimeo.com/19994114
[iv] Gilles Mantel,  « Automatisation des tests, Le mythe du ROI », Mix-It 2012 - http://www.youtube.com/watch?v=hf__zcdihwI
[v] Peter Provost, Kata – the only way to learn TDD, http://www.peterprovost.org/blog/2012/05/02/kata-the-only-way-to-learn-tdd
[vi] Michael C. Feathers, « Working Effectively with Legacy Code »

How we built our taskboard...

...during our sprint retrospective 

(Or how this ... became that!)

That : our brand new board

This : our old board

 

 

Context

The team I'm working with is quite new doing the Scrum. We decided to use this retrospective to ameliorate our scrum process and the team agreed that this time it would be me choosing the topics. During the previous sprint retro we have defined our process of work building a VSM [1]. During this one I wanted to fix the following items:

1.       The technical nature of the daily meeting which needs to get more comprehensible .
2.       The impossibility of knowing who is working on what and if we are on the tracks.
3.       The lack of a shared vision of the product, on what's coming next.
4.       The lack of the priorities set.

I tried to search for the root causes of those problems and finally all my thoughts converged on the task board! The board is a mess... You cannot see anything, everything is confusing... My point is: you can't have clean water coming from a dirty pipe. The board is the mirror of the team daily work. Mirror my sweet mirror...

The aim of the retro workshop was to build a GOOD board. Helping us to have an understandable daily meeting, allowing us to know who is working on what, sharing a vision and the priorities.

The purpose was to have a bottom-up approach, making the team build their own board. By experience this is the better way to have a practice approved and well adopted.

Running the meeting

• Gather knowledge on boards

First we needed to gather some knowledge on boards. What can a board represent , how, and mainly why are we using boards and visual management [1]? I always try to explain why we do the things we do! It's making sure that things will get done effectively. So we spend 30mn gathering knowledge about boards. We used a couple of docs [2].
After this we shared with the team what a GOOD board is, we take as definition of done:
-         The board should represent our process of work.
-         The board should clarify what we wanted to transmit.
-         The board should be build and used collectively.
So we needed as input a representation of our process of work. Cool this was the purpose of our previous retrospective, the way we work [3]!

Our way of work represented with kind of a VSM

• Generate insight : building the GOOD board

We used a workshop created by Xavier Quesada [4]. The purpose is to build a board as you build a project using Scrum. So as a team we made 3 sprints (20mn) with demo and retro (10mn), using a backlog of requirements illustrating classical situations on a project such as:
-         A critical bug was found in production! ASAP action is required by the team. Task board must show this. Name of the issue: "Invoices not being sent".
-         A task needs a large database to be finished. We thought we had enough disk space, but we don't! The team doesn't have the means to solve this on their own. Task is blocked. Task board must show this.
-         ...
I added another constraint: the board had to represent our VSM (previously built) and I put the VSM on the room.

The build process

a. Sprint 01

Objective: The team built a basic board representing the feature, the sprint goal, the process, some DoD.

Sprint review: The team talked about the board; some criticism came up

• How to know in which environment an U.S is deployed?
• How to manage an unplanned task?
• How to manage a correction on done U.S with bug found during Q.A

b. Sprint 02

Objective: address the sprint 01 issues, identify lost time (aka waste), impediments, show the all picture from the preparation. 

c. Sprint 03

Objective: We look for typical situations occurring during our project.

d. Result

Incoming ...

Work In Progress ... part 01

Work In Progress ... part 02 & done

toolkit

Benefits:

• You feel the added value of Scrum building your own task board: the board appears, sprint after sprint handling more and more constraints. After each sprint you challenge your board in a demo finding stuff to ameliorate.
• The bottom up approach: the team builds the board they will use.
• We really appreciate to see our real workflow
• The result is great!

Constraints:

• The definition of done(DoD) we created during the meeting is not clear (It will be the subject of my next retro, and I suggest you do so too; DoD is too important to be treated in the same workshop)
• We only built a draft and I spent one entire day to realize the final board, it took me 4 entire days to prepare, animate and close the workshop
• We appreciate too much to see our real workflow... and we decide to break a principle: moving cards back [5]. Next step should be to rearrange our board to manage this.
• We do not have limits on the board. We need more maturity to do this. Maybe during a future retro.

In a nutshell ...

The improvement of this retro was done: the new board was built. As a team we had some constraints:

1. The technicality of the meeting and the lack of comprehensiveness
2. The impossibility to know who is working on what, where and if we are on the tracks.
3. The lack of a shared vision on what's coming next
4. The absence of setting priorities
   

We respectively tackle with

1. Story focus stand-up: After one week of work with the new board instead of manipulating technical task related to user stories, the team manipulates the stories making them moving from one column to another.
2. We have some good tips like tagging the stories with the environment where they are deployed, or the status (fix me, bloc ...)
3. We have the all story lifecycle on the board, from the idea to the product. The all team share a global vision of the product
4. We have some rules for priorities (top, down) and a golden line for the top priority work to do (prod incident for example)

Closing

We did a ROTI to close the retrospective. The team vote for 4/5 (We gain more time than we spent). The team is using the board for few months now, we added some tools, fixed some stuff. The feedback is great. Some fellows come into the project room and look at the board, talk about the board. Feels good...!

(Re)-learn

-         GOOD Board == represent the process of work, clarify what we wanted to transmit, build and used collectively.
-         Mess on board == mess on the team process // the board is the mirror of the daily work.
-         How to build a board == use Scrum.
-         The board == from idea to product.
-         Actions to do on a task == use tags on your user stories.
-         Top priorities == golden line.
-         Brake technical daily meeting == story focus stand up.
-         Closing a meeting == R.O.T.I

Thank you for reading. Feel free to give me some feedback.
Special thanks to Mister Xavier Quesada for the workshop [4], Colin, Stephane, Jean-Baptiste and Anna for the review!

Sources

[1] Roots cause of visual management: http://www.ted.com/talks/lang/en/tom_wujec_on_3_ways_the_brain_creates_meaning.html

[2] Knowledge material:
-         Support Agile avec Kanban : http://www.fabrice-aimetti.fr/dotclear/public/traductions/Support-Agile-avec-Kanban-FR.pdf
-         Agile Support with Kanban : http://blog.crisp.se/2010/02/17/tomasbjorkholm/1266404160000
-         Kanban boards : http://www.xqa.com.ar/visualmanagement/2009/06/kanban-boards/
-         Comment améliorer votre Kanban : http://blog.soat.fr/2011/12/agile-tour-paris-2011-2/
-         Element for taskboard design : http://www.xqa.com.ar/visualmanagement/elements-of-taskboard-design/
-         Kanban & Scrum:http://www.infoq.com/minibooks/kanban-scrum-minibook

[3] the way we work : retrospective dedicated to build a VSM

[4] http://www.xqa.com.ar/visualmanagement/

[5] Kanban moving cards back: http://blog.brodzinski.com/2010/10/kanban-moving-cards-back.html