Recently I have been involved in an “Agile” transformation. I have been forced to look at a number of teams and assess which tools from the Agile toolkit are relevant.
Rather than simply foist Agile goodness on the teams, I’ve been engaged in a discussion about the relevance of the tools… especially from a risk perspective (what a surprise!).
To help people understand Agile I have been casting it in a historical perspective against the (perverted version of) Royce’s Waterfall approach with sign-offs and change control mechanisms. Adopting an iterative development approach mitigates the need for those sign-offs and change control. In addition, the iterations help business investors mitigate a whole bunch of other risks.
Multiple iterations cause some additional issues. There are transaction costs associated with an iteration such as regression testing and releases. Without careful management, some of these transaction costs (regression testing) will increase with each iteration which will start to have a negative impact on the teams ability to deliver new functionality.
The traditional approach is to perform extensive manual regression testing. This increases with each iteration and as a result the economics of the situation lead to larger iterations which breaks the risk management goodness of small iterations.
The agile approach to this problem is to automate testing. To ensure comprehensive and effective coverage, it is necessary to consider testing as a primary concern. Thus test driven development comes to the fore.
The team I work on has an interesting context. The system is not seen outisde of the organisation. It is not used to make payments to external entities. The users do not use the information solely to make decisions. As a result the users are tolerant to bugs… provided they can be fixed quickly. The team can turn around bug fixes within a few minutes to an hour which is acceptable to the business. So this presents a “third way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly (including by roll back which has not yet happened).
It will cost the business investor a lot of time and money to retro-fit effective automated tests. It will take time and effort for the team to develop the skills to learn TDD and build automated tests around the legacy code base. Time and effort that the business investor could spend on other things.
Currently the team are able to produce several releases a week. The business investors are happy with the performance of the team.
Would TDD be the best approach or would the “Third Way” be more appropriate to this context? Would the introduction of TDD break the existing development process from the business investor’s perspective?
Should TDD always be foisted on a team? Or not? Would a team be Agile if they did not do TDD?
Thoughts?
P.S. I am aware that TDD brings many other benefits beyond regression testing.
The IT Risk Manager 
April 11th, 2012 at 3:57 pm
I use TDD as a design tool to make sure the requirements are satisfied in the least amount of code possible, so I think it’s worth it to learn going forward, but going backward just for the sake of having regression tests? Probably not worth it.
April 11th, 2012 at 4:47 pm
Hi Phil
Your thinking is alligned with my own.
My problem is that as a TDD advocate I’ve encountered a situation where I’m questioning its universal applicability.
Thank you
Chris
April 11th, 2012 at 4:58 pm
I’d never say a practice had universal applicability, but I’d say that, in your situation, it has a lot of value moving forward – value that may have little to do with bug fixing or regression testing. In other words, your presented situation is not different in the ways that would make me say TDD wasn’t necessary or useful.
However, it -is- different in ways that would make me question the value of writing a bunch of automated tests after the fact. I’m not sure there is. As time goes on, I see less and less value in coding tests afterward just to have them around.
April 11th, 2012 at 5:08 pm
Phil
Thank you.
You have helped me clarify my thoughts on the matter.
We should not be selling TDD primarily as a regression testing mechanism. Rather we should also promote it as a way of efficiently writing better code going forward. It means I have to explain that bit as well. 😉
Regards
Chris
April 11th, 2012 at 4:09 pm
A fair post. I’d give it a five out of ten.
I like the notion of risk-awareness in software development approaches (see: http://www.fallingblossoms.com/opinion/content?id=1008)
I like the considered tone and the allusions to teams being smart enough (sometimes with a little help or support) to make up their own minds about specific practices, etc.)
I like the open questions.
To make this post a ten, it would have to stress that TDD is much more a design approach than it is about producing a regression suite. If this team eschews TDD, would the quality of their design suffer? By how much? In what regards? Could this be fixed-up later? At what cost? For me, the production of an automated regression suite is a fortuitous by-product of TDD, not the justification for TDD.
P.S. Your P.S. seems an afterthought?
– Bob
April 11th, 2012 at 4:55 pm
Hi Bob
Thank you for the comment.
The P.S. was an after thought. I was focusing in on the regression testing aspects of TDD and realised I had not presented a balanced view of TDD, hence the P.S.
I’ve focused on the automated regression aspect as one of the things that is sold to “management” or the “business investors” to justify investment in an agile transformation.
I would agree that much of the benefit of TDD is its efficient nature, allowing the developer to know when to stop.
Regards
Chris
April 11th, 2012 at 5:16 pm
I think a professional developer should use his discretion and apply TDD for most new software development, in the ideal case. He may not do that if there is already a design provided, there might be a lot of copy-paste of boilerplate or GUI work that might not be suitable for TDD, but he should just do it for the majority of business logic, without even asking for permission. Generally a software developer would also invest a little bit of time maintaining existing code, writing a test if none exists, doing a bit of refactoring etc, as he goes.
I don’t think your third way really is a third way, teams should be fixing bugs quickly (which requires a good design that you get with TDD), AND doing TDD generally for new development, AND writing new unit tests, AND refactoring. In fact if you need to fix bugs quickly and want to continue fixing them quickly then you really need to have good testable, maintainable software with good test coverage, so all those things go together.
There are 3 ways:
1. No TDD, no tests, hack everything quickly, you soon get to an unmaintainable design.
2. Some experts come up with a predefined standard architecture, a big complex framework with lots of features you might need some day, that all new development is supposed to use. “Normal” developers can use wizards to “simplify” generation of new standard components. TDD could be hard in this situation, and I doubt it will result in more maintainable code over time than way 1.
3. Clean code craftsmen allow a good software design to grow via TDD and refactoring. They are all trusted to make small design decisions just-in-time. This results in a clean maintainable minimal code basis.
Now to answer your questions:
“Would TDD be the best approach or would the “Third Way” be more appropriate to this context?”
Well if quality is truly less critical, and design decisions already made mean TDD costs more than it is worth, then I expect a good developer to decide not to use it in many cases. He would probably focus on writing tests and refactoring areas that result in bugs though. Perhaps if it is known that the software will reach end of life within a year it might be sensible to continue hacking away, but some effort needs to go into keeping future maintenance costs down. But don’t worry, a good developer knows when to use TDD and when not to, standardizing on one way or another way doesn’t make sense. Every situation is different, and if it is possible to use TDD it probably should be used.
“Would the introduction of TDD break the existing development process from the business investor’s perspective?” Sure but I see it the other way around. Design decisions already made, could make starting with TDD in the middle of a project difficult, and perhaps not even worth it. Depends on the degree to which new functionality needs to touch old code. There is probably going to be a time investment initially that only pays itself back long term in reduced maintenance costs, and higher quality overall.
“Should TDD always be foisted on a team” No never. It is a developer decision that he can make with the team.
“Would a team be Agile if they did not do TDD?” Yes I don’t think TDD is compulsory. However it is one of the things to look at if a team wants to improve, probably number 2 on my list after continuous integration.
April 11th, 2012 at 5:33 pm
Kurt
Thank you. Great response.
In the situation I’m in I’m reminded of that classic response. “If you want to get there, I would not start from where you are.”
The challenge will be to get a team who are happy with their performance to learn TDD which is a tool they do not feel they need.
Many thanks again.
Chris
April 12th, 2012 at 7:01 am
Most telling phrase: “a team who are happy with their performance”. Nowhere to go from there.
– Bob
April 11th, 2012 at 5:21 pm
The Agile Manifesto doesn’t mandate the use of TDD. It does, however, stipulate (the 12th of the Principles) “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” I’d suggest that introducing some aspects of test automation (perhaps along the lines of Feathers’ “Working Effectively With Legacy Code”) might be an approach?
April 11th, 2012 at 5:35 pm
Mike
Thank you.
Its a case of don’t know what you don’t know.
The team should perhaps use the transformation as a means to add TDD to their tool kit so that it is an option for how they can improve their practices.
Regards
Chris
May 18th, 2012 at 4:17 pm
Watch this, Chris, I think it’s a very good talk. It may give you some insights.
http://www.infoq.com/presentations/Simple-Made-Easy