I have a bad feeling that this Spolsky post --- which I liked, for what it's worth --- is going to spark a very long, very boring testing vs. shipping controversy, in which both sides, by taking sides, are going to come off grating and sanctimonius.
What I took from Spolsky article wasnt that you shouldn't touch X/Y/Z, but that you should approach new code in the simplest and quickest way you can.
I have often been guilty over engineering things to do them the "right way", to either end up throwing the feature out anyway or having to reengineer because I didnt know enough about the problem to fulfill its design from the outset anyway.
while prototype fast is a reasonably mundane point, Its still one we get wrong time and time again, and I think programmers pay lip service to it as an ideal, but when it comes to actually having to leave that nasty bit of code alone because it does its job, too often we go down the wrong path of making it "right"
I have a friend who develops OpenGL drivers in C for various aerospace uses and his perspective on code quality was very interesting to hear.
As a web-developer I used to think of tests as frivolous wastes of time, whereas he saw them as an essential and necessary part of the development process.
Since I started implementing testing on all my projects I have noticed that my ability to change the code has actually gone up, which in in stark contrast with my initial expectations. I think that having a sounds testing methodology definitely has benefits even in cases where financial/life loss due to software failure is an impossibility...
But the point was that the decision interacts with the market, though. Obviously testing is good and omitting testing completely is bad. But at the end of the day the someone or something needs to pay for the software. Software vendors that ship late due to elaborate design and testing regimes get paid less, in practice, than the "duct tape" folks who beat them to market.
The video drivers you mention are a good example: an aerospace customer is almost certainly operating on a slower schedule, and probably has contracted the revenue already. The value of time to market here is less. And of course the value of code quality in a vehicle safety application is much higher than it is in a consumer web app.
So I'm not surprised at all that sane and smart people would make different testing decisions in the two regimes. They're both right.
When I was a child, my father got the Pinto. even though it is a unsafe car as we knew later. But in a then poor Taiwan, it is like a luxury and it helped him to do his job.
I wrote "the ford pinto memo", not "the ford pinto".
The point wasn't whether the car was okay, it was about the amoral behaviour of the company. Ford crossed a line between wanting to produce something to a given spec; and depraved indifference to the damage the errors in the design would cause. It only sounds like a subtle difference.
Please read the article about the myth. The memo is not specially evil. Its decision is based on NHTSA. The company and the government agency are pretty rational. But it just tells us even we are rational, the complexity of the world is much more unpredictable.
Companies and governments should be amoral but rational. otherwise church and state will not be able to separated from each other. To me, moralistic companies/governments do more harm than welfare to humanity.
This is, indeed, an excellent article and, I would suggest, a must-read for anyone with an analytical mind.
Please do so and form your own understanding before reading the rest of my comment.
What I find fascinating adn distressing is the reliance on potentially meaningless statistics to determine relative safety of cars. (Fortunately, today, one can query quite a bit of data at http://www-fars.nhtsa.dot.gov though there may still be a GIGO).
Specifically, I'm uncomfortable with denominators used in such statistics, as they appear to assume consistent driving patterns within a certain "class" of cars. (Is there a study which shows this?) For example, the denominator of units of a model in operation is deceptive, if one model is driven far more than another, or, perhaps, rather, that one model is overwhelmingly favored by those who drive more less.
I have, admittedly, begged defining what driving "more" or "less" means. It seems that miles traveled is commonly accepted, but overall time spent traveling may be equally, if not more, meaningful. Sadly, this latter information isn't readily available, contributing to the GIGO problem. Even person-mile data are elusive.
Since the "nonmotorist" number of fatalities shown by the FARS front page ("Sub Total2") is around one sixth the total, this could easily mask differences in safety between models, since, presumably, car safety features are, first and foremost, for the protection of the occupants, not nonmotorists (or motorcyclists).
Back to the (sub-)topic at hand, my biggest take-away is that the Pinto was not defective or inherently unsafe, as it did not burst into flames on its own, nor is a cost-benefit analysis (which requires putting a price tag on human injury and death when it comes to safety features) reprehensible. Instead, such an analysis is necessary and beneficial.
The most important concept here is risk. How systems behave "in the wild," especially when human users/operators are part of these systems, is far from a certainty. This is the uncertainty is the basis of the polish vs. ship tension, as well as, apparently, the basis for negligence jurisprudence.
That I consider the whole Pinto debacle to be primarily about imposed versus consented risk on the part of the purchaser might make this whole sub-thread irrelevant to the OP's topic, but I think a contrast would be instructive in any case.
Regardless, I personally do not believe it is up to an engineer to make a risk analysis. Rather, I am a firm believer in more "sunshine" on such information, despite the current practice of graffitying products with warning labels.
Let's take a step back here. Joel's blog entry was about ONE chapter in Codes at Work. If the anti-duct bloggers were to read the entire book, they would see that TDD, C++, templates, pattern fever, OO oration, IDEs--don't get much credibility with the whole lot of them.
But there are self-interest forces at work in arguing against the duct-tape arguments--advocates of patterns, tdd and so on.
Two parties in this argument here have shipped real software in crunch mode. Perhaps others in this argument have not.
"Two parties in this argument here have shipped real software in crunch mode. Perhaps others in this argument have not."
This is a key point. Agile methodology vendors who wax loudest about TDD and other "core practices" have never shipped any significant software [1]. None of them ever built a software product or a startup. Neither do they show us any decent body of code written with their superior approach [2] (vs talk interminable about how our code will be better if we adopt their advice and pay them the big consulting $$).
After all the book is about Coders At Work, not "Methodology Vendors at Work" (which might be an interesting book, in a very horrific fashion).
[1] How many of the creators of the Agile Manifesto have shipped any significant software or built a software company? Most (All?) of them are methodology consultants.
[2] Kent beck has JUnit. Robert Martin has Finesse. The rest of the agile gurus don't have anything. I leave it to the readers to decide if JUnit and Finesse are significantly superior, designwise, to other Open Source code bases, written without TDD.
What a load of tosh. What is this, the "if it's not on the web, it's not real code" party? Show me an embedded system codebase that any of the duct tape people have put together, systems where failures kill people, and tell me you'd be happy depending on those systems, for your chemotherapy or your ABS brakes or your ATM code or your airliner's fly-by-wire system.
Bugs in systems like that will kill people, and we're getting to a state these days where those systems are becoming more and more embedded in day to day life and the risk to Joe Soap is getting higher. How long until a bad UI in a GPS iPhone app turns out to be a contributing factor in a bad car accident? Or until a web-based medical app with a hidden bug screws up a dosing calculation for a patient?
This whole duct-tape-it-and-ship-the-sucker mentality has got to stop, before someone gets badly hurt.
"Show me an embedded system codebase that any of the duct tape people have put together,"
StrawMan.
replace "duct tape programmers" by "TDD/Agile consultants" What do you get? ;-)
"tell me you'd be happy depending on those systems, for your chemotherapy or your ABS brakes or your ATM code or your airliner's fly-by-wire system."
More Strawmen.
Would you be happier if methodology consultants whose only programming experience in the last 10 years was working on a PayRoll System at Chrysler (which failed btw) used TDD to develop your chemotherapy/fly-by-wire/ABS brakes? ;-)
Are you asking if I would be happier if there was testing done on my ABS brakes/chemotherapy/fly-by-wire software than if there wasn't?
Seriously?
You're suggesting there's an alternative?
Or are you saying you base your level of happiness on the experience level of the coders involved rather than their test coverage stats? Because last I checked, experience levels meant very little. The Tacoma Narrows bridge was designed by - at the time - the foremost structural engineers in the world. Their paper on its design was recognised as the single most important paper of its kind in that decade. The bridge still fell, not because they were wrong, but because of something nobody had encountered yet (in this case, aerodynamic flutter).
Critical code is precisely that - critical. You don't leave it to the "gifted engineer" model of development because that always fails. Noone is gifted and diligent 100% of the time, everyone screws up eventually. It's dangerously arrogant to think otherwise. You build a system of work that lowers the probability that those screwups make it into a working system, that's all. Civil and Mech. and other engineering branches have been doing it for decades. We're just getting started on it in this branch of engineering.
You can not write tests that test/discover defects that you don't know as defects. The Tacoma Bridge case proves exactly the above statement. If we all can conceive tests that remove all defects from past, now and future, then we all live perfect lives. But that is impossible, though we can try to minimize the impact. The problem is probably is undecidable in computational complexity sense.
And to some point, we need people, maybe our beloved one, maybe ourselves to be sacrificed in order for scientists and engineers in the next generation to figure out and put them into their test cases to avoid repetition of tragedies.
I think you misunderstand gifted engineers. Gifted engineers understand what they shipped are imperfect, but they have guts to ship them and see how they work. The failure will be lessons for next generation to learn. I feel more scared of people who think their tests have removed all known defects while they forget about unknown defects. They have committed the sin of "confirmation bias" and is much worse to welfare of humanity.
There are part of computer science and engineering dealing with well defined problems. then it is easier to judge solutions to problems by proofs and test cases. Whoever can't prove their code is right or writing test cases are really criminal in their professions.
But there are another part of computer science and engineering dealing with fuzzy requirements or unknown problems. I don't see any better way to explore that area except trials and errors and the scientists and engineers have to face possible charges of "piss-poor", unethical, criminal and evil for the consequences of their solutions.
I feel that you have a very clear dichotomy of moral system. Unfortunately I deal with fuzzy things daily and see the harms of dichotomy in creating new stuffs and burning through money without any fruit of labor. It is easy for you to improve existing technologies because you can define criteria for improvements. But it is so hard in underfunding web startups that you can't expect in advance what a new solutions will be to users unless you make it and test it with them.
If I need someone to preserve and maintain existing software systems, because specifications and use cases are well defined, people like you are well fit for the task. But if someone has a new idea to solve a new problem that no one solved before, I would rather to see an unsophisticated solution first to test against the problem and start from there, and I am ready to rewrite the system when the problem becomes clearer.
I've spent a few years dealing with fuzzy requirement myself. And we tell outselves it hurts noone when a bug happens, but we're being deliberately obtuse in those cases I think. Does it really hurt noone if an online test which comprises a percentage of someones final degree grade, crashes repeatedly while the course tries to take the test? Does it really hurt noone if bad UIs cause people to lose time or money, especially when you add up all that time?
I'm not suffering from an absolutist morality here - I'm pointing out that there are consequences to piss-poor engineering, and it is necessary for engineers to take the viewpoint of those being hurt rather than that of those trying to make money selling the product; because after four hundred years of training engineers, that's the best way we've found of preventing disasters. And the same should be true of developers, because just as civil and mechanical engineering before it, computer and software engineering are increasingly forming an integral part of the infrastructure we live our lives in.
And that means there are tons of improvements to be done and lessons to be learned while you need to make money to support your live and maybe that's why you can exploit those bad products and design better one to kill off them!
You seem to hang up at the concept that it has to be perfect from the first time from your writing.
You don't actually say that no real developers use TDD, but you seem to imply that by focusing on the fact that the teachers, authors, consultants and trainers that promote it "loudest" are earning money via the teaching, authoring, consulting and training in which they loudly promote it. Bit of a catch-22 there, no? Since if their job involves communication about programming topics you automatically ignore the message and you'll only listen to those not trying to communicate.
Luckily, I just finished reading this article by one of the Twisted guys in relation to the recent release of Tornado which, in passing, sings the praises of TDD, basically calling it essential for the work they do:
http://glyph.twistedmatrix.com/2009/09/diesel-case-study-in-...
"You don't actually say that no real developers use TDD, but you seem to imply that by focusing on the fact that the teachers, authors, consultants and trainers that promote it "loudest" are earning money via the teaching, authoring, consulting and training in which they loudly promote it."
No that isn't what I am saying. I am saying many of these people have no programming experience worth a damn. If someone were to tell me how to program(which is what all these methodology sellers do), they would be much more credible if they actually had the programming chops to back it up.
Musashi writing a book on swordsmanship is one thing. If I (who couldn't wield a samurai sword to save his life) write a book that claims my "style" of swordsmanship is better than anything all the others existent today, I shouldn't be surprised if practicing swordsmen don't pay me much heed.
It is the juxtaposition of people with no demonstrated/demonstrable programming skills being loud and obnoxious about the superiority of their advice about programming which I pointed to.
I was also careful to distinguish between Beck and Martin (who at least walk the walk by putting out code developed using the methods they preach) from the other more "all hat no cattle" folks like Jeffries. so to answer your question,
"Bit of a catch-22 there, no? "
No, Not at all. :-). I don't have any problems with people telling me how to program. But I don't trust people who demonstrate convincingly that they know nothing about programming (like Ron Jeffries does with his Sudoku code) who also claim to be "masters" of programming.
That would be like trying to learn fighting skills from a martial arts "master" you saw getting bashed around by a couple of punks at the bar last night, and ended up being taken to hospital and is still covered with bandages and walks with a limp! A master who wins fights would be much more convincing if he claimed he discovered a new martial art which makes existing ones obsolete.
I explicitly said
"None of them ever built a software product or a startup. Neither do they show us any decent body of code written with their superior approach (vs talk interminably about how our code will be better if we adopt their advice and pay them the big consulting $$)."
I stand by this. These guys write books that are the programming equivalent of "Who Moved My Cheese"?. "Coders at Work" otoh, focusses on people who actually are great programmers. Each interview contains more wisdom on programming than reams of "agile" (and these days "lean") textbooks.
FWIW, I did TDD almost exclusively for three years (when I worked at ThoughtWorks), but I don't do TDD anymore, though I still have unit tests.
I don't know about the specific people named in swordmanship but generally in sport the best teachers, coaches and managers will not necessarily be the best at the actual sport. Maybe that doesn't apply in the martial arts though.
And that's at the top level. For ordinary folks I'd suggest that the advice given by a legend may be inspirational, but that pragmatic advice from a standard teacher which is aimed at their level (e.g. "walk away from bar fights if at all possible") may be more appropriate. So I have no problem with methodology suggestions provided by the less than leet.
Also, your reply suggests you unit test, you just don't write the tests first. Which means, compared with the average programmer, you're like Greenpeace complaining about PETA and while you see a big difference the 'average' person just lumps you in together under the general heading of "extremist weirdos".
Fair Enough. You have a right to form your own opinions.
Asking for proof of competence is not "ad hominem", especially in a context where that competence is what is under discussion. But yeah,, whatever.
I think most agile guru can't code for nuts. If you have difference of opinion, show me some great code these agile gurus wrote that in your opinion gives them the "level" to teach us how to code. Or a wildly successful project they managed(No the original XP project at Chrysler that failed doesn't count! ;-)) to the point they can give us the rest of us lessons.
"So I have no problem with methodology suggestions provided by the less than leet."
I don't either. But I do have problems with people who can't put one foot in front of the other masquerading as martial arts teachers. A teacher should know something beyond the students they purport to teach.
Shifting form martial arts to music, If someone doesn't know how to string a guitar and sets himself up as a "master guitarist" , should we all automatically respect him as a maestro just because he calls himself one?
Programming like music depends on quality of the output, not what title is on a visiting card. if someone can't sing "Jingle Bells" in tune, he is hardly a "musician" no matter what he calls himself or how much he blogs/tweets/writes self promoting pamphlets. Likewise for the programmatic equivalent of what (agile "guru" Ron Jeffries's attempts at TDDIng a Sudoku solver are. Please do a web search for "Ron Jeffries Sudoku Norvig" for the gory details).
A master musician is someone who can play very well. A music teacher should play much better than the untrained/untalented, if not at superhuman levels. There is a minimu,m bar of performance where it is criminal to let such a person "teach".
" your reply suggests you unit test, you just don't write the tests first."
I use what works.
But your vague Green peace/PETA analogy doesn't hold. You have your history mixed up.
You imply that anyone who uses unit tests is different from a an Agile Kool Aid drinking, propaganda spouting full time TDDer only as matter of degree. So Unit tests = agile to some degree !
Yeah Right :-).
Unit (and other) tests were used much before the agile weirdos came on the scene and blew it up into some ultimate design method (which I don't believe in - I've stated what I believe in - I think the Agile movement is largely inhabited and sustained by consultants and "scrum masters" and people who write books but no/very bad code.).
Using unit tests is not supporting Agile/XP/whatever. Drawing an occasional UML ish diagram on the whiteboard doesn't mean you support the RUP methodology salesmen.
FYI. I am done with this thread. I have no intention of going back and forth with you on a long and tedious thread. The PlinkPonk Thread Depth Alarm just went off. If you are really interested in discussing this offline, my email is in my profile. Else, well met and have a great day.
The amount of false dichotomies and developer stereotypes Spolsky and Atwood manage to get spewed from ordinarily calm and logical people is astounding.
