Saturday, July 28, 2007
The First Strategy: Declare War on Your Enemies (The Polarity Strategy)

Software is endless battle and conflict, and you cannot develop effectively unless you can identify the enemies of your project. Obstacles are subtle and evasive, sometimes appearing to be strengths and not distractions. You need clarity. Learn to smoke out your obstacles, to spot them by the signs and patterns that reveal hostility and opposition to your success. Then, once you have them in your sights, have your team declare war. As the opposite poles of a magnet create motion, your enemies--your opposites--can fill you with purpose and direction. As people and problems that stand in your way, who represent what you loathe, oppositions to react against, they are a source of energy. Do not be naive: with some problems, there can be no compromise, no middle ground.

In the early 1970s, Margaret Thatcher shook up British politics by refusing to take the style of the politicians before her: where they were smooth and conciliatory, she was confrontational, attacking her opponents directly. She bucked the conventional wisdom, attacking her opponents mercilessly where historically politicians sought to reassure and compromise. Her opponents had no choice but to respond in kind. Thatcher's approach polarized the population, seizing the attention, attracting the undecided and winning a sizable victory.

But now she had to rule, and as she continued down her obstinate, all-in style of "radicalism" politics, she seemed to gain more enemies than any one politician could hold off. Then, in 1982, Argentina attacked the British Falkland Islands in the South Atlantic. Despite the distance--the Falklands were almost diametrically opposite the globe from the British home islands, off the tip of South America--Thatcher never hesitated, dispatching the British armed forces to respond to the incursion with deadly force, and the Argentinians were beaten. Suddenly, her obstinacy and radicalism were seen in a different light: courage, nobility, resolute and confident.

Thatcher, as an outsider (a middle-class woman and a right-wing radical), chose not to try and "fit in" with the crowd, but to stake her territory clearly and loudly. Life as an outsider can be hard, but she knew that if she tried to blend in, she could easily be replaced. Instead, she set herself up at every turn as one woman against an army of men.

As Greene notes,

We live in an era in which people are seldom directly hostile. The rules of engagement--social, political, military--have changed, and so must your notion of the enemy. An up-front enemy is rare now and is actually a blessing. ... Although the world  is more competitive than ever, outward aggression is discouraged, so people have learned to go underground, to attack unpredictably and craftily. ... Understand: the word 'enemy'--from the Latin inimicus, "not a friend"--has been demonized and politicized. Your first task as a strategist is to widen your concept of the enemy, to include in that group those who are working against you, thwarting you, even in subtle ways.

Software projects face much the same kind of problem: there are numerous forces that are at work trying to drag the project down into failure. While agile books love to assume an environment in which the agile methodology is widely accepted and embraced, reality often intrudes in very rude ways. Sometimes management's decision to "go agile" is based not to try and deliver software more successfully, but to simply take the latest "fad" and throw it into the mix, such that when it fails, management can say, "But we followed industry best practices, it clearly can't be management at fault." (This is the same idea behind the statement, "Nobody ever got fired for buying IBM (or Microsoft or Java or ...)." Sometimes the users are not as committed to the project as we might hope, and at times, the users are even contradictory to one another, as each seeks to promote their own agenda within the project.

As a software development lead (or architect, or technical lead, or project manager, or agile coach, or whatever), you need to learn how to spot these enemies to your project, identify them clearly, and make it clear that you see them as an enemy that will not be tolerated. This doesn't mean always treat them openly hostile--sometimes the worst enemies can be turned into the best friends, if you can identify what drives them to the position that they take, and work to meet their needs. Case in point: system administrators frequently find themselves at odds with developers, because the developer seeks (by nature) to change the system, and sysadmins seek (by nature) to keep everything status quo. Recognizing that sysadmins have historically been blindsided by projects that essentially ignored their needs (such as a need to know that the system is still running, or the need to be able to easily add users, change security policies, or diagnose failures quickly at 3AM in the morning) means that we as developers can either treat them as enemies to be overcome, or win them as friends by incorporating their needs into the project. But they are either "for us" or "against us", and not just a group to be ignored.

Other enemies are not to be tolerated at any level: apathy, sloth, or ignorance are all too common among developer teams. Ignorance of how underlying technologies work. Apathy as to the correctness of the code being created. Sloth in the documentation or tests. These are enemies that, given enough time and inattention, will drag the project down into the tar pits of potential failure. They cannot be given any quarter. Face them squarely, with no compromise. Your team, if they hold these qualities, must be shown that there is no tolerance for them. Hold brown-bag lunches once a week to talk about new technologies, and their poential impact on the team or company or project. Conduct code reviews religiously, during active development (rather that at the end as a token gesture), with no eye towards criticizing the author of the code, but the code itself. Demand perfection in the surrounding artifacts of the project: the help files, the user documentation, the graphics used for buttons and backdrops and menus.

Do not wait for the enemies of your project to show themselves, either--actively seek them out and crush them. Take ignorance, for example. Do not just "allow" each of your developers to research new technologies, but demand it of them: have each one give a brown-bag presentation in turn. In a four-man team, this means that each developer will have a month in which to find something new to discover, analyze, discuss and present. They do not have to have all the answers to the technology, and in fact, if the technology is sufficiently large or interesting, they can spend the next month investigating a new element or new use of the technology. Demanding this of your developers means they are forced to educate themselves, and forced to raise their own awareness of the changing world. (Naturally, developers must be given time to do this research; anecdotally, giving them Friday afternoons to do this experimentation and research, when energy and interest in the work week is already typically at an ebb, works well.)

Wherever possible, avoid enemies that are large and hard to pinpoint. Simply saying "We need to have better quality code" is too amorphous and too vague. Developers have nothing to measure against. Personalize your enemies, eyeball to eyeball. Put names to them, make them clearly visible to all involved. "We will have 100% code coverage in our unit tests" is a clearly-defined goal, and anything that prevents that goal from being reached will be clearly visible. "We will not ship code that fails to pass any unit test" is another clear goal, but must be paired with something that avoids the natural "Well, then, we'll not write any unit tests, and we'll have met that goal!" response. Demanding a ratio of unit-test-lines-to-lines ratio is a good start: "We will have three lines of unit test code per line of code" now offers a measurable, identifiable enemy that can be stared in the face. Go so far as to make a ceremony out of it: call the developers into a room, put a poster on a wall, and make your intentions clear. Motivate them. "When we presented the release of the payroll system to the HR department last year, the users called it 'barely acceptable' and 'hard to use'. I refuse to allow that to happen again. The system we build for them this year will be amazing. It will be reliable. It will have those features they need to get their job done (and be specific here), and we will accept no excuse otherwise."

One of the world's most successful software companies, Microsoft is no stranger to the polarity strategy. The company as a whole as declared war on its enemies in a variety of fields, and with few exceptions, has won in almost every conflict. Microsoft actively courts conflict and confrontation. The presence of a well-established competitor in a particular field is no barrier to entry; Microsoft has routinely entered fields with dominant competitors and come out ahead in the end. Witness their entries into the word-processor and spreadsheet markets held at the time by dominant competitors WordPerfect and Lotus 1-2-3, their entry into the video-game console market against well-established competitors Sega and Nintendo, and more recently, the mobile entertainment device market (the Zune) against the iPod. In the latter, the battle has just begun, and the market remains firmly in the hands of the iPod, but let it not be forgotten that Microsoft is not one to retreat quickly from a battle.

Microsoft is also known to do this within their projects; developers who are committed to a project yet seem hesitant or lax in their work are asked if they are really "on board" with this project. The legends of Microsoft developers putting in 80-plus hours a week on a project are somewhat true, but not because Microsoft management expects it of them, but because developers have been willing to put that kind of time into the project in order to succeed.

And Microsoft management itself has declared war on its enemies, time and again, looking to eliminate any and all opposition the successful release of software. Distractions? Every developer gets his own office. Household chores? Microsoft has been known to offer their developers laundry services at work. Computing resources? It's not at all uncommon to walk into a Microsoft developers' office and see multiple CPUs (under desks, in corners, laptops, and so on) and monitors all over the room. Bodily needs? Refrigerators on every floor, stocked with sodas of every variety, water, juices, anything that the thirsty developer could need, all free for the taking. Even fatigue is an enemy: Microsoft buildings have video game consoles, foosball tables, bean bag chairs, and more tools of relaxational activities to help the developer take a break when necessary, so that they can resume the fight refreshed.

Greene notes further,

There are always hostile people and destructive relationships. The only way to break out of a negative dynamic is to confront it. Repressing your anger, avoiding the person threatening you, always looking to conciliate--these common strategies spell ruin. Avoidance of conflict becomes a habit, and you lose the taste for battle. Feeling guilty is pointless; it is not your fault you have enemies. Feeling wronged or victimized is equally futile. In both cases you are looking inward, concentrating on yourself and your feelings. Instead of internalizing a bad situation, externalize it and face your enemy. It is the only way out.

To adapt this to software, instead of simply talking about the hopeless situation in which you find yourself--your company has no interest in agile, your team is just too "inexperienced" to tackle the kinds of projects you are being given, and so on--externalize it. Face the enemy. Your company has no interest in agile? Fine--instead of trying to talk them into it, take the radical approach, do a project in an agile fashion (even without upper management's knowledge if necessary), and show them the results. Can't get the IT budget to allow for a source-control server or continuous integration server? Use your desktop machine instead. Face the enemy, confront it, and defeat it.

Enemies are not evil, and should not be seen as something to avoid. Enemies give us something against which to measure ourselves, to use as a foil against which to better ourselves. They motivate and focus your beliefs. Have a co-worker who refuses to see the benefits of dynamic languages? Avoiding him simply avoids an opportunity for you to learn from him and to better your arguments and approaches. Have a boss who doesn't see what the big deal behind a domain-specific language is? Have conversations on the subject, to understand her reluctance and opposition, and build a small DSL to show her the benefits of doing so. Don't avoid these people, for they offer you opportunities to better yourself and your understanding.

Enemies also give you a standard against which to judge yourself. It took Joe Frazier to make Muhammad Ali a truly great boxer. The samurai of Japan had no guage of their excellence unless they challenged the best swordsmen they could find. For the Indianapolis Colts of last year, each victory was hollow unless they could beat their arch-rivals, the New England Patriots. The bigger the opponent, the greater your reward, even in defeat, for if you lose, you have opportunities to analyze the results and discover how and why you lost, then correct your strategy for the next time. For there will always be a next time.

Don't seek to eschew enemies, either. Enemies give us options and purpose. Julius Caesar identified Pompey as his enemy early in his rise to the throne. Everything he did from then on was measured against Pompey, to put him in a stronger position to meet his enemy. Once he defeated Pompey, however, Caesar lost his way, and eventually fell because he viewed himself as a god and above all other men. Enemies force on you a sense of realism and humility.

Remember, enemies surround you and your project, and sometimes even within your project. Keep a sharp eye out, so that once spotted, they can be identified, analyzed, and handled. Show no quarter to those enemies: they must either join you to help you in your quest to build great software, or be ruthlessly eliminated from your path. They can either benefit from the project, or they can be removed from the battlefield entirely. Some enemies--ignorance, apathy, sloth--are not easily defeated, nor once defeated will they remain so. Never lay down your arms against them or trust your arms to someone else--you are the last line of your own defense.

Development Processes

Saturday, July 28, 2007 2:42:31 PM (Pacific Standard Time, UTC-08:00)
Comments [1]  | 
 Monday, July 23, 2007
The Strange Things That Go On Behind The Scenes

I've been doing a series of video interviews for Pearson (the group behind the publishers Addison-Wesley and Prentice-Hall, among other titles), starting with a series of about a dozen or so we took at the SDWest show back in March. At said show... well... Barbara's blog says it best. (Warning--partial nudity here. Not suitable for work. ;-) )

And yes, it really did happen that way--Bjarne and Herb weren't entirely sure if having a T-shirt emblazoned with "I love C#" on it would go well with their fans, so... *shrug* I reversed it and we went on.

Of course, they *were* probably half-joking, and we *could* probably have done it without a problem... but when you're interviewing one of your childhood heroes (actually, two of them), you don't look to let obstacles stand in the way. :-)

Meanwhile, the entire series is now up on iTunes, under "On Software" (or subscribe), or subscribe to an RSS feed. More are coming, and unfortunately I can't tell you who I was interviewing this week at OSCon, or a few months ago at Microsoft TechEd, but they're all looking pretty good....

Oh, and if you have any suggestions of questions to ask these kinds of folks for these video podcasts, by all means, drop me a line--Pearson's planning a bunch more, and I can always use good ideas for questions....

Monday, July 23, 2007 10:57:12 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  | 
 Saturday, July 14, 2007
Yellow Journalism Meets The Web... again...

For those who aren't familiar with the term, "yellow journalism" was a moniker applied to journalism (newspapers, at the time) articles that were written with little attention to the facts, and maximum attention to gathering attention and selling newspapers. Articles were sensationalist, highly incorrect or unvalidated, seeking to draw at the emotional strings the readers would fear or want pulled. Popular at the turn of the last century, perhaps the most notable example of yellow journalism was the sinking of the Maine, a US battleship that exploded in harbor while visiting Cuba (then, ironically, a very US-friendly place). Papers at the time attributed the explosion to sabotage work by Spain, despite the fact that no cause or proof of sabotage was ever produced, leading the US to declare war on the Spanish, seize several Spanish colonies (including the Phillipines in the Pacific, which would turn out to be important to US Pacific Naval interests during World War Two), and in general pronouce anything Spanish to be "enemies of the state" and all that.

Vaguely reminiscent of Fox News, now that I think of it.

In this case, however, yellow journalism meets the Web in two recent "IT magazine" pieces that have come to my attention: this one, which blasts Sun for not rolling out updates in a more timely fashion to its consumers, despite the many issues that constant update rollouts pose for those same consumers, but more flagrantly, this one, which states that Google researchers have found a vulnerability in the Java Runtime Environment that "threatens the security of all platforms, browsers, and even mobile devices". As if that wasn't enough, check out these "sky-is-falling" quotes:

" 'It’s a pretty significant weakness, which will have a considerable impact if the exploit codes come to fruition quickly. It could affect a lot of organizations and users.'

"... anyone using the Java Runtime Environment or Java Development Kit is at risk.

" 'Delivery of exploits in this manner is attractive to attackers because even though the browser may be fully patched, some people neglect to also patch programs invoked by browsers to render specific types of content.'

"... the bugs threaten pretty much every modern device.

" '... this exploit is browser independent, as long as it invokes a vulnerable Java Runtime Environment.'

"... the problem is compounded by the slim chance of an enterprise patching Java Runtime vulnerabilities.

Now, I have no problems with the media reporting security vulnerabilities; in fact, I encourage it (as any security professional should), because consumers and administrators can only take action to protect against vulnerabilities when we know about them. But here's the thing: nowhere, not one place in the article, describes what the vulnerability actually is. Is this a class verifier problem? Is this a buffer overflow attack? A luring attack? A flaw in the platform security model? A flaw in how Java parses and consumes image formats (a la the infamous "picture attachment attack" that bedevils Outlook every so often)?

No details are given in this article, just fear, uncertainty and doubt. No quote, no vague description of how the vulnerability can be exploited, not even a link to the original report from Google's Security team.

Folks, that is sensationalist journalism at its best. Or worst, if you prefer.

Mr. Tung, who authored the article, should have titled it "The Sky is Falling! The Sky is Falling!" instead. Frankly, if I were Mr. Tung's editor, this drivel would never have been published. If I were given the editor's job tomorrow, I'd thank Mr. Tung for his efforts and send him over to a competitor's publication. Blatant, irresponsible, and reckless.

Now, if you'll excuse me, I'm going to try and find some hard data on this vulnerability. Any vulnerability that can somehow strike across every JVM ever written (according to the article above) must be some kinda doozy. After all, I need to learn how to defend myself before al Qaeda gets hold of this and takes over "pretty much every modern device" and uses them to take over the world, which surely must be next....

Development Processes | Java/J2EE | Reading

Saturday, July 14, 2007 10:07:48 PM (Pacific Standard Time, UTC-08:00)
Comments [4]  | 
 Friday, July 13, 2007
The Strategies of Software Development

At a software conference not too long ago, I was asked what book I was currently reading that I'd recommend, and I responded, "Robert Greene's The 33 Strategies of War". When asked why I'd recommend this, the response was pretty simple: "Because I believe that there's more parallels to what we do in military history than in constructing buildings."

Greene's book is an attempt at a distillation of what all the most successful generals and military leaders throughout history used to make them so successful. A lot of these concepts and ideas are just generally good practices, but a fair amount of them actually apply pretty directly to software development (whether you call it "agile" or not). Consider this excerpt from the Preface, for example:

The war [that exists in the real world] exists on several levels. Most obviously, we have our rivals on the other side. The world has become increasingly competitive and nasty. In politics, business, even the arts, we face opponents who will do almost anything to gain an edge. More troubling and complex, however, are the battles we face with those who are supposedly on our side. There are those who outwardly play the team game, who act very friendly and agreeable, but who sabotage us behind the scenes, ues the group to promote their own agenda. Others, more difficult to spot, play subtle games of passive aggression, offering help that never comes, instilling guilt as a secret weapon. On the surface everything seems peaceful enough, but just below it, it is every man and woman for him- or herself, this dynamic infecting even families and relationships. The culture may deny this reality and promote a gentler picture, but we know it and feel it, in our battle scars.

Without trying to paint a paranoid picture, this "dynamic of war" frequently infects software development teams and organizations; developers vs. management, developers vs. system adminstrators, developers vs. DBAs, even developers vs. architects or developers vs. developers. His book, then, suggests that we need to face this reality and learn how to deal with it:

What we need are not impossible and inhuman ideals of peace and cooperation to live up to, and the confusion that brings us, but rather practical knowledge on how to deal with conflict and the daily battles we face. And this knowledge is not about how to be more forceful in getting what we want or defending ourselves but rather how to be more rational and strategic when it comes to conflict, channeling our aggressive impulses instead of denying or repressing them. If there is an ideal to aim for, it should be that of the strategic warrior, the man or woman who manages difficult situations and people through deft and intelligent maneuver.

... and I want that man or woman heading up my project team.

It may seem incongruous to draw parallels between war and software development, because in war there is an obvious "enemy", an obvious target for our aggression and intentions and strategies and tactics. It turns out, however, that the "enemy" in software development is far more nebulous and amorphous, that of "failure", which can be just as tenacious and subversive. This enemy won't ever try to storm your cubicles and kill you or try to hold you for ransom, but a lot of the strategies that Greene talks about aren't so much about how to kill people, but how to think strategically, which is, to my mind, something we all of us have to do more of.

Consider this, for example; Greene suggests "six fundamental ideals you should aim for in transforming yourself into a strategic warrior in daily life":

  • Look at things as they are, not as your emotions color them. Too often, it's easy to "lose your head" and see the situation in emotional terms, rather than rational ones. "Fear will make you overestimate the enemy and act too defensively"; in other words, fear will cause you to act too conservatively and resist taking the necessary gamble on a technology or idea that will lead to success. "Anger and impatience will draw you into rash actions that will cut off your options"; or, anger and impatience will cause you to act rashly with respect to co-workers (such as DBAs and sysadmins) or technology decisions that may leave you with no clear path forward. "The only remedy is to be aware that the pull of emotion is inevitable, to notice it when it is happening, and to compensate for it."
  • Judge people by their actions. "What people say about themselves [on resumes, in meetings, during conversations] does not matter; people will say anything. Look at what they have done; deeds do not lie." Which means, you have to have a way by which to measure those deeds, meaning you have to have a good "feel" for what's going on in your department--simply listening to reports in meetings is often not enough. "In looking back at a defeat [failed project], you must identify the things you could have done differently. It is your own bad strategies, not the unfair opponent [or management decisions or unhelpful IT department, or whatever], that are to blame for your failures. You are responsible for the good and bad in your life."
  • Depend on your own arms. "... people tend to rely on things that seem simple and easy or that have worked before. ... But true strategy is psychological--a matter of intelligence, not material force. ... But if your mind is armed with the art of war, there is no power that can take that away. In the middle of a crisis, your mind will find its way to the right solution. ... As Sun-tzu says, 'Being unconquerable lies with yourself.' "
  • Worship Athena, not Ares. This one probably doesn't translate directly; Athena was the goddess of war in its form seen in guile, wisdom, and cleverness, whereas Ares was the god of war in its direct and brutal form. Athena always fought with the utmost intelligence and subtlety; Ares fought for the sheer joy of blood. Probably the closest parallel here would be to suggest that we seek subtle solutions, not brute force ones, meaning look for answers that don't require hiring thousands of consultants and developers. But that's a stretch.
  • Elevate yourself above the battlefield. "In war, strategy is the art of commanding the entire miliary operation. Tactics, on the other hand, is the skill of forming up the army for battle [project] itself and dealing with the immediate needs of the battlefield. Most of us in life are tacticians, not strategists." Too many project managers (and team members) never look beyond the immediate project in front of them to consider the wider implications of their actions. "To have the power that only strategy can bring, you must be able to elevate yourself above the battlefield, to focus on your long-term objectives, to craft an entire campaign, to get out of the reactive mode that so many battles in life lock you into. Keeping your overall goals in mind, it becomes much easier to decide when to fight [or accept a job or accept a project] and when to walk away."
  • Spiritualize your warfare. "... the greatest battle is with yourself--your weaknesses, your emotions, your lack of resolution in seeing things through to the end. You must declare unceasing war on yourself. As a warrior in life, you welcome combat and conflict as ways to prove yourself, to better your skills, to gain courage, confidence and experience." That means we should never let fear or doubt stop us from tackling a new challenge (but, similarly, we shouldn't risk others' welfare on wild risks). "You want more challenges, and you invite more war [or projects]."

Granted, it's not a complete 1-to-1 match, but there's a lot that the average developer can learn from the likes of Sun-Tzu, MacArthur, Julies Caesar, Genghis Khan, Miyamoto Musashi, Erwin Rommel, or Carl von Clausewitz.

Just for reference purposes, the original 33 strategies (some of which may not be easy or even possible to adapt) are:

  1. Declare war on your enemies: The Polarity Strategy
  2. Do not fight the last war: The Guerilla-War-of-the-Mind Strategy
  3. Amidst the turmoil of events, do not lose your presence of mind: The Counterbalance Strategy
  4. Create a sense of urgency and desperation: The Death-Ground Strategy
  5. Avoid the snares of groupthink: The Command-and-Control Strategy
  6. Segment your forces: The Controlled-Chaos Strategy
  7. Transform your war into a crusade: Morale Strategies
  8. Pick your battles carefully: The Perfect-Economy Strategy
  9. Turn the Tables: The Counterattack Strategy
  10. Create a threatening presence: Deterrence Strategies
  11. Trade space for time: The Nonengagement Strategy
  12. Lose battles but win the war: Grand Strategy
  13. Know your enemy: The Intelligence Strategy
  14. Overwhelm resistance with speed and suddenness: The Blitzkrieg Strategy
  15. Control the dynamic: Forcing Strategies
  16. Hit them where it hurts: The Center-of-Gravity Strategy
  17. Defeat them in detail: The Divide-and-Conquer Strategy
  18. Expose and attack your opponent's soft flank: The Turning Strategy
  19. Envelop the enemy: The Annihiliation Strategy
  20. Maneuver them into weakness: The Ripening-for-the-sickle Strategy
  21. Negotiate while advancing: The Diplomatic-War Strategy
  22. Know how to end things: The Exit Strategy
  23. Weave a seamless blend of fact and fiction: Misperception Strategies
  24. Take the line of least expectation: The Ordinary Extraordinary Strategy
  25. Occupy the moral high ground: The Righteous Strategy
  26. Deny them targets: The Strategy of the Void
  27. Seem to work for the interests of others while furthering your own: The Alliance Strategy
  28. Give your rivals enough rope to hang themselves: The One-Upmanship Strategy
  29. Take small bites: The Fait Accompli Strategy
  30. Penetrate their minds: Communication Strategies
  31. Destroy them from within: The Inner-Front Strategy
  32. Dominate while seeming to submit: The Passive-Aggression Strategy
  33. Sow uncertainty and panic through acts of terror: The Chain-Reaction Strategy

What I'm planning to do, then, is go through the 33 strategies of war, analogize as necessary/possible, and publishthe results. Hopefully people find it useful, but even if you don't think it's going to help, it'll help me internalize the elements I want to through the process just for my own use. And, in the end, that's the point of "spiritualize your warfare": trying to continuously enhance yourself.

Naturally, I invite comment and debate; in fact, I'd really encourage it, because I'm not going to promise that these are 100%-polished ideas or concepts, at least as how they apply to software. So please, feel free to comment, either publicly on the blog or privately through email, whether you agree or not. (Particularly if you don't agree--the more the idea is tested, the better it stands, or the sooner it gets refactored.)

Conferences | Development Processes | Reading

Friday, July 13, 2007 9:41:02 PM (Pacific Standard Time, UTC-08:00)
Comments [8]  | 
The Korean Conflict... and why SOAP and REST were never a "war"

David Chappelle, a man I greatly respect and admire, recently blogged that

... the war between REST and WS-* is over. The war ended in a truce rather than crushing victory for one side--it's Korea, not World War II. The now-obvious truth is that both technologies have value, and both will be used going forward.

While I agree with his conclusion (that both technologies have a place and can we please just move along here?), I think the analogy is a bit misplaced. I'll get to that in a second.

Elliott Rusty Harold, never one to let a conclusion go by without putting his name on it somewhere, then tries to take David's conclusion and, in his own unique and partisan style, subvert the discussion and David's conclusion entirely:

That’s a nice analogy. Take it one step further though. WS-* is North Korea and REST is South Korea. While REST will go on to become an economic powerhouse with steadily increasing standards of living for all its citizens, WS-* is doomed to sixty+ years of starvation, poverty, tyranny, and defections until it eventually collapses from its own fundamental inadequacies and is absorbed into the more sensible policies of its neighbor to the South.

The analogy isn’t as silly as it sounds either. North Korean/Soviet style “communism” fails because it believes massive central planning works better than the individual decisions of millions of economic actors. WS-* fails because it believes massive central planning works better than the individual decisions of millions of web sites. It’s no coincidence that the WS-* community constantly churns out volume after volume of specification and one tool after another. The WS-* community really believes that developers are too stupid to be allowed to manage themselves. Developers have to be told what to do and kept from getting their grubby little hands all over the network protocols because they can’t be trusted to make the right choices.

By contrast you don’t see a lot of complicated REST frameworks or specifications. You could read all the relevant REST specifications in a slow afternoon (mostly the HTTP spec and a couple of subsidiary RFCs, plus XML and Namespaces in XML. Maybe Atom syntax and Atom-pub too if you feel ambitious.). REST/HTTP sets up a simple economic system based on a few clear rules, and then pretty much gets out of the way to let people do their own thing. It doesn’t even get too upset when people break the rules, and start tunneling everything through POST or deleting items with GET. The main people harmed by such bad decisions will be the sites themselves, and they will be dealt with by the RESTful market.

Let's get a few things straight: as any student of modern political theory will tell you, communism is a political system built around an economic model. The economic model, in many ways, is an idealistic one, somewhat reminiscent of the open source model: everybody produces what they can, contributes it to the whole, and partakes of that collection only as much as they need to in order to meet their needs/requirements. "From each, according to their abilities, to each, according to their needs". In a perfect world, it's a far more sensible system than this capitalistic system in which we find ourselves--it's a system that's predicated on cooperation rather than competition, looking to avoid the extremes of the capitalistic system: no poverty, and no blindingly rich.

The political system, however, is where communist theory breaks down completely. The governing body, as any group invested with absolute power will, in Hobbesian fashion, becomes corrupt and looks to ensure that it receives the lion's share of the benefits. This distorts the entire basis of the system, leaving those who are producing at their fullest potential to wonder, "What's in it for me?" and look to get away with doing as little as possible in order to obtain the maximum possible. In short, the commonly-attributed failure of communism is "human nature".

(Sort of puts a rather dark horizon on the whole open-source community, if you ask me.)

I don't think anyone is going to hold up North Korea as a model Communist society; in fact, of all those governments still following the communist political doctrine (of which the current count, by most sources, is two: Cuba and North Korea), it's fair to say that none of them are exactly winning admirers either among their populations, the political watchdog groups, or the academics. So drawing any kind of analogy in which "communists" are one end of the analogy is almost guaranteed to make your partisaned point of view pretty clear. In fact, politically, the problem with North Korea isn't so much with its economic system as its totalitarian form of government, which is almost entirely orthogonal to its centrally-planned economy; several European states--Sweden being one of them--have high degrees of central planning and they seem to get along quite well with the rest of the world. So let's put the "North Korea is bad because it's communist" argument on the shelf, shall we? It's the fact that (a) they're totalitarian, and more importantly, (b) their leader doesn't like us, that makes them one of the "Axis of Evil" states. Saudi Arabia is also totalitarian, and their populace doesn't exactly rank at the top of most standards-of-living charts, either. In fact, the Saudis (76) rank below Cuba (50) on one measurement of standard of living. (Data drawn from the Human Development Report 2006.)

All that said, it's probably obvious that (a) I don't think the communistic argument here is relevant, and (b) I don't think the WS-* set of agreements are at all communistic or totalitarian. In fact, I'd argue they much more closely follow the model of the European Union, rather than communistic practice at either the economic or political levels.

Look at it this way: the WS-* set of documents intend to provide points of agreed interoperability, not a set of practices that platform developers must follow. Assume you, a Ruby shop, and I, a Java shop, want to integrate systems. Assume we need to ensure that the communication cannot be hijacked, must have certain atomic guarantees, and can't just "disappear" into the ether of the Internet. I don't particularly care how you write your code, and you don't particularly care how I write mine, but in order for us to get our work done between us, we have to agree on how the stuff across the wire will look.

Consider, for a moment, the issue of security: We can certainly agree on using HTTP/S, which requires that we both establish digital certificates that will be validated by code every time the communication is established. (We can't rely on the Web's traditional one-sided certificate approach, because that only verifies that the client knows who the server is; the server has no mathematical guarantees that the client is who they claim.) HTTP/S also means a new shared session key must be exchanged between the principals, meaning that no intermediaries (a fundamental feature of TCP/IP, and of the RESTful architecture itself) are possible, since now the intermediary has to have its own certificate, if it's going to be able to influence the communication somehow. That means now that you and I don't trust each other directly, but a third party, which is a wholly different set of discussions.

WS-Security (or, more appropriately, WS-SecureConversation) addresses this.

On the issue of reliable communication, we can either assume that the TCP/IP "best delivery" semantics are sufficient, but given how many of us have seen the "ghost request", the HTTP request that gets sent from the browser but never gets a response, or the "phantom email" that get sent and never received, most of us are going to be somewhat skeptical about the idea of our bank using TCP/IP for transfers and deposits without some degree of additional reliability requirements. Using REST, then, we build our own ACK system, identifying messages with unique values and requiring recipients to respond when a message is received, just to make sure the message was received.

WS-ReliableMessaging addresses this.

These are not requirements, by the way--you do not have to use these facilities if you are building a SOAP-based service. They are opt-in, if your payload is wrapped in an SOAP envelope (which consists of two additional elements around your POX data, by the way). In fact, if you build your own REST/POX service, chances are you're going to end up re-designing something very similar to the SOAP envelope anyway, when you build in some kind of structure for handling out-of-band data (like authentication information). You're free to use HTTP headers, but they're limited to simple name-value pairs, which can be somewhat constraining at times. (Look at what cookies look like sometime; clearly they could benefit in a big way from some larger structure.)

The key thing here is, when you want these facilities, people have already agreed on how they should look so that you don't have to work out all those details for yourself. In this way, it's something like the Euro: a nation doesn't have to use the Euro as its currency (look at Great Britain), but it benefits from doing so. Or, for a different analogy, the WS-* set of specifications are like NATO: an agreed-upon set of protocols and understandings designed to allow for maximum interoperability during a time of crisis (namely, the presumed invasion of Germany by the Warsaw Pact nations). It didn't mean that each nation wasn't able to purchase whatever arms it desired, or train their soldiers in whatever fashion they desired, it was simply an established logistical structure to help everybody "get along" and work together effectively in the face of the crushing numbers of Soviet, Polish, and East German (among others) tanks and soldiers.

In essence, the goal of REST was to create a loose-as-possible coupling between client and server for the purpose of distributed hypermedia. Fielding's thesis makes this eminently clear from the beginning. Trying to create a generalized distributed communication system was never part of the goal, at least according to the thesis itself. More importantly, there are numerous places where tighter coupling, or greater interaction, or some kind of transactional capability, are necessary. Not in all situations, but as Michael Nygard points out in Release It!, a lot of the integration scenarios that developers face aren't across HTTP boundaries, but across departmental boundaries.

Oh, and it's not just the big vendors who get to propose these SOAP-based standards (though if you want yours to gain traction with the industry as a whole, you're obviously going to have to elicit their support); you're more than welcome/free to use the SOAP Envelope/Header area for your own purposes, just as we do with HTTP headers. Just make sure your tags don't conflict with anybody else's (ah, yes, that's what XML namespaces were for), and party on.

Put that way, the WS-* stack doesn't seem so bad, does it? Clearly the Euro and NATO function a lot better than communism does...

Meanwhile, back to Korea.

It may seem odd to many that I'm criticizing the use of a war as an analogy to Computer Science; after all, I was the one who let the genie out of the bottle in the first place. My issue here isn't with the use of a war as an analogy to what we do (I think there's a lot more analogy to war in what we do than there is to building a house, in fact), but with the particular analogy in question. The SOAP-vs-REST debates helped frame a large part of how we view distributed systems. Korea, "the forgotten war", gave us a legacy of...

... well, nothing.

Apologies to all who served in Korea (as did my uncles), but the basic fact was that Korea really didn't change anything. At the close of World War Two, the Koreans, who'd lost their national independence to Japan in the mid-30's, were eager to re-form their nation and resume governance. Unfortunately, the two superpowers had their own ideas: the Soviet Union (and, more notably, China) refused to consider a united Korea under Western influence, and the US refused to consider a Korea under Communist influence, regardless of what the people themselves might have wanted. (Truthfully, it's hard to tell now what the general populace wanted at the time--sources are pretty biased and partisan and entirely conflicting.) This was how things were left, the Korean peninsula split at the 38th parallel, until 1950.

On June 25th, 1950, the North invaded the South in a surprise attack, pushed the ROK (Republic of Korea) and allied forces back down the peninsula until General MacArthur, the Pacific War hero of World War Two, dared an amphibious landing at Seoul, far to the rear of the front lines, and the Allies were able to push the North's forces back up the peninsula. Then, pursuing a strategy of "total war", MacArthur chose to continue north, pushing the Communist forces all the way back to their border with China. MacArthur, in fact, wanted to push even further, taking the war into China if necessary, to achieve the kind of total victory he'd been able to create in World War Two. President Truman, mindful of the fact that the Soviet Union was presumably willing to go to war globally to prevent the fall of its (presumably) close ally, refused to allow Truman to slip his leash, and ultimately dismissed him, an entirely unpopular move with the US population. (MacArthur was widely supported in the US, and there were very real fears he would stage a coup and remove Truman from office. MacArthur turned out to be more patriot than populist leader, however, and wrote one of the most stirring speeches ever delivered instead.) China then dispatched troops to aid the North, pushing the Allies back to the 38th parallel, where things remained until an armistice (and later a peace) was signed in 1953.

Militarily, nothing much changed. The front lines remained as they are today, fifty years later, and the war did nothing to change the balance of power in the region, or in the world. The US still feared and distrusted Communist China and Communist USSR (who were in fact very dubious allies, though it wasn't known at the time), ultimately leading in part to the Red Scare and the Committee on Un-American Activities, what came to be known as "McCarthyism". For most of the domestic US population, once the troops came home, the desire was simply to "go back to the way things were", and forget that the conflict ever happened.

This is why I disagree with the analogy: for many of us who've participated in this debate, the "REST-vs-SOAP" discussion (which really was more of a "REST-vs-RPC" discussion) helped frame several key concepts: the idea of coarse-grained communication, the desire for loose coupling, the discussion around XML as a lingua franca for data, and so on. Clearly the REST-vs-RPC debate helped frame a great deal of the conversation around distributed systems for decades to come, and in that sense, it makes zero sense whatsoever to analogize that debate as being similar to the Korean War.

Friday, July 13, 2007 7:58:20 PM (Pacific Standard Time, UTC-08:00)
Comments [2]  | 
 Tuesday, July 10, 2007
Shouting out to the Sun JDK team

Those who know me or who've seen me speak know that I don't pull any punches; this is a deliberate stance on my part, as I'm generally way too busy to bother with soft-shoeing around topical areas that might be sensitive to certain groups or teams. I call 'em as I see 'em, and if people don't like the results, I'm always open to being convinced otherwise. (Strong opinionation and high open-mindedness have to go hand-in-hand, if you're going to work to avoid being proven a complete idiot repeatedly in your life.)

That's why I have to give a huge shout out to the Sun build and source-repository engineers who've been working over the last half-year or so (probably longer, but I don't know for certain) to make the OpenJDK project a reality. Where they could have simply tossed the source and build state into a Subversion or CVS respository, washed their hands and said, "There, the source is out there, enjoy", they've instead slowly-and-steadily taken what was a pretty ugly setup and build process and whittled it down to a pretty dirt-simple set of instructions to get a JDK build up and running on your local machine.

If you ever took a moment to pull down the SCSL or JCL sources for JDK 1.2 or 1.3, particularly if you were on a Windows box (as I am), you probably fled screaming from the room (as I did, more than once). The old builds required out-of-date versions of Microsoft Visual C++ (5.0!), and a commercial UNIX-like toolset (MKS, and not even a version you could purchase anywhere, from what I could see). Clearly, the build process in those days was geared specifically around the environment that was existing inside of Sun, and if you weren't a Sun employee with access to those specific versions of those tools, forget it. You still had the source, but...

I pulled down the OpenJDK bits again last night (fresh SVN checkout), and I realized as I was going through the steps to rebrick the build environment that it's been getting steadily simpler and simpler. First, the build tools for Windows now need nothing more complicated than a few tools out of Cygwin and the GNU Make utility (largely because the Windows NMAKE utility is pretty weak compared to GNU's, not to mention the fact that NMAKE doesn't really exist for non-Windows platforms... the SSCLI-built version being the only exception I know). Second, the version of the Microsoft compiler needed has been upgraded to Visual Studio 2003 (not 2005, as building native apps under 2005 took a left turn, as anyone who's ever wrestled with manifests and DLLs can tell you), and a version of the DirectX9 SDK (which is a free download from MSDN). As a matter of fact, if you just want to build the various flavors of Hotspot and not the rt.jar bits from sources, you can even skip GNU make and the DirectX SDK. It's almost turnkey from there.

If you're any kind of plumbing wonk, as I aspire/desire/claim to be, this is a huge step in the right direction, and it's easy to repeat: fire up your Subversion client, point it to the OpenJDK SVN respository, pull down the trunk, and start building. Particularly fun is to build a 'debug' build of Hotspot in order to get the symbols, then build a custom Java launcher, and step through it in the debugger. And I mean, right out of your launcher and into the JDK itself. Or, drop the compiled 'debug' JVM.DLL into your JRE's bin\client or bin\server (or, as I do, create new subdirectories in the bin dir and create some custom -debugclient and -debugserver options, so you can switch back and forth as desired). Or, take the compiled 'fastdebug' build, and run it with the '-Xprintflags' option, in case you needed to be convinced that you really don't know all the -XX options that are available to you...

Kudos, Sun build team. Major, major kudos.

And yes, for those in the .NET space that were wondering, SSCLI Essentials, 2nd Ed, is under way as I write this...

Tuesday, July 10, 2007 6:10:19 PM (Pacific Standard Time, UTC-08:00)
Comments [2]  | 
 Monday, July 9, 2007
Ted and Ted at TheServerSide in Barcelona

It's not often that I meet another "Ted" in the world, much less in my own industry. The last one I knew (besides my great-uncle Ted, who unfortunately passed away a number of years ago) was Ted Rallis, a buddy of mine from my college days who was equal parts philosopher and contemplator-of-navel lint and computer scientist (whom I haven't seen in years, if you're out there Mr. Rallis, drop me a line).

I'd heard rumors of an evil twin brother, though. People claimed that they'd met me at other shows, and while I won't claim to remember everybody I've ever met at a conference (much as I'd like to, my memory just isn't that good), I was pretty convinced that at least a percentage of those folks claiming to have met me before were somehow mistaken. Now I'm sure of it.

While at TSSJS 2007 in Barcelona, I met Ted Goddard, the architect of ICEFaces, and damn if we don't look alike. One of his compatriots snapped a photo of the two of us, and Ted blogged it.

Oh, and by the way? If you're using JSF (which a rising number of people appear to be doing, according to the NFJS polls we take during the speaker panel), you should check out the fruit of Ted's labors. I'm no JSF expert, but people I trust tell me it's pretty good stuff....

Monday, July 9, 2007 8:06:54 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  |