This is not a title I convey lightly, but Michael Nygard's Release It! deserves the honor. It's the first book I've ever seen that addresses the issues of building software that's Production-friendly and sysadmin-approachable. He describes a series of antipatterns describing a variety of software failures, and offers up a series of solutions (patterns, if you will) to building software systems designed to combat said failures.

From the back cover:

Every website project is really an enterprise integration project: the stakes are high and the projects complex. In this world where good marketing can be fatal to yor website, where networks are unreliable, and where astronomically unlikely coincidences happen daily, you need all the help you can get.

...

You're a whiz at development. But 80% of typical project lifecyle cost can occur in production--not in development.

Although Michael's personal experience stems mostly from the Java space, the lessons and stories he offers up are equally relevant to Java, .NET, C++, Ruby, PHP, and any other language or platform you can imagine. Michael Nygard not only knows the Ten Fallacies of Enterprise Development, he breathes them.

Go. Now. Buy. Read. Don't write another line of code until you do.

I'm certain I'm not the first one to blog this, but I wanted to help fan the information out to the world: this email crossed my virtual desk today, and it indicates a subtle shift that many probably didn't see coming:

This morning we announced that later this year, with the final release of Visual Studio 2008, we will make available the source to much of the .NET Framework Libraries under the Microsoft Reference License.  This means that “anyone” who accepts the license will be able to browse and view source code.  The set of libraries initially includes  the Base Class Libraries (System namespace, IO, Text, Collections, CodeDom, Regular Expressions, etc), ASP.NET, WinForms, and WPF .  Microsoft will add to this list as time goes on.

There are two ways people will access the source code:

1) They will download a package with all the source, and then they will be able to install and browse locally

2) VS 2008 integration will enable developers to debug from their own source code into the .NET Framework source code.  We’ll provide symbols for our source on an internet-accessible source-server; to enable this experience, the developer needs to set up the URI for the server.  This second option is really cool, and enables a kick-ass developer experience.

We’re announcing this via Scott Guthrie’s blog, with screenshots of the VS experience.

Distributed with the email was a Q&A document, which (in the interests of minimizing the accusations of FUD on my part) I will paste in its entirety:

Reference Source Code Availability for the .NET Framework

--Microsoft Confidential until October 3^rd, 9:00am Pacific time--

Key Points

· Microsoft is releasing the source code for .NET Framework libraries under the Microsoft Reference License.  This license allows viewing of source code, but not modification or redistribution. The source code will be downloadable and viewable by anyone who accepts the license agreement. 

· Microsoft will introduce a capability in Visual Studio 2008 to allow .NET developers who are debugging applications, to debug not only into their own source code, but also into .NET Framework source code using Visual Studio. 

· This release falls under Microsoft’s Shared Source Initiative, which encompasses a spectrum of source code offerings, complementing the company’s other activities around sharing source code.  This is another example of Microsoft’s continued commitment to increasing transparency and addressing developer needs. 

Q&A

Q. What are you announcing?

Microsoft is releasing the source code for .NET Framework base class libraries under the Microsoft Reference License, which is available here: http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx .  This license allows viewing of source code, but not copying or re-compiling. The source code will be downloadable and viewable by anyone who accepts the license agreement. 

In addition, Microsoft will introduce a capability in Visual Studio 2008 to allow .NET developers who are debugging applications, to debug not only into their own source code, but also debug into the .NET Framework source code using Visual Studio.  With this capability, when developers are stepping through code, if they wish, they will be able to step into the source code for the .NET base class libraries.

Q. When will this become available to developers?

We expect the debugging capability to be available to .NET developers who use the RTM version of Visual Studio 2008, when Visual Studio 2008 releases. Our current plan is to release Visual Studio 2008 before the end of the 2007.

Developers will be able to download the source code package for the .NET libraries around the same time, or soon after.

Q: Which libraries are included in the source-code release?

This release will include the Base Class Libraries (BCL), Windows Forms, ASP.NET, System.Data, and WPF. BCL includes many of the basic classes in the framework including collections, string and text handling, IO,serialization, remoting, and others. We plan to include additional libraries into the set as time goes on.

Q: Why are you releasing only a subset of the .NET Framework libraries under the Microsoft Reference License?

Our intention is to make .NET Framework library source code available broadly, in response to customer demand. Before release under the Microsoft Reference License, each library is subject to a code review. As time allows, we will complete additional code reviews and release additional modules under this license.

Q: Will the libraries distributed under the Microsoft Reference License be limited to “managed code” libraries – in other words, .NET Framework libraries?

Microsoft’s intention in releasing developer libraries under the Microsoft Reference License (Ms-RL) is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code. Microsoft may release additional libraries under the license, potentially including unmanaged libraries, and these additional libraries would also deliver the same debugging experience described previously. Microsoft has no announcement at this time regarding additional libraries being released under the Microsoft Reference License.

Q: How does this relate to CodePlex, and other source-code sharing programs offered by Microsoft?

Microsoft has an initiative called the Shared Source Initiative which encompasses a spectrum of source code offerings.  At one end of the spectrum, Microsoft shares source code for commercial products like Windows and Office through programs offered to governments, enterprises, MVPs, OEMs, SIs and others.  On the other end of the spectrum, Microsoft encourages open source development through the CodePlex project management site, as well as direct contribution of source code to community projects on CodePlex.  Today, we’re announcing a new offering: the source code for the .NET Framework libraries will be available under the Microsoft Reference License. This offering will enable developers to view and even debug into the source code for the .NET Framework libraries.

Q: Allowing anyone to view the source code of a library – does this present a security risk?

The security of the .NET Framework does not depend on the obscurity of the .NET Framework source code. A realistic threat model must assume that any bad actor who wishes to subvert the security of any library, for practical purposes, has full access to the source code. This release will not change that.

Q: What do the developers need in order to take advantage of the debugging capability?

The debugging capability will be available to developers who use the RTM version of the Professional or Standard editions of Visual Studio 2008. Microsoft will also publish instructions for how to enable other .NET development or debugging tools to take advantage of the same capability. The Express versions of Visual Studio will not include this capability.

Q: What do the developers need in order to simply view the .NET source code?

Developers will need to download the source code package, and accept the license agreement before unpacking the archive. At that point, the source files will be viewable in any text editor.

Q. Why is Microsoft making this available now?

This is another step along the path of increasing transparency and addressing customer needs. Generally speaking, we are always looking for ways to continue to improve and enhance the developer experience for people who build applications using Microsoft tools. Delivering this particular capability for our customers requires changes in the developer tool itself, and the introduction of Visual Studio 2008 gives us the opportunity to do that.

Q. Why didn’t you do this sooner?

Our primary goal was to enable developers to build better applications. A new capability like this requires changes in the developer tool, and the release of Visual Studio 2008 presents us with an opportunity to deliver the right experience for developers. Maintaining the right developer experience, while preserving the security stability and serviceability of the .NET Framework, were paramount requirements for us. We believe we’ll have a simple and easy developer experience that will empower developers to build better more reliable applications. Having said this, the debugging experience is very transparent, and developers won’t even notice the change.

Q. Can developers modify and rebuild the .NET Framework source code, and redistribute the results of that action?

The source code of the developer libraries being made available, including the libraries in the .NET Framework, will be released under the Microsoft Reference License. http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx

The Microsoft Reference License allows viewing of source code for reference purposes, but does not allow editing, copying, or rebuilding. Microsoft’s intention in releasing developer libraries under this license is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code.

Q. How is the license accepted?

The license is accepted differently depending on how the person accesses the source. If it is through the VS2008 experience, there is a popup “do you accept this license?” dialog. If it is through the source package, then the MSI installer will require the person to accept the license before installing.

Q. Is this an open-source release?

Microsoft believes that a consistent framework for releasing source code – one the community can rely on – delivers the best value for customers and minimizes confusion. This is why use a set of source licenses, with clearly differentiated purposes, for source releases. (See http://www.microsoft.com/resources/sharedsource/licensingbasics/sharedsourcelicenses.mspx ) This particular release is being offered under the Microsoft Reference License.

Q. {Java,Ruby, Python} is truly open source and lets me rebuild the source code. Why can I not do this with .NET?

We’re focused on delivering value to developers. With the .NET Framework library source code, our goal is to enable developers to view the source of the platform, learn from that, and potentially provide feedback into Microsoft. (This would commonly be done via the Product Feedback center Microsoft has used for years – see http://connect.microsoft.com/feedback/default.aspx?SiteID=210 ) The Visual Studio debugging experience extends that view capability to a debugging session. With this capability, we think we’re delivering an easy, seamless experience for developers, and good value to the community.

Allowing developers to rebuild a framework, any framework, from source, and then redistribute the modified result, can introduce problems with providing support, serviceability, integrity and security of the framework itself. This is true with .NET, Java, Ruby, and other frameworks. Multiple independent redistributed modified versions of a framework can decrease the reliability and dependability of the common platform, which is not desirable.

In the path we’ve taken here, we are seeking to balance the requirements we hear from the developer community for transparency and of reliability and dependability of the platform. In striking that balance, we believe the Microsoft Reference License is the right license for this release.

Q. How does the debugging experience really work? Will the source code remain resident on the machine?

Visual Studio will request source from a Microsoft (MSDN) server as needed when a developer steps into .NET Framework library code during in a debugging session. In more detail, the debugger queries the remote server for the version of symbols that matches the binaries used in the application being debugged. If the debugger finds the correct version of symbols, it downloads the source files on-demand as the developer debugs into them. In all cases, the first download of source code requires explicit acceptance of the license. This is the same flow works for all source code that we will add in the future to this program, whether managed, unmanaged, 32 bit, 64 bit, desktop, mobile, and so on.

Q. What if a company does not want to allow the .NET Framework source code or other library source code, to be viewed by its developers?

Viewing or debugging into the Microsoft library source code is an action that will require an explicit acceptance of the license, regardless of the mechanism by which developers view the code (whether in a developer tool like Visual Studio, or as a separate archive download). Companies that do not wish to view the source code should instruct their employees to not accept this license.

Q. What are the terms of support with this source code? What if the developer finds a bug in the code?

The license spells out what the developer should expect. Essentially, people can look at the source code, but they may not copy it. If a person finds what they believe to be a bug in the code, Microsoft would encourage the person to submit feedback via the product feedback center, see http://connect.microsoft.com/feedback/default.aspx?SiteID=210 .

As you can well see, there's a lot of opportunities for FUD here--the Q&A document makes it clear that the source license is a "read-only" license, in that you can't recompile the code yourself. However, for those who are hell-bent on doing so, two options are available to you:

1. Build the code without telling Microsoft. This is not a trivial undertaking, and I'd be shocked if it turns out to be as simple as "create a solution, import the files, press the big green button". It may not turn out to be worth the effort.
2. Download Rotor Whidbey, also known as SSCLI 2.0, and build that, instead. It won't have ASP.NET, WinForms, or WPF, but it will have much of the "core" FCL, and the core CLR (modulo GC and JIT, which Microsoft re-wrote for Rotor v1) to boot.

Speaking of the latter, Joel Pobar and I are doing a revision of the SSCLI Essentials book to bring it up to date with the Rotor Whidbey sources; I'm doing an editing pass on the chapters now, and we'll be hunting up reviewers to take a look probably before the end of the month. (If you're interested, you know how to get a hold of me.)

Some additional comments to go along with the Q&A from above:

• "Allowing anyone to view the source code of a library – does this present a security risk? The security of the .NET Framework does not depend on the obscurity of the .NET Framework source code. A realistic threat model must assume that any bad actor who wishes to subvert the security of any library, for practical purposes, has full access to the source code." Amen, brother! Sing it loud, sing it proud: obscurity is not security!
• "Why is Microsoft making this available now? This is another step along the path of increasing transparency and addressing customer needs. Generally speaking, we are always looking for ways to continue to improve and enhance the developer experience for people who build applications using Microsoft tools. Delivering this particular capability for our customers requires changes in the developer tool itself, and the introduction of Visual Studio 2008 gives us the opportunity to do that." There's a small amount of shuffle-and-dance going on here; to allow developers to debug .NET in the manner Microsoft describes above--Visual Studio fetching symbols and/or source from MSDN when the developer steps-in to the source--did, probably, require a tool change. Just using the existing facility to talk to a symbol server over the Internet would most likely not be a great experience, though I'd defer to John Robbins on that one. But nothing prevented them from making sources available in the traditional (a la SSCLI) manner (a big tarball) long before this. Instead, I think what we're seeing is something Stu Halloway predicted three years ago at a No Fluff Just Stuff speaker panel: "In five years, Microsoft will be the biggest open-source company on the planet". I think Microsoft just needed to figure out how it wanted to embrace open source, because at the end of the day, Microsoft needs to make money, and it wasn't sure how to do open source and make money at the same time--after all, the "golden bailout" that both Red Hat and JBoss both used won't be available to Microsoft.
• "Can developers modify and rebuild the .NET Framework source code, and redistribute the results of that action? The source code of the developer libraries being made available, including the libraries in the .NET Framework, will be released under the Microsoft Reference License. http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx

  The Microsoft Reference License allows viewing of source code for reference purposes, but does not allow editing, copying, or rebuilding. Microsoft’s intention in releasing developer libraries under this license is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code." In other words, no, you can't make a forked copy of the library and ship it yourself. At least, not for now. Mark my words, though, Microsoft will pay very close attention to this issue, and if customers find themselves wanting/needing to do so, Microsoft will find a way to make this work. It just may not be in the same manner that traditional open-source projects would do so. (Remember, Microsoft is very aware of their bottom line, and they need to tread lightly anywhere the goals of "making money" might possibly be threatened or thrown for a loop. They're not risk-avoidant, but they aren't going to take risks unnecessarily, either.)

• "{Java,Ruby, Python} is truly open source and lets me rebuild the source code. Why can I not do this with .NET?" This is an interesting and tricky subject. On the one hand, allowing developers "full reign" over the source provides the greatest amount of power and flexibility, and opens up the possibility of customizing the .NET Framework to behave exactly the way you want it to. On the other hand, being able to do that also means that the versioning problem just took on a whole new order of magnitude of complexity, because now you have some serious problems regarding the FCL assemblies. Remember, the FCL assemblies are signed with Microsoft's private key, so either Microsoft has to release that in order to allow you to rebuild the assemblies to match up with the ones already built by Microsoft (and we already know that releasing a private key is a Bad Thing, right?), or your custom-built FCL assemblies would be built with your own private key, which means now that your library is only used when code is compiled directly against it--and what happens when you build your application with a third-party library which itself was compiled against Microsoft's code? Trust me on this--the resulting picture is not pretty. Don't imagine that you have answers to this problem, either: it's a tricky problem, one that no programming platform has solved in a transparent and universal manner. Where a solution might work for your particular situation, be assured that it'll fail for somebody else's.

 The one thing I wish they'd do, and I can't tell from the announcement or the Q&A document if they will, is release the sources in a single tarball for download and offline use; I don't want to have to be tethered to an Internet connection all the time while debugging code. (Remember that the first Fallacy, "The network is always available", doesn't apply at 39,000 feet.)

All in all, this makes the latest marker in Microsoft's slow-but-steady shift towards a more open-source-friendly position. Whether or not the developer community meets them halfway, of course, resides entirely with your point of view on the statement, "Microsoft is an evil empire that does nothing good for anybody but themselves". If you agree with that statement, you and the other Microsoft Conspiracy Theorists will probably conclude that Lee Harvey Oswald was not only not alone, but he was paid by Microsoft. If you dsiagree with that statement, then you and the other Microsoft-sponsored Microsoft Certified Partners, Microsoft-sponsored Regional Directors, Microsoft-sponsored Most Valuable Professionals, and Microsoft-sponsored development teams will be happy to note that the checks still cash despite this move to open source. (And yes, there's likely to be a few of you out there who are somewhere in the middle on this, and you will probably get killed in the crossfire between those two rival gangs as a result of all this.)

I just got back from the No Fluff Just Stuff show in St Louis, where I gave my "Why the Next Five Years Will Be About Programming Languages" keynote, and a fellow speaker emailed me to point out the Code To Joy blog, which says some things that were... um... well, rather than try and select an adjective, I'll let you look for yourself:

Ted Neward talked about how the next 5 years will be about languages. (Fellow speaker Alex Miller has a post which contains a link to a similar talk and some of his own commentary).

Ted's thesis was that the Big Runtimes (the JVM and CLR) act as a bridge between the academic language lawyers and the problem solvers in the field: the academics can experiment with language syntax and exotic semantics; the problem solvers can use these languages and rely on the venerable runtime libraries to get things done.

I enjoyed the talk thoroughly. Ted paced to-and-fro like a giant cat: if NFJS is Narnia, then Ted is Aslan, except very hungry, and with a mild-case of rabies. Many good jabs toward, well, everybody. He's an equal-opportunity offender (and a big softie underneath it all.)

A giant cat. A very hungry giant cat. A very hungry giant cat with a mild case of rabies.

*sniff* It's like I can retire now. *sniff* I'd like to thank the Academy, and my parents, and ...

I get e-mail from blog readers, and this one--literally--stopped me in my tracks as I was reading. Rather than interpret, I'll just quote (with permission) the e-mail and respond afterwards

Hi Ted,

I had a job interview last Friday which I wanted to share with you. It was for a “Solutions Architect” role with a large Airline here in New Zealand. I had a preliminary interview with the head Architect which went extremely well, and I was called in a few days later for an interview with the other three guys on the Architecture team.

The second interview started off with the usual pleasantries, and then the technical grilling began with:

“What are the best practices that you would put in place on a project?”

I replied “I’ve come to realise that there is no such thing as ’Best Practice’ in architecture – everything is contextual.”

Well, it went down like a sh*t sandwich! The young German guy who asked the question looked at me like some sort of heretic and said – “ I disagree”. I thought to myself, no damn it – I’m going to push it to see where this guys argument goes, so I said “I can take any best practice you can think of, and by changing the context, I can render that best practice a worst practice”. He didn’t like that very much, but I thought, ‘bugger him, I know I’m right’.

He then came out with the next question “What do you do when your architectural principles are compromised”. I asked “what do you mean”, and he indicated that an application he designed recently, was found to be far too expensive to implement So he asked again ‘what would you do’?

I replied “redesign it”. He scoffed at this answer, and reiterated “but what if the redesign was compromising your architectural principles”? So I asked “what is more important. Your principles, or achieving a business objective?”. I don’t remember exactly what his answer was, but it was along the lines of “you have to maintain a corporate standard”.

His next attack was at extreme programming. Having seen that I had used extreme programming one of my recent projects (in addition to also using waterfall, more recently, on others), he asked “don’t you think that the very risky nature of extreme programming is at odds with it’s ability to deliver software consistently?”. This was a bit of a stunner. I indicated that, once again, it was contextual. XP is appropriate on some projects, but it is not on others. On the XP project I worked on, it was entirely appropriate. We delivered early, within budget, the client got what he wanted, and we got a few million dollars worth of work out of it. Not surprisingly, he didn’t have a lot to say to me after that.

Having worked as a consultant for a number of years now, I have been entirely focused on adding business value. I was stunned to hear first hand, how divorced from the business this “architect” was. Clearly he has to maintain some sort of structure with their corporate systems, but surely each business solution should be assessed primarily in the context of it’s own business objectives.

The interview was good in the respect that I was able to quickly establish that it wasn’t the place for me, but it did leave me with some unanswered questions:

1. How could their idea of an architect (being the policemen of corporate best practice) be so far removed from someone like myself, who aims to make case by case judgements based on pragmatism and experience?
2. Is architecture supposed to be facilitative or restrictive?
3. What relevance do architects have today? Are they just overpaid, out of touch developers?

Regards,

Shane Paterson

Hands on Architect Type

(for lack of a more relevant title)

Wow.

For starters, Shane, kudos to you for sticking to your guns, and for figuring out really quickly that this was clearly not a place you wanted to work--a lot of developers have a mentality that says that they need the company more than the company needs them, sort of a "job at any price" mindset. Interviews aren't supposed to be the place where candidates grovel and say whatever the company wants them to hear--an interview is supposed to be a vetting process for both sides.

But on to your questions:

How could their idea of an architect be so far removed from someone like myself? I can't answer this one solidly, but I can say that the definition of an architect seems to be vague and indiscriminate a term, only exceeded in opacity by the term "software" itself. For some companies I've worked for, the "architect" was as you describe yourself, someone whose hands were dirty with code, acting as technical lead, developer, sometimes-project-manager, and always focused on customer/business value as well as technical details. At other places, the architect (or "architect team") was a group of developers who had to be promoted (usually due to longevity) with no clear promotion path available to them other than management. This "architect team" then lays down "corporate standards", usually based on "industry standards", with little to no feedback as to the applicability of their standards to the problems faced by the developers underneath them. A friend of mine on the NFJS tour, Brian Sletten, tells a story of how he consulted on a project, implementing the (powerful) 1060 Netkernel toolkit at the core of the system, to resounding success. Then, on deployment, the "architecture team" took a look, pronounced the system to be incompatible with their "official standards", and forced new development of a working product. In other words, the fact that it worked (and could easily be turned to interoperate with their SOAP-based standard, of which there were zero existing services) was in no way going to stand as an impediment to their enforcement of the corporate standard.

Is architecture supposed to be facilitative or restrictive? Ah, this is a harder one to answer. In essence, both. Now, before the crowd starts getting out their torches and pitchforks to have a good old-fashioned lynching, hear me out.

Architecture is intended to be facilitative, of course, in that a good architecture should enable developers to build applications quickly and easily, without having to spend significant amounts of time re-inventing similar infrastructure across multiple projects. A good architecture will also facilitate interoperability across applications, ensure a good code quality, ensure good maintainability, provide for future extensibility, and so on. All of this, I would argue, falls under the heading of "facilitation".

But an architecture is also intended to be restrictive, in that it should channel software developers in a direction that leads to all of these successes, and away from potential decisions that would lead to prolems later. In other words, as Microsoft's CLR architect Rico Mariani put it, a good architecture should enable developers to "fall into the pit of success", where if you just (to quote the proverbial surfer) "go with the flow", you make decisions that lead to all of those good qualities we just discussed.

This is asking a lot of an architecture, granted. But that's the ideal.

What relevance do architects have today? Well, this is a dangerous question, in that you're asking it of one who considers himself an architect and technologist, so take this with the usual grain of salt. Are we just overpaid out-of-touch developers? God, I hope not. Fowler talks about architecture being irrelevant in an agile project, but I disagree with that notion pretty fundamentally: an architect is the captain of the ship, making the decisions that cross multiple areas of concern (navigation, engineering, and so on), taking final responsibility for the overall health of the ship and its crew (project and its members), able to step into any station to perform those duties as the need arises (write code for any part of the project should they lose a member). He has to be familiar with the problem domain, the technology involved, and keep an eye out on new technologies that might make the project easier or answer new customers' feature requests.

And if anybody stands up at this point and says, "Hey, wait a minute, that's a pretty tall order for anybody to fill!", then you start to get an idea of why architects do, frequently, get paid more than developers do. Having to know the business, the technology at a high and low level of detail, keeping your hands in the code, and watching the horizon for new developments in industry, is a pretty good way to burn out any free time you might have thought you'd have.

Granted, all of these answers notwithstanding, there's a large number of "architects" out there whose principal goal is to simply remain employed. To do that, they cite "best practices" established by "industry experts" as a cover for making decisions of their own, because nobody ever gets fired for choosing what industry "best practices" dictate. That's partly why I hate that term: it's a cop-out. It's basically relying on articles on popular websites and magazines to do your thinking for you. Inevitably, when somebody at a conference says the word, "Best Practice", listeners' minds turn off, their pens turn on, and they dutifully enscribe this bit of knowledge into their projects at home, without considering the applicability to their project or corporate culture. Nothing, not a single technology, not a single development methodology, not even a single tool, is always the right answer.

In the end, I think what Shane ran into was an "architect" with an agenda and an alpha-geek complex. He refused to consider somebody with a competing point of view, because God forbid somebody show him not to be the expert he's hoodwinked everybody else at work to think he is. Unfortunately I've run across this phenomenon too often to call it statistical error, and the only thing you can do is to do exactly what you did, Shane: get the hell out of Dodge.

Well, those of you who've seen me at the various No Fluff Just Stuff shows probably already knew this was coming, but today I finally made the switch, bought a Mac, and started the (rather scary) experiment of converting my IT life over to a brand new operating system, something I haven't done in close to two decades. (Not since I bought my first 286 PC, in fact, back in high school.) I am now the new owner of a 17" MacBookPro (4GB, 1900x1200 matte, for those who want such details), and I'm slowly starting the transition.

Thus far, the Mac experience isn't all that exciting--I unwrapped the box, plugged the thing into the wall outlet (hey, now there's a switch--Macs come with pre-powered batteries, it seems), booted and walked through the opening steps of Mac OS X. (Don't ask me which version it is--I honestly don't know yet, nor do I know how to find out. I will, in time, but for now...) Vaguely reminiscent of Vista's installation (or maybe that should be the other way around), if a touch smoother.

Next steps, install iWork, so I can see if it's the Office-killer that some of my Mac-zealot friends say it is, install Firefox (because Safari just doesn't work for me, not even for the fifteen minutes I spent using it), and purchase VMWare Fusion. This is the secret to my $4000 experiment--I don't have to switch over completely in one day, because I have everything I do bundled up in VMWare images (one for Java work, one for .NET 2005 work, one for NetFX 3 work, one for playing-around-with-new-programming-languages work, and so on), so as long as VMWare images are cross-compatible between Intel Macs and Intel PCs, I'm good.

Unfortunately, this isn't going to be quite as smooth as I'd hoped; I'd forgotten that the Mac OS and Windows don't exactly share the same filesystem--Windows prefers NTFS (and has for years now), but apparently the Mac OS doesn't have write capabilities to NTFS. This presents a small problem, as my external 160GB USB drives are all NTFS-formatted. So I can suck the images off the drives, but can't write to them. Ugh. Possible salvation lies in the Ext2FS filesystem, which apparently is supported (through open-source efforts) on both Mac OS and Windows. I'll pick up a new 160GB drive, format it Ext2FS, and see how well it works on both machines.

(Curious readers may wonder why I need a filesystem that's writable in both directions; frankly, I need the full-fidelity because I'm not entirely convinced of the sanity of this move just yet, and I want to be able to go back if I need or want to. In fact, I'm going to be carrying the trusty T42p around with me, snuggled right up against the MBP, for a few months yet.)

So, as I write this, I'm running my "Work" guest image on top of VMWare Fusion. (Had to buy the upgrade to VMWare Workstation 6 in order to support the move to Fusion, since Fusion uses the hardware format of Workstation 6. Besides, Fusion/6 has some support for DirectX, and yes, I'm also interested in getting some games installed in here and playing around that way too. ) I have to say, I really don't care for the touchpad, having spent the last half-decade training my index finger to use the nipple on the Thinkpads, but I picked up a Bluetooth mouse for serious work sessions, and like all good mice, it has two buttons. (Anybody know how to emulate a right-mouse-button-click on a MacBookPro?)

Fortunately as well, I have friends who are serious Macophiles, and I'm sure they will take my n00b questions with patience and understanding... since it's their fault I'm here anyway.

Wish me luck....

I know many of you, over the past couple of years, have sent me LinkedIn invitations that I've turned down (mostly because I've been nervous about what LinkedIn would do with the personal information), but a recent email discussion between myself and Van Meyer (of Meyer Strategic, Inc) has finally convinced me to jump in. The profile is public, in case anybody wanted to have a look.

So... to all those who sent me LinkedIn invitations that I turned down, I'm now online, and will accept invitations offered me.

Assuming, of course, I know you.

Software is endless battle and conflict, and you cannot develop effectively unless you can identify the enemies of your project. Obstacles are subtle and evasive, sometimes appearing to be strengths and not distractions. You need clarity. Learn to smoke out your obstacles, to spot them by the signs and patterns that reveal hostility and opposition to your success. Then, once you have them in your sights, have your team declare war. As the opposite poles of a magnet create motion, your enemies--your opposites--can fill you with purpose and direction. As people and problems that stand in your way, who represent what you loathe, oppositions to react against, they are a source of energy. Do not be naive: with some problems, there can be no compromise, no middle ground.

In the early 1970s, Margaret Thatcher shook up British politics by refusing to take the style of the politicians before her: where they were smooth and conciliatory, she was confrontational, attacking her opponents directly. She bucked the conventional wisdom, attacking her opponents mercilessly where historically politicians sought to reassure and compromise. Her opponents had no choice but to respond in kind. Thatcher's approach polarized the population, seizing the attention, attracting the undecided and winning a sizable victory.

But now she had to rule, and as she continued down her obstinate, all-in style of "radicalism" politics, she seemed to gain more enemies than any one politician could hold off. Then, in 1982, Argentina attacked the British Falkland Islands in the South Atlantic. Despite the distance--the Falklands were almost diametrically opposite the globe from the British home islands, off the tip of South America--Thatcher never hesitated, dispatching the British armed forces to respond to the incursion with deadly force, and the Argentinians were beaten. Suddenly, her obstinacy and radicalism were seen in a different light: courage, nobility, resolute and confident.

Thatcher, as an outsider (a middle-class woman and a right-wing radical), chose not to try and "fit in" with the crowd, but to stake her territory clearly and loudly. Life as an outsider can be hard, but she knew that if she tried to blend in, she could easily be replaced. Instead, she set herself up at every turn as one woman against an army of men.

As Greene notes,

We live in an era in which people are seldom directly hostile. The rules of engagement--social, political, military--have changed, and so must your notion of the enemy. An up-front enemy is rare now and is actually a blessing. ... Although the world  is more competitive than ever, outward aggression is discouraged, so people have learned to go underground, to attack unpredictably and craftily. ... Understand: the word 'enemy'--from the Latin inimicus, "not a friend"--has been demonized and politicized. Your first task as a strategist is to widen your concept of the enemy, to include in that group those who are working against you, thwarting you, even in subtle ways.

Software projects face much the same kind of problem: there are numerous forces that are at work trying to drag the project down into failure. While agile books love to assume an environment in which the agile methodology is widely accepted and embraced, reality often intrudes in very rude ways. Sometimes management's decision to "go agile" is based not to try and deliver software more successfully, but to simply take the latest "fad" and throw it into the mix, such that when it fails, management can say, "But we followed industry best practices, it clearly can't be management at fault." (This is the same idea behind the statement, "Nobody ever got fired for buying IBM (or Microsoft or Java or ...)." Sometimes the users are not as committed to the project as we might hope, and at times, the users are even contradictory to one another, as each seeks to promote their own agenda within the project.

As a software development lead (or architect, or technical lead, or project manager, or agile coach, or whatever), you need to learn how to spot these enemies to your project, identify them clearly, and make it clear that you see them as an enemy that will not be tolerated. This doesn't mean always treat them openly hostile--sometimes the worst enemies can be turned into the best friends, if you can identify what drives them to the position that they take, and work to meet their needs. Case in point: system administrators frequently find themselves at odds with developers, because the developer seeks (by nature) to change the system, and sysadmins seek (by nature) to keep everything status quo. Recognizing that sysadmins have historically been blindsided by projects that essentially ignored their needs (such as a need to know that the system is still running, or the need to be able to easily add users, change security policies, or diagnose failures quickly at 3AM in the morning) means that we as developers can either treat them as enemies to be overcome, or win them as friends by incorporating their needs into the project. But they are either "for us" or "against us", and not just a group to be ignored.

Other enemies are not to be tolerated at any level: apathy, sloth, or ignorance are all too common among developer teams. Ignorance of how underlying technologies work. Apathy as to the correctness of the code being created. Sloth in the documentation or tests. These are enemies that, given enough time and inattention, will drag the project down into the tar pits of potential failure. They cannot be given any quarter. Face them squarely, with no compromise. Your team, if they hold these qualities, must be shown that there is no tolerance for them. Hold brown-bag lunches once a week to talk about new technologies, and their poential impact on the team or company or project. Conduct code reviews religiously, during active development (rather that at the end as a token gesture), with no eye towards criticizing the author of the code, but the code itself. Demand perfection in the surrounding artifacts of the project: the help files, the user documentation, the graphics used for buttons and backdrops and menus.

Do not wait for the enemies of your project to show themselves, either--actively seek them out and crush them. Take ignorance, for example. Do not just "allow" each of your developers to research new technologies, but demand it of them: have each one give a brown-bag presentation in turn. In a four-man team, this means that each developer will have a month in which to find something new to discover, analyze, discuss and present. They do not have to have all the answers to the technology, and in fact, if the technology is sufficiently large or interesting, they can spend the next month investigating a new element or new use of the technology. Demanding this of your developers means they are forced to educate themselves, and forced to raise their own awareness of the changing world. (Naturally, developers must be given time to do this research; anecdotally, giving them Friday afternoons to do this experimentation and research, when energy and interest in the work week is already typically at an ebb, works well.)

Wherever possible, avoid enemies that are large and hard to pinpoint. Simply saying "We need to have better quality code" is too amorphous and too vague. Developers have nothing to measure against. Personalize your enemies, eyeball to eyeball. Put names to them, make them clearly visible to all involved. "We will have 100% code coverage in our unit tests" is a clearly-defined goal, and anything that prevents that goal from being reached will be clearly visible. "We will not ship code that fails to pass any unit test" is another clear goal, but must be paired with something that avoids the natural "Well, then, we'll not write any unit tests, and we'll have met that goal!" response. Demanding a ratio of unit-test-lines-to-lines ratio is a good start: "We will have three lines of unit test code per line of code" now offers a measurable, identifiable enemy that can be stared in the face. Go so far as to make a ceremony out of it: call the developers into a room, put a poster on a wall, and make your intentions clear. Motivate them. "When we presented the release of the payroll system to the HR department last year, the users called it 'barely acceptable' and 'hard to use'. I refuse to allow that to happen again. The system we build for them this year will be amazing. It will be reliable. It will have those features they need to get their job done (and be specific here), and we will accept no excuse otherwise."

One of the world's most successful software companies, Microsoft is no stranger to the polarity strategy. The company as a whole as declared war on its enemies in a variety of fields, and with few exceptions, has won in almost every conflict. Microsoft actively courts conflict and confrontation. The presence of a well-established competitor in a particular field is no barrier to entry; Microsoft has routinely entered fields with dominant competitors and come out ahead in the end. Witness their entries into the word-processor and spreadsheet markets held at the time by dominant competitors WordPerfect and Lotus 1-2-3, their entry into the video-game console market against well-established competitors Sega and Nintendo, and more recently, the mobile entertainment device market (the Zune) against the iPod. In the latter, the battle has just begun, and the market remains firmly in the hands of the iPod, but let it not be forgotten that Microsoft is not one to retreat quickly from a battle.

Microsoft is also known to do this within their projects; developers who are committed to a project yet seem hesitant or lax in their work are asked if they are really "on board" with this project. The legends of Microsoft developers putting in 80-plus hours a week on a project are somewhat true, but not because Microsoft management expects it of them, but because developers have been willing to put that kind of time into the project in order to succeed.

And Microsoft management itself has declared war on its enemies, time and again, looking to eliminate any and all opposition the successful release of software. Distractions? Every developer gets his own office. Household chores? Microsoft has been known to offer their developers laundry services at work. Computing resources? It's not at all uncommon to walk into a Microsoft developers' office and see multiple CPUs (under desks, in corners, laptops, and so on) and monitors all over the room. Bodily needs? Refrigerators on every floor, stocked with sodas of every variety, water, juices, anything that the thirsty developer could need, all free for the taking. Even fatigue is an enemy: Microsoft buildings have video game consoles, foosball tables, bean bag chairs, and more tools of relaxational activities to help the developer take a break when necessary, so that they can resume the fight refreshed.

Greene notes further,

There are always hostile people and destructive relationships. The only way to break out of a negative dynamic is to confront it. Repressing your anger, avoiding the person threatening you, always looking to conciliate--these common strategies spell ruin. Avoidance of conflict becomes a habit, and you lose the taste for battle. Feeling guilty is pointless; it is not your fault you have enemies. Feeling wronged or victimized is equally futile. In both cases you are looking inward, concentrating on yourself and your feelings. Instead of internalizing a bad situation, externalize it and face your enemy. It is the only way out.

To adapt this to software, instead of simply talking about the hopeless situation in which you find yourself--your company has no interest in agile, your team is just too "inexperienced" to tackle the kinds of projects you are being given, and so on--externalize it. Face the enemy. Your company has no interest in agile? Fine--instead of trying to talk them into it, take the radical approach, do a project in an agile fashion (even without upper management's knowledge if necessary), and show them the results. Can't get the IT budget to allow for a source-control server or continuous integration server? Use your desktop machine instead. Face the enemy, confront it, and defeat it.

Enemies are not evil, and should not be seen as something to avoid. Enemies give us something against which to measure ourselves, to use as a foil against which to better ourselves. They motivate and focus your beliefs. Have a co-worker who refuses to see the benefits of dynamic languages? Avoiding him simply avoids an opportunity for you to learn from him and to better your arguments and approaches. Have a boss who doesn't see what the big deal behind a domain-specific language is? Have conversations on the subject, to understand her reluctance and opposition, and build a small DSL to show her the benefits of doing so. Don't avoid these people, for they offer you opportunities to better yourself and your understanding.

Enemies also give you a standard against which to judge yourself. It took Joe Frazier to make Muhammad Ali a truly great boxer. The samurai of Japan had no guage of their excellence unless they challenged the best swordsmen they could find. For the Indianapolis Colts of last year, each victory was hollow unless they could beat their arch-rivals, the New England Patriots. The bigger the opponent, the greater your reward, even in defeat, for if you lose, you have opportunities to analyze the results and discover how and why you lost, then correct your strategy for the next time. For there will always be a next time.

Don't seek to eschew enemies, either. Enemies give us options and purpose. Julius Caesar identified Pompey as his enemy early in his rise to the throne. Everything he did from then on was measured against Pompey, to put him in a stronger position to meet his enemy. Once he defeated Pompey, however, Caesar lost his way, and eventually fell because he viewed himself as a god and above all other men. Enemies force on you a sense of realism and humility.

Remember, enemies surround you and your project, and sometimes even within your project. Keep a sharp eye out, so that once spotted, they can be identified, analyzed, and handled. Show no quarter to those enemies: they must either join you to help you in your quest to build great software, or be ruthlessly eliminated from your path. They can either benefit from the project, or they can be removed from the battlefield entirely. Some enemies--ignorance, apathy, sloth--are not easily defeated, nor once defeated will they remain so. Never lay down your arms against them or trust your arms to someone else--you are the last line of your own defense.

I've been doing a series of video interviews for Pearson (the group behind the publishers Addison-Wesley and Prentice-Hall, among other titles), starting with a series of about a dozen or so we took at the SDWest show back in March. At said show... well... Barbara's blog says it best. (Warning--partial nudity here. Not suitable for work. )

And yes, it really did happen that way--Bjarne and Herb weren't entirely sure if having a T-shirt emblazoned with "I love C#" on it would go well with their fans, so... *shrug* I reversed it and we went on.

Of course, they *were* probably half-joking, and we *could* probably have done it without a problem... but when you're interviewing one of your childhood heroes (actually, two of them), you don't look to let obstacles stand in the way.

Meanwhile, the entire series is now up on iTunes, under "On Software" (or subscribe), or subscribe to an RSS feed. More are coming, and unfortunately I can't tell you who I was interviewing this week at OSCon, or a few months ago at Microsoft TechEd, but they're all looking pretty good....

Oh, and if you have any suggestions of questions to ask these kinds of folks for these video podcasts, by all means, drop me a line--Pearson's planning a bunch more, and I can always use good ideas for questions....

For those who aren't familiar with the term, "yellow journalism" was a moniker applied to journalism (newspapers, at the time) articles that were written with little attention to the facts, and maximum attention to gathering attention and selling newspapers. Articles were sensationalist, highly incorrect or unvalidated, seeking to draw at the emotional strings the readers would fear or want pulled. Popular at the turn of the last century, perhaps the most notable example of yellow journalism was the sinking of the Maine, a US battleship that exploded in harbor while visiting Cuba (then, ironically, a very US-friendly place). Papers at the time attributed the explosion to sabotage work by Spain, despite the fact that no cause or proof of sabotage was ever produced, leading the US to declare war on the Spanish, seize several Spanish colonies (including the Phillipines in the Pacific, which would turn out to be important to US Pacific Naval interests during World War Two), and in general pronouce anything Spanish to be "enemies of the state" and all that.

Vaguely reminiscent of Fox News, now that I think of it.

In this case, however, yellow journalism meets the Web in two recent "IT magazine" pieces that have come to my attention: this one, which blasts Sun for not rolling out updates in a more timely fashion to its consumers, despite the many issues that constant update rollouts pose for those same consumers, but more flagrantly, this one, which states that Google researchers have found a vulnerability in the Java Runtime Environment that "threatens the security of all platforms, browsers, and even mobile devices". As if that wasn't enough, check out these "sky-is-falling" quotes:

" 'It’s a pretty significant weakness, which will have a considerable impact if the exploit codes come to fruition quickly. It could affect a lot of organizations and users.'

"... anyone using the Java Runtime Environment or Java Development Kit is at risk.

" 'Delivery of exploits in this manner is attractive to attackers because even though the browser may be fully patched, some people neglect to also patch programs invoked by browsers to render specific types of content.'

"... the bugs threaten pretty much every modern device.

" '... this exploit is browser independent, as long as it invokes a vulnerable Java Runtime Environment.'

"... the problem is compounded by the slim chance of an enterprise patching Java Runtime vulnerabilities.

Now, I have no problems with the media reporting security vulnerabilities; in fact, I encourage it (as any security professional should), because consumers and administrators can only take action to protect against vulnerabilities when we know about them. But here's the thing: nowhere, not one place in the article, describes what the vulnerability actually is. Is this a class verifier problem? Is this a buffer overflow attack? A luring attack? A flaw in the platform security model? A flaw in how Java parses and consumes image formats (a la the infamous "picture attachment attack" that bedevils Outlook every so often)?

No details are given in this article, just fear, uncertainty and doubt. No quote, no vague description of how the vulnerability can be exploited, not even a link to the original report from Google's Security team.

Folks, that is sensationalist journalism at its best. Or worst, if you prefer.

Mr. Tung, who authored the article, should have titled it "The Sky is Falling! The Sky is Falling!" instead. Frankly, if I were Mr. Tung's editor, this drivel would never have been published. If I were given the editor's job tomorrow, I'd thank Mr. Tung for his efforts and send him over to a competitor's publication. Blatant, irresponsible, and reckless.

Now, if you'll excuse me, I'm going to try and find some hard data on this vulnerability. Any vulnerability that can somehow strike across every JVM ever written (according to the article above) must be some kinda doozy. After all, I need to learn how to defend myself before al Qaeda gets hold of this and takes over "pretty much every modern device" and uses them to take over the world, which surely must be next....

At a software conference not too long ago, I was asked what book I was currently reading that I'd recommend, and I responded, "Robert Greene's The 33 Strategies of War". When asked why I'd recommend this, the response was pretty simple: "Because I believe that there's more parallels to what we do in military history than in constructing buildings."

Greene's book is an attempt at a distillation of what all the most successful generals and military leaders throughout history used to make them so successful. A lot of these concepts and ideas are just generally good practices, but a fair amount of them actually apply pretty directly to software development (whether you call it "agile" or not). Consider this excerpt from the Preface, for example:

The war [that exists in the real world] exists on several levels. Most obviously, we have our rivals on the other side. The world has become increasingly competitive and nasty. In politics, business, even the arts, we face opponents who will do almost anything to gain an edge. More troubling and complex, however, are the battles we face with those who are supposedly on our side. There are those who outwardly play the team game, who act very friendly and agreeable, but who sabotage us behind the scenes, ues the group to promote their own agenda. Others, more difficult to spot, play subtle games of passive aggression, offering help that never comes, instilling guilt as a secret weapon. On the surface everything seems peaceful enough, but just below it, it is every man and woman for him- or herself, this dynamic infecting even families and relationships. The culture may deny this reality and promote a gentler picture, but we know it and feel it, in our battle scars.

Without trying to paint a paranoid picture, this "dynamic of war" frequently infects software development teams and organizations; developers vs. management, developers vs. system adminstrators, developers vs. DBAs, even developers vs. architects or developers vs. developers. His book, then, suggests that we need to face this reality and learn how to deal with it:

What we need are not impossible and inhuman ideals of peace and cooperation to live up to, and the confusion that brings us, but rather practical knowledge on how to deal with conflict and the daily battles we face. And this knowledge is not about how to be more forceful in getting what we want or defending ourselves but rather how to be more rational and strategic when it comes to conflict, channeling our aggressive impulses instead of denying or repressing them. If there is an ideal to aim for, it should be that of the strategic warrior, the man or woman who manages difficult situations and people through deft and intelligent maneuver.

... and I want that man or woman heading up my project team.

It may seem incongruous to draw parallels between war and software development, because in war there is an obvious "enemy", an obvious target for our aggression and intentions and strategies and tactics. It turns out, however, that the "enemy" in software development is far more nebulous and amorphous, that of "failure", which can be just as tenacious and subversive. This enemy won't ever try to storm your cubicles and kill you or try to hold you for ransom, but a lot of the strategies that Greene talks about aren't so much about how to kill people, but how to think strategically, which is, to my mind, something we all of us have to do more of.

Consider this, for example; Greene suggests "six fundamental ideals you should aim for in transforming yourself into a strategic warrior in daily life":

• Look at things as they are, not as your emotions color them. Too often, it's easy to "lose your head" and see the situation in emotional terms, rather than rational ones. "Fear will make you overestimate the enemy and act too defensively"; in other words, fear will cause you to act too conservatively and resist taking the necessary gamble on a technology or idea that will lead to success. "Anger and impatience will draw you into rash actions that will cut off your options"; or, anger and impatience will cause you to act rashly with respect to co-workers (such as DBAs and sysadmins) or technology decisions that may leave you with no clear path forward. "The only remedy is to be aware that the pull of emotion is inevitable, to notice it when it is happening, and to compensate for it."
• Judge people by their actions. "What people say about themselves [on resumes, in meetings, during conversations] does not matter; people will say anything. Look at what they have done; deeds do not lie." Which means, you have to have a way by which to measure those deeds, meaning you have to have a good "feel" for what's going on in your department--simply listening to reports in meetings is often not enough. "In looking back at a defeat [failed project], you must identify the things you could have done differently. It is your own bad strategies, not the unfair opponent [or management decisions or unhelpful IT department, or whatever], that are to blame for your failures. You are responsible for the good and bad in your life."
• Depend on your own arms. "... people tend to rely on things that seem simple and easy or that have worked before. ... But true strategy is psychological--a matter of intelligence, not material force. ... But if your mind is armed with the art of war, there is no power that can take that away. In the middle of a crisis, your mind will find its way to the right solution. ... As Sun-tzu says, 'Being unconquerable lies with yourself.' "
• Worship Athena, not Ares. This one probably doesn't translate directly; Athena was the goddess of war