Michael Nygard (author of the great book Release It!), writes that "[his] definition of 'done' continues to expand". Currently, his definition reads:

A feature is not "done" until all of the following can be said about it:

1. All unit tests are green.
2. The code is as simple as it can be.
3. It communicates clearly.
4. It compiles in the automated build from a clean checkout.
5. It has passed unit, functional, integration, stress, longevity, load, and resilience testing.
6. The customer has accepted the feature.
7. It is included in a release that has been branched in version control.
8. The feature's impact on capacity is well-understood.
9. Deployment instructions for the release are defined and do not include a "point of no return".
10. Rollback instructions for the release are defined and tested.
11. It has been deployed and verified.
12. It is generating revenue.

Until all of these are true, the feature is just unfinished inventory.

As much as I agree with the first 11, I'm not sure I agree with #12. Not because it's not important--too many software features are added with no positive result--but because it's too hard to measure the revenue a particular program, much less a particular software feature, is generating.

My guess is that this is also conflating the differences between "features" and "releases", since they aren't always one and the same, and that not all "features" will be ones mandated by the customer (making #6 somewhat irrelevant). Still, this is an important point to any and all development shops:

What do you call "done"?

For those who've seen me give presentations at the No Fluff Just Stuff symposia over the year who've also asked to get the demo code that I write during the presentations, I've put it all up online. Simply find the show you attended below, and the hyperlink downloads the associated .zip file:

• Boulder JUG
• Denver JUG
• Phoenix JUG
• Atlanta
• Austin
• Boston
• Chicago
• Columbus
• Dallas
• Denver
• Green Bay
• London
• Miami
• Milwaukee
• Minneapolis (1st show)
• Minneapolis (2nd show)
• Oklahoma City
• Omaha
• Orlando
• Phoenix
• Portland
• Princeton, NJ
• Raleigh, NC
• Reston, DC
• Seattle
• St Louis
• Toronto

Usual disclaimer applies, as with all demo code: I don't guarantee the code for any particular purpose except as a supplement to the lecture that was given at that time in that city, the code isn't guaranteed to do anything except soak up space on your hard drive, I can't be held responsible if you use it in production, and so on.

If any of the links are broken, email me and let me know, but I think they should all work.

... for Ruby, or PowerShell/.NET? I'm looking for something to make it easier to use WebDAV from a shell scripting language on Windows; Ruby and PowerShell are the two that come to mind as the easiest to use on Windows. For some reason, Google doesn't yield much by way of results, and I've got to believe there's better WebDAV support out there than what I'm finding.

(Yes, I could write one, but why bother, if one is out there that already exists? DRY!)

BTW, anybody who finds one and wants credit for it, I'll be happy to post here. If you're a commercial vendor and you send me a license to go with it, I'll post that, too, with some code and explanation on how I'm using it, though I doubt it's going to be all that different from how anybody else would use it.

Some of you knew me way back when I started the domain name "javageeks.com". Obviously, it's been a while since I updated that site, and in fact, if you go try to read some of the papers there, you'll get redirected to "neward.net", which has been on hiatus ever since the ISP there decided that my use of JSP was somehow trashing their server. (This is what happens when college students run an ISP in the spare time, I guess.)

Anyway, for a variety of reasons, I've since decided that my online presence should coalesce into one place, this domain (tedneward.com), and I've been slowly moving things over to this site in recognition of that fact. As a result, a bunch of the papers that I'd put up for reading on javageeks.com are now here on tedneward.com, in the "Writing" section of the website.

For those of you too tied to RSS to go visit the site itself, here's a list of the papers I've moved over thus far:

• Using the Boot Classpath
• Finding Loaded Classes
• Using Java Policy
• When is a Static Not Static?
• Multiple Java Homes
• X-Power: Using XML to Write Papers

It's not the complete list, since I'm also (trying to) move the authoring format over from my own home-grown XML+XSLT system into DocBook, but that means either developing an XSLT to transform my home-grown format into DocBook, or else just manually switching them over. I'm not convinced yet which is the faster way to go.

Part of it isn't just transmuting their form, though. Some of them could use with a touch of updating (such as the Java Policy one--there's been a few changes there since I wrote that paper, back in Java 1.2 days), so I think I'm going to have to labor through and transform and update them manually. Still, it could be worse: I could have written them in Lotus Ami Pro....

Now, I've had my issues with corporate management before (some of you may recall my run-in with Sun lawyers over the domain "javageeks.com"), and I've seen corporations behave badly with cease-and-desist letters (Hey, Microsoft... you and RedHat can come out of the corner now...), but this action on Sun's part to effectively muscle out the four project leads on the open-source OpenDS project definitely deserves some kind of award.

(For those of you who want the short version: four Sun employees, including Neil Wilson, the poster, have been working on the OpenDS project for the last n number of years. Sun decides to offshore its directory services development to Europe (WTF?!?), and pink-slips the four project owners. The project owners decide to continue working on the project, and decide that there should be a fifth owner, a Sun employee, in order to make sure Sun keeps a hand in a project it vested time & energy & money into. Sun decides instead to require that ALL the project owners be Sun employees. Project owners, a tad distrubed, circle back to discuss things. Sun management threaten to yank all four ex-employees' severance benefits if they fight this. Project owners basically decide to bail entirely, and not anger the Sun Gods any further, until after the benefits have been paid out. Project owner Wilson blogs the aforementioned story, which annoys a few dark corners of the blogosphere, including yours truly.)

In fact, in honor of the book "In Search of Stupidity", I hereby nominate Sun for a Stupidity Award, in this case, a two-parter: one, "For having the incredible temerity to threaten with termination the very core group of people who built the product into what it was", and two, "For having the absolute chutzpah to effectively backstab the very community-oriented efforts that Sun upper management are seeking to build as a way to make the company once again relevant in the computing industry".

I dunno if anybody at Sun reads my blog, but if you do, forward this over to Simon Phipps and Jonathan Schwartz, because if they want to be taken seriously as an open-source company, they need to find the morons who acted with this kind of heavy-handedness and defenestrate them at the earliest opportunity. An open letter of apology and response should be posted on one or both blogs. Then, they need to go back to Mr. Wilson, apologize profusely, and ask him and the other three--very nicely--if they'll agree to take up the reins of the OpenDS project again and allow Sun to demonstrate--by actions, not words--that they are a community-focused company.

Or, I suppose, they could do none of this, and prove--by actions, not words--that they don't give a sh*t. (Frankly, I could care less about directory services projects, but I would think Sun cares far more about its reputation at this point than the project itself.)

'Tis your move, Sun. Are you "profoundly involved in the open source world", or not?

Update: Simon Phipps writes, in a comment here (which may not be accessible to some RSS readers, so I will quote in full):

Ted: I have investigated this situation. It's very regerttable that Sun had to let some great engineers go as part of a staff reduction, unrelated to the governance issue, but sadly that happens in every company. The real issue I was concerned about here was the discussion of governance. I discovered that Neil did not tell the whole story.

The original governance of OpenDS said this:

<blockquote>The OpenDS project has single, overall Project Lead. This Project Lead, who is appointed by Sun Microsystems, is responsible for managing the entire project, and is the final arbiter of all decisions.</blockquote>

At some point, without any community discussion I can find, and without management discussion, while still Sun employees, the leaders of OpenDS changed the governance to say this:

<<The OpenDS project has single, overall Project Lead. This Project Lead, who is appointed and removed by a majority vote of the Project Owners, is responsible for managing the entire project, and is the final arbiter of all decisions in the development process.>>

You can see the change in the Subversion system for OpenDS here.

When Neil says:

<<On November 14, 2007, a member of executive management within Sun’s software division contacted one of the recently-laid-off OpenDS project owners and demanded that the owners approve a governance change that would grant Sun full control of the OpenDS project.>> that is misleading. What actually happened is the leaders were asked to revert their arbitrary change of the governance (they also removed the phrase "Project Owners will always attempt to reach consensus before making decisions") so that the matter could be openly discussed.

I'm continuing to investigate and monitor, but right now the evidence I see is that Sun has not acted in bad-faith towards the governance. I'd welcome evidence to the contrary if it exists, but that SVN dif seems pretty conclusive to me.

There are, I think, two issues here that are getting conflated, and I want to tease them apart. What is not an issue, as far as I'm concerned, was Sun's decision to let these engineers go in the first place. Business decisions that involve letting people go are always hard decisions to make, and I won't second-guess the thought process that led up to it. I'm willing to believe that if Sun management felt they could have kept them in place, they would have. (You may choose to disagree but that's your own value judgement to make.) That doesn't take away from the two issues I want to discuss here.

First, there is the issue of whether or not Sun's actions were in accordance with the project governance model for the OpenDS project, which Simon seems to focus on above. Frankly, I could care less about this aspect of things; whether that change was appropriate, necessary, legal, whatever, is not the issue I want to discuss or point out. Yes, there is the possibility that Neil-and-company's actions were regarded as being contrary to the charter of the project by the legitimately-selected Project Owner, and Sun felt it necessary to retaliate legally. Even in that scenario, it strikes me as a heavy-handed reaction.

(Quite footnote: on a whim, after posting my update, I checked the SVN revisions beyond what Simon posted. The revision change in question, from 1622 to 1739, occured on April 28th, 2007, which is close enough to have been part of this whole fiasco, but obviously I'm wasn't there, I can't know that for certain. Needless to say, if all these changes occurred without any community input, that's clearly not playing to the spirit of the open source community, IMHO.)

Secondly, there is the issue of Sun's threat to revoke their severance benefits. To me, this is unconscionable. Severance is a way of providing valuable employees with a "cover period" while they seek new employment ("we were encouraged to use this time to find a new job", Neil reports). This is a nice gesture, a way of saying "thank you" for the valuable efforts of a valued employee. But to turn around and threaten to remove those benefits if the employees don't play ball on a job-related issue when those employees are seeking to further the company's stated cause seems heavy-handed and unnecessary.

Look, in order to be seen as effective without being heavy-handed, keep the scope of the response within the frame of the supposed offense: If they are violating the governance model, then threaten to remove them from the project. If they are seeking to use the code in a way that is violating the license of the original, then threaten legal action over the licensing model and those actions. To go straight for the severance benefits here seems akin to Pres. Bush reaching for the nuclear codes when a foreign national jaywalks. The punishment must fit the crime. And the crime has to be proven first, in any event. The risks of a "preventative first strike" are only acceptable when the liabilities are equally as high, if not higher.

Look at it this way: the way this played out, Sun...

1. ... kept control over the OpenDS project... which they were going to have anyway, under either side's story.
2. ... still paid out the severance benefits for those engineers.
3. ... earned a negative PR hit among the very people they're trying to convince, the open-source community.
4. ... earned the enmity of the core developers working on the OpenDS project, which I presume they would have preferred to have continue working on the project, even if they no longer paid their salary.
5. ... forced their Chief Open Source Officer to spend time investigating this whole sordid mess.

What might have happened if Sun had simply approached the engineers in question and instead of threatening to yank their benefits, had said "Look, we understand what you're trying to do, but as we understand it, the project's governance statement says otherwise... We value you guys, we value what you did, we want you to continue to contribute, how do we resolve this amicably?" Sun might have had to compromise--noting, by the way, that they might have done this even when legally or morally they didn't have to--and would have instead...

1. ... kept control over the OpenDS project.
2. ... still paid out the severance benefits for those engineers.
3. ... earned a positive PR boost at best, no PR at worst, among the community.
4. ... kept the goodwill of the developers who represented Sun's efforts on the project.
5. ... had a "feel-good" story their Chief Open Source Officer could use when addressing the OpenDS group at the next directory services conference. ("Look, Sun is committed to open source, even when we can't contribute the resources financially, we can prove this by...")

Sometimes, even when you're right, you get more from taking the position that there is room to compromise on your end. Flies, honey and vinegar, and all that. I applaud Simon's efforts to get involved and track this one down, but I don't think this is an open-source issue.

I think this is a management issue.

When you can't do something simple with it.

Check out this absurdly simple Groovy script:

import javax.management.ObjectName
import javax.management.MBeanServerConnection
import javax.management.remote.JMXConnectorFactory as JMXFactory
import javax.management.remote.JMXServiceURL as JMXUrl

def serverUrl = 'service:jmx:rmi:///jndi/rmi://127.0.0.1:9004/jmxrmi'
def serv = JMXFactory.connect(new JMXUrl(serverUrl))
def on = new ObjectName('Catalina:type=Server')
def gmb = new GroovyMBean(serv, new ObjectName('Catalina:type=Server')).serverInfo

For those of you not up on your JMX, this is a simple connection via an RMI connector to the JMX server running at port 9004 (which happens to be my local Tomcat installation). This is straight off the Groovy-JMX documentation page, but slimmed down because the GroovyMBean constructor throws an exception, claiming that the desired constructor cannot be found:

C:\Projects\Exploration\Groovy>groovy GroovyJMX.groovy
Caught: groovy.lang.GroovyRuntimeException: Could not find matching constructor
for: groovy.util.GroovyMBean(javax.management.remote.rmi.RMIConnector, javax.management.ObjectName)
at GroovyJMX.run(GroovyJMX.groovy:9)
at GroovyJMX.main(GroovyJMX.groovy)
C:\Projects\Exploration\Groovy>

C'mon, folks. I have no idea what the problem is, and debugging this is a nightmare. I've looked around various Groovy forums, and nobody appears to have any real idea what's going on. Either nobody is really using Groovy as a JMX client (in which case, just remove the GroovyMBean from the library), or else Groovy has a bug within it (thus reducing its efficicacy as a production-ready language).

I'm fully willing to accept that the problem is with me or my environment somehow. The challenge, however, is for somebody to take a stock JDK 1.6 and Groovy 1.0 download (oh, and 1.1-rc2 fails with the same error, so that's not the issue, either), run the above 8-line script, and tell me why mine isn't working. (I've already done the suggested step of removing the mx4j jar out of the groovy-1.0/lib directory, so that doesn't help, either.)

Oh, and if you're going to write in claiming that this is a ClassLoader issue or something, you'd be wrong--all of the JMX types being loaded are coming out of rt.jar (which I verified using -verbose:class, doing which required me to edit the Groovy launcher scripts, which I find to be just silly--don't make it hard for me to use the basic management & monitoring facilities of the JVM). The only ClassLoader player I don't know for certain is the org.codehaus ClassLoader that's established by Groovy itself, so if the problem is in ClassLoaders, it's inside of the Groovy implementation, which means it's a Groovy problem, not mine.

I've even taken the step to compile the Groovy code into .class files, and run those:

C:\Projects\Exploration\Groovy>groovyc -d compiled GroovyJMX.groovy
C:\Projects\Exploration\Groovy>cd compiled

C:\Projects\Exploration\Groovy\compiled>java -classpath .;libg\groovy-1.0.jar;li
bg\asm-2.2.jar GroovyJMX
Exception in thread "main" groovy.lang.GroovyRuntimeException: Could not find ma
tching constructor for: groovy.util.GroovyMBean(javax.management.remote.rmi.RMIC
onnector, javax.management.ObjectName)
at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:776)
at groovy.lang.MetaClassImpl.invokeConstructor(MetaClassImpl.java:688)
at org.codehaus.groovy.runtime.Invoker.invokeConstructorOf(Invoker.java:
163)
at org.codehaus.groovy.runtime.InvokerHelper.invokeConstructorOf(Invoker
Helper.java:140)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeNewN(ScriptBy
tecodeAdapter.java:243)
at GroovyJMX.run(GroovyJMX.groovy:9)
at gjdk.GroovyJMX_GroovyReflector.invoke(Unknown Source)
at groovy.lang.MetaMethod.invoke(MetaMethod.java:115)
at org.codehaus.groovy.runtime.MetaClassHelper.doMethodInvoke(MetaClassH
elper.java:713)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:560)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:450)
at org.codehaus.groovy.runtime.Invoker.invokeMethod(Invoker.java:131)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.
java:111)
at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.jav
a:408)
at gjdk.org.codehaus.groovy.runtime.InvokerHelper_GroovyReflector.invoke
(Unknown Source)
at groovy.lang.MetaMethod.invoke(MetaMethod.java:115)
at org.codehaus.groovy.runtime.MetaClassHelper.doMethodInvoke(MetaClassH
elper.java:713)
at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:664)
at org.codehaus.groovy.runtime.Invoker.invokeMethod(Invoker.java:111)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.
java:111)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(Scrip
tBytecodeAdapter.java:187)
at GroovyJMX.main(GroovyJMX.groovy)

C:\Projects\Exploration\Groovy\compiled>

And here we can obviously see, by the light of the .jars passed in, that the mx4j jars are nowhere to be found (unless, of course, Groovy is explicitly searching the hard drive for them, because I deleted them entirely out of Groovy's lib directory).

I've asked a couple of the Groovy heavyweights (not mentioning anybody by name) if they know what's up. Silence. Not a good sign.

I'm about to go off and try the same thing using JRuby. If it works, out of the box, then the problem is with Groovy, not with me. If you're a Groovy expert, or know someone who is, then have them email me the solution (assuming they can find one), because this is a point where I'm about to close the door on Groovy forever: if you can't do something simple with it, it's not ready for prime-time. It may be OK to whip up dirt-simple websites where 90% of the stuff is pre-generated, but if I can't use it for something like being a JMX client, then it ain't worth my time.

'Nuff said.

Update: OK, I may have to eat my words.

Playing around with some JRuby/JMX stuff (which is absurdly simple, even without jmx4r, which I'm trying to get gem to install right now), I went back and decided to try a little JMX without using GroovyMBean:

import javax.management.*
import javax.management.remote.*
import java.lang.management.*

def serverUrl = 'service:jmx:rmi:///jndi/rmi://127.0.0.1:9004/jmxrmi'
def connector = JMXConnectorFactory.connect(new JMXServiceURL(serverUrl))
def mbsc = connector.mBeanServerConnection

def memory_mbean =
ManagementFactory.newPlatformMXBeanProxy(mbsc, "java.lang:type=Memory", MemoryMXBean.class)

This, by the way, is almost a straight port of Jeff's corresponding JRuby/JMX example code. Works just fine. From there, I thought, "Let's see if I can get past the problem I was having with GroovyMBean a few minutes ago." So I add the one-line GroovyMBean constructor (taking the mBeanServerConnection as the first parameter):

def gmb = new GroovyMBean(mbsc, new ObjectName('Catalina:type=Server')).serverInfo

Voila! Success! A quick execution test verifies that I'm all good:

def query = new ObjectName('Catalina:*')
String[] allNames = mbsc.queryNames(query, null)
def modules = allNames.findAll{ name ->
    name.contains('j2eeType=WebModule')
}.collect{ new GroovyMBean(mbsc, it) }

println "Found ${modules.size()} web modules. Processing ..."

modules.each{ m ->
    println "Found ${m.name()} at ${m.path} (${m.processingTime})"
}

returns:

C:\Projects\Exploration\Groovy>groovy GroovyJMX.groovy
Found 5 web modules. Processing ...
Found Catalina:j2eeType=WebModule,name=//localhost/docs,J2EEApplication=none,J2E
EServer=none at /docs (0)
Found Catalina:j2eeType=WebModule,name=//localhost/host-manager,J2EEApplication=
none,J2EEServer=none at /host-manager (0)
Found Catalina:j2eeType=WebModule,name=//localhost/examples,J2EEApplication=none
,J2EEServer=none at /examples (0)
Found Catalina:j2eeType=WebModule,name=//localhost/,J2EEApplication=none,J2EESer
ver=none at (0)
Found Catalina:j2eeType=WebModule,name=//localhost/manager,J2EEApplication=none,
J2EEServer=none at /manager (0)
C:\Projects\Exploration\Groovy>

Now if that ain't cool, I don't know what is.

Now for the hard part: What happened? Why'd my earlier code fail?

Looking back at my example script above, I clearly see that somewhere along the way in my debugging/exploration, I accidentally left out the call to obtain the MBeanServerConnection from the Connector and pass that in to the GroovyMBean constructor. Ugh.

Formal apologies to the Groovy crowd. However, I stipulate, for your own consideration, that

1. Originally my script was not so--I copied it line-for-line from the Groovy/JMX page, and I got the GroovyCastException that led me down this path. (You decide whether you believe me or not. )
2. The GroovyMBean constructor could (and should) be overloaded to take a Connector and extract the MBeanServerConnection from it and proceed. In fact, I'd suggest that the GroovyMBean should be written to take a JMXServiceURL as a parameter and handle all the details of connecting to the remote server.
3. Exploration tests would have yielded a better record of my efforts, and maybe found the point where my coding got led astray. It would also help track down the problem for others, if there was a language issue at stake here. (There may be--one noticeable difference is that in the working version, I don't use the import-redeclaration feature, whereas in the non-working version I did. More research is required.) At the time, I was just trying to debug a simple script problem, but at some point, greater rigor on my part should have kicked in so I could have better results to work from. Sigh.
4. Debugging this was way too hard. As we move into an era of more languages-atop-VMs, better debugging/spelunking facilities has to be a top priority.
5. Groovy (and JRuby) need to make sure they honor--somehow--the various flags I want to pass to the Java VM, such as -verbose or -client/-server, or even -ms or -mx. Java does have the JAVA_TOOLS_OPTIONS environment variable that will be picked up by the Java launcher (java.exe), but that only works if the language in question uses the launcher itself as part of its startup script. Both Groovy and JRuby seem to fail this test (though I admit I didn't look super-hard for undocumented/underdocumented ways to do it).

"Hi. My name's Ted, and I write shitty code."

With this opening, a group of us earlier this year opened a panel (back in March, as I recall) at the No Fluff Just Stuff conference in Minneapolis. Neal Ford started the idea, whispering it to me as we sat down for the panel, and I immediately followed his opening statement in the same vein. Poor Charles Nutter, who was new to the tour, didn't get the whispered-down-the-line instruction, and tried valiantly to recover the panel's apparent collective discard of dignity--"Hi, I'm Charles, and I write Ruby code"--to no avail. (He's since stopped trying.)

The reason for our declaration of impotent implementation, of course, was, as this post states so well, a Zen-like celebration of our inadequacies: To be a Great Programmer, you must admit that you are a Terrible Programmer.

To those who count themselves as the uninitiated into our particular brand of philosophy (or religion, or just plain weirdness), the declaration is a statement of anti-Perfectionism. "I am human, therefore I make mistakes. If I make mistakes, then I cannot assume that I will write code that has no mistakes. If I cannot write code that has no mistakes, then I must assume that mistakes are rampant within the code. If mistakes are rampant within the code, then I must find them. But because I make mistakes, then I must also assume that I make mistakes trying to identify the mistakes in the code. Therefore, I will seek the best support I can find in helping me find the mistakes in my code."

This support can come in a variety of forms. The Terrible Programmer cites several of his favorites: use of the Statically-Typed Language (in his case, Ada), Virulent Assertions, Testing Masochism, the Brutally Honest Code Review, and Zeal for the Visible Crash. Myself, I like to consider other tools as well: the Static Analysis Tool Suite, a Commitment to Confront the Uncomfortable Truth, and the Abject Rejection of Best Practices.

By this point in time, most developers have at least heard of, if not considered adoption of, the Masochistic Testing meme. Fellow NFJS'ers Stuart Halloway and Justin Gehtland have founded a consultancy firm, Relevance, that sets a high bar as a corporate cultural standard: 100% test coverage of their code. Neal Ford has reported that ThoughtWorks makes similar statements, though it's my understanding that clients sometimes put accidental obstacles in their way of achieving said goal. It's amibtious, but as the ancient American Indian proverb is said to state, If you aim your arrow at the sun, it will fly higher and father than if you aim it at the ground.

In fact, on a panel this weekend in Dallas, fellow NFJS'er David Bock attributed the rise of interest in dynamic languages to the growth of the unit-testing meme, since faith in the quality of code authored in a dynamic language can only follow if the code has been rigorously tested, since we have no tool checking the syntactic or semantic correctness before it is executed.

Among the jet-setting Ruby crowd, this means a near-slavish devotion to unit tests. Interestingly enough, I find this attitude curiously incomplete: if we assume that we make mistakes, and that we therefore require unit tests to prove to ourselves (and, by comfortable side effect, the world around us) that the code is correct, would we not also benefit from the automated--and in some ways far more comprehensive--checking that a static-analysis tool can provide? Stu Halloway once stated, "In five years, we will look back upon the Java compiler as a weak form of unit testing." I have no doubt that he's right--but I draw an entirely different conclusion from his statement: we need better static analysis tools, not to abandon them entirely. 

Consider, if you will, the static-analysis tool known as FindBugs. Fellow NFJS'er (and author of the JavaOne bestselling book Java Concurrency in Practice) Brian Goetz offered a presentation last year on the use of FindBugs in which he cited the use of the tool over the existing JDK Swing code base. Swing has been in use since 1998, has had close to a decade of debugging driven by actual field use, and (despite what personal misgivings the average Java programmer may have about building a "rich client" application instead of a browser-based one) can legitimately call itself a successful library in widespread use.

If memory serves, Brian's presentation noted that when run over the Swing code base, FindBugs discovered 70-something concurrency errors that remained in the code base, some since JDK 1.2. In close to a decade of use, 70-something concurrency bugs had been neither fixed nor found. Even if I misremember that number, and it is off by an order of magnitude--7 bugs instead of 70--the implication remains clear: simple usage cannot reveal all bugs.

The effect of this on the strength of the unit-testing argument cannot be understated--if the quality of dynamic-language-authored code rests on the exercise of that code under constrained circumstances in order to prove its correctness, then we have a major problem facing us, because the interleaving of execution paths that define concurrency bugs remain beyond the ability of most (arguably all) languages and/or platforms to control. It thus follows that concurrent code cannot be effectively unit tested, and thus the premise that dynamic-language-authored code can be proven correct by simple use of unit tests is flawed.

Some may take this argument to imply a rejection of unit tests. Those who do would be seeking evidence to support a position that is equally untenable, that unit-testing is somehow "wrong" or unnecessary. No such statement is implied; quite the opposite, in fact. We can neither reject the necessitary of unit testing any more than we can the beneficence of static analysis tools; each is, in its own way, incomplete in its ability to prove code correct, at least in current incarnations. In fact, although I will not speak for them, many of the Apostles of UnitTestitarianism in fact indirectly support this belief, arguing that unit tests do not obviate the need for a quality-analysis phase after a development iteration, because correct unit tests cannot imply completely correct code.

Currently much research is taking place in the statically-typed languages to make them more typesafe without sacrificing the productivity enhancements seen in the dynamically-typed language world. Scala, for example, makes heavy use of type-inferencing to reduce the burden of type declarations by the programmer, requiring such declarations only when the inferencing yields ambiguity. As a result, Scala's syntax--at a first glance--looks remarkably similar in places to Ruby's, yet the Scala compiler is still fully type-safe, ensuring that accidental coercion doesn't yield confusion. Microsoft is pursuing much the same route as part of their LINQ strategy for Visual Studio 2008/.NET 3.5 (the "Orcas" release), and some additional research is being done around functional languages in the form of F#.

At the end of the day, the fact remains that I write shitty code. That means I need all the help I can get--human-centric or otherwise--to make sure that my code is correct. I will take that help in the form of unit tests that I force myself (and others around me force myself) to write. But I also have to accept the possibility that my unit tests themselves may be flawed, and that therefore other tools--which do not suffer from my inherent human capacity to forget or brush off--are a powerful and necessary component of my toolbox.

ROTFL!

It seems that starting up a conversation about architecture is the topic of the month. Another email, this time from anattendee at a recent Denver Java Users Group meeting I spoke at a few weeks ago:

I just finished another interview to act as a consultant on a enterprise Java project at a large (Fortune 500) Denver company.  It went pretty well from a technical perspective, but I couldn't convince the interviewer of a few points.  I wanted your perspective so that I can share it at the Denver Open Source Users Group this Thursday with our members...

- Interviewer said that neither him nor his team attend any events in person (DJUG, Boulder Java Users Group, DOSUG, NFJS, ApacheCon).

- Interviewer said the budget wasn't there for these events.

- I pointed out that ones like DJUG were free.  He said "OH" like he didn't even know. (warning bells ringing in my head)

- I noted the value that (as you also stated) an attendee gets from not just the presentation, but the interaction with the speaker AND audience.  He was not impressed.

- I asked if he or anyone on his team had ever contributed to anything open source.  He said that none of his current employees or himself do.  But a guy that left last year did.  (warning bells ringing again)

- He said that they are primarily a consumer of open source, not a producer and that if they really wanted to change code, they would grow the tool in-house anyway.

- They are still on Junit 3.8 because they haven't had time to upgrade (this was a greenfield project as of 6 months ago)

- They are sticking with Ant because Maven looks daunting.

I made some attempts to probe why he thought this way, but of course, didn't make much headway.  I think I'll stay with my current contracts for a while longer.  This didn't appear to be "the place" I would want to work.

Probably a smart idea; as a general rule, if you strongly disagree with the software development practices in place at a particular company (or any part of the company's culture, for that matter), it's often not worth it to jump in--you just have to jump back out within six months or so. Granted, there are those who can stomach that kind of mismatch between internal values and external culture, but generally only by "checking out" of the environment around them and pursuing other things on the side. (As a side note, it seems to me that this "checking out" at work is probably the number one reason and source for developers working on open-source projects: if you're mentally and emotionally engaged at work, your desire to write code when you get home seems significantly diminished. It's almost like a drug addict's wet dream: getting high at work, so you don't have to bother with it at home.)

Anyway, the answers I'd love to hear your quick opinion on are:

1) Is this the norm for Fortune 500 sized companies (I've only got experience of a dataset of 10, and I'm sure you have more)

There's a conflation of topics here, so let's take them in order.

1. Do many/most F500 companies act as open-source consumer? Absolutely.
2. Do many/most F500 companies act as open-source producer? Not even close, but this makes sense. Most F500 companies are not in the business of contributing software; in fact, a significant majority of them don't make money on IT in the slightest, so it's not to their competitive advantage to offer up contributions to open source. (That's going to piss off the open-source zealots to no end, I'm sure, but in the end, which would you rather a drug manufacturer focus on: finding a cure to cancer, or patching security bugs in Tomcat? The benefit of an economy of specialization, is specialization. From each according to their abilities, and all....)
3. Do many/most developers at F500 companies not go to events like local JUGs or conferences like NFJS? Unfortunately, far too many developers don't treat their careers as a work-in-progress, but believe that once they've landed the job, they're on a track to moderate financial success for life. In fact, lots of developers fit the demographic of "male, 18-30, and single", which is the absolute worst demographic for future planning. Lots of these guys think that "Hey, I'm smart, the money will always flow in, right? Java/.NET/C++/COBOL will always be the tool of choice, right? What, me worry?" Unfortunately, it's my experience that it doesn't get better with age. Developers in the age range of 31 and up have seen one (or two) generations of languages/platforms go by, and have had to re-tool themselves, and are still pissed about it. The ones who realize that no matter how much they learn, there's still a lot more to take on, those are the ones going to JUGs and NFJS and TechEd and JavaOne and whatever else comes their way. Those are the same ones who see training classes as opportunities for advancement, not opportunities to play 8 hours of uninterrupted Solitaire. And, unfortunately, those developers are the exception, not the rule, it seems. Which means, if you take the time to invest in yourself, you will never be in the bottom of the candidate pool for your next job. Period.
4. Do many/most F500 projects stay with outdated things like JUnit 3.8? Honestly, this is the least of their sins. Over half of the code being written out there is being done without benefit of unit tests--frankly, the fact that they're using JUnit at all is a point in their favor, not against them. But I see your concern, the "we're frozen in time" syndrome when it comes to technology choices can often atrophy over long periods of time, and yes, that is a concern. But not in this case, no. If they're still using JDK 1.4 because they're still on WebSphere and have no plans to move forward until IBM makes them do it (because of end-of-support concerns--are you listening out there, VB6 world?), then yes, it's a concern. But only if they refuse to do the risk analysis of upgrading; if they do that analysis (and the architect or project manager should be able to tell you, if not show you, that analysis), and still decide to stay with what they've got for reasons they can explicate, then that's just being smart.
5. Do many/most F500 projects stick with Ant because Maven looks daunting? Dear Lord, man, I don't use Maven, for the same reason! Be careful not to project your own value judgements ("Maven is better than Ant!" "Maven is just a pile of crap on top of the goodness that is Ant!" and so on) on the company's decisions around build process. I've seen a few teams that used Maven and loved it, and I've seen a few teams that used Maven and hated it. I've seen some teams use Ant (or MSBuild) in ways that were simple, clean, and elegant, and I've seen Ant used in ways that would keep you up at night, shivering. (I was an accomplice to one of those latter Ant-trocides. Therapist says with time, I may be able to sleep through the night.) So long as a company can offer an articulate reason for their technology decisions, you can disagree with them if you wish, but you have to honor them. If you have serious concerns still, subvert them from within. Once you've lived their build process for a while, you'll know where the pain points are and can raise suggestions, perhaps suggesting Maven as a solution, perhaps not, to help address those pain points. But never forget, as the newly-installed outsider, your credibility will be at its lowest on the very first day you start there[1].

2) Would you also hear warning bells going off in your head after hearing this guy reply as he did above?

I hear warning bells all the time, but that's because I don't expect any place I go into to be perfect on all three axes of the triangle (People, Process, Technology). I'm looking for where the red flags are, because nobody's perfect. Having said that, it's critical for your sanity as a developer to know which of those three axes you need to be on the "high" end of the scale in order to preserve your own sanity, and which ones you can live without (or work to change).

3) Is it worth being an evangelist to folks like this convincing them that events AND contributing to open source are personal-improvement exercises that lead to better code and architecture at your main job?

Depends on a couple of factors. One, how much do you want to be an influencer? Some people derive great joy from changing the way a company does business, others see that as an impediment to their larger goal of "building cool stuff". And two, how much do you want to emotionally invest in this group? If you're just there for the short-term, certainly tell them and quote the statistics, but don't lose sleep over it. If you're really invested in this company (because of its location, the work its doing, you own the firm, whatever), then obviously put your heart out there on the line.

4) How far back on that continuum that we talked about at DJUG do the big companies have to be to start missing out on the early-adopter advantage? (1, 3 or 5 years back on mature technologies?)

That's a hard one to answer. "That continuum" is the technology-adoption continuum I mentioned in the JUG talk: every company is pegged somewhere on the technology adoption continuum, despite our tendencies to classify companies as either "(b)leading-edge/early-adopters" or "legacy players". Frankly, I think the hard part of that question is the phrase "mature technologies"--what's "mature" these days? What does the definition of "mature" mean for a technology? It's a hard thing to nail down, and until we do, I can't really answer the question. In the spirit of the question, though, I'd say that any firm who isn't at the very least watching forums like TheServerSide or InfoQ, and/or building a corporate bookshelf on a yearly/quarterly/monthly basis (and Safari, the joint Pearson/OReilly online book project is a HUGE win here for firms seeking to do this) is missing on golden opportunities to keep their ear to the ground. I'm not saying they need to adopt them right away, but every firm that writes software should have prototyped a Ruby/Rails app at this point, in order to get an idea of what Ruby and Rails bring to the playground, and whether or not they can play nicely with the rest of the firm's Java/.NET/C++/whatever apps.

[1] Ironically, the reverse is true of consultants--their credibility is at its highest on day one, and generally drops off from there, unless the consultant can prove his worth and earn more credibility. 'Tis horribly unfair, and exactly the reverse of what it should be, but I don't make these rules, I just observe them.

By the way, just a reminder, if you've got questions on architecture or other software-development-related topics that you'd love to see answered here, feel free to drop me a line: "ted@" this domain name. Or, blog them, and shoot me an email with the blog link.

This is not a title I convey lightly, but Michael Nygard's Release It! deserves the honor. It's the first book I've ever seen that addresses the issues of building software that's Production-friendly and sysadmin-approachable. He describes a series of antipatterns describing a variety of software failures, and offers up a series of solutions (patterns, if you will) to building software systems designed to combat said failures.

From the back cover:

Every website project is really an enterprise integration project: the stakes are high and the projects complex. In this world where good marketing can be fatal to yor website, where networks are unreliable, and where astronomically unlikely coincidences happen daily, you need all the help you can get.

...

You're a whiz at development. But 80% of typical project lifecyle cost can occur in production--not in development.

Although Michael's personal experience stems mostly from the Java space, the lessons and stories he offers up are equally relevant to Java, .NET, C++, Ruby, PHP, and any other language or platform you can imagine. Michael Nygard not only knows the Ten Fallacies of Enterprise Development, he breathes them.

Go. Now. Buy. Read. Don't write another line of code until you do.

I'm certain I'm not the first one to blog this, but I wanted to help fan the information out to the world: this email crossed my virtual desk today, and it indicates a subtle shift that many probably didn't see coming:

This morning we announced that later this year, with the final release of Visual Studio 2008, we will make available the source to much of the .NET Framework Libraries under the Microsoft Reference License.  This means that “anyone” who accepts the license will be able to browse and view source code.  The set of libraries initially includes  the Base Class Libraries (System namespace, IO, Text, Collections, CodeDom, Regular Expressions, etc), ASP.NET, WinForms, and WPF .  Microsoft will add to this list as time goes on.

There are two ways people will access the source code:

1) They will download a package with all the source, and then they will be able to install and browse locally

2) VS 2008 integration will enable developers to debug from their own source code into the .NET Framework source code.  We’ll provide symbols for our source on an internet-accessible source-server; to enable this experience, the developer needs to set up the URI for the server.  This second option is really cool, and enables a kick-ass developer experience.

We’re announcing this via Scott Guthrie’s blog, with screenshots of the VS experience.

Distributed with the email was a Q&A document, which (in the interests of minimizing the accusations of FUD on my part) I will paste in its entirety:

Reference Source Code Availability for the .NET Framework

--Microsoft Confidential until October 3^rd, 9:00am Pacific time--

Key Points

· Microsoft is releasing the source code for .NET Framework libraries under the Microsoft Reference License.  This license allows viewing of source code, but not modification or redistribution. The source code will be downloadable and viewable by anyone who accepts the license agreement. 

· Microsoft will introduce a capability in Visual Studio 2008 to allow .NET developers who are debugging applications, to debug not only into their own source code, but also into .NET Framework source code using Visual Studio. 

· This release falls under Microsoft’s Shared Source Initiative, which encompasses a spectrum of source code offerings, complementing the company’s other activities around sharing source code.  This is another example of Microsoft’s continued commitment to increasing transparency and addressing developer needs. 

Q&A

Q. What are you announcing?

Microsoft is releasing the source code for .NET Framework base class libraries under the Microsoft Reference License, which is available here: http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx .  This license allows viewing of source code, but not copying or re-compiling. The source code will be downloadable and viewable by anyone who accepts the license agreement. 

In addition, Microsoft will introduce a capability in Visual Studio 2008 to allow .NET developers who are debugging applications, to debug not only into their own source code, but also debug into the .NET Framework source code using Visual Studio.  With this capability, when developers are stepping through code, if they wish, they will be able to step into the source code for the .NET base class libraries.

Q. When will this become available to developers?

We expect the debugging capability to be available to .NET developers who use the RTM version of Visual Studio 2008, when Visual Studio 2008 releases. Our current plan is to release Visual Studio 2008 before the end of the 2007.

Developers will be able to download the source code package for the .NET libraries around the same time, or soon after.

Q: Which libraries are included in the source-code release?

This release will include the Base Class Libraries (BCL), Windows Forms, ASP.NET, System.Data, and WPF. BCL includes many of the basic classes in the framework including collections, string and text handling, IO,serialization, remoting, and others. We plan to include additional libraries into the set as time goes on.

Q: Why are you releasing only a subset of the .NET Framework libraries under the Microsoft Reference License?

Our intention is to make .NET Framework library source code available broadly, in response to customer demand. Before release under the Microsoft Reference License, each library is subject to a code review. As time allows, we will complete additional code reviews and release additional modules under this license.

Q: Will the libraries distributed under the Microsoft Reference License be limited to “managed code” libraries – in other words, .NET Framework libraries?

Microsoft’s intention in releasing developer libraries under the Microsoft Reference License (Ms-RL) is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code. Microsoft may release additional libraries under the license, potentially including unmanaged libraries, and these additional libraries would also deliver the same debugging experience described previously. Microsoft has no announcement at this time regarding additional libraries being released under the Microsoft Reference License.

Q: How does this relate to CodePlex, and other source-code sharing programs offered by Microsoft?

Microsoft has an initiative called the Shared Source Initiative which encompasses a spectrum of source code offerings.  At one end of the spectrum, Microsoft shares source code for commercial products like Windows and Office through programs offered to governments, enterprises, MVPs, OEMs, SIs and others.  On the other end of the spectrum, Microsoft encourages open source development through the CodePlex project management site, as well as direct contribution of source code to community projects on CodePlex.  Today, we’re announcing a new offering: the source code for the .NET Framework libraries will be available under the Microsoft Reference License. This offering will enable developers to view and even debug into the source code for the .NET Framework libraries.

Q: Allowing anyone to view the source code of a library – does this present a security risk?

The security of the .NET Framework does not depend on the obscurity of the .NET Framework source code. A realistic threat model must assume that any bad actor who wishes to subvert the security of any library, for practical purposes, has full access to the source code. This release will not change that.

Q: What do the developers need in order to take advantage of the debugging capability?

The debugging capability will be available to developers who use the RTM version of the Professional or Standard editions of Visual Studio 2008. Microsoft will also publish instructions for how to enable other .NET development or debugging tools to take advantage of the same capability. The Express versions of Visual Studio will not include this capability.

Q: What do the developers need in order to simply view the .NET source code?

Developers will need to download the source code package, and accept the license agreement before unpacking the archive. At that point, the source files will be viewable in any text editor.

Q. Why is Microsoft making this available now?

This is another step along the path of increasing transparency and addressing customer needs. Generally speaking, we are always looking for ways to continue to improve and enhance the developer experience for people who build applications using Microsoft tools. Delivering this particular capability for our customers requires changes in the developer tool itself, and the introduction of Visual Studio 2008 gives us the opportunity to do that.

Q. Why didn’t you do this sooner?

Our primary goal was to enable developers to build better applications. A new capability like this requires changes in the developer tool, and the release of Visual Studio 2008 presents us with an opportunity to deliver the right experience for developers. Maintaining the right developer experience, while preserving the security stability and serviceability of the .NET Framework, were paramount requirements for us. We believe we’ll have a simple and easy developer experience that will empower developers to build better more reliable applications. Having said this, the debugging experience is very transparent, and developers won’t even notice the change.

Q. Can developers modify and rebuild the .NET Framework source code, and redistribute the results of that action?

The source code of the developer libraries being made available, including the libraries in the .NET Framework, will be released under the Microsoft Reference License. http://ww