JOB REFERRALS
    ON THIS PAGE
    ARCHIVES
    CATEGORIES
    BLOGROLL
    LINKS
    SEARCH
    MY BOOKS
    DISCLAIMER
 
 Monday, October 29, 2007
Welcome to the Shitty Code Support Group

"Hi. My name's Ted, and I write shitty code."

With this opening, a group of us earlier this year opened a panel (back in March, as I recall) at the No Fluff Just Stuff conference in Minneapolis. Neal Ford started the idea, whispering it to me as we sat down for the panel, and I immediately followed his opening statement in the same vein. Poor Charles Nutter, who was new to the tour, didn't get the whispered-down-the-line instruction, and tried valiantly to recover the panel's apparent collective discard of dignity--"Hi, I'm Charles, and I write Ruby code"--to no avail. (He's since stopped trying.)

The reason for our declaration of impotent implementation, of course, was, as this post states so well, a Zen-like celebration of our inadequacies: To be a Great Programmer, you must admit that you are a Terrible Programmer.

To those who count themselves as the uninitiated into our particular brand of philosophy (or religion, or just plain weirdness), the declaration is a statement of anti-Perfectionism. "I am human, therefore I make mistakes. If I make mistakes, then I cannot assume that I will write code that has no mistakes. If I cannot write code that has no mistakes, then I must assume that mistakes are rampant within the code. If mistakes are rampant within the code, then I must find them. But because I make mistakes, then I must also assume that I make mistakes trying to identify the mistakes in the code. Therefore, I will seek the best support I can find in helping me find the mistakes in my code."

This support can come in a variety of forms. The Terrible Programmer cites several of his favorites: use of the Statically-Typed Language (in his case, Ada), Virulent Assertions, Testing Masochism, the Brutally Honest Code Review, and Zeal for the Visible Crash. Myself, I like to consider other tools as well: the Static Analysis Tool Suite, a Commitment to Confront the Uncomfortable Truth, and the Abject Rejection of Best Practices.

By this point in time, most developers have at least heard of, if not considered adoption of, the Masochistic Testing meme. Fellow NFJS'ers Stuart Halloway and Justin Gehtland have founded a consultancy firm, Relevance, that sets a high bar as a corporate cultural standard: 100% test coverage of their code. Neal Ford has reported that ThoughtWorks makes similar statements, though it's my understanding that clients sometimes put accidental obstacles in their way of achieving said goal. It's amibtious, but as the ancient American Indian proverb is said to state, If you aim your arrow at the sun, it will fly higher and father than if you aim it at the ground.

In fact, on a panel this weekend in Dallas, fellow NFJS'er David Bock attributed the rise of interest in dynamic languages to the growth of the unit-testing meme, since faith in the quality of code authored in a dynamic language can only follow if the code has been rigorously tested, since we have no tool checking the syntactic or semantic correctness before it is executed.

Among the jet-setting Ruby crowd, this means a near-slavish devotion to unit tests. Interestingly enough, I find this attitude curiously incomplete: if we assume that we make mistakes, and that we therefore require unit tests to prove to ourselves (and, by comfortable side effect, the world around us) that the code is correct, would we not also benefit from the automated--and in some ways far more comprehensive--checking that a static-analysis tool can provide? Stu Halloway once stated, "In five years, we will look back upon the Java compiler as a weak form of unit testing." I have no doubt that he's right--but I draw an entirely different conclusion from his statement: we need better static analysis tools, not to abandon them entirely. 

Consider, if you will, the static-analysis tool known as FindBugs. Fellow NFJS'er (and author of the JavaOne bestselling book Java Concurrency in Practice) Brian Goetz offered a presentation last year on the use of FindBugs in which he cited the use of the tool over the existing JDK Swing code base. Swing has been in use since 1998, has had close to a decade of debugging driven by actual field use, and (despite what personal misgivings the average Java programmer may have about building a "rich client" application instead of a browser-based one) can legitimately call itself a successful library in widespread use.

If memory serves, Brian's presentation noted that when run over the Swing code base, FindBugs discovered 70-something concurrency errors that remained in the code base, some since JDK 1.2. In close to a decade of use, 70-something concurrency bugs had been neither fixed nor found. Even if I misremember that number, and it is off by an order of magnitude--7 bugs instead of 70--the implication remains clear: simple usage cannot reveal all bugs.

The effect of this on the strength of the unit-testing argument cannot be understated--if the quality of dynamic-language-authored code rests on the exercise of that code under constrained circumstances in order to prove its correctness, then we have a major problem facing us, because the interleaving of execution paths that define concurrency bugs remain beyond the ability of most (arguably all) languages and/or platforms to control. It thus follows that concurrent code cannot be effectively unit tested, and thus the premise that dynamic-language-authored code can be proven correct by simple use of unit tests is flawed.

Some may take this argument to imply a rejection of unit tests. Those who do would be seeking evidence to support a position that is equally untenable, that unit-testing is somehow "wrong" or unnecessary. No such statement is implied; quite the opposite, in fact. We can neither reject the necessitary of unit testing any more than we can the beneficence of static analysis tools; each is, in its own way, incomplete in its ability to prove code correct, at least in current incarnations. In fact, although I will not speak for them, many of the Apostles of UnitTestitarianism in fact indirectly support this belief, arguing that unit tests do not obviate the need for a quality-analysis phase after a development iteration, because correct unit tests cannot imply completely correct code.

Currently much research is taking place in the statically-typed languages to make them more typesafe without sacrificing the productivity enhancements seen in the dynamically-typed language world. Scala, for example, makes heavy use of type-inferencing to reduce the burden of type declarations by the programmer, requiring such declarations only when the inferencing yields ambiguity. As a result, Scala's syntax--at a first glance--looks remarkably similar in places to Ruby's, yet the Scala compiler is still fully type-safe, ensuring that accidental coercion doesn't yield confusion. Microsoft is pursuing much the same route as part of their LINQ strategy for Visual Studio 2008/.NET 3.5 (the "Orcas" release), and some additional research is being done around functional languages in the form of F#.

At the end of the day, the fact remains that I write shitty code. That means I need all the help I can get--human-centric or otherwise--to make sure that my code is correct. I will take that help in the form of unit tests that I force myself (and others around me force myself) to write. But I also have to accept the possibility that my unit tests themselves may be flawed, and that therefore other tools--which do not suffer from my inherent human capacity to forget or brush off--are a powerful and necessary component of my toolbox.


.NET | C++ | Conferences | Development Processes | Java/J2EE | Ruby

Monday, October 29, 2007 5:12:32 PM (Pacific Daylight Time, UTC-07:00)
Comments [4]  | 
 Saturday, October 20, 2007
Link: The Last Language War/Language Trolling Post You'll Ever Need to Read

ROTFL!




Saturday, October 20, 2007 1:50:11 AM (Pacific Daylight Time, UTC-07:00)
Comments [1]  | 
 Thursday, October 18, 2007
A Conversation on Architecture

It seems that starting up a conversation about architecture is the topic of the month. Another email, this time from anattendee at a recent Denver Java Users Group meeting I spoke at a few weeks ago:

I just finished another interview to act as a consultant on a enterprise Java project at a large (Fortune 500) Denver company.  It went pretty well from a technical perspective, but I couldn't convince the interviewer of a few points.  I wanted your perspective so that I can share it at the Denver Open Source Users Group this Thursday with our members...

- Interviewer said that neither him nor his team attend any events in person (DJUG, Boulder Java Users Group, DOSUG, NFJS, ApacheCon).

- Interviewer said the budget wasn't there for these events.

- I pointed out that ones like DJUG were free.  He said "OH" like he didn't even know. (warning bells ringing in my head)

- I noted the value that (as you also stated) an attendee gets from not just the presentation, but the interaction with the speaker AND audience.  He was not impressed.

- I asked if he or anyone on his team had ever contributed to anything open source.  He said that none of his current employees or himself do.  But a guy that left last year did.  (warning bells ringing again)

- He said that they are primarily a consumer of open source, not a producer and that if they really wanted to change code, they would grow the tool in-house anyway.

- They are still on Junit 3.8 because they haven't had time to upgrade (this was a greenfield project as of 6 months ago)

- They are sticking with Ant because Maven looks daunting.

I made some attempts to probe why he thought this way, but of course, didn't make much headway.  I think I'll stay with my current contracts for a while longer.  This didn't appear to be "the place" I would want to work.

Probably a smart idea; as a general rule, if you strongly disagree with the software development practices in place at a particular company (or any part of the company's culture, for that matter), it's often not worth it to jump in--you just have to jump back out within six months or so. Granted, there are those who can stomach that kind of mismatch between internal values and external culture, but generally only by "checking out" of the environment around them and pursuing other things on the side. (As a side note, it seems to me that this "checking out" at work is probably the number one reason and source for developers working on open-source projects: if you're mentally and emotionally engaged at work, your desire to write code when you get home seems significantly diminished. It's almost like a drug addict's wet dream: getting high at work, so you don't have to bother with it at home.)

Anyway, the answers I'd love to hear your quick opinion on are:

1) Is this the norm for Fortune 500 sized companies (I've only got experience of a dataset of 10, and I'm sure you have more)

There's a conflation of topics here, so let's take them in order.

  1. Do many/most F500 companies act as open-source consumer? Absolutely.
  2. Do many/most F500 companies act as open-source producer? Not even close, but this makes sense. Most F500 companies are not in the business of contributing software; in fact, a significant majority of them don't make money on IT in the slightest, so it's not to their competitive advantage to offer up contributions to open source. (That's going to piss off the open-source zealots to no end, I'm sure, but in the end, which would you rather a drug manufacturer focus on: finding a cure to cancer, or patching security bugs in Tomcat? The benefit of an economy of specialization, is specialization. From each according to their abilities, and all....)
  3. Do many/most developers at F500 companies not go to events like local JUGs or conferences like NFJS? Unfortunately, far too many developers don't treat their careers as a work-in-progress, but believe that once they've landed the job, they're on a track to moderate financial success for life. In fact, lots of developers fit the demographic of "male, 18-30, and single", which is the absolute worst demographic for future planning. Lots of these guys think that "Hey, I'm smart, the money will always flow in, right? Java/.NET/C++/COBOL will always be the tool of choice, right? What, me worry?" Unfortunately, it's my experience that it doesn't get better with age. Developers in the age range of 31 and up have seen one (or two) generations of languages/platforms go by, and have had to re-tool themselves, and are still pissed about it. The ones who realize that no matter how much they learn, there's still a lot more to take on, those are the ones going to JUGs and NFJS and TechEd and JavaOne and whatever else comes their way. Those are the same ones who see training classes as opportunities for advancement, not opportunities to play 8 hours of uninterrupted Solitaire. And, unfortunately, those developers are the exception, not the rule, it seems. Which means, if you take the time to invest in yourself, you will never be in the bottom of the candidate pool for your next job. Period.
  4. Do many/most F500 projects stay with outdated things like JUnit 3.8? Honestly, this is the least of their sins. Over half of the code being written out there is being done without benefit of unit tests--frankly, the fact that they're using JUnit at all is a point in their favor, not against them. But I see your concern, the "we're frozen in time" syndrome when it comes to technology choices can often atrophy over long periods of time, and yes, that is a concern. But not in this case, no. If they're still using JDK 1.4 because they're still on WebSphere and have no plans to move forward until IBM makes them do it (because of end-of-support concerns--are you listening out there, VB6 world?), then yes, it's a concern. But only if they refuse to do the risk analysis of upgrading; if they do that analysis (and the architect or project manager should be able to tell you, if not show you, that analysis), and still decide to stay with what they've got for reasons they can explicate, then that's just being smart.
  5. Do many/most F500 projects stick with Ant because Maven looks daunting? Dear Lord, man, I don't use Maven, for the same reason! :-) Be careful not to project your own value judgements ("Maven is better than Ant!" "Maven is just a pile of crap on top of the goodness that is Ant!" and so on) on the company's decisions around build process. I've seen a few teams that used Maven and loved it, and I've seen a few teams that used Maven and hated it. I've seen some teams use Ant (or MSBuild) in ways that were simple, clean, and elegant, and I've seen Ant used in ways that would keep you up at night, shivering. (I was an accomplice to one of those latter Ant-trocides. Therapist says with time, I may be able to sleep through the night.) So long as a company can offer an articulate reason for their technology decisions, you can disagree with them if you wish, but you have to honor them. If you have serious concerns still, subvert them from within. Once you've lived their build process for a while, you'll know where the pain points are and can raise suggestions, perhaps suggesting Maven as a solution, perhaps not, to help address those pain points. But never forget, as the newly-installed outsider, your credibility will be at its lowest on the very first day you start there[1].

2) Would you also hear warning bells going off in your head after hearing this guy reply as he did above?

I hear warning bells all the time, but that's because I don't expect any place I go into to be perfect on all three axes of the triangle (People, Process, Technology). I'm looking for where the red flags are, because nobody's perfect. Having said that, it's critical for your sanity as a developer to know which of those three axes you need to be on the "high" end of the scale in order to preserve your own sanity, and which ones you can live without (or work to change).

3) Is it worth being an evangelist to folks like this convincing them that events AND contributing to open source are personal-improvement exercises that lead to better code and architecture at your main job?

Depends on a couple of factors. One, how much do you want to be an influencer? Some people derive great joy from changing the way a company does business, others see that as an impediment to their larger goal of "building cool stuff". And two, how much do you want to emotionally invest in this group? If you're just there for the short-term, certainly tell them and quote the statistics, but don't lose sleep over it. If you're really invested in this company (because of its location, the work its doing, you own the firm, whatever), then obviously put your heart out there on the line.

4) How far back on that continuum that we talked about at DJUG do the big companies have to be to start missing out on the early-adopter advantage? (1, 3 or 5 years back on mature technologies?)

That's a hard one to answer. "That continuum" is the technology-adoption continuum I mentioned in the JUG talk: every company is pegged somewhere on the technology adoption continuum, despite our tendencies to classify companies as either "(b)leading-edge/early-adopters" or "legacy players". Frankly, I think the hard part of that question is the phrase "mature technologies"--what's "mature" these days? What does the definition of "mature" mean for a technology? It's a hard thing to nail down, and until we do, I can't really answer the question. In the spirit of the question, though, I'd say that any firm who isn't at the very least watching forums like TheServerSide or InfoQ, and/or building a corporate bookshelf on a yearly/quarterly/monthly basis (and Safari, the joint Pearson/OReilly online book project is a HUGE win here for firms seeking to do this) is missing on golden opportunities to keep their ear to the ground. I'm not saying they need to adopt them right away, but every firm that writes software should have prototyped a Ruby/Rails app at this point, in order to get an idea of what Ruby and Rails bring to the playground, and whether or not they can play nicely with the rest of the firm's Java/.NET/C++/whatever apps.

 

[1] Ironically, the reverse is true of consultants--their credibility is at its highest on day one, and generally drops off from there, unless the consultant can prove his worth and earn more credibility. 'Tis horribly unfair, and exactly the reverse of what it should be, but I don't make these rules, I just observe them.


By the way, just a reminder, if you've got questions on architecture or other software-development-related topics that you'd love to see answered here, feel free to drop me a line: "ted@" this domain name. Or, blog them, and shoot me an email with the blog link.




Thursday, October 18, 2007 11:33:30 PM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
 Sunday, October 07, 2007
A Book Every Developer Must Read

This is not a title I convey lightly, but Michael Nygard's Release It! deserves the honor. It's the first book I've ever seen that addresses the issues of building software that's Production-friendly and sysadmin-approachable. He describes a series of antipatterns describing a variety of software failures, and offers up a series of solutions (patterns, if you will) to building software systems designed to combat said failures.

From the back cover:

Every website project is really an enterprise integration project: the stakes are high and the projects complex. In this world where good marketing can be fatal to yor website, where networks are unreliable, and where astronomically unlikely coincidences happen daily, you need all the help you can get.

...

You're a whiz at development. But 80% of typical project lifecyle cost can occur in production--not in development.

Although Michael's personal experience stems mostly from the Java space, the lessons and stories he offers up are equally relevant to Java, .NET, C++, Ruby, PHP, and any other language or platform you can imagine. Michael Nygard not only knows the Ten Fallacies of Enterprise Development, he breathes them.

Go. Now. Buy. Read. Don't write another line of code until you do.


.NET | C++ | Development Processes | Java/J2EE | Reading | Ruby | Windows | XML Services

Sunday, October 07, 2007 5:41:29 PM (Pacific Daylight Time, UTC-07:00)
Comments [3]  | 
 Wednesday, October 03, 2007
Reports of Snowballs In Hell...

I'm certain I'm not the first one to blog this, but I wanted to help fan the information out to the world: this email crossed my virtual desk today, and it indicates a subtle shift that many probably didn't see coming:

This morning we announced that later this year, with the final release of Visual Studio 2008, we will make available the source to much of the .NET Framework Libraries under the Microsoft Reference License.  This means that “anyone” who accepts the license will be able to browse and view source code.  The set of libraries initially includes  the Base Class Libraries (System namespace, IO, Text, Collections, CodeDom, Regular Expressions, etc), ASP.NET, WinForms, and WPF .  Microsoft will add to this list as time goes on.

There are two ways people will access the source code:

1) They will download a package with all the source, and then they will be able to install and browse locally

2) VS 2008 integration will enable developers to debug from their own source code into the .NET Framework source code.  We’ll provide symbols for our source on an internet-accessible source-server; to enable this experience, the developer needs to set up the URI for the server.  This second option is really cool, and enables a kick-ass developer experience.

We’re announcing this via Scott Guthrie’s blog, with screenshots of the VS experience.

Distributed with the email was a Q&A document, which (in the interests of minimizing the accusations of FUD on my part) I will paste in its entirety:

Reference Source Code Availability for the .NET Framework

--Microsoft Confidential until October 3rd, 9:00am Pacific time--

Key Points

· Microsoft is releasing the source code for .NET Framework libraries under the Microsoft Reference License.  This license allows viewing of source code, but not modification or redistribution. The source code will be downloadable and viewable by anyone who accepts the license agreement. 

· Microsoft will introduce a capability in Visual Studio 2008 to allow .NET developers who are debugging applications, to debug not only into their own source code, but also into .NET Framework source code using Visual Studio. 

· This release falls under Microsoft’s Shared Source Initiative, which encompasses a spectrum of source code offerings, complementing the company’s other activities around sharing source code.  This is another example of Microsoft’s continued commitment to increasing transparency and addressing developer needs. 

Q&A

Q. What are you announcing?

Microsoft is releasing the source code for .NET Framework base class libraries under the Microsoft Reference License, which is available here: http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx .  This license allows viewing of source code, but not copying or re-compiling. The source code will be downloadable and viewable by anyone who accepts the license agreement. 

In addition, Microsoft will introduce a capability in Visual Studio 2008 to allow .NET developers who are debugging applications, to debug not only into their own source code, but also debug into the .NET Framework source code using Visual Studio.  With this capability, when developers are stepping through code, if they wish, they will be able to step into the source code for the .NET base class libraries.

Q. When will this become available to developers?

We expect the debugging capability to be available to .NET developers who use the RTM version of Visual Studio 2008, when Visual Studio 2008 releases. Our current plan is to release Visual Studio 2008 before the end of the 2007.

Developers will be able to download the source code package for the .NET libraries around the same time, or soon after.

Q: Which libraries are included in the source-code release?

This release will include the Base Class Libraries (BCL), Windows Forms, ASP.NET, System.Data, and WPF. BCL includes many of the basic classes in the framework including collections, string and text handling, IO,serialization, remoting, and others. We plan to include additional libraries into the set as time goes on.

Q: Why are you releasing only a subset of the .NET Framework libraries under the Microsoft Reference License?

Our intention is to make .NET Framework library source code available broadly, in response to customer demand. Before release under the Microsoft Reference License, each library is subject to a code review. As time allows, we will complete additional code reviews and release additional modules under this license.

Q: Will the libraries distributed under the Microsoft Reference License be limited to “managed code” libraries – in other words, .NET Framework libraries?

Microsoft’s intention in releasing developer libraries under the Microsoft Reference License (Ms-RL) is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code. Microsoft may release additional libraries under the license, potentially including unmanaged libraries, and these additional libraries would also deliver the same debugging experience described previously. Microsoft has no announcement at this time regarding additional libraries being released under the Microsoft Reference License.

Q: How does this relate to CodePlex, and other source-code sharing programs offered by Microsoft?

Microsoft has an initiative called the Shared Source Initiative which encompasses a spectrum of source code offerings.  At one end of the spectrum, Microsoft shares source code for commercial products like Windows and Office through programs offered to governments, enterprises, MVPs, OEMs, SIs and others.  On the other end of the spectrum, Microsoft encourages open source development through the CodePlex project management site, as well as direct contribution of source code to community projects on CodePlex.  Today, we’re announcing a new offering: the source code for the .NET Framework libraries will be available under the Microsoft Reference License. This offering will enable developers to view and even debug into the source code for the .NET Framework libraries.

Q: Allowing anyone to view the source code of a library – does this present a security risk?

The security of the .NET Framework does not depend on the obscurity of the .NET Framework source code. A realistic threat model must assume that any bad actor who wishes to subvert the security of any library, for practical purposes, has full access to the source code. This release will not change that.

Q: What do the developers need in order to take advantage of the debugging capability?

The debugging capability will be available to developers who use the RTM version of the Professional or Standard editions of Visual Studio 2008. Microsoft will also publish instructions for how to enable other .NET development or debugging tools to take advantage of the same capability. The Express versions of Visual Studio will not include this capability.

Q: What do the developers need in order to simply view the .NET source code?

Developers will need to download the source code package, and accept the license agreement before unpacking the archive. At that point, the source files will be viewable in any text editor.

Q. Why is Microsoft making this available now?

This is another step along the path of increasing transparency and addressing customer needs. Generally speaking, we are always looking for ways to continue to improve and enhance the developer experience for people who build applications using Microsoft tools. Delivering this particular capability for our customers requires changes in the developer tool itself, and the introduction of Visual Studio 2008 gives us the opportunity to do that.

Q. Why didn’t you do this sooner?

Our primary goal was to enable developers to build better applications. A new capability like this requires changes in the developer tool, and the release of Visual Studio 2008 presents us with an opportunity to deliver the right experience for developers. Maintaining the right developer experience, while preserving the security stability and serviceability of the .NET Framework, were paramount requirements for us. We believe we’ll have a simple and easy developer experience that will empower developers to build better more reliable applications. Having said this, the debugging experience is very transparent, and developers won’t even notice the change.

Q. Can developers modify and rebuild the .NET Framework source code, and redistribute the results of that action?

The source code of the developer libraries being made available, including the libraries in the .NET Framework, will be released under the Microsoft Reference License. http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx

The Microsoft Reference License allows viewing of source code for reference purposes, but does not allow editing, copying, or rebuilding. Microsoft’s intention in releasing developer libraries under this license is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code.

Q. How is the license accepted?

The license is accepted differently depending on how the person accesses the source. If it is through the VS2008 experience, there is a popup “do you accept this license?” dialog. If it is through the source package, then the MSI installer will require the person to accept the license before installing.

Q. Is this an open-source release?

Microsoft believes that a consistent framework for releasing source code – one the community can rely on – delivers the best value for customers and minimizes confusion. This is why use a set of source licenses, with clearly differentiated purposes, for source releases. (See http://www.microsoft.com/resources/sharedsource/licensingbasics/sharedsourcelicenses.mspx ) This particular release is being offered under the Microsoft Reference License.

Q. {Java,Ruby, Python} is truly open source and lets me rebuild the source code. Why can I not do this with .NET?

We’re focused on delivering value to developers. With the .NET Framework library source code, our goal is to enable developers to view the source of the platform, learn from that, and potentially provide feedback into Microsoft. (This would commonly be done via the Product Feedback center Microsoft has used for years – see http://connect.microsoft.com/feedback/default.aspx?SiteID=210 ) The Visual Studio debugging experience extends that view capability to a debugging session. With this capability, we think we’re delivering an easy, seamless experience for developers, and good value to the community.

Allowing developers to rebuild a framework, any framework, from source, and then redistribute the modified result, can introduce problems with providing support, serviceability, integrity and security of the framework itself. This is true with .NET, Java, Ruby, and other frameworks. Multiple independent redistributed modified versions of a framework can decrease the reliability and dependability of the common platform, which is not desirable.

In the path we’ve taken here, we are seeking to balance the requirements we hear from the developer community for transparency and of reliability and dependability of the platform. In striking that balance, we believe the Microsoft Reference License is the right license for this release.

Q. How does the debugging experience really work? Will the source code remain resident on the machine?

Visual Studio will request source from a Microsoft (MSDN) server as needed when a developer steps into .NET Framework library code during in a debugging session. In more detail, the debugger queries the remote server for the version of symbols that matches the binaries used in the application being debugged. If the debugger finds the correct version of symbols, it downloads the source files on-demand as the developer debugs into them. In all cases, the first download of source code requires explicit acceptance of the license. This is the same flow works for all source code that we will add in the future to this program, whether managed, unmanaged, 32 bit, 64 bit, desktop, mobile, and so on.

Q. What if a company does not want to allow the .NET Framework source code or other library source code, to be viewed by its developers?

Viewing or debugging into the Microsoft library source code is an action that will require an explicit acceptance of the license, regardless of the mechanism by which developers view the code (whether in a developer tool like Visual Studio, or as a separate archive download). Companies that do not wish to view the source code should instruct their employees to not accept this license.

Q. What are the terms of support with this source code? What if the developer finds a bug in the code?

The license spells out what the developer should expect. Essentially, people can look at the source code, but they may not copy it. If a person finds what they believe to be a bug in the code, Microsoft would encourage the person to submit feedback via the product feedback center, see http://connect.microsoft.com/feedback/default.aspx?SiteID=210 .

As you can well see, there's a lot of opportunities for FUD here--the Q&A document makes it clear that the source license is a "read-only" license, in that you can't recompile the code yourself. However, for those who are hell-bent on doing so, two options are available to you:

  1. Build the code without telling Microsoft. This is not a trivial undertaking, and I'd be shocked if it turns out to be as simple as "create a solution, import the files, press the big green button". It may not turn out to be worth the effort.
  2. Download Rotor Whidbey, also known as SSCLI 2.0, and build that, instead. It won't have ASP.NET, WinForms, or WPF, but it will have much of the "core" FCL, and the core CLR (modulo GC and JIT, which Microsoft re-wrote for Rotor v1) to boot.

Speaking of the latter, Joel Pobar and I are doing a revision of the SSCLI Essentials book to bring it up to date with the Rotor Whidbey sources; I'm doing an editing pass on the chapters now, and we'll be hunting up reviewers to take a look probably before the end of the month. (If you're interested, you know how to get a hold of me.)

Some additional comments to go along with the Q&A from above:

  • "Allowing anyone to view the source code of a library – does this present a security risk? The security of the .NET Framework does not depend on the obscurity of the .NET Framework source code. A realistic threat model must assume that any bad actor who wishes to subvert the security of any library, for practical purposes, has full access to the source code." Amen, brother! Sing it loud, sing it proud: obscurity is not security!
  • "Why is Microsoft making this available now? This is another step along the path of increasing transparency and addressing customer needs. Generally speaking, we are always looking for ways to continue to improve and enhance the developer experience for people who build applications using Microsoft tools. Delivering this particular capability for our customers requires changes in the developer tool itself, and the introduction of Visual Studio 2008 gives us the opportunity to do that." There's a small amount of shuffle-and-dance going on here; to allow developers to debug .NET in the manner Microsoft describes above--Visual Studio fetching symbols and/or source from MSDN when the developer steps-in to the source--did, probably, require a tool change. Just using the existing facility to talk to a symbol server over the Internet would most likely not be a great experience, though I'd defer to John Robbins on that one. But nothing prevented them from making sources available in the traditional (a la SSCLI) manner (a big tarball) long before this. Instead, I think what we're seeing is something Stu Halloway predicted three years ago at a No Fluff Just Stuff speaker panel: "In five years, Microsoft will be the biggest open-source company on the planet". I think Microsoft just needed to figure out how it wanted to embrace open source, because at the end of the day, Microsoft needs to make money, and it wasn't sure how to do open source and make money at the same time--after all, the "golden bailout" that both Red Hat and JBoss both used won't be available to Microsoft.
  • "Can developers modify and rebuild the .NET Framework source code, and redistribute the results of that action? The source code of the developer libraries being made available, including the libraries in the .NET Framework, will be released under the Microsoft Reference License. http://www.microsoft.com/resources/sharedsource/licensingbasics/referencelicense.mspx

    The Microsoft Reference License allows viewing of source code for reference purposes, but does not allow editing, copying, or rebuilding. Microsoft’s intention in releasing developer libraries under this license is to provide transparency and allow developers to more deeply understand the inner workings of the source code. Developers who better understand the source code will be more effective in writing software that makes use of the licensed source code." In other words, no, you can't make a forked copy of the library and ship it yourself. At least, not for now. Mark my words, though, Microsoft will pay very close attention to this issue, and if customers find themselves wanting/needing to do so, Microsoft will find a way to make this work. It just may not be in the same manner that traditional open-source projects would do so. (Remember, Microsoft is very aware of their bottom line, and they need to tread lightly anywhere the goals of "making money" might possibly be threatened or thrown for a loop. They're not risk-avoidant, but they aren't going to take risks unnecessarily, either.)

  • "{Java,Ruby, Python} is truly open source and lets me rebuild the source code. Why can I not do this with .NET?" This is an interesting and tricky subject. On the one hand, allowing developers "full reign" over the source provides the greatest amount of power and flexibility, and opens up the possibility of customizing the .NET Framework to behave exactly the way you want it to. On the other hand, being able to do that also means that the versioning problem just took on a whole new order of magnitude of complexity, because now you have some serious problems regarding the FCL assemblies. Remember, the FCL assemblies are signed with Microsoft's private key, so either Microsoft has to release that in order to allow you to rebuild the assemblies to match up with the ones already built by Microsoft (and we already know that releasing a private key is a Bad Thing, right?), or your custom-built FCL assemblies would be built with your own private key, which means now that your library is only used when code is compiled directly against it--and what happens when you build your application with a third-party library which itself was compiled against Microsoft's code? Trust me on this--the resulting picture is not pretty. Don't imagine that you have answers to this problem, either: it's a tricky problem, one that no programming platform has solved in a transparent and universal manner. Where a solution might work for your particular situation, be assured that it'll fail for somebody else's.

 The one thing I wish they'd do, and I can't tell from the announcement or the Q&A document if they will, is release the sources in a single tarball for download and offline use; I don't want to have to be tethered to an Internet connection all the time while debugging code. (Remember that the first Fallacy, "The network is always available", doesn't apply at 39,000 feet.)

All in all, this makes the latest marker in Microsoft's slow-but-steady shift towards a more open-source-friendly position. Whether or not the developer community meets them halfway, of course, resides entirely with your point of view on the statement, "Microsoft is an evil empire that does nothing good for anybody but themselves". If you agree with that statement, you and the other Microsoft Conspiracy Theorists will probably conclude that Lee Harvey Oswald was not only not alone, but he was paid by Microsoft. If you dsiagree with that statement, then you and the other Microsoft-sponsored Microsoft Certified Partners, Microsoft-sponsored Regional Directors, Microsoft-sponsored Most Valuable Professionals, and Microsoft-sponsored development teams will be happy to note that the checks still cash despite this move to open source. (And yes, there's likely to be a few of you out there who are somewhere in the middle on this, and you will probably get killed in the crossfire between those two rival gangs as a result of all this.)




Wednesday, October 03, 2007 10:43:49 AM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
 Tuesday, October 02, 2007
Wow.

I just got back from the No Fluff Just Stuff show in St Louis, where I gave my "Why the Next Five Years Will Be About Programming Languages" keynote, and a fellow speaker emailed me to point out the Code To Joy blog, which says some things that were... um... well, rather than try and select an adjective, I'll let you look for yourself:

Ted Neward talked about how the next 5 years will be about languages. (Fellow speaker Alex Miller has a post which contains a link to a similar talk and some of his own commentary).

Ted's thesis was that the Big Runtimes (the JVM and CLR) act as a bridge between the academic language lawyers and the problem solvers in the field: the academics can experiment with language syntax and exotic semantics; the problem solvers can use these languages and rely on the venerable runtime libraries to get things done.

I enjoyed the talk thoroughly. Ted paced to-and-fro like a giant cat: if NFJS is Narnia, then Ted is Aslan, except very hungry, and with a mild-case of rabies. Many good jabs toward, well, everybody. He's an equal-opportunity offender (and a big softie underneath it all.)

A giant cat. A very hungry giant cat. A very hungry giant cat with a mild case of rabies.

*sniff* It's like I can retire now. *sniff* I'd like to thank the Academy, and my parents, and ... :-)




Tuesday, October 02, 2007 8:44:00 PM (Pacific Daylight Time, UTC-07:00)
Comments [4]  |