So the December CTP of $g{Vista} is available, and I'm finding a recurring problem trying to install it into VMWare. When I tried this with the Beta1 of Vista into VMWare 5.0, the same problem occurs as what I'm getting now: when trying to boot off of the .ISO inside of VMWare (in other words, not physically off a burned CD/DVD, but out of the .ISO image itself mapped into the virtual CD in VMWare), I get a BSOD every... single... time. Is this indicative of a f-ed up installation of my VMWare image, is this a known bug in Vista (sidenote: yes, I know about the raw partition issues with Vista, but doing an XP-first install of Vista gives the same BSOD on reboot), or is it just the combination of ISO-image-and-VMWare? Anybody got ideas for a workaround or a fix?

I've been contacted by a third party (not a recruiting agency) who's having a hard time finding a J2EE architect with familiarity/expertise with concepts of architecture and the low-level chops not to lose sight of the code. To put it in their words:

This is strongly influenced by component-based methodologies such as OpenDoc, but would be extending the RCP Platform developed by Eclipse, and encompassing other concepts from systems such as Squeak... We are having trouble finding someone senior enough to understand the conceptual architectural issues, but technical enough to know important lower-level details such as how design choices will effect performance and scalability.

By the way, DON'T send me your resumes--in your email to me, tell me what your favorite alternative language or platform is, and why. I'll use that to know if you're somewhere in the ballpark.

And no, job postings are NOT going to become a regular part of this blog; I'm doing this as a once-off. (My motivation? I just want to help these guys; this isn't an easy requirement to fill, and I'm hoping I get to meet whomever it is they end up selecting, 'cuz I'm thinking they'll be a kindred spirit. )

Update: They sent me a formal job description, and it seems the desires changed just a bit. *shrug* See for yourself:

A lead software architect for [[name snipped]]. Applicant must be able to take a high level conceptual framework and architect a conceptually sound implementation using open standards in collaboration with an existing team. Experience with Java, Eclipse, and the Rich Client Platform (RCP) sub-project in Eclipse is a significant advantage. Experience with Aspect Oriented Programming is a plus. Participation in the open standards community a plus (e.g. W3C and Eclipse standards communities). Five or more years of experience required with experience as a lead software architect, industry experience preferred. Opening is for a full-time onsite position.

Jim Hugunin strikes again: IronPython 1.0 beta 1 is out. I think Jim should get an award for "first language designer to operate on both the JVM and CLR". Now he just needs to do one more implementation to get the hat trick.

Flame away...

The Pragmatic guys are at it again... This time it's a whole course, taught by two of the finest instructors I have had the privilege to know (and, quite honestly, argue with), on everybody's favorite presentation-layer hot topic, Ajax.

By the way, dear audience, this is one class you can attend regardless of which camp you prefer--both Stu and Justin are equally adept on both enterprise platforms (Java and .NET) and the new hot language, as is clear when they say that they will show you how "to use frameworks such as Rails, Spring, and ASP.NET"; Justin, for example, co-authored "Better, Faster, Lighter Java" and the "Spring Developer's Handbook", as well as built DevelopMentor's ASP.NET website and infrastructure. Stu was one of the COM cognoscenti back in the day (his poems on the subject (towards the bottom, search for Stu) are legend), and then took over as Java curriculum lead when DevelopMentor... well, created a Java curriculum. Two brighter guys--and better instructors--you're not likely to meet.

Oh, and, uh, they seem to know a fair amount about Ajax, too. Enough that I'd attend the course, were I not already busy that week.... Do yourself the favor, if you want to know more about Ajax, go see them. I can think of a lot worse ways to spend a grand in cash and 3 days...

An interesting question emerged during a discussion with some buddies/co-workers/peers/whatever-you-want-to-call-them today:

"Which conferences have you attended in the past that you thought was really good, and why? Which sessions were your favorites, and why? What made them that way?"

Yes, this is somewhat selfish, since the new conference season is amping up, and obviously I'd like to make sure my sessions are ones that people find interesting and recommend to others, but the question actually stemmed from an unrelated discussion to that. I promise.

Microsoft has done it again--this time, they're previewing the next release of C# planned for after LINQ/C# 3.0. They call it, for obvious reasons, the YODA programming langauge. Judge me by my size, do you? As well you should not, for my ally is the Source, and a powerful ally it is, indeed.

Remember: A Jedi uses the Source only for knowledge and defense, never for a hack.

In a recent thread on TheServerSide.com, Rick Hightower, a fellow NFJS speaker, commented on the JSR-175/annotations specification, and I felt a little obligated to respond, since this is a common critique/criticism:

Why don't you like the implementation? I hate the fact that your code has to import the annotations and then your code is tied to the annotation. It does not seem that different than depending on a interface (i.e., a marker interface). I'd like to see a soft import for annotations that does not impact compilation. (from TheServerSide.com)

The basic problem was this: if the compiler runs across an annotation, and it doesn't match an annotation type defined anywhere in the compilation classpath of imported symbols, what is the proper behavior? In any other scenario, such as:

public class App
{
  public static main(String[] args)
  {
    Systme.out.println("Hello, world!");
  }
}

public class App extends Object
{
  @Overide public String toString(Object obj) { return "App"; }
}

The big IDE vendors were particularly upset at this idea, leading one to claim, "If we cannot solve this problem we will consider JSR 175 to have been a failure." Unfortunately, then, we failed--there is no good way to solve this problem without breaking the fundamental vision of the Java platform. We tried a variety of ideas, including a few centered around the JDK 1.4 assertion idea (some kind of runtime flag indicating which annotations were safe to ignore), but couldn't work out the basic semantics of such without requiring a definition of the annotation to be present on the compilation path. And frankly, in the grander scheme of things, it makes sense to me that annotations ARE required at compilation time--just as interfaces, helper classes, member field types, and other types are required to be present at compilation time, as well.

Rod Johnson continued his critique of annotations by citing the following two reasons:

1. No proper mechanism for overriding annotations at runtime, despite the fact that just about any framework that uses annotations is going to need to consider doing that.
2. Inability for an annotation to extend an existing interface (even if that interface is simple enough to sit within annotations). Of course there are implementation issues around this one, I guess. But it means that it's hard to avoid code duplication when working with annotations and alternative metadata sources--something that's going to be particularly important until everyone and their dog uses Java 5, and anyway will remain important to work with existing code that may not have the right annotation.

Nutshell version of all this, the JSR 175 EG did, in fact, think long and hard about "soft imports" and "runtime annotation modification", and both ideas were shot down for what we felt were good reasons.

In keeping with the tradition, I'm suggesting the following will take place for 2006:

1. The hype surrounding Ajax will slowly fade, as people come to realize that there's really nothing new here, just that DHTML is cool again. As Dion points out, Ajax will become a toolbox that you use in web development without thinking that "I am doing Ajax". Just as we don't think about "doing HTML" vs "doing DOM".
2. The release of EJB 3 may actually start people thinking about EJB again, but hopefully this time in a more pragmatic and less hype-driven fashion. (Yes, EJB does have its place in the world, folks--it's just a much smaller place than most of the EJB vendors and book authors wanted it to be.)
3. Vista will be slipped to 2007, despite Microsoft's best efforts. In the meantime, however, WinFX (which is effectively .NET 3.0) will ship, and people will discover that Workflow (WWF) is by far the more interesting of the WPF/WCF/WWF triplet. Notice that I don't say "powerful" or "important", but "interesting".
4. Scripting languages will hit their peak interest period in 2006; Ruby conversions will be at its apogee, and its likely that somewhere in the latter half of 2006 we'll hear about the first major Ruby project failure, most likely from a large consulting firm that tries to duplicate the success of Ruby's evangelists (Dave Thomas, David Geary, and the other Rubyists I know of from the NFJS tour) by throwing Ruby at a project without really understanding it. In other words, same story, different technology, same result. By 2007 the Ruby Backlash will have begun.
5. Interest in building languages that somehow bridge the gap between static and dynamic languages will start to grow, most likely beginning with E4X, the variant of ECMAScript (Javascript to those of you unfamiliar with the standards) that integrates XML into the language.
6. Java developers will start gaining interest in building rich Java apps again. (Freely admit, this is a long shot, but the work being done by the Swing researchers at Sun, not least of which is Romain Guy, will by the middle of 2006 probably be ready for prime-time consumption, and there's some seriously interesting sh*t in there.)
7. Somebody at Microsoft starts seriously hammering on the CLR team to support continuations. Talk emerges about supporting it in the 4.0 (post-WinFX) release.
8. Effective Java (2nd Edition) will ship. (Hardly a difficult prediction to make--Josh said as much in the Javapolis interview I did with him and Neal Gafter.)
9. Effective .NET will ship.
10. Pragmatic XML Services will ship.
11. JDK 6 will ship, and a good chunk of the Java community self-proclaimed experts and cognoscente will claim it sucks.
12. Java developers will seriously begin to talk about what changes we want/need to Java for JDK 7 ("Dolphin"). Lots of ideas will be put forth. Hopefully most will be shot down. With any luck, Joshua Bloch and Neal Gafter will still be involved in the process, and will keep tight rein on the more... aggressive... ideas and turn them into useful things that won't break the spirit of the platform.
13. My long-shot hope, rather than prediction, for 2006: Sun comes to realize that the Java platform isn't about the language, but the platform, and begin to give serious credence and hope behind a multi-linguistic JVM ecosystem.
14. My long-shot dream: JBoss goes out of business, the JBoss source code goes back to being maintained by developers whose principal interest is in maintaining open-source projects rather than making money, and it all gets folded together with what the Geronimo folks are doing. In other words, the open-source community stops the infighting and starts pulling oars in the same direction at the same time. For once.

Whilst perusing the latest VMWare Workstation offering from their website, I noticed that not only does VMWare offer a free VMWare player (in other words, take a VMWare disk image created by somebody else and use it), but the VMWare site also has links to various pre-built VMWare disk images, including one for BEA's complete WebLogic 8.1 environment.... Whoever thought this idea up deserves to be knighted--what a great way to make it trivially simple for somebody to get started with a rather intimidating task (be that either installing a new O/S or a new app server).

Are you listening, Microsofties? VPCs of Vista, Visual Studio Team System and, heck, even just a base Visual Studio Express (pick a language, C# and VB sound like good starters) image are definitely something to consider if you want to make it easy for dev's to play with your tools.... Particularly people who DON'T want to install Windows just to play with Microsoft's implementation of .NET....

If you read no other book this coming year, you must read "Blink", by Malcolm Gladwell, the same author who wrote "The Tipping Point" (which is about why certain trends seem to just "take off" with no prior warning--case in point, the incredible rise of certain fashion trends, such as "Hush Puppies")..

I won't tell you what it's about except to quote the back cover; to do so would ruin the book's effect, to be blunt. The inside jacket reads,

In his landmark bestseller The Tipping Point, Malcolm Gladwell redefined how we understand the world around us. Now, in Blink, he revolutionizes the way we understand the world within. Blink is a book about how we think without thinking, about choices that seem to be made in an instance--in a blink of an eye--that actually aren't as simple as they seem. Why are some people brilliant decision-makers, while others are consistently inept? Why do some people follow their instincts and win, while others end up stumbling into error? How do our brains really work--in the office, in the classroom, in the kitchen and in the bedroom? And why are the best decisions often the ones that are impossible to explain to others?

In Blink we meet the psychologist who has learned to predict whether a marriage will last, based on a few minutes of observing a couple; the tennis coach who knows when a player will double-fault before the racket even makes contact with the ball; the antiquities experts who recognize a fake at a glance. Here, too, are great failures of "blink": the election of Warren Harding; New Coke; and the shooting of Amadou Diallo by police. Blink reveals that great decision makers aren't those who possess the most information or spend the most time deliberating, but those who have perfected the art of "thin-slicing"--filtering the very few factors that matter from an overwhelming number of variables.

Drawing on cutting-edge neuroscience and psychology and displaying all of the brilliance that made The Tipping Point a classic, Blink changes the way you understand every decision you make. Never again will you think about thinking the same way.

You make the call on this one... cut & pasted directly out of the email (after the horizonal rule):

Mark Michaelis posted a challenge: modify a string such that the following would print "Smile":

class Program
{
  static void Main()
  {
      string text;

      // ...
      // Place code here
      // ...
      text = "S5280ft";
      System.Console.WriteLine(text);
  }
}

His solution?

class Program
{
  static void Main()
  {
      string text;
      unsafe {
          fixed (char* pText = text) {
              pText[1] = 'm';
              pText[2] = 'i';
              pText[3] = 'l';
              pText[4] = 'e';
          }
      }
      text = "S5280ft";
      System.Console.WriteLine(text);
  }
}

My answer; note that I believe mine to be cleaner, more elegant, and far far more dangerous, since it never uses any sort of unsafe code:

class Program
{
  static void Main()
  {
      string text;

      string internedText = "S5280ft";
      String.Intern(internedText);
      MethodInfo mi = typeof(string).GetMethod("InsertInPlace", 
        BindingFlags.NonPublic | BindingFlags.Instance, null,
        new Type[] { typeof(Int32), typeof(string), typeof(Int32), typeof(Int32), typeof(Int32) }, null);
      mi.Invoke(internedText, new object[] {0, "Smile", 1, 7, 5});      

      text = "S5280ft";
      System.Console.WriteLine(text);
  }
}

The point? Playing with Reflection can be dangerous... oh, and it helps to know that strings are only as immutable as the platform forces them to be. In this case, my little hack would only be possible because under the covers, .NET doesn't really have immutable strings--it just doesn't let YOU modify them.

(By the way, same trick is available in Java, using the same approach. Or you could write JNI code to sort of duplicate Mark's trick, but who'd want to do that? Brrr.)

Matt Cassell is putting on an Academic .NET radio show (something in the vein of .NET Rocks! but aimed at students), and asked me to be the opening episode. It's up online now, so have a listen and see if I managed to steer the kids straight....

A good friend of mine and I are looking at taking on a new project together, and as part of the discussion we were exploring some of the differences of taking a relational perspective against an object perspective, and one of the comments she made was that in a relational model, you can always "filter" the data you want based on some predicate. "Ha!", I said, "If that's what you want, I can give you that over objects, too!" What's more, thanks to generics, I can do this for any collection type in the system without having to introduce it on some kind of base class:

    static class SetUtils
    {
        public static List<T> Project<T>(List<T> list, Predicate<T> pred)
        {
            List<T> results = new List<T>();

            foreach (T p in list)
                if (pred(p))
                    results.Add(p);

            return results;
        }

        // Not too hard to imagine the other relational operators here, too
    }

    // Usage:
    class Person
    {
        private string firstName;
        private string lastName;

        public Person(string fn, string ln, int age) {
            this.firstName = fn;
            this.lastName = ln;
        }

        public string FirstName {
            get { return firstName; }
            set { firstName = value; }
        }
        public string LastName {
            get { return lastName; }
            set { lastName = value; }
        }
        public override string ToString() {
            return "[Person [" + firstName + "]" + " " + "[" + lastName + "]" + "]";
        }
    }

    class Program {
        static void Main(string[] args) {
            Person cg = new Person("Cathi", "Gero", 35);
            Person tn = new Person("Ted", "Neward", 35);
            Person sg = new Person("Stephanie", "Gero", 12);
            Person mn = new Person("Michael", "Neward", 12);

            List<Person> list = new List<Person>();
            list.Add(cg);
            list.Add(tn);
            list.Add(sg);
            list.Add(mn);

            List<Person> newards = 
                SetUtils.Project<Person>(list, 
                    delegate (Person p) { if (p.LastName == "Neward") return true; else return false; } );
            foreach (Person p in newards)
                Console.WriteLine(p);
        }
    }

Now if I could just figure out how C# 3.0 manages to differentiate/overload between delegate instances and Expression objects in LINQ/DLinq, I might be able to backport that to C# 2.0, too, and be able to pass these Predicate instances across the wire for execution on other machines.

In a lot of ways, the Predicate delegate type is an example of using C#'s anonymous methods as a form of closure or lambda expression. (It's been argued that anonymous methods-as-delegates aren't "true" closures, since the local variables referenced in a closure will only be references to the objects, not complete copies, but to my mind that's exactly as it should be, as any time you pass a reference to an object, you're passing just that--a reference to an object, not a complete copy of the object. To do otherwise in anonymous methods would violate the Principle of Least Surprise, IMHO.) The Ruby syntax arguably isn't any more elegant or terse, and I suspect similar things could be done in C++ using templates; probably something along these lines already exists in Boost. But alas, I see no way to do this in Java given the current state of the JVM, namely the aforementioned lack of "managed functors" and type-preserving generics. If any out there in Java-land know otherwise, please holler, because I would really love to know how to do this as elegantly.

This is a bit of old news, but the discussion came up during the Seattle Code Camp, so I thought I'd go through the problem, and use it as an example of the issues that can come up when trying to map language concepts on top of a platform that doesn't support the idea natively. Hopefully, this will cause developers looking to build DSLs or other languages on top of the .NET (or JVM) platform to see some of the edge cases a bit more clearly and a bit sooner.

To lay down the background first: dealing with NULLs has always been somewhat problematic; the most obvious example of this is the mapping between relational databases, where even an INTEGER column can either have a value, or be empty, or be NULL, each of those being separate and distinct states. Trying to map NULL integer column values to integer values in the language has always been difficult in Java. C++, and C#, since primitive types / value types generally don't support null values, and Anders (among others) decided that it was time to try and integrate nullability more deeply into the language. The .NET team saw an opportunity to support nullability by creating a generic/templatized type to represent the possibility of nullability, and the C# language team took it further to try and make nullability feel "more at home" within the language. It was a bold, if at first seemingly-trivial, step.

Initially, the Nullable<T> type was pretty simple: it captured an instance of T internally, and if T was null it tripped an internal flag such that the IsNull property would return true. So, using a nullable int would work something like this:

Nullable<int> ni = new Nullable<int>(null);
if (ni.IsNull)
  Console.WriteLine("It's null!");
else
  Console.WriteLine(ni.Value);

int? ni = null;
if (ni == null)
  Console.WriteLine("It's null!");
else
  Console.WriteLine((int)ni);

int? ni = null;
int i = (int)ni;

OK, so far, so good. The problem is, however, that people were going to ask these nullable types to do things that subtly were different from what they'd ask of Nullable<T> instances. For example, the following snippet of code wouldn't behave as expected:

int? ni = null;
object o = ni; // What should this conversion be?
if (o == null) {
  // Should we be in this block?
}

Somasegar's weblog describes what happened next in some detail:

Clearly this had to change. We had a solution in Visual Studio 2005 Beta2 that gave users static methods that could determine the correct null-ness for nullable types in these more or less untyped scenarios. However, these methods were costly to call and difficult to remember to use. The feedback you gave us was that you expected it to simply work right by default.

So we went back to the drawing board. After looking at several different workarounds and options, it became clear to all that no amount of tweaking of the languages or framework code was ever going to get this type to work as expected.

The only viable solution was one that needed the runtime to change.

int? x = 10;
object y = x;
int? z = (int?)y; // unbox into a Nullable<int>

The lessons here? When building languages to run on top of another platform or runtime, the decisions that runtime makes often put some serious constraints around what you can do within your language. For example, looking to support first-class functors on a JVM or CLR will run into the fact that functions aren't first-class in the runtime, but instead have to be handled with object wrappers around the functions. Hiding those differences in language semantics can only get you so far, and that sometimes you need to involve the runtime team a bit more deeply if you want to close all those edge cases. (Hint to Sun: you really need to start thinking about revising and extending the JVM, instead of this current policy that essentially describes the JVM as perfect as-is. The changes made to support annotations were minor, but a good first step; it's time to open that Pandora's box wider if you want to keep up with the CLR, to be blunt about it.)

Dion Almaer made an interesting point recently:

A friend ... talked about how it is interesting that HTML is not statically typed, yet it has scaled pretty well. The internet architecture has made this happen. We are loosely coupled and modules (pages/site) are seperated out.

Ruby has its merits, Dion--you don't need to make spurious comparisons to try and justify it. Let programmers discover the beauty that is dynamically-typed programming on their own.

Matt Davey poses an interesting question:

The problem:

• C++ Corba legacy codebase (5+ years old, 1 million lines)
• No unit tests
• Little test data
• Limited knowledge transfer from the original development team.
• A flake environment to run the application in.

The requirement:

• Port the C++ result accumulation and session management code to Java

Do you:

1. Write C+ unit tests to understand the current system, then write Java equivalent code using TDD
2. Write Java tests using TDD based on your understanding of the C++ code
3. Hope you understand the C++ code, and JFDI in Java
4. Give up and go home
5. Get the original development team to do the work

My answer: depends. (Typical.) Here's what I mean:

• Option 1 is clearly the "best" answer if the goal is to produce code that will most accurately match what the current C++ code is doing, but also represents the greatest time and energy commitment, as well as making the fundamental assumption that what the C++ code does today is correct in the first place.
• Option 2 is the approach to take if the time crunch is a bit tighter and/or if the C++ unit tests can't be sold to management ("You're just going to throw them away anyway!"), particularly if the team working on the port has many or all of the original C++ devs. It also allows for the inevitable "You know, we always wanted to change how that code worked, so why don't we...." requirements changes.
• Option 3 is probably appropriate in those shops where WHISKEY (Why the Hell Isn't Somebody Koding Everything Yet) is considered an acceptable development methodology, but the lack of unit tests for the Java port will catch up to you someday (as it always does).
• Option 4 is probably best if the company you work for is seriously considering Option 3.
• Option 5 is only viable if the original development team is available (not going to happen if you outsourced it, by the way), able to work on it (meaning they've flipped the switch to Java at both a syntactic and semantic level), and isn't otherwise engaged on another project (which is probably the dealbreaker).

• 6. Let management believe in the whizzy-bang code conversion wizard that such-and-such company is trying to sell them on that "guarantees" 99% code translation and compatibility
• 7. Let management outsource the port, and let them worry about it
• 8. Give it all up and start from scratch--who needs that system anyway? It's not like anybody ever really used it, right?

Porting legacy code is one of the least-favorite projects of any software developer, but what few developers seem to realize is that they're also the least-favorite of management, too: it's a project that has no discernible ROI beyond that of "getting us out of the Stone Age". You might argue that the code becomes more maintainable if it's written in whatever-the-latest-technology-flavor-is-today, but the truth of the matter is, today's hot language is tomorrow's legacy language, subject to being rewritten in tommorrow's hot language. (Any programmer who's been writing code for more than five years probably already knows this, and any programmer who's been writing code for more than 10 years almost certainly knows this.)

Companies have been on this hamster wheel for far too long. Having gone through several transitions, particularly the C++-to-COM/CORBA-to-Java/EJB transitions over the last decade--and they're starting to resist if not outright reject the idea. Instead, they're preferring to find ways to create interoperable solutions rather than ported solutions--hence the huge interest in Web services when they first came out (and the interest in CORBA when it first came out, and the interest in middleware products in general like Tuxedo when they first came out, and so on). Integration still remains the "hard problem" of our industry, one that none of the new languages or platforms seem to want to address until they have to. Witness, for example, Sun's reluctance to really adopt any sort of external-facing technology into Java until they had to (meaning the Java Connector Architecture; their adoption of CORBA was half-hearted at best and a PR move at worst). .NET suffers the same problem, though fortunately Microsoft was wise enogh to realize that shipping .NET without a good Win32/COM interop story was going to kill it before it left the gate. C++ at least had the advantage of being call-compatible with C (if you declared the prototypes correctly), and so could automatically interop against the operating system's libraries pretty easily. In fact, it could be argued that C has long been the de-facto call-level compatibility interoperability standard (Python has C bindings, Ruby has C bindings, Java reluctantly, it seems, support C bindings through JNI, and so on), but of course that only works to a given platform/OS, since C offers so little by way of standardization and the operating systems have never been able to create a portable OS layer beyond the simple stuff; POSIX was arguably the closest they came, and many's the POSIX programmer who will tell you just how successful THAT was.

My point? I hereby declare a rule that any new language developed should think first about its interoperability bindings, and developers contemplating the adoption of a new language must flesh out, in concrete form, how they will integrate the hot new language into their existing architecture, or else they can't use it. (Yes, this applies equally to Ruby, Java, .NET, C++, and all the rest, even FORTRAN--no exceptions.) If you can't describe how it'll integrate into your current stuff, then you're just fascinated with the bright shiny new toy and need to grow up. It doesn't really matter to me how it integrates--through a database, through files on a filesystem, through a message-passing interface like JMS, or through a call-level interface, just have SOME kind of plan for hooking your new <technology X> project into the rest of the enterprise. (And yes, those answers are there for each of those languages/platforms; the test is not whether such answers exist, but how they map into your existing infrastructure.)

What's more, I hereby rededicate this blog to finding interoperabilty solutions across the technology spectrum--got an interop problem you're not sure how to solve? Email me and (with your permission) I'll post the response--sort of an "Ann Landers" for interop geeks.

By the way, this conundrum can be genericized pretty easily using generics/templates:

enum Q
{
  No, Bad, Little, Flakey, Untouchable
};
enum technology
{
  C, C++, Java, C#, C++/CLI, VB 6, VB 7, VB 8, FORTRAN, COBOL, Smalltalk, Lisp, ...
};

Problem<technology X, technology Y, type T extends AbstractTest, enum Q>:
{

<X> legacy codebase (<int N where N > 1> years old, <int L where L > 1000> lines)
No <type T> tests
<Q> test data
<Q> knowledge transfer from the original development team
<Q> environment to run the application in.

} returning requirement:

Port the <X> project to <Y>

Ever since the Seattle Code Camp, where I hosted a discussion (hardly can call it a lecture--I didn't do most of the talking this time, as it turned out) on language innovations, one of the topics that came up was the notion of concurrency, and of course Herb Sutter's "No More Free Lunch" article from DDJ from some months ago. That put a bug in my ear: what sort of languages out there support concurrency in some form, baked into the language? I've started to compile a list, but any other suggestions/references would be welcome; I'd like to keep it to "active" languages (as opposed to languages no longer under active development), but if there's a particular concurrent language that had some kind of major influence on a branch of thinking, I'd love to see it listed. And by "language" here I'm willing to be flexible--extensions to preexisting languages (a la OpenMP) are interesting in their own right. But, I'd like to keep it to language-level constructs, not library-level constructs--so C-with-POSIX, C++-with-BOOST or Java-with-java.util.concurrent aren't going to make the list, since they mostly support concurrency through the low-level mechanism of "start yer own thread". I'm interested in languages that do more than that.

So far, what I've come up with includes:

• Cw (aka C-omega): a combination of X#/Xen and Polyphonic C#, Cw provides an interesting concept called "chords" that suggests that methods of classes "work together" in pairs to handle concurrent access.
• OpenMP: an extension to FORTRAN and C++, OpenMP uses #pragmas (in C++) to declare regions of code where an OpenMP compiler can spawn off threads and provide concurrent execution. What makes this interesting is its intersection to the mainstream: Visual Studio 2005 is an OpenMP compiler, and works for both unmanaged and C++/CLI code, meaning that this may be an interesting approach to handling concurrency inside of .NET apps. I know there's more out there--fire away! Regardless of whether they compile for .NET, JVM, or unmanaged code, I'm interested in seeing what others have been exploring and/or playing around with. Academic links particularly wanted--they have a tendency to push the edge of the envelope (and some would say sanity) when it comes to areas like this.

I asked Jan Kotas, about a patch he'd made for Rotor (SSCLI) to run on XP SP2, Windows 2003, FreeBSD 5.2 and MacOS/X, since the location Joel had blogged about is no longer available--the www.sscli.net server has been shut down--and he was gracious enough to send it to me. Figuring that others would like to find the same patch, I'm posting it here (which hopefully isn't in violation of the Shared Source license, email me if you're Microsoft and want me to cease-and-desist). This patch, I believe, is to the last official release of the SSCLI tarball (which you can get from microsoft.com).

ssclipatch_20040514.diff.gz (104.21 KB)

By the way, guys, we're all eagerly looking forward to Rotor Whidbey!

Christian Weyer has created a pretty comprehensive chart of WS-* specs and how they map to .NET technologies (which specs are supported in which product), and I realized that I've not seen a similar chart in the Java space detailing WS-* spec to JCP spec, nor how the WS-* specs and/or JCP specs map to various XML service providers (Axis 1.x, 2.x, WebLogic, and so on). So I thought I'd draft one up, but before I do, does anybody know of a similar writeup already existing in the Java space?

I noticed a referrer URL in my logs from a Lisp chat channel, where apparently a collection of Lisp programmers found my dynamic languages blog entry and were a little less than impressed at my Lisp knowledge. Let's make something REALLY clear right now:

I know almost nothing about Lisp.

Seriously, my proposal for giving a talk on Lisp was to be the take of a guy who's a statically-typed guy for a decade who's coming to see Lisp and try to explain its concepts to other statically-typed guys, not as a Lisp expert to other Lisp experts. In fact, I'd love it if those who were on the chat emailed me privately so I can try to understand it better.

In the meantime, though, I do know what I've begun to pick up out of books (my current tome being Practical Common Lisp, from APress) and the various Lispers I've talked to in the past, and I do know (until somebody can prove otherwise) that Lisp has a small set of core primitives from which the remainder of the language is built. If that's not the case, show me otherwise.

Stu Halloway has responded to my earlier post about dynamic languages, and Stu refines his argument. Still wrong, but at least now it's refined.

Stu writes that we're "talking past one another", and in particular notes that

The criticial point is that these abstractions are implemented in the language itself. Developers can (and do!) modify these core abstractions to work in different ways.

Where Stu, I think, is being fallacious with this is that he presumes a bit much with respect to at least a few of these languages; in particular Ruby has some facility for self-modification and language evolution, but still relies on a core set of principles that are implemented in native code inside the Ruby interpreter. Ditto for Smalltalk, ditto for Python, and even for Lisp, the poster child for dynamic languages. (In all fairness, Stu does admit this--in a backhanded sort of way--when he notes that "The rules for adding new methods to existing classes aren’t (for the most part) in the core of ruby — they are implemented in Ruby source code.")

What Stu's point does raise, however, is still the valid point that languages offer a continuum of self-modification and/or evolution, and that languages like Ruby, Smalltalk, Python or Lisp clearly come in on the "more" end of that continuum as opposed to languages like C# or Java or C++. And this plays into his later comment when he states, "It’s all about control. With a vendor-oriented language like C#, core abstractions are much more firmly controlled by the language vendor. Conversely, developer-oriented langauges like Python leave more of these choices to the developer (although they tend to provide reasonable defaults). So, again, who do you trust?"

There's two points I want to raise here. One is technical, the other political/cultural.

First, the technical: dynamic languages may choose to expose more meta-control over the language, but there's nothing inherent in the dynamic language that requires it, nor is there anything in a static language that prevents it. Languages/tools like Shigeru Chiba's OpenC++ or Javassist, or Michiaki Tatsubori's OpenJava clearly demonstrates that we can have a great deal of flexibility in how the language looks without losing the benefits of statically-typed environments. So to attribute this meta-linguistic capability exclusively to dynamic languages is a fallacy.

Secondly is the cultural issue: is the idea of granting meta-linguistic power (known as meta-object protocol, or MOP) to a language a good thing? Stu asserts that it is: "My concern is who controls the abstractions. Developer-oriented languages (like Scheme) give a lot of control (and responsibility) to developers. Vendor-oriented languages (like Java) leave that control more firmly in the hands of the vendor." So in whose hands are these abilities to change the language best placed?

*deep breath* I don't trust developers. There, I've said it.

I say this not because I think developers are all 5-year-olds who need to be carefully watched and monitored and chastised gently when they actually run with scissors, but because in some cases, we don't necessarily know what we're doing when we start adopting certain features or ideas. Here's an example of what I mean: about eight years ago, when servlets were new and Reflection was still a Brand New Topic amongst developers, I read an article on building a servlet-based system that was touted as "dynamic" and "powerful": in essence, the servlet would look for a query parameter in the request URL and Reflect for that method name on the servlet and/or alternate class, and execute it.

This is a Good Thing?!? Incredibly dynamic, granted, but given the overhead and performance implications (not to mention security concerns), I can't see this as a great way to build scalable, dynamic systems.

Gregor Kiczales, the inventor of AspectJ and long-time CLOS wonk--so you know he has experience on both sides of this fence--told me once that one of the greatest flaws of CLOS (I don't know if he used the word "flaw", per se, but that was my takeaway) was that it allowed developers too much power. Developers writing CLOS systems apparently had this tendency to do too many wild-and-crazy things that ultimately (in his view) led to a number of write-only CLOS codebases. AspectJ was deliberately constrained to prevent these sorts of things, and whether or not he's succeeded in that remains to be seen--many long-time O-O advocates still see AspectJ as "an evil hacking language", despite those constraints.

I see the same concern every time a developer starts talking about doing bytecode manipulation at load-time--just because you can doesn't mean you should. In this respect, I trust the guys who've been down this road before much more so than developers who are just coming to this and are starting to flex their new-found freedom and will (undoubtedly) start building systems that exercise this power.

In the end, Stu's right, in that he and I share a lot of common ground--working together for four years has a tendency to do that to you. And I won't even suggest that he's "wrong" so much as that he and I simply disagree on how much meta-control should be baked into a language, dynamic or otherwise.

Long-time blog reader Dilip Ranganathan pointed me to this discussion over on Steve Vinoski's blog about the history of CORBA, and in particular the discussion that ensued in the comments section on the entry. I found it interesting from two perspectives:

1. The idea that two people could look at the history of CORBA (having presumably lived through it) and come away with entirely different ideas of what that history was, and
2. The discussion over CORBA's role and influence on the current XML services environment.

For starters, Steve Vinoski was a bit miffed at the idea posited by Mark Baker that CORBA failed. Sorry, Steve, I have to say it, but I agree with Mark--CORBA never fulfilled on its intended promise of seamless middleware interoperability and integration capabilities, and certainly not over the Internet in any meaningful way. By the time CORBA began to address some of those issues--firewalls being a big one--the world had already pretty much abandoned both the "distributed object brokers" (the other being COM/DCOM) and were starting to explore HTTP as the be-all, end-all transport protocol.

But the discussion that comes out of Steve's challenge that CORBA didn't fail is to me the far more interesting point--the discussion of whether the WS-* stack is loosely coupled or not. See, if CORBA's failure was that it was a too tightly-coupling technology to allow for good integration between companies (as Mark Baker asserts in the discussion), then we have to be careful regarding how tightly we couple endpoints and interfaces in the WSDL world, as well. And this is where I wholly agree with Mr. Baker: I look at the current crop of WSDL-based implementations, and their IDL-cum-WSDL interface descriptions (usually generated from shudder a language interface), and I see the same mistakes being made.

The discussion continues, but rather than try to summarize it (and probably get it wrong, given my current state of exhaustion), I suggest you head over and have a look. If you're into the XML services space at all, you owe it to yourself... and your clients... to do so.

For those in the blogosphere living in the Seattle area, wondering about details on Seattle's Code Camp 2005 experience, the schedule and agenda have been posted. It's looking to be an interesting set of talks, including discussions on MacOS/Cocoa development, Ruby, an Intro to Perl, Monad, Objective C, and LINQ/C# 3... and that's just in the languages/frameworks track.

Observant Blog Ride Readers will note, however, that the sessions page doesn't list anything from me. This is not a snub from the Code Camp HR Department, this is me not being entirely sure what to present on. Got any suggestions or votes? I'm thinking about talking about (in no particular order or preference, and yes, this is totally just "brain dump" ideas, as I want to do something totally experimental for Code Camp and not one of my regular sessions):

• ECMAScript and/or E4X
• Lisp
• Smalltalk (via Squeak and/or Cincomm Smalltalk)
• the new script engine support inside of JDK 1.6 ("Mustang")
• C-omega (also sometimes known as Cw)
• Boo and/or Groovy
• JRuby and/or Ruby.NET (Ruby-on-VMs)
• ANTLR and building your own language
• Reversing malware (the talk I did with my brother-in-law at the Portland Code Camp)
• Intro to C++/CLI
• F#
• Windows internals

Got your own suggestion, maybe based on something loosely related to what I've talked on before? Fire away!

Here in Orlando, Land of the Hurricanes, and just gave a talk on Hosting ASP.NET, and I've posted both the slides and sample code (what there is of it). If I find time, I'll come back and update the entry to include a link to the article Aaron Skonnard wrote on MSDN about hosting the ASP.NET runtime, but I wanted to get this up ASAP.

A number of folks have pinged me about my slides for the above two shows; they're not found on (either) conference's CD nor their website, for which I accept 100% blame. (I missed the cutoff date for including them on both.)

To make it as easy as possible, I've posted them here, for your viewing pleasure.

SD Best Practices 2005 (Boston)

• FallaciesOfEnterpriseComputing.ppt (102 KB)
• Messaging.ppt (433 KB)

JAOO 2005 (Aarhus, Denmark)

• CoreIndigoPatterns.ppt (40.5 KB)
• EffectiveJava1.ppt (760.5 KB)
• Extending System.Xml.ppt (62 KB)

As usual, if you weren't at the shows, the slides may not make complete sense, but if you find them intriguing, by all means, come on by one of the same conferences next year.

For many developers, it's been a while since they got together with their current programming environment. They've hit the 7-year-itch mark with their current language/platform partner. They find themselves in a rut. Coding is mundane. Routine. Boring, even. It's the same old roll-over, perfunctory foreplay about which frameworks to use, same decisions and scripts every time, same results, same good-night kiss and back-to-sleep as the last project, and the project before that and the project before that and the project before that...

Ruby is new. Exciting. It makes you feel alive again. You feel appreciated. You feel loved. Like the language was made just for you. It caresses your desires, gives you new ideas, molds itself to what you want it to be. It makes your jaw drop and say, "I didn't know you could do that!". It leaps to your will, and does so much more than you thought a partner could do. You wonder what you ever saw in that language you left behind.

At least at first.

Over time, though, the infatuation ends as most affairs do--in time, you discover a certain comfort in your language of choice. Sure, it's not perfect, but you know it well, you can get the job done, and what's more, everybody's content. Not ecstatically happy, sure, but "good enough", and besides, it's hard work trying to learn the nuances of a new partner. Nobody likes to admit it, but sometimes comfortable is better than exciting.

You never forget those heady days, feeling the wind in your hair and reliving your younger days as a programmer. It reinvigorates you, reenergizes you, makes you feel alive. It gives you something you didn't know you needed, but in the end, fires you up to go back to what you know best, brimming with fresh ideas and energy, ready to spice up your partnership so that you can remain happy for the next five, ten, even twenty years.

Ruby is a love affair.

A couple of commenters have asked me, via comments and email, what my particular story with respect to my ADD and medication is. Put bluntly, I don't like pills, and generally try to stay away from them unless absolutely necessary. (I think we as a society--that is, the US--have a strong tendency to overmedicate ourselves, so I only want to be popping pills if it's a necessity. That said, I'm not a religious zealot over this; for example, a migraine right before a talk clearly counts as a necessity. )

I tried a few medications for the ADD for a while, but didn't really feel that sense of "Oh, wow..." that many of the other ADD/ADHD-gifted people I'd met had claimed. So I pretty much stopped trying. In fact, one working theory I've come up with since is that the vast amounts of Diet Coke that I drink on a daily basis is a form of self-medication, since most ADD/ADHD meds are stimulants, as is caffeine.

That said, I've met others who absolutely could not function without their meds, and so while I'm not opposed to meds in general, I want to try and manage it without them if I can.

Let me be clear, though: knowing you have it is a HUGE relief--to those of you who were thinking of getting tested, do it, but do it through an accredited pyschologist, preferably one who's very familiar with the symptoms. (It's very easy to think you have it if you find you have one or two of the symptoms, but true ADD-gifted people have ALL of them.) It can literally mean the difference in your life--for example, it almost completely changed mine, because suddenly, now I understood why I was the way I was, and more importantly, that others didn't see the world the same way I do. That, more than anything, more than any meds I could ever take, makes the difference.

EVer since being diagnosed as an adult with ADD (Attention Deficit Disorder), I've been actually pretty cool with the idea--it lets me multitask far more easily than my non-ADD compatriots, and I've always enjoyed the creativity that goes with an imagination run wild.

Now, apparently, MSN thinks so too.

The good folks over at Relevance have blogged again, offering something of a backhanded compliment to the new features of C# 3.0:

The argument that I infer from Ted’s piece is "Look! now we can have (some of) the expressiveness of dynamic languages with (most of) the safety of a statically typed language." ... But just because C# now looks a little more like some dynamic languages, don’t make the mistake of assuming that two worlds are converging. In the most important ways, they are as different as ever. Here’s why: Languages like C# "bake in" specific and detailed rules for inheritance, encapsulation, delegation, how symbols are interpreted, etc. In dynamic languages, similar rules exist, but they are not part of the language core. Instead, they are idiomatic extensions built within the language itself. Development teams can follow these idiomatic rules. Or, they can build (and enforce!) their own rules, specifically tailored to their needs. This has huge implications for productivity. In dynamic languages, you get to build the language up toward your domain, while you build the solution down.

Folks, we need to realize something: all this "expressiveness" is like putting craftsman's tools in your hands; in the hands of a master craftsman, amazing things can result, but in anybody else's hands, it's putting a loaded gun into the hands of a child. YOU may be good enough to be disciplined enough to keep the rules of your types in your head when programming with Ruby, but are all of the programmers on your team equally gifted? Are all of the programmers that will follow you so gifted?

There's something else that they call out here, though, and that's the part that irks me:

So why has the static/dynamic debate staggered on for so long? I think we could get closer to some answers with better choice of terms. "Static" vs. "dynamic" is highly misleading. I propose we use a new set of names: vendor-oriented vs. developer-oriented programming, or VOP vs. DOP. So who do you trust most: vendors or developers?

... should I wait for Indigo/WCF to ship?

I'm a bit late to this, but they've just started putting together the logistics for Seattle Code Camp (Oct 22-23), a community-driven event bringing programming speakers and interested attendees together for a couple of days, gratis. Who is "they", you ask? It's that Evil Empire, Microsoft, out to steal your souls. Be warned, Java faithful, lest ye lose your chance at the Afterlife and Good Code!

Not.

Code Camps are a recent invention of Microsoft's, and they're intended to be technology-agnostic. (In other words, no evangelism, no hard-sells to convert you to .NET. As a matter of fact, I think I've heard more anti-Microsoft jokes from the Microsoft Developer Evangelist team than any other organized body, including the JBoss folks.) Microsoft is doing what it can to improve it's relationship with developers on the whole, and this is one of those efforts. It's on the up-and-up, believe me--we had a number of non-.NET/non-Microsoft talks at the Portland Code Camp a few months ago, for example.

I'm the track chair of the Java technology area, for example, and already a friend of mine (whose name I'm not sure I have permission to mention here, so I'll play it safe and not say it, but you'd recognize it if you heard it--he's behind a couple of good XML open-source frameworks, on which he'll be speaking) has agreed to brave the waters and come speak. If you're interested in Java, Ruby, .NET, XML, or anything else code-related, come on by; details of where will be posted soon. If you're interested in speaking at said event--and this is not open to just professional speakers, but anyone with something interesting and code-related to show other programmers--contact me and I'll either put you in touch with the right folks, or (if it's Java-related), it's me you deal with.

And just for the record, I would LOVE it if the Seattle Java community stormed the show and outnumbered the .NET talks. Email me and let's make it happen.

For those of you who don't know this, the blog at the root of the neward.net domain is one that my wife maintains--all I can claim is inspiration, providing her with plenty of material to write about, like the stories about her kids and her uber-geek husband. A regular Muse, that's me.

The reason I bring it up here, in this channel, is that I've had more speaker-friends of mine come to me and tell me that while they like reading my blog, they love reading Charlotte's blog. What's more, their spouses find Charlotte's blog to be highly entertaining, probably because they can relate so deeply to Charlotte's dilemma as Geek Widow. So if you've got a girlfriend or wife who'd like to check out a non-technical blog, or if you're looking for a bit more insight into the personal world of Ted, or maybe you just want to read a pretty good writer, check out The Neward Family Weblog.

G'wan--the geek blogs will still be waiting for you when you get back.

Whomever designed the JAOO conference should be knighted by the Queen. Or King. Or whatever it is they have in Denmark (forgive my lack of background on Danish monarchist traditions; disturbing for a former International Relations major and future diplomat, I know, but...).

I've got to admit, I'm rapidly falling in love with the European shows--first JavaZone, then JAOO, not to mention SDC earlier this year, it's really becoming apparent that European shows (despite their reputation to the contrary, apparently, an attitude I completely don't understand) are every bit as interesting and exciting as US ones. In fact, I might go so far as to say they're even better than their US counterparts. I'm not certain exactly why, it's just they seem to have more "character" than the US shows I've been to over the years, the sole exception to that being the NFJS shows and DevTeach (which I think has more of a European flavor to it thanks to its Canadian heritage). Organizers of a show in Bergen, Norway, have invited me to their get-together in Bergen in April, and I'm already looking forward to it, not to mention Javapolis in December. Oh, and now that I think about it, DevWeek is coming up, too.

I don't know if they're worth flying out from the States to go see (not when NFJS brings so much of that same feel to your backyard), but they're definitely fun to speak at, despite the nine-hour or eleven-hour flights from Seattle to Continental Europe. And to those who might suggest that European shows are somehow inferior to US ones... fie! fie! fie on thee!

One of my favorite questions to ask during my Enterprise Fallacies presentation is how you're going to use your thin-client application at 37,000 feet, because the airlines don't have network access. Now, as I write this, I'm on board a Scandinavian Air Service flight to Copenhagen (on my way to JAOO), using the wireless service on the flight to access the thin-client blog-entry interface on the site--this wasn't written offline and posted later, as so many of my other blog entries have been.

Which means, of course, that I now face a dilemma--do I retract what I say in that part of the Fallacies talk, and admit that, finally, the network really is available everywhere? After all, even though it's only the European carriers that are offering it (Lufthansa and SAS are the only two I know of thus far), and even then only on their international flights (so far as I know), the actual connection is "Connexion By Boeing", so you know Boeing is going to offer it as a retrofit on US aircraft before too long--it's just a matter of the FAA getting around to realizing that the signal isn't nearly as much of a danger as they've made it out to be.

So, is it time to abandon the first fallacy?

Duh--of course not.

Truth is, the network access from the plane is horrendous--latency is terrible, which makes a lot of sense, given how far these poor little bytes have to travel in order to actually reach the site. In fact, if you consider the fact that they're traveling through a tight-band satellite connection, which has been known to be somewhat flaky due to nothing more than aircraft movement, it's pretty amazing that they get there at all. But more importantly than that, the point still remains that even if the network is there most of the time, it's not there all of the time, and the partial-failure scenarios that have been with us from the beginning are still scenarios we need to worry about for the enterprise systems that we build. And, more importantly, by taking network outages into the design/architecture of the system, we build not only redundancy in case of accidental failures but also ability to function even in the face of administrative outages (patches, upgrades, hardware replacements, etc).

The First Fallacy isn't just about network availability, it's about network outages, and the more we spread wireless around (and become dependent on it), the more we'll find that network outages are more and more common, something that we'll have to take into account when building systems. So don't expect the First Fallacy to go away any time soon.

Update: Well, turned out I was more right than I knew; while I was able to surf to the entry page to fill this entry out, I couldn't manage to get it submitted--kept timing out when I'd push the button to send it in. A couple of other States-based sites were timing out, too, so I'm guessing that the gateway (whether that's on the plane or on the ground, I'm not sure) is giving up because the latency is so high. So apparently the First Fallacy is still with us, airplane networking or no. (Interestingly enough, though, MSMessenger and Google Talk worked just fine, so apparently the latency either doesn't bother them or the conversations were just that much slower and I didn't realize it.)

Apparently there's been quite a stir started by my use of the term "syntactic sugar" to describe the featureset of C# 3.0, and more than a few people are wondering what I mean by that. Simply this: that the C# compiler isn't doing anything fundamentally *different* than what you could easily do using the existing facilities of the language--in essence, it is making certain things easier, not possible. So, for example, right now the C# compiler does not allow for inline assembly CIL expressions (though I wish it would, quite honestly), so adding this as a language feature would be a non-sugar feature. The implicitly-typed local variable, on the other hand, is just an easier way to declare a local, nothing else changes once the compiler has finished its pass over the keyword "var".

It's probably not the most rigorous definition of the term, and I'm probably using it wrong, but that's the beauty of expressing an opinion--you get to learn just how wrong you are from a variety of different sources. Thank God for the Internet!

For those in the Java community who've heard brief rumors about the suggested feature set of C# 3.0 announced last week at PDC, let me be the first to point out that nothing in the language (aside from generics, which Microsoft did right in C# 2.0, integrating them into the virtual machine rather than the type-erasure-based approach that Java chose) that's proposed couldn't be done in the Java language or on top of the JVM; in fact, most of the features of C# 3.0 are, arguably, nothing but syntactic sugar designed to make programming more productive. What I plan to do here is explain each of the features of C# 3, show how they're implemented (by examining the generated CIL), at least in the PDC preview Microsoft handed out at PDC, and by doing so demonstrate how Java could be extended in turn to support exactly the same sorts of features.

Standard disclaimer applies: all of this is based on the PDC preview of C# 3.0, no guarantees or warranties implied, use at your own risk, yadda yadda yadda. In short, if you install it, and it blows up your hard drive, it's your own fault.

Implicitly typed variables

For starters, C# 3.0 will support implicitly typed local variables, meaning that programmers can now write code in a more "ignorant" fashion--programmers need not worry so much about getting the types exactly correct when working with local variables:

var i = 5;
var s = "This is an implicitly typed local variable";
var a = new int[] { 1, 2, 3 };

int i = 5;
string s = "This is an implicitly typed local variable";
int[] a = new int[] { 1, 2, 3 };

.method private hidebysig static void  Main() cil managed
{
  .entrypoint
  // Code size       28 (0x1c)
  .maxstack  3
  .locals init (int32 V_0,
           string V_1,
           int32[] V_2)
  IL_0000:  nop
  IL_0001:  ldc.i4.5
  IL_0002:  stloc.0
  IL_0003:  ldstr      "This is an implicitly typed local variable"
  IL_0008:  stloc.1
  IL_0009:  ldc.i4.3
  IL_000a:  newarr     [mscorlib]System.Int32
  IL_000f:  dup
  IL_0010:  ldtoken    field valuetype 
'{E4ADF86B-1985-4CA3-90AF-B705A8279423}'/'__StaticArrayInitTypeSize=12' 
'{E4ADF86B-1985-4CA3-90AF-B705A8279423}'::'$$method0x6000001-1'
  IL_0015:  call       
    void [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array,
        valuetype [mscorlib]System.RuntimeFieldHandle)
  IL_001a:  stloc.2
  IL_001b:  ret
} // end of method Sample::Main

It may seem odd and a trivial feature to add, but this will turn out to be a profound feature of the language when coupled with object initializers, next.

Object initializers

One of the more annoying aspects of C# (and Java, or C++ for that matter) is that we end up having to write a lot of redundant code, particularly when frequently it's all effectively the same basic conceptual idea. One such area of redundancy is constructors--far too often, we write classes whose constructors do the most basic thing a constructor can do, which is of course to initialize its fields to their desired values. Object initializer syntax allows for simple initialization of types without requiring an explicit constructor to be written:

public class Point
{
  int x; int y;

  public int X { get { return x; } set { x = value; } }
  public int Y { get { return y; } set { y = value; } }
}

Point p = new Point { X = 0, Y = 1 };

Point p = new Point();
p.X = 0;
p.Y = 1;

.method private hidebysig static void  Main() cil managed
{
  .entrypoint
  // Code size       28 (0x1c)
  .maxstack  2
  .locals init (class Point V_0,
           class Point V_1)
  IL_0000:  nop
  IL_0001:  nop
  IL_0002:  newobj     instance void Point::.ctor()
  IL_0007:  stloc.1
  IL_0008:  ldloc.1
  IL_0009:  ldc.i4.0
  IL_000a:  callvirt   instance void Point::set_X(int32)
  IL_000f:  nop
  IL_0010:  ldloc.1
  IL_0011:  ldc.i4.1
  IL_0012:  callvirt   instance void Point::set_Y(int32)
  IL_0017:  nop
  IL_0018:  ldloc.1
  IL_0019:  nop
  IL_001a:  stloc.0
  IL_001b:  ret
} // end of method Program::Main

And this isn't limited to primitive type fields, either; we can do the same for complex fields, as in:

public class Rectangle
{
  Point p1; Point p2;

  public Point UpperLeft { get { return p1; } set { p1 = value; } }
  public Point LowerRight { get { return p2; } set { p2 = value; } }
}

Rectangle r = new Rectangle { 
    UpperLeft = new Point { X = 0, Y = 0 },
    LowerRight = new Point { X = 5, Y = 5 }
};

Note that along with object initializers, C# 3 also introduces a similar syntax for initializing arrays and collections of various forms; this is more fully documented in the C# 3.0 Language Specification that ships with the PDC Preview bits, but lexically looks pretty similar to object initializers, so I'll just refer you to that document for details.

Anonymous types

var x = new { UpperLeft = new Point { X = 0, Y = 0 }, LowerRight = new Point { X = 5, Y = 5 } };

class __This_Name_Really_Doesnt_Matter
{
  private Point _Field1;
  private Point _Field2;

  public Point UpperLeft { get { return _Field1; } set { _Field1 = value; } }
  public Point LowerRight { get { return _Field2; } set { _Field2 = value; } }

  public override bool Equals(bool rhs) { ... }
  public override string ToString() { ... }
  public override int HashCode() { ... }
}

Extension methods

Another significant addition to the C# 3.0 language will be extension methods, whereby one class can lexically "inject" methods into another class by declaring a specific form of static method on a static class. Once again, however, it will be pretty clear that this is pretty much all just compiler syntactic sugar, and once again will play a significant role in DLinq.

To declare an extension method, create a static class (a new feature of C# 2.0, a static class is a class that can never be instantiated--in many respects, it is a formalization of the old procedural library concept from C or Pascal) that contains a static method as usual, but with one minor difference. To make this method an extension method, declare the first parameter to have an additional modifier, the this keyword, to indicate the type to which this method will extend.

This is a bit confusing, but bear with me--a few examples will make it clearer.

From time to time, every object programmer has lamented the inability to "slip in" functionality on a base class they do not control--one of the classes from the Framework Class Library, perhaps, or a class that comes out of a commercial third-party library to which they do not own the source. (Even open-source projects are resistant to this kind of injected change, because forking an open-source project is not a task undertaken lightly--you will have to make the same changes to every successive version of the library, an unenviable task.) Using an extension method, the compiler will effectively "pretend" that the extension method is declared on that class, and allow for invocation of the extension method as an instance method of the object.

Begin with a basic class, perhaps our Point class from before:

public class Point
{
  int x, y;

  public int X { get { return x; } set { x = value; } }
  public int Y { get { return y; } set { y = value; } }
}

Extension methods offer a way out:

namespace Extender
{
  public static class XMLUtil
  {
    public static string ToXML(this Point pt)
    { 
      Console.WriteLine("Imagine cool XML code here"); }
    }
  }
}

Point pt = new Point { X = 0, Y = 1 };
Console.WriteLine("pt.ToXML = {0}", pt.ToXML());

.method private hidebysig static void  Main() cil managed
{
  .entrypoint
  // Code size       45 (0x2d)
  .maxstack  2
  .locals init (class Point V_0,
           class Point V_1)
  IL_0000:  nop
  IL_0001:  nop
  IL_0002:  newobj     instance void Point::.ctor()
  IL_0007:  stloc.1
  IL_0008:  ldloc.1
  IL_0009:  ldc.i4.0
  IL_000a:  callvirt   instance void Point::set_X(int32)
  IL_000f:  nop
  IL_0010:  ldloc.1
  IL_0011:  ldc.i4.1
  IL_0012:  callvirt   instance void Point::set_Y(int32)
  IL_0017:  nop
  IL_0018:  ldloc.1
  IL_0019:  nop
  IL_001a:  stloc.0
  IL_001b:  ldstr      "p.ToXML() = {0}"
  IL_0020:  ldloc.0
  IL_0021:  call       string Extender.XMLUtil::ToXML(class Point)
  IL_0026:  call       void [mscorlib]System.Console::WriteLine(string,
                                                                object)
  IL_002b:  nop
  IL_002c:  ret
} // end of method Program::Main

Note that of course extension methods introduce some interesting method-overload-resolution rules, such as when the extension method clashes with a method on the extended type (the extended type wins) or when two extension methods of the same name and signature are both brought in via a using clause (in which case the "most nested" using expression, inside namespace declarations, wins). These rules are likely to change as feedback filters in on the released PDC bits, so if you're to bet the farm on this particular aspect of the language (pun intended), make sure to keep up with the latest C# 3.0 specification changes as well.

Note also that as of this writing, the PDC Preview bits also come with this note in the documentation:

Extension methods are less discoverable and more limited in functionality than instance methods. For those reasons, it is recommended that extension methods be used sparingly and only in situations where instance methods are not feasible or possible. ... Extension members of other kinds, such as properties, events, and operators, are being considered but are currently not supported.

Lambda Expressions

The lambda expression, long a favorite of Lisp programmers, has come to C#. While the .NET platform has always had the capability to create delegates, which are essentially managed function pointers, and while delegates could always be used as a poor man's subsitute for lambda expressions, former Lisp programmers have always had a yearning in their heart to see real lamba expressions in their favorite .NET language. Anders heard the call, and answered: where C# 2.0 introduced the ability to create anonymous delegates, method bodies that are implicitly converted into a class with a single method (the anonymous method itself), C# 3.0 introduces lambda expressions, the ability to define a method body--or, more accurately, just a block of code--in a fairly terse and elegant way. The lamba expressions are probably the hardest part of the C# 3.0 specification to grok if you've not nseen it before, however, so be prepared to spend a little time with it before it all makes intuitive sense.

In essence, a lambda expression follows the pattern aid down by a delegate, so to begin we start by declaring a delegate type to which lamba expressions should be assigned; in the PDC preview documentation, for example, they use this example:

delegate R Func<A, R>(A arg);

Func<int, int> f1 = new Func<int, int>(MyClass.MyMethodTakingAnIntAndReturningAnInt);

Func<int, int> f1 = delegate(int i) { return i * i; };

Func<int, int> f1 = x => x + 1;

Console.WriteLine(f1(12)); // prints 144

• "Lambda expressions permit parameter types to be omitted and inferred whereas anonymous methods require parameter types to be explicitly stated.
• "The body of a lambda expression can be an expression or a statement block whereas the body of an anonymous method can only be a statement block.
• "Lambda expressions passed as arguments participate in type argument inference and in method overload resolution.
• "Lambda expressions with an expression body can be converted to expression trees."

The last of the new features of C# 3, the query language features, isn't really a language feature per se, but a close integration of the compiler and expected library support it's compiling against, and as such doesn't really openly qualify as "language innovation", in my opinion. That said, though, it's damn useful, and what's more interesting, Java actually has a tool that can provide this kind of capability already--the OpenJava compiler tool (from the same folks that brought you Javassist, the bytecode manipulation tool that is at the heart of JBoss, among other open-source projects), which allows you full metaobject protocol capabilities, including the ability to add new keywords to the language.

And that's ultimately my point here: as you've seen, nothing that C# 3.0 introduces is really all that revolutionary once we get past the compiler--even the extension methods and lambda expressions are defined in terms of what's already present within the language and framework, making the entire exercise one in compiler syntactic sugar. Very sweet, very addictive sugar, perhaps, but just syntactic sugar nonetheless. And yet, because these features are still built in terms of the CLR, it means that we have full fidelity static-typing, even through the syntactic sugar (unlike what happens in the case of Java generics).

For ten years, Sun has insisted that Java Language and Java Virtual Machine must remain in lockstep, and as a result the language innovation in Java has either completely stagnated (the only real language innovation in Java 5 was the custom annotations model, and that was almost a direct copy of what .NET had done before), or else occurred outside of Sun's--and therefore "official Java"'s--boundaries. Sun needs to realize that the strength of the JVM by far exceeds the limited language potential of the Java language, and if they don't want to watch Java's popularity begin a steady decline, they need to cut the umbilical and let the JVM run free and the language innovation truly begin. Otherwise, it's looking like a very CLR world ahead of us.

Now, I own a Windows box (which runs VMWare, which runs three other Windows images and a Linux image, so perhaps it is fairer to say that I own lots of different virtual boxes but I still feel most at home in Windows), and I've tried to get the JDK (since version 1.3? 1.4? when they first introduced the SCSL licensed-source) to build under Windows on my own. Oh, I've managed to get pieces of it to build--most notably the VM--but I want the whole thing, lock-stock-and-barrel, so I can start doing some major spelunking across the entire JVM-and-related-libraries, and maybe even do a book on it.

Enter this page.  I haven't tried it yet, but wow, talk about step-by-step. Have to give that a spin, maybe on a fresh VMWare image (just to avoid cluttering up the others).

John Haren, of CodeSnipers.com, recently blogged about something I feel pretty strongly about:

There's a common trope in CS education that goes something like this: "All software evolves, so be prepared for it."

Far be it from me to imply that one shouldn't be able to respond to change; that's not my intention. But the idea expressed above contains a flaw: software does not evolve.

Duh, John… everyone knows that software changes. Features creep. Scope broadens. New platforms and whizbangs are targeted. Get with it!

I concede the obvious: of course software changes. But repeat after me: software does not evolve. Because change != evolution.

Evolution is a blind, natural process; the result of random mutations in an organism. Now it may just so happen that the result of the mutation is beneficial to the reproductive success of the organism, meaning we’d expect to see creatures with such a trait outperform others without it. That's how traits are selected for. In the overwhelming majority of cases, mutations are detrimental, and they don't stick around for long (since there are many, many more ways of being dead than alive).

Now in order to say that software evolves, you'd have to accept that your development process goes something like this: Developer opens a file at random, positions the cursor at random, punches a few keys at random. Developer then recompiles and sees what happens, hoping for the staggering luck that the resulting change actually does improve the software, and everybody loves it, so they buy it, and you'd expect to see more of it.

Okay, insert joke here about how your development process seems that way from time to time.

Jocularity aside, there's more at issue here than a flawed analogy. Of more significance is the type of thinking it can engender. Nothing "just happens" in software. Whatever it is, somebody made it happen. Someone decided. They may very well have decided in error, but they decided. They decided "well, let's just try and fit that feature in; it shouldn't cause too many problems if it goes out only 70% tested... if it breaks, we'll deal with it then." Or they think "yeah, a talking paperclip… why not?" In other words, magical thinking. Don’t do that.

And CS departments should stop teaching that. Let them stress peopleware instead.

His presumption here, which may seem fair at first, is that all evolution is basically random. And, frankly, that's not entirely without truth. But what he sees as the randomness in the system is different from the randomness that I see, and that's that the users are what bring the randomness into the system.

Look, how many times hasn't a user told you, "We need this feature", only to discover six months after shipping that feature that nobody's really used it, but that it in turn sort of answered a different problem that you ended up providing for them as a new feature? See, the software itself doesn't evolve randomly, but the users' interactions with the software do. That's evolution, that's healthy, and that's how software evolves.

In short, it's recognizing that the users are part of the system, too, part of the organism that makes up this bizaare and wonderful world we live in, and their input is often exactly that: random. Which is probably why it's so important to have the on-site customer, as per agile development's recommendation, because you never know when randomness will strike and make your life better/faster/easier.

I gave two talks at the JavaZone 2005 conference, which I've made available here, "Concrete Services" and a few items from "Effective Enterprise Java", because I didn't get the slides into the organizers in time for them to include on their site. Enjoy.

The title is a bit scary, but "Rootkits", by Hoglund and Butler, really is anything but. Oh, I'll admit, their talk of how rootkits--programs that hackers install onto your system that patch into kernel space and thus are undetectable by any user-mode program--is scary, but then they walk you through the process of developing your own rootkit, thereby giving you some awareness of what a rootkit looks like, acts like, and therefore can be discovered and killed.

Well, in theory, anyway.

To put it bluntly, I'm loving this book, if only because it's the first book I've run into that really sits down and explains how to write a device driver, not to mention how to communicate with it from user mode. I've been fascinated with that very idea for many years now, but all the DDK-based material I've found--books, articles, etc--all assumed that you wanted to write some variation on the SCSI driver or something, implying that you care more about device-level details than you do in writing kernel-mode code. Rootkits, of course, are nothing like real device drivers, but a lot more like what I'm interested in exploring and displaying (that is, getting at program information from within the kernel--very useful for debugging scenarios, for example).

By page 30, you've already written and compiled a basic kernel driver, and by page 39 they've discussed how you can have your driver expose itself as a special file handle for communication with user-mode code. Pages 40-43 talk about loading the driver from code, and 43-46, how to extract your driver from a user-mode program as a resource, suitable for loading (because, of course, rootkits need to piggyback on top of other code to install themselves, stealthy-like). Pages 46-47 talk about how to make your rootkit survive reboot, and that concludes Chapter Two.

Wow. I'm in love.

It's not the be-all-end-all book on drivers, nor is it going to necessarily turn you into a l33t hax0r, but if you ever wanted to get started understanding how rootkits work (so as to start looking for them on your own system in order to remove them) or just use that knowledge for more benign purposes (such as trying to figure out NT internals so you can more efficiently--and automatedly--debug services or server-style programs), this book rocks. Easily a classic, and one I'm probably going to carry around with me as much as I do Hoglund's other book (with Gary McGraw, one of my favorite security authors), "Exploiting Software".

I'm in Oslo, Norway, for the next four days, ostensibly to speak at the JavaZone 2005 conference, but the truth is, I don't really care why I'm here.

Truth is, I've discovered that Oslo is quite possibly the closest place on the planet to claim being a real-life Norman Rockwell scene.

The drive from the Oslo airport to downtown Oslo (to the Radisson SAS hotel) is quite possibly one of the most beautiful drives I've ever had the pleasure of making--it really is like driving through a Norman Rockwell painting, with the farm fields off to both sides, thick lush forests rising on the hills, and the buildings just barely visible, nestled in amongst the trees and rising slopes. If it weren't for the fact that I know this place is going to be just buried in snow in the next month or two, I'd seriously consider living here for a while.

Oh, and I've also discovered that Norway is one of the western European nations that doesn't (yet?) take the Euro--tip to American travelers, don't take cash out at the Amsterdam airport when you're headed off to a non-Euro country. It's an expensive mistake.

For anybody who's not been paying attention to the technical news front, this week is Microsoft's PDC in LA, and one of the things they've announced for the next release of Visual Studio is Project LINQ, short for Language INtegrated Query. In essence, C# 3 and VB 9 are going to integrate (through a variety of language extensions, such as lambda expressions) query capabilities directly into the language, making much of the need for an automatted O/R mapping layer (such as Hibernate or JDO) a thing of the past (at least, in theory).

If you haven't had a look, check out Dion's or Ben's weblogs for details; there's also a paper coming out (by Don Box and Anders Hjalsberg) with the LINQ Preview bits that contains the best description of the language/tools I've seen thus far.