OK, OK, I admit it. Maybe significant whitespace isn't all bad. (But don't let me ever catch you quoting me say that.)

The reason for my (maybe) shift in thinking? Manning Publications sent me a copy of Iron Python in Action, and I have to say, I like the book and its approach. Getting me to like Python as a primary language for development will probably take more than just one book can give, but... *shrug* Who knows?

Bear in mind, I have plenty of reasons to like IronPython (Microsoft's Python implementation for the .NET environment):

• A good friend of mine, Harry Pierson (aka @DevHawk), is the PM on the IPy project, and I'm generally prejudiced in favor of those things that people I know and respect.
• I'm generally a fan of dynamic languages, particularly those that let you do strange and twisted things to the type system and its instances at runtime. (Yes, I'm looking at you, ECMAScript...)
• I spent some quality time with IronPython Studio last year while researching a Visual Studio Extensibility "Deep Dive" paper.
• I've known Jim Hugunin (the creator of IronPython, and Jython before that) for some years, ever since his days working on AspectJ, and he's one of those scary-smart guys that, despite knowing they're scary-smart, still render me stunned when I listen to them.
• I'm a huge fan of the DLR. It's like having Parrot, but without having to wait a decade (give or take).

But, just to counterbalance the scales, I have plenty of good reasons to dislike IronPython, too:

• Significant whitespace.
• The "There's only one way to do it" oath that Pythonistas seem to hold as religion. (Somebody told me that building C-Python—the original implementation—only works for you if you swear a holy oath to The One True Way on the One True Way Bible. Needless to say, I believe them, and have never tried to build C-Python from sources as a result.)
• Significant whitespace.
• Uh.... did I mention significant whitespace yet?

I admit, it was with some hesitation that I cracked open the book. Actually, to be honest, I was really ready to just take out all my dislike of significant whitespace and pour it into a heated, vitriolic diatribe on everything that was just wrong with Python.

And...?

Well, OK, I admit it. Maybe significant whitespace isn't all bad.

But this is a review of the book, not the technology. So, on we go.

What I liked about the book

• The focus is on both .NET and Python, and doesn't try to short-change either the "Python"-ness or the ".NET'-ness by trying to be a "Python book (that happens to run on .NET)" or a ".NET book (that happens to use Python for code samples)". The authors, I think, did a very good job of balancing the two, making this the book to get if you're in that area on the Venn diagram where "Python" overlaps with ".NET".
• Part 2, "Core development techniques", starts down the "feed you the Python Kool-Ade" pretty quickly, heading straight into Chapter 4 ("Writing an application and design patterns with IronPython") without much of a pause for breath. The authors get into duck typing, protocols, and Model-View-Controller within the first four pages, and begin working on a running example to highlight some of the ideas. (Interestingly enough, they also take a few moments to point out that IronPython on Mono works, and include a couple of screen shots to that effect as we go, though I personally wonder just how many people are really going down this path.) I like the no-holds-barred, show-you-the-code style, but only because they also take time throughout the prose to talk about some of the concepts at work underneath and laced throughout the code. "Show me then tell me" is a time-honored tradition, but too many authors forget the "tell me" part and stop with code. These guys do a good job of following through.
• The chapters in Part 3, "IronPython and advanced .NET", form an interesting collection of how IronPython can fit into the rest of the .NET stack, demonstrating how to use IronPython with WPF, ASP.NET, and IronPython's crowning glory, Silverlight. If you're into front-end stuff, this is the section where I think you're going to have the most fun.
• The chapters in Part 4, "Reaching out with IronPython", is I think the most important part of the book, showing how to extend IronPython (chapter 14) with C#/VB extensions (similar to how a C-Python developer would extend Python by writing C code, but much much simpler) and the opposite—how to embed IronPython inside of existing C#/VB applications (chapter 15), which is really an exercise in using the DLR Hosting APIs. While the discussion in chapter 15 is good, I wish it'd had a bit more thorough discussion of how the DLR could be hosted regardless of the scripting language, though I admit that's pretty beyond the scope of this book (which is focused, after all, entirely on IronPython, and as a result should stay focused on how to host IPy).

What I found "Meh" about the book

• Part 1 ("A new language for .NET", "Introduction to Python", and ".NET objects and IronPythong") does a good job of bringing the rank beginner up to speed, getting some basic Python ideas across in the same breath that they bring .NET home. The only problem is, it only works well if you're neither a Python programmer nor a .NET programmer. Chapter 1, for example, does a sort of Cannonball-into-the-pool kind of dive into Python, but dives equally into the "Iron" parts as it does the "Python" parts. If you're either a Pythonista or a .NETter, I suspect you're going to be tempted to flip pages pretty quickly, and (I suspect) miss a few things. Chapter 2 is all about Python (meaning .NETters will probably spend some time here), but it certainly doesn't feel like an exhaustive reference, nor does Chapter 3 stand as an exhaustive discussion about all things .NET, either. I almost wish all three chapters had been collapsed into one—suffice it to say, I don't feel like I know the Python language, and don't feel like this book could be my Python reference next to me as I learn it, and I know that it's not a great .NET reference, either. Fortunately, the goal of these three chapters feels pretty clearly to be "Teach you just enough to make you dangerous (and able to understand the rest of the book)", and once we hit Part 2, rubber meets road pretty quickly.
• By the time you hit Chapter 7, less than halfway through the book, the authors have created a fairly nice, if simplistic, application for later dissection, but it's not until you hit Chapter 7 that they begin to start unit-testing, even though they insist (on page 17) that "Dynamic language programmers are often proponents of strong testing rather than strong typing" (a quote they attribute to Bruce Eckel, though I'm relatively certain I heard Dave Thomas and Neal Ford say it with respect to Ruby, long before Eckel started "Thinking in Python... or Flex... or whatever"). If unit-testing is that important, why wait three chapters into the application's development before writing a single unit-test? This doesn't jibe with me, somehow.
• If you're into back-end stuff, chapter 12 on "Databases and web services" is pretty bland. The fact that the two are combined into a single chapter is indicative, all by itself, of how deep or intensive the coverage goes, and there's zero mention of anything beyond basic ADO.NET. The coverage on web services covers REST relatively well, but there's zero coverage of WCF, and the whole of SOAP-based services is all of four or five pages. And Workflow? Doesn't exist, isn't even mentioned (except for an appearance in a table, "The major new APIs of .NET 3.0"). Yikes.

What I actively disliked about the book

Actually, not much. Manning did their usual superb job of arrowed callouts to point out particular concepts in the code listings, the copyediting is professional (meaning there's no obvious typos or misspellings that just break up the flow of prose, something that not all publishers seem to take seriously), and the graphics flow nicely alongside the prose, not dominating the page but accentuating it.

In fact, about the only thing I'd care to criticize is the huge number of footnotes, particularly in the first chapter. (By page 20 in the book, there have already been 30 footnotes.) When you have three footnotes per page, on average (and sometimes more), it does tend to distract, at least to me it does. It feels like there were ways, for most of them, to inject the idea or concept into the main prose, or leave it out entirely, but that could just be a difference of writing style, too.

Summation

If you're a .NET developer interested in learning/using IronPython on your next project, this is a definite winner. If you're a Python developer looking to see how to break into .NET, I'm not so sure this is your book, but I say that mostly because I'm not a Pythonista and can't really speak to how that mindset will find this as an introduction to the .NET space. My intuition tells me that this would be a good springboard into another book on .NET for the Python programmer, but I'll have to leave that to Pythonistas who've read this book to comment one way or another.

(Disclaimer: In the spirit of full disclosure, Stu is a friend, fellow NFJS speaker, and former co-worker of mine from DevelopMentor.)

I present this review to you in two parts.

Short version: If you want to learn Clojure, and you're familiar with at least one programming language, you'll find this a great resource. If you don't already know a programming language, or if you already know Clojure, or if you're looking for "best practices" to cut-and-paste, you're going to be disappointed.

Long version: Recently, fellow NFJS speaker Stu Halloway decided to take up a new language, and came to Clojure. He found the language interesting enough to write a book on it, something he hasn't done since his Java days, and the result is a nice walk through the language and its environment for experienced Java developers who want to understand Clojure's language, concurrency concepts, and programming model.

Now, let's be 100% honest about this: if you're coming at this book expecting it to be a language reference, you will probably be disappointed (as this guy obviously is). Stu's not like that—he's not going to re-create material that's available elsewhere, or that can be found with an easy Google search. Stu will not waste your time that way—he wants to tell you a story, one that takes you from "I'm a Java guy, but clueless about Lisp, dynamic languages, functional programming, concurrency, or macros" to "Wow. I know kung-fu." in the shortest path possible, but without trying to lobotomize you. He wants—no, expects—the readers of his book to be propping the text open with a cell phone on one side and the dinner plate on the other, craning your neck over to scan the pages and type in the examples into the REPL shell to try them out, see them work, then spend a few minutes experimenting with them before moving on to the next paragraph or page.

(Oh, I suppose you could just cut and paste them from the PDF version of the book, but where's the fun in that?)

The fact is, the concepts behind Clojure make up what's important to learn here, and readers of this book will come away like the panda from the movie, realizing that "There is no Secret Ingredient", that the power of Clojure comes not from its super-secret language sauce or special libraries, but in the way Clojure programmers approach problems and think about programming. And for that reason, if you're a programmer—even if you don't program on the JVM—you really want to take a look at what Stu's talking about (and Rich Hickey is creating).

Just remember, cellphone and dinner plate. Otherwise you'll be missing out on so much.

Well, OK, the title is trolling ever so slightly, but there is an interesting trend at work, and I'm genuinely concerned about its ultimate expression if the trend continues to its logical conclusion. Have a look and tell me if you agree or disagree.

Apparently the Rails community isn't the only one pursuing that ephemeral goal of "edginess"—another blatantly sexist presentation came off without a hitch, this time at a Flash conference, and if anything, it was worse than the Rails/CouchDB presentation. I excerpt a few choice tidbits from an eyewitness here, but be warned—if you're not comfortable with language, skip the next block paragraph.

Yesterday's afternoon keynote is this guy named Hoss Gifford — I believe his major claim to fame is that viral "spank the monkey" thing that went around a few years back.  Highlights of his talk:

• He opens his keynote with one of those "Ignite"-esque presentations — where you have 5-minutes and 20 slides to tell a story — and the first and last are a close-up of a woman's lower half, her legs spread (wearing stilettos, of course) and her shaved vagina visible through some see-thru panties that say "drink me," with Hoss's Photoshopped, upward-looking face placed below it.
• He later demos a drawing tool he has created (admittedly with someone else's code) and invites a woman to come up to try it.  After she sits back down, he points out that in her doodles she's drawn a "cock."
• Then he decides he wants to give a try at using the tool to draw a "cock" (he loves this word) — and draws a face, then a giant dick (he redraws it three times) that ultimately cums all over the face.
• A multitude of references to penises and lots of swearing — and also "If you are easily offended, fuck you!"
• And then, to top it off, a self-made flash movie of an animated woman's face, positioned as if she's having sex with you, who gradually orgasms based on the speed of your mouse movement on the page.

Wow. Just... wow. To call this unprofessional smacks of calling Hitler a "socially awkward individual"... or using a euphemism like "mild medical condition" to refer to death. This is so far "over the line" that it's unbelievable. Even Mr. Aimonetti's "CouchDB" presentation, as bad as it was, at least tried to tie the analogy together in a meaningful, if offensive, way. This is just male posturing at its worst. (I'm shocked Hoss didn't whip off his pants and demand the women in the room bow down in worship to his obviously superior manhood.)

Fortunately, according to the source, the conference organizer seems to be pretty responsive, so kudos to the one adult in the room, but....

What's worse, apparently the presenter and more than a few of his pals are (in the best traditions of assholery) blatantly unrepentant about the whole thing, claiming the moral high ground in much the same way that the Rails idiots did—it's all in good fun, if you don't find it funny you're a prude, and so on:

I checked Twitter (hashtag #flashbelt) to see what the responses were.  Here are some notable remarks:

• Fonx is reading the #flashbelt rants on Hoss offending the ladies w/ a few swear words & a penis drawing - r u really that prudish & sexist?
• nthitz lol @hoss69 "If you are easily offended, fuck you" #flashbelt
• livenootrac Ladies of #flashbelt , I am sorry for the Hoss preso, but in the flash community he gets a pass, kinda like Don Rickles - that's just Hoss.
• CujoJpn @livenootrac And there were many ladies at #flashbelt who were offended by Hoss' Preso some were thick skinned and took it as is.

So, if you didn't like it then
a) you are a prude - and sexist (?)
b) fuck you
c) suck it because Hoss gets a pass here in the boy's club known as "the flash community" and
d) you are a wimpy girl who isn't strong enough / man enough / "thick-skinned" enough  to deal with it.

Even more... wow. Talk about justification and marginalization. Amazing.

Before I figuratively smack this Hoss guy around the blog for a while, let's take a brief moment for reflection—what's going on here? Why all the misogynistic presentations recently? Is this reflective of a general trend in the programming industry? Of society in general? Is the world coming to an end?

A few possibilities present themselves:

• The lack of women in the IT industry means there's nobody around to act as a "gender filter" to keep things on an even keel. In other words, the genders constantly filter themselves based on the company they keep, and because the boys who put these presentations together don't have female input, they simply don't know where to draw the line for mixed company. This theory also presumes that an industry that's made up primarily of women will also lack such a filter and "girls will be girls" as a result. Unfortunately I have no good counterexamples at hand to examine—anybody know of an industry populated primarily by women, and can weigh in with experience there? The closest I get is my brief experience working in a restaurant with an almost-all-woman serving staff, and from that brief experience, yep, the theory holds. Solution? Easy: get more women in IT, and things will re-balance themselves naturally.
• Programmers are principally males who have no redeeming social skills. In other words, the industry gathers up exactly the kind of men who find objectifying women and reveling in late-acquired testosterone overdoses to be gratifying, and this kind of behavior is the result. If true, it leads to the conclusion that programmers are no more evolved than the Navy sailors involved in the Tailhook scandal of a few years ago. So go ahead, smack your wives and girlfriends around a little if they get a little "uppity", it's OK, 'cuz u r a l33t d00d. Personally? I find the idea ludicrous—there is definitely a strong antisocial streak that runs through the IT ecosystem (how many of you met your friends via World of Warcraft again?), but like all stereotypes, there's some elements of truth to it, and a lot of exaggeration. And frankly, anybody who believes in this theory is welcome to come with me to dinner at a No Fluff Just Stuff show and meet the other speakers, and listen in on our "boys club" conversations, including questions like, "Which movie best represents the book it was made after?" and "If given a mandate to create a programming language, what language would your language most resemble?". Oh, and the odd fart joke. We are boys, after all.
• We're hypersensitive to the subject right now. In other words, these kind of presentations have always been going on, and it's just that we notice them now, in the same way that you notice a particular brand of car on the road a lot more when you're thinking about buying that brand and model of car. Frankly, I don't buy this argument—I've been to a lot of presentations over the past decade, and I've never seen any that were anything like this.
• This is the YouTube generation, with access to everything the Internet has to offer, and this is "just how they do things". After all, how much maturity, sexual discretion and adult behavior can we expect of the generation that gave us "Girls Gone Wild" and its ilk? It's just a "generation gap" thing, and we old fogies who didn't grow up with Internet porn just a browser-click away just don't "get it". Hmm.... somehow, I just don't buy it. Sure, there may be some elements of this involved here (I'm really curious to see what all these "Girls Gone Wild" girls are going to say to their own daughters in a decade or so...), but I think that's too easy an answer, and an eminently unhelpful one.
• We have copycatters out there trying to follow the path of people they respect. If you're looking up at this Hoss character and thinking, "I want to be just like him!", you really should see a therapist and develop a sense of self, before you find yourself without friends. Hoss gets a pass because of your misguided fan-boi hero-worship. So does Paris Hilton. You want to be the Paris Hilton of your social circle? Go for it. After all, she's highly respected and loved, right? Take a clue from the next car wreck you drive past—everybody's slowing to look not because they wish they were in the body bag, folks, but because we have a ghoulish fascination with it. In the case of Ms. Hilton, that ghoulish fascination is with those who self-destruct in spectacular fashion. (Me, I'd love to be the fly on the wall at the Hoss residence when he tries to explain this whole thing to his daughter or his date/girlfriend/wife, if he ever finds one.)
• The presenters taking this tack are looking for an easy path to fame. In the grand traditions of Andrew Dice Clay ("Oh!"), the easiest way for a presenter to "stand out" from the rest of the crowd of presenters is to do something outrageous and call it "edgy", and stake out a claim on the edge of the civilization, rather than try to integrate with the rest of the crowd and build something up slowly. Don Box has already claimed "HTTP is dead", I made the analogy between a technology and a military conflict, and Matt Aimonetti claimed a data storage framework "performs like a pr0n star", so what's left but to stake out ground even further out on the fringe and just be misogynistic? Fortunately, history suggests that people with content-free/shock-heavy presentations (or even content-heavy/shock-heavy ones) don't go the distance, so to speak, and that once there's nowhere more shocking left to go, the audience comes back to the content-heavy/shock-light discussions and stays there for a while. Unfortunately, this means we're going to have to suffer through somebody's "Live YouPorn filming" talk first, which I'm not looking forward to.

And now for the smacking around... but you know, I suddenly realize that the volume of comments on the original post leave with nothing to do or say that's not already being said, so to just "pile on" would only serve to let me vent, and I have other outlets for that. But it would be inappropriate to just "walk away", so to speak, so with that in mind....

Hoss, you're an idiot. Like any sprinter, you're going to head up the pack for a bit, but soon enough, your "shtick" is going to flame out and you'll be left behind with all the other "shock jocks" of the 80's who found their material unwelcome after a while. So enjoy the spotlight (such as it is) while you can. In the meantime, I'm off to revise a few presentations, and stick with solid ideas and analogies, and maybe dropping the odd F-bomb when I want to make a point, just for emphasis, because I know something you apparently don't:

Shock makes a point because of the contrast to the rest of the talk, not because of its inherent "edginess".

Meanwhile, by all means, continue to be an idiot. You just make me look better by comparison, for which I thank you.

Update: See below, but I wanted to include the text Mike Abercrombie (DM's owner) posted as a comment to this post, in the body of the blog post itself. "Ted - All of us at DevelopMentor greatly appreciate your admiration. We're also grateful for your contributions to DevelopMentor when you were part of our staff. However, all of us that work here, especially our technical staff that write and delivery our courses today, would appreciate it if you would check your sources before writing our eulogy. DevelopMentor is open for business and delivering courses this week and we intend to remain doing so." Duly noted, Mike. Apology offered (and hopefully accepted).

An email crossed my desk today, announcing that DevelopMentor, home to so many good people and fond memories, has (at least temporarily) closed its doors.

I admit to a small, carefully-cushioned place in my heart where I mourn over this.

DevelopMentor was such a transcendent place for me. Much, if not most or all, of the acceleration that came in my career came not only while I was there, but because I was there.

So much of my speaking persona and skill I owe to Ron Sumida, who took a half-baked neophyte of intermediate speaking skill, and in an eight-hour marathon session still referred to in my mental memoirs as my "Night with Scary Ron", shaped me and taught me tricks about speaking that I continue to use to this day. That I got to know him as a friend and confidant later still to this day ranks as one of my greatest blessings.

I remember my first DM Instructor Retreat, where I met so many of the names I'd read about or heard about, and feeling "Oh, my God" fanboy-ish. I remember Tim Ewald giving a talk on transactions at that retreat that left me agape—I seriously didn't understand half of what he was saying, and rather than feeling overwhelmed or ashamed, I remember distinctly thinking, "Wow—I have found a home where I can learn SO much more." It was like waking up one morning to find that your writing workshop group suddenly included Neal Stephenson, Stephen Pinker, C.S. Lewis and Ernest Hemingway. (Yes, I know those last two are dead. Work with me here.)

I remember the day that Lorie (the ops manager at the time) called me to say that Don Box wanted me to work with him on the C# course. I was convinced that she'd called the wrong Ted, meaning instead to reach for Ted Pattison in her Rolodex and coming up a few letters shy. She tartly informed me, "No, I know exactly who I'm talking to, and are you interested or not?" How could I refuse? Help the Diety of COM write DM's flagship course on Microsoft's flagship technology for the next decade? "Hmm...", I say out loud, not because I needed time to think about it, but because a thread in the back of my head says, "Is there any scenario here where I say no?"

I still fondly recall doing a Guerilla .NET at the Torrance Hilton shortly after the .NET 1.0 release, and having a conversation with Don in my hotel room later that night; that was when he told me "Microsoft is working on an open-source version of the CLR". I was stunned—I had no idea that said version would factor pretty largely in my life later. But it opened my eyes, in a very practical way, to how deeply-connected DevelopMentor was to Microsoft, and how that could play out in a direct fashion.

When Peter Drayton joined, he asked me to do a quick review pass on the reference section of his C# in a Nutshell, and I agreed because Peter was a good guy (and somebody I'd hoped would become a friend), and wanted to see the book do well. That went from informal review to formal review to "well, could you maybe make it an editing pass?" to "Would you like to write a few chapters?" to "Well, let's sign you up as a co-author...". That project is what introduced me to John Osborn, which in turn led him to call me one day and say, "Some guys at Microsoft are working on an open-source version of the CLR, and would like to have a 'professional writer' help them write a book on it. Interested?" That led to SSCLI Internals, working with David Stutz, and wow, did I learn a helluvalot from that project, too.

Effective Enterprise Java came through DevelopMentor, thanks again to Don Box, who introduced me to the folks at Addison-Wesley that put the contract (and Scott Meyers, another blessing) in front of me.

DM got me my start in the conference circuit, as well. In 2002, John Lam pinged me over email—he'd recently become track chair for Connections down in Orlando, and was I interested in speaking there? I was such a newbie to the whole idea, but having taught classes roughly twice every month, I wasn't worried about the speaking part, but the rest of the process. John walked me through the process, and in doing so, set me down a path that would almost completely redefine my career within a year or so.

Even my Java chops got built up—the head of our Java curriculum was Stu Halloway (recently of Clojure fame), and between him, Kevin Jones, Si Horrell, Brian Maso and Owen Tallman, man, did I feel simultaneously like a small child among giants and like a kid in a candy store. Every time I turned around, they'd discovered something new about the Java platform that floored me. Bob Beauchemin has forgotten more about databases in general than I will ever learn, and he had some insights on the intersection of Java + databases that still hang with me today.

And my start with No Fluff Just Stuff came through DevelopMentor, too. Jason Whittington heard through a mutual friend (Erik Hatcher, of Ant fame) about this cool little conference being held in Denver, and maybe I should look into it. That led to an email intro to Jay Zimmerman, a dinner together while I was teaching in Denver a few weeks later, and before I knew it, I was on the Denver NFJS schedule, including the speaker panel, where I uttered the then-infamous line, "Swing sucks. Get over it."

DevelopMentor, you shaped my career—and my life—in so many ways, you will always be a source of pleasant memories and a group of friends and acquaintances that I would never have had otherwise. Thank you so much.

Rest in peace.

Update: Well, as it turns out, I have to rescind at least part of my eulogy, as the post itself generated quite a stir—the folks at DevelopMentor were pretty quick to email me, pointing out that they're still alive and well. In fact, as one of them (a friend of mine still working there) put it, "We were all kinda surprised when we came to work this morning and discovered that we could go home." Fortunately, the DevelopMentor folks were pretty gracious about what could've been a very ugly situation, and I apologize for to them for the misunderstanding—all I can say is that my "source" must've also been mistaken, and I'm glad that we're all still good. And lest it need to be said out loud, I heartily want nothing but the best for DM, and hope that I never have to write this message again.

Joel's weblog appears to be down, so in response to some emails I've posted my draft copy of SSCLI 2.0 Internals here. I think it's the same PDF that Joel had on his weblog, but I haven't made absolutely certain of the fact. :-/

If you've not checked out the first version of SSCLI Internals, it's cool—the second edition is basically everything that the first edition is, plus a new chapter on Generics (and how they changed the internals of the CLR to reflect generics all the way through the system), so you're good. And if you're not sure where to get the codebase for Rotor 2.0 (the SSCLI), well, here, I'll make it easy for you.

Gotta say, this is almost without question my favorite book to have written. Just wish Microsoft would've kept Rotor up with the successive CLR releases (3.5 SP 1 and now the forthcoming 4.0). Maybe, if I can find that wishing ring....

Yep, you heard that right—Aaron Erickson, author of The Nomadic Developer, is now a ThoughtWorker.

For those of who you don't know Aaron, he's been a consultant at another consulting company for a while, and has been exploring a number of different topics in the .NET space for a few years now, not least of which is one of my favorites (F#) and one of THoughtWorks' favorites (agile). He's been speaking at a number of events, including the Connections conferences, and he's going to bring some serious market-development potential to our Chicago office, something that's obviously of concern right now in these current economic conditions.

He also cooks a mean bacon-wrapped scallop, but that's another story for another day.

I'm looking forward to having him be a part of the growing collection of .NET rock stars at ThoughtWorks. Wanna come join us? Always room for a few more....

These are the things I think as I wing my way out of LA fresh from this year's TechEd 2009 conference:

• I think I owe the attendees at DTL309 ("Busy .NET Developer's Guide to F#") an explanation. It's always embarrassing when your brain freezes during a presentation, and that's precisely what happened during the F# talk—I completely spaced on the syntax for implementing an interface on a class in F#. (To the attendees who commented "consider preparing a bit better so you dont forget the sintax :)" and "Not remembering the language syntax sorta comes across bad doesn't it?", you're absolutely right, which prompts this next sentence.) I apologize profusely to those who were there—I just blew it. For the record, the missing syntax looks like this:

  #light
  
  type IStudy =
      abstract Study: string -> unit
  
  type Person(firstName : string, lastName : string, age : int) =
      member p.FirstName = firstName
      member p.LastName = lastName
      member p.Age = age
      override p.ToString() = 
          System.String.Format("[Person: firstName={0}, lastName={1}, age={2}]",
              p.FirstName, p.LastName, p.Age);
              
  type Student(firstName : string, lastName : string, age : int, subject : string) =
      inherit Person(firstName, lastName, age)
      interface IStudy with
          member s.Study(sub : string) =
              System.Console.WriteLine("Hey, Ma, I'm studying {0}!", sub)
      member s.Subject = subject
      override s.ToString() =
          System.String.Format("[Student: " + base.ToString() + " subject={0}]", s.Subject);
  

  
  
  Truth is, though, right now not a lot of people (myself included) are writing types that formally implement a given interface—the current common practice appears to be an object expression instead, something along these lines:

  let monkey =
      { new IStudy with
          member p.Study(subject : string) =
              System.Console.WriteLine("Oook eeek aah aah {0}!", subject) }
  monkey.Study("Visual Basic")
  

  
  
  In this way, the object handed back still implements the interface type that the client wants to call through, but the defined type remains anonymous (and thus provides an extra layer of encapsulation against implementation details leaking out). The most frustrating part about that particular snafu? I had a Notepad window open with some prepared code snippets waiting for me (a fully-defined Person type, a fully-defined Student type inheriting from Person, and so on) if I needed to grab that code because typing it out was taking too long. Why didn't I use it? I just forgot. Oy.....
• Clearly Microsoft is thinking big things about Azure. There were a lot of sessions around Azure and cloud computing, far more than I'd honestly expected, given how new (and unreleased) the Azure bits are. This is a subject I would have expected to see covered this deeply at PDC, not TechEd.
• TechEd Speaker Idol is a definite win, to me. I watched the final round of Speaker Idol on Thursday night (before catching the redeye out to Atlanta for the NFJS show there this weekend), and quite honestly, I was blown away by the quality of the presentations—they were all of them better than some of the TechEd speakers I'd seen, and it was great to hear that not only will the winner, who did a great presentation on legacy application support in Windows7 (and whose name I didn't catch, sorry) be guaranteed a slot at TechEd, but I overheard that the runner-up, a Polish security expert who demoed how to break Process Explorer (in front of Mark Russinovich, no less!), will also be speaking at TechEd Berlin this year.
• As always, the parties at TechEd were where the real value lay. This may seem like an odd statement to those whose heads are a bit full right now from five days' worth of material (six, if you attended a pre-con), but remember that I'm a speaker, so the sessions aren't always as useful as they are to people who've not seen this content before (or have the kind of easy access to the people building it and/or presenting it that I'm fortunate and privileged to have). Any future attendees should take serious note, though: networking is a serious part of this business, and if you're not going out to the parties (or creating a few of your own while you're there) and handing out business cards left and right, you're missing a valuable opportunity.
• I'm looking forward to TechEd 2010. Particularly because, thanks to a few technical snafus, I had the chance to sit down with the folks who organize and run TechEd and vent for a little bit about everything I found annoying (as a speaker). Not only were my comments not blown off, but it started a really productive discussion about how to make the behind-the-scenes experience for the TechEd speakers a more pleasant and streamlined one. What's more, we're planning to revisit some of these discussions in the months to come as they start their preparations for TechEd 2010 (in New Orleans). I'm looking forward to those conversations and (hopefully) helping them eliminate some of the awkwardness that I've seethed over in the past.

New Orleans in the summer will not be an entirely wonderful experience (I'm told it gets monstrously humid there in the summers, but it can't be any worse than Orlando is/was), but I'm honestly very curious to get back there to see what post-Katrina New Orleans looks and feels like, and to maybe do my (very little) part to help the area claw its way back by maybe staying an extra day or two and taking in some of the sights. (I'm hoping that Sara Ford will be willing to act as tour guide.....)

In the meantime, thanks to all of you who came, and remember—if you attended a talk and you want to say "thanks" to the speaker who gave it, the best way is to take the five minutes to fill out the evals for that talk. (Speaking personally, I don't even care so much about the scores you give me, but the comments are absolutely invaluable.)

See y'all next year!

Recently, NFJS alum and buddy Dion Almaer questioned the widespread, almost default, usage of a relational database for all things storage related:

Ian Hickson: “I expect I’ll be reverse-engineering SQLite and speccing that, if nothing better is picked first. As it is, people are starting to use the database feature in actual Web apps (e.g. mobile GMail, iirc).”

When I read that comment to Vlad’s post on HTML 5 Web Storage I gulped. This would basically make SQLite the HTML 5 for storage in the browser. You would have to be a little crazy to re-write the exact semantics (including bugs) of SQLite and its dialect. What if you couldn’t use the public domain code?

Gears lead out strong with making a relational database part of the toolbox for developers. It embedded its own SQLite, in fact one that was customized to have the very cool full text search ability. However, this brings up the point of “which SQLite do you standardize on?”

The beauty of using SQL and SQLite is that many developers already know it. RDBMS has been mainstream for donkey’s years; we have tools to manage SQL, to view the model, and to tweak for performance. It has gone through the test of time.

However, SQL has always been at odds with many developers. Ted Neward brought up ORM as the vietnam of computer science (which is going a touch far ;). I was just lamenting with a friend at Microsoft on how developers spend 90% of their time munging data. Our life is one of transformations, and that is why I am interested in a world of JavaScript on client and server AND database. We aren’t there yet, but hopefully we can make progress.

One of Vlad’s main questions is “Is SQL the right API for Web developers?” and it is a valid one. I quickly found that for most of my tasks with the DB I just wanted to deal with JSON and hence created a wrapper GearsDB to let me insert/update/select/delete the database with a JSON view of the world. You probably wouldn’t want to do this on large production applications for performance reasons, but it works well for me.

Now a days, we have interesting APIs such as JSONQuery which Persevere (and other databases) use. I would love to see Firefox and other browsers support something like this and let us live in JSON throughout the stack. It feels so much more Webby, and also, some of the reasons that made us stay with SQL don’t matter as much in the client side world. For example, when OODBMS took off in some Enterprises, I remember having all of these Versant to Oracle exports just so people could report on the darn data. On the client the database is used for a very different reason (local storage) so lets use JSON!

That being said, at this point there are applications such as Gmail, MySpace search, Zoho, and many iPhone Web applications that use the SQL storage in browsers. In fact, if we had the API in Firefox I would have Bespin using it right now! We had a version of this that abstracted on top of stores, but it was a pain. I would love to just use HTML 5 storage and be done.

So, I think that Firefox should actually support this for practical reasons (and we have SQLite right there!) but should push JSON APIs and let developers decide. I hope that JSON wins, you? I also hope that Hixie doesn’t have to spec SQLite :/

Dion's right when he says "developers spend 90% of their time munging data" and that "Our life is one of transformations", but I think he's being short-sighted and entirely narrow-minded when he says, "I am interested in a world of JavaScript on client and server AND database." Dion, I love you, man, but you're falling prey to the Fallacy of the One True Language. JavaScript (or ECMAScript, as its official name is given) is an interesting and powerful language, but why do you want to force your biases and perceptions on the rest of the world, man? You're being just as bad as the C++ or Java guys were in their heyday—remember when Java stored procedures were all the rage because "everybody knows that Java is the wave of the future"?

The fact is, from where I stand, there is no one storage solution or language solution or user-interface solution that is the Right Thing To Do in all situations. Not even inside the browser. There will be situations where a SQLite is the Right Thing, and other situations where a document-oriented JSON-like or CouchDB-like thing will be the Right Thing, and trying to force-feed one into a situation that's best solved by the other is a bad idea.

Dion alludes to my article about the Vietnam of Computer Science, but in fact, his suggestion charges right into another quagmire—how long before somebody starts trying to create a JSON-to-RDBMS adaption layer? Or JSON-to-CouchDB? Or things equally ridiculous? The fact is, data has three fundamentally different "shapes" to it, and trying to pound data from one shape into the other has all the efficacy and elegance to it just as much as pounding round pegs into square holes does. Dion even alludes to this with this paragraph:

One of Vlad’s main questions is “Is SQL the right API for Web developers?” and it is a valid one. I quickly found that for most of my tasks with the DB I just wanted to deal with JSON and hence created a wrapper GearsDB to let me insert/update/select/delete the database with a JSON view of the world. You probably wouldn’t want to do this on large production applications for performance reasons, but it works well for me.

JSON is certainly an attractive representation format for ECMAScript objects, thanks to its fundamental roots in ECMAScript's object literal syntax, and the powerful/dangerous eval() functionality offered by ECMAScript environments, but JSON also lacks a number of things a SQL-based dialect has, including a powerful query syntax for selecting individual and subsets of entities from the whole, which only becomes more and more necessary as the data base itself gets larger and larger. (Anybody who suggests that a local browser store would only remain within a certain size is clearly not thinking further ahead than the current day. Look at how cookies are outrageously abused as local storage for a lot of sites, or how Viewstate was abused in early ASP.NET apps—if you give the HTML/front-end developer a local storage mechanism, they will use it, and use it as far and as long and as hard as they can.) On top of which, JSON simply doesn't have the years of solid backing behind it than a SQL-based storage format does. And so on, and so on, and so on.

Ironically, just as JSON is a scheme for representing native objects in some kind of data format (in this case, a plain-text one), developers casually ignore the idea of storing objects in a native data format with all of the other bells-and-whistles that a database provides. Naturally, I'm referring to the idea of an object database—if JSON is appropriate for storing certain kinds of data in certain scenarios, then why isn't it appropriate to consider a native object database for some of those same certain kinds of scenarios? Not that I have anything against a JSON-based database scenario—in fact, I can easily imagine a JSON database that indexes the properties of the stored objects and takes ECMAScript functions as "native queries" in the same way that db4o doe. But let's stop with the repeated attempts at "one size fits all", and just accept that the world is a polyglot world, and that no one language—or data storage format, or data access API—will be the Right Thing To Do for all scenarios. Each language, format, API or tool has a reason to exist, a particular way it looks at the world, and optimizes itself to work best when used in that particular style. Trying to force one into the terms of the other is the road to another Computer Science quagmire.

Viva la Polyglot!

The Simple-Talk newsletter is a monthly e-zine that the folks over at Red Gate Software (makers of some pretty cool toys, including their ANTS Profiler, and recent inheritors of the Reflector utility legacy) produce, usually to good effect.

But this month carried with it an interesting editorial piece, which I reproduce in its entirety here:

When the market is slack, nothing succeeds better at tightening it up than promoting serial group-panic within the community. As an example of this, a wave of multi-core panic spread across the Internet about 18 months ago. IT organizations, it was said, urgently had to improve application performance by an order of magnitude in order to cope with rising demand. We wouldn't be able to meet that need because we were at the "end of the road" with regard to step changes in processor power and clock speed. Multi-core technology was the only sure route to improving the speed of applications but, unfortunately, our current "serial" programming techniques, and the limited multithreading capabilities of our programming languages and programmers, left us ill-equipped to exploit it. Multi-core mania gripped the industry.

However, the fever was surprisingly short-lived. Intel's "largest open-source effort ever" to provide a standard tool for writing multi-threaded code, caused little more than a ripple of interest. Various books, rushed out while the temperature soared, advocated the urgent need for new "multi-core-friendly" programming models, involving such things as "software pipelines". Interesting as they undoubtedly are, they sit stolidly on bookshelves, unread.

The truth is that it's simply not a big issue for the majority of people. Writing truly "concurrent" applications in languages such as C# is difficult, as you get very little help from the language. It means getting involved with low-level concurrency primitives, such as lock statements and so on.

Many programmers lack the skills to do this, but more pertinently lack the need. Increasingly, programmers work in a web environment. As long as these web applications are deployed to a load-balanced web farm, then page requests can be handled in parallel so all available cores will be used efficiently without the need for the programmer to be concerned with fine-grained parallelism.

Furthermore, the SQL Server engine behind these web applications is intrinsically "parallel", and can handle and use effectively about as many cores as you care to throw at it. SQL itself is a declarative rather than procedural language, so it is fundamentally concurrent.

A minority of programmers, for example games programmers or those who deal with "embarrassingly parallel" desktop applications such as Photoshop, do need to start working with the current tools and 'low-level' coding techniques that will allow them to exploit multi-core technology. Although currently perceived to be more of "academic" interest, concurrent languages such as Erlang, and concurrency techniques such as "software transactional memory", may yet prove to be significant.

For most programmers and for most web applications, however, the multi-core furore is a storm in a teacup; it's just not relevant. The web and database platforms already cope with concurrency requirements. We are already doing it.

My hope is that this newsletter, sent on April 1st, was intended to be a joke. Having said that, I can’t find any verbage in the email that suggests that it is, in which case, I have to treat it as a legitimate editorial.

And frankly, I think it’s all crap.

It's dangerously ostrichian in nature—it encourages developers to simply bury their heads in the sand and ignore the freight train that's coming their way. Permit me, if you will, a few minutes of your time, that I may be allowed to go through and demonstrate the reasons why I say this.

To begin ...

When the market is slack, nothing succeeds better at tightening it up than promoting serial group-panic within the community. As an example of this, a wave of multi-core panic spread across the Internet about 18 months ago. IT organizations, it was said, urgently had to improve application performance by an order of magnitude in order to cope with rising demand. [...] Multi-core mania gripped the industry.

Point of fact: The “panic” cited here didn’t start about 18 months ago, it started with Herb Sutter’s most excellent (and not only highly recommended but highly required) article, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software”, appeared in the pages of Dr. Dobb’s Journal in March of 2005. (Herb’s website notes that “a much briefer version under the title “The Concurrency Revolution” appeared in C/C++ User’s Journal” the previous month.) And the panic itself wasn’t rooted in the idea that we weren’t going to be able to cope with rising demand, but that multi-core CPUs, back then a rarity and reserved only for hardware systems in highly-specialized roles, were in fact becoming commonplace in servers, and worse, as they migrated into desktops, they would quickly a fact of life that every developer would need to face. Herb demonstrated this by pointing out that CPU speeds had taken an interesting change of pace in early 2003:

Around the beginning of 2003, [looking at the website Figure 1 graph] you’ll note a disturbing sharp turn in the previous trend toward ever-faster CPU clock speeds. I’ve added lines to show the limit trends in maximum clock speed; instead of continuing on the previous path, as indicated by the thin dotted line, there is a sharp flattening. It has become harder and harder to exploit higher clock speeds due to not just one but several physical issues, notably heat (too much of it and too hard to dissipate), power consumption (too high), and current leakage problems.

Joe Armstrong, creator of Erlang, noted in a presentation at QCon London 2007 that another of those physical limitations was the speed of light—that for the first time, CPU signal couldn't get from one end of the chip to the other in a single clock cycle.

Quick: What’s the clock speed on the CPU(s) in your current workstation? Are you running at 10GHz? On Intel chips, we reached 2GHz a long time ago (August 2001), and according to CPU trends before 2003, now in early 2005 we should have the first 10GHz Pentium-family chips.

Just to (re-)emphasize the point, here, now, in early 2009, we should be seeing the first 20 or 40 GHz processors, and clearly we’re still plodding along in the 2 – 3 GHz range. The "Quake Rule" (when asked about perf problems, tell your boss you'll need eighteen months to get a 2X improvement, then bury yourselves in a closet for 18 months playing Quake until the next gen of Intel hardware comes out) no longer works.

For the near-term future, meaning for the next few years, the performance gains in new chips will be fueled by three main approaches, only one of which is the same as in the past. The near-term future performance growth drivers are:

• hyperthreading
• multicore
• cache

Hyperthreading is about running two or more threads in parallel inside a single CPU. Hyperthreaded CPUs are already available today, and they do allow some instructions to run in parallel. A limiting factor, however, is that although a hyper-threaded CPU has some extra hardware including extra registers, it still has just one cache, one integer math unit, one FPU, and in general just one each of most basic CPU features. Hyperthreading is sometimes cited as offering a 5% to 15% performance boost for reasonably well-written multi-threaded applications, or even as much as 40% under ideal conditions for carefully written multi-threaded applications. That’s good, but it’s hardly double, and it doesn’t help single-threaded applications.

Multicore is about running two or more actual CPUs on one chip. Some chips, including Sparc and PowerPC, have multicore versions available already. The initial Intel and AMD designs, both due in 2005, vary in their level of integration but are functionally similar. AMD’s seems to have some initial performance design advantages, such as better integration of support functions on the same die, whereas Intel’s initial entry basically just glues together two Xeons on a single die. The performance gains should initially be about the same as having a true dual-CPU system (only the system will be cheaper because the motherboard doesn’t have to have two sockets and associated “glue” chippery), which means something less than double the speed even in the ideal case, and just like today it will boost reasonably well-written multi-threaded applications. Not single-threaded ones.

Finally, on-die cache sizes can be expected to continue to grow, at least in the near term. Of these three areas, only this one will broadly benefit most existing applications. The continuing growth in on-die cache sizes is an incredibly important and highly applicable benefit for many applications, simply because space is speed. Accessing main memory is expensive, and you really don’t want to touch RAM if you can help it. On today’s systems, a cache miss that goes out to main memory often costs 10 to 50 times as much getting the information from the cache; this, incidentally, continues to surprise people because we all think of memory as fast, and it is fast compared to disks and networks, but not compared to on-board cache which runs at faster speeds. If an application’s working set fits into cache, we’re golden, and if it doesn’t, we’re not. That is why increased cache sizes will save some existing applications and breathe life into them for a few more years without requiring significant redesign: As existing applications manipulate more and more data, and as they are incrementally updated to include more code for new features, performance-sensitive operations need to continue to fit into cache. As the Depression-era old-timers will be quick to remind you, “Cache is king.”

Herb’s article was a pretty serious wake-up call to programmers who hadn’t noticed the trend themselves. (Being one of those who hadn’t noticed, I remember reading his piece, looking at that graph, glancing at the open ad from Fry’s Electronics sitting on the dining room table next to me, and saying to myself, “Holy sh*t, he’s right!”.) Does that qualify it as a “mania”? Perhaps if you’re trying to pooh-pooh the concern, sure. But if you’re a developer who’s wondering where you’re going to get the processing power to address the ever-expanding list of features your users want, something Herb points out as a basic fact of life in the software development world ...

There’s an interesting phenomenon that’s known as “Andy giveth, and Bill taketh away.” No matter how fast processors get, software consistently finds new ways to eat up the extra speed. Make a CPU ten times as fast, and software will usually find ten times as much to do (or, in some cases, will feel at liberty to do it ten times less efficiently).

...  then eking out the best performance from an application is going to remain at the top of the priority list. Users are classic consumers: they will always want more and more for the same money as before. Ignore this truth of software (actually, of basic microeconomics) at your peril.

To get back to the editorial, we next come to ...

However, the fever was surprisingly short-lived. Intel's "largest open-source effort ever" to provide a standard tool for writing multi-threaded code, caused little more than a ripple of interest. Various books, rushed out while the temperature soared, advocated the urgent need for new "multi-core-friendly" programming models, involving such things as "software pipelines". Interesting as they undoubtedly are, they sit stolidly on bookshelves, unread.

Wow. Talk about your pretty aggressive accusation without any supporting evidence or citation whatsoever.

Intel's not big into the open-source space, so it doesn't take much for an open-source project from them to be their "largest open-source effort ever". (What, they're going to open-source the schematics for the Intel chipline? Who could read them even if they did? Who would offer up a patch? What good would it do?) The fact that Intel made the software available in the first place meant that they knew the hurdle that had yet to be overcome, and wanted to aid developers in overcoming it. They're members of the OpenMP group for the same reason.

Rogue Wave's software pipelines programming model is another case where real benefits have accrued, backed by case studies. (Disclaimer: I know this because I ghost-wrote an article for them on their Software Pipelines implementation.) Let's not knock something that's actually delivered value. Pipelines aren't going to be the solution to every problem, granted, but they're a useful way of structuring a design, one that's curiously similar to what I see in functional programming languages.

But simply defending Intel's generosity or the validity of an alternative programming model doesn't support the idea that concurrency is still a hot topic. No, for that, I need real evidence, something with actual concrete numbers and verifiable fact to it.

Thus, I point to Brian Goetz’s Java Concurrency in Practice, one of those “books, rushed out while the temperature soared”, which also turned out to be the best-selling book at Java One 2007, and the second-best-selling book (behind only Joshua Bloch’s unbelievably good Effective Java (2nd Ed) ) at Java One 2008. Clearly, yes, bestselling concurrency books are just a myth, alongside the magical device that will receive messages from all over the world and play them into your brain (by way of your ears) on demand, or the magical silver bird that can wing its way through the air with no visible means of support as it does so. Myths, clearly, all of them.

To continue...

The truth is that it's simply not a big issue for the majority of people. Writing truly "concurrent" applications in languages such as C# is difficult, as you get very little help from the language. It means getting involved with low-level concurrency primitives, such as lock statements and so on.

Many programmers lack the skills to do this, but more pertinently lack the need. Increasingly, programmers work in a web environment. As long as these web applications are deployed to a load-balanced web farm, then page requests can be handled in parallel so all available cores will be used efficiently without the need for the programmer to be concerned with fine-grained parallelism.

He’s right when he says you get very little help from the language, be it C# or Java or C++. And getting involved with low-level concurrency primitives is clearly not in anybody’s best interests, particularly if you’re not a concurrency guru like Brian. (And let’s be honest, even low-level concurrency gurus like Brian, or Joe Duffy, who wrote Concurrent Programming on Windows, or Mike Woodring, who co-authored Win32 Multithreaded Programming, have better things to do.) But to say that they “pertinently lack the need” is a rather impertinent statement. “As long as these web applications are deployed to a load-balanced web farm", which is very likely to continue to happen, “then page requests can be handled in parallel so all available cores will be used …”

Um... excuse me?

Didn’t you just say that programmers didn’t need to learn concurrency constructs? It would strike me that if their page requests are being handled in parallel that they have to learn how to write code that won’t break when it’s accessed in parallel or lead to data-corruption problems or race conditions when their pages are accessed in parallel. If parallelism is a fundamental part of the Web, don’t you think it’s important for them to learn how to write programs that can behave correctly in parallel?

Look for just a moment at the average web application: if data is stored in a per-user collection, and two simultaneous requests come in from a given user (perhaps because the page has AJAX requests being generated by the user on the page, or perhaps because there’s a frameset that’s generating requests for each sub-frame, or ...), what happens if the code is written to read a value from the session, increment it, and store it back? ASP.NET can save you here, a little, in that it used to establish a per-user lock on the entirety of the page request (I don’t know if it still does this—I really have lost any desire to build web apps ever again), but that essentially puts an artificial throttle on the scalability of your system, and makes the end-users’ experience that much slower. Load-balancer going to spray the request all over the farm? So long as the user session state is stored on every machine in the farm, that’ll work... But of course if you store the user’s state in the SQL instance behind each of those machines on the farm, then you take the performance hit of an extra network round-trip (at which point we’re back to concurrency in the database) ...

... all because the programmer couldn’t figure out how to make “lock” work? This is progress?

The Java Servlet specification specifically backed away from this "lock on every request" approach because of the performance implications. I heard a fair amount of wailing and gnashing during the early ASP.NET days over this. I heard the ASP.NET dev team say they made their decision because the average developer can't figure out concurrency correctly anyway.

And, by the way folks, this editorial completely ignores XML services. I guess "real" applications don't write services much, either.

The next part is even better:

Furthermore, the SQL Server engine behind these web applications is intrinsically "parallel", and can handle and use effectively about as many cores as you care to throw at it. SQL itself is a declarative rather than procedural language, so it is fundamentally concurrent.

True… and false. SQL is fundamentally “parallel” (largely because SQL is a non-strict functional language, not just a “declarative” one), but T-SQL isn’t. And how many developers actually know where the line is drawn between SQL and T-SQL? More importantly, though, how many effective applications can be written with a complete ignorance of the underlying locking model? Why do DBAs spend hours tuning the database’s physical constructs, establishing where isolation levels can be turned down, establishing where the scope of a transaction is too large, putting in indexed columns where necessary, and figuring out where page, row, or table locking will be most efficient? Because despite the view that a relational database presents, these queries are being executed in parallel, and if a developer wants to avoid writing an application that requires a new server for each and every new user added to the system, they need to learn how to maximize their use of the database’s parallelism. So even if the language is "fundamentally concurrent" and can thus be relied upon to do the right thing on behalf of the developer, the implementation isn't, and needs to be understood in order to be implemented efficiently.

He finishes:

For most programmers and for most web applications, however, the multi-core furore is a storm in a teacup; it's just not relevant. The web and database platforms already cope with concurrency requirements. We are already doing it.

This is one of those times I wish I had a time machine handy—I'd love to step forward five years, have a look around, then come back and report the findings. I'm tempted to close with the challenge to just let’s come back in five years and see what the programming language landscape and hardware landscape looks like. But that's too easy an "out", and frankly, doesn't do much to really instill confidence, in my opinion.

To ignore the developers building "rich" applications (be they being done in Flex/Flash, Cocoa/iPhone, WinForms, Swing, WPF, or what-have-you) is to also ignore a relatively large segment of the market. Not every application is being built on the web and is backed by a relational database—to simply brush those off and not even consider them as part of the editorial reveals a dangerous bias on the editor's part. And those applications aren't hosted in an "intrinsically 'parallel'" container that developers can just bury their head inside.

Like it or not, folks, the path forward isn't one that you get to choose. Intel, AMD, and other chip manufacturers have already made that clear. They're not going to abandon the multicore approach now, not when doing so would mean trying to wrestle with so many problems (including trying to change the speed of light) that simply aren't there when using a multicore foundation. That isn't up for debate anymore. Multicore has won for the forseeable future. And, as a result, multicore is going to be a fact of the developer's life for the forseeable future. Concurrency is thus also a fact of the developer's life for the forseeable future.

The web and database platforms “cope” with concurrency requirements by either making "one-size-fits-all" decisions that almost always end up being the wrong decision for high-scale systems (but I'm sure your new startup-based idea, like a system that allows people to push "micro-entries" of no more than 140 characters in length to a publicly-trackable feed would never actually take off and start carrying millions and millions of messages every day, right?), or by punting entirely and forcing developers to dig deeper beneath the covers to see the concurrency there. So if you're happy with your applications running no faster than 2GHz for the rest of the forseeable future, then sure, you don't need to worry about learning concurrency-friendly kinds of programming techniques. Bear in mind, by the way, that this essentially locks you in to small-scale, web-plus-database systems for the forseeable future, and clearly nothing with any sort of CPU intensiveness to it whatsoever. Be happy in your niche, and wave to the other COBOL programmers who made the same decision.

This is a leaky abstraction, full stop, end of story. Anyone who tells you otherwise is either trolling for hits, trying to sell you something, or striving to persuade developers that ignorance isn't such a bad place to be.

All you ignorant developers, this is the phrase you will be forced to learn before you start your next job: "Would you like fries with that?"

While playing around with a recent research-oriented project for myself (more on that later), I discovered something that I haven't seen mentioned anywhere in the Scala universe before. (OK, not really--as you'll see towards the end of this piece, it really is documented, but allow me my brief delusions of grandeur as I write this. They'll get deflated quickly enough.)

So the core of the thing was a stack-oriented execution engine; essentially I'm processing commands delivered in a postfix manner. Since some of these commands are relational operators, it's important that there be two things to relationally operate on the execution stack, after which I want to evaluate the relational operation and push its result (1 if true, 0 if false) back on the stack; this is pretty easily done via the following:

def compareOp(op : (Int, Int) => Boolean) =
{
  checkStack(2)
  val v1 = (execStack.pop()).asInstanceOf[Int]
  val v2 = (execStack.pop()).asInstanceOf[Int]
  val vr = op(v1, v2)
  execStack.push(if (vr) 1 else 0)
} 

where "execStack" is a mutable.Stack[Any] held in an enclosing function.

Interestingly enough, however, when I wrote this the first time, I wrote it like this, which is a very different sequence of operations:

def compareOp(op : (Int, Int) => Boolean) =
{
    checkStack(2)
    def v1 = (execStack.pop()).asInstanceOf[Int]
    def v2 = (execStack.pop()).asInstanceOf[Int]
    def vr = op(v1, v2)
    execStack.push(if (vr) 1 else 0)
}

See the difference? Subtle, is it not? But the actual code is significantly different, something that's more easily seen with a much simpler (and standalone) example:

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        def v1 = (stack.pop()).asInstanceOf[Int]
        def v2 = (stack.pop()).asInstanceOf[Int]
        def vr = v1 + v2
        System.out.println(vr)
    }
} 

When run, the console prints out "36", as we'd well expect.

But suppose we want to look at those values of v1 and v2 along the way, perhaps as part of a logging operation, or perhaps because you're just screwing around with some ideas in your head and you don't want to bother to fire up an IDE with Scala support in it. So you decide to spit those values to a console:

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        def v1 = (stack.pop()).asInstanceOf[Int]
        def v2 = (stack.pop()).asInstanceOf[Int]
        System.out.println(v1)
        System.out.println(v2)
        def vr = v1 + v2
        System.out.println(vr)
    }
} 

And then something *very* different happens; you get "24", "12", and then a NoSuchElementException.

If you're like me the first time I ran into this, your first reaction is, "Eh?". Actually, if you're like me, when you're programming, your profanity filters are probaby at an ebb, so your first reaction is "WTF?!?", said with great gusto and emphasis. Which has a tendency to get some strange looks when you're at a Denny's doing your research, I will admit. Particularly when it's at 3 AM in the morning. And the bar crowd is in full alcoholic haze and slightly nervous about the long-haired, goatee-sporting guy in his headphones, wearing his black leather jacket and swearing like a drunken sailor at his laptop. But I digress.

What is Scala doing here?

Turns out this is exactly as the language designers intended, but it's subtle. (Or maybe it's just subtle to me at 3AM when I'm pumped full of caffeine.)

Let's take this a different way:

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        def v1 = (stack.pop()).asInstanceOf[Int]
        def v2 = (stack.pop()).asInstanceOf[Int]
        System.out.println(stack)
    }
} 

When run, the console prints "Stack(12, 24)", which *really* starts to play with your mind when you're a little short on sleep and a little high on Diet Coke. At first glance, it looks like Scala is broken somehow--after all, those "pop" operations are supposed to modify the Stack against which they're operating, just as the push()es do. So why is the stack convinced that it still holds the values of 12 and 24?

Because Scala hasn't actually executed those pop()s yet.

The "def" keyword, it turns out, isn't what I wanted here--what I wanted (and in retrospect it’s painfully obvious) was a "val", instead, in order to force the execution of those statements and capture the value into a local value (an immutable local variable). The "def" keyword, instead, creates a function binding that waits for formal execution before evaluating. So that when I previously said

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        def v1 = (stack.pop()).asInstanceOf[Int]
        def v2 = (stack.pop()).asInstanceOf[Int]
        def vr = v1 + v2
        System.out.println(vr)
    }
} 

… what in fact I was saying was this:

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        def v1 = (stack.pop()).asInstanceOf[Int]
        def v2 = (stack.pop()).asInstanceOf[Int]
        System.out.println(v1 + v2)
    }
} 

… which is the same as:

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        System.out.println((stack.pop()).asInstanceOf[Int] + (stack.pop()).asInstanceOf[Int])
    }
} 

… which, when we look back at my most recent "debugging" version of the code, substituting the "def"ed versions of v1 and v2 (and vr) where they're used, makes the reason for the NoSuchElementException become entirely more clear:

object App
{
    def main(args : Array[String]) =
    {
        import scala.collection.mutable.Stack
        var stack : Stack[Any] = new Stack()
        stack.push(12)
        stack.push(24)
        System.out.println((stack.pop()).asInstanceOf[Int])
        System.out.println((stack.pop()).asInstanceOf[Int])
        System.out.println((stack.pop()).asInstanceOf[Int] + (stack.pop()).asInstanceOf[Int])
    }
} 

Now, normally, this would probably set off all kinds of alarm bells in your head, but the reaction that went off in mine was "COOL!", the reasons for which revolve around the concept of "laziness"; in a functional language, we frequently don't want to evaluate the results right away, instead preferring to defer their execution until actually requiring it. In fact, many functional languages—such as Haskell—take laziness to new heights, baking it directly into the language definition and assuming laziness everywhere, so much so that you have to take special steps to avoid it. There’s a variety of reasons why this is advantageous, but I’ll leave those discussions to the Haskellians of the world, like Matt Podwysocki and Simon Peyton-Jones.

From a Scalist’s perspective, laziness is still a useful tool to have in your toolbox. Suppose you have a really powerful function that calculates PI to a ridiculous number of decimal places. In Java, you might be tempted to do something like this:

class MyMath
{
  public static final double PI = calculatePiToARidiculousNumberOfPlaces();
  private static double calculatePiToARidiculousNumberOfPlaces()
  {
    // implementation left to the reader's imagination
    // imagine it being "really cool"
  }
} 

The problem with this is that if that method takes any length of time to execute, it's being done during class initialization during its ClassLoading phase, and aside from introducing a window of time where the class *could* be used before that initialization is finished (it's subtle, it's not going to happen very often, but it can, according to older versions of the JVM Spec), the problem is that the time required to do that initialization is paid for *regardless of whether you use PI*. In other words, the classic Stroustrup-ian "Don't pay for it if you don't use it" principle is being completely tossed aside.

In Scala, using the "def" keyword here, aside from avoiding the need for the additional decorators, completely eliminates this cost--people won't need the value of PI until it becomes used:

object App
{
    def PI = calculatePiToARidiculousNumberOfPlaces()
    def calculatePiToARidiculousNumberOfPlaces() =
    {
        System.out.println("Calculating PI")
        3 + 0.14
    }
    def main(args : Array[String]) =
    {
        System.out.println("Entering main")
        System.out.println("PI = " + PI)
    }
} 

(In fact, you'd probably just write it without the calculating method definition, since it's easier that way, but bear with me.)

When you run this, of course, we see PI being calculated after main()'s been entered, thus proving that PI is being calculated only on demand, not ahead of time, as a public-static-final-constant would be.

The problem with this approach is, you end up calculating PI on each access:

object App
{
    def PI = calculatePiToARidiculousNumberOfPlaces()
    def calculatePiToARidiculousNumberOfPlaces() =
    {
        System.out.println("Calculating PI")
        3 + 0.14
    }
    def main(args : Array[String]) =
    {
        System.out.println("Entering main")
        System.out.println("PI = " + PI)
        System.out.println("PI = " + PI)
            // prints twice! Not good!
    }
} 

Which sort of defeats the advantage of lazy evaluation.

This got me wondering--in F#, we have lazy as a baked-in concept (sort of), such that when I write

#light
let sixty = lazy (30 + 30)
System.Console.WriteLine(sixty) 

What I see on the console is not 60, but a Lazy<T> type instance, which effectively defers execution until it's Force() method is invoked (among other scenarios). This means I can write things like

let reallyBigList = lazy ([1..1000000000000] |> complexCalculation |> anotherComplexCalcuation)

without fear of blowing the stack or heap apart, since laziness means the list won't actually be calculated until it's forced; we can see this from the following (from the F# interactive console):

> let sixtyWithSideEffect = lazy (printfn "Hello world"; 30+30);;
val sixtyWithSideEffect: Lazy<int>
> sixtyWithSideEffect.Force();;
Hello world
val it : int = 60
> sixtyWithSideEffect.Force();;
val it : int = 60 

(Examples taken from the excellent Expert F# by Syme/Granicz/Cisternino; highly recommended, if a touch out-of-date to the current language definition. I expect Chris Smith’s Programming F#, from O’Reilly, to correct that before too long.)

It would be nice if something similar were doable in Scala. Of course, once I start looking for it, it makes itself visible, in the wonderful Venners/Odersky/Spoon book, Programming In Scala, p. 444:

You can use pre-initialized fields to simulate precisely the initialization behavior
of class constructor arguments. Sometimes, however, you might prefer
to let the system itself sort out how things should be initialized. This can
be achieved by making your val definitions lazy. If you prefix a val definition
with a lazy modifier, the initializing expression on the right-hand side
will only be evaluated the first time the val is used.

[...]

This is similar to the situation where x is defined as a parameterless
method, using a def. However, unlike a def a lazy val is never evaluated
more than once. In fact, after the first evaluation of a lazy val the result of the
evaluation is stored, to be reused when the same val is used subsequently.

Perfect! The key, then, is to define PI like so:

object App
{
    lazy val PI = calculatePiToARidiculousNumberOfPlaces()
    def calculatePiToARidiculousNumberOfPlaces() =
    {
        System.out.println("Calculating PI")
        3 + 0.14
    }
    def main(args : Array[String]) =
    {
        System.out.println("Entering main")
        System.out.println("PI = " + PI)
        System.out.println("PI = " + PI)
            // prints once! Awesome!
    }
} 

That means, if I apply it to my Stack example from before, I should get the same deferred-execution properties of the "def"-based version ...

def main(args : Array[String]) =
{
    import scala.collection.mutable.Stack
    var stack : Stack[Any] = new Stack()
    stack.push(12)
    stack.push(24)
    lazy val v1 = (stack.pop()).asInstanceOf[Int]
    lazy val v2 = (stack.pop()).asInstanceOf[Int]
    System.out.println(stack)
        // prints out "Stack(12,24)
} 

... but if I go back to the version that blows up because the stack is empty, using lazy val works exactly the way I would want it to:

def main(args : Array[String]) =
{
    import scala.collection.mutable.Stack
    var stack : Stack[Any] = new Stack()
    stack.push(12)
    stack.push(24)
    lazy val v1 = (stack.pop()).asInstanceOf[Int]
    lazy val v2 = (stack.pop()).asInstanceOf[Int]
    System.out.println(v1)
    System.out.println(v2)
    lazy val vr = v1 + v2
    System.out.println(vr)
        // prints 12, 24, then 36
        // and no exception!
} 

Nice.

So, it turns out that my accidental use of "def" inside the compareOp function behaves exactly the way the language designers wanted it to, which is not surprising, and that Scala provides nifty abilities to defer processing or extraction of values until called for.

Curiously, the two languages differ in how laziness is implemented; in F#, the lazy modifier defines the type to be a Lazy<T> instance, an ordinary type that we can pass around from F# to C# and back again as necessary (in much the same way that C# defined nullable types to be instances of Nullable<T> under the hood). We can see that from the interactive console output above, and from the fact that we call Force() on the instance to evaluate its value.

In Scala, however, there is no corresponding Lazy[T] type; instead, the PI() method is defined to determine whether or not the value has already been evaluated:

public double PI();
  Code:
   0:   aload_0
   1:   getfield        #135; //Field bitmap$0:I
   4:   iconst_1
   5:   iand
   6:   iconst_0
   7:   if_icmpne       48
   10:  aload_0
   11:  dup
   12:  astore_1
   13:  monitorenter
   14:  aload_0
   15:  getfield        #135; //Field bitmap$0:I
   18:  iconst_1
   19:  iand
   20:  iconst_0
   21:  if_icmpne       42
   24:  aload_0
   25:  aload_0
   26:  invokevirtual   #137; //Method calculatePiToARidiculousNumberOfPlaces:()D
   29:  putfield        #139; //Field PI:D
   32:  aload_0
   33:  aload_0
   34:  getfield        #135; //Field bitmap$0:I
   37:  iconst_1
   38:  ior
   39:  putfield        #135; //Field bitmap$0:I
   42:  getstatic       #145; //Field scala/runtime/BoxedUnit.UNIT:Lscala/runtime/BoxedUnit;
   45:  pop
   46:  aload_1
   47:  monitorexit
   48:  aload_0
   49:  getfield        #139; //Field PI:D
   52:  dreturn
   53:  aload_1
   54:  monitorexit
   55:  athrow
  Exception table:
   from   to  target type
    14    48    53   any

If you look carefully at the bytecode, the implementation of PI is checking a bitmask field (!) to determine if the first bit is flipped (!) to know whether or not the value is held in the local field PI, and if not, calculate it and store it there. This means that Java developers will just need to call PI() over and over again, rather than have to know that the instance is actually a Lazy[T] on which they need to call Value or Force (such as one would from C# in the F# case). Frankly, I don’t know at this point which approach I prefer, but I’m slightly leaning towards the Scala version for now. (If only Java supported properties, then the syntax “MyMath.PI” would look like a constant, act lazily, and everything would be great.)

(It strikes me that the F# developer looking to write something C#-accessible need only tuck the Lazy<T> instance behind a property accessor and the problem goes away, by the way; it would just be nicer to not have to do anything special on either side, to have my laziness and Force() it, too. Pipe dream, perhaps.)

In retrospect, I could wish that Scala weren't *quite* so subtle in its treatment of "def" vs "val", but now that I'm aware of it, it'll (hopefully) not bite me quite so subtly in the sensitive spots of my anatomy again.

And any experience in which you learn something is a good one, right?

An interesting blog post was forwarded to me by another of my fellow ThoughtWorkers, which suggests a new software stack for building an enterprise system, acronymized as “JOSH”:

The Book Of JOSH

Through a marvelous, even devious, set of circumstances, I'm presented with the opportunity to address my little problem without proscribed constraints, a true green field opportunity.

Json OSGi Scala HTTP

Json delivers on what XML promised. Simple to understand, effective data markup accessible and usable by human and computer alike. Serialization/Deserialization is on par with or faster then XML, Thrift and Protocol Buffers. Sure I'm losing XSD Schema type checking, SOAP and WS-* standardization. I'm taking that trade.

OSGi a standardized dynamic, modular framework for versioned components and services. Pick a logger component, a HTTP server component, a ??? component, add your own internal components and you have a dedicated application solution. Micro deployment with true replacement. What am I giving up? The monolithic J2EE application servlet loaded with 25 frameworks, SCA and XML configuration hell. Taking the trade.

HTTP is simple, effective, fast enough, and widely supported. I'm tired of needlessly complex and endless proprietary protocols to move simple data from A to B with all the accompanying firewall port insanity. Yes, HTTP is not perfect. But I'm taking this trade where I can as well.

All interfaces will be simple REST inspired APIs based on HTTP+JSON. This is an immediate consequence of the JOSH stack.

Scala is by far the toughest, yet the easiest selection in the JOSH stack. I wrestled far more with the JSON or XML or Thrift or Protocol Buffers decision.

And, let’s be honest, the stack sounds a lot better than what he was working with before....

[...] Yes, you see, I have a small problem.

So whats the issue, you say? I write a whole blog about nothing, you say? We all know the right answer, you're pointing out? Yea, I know, its intuitively obvious to the casual observer.

We'll rewrite it from scratch.

Course we'll need a cluster of WebSphere Application Servers, and an Oracle RAC cluster for all that data. Don't forget the middleware needed to transition over from the legacy systems, so toss in an ESB cluster, and what heck a couple of BPEL servers too.

Need a SOA Center of Excellence of course too. Can't integrate without some common XML Business Object Schemas. Also need to roll the Rational RUP suite and some beefy IDE environments and for that shiny look, sprinkle the works with lots of WS-* sparkly dust. Bake 3-5 years or until done, whenever.

My presentation slides for all this will be killer. I can sell this stuff. I'm good at it. I'll look like a bloody genius. I'll have Vendors fawning all over me. And the best part is the bubble on this mess won't pop for YEARS, when I'll have plenty of plausible deniability. "Hey the plan was perfect, the business, IT managers and their people were incapable of executing it."

I feel like the enterprise IT equivalent of an AIG trader pocketing ill gotten gains from writing Credit Default Swaps that we can't pay off.

Ewww... even thinking about all that makes me want to go upstairs, step into the shower, turn the water as hot as it will go, and wash. Scrub my skin raw with soap and sponge until the top five layers of epidermis are gone, and still not feel clean.

On the surface of things, the stack sounds pretty good. OSGi is a pretty solid spec for managing versioning and modularity within a running Java system, and more importantly, it’s well-known, relatively reliable, and pretty well-proven to handle the classic problems well. And of course, anybody who knows me knows that I’m a fan of the Scala language as a potential complement or supplement to the Java programming language, so that’s hardly in debate.

But there are a few concerns. JSON is a simple wire protocol, granted, but that is both a good thing and a bad thing (it’s object-centric, for one, and will run into some of the same issues as objects do with certain relationships), and it lacks the ubiquity that XML provides. Granted, XML clearly suffered from an overabundance of adoption, but it still doesn’t take away the fact that ubiquity is really necessary if you’re building a baseline for something that will talk to a variety of different systems. Which, I admit, may not be in his list of requirements, I don’t know. And HTTP is great for long-haul, client-initiated communication, but it definitely has its limitations (which he acknowledges, openly, to his credit), at least to internal-facing consumers. There is no peer for external-facing consumers, that’s a given.

And the stack is clearly also missing something else...

The JOSH stack is lacking a letter, because a solution for persisted data is missing in the stack.

A great deal of what needs to be done does not require a ACID RDB cluster. Some of it does and I'm kicking that can down the road.

For the rest, either the data is ReadOnly and loaded a 1-3 times a day or is best persisted by a distributed Key-Value storage system. A number of these are now available as open source solutions and at the right moment I'll need to pick one and add that letter to the JOSH stack.

As a commenter suggested, CouchDB might be a solution here, or I’ll even throw db4o into the ring for discussion as an option. Again, it’ll depend on how far-and-wide the data will be seen by other systems—the more other systems need to see it, the less further away from a “regular” RDBMS we can go.

Certainly, it’s a great start for discussion, even if the acronym is likely to give those named Joshua an unhealthy ego boost.

Part of me wonders, though... what would the equivalent on .NET look like? JSON + Assemblies + F# + HTTP = JAFH?

This crossed my Inbox:

I read your article entitled: The Polyglot Programmer. How about the thought that rather than becoming a polyglot-software engineer; pick a polyglot-language. For example, C# is borrowing techniques from functional and dynamic languages. Let the compiler designer worry about mixing features and software engineers worry about keep up with the mixture. Is this a good approach? [From Phil, at http://greensoftwareengineer.spaces.live.com/]

Phil, it’s an interesting thought you’ve raised—which is the better/easier approach to take, that of incorporating the language features we want into a single language, rather than needing to learn all those different languages (and their own unique syntaxes) in order to take advantage of those features we want?

After all, we’re starting to see this taking place within a certain number of languages already, particularly C#; first, in 3.0, they introduced a number of features in support of LINQ that make C# a useful starting point for working with a functional language. Extension methods, for example, allow us to add a number of different methods to the collection classes that provide some functional capabilities (Select<>, GroupBy<>, and so on), as Matt Podwysocki demonstrates, generics contribute the type-safety that most functional languages embrace, anonymous methods and delegates provide better functions-as-first-class-constructs (including lambdas), and anonymous types make it vastly easier to return and pass tuples. And now, in 4.0, we’re getting the “dynamic” keyword, which will add support for invoking methods and properties dynamically, in the grand tradition of most dynamic languages (like Python and Ruby), and 3.0’s local variable type inference allows us to write “var x = ...”, which feels pretty dynamic (even if it’s not, under the hood).

Unfortunately, I think for the most part, the answer’s going to be, “Yes, it would be nice, if it weren’t for the fact that there are very few languages that won’t collapse underneath their own weight if they did so.”

Consider, for example, the C# language. Already, with the C# 3.0 definition, the language specification weighs in at close to a thousand pages. The additional features in 4.0 could easily push it over a thousand and possibly, with all the places where “dynamic” behavior will need to be factored into the existing specification, could push that well into the 1200 to 1300 page range. What’s the upper limit on a language’s complexity to maintain and enhance, much less for its programmers to comprehend?

(By comparison, the C++ specification, as I can best remember, didn’t weigh in at more than a thousand pages, but given that the current working draft is under password protection, and I can’t find the prior spec as a freely-available download, I can’t see if memory is correct or not.)

Or, consider the various edge cases that came up around the introduction of nullable types in C# 2.0. What started out as a fairly simple suggestion—“let’s let T? represent the idea that this instance of T could be nullable, and at runtime it’ll be a Nullable<T> instance behind the scenes”—turned into a pretty ugly morass of edge cases at the language level that resulted in some serious bug-fixing right up until the final ship date.

Thing is, languages that aren’t written deliberately to allow their own modification and evolution tend to fail over time. C++ was one such example, and I think both Java and C# will stand as successor examples before long.

Right now, in C# 3.0, type inference is limited entirely to local variables because the language isn’t syntactically set up to leave out type names wherever possible—the “var” token is a type placeholder, largely because the parser has to have a type first. (This is the same purpose the “dynamic” keyword seems to be playing for 4.0, though I can’t say so for certain.) In F# and Scala, this syntax is deliberately written Pascal-style, with the name first, optionally followed by a colon and the type, because the parser can see the colon and realize the type is already specified, or see no colon and realize the type should be inferred. That syntax is used consistently throughout the F# and Scala languages, and that means it’s pretty easy, lexically speaking, for the languages to recognize when type inference should kick in.

What’s more, both F# and Scala don’t really support the O-O notion of method overloading, because again, it gets confusing when trying to kick in type inference—something about too many possibilities confusing the type-inferencer. (I’m not entirely positive of this point, by the way, it’s based on some conversations I’ve had with language designers over the last few years. I could be wrong, and would love to see a language that supports both.) Instead, they force developers to be more explicit about parameters being passed—F# won’t even do implicit widening conversions, in fact, such as automatically widening ints to longs.

But both F# and Scala have a very interesting facility to allow definitions of methods/functions using very flexible syntactic rules, such that they look like operators or keywords built into the language; F# defines its pipeline operator ( |> ) in its library definitions, for example. Scala defines numerous “keywords”, like synchronized or transient, as classes in the Scala package extending “StaticAnnotation”—in other words, their syntax and behavior is defined as an annotation, rather than as a built-in part of the language. Ditto for Scala’s XML support.

Lisp, of course, was one of the first (if not the first) language to do this, and it’s my understanding that this has been one of the principal reasons it has survived all these years as a language—because it’s an abstraction built on top of an abstraction built on top of an abstraction, et al, it makes it easier to change those underlying abstractions when the context changes.

This doesn’t mean those “polyactivist” languages like C# are bad things, it just means that there’s a danger that they’ll eventually collapse from too many moving parts all trying to talk to each other at the same time. As an exercise, open the C# 3.0 spec, and start checking off all the sections that will need to be touched by the introduction of the “dynamic” keyword as a new type.

Or, to put it analagously, yes, for a lot of work, a single multifunction tool can be useful, but for a lot of other work, you want tools that are specialized to the task at hand. Let’s not minimize the usefulness of that multifunction tool, but let’s not try to use a Swiss Army knife where a jeweler’s screwdriver is really needed.

This email crossed my Inbox last week while I was on the road:

Due to the current economic situation, TechWeb has made the difficult decision to discontinue the Software Development events, including SD West, SD Best Practices and Architecture & Design World. We are grateful for your support during SD's twenty-four year history and are disappointed to see the events end.

This really bums me out, because the SD shows were some of the best shows I’ve been to, particularly SD West, which always had a great cross-cutting collection of experts from all across the industry’s big technical areas: C++, Java, .NET, security, agile, and more. It was also where I got to meet and interview Bjarne Stroustrup, a personal hero of mine from back in my days as a C++ developer, where I got to hang out each year with Scott Meyers, another personal hero (and now a good friend) as well as editor on Effective Enterprise Java, and Mike Cohn, another good friend as well as a great guy to work for. It was where I first met Gary McGraw, in a rather embarrassing fashion—in the middle of his presentation on security, my cell phone went off with a klaxon alarm ring tone loud enough to be heard throughout the entire room, and as every head turned to look at me, he commented dryly, “That’s the buffer overrun alarm—somewhere in the world, a buffer overrun attack is taking place.”

On a positive note, however, the email goes on to say that “Cloud Connect [will] take over SD West's dates in March 2010 at the Santa Clara Convention Center”, which is good news, since it means (hopefully) that I’ll still get a chance to make my yearly pilgrimage to In-N-Out....

Rest in peace, SD. You will be missed.

Just got this email from Chris Sells:

For twelve 45-minute slots at this year’s DSL DevCon (April 16-17 in Redmond, WA), we had 49 proposals. You have been selected as speakers for the following talks. Please confirm that you’ll be there for both days so that I can put together the schedule and post it on the conference site. This DevCon should rock. Thanks!

Martin Fowler - Keynote

Paul Vick + Gio - Mgrammar Deep Dive

Tom Rodgers - Domain Specific Languages for automated testing of equity order management systems and trading machines

Paul Cowan - DSLs in the Horn Package Manager

Guillaume Laforge - How to implement DSLs with Groovy

Markus Voelter - Eclipse tooling for Model-Driven stuff

Dionysios G. Synodinos - JavaScript DSLs for the Client Side

Ted Neward, Bradford Cross - Functional vs. Dynamic DSLs: The Smackdown

Gilad Bracha - embedding EBNF in a general purpose language

Umit Yalcinalp, Tilman Giese - RUMBA: RUby Managed Business data for Applications

Bob Archer - A DSL for Cool Effects in Adobe Pixel Blender

Chance Coble - Language Oriented Programming in F#

As my 15-year-old son Michael has grown fond of saying... w00t! The list of topics is fascinating, and I'm really looking forward to most, if not all, of them. Chance's talk on LOP in F# should be good, I'm really curious to see Gilad's discussion of EBNF (and wondering if this is Newspeak we'll be seeing), and Guillaume is always fun to watch when he's going on about Groovy. Of course, I'm also excited to be paired up with Brad, who's an insanely smart guy--I have a feeling I'll learn a lot just by standing next to him. (Sort of a speakers' osmosis.)

If you're not planning to be here for this (and the Lang.NET Symposium), either you have life-saving surgery scheduled that can't be pushed back, or you're clearly not interested in DSLs. For your own sake, I hope it's the latter.

Seriously, come for the full week. The Lang.NET Symposium last year was an amazing event, for a number of reasons, not the least of which is that it saw Sun celebrities John Rose, Charlie Nutter and Brian Goetz step on to the Microsoft campus, deliver a great presentation on the JVM, MLVM/invokedynamic, and JRuby, and get good feedback and discussion from Microsoft engineers and other notables. You don't get to see that every day.

Answer: "I don't know, but I'm damn well going to find out!"

(Now I really wish I were in London. Ah, well, will just have to go see Ward Cunningham speak at Alt.NET Seattle, instead.)

Matt Podwysocki makes it all clear:

Hey, I'd have learned Haskell a LONG time ago if I'd known it could yield up a beer!

Last year I had the opportunity to return to the land of my roots, Poland, and speak at Java Developer Days (JDD). Just today, the organizers from JDD sent me a link with a nice little photo montage from the conference. (I did notice a few photos from the after-party were selectively left out of the montage, however, which is probably a good thing because that was the first time I'd ever met a Polish Mad Dog, and boy did they all go down easy...)

If you're anywhere in the area around Krakow in March, you definitely should swing by for their follow-up conference, 4Developers--it sounds like it's going to be another fun event, and this time it's going to reach out to more than just the Java folks, but also the .NET crowd (and a few others), as well.

(I don't really expect any of the readers of this blog living outside Poland to really pack up and head over to Krakow for a weekend, mind you, but if you're a technology speaker and you're interested in hanging with an extremely good group of people, the people who put these shows on--ProIdea--are top-notch, take great care of the speakers, and overall make the entire experience well worth the trip.)

Because I’m about to start my third part in the WCF/F# series, I realized that I’ve now hit the “rule of three” mark: in this particular case, this will mark the third project I’m creating that unifies WCF and F#, and frankly, it’s a pain in the *ss to do it all by hand each time: create an F# Library, add the System.ServiceModel and System.Runtime.Serialization assemblies, go create an App.config file and add it to the project as an Existing Item…. Painful.

So… as a brief interlude, I decided to go re-acquaint myself with the Visual Studio project template system, and sure enough, it’s basically what I remember: a collection of files with some template-style functionality, bundled into a .zip file and stored in the Visual Studio directory, under <VSDir>\Common7\IDE\ProjectTemplates. What was new to me, however, was the highly useful “File | Export Template…” menu option, allowing me to take an existing F#/WCF project and use it as a template to create the .zip bundle. (Naturally, I didn’t discover this until I’d built the silly thing by hand.)

Sara Ford has more on creating a VS template on her Visual Studio Tools blog/column, number 336 to be precise. (You should read all of them, by the way—start with #1 and work your way there. When you’re done, you’ll have a much better appreciation of everything Visual Studio can do, and you’ll be able to find a ton of ways to save yourself and your team some time and effort.)

You can always take a .zip bundle like this and drop it into the Visual Studio 2008 “My Exported Templates” directory, but quite frankly, I didn’t want that. I wanted my template to appear in a subcategory of Visual F# in the New Project dialog box, under “WCF”, just as the C# versions do. The easiest way to do this is to manually create the “WCF” directory (full path thus being <VSDir>\Common7\IDE\ProjectTemplates\FSharp\WCF), and drop the .zip file there. Note that if you restart Visual Studio at this point, you won’t see the new template; it builds a cache of the .zip templates in a sister directory (ProjectTemplatesCache), so instead, you have to tell Visual Studio to reset that cache by firing “devenv /setup” from the command-line. (This will require admin privileges, by the way.)

After that, you have an F#/WCF project template, and you’re good to go.

If you’ve not read the first part in the series, take a look there first.

While it’s always easier to build WCF services with nothing but primitive types understood by all the platforms to which you’re communicating (be it Java through XML services or other .NET systems via WCF’s more efficient binding types), this gets old and limiting very quickly. The WCF service author will want to develop whole composite types that can be exchanged across the wire, and this is most often done via the DataContract attribute applied to the types that will be exchanged.

In Michele Leroux Bustamente’s Learning WCF examples, this is covered in Chapter #2, and the corresponding code I’m using as a basis for conversion to F# is in Labs\Chapter2\DataContracts_Part1.

One notable difference between this example and the previous one is that the type definitions are stored in a separate assembly, ContentTypes.dll. There’s two basic choices to use here: one, to use the C# types as defined, from a service written in F#, or two, to define the types in F# and use them from the service. A third choice, defining the types in F# and using them from C#, also presents itself, but is uninteresting to us from a purely instructional standpoint—if you know how to write C#, then you can take the types defined in F# and use them just as you would have the C# types.

For instructional purposes, I’m going to take the second approach: I’m first going to convert the ContentTypes.dll assembly over to F#, again to show how to create types in F# that are structurally equivalent to the types defined in C#, since that’s something that has changed since Nick Holmes blogged about this last year), then I’m going to flip the service over to F# as well.

Defining the Data Types

The first step, for many service authors, is to define the interfaces for the service(s) and the types that will be exchanged; in this case, since I’m building from Michele’s example, these have already been defined as:

   1: using System;

   2: using System.ServiceModel;

   3: using System.Runtime.Serialization;

   4:  

   5: namespace ContentTypes

   6: {

   7:     

   8:    [DataContract(Namespace="http://schemas.thatindigogirl.com/samples/2006/06")]

   9:     public class LinkItem

  10:     {

  11:  

  12:         [DataMember(Name = "Id", IsRequired = false, Order = 0)]

  13:         private long m_id;

  14:         [DataMember(Name = "Title", IsRequired = true, Order = 1)]

  15:         private string m_title;

  16:         [DataMember(Name = "Description", IsRequired = true, Order = 2)]

  17:         private string m_description;

  18:         [DataMember(Name = "DateStart", IsRequired = true, Order = 3)]

  19:         private DateTime m_dateStart;

  20:         [DataMember(Name = "DateEnd", IsRequired = false, Order = 4)]

  21:         private DateTime m_dateEnd;

  22:         [DataMember(Name = "Url", IsRequired = false, Order = 5)]

  23:         private string m_url;

  24:  

  25:         public DateTime DateStart

  26:         {

  27:             get { return m_dateStart; }

  28:             set { m_dateStart = value; }

  29:         } 

  30:  

  31:         public DateTime DateEnd

  32:         {

  33:             get { return m_dateEnd; }

  34:             set { m_dateEnd = value; }

  35:         }

  36:        

  37:         public string Url

  38:         {

  39:             get { return m_url; }

  40:             set { m_url = value; }

  41:         }

  42:         

  43:         public long Id

  44:         {

  45:             get { return m_id; }

  46:             set { m_id = value; }

  47:         }

  48:  

  49:         public string Title

  50:         {

  51:             get { return m_title; }

  52:             set { m_title = value; }

  53:         }

  54:  

  55:         public string Description

  56:         {

  57:             get { return m_description; }

  58:             set { m_description = value; }

  59:         }

  60:     }

  61: }

Note that now, in a C#3-friendly world, we can slim the definition of the LinkItem down to a much smaller level thanks to the power of automatic properties:

   1: using System;

   2: using System.ServiceModel;

   3: using System.Runtime.Serialization;

   4:  

   5: namespace ContentTypes

   6: {    

   7:     [DataContract(Namespace="http://schemas.thatindigogirl.com/samples/2006/06")]

   8:     public class LinkItem

   9:     {

  10:         [DataMember(Name = "Id", IsRequired = false, Order = 0)]

  11:         public long Id { get; set; }

  12:         [DataMember(Name = "Title", IsRequired = true, Order = 1)]

  13:         public string Title { get; set; }

  14:         [DataMember(Name = "Description", IsRequired = true, Order = 2)]

  15:         public string Description { get; set; }

  16:         [DataMember(Name = "DateStart", IsRequired = true, Order = 3)]

  17:         public DateTime DateStart { get; set; }

  18:         [DataMember(Name = "DateEnd", IsRequired = false, Order = 4)]

  19:         public DateTime DateEnd { get; set; }

  20:         [DataMember(Name = "Url", IsRequired = false, Order = 5)]

  21:         public string Url { get; set; }

  22:     }

  23: }

… but either way, the type ends up looking the same. Converting this over to F# is relatively easy, if not any shorter or more convenient than the C# 3.0 version, owing to the fact that, by default, F# will not generate mutable properties by default:

   1: #light

   2:  

   3: namespace ContentTypes

   4:     

   5: open System

   6: open System.Runtime.Serialization

   7: open System.ServiceModel

   8:  

   9: [<DataContract(Namespace="http://schemas.thatindigogirl.com/samples/2006/06")>]

  10: type LinkItem() =

  11:     let mutable id : int64 = 0L

  12:     let mutable title : string = String.Empty

  13:     let mutable description : string = String.Empty

  14:     let mutable dateStart : DateTime = DateTime.Now

  15:     let mutable dateEnd : DateTime = DateTime.Now

  16:     let mutable url : string = String.Empty

  17:  

  18:     [<DataMember(Name = "Id", IsRequired = false, Order = 0)>]

  19:     member public l.Id

  20:         with get() = id

  21:         and set(value) = id <- value

  22:     [<DataMember(Name = "Title", IsRequired = true, Order = 1)>]

  23:     member public l.Title

  24:         with get() = title

  25:         and set(value) = title <- value

  26:     [<DataMember(Name = "Description", IsRequired = true, Order = 2)>]

  27:     member public l.Description

  28:         with get() = description

  29:         and set(value) = description <- value

  30:     [<DataMember(Name = "DateStart", IsRequired = true, Order = 3)>]

  31:     member public l.DateStart

  32:         with get() = dateStart

  33:         and set(value) = dateStart <- value

  34:     [<DataMember(Name = "DateEnd", IsRequired = false, Order = 4)>]

  35:     member public l.DateEnd

  36:         with get() = dateEnd

  37:         and set(value) = dateEnd <- value

  38:     [<DataMember(Name = "Url", IsRequired = false, Order = 5)>]

  39:     member public l.Url

  40:         with get() = url

  41:         and set(value) = url <- value

Notice that I have to create a mutable backing field, and define the properties in the F# LinkItem type to explicitly access and mutate those values. This is a bit frustrating, because it seems like F# should be able to infer what I want from a simple property declaration, in the same way that C# can, but perhaps that’s asking too much from the language right now, considering the silly thing hasn’t even shipped yet.

(Psssst, Luke, Don, if you’re listening, automatic property generation in F# would be a nifty feature to add between now and then, if you guys can ninja it in there before the next CTP…)

Notice, by the way, the namespace directive at the top of the F# code; this is necessary to set the prefix around the LinkItem type. Without it, remember, the F# code is going to be slipped inside an outer class declaration matching the filename, effectively naming the class Module1+LinkItem, which would not be structurally equivalent to the C# type.

Lesson #4: Always put a namespace or module declaration around the types exported from a service.

Notice that LinkItem also has a default constructor, as per Lesson #2; this is necessary because the DataContract-related code inside of WCF is going to need to be able to construct one of these and set its properties. If we want to set any reasonable defaults, that’s easily done in the mutable member definitions.

One principal difference between the F# version and the C# version is that the DataMember attributes are applied to the properties, instead of the fields, largely because the F# language wants to keep a layer of encapsulation between the code you write as an F# programmer, and the actual code generated. So, for example, the “field” id, above, doesn’t actually get generated exactly as described—in truth, it turns into a field called “id@11”. This is a marked difference from C# (or even VB), which deliberately gives us more control over how the physical structure of classes looks. This is even more obvious in a basic F# program where a top-level declaration reads, “let x = 12”; where it might be tempting to assume that x will be a static field on the class surrounding the declaration, the F# compiler actually generates a property.

In this particular case, whether the attribute applies to the fields or the property declarations isn’t going to make a large difference, but in more sophisticated classes, it might, so it’s better to apply the attribute to the property and not the field, at least, from what I’ve found so far.

Lesson #5: Put DataMember attributes on the properties of the DataContract, not the fields.

Defining the Service

The definition of the service is actually pretty straightforward. Add either the C# ContentTypes.dll or the F# ContentTypes.dll as an assembly reference, and where the C# code (GigManagerService.cs) reads:

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Text;

   4: using System.ServiceModel;

   5: using ContentTypes;

   6:  

   7: namespace GigManager

   8: {

   9:     [ServiceContract(Name = "GigManagerServiceContract", Namespace = "http://www.thatindigogirl.com/samples/2006/06", SessionMode = SessionMode.Required)]

  10:     public interface IGigManagerService

  11:     {

  12:         [OperationContract]

  13:         void SaveGig(LinkItem item);

  14:  

  15:         [OperationContract]

  16:         LinkItem GetGig();

  17:     }

  18:  

  19:     [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession)]

  20:     public class GigManagerService : IGigManagerService

  21:     {

  22:  

  23:         private LinkItem m_linkItem;

  24:  

  25:         public void SaveGig(LinkItem item)

  26:         {

  27:             m_linkItem = item;

  28:         }

  29:  

  30:         public LinkItem GetGig()

  31:         {

  32:             return m_linkItem;

  33:         }

  34:     }

  35: }

… the corresponding F# code (Program.fs) reads like so:

   1: #light

   2:  

   3: module GigManager =

   4:     open System

   5:     open System.Runtime.Serialization

   6:     open System.ServiceModel

   7:     

   8:     open ContentTypes

   9:     

  10:     [<ServiceContract(Name = "GigManagerServiceContract", 

  11:         ConfigurationName = "IGigManagerService",

  12:         Namespace = "http://www.thatindigogirl.com/samples/2006/06", 

  13:         SessionMode = SessionMode.Required)>]

  14:     type IGigManagerService =

  15:         [<OperationContract>]

  16:         abstract SaveGig: item : LinkItem -> unit

  17:         [<OperationContract>]

  18:         abstract GetGig: unit -> LinkItem

  19:         

  20:     [<ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession)>]

  21:     type GigManagerService() =

  22:         let mutable li : LinkItem = LinkItem()

  23:         interface IGigManagerService with

  24:             member gms.SaveGig(item) = li <- item                

  25:             member gms.GetGig() = li

Careful readers will notice that there’s one additional element in the F# version that isn’t in the C# version; specifically, on line 11, I’ve added a “ConfigurationName” element to the IGigManagerService’s ServiceContract attribute. I do this because, again, the F# compiler is doing some interesting things to the code under the hood. In particular, the interface IGigManagerService is actually exposed under a slightly different name—remember, F# likes to use nested classes, not namespaces, so where the C# version of IGigManagerService is formally known as “GigManager::IGigManagerService”, the F# version is “Program/GigManager/GigManagerService”, where Program is the name of the .fs file. This seems to cause WCF some heartache when it starts looking through the App.config file and matching it up against the names exported from the actual class—it won’t match up correctly. So, by giving it a ConfigurationName that matches the human-readable interface name, WCF is happy again.

Lesson #5: Use ConfigurationName on ServiceContract to avoid having to learn F#’s naming bindings to the CLR.

The rest of the code in Program.fs is the hosting code, which structurally is no different than that of the previous post.

One key thing to remember, however, is that the host “service” element will also be looking at type names, so if you forget to set the name of the service, you’ll need to use a type-investigation tool (ILDasm or Reflector) to figure out what the host class name is; in the case above, it would be “Program+GigManager+GigManagerService”, forcing the App.config file to read as follows:

   1: <?xml version="1.0" encoding="utf-8" ?>

   2: <configuration>

   3:   <system.serviceModel>

   4:     <services>

   5:       <service name="Program+GigManager+GigManagerService" 

   6:                behaviorConfiguration="serviceBehavior">

   7:         <host>

   8:           <baseAddresses>

   9:             <add baseAddress="http://localhost:8000"/>

  10:             <add baseAddress="net.tcp://localhost:9000"/>

  11:           </baseAddresses>

  12:         </host>

  13:         <endpoint address="GigManagerService"

  14:                   binding="netTcpBinding"

  15:                   contract="IGigManagerService" />

  16:         <endpoint address="mex"

  17:                   binding="mexHttpBinding"

  18:                   contract="IMetadataExchange" />

  19:       </service>

  20:     </services>

  21:       <behaviors>

  22:           <serviceBehaviors>

  23:               <behavior name="serviceBehavior">

  24:                   <serviceMetadata httpGetEnabled="true"/>

  25:               </behavior>

  26:           </serviceBehaviors>

  27:       </behaviors>

  28:     <!-- This <diagnostics> section should be placed inside the <system.serviceModel> section. In addition, you'll need to add the <system.diagnostics> snippet to specify service model trace listeners and a file for output. -->

  29:     <diagnostics performanceCounters="All" wmiProviderEnabled="true" >

  30:       <messageLogging logEntireMessage="true" logMalformedMessages="true" logMessagesAtServiceLevel="true" logMessagesAtTransportLevel="true" maxMessagesToLog="100000"  />

  31:     </diagnostics>

  32:   </system.serviceModel>

  33:   <!-- This <system.diagnostics> section illustrates the use of a shared listener for service model output. It requires you to also add the <diagnostics> snippet for the <system.serviceModel> section. -->

  34:   <system.diagnostics >

  35:     <sharedListeners>

  36:       <add name="sharedListener" 

  37:                  type="System.Diagnostics.XmlWriterTraceListener"

  38:                  initializeData="c:\logs\servicetrace.svclog" />

  39:     </sharedListeners>

  40:     <sources>

  41:       <source name="System.ServiceModel" switchValue="Verbose, ActivityTracing" >

  42:         <listeners>

  43:           <add name="sharedListener" />

  44:         </listeners>

  45:       </source>

  46:       <source name="System.ServiceModel.MessageLogging" switchValue="Verbose">

  47:         <listeners>

  48:           <add name="sharedListener" />

  49:         </listeners>

  50:       </source>

  51:     </sources>

  52:   </system.diagnostics>

  53: </configuration>

Caveat emptor. In all honesty, despite the motivation of Lesson #5, I don’t think there’s any way around learning at least a little bit of F#’s name-mapping scheme, but at least we can be selective about where and when we apply it.

From Scott Hanselman's blog:

Are you in King County/Seattle/Redmond/Bellevue Washington and surrounding areas? Are you a huge nerd? Perhaps a geek? No? Maybe a dork, dweeb or wonk. Maybe you're in town for an SDR (Software Design Review) visiting BillG. Quite possibly you're just a normal person.

Regardless, why not join us for some Mall Food at the Crossroads Bellevue Mall Food Court on Monday, January 19th around 6:30pm?

...

NOTE: RSVP by leaving a comment here and show up on January 19th at 6:30pm! Feel free to bring friends, kids or family. Bring a Ruby or Java person!

Any of the SeaJUG want to attend? (Anybody know of a Ruby JUG in the Eastside area, by the way?) I'm game....

For a while now, I’ve held the opinion that the “sweet spot” for functional languages on the JVM and CLR will be in the services space, since services and functions seem pretty similar to one another in spirit—a given input produces a given output, with (ideally) no shared state, high concurrency expectations, idempotent processing, and so on. This isn’t to say that a functional language is going to make a non-trivial service trivial, but I think it will make it simpler and more likely to scale better over time, particularly as the service gets more complicated.

As part those explorations into the union of services and functional languages, I’ve been taking some of Michele Leroux Bustamente’s excellent labs from Learning WCF and flipping the services over to F#. Along the way, I’ve discovered a few “quirks” of F# that make building a WCF service a tad more complicated than it needs to be, so I’ve decided to blog what’s going on so others can find it easier.

(Many thanks to Nick Holmes’ blog, which helped identify one of the first problems I ran into, though a few things have changed since he blogged back in February, so I thought I’d catch everything up to the Sep 08 CTP of F#.)

This isn’t intended to be a tutorial on WCF, so if you’re not familiar with WCF, I strongly suggest you go get Michele’s book. I’m assuming you’ll know the WCF basics (address, binding, contract, config files, behaviors, etc), and I just want to show the deltas necessary to make F# work. Note that I’m just doing the service side of things—I believe clients will probably continue to be written in C# or VB or some other OO language, in keeping with the theory that OO will remain the predominant way of developing client-facing stuff. (Note that this also neatly avoids the basic problem that svcutil.exe only generates either C# or VB proxy code, and that “Add Service Reference” isn’t available inside an F# project, as of this writing.)

Defining Contracts in F#

The first step in any straight-up WCF service is, of course, to define the contract that both sides will agree to. (Yes, I know, we could do everything in terms of picking Message types apart; I’ll get to that in a later piece.) First things first: taking Michele’s HelloIndigo_Part1 solution, I add a new project to it, “FHost”, an F# application. Add the System.ServiceModel and System.Runtime.Serialization assemblies, and we’re good to get going.

Michele’s “HelloIndigo_Part1” solution defines the contract between client and service this way:

   1: namespace Host

   2: {

   3:     [ServiceContract(Namespace = "http://www.thatindigogirl.com/samples/2006/06")]

   4:     public interface IHelloIndigoService

   5:     {

   6:         [OperationContract]

   7:         string HelloIndigo();

   8:     }

   9:     // ...

  10: }

This contract can be consumed in two ways; one is to build this interface into its own assembly that’s linked to both the WCF service host and to the WCF client, but in her example (as is perfectly reasonable in a WCF project), she repeats the interface in both the client and the service, so to be faithful to that, let’s define the interface in the F# code:

   1: #light

   2:  

   3: open System

   4: open System.ServiceModel

   5:  

   6: [< ServiceContract(Namespace = "http://www.thatindigogirl.com/samples/2006/06") >]

   7: type IHelloIndigoService =

   8:     [< OperationContract >]

   9:     abstract HelloIndigo: unit -> string

(The color syntax highlighting is off because I’m using the C# mode of the “Code Snippet” plugin in Windows Live Writer to post the code, and it doesn’t have an F# mode. Yet.)

Pay very close attention to the interface definition in F#, because there is a subtle WCF “bug” that F# exposes by accident. When F# compiles an interface, if a method in the interface has parameters, if no name is specified for that parameter, then WCF will throw an ArgumentNullException when you try to run svcutil.exe over the compiled assembly, or when you pass the type in to the ServiceHost constructor, claiming “Value cannot be null. Parameter name: name”. The problem is that F#, unlike C# or VB, allows methods to have parameters without names, and WCF can’t handle this. Verifying this is a b*tch; if you use ILDasm to view the F#-compiled assembly, it looks like there are parameter names there, because ILDasm generates them as placeholders for display. (Reflector is your friend here.)

The WCF team has basically said that this behavior is by design—SOAP, which is a key concept to the WCF stack, doesn’t really have great support for unnamed parameters (and yes, I know, this is not exactly true, but I’m not going to get into that debate here), so the WCF team has basically said there’s really nothing they can do but maybe issue a better error message than ArgumentNullException.

Lesson #1: Always name your WCF contract interface params.

Caveat emptor.

Defining the Service Implementation

Next step is to define the service implementation. Again, Michele’s code looks like so:

   1: public class HelloIndigoService : IHelloIndigoService

   2: {

   3:     public string HelloIndigo()

   4:     {

   5:         return "Hello Indigo";

   6:     }

   7: }

Not a really difficult operation, so converting that to F# is pretty straightforward:

   1: [< ServiceBehavior(ConfigurationName="HIS") >]    

   2: type HelloIndigoService() =

   3:     interface IHelloIndigoService with

   4:         member s.HelloIndigo() : string =

   5:             "Hello Indigo"

There are two things important to this definition. First, the parentheses at the end of the “type” declaration line create a default no-argument constructor for the HelloIndigoService, which is required—without it, WCF is going to complain about being unable to construct an instance of this type.

Lesson #2: Always provide the default type constructor in the service implementation.

Second, the ServiceBehavior attribute is one I’ve added, because F# does some funky things with the type names during compilation; for example, since my F# code is in a file called “Host.fs”, the F# compiler synthesizes a class called “Host” which acts as a nesting wrapper around everything else in the file, so technically the typename of HelloIndigoService is now “Host+HelloIndigoService”, which will cause some chaos when WCF tries to match up the service name with the appropriate entry in the App.config file. You can either make sure the App.config matches the CLR-level type names generated by the F# compiler, or you can explicitly specify the configuration names; I choose the latter, so that it’s a bit more clear what’s going on.

Lesson #3: Always specify the configuration name on the service implementation.

The App.config file, by the way, now looks like this, the only change from Michele’s labs being the changes to the configuration name of the service behavior (line 13):

   1: <?xml version="1.0" encoding="utf-8" ?>

   2: <configuration>

   3:     <system.serviceModel>

   4:         <behaviors>

   5:             <serviceBehaviors>

   6:                 <behavior name="serviceBehavior">

   7:                     <serviceMetadata httpGetEnabled="false" />

   8:                 </behavior>

   9:             </serviceBehaviors>

  10:         </behaviors>

  11:         <services>

  12:             <service behaviorConfiguration="serviceBehavior" 

  13:                      name="HIS">

  14:                 <clear />

  15:                 <endpoint address="HelloIndigoService" 

  16:                           binding="basicHttpBinding"

  17:                           name="basicHttp" 

  18:                           contract="Host+IHelloIndigoService" />

  19:                 <endpoint binding="mexHttpBinding" 

  20:                           name="mex" 

  21:                           contract="IMetadataExchange" />

  22:                 <host>

  23:                     <baseAddresses>

  24:                         <add baseAddress="http://localhost:8000/HelloIndigo" />

  25:                     </baseAddresses>

  26:                 </host>

  27:             </service>

  28:         </services>

  29:     </system.serviceModel>

  30: </configuration>

Still with me? One last part to go, defining the (self-hosting) host.

Defining the Self-Hosting Host

In simple examples, frequently the service code self-hosts, meaning it doesn’ t need to be deployed into IIS. Michele uses a wrapper class to defer some of the hosting details, a la:

   1: internal class MyServiceHost

   2: {

   3:     internal static ServiceHost myServiceHost = null;

   4:  

   5:     internal static void StartService()

   6:     {

   7:         // Instantiate new ServiceHost 

   8:         myServiceHost = new ServiceHost(typeof(HelloIndigoService));

   9:         myServiceHost.Open();

  10:     }

  11:  

  12:     internal static void StopService()

  13:     {

  14:         // Call StopService from your shutdown logic (i.e. dispose method)

  15:         if (myServiceHost.State != CommunicationState.Closed)

  16:             myServiceHost.Close();

  17:     }

  18: }

  19:  

  20: class Program

  21: {

  22:     static void Main(string[] args)

  23:     {

  24:         try

  25:         {

  26:             MyServiceHost.StartService();

  27:             Console.WriteLine("Press <ENTER> to terminate the host application");

  28:             Console.ReadLine();

  29:         }

  30:         finally

  31:         {

  32:             MyServiceHost.StopService();

  33:         }

  34:     }

  35: }

I don’t quite think the wrapper is necessary, so I simplified it down to:

   1: let main() =

   2:     Console.WriteLine("IHelloIndigoService = " + typeof<IHelloIndigoService>.ToString() )

   3:  

   4:     let hisType = typeof<HelloIndigoService>

   5:     let host = new ServiceHost(hisType, ([| |] : Uri[] ) )

   6:     host.Open()

   7:     Console.WriteLine("Press <ENTER> to terminate the host application")

   8:     Console.ReadLine() |> ignore

   9:     host.Close()

  10:  

  11: main()

One quirk of the current (Sept 08) F# CTP is that when working with variable-argument parameters (like the second argument of the ServiceHost constructor), F# doesn’t have a great syntax. We have to explicitly specify, in this case, an empty array of Uri objects, but simply specifying an empty array (“[| |]”) will be interpreted as an empty array of objects, and thus generate a compile error. We have to explicitly set the type of the array to be an array of Uri, hence the type specifier.

Oh, and don’t forget, if you’re running as a non-Administrator on Vista or XP, you’ll need to create a URL ACL to allow a non-Administrator user to create an HTTP endpoint; the relevant command for the example above is this:

netsh http add urlacl url=http://+:8000/HelloIndigo user=devtop-t42p\ted

(Obviously, you substitute in your own domain and username for mine.) Make sure to do this from an Administrator-enabled command prompt, or you’ll just get another security error.

The beautiful thing about this example is that if it works, you can use the Client written in C# without a hitch of a problem, thus demonstrating quite clearly that WCF isn’t sharing assemblies between client and service. Given that this service also sets up the MEX endpoint, you should also be able to run svcutil against the running service and generate proxy code if you want to prove that it’s doable; I didn’t do it for this example, since I trust that the App.config-specified MEX endpoint will still be there, and because I was more interested in taking the existing Client and making it work as-is.

More to come, but this should get you started, anyway. Thanks again to Michele for letting me scaffold off of her!

Chris Sells, an acquaintance (and perhaps friend, when he's not picking on me for my Java leanings) of mine from my DevelopMentor days, has a habit of putting on a "DevCon" whenever a technology seems to have reached a certain maturity level. He did it with XML a few years ago, and ATL before that, both of which were pretty amazing events, filled with the sharpest guys in the subject, gathered into a single room to share ideas and shoot each others' pet theories full of holes.

He's at it again, this time with DSLs; from the announcement on his blog:

Are you interested in presenting a 45-minute talk on some Domain Specific Language (DSL) related topic? It doesn't matter which platform or OS you're targeting. It also doesn't matter whether you're an author, a vendor, a professional speaker or a developer in the trenches (in fact, I tend to be biased toward the latter). We're after interesting and unique applications of DSL technology and if you're doing good work in that area, then I need you to send me a session topic and 2-4 sentence abstract along with a little bit about yourself.

I'll be taking submissions 'til February 9th, 2009, but don't delay. Passion and a burning story to tell count twice as much as anything else.

And don't be shy about spreading this announcement around! I've got good coverage in the .NET and Windows communities, but don't know very many folks in the Java or Unix or hardcore modeling worlds, so if you're in that world, let those guys know! Thanks.

The DSL DevCon itself will be in Redmond, WA on the Microsoft campus April 16-17, 2009, right after the Lang.NET conference. Lang.NET will be focused on general-purpose languages, whereas the DSL DevCon will focus on domain-specific languages. The idea is that if you want to attend one or the other or both, that's totally fine. We'll have 2.5 days of Lang.NET on April 14-16 and then 1.5 days of DSL DevCon content.

Oh, and the cost for both conferences is the same: $0.

We're only accepting 150 attendees to either conference. Every one of the five previous DevCons have sold out, so when we open registration, you'll want to be quick about getting your name on the list.

Submit your DSL-related talk idea!

For those of you who are deep in the Java or Ruby space, I really urge you to take a chance here and come to the event--just because it's being held on the Microsoft campus doesn't mean you're going to be forcibly plugged into the Matrix; the same goes for the Lang.NET event in the earlier part of the week, too. Don't believe me? I have proof: Brian Goetz, John Rose, and Charlie Nutter, Sun employees all, attended last years Lang.NET event, talked about the JVM and JRuby, and not only did they not have to give up their "sun.com" email addresses, but they came away with some new appreciations for the CLR, the ecosystem there, and even a few insights about their own platform in comparison to the JVM. (I won't say this as an absolute fact, but I think a lot of John's work on method handles for Java7 came out of conversations he'd had with some of the CLR guys that week.)

This is a DevCon, not a MarCon or a SaleCon. If you're a dev, you're welcome to come here. Frankly, I'd love to see the Java and Ruby (and LLVM and Parrot and ...) guys storm the castle, so to speak, if for no other reason than so Chris will stop teasing me about being a Java guy.

This is more a continuation of my earlier Windows7 post, but I've installed the new Windows7 beta into a VMWare Fusion VM with zero difficulties, and I just finished putting VS2008 (and the SP1 patch) on it, then the latest F# CTP on top of that, and so far it all looks pretty smooth. Put in the DDK and the SDK, and I've got a nice Windows7 development image to play with.

I've had a few people ask me if I've still had problems with the mouse, but to be honest I installed it without the driver installed in the VMWare Tools install, so as soon as I copy off the .vmdk and .vmss files to a quiet little corner of the hard drive as backup, I'll try installing the mouse driver to see if it works, and report back here soon.

An open message to the Visual Studio installation team: One thing I'd like to see changed for VS2010--instead of giving me a "cmd.exe" environment for using VS from the command-line, can you at least give me a PowerShell .ps1 shell link to go alongside it? And why does the VS2008 SP1 patch require me to put Visual Studio in the CD tray to reference the vs_setup.msi about halfway through?

Update: Mouse driver works flawlessly. Dunno if it was a bug they fixed, or just random good VM karma, but the entire VMWare Tools package now works perfectly, as far as I can tell. Note: I haven't heard any sound out of it, but sometimes the sound driver in Fusion cuts out for reasons beyond my understanding, and after a reboot, sound is back without a problem. Besides, sound is not as important to me in a work VM as mouse or network, anyway, so....

For a couple of years now, I've been going around the world and giving a talk entitled "Pragmatic Architecture", talking both about what architecture is (and what architects really do), and ending the talk with my own "catalog" of architectural elements and ideas, in an attempt to take some of the mystery and "cloud" nature of architecture out of the discussion. If you've read Effective Enterprise Java, then you've read the first version of that discussion, where Pragmatic Architecture was a second-generation thought process.

Recently, the patterns & practices group at Microsoft went back and refined their Application Architecture Guide, and while there's a lot about it that I wish they'd done differently (less of a Microsoft-centric focus, for one), I think it's a great book for Microsoft-centric architects to pick up and have nearby. In a lot of ways, this is something similar to what I had in mind when I thought about the architectural catalog, though I'll admit that I'd prefer to go one level "deeper" and find more of the "atoms" that make up an architecture.

Nevertheless, I think this is a good PDF to pull down and put somewhere on your reference list.

Notes and caveats: Firstly, this is a book for solution architects; if you're the VP or CTO, don't bother with it, just hand it to somebody further on down the food chain. Secondly, if you're not an architect, this is not the book to pick up to learn how to be one. It's more in the way of a reference guide for existing architects. In fact, my vision is that an architect faced with a new project (that is, a new architecture to create) will think about the problem, sketch out a rough solution in his head, then look at the book to find both potential alternatives (to see if they fit better or worse than the one s/he has in her/his head), and potential consequences (to the one s/he has in her/his head). Thirdly, even if you're a Java or Ruby architect, most of the book is pretty technology-neutral. Just take a black Sharpie to the parts that have the Microsoft trademark around them, and you'll find it a pretty decent reference, too. Fourthly, in the spirit of full disclosure, the p&p guys brought me in for a day of discussion on the Guide, so I can't say that I'm completely unbiased, but I can honestly say that I didn't write any of it, just offered critique (in case that matters to any potential readers).

It's once again that time of year, and in keeping with my tradition, I'll revisit the 2008 predictions to see how close I came before I start waxing prophetic on the coming year. (I'm thinking that maybe the next year--2010's edition--I should actually take a shot at predicting the next decade, but I'm not sure if I'd remember to go back and revisit it in 2020 to see how I did. Anybody want to set a calendar reminder for Dec 31 2019 and remind me, complete with URL? )

Without further preamble, here's what I said for 2008:

• THEN: General: The buzz around building custom languages will only continue to build. More and more tools are emerging to support the creation of custom programming languages, like Microsoft's Phoenix, Scala's parser combinators, the Microsoft DLR, SOOT, Javassist, JParsec/NParsec, and so on. Suddenly, the whole "write your own lexer and parser and AST from scratch" idea seems about as outmoded as the idea of building your own String class. Granted, there are cases where a from-hand scanner/lexer/parser/AST/etc is the Right Thing To Do, but there are times when building your own String class is the Right Thing To Do, too. Between the rich ecosystem of dynamic languages that could be ported to the JVM/CLR, and the interesting strides being made on both platforms (JVM and CLR) to make them more "dynamic-friendly" (such as being able to reify classes or access the call stack directly), the probability that your company will find a need that is best answered by building a custom language are only going to rise. NOW: The buzz has definitely continued to build, but buzz can only take us so far. There's been some scattershot use of custom languages in a few scattershot situations, but it's certainly not "taken the world by storm" in any meaningful way yet.
• THEN: General: The hype surrounding "domain-specific languages" will peak in 2008, and start to generate a backlash. Let's be honest: when somebody looks you straight in the eye and suggests that "scattered, smothered and covered" is a domain-specific language, the term has lost all meaning. A lexicon unique to an industry is not a domain-specific language; it's a lexicon. Period. If you can incorporate said lexicon into your software, thus making it accessible to non-technical professionals, that's a good thing. But simply using the lexicon doesn't make it a domain-specific language. Or, alternatively, if you like, every single API designed for a particular purpose is itself a domain-specific language. This means that Spring configuration files are a DSL. Deployment descriptors are a DSL. The Java language is a DSL (since the domain is that of programmers familiar with the Java language). See how nonsensical this can get? Until somebody comes up with a workable definition of the term "domain" in "domain-specific language", it's a nonsensical term. The idea is a powerful one, mind you--creating something that's more "in tune" with what users understand and can use easily is a technique that's been proven for decades now. Anybody who's ever watched an accountant rip an entirely new set of predictions for the new fiscal outlook based entirely on a few seed numbers and a deeply-nested set of Excel macros knows this already. Whether you call them domain-specific languages or "little languages" or "user-centric languages" or "macro language" is really up to you. NOW: The backlash hasn't begun, but only because the DSL buzz hasn't materialized in much way yet--see previous note. It generally takes a year or two of deployments (and hard-earned experience) before a backlash begins, and we haven't hit that "deployments" stage yet in anything yet resembling "critical mass" yet. But the DSL/custom language buzz continues to grow, and the more the buzz grows, the more the backlash is likey.
• THEN: General: Functional languages will begin to make their presence felt. Between Microsoft's productization plans for F# and the growing community of Scala programmers, not to mention the inherently functional concepts buried inside of LINQ and the concurrency-friendly capabilities of side-effect-free programming, the world is going to find itself working its way into functional thinking either directly or indirectly. And when programmers start to see the inherent capabilities inside of Scala (such as Actors) and/or F# (such as asynchronous workflows), they're going to embrace the strange new world of functional/object hybrid and never look back. NOW: Several books on F# and Scala (and even one or two on Haskell!) were published in 2008, and several more (including one of my own) are on the way. The functional buzz is building, and lots of disparate groups are each evaluating it (functional programming) independently.
• THEN: General: MacOS is going to start posting some serious market share numbers, leading lots of analysts to predict that Microsoft Windows has peaked and is due to collapse sometime within the remainder of the decade. Mac's not only a wonderful OS, but it's some of the best hardware to run Vista on. That will lead not a few customers to buy Mac hardware, wipe the machine, and install Vista, as many of the uber-geeks in the Windows world are already doing. This will in turn lead Gartner (always on the lookout for an established trend they can "predict" on) to suggest that Mac is going to end up with 115% market share by 2012 (.8 probability), then sell you this wisdom for a mere price of $1.5 million (per copy). NOW: Can't speak to the Gartner report--I didn't have $1.5 million handy--but certainly the MacOS is growing in popularity. More on that later.
• THEN: General: Ted will be hired by Gartner... if only to keep him from smacking them around so much. .0001 probability, with probability going up exponentially as my salary offer goes up exponentially. (Hey, I've got kids headed for college in a few years.) NOW: Well, Gartner appears to have lost my email address and phone number, but I'm sure they were planning to make me that offer.
• THEN: General: MacOS is going to start creaking in a few places. The Mac OS is a wonderful OS, but it's got its own creaky parts, and the more users that come to Mac OS, the more that software packages are going to exploit some of those creaky parts, leading to some instability in the Mac OS. It won't be widespread, but for those who are interested in finding it, they're there. Assuming current trends (of customers adopting Mac OS) hold, the Mac OS 10.6 upgrade is going to be a very interesting process, indeed. NOW: Shhh. Don't tell anybody, but I've been seeing it starting to happen. Don't get me wrong, Apple still does a pretty good job with the OS, but the law of numbers has started to create some bad upgrade scenarios for some people.
• THEN: General: Somebody is going to realize that iTunes is the world's biggest monopoly on music, and Apple will be forced to defend itself in the court of law, the court of public opinion, or both. Let's be frank: if this were Microsoft, offering music that can only be played on Microsoft music players, the world would be through the roof. All UI goodness to one side, the iPod represents just as much of a monopoly in the music player business as Internet Explorer did in the operating system business, and if the world doesn't start taking Apple to task over this, then "justice" is a word that only applies when losers in an industry want to drag down the market leader (which I firmly believe to be the case--nobody likes more than to pile on the successful guy). NOW: Nothing this year.
• THEN: General: Somebody is going to realize that the iPhone's "nothing we didn't write will survive the next upgrade process" policy is nothing short of draconian. As my father, who gets it right every once in a while, says, "If I put a third-party stereo in my car, the dealer doesn't get to rip it out and replace it with one of their own (or nothing at all!) the next time I take it in for an oil change". Fact is, if I buy the phone, I own the phone, and I own what's on it. Unfortunately, this takes us squarely into the realm of DRM and IP ownership, and we all know how clear-cut that is... But once the general public starts to understand some of these issues--and I think the iPhone and iTunes may just be the vehicle that will teach them--look out, folks, because the backlash will be huge. As in, "Move over, Mr. Gates, you're about to be joined in infamy by your other buddy Steve...." NOW: Apple released iPhone 2.0, and with it, the iPhone SDK, so at least Apple has opened the dashboard to third-party stereos. But the deployment model (AppStore) is still a bit draconian, and Apple still jealously holds the reins over which apps can be deployed there and which ones can't, so maybe they haven't learned their lesson yet, after all....
• THEN: Java: The OpenJDK in Mercurial will slowly start to see some external contributions. The whole point of Mercurial is to allow for deeper control over which changes you incorporate into your build tree, so once people figure out how to build the JDK and how to hack on it, the local modifications will start to seep across the Internet.... NOW: OpenJDK has started to collect contributions from external (to Sun) sources, but still in relatively small doses, it seems. None of the local modifications I envisioned creeping across the 'Net have begun, that I can see, so maybe it's still waiting to happen. Or maybe the OpenJDK is too complicated to really allow for that kind of customization, and it never will.
• THEN: Java: SpringSource will soon be seen as a vendor like BEA or IBM or Sun. Perhaps with a bit better reputation to begin, but a vendor all the same. NOW: SpringSource's acquisition of G2One (the company behind Groovy just as SpringSource backs Spring) only reinforced this image, but it seems it's still something that some fail to realize or acknowledge due to Spring's open-source (?) nature. (I'm not a Spring expert by any means, but apparently Spring 3 was pulled back inside the SpringSource borders, leading some people to wonder what SpringSource is up to, and whether or not Spring will continue to be open source after all.)
• THEN: .NET: Interest in OpenJDK will bootstrap similar interest in Rotor/SSCLI. After all, they're both VMs, with lots of interesting ideas and information about how the managed platforms work. NOW: Nope, hasn't really happened yet, that I can see. Not even the 2nd edition of the SSCLI book (by Joel Pobar and yours truly, yes that was a plug) seemed to foster the kind of attention or interest that I'd expected, or at least, not on the scale I'd thought might happen.
• THEN: C++/Native: If you've not heard of LLVM before this, you will. It's a compiler and bytecode toolchain aimed at the native platforms, complete with JIT and GC. NOW: Apple sank a lot of investment into LLVM, including hosting an LLVM conference at the corporate headquarters.
• THEN: Java: Somebody will create Yet Another Rails-Killer Web Framework. 'Nuff said. NOW: You know what? I honestly can't say whether this happened or not; I was completely not paying attention.
• THEN: Native: Developers looking for a native programming language will discover D, and be happy. Considering D is from the same mind that was the core behind the Zortech C++ compiler suite, and that D has great native platform integration (building DLLs, calling into DLLs easily, and so on), not to mention automatic memory management (except for those areas where you want manual memory management), it's definitely worth looking into. www.digitalmars.com NOW: D had its own get-together as well, and appears to still be going strong, among the group of developers who still work on native apps (and aren't simply maintaining legacy C/C++ apps).

Now, for the 2009 predictions. The last set was a little verbose, so let me see if I can trim the list down a little and keep it short and sweet:

• General: "Cloud" will become the next "ESB" or "SOA", in that it will be something that everybody will talk about, but few will understand and even fewer will do anything with. (Considering the widespread disparity in the definition of the term, this seems like a no-brainer.)
• Java: Interest in Scala will continue to rise, as will the number of detractors who point out that Scala is too hard to learn.
• .NET: Interest in F# will continue to rise, as will the number of detractors who point out that F# is too hard to learn. (Hey, the two really are cousins, and the fortunes of one will serve as a pretty good indication of the fortunes of the other, and both really seem to be on the same arc right now.)
• General: Interest in all kinds of functional languages will continue to rise, and more than one person will take a hint from Bob "crazybob" Lee and liken functional programming to AOP, for good and for ill. People who took classes on Haskell in college will find themselves reaching for their old college textbooks again.
• General: The iPhone is going to be hailed as "the enterprise development platform of the future", and companies will be rolling out apps to it. Look for Quicken iPhone edition, PowerPoint and/or Keynote iPhone edition, along with connectors to hook the iPhone up to a presentation device, and (I'll bet) a World of Warcraft iPhone client (legit or otherwise). iPhone is the new hotness in the mobile space, and people will flock to it madly.
• .NET: Another Oslo CTP will come out, and it will bear only a superficial resemblance to the one that came out in October at PDC. Betting on Oslo right now is a fools' bet, not because of any inherent weakness in the technology, but just because it's way too early in the cycle to be thinking about for anything vaguely resembling production code.
• .NET: The IronPython and IronRuby teams will find some serious versioning issues as they try to manage the DLR versioning story between themselves and the CLR as a whole. An initial hack will result, which will be codified into a standard practice when .NET 4.0 ships. Then the next release of IPy or IRb will have to try and slip around its restrictions in 2010/2011. By 2012, IPy and IRb will have to be shipping as part of Visual Studio just to put the releases back into lockstep with one another (and the rest of the .NET universe).
• Java: The death of JSR-277 will spark an uprising among the two leading groups hoping to foist it off on the Java community--OSGi and Maven--while the rest of the Java world will breathe a huge sigh of relief and look to see what "modularity" means in Java 7. Some of the alpha geeks in Java will start using--if not building--JDK 7 builds just to get a heads-up on its impact, and be quietly surprised and, I dare say, perhaps even pleased.
• Java: The invokedynamic JSR will leapfrog in importance to the top of the list.
• Windows: Another Windows 7 CTP will come out, and it will spawn huge media interest that will eventually be remembered as Microsoft promises, that will eventually be remembered as Microsoft guarantees, that will eventually be remembered as Microsoft FUD and "promising much, delivering little". Microsoft ain't always at fault for the inflated expectations people have--sometimes, yes, perhaps even a lot of times, but not always.
• Mac OS: Apple will begin to legally threaten the clone market again, except this time somebody's going to get the DOJ involved. (Yes, this is the iPhone/iTunes prediction from last year, carrying over. I still expect this to happen.)
• Languages: Alpha-geek developers will start creating their own languages (even if they're obscure or bizarre ones like Shakespeare or Ook#) just to have that listed on their resume as the DSL/custom language buzz continues to build.
• XML Services: Roy Fielding will officially disown most of the "REST"ful authors and software packages available. Nobody will care--or worse, somebody looking to make a name for themselves will proclaim that Roy "doesn't really understand REST". And they'll be right--Roy doesn't understand what they consider to be REST, and the fact that he created the term will be of no importance anymore. Being "REST"ful will equate to "I did it myself!", complete with expectations of a gold star and a lollipop.
• Parrot: The Parrot guys will make at least one more minor point release. Nobody will notice or care, except for a few doggedly stubborn Perl hackers. They will find themselves having nightmares of previous lives carrying around OS/2 books and Amiga paraphernalia. Perl 6 will celebrate it's seventh... or is it eighth?... anniversary of being announced, and nobody will notice.
• Agile: The debate around "Scrum Certification" will rise to a fever pitch as short-sighted money-tight companies start looking for reasons to cut costs and either buy into agile at a superficial level and watch it fail, or start looking to cut the agilists from their company in order to replace them with cheaper labor.
• Flash: Adobe will continue to make Flex and AIR look more like C# and the CLR even as Microsoft tries to make Silverlight look more like Flash and AIR. Web designers will now get to experience the same fun that back-end web developers have enjoyed for near-on a decade, as shops begin to artificially partition themselves up as either "Flash" shops or "Silverlight" shops.
• Personal: Gartner will still come knocking, looking to hire me for outrageous sums of money to do nothing but blog and wax prophetic.

Well, so much for brief or short. See you all again next year....

It amazes me how insular and inward-facing the software industry is. And how the "agile" movement is reaping the benefits of a very simple characteristic.

For example, consider Jeff Palermo's essay on "The Myth of Self-Organizing Teams". Now, nothing against Jeff, or his post, per se, but it amazes me how our industry believes that they are somehow inventing new concepts, such as, in this case the "self-organizing team". Team dynamics have been a subject of study for decades, and anyone with a background in psychology, business, or sales has probably already been through much of the material on it. The best teams are those that find their own sense of identity, that grow from within, but still accept some leadership from the outside--the classic example here being the championship sports team. Most often, that sense of identity is born of a string of successes, which is why teams without a winning tradition have such a hard time creating the esprit de corps that so often defines the difference between success and failure.

(Editor's note: Here's a free lesson to all of you out there who want to help your team grow its own sense of identity: give them a chance to win a few successes, and they'll start coming together pretty quickly. It's not always that easy, but it works more often than not.)

How many software development managers--much less technical leads or project managers--have actually gone and looked through the management aisle at the local bookstore?

Tom and Mary Poppendieck have been spending years now talking about "lean" software development, which itself (at a casual glance) seems to be a refinement of the concepts Toyota and other Japanese manufacturers were pursuing close to two decades ago. "Total quality management" was a concept introduced in those days, the idea that anyone on the production line was empowered to stop the line if they found something that wasn't right. (My father was one of those "lean" manufacturing advocates back in the 80's, in fact, and has some great stories he can tell to its successes, and failures.)

How many software development managers or project leads give their developers the chance to say, "No, it's not right yet, we can't ship", and back them on it? Wouldn't you, as a developer, feel far more involved in the project if you knew you had that power--and that responsibility?

Or consider the "agile" notion of customer involvement, the classic XP "On-Site Customer" principle. Sales people have known for years, even decades (if not centuries), that if you involve the customer in the process, they are much more likely to feel an ownership stake sooner than if they just take what's on the lot or the shelf. Skilled salespeople have done the "let's walk through what you might buy, if you were buying, of course" trick countless numbers of times, and ended up with a sale where the customer didn't even intend to buy.

How many software development managers or project leads have read a book on basic salesmanship? And yet, isn't that notion of extracting what the customer wants endemic to both software development and basic sales (of anything)?

What is it about the software industry that just collectively refuses to accept that there might be lots of interesting research on topics that aren't technical yet still something that we can use? Why do we feel so compelled to trumpet our own "innovations" to ourselves, when in fact, they've been long-known in dozens of other contexts? When will we wake up and realize that we can learn a lot more if we cross-train in other areas... like, for example, getting your MBA?

Dustin Campbell, a self-professed "IDE guy", is speaking at the .NET Developer's Association of Redmond this evening, on the future of Visual Basic in Visual Studio 2010, and I feel compelled, based on my earlier "dissing" of VB in my thoughts of PDC post, to give VB a little love here.

First of all, he notes publicly that the VB and C# teams have been brought together under one roof, organizationally, so that the two languages can evolve in parallel to one another. I have my concerns about this. Frankly, I think the Managed Languages team at Microsoft is making a mistake by making these two languages mirror images of one another, no matter what their customers are telling them; it's creating an artificial competition between them, because if you can't differentiate between the two on a technical level, then the only thing left to differentiate them on is an aesthetic level (do you prefer curly braces and semicolons, or keywords?). Unfortunately, the market has already done so, to the tune of "C# developers make more than VB developers do (on average)", leaving little doubt in the minds of VB developers where they'd rather be... and even less doubt in the minds of C# developers where they'd rather the VB developers remain, lest the supply and demand curves shift and move the equilibrium point of C# developer salaries further south.

Besides, think about this for a moment: how much time and energy has Microsoft (and other .NET authors) had to invest in making sure that every SDK and every article ever written has both C# and VB sample code? All because Microsoft refuses to simply draw a line in the sand and say, once and for all, "C# is the best statically-typed object-oriented language for the CLR on the planet, and Visual Basic is the best dynamically-typed object-oriented language for the CLR on the planet", and run with it. Then at least there would be solid technical reasons for using one or the other, and at least we could take this out of the realm of aesthetics.

Or, contrarily, do the logical thing and create one language with two parsers, switching between them based on the file extension. That guarantees that the two evolve in parallel, and releases resources from the languages team to work on other things.

Next, he shows some simple spin-off-a-thread code, with the Thread constructor taking a parameter to a function name, traditional delegate kinds of stuff, then notes the disjoint nature of referencing a method defined elsewhere in the class but only to be used once. Yes, he's setting up for the punchline: VB gets anonymous methods, and "VB's support for lambda (expressions) reaches parity with C#'s" in this next release. I don't know if this was a feature that VB really needed to get, since I don't know that the target audience for VB is really one that cares about such things (and, before the VB community tries to lynch me, let me be honest and say that I'm not sure the target audience for C# does, either), but at least it's nice that such a powerful feature is now present in the VB language. Subject to the concerns of last paragraph, of course.

Look, at the end of the day, I want C# and VB to be full-featured languages each with their own raison d'etre, as the French say, their own "reason to be". Having these two "evolve in parallel" or "evolve in concert" with one another is only bound to keep the C#-vs-VB language wars going for far too long.

Along the way, he's showing off some IDE features, which presumably will be in place for both C# and VB (since the teams are now unified under a single banner), what he's calling