Numerous folks have taken me to task (some here in comments, some through private email, some through still other channels) over the last blog post; rather than try to respond to all individually, I figured it makes more sense to address the more salient points here:

1. "How dare you use the Vietnam War as an analogy for something so trivial as object/relational mapping?" First of all, let's make a few facts clear. My father served in Vietnam. I have friends in Iraq right now. My best friend from high school served in the Navy during the first Iraq. I studied Vietnam--along with numerous other wars and coflicts--for several years as an International Relations major in college, focused specifically on military history. I have nothing but deep respect for all soldiers, of all nations, who go off to risk their lives in services to their country. I am appalled at how quickly governments (ours and others) chuck troops into a situation without thinking of the long-term strategy. I've spent more time studying war and its effects on the solidiers, the governments and the people than most people have spent watching TV. I am very aware of the ghosts I'm treading upon when I use the word "Vietnam", and quite frankly, folks, we as a nation have yet to come to terms with what happened there. Rambo films don't exorcise ghosts, much as we might want them to. POW-MIA flags don't, either. Please don't bring your ghosts in with you when approaching this subject, and I'll leave mine behind as well.
2. "The Vietnam War is a bad analogy for O/R-M." Vietnam remains, for most Americans, as the quintessential symbol for "bloody, ugly, unresolvable quagmire". And, as some have pointed out in comments on the blog post already, all analogies break down eventually, and this one is no different--as one commenter put it, nobody ever died from a bad O/R-M tool. (Though the day is not far off when such could occur, given the incredible spread of technology into all corners of our lives--it's not too hard to imagine a day when a patient dies because a doctor received incorrect information about a medical allergy from the enterprise system he/she uses to call up patient records.) That said, however, I assert that the analogy is appropriate, and relevant, for a variety of reasons. One, because just as development teams frequently believe that the object/relational problem is "solvable", so too did the US government believe that the Communist insurgency (which was more of an independence movement than a Communist movement, we've since realized) was "solvable" in South Indochina. Two, development teams frequently believe that with "just a little bit more work, we're almost there..." (wherever "there" is, in the minds of the architect or team lead), just as the US government frequently predicted that the Viet Cong were on the verge of defeat, just a few more troops and the war is over... Three, the analogy holds because even as team leads and architects approach this problem having been burned before, they still attempt solutions to the problem, just as many of the US administrations' advisors believed that Vietnam was a dead-end and ill-fated, they still went in there anyway.
3. "You aren't being fair--after all, {insert-name-of-favorite-O/R-M-tool-here} doesn't suffer from that problem." Not yet, it doesn't. Or it does, but you just haven't run into it yet. Either answer is possible. And in the early years of the Vietnam conflict, we didn't suffer the problems that we commonly associate with the War--the poor morale, the rampant drug use among the military, the widespread unpopularity of the conflict back home, and so on. The danger here is on the far end of the Slippery Slope, not the near end.
4. "You aren't being fair--when you balance the pros and cons..." Perhaps not. But as someone who's built three O/R-M's in his lifetime, and refuses to build another one because they all faced the same end, despite very different beginnings, I worry more about the Slippery Slope and where it leaves us in the end. If your team can stay perched on the side of the Slope that yields the most benefits, then more power to you; but I worry about the day when the new college intern says to himself, "You know, with a bit more investment, I bet we could add inheritance...."
5. "Some languages do allow for varying numbers of fields." Actually, no, most of the languages cited as examples, including Ruby, don't allow for varying fields. Ruby has a feature called "open classes", in which you can change the definition of the class at any time, but it's still (very loosely) a class-based language. (The implementation of Ruby, from what I can see, seems to back this point--each object holds a pointer back to the class object it stems from, which means, at least to me, it's loosely class-based.) We can debate the semantics of this point for days, and frankly I welcome the discussion, but not in the context of this one. We can save that for another post/thread at another time.
6. "OK, but where can I go to get more info about O/R-M so I don't fall into the quagmire?" Excellent question. Roy Osherove has started a community site about O/R-Ms, which I think holds promise for discussion on the topic. The JDO crowd had several resources available at JDOCentral, and there's lots of discussion about O/R-M (stretching back several years) on TheServerSide. BEA, with its acquisition of Solarmetric, now owns one of the better O/R-M tools on the market, Kodo, and they're likely to still have numerous white papers and such on the subject.
7. "OK, but where can I go to get more info about object persistence tools?" Right now, the only one I have any faith in is the db4o project; in fact, I'm speaking at their first user/developer conference in London in a few weeks. I've used others (such as Versant) in the past, and frankly, wasn't incredibly impressed.
8. "OK, but where can I go to get more info about these other languages/approaches?" Keep your eye on LINQ, for starters, as that's one of the first mainstream attempts to bring some of these ideas into traditional statically-typed object platforms. Scala and F# I already mentioned. Ruby is another place to spend some time, as there's a lot of features Ruby has that are trying to make their way into other languages. And, although I will likely gather some serious heat for saying this, Visual FoxPro may have some of the most interesting "best of both worlds" mojo in the entire language space on this subject.
9. "Great post!" Thanks.

Make no mistake about it: I am deeply sympathetic to anyone who lost somebody--figuratively or literally--to the Vietnam conflict. I feel equally sympathetic to anyone who lost somebody in the Korean War (as my family did), World War Two, or even World War One before that. In fact, my sympathies go out to anyone lost in any of the conflicts across history and the globe in which men and women die for an ideal or symbol. It is an unfortunate statement about human affairs that we see war as the ultimate arbiter over power disputes between nations, but this is the world we live in now. If you don't care for that, then I encourage you to actively work to change it, regardless of your politics. I have far more respect for someone who virulently disagrees with my political viewpoints and actively promotes their own, than I do for those who agree with my politics and do nothing but complain.

Perhaps history will record Vietnam as America's greatest military failure, perhaps not. There is ample evidence to suggest that Vietnam will forever act as a check on American territorial expansionism (remember, Hawaii and Alaska gained statehood after World War Two), and more importantly, as a checkpoint to hold flagrant use of American military muscle in place. But be that as it may, the fact remains that Vietnam had an incalculable effect on American foreign policy and domestic agenda, and will continue to do so for the next several generations. And, as numerous examples from my own experience and others can attest, the use of O/R-M can have the same effect (relativisitically speaking) on a development team's efforts.

(Two years ago, at Microsoft's TechEd in San Diego, I was involved in a conversation at an after-conference event with Harry Pierson and Clemens Vasters, and as is typical when the three of us get together, architectural topics were at the forefront of our discussions. An crowd gathered around us, and it turned into an impromptu birds-of-a-feather session. The subject of object/relational mapping technologies came up, and it was there and then that I first coined the phrase, "Object/relational mapping is the Vietnam of Computer Science". In the intervening time, I've received numerous requests to flesh out the discussion behind that statement, and given Microsoft's recent announcement regarding "entity support" in ADO.NET 3.0 and the acceptance of the Java Persistence API as a replacement for both EJB Entity Beans and JDO, it seemed time to do exactly that.)

No armed conflict in US history haunts the American military more than $g(Vietnam). So many divergent elements coalesced to create the most decisive turning point in modern American history that it defies any layman's attempt to tease them apart. And yet, the story of Vietnam is fundamentally a simple one: The United States began a military project with simple yet unclear and conflicting goals, and quickly became enmeshed in a quagmire that not only brought down two governments (one legally, one through force of arms), but also deeply scarred American military doctrine for the next four decades (at least).

Although it may seem trite to say it, $g(Object/Relational Mapping) is the Vietnam of Computer Science. It represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy.

History

PBS has a good synopsis of the war, but for those who are more interested in Computer Science than Political/Military History, the short version goes like this:

$g(South Indochina), now known as Vietnam, Thailand, Laos and Cambodia, has a long history of struggle for autonomy. Before French colonial rule (which began in the mid-1800s), South Indochina wrestled for regional independence from China. During World War Two, the Japanese conquered the area, only to be later "liberated" by the Allies, leading France to resume their colonial rule (as did the British in their colonial territories elsewhere in Asia and India). Following WWII, however, the people of South Indochina, having thrown off one oppressor, extended their anti-occupation efforts to fight the French instead of the Japanese, and in 1954 the French capitulated, signing the $g(Geneva Peace Accords) to formally grant Vietnam its independence. Unfortunately, global pressures perverted the efforts somewhat, and instead of a lasting peace agreement a temporary solution was created, dividing the nation at the 17th parallel, creating two nations where formerly no such division existed. Elections were to be held in 1956 to reunify the country, but the US feared that too much power would be given to the $g(Communist Party of Vietnam) through these elections, and instead backed a counter-Communist state south of the 17th parallel and formed a series of multilateral agreements around it, such as $g(SEATO). The new nation of $g(South Vietnam) was born, and its first (dubiously) elected leader was $g(Ngo Dinh Diem), a staunchly anti-Communist who almost immediately declared his country under Communist attack. The $g(Eisenhower Administration) remained supportive of the Diem government, but Diem's loyalty with the people was almost nonexistent from the beginning.

By the time the US Democratic Party's $g(John F Kennedy) came to the White House, things were coming to a head in South Vietnam. Kennedy sent a team to Vietnam to research the conditions there and help formulate his strategy on the issue. In what's now known as the "$g(December 1961 White Paper)", an argument for an increase in military, technical and economic aid was presented, along with large-scale American "advisers" to help stabilize the Diem government and eliminate the $g(National Liberation Front), dubbed the $g(Viet Cong) by the US. What's not as widely known, however, is that a number of Kennedy's advisers argued against that buildup, calling Vietnam a "dead-end alley".

Faced with two diametrically opposite paths, Kennedy, as was typical for his administration, chose a middle path: instead of either a massive commitment or a complete withdrawal, Kennedy instead chose to seek a limited settlement, sending aid but not large numbers of troops, a path that was almost doomed from the beginning. Through a series of strategic blunders, including the forced relocation of rural villagers (known as the $g(Strategic Hamlet Program)), Diem's support was so deeply eroded that Kennedy hesitatingly and haltingly supported a coup, during which Diem was killed. Three weeks later, Kennedy was also assassinated, throwing the domestic US political scene into turmoil as well. Ironically, the conflict began by Kennedy would in fact later be associated most closely with his replacement.

Johnson's War

At the time of the Kennedy assassination, Vietnam had 16,000 American advisers in place, most of whom weren't involved in daily combat operations. Kennedy's Vice President and new replacement, however, $g(Lyndon Baines Johnson), was not convinced that this path was leading to success, and came to believe that more aggressive action was needed. Seizing on a dubious incident in which Vietnamese patrol boats attacked American destroyers¹ in the $g(Gulf of Tonkin), Johnson used pro-war sentiment in Congress to pass a resolution that gave him powers to conduct military action without an explicit declaration of war. To put it simply, Johnson wanted to fight this war "in cold blood": "This meant that America would go to war in Vietnam with the precision of a surgeon with little noticeable impact on domestic culture. A limited war called for limited mobilization of resources, material and human, and caused little disruption in everyday life in America." (source) In essence, it would be a war whose only impact would be felt by the Vietnamese--American life and society would go on without any notice of the events in Vietnam, thus leaving Johnson to pursue his first great love, his "Great Society", a domestic agenda designed to fix many of US society's ills, such as poverty². History, of course, knows better, and--perhaps cruelly--calls the Vietnam conflict "Johnson's War".

Initially, it must be noted that Vietnam-as-disaster is a more recent perception; Americans polled as late as 1967 were convinced that the war was a good thing, that Communism needed to be stopped and that Vietnam, should it fall, would be the first of a series of nations to succumb to Communist subversion. This "$g(Domino Theory)" was a common refrain for American politics in the latter half of the 20th century. Concerns of this sort plagued American foreign policy ever since the Communists successfully or nearly-successfully subverted several European governments during hte latter half of the 1940's, and then China in the 50's. (It must be noted that Eisenhower and $g(John Foster Dulles), formulators of the theory, never included Vietnam in their ring of dominos that must be preserved, and in fact Eisenhower was surprisingly apathetic about Vietnam during some of his meetings with Kennedy during the White House transition.)

In 1968, however, the Vietnam experience turned significantly, as the North Vietnamese and Viet Cong launched the $g(Tet Offensive), a campaign that put to lie all of the reassurances of the American government that it was winning the war in Vietnam. Ironically, as had been the case for much of the war, the NVA/VC forces lost a substantial number of troops, far more than their American opponents, yet the Tet Offensive is widely considered by historians to be the breaking point of American will in the war. Following that, popular opinion turned on Johnson, and in a dramatic news conference, he announced that he would not seek re-election. Furthermore, he announced that he would seek a negotiated settlement with the Vietnamese.

Nixon's Promise

Unfortunately, American negotiating position was seriously weakened by the very protests that had brought the Americans to the negotiating table in the first place; NVA/VC leadership recognized that the NVA/VC forces, despite staggering military losses that nearly broke them (several times), could simply continue to do as they were doing, and wring concessions from the Americans without offering any in return. Running on a platform that consisted mostly of a promise to "Get America out of Vietnam", Johnson's successor, Republican $g(Richard Nixon), tried several tactics to bring pressure to the NVA/VC forces to bargain, including increased air-combat presence (such as the $g(Christmas bombings) and $g(Operation Menu) ) and regular violations of nearby Laos and Cambodia, pursuing the line of supplies from North Vietnam to cells in South Vietnam. Nothing worked, however, and in 1973 Nixon's administration signed the $g(Paris Peace Agreement), ending American involvement in that conflict. Two years later, South Vietnam had been overrun, and on April 30, 1975, Communist forces captured Saigon, the capital of Vietnam, forcing the evacuation of the American embassy and the most memorable image of the war, that of streams of fleeing people seeking space on the Huey helicopter perched on the roof of the embassy.

War's End

The Second South Indochina War was over, America had experienced its most profound defeat ever in its history, and Vietnam became synonymous with "quagmire". Its impact on American culture was immeasurable, as it taught an entire generation of Americans to fear and mistrust their government, it taught American leaders to fear any amount of US military casualties, and brought the phrase "clear exit strategy" directly into the American political lexicon. Not until $g(Ronald Reagan) used the American military to "liberate" the small island nation of $g(Grenada) would American military intervention be considered a possible tool of diplomacy by American presidents, and even then only with great sensitivity to domestic concern, as $g(Bill Clinton) would find out during his peacekeeping missions to $g(Somalia) and $g(Kosovo). In quantifiable terms, too, Vietnam's effects clearly fell short of Johnson's goal of a war in "cold blood". Final tally: 3 million Americans served in the war, 150,000 seriously wounded, 58,000 dead, and over 1,000 MIA, not to mention nearly a million NVA/Viet Cong troop casualties, 250,000 South Vietnamese casualties, and hundreds of thousands--if not millions, as some historians advocated--of civilian casualties.

Lessons of Vietnam

Vietnam presents an interesting problem to the student of military and political history--exactly what went wrong, when, and where? Obviously, the US government's unwillingness to admit its failures during the war makes for an easy scapegoat, but no government in the history of modern society has ever been entirely truthful with its population about its fortunes of war; one such example includes (but is not limited to) the same US government's careful censorship of activities during World War Two, fifty years earlier, known in American history as "the last 'good' war". It's also tempting to point to the lack of a military objective as the crucial failing point of Vietnam, but other non-military objectives have been successfully executed by the US and other governments without the kind of colossal failure accompanying Vietnam's story. Moreover, it's important to note that the US did, in fact, have a clear objective in what it wanted out of the conflict in South Indochina: to stop the fall of the South Vietnam government, and, barring that, the cessation of the "spread" of Communism. Was it the reluctance of the US government to unleash the military to its fullest capabilities, as $g(General William Westmoreland) always claimed? Certainly the failure in Vietnam was not a military one; the casualty figures make it clear that the US, by any other measure, was clearly winning.

So what were the principal failures in Vietnam? And, more importantly, what does all this have to do with O/R Mapping?

Vietnam and O/R mapping

In the case of Vietnam, the United States political and military apparatus was faced with a deadly form of the $g(Law of Diminishing Returns). In the case of automated Object/Relational Mapping, it's the same concern--that early successes yield a commitment to use O/R-M in places where success becomes more elusive, and over time, isn't a success at all due to the overhead of time and energy required to support it through all possible use-cases. In essence, the biggest lesson of Vietnam--for any group, political or otherwise--is to know when to "cut bait and run", as fishermen say. Too often, as was the case in Vietnam, it is easy to justify further investment in a particular course of action by suggestion that abandoning that course somehow invalidates all the work--or, in Vietnam's case, the lives of American soldiers--that have already been paid. Phrases like "We've gone this far, surely we can see this thing through" and "To back out now is to throw away everything we've sacrificed up until this point" become commonplace. At least during the later, deeply bitter years of the second half of Vietnam, questions of patriotism came into question: if you didn't support the war, you were clearly a traitor, a Communist, obviously "unAmerican", disrespectful of all American veterans of any war fought on any soil for whatever reason, and you probably kicked your dog to boot. (It didn't help the protestors' cause that they blamed the soldiers for the war, holding them accountable--sometimes personally--for the decisions made by military and political leaders, most of whom neither the soldiers nor the protestors had ever met.)

Recognizing that all analogies fail eventually, and that the subject of Vietnam is deeper than this essay can examine, there are still lessons to be learned here in an entirely different arena. One of the key lessons of Vietnam was the danger of what's colloquially called "the Slippery Slope": that a given course of action might yield some early success, yet further investment into that action yields decreasingly commensurate results and increasibly dangerous obstacles whose only solution appears to be greater and greater commitment of resources and/or action. Some have called this "the Drug Trap", after the way pharmaceuticals (legal or illegal) can have diminished effect after prolonged use, requiring upped dosage in order to yield the same results. Others call this "the Last Mile Problem": that as one nears the end of a problem, it becomes increasingly difficult in cost terms (both monetary and abstract) to find a 100% complete solution. All are basically speaking of the same thing--the difficulty of finding an answer that allows our hero to "finish off" the problem in question, completely and satisfactorily.

We begin the analysis of Object/Relational Mapping--and its relationship to the Second South Indochina War--by examining the reasons for it in the first place. What drives developers away from using traditional relational tools to access a relational database, and to prefer instead tools such as O/R-M's?

The Object-Relational Impedence Mismatch

To say that objects and relational data sets are somehow constructed differently is typically not a surprise to any developer who's ever used both; except in extremely simplistic situations, it becomes fairly obvious to recognize that the way in which a relational data store is designed is subtly--and yet profoundly--different than how an object system is designed.

Object systems are typically characterized by four basic components: identity, state, behavior and encapsulation. Identity is an implicit concept in most O-O languages, in that a given object has a unique identity that is distinct from its state (the value of its internal fields)--two objects with the same state are still separate and distinct objects, despite being bit-for-bit mirrors of one another. This is the "identity vs. equivalence" discussion that occurs in languages like C++, C# or Java, where developers must distinguish between "a == b" and "a.equals(b)". The behavior of an object is fairly easy to see, a collection of operations clients can invoke to manipulate, examine, or interact with objects in some fashion. (This is what distinguishes objects from passive data structures in a procedural language like C.) Encapsulation is a key detail, preventing outside parties from manipulating internal object details, thus providing evolutionary capabilities to the object's interface to clients.³. From this we can derive more interesting concepts, such as type, a formal declaration of object state and behavior, association, allowing types to reference one another through a lightweight reference rather than complete by-value ownership (sometimes called composition), inheritance, the ability to relate one type to another such that the relating type incorporates all of the related type's state and behavior as part of its own, and polymorphism, the ability to substitute an object in where a different type is expected.

Relational systems describe a form of knowledge storage and retrieval based on predicate logic and truth statements. In essence, each row within a table is a declaration about a fact in the world, and SQL allows for operator-efficient data retrieval of those facts using predicate logic to create inferences from those facts. [Date04] and [Fussell] define the relational model as characterized by relation, attribute, tuple, relation value and relation variable. A relation is, at its heart, a truth predicate about the world, a statement of facts (attributes) that provide meaning to the predicate. For example, we may define the relation "PERSON" as {SSN, Name, City}, which states that "there exists a PERSON with a Social Security Number SSN who lives in City and is called Name". Note that in a relation, attribute ordering is entirely unspecified. A tuple is a truth statement within the context of a relation, a set of attribute values that match the required set of attributes in the relation, such as "{PERSON SSN='123-45-6789' Name='Catherine Kennedy' City='Seattle'}". Note that two tuples are considered identical if their relation and attribute values are also identical. A relation value, then, is a combination of a relation and a set of tuples that match that relation, and a relation variable is, like most variables, a placeholder for a given relation, but can change value over time. Thus, a relation variable People can be written to hold the relation {PERSON}, and consist of the relation value

{ {PERSON SSN='123-45-6789' Name='Catherine Kennedy' City='Seattle'},
  {PERSON SSN='321-54-9876' Name='Charlotte Neward' City='Redmond'},
  {PERSON SSN='213-45-6978' Name='Cathi Gero' City='Redmond'} }

Already, it's fairly clear to see that there are distinct differences between how the relational world and object world view the "proper" design of a system, and more will become apparent as time progresses. It's important to note, however, that so long as programmers prefer to use object-oriented programming languages to access relational data stores, there will always be some kind of object-relational mapping taking place--the two models are simply too different to bridge silently. (Arguably, the same is true of object-oriented and procedural programming, but that's another argument for another day.) O/R mappings can take place in a variety of forms, the easiest of which to recognize is the automated O/R mapping tool, such as $g(TopLink), $g(Hibernate) / $g(NHibernate), or $g(Gentle.NET). Another form of mapping is the hand-coded one, in which programmers use relational-oriented tools, such as JDBC or ADO.NET, to access relational data and extract it into a form more pleasing to object-minded developers "by hand". A third is to simply accept the shape of the relational data as "the" model from which to operate, and slave the objects around it to this approach; this is also known in the patterns lexicon as Table Data Gateway [PEAA, 144] or Row Data Gateway [PEAA 152]; many data-access layers in both Java and .NET use this approach and combine it with code-generation to simplify the development of that layer. Sometimes we build objects around the relational/table model, put some additional behavior around it, and call it Active Record [PEAA, 160].

In truth, this basic approach--to slave one model into the terms and approach of the other--has been the traditional answer to the impedance mismatch, effectively "solving" the problem by ignoring one half of it. Unfortunately, most development efforts, like the Kennedy Administration, aren't willing to see this through to its logical conclusion with a wholesale commitment to one approach over the other. For example, while most development teams would be happy to adopt an "objects-only" approach, doing so at the storage level implies the use of an Object Oriented DataBase Management System (OODBMS), a topic that frequently has no traction within upper management or the corporate data management team. The opposite approach--a "relational-only" approach--is almost nonsensical to consider, given the technology of the day at the time this was written⁴.

Given that it's impossible, then, to "unleash the objects to their fullest capabilities", as General Westmoreland might call it, we're left with some kind of hybrid object-to-relational mapping approach, preferably one that's automated as much as possible, so that developers can focus on their Domain Model, rather than on the details of the object-to-table(s) mapping. And here, unfortunately, is where the potential quagmire begins.

The Object-to-Table Mapping Problem

One of the first and most easily-recognizable problems in using objects as a front-end to a relational data store is that of how to map classes to tables. At first, it seems a fairly straightforward exercise--tables map to types, columns to fields. Even the field types appear to line up directly against the relational column types, at least to a fairly isomorphic degree: VARCHARs to Strings, INTEGERs to ints, and so on. So it makes sense that for any given class defined in the system, a corresponding table--likely to be of the same or closely related name--is defined to go with it. Or, perhaps, if the object code is being written to an already existing schema, then the class maps to the table.

But as time progresses, it's only natural that a well-trained object-oriented developer will seek to leverage inheritance in the object system, and seek ways to do the same in the relational model. Unfortunately, the relational model does not support any sort of polymorphism or IS-A kind of relation, and so developers eventually find themselves adopting one of three possible options to map inheritance into the relational world: table-per-class, table-per-concrete-class, or table-per-class-family. Each of them carries potentially significant drawbacks.

The table-per-class approach is perhaps the most easily understood, for it seeks to minimize the "distance" between the object model and the relational model; each class in the inheritance hierarchy gets its own relational table, and objects of derived types are stitched together from relational JOINs across the various inheritance-based tables. So, for example, if an object model has the base class Person, with Student derived from Person and GraduateState derived from Student, then there will be three tables required to hold this model, PERSON, STUDENT, and GRADUATESTUDENT, each holding the fields corresponding to the class of the same name. Relating these tables together, however, requires each to have an independent primary key (one whose value is not actually stored in the object entity) so that each derived class can have a foreign key relation to its superclass's table. The reason for this is clear: a GraduateStudent object, by virtue of its IS-A relationship to Student and Person, is a collection of all three sets of state, and the distinction between the classes is largely removed by the time an object of this type is created--in both Java and .NET, for example, the object itself is a chunk of memory that holds the instance fields defined in all of its classes and superclasses, along with a pointer to the table of methods defined by that same hierarchy. This means that when querying for a particular instance at the relational level, at least three JOINs must be made in order to bring all of the object's state into the object program's working memory.

Actually, it gets worse than that--if the object hierarchy continues to grow, say to include Professor, Staff, Undergrad (inherits from Student), and a whole hierarchy of AdjunctEmployees (inheriting from Staff), and the program wants to find all Persons whose last name is Smith, then JOINs must be done for every derived class in the system, since the semantics of "find all Persons" means that the query must seek data on the PERSON table, but then do an expensive set of JOINs to bring in the rest of the data from across the rest of the database, pulling in the PROFESSOR table to fetch the rest of the data, not to mention the UNDERGRAD, ADJUCTEMPLOYEE, STAFF, and other tables. Considering that JOINs are among the most expensive expressions in RDBMS queries, this is clearly not something to undertake lightly.

As a result, developers typically adopt one of the other two approaches, more complex in outlook but more efficient when dealing with relational storage: they either create a table per concrete (most-derived) class, preferring to adopt denormalization and its costs, or else they create a single table for the entire hierarchy, often in either case creating a discriminator column to indicate to which class each row in the table belongs. (Various hybrids of these schemes are also possible, but typically don't create results that are significantly different from these two.) Unfortunately, the denormalization costs are often significant for a large volume of data, and/or the table(s) will contain significant amounts of empty columns, which will need NULLability constraints on all columns, eliminating the powerful integrity constraints offered by an RDBMS.

Inheritance mapping isn't the end of it; associations between objects, the typical 1:n or m:n cardinality associations so commonly used in both SQL and/or UML, are handled entirely differently: in object systems, association is unidirectional, from the associator to the associatee (meaning the associated object(s) have no idea they are in fact associated unless an explicit bidirectional association is established), whereas in relational systems the association is actually reversed, from the associatee to the associator (via foreign key columns). This turns out to be surprisingly important, as it means that for m:n associations, a third table must be used to store the actual relationship between associator and associatee, and even for the simpler 1:n relationships the associator has no inherent knowledge of the relations to which it associates--discovering that data requires a JOIN against any or all associated tables at some point. (When to actually retrieve that data is a subject of some debate--see the Loading Paradox, below).

The Schema-Ownership Conflict

Discussions of inheritance-to-table and association mapping schemes also reveals a basic flaw: At heart, many object-relational mapping tools assume that the schema is something that can be defined according to schemes that help optimize the O/R-M's queries against the relational data. But this belies a basic problem, that often the database schema itself is not under the direct control of developers, but instead is owned by another group within the company, typically the database administration (DBA) group. To whom does responsibility for designing the database--and deciding when schema changes are permissible--belong?

In many cases, developers begin a new project with a "clean slate", an empty relational database whose schema is theirs to define as they see fit. But, soon after the project has shipped (sometimes even earlier than that, due to political and/or "turf war" issues), it becomes apparent that the developers' ownership of the schema is temporary at best--various departments begin clamoring for reports against the database, DBAs are held accountable to the performance of the database thereby giving them cause to call for "refactoring" and denormalization of the data, and other development teams may start inquiring about how they might make use of the data stored therein. Before too long, the schema must be "frozen", thereby potentially creating a barrier to object model refactoring (see The Coupling Concern, below). In addition, these other teams will expect to see a relational model defined in relational terms, not one which supports an entirely orthogonal form of persistence--for example, the "discriminator" column from the Inheritance-to-Table Mapping Problem will represent difficulties, and arguably be all but unusable, to relational report generators such as Crystal Reports. Unless developers are willing to write all reports (and their UIs, and their printing code, and their ad-hoc capabilities...) by hand, this is usually going to be an unacceptable state of affairs.

(To be fair, this is not so much a technical problem as it is a political problem, but it still represents a serious problem regardless of its source--or solution. And as such, it still represents an impediment to an object/relational mapping solution.)

The Dual-Schema Problem

A related issue to the question of schema ownership is that in an O/R-M solution, the metadata to the system is held fundamentally in two different places: once in the database schema, and once in the object model (another schema, if you will, expressed in Java or C# instead of DDL). Updates or refactorings to one will likely require similar updates or refactorings to the other. Refactoring code to match database schema changes is widely considered to be the easier of the two--refactoring the database frequently requires some kind of migration and/or adaptation of data already within the database, where code has no such requirement. (Objects, at least in this discussion, are ephemeral in-memory instances that will disappear once the process holding them terminates. If the objects are stored in some kind of object form that can persist across process execution--such as serialized object instances stored to disk--then refactoring objects becomes equally problematic.)

More importantly, while it's not uncommon for code to be deployed specifically to a single application, frequently database instances are used by more than one application, and it's frequently unacceptable to business to trigger a company-wide refactoring of code simply because a refactoring on one application requires a similar database-driven refactoring. As a result, as the system grows over time, there will be increasing pressure on the developers to "tie off" the object model from the database schema, such that schema changes won't require similar object model refactorings, and vice versa. In some cases, where the O/R-M doesn't permit such disconnection, an entirely private database instance may have to be deployed, with the exact schema the O/R-M-based solution was built against, creating yet another silo of data in an IT environment where pressure is building to reduce such silos.

Entity Identity Issues

As if these problems weren't enough, we then walk into another problem, that of identity of objects and relations. As noted above, object systems use an implicit sense of identity, typically based on the object's location in memory (the ubiquitous this pointer); alternatively, this is sometimes referred to as an OID (Object IDentifier), usually in systems which don't directly expose memory locations, such as the object database (where an in-memory pointer is pretty useless as an identifier outside of the database process). In a relational model, however, identity is implicit in the state itself--two rows with the exact same state are typically considered a relational data corruption, as the same fact asserted twice is redundant and counterproductive. To be fair, we should be a bit more explicit here; a relational system can, in fact, permit duplicate tuples (as described above), but this is often explicitly disallowed by explicit relational constraints, such as PRIMARY KEY constraints. In those situations where duplicate values are allowed, there is no way for a relational system to determine which of the two duplicate rows are being retrieved--there is no implicit sense of identity to the relation except that offered by its attributes. The same is not true of object systems, where two objects that contain precisely identical bit patterns in two different locations of memory are in fact separate objects. (This is the reason for the distinction between "==" and ".equals()" in Java or C#.) The implication here is simple: if the two systems are going to agree on the sense of identity, the relational system must offer some kind of unique identity concept (usually an auto-incrementing integer column) to match that of the notion of object identity.

This causes some serious concerns regarding automated O/R systems, because the sense of identity is entirely different--if two separate user sessions interact with the same relation in storage, the relational database system's concurrency systems kick in and ensure some form of concurrent access, typically via the transactional metaphor (ACID). If an O/R system retrieves a relation out of storage (essentially forming a "view" over the data), we now have a second source of data identity, one in the database (protected by the aforementioned transactional scheme), and one in the in-memory object representation of that data, which has no consistent transactional support aside from that built into the language (such as the monitors concept in Java and .NET) or libraries (such as System.Transactions in .NET 2.0), either of which can be--and unfortuantely frequently are--easily ignored by developers. Managing isolation and concurrency is not an easy problem to solve, and unfortunately the languages and platforms commonly available to developers aren't yet as consistent or flexible as the database transaction metaphor.

What complicates this problem further is that many O/R systems introduce significant caching support into the O/R layer (usually in an attempt to improve performance and avoid round-trips to the database), and this in turn presents some problems, particularly if the caching system is not a write-through cache: when does the actual "flush" to the database take place, and what does this say about transactional integrity if the application code believes the write to have occurred when in fact it hasn't? This problem in turn only compounds when the O/R system runs in multiple processes in front of the database engine, commonly found in clustered or farmed application server scenarios. Now the data identity is spread across n+1 locations, n being the number of application server nodes, and 1 being the database itself. Each node must somehow signal its intent to do an update to the other nodes in order to obtain some kind of concurrency construct to prevent simultaneous access (by another instance of the same session, or by an instance of a different session accessing the same data), which takes time, killing performance. Even in the case of a read-only cache, updates to the data store must somehow be signaled to the caches running in the application server nodes, requiring server-to-client communication originating from the database; support for this is not well-understood or documented in the current crop of modern relational databases.

The Data Retrieval Mechansim Concern

So once the entity is stored within the database, how exactly do we retrieve it? In all honesty, a purely object-oriented approach would make use of object approaches for retrieval, ideally using constructor-style syntax identifying the object(s) desired, but unfortunately constructor syntax isn't generic enough to allow for something that flexible; in particular, it lacks the ability to initialize a collection of objects, and queries frequently need to return a collection, rather than just a single entity. (Multiple trips to the database to fetch entities individually is generally considered too wasteful, in both latency and bandwidth, to consider credibly as an alternative--see the Load-Time Paradox, below, for more.) As a result, we typically end up with one of Query-By-Example (QBE), Query-By-API (QBA), or Query-By-Language (QBL) approaches.

A QBE approach states that you fill out an object template of the type of object you're looking for, with fields in the object set to a particular value to use as part of the query-filtration process. So, for example, if you're querying the Person object/table for people with the last name of Smith, you set up the query like so:

Person p = new Person(); // assumes all fields are set to null by default
p.LastName = "Smith";
ObjectCollection oc = QueryExecutor.execute(p);

As a result, usually the second step is to have the object system support a "Query-By-API" approach, in which queries are constructed by query objects, usually something of the form:

Query q = new Query();
q.From("PERSON").Where(
  new EqualsCriteria("PERSON.LAST_NAME", "Smith"));
ObjectCollection oc = QueryExecutor.execute(q);

Developers quickly note that the above approach is (generally) much more verbose than the traditional SQL approach, and certain styles of queries (particularly the more unconventional joins, such as outer joins) are much more difficult--if not impossible--to represent in the QBA approach.

On top of this, we have a more subtle problem, that of the reliance on developers' dicipline: both the table name ("PERSON") and the column name in the criteria ("PERSON.LAST_NAME") are standard strings, taken as-is and fed to the system at runtime with no sort of validity-checking until then. This presents a classic problem in programming, that of the "fat-finger" error, where a developer doesn't actually query the "PERSON" table, but the "PRESON" table instead. While a quick unit-test against a live database instance will reveal the error during unit-testing, this presumes two facts--that the developers are religious about adopting unit-testing, and that the unit-tests are run against database instances. While the former is slowly becoming more of a guarantee as more and more developers become "test-infected" (borrowing Gamma's and Beck's choice of terminology), the latter is still entirely open to discussion and interpretation, owing to the fact that setting-up and tearing-down the database instance appropriately for unit tests is still difficult to do in a database. (While there are a variety of ways to circumvent this problem, few of them seem to be in use.)

We're also faced with the basic problem that greater awareness of the logical--or physical--data representation is required on the part of the developer--instead of simply focusing on how the objects are related to one another (through simple associations such as arrays or collection instances), the developer must now have greater awareness of the form in which the objects are stored, leaving the system somewhat vulnerable to database schema changes. This is sometimes obviated by a hybrid approach between the two, whereby the system will take responsibility for interpreting the associations, leaving the developer to write something like this:

Query q = new Query();
Field lastNameFieldFromPerson = Person.class.getDeclaredField("lastName");
q.From(Person.class).Where(new EqualsCriteria(lastNameFieldFromPerson, "Smith"));
ObjectCollection oc = QueryExecutor.execute(q);

So, then, the next task is to create a "Query-By-Language" approach, in which a new language, similar to SQL but "better" somehow, is written to support the kind of complex and powerful queries normally supported by SQL; OQL and HQL are two examples of this. The problem here is that frequently these languages are a subset of SQL and thus don't offer the full power of SQL. More importantly, the O/R layer has now lost an important "selling point", that of the "objects and only objects" mantra that begat it in the first place; using a SQL-like language is almost just like using SQL itself, so how can it be more "objectish"? While developers may not need to be aware of the physical schema of the data model (the query language interpreter/executor can do the mapping discussed earlier), developers will need to be aware of how object associations and properties are represented within the language, and the subset of the object's capabilities within the query language--for example, is it possible to write something like this?

SELECT Person p1, Person p2 
FROM Person 
WHERE p1.getSpouse() == null 
  AND p2.getSpouse() == null 
  AND p1.isThisAnAcceptableSpouse(p2) 
  AND p2.isThisAnAcceptableSpouse(p1);

(Critics will argue that this is a workable problem, proposing two possible solutions. One is to encode the preference data in a separate table and make that part of the query; this will result in a hideously complicated query that will take several pages in length and likely require a SQL expert to untangle later when new preferential criteria want to be added. The other is to encode this "acceptability" implementation in a stored procedure within the database, which now removes code entirely from the object model and leaves us without an "object"-based solution whatsoever--acceptable, but only if you accept the premise that not all implementation can rest inside the object model itself, which rejects the "objects and nothing but objects" premise with which many O/R advocates open their arguments.)

The Partial-Object Problem and the Load-Time Paradox

It has long been known that network traversal, such as that done when making a traditional SQL request, takes a significant amount of time to process. (Rough benchmarks have placed this value at anywhere from three to five orders of magnitude, compared against a simple method call on either the Java or .NET platform⁵; roughly analogous, if it takes you twenty minutes to drive to work in the morning, and we call that the time required to execute a local method call, four orders of magnitude to that is roughly the time it takes to travel to Pluto, or just shy of fourteen years, one way.) This cost is clearly non-trivial, so as a result, developers look for ways to minimize this cost by optimizing the number of round trips and data retrieved.

In SQL, this optimization is achieved by carefully structuring the SQL request, making sure to retrieve only the columns and/or tables desired, rather than entire tables or sets of tables. For example, when constructing a traditional drill-down user interface, the developer presents a summary display of all the records from which the user can select one, and once selected, the developer then displays the complete set of data for that particular record. Given that we wish to do a drill-down of the Persons relational type described earlier, for example, the two queries to do so would be, in order (assuming the first one is selected):

SELECT id, first_name, last_name FROM person;

SELECT * FROM person WHERE id = 1;

This presents a problem for most, if not all, object/relational mapping layers: the goal of any O/R is to enable the developer to see "nothing but objects", and yet the O/R layer cannot tell, from one request to another, how the objects returned by the query will be used. For example, it is entirely feasible that most developers will want to write something along the lines of:

Person[] all = QueryManager.execute(...);
Person selected = DisplayPersonsForSelection(all);
DisplayPersonData(selected);

The problem here is that the data to be displayed in the first Display...() call is not the complete Person, but a subset of that data; here we face our first problem, in that an object-oriented system like C# or Java cannot return just "parts" of an object--an object is an object, and if the Person object consists of 12 fields, then all 12 fields will be present in every Person returned. This means that the system faces one of three uncomfortable choices: one, require that Person objects must be able to accomodate "nullable" fields, regardless of the domain restrictions against that; two, return the Person completely filled out with all the data comprising a Person object; or three, provide some kind of on-demand load that will obtain those fields if and when the developer accesses those fields, even indirectly, perhaps through a method call.

(Note that some object-based languages, such as ECMAScript, view objects differently than class-based languages, such as Java or C# or C++, and as a result, it is entirely possible to return objects which contain varying numbers of fields. That said, however, few languages possess such an approach, not even everybody's favorite dynamic-language poster child, Ruby, and until such languages become widespread, such discussion remains outside the realm of this essay.)

For most O/R layers, this means that objects and/or fields of objects must be retrieved in a lazy-loaded manner, obtaining the field data on demand, because retrieving all of the fields of all of the Person objects/relations would "clearly" be a huge waste of bandwidth for this particular scenario. Typically, the object's entire set of fields will be retrieved when any field not-yet-returned is accessed. (This approach is preferred to a field-by-field approach because there's less chance of the "N+1 query problem", in which retrieving all the data from an object requires 1 query to retrieve the primary key + N queries to retrieve each field from the table as necessary. This minimizes the bandwidth consumed to retrieve data--no unaccessed field will have its data retrieved--but clearly fails to minimize network round trips.)

Unfortunately, fields within the object are only part of the problem--the other problem we face is that objects are frequently associated with other objects, in various cardinalities (one-to-one, one-to-many, many-to-one, many-to-many), and an O/R mapping has to make some up-front decisions about when to retrieve these associated objects, and despite the best efforts of the O/R-M's developers, there will always be common use-cases where the decision made will be exactly the wrong thing to do. Most O/R-M's offer some kind of developer-driven decision-making support, usually some kind of configuration or mapping file, to identify exactly what kind of retrieval policy will be, but this setting is global to the class, and as such can't be changed on a situational basis.

Summary

Given, then, that objects-to-relational mapping is a necessity in a modern enterprise system, how can anyone proclaim it a quagmire from which there is no escape? Again, Vietnam serves as a useful analogy here--while the situation in South Indochina required a response from the Americans, there were a variety of responses available to the Kennedy and Johson Administrations, including the same kind of response that the recent fall of Suharto in Malaysia generated from the US, which is to say, none at all. (Remember, Eisenhower and Dulles didn't consider South Indochina to be a part of the Domino Theory in the first place; they were far more concerned about Japan and Europe.)

Several possible solutions present themselves to the O/R-M problem, some requiring some kind of "global" action by the community as a whole, some more approachable to development teams "in the trenches":

1. Abandonment. Developers simply give up on objects entirely, and return to a programming model that doesn't create the object/relational impedance mismatch. While distasteful, in certain scenarios an object-oriented approach creates more overhead than it saves, and the ROI simply isn't there to justify the cost of creating a rich domain model. ([Fowler] talks about this to some depth.) This eliminates the problem quite neatly, because if there are no objects, there is no impedance mismatch.
2. Wholehearted acceptance. Developers simply give up on relational storage entirely, and use a storage model that fits the way their languages of choice look at the world. Object-storage systems, such as the db4o project, solve the problem neatly by storing objects directly to disk, eliminating many (but not all) of the aforementioned issues; there is no "second schema", for example, because the only schema used is that of the object definitions themselves. While many DBAs will faint dead away at the thought, in an increasingly service-oriented world, which eschews the idea of direct data access but instead requires all access go through the service gateway thus encapsulating the storage mechanism away from prying eyes, it becomes entirely feasible to imagine developers storing data in a form that's much easier for them to use, rather than DBAs.
3. Manual mapping. Developers simply accept that it's not such a hard problem to solve manually after all, and write straight relational-access code to return relations to the language, access the tuples, and populate objects as necessary. In many cases, this code might even be automatically generated by a tool examining database metadata, eliminating some of the principal criticism of this approach (that being, "It's too much code to write and maintain").
4. Acceptance of O/R-M limitations. Developers simply accept that there is no way to efficiently and easily close the loop on the O/R mismatch, and use an O/R-M to solve 80% (or 50% or 95%, or whatever percentage seems appropriate) of the problem and make use of SQL and relational-based access (such as "raw" JDBC or ADO.NET) to carry them past those areas where an O/R-M would create problems. Doing so carries its own fair share of risks, however, as developers using an O/R-M must be aware of any caching the O/R-M solution does within it, because the "raw" relational access will clearly not be able to take advantage of that caching layer.
5. Integration of relational concepts into the languages. Developers simply accept that this is a problem that should be solved by the language, not by a library or framework. For the last decade or more, the emphasis on solutions to the O/R problem have focused on trying to bring objects closer to the database, so that developers can focus exclusively on programming in a single paradigm (that paradigm being, of course, objects). Over the last several years, however, interest in "scripting" languages with far stronger set and list support, like Ruby, has sparked the idea that perhaps another solution is appropriate: bring relational concepts (which, at heart, are set-based) into mainstream programming languages, making it easier to bridge the gap between "sets" and "objects". Work in this space has thus far been limited, constrained mostly to research projects and/or "fringe" languages, but several interesting efforts are gaining visibility within the community, such as functional/object hybrid languages like Scala or F#, as well as direct integration into traditional O-O languages, such as the LINQ project from Microsoft for C# and Visual Basic. One such effort that failed, unfortunately, was the SQL/J strategy; even there, the approach was limited, not seeking to incorporate sets into Java, but simply allow for embedded SQL calls to be preprocessed and translated into JDBC code by a translator.
6. Integration of relational concepts into frameworks. Developers simply accept that this problem is solvable, but only with a change of perspective. Instead of relying on language or library designers to solve this problem, developers take a different view of "objects" that is more relational in nature, building domain frameworks that are more directly built around relational constructs. For example, instead of creating a Person class that holds its instance data directly in fields inside the object, developers create a Person class that holds its instance data in a RowSet (Java) or DataSet (C#) instance, which can be assembled with other RowSets/DataSets into an easy-to-ship block of data for update against the database, or unpacked from the database into the individual objects.

Just as it's conceivable that the US could have achieved some measure of "success" in Vietnam had it kept to a clear strategy and understood a more clear relationship between commitment and results (ROI, if you will), it's conceivable that the object/relational problem can be "won" through careful and judicious application of a strategy that is celarly aware of its own limitations. Developers must be willing to take the "wins" where they can get them, and not fall into the trap of the Slippery Slope by looking to create solutions that increasingly cost more and yield less. Unfortunately, as the history of the Vietnam War shows, even an awareness of the dangers of the Slippery Slope is often not enough to avoid getting bogged down in a quagmire. Worse, it is a quagmire that is simply too attractive to pass up, a Siren song that continues to draw development teams from all sizes of corporations (including those at Microsoft, IBM, Oracle, and Sun, to name a few) against the rocks, with spectacular results. Lash yourself to the mast if you wish to hear the song, but let the sailors row.

Endnotes

¹ Later analysis by the principals involved--including then-Secretary of Defense Robert McNamara--concluded that half of the attack never even took place.

² It is perhaps the greatest irony of the war, that the man Fate selected to lead during America's largest foreign entanglement was a leader whose principal focus was entirely aimed within his own shores. Had circumstances not conspired otherwise, the hippies chanting "Hey, hey LBJ, how many boys did you kill today" outside the Oval Office could very well have been Johnson's staunchest supporters.

³ Ironically, encapsulation, for purposes of maintenance simplicity, turns out to be a major motivation for almost all of the major innovations in Linguistic Computer Science--procedural, functional, object, aspect, even relational technologies ([Date02]) and other languages all cite "encapsulation" as major driving factors.

⁴ We could, perhaps, consider stored procedure languages like T-SQL or PL/SQL to be "relational" programming languages, but even then, it's extremely difficult to build a UI in PL/SQL.

⁵ In this case, I was measuring Java RMI method calls against local method calls. Similar results are pretty easily obtainable for SQL-based data access by measuring out-of-process calls against in-process calls using a database product that supports both, such as Cloudscape/Derby or HSQL (Hypersonic SQL).

References

[Fussell]: Foundations of Object Relational Mapping, by Mark L. Fussell, v0.2 (mlf-970703)

[Fowler] Patterns of Enterprise Application Architecture, by Martin Fowler

[Date04]: Introduction to Database Systems, 8th Edition, by Chris Date.

[Neward04]: Effective Enterprise Java

Cathi Gero's and my session from TechEd, "Pragmatic Architecture", is now available as a webcast for your viewing and listening pleasure. We had a few issues with the audio, which got us started late, but overall the general feedback was positive. Enjoy...

You are a dead man if I ever find you.

That said, it was rather amusing to see which of the speakers were big enough geeks to bring their laptops outside when we were all evacuated....

Larry O'Brien asks

Are you confident that continuations can be even semi-efficiently implemented on the CLR? I'm not.

1. If Microsoft thinks Ruby is important, they're ignoring the threat to them posed by X (where, I suspect, X = LISP), or
2. If Microsoft thinks Ruby is competition, they will not implement it and therefore be doomed

Not long ago, Microsoft posted a job opening for a developer "first task will be to drive the exploration of other dynamic languages such as Ruby and JavaScript on the CLR", so my feeling is that if Microsoft could get a Ruby on the CLR, they'd be thrilled.

Larry goes on to say:

Ruby is not easy to implement on the CLR, at least in part because a complete Ruby implementation requires continuations, which are not modeled within the CLR. This isn't just laziness on the part of langauge implementors. The CLR presents a machine architecture different than the wide-open architecture in which most compiler experience has been gathered. The CLR architecture is safer, but more restrictive, when it comes to manipulating the stack, which is central to continuations.

Continuations are not impossible to support, however they are currently more or less impossible to support given the current lack of access to the underlying stack frames in the managed environment--you'd need some support from the runtimes (either the JVM or the CLR) to make it work. Such runtime support would not be too difficult to add, however, as both environments already have rich and powerful stack-walking mechanisms (because both environments use the thread stack as bookkeeping tools, among other things, and need to be able to crawl through those stack markers for a variety of reasons, such as security checks), and it would not be hard to create a runtime-level mechanism that allowed code to "take a snapshot" of the stack--and its related object graph--from a certain point to a certain point, and save off that state to some arbitrary location. In many respects, it would be similar to serializing an object, I believe. In fact, we could imagine something along the lines of:

// All this is totally C#-like pseudocode. Imagine 
// something similar for Java if you like. 
// 
public int ContinuationedMethod() { 
  SnapshotMarker sm = new SnapshotMarker(); 
    // in other words, the StackSnapshot will only crawl 
    // back to the SnapshotMarker referenced when we 
    // take the thread's snapshot; this way we don't crawl 
    // all the way back to the Thread's starting point (unless 
    // you really wanted to). 
  int x = 1; 
  for (int i=0; i<10; i++) { 
    x = x * i; 
    if (i == 5) { 
      StackSnapshot ss = Thread.Current.TakeSnapshot(sm); 
        // At this point, the managed stack is walked, heap-referenced 
        // objects are captured, the instruction label that we're on is 
        // saved, and a StackSnapshot is allocated and returned. 
        // However, when ss is later rehydrated--using, say, ss.Resume(), 
        // we need some way of knowing that. So, following the lead of the 
        // old Unix fork() call, I presume that a "null" return value from 
        // TakeSnapshot is our way of knowing that we are resuming. 
        // 
      if (ss == null) 
        continue; 
      else 
      {
        // store ss someplace for later retrieval and return, either by 
        // throwing an exception if you like or just plain-old-"return 0"
        // or something
        //
      }
    } 
  } 
  return x; 
}

In the end, continuations are not, I believe, nearly as hard to implement--on either the JVM or the CLR--as some might suggest. Had I the money, I would go off and build the necessary Ruby-esque features we'd want into the CLR (through Rotor) or the JVM (through... uh... the JCL source, I guess, though the licensing there bothers me) for use. Anybody got some cash laying around to cover my mortgage while I do this?

The guys over at Software Engineering Radio asked me to do a podcast a few months back, and it's now live on their website. They were particularly interested in language and new language development, so we spent a fair amount of time talking about Scala, F#, LINQ, and other interesting language developments in the world of the JVM and CLR. Have a listen, if you like...

A while back, I blogged how MonadWindows PowerShell can be used to do a lot of the things the Ruby advocates are saying is one of Ruby's biggest strengths, that of "scripting" and driving things from the REPL environment. Glenn Vanderburg jumped all over me, believing I was suggesting that this was some kind of contest by which Ruby was supposed to come out in the Negative Points Zone. Had that been my intent, I would heartily agree with his critique; unfortunately, that wasn't the point.

For a while now, people have been holding up Ruby as this "incredibly productive tool", with the implication that such productivity cannot be achieved on the platforms that are currently the standards among the industry--that is, the CLR and Java virtual machine. Dynamically-, weak- or non-typed languages, for example, are all the rage now because they mean we don't need to try and "fool" the compiler on a regular basis with typecasts, we can have things like closures and continuations, and so on. My point simply was to point out that such argument is FUD--we can have closures, continuations, and so on, in platforms like the CLR and JVM--it's just that the major languages of the day don't provide those features (yet).

The Ruby advocate may snicker at my splitting of hairs here--"OK, so your platform may support it, but your languages don't. I'd rather use a language and platform that does support these features, instead of waiting for somebody else to come along and give them to me." It's a fair question--why is this an important distinction? Because asking existing infrastructure and applications to switch to a new platform is almost impossible. Asking them to integrate with a new platform is painful. Asking developers--particularly those on the team who don't get the beauty of dynamic/weak/non-typed languages--to switch to an entirely new way of thinking is going to mean you're going to spend at least five years learning the "new way".

If we can get those features we want from languages like Ruby onto a platform that we've already standardized on gives us a best-of-both-worlds result. Those developers who are comfortable with statically-typed objects can stay with statically-typed objects. Those who want the more dynamic features of "scripting" languages can do so. We can then blend the two together, to form an interesting and seamless whole: "Give me my Rails, but let me call into a J2EE Connector implementation to talk to the mainframe while we're at it." Getting Ruby to run on the JVM or CLR would be a Very Good Thing. It would answer one of the principal criticisms of Rails, for example, that of the idea that it doesn't do well when dealing with Large Enterprise Things--if Rails could create a javax.transaction.XATransaction before it kicks into ActiveRecord code, for example, suddenly we have a two-phase commit possibility that includes not just databases but mainframes and messaging systems. And yes, people DO need those kinds of Large Enterprise Things in the real world sometimes. Would it not be a win for Ruby to be able to hook into those without having to write all that code themselves?

As a particular footnote to the discussion, Lee Holmes (whose blog entry I quoted to start all this) points out that there is a more concise version of the MonadWindows PowerShell script that compares more favorably (in terms of the "lines of code" metric, which, as Glenn notes, I don't really put much stock into) to the Ruby version. But that's not the point--the point is, there *is* a way to do Ruby-esque things in the Windows world, even if you choose not to learn the Ruby syntax. If that makes your life easier, hoo-hah, Sargeant! If you instead want to use JScript.NET and compile into IL, hoo-hah, Sargeant! If you choose to do Ruby, hoo-hah, Sargeant! Whatever makes life easier for you to Get The Work Done.

But don't underestimate the costs of integration, and certainly don't sacrifice integration on the altar of productivity--the time that you saved up front writing the thing will be more than spent when you try to make integration with the rest of your company's IT systems (or your new partner's IT systems, or your new consortium's standards, or....) work. Integration has been, is now, and will remain the Number One challenge to companies today, and that's not going to change in the future, Ruby or no Ruby.

Best link to it is from their TechTalk page; the link to the actual stream itself redirects several times....

Some of you've been asking, where the heck have I been lately? Short story--I've been insanely busy. Long story--I've been insanely busy with a bunch of conferences and on-the-road consulting, not to mention the various writing projects I have afloat. Fear not, the blogging will resume shortly....

My son misses me terribly... or, apparently, at least he misses certain parts of me more than others, according to Charlotte's blog... (Warning, only for people who want personal life details. )

The Javapolis folks were kind (deluded? crazy? you pick the word) enough to put the interview of me that Dion did at the show online. Have a listen....

I happened across this blog entry while doing some research on Monad, Microsoft's new command shell (meaning, think "bash" or "csh", not "Explorer"), and found it so similar in many ways to what guys in the Ruby space have been hyping for a year or two now, that I just had to pass it on. Reproducing directly from the site:

One of the scripts I like the most in my toolbox is the one that gives me answers to questions from the command line.

For the past 2 years or so, Encarta has offered an extremely useful Instant Answers feature. Its since been integrated into MSN Search, as well as a wildly popular Chat Bot. MoW showed how to use that feature through a Monad IM interface (via the Conversagent bot,) but we can do a great job with good ol screen scraping.

[C:\temp]
MSH:70 > get-answer "What is the population of China?"

Answer: China: Population, total: 1,313,973,700
2006 estimate
United States Census International Programs Center


Answer: China : Population:
More than 20 percent of the worldâ?Ts population lives in China. Of the co
untryâ?Ts inhabitants, 92 percent are ethnic Han Chinese. The Han are desc
endants...

[C:\temp]
MSH:71 > get-answer "5^(e^(x^2))=50"

Answer: 5^(e ^( x^2))=50 = x=0.942428  x=-0.942428

[C:\temp]
MSH:72 > get-answer "define: canadian bacon"

Definition: Canadian bacon lean bacon

[C:\temp]
MSH:73 > get-answer "How many calories in an Apple?"

Answer: Apples: calories:
1.0 cup, quartered or chopped has 65 calories 1.0 NLEA serving has 80 calo
ries 1.0 small (2-1/2" dia) (approx 4 per lb) has 55 calories 1.0 medium (
2-3/4" dia) (approx 3 per lb) has 72 calories 1.0 large (3-1/4" dia) (appr
ox 2 per lb) has 110 calories 1.0 cup slices has 57 calories
USDA

[C:\temp]
MSH:74 > get-answer "How many inches in a light year?"

Answer: 1 lightyear = 372,461,748,226,857,000 inches

Here is the script, should you require your own command-line oracle:

## Get-Answer.msh 
## Use Encarta's Instant Answers to answer your question. 

param([string] $question = $(throw "Please ask a question.")) 

function Main 
{ 
   # Load the System.Web.HttpUtility DLL, to let us URLEncode 
   [void] [System.Reflection.Assembly]::LoadWithPartialName("System.Web") 

   ## Get the web page into a single string 
   $encoded = [System.Web.HttpUtility]::UrlEncode($question) 
   $text = get-webpage "http://search.msn.com/encarta/results.aspx?q=$encoded" 

   ## Get the answer with annotations 
   $startIndex = $text.IndexOf('<div id="results">') 
   $endIndex = $text.IndexOf('</div></div><h2>Results</h2>') 

   ## If we found a result, then filter the result 
   if(($startIndex -ge 0) -and ($endIndex -ge 0)) 
   { 
      $partialText = $text.Substring($startIndex, $endIndex - $startIndex) 
    
      ## Very fragile, voodoo screen scraping here 
      $regex = "<\s*a\s*[^>]*?href\s*=\s*[`"']*[^`"'>]+[^>]*>.*?</a>" 
      $partialText = [Regex]::Replace("$partialText", $regex, "") 
      $partialText = $partialText -replace "</div>", "`n" 
      $partialText = $partialText -replace "</span>", "`n" 
      $partialText = clean-html $partialText 
      $partialText = $partialText -replace "`n`n", "`n" 
     
      "" 
      $partialText.TrimEnd() 
   } 
   else 
   { 
      "" 
      "No answer found." 
   } 
} 

## Get a web page
function Get-WebPage ($url=$(throw "need to specify the URL to fetch")) 
{ 
    # canonicalize the url 
    if ($url -notmatch "^[a-z]+://") { $url = "http://$url" } 
     
    $wc = new-object System.Net.WebClient  
    $wc.Headers.Add("user-agent", $userAgent) 
    $wc.DownloadString($url) 
} 

## Clean HTML from a text chunk 
function Clean-Html ($htmlInput) 
{ 
    [Regex]::Replace($htmlInput, "<[^>]*>", "") 
} 

. Main

Added to my list of developer interview questions: "What is Monad, and why do I care?"

Recently, while engaging in my other passion (international relations), I was reading the latest issue of Foreign Affairs, and ran across an interesting essay regarding the increasing outsourcing--or, the term they introduce which I prefer in this case, "offshoring"--of technical work, and I found some interesting analysis there that I think solidifies why I think programmers shouldn't fear offshoring, but instead embrace it and ride the wave to a better life for both us and consumers. Permit me to explain.

The essay, entitled "Offshoring: The Next Industrial Revolution?" (by Alan S. Blinder), opens with an interesting point, made subtly, that offshoring (or "offshore outsourcing"), is really a natural economic consequence:

In February 2004, when N. Gregory Mankiw, a Harvard professor then serving as chairman of the White House Council of Economic Advisers, caused a national uproar with a "textbook" statement about trade, economists rushed to his defense. Mankiw was commenting on the phenomenon that has been clumsily dubbed "offshoring" (or "offshore outsourcing")--the migration of jobs, but not the people who perform them, from rich countries to poor ones. Offshoring, Mankiw said, is only "the latest manifestation of the gains from trade that economists have talked about at least since Adam Smith. ... More things are tradable than were tradable in the past, and that's a good thing." Although Democratic and Republican politicians alike excoriated Mankiw for his callous attitude toward American jobs, economists lined up to support his claim that offshoring is simply international business as usual.

Their economics were basically sound: the well-known principle of comparative advantage implies that trade in new kinds of products will bring overall improvements in productivity and well-being. But Mankiw and his defenders underestimated both the importance of offshoring and its disruptive effect on wealthy countries. Sometimes a quantitative change is so large that it brings qualitative changes, as offshoring likely will. We have so far barely seen the tip of the offshoring iceberg, the eventual dimensions of which may be staggering.

To be sure, the furor over Mankiw's remark was grotesquely out of proportion to the current importance of offshoring, which is still largely a prospective phenomenon. Although there are no reliable national data, fragmentary studies indicate that well under a million service-sector jobs have been lost to offshoring to date. (A million seems impressive, but in the gigantic and rapidly churning U.S. labor market, a million jobs is less than two weeks' worth of normal gross job losses.)¹ However, constant improvements in technology and global communications will bring much more offshoring of "impersonal services"--that is, services that can be delivered electronically over long distances, with little or no degradation in quality.

That said, we should not view the coming wave of offshoring as an impending catastrophe. Nor should we try to stop it. The normal gains from trade mean that the world as a whole cannot lose from increases in productivity, and the United States and other industrial countries have not only weathered but also benefited from comparable changes in the past. But in order to do so again, the governments and societies of the developed world must face up to the massive, complex, and multifaceted challenges that offshoring will bring. National data systems, trade policies, educational systems, social welfare programs, and politics must all adapt to new realities. Unfortunately, none of this is happening now.

Why Offshoring

Blinder first describes the basics of "comparative advantage" and why it's important in this context:

Countries trade with one another for the same reasons that individuals, businesses and regions do: to exploit their comparative advantages. Some advantages are "natural": Texas and Saudi Arabia sit atop massive deposits of oil that are entirely lacking in New York and Japan, and nature has conspired to make Hawaii a more attractive tourist destination than Greenland. Ther eis not much anyone can do about such natural advantages.

But in modern economics, nature's whimsy is far less important than it was in the past. Today, much comparative advantage derives from human effort rather than natural conditions. The concentration of computer companies around Silicon Valley, for example, has nothing to do with bountiful natural deposits of silicon; it has to do with Xerox's fabled Palo Alto Research Center, the proximity of Stanford University, and the arrival of two young men named Hewlett and Packard. Silicon Valley could have sprouted up anywhere.

One important aspect of this modern reality is that patterns of man-made comparative advantage can and do change over time. The economist Jagdish Bhagwait has labeled this phenomenon "kaleidoscopic comparative advantage", and it is critical to understanding offshoring. Once upon a time, the United Kingdom had a comparative advantage in textile manufacturing. Then that advantage shifted to New England, and so jobs were moved from the United Kingdom to the United States.² Then the comparative advantage in textile manufacturing shifted once again--this time to the Carolinas--and jobs migrated south within the United States.³ Now the comparative advantage in textile manufacturing resides in China and other low-wage countries, and what many are wont to call "American jobs" have been moved there as a result.

Of course, not everything can be traded across long distances. At any point in time, the available technology--especially in transportation and communications⁴--largely determines what can be traded internationally and what cannot. Economic theorists accordingly divide the world's goods and services into two bins: tradable and non-tradable. Traditionally, any item that could be put in a box and shipped (roughly, manufactured goods) was considered tradable, and anything that could not be put into a box (such as services) or was too heavy to ship (such as houses) was thought of as nontradable. But because technology is always improving and transportation is becoming cheaper and easier, the boundary between what is tradable and what is not is constantly shifting. And unlike comparative advantage, this change is not kaleidoscopic; it moves in only one direction, with more and more items becoming tradable.

The old assumption that if you cannot put it in a box, you cannot trade it is thus hopelessly obsolete. Because packets of digitized information play the role that boxes used to play, many more services are now tradable and many more will surely become so. In the future, and to a great extent already, the key distinction will no longer be between things that can be put in a box and things that cannot. Rather, it will be between services that can be delivered electronically and those that cannot.

Then came the second Industrial Revolution, and jobs shifted once again--this time away from manufacturing and toward services. The shift to services is still viewed with alarm in the United States and other rich countries, where people bemoan rather than welcome the resulting loss of manufacturing jobs⁵. But in reality, new service-sector jobs have been created far more rapidly than old manufacturing jobs have disappeared. In 1960, about 35 percent of nonagricultural workers in the United States produced goods and 65 percent produced services. By 2004, only about one-sixth of the United States' nonagricultural jobs were in goods-producing industries, while five-sixths produced services. This trend is worldwide and continuing.

We are now i nthe arly stages of a third Industrial Revolution--the information age. The cheap and easy flow of information around the globe has vastly expanded the scope of tradable services, and there is much more to come. Industrial revolutions are big deals. And just like the previous two, the third Industrial Revolution will require vast and usettling adjustments in the way Americans and residents of other developed countries work, live, and educate their children.

So what does this mean?

But now we start to come to the interesting part of the article.

But a bit of historical perspective should help temper fears of offshoring. The first Industrial Revolution did not spell the end of agriculture, or even the end of food production, in the United States. It jus tmean that a much smaller percentage of Americans had to work on farms to feed the population. (By charming historical coincidence, the actual number of Americans working on farms today--around 2 million--is about what it was in 1810.) The main reason for this shift was not foreign trade, but soaring farm productivity. And most important, the massive movement of labor off the farms did not result in mass unemployment. Rather, it led to a large-scale reallocation of labor to factories.

Therein lies the flaw in the argument: the amount of productivity required to achieve the desired ends is constant in the agriculture industry, yet a constantly-changing dynamic value in software. This is also known as what I will posit as the Groves-Gates Maxim: "What Andy Groves giveth, Bill Gates taketh away."

The Groves-Gates Maxim

The argument here is simple: the process of growing food is a pretty constant one: put seed in ground, wait until it comes up, then harvest the results and wait until next year to start again. Although we might have numerous tools that can help make it easier to put seeds into the ground, or harvesting the results, or even helping to increase the yield of the crop when it comes up, the basic amount of productivity required is pretty much constant. (My cousin, the FFA Farmer of the Year from some years back and a seed hybrid researcher in Iowa might disagree with me, mind you.) Compare this with the software industry: the basic differences between what's an acceptable application to our users today, compared to even ten years ago, is an order of magnitude different. Gains in productivity have not yielded the ability to build applications faster and faster, but instead have created a situation where users and managers ask more of us with each successive application.

The Groves-Gates Maxim is an example of that: every time Intel (where Andy Groves is CEO) releases new hardware that accelerates the power and potential of what the "average" computer (meaning, priced at somewhere between $1500-$2000) is capable of, it seems that Microsoft (Mr. Gates' little firm) releases a new version of Windows that sucks up that power by providing a spiffier user interface and "eye-candy" features, be they useful/important or not. In other words, the more the hardware creates possibilities, the more software is created to exploit and explore those possibilities. The additional productivity is spent not in reducing the time required to produce the thing desired (food in the case of agriculture, an operating system or other non-trivial program in the case of software), but in the expansion of the functionality of the product.

This basic fact, the Groves-Gates Maxim, is what saves us from the bloody axe of forced migration. Because what's expected of software is constantly on the same meteoric rise as what productivity gains provide us, the need for programmer time remains pretty close to constant. Now, once the desire for exponentially complicated features starts to level off, the exponentially increasing gains in productivity will have the same effect as they did in the agricultural industry, and we will start seeing a migration of programmers into other, "personal service" industries (which are hard to offshore, as opposed to "impersonal service" industries which can be easily shipped overseas).

Implications

What does this mean for programmers? For starters, as Dave Thomas has already frequently pointed out on NFJS panels, programmers need to start finding ways to make their service a "personal service" position rather than an "impersonal service" one. Blinder points out that the services industry is facing a split down the middle along this distinction, and it's not necessarily a high-paying vs low-paying divide:

Many people blithely assume that the critical labor-market distinction is, and will remain, between highly educated (or highly-skilled) people and less-educated (or less-skilled) people--doctors versus call-center operators, for example. The supposed remedy for the rich countries, accordingly, is more education and a general "upskilling" of the work force. But this view may be mistaken. Other things being equal, education and skills are, of course, good things; education yields higher returns in advanced societies, and more schooling probably makes workers more flexible and more adaptable to change. But the problem with relying on education as the remedy for potential job losses is that "other things" are not remotely close to equal. The critical divide in the future may instead be between those types are work that are easily deliverable through a wire (or via wireless connections) with little or no diminution in quality and those that are not. And this unconventional divide does not correspond well to traditional distinctions between jobs that require high levels of education and jobs that do not.

A few disparate examples will illustrate just how complex--or, rather, how untraditional--the new divide is. It is unlikely that the services of either taxi drivers or airline pilots will ever be delivered electronically over long distances. The first is a "bad job" with negligible educational requirements; the second is quite the reverse. On the other hand, typing services (a low-skill job) and security analysis (a high-skill job) are already being delivered electronically from India--albeit on a small scale so far. Most physicians need not fear that their jobs will be moved offshore, but radiologists are beginning to see this happening already. Police officers will not be replaced by electronic monitoring, but some security guards will be. Janitors and crane operators are probably immune to foreign competition; accountants and computer programmers are not. In short, the dividing line between the jobs that produce services that are suitable for electronic delivery (and are thus threatened by offshoring) and those that do not does not correspond to traditional distinctions between high-end and low-end work.

There is currently not even a vocabulary, much less any systematic data, to help society come to grips with the coming labor-market reality. So here is some suggested nomenclature. Service that cannot be delivered electronically, or that are notably inferior when so delivered, have one essential characteristic: personal, face-to-face contact is either imperative or highly desirable. Think of hte waiter who serves you dinner, the doctor you gives you your annual physical, or the cop on the beat. Now think of any of those tasks being performed by robots controlled from India--not quite the same. But such face-to-face human contact is not necessary in the relationship you have with the telephone operator who arranges your conference call or the clerk who takes your airline reservation over the phone. He or she may be in India already.

The first group of tasks can be called personally-delivered services, or simply personal services, and the second group of impersonally delivered services, or impersonal services. In the brave new world of globalized electronic commerce, impersonal services have more in common with manufactured goods that can be put in boxes than they do with personal services. Thus, many impersonal services are destined to become tradable and therefore vulnerable to offshoring. By contrast, most personal services have attributes that cannot be transmitted through a wire. Some require face-to-face contact (child care), some are inherently "high-risk" (nursing), some involve high levels of personal trust (psychotherapy), and some depend on location-specific attributes (lobbying).

• Face-to-face contact. Agile methodologies demand that customers be easily accessible in order to answer questions regarding implementation decisions or to resolve lack of understanding of the requirements. Instead of demanding customers be present at your site, you may find yourself in a better position if you put yourself close to your customers.
• "High-risk". This is a bit harder to do with software projects--either the project is inherently high-risk in its makeup (perhaps this is a mission-critical app that the company depends on, such as the e-commerce portal for an online e-tailer), or it's not. There's not much you can do to change this, unless you are politically savvy enough to "sell" your project to a group that would make it mission-critical.
• High levels of personal trust. This is actually easier than you might think--trust in this case refers not to the privileged nature of therapist-patient communication, but in the credibility the organization has in you to carry out the work required. One way to build this trust is to understand the business domain of the client, rather than remaining aloof and "staying focused on the technology". This trust-based approach is already present in a variety of forms outside our industry--regardless of the statistical ratings that might be available, most people find that they have a favorite auto repair mechanic or shop not for any quantitatively-measurable reason, but beceause the mechanic "understands" them somehow. The best customer-service shops understand this, and have done so for years. The restaurant that recognizes me as a regular after just a few visits and has my Diet Coke ready for me at my favorite table is far likelier to get my business on a regular basis than the one that never does. Learn your customers, learn their concerns, learn their business model and business plan, and get yourself into the habit of trying to predict what they might need next--not so you can build it already, but so that you can demonstrate to them that you understand them, and by extension, their needs.
• Location-specific attributes. Sometimes, the software being built is localized to a particular geographic area, and simply being in that same area can yield significant benefits, particularly when heroic efforts are called for. (It's very hard to flip the reset switch on a server in Indiana from a console in India, for example.)

But even if this analysis doesn't make you feel a little more comfortable, consider this: there are 1 billion people in China alone, and close to the same in India. Instead of seeing them as potential competition, imagine what happens when the wages from the offshored jobs start to create a demand for goods and services in those countries--if you think the software market in the U.S. was hot a decade ago, where only a half-billion (across both the U.S. and Europe) people were demanding software, now think about it when four times that many start looking for it.

Footnotes

¹ Which in of itself is an interesting statistic--it implies that offshoring is far less prevalent than some of people worried about it believe it to be, including me.

² Interesting bit of trivia--part of the reason that advantage shifted was because the US stole (yes, stole, as in industrial espionage, one of the first recorded cases of modern industrial espionage) the plans for modern textile machinery from the UK. Remember that, next time you get upset at China's rather loose grip of intellectual property law....

³ Which, by the way, was a large part of the reason we fought the Civil War (the "War Between the States" to some, or the "War of Northern Aggression" to others)--the Carolinas depended on slave labor to pick their cotton cheaply, and couldn't acquire Northern-made machinery cheaply to replace the slaves. Hence, for that (and a lot of other reasons), war followed.

⁴ An interesting argument--is there any real difference between transportation and communications? One ships "stuff", the other "data", beyond that, is there any difference?

⁵ And, I'd like to point out, the shrinking environmental damage that can arise from a manufacturing-based economy. Services rarely generate pollution, which is part of the clash between the industrialized "Western" nations and the developing "Southern" ones over environmental issues.

Resources

"Offshoring: The Next Industrial Revolutoin?", by Alan S. Blinder, Foreign Affairs (March/April 2006), pp 113 - 128.

Scott Hanselman has diabetes. So does a good friend of mine, who discovered it during our third year of college. Take a moment and spare US$20 to support Scott in his fight against it. Do it because you probably know somebody who has it, or will before long. If nothing else, do it because it's a cheap way to support somebody who's indirectly responsible for this blog (Scott maintains dasBlog, which is the blogging engine I use now).

Out.

CLASSPATH now supports wildcards to pick up multiple .jar files. Finally. But given that the AppClassLoader is a derivative of the standard UrlClassLoader, I still don't see why I can't put full URLs on the CLASSPATH and expect them to be resolved correctly....

Oh, and while we're at it, you shouldn't be using CLASSPATH-the-environment variable anymore, anyway. There's far too many ways to manage .jar file resolution to be falling back to that old hack. Prefer instead to put your .jar file dependencies inside the manifest of your application's .jar file, or at the very least, specify the classpath to the java launcher when you kick it off. If you're still doing "set CLASSPATH=..." at the command-line, you're about ten years behind the times.

Mustang, the latest JDK (to be called Java6 on its release), fixes a minor but very annoying nit that's bugged Java developers for years: "if a JDBC driver is packaged as a service, you can simply (leave out the call to Class.forName() to bootstrap the driver class into the JVM). ... The DriverManager code will look for implementations of java.sql.Driver in classpath and do Class.forName() implicitly." It's never been a huge deal in Java, to have to explicitly bootstrap the JDBC driver into the JVM before being able to obtain a Connection from it, but it's always been annoying, and inexplicable, given the Service Provider mechanism that's been there for a couple of releases now.

Next up: do the same for JNDI and XML parsers (and do away with the extensions directory while we're at it!), and let's start making all of these factories a bit more practical to use on a larger basis. It would be nice if this mechanism had percolated through other tools and areas, such as servlets, but it's probably too late to correct for that now.

From the "I'm not sure why you'd want to do this, but..." Department: How to install Windows XP onto your brand new Intel Mac Pro. (Given that so much of the Windows O/S relies on the right-mouse-button, and given that it'd be so much cheaper to buy a Dell or even a Thinkpad of equivalent power... why would you bother to buy a Mac Pro, then turn around and install--or even dual-boot install--XP on it? Maybe I'm just not "getting it" or something....)

Needless to say, despite the hefty geekiness factor inherent in the idea of doing .NET demos with a Mac Pro, I'll probably not be buying one any time soon... Instead, somebody point me to Tiger running in a VMWare image and then stand back.

Bruce Tate is at it again, this time writing for the Pragmatic Press, called "Java to Ruby", on... well, the title kinda says it all, on migrating from Java to Ruby. It's not just a "blind adoption" book, meaning it just blindly advocates the transition, but instead Bruce discusses the risks involved with making the switch, and how to justify it to upper management. Don't get me wrong, he's operating from the basic conclusion that you want to make the switch, so if you're not yet convinced that Ruby or Ruby-on-Rails is the way to go, you're not necessarily going to be any more convinced by this book. But if you are already convinced (and in Bruce's defense, there's lots of good reasons to be), this looks to be the book to help you convince others, most notably your management.

While training last week, the group I was training asked for some help in interviewing candidates for some openings. I came up with the following, and thought I'd post it in the interests of giving teams looking to hire some new folks. This was created specifically to find candidates with 2-3 years' experience with some familiarity with web applications.

• What was the most interesting computer/technical book you read last year? Why?
• What was the last new programming language you learned?
• How do you convince a customer that what they want is wrong?
• Describe the last elegant hack you wrote. Why, where, how, when, ...?
• Write a servlet that prints all parameters and HTTP headers to the browser. Build an Ant script that compiles, and creates the appropriate .war file for deployment into Tomcat.
• Name all of the verbs in the HTTP/1.1 specification.
• Given a tag library, "utilitytags.jar" which is described by the tag library descriptor file "utilitytags.tld", please write a JSP that uses the "foobar" tag from that library.
• What keyword in Java do we use to grant package-level access to class members?
• When do you think you should use an abstract base class as opposed to using an interface?
• How do I protect code in a multithreaded environment from being executed by more than one thread at a time?
• Please fill in the necessary code to return the contents of the "last name" column using the following SQL statement; make all necessary changes so that this code will compile:

  public void printUsers(Connection conn)
  {
    String sql = "SELECT last_name FROM users WHERE first_name = 'Sean'";
    ...
  }
  

• What is a design pattern? Please describe three patterns that you have used and why you used them and what the consequences of using them were, both good and bad.

Update: A couple of commenters have pointed out that a few of the questions are answered by simply looking up stuff (in the HTTP specification, for example). Two answers come to mind why I want people to know this without having to look it up--one, sometimes I want to pitch a slowball just to see how they'll react and answer, and two, if they can answer the question without having to look it up, it means they know the spec, which is a far, far different thing than being able to look something up.

Much of the power, it seems, of languages like Ruby or Nemerle or LISP derives from the ability to create chunks of code that operate in turn on the code itself; a long-standing meme of the LISP world is that "code is data", meaning it can be manipulated and twisted and tweaked in structural ways before being executed. And this isn't the first time we've experimented with this idea: CLOS, Common List Object System, was where Gregor Kiczales, of AspectJ fame, cut his teeth on the AOP concepts, largely because it seemed to him that having a completely open meta-object protocol was too dangerous--but that's another story.

So given that everybody's going ga-ga over Ruby's MOP/metaprogramming facilities, it occurs to me, is what we're really after an MOP for Java? Because such things exist already in the academic world--see OpenJava, for example. Is that what Java would need to do to evolve and take back the productivity label? Is the lack of productivity in Java (the chief complaint of Java today, according to Bruce Tate) due directly to its statically-typed nature, or is it simply the inability to twist the language in ways that are closer to what the programming staff really needs and wants? After all, if you take away some of the MOP features that Ruby uses to make the Person class pretty brief, such as the attr_reader and attr_writer manipulators, then a lot of Ruby's terseness goes away, too.

Not that functional languages aren't still a good way to go--and I will continue the discussion of Scala, largely because there's some nifty stuff there we haven't touched yet--but I'm becoming more and more convinced that the problem with Java isn't in its statically-typed nature, but in the language we use to generate bytecode for the platform. (.NET may have some better answers here, given Microsoft's greater friendliness to languages on their platform, but it's just another platform similar to the JVM in a lot of ways, and as such has the same drawbacks and benefits as Java does in this regard.)

So where are all the good macro-friendly tools/languages for Java? (And that means, "macros as from LISP", not "macros as from C". Frankly, C++ could use a good MOP, too, but that's another story for another day...)

"The request could not be completed because the service is too busy. Please try again later." (From MSMessenger, about thirty seconds ago.)

The new home page is alive and kicking...

My dad sent me this:

In today's world, we typically spend a good deal of time traveling, and with a lot of that travel by air, I thought you might enjoy the attached. Jerry Cosley is an acquaintance of mine, someone I worked with at TWA, and among his other positions he was the Staff Vice President of Public Relations. Now that TWA is gone, Jerry is involved with others in sifting through some of the memorabilia and historical items. He ran across this item, provided by Stout Air Services, a predecessor of TWA, and wanted to share it. In 1929, TWA was the hot ticket - you could get from New York to California in only 48 hours, flying by day and riding on the train through the night. #4 of course is difficult on today's airplanes. By way of comparison, in 1979 I flew in a 747 from LA to JFK in 3 hours, 58 minutes.

How to Get the Maximum Enjoyment Out of Flying

(These simple rules are provided by the Stout Air Services, Inc.)

1. DONT WORRY. Relax. Settle back and enjoy life. If theres any worrying to do let the pilot do it. Thats what hes paid for.
2. The pilot always takes off and lands into the wind. Be patient while the plane taxies to a corner of the field before taking off.
3. The pilot always banks the plane when turning in the air. Just as a race track is banked at the corners, so is an airplane tilted when making a perfect turn. Take the turns naturally with the plane. Dont try to hold up the lower wing with the muscles of your abdomen; its unfair to yourself and an unjust criticism of your pilot.
4. The atmosphere is like an ocean. It supports the plane just as firmly as the ocean supports a ship. At the speed you are traveling the air has a densitypractically equivalent to water. To satisfy yourself, put your hand out of the window and feel the tremendous pressure. That ever-present pressure is your guarantee of absolute safety.
5. The wind is similar to an ocean current. Once in a while the wind is gusty and rough, like the Gulf Stream off the coast of Florida. These gusts used to be called air pockets, but they are nothing more than billows of warm and cool air and are nothing to be alarmed over.
6. The air pressure changes with the altitude. Some people have ears that are sensitive to the slightest change in air density at different altitudes. If yours are, swallow once in a while, or breathe a little through the mouth. If you hold your nose and swallow, your will hear a little crack in your ears, caused by the suction of air on the ear drums. Try it.
7. Dizziness is unknown in airplanes. There is no discomfort in looking downward while flying, because there is no connection with the earth. Owing to the altitude you may think you are moving veryslowly, although the normal flying speed is above 105 miles an hour.
8. WHEN ABOUT TO LAND: The pilot throttles the engine preparatory to gliding down to the airport. The engine is not needed in landing and the plane can be landed perfectly with the engine entirely cut off. FROM AN ALTITUDE 0F 2,500 FEET IT IS POSSIBLE TO GLIDE WITH ENGINE STOPPED TO ANY FIELD WITHIN A RADIUS OF FOUR MILES!

In the Scala documentation, they make a point of calling out the idea that "everything's an object", including numbers and (most importantly) functions. Smalltalk had this same perception/assumption in its design, and Scala, as a result, sometimes feels very Smalltalk-ish.

For example, when Scala sees this expression:

2 + 4 * 7

2.+(4.*(7))

val nums = 1 :: 2 :: 3 :: 4 :: Nil;

val nums = 1.::(2.::(3.::(4.::(Nil))));

> val msg = "Hello World"
val msg: java.lang.String("Hello World") = Hello World
> msg equals "Hello World"
true: scala.Boolean

Thus far, this concept of everything being an object may not seem all that powerful; in fact, arguably, the above section should probably have been mentioned in the previous post than this one, since the ability to recognize new operators is itself an extension of the idea of expressing exactly what you want, nothing more. In fact, in the "Scala by Example" document that comes with the Scala download, they show a traditional Java (but which could easily be written to read in C++ or C#) quicksort implementation:

def sort(xs: Array[int]): unit = {
  def swap(i: int, j: int): unit = {
    val t = xs(i); xs(i) = xs(j); xs(j) = t;
  }
  def sort1(l: int, r: int): unit = {
    val pivot = xs((l + r) / 2);
    var i = l, j = r;
    while (i <= j) {
      while (xs(i) < pivot) { i = i + 1 }
      while (xs(j) > pivot) { j = j - 1 }
      if (i <= j) {
        swap(i, j);
        i = i + 1;
        j = j - 1;
      }
    }
    if (l < j) sort1(l, j);
    if (j < r) sort1(i, r);
  }
  sort1(0, xs.length - 1);
}

def sort(xs: List[int]): List[int] =
  if (xs.length <= 1) xs
  else {
    val pivot = xs(xs.length / 2);
    sort(xs.filter(x => x < pivot))
    ::: xs.filter(x => x == pivot)
    ::: sort(xs.filter(x => x > pivot))
  }

• If the list is empty or consists of a single element, it is already sorted, so return it immediately.
• If the list is not empty, pick an an element in the middle of it as a pivot.
• Partition the lists into two sub-lists containing elements that are less than, respectively greater than the pivot element, and a third list which contains elements equal to pivot.
• Sort the first two sub-lists by a recursive invocation of the sort function.
• The result is obtained by appending the three sub-lists together.

As hinted, functions are full objects in of their own right, and are just as easily accessible as parameters as any other object passed into a method. So, for example, consider the above sort implementation again. The only thing that really "ties" it to sorting lists of integers is the comparison that goes on to determine if the item inside the list is less-than, equal-to, or greater-than other elements in the list. If we could somehow genericize that decision-making, we could make the quicksort be entirely generic and applicable to lists-of-anything. (As it turns out, it's sometimes easier to do this by simply having any types that wish to be sorted implement the <, == and > methods in Scala, and this is possible to enforce via interfaces and mixins and such, but bear with me on this example.)

We'll start by making sort generic:

def sort(xs: List[T]): List[T] =
  if (xs.length <= 1) xs
  else {
    // ...
  }

def sort(xs: List[T]): List[T] =
  if (xs.length <= 1) xs
  else {
    val pivot = xs(xs.length / 2);
    sort(xs.filter(x => (x less-than pivot) )
    ::: xs.filter(x => (x equal-to pivot) )
    ::: sort(xs.filter(x => (x greather-than pivot) )
  }

def sort[T](xs: List[T], lt: (T, T) => boolean, 
            eq: (T, T) => boolean, gt: (T, T) => boolean) : List[T] =
  if (xs.length <= 1) xs
  else {
    val pivot = xs(xs.length / 2);
    sort(xs.filter(x => lt(x, pivot)), lt, eq, gt)
      ::: xs.filter(x => eq(x, pivot))
      ::: sort(xs.filter(x => gt(x, pivot)), lt, eq, gt)
  }

object App with Application {

  def lessThan(lhs: int, rhs: int) : boolean =
    if (lhs < rhs) true else false;

  def equalTo(lhs: int, rhs: int) : boolean =
    if (lhs == rhs) true else false;

  def greaterThan(lhs: int, rhs: int) : boolean =
    if (lhs > rhs) true else false;

  val nums : List[int] = 1 :: 4 :: 3 :: 2 :: Nil;
  val sorted = Test.sort(nums, lessThan, equalTo, greaterThan);
  System.out.println(sorted);
}

This is where the notion of an anonymous function becomes important, however. Instead of defining those three functions outright and referencing them by name in the sort call, we can instead define them "on the fly" in the call itself:

object App with Application {
  val nums : List[int] = 1 :: 4 :: 3 :: 2 :: Nil;
  val sorted = Test.sort(nums, (lhs:int, rhs:int) => if (lhs < rhs) true else false, 
                               (lhs:int, rhs:int) => if (lhs == rhs) true else false,
                               (lhs:int, rhs:int) => if (lhs > rhs) true else false );
  System.out.println(sorted);
}

Er... maybe.

If you're like a lot of developers, you're looking at the above and you're not necessarily won over. There's a couple of things that could be red-flagging at the back of your head:

• "I thought that the Don't-Repeat-Yourself principle says it's better to collect this stuff into one place, for easier maintenance?" True. But consider this: how often have you written a method in a class because you HAD to, not because it satisfied the DRY principle? Java's use of nested inner classes (and C++'s inability to allow you to define functions in-line) forces the use of methods, even in those situations where you know, without a doubt, that you will never execute or reference this code more than once.
• "Shouldn't this be making the sort shorter to use?" False. Anytime you genericize something, you run a (strong) risk that you're actually making it more complicated, and therefore more difficult to use. If we really wanted to make sort easier, I'd ask for a function parameter "compare" that does the traditional C-style comparison (return -1 for less-than, 0 for equal, 1 for greater-than), and write the necessary code inline in the sort() to use that method to determine if something fit the less-than, equal, or greater-than filters. That's not really the point, though... so I didn't.
• "That syntax is just ugly." True. To a Java, C++ or C# programmer. To a LISP programmer, though, there's clearly some parentheses missing. Seriously, the syntax isn't what you're used to, perhaps, but like most syntactic decisions, it has deeper meaning and rules around it that makes it look that way. The same is true of Java, C++ (remember the ">>" rule in multiple template usage?) and C#, and will remain so for as long as computers are what's interpreting our source code.
The thing is, the use of functions-as-objects is just the tip of the iceberg. Turns out there's some more interesting tidbits that we can make use of when using functions as objects, one of which is called "currying".

One frequently useful idiom in functional languages is to return a function, rather than the results of applying that function. This means that we can delay actually executing the function until later--this is what we're doing (sort of in reverse, passing it in rather than returning it) in the sort example above. We pass in the comparator function into the filter routine, who then executes it. Returning a function is of the same mindset--hand back a function to be executed by the caller (either implicitly or explicitly) that produces the results desired.

In some situations, however, the full inputs of the function aren't known at the time the function is returned. Or, as is often the case, some of the inputs are known, but others aren't. So the compiler, when handed a partially-called function, defers execution and uses parameters found in the caller's scope (wherever that may be) to fill out the remainder of the necessary functional inputs and carry out execution. Make sense?

Probably not; currying takes a while to ingest. At least it did for me. An example may serve to help cement this down.

def sum(f: int => int) = {
  def sumF(a: int, b: int): int =
    if (a > b) 0 else f(a) + sumF(a + 1, b);
  sumF
}

Here, we see a function "sum", which takes a function "f" that takes an int and returns an int. It in turn uses a nested function, "sumF", that takes two integer arguments "a" and "b" and applies the function "f" to them so long as "a > b". Notice, however, that sum neither takes the parameters "a" and "b", nor does it manufacture them from someplace in order to pass them in; in fact, it noticeably excludes them when it returns, without decoration, the function "sumF" as the return value of the "sum" function. (Notice again how we don't need to specify the return value of "sum", because Scala's type inference can figure it out without additional help from the programmer.)

So how does one use the sum function? By applying both parameters--the function to apply to each argument, and a pair of ints to supply bounds to be summed up:

sum(x => x * x)(1, 10)

Which, in this case, summarizes the expression 1^1 + 2^2 + 3^3 + ... + 10^10. (Readers with a background in mathematics will recognize it as a sequence, the "big E" notation, as I used to call it back in sophomore Advanced Algebra. Probably has a more formal name than that, but my background isn't in math.) To understand where the currying takes place, look at how the compiler sees this expression:

(sum(x => x * s))(1,10)

Which means, of course, pass the function "x * x" into sum, which then returns the sumF function with f(a) replaced by "a * a". sumF still expects two integer parameters, however, so the compiler takes the next expression "(1, 10)" and applies those as "a" and "b", respectively. From there, it's a simple exercise in recursion to arrive at the answer.

The power of currying becomes more apparent when you see that because the compiler is willing to accept partially-evaluated functions as first-order types, we can partially-define functions in terms of other functions, as in:

def sumInts = sum(x => x);
def sumSquares = sum(x => x * x);
def sumPowersOfTwo = sum(powerOfTwo);

and then use them as top-level functions without any special syntax:

> sumSquares(1, 10) + sumPowersOfTwo(10, 20)
267632001: scala.Int

Now, if for some reason the definition of sum() needed to change, it would ripple throughout this tiny framework by making one change in one place. (Don't know why sum() would need to change, mind you, but that's the problem with simple examples--it's sometimes hard to see the really positive benefits unless you get more complicated, but more complicated examples are harder to use to present the concept.) And I'd be lying to you if I said that I "get" how to use this in code more practical than summations yet--I still have a lot of internalizing to do. But I can see the outskirts of where it might be useful, and if I can get working samples of how and where it would, believe me, they're going up here.

In the meantime, next is traits and mixins, which are both features that are definitely easier to see applicability.



Java/J2EE

Saturday, March 04, 2006 7:52:49 PM (Pacific Standard Time, UTC-08:00)
Comments [5]  | 

My father, whom I've often used (somewhat disparagingly...) as an example of the classic "power user", meaning "he-thinks-he-knows-what-he's-doing-but-usually-ends-up-needing-me-to-fix-his-computer-afterwards" (sorry Dad, but it's true...), often forwards me emails that turn out to be one hoax or another. This time, though, he found a winner--he sent me this article, warning against the latest caller identity scam: this time, they call claiming to be clerks of the local court, threatening that because the victim hasn't reported in for jury duty, arrest warrants have been issued. When the victim protests, the "clerk" asks for confidential info to verify the records. Highly credible attack, if you ask me.

Net result (from the article):

• Court workers will not telephone to say you've missed jury duty or that they are assembling juries and need to pre-screen those who might be selected to serve on them, so dismiss as fraudulent phones call of this nature. About the only time you would hear by telephone (rather than by mail) about anything having to do with jury service would be after you have mailed back your completed questionnaire, and even then only rarely.
• Do not give out bank account, social security, or credit card numbers over the phone if you didn't initiate the call, whether it be to someone trying to sell you something or to someone who claims to be from a bank or government department. If such callers insist upon "verifying" such information with you, have them read the data to you from their notes, with you saying yea or nay to it rather than the other way around.
• Examine your credit card and bank account statements every month, keeping an eye peeled for unauthorized charges. Immediately challenge items you did not approve.

Remember, it's always acceptable to ask for verification of THEIR identity if they're asking for confidential information. And most credible organizations are taking great pains to not ask for that information over the phone in the first place. Practice the same discretion over the phone that you would over IM or email; the phone can be just as anonymous as the Internet can.

Apparently, I touched a nerve with that last post; predictably, people started counting the keystrokes and missing my point. For example, Mark Blomsma wrote:

Looks to me like you're comparing apples and pears.

C# does not force you to use accessors. The following is already a lot closer to Scala.

public class Person
{
public string firstName; public string lastName; public Person spouse;

public Person(string fn, string ln, Person s)
{
firstName = fn; lastName = ln; spouse = s;
}

public Person(string fn, string ln) : this(gn, ln, null) { }

public string Introduction()
{
return "Hi, my name is " + firstName + " " + lastName +
(spouse != null ?
" and this is my spouse, " + spouse.firstName + " " + spouse.lastName + "." :
".");
}
}

This is only 356 keystrokes, compared to 287 for Scala. Now in Scala the default accessor for classes and members seems to be public, if this were not the case then you'd need 323 keystrokes in Scala.
Only a very minor difference. And definately not enough to make a case that Scala is more efficient for a developer.

Another consideration if you start talking keystrokes is that the tooling suddenly becomes a factor. With C# and VS2005 I only type 'prop,tab,tab' and then the type and name info. Skipping quite some keystrokes.

All that sort of misses the point, though: the purpose of the comparison was not to count keystrokes, per se, but to look at the expressiveness of the language and how concisely the language can express a concept without requiring a great deal of scaffolding. C, for example, could always be used to build object-oriented systems... but you had a lot of work to do on your own to do it. As a result, a huge amount of complexity was spent in manaing the relationships between "classes" by hand (by tracking pointer relationships and so on). C++ solved a lot of that by baking those concepts in as a first-class concept, thus reducing the surface area requirment in the programmer's mind devoted to "plumbing", and making room for more business-level complexity. Java did the same to C++ by introducing GC and other VM-level support, and so on. Scala and Ruby (and other hybrid and/or dynamic languages) are now seeking to do the same to Java and .NET.

The question of tooling is an interesting one, though: is a language just the language by itself, or the language plus the tools that support it? Is Lisp still Lisp if you take Emacs out of the equation? Or is Smalltalk interesting without the Smalltalk environment and/or browser? Can we separate the two? Should we? That's a question to which I don't have an easy answer.

While speaking at a conference in the .NET space (the patterns & practices Summit, to be precise), Rocky Lhotka once offered an interesting benchmark for language productivity, a variation on the kLOC metric, what I would suggest is the "CLOC" idea: how many lines of code required to express a concept. (Or, since we could argue over formatting and style until the cows come home, how many keystrokes rather than lines of code.)

Let's start with a simple comparison. The basic concept we want to express is that of a domain object type, my favorite example, that of a Person type. In domain lingo,

A Person has a first name, a last name, and a spouse. Persons always have a first and last name, but may not have a spouse. Persons know how to say hi, by introducing themselves and their spouse.

In Java, we express this class like so:

public class Person
{
    private String lastName;
    private String firstName;
    private Person spouse;
    
    public Person(String fn, String ln, Person s)
    {
        lastName = ln; firstName = fn; spouse = s;
    }
    public Person(String fn, String ln)
    {
        this(fn, ln, null);
    }
    
    public String getFirstName()
    { 
        return firstName;
    }
    
    public String getLastName()
    { 
        return lastName;
    }
    
    public Person getSpouse()
    { 
        return spouse;
    }
    public void setSpouse(Person p) 
    { 
        spouse = p;
            // We ignore sticky questions of reflexivity and
            // changing last names in this method for simplicity
    }
    
    public String introduction()
    {
        return "Hi, my name is " + firstName + " " + lastName +
            (spouse != null ? 
            " and this is my spouse, " + spouse.firstName + " " + spouse.lastName + "." :
            ".");
    }
}

This isn't just a Java problem; the C# version of this type isn't much better:

public class Person
{
    private string lastName;
    private string firstName;
    private Person spouse;
    
    public Person(string fn, string ln, Person s)
    {
        lastName = ln; firstName = fn; spouse = s;
    }
    public Person(string fn, string ln)
        : this(fn, ln, null)
    {
    }
    
    public string FirstName
    {
        get { return firstName; }
        set { firstName = value; }
    }
    
    public string LastName
    {
        get { return lastName; }
    }
    
    public Person Spouse
    {
        get { return spouse; }
        set { spouse = value; }
    }
    
    public string Introduction()
    {
        return "Hi, my name is " + firstName + " " + lastName +
            (spouse != null ? 
            " and this is my spouse, " + spouse.firstName + " " + spouse.lastName + "." :
            ".");
    }
}

Class Person
  Dim _FirstName As String
  Dim _LastName As String
  Dim _Spouse As Person

  Public Sub New(ByVal FirstName As String, ByVal LastName As String, ByVal Spouse As Person)
    Me._LastName = LastName
    Me._FirstName = FirstName
    Me._Spouse = Spouse
  End Sub

  Public Sub New(ByVal FirstName As String, ByVal LastName As String)
    Me.New(FirstName, LastName, Nothing)
  End Sub

  Public ReadOnly Property LastName() As String
    Get
      Return _LastName
    End Get
  End Property

  Public Property FirstName() As String
    Get
      Return _FirstName
    End Get
    Set (ByVal Value As String)
      Me._FirstName = Value
    End Set
  End Property

  Public Property Spouse() As String
    Get
      Return _Spouse
    End Get
    Set (ByVal Value As Person)
      Me._Spouse = Value
    End Set
  End Property

  Public Function Introduction As String
    Dim temp As String
    temp = "Hi, my name is " & _FirstName & " " & _LastName
    If _Spouse <> Nothing Then
      temp = temp & " and this is my spouse, " & _Spouse.FirstName() & " " & _Spouse.LastName() & "."
    Else
      temp = temp & "."
    End If
    Return temp
  End Function
End Class

class Person
  def initialize(firstname, lastname, spouse = null)
    @firstname = firstname
    @lastname = lastname
    @spouse = spouse
  end

  attr_reader :lastName
  attr_writer :firstName, :spouse
  
  def introduction
    if spouse == nil
      "Hello, my name is #{firstName} #{lastName}"
    else
      "Hello, my name is #{firstName} #{lastName} and this is my spouse, #{spouse.firstName} #{spouse.lastName}"
    end
  end
end

class Person(ln : String, fn : String, s : Person)
{
    def lastName = ln;
    def firstName = fn;
    def spouse = s;
    
    def this(ln : String, fn : String) = { this(ln, fn, null); }

    def introduction() : String = 
        return "Hi, my name is " + firstName + " " + lastName +
            (if (spouse != null) " and this is my spouse, " + spouse.firstName + " " + spouse.lastName + "." 
             else ".");
}

This is hardly an exhaustive comparison of the languages, but it does give you a little taste of Scala's object flavor. Ruby's syntax is arguably of the same length as Scala's (and frankly, to my mind, they're too close to call... or care), but clearly Scala's length is much much smaller than that of the equivalent C#, Java, Visual Basic or C++ class. (C++ could make things interesting with judicious use of templates to handle backing store, accessor and mutator, but that's considered too advanced by many C++ devs, and therefore too obscure to use in common practice, rightly or wrongly.)

When next we look at this, we'll look at what Scala means when they say "everything's an object"... and how that, in many ways, this means that Scala is more object-oriented than Java itself.

Update: Glenn Vanderburg pointed out that my Ruby wasn't quite correct, and also suggested a bit more "Rubification":

     class Person
       def initialize(firstname, lastname, spouse = null)
         @firstname, @lastname, @spouse = firstname, lastname, spouse
       end

       attr_reader :lastName
       attr_accessor :firstName, :spouse  # attr_writer *just* makes a writer.  You really want this.

       # I would typically use the more explicit "if" that you used here, but for terseness I've
       # put this in the form you used with the Scala version:
       def introduction
         "Hello, my name is #{firstName} #{lastName}" + (spouse ? " and this is my spouse, #{spouse.firstName} #{spouse.lastName}" : "")
       end
     end

Thanks, Glenn. Again, I'm struck by how Ruby's strength lies not in the core language itself, but the various "macros" that they've defined (such as attr_reader and attr_accessor or attr_writer). This notion of "core language with user-defined extensions" is a powerful one, and I hope to show how Scala does much the same in its language definition.

As Joel before me, I'm going to be in beautiful Victoria, British Columbia, on April 4th to present at the Victoria .NET Developers Association, and as usual, the topic of what to present has come up.

Normally, this is a subject that the user group lead and I sort of hash out in private beforehand, but in this case, Nolan Zak (the user group lead) suggested I post here and call for suggestions. So, here's a list of topics I can present on, send me your thoughts.

1. Pragmatic XML Services: you know about SOA, you've heard the Four Tenets.... now what?
2. Intro to WCF: All you need to know about Microsoft's latest communication stack.
3. Web Services: Overview of the specs, the stacks, and the standards. What's critical, what's useful, what's vendor hype and fluff.
4. C# 3/LINQ: What's in their heads for C# v.Next
5. (Or suggest your own idea.)

I know that many readers of this blog complain when I take time out from technical topics to talk about personal stuff, so if you're one of those folks, move along. This is about as personal as it gets, and fair warning: if you're going to complain about this post, I'm going to ignore you, at best.

Daniel Steinberg, a fellow No Fluff Just Stuff speaker, lost his seven-year-old daughter not too long ago, and he wrote the story up in a really poignant and moving piece he called "Dear Elena".

Dan, you can't imagine how terrible I feel for you right now--that's every parent's worst nightmare. I wish there were something I could do or say to make this time easier or less tragic for you, but of course there isn't. She sounds like she was a wonderful little girl, and I'm saddened by the fact that I didn't get the chance to meet her. My thoughts and prayers are with you and your family right now.

Now, if you'll all excuse me, I'm going to Skype my six-year-old son at home. For what I would hope to be a fairly obvious reason, I feel the need to give him a hug from here in London.

Several people have commented on my recent posting about Spring, and I want to make something clear: I'm not saying that Spring (or Hibernate, or EJB, or anything else) is a bad technology. I'm saying that walking up to every project, assuming that Spring will be THE answer, is bad. This kind of dogmatic approach--which, by the way, more than anything else is what led to the downfall of EJB as a popular technology--is bound to bite you in an uncomfortable place sooner or later.

One commenter, in particular, chastised me for not providing specific examples regarding where Spring may fail; I'm not going to stand here and make an exhaustive analysis of Spring's strengths and weaknesses. Besides being something that's already being done elsewhere, it would be beside the point that I'm trying to make--that dogma of any form is bad.

Look, so you've been successful with Spring on a few projects--that's good, and I encourage you to consider Spring again for your next couple. But don't make the dangerous assumption that using Spring will always yield success. In fact, let's take this out of the realm of Spring entirely and restate the point: "Look, so you've been successful with [[TECHNOLOGY-X]] on a few projects--that's good, and I encourage you to consider [[TECHNOLOGY-X]] again for your next couple. But don't make the dangerous assumption that using [[TECHNOLOGY-X]] will always yield success." (Where [[TECHNOLOGY-X]] can be, but isn't limited to, one of Spring, Hibernate, EJB, J2EE, COM+, WCF, CORBA, XML services, relational databases, stored procedures, managed-code-inside-the-database, highly denormalized relational data, highly normalized relational data, ....)

Recently, while cruising the Internet (and, in particular, the Lambda-the-Ultimate site), I ran across the Scala programming language, latest brainchild of Martin Odersky (of GJ fame, which of course was derived from Pizza, among others). It's another entry in the hybrid functional/object language space, and as such, has a lot of interesting features that Ruby holds, but runs on the JVM (and can actually cross-compile into a .NET assembly, though it does require some slightly different mappings), and as such means developers don't have to make a wholesale commitment to the Ruby interpreter.

I thought I'd share some of the more interesting bits of Scala in this and a few more blog posts.

The high-level stuff

First of all, from the Scala website, let's get the high-level overview stuff out of the way:

• Scala is object-oriented. Scala is a pure object-oriented language in the sense that every value is an object. Types and behavior of objects are described by classes and traits. Class abstractions are extended by subclassing and a flexible mixin-based composition mechanism as a clean replacement for multiple inheritance.
• Scala is functional. Scala is also a functional language in the sense that every function is a value. Scala provides a lightweight syntax for defining anonymous functions, it supports higher-order functions, it allows functions to be nested, and supports currying. Scala's case classes and its built-in support for pattern matching model algebraic types used in many functional programming languages. Furthermore, Scala's notion of pattern matching naturally extends to the processing of XML data with the help of regular expression patterns. In this context, sequence comprehensions are useful for formulating queries. These features make Scala ideal for developing applications like web services.
• Scala is statically typed. Scala is equipped with an expressive type system that enforces statically that abstractions are used in a safe and coherent manner. In particular, the type system supports generic classes, variance annotations, upper and lower type bounds, inner classes and abstract types as object members, compound types, explicitly typed self references, views and polymorphic methods. A local type inference mechanism takes care that the user is not required to annotate the program with redundant type information. In combination, these features provide a powerful basis for the safe reuse of programming abstractions and for the type-safe extension of software.
• Scala is extensible. The design of Scala acknowledges the fact that in practice, the development of domain-specific applications often requires domain-specific language extensions. Scala provides a unique combination of language mechanisms that make it easy to smoothly add new language constructs in form of libraries: any method may be used as an infix or postfix operator, and closures are constructed automatically depending on the expected type (target typing). A joint use of both features facilitiates the definition of new statements without extending the syntax and without using macro-like meta-programming facilities.

Enough overview. Let's look at code.

Hello, Scala

OK, Scala really isn't all that interesting as a Hello World program, but it does highlight one of the more interesting elements of Scala that I already like:

object Hello {
  def main(args: Array[String]): Unit = {
    Console.println("Hello, Scala!");
  }
}

Some of what's interesting to see here is that the Scala compiler actually produced two .class files--one entitled App.class, another called App$.class, the second App$ class apparently to provide "module" behavior (which I suspect is related to the singleton-ness of the object declaration in the code). As you might expect, Scala injects some additional support methods into both classes, including getScalaType, which is obviously intended to return the type of the object to Scala, just as the .class or getClass does for Java. Which brings up another interesting point.

Scala presents a unified type hierarchy, such that scala.Any is the root of the type system, and (like the CLR) is bifurcated into two basic elements, one being the object-family of types (java.lang.Object, known to Scala as scala.AnyRef) and the "primitive type" family of types, known to Scala as scala.AnyVal. Scala calls these reference classes and value classes, respectively--the same monikers the CLR uses. There's also reference to a type scala.All, which the introduction/tutorial page puts at the bottom of the type hierarchy, apparently inheriting from everything, but I'm can't find documentation on it or what purpose it serves. *shrug* More on that later, I guess.

Another interesting tidbit is that we can run Scala interpretively, the same way we can do to Groovy:

> scalaint -nologo HelloWorld.scala
> HelloWorld.main(null)
Hello, world!
(): scala.Unit
>:q

Update: Forgot to mention, Scala has another interesting element to it that makes it very interesting to Ruby in much the same way:

object HelloWorld2 with Application { 
  Console.println("Hello, world!"); 
}

I consider myself a relative newbie to Scala, but as I progress through the language and see some useful applications of features, I will blog more. I'll also blog some of the features themselves, but you can find that for yourself by working through the Scala tutorial material on the site, if you're so inclined. In the meantime, catch the presentation I'm doing on Scala at the No Fluff Just Stuff symposiums, starting 2Q this year.

And, by the way, for those of you in the .NET space, Scala does, as I mentioned before, cross-compile to .NET assemblies, though I haven't spent much time exploring this. Frankly, I'd be more comfortable using Scala in the .NET world if there was a .NET-based compiler for it, rather than having to install a JRE just to run the compiler, but F# serves much the same space in the .NET world that Scala does here, and that's another language I'm pursuing with some vigor, as well. More on that later.

There are a few things, I've found, that are fun about being a speaker and general rabble-rouser, but none of them are nearly as much fun as when I get an opportunity to interview industry icons and ask them all my questions on camera. In this case, while at JavaPolis2005, my victims were the well-known pair Joshua Bloch and Neal Gafter, who, more than anyone else in the world, are most directly responsible for the language features that came in Java5. I tried to keep the interview pleasant and friendly, but I did ask the questions that've bothered me for a while, like "Why did generics end up the way they did?", "Is Java too complicated and hard to use now?" and "What are you doing at Google these days, anyway?"

Online (registration required) at the JavaPolis2005 site. Keep an eye on the site for the other interviews Dion did, as well as one more I did with Brian Goetz, who's got a GREAT book on Java Concurrency coming out in 2006.

CrazyBob just wrote about how he "doesn't get Spring", and although it runs the risk of sounding like something from the "Me, too" bandwagon, I have to say, I agree with him (and have been saying this in conferences and panels for a while now):

Even worse, I've noticed what I consider to be a dangerous and blind increase in the rate of Spring adoption. I've yet to read a critical article or book on Spring. It seems like everyone loves Spring except me.

Maybe Spring adoption is a knee-jerk reaction to J2EE. "J2EE is bad, and the Spring guys say their stuff is better, so Spring must be good." It doesn't work that way.

For starters, I'm with Bob on the statement that blind adoption of Spring is dangerous. I wrote once before that dogma of any form is bad, and Spring dogma is just as bad and just as dangerous as J2EE dogma ever was, for much the same reason: dogma discourages thinking. Walking onto a project, prepared already to believe that Spring is the best solution to the problem, without considering the context, is just as bad as when we did that with J2EE. In fact, any technology can fall into that trap, be it Ruby, .NET, J2EE, Spring, LAMP, Vista, COM/DCOM/COM+, you name it. ANY kind of dogma that allows developers to shut off the analytical part of their brain is dangerous. Spring is a useful technology, no question. But so is J2EE, and so is .NET, and so is LAMP, and...

Don't ever make the mistake of letting dogma drive your technology decisions, period. No matter who justifies them. I thinke states it best when he says

If you do [adopt Spring], go in with your eyes wide open. Be skeptical, critical. Just because someone has a popular Open Source framework, they have slick marketing, and they're supported by a big vendor (IBM pushed Struts on me for a number of years after all), it doesn't necessarily mean they know what's best for you or even that they know better than you.

Bob then goes into how he likes the setter-injection that Spring provides; I personally don't have the same degree of fondness for dependency injection, to be honest: I find JNDI (and Service Locator) to be a superior approach, mostly because with the Service Locator, I can control when the resource moniker is resolved, meaning that if the resource should fail somewhere during the call, I can control where and how I go back to the Service Locator for a failover attempt. More importantly, I can re-resolve the resource as my failover policies permit, and I'm not held hostage to how--or mre importantly when--the container decides to inject the dependency.

At the end of the day, it's important to remember that "lightweight" and "testable" doesn't have to mean "Spring". In-process testing of EJB components is possible thanks to the in-proc nature of the OpenEJB stack. Testability of JNDI is easily accomplished with unit tests that use the Hashtable JNDI provider that Sun makes available in the Java Tutorial, if you want or need to test the Service Locator code itself. Or, you take the "black box" approach (as some recommend for servlet containers), and test your code through the container itself by doing the heavierweight communication through the communication stack from out-of-process calls. In the end, it's not the APIs that define the tool's testability, but the ability to embed the tool inside a unit-test environment.

Would I recommend Spring? Certainly, under the same circumstances and for the same reasons I'd recommend J2EE: when it's appropriate, because there's some good stuff there, and it's well-known and an official (in J2EE's case) or de-facto (in Spring's case) standard.

Oh, and let's not forget, this applies to any technology, including the upcoming rise of dynamic languages...

Billy Hollis, famed Visual Basic lecturer and secret programming language anthropologist, has compiled a succinct history of programming languages. As he puts it,

If you like VB, look at the history of the C family [of languages] first. If you like C#, Java or C++, look at the history of the BASIC family first.

Happy Friday.

A number of you have made comments about my LINQ paper, and rather than respond in comments in turn, I thought I'd gather them up and respond to them en masse. So, without ado....

Stu Smith said:

Nice article. Two things occur to me immediately...

1. Based on my current understanding of LINQ, it's purely for querying, and so compared to most O/R systems it lacks caching support. (ie its queries may be optimal but that's not much consolation if it keeps re-executing them).
2. The O/R system we use (which admittedly only needs to support a particular kind of application) solves the 'lazy load vs N+1' issue by generating joined queries based on either the path taken to the data, or on particular routes to a marked-up tables. ie, if you navigate from a Customer to a collection of Orders, that's a single select. If you then start iterating the Orders and inner-iterating the OrderDetails, then on the first one a second joined select is issued, and for subsequent iterations the data is already cached and thus no further SQL statements are emitted.

1. No, LINQ can, in fact, do any sort of relational manipulation, including INSERTs, UPDATEs and DELETEs, but the real strength of the language integration is in the query aspects, particularly the fact that LINQ can do these queries across any rectangular data store, so it's fair to say that LINQ is mostly about query.
2. I'd be interested to hear more about your solution to solve the lazy-load issues, partcularly how you handle the situation where you need to display only a small part of the full OrderDetails data--remember, part of the criticism of O/R is that they either eager-load too much, or lazy-load too much, and can't infer the amount or areas of data to retrieve that's "just right".

Bryant wrote:

I thought the article was well written and informative. While I think it's cool that I might one day be able to use the same object model to query databases, XML documents, and even the file system I still feel compelled to look back upon my days as a DBA. This looks a lot like building ad hoc SQL statements in the code except we're a little more type safe here. LINQ still does not answer the question of storage abstraction. The developer still needs an intimate knowledge of the database structure. So, LINQ appears to cover "Conflicting type systems" and maybe "Transactional boundaries" and "Query/access capabilities". There are still four more items in your list that I don't see being solved with LINQ. Your article was a great read but sorry, I am still not excited.

James commented:

I thought the article was great! I'm a little unclear as to why it's not getting ranked better on MSDN but for someone who didn't really understand/appreciate the problem domain LINQ is serving, your article really got me thinking and cleared up a lot of fogginess in my mind. Excellent work!

Andy Maule said:

Very interesting! I liked the discussion of Rail's ActiveRecord which I think is an approach that most people miss when talking about OR Mappings.

There's a good research paper discussing the same stuff here. It mentions something recently developed for doing statically typed queries in Java 'Native Queries' which is an interesting comparison to LINQ. Anyone interested in this area should take a look.

I'm currently doing a PhD in this area, and I have to say that LINQ is making things very interesting.

Karmencita is my lightweight alternative for in memory object querying.

As for the rest of you who offered kudos (Bart De Boeck, Dan Kahler, Paul Wilson, Eric Bachtal), thanks; every author likes to know that their work is appreciated, particularly when so much of what they say seems to stir up more controversy than discussion.

Matt Morton commented, "One might be able to say that Ted Neward is cynical about any new technology. You might also say he puts himself in the position of the "old" kermudgeon (sp) who opposes anything new and cool." Yep, guilty as charged, for a very specific reason.

Ages ago, when EJB first shipped, I was one of the first who looked at it with stars in my eyes. It seemed like such a great, easy solution to all the problems of developers building server-side systems (and I'd done a C++-based 2-tier, CORBA-based 2-tier and Java/NetDynamics-based 3-tier system before this, so I kinda fit into that space already). I was excited. I was ready to swear it to all my friends. And then....

It was a lunch with Don Box and Kevin Jones, shortly after I'd joined DevelopMentor, and I asked Don and Kev about EJB. Don looked at Kev with this silly grin on his face, and Kev just shook his head and said, "It's a thing on a thing. That's always slower than just a thing." Slowly, the light dawned. Over the course of several conversations with Kev and Don later, I came to realize that EJB wasn't a distributed object technology, but a component technology focused on transactions. Over time, developers' stupidity in using EJB for their single-database simple-HTML-form apps drove me to be widely proclaimed as an "EJB expert who consults against EJB", which isn't exactly correct--I've recommended EJB in scenarios, but only where it seems appropriate.

Which, if you think about it, is what we're supposed to do: recommend tools where they're appropriate.

What does this have to do with being a curmudgeon? Simple: I trust no technology until it proves itself to me. Our industry is SO filled with hype, it's surprisingly easy to get caught up in the energy and excitement that surrounds a new tool, particularly those that deal with presentation issues. (Face it, folks, the "jazz factor" surrounding Avalon/WPF or Ajax is exponentially higher than that surrounding RIFE/Continuations, despite the fact that the latter is far more interesting from a technology perspective. Why is this? Because you can SEE the niftiness in Ajax; the continuations story isn't something you can show off to your mom or impress your significant other.) I look at new tools, new technologies and deliberately look to find the flaws, the various fallacies they fall prey to, and I routinely caution people against them JUST because they're new. I would much rather err on the side of caution and hestiation than fall into the trap of hyperbole and bandwagon, because I think, ultimately, it's a more responsible position to my clients and audience.

I don't think I'm unjustified in this position: there's an unhealthy absence of cynicism in our world right now. My two big examples: the terrible tragedy of the miners in West Virginia (why didn't anybody in the media CONFIRM the story of their rescue before reporting it?) and the South Korean human cloning story. Or the supposed report of "cold fusion" from a decade or two ago. Our industry could use, I think, from a large dose of, "OK, so you've created a new framework. What's it to me?" right now.

Matt went on to say,

I dont totally agree with #4 though. When a large scale Ruby project fails it wont be because of the language. Just as it is faulty to claim a language will reduce project failures, it is just as faulty to claim that a language or platform will be the cause. Projects fail because of people, plain and simple. People in general have trouble being honest especially when they have something vested emotionally (or financially) in a project. Perhaps this is what he is getting at. The proponents are so emotionally involved with Ruby that when less experienced folks try to apply it to a larger scale the project, it will blow up.
Personally I have found that Rails and Ruby are a joy to use. I guess then you could say that I get emotionally involved with things that bring me joy. Perhaps Rails strength is its weakness. It is such a joy that it blinds you to its true uses.

Which, if you think about it, is exactly the same thing that happened with EJB before it.

Lego, those folks who brought you Mindstorms in the first place, have done it again--they've announced a next-generation programmable brick, the NXT, that offers some really cool enhancements, including Bluetooth capability. Suddenly, the old Robotwar clones from ages ago take on an entirely new meaning....

More interestingly, supposedly they've announced that the VM itself will be documented, which leads me to wonder how long before we see J-NXT and NXT#. It's amazing how the managed environment is just spreading like wildfire through the world, if'n you ask me....

Hey, Santa, if you're listening, this just went on my list in a BIG way.

So the December CTP of $g{Vista} is available, and I'm finding a recurring problem trying to install it into VMWare. When I tried this with the Beta1 of Vista into VMWare 5.0, the same problem occurs as what I'm getting now: when trying to boot off of the .ISO inside of VMWare (in other words, not physically off a burned CD/DVD, but out of the .ISO image itself mapped into the virtual CD in VMWare), I get a BSOD every... single... time. Is this indicative of a f-ed up installation of my VMWare image, is this a known bug in Vista (sidenote: yes, I know about the raw partition issues with Vista, but doing an XP-first install of Vista gives the same BSOD on reboot), or is it just the combination of ISO-image-and-VMWare? Anybody got ideas for a workaround or a fix?

I've been contacted by a third party (not a recruiting agency) who's having a hard time finding a J2EE architect with familiarity/expertise with concepts of architecture and the low-level chops not to lose sight of the code. To put it in their words:

This is strongly influenced by component-based methodologies such as OpenDoc, but would be extending the RCP Platform developed by Eclipse, and encompassing other concepts from systems such as Squeak... We are having trouble finding someone senior enough to understand the conceptual architectural issues, but technical enough to know important lower-level details such as how design choices will effect performance and scalability.

By the way, DON'T send me your resumes--in your email to me, tell me what your favorite alternative language or platform is, and why. I'll use that to know if you're somewhere in the ballpark.

And no, job postings are NOT going to become a regular part of this blog; I'm doing this as a once-off. (My motivation? I just want to help these guys; this isn't an easy requirement to fill, and I'm hoping I get to meet whomever it is they end up selecting, 'cuz I'm thinking they'll be a kindred spirit. )

Update: They sent me a formal job description, and it seems the desires changed just a bit. *shrug* See for yourself:

A lead software architect for [[name snipped]]. Applicant must be able to take a high level conceptual framework and architect a conceptually sound implementation using open standards in collaboration with an existing team. Experience with Java, Eclipse, and the Rich Client Platform (RCP) sub-project in Eclipse is a significant advantage. Experience with Aspect Oriented Programming is a plus. Participation in the open standards community a plus (e.g. W3C and Eclipse standards communities). Five or more years of experience required with experience as a lead software architect, industry experience preferred. Opening is for a full-time onsite position.

Jim Hugunin strikes again: IronPython 1.0 beta 1 is out. I think Jim should get an award for "first language designer to operate on both the JVM and CLR". Now he just needs to do one more implementation to get the hat trick.

Flame away...

The Pragmatic guys are at it again... This time it's a whole course, taught by two of the finest instructors I have had the privilege to know (and, quite honestly, argue with), on everybody's favorite presentation-layer hot topic, Ajax.

By the way, dear audience, this is one class you can attend regardless of which camp you prefer--both Stu and Justin are equally adept on both enterprise platforms (Java and .NET) and the new hot language, as is clear when they say that they will show you how "to use frameworks such as Rails, Spring, and ASP.NET"; Justin, for example, co-authored "Better, Faster, Lighter Java" and the "Spring Developer's Handbook", as well as built DevelopMentor's ASP.NET website and infrastructure. Stu was one of the COM cognoscenti back in the day (his poems on the subject (towards the bottom, search for Stu) are legend), and then took over as Java curriculum lead when DevelopMentor... well, created a Java curriculum. Two brighter guys--and better instructors--you're not likely to meet.

Oh, and, uh, they seem to know a fair amount about Ajax, too. Enough that I'd attend the course, were I not already busy that week.... Do yourself the favor, if you want to know more about Ajax, go see them. I can think of a lot worse ways to spend a grand in cash and 3 days...

An interesting question emerged during a discussion with some buddies/co-workers/peers/whatever-you-want-to-call-them today:

"Which conferences have you attended in the past that you thought was really good, and why? Which sessions were your favorites, and why? What made them that way?"

Yes, this is somewhat selfish, since the new conference season is amping up, and obviously I'd like to make sure my sessions are ones that people find interesting and recommend to others, but the question actually stemmed from an unrelated discussion to that. I promise.