ON THIS PAGE
    ARCHIVES
    CATEGORIES
    BLOGROLL
    LINKS
    SEARCH
    MY BOOKS
    DISCLAIMER
 
 Sunday, June 18, 2006
"Pragmatic Architecture" TechEd Webcast now up

Cathi Gero's and my session from TechEd, "Pragmatic Architecture", is now available as a webcast for your viewing and listening pleasure. We had a few issues with the audio, which got us started late, but overall the general feedback was positive. Enjoy...


.NET | C++ | Conferences | Java/J2EE | Windows | XML Services

Sunday, June 18, 2006 10:01:47 AM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
 Thursday, June 15, 2006
To whatever moron pulled the fire alarm at the speaker hotel at TechEd....

You are a dead man if I ever find you.

That said, it was rather amusing to see which of the speakers were big enough geeks to bring their laptops outside when we were all evacuated.... :-)


Conferences

Thursday, June 15, 2006 1:09:20 AM (Pacific Daylight Time, UTC-07:00)
Comments [5]  | 
 Saturday, May 06, 2006
Can the CLR "go dynamic"? Absolutely... and arguably, already is

Larry O'Brien asks

Are you confident that continuations can be even semi-efficiently implemented on the CLR? I'm not.
and in turn references his blog, where he points out a quote from Patrick Logan that says "If Microsoft really looks at Ruby as competition then Microsoft has already lost the war" and offers this:
  1. If Microsoft thinks Ruby is important, they're ignoring the threat to them posed by X (where, I suspect, X = LISP), or
  2. If Microsoft thinks Ruby is competition, they will not implement it and therefore be doomed
Not long ago, Microsoft posted a job opening for a developer "first task will be to drive the exploration of other dynamic languages such as Ruby and JavaScript on the CLR", so my feeling is that if Microsoft could get a Ruby on the CLR, they'd be thrilled.
First of all, said job has already been filled--Jim Hugunin, of Jython fame, joined Microsoft some months ago on the CLR team and has since pushed a 1.0 beta of his IronPython implementation (which, according to Python benchmarks, is already faster than the corresponding native C Python implementation), available from Microsoft. Second of all, I won't suggest that I know what Mr. Logan was thinking when he made his comment, but I suspect he's thinking more about development process than technological issues. What's more, I don't agree with the comment at all: I think Microsoft needs to pursue high-level "scripting" languages on the CLR, if only because they ARE more productive than statically-typed languages like C#/Java/C++; this is the lesson we forgot, and inadvertently abandoned, from VB. Which leads me to suggest that Ruby is the VB of the next decade. Or, if not Ruby, then something like it.

Larry goes on to say:

Ruby is not easy to implement on the CLR, at least in part because a complete Ruby implementation requires continuations, which are not modeled within the CLR. This isn't just laziness on the part of langauge implementors. The CLR presents a machine architecture different than the wide-open architecture in which most compiler experience has been gathered. The CLR architecture is safer, but more restrictive, when it comes to manipulating the stack, which is central to continuations.
Ruby actually requires more in the way of support than just continuations, but it's not necessarily impossible to implement on the CLR; it's just hard to implement on the CLR in a high-performing manner using today's CLR. That's part of what Jim is there to do, evolve the CLR to better support languages with Ruby's interesting featureset (like open classes and the "missing_method" method) in such a way that it doesn't tear down perf.

Continuations are not impossible to support, however they are currently more or less impossible to support given the current lack of access to the underlying stack frames in the managed environment--you'd need some support from the runtimes (either the JVM or the CLR) to make it work. Such runtime support would not be too difficult to add, however, as both environments already have rich and powerful stack-walking mechanisms (because both environments use the thread stack as bookkeeping tools, among other things, and need to be able to crawl through those stack markers for a variety of reasons, such as security checks), and it would not be hard to create a runtime-level mechanism that allowed code to "take a snapshot" of the stack--and its related object graph--from a certain point to a certain point, and save off that state to some arbitrary location. In many respects, it would be similar to serializing an object, I believe. In fact, we could imagine something along the lines of:

// All this is totally C#-like pseudocode. Imagine 
// something similar for Java if you like. 
// 
public int ContinuationedMethod() { 
  SnapshotMarker sm = new SnapshotMarker(); 
    // in other words, the StackSnapshot will only crawl 
    // back to the SnapshotMarker referenced when we 
    // take the thread's snapshot; this way we don't crawl 
    // all the way back to the Thread's starting point (unless 
    // you really wanted to). 
  int x = 1; 
  for (int i=0; i<10; i++) { 
    x = x * i; 
    if (i == 5) { 
      StackSnapshot ss = Thread.Current.TakeSnapshot(sm); 
        // At this point, the managed stack is walked, heap-referenced 
        // objects are captured, the instruction label that we're on is 
        // saved, and a StackSnapshot is allocated and returned. 
        // However, when ss is later rehydrated--using, say, ss.Resume(), 
        // we need some way of knowing that. So, following the lead of the 
        // old Unix fork() call, I presume that a "null" return value from 
        // TakeSnapshot is our way of knowing that we are resuming. 
        // 
      if (ss == null) 
        continue; 
      else 
      {
        // store ss someplace for later retrieval and return, either by 
        // throwing an exception if you like or just plain-old-"return 0"
        // or something
        //
      }
    } 
  } 
  return x; 
}
This is the API that I cooked up in all of thirty seconds, but hopefully you get the idea--it would be difficult to do from outside the runtime, as the many exception-trace stack-frame approaches suggest.

In the end, continuations are not, I believe, nearly as hard to implement--on either the JVM or the CLR--as some might suggest. Had I the money, I would go off and build the necessary Ruby-esque features we'd want into the CLR (through Rotor) or the JVM (through... uh... the JCL source, I guess, though the licensing there bothers me) for use. Anybody got some cash laying around to cover my mortgage while I do this? :-)


.NET | C++ | Java/J2EE | Ruby | XML Services

Saturday, May 06, 2006 11:19:09 PM (Pacific Daylight Time, UTC-07:00)
Comments [10]  | 
Another podcast with me goes live...

The guys over at Software Engineering Radio asked me to do a podcast a few months back, and it's now live on their website. They were particularly interested in language and new language development, so we spent a fair amount of time talking about Scala, F#, LINQ, and other interesting language developments in the world of the JVM and CLR. Have a listen, if you like...


.NET | Java/J2EE | Ruby | XML Services

Saturday, May 06, 2006 10:21:04 PM (Pacific Daylight Time, UTC-07:00)
Comments [0]  | 
 Friday, May 05, 2006
More on "Monad vs Ruby"... which really wasn't supposed to be a "vs" at all...

A while back, I blogged how MonadWindows PowerShell can be used to do a lot of the things the Ruby advocates are saying is one of Ruby's biggest strengths, that of "scripting" and driving things from the REPL environment. Glenn Vanderburg jumped all over me, believing I was suggesting that this was some kind of contest by which Ruby was supposed to come out in the Negative Points Zone. Had that been my intent, I would heartily agree with his critique; unfortunately, that wasn't the point.

For a while now, people have been holding up Ruby as this "incredibly productive tool", with the implication that such productivity cannot be achieved on the platforms that are currently the standards among the industry--that is, the CLR and Java virtual machine. Dynamically-, weak- or non-typed languages, for example, are all the rage now because they mean we don't need to try and "fool" the compiler on a regular basis with typecasts, we can have things like closures and continuations, and so on. My point simply was to point out that such argument is FUD--we can have closures, continuations, and so on, in platforms like the CLR and JVM--it's just that the major languages of the day don't provide those features (yet).

The Ruby advocate may snicker at my splitting of hairs here--"OK, so your platform may support it, but your languages don't. I'd rather use a language and platform that does support these features, instead of waiting for somebody else to come along and give them to me." It's a fair question--why is this an important distinction? Because asking existing infrastructure and applications to switch to a new platform is almost impossible. Asking them to integrate with a new platform is painful. Asking developers--particularly those on the team who don't get the beauty of dynamic/weak/non-typed languages--to switch to an entirely new way of thinking is going to mean you're going to spend at least five years learning the "new way".

If we can get those features we want from languages like Ruby onto a platform that we've already standardized on gives us a best-of-both-worlds result. Those developers who are comfortable with statically-typed objects can stay with statically-typed objects. Those who want the more dynamic features of "scripting" languages can do so. We can then blend the two together, to form an interesting and seamless whole: "Give me my Rails, but let me call into a J2EE Connector implementation to talk to the mainframe while we're at it." Getting Ruby to run on the JVM or CLR would be a Very Good Thing. It would answer one of the principal criticisms of Rails, for example, that of the idea that it doesn't do well when dealing with Large Enterprise Things--if Rails could create a javax.transaction.XATransaction before it kicks into ActiveRecord code, for example, suddenly we have a two-phase commit possibility that includes not just databases but mainframes and messaging systems. And yes, people DO need those kinds of Large Enterprise Things in the real world sometimes. Would it not be a win for Ruby to be able to hook into those without having to write all that code themselves? :-)

As a particular footnote to the discussion, Lee Holmes (whose blog entry I quoted to start all this) points out that there is a more concise version of the MonadWindows PowerShell script that compares more favorably (in terms of the "lines of code" metric, which, as Glenn notes, I don't really put much stock into) to the Ruby version. But that's not the point--the point is, there *is* a way to do Ruby-esque things in the Windows world, even if you choose not to learn the Ruby syntax. If that makes your life easier, hoo-hah, Sargeant! If you instead want to use JScript.NET and compile into IL, hoo-hah, Sargeant! If you choose to do Ruby, hoo-hah, Sargeant! Whatever makes life easier for you to Get The Work Done.

But don't underestimate the costs of integration, and certainly don't sacrifice integration on the altar of productivity--the time that you saved up front writing the thing will be more than spent when you try to make integration with the rest of your company's IT systems (or your new partner's IT systems, or your new consortium's standards, or....) work. Integration has been, is now, and will remain the Number One challenge to companies today, and that's not going to change in the future, Ruby or no Ruby. :-)


.NET | Java/J2EE | Ruby | XML Services

Friday, May 05, 2006 11:03:12 AM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
 Thursday, April 27, 2006
TheServerSide releases a TechTalk with yours truly

Best link to it is from their TechTalk page; the link to the actual stream itself redirects several times....

Some of you've been asking, where the heck have I been lately? Short story--I've been insanely busy. Long story--I've been insanely busy with a bunch of conferences and on-the-road consulting, not to mention the various writing projects I have afloat. Fear not, the blogging will resume shortly....


Java/J2EE

Thursday, April 27, 2006 11:29:19 PM (Pacific Daylight Time, UTC-07:00)
Comments [0]  | 
 Monday, April 03, 2006
Nice to know I'm missed

My son misses me terribly... or, apparently, at least he misses certain parts of me more than others, according to Charlotte's blog... (Warning, only for people who want personal life details. ;-) )




Monday, April 03, 2006 12:32:39 AM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
 Sunday, April 02, 2006
Javapolis interview with me is now up and available

The Javapolis folks were kind (deluded? crazy? you pick the word) enough to put the interview of me that Dion did at the show online. Have a listen....


.NET | C++ | Java/J2EE | Ruby | XML Services

Sunday, April 02, 2006 10:32:00 PM (Pacific Daylight Time, UTC-07:00)
Comments [1]  | 
 Friday, March 31, 2006
Need Ruby? Nah--get Monad instead

I happened across this blog entry while doing some research on Monad, Microsoft's new command shell (meaning, think "bash" or "csh", not "Explorer"), and found it so similar in many ways to what guys in the Ruby space have been hyping for a year or two now, that I just had to pass it on. Reproducing directly from the site:

One of the scripts I like the most in my toolbox is the one that gives me answers to questions from the command line.

For the past 2 years or so, Encarta has offered an extremely useful Instant Answers feature. Its since been integrated into MSN Search, as well as a wildly popular Chat Bot. MoW showed how to use that feature through a Monad IM interface (via the Conversagent bot,) but we can do a great job with good ol screen scraping.

[C:\temp]
MSH:70 > get-answer "What is the population of China?"

Answer: China: Population, total: 1,313,973,700
2006 estimate
United States Census International Programs Center


Answer: China : Population:
More than 20 percent of the worldâ?Ts population lives in China. Of the co
untryâ?Ts inhabitants, 92 percent are ethnic Han Chinese. The Han are desc
endants...

[C:\temp]
MSH:71 > get-answer "5^(e^(x^2))=50"

Answer: 5^(e ^( x^2))=50 = x=0.942428  x=-0.942428

[C:\temp]
MSH:72 > get-answer "define: canadian bacon"

Definition: Canadian bacon lean bacon

[C:\temp]
MSH:73 > get-answer "How many calories in an Apple?"

Answer: Apples: calories:
1.0 cup, quartered or chopped has 65 calories 1.0 NLEA serving has 80 calo
ries 1.0 small (2-1/2" dia) (approx 4 per lb) has 55 calories 1.0 medium (
2-3/4" dia) (approx 3 per lb) has 72 calories 1.0 large (3-1/4" dia) (appr
ox 2 per lb) has 110 calories 1.0 cup slices has 57 calories
USDA

[C:\temp]
MSH:74 > get-answer "How many inches in a light year?"

Answer: 1 lightyear = 372,461,748,226,857,000 inches
Here is the script, should you require your own command-line oracle:
## Get-Answer.msh 
## Use Encarta's Instant Answers to answer your question. 

param([string] $question = $(throw "Please ask a question.")) 

function Main 
{ 
   # Load the System.Web.HttpUtility DLL, to let us URLEncode 
   [void] [System.Reflection.Assembly]::LoadWithPartialName("System.Web") 

   ## Get the web page into a single string 
   $encoded = [System.Web.HttpUtility]::UrlEncode($question) 
   $text = get-webpage "http://search.msn.com/encarta/results.aspx?q=$encoded" 

   ## Get the answer with annotations 
   $startIndex = $text.IndexOf('<div id="results">') 
   $endIndex = $text.IndexOf('</div></div><h2>Results</h2>') 

   ## If we found a result, then filter the result 
   if(($startIndex -ge 0) -and ($endIndex -ge 0)) 
   { 
      $partialText = $text.Substring($startIndex, $endIndex - $startIndex) 
    
      ## Very fragile, voodoo screen scraping here 
      $regex = "<\s*a\s*[^>]*?href\s*=\s*[`"']*[^`"'>]+[^>]*>.*?</a>" 
      $partialText = [Regex]::Replace("$partialText", $regex, "") 
      $partialText = $partialText -replace "</div>", "`n" 
      $partialText = $partialText -replace "</span>", "`n" 
      $partialText = clean-html $partialText 
      $partialText = $partialText -replace "`n`n", "`n" 
     
      "" 
      $partialText.TrimEnd() 
   } 
   else 
   { 
      "" 
      "No answer found." 
   } 
} 

## Get a web page
function Get-WebPage ($url=$(throw "need to specify the URL to fetch")) 
{ 
    # canonicalize the url 
    if ($url -notmatch "^[a-z]+://") { $url = "http://$url" } 
     
    $wc = new-object System.Net.WebClient  
    $wc.Headers.Add("user-agent", $userAgent) 
    $wc.DownloadString($url) 
} 

## Clean HTML from a text chunk 
function Clean-Html ($htmlInput) 
{ 
    [Regex]::Replace($htmlInput, "<[^>]*>", "") 
} 

. Main
That's nifty stuff, if you ask me. And, what's best, this is a loosely-typed, dynamic language every bit as interesting and powerful as Ruby, though admittedly without some of the metaprogramming capabilities that Ruby has. But notice how we're making use of the vast power underneath the .NET framework to lay out a pretty straightforward use of the code, in a way that's entirely dynamic and loosely-typed, including the assumed return value from the if/else block, and so on. It's going to be a whole new world for automating projects (among other things) once Monad ships, and the saavy .NET developer (and even the saavy Java developer who builds on Windows) will already be looking at Monad for ways to streamline the things they need to do on Windows.

Added to my list of developer interview questions: "What is Monad, and why do I care?"


.NET | Ruby

Friday, March 31, 2006 7:20:34 PM (Pacific Daylight Time, UTC-07:00)
Comments [7]  | 
 Friday, March 24, 2006
Why programmers shouldn't fear offshoring

Recently, while engaging in my other passion (international relations), I was reading the latest issue of Foreign Affairs, and ran across an interesting essay regarding the increasing outsourcing--or, the term they introduce which I prefer in this case, "offshoring"--of technical work, and I found some interesting analysis there that I think solidifies why I think programmers shouldn't fear offshoring, but instead embrace it and ride the wave to a better life for both us and consumers. Permit me to explain.

The essay, entitled "Offshoring: The Next Industrial Revolution?" (by Alan S. Blinder), opens with an interesting point, made subtly, that offshoring (or "offshore outsourcing"), is really a natural economic consequence:

In February 2004, when N. Gregory Mankiw, a Harvard professor then serving as chairman of the White House Council of Economic Advisers, caused a national uproar with a "textbook" statement about trade, economists rushed to his defense. Mankiw was commenting on the phenomenon that has been clumsily dubbed "offshoring" (or "offshore outsourcing")--the migration of jobs, but not the people who perform them, from rich countries to poor ones. Offshoring, Mankiw said, is only "the latest manifestation of the gains from trade that economists have talked about at least since Adam Smith. ... More things are tradable than were tradable in the past, and that's a good thing." Although Democratic and Republican politicians alike excoriated Mankiw for his callous attitude toward American jobs, economists lined up to support his claim that offshoring is simply international business as usual.

Their economics were basically sound: the well-known principle of comparative advantage implies that trade in new kinds of products will bring overall improvements in productivity and well-being. But Mankiw and his defenders underestimated both the importance of offshoring and its disruptive effect on wealthy countries. Sometimes a quantitative change is so large that it brings qualitative changes, as offshoring likely will. We have so far barely seen the tip of the offshoring iceberg, the eventual dimensions of which may be staggering.

So far, you're not likely convinced that this is a good thing, and Blinder's article doesn't really offer much reassurance as you go on:

To be sure, the furor over Mankiw's remark was grotesquely out of proportion to the current importance of offshoring, which is still largely a prospective phenomenon. Although there are no reliable national data, fragmentary studies indicate that well under a million service-sector jobs have been lost to offshoring to date. (A million seems impressive, but in the gigantic and rapidly churning U.S. labor market, a million jobs is less than two weeks' worth of normal gross job losses.)1 However, constant improvements in technology and global communications will bring much more offshoring of "impersonal services"--that is, services that can be delivered electronically over long distances, with little or no degradation in quality.

That said, we should not view the coming wave of offshoring as an impending catastrophe. Nor should we try to stop it. The normal gains from trade mean that the world as a whole cannot lose from increases in productivity, and the United States and other industrial countries have not only weathered but also benefited from comparable changes in the past. But in order to do so again, the governments and societies of the developed world must face up to the massive, complex, and multifaceted challenges that offshoring will bring. National data systems, trade policies, educational systems, social welfare programs, and politics must all adapt to new realities. Unfortunately, none of this is happening now.

Phrases like "the world cannot lose from increases in productivity" are hardly comforting to programmers who are concerned about their jobs, and hearing "nor should we try to stop" the impending wave of offshoring is not what most programmers want to hear. But there's an interesting analytical point that I think Blinder misses about the software industry, and in order to make the point I have to walk through his argument a bit to get to it. I'm not going to quote the entirety of the article to you, don't worry, but I do have to walk through a few points to get there. Bear with me, it's worth the ride, I think.

Why Offshoring

Blinder first describes the basics of "comparative advantage" and why it's important in this context:

Countries trade with one another for the same reasons that individuals, businesses and regions do: to exploit their comparative advantages. Some advantages are "natural": Texas and Saudi Arabia sit atop massive deposits of oil that are entirely lacking in New York and Japan, and nature has conspired to make Hawaii a more attractive tourist destination than Greenland. Ther eis not much anyone can do about such natural advantages.

But in modern economics, nature's whimsy is far less important than it was in the past. Today, much comparative advantage derives from human effort rather than natural conditions. The concentration of computer companies around Silicon Valley, for example, has nothing to do with bountiful natural deposits of silicon; it has to do with Xerox's fabled Palo Alto Research Center, the proximity of Stanford University, and the arrival of two young men named Hewlett and Packard. Silicon Valley could have sprouted up anywhere.

One important aspect of this modern reality is that patterns of man-made comparative advantage can and do change over time. The economist Jagdish Bhagwait has labeled this phenomenon "kaleidoscopic comparative advantage", and it is critical to understanding offshoring. Once upon a time, the United Kingdom had a comparative advantage in textile manufacturing. Then that advantage shifted to New England, and so jobs were moved from the United Kingdom to the United States.2 Then the comparative advantage in textile manufacturing shifted once again--this time to the Carolinas--and jobs migrated south within the United States.3 Now the comparative advantage in textile manufacturing resides in China and other low-wage countries, and what many are wont to call "American jobs" have been moved there as a result.

Of course, not everything can be traded across long distances. At any point in time, the available technology--especially in transportation and communications4--largely determines what can be traded internationally and what cannot. Economic theorists accordingly divide the world's goods and services into two bins: tradable and non-tradable. Traditionally, any item that could be put in a box and shipped (roughly, manufactured goods) was considered tradable, and anything that could not be put into a box (such as services) or was too heavy to ship (such as houses) was thought of as nontradable. But because technology is always improving and transportation is becoming cheaper and easier, the boundary between what is tradable and what is not is constantly shifting. And unlike comparative advantage, this change is not kaleidoscopic; it moves in only one direction, with more and more items becoming tradable.

The old assumption that if you cannot put it in a box, you cannot trade it is thus hopelessly obsolete. Because packets of digitized information play the role that boxes used to play, many more services are now tradable and many more will surely become so. In the future, and to a great extent already, the key distinction will no longer be between things that can be put in a box and things that cannot. Rather, it will be between services that can be delivered electronically and those that cannot.

Blinder goes on to describe the three industrial revolutions, the first being the one we all learned in school, coming at the end of the 18th century and marked by Adam Smith's The Wealth of Nations in 1776. It was a massive shift in the economic system, as workers in industrializing countries migrated from farm to factory. "It has been estimated that in 1810, 84 percent of the U.S. work force was engaged in agriculture, compared to a paltry 3 percent in manufacturing. By 1960, manufacturing's share had rised to almost 25 percent and agriculture's had dwindled to just 8 percent. (Today, agriculture's share is under 2 percent.)" (This statistic is important, by the way--keep it in mind as we go.) He goes on to point out the second Revolution, the shift from manufacturing to services:

Then came the second Industrial Revolution, and jobs shifted once again--this time away from manufacturing and toward services. The shift to services is still viewed with alarm in the United States and other rich countries, where people bemoan rather than welcome the resulting loss of manufacturing jobs5. But in reality, new service-sector jobs have been created far more rapidly than old manufacturing jobs have disappeared. In 1960, about 35 percent of nonagricultural workers in the United States produced goods and 65 percent produced services. By 2004, only about one-sixth of the United States' nonagricultural jobs were in goods-producing industries, while five-sixths produced services. This trend is worldwide and continuing.

It's also important to point out that the years from 1960 to 2004 saw a meteoric rise in the average standard of living for the United States, on a scale that's basically unheard of in history. In fact, it was SUCH a huge rise that it became an expectation that your children would live better than you did, and the inability to keep that basic expectation in place (which has become a core part of the so-called "American Dream") that creates major societal angst on the part of the United States today.

We are now i nthe arly stages of a third Industrial Revolution--the information age. The cheap and easy flow of information around the globe has vastly expanded the scope of tradable services, and there is much more to come. Industrial revolutions are big deals. And just like the previous two, the third Industrial Revolution will require vast and usettling adjustments in the way Americans and residents of other developed countries work, live, and educate their children.

Wow, nothing unsettles people more than statements like "the world you know will cease to exist" and the like. But think about this for a second: despite the basic "growing pains" that accompanied the transitions themselves, on just about every quantifiable scale imaginable, we live a much better life today than our forebears did just two hundred years ago, and orders of magnitude better than our forebears did three hundred or more years ago (before the first Industrial Revolution). And if you still hearken back to the days of the "American farmer" with some kind of nostalgia, you never worked on a farm. Trust me on this.

So what does this mean?

But now we start to come to the interesting part of the article.

But a bit of historical perspective should help temper fears of offshoring. The first Industrial Revolution did not spell the end of agriculture, or even the end of food production, in the United States. It jus tmean that a much smaller percentage of Americans had to work on farms to feed the population. (By charming historical coincidence, the actual number of Americans working on farms today--around 2 million--is about what it was in 1810.) The main reason for this shift was not foreign trade, but soaring farm productivity. And most important, the massive movement of labor off the farms did not result in mass unemployment. Rather, it led to a large-scale reallocation of labor to factories.
Here's where we get to the "hole" in the argument. Most readers will read that paragraph, do the simple per-capita math, and conclude that thanks to soaring productivity gains in the programming industry (cite whatever technology you want here--Ruby, objects, hardware gains, it really doesn't matter what), the percentage of programmers in the country is about to fall into a black hole. After all, if we can go from 84 percent of the population involved in agriculture to less than 2% or so, thanks to that soaring productivity, why wouldn't it happen here again?

Therein lies the flaw in the argument: the amount of productivity required to achieve the desired ends is constant in the agriculture industry, yet a constantly-changing dynamic value in software. This is also known as what I will posit as the Groves-Gates Maxim: "What Andy Groves giveth, Bill Gates taketh away."

The Groves-Gates Maxim

The argument here is simple: the process of growing food is a pretty constant one: put seed in ground, wait until it comes up, then harvest the results and wait until next year to start again. Although we might have numerous tools that can help make it easier to put seeds into the ground, or harvesting the results, or even helping to increase the yield of the crop when it comes up, the basic amount of productivity required is pretty much constant. (My cousin, the FFA Farmer of the Year from some years back and a seed hybrid researcher in Iowa might disagree with me, mind you.) Compare this with the software industry: the basic differences between what's an acceptable application to our users today, compared to even ten years ago, is an order of magnitude different. Gains in productivity have not yielded the ability to build applications faster and faster, but instead have created a situation where users and managers ask more of us with each successive application.

The Groves-Gates Maxim is an example of that: every time Intel (where Andy Groves is CEO) releases new hardware that accelerates the power and potential of what the "average" computer (meaning, priced at somewhere between $1500-$2000) is capable of, it seems that Microsoft (Mr. Gates' little firm) releases a new version of Windows that sucks up that power by providing a spiffier user interface and "eye-candy" features, be they useful/important or not. In other words, the more the hardware creates possibilities, the more software is created to exploit and explore those possibilities. The additional productivity is spent not in reducing the time required to produce the thing desired (food in the case of agriculture, an operating system or other non-trivial program in the case of software), but in the expansion of the functionality of the product.

This basic fact, the Groves-Gates Maxim, is what saves us from the bloody axe of forced migration. Because what's expected of software is constantly on the same meteoric rise as what productivity gains provide us, the need for programmer time remains pretty close to constant. Now, once the desire for exponentially complicated features starts to level off, the exponentially increasing gains in productivity will have the same effect as they did in the agricultural industry, and we will start seeing a migration of programmers into other, "personal service" industries (which are hard to offshore, as opposed to "impersonal service" industries which can be easily shipped overseas).

Implications

What does this mean for programmers? For starters, as Dave Thomas has already frequently pointed out on NFJS panels, programmers need to start finding ways to make their service a "personal service" position rather than an "impersonal service" one. Blinder points out that the services industry is facing a split down the middle along this distinction, and it's not necessarily a high-paying vs low-paying divide:

Many people blithely assume that the critical labor-market distinction is, and will remain, between highly educated (or highly-skilled) people and less-educated (or less-skilled) people--doctors versus call-center operators, for example. The supposed remedy for the rich countries, accordingly, is more education and a general "upskilling" of the work force. But this view may be mistaken. Other things being equal, education and skills are, of course, good things; education yields higher returns in advanced societies, and more schooling probably makes workers more flexible and more adaptable to change. But the problem with relying on education as the remedy for potential job losses is that "other things" are not remotely close to equal. The critical divide in the future may instead be between those types are work that are easily deliverable through a wire (or via wireless connections) with little or no diminution in quality and those that are not. And this unconventional divide does not correspond well to traditional distinctions between jobs that require high levels of education and jobs that do not.

A few disparate examples will illustrate just how complex--or, rather, how untraditional--the new divide is. It is unlikely that the services of either taxi drivers or airline pilots will ever be delivered electronically over long distances. The first is a "bad job" with negligible educational requirements; the second is quite the reverse. On the other hand, typing services (a low-skill job) and security analysis (a high-skill job) are already being delivered electronically from India--albeit on a small scale so far. Most physicians need not fear that their jobs will be moved offshore, but radiologists are beginning to see this happening already. Police officers will not be replaced by electronic monitoring, but some security guards will be. Janitors and crane operators are probably immune to foreign competition; accountants and computer programmers are not. In short, the dividing line between the jobs that produce services that are suitable for electronic delivery (and are thus threatened by offshoring) and those that do not does not correspond to traditional distinctions between high-end and low-end work.

What's the implications here for somebody deep in our industry? Pay close attention to Blinder's conclusion, that computer programmers are highly vulnerable to foreign competition, based on the assumption that the product we deliver is easily transferable across electronic media. But there is hope:

There is currently not even a vocabulary, much less any systematic data, to help society come to grips with the coming labor-market reality. So here is some suggested nomenclature. Service that cannot be delivered electronically, or that are notably inferior when so delivered, have one essential characteristic: personal, face-to-face contact is either imperative or highly desirable. Think of hte waiter who serves you dinner, the doctor you gives you your annual physical, or the cop on the beat. Now think of any of those tasks being performed by robots controlled from India--not quite the same. But such face-to-face human contact is not necessary in the relationship you have with the telephone operator who arranges your conference call or the clerk who takes your airline reservation over the phone. He or she may be in India already.

The first group of tasks can be called personally-delivered services, or simply personal services, and the second group of impersonally delivered services, or impersonal services. In the brave new world of globalized electronic commerce, impersonal services have more in common with manufactured goods that can be put in boxes than they do with personal services. Thus, many impersonal services are destined to become tradable and therefore vulnerable to offshoring. By contrast, most personal services have attributes that cannot be transmitted through a wire. Some require face-to-face contact (child care), some are inherently "high-risk" (nursing), some involve high levels of personal trust (psychotherapy), and some depend on location-specific attributes (lobbying).

In other words, programmers that want to remain less vulnerable to foreign competition need to find ways to stress the personal, face-to-face contact between themselves and their clients, regardless of whether you are a full-time employee of a company, a contractor, or a consultant (or part of a team of consultants) working on a project for a client. Look for ways to maximize the four cardinalities he points out:
  • Face-to-face contact. Agile methodologies demand that customers be easily accessible in order to answer questions regarding implementation decisions or to resolve lack of understanding of the requirements. Instead of demanding customers be present at your site, you may find yourself in a better position if you put yourself close to your customers.
  • "High-risk". This is a bit harder to do with software projects--either the project is inherently high-risk in its makeup (perhaps this is a mission-critical app that the company depends on, such as the e-commerce portal for an online e-tailer), or it's not. There's not much you can do to change this, unless you are politically savvy enough to "sell" your project to a group that would make it mission-critical.
  • High levels of personal trust. This is actually easier than you might think--trust in this case refers not to the privileged nature of therapist-patient communication, but in the credibility the organization has in you to carry out the work required. One way to build this trust is to understand the business domain of the client, rather than remaining aloof and "staying focused on the technology". This trust-based approach is already present in a variety of forms outside our industry--regardless of the statistical ratings that might be available, most people find that they have a favorite auto repair mechanic or shop not for any quantitatively-measurable reason, but beceause the mechanic "understands" them somehow. The best customer-service shops understand this, and have done so for years. The restaurant that recognizes me as a regular after just a few visits and has my Diet Coke ready for me at my favorite table is far likelier to get my business on a regular basis than the one that never does. Learn your customers, learn their concerns, learn their business model and business plan, and get yourself into the habit of trying to predict what they might need next--not so you can build it already, but so that you can demonstrate to them that you understand them, and by extension, their needs.
  • Location-specific attributes. Sometimes, the software being built is localized to a particular geographic area, and simply being in that same area can yield significant benefits, particularly when heroic efforts are called for. (It's very hard to flip the reset switch on a server in Indiana from a console in India, for example.)
In general, what you're looking to do is demonstrate how your value to the company arises out of more than just your technical skill, but also some other qualities that you can provide in better and more valuable form than somebody in India (or China, or Brazil, or across the country for that matter, wherever the offshoring goes). It's not a guarantee that you might still be offshored--some management folks will just see bottom-line dollars and not recognize the intangible value-add that high levels of personal trust or locality provides--but it'll minimize it on the whole.

But even if this analysis doesn't make you feel a little more comfortable, consider this: there are 1 billion people in China alone, and close to the same in India. Instead of seeing them as potential competition, imagine what happens when the wages from the offshored jobs start to create a demand for goods and services in those countries--if you think the software market in the U.S. was hot a decade ago, where only a half-billion (across both the U.S. and Europe) people were demanding software, now think about it when four times that many start looking for it.


Footnotes

1 Which in of itself is an interesting statistic--it implies that offshoring is far less prevalent than some of people worried about it believe it to be, including me.

2 Interesting bit of trivia--part of the reason that advantage shifted was because the US stole (yes, stole, as in industrial espionage, one of the first recorded cases of modern industrial espionage) the plans for modern textile machinery from the UK. Remember that, next time you get upset at China's rather loose grip of intellectual property law....

3 Which, by the way, was a large part of the reason we fought the Civil War (the "War Between the States" to some, or the "War of Northern Aggression" to others)--the Carolinas depended on slave labor to pick their cotton cheaply, and couldn't acquire Northern-made machinery cheaply to replace the slaves. Hence, for that (and a lot of other reasons), war followed.

4 An interesting argument--is there any real difference between transportation and communications? One ships "stuff", the other "data", beyond that, is there any difference?

5 And, I'd like to point out, the shrinking environmental damage that can arise from a manufacturing-based economy. Services rarely generate pollution, which is part of the clash between the industrialized "Western" nations and the developing "Southern" ones over environmental issues.

Resources

"Offshoring: The Next Industrial Revolutoin?", by Alan S. Blinder, Foreign Affairs (March/April 2006), pp 113 - 128.


.NET | C++ | Development Processes | Java/J2EE | Reading | Ruby | XML Services

Friday, March 24, 2006 2:43:00 AM (Pacific Daylight Time, UTC-07:00)
Comments [6]  | 
 Monday, March 20, 2006
Take a non-technical moment and support the fight against diabetes

Scott Hanselman has diabetes. So does a good friend of mine, who discovered it during our third year of college. Take a moment and spare US$20 to support Scott in his fight against it. Do it because you probably know somebody who has it, or will before long. If nothing else, do it because it's a cheap way to support somebody who's indirectly responsible for this blog (Scott maintains dasBlog, which is the blogging engine I use now).

Out.




Monday, March 20, 2006 11:10:22 PM (Pacific Daylight Time, UTC-07:00)
Comments [0]  | 
 Sunday, March 19, 2006
Another annoying nit in Java, fixed

CLASSPATH now supports wildcards to pick up multiple .jar files. Finally. But given that the AppClassLoader is a derivative of the standard UrlClassLoader, I still don't see why I can't put full URLs on the CLASSPATH and expect them to be resolved correctly....

Oh, and while we're at it, you shouldn't be using CLASSPATH-the-environment variable anymore, anyway. There's far too many ways to manage .jar file resolution to be falling back to that old hack. Prefer instead to put your .jar file dependencies inside the manifest of your application's .jar file, or at the very least, specify the classpath to the java launcher when you kick it off. If you're still doing "set CLASSPATH=..." at the command-line, you're about ten years behind the times.


Java/J2EE

Sunday, March 19, 2006 2:11:35 AM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
At last, a minor but annoying nit in Java, fixed

Mustang, the latest JDK (to be called Java6 on its release), fixes a minor but very annoying nit that's bugged Java developers for years: "if a JDBC driver is packaged as a service, you can simply (leave out the call to Class.forName() to bootstrap the driver class into the JVM). ... The DriverManager code will look for implementations of java.sql.Driver in classpath and do Class.forName() implicitly." It's never been a huge deal in Java, to have to explicitly bootstrap the JDBC driver into the JVM before being able to obtain a Connection from it, but it's always been annoying, and inexplicable, given the Service Provider mechanism that's been there for a couple of releases now.

Next up: do the same for JNDI and XML parsers (and do away with the extensions directory while we're at it!), and let's start making all of these factories a bit more practical to use on a larger basis. It would be nice if this mechanism had percolated through other tools and areas, such as servlets, but it's probably too late to correct for that now.


Java/J2EE

Sunday, March 19, 2006 2:07:44 AM (Pacific Daylight Time, UTC-07:00)
Comments [0]  | 
 Friday, March 17, 2006
XP on an Intel Mac...

From the "I'm not sure why you'd want to do this, but..." Department: How to install Windows XP onto your brand new Intel Mac Pro. (Given that so much of the Windows O/S relies on the right-mouse-button, and given that it'd be so much cheaper to buy a Dell or even a Thinkpad of equivalent power... why would you bother to buy a Mac Pro, then turn around and install--or even dual-boot install--XP on it? Maybe I'm just not "getting it" or something....)

Needless to say, despite the hefty geekiness factor inherent in the idea of doing .NET demos with a Mac Pro, I'll probably not be buying one any time soon... Instead, somebody point me to Tiger running in a VMWare image and then stand back. :-)




Friday, March 17, 2006 3:20:08 AM (Pacific Daylight Time, UTC-07:00)
Comments [2]  | 
 Tuesday, March 14, 2006
Bruce Tate, this time on moving from Java to Ruby

Bruce Tate is at it again, this time writing for the Pragmatic Press, called "Java to Ruby", on... well, the title kinda says it all, on migrating from Java to Ruby. It's not just a "blind adoption" book, meaning it just blindly advocates the transition, but instead Bruce discusses the risks involved with making the switch, and how to justify it to upper management. Don't get me wrong, he's operating from the basic conclusion that you want to make the switch, so if you're not yet convinced that Ruby or Ruby-on-Rails is the way to go, you're not necessarily going to be any more convinced by this book. But if you are already convinced (and in Bruce's defense, there's lots of good reasons to be), this looks to be the book to help you convince others, most notably your management.


Java/J2EE | Ruby

Tuesday, March 14, 2006 12:07:20 AM (Pacific Daylight Time, UTC-07:00)
Comments [1]  | 
 Saturday, March 11, 2006
Sample programmers' quiz

While training last week, the group I was training asked for some help in interviewing candidates for some openings. I came up with the following, and thought I'd post it in the interests of giving teams looking to hire some new folks. This was created specifically to find candidates with 2-3 years' experience with some familiarity with web applications.

  • What was the most interesting computer/technical book you read last year? Why?
  • What was the last new programming language you learned?
  • How do you convince a customer that what they want is wrong?
  • Describe the last elegant hack you wrote. Why, where, how, when, ...?
  • Write a servlet that prints all parameters and HTTP headers to the browser. Build an Ant script that compiles, and creates the appropriate .war file for deployment into Tomcat.
  • Name all of the verbs in the HTTP/1.1 specification.
  • Given a tag library, "utilitytags.jar" which is described by the tag library descriptor file "utilitytags.tld", please write a JSP that uses the "foobar" tag from that library.
  • What keyword in Java do we use to grant package-level access to class members?
  • When do you think you should use an abstract base class as opposed to using an interface?
  • How do I protect code in a multithreaded environment from being executed by more than one thread at a time?
  • Please fill in the necessary code to return the contents of the "last name" column using the following SQL statement; make all necessary changes so that this code will compile:
    public void printUsers(Connection conn)
    {
      String sql = "SELECT last_name FROM users WHERE first_name = 'Sean'";
      ...
    }
    
  • What is a design pattern? Please describe three patterns that you have used and why you used them and what the consequences of using them were, both good and bad.
More suggestions, questions, and comments welcome. Note that some of the questions are deliberate traps, some of them are deliberately open-ended in order to encourage discussion and opportunities to flesh out what's on the resume, and some of them are intended not to hear the answers but to watch the candidates' reaction. (Yeah, I'm a hard interviewer. :-) )


Update: A couple of commenters have pointed out that a few of the questions are answered by simply looking up stuff (in the HTTP specification, for example). Two answers come to mind why I want people to know this without having to look it up--one, sometimes I want to pitch a slowball just to see how they'll react and answer, and two, if they can answer the question without having to look it up, it means they know the spec, which is a far, far different thing than being able to look something up.


Java/J2EE

Saturday, March 11, 2006 10:27:20 PM (Pacific Standard Time, UTC-07:00)
Comments [6]  | 
My kingdom for a good macro language!

Much of the power, it seems, of languages like Ruby or Nemerle or LISP derives from the ability to create chunks of code that operate in turn on the code itself; a long-standing meme of the LISP world is that "code is data", meaning it can be manipulated and twisted and tweaked in structural ways before being executed. And this isn't the first time we've experimented with this idea: CLOS, Common List Object System, was where Gregor Kiczales, of AspectJ fame, cut his teeth on the AOP concepts, largely because it seemed to him that having a completely open meta-object protocol was too dangerous--but that's another story.

So given that everybody's going ga-ga over Ruby's MOP/metaprogramming facilities, it occurs to me, is what we're really after an MOP for Java? Because such things exist already in the academic world--see OpenJava, for example. Is that what Java would need to do to evolve and take back the productivity label? Is the lack of productivity in Java (the chief complaint of Java today, according to Bruce Tate) due directly to its statically-typed nature, or is it simply the inability to twist the language in ways that are closer to what the programming staff really needs and wants? After all, if you take away some of the MOP features that Ruby uses