Saturday, May 6, 2006
Can the CLR "go dynamic"? Absolutely... and arguably, already is

Larry O'Brien asks

Are you confident that continuations can be even semi-efficiently implemented on the CLR? I'm not.
and in turn references his blog, where he points out a quote from Patrick Logan that says "If Microsoft really looks at Ruby as competition then Microsoft has already lost the war" and offers this:
  1. If Microsoft thinks Ruby is important, they're ignoring the threat to them posed by X (where, I suspect, X = LISP), or
  2. If Microsoft thinks Ruby is competition, they will not implement it and therefore be doomed
Not long ago, Microsoft posted a job opening for a developer "first task will be to drive the exploration of other dynamic languages such as Ruby and JavaScript on the CLR", so my feeling is that if Microsoft could get a Ruby on the CLR, they'd be thrilled.
First of all, said job has already been filled--Jim Hugunin, of Jython fame, joined Microsoft some months ago on the CLR team and has since pushed a 1.0 beta of his IronPython implementation (which, according to Python benchmarks, is already faster than the corresponding native C Python implementation), available from Microsoft. Second of all, I won't suggest that I know what Mr. Logan was thinking when he made his comment, but I suspect he's thinking more about development process than technological issues. What's more, I don't agree with the comment at all: I think Microsoft needs to pursue high-level "scripting" languages on the CLR, if only because they ARE more productive than statically-typed languages like C#/Java/C++; this is the lesson we forgot, and inadvertently abandoned, from VB. Which leads me to suggest that Ruby is the VB of the next decade. Or, if not Ruby, then something like it.

Larry goes on to say:

Ruby is not easy to implement on the CLR, at least in part because a complete Ruby implementation requires continuations, which are not modeled within the CLR. This isn't just laziness on the part of langauge implementors. The CLR presents a machine architecture different than the wide-open architecture in which most compiler experience has been gathered. The CLR architecture is safer, but more restrictive, when it comes to manipulating the stack, which is central to continuations.
Ruby actually requires more in the way of support than just continuations, but it's not necessarily impossible to implement on the CLR; it's just hard to implement on the CLR in a high-performing manner using today's CLR. That's part of what Jim is there to do, evolve the CLR to better support languages with Ruby's interesting featureset (like open classes and the "missing_method" method) in such a way that it doesn't tear down perf.

Continuations are not impossible to support, however they are currently more or less impossible to support given the current lack of access to the underlying stack frames in the managed environment--you'd need some support from the runtimes (either the JVM or the CLR) to make it work. Such runtime support would not be too difficult to add, however, as both environments already have rich and powerful stack-walking mechanisms (because both environments use the thread stack as bookkeeping tools, among other things, and need to be able to crawl through those stack markers for a variety of reasons, such as security checks), and it would not be hard to create a runtime-level mechanism that allowed code to "take a snapshot" of the stack--and its related object graph--from a certain point to a certain point, and save off that state to some arbitrary location. In many respects, it would be similar to serializing an object, I believe. In fact, we could imagine something along the lines of:

// All this is totally C#-like pseudocode. Imagine 
// something similar for Java if you like. 
public int ContinuationedMethod() { 
  SnapshotMarker sm = new SnapshotMarker(); 
    // in other words, the StackSnapshot will only crawl 
    // back to the SnapshotMarker referenced when we 
    // take the thread's snapshot; this way we don't crawl 
    // all the way back to the Thread's starting point (unless 
    // you really wanted to). 
  int x = 1; 
  for (int i=0; i<10; i++) { 
    x = x * i; 
    if (i == 5) { 
      StackSnapshot ss = Thread.Current.TakeSnapshot(sm); 
        // At this point, the managed stack is walked, heap-referenced 
        // objects are captured, the instruction label that we're on is 
        // saved, and a StackSnapshot is allocated and returned. 
        // However, when ss is later rehydrated--using, say, ss.Resume(), 
        // we need some way of knowing that. So, following the lead of the 
        // old Unix fork() call, I presume that a "null" return value from 
        // TakeSnapshot is our way of knowing that we are resuming. 
      if (ss == null) 
        // store ss someplace for later retrieval and return, either by 
        // throwing an exception if you like or just plain-old-"return 0"
        // or something
  return x; 
This is the API that I cooked up in all of thirty seconds, but hopefully you get the idea--it would be difficult to do from outside the runtime, as the many exception-trace stack-frame approaches suggest.

In the end, continuations are not, I believe, nearly as hard to implement--on either the JVM or the CLR--as some might suggest. Had I the money, I would go off and build the necessary Ruby-esque features we'd want into the CLR (through Rotor) or the JVM (through... uh... the JCL source, I guess, though the licensing there bothers me) for use. Anybody got some cash laying around to cover my mortgage while I do this? :-)

.NET | C++ | Java/J2EE | Ruby | XML Services

Saturday, May 6, 2006 10:19:09 PM (Pacific Standard Time, UTC-08:00)
Comments [10]  | 
Another podcast with me goes live...

The guys over at Software Engineering Radio asked me to do a podcast a few months back, and it's now live on their website. They were particularly interested in language and new language development, so we spent a fair amount of time talking about Scala, F#, LINQ, and other interesting language developments in the world of the JVM and CLR. Have a listen, if you like...

.NET | Java/J2EE | Ruby | XML Services

Saturday, May 6, 2006 9:21:04 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  | 
 Friday, May 5, 2006
More on "Monad vs Ruby"... which really wasn't supposed to be a "vs" at all...

A while back, I blogged how MonadWindows PowerShell can be used to do a lot of the things the Ruby advocates are saying is one of Ruby's biggest strengths, that of "scripting" and driving things from the REPL environment. Glenn Vanderburg jumped all over me, believing I was suggesting that this was some kind of contest by which Ruby was supposed to come out in the Negative Points Zone. Had that been my intent, I would heartily agree with his critique; unfortunately, that wasn't the point.

For a while now, people have been holding up Ruby as this "incredibly productive tool", with the implication that such productivity cannot be achieved on the platforms that are currently the standards among the industry--that is, the CLR and Java virtual machine. Dynamically-, weak- or non-typed languages, for example, are all the rage now because they mean we don't need to try and "fool" the compiler on a regular basis with typecasts, we can have things like closures and continuations, and so on. My point simply was to point out that such argument is FUD--we can have closures, continuations, and so on, in platforms like the CLR and JVM--it's just that the major languages of the day don't provide those features (yet).

The Ruby advocate may snicker at my splitting of hairs here--"OK, so your platform may support it, but your languages don't. I'd rather use a language and platform that does support these features, instead of waiting for somebody else to come along and give them to me." It's a fair question--why is this an important distinction? Because asking existing infrastructure and applications to switch to a new platform is almost impossible. Asking them to integrate with a new platform is painful. Asking developers--particularly those on the team who don't get the beauty of dynamic/weak/non-typed languages--to switch to an entirely new way of thinking is going to mean you're going to spend at least five years learning the "new way".

If we can get those features we want from languages like Ruby onto a platform that we've already standardized on gives us a best-of-both-worlds result. Those developers who are comfortable with statically-typed objects can stay with statically-typed objects. Those who want the more dynamic features of "scripting" languages can do so. We can then blend the two together, to form an interesting and seamless whole: "Give me my Rails, but let me call into a J2EE Connector implementation to talk to the mainframe while we're at it." Getting Ruby to run on the JVM or CLR would be a Very Good Thing. It would answer one of the principal criticisms of Rails, for example, that of the idea that it doesn't do well when dealing with Large Enterprise Things--if Rails could create a javax.transaction.XATransaction before it kicks into ActiveRecord code, for example, suddenly we have a two-phase commit possibility that includes not just databases but mainframes and messaging systems. And yes, people DO need those kinds of Large Enterprise Things in the real world sometimes. Would it not be a win for Ruby to be able to hook into those without having to write all that code themselves? :-)

As a particular footnote to the discussion, Lee Holmes (whose blog entry I quoted to start all this) points out that there is a more concise version of the MonadWindows PowerShell script that compares more favorably (in terms of the "lines of code" metric, which, as Glenn notes, I don't really put much stock into) to the Ruby version. But that's not the point--the point is, there *is* a way to do Ruby-esque things in the Windows world, even if you choose not to learn the Ruby syntax. If that makes your life easier, hoo-hah, Sargeant! If you instead want to use JScript.NET and compile into IL, hoo-hah, Sargeant! If you choose to do Ruby, hoo-hah, Sargeant! Whatever makes life easier for you to Get The Work Done.

But don't underestimate the costs of integration, and certainly don't sacrifice integration on the altar of productivity--the time that you saved up front writing the thing will be more than spent when you try to make integration with the rest of your company's IT systems (or your new partner's IT systems, or your new consortium's standards, or....) work. Integration has been, is now, and will remain the Number One challenge to companies today, and that's not going to change in the future, Ruby or no Ruby. :-)

.NET | Java/J2EE | Ruby | XML Services

Friday, May 5, 2006 10:03:12 AM (Pacific Standard Time, UTC-08:00)
Comments [2]  |