JOB REFERRALS
    ON THIS PAGE
    ARCHIVES
    CATEGORIES
    BLOGROLL
    LINKS
    SEARCH
    MY BOOKS
    DISCLAIMER
 
 Saturday, December 29, 2007
I Refused to be Terrorized

Bruce Schneier has a great blog post on this. I'm joining the movement, with this declaration:

I am not afraid of terrorism, and I want you to stop being afraid on my behalf. Please start scaling back the official government war on terror. Please replace it with a smaller, more focused anti-terrorist police effort in keeping with the rule of law. Please stop overreacting. I understand that it will not be possible to stop all terrorist acts. I accept that. I am not afraid.

In fact, I would amend this a little to include more than just the politically-correct discussion of terrorism and the government:

I am not afraid of security discussions, and I want you to stop being afraid on my behalf. Please start scaling back the draconian requirements on my passwords and connection options. Not everything has to run over HTTPS and require passwords that must be 12 characters long and contain an upper-case letter, a lower-case letter, a number, a punctuation mark, and a letter from the Klingon alphabet. Please replace it with a smaller, more focused security effort in keeping with the risk involved. Please stop overreacting. I understand that it will not be possible to stop all acts of security attack. I accept that. I am not afraid.

I want companies not to abandon their security efforts, but to put the effort into more targeted efforts. Don't spend millions instituting a VPN; instead, spend that time and money getting developers to find and fix all the command injection and/or cross-site scripting attacks that plague web applications.


.NET | C++ | Development Processes | Java/J2EE | Windows | XML Services

Saturday, December 29, 2007 4:46:57 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  | 
 Wednesday, December 26, 2007
2008 Predictions, 2007 Predictions Redux

As has become a blogs.tedneward.com tradition, 'tis time to offer up my predictions for the technical year 2008, and, more importantly, revisit my predictions for the technical year 2007 and see how close to the mark I was. (As usual, I'll simply offer up commentary and my own opinion--you're free to draw whatever conclusions you like about my abilities as a prognosticator.)

First, the predictions from 2007, in retrospect:

  • THEN: General: Analysts will call 2007 the Year of the {Something}, where I bet that {Something} will be either "ESB" or "SOA". They will predict that companies adopting {Something} will save millions, if not billions, if only they rush to implement it now. They will tag this with a probability of .8 in order to CYA in case {Something} doesn't pan out. (Yes, I've read far too many of these reports--I'm personally convinced that each of the analyst companies has a template buried away in their basement that they pull out each time they need a new one, and they just do a global search-and-replace of "{Something}" with whatever the technology du jour happens to be.) NOW: Wasn't this the Year of the ESB?
  • THEN: .NET: Thousands of developers will horribly abuse WPF in ways that can only be called nightmarish, thus once again proving the old adage that "just because you can doesn't mean you should" still holds. WPF's capabilities with video will prove, in many ways, to be the modern equivalent to the "blink" tag in HTML. This will provide some author with a golden opportunity: "WPF Applications That Suck". Alan Cooper will re-release "About Face", updated to include WPF UI elements. NOW: Well, it wasn't Alan Cooper, but Charles Petzold re-released his classics, updated for WPF. Fact is, WPF hasn't seemed to penetrate all that deeply into the programming world, so maybe this is better as a 2008 prediction, now that Visual Studio 2008 is in the hands of developers.
  • THEN: .NET: Thousands of developers will look to Redmond for an answer to the question, "Which should I use? BizTalk, Windows Workflow, or SQL Server Service Broker?", and get no clear answer. NOW: How many presentations at DevConnections, VSLive!, TechEd, and other .NET shows addressed this exact question? How much guidance from Microsoft was there on the subject? 'Nuff said.
  • THEN: Windows: Microsoft will try, once again, to kill off the abomination that was called the Windows 95/98/Me line of operating systems, and will once again have to back off as industry outcries of protest (on behalf of little old ladies who are the only ones left running Windows 95/98/Me and probably haven't turned their machine on in months, at least not since the grandkids last visited) go ballistic. NOW: Granny Schottenwilkins in East Duluth, Minnesota died last year, so maybe next year Microsoft will finally have enough momentum to kill Windows 16-bit.
  • THEN: Windows: Ditto for Visual Basic 6.0, except now the outcry will be on behalf of developers who aren't capable of learning anything new. Sun will use the resulting PR to announce Project YAVKRWMITT (Yet Another VB Killer Really We Mean It This Time, pronounced "YAV-kermit") on java.net. Meanwhile, efforts to make CLASSPATH into something a VB 6 guy actually has a prayer of understanding will go quietly ignored. NOW: Apparently Microsoft had enough problems getting Vista adoption going, so they didn't bother trying to kill VB 6 this year. Meanwhile, Sun continues to build NetBeans to attract developers from "across the fence", except now its Ruby developers. (Apparently somebody finally showed Simon Phipps what VB 6 code really looks like, and he made the prudent decision to leave it in Microsoft's hands....)
  • THEN: Java: JSR 277 will continue to churn along, and once the next draft ships, publicly nobody will like what we produce, though quietly everybody will admit it's a far cry better than what we have now, and when it ships in JDK 7 will be adopted widely and quietly. NOW: We shipped a straw man implementation, which so far as I can tell, nobody bothered to look at, probably because it was a source patch on top of the OpenJDK bits, which nobody else knew how to build....
  • THEN: Java: Thousands of new ideas and proposals to extend the Java language in various ways will flood into the community, now that developers can start hacking on it for themselves thanks to the OpenJDK. Only a small fraction of these will ever get beyond the concept stage, and maybe one or two will actually be finished and released to the Web for consideration by the community and the JCP. Thousands more Java developers craving Alpha-Geek status will stick a "Hello, world" message into the compiler's startup sequence, then claim "experienced with modifying the OpenJDK Java compiler" on their resume and roundly criticize Java in one way or another by saying, "Well, I've looked at the code, and let me tell you....". NOW: Numerous language-extension projects emerging on Sourceforge and java.net, none of which appear to have any code behind them... What's it say on your resume, by the way?
  • THEN: .NET: Somewhere, a developer will realize that SQL Server 2005 can be a SOAP/WSDL XML service endpoint, and open it up as a private back-channel for his application to communicate with the database through the firewall "for performance reasons" (meaning, "So I can avoid having to talk to the app server in between my web server and my database"). With any luck, the DBA will kill him and hide the body before anybody can find and exploit it. NOW: "Body of dead developer found underneath raised floor in server room. Story, at 11. In related news, WhizzBang Corp filed for bankruptcy, owing to the complete collapse of their stock after news of their entire customer base personal details database being discovered on the Web was made public...."
  • THEN: General: Yet Another Virus That's Microsoft's Fault will rip through the Internet, and nobody will notice that the machines affected are the ones that aren't routinely administered or receive updates/patches. Companies will threaten Microsoft with million-dollar lawsuits, yet will fire none of their system administrators who lovingly lavish whole days tuning their Linux IRC servers yet leave the Windows Exchange Server still running Windows NT 4.0. NOW: Surprisingly, this didn't happen. Either we're shockingly bland to the viruses ripping through the Internet now, or else the virus creators are falling behind the anti-virus defenders, or else the viruses have learned to lay low, waiting for the master signal to go off simultaneously.
  • THEN: General: Interest in JSON will escalate wildly, hyped as the "natural replacement for XML" in building browser-to-server connections, owing to its incredible simplicity in expressing "object" data. Folks, JSON is a useful format, but it's not a replacement for XML (nor is XML a replacement for it, either). What made XML so popular was not is hierarchical format (Lord above, that's probably the worst part of it, from where we as developers sit), nor its HTML-like simplified-SGML syntax. What made XML interesting was the fact that everybody lined up behind it--Microsoft, Sun, BEA, Oracle, IBM, there's not a big vendor that didn't express its undying love and devotion to XML. I sincerely doubt JSON will get that kind of rallying effect. (And if you're going to stand there and suggest that JSON is better because its simpler and therefore more approachable for developers to build support for themselves, quite honestly, I thought we were trying to get out of developers building all this communications infrastructure--isn't that what the app servers and such taught us?) NOW: Well, JSON continues to be of interest to those who are also hand-writing their JavaScript code on their web pages, which means that to the top-of-the-web-pyramid ubergeek, JSON is "clearly the right way to go", while the rest of the industry just doesn't give a damn so long as the code works.
  • THEN: General: Interest in Java/.NET interopability will rise as companies start to realize that (a) the WS-* "silver bullet" isn't, (b) ESB, XML, and SOA are just acronyms and won't, in of themselves, solve all the integration problems, and (c) we have lots of code in both Java and .NET that need to talk to each other. This may be a self-serving prediction, but I got a LOT of interest towards the end of this year in the subject, so I'm guessing that this is going to only get bigger as the WS-* hype continues to lose its shine in the coming years. NOW: It got bigger, then drained away. I guess either the world figured out their story here, or everybody's busy rewriting all their apps in Ruby.
  • THEN: Ruby: Interest in Java/Ruby and .NET/Ruby interoperability is going to start quietly making its presence felt, as people start trying to wire up their quick-to-write "stovepipe" RAILS apps against other systems in their production data center, and find that Ruby really is a platform of its own. RubyCLR or JRuby may be part of the answer here, but there's likely some hidden mines there we haven't seen yet. NOW: JRuby now runs faster than the native Ruby interpreter. IronRuby (the logical successor to RubyCLR) looks to do the same for the CLR side, plus we get a dynamic language runtime out of it. Ruby may be the best language running on somebody else's platform in 2008.
  • THEN: Languages: A new meme will get started: "JavaScript was that thing, that little toy language, that you used to do stuff in the HTML browser. ECMAScript, on the other hand, is a powerful and flexible dynamic programming language suitable for use in all sorts of situations." Pass it on. If you get it, don't tell anybody else. (Don't laugh--it worked for "The Crying Game".) It's the only way ECMAScript will gain widespread acceptance and shed the "toy" label that JavaScript has. NOW: And now ECMAScript 4 is coming, with a whole new set of features that make even the Ruby guys shiver in anticipation.
  • THEN: Languages: Interest in functional-object hybrid languages will grow. Scala, Jaskell, F#, and others not-yet-invented will start to capture developers' attention, particularly when they hear the part about functional languages being easier to use in multi-core systems because it encourages immutable objects and discourages side effects (meaning we don't have to worry nearly so much about writing thread-safe code). NOW: It grew, and it continues to grow. New languages in general are being released (see Clojure, Jaskell, dynalang, and others), and lots of them are incorporating functional language concepts. If you've never programmed in Haskell, now's a good time to learn, because those concepts and syntax are fast making their way towards you...
  • THEN: Languages: Interest in Domain-specific languages will reach a peak this year, but a small backlash will begin next year. Meanwhile, more and more developers will realize that one man's "DSL" is another man's "little language", something UNIX has been doing since the early 70's. This will immediately take the shine off of DSLs, since anything that we did in the 70's must be bad, somehow. (Remember disco?) NOW: Apparently people haven't made the connection yet. Give it time, give it time...
  • THEN: General: Rails will continue to draw developers who want quick-fix solutions/technologies, and largely that community will ignore the underlying power of Ruby itself. The draw will start to die down once Rails-esque feature ideas get folded into Java toolkits. (Rails will largely be a non-issue with the .NET community, owing to the high-productivity nature of the drag-and-drop interface in Visual Studio.) NOW: In the Java world, see Grails. In the .NET world, see the new MVC container for ASP.NET. In both worlds? Sit back and chortle at how each side chases the other, and quietly look for the best of both. Look like a genius. Retire to Caribbean country on your luxury yacht and supermodel girlfriends.
  • THEN: Java: Interface21 is going to start looking like a "big vendor" alongside BEA and IBM. I was talking with some of the I21 folks in Aarhus, Denmark at JAOO, and one of them casually mentioned that they were looking at a Spring 2.1 release somewhere in mid-2008. Clearly Spring is settling into eighteen-month major-version release cycles like all the big (meaning popular), established software systems have a tendency to do. This is both a good thing and a bad thing--it's good in that it means that Spring is now becoming an established part of the Java landscape and thus more acceptable to use in production environments, but it's bad in that Spring is now going to face the inevitable problem all big vendors face: trying to be all things to all people. This is dangerous, both for Interface21 and the people relying on Spring, largely because it means that Spring faces a very real future of greater complexity (and there are those, myself included, who believe that Spring is too complex already, easily on par with the complexity seen in EJB, POJOs notwithstanding). NOW: Spring hit the 2.1 release earlier than 2008... sort of. They actually went one step better, deciding that the next release of Spring isn't a "2.1", but a "2.5". (Apparently Microsoft marketing isn't the only one who likes to jerk around version numbers.) When combined with their re-branding as "SpringSource", they're just a re-org away from being a Real Corporation.
  • THEN: General: Marc Fleury will get a golden parachute from Red Hat (at their request and to their immense relief), and hopefully will retire to his own small island (might I suggest Elba, la petite corporal?) to quietly enjoy his millions. A shame that the people who did most of the real work on JBoss won't see a commensurate reward, but that's the way the business world works, I guess. NOW: Has anybody paid any attention to Fleury since his "extended paternity leave"? Maybe it's just me, but the Internet has been blissfully quiet the last half-year or so....
  • THEN: General: Some company will get millions to build an enterprise product on the backs of RSS and/or Atom, thus proving that VCs are just as stupid and just as vulnerable to hype now as they were back in the DotCom era. NOW: I think I jumped the gun on this one. Unless, of course, you think about Flickr and Twittr as extended RSS/ATOM feeds....
  • THEN: General: Somebody will attempt to use the phrase "Web 2.0" in a serious discussion, and I will be forced to kill them for attempting to use a vague term in a vain effort to sound intelligent. NOW: Tim O'Reilly must be dipped in lemon juice after being subjected to two dozen papercuts for creating this term. I actually heard somebody who should have known better ask, "So how do I log on to this Web 2.0 thing?" Apparently he got it confused with Internet2 (which is a boondoggle in its own right by this point.)
  • THEN: Web clients: Ajax will start to lose its luster when developers realize the power of Google Maps isn't in Ajax, but in the fact that it's got some seriously cool graphics and maps. (Or, put another way, when developers realize that Ajax alone won't make their apps as cool as Google Maps, that's it's the same old DHTML from 1998, the hype will start to die down.) NOW:  Have people started building real apps with this stuff yet? Everything so far looks like demos and research prototypes to me....
  • THEN: XML: Somebody, somewhere, will realize that REST != HTTP. He will be roundly criticized by hordes of HTTP zealots, and quietly crawl away to go build simpler and more robust systems that use transports other than HTTP. NOW: Anybody? Anybody? Bueller?
  • THEN: XML: Somebody, somewhere, will read the SOAP 1.2 specification. H.P. Lovecraft once suggested, loosely paraphrased, the the day Man understands the nature of the universe, he will either be driven into gibbering insanity, or flee back into ignorance in self-preservation. Ditto for the day Man reads the SOAP 1.2 spec and realizes that SOAP is, in fact, RESTful. NOW: Paging Mr. Davis, Mr. Scott Davis, please answer the white courtesy telephone...
  • THEN: Security: The US Government will continue its unbelievable quest to waste money on "security" by engaging in yet more perimeter security around airports and other indefensible locations, thus proving that none of them have bothered to read Schneier and learn that real security is a three-part tuple: prevention, detection, and response. NOW: Paging Mr. [DELETED], Mr. [REDACTED] [DELETED], you have a [INSUFFICIENT CLEARANCE]....
  • THEN: Security: Thousands of companies will follow in the US Government's footsteps by doing exactly the same thing. (Folks, you can't solve all your problems with cryptography, no matter how big the key size--you just end up with the basic problem of where to store the keys, and no, burying them inside the code isn't going to hide them effectively.) NOW: Anybody had a look at some of these "secure" website approaches recently?
  • THEN: Security: More and more rootkits-shipping-with-a-product will be discovered. We used to call it "getting close to the metal", now it's a "rootkit". With great power comes great responsibility... and, as many consumers have already discovered, with great power also comes a tendency to create greater instability... NOW: Rootkits, extensions, call it what you will, but they're growing all over the place....
  • THEN: General: Parrot will ship a 1.0 release. Oh, wait, hang on, sorry, I bumped into the crystal ball and accidentally set it to 2017. NOW: Parrot is all but dead, near as I can tell. Too bad, too, but maybe if they'd had a more realistic approach to building a unified type system....
  • THEN: .NET: Microsoft will ship Orcas (NetFX 3.5). (Sorry, crystal ball's still set on 2017. Trying to fix it...) NOW: They shipped. 'Nuff said.
  • THEN: .NET: Vista will surpass Windows XP in market penetration. (Let's see, almost got it set back to 2007, bear with me... There. Got it.) NOW: They wish. Cue Vista SP1. 'Nuff said.
  • THEN: General: I will blog more than I did this year. (Hell, I couldn't blog less, even if I tried.) NOW: Well....
  • THEN: General: Pragmatic XML Services, Pragmatic .NET Project Automation and Effective .NET will ship. (Wait, is the crystal ball still on 2017...?) NOW: I wish. 'Nuff said.

Next, my predictions for 2008:

  • General: The buzz around building custom languages will only continue to build. More and more tools are emerging to support the creation of custom programming languages, like Microsoft's Phoenix, Scala's parser combinators, the Microsoft DLR, SOOT, Javassist, JParsec/NParsec, and so on. Suddenly, the whole "write your own lexer and parser and AST from scratch" idea seems about as outmoded as the idea of building your own String class. Granted, there are cases where a from-hand scanner/lexer/parser/AST/etc is the Right Thing To Do, but there are times when building your own String class is the Right Thing To Do, too. Between the rich ecosystem of dynamic languages that could be ported to the JVM/CLR, and the interesting strides being made on both platforms (JVM and CLR) to make them more "dynamic-friendly" (such as being able to reify classes or access the call stack directly), the probability that your company will find a need that is best answered by building a custom language are only going to rise.
  • General: The hype surrounding "domain-specific languages" will peak in 2008, and start to generate a backlash. Let's be honest: when somebody looks you straight in the eye and suggests that "scattered, smothered and covered" is a domain-specific language, the term has lost all meaning. A lexicon unique to an industry is not a domain-specific language; it's a lexicon. Period. If you can incorporate said lexicon into your software, thus making it accessible to non-technical professionals, that's a good thing. But simply using the lexicon doesn't make it a domain-specific language. Or, alternatively, if you like, every single API designed for a particular purpose is itself a domain-specific language. This means that Spring configuration files are a DSL. Deployment descriptors are a DSL. The Java language is a DSL (since the domain is that of programmers familiar with the Java language). See how nonsensical this can get? Until somebody comes up with a workable definition of the term "domain" in "domain-specific language", it's a nonsensical term. The idea is a powerful one, mind you--creating something that's more "in tune" with what users understand and can use easily is a technique that's been proven for decades now. Anybody who's ever watched an accountant rip an entirely new set of predictions for the new fiscal outlook based entirely on a few seed numbers and a deeply-nested set of Excel macros knows this already. Whether you call them domain-specific languages or "little languages" or "user-centric languages" or "macro language" is really up to you.
  • General: Functional languages will begin to make their presence felt. Between Microsoft's productization plans for F# and the growing community of Scala programmers, not to mention the inherently functional concepts buried inside of LINQ and the concurrency-friendly capabilities of side-effect-free programming, the world is going to find itself working its way into functional thinking either directly or indirectly. And when programmers start to see the inherent capabilities inside of Scala (such as Actors) and/or F# (such as asynchronous workflows), they're going to embrace the strange new world of functional/object hybrid and never look back.
  • General: MacOS is going to start posting some serious market share numbers, leading lots of analysts to predict that Microsoft Windows has peaked and is due to collapse sometime within the remainder of the decade. Mac's not only a wonderful OS, but it's some of the best hardware to run Vista on. That will lead not a few customers to buy Mac hardware, wipe the machine, and install Vista, as many of the uber-geeks in the Windows world are already doing. This will in turn lead Gartner (always on the lookout for an established trend they can "predict" on) to suggest that Mac is going to end up with 115% market share by 2012 (.8 probability), then sell you this wisdom for a mere price of $1.5 million (per copy).
  • General: Ted will be hired by Gartner... if only to keep him from smacking them around so much. .0001 probability, with probability going up exponentially as my salary offer goes up exponentially. (Hey, I've got kids headed for college in a few years.) ;-)
  • General: MacOS is going to start creaking in a few places. The Mac OS is a wonderful OS, but it's got its own creaky parts, and the more users that come to Mac OS, the more that software packages are going to exploit some of those creaky parts, leading to some instability in the Mac OS. It won't be widespread, but for those who are interested in finding it, they're there. Assuming current trends (of customers adopting Mac OS) hold, the Mac OS 10.6 upgrade is going to be a very interesting process, indeed.
  • General: Somebody is going to realize that iTunes is the world's biggest monopoly on music, and Apple will be forced to defend itself in the court of law, the court of public opinion, or both. Let's be frank: if this were Microsoft, offering music that can only be played on Microsoft music players, the world would be through the roof. All UI goodness to one side, the iPod represents just as much of a monopoly in the music player business as Internet Explorer did in the operating system business, and if the world doesn't start taking Apple to task over this, then "justice" is a word that only applies when losers in an industry want to drag down the market leader (which I firmly believe to be the case--nobody likes more than to pile on the successful guy).
  • General: Somebody is going to realize that the iPhone's "nothing we didn't write will survive the next upgrade process" policy is nothing short of draconian. As my father, who gets it right every once in a while, says, "If I put a third-party stereo in my car, the dealer doesn't get to rip it out and replace it with one of their own (or nothing at all!) the next time I take it in for an oil change". Fact is, if I buy the phone, I own the phone, and I own what's on it. Unfortunately, this takes us squarely into the realm of DRM and IP ownership, and we all know how clear-cut that is... But once the general public starts to understand some of these issues--and I think the iPhone and iTunes may just be the vehicle that will teach them--look out, folks, because the backlash will be huge. As in, "Move over, Mr. Gates, you're about to be joined in infamy by your other buddy Steve...."
  • Java: The OpenJDK in Mercurial will slowly start to see some external contributions. The whole point of Mercurial is to allow for deeper control over which changes you incorporate into your build tree, so once people figure out how to build the JDK and how to hack on it, the local modifications will start to seep across the Internet....
  • Java: SpringSource will soon be seen as a vendor like BEA or IBM or Sun. Perhaps with a bit better reputation to begin, but a vendor all the same.
  • .NET: Interest in OpenJDK will bootstrap similar interest in Rotor/SSCLI. After all, they're both VMs, with lots of interesting ideas and information about how the managed platforms work.
  • C++/Native: If you've not heard of LLVM before this, you will. It's a compiler and bytecode toolchain aimed at the native platforms, complete with JIT and GC.
  • Java: Somebody will create Yet Another Rails-Killer Web Framework. 'Nuff said.
  • Native: Developers looking for a native programming language will discover D, and be happy. Considering D is from the same mind that was the core behind the Zortech C++ compiler suite, and that D has great native platform integration (building DLLs, calling into DLLs easily, and so on), not to mention automatic memory management (except for those areas where you want manual memory management), it's definitely worth looking into. www.digitalmars.com

See you all next year....




Wednesday, December 26, 2007 4:28:47 PM (Pacific Standard Time, UTC-08:00)
Comments [1]  | 
 Saturday, December 15, 2007
Let the JDK Hacking Begin...

OpenJDK, the open-source JDK 7 release (and no, I don't know if there's any practical difference between the two) has officially opened for business with the promotion of the "real, live" Mercurial repositories. These are the real deal, the same repositories that Sun employees will be working on as they modify the code... which means, in all reality, that there is a very tiny window of opportunity for you to check out code between changesets that are dependent on one another due to the way they've got the forest set up--if you get weird build errors, try re-fetching... but more on that later.

Think about it for a second--now, thanks to the way Mercurial handles source (a distributed source code system is different from a centralized client/server one like SVN or CVS), you can hack away on your own private build of the JDK, and still keep up to date with what Sun's doing. And, if Sun likes what you're doing, you could end up contributing back to the JDK as a whole.

You could make changes to the langtools bundle to start adding new features you think should be in Java.

You could make changes to the hotspot bundle to start exploring ways to optimize GC better.

You could fix bugs you find in the various Java libraries.

You could even add new features to the core classes that you've been saying needed to be there since Day 1.

You can start exploring the new Modules system (JSR-277) strawman implementation.

Then, you can post your diffs, or just tell people where your changesets are, and others can play with them.

Cool.

Getting Started

Meanwhile, for those of you looking to get started with hacking on the JDK, there's a couple of things you need to do:

  1. Fetch the prereqs.
  2. Fetch the source through Mercurial.
  3. Set up your build prompt.
  4. Type "make" and go away for an hour or two.

Naturally, things are just a teeny bit more complicated than that, particularly if you're on a Windows box. (Update: Volker Simonis has blogged, similar to this post, his experience in configuring a Suse Enterprise Linux 9.3 box to build the OpenJDK.) Thus, as one who's labored through this process on said platform, I'll show you how I got it all up and running on my machine (well, technically, pseudo-machine, it's a VMWare image with Windows XP SP2 in it). If you have questions, you're free to send me email, but chances are I'll just redirect you to build-dev, where the amazing Kelly O'Hair hangs out and answers build questions. (MAJOR kudos again to Kelly for getting this done!)

Note that Sun is looking at other platforms, and has some good docs on building for Ubuntu 6.x and 7.x in the README already known--I've managed that with very little difficulty, so if you're starting from scratch, I'd suggest grabbing the free VMWare Player, an Ubuntu 7.04 Desktop image, and use that as a JDK build environment.

While we're at it, along the way, I'll be pointing out areas that I think Sun could use and would appreciate some community contributions. (People are always asking me this at conferences, as a way of making their name more well-known and building credibility in the industry.) Note that I have no "pull" with Sun and can't begin to guess where they want the help, I'm simply suggesting where I think the help would be useful--to the community if not to Sun directly.

By the way, the official document for build process requirements is the OpenJDK Build README; I only offer this up as a supplement and to offer up my own experiences. Note that I didn't go "all the way", in that I don't care about building the Java Plug-In or some of the other ancillary stuff, such as the JDK installer itself, so I didn't necessarily install all the bits documented on the OpenJDK README page. I wanted source that I could hack, step into, debug, and so on.

Without further ado....

1. Fetch the prereqs

This sort of goes without saying, but in order to build the JDK, you need to have the necessary tools in place.

  • Environment. The build needs to be portable across both Windows and *nix, obviously, so OpenJDK needs a *nix-like environment on the Windows platform. Originally, back in the JDK 1.2 era, this was a commercial toolkit Sun purchased, the MKS Toolkit, but obviously that doesn't work for open source, so they've chosen to use Cygwin. You'll need to make sure you have a couple of tools explicitly documented by Sun (ar.exe, cpio.exe, make.exe, file.exe, and m4.exe), but realistically you'll want the rest of the Cygwin developer tools just to have a complete environment. Cygwin afficionados will (hopefully) weigh in with what the best packages mix to grab are; I just grabbed everything that looked tasty at the time, including stuff that had nothing to do with building OpenJDK. Note that there is a big problem with the make.exe that comes with Cygwin, however...  (Update: Poonam's Weblog notes, "Along with the default installation, we need to install Devel, Interpreters and Utils packages.")
    • On my system, I put this in C:\Prg\cygwin.
    • CONTRIBUTION: There's a lot of MSYS developers who would love it if you could figure out how to make the build work with their toolchain, too.... and don't ask me how, because I have no real idea; I'm totally n00b when it comes to the differences between the two.
  • GNU Make 3.78.1 to 3.80. NOT the 3.81 or later version, because apparently there's problems with DOS-like paths, like C:/ (which is obviously going to be a factor on Windows; see some of Kelly's problems with shells for examples of what he's had to wrestle with). Getting this turned out to be a pain, and required some searching; Kelly pointed out a patched make.exe on Cygwin that should work. Make sure this make appears before the regular Cygwin make in your shell prompt (later). (Update: Find the right one here, as well.)
    • On my system, I created a directory for the OpenJDK project, in C:\Prg\OpenJDK, and put GNU make in C:\Prg\OpenJDK\bin.
    • CONTRIBUTION: Either fix the damn PATH issues ("Dear Mr. Gates, Would you please tell your developers to use the right kind of slash, already?"), or else fix Cygwin's make to recognize C:/ (note the forward slash) paths. Not sure what else could be done here.
  • Compiler. Sun originally was using the Microsoft Visual C++ 6.0 environment (which made sense at the time, since they were a company and could pay for it); with the open-source release... well... turns out that they're still using the Microsoft tools, only they've also added Microsoft Visual Studio 2003 to the list of supported compilers. (I'm using MSVS2003 myself.) This is a pain because both of those environments are commercial, and the Visual C++ 2005 Express (which is free) doesn't seem to work yet. People have suggested other compilers (Watcom's OpenC++, Cygwin's gcc, and so on), but right now that doesn't sound like a high priority for Sun. Of course, it's fair to suggest that if you're building for Windows, you probably have a MSVS installation somewhere, but still, it'd be nice....
    • On my system, I put MSVS2003 in C:\Prg\MSVS2003. Note that I *just* installed the Visual C++ bits, and left out the rest of the .NET stuff. (I do .NET in another VMWare image, so I don't need it here.) To make our life simpler, reister the environment variables globally. (This is an install option.)
    • CONTRIBUTION: Port the makefiles to use Visual C++ 2005 Express, or one of the other free compilers. I would think the easiest transition would be to VC2005Ex, but there may be some tools missing from the Express editions that the build needs; I haven't spent much time here researching what's working and not. This would likely be (much) harder for other compilers, owing to the differences in toolchains.
    • CONTRIBUTION: Port the makefiles to use Visual Studio 2008.
  • DirectX SDK. Yes, you need the DirectX SDK, specifically the Summer 2004 9.X Update, for building some of the advanced graphics stuff. The Build README has it linked there, as well, and the link, as of this writing, is still good. Install it, then take note of the DXSDK environment variable--we'll need it later.
    • On my system, I put DirectX in C:\Prg\DirectX9SDK_062005.
  • FreeType 2. This is something to do with fonts, and is needed to remove a commercial dependency that was holding up the OpenJDK from building about a year ago. Grab it, install it, and note the headers and lib directory. The FreeType download page has some links to pre-built stuff, but in the end, you just want freetype.dll, freetype.lib, and the various FreeType headers.
    • On my system, I put these in C:\Prg\OpenJDK\deps\freetype-igor (named after the helpful soul on build-dev who was kind enough to send me his pre-built FreeType bits). Note that underneath that directory, I have windows/freetype-i586/headers and /lib, which I'll use later for environment variables.
    • CONTRIBUTION: Put a "JDK-specific" bundle of FreeType somewhere on the Web for people to download and not have to build. :-)
  • A "bootstrap" JDK. Go grab JDK 1.6; you'll need this for building the Java bits. (I would hope this isn't a problem for you; if it is, you may want to quickly go to some other Web page. Any web page.)
    • On my system, this resides in C:\Prg\jdk1.6.0.
  • Apache Ant. At least version 1.6.3, I'm using 1.7 without a problem.
    • On my system, this resides in C:\Prg\apache-ant-1.7.0.
  • The "binary plugs". As much work has Sun has done to unencumber the JDK, there's still little bits and pieces that are commercial that they can't release the source to, so they put them into a "binary plugs" package and have you install it, then point to it in the build scripts, and copy those files over. They get versioned every time Sun releases a built JDK 7 bundle, but I don't think you need to grab this more than once; just the same, keep an eye on that as time goes on, and if you get weird build errors, check build-dev to see if the plugs changed.
    • On my system, this resides in C:\Prg\OpenJDK\deps\openjdk-binary-plugs. (The .jar file is an executable jar, so just "java -jar" it and tell it to install in C:\Prg\OpenJDK\deps; it adds the rest. It's done in 3 seconds, and consists of 1 file. Now you see why I wouldn't worry too much about this.)
  • Mercurial. This is a distributed revision control system, and there's lots more to say about that at the Mercurial website. Its commands look somewhat similar to SVN, though definitely read "Distributed Revision Control with Mercurial" if you're going to be keeping up with the source trees as they go. You *want* the "forest" extension as part of your Mercurial binary, so grab the "Batteries Included" installer version. I went with the non-TortoiseHG version, and had to download all four of the released files off that page and install and uninstall and install and uninstall until I found one that worked (the "win32extrasp1.exe" version in the "dec" release list on Sourceforge).
    • On my system, Mercurial lives in C:\Prg\Mercurial. Put in on the PATH so you have access to "hg.exe".
    • CONTRIBUTION: Figure out what the differences are and post it someplace--how to get the "forest" extension installed and turned on in Mercurial was a pain; Google was of little to no help here. (Tag it as a comment to this blog entry, if you like, and I'll update the entry itself once I see it.)
    • Update: Daniel Fuchs blogs about how to get Mercurial's "forest" extension installed in your installation, in case you don't get the "Batteries Included" version:

      I simply cloned the forest repository in c:\Mercurial. In a cygwin terminal:

        cd c:/Mercurial
        hg clone http://www.terminus.org/hg/hgforest  hgforest
      

      Then I edited c:/Mercurial/Mercurial.ini and added the lines:

        [extensions]
        forest=c:/Mercurial/hgforest/forest.py
      

      as documented in the Mercurial Wiki.

  • Optional: FindBugs. The build will start using FindBugs to do source-analysis to find bugs before they happen, so it's not a bad idea to have that as well.
    • On my system, this resides in C:\Prg\findbugs-1.2.1.

At this point, you should be ready to go.

2. Fetch the source.

Ivan's got it (mostly) right: just do this:

cd C:\Prg\OpenJDK

md jdk7

hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7

(Don't use MASTER as he does, I don't think that works--that was just for the experimental repositories.) Don't forget the trailing slash, either, or you'll get an error saying something along the lines of http://hg.openjdk.java.net/jdk7%5Ccorba is not a valid URL or somesuch.

If your Mercurial doesn't have the "forest" extension, "fclone" won't work; Ivan's got tips on how to pull down the sub-repositories by hand, but I get nervous doing that because what if the list changes and I wasn't paying attention?

Your network will go wild for about twenty minutes or so, maybe longer, pulling lots of stuff down from the URL above. The sources should now reside in C:\Prg\OpenJDK\jdk7. Go browse 'em for a bit, if you feel so inclined. Get a good rush going, because this next step can be the tricky one.

Update: Volker Simonis ran into some issues with using Mercurial and an HTTP proxy, and found it difficult to find assistance across the Web. In the interests of making it easier for others, he's allowed me to republish his experience here:

I just had a real hard time to get the forest extension working and finally found out that it was because the forest extension doesn't honor the "http_proxy" environment variable. So I thought I'll post it here in case anybody else will face the same problem in order to save him some time. (If you're only interested in the solution of the problem, you can skip the next paragraphs and jump right to the end of this post).

I installed Mercurial and the forest extension as described in various places, here on the list and on the Web - that was the easy part:) Afterwards I could do:

/share/software/OpenJDK> hg clone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
requesting all changes
adding changesets
adding manifests 
adding file changes
added 2 changesets with 26 changes to 26 files
26 files updated, 0 files merged, 0 files removed, 0 files unresolved

So everything worked fine! Note that I'm behind a firewall, but Mercurial correctly picked up my http proxy from the "http_proxy" environment variable!

But now, everytime I tried 'fclone', I got the following error:

/share/software/OpenJDK> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 
[.]
abort: error: Name or service not known

That was not amusing. First I thought I got something wrong during the installation of the forest extension. I than used the '--traceback' option to "hg" which told me that the error was in keepalive.py:

/share/software/OpenJDK> hg --traceback fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 
[.] 
Traceback (most recent call last):
...
File "/share/software/Python-2.5.1_bin/lib/python2.5/site-packages/mercurial/keepalive.py", 
line 328, in _start_transaction raise urllib2.URLError(err)
URLError: <urlopen error (-2, 'Name or service not known')>
abort: error: Name or service not known 

So I enabled the debugging output in keepalive.py and realized, that the first two connections to hg.openjdk.java.net where made trough the proxy, while the third one (the first that actually fetches files), wants to go directly to hg.openjdk.java.net, which badly fails:

/share/software/OpenJDK> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
keepalive.py - creating new connection to proxy:8080 (1078835788)
keepalive.py - STATUS: 200, OK
keepalive.py - re-using connection to proxy:8080 (1078835788)
keepalive.py - STATUS: 200, OK 
[.]
keepalive.py - creating new connection to hg.openjdk.java.net (1078970092)
abort: error: Name or service not known

The problem can be fixed by adding the proxy settings to your .hgrc file, like this:

[http_proxy]
host=proxy:8080

where you have to replace "proxy:8080" with the name and the port of your real proxy host!

Volker's original email came from the buid-dev list, and if you have further questions about Mercrial and HTTP proxies, I'd suggest that as a resource.

3. Set up your environment.

This is where you'll spend a fair amount of time, because getting this right can be ugly. There's some environment variables that tell the build script where to find things, and we have to point out those things, like compiler location and such. If you installed everything in the same places I did, then the following, which I put into C:\Prg\OpenJDK\build_shell.sh, should work for you:

#!/bin/sh

# "External" bits (outside of OpenJDK path structure)
#
export ALT_BOOTDIR=C:/Prg/jdk1.6.0
export ANT_HOME=c:/Prg/apache-ant-1.7.0
export FINDBUGS_HOME=/cygdrive/c/Prg/findbugs-1.2.1

# OpenJDK flag (to make FreeType check pass)
#
export OPENJDK=true

export OPENJDK_HOME=C:/Prg/OpenJDK
openjdkpath=$(cygpath --unix $OPENJDK_HOME)

# OpenJDK-related bits
# (ALT_JDK_IMPORT_PATH fixes a corba bug; remove it later)
#
export ALT_JDK_IMPORT_PATH=$(cygpath --unix C:/Prg/jdk1.6.0)
export ALT_BINARY_PLUGS_PATH=$OPENJDK_HOME/openjdk-binary-plugs
export ALT_UNICOWS_DLL_PATH=$OPENJDK_HOME/deps/unicows
export ALT_FREETYPE_LIB_PATH=$OPENJDK_HOME/deps/freetype-igor/windows/freetype-i586/lib
export ALT_FREETYPE_HEADERS_PATH=$OPENJDK_HOME/deps/freetype-igor/windows/freetype-i586/include/freetype2
. $openjdkpath/jdk7/jdk/make/jdk_generic_profile.sh

# Need GNU make in front of Cygwin's; this is the only practical way to do it
#
PATH=$openjdkpath/bin:$PATH
export PATH

# Let people know this is an OpenJDK-savvy prompt
#
export PS1='OpenJDK:\[\e]0;\w\a\]\[\e[32m\]\u@${COMPUTERNAME}:\[\e[33m\]\w\[\e[0m\]\n\$ '

Note the UNICOWS_DLL thing in the middle; this was necessary in earlier builds, and I don't know if it still is. For now I'm not messing with it; if you discover that you need it, the Build README has links. (Update: Kelly confirmed that this is no longer necessary in the OpenJDK build. Yay!)

Note that I set the COMPILER_VERSION flag to tell the build script which compiler I'm using--if that's not set, the build fails pretty quickly, complaining that "COMPILER_VERSION" cannot be empty. (Update: Kelly mentions, "I suspect the reason you are having the COMPILER_VERSION problem is that the makefiles are trying to run cl.exe to get the version, and if the PATH/LIB/INCLUDE env variables are not setup right, cl.exe fails. Several people have run into that problem.")

Note that OPENJDK must be set, or the build process thinks this is a commercial build, and an early sanity-check to see what version of FreeType is running will fail. (Specfically, Sun builds a tool just to see if the code compiles; if it fails to compile, chances are you forgot to set this flag. That's been my problem, each and every time I tried to rebuild the OpenJDK build space. Hopefully I never forget it again.)

Note that I call into jdk_generic_profile.sh to do some more setup work; this gets all the MSVS2003 stuff into the PATH as well.

Be very careful with which path you use; sometimes the build wants C:/Prg style paths, and sometimes it wants /cygdrive/c/Prg style paths. Don't assume the script above is perfect--I'm still testing it, and will update this entry as necessary as I find bugs in it.

(Update: Kelly mentions, "Be careful putting cygwin before VS2003 in the PATH, cygwin has a link.exe and so does VS2003, you need the one from VS2003.")

From a Cygwin bash prompt,

cd /cygdrive/c/Prg/OpenJDK

. ./build_shell.sh

cd jdk7

make sanity

It will churn, think, text will go flying by, and you will (hopefully) see "Sanity check passed". If not, look at the (voluminous) output to figure out what paths are wrong, and correct them. Note that certain paths may be reported as warnings and yet the buld will still succeed, that's OK, as far as I can tell.

And no, I don't know what all of those environment variables are for. Kelly might, but I suspect there's a lot of built-up cruft from over the years that they'd like to start paring down. Let's hope.

4. Type "make" and go away for a while.

Specifically, type "make help" to see the list of targets.

OpenJDK:Ted@XPJAVA:/cygdrive/c/Prg/OpenJDK/jdk7
$ make help
Makefile for the JDK builds (all the JDK).

--- Common Targets ---
all -- build the core JDK (default target)
help -- Print out help information
check -- Check make variable values for correctness
sanity -- Perform detailed sanity checks on system and settings
fastdebug_build -- build the core JDK in 'fastdebug' mode (-g -O)
debug_build -- build the core JDK in 'debug' mode (-g)
clean -- remove all built and imported files
clobber -- same as clean

--- Common Variables ---
ALT_OUTPUTDIR - Output directory
OUTPUTDIR=./build/windows-i586
ALT_PARALLEL_COMPILE_JOBS - Solaris/Linux parallel compile run count
PARALLEL_COMPILE_JOBS=2
ALT_SLASH_JAVA - Root of all build tools, e.g. /java or J:
SLASH_JAVA=J:
ALT_BOOTDIR - JDK used to boot the build
BOOTDIR=C:/Prg/jdk1.6.0
ALT_JDK_IMPORT_PATH - JDK used to import components of the build
JDK_IMPORT_PATH=c:/Prg/JDK16~1.0
ALT_COMPILER_PATH - Compiler install directory
COMPILER_PATH=C:/Prg/MSVS2003/Common7/Tools/../../Vc7/Bin/
ALT_CACERTS_FILE - Location of certificates file
CACERTS_FILE=/lib/security/cacerts
ALT_DEVTOOLS_PATH - Directory containing zip and gnumake
DEVTOOLS_PATH=/usr/bin/
ALT_DXSDK_PATH - Root directory of DirectX SDK
DXSDK_PATH=C:/Prg/DIRECT~1
ALT_MSDEVTOOLS_PATH - Root directory of VC++ tools (e.g. rc.exe)
MSDEVTOOLS_PATH=C:/Prg/MSVS2003/Common7/Tools/../../Vc7/Bin/
ALT_MSVCRT_DLL_PATH - Directory containing mscvrt.dll
MSVCRT_DLL_PATH=C:/WINDOWS/system32
WARNING: SLASH_JAVA does not exist, try make sanity
WARNING: CACERTS_FILE does not exist, try make sanity

--- Notes ---
- All builds use same output directory unless overridden with
ALT_OUTPUTDIR=<dir>, changing from product to fastdebug you may want
to use the clean target first.
- JDK_IMPORT_PATH must refer to a compatible build, not all past promoted
builds or previous release JDK builds will work.
- The fastest builds have been when the sources and the BOOTDIR are on
local disk.

--- Examples ---
make fastdebug_build
make ALT_OUTPUTDIR=/tmp/foobar all
make ALT_OUTPUTDIR=/tmp/foobar fastdebug_build
make ALT_OUTPUTDIR=/tmp/foobar all
make ALT_BOOTDIR=/opt/java/jdk1.5.0
make ALT_JDK_IMPORT_PATH=/opt/java/jdk1.6.0

OpenJDK:Ted@XPJAVA:/cygdrive/c/Prg/OpenJDK/jdk7
$

The one I want is "make fastdebug_build" or "make debug_build" (so I have debug symbols and can go spelunking). Do it.

Watch the stuff go flying by.

Get bored.

Get lunch.

Come back, it might be done. :-)

If it's a successful build, you'll have "stuff" in the C:\Prg\OpenJDK\jdk7\build directory, corresponding to the kind of build you kicked off; for a "fastdebug" build, for example, there'll be a "windows-i586-fastdebug" directory in which you'll find a "j2sdk-image" directory in which there should be a complete JDK environment. Try running Java and see if it works (make sure your PATH is updated to point to the right place before you do!)

If not, it's debugging time. :-) Note that the "build" directory is completely built from scratch, so if you get a partial build and want to start over from scratch, just "rd /s build" from the jdk7 directory. (It's easier than "make clean" or "make clobber", I've found.)

More About Builds

When building the JDK, you may want to build bits "underneath" the top-level directory, but doing this is a tad tricky. I asked about this on the build-dev list, and Kelly responded with a great email about the build process, specifically about launching "sub" makes within the system:

Due to history, a build directly from the jdk/make directories uses a default OUTPUTDIR of jdk/build/* but if FASTDEBUG=true, it's jdk/build/*-fastdebug, or if a plain debug build with just VARIANT=DBG it would be jdk/build/*-debug The variant builds leave results in a completely separate outputdir.

If you used the very top level makefile (which came from the now defunct control/make area) the default OUTPUTDIR is ./build/* (at the very top of the repositories). When this top level Makefile runs the jdk/make Makefiles, it passes in a ALT_OUTPUTDIR to refer to this top level build result area because it's default outputdir is not the same place.

I don't know the complete history as to why this was done this way, but my tendency is to try and get us back to a single default OUTPUTDIR for all the repositories. Someday...

This is what I do when I work on just the jdk repository:
cd jdk/make && gnumake

That primes the outputdir area, then I can drop down in:
cd jdk/make/java && gnumake

Or even drop in and clean an area and re-build it:
cd jdk/make/jpda && gnumake clean && gnumake

Or just repeat the entire build (incremental build)
cd jdk/make && gnumake

If I wanted the jdk image (j2sdk-image), I would need to:
cd jdk/make && gnumake image

But the output by default will go to jdk/build/* and a different directory if VARIANT=DBG or FASTDEBUG=true.

This should help having to go through the whole process for incremental updates. (Note that on my system, I had to call it "make" instead of "gnumake".)

Futures

As time goes on, I'll hopefully find the time to blog about how to find various little things in the JDK and make source-modifications to prove that they work, and use that as a springboard from which to let you start hacking on the JDK. In the meantime, please, if you run into trouble and find fixes to any of the above, comment or email me, and I'll correct this.

Contributions/Suggested TODOs

  • Have the make system automatically capture the results of the build in a log file, for easier debugging. Granted, you can "make >| build.output", but that seems tedious when it could easily be captured automagically for you each time. ("make sanity" does this, capturing the results in build/windows-i586-fastdebug/sanityCheckMessages.txt and sanityCheckWarnings.txt.
  • Documentation, documentation, documentation. This thing does a lot of recursive makes and invokes a lot of tools (some of them built as part of the build process itself), and it would be much easier to debug and understand if the process were better documented. Even a simple flowchart/tree of the various Make invocations and what each does (in practice) would be helpful when trying to track down issues.
  • Add support for all output to be captured into a build log file. This can obviously be done at the command-line via "make |& tee build.log" or ">& build.log", but I think it'd be nice if it were somehow folded in as part of the build process so it happened automatically.

Updates

  1. Kelly OHair sent me some updated information, such as the right link to use for the README file (from the HG repository instead of the SVN one).
  2. Kelly also mentioned that the plugin and installers are not part of the OpenJDK build yet, so that's not something I could build even if I wanted to. Which is fine, for me. :-)
  3. Kelly confirmed that UNICOWS is not needed in OpenJDK.
  4. Kelly mentions that link.exe on the PATH must be VS2003's, not Cygwin's.
  5. Kelly mentions the COMPILER_VERSION problem might be PATH issues.
  6. Kelly notes, "On the C:\ vs C:/ vs. /cyg*/ paths, I try and use the C:/ all the time everywhere, and anywhere in the Makefiles where I know I need C:\ or the /cyg*/ paths, I try and use cygpath or MKS dosname to force them into the right form. NMAKE.EXE only likes C:\ paths, and cygwin PATH only wants /cyg*/ paths, and Windows/Java/MKS never want /cyg*/ paths. :^( I wish we had a better solution on Windows for this shell/path mania."
  7. Poonam's Weblog has a good page on building the OpenJDK on Windows with NetBeans, from which I've stolen... ahem, leveraged... some links. His webpage has a link for the UNICOWS download page, but that only includes the DLL, not the .LIB, which you will also need. (It's an import library for the DLL; you need both.) The only place I know to get the .LIB is from the Platform SDK, and you need an old one, circa November 2001 or so. I don't know if it's kosher to redistribute any other way (meaning, Microsoft won't let us, as far as I know).
  8. Added Volker Simonis' experiences with HTTP proxies in Mercurial
  9. Added the "sub" make build discussion from Kelly on build-dev
  10. Added the link to Volker's blog about building on Suse Linux

C++ | Development Processes | Java/J2EE

Saturday, December 15, 2007 2:35:59 AM (Pacific Standard Time, UTC-08:00)
Comments [2]  | 
 Thursday, December 13, 2007
Them's fightin' words

From the cover of Dr. Dobb's Journal (Jan/2008):

PHP: The Power Behind Web 2.0
The article goes on to take a much less aggressive tone, simply saying that PHP is a good language for building web sites/applications that make use of Ajax and Web services, but let's be honest: you walk into a bar anywhere in the San Jose, Burlington or Redmond areas and say that kind of thing out loud, yer gonna get tossed out on yer keester.

Only because you're saying "Web 2.0", mind you. *shudder* All those in favor of "alt.death.to.tim.oreilly.for.coining.that.phrase" newsgroup, say Aye....


.NET | C++ | Conferences | Java/J2EE | Ruby | XML Services

Thursday, December 13, 2007 7:57:24 PM (Pacific Standard Time, UTC-08:00)
Comments [2]  | 
 Monday, December 10, 2007
Dear Santa...

You know, for years now, I've had just a terrible time making out Christmas lists.

The fact is, I'm fortunate enough to be in a position where if I really want something, I'm usually able to just go out and get it. If it's more expensive than what a single trip to the store armed with a credit card can obtain, then it's probably something I don't really need. (High-end gaming desktop machines do not fall into that category, no matter what my family might suggest.) So, it's rather difficult to come up with a list of things for friends and family to buy for me, particularly since impulse control is... well... almost entirely lacking in my case.

One year, I just said, "Make a charitable donation in my name to your favorite charity." I thought this was a nice solution, but it didn't go over incredibly well, probably because my family likes to see the whole "wow, thanks!" while we're all sitting underneath the tree. I ended up with the usual range of CDs, books, games, and so on. Now, mind you, I didn't return any of those things, but still, I felt vaguely guilty at my rampant consumerism when it's apparent with just a glance through any world-facing publication that there's a lot of people who could've used that money to survive for the year on what one CD cost.

I just figured out what I want for Christmas.

Those few of you who are non-geeks may not have heard of the "One Laptop Per Child" project: it's a charitable group that has created a (relatively) inexpensive laptop designed for use by underprivileged children to help close the gap between the technology "haves" and "have-nots". Now, in of itself, this is a Good Thing, but for a limited time, the foundation is sweetening the pot significantly:

From now through December 31, 2007, OLPC is offering a Give One Get One program in the United States and Canada. This is the first time the revolutionary XO laptop has been made available to the general public. For a donation of $399, one XO laptop will be sent to empower a child in a developing nation and one will be sent to the child in your life in recognition of your contribution. Up to $200 of your donation may be tax-deductible (your $399 donation minus the fair market value of the XO laptop you will be receiving).

For all U.S. donors who participate in the Give One Get One program, T-Mobile is offering one year of complimentary HotSpot access.

If you participate in Give One Get One today, you will receive your XO laptop(s) in early 2008. Your donated laptop will reach a child in Afghanistan, Cambodia, Haiti, Mongolia or Rwanda in the same early 2008 timeframe.

Aside from the fact that the T-Mobile access is a useful thing for a guy who's one the road close to 75% of his life, and aside from the fact that my family can take the "get one" laptop and give it to my 8-year-old as a starter laptop (or take it down to the local women's shelter and give it to a child there who probably would otherwise never set fingers on a working computer keyboard), the simple fact is, it's a nifty little gadget. Granted, I'm not going to give up my MacBookPro anytime soon over it, but it's got enough functionality to access wireless networks, and provides a Smalltalk-based environment and Python-based environment in which people can learn to program, and what's not to like about that?

 

Dear Santa--

I think I've been a good boy this year, but there are others in the world who've been better than me in circumstances that I can't even begin to imagine. So please, this year, make a donation to the One Laptop Per Child initiative, in either my name or yours.

Thanks, from both myself and someone neither of us has met yet,

Ted




Monday, December 10, 2007 3:01:57 AM (Pacific Standard Time, UTC-08:00)
Comments [1]  | 
 Sunday, December 9, 2007
Brian Grieves...

I grieve with him. I can't imagine how the babysitters feel right now--at age 13 and 11, to have to make the hard choices any babysitter has to make in a the event of a fire, and to have to live with that choice for the rest of your life. To be the parents, to have to live with the "what-might-have-beens" and the "if-onlys".

And to have to learn, as Brian talks about, how to go on.

No parent should ever have to bury their child.




Sunday, December 9, 2007 5:32:05 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  | 
Does this get an "A" on the Turing test, then?

From wikipedia:

The Turing test is a proposal for a test of a machine's capability to demonstrate intelligence. Described by Alan Turing in the 1950 paper "Computing machinery and intelligence," it proceeds as follows: a human judge engages in a natural language conversation with one human and one machine, each of which try to appear human; if the judge cannot reliably tell which is which, then the machine is said to pass the test.

From C.NET: "Warning sounded over 'flirting robots'"

Those entering online dating forums risk having more than their hearts stolen.

A program that can mimic online flirtation and then extract personal information from its unsuspecting conversation partners is making the rounds in Russian chat forums, according to security software firm PC Tools.

The artificial intelligence of CyberLover's automated chats is good enough that victims have a tough time distinguishing the "bot" from a real potential suitor, PC Tools said. The software can work quickly too, establishing up to 10 relationships in 30 minutes, PC Tools said. It compiles a report on every person it meets complete with name, contact information, and photos.

"As a tool that can be used by hackers to conduct identity fraud, CyberLover demonstrates an unprecedented level of social engineering," PC Tools senior malware analyst Sergei Shevchenko said in a statement.

Among CyberLover's creepy features is its ability to offer a range of different profiles from "romantic lover" to "sexual predator." It can also lead victims to a "personal" Web site, which could be used to deliver malware, PC Tools said.

Daaaaaamnnnnnn... you can offer a range of different profiles. From "romantic lover" to "sexual predator".

Can the "Make You Happy All Night Long" Web Service be far behind?

I would love to get my hands on this code, though. Think about what the natural language parsing routines must be like, particularly considering they have to be able to parse smileys, too.

If they can mimic flirting, how far away are we from code that mimics online help agents? It'd be like voicemail, but worse, since you could never know if you were talking to a real person or not....




Sunday, December 9, 2007 4:23:42 PM (Pacific Standard Time, UTC-08:00)
Comments [2]  | 
 Saturday, December 8, 2007
Quotes on writing

This is, without a doubt, the most accurate quote ever about the "fun" of writing a book:

Writing a book is an adventure. To begin with, it is a toy and an amusement; then it becomes a mistress, and then it becomes a master, and then a tyrant. The last phase is that just as you are about to be reconciled to your servitude, you kill the monster, and fling him out to the public. (Source: Winston Churchill)

Keep that in mind, all you who are considering authoring as a career or career supplement.

Were I to offer my own, it would be like so:

Writing a book is like having a child.

Trying is the best part, in some ways. You have this idea, this burning sensation in your heart, that just has to get out into the world. But you need a partner, a publisher who will help you bring your vision to life. You write proposals, you write tables of contents, you imagine the book cover in your mind. Then, YES! You get a publisher to agree. You sign the contract, fax it in, and you are on the way! We are authoring!

At first, it is wonderful and exciting and full of potential. You run into a few hangups, a few periods of nausea as you realize the magnitude of what you're really doing. You resolve to press on. As you continue, you begin to feel like you're in control again, but you start to get this sense like it's an albatross, a weight around your neck. Before long, you're dragging your feet, you can't seem to muster the energy to do anything, just get this thing done. The deadline approaches, the sheer horror of what's left to be done paralyzes you. You look your editor in the eye (literally or figuratively) and say, "I can't do this." The editor says, "Push". You whimper, "Don't make me do this, just cancel the contract." The editor says, "Push". You scream at them, "This is YOUR fault, you MADE me do this!" The editor says, "Push". Then, all of a sudden, it's done, it's out, it's on the shelf, and you take photos and show it off to all the friends, neighbors and family, who look at you a little sympathetically, and don't mention how awful you really look in that photo.

As the book is out in the world, you feel a sense of pride an joy at it. You imagine it profoundly changing the way people look at the world. You imagine it reaching bestseller lists. You're already practicing the speech for the Nobel. You're sitting in your study, you reach out and grab one of the free copies still sitting on your desk, and you open to a random page. Uh, oh. There's a typo, or a mistake, or something that clearly got past you and the technical reviewers and the copyeditors. Damn. Oh, well, one mistake can't make that much difference.

Then the reviews come in on Amazon. People like it. People post good reviews. One of them is not positive. You get angry: this is your baby they are attacking. How DARE they. You make plans to find large men with Italian names and track down that reviewer. You suddenly realize your overprotectiveness. You laugh at yourself weakly. You try to convince yourself that there's no pleasing some people.

Then someone comes up to you at a conference or interview or other gathering, and says, "Wow, you wrote that? I have that book on my shelf!" and suddenly it's all OK. It may not be perfect, but it's yours, and you love it all the same, warts and all.

Nearly a dozen books later, it's always the same.


.NET | C++ | Conferences | Development Processes | Java/J2EE | Reading | Ruby | Windows | XML Services

Saturday, December 8, 2007 2:48:51 AM (Pacific Standard Time, UTC-08:00)
Comments [1]  | 
 Wednesday, December 5, 2007
A Dozen Levels of Done

Michael Nygard (author of the great book Release It!), writes that "[his] definition of 'done' continues to expand". Currently, his definition reads:

A feature is not "done" until all of the following can be said about it:

  1. All unit tests are green.
  2. The code is as simple as it can be.
  3. It communicates clearly.
  4. It compiles in the automated build from a clean checkout.
  5. It has passed unit, functional, integration, stress, longevity, load, and resilience testing.
  6. The customer has accepted the feature.
  7. It is included in a release that has been branched in version control.
  8. The feature's impact on capacity is well-understood.
  9. Deployment instructions for the release are defined and do not include a "point of no return".
  10. Rollback instructions for the release are defined and tested.
  11. It has been deployed and verified.
  12. It is generating revenue.

Until all of these are true, the feature is just unfinished inventory.

As much as I agree with the first 11, I'm not sure I agree with #12. Not because it's not important--too many software features are added with no positive result--but because it's too hard to measure the revenue a particular program, much less a particular software feature, is generating.

My guess is that this is also conflating the differences between "features" and "releases", since they aren't always one and the same, and that not all "features" will be ones mandated by the customer (making #6 somewhat irrelevant). Still, this is an important point to any and all development shops:

What do you call "done"?


Development Processes | Reading

Wednesday, December 5, 2007 2:44:41 AM (Pacific Standard Time, UTC-08:00)
Comments [4]  | 
 Monday, December 3, 2007
NFJS 2007 Demo Code Posted

For those who've seen me give presentations at the No Fluff Just Stuff symposia over the year who've also asked to get the demo code that I write during the presentations, I've put it all up online. Simply find the show you attended below, and the hyperlink downloads the associated .zip file:

Usual disclaimer applies, as with all demo code: I don't guarantee the code for any particular purpose except as a supplement to the lecture that was given at that time in that city, the code isn't guaranteed to do anything except soak up space on your hard drive, I can't be held responsible if you use it in production, and so on.

If any of the links are broken, email me and let me know, but I think they should all work.




Monday, December 3, 2007 5:27:29 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  | 
Anybody know of a good WebDAV client library ...

... for Ruby, or PowerShell/.NET? I'm looking for something to make it easier to use WebDAV from a shell scripting language on Windows; Ruby and PowerShell are the two that come to mind as the easiest to use on Windows. For some reason, Google doesn't yield much by way of results, and I've got to believe there's better WebDAV support out there than what I'm finding.

(Yes, I could write one, but why bother, if one is out there that already exists? DRY!)

BTW, anybody who finds one and wants credit for it, I'll be happy to post here. If you're a commercial vendor and you send me a license to go with it, I'll post that, too, with some code and explanation on how I'm using it, though I doubt it's going to be all that different from how anybody else would use it. ;-)


.NET | C++ | Development Processes | Java/J2EE | Ruby | Windows

Monday, December 3, 2007 5:03:19 PM (Pacific Standard Time, UTC-08:00)
Comments [1]  | 
Javageeks papers moving here

Some of you knew me way back when I started the domain name "javageeks.com". Obviously, it's been a while since I updated that site, and in fact, if you go try to read some of the papers there, you'll get redirected to "neward.net", which has been on hiatus ever since the ISP there decided that my use of JSP was somehow trashing their server. (This is what happens when college students run an ISP in the spare time, I guess.)

Anyway, for a variety of reasons, I've since decided that my online presence should coalesce into one place, this domain (tedneward.com), and I've been slowly moving things over to this site in recognition of that fact. As a result, a bunch of the papers that I'd put up for reading on javageeks.com are now here on tedneward.com, in the "Writing" section of the website.

For those of you too tied to RSS to go visit the site itself, here's a list of the papers I've moved over thus far:

It's not the complete list, since I'm also (trying to) move the authoring format over from my own home-grown XML+XSLT system into DocBook, but that means either developing an XSLT to transform my home-grown format into DocBook, or else just manually switching them over. I'm not convinced yet which is the faster way to go. :-)

Part of it isn't just transmuting their form, though. Some of them could use with a touch of updating (such as the Java Policy one--there's been a few changes there since I wrote that paper, back in Java 1.2 days), so I think I'm going to have to labor through and transform and update them manually. Still, it could be worse: I could have written them in Lotus Ami Pro....




Monday, December 3, 2007 1:33:32 PM (Pacific Standard Time, UTC-08:00)
Comments [0]  |