OpenJDK, the open-source JDK 7 release (and no, I don't know if there's any practical difference between the two) has officially opened for business with the promotion of the "real, live" Mercurial repositories. These are the real deal, the same repositories that Sun employees will be working on as they modify the code... which means, in all reality, that there is a very tiny window of opportunity for you to check out code between changesets that are dependent on one another due to the way they've got the forest set up--if you get weird build errors, try re-fetching... but more on that later.

Think about it for a second--now, thanks to the way Mercurial handles source (a distributed source code system is different from a centralized client/server one like SVN or CVS), you can hack away on your own private build of the JDK, and still keep up to date with what Sun's doing. And, if Sun likes what you're doing, you could end up contributing back to the JDK as a whole.

You could make changes to the langtools bundle to start adding new features you think should be in Java.

You could make changes to the hotspot bundle to start exploring ways to optimize GC better.

You could fix bugs you find in the various Java libraries.

You could even add new features to the core classes that you've been saying needed to be there since Day 1.

You can start exploring the new Modules system (JSR-277) strawman implementation.

Then, you can post your diffs, or just tell people where your changesets are, and others can play with them.

Cool.

Getting Started

Meanwhile, for those of you looking to get started with hacking on the JDK, there's a couple of things you need to do:

1. Fetch the prereqs.
2. Fetch the source through Mercurial.
3. Set up your build prompt.
4. Type "make" and go away for an hour or two.

Naturally, things are just a teeny bit more complicated than that, particularly if you're on a Windows box. (Update: Volker Simonis has blogged, similar to this post, his experience in configuring a Suse Enterprise Linux 9.3 box to build the OpenJDK.) Thus, as one who's labored through this process on said platform, I'll show you how I got it all up and running on my machine (well, technically, pseudo-machine, it's a VMWare image with Windows XP SP2 in it). If you have questions, you're free to send me email, but chances are I'll just redirect you to build-dev, where the amazing Kelly O'Hair hangs out and answers build questions. (MAJOR kudos again to Kelly for getting this done!)

Note that Sun is looking at other platforms, and has some good docs on building for Ubuntu 6.x and 7.x in the README already known--I've managed that with very little difficulty, so if you're starting from scratch, I'd suggest grabbing the free VMWare Player, an Ubuntu 7.04 Desktop image, and use that as a JDK build environment.

While we're at it, along the way, I'll be pointing out areas that I think Sun could use and would appreciate some community contributions. (People are always asking me this at conferences, as a way of making their name more well-known and building credibility in the industry.) Note that I have no "pull" with Sun and can't begin to guess where they want the help, I'm simply suggesting where I think the help would be useful--to the community if not to Sun directly.

By the way, the official document for build process requirements is the OpenJDK Build README; I only offer this up as a supplement and to offer up my own experiences. Note that I didn't go "all the way", in that I don't care about building the Java Plug-In or some of the other ancillary stuff, such as the JDK installer itself, so I didn't necessarily install all the bits documented on the OpenJDK README page. I wanted source that I could hack, step into, debug, and so on.

Without further ado....

1. Fetch the prereqs

This sort of goes without saying, but in order to build the JDK, you need to have the necessary tools in place.

• Environment. The build needs to be portable across both Windows and *nix, obviously, so OpenJDK needs a *nix-like environment on the Windows platform. Originally, back in the JDK 1.2 era, this was a commercial toolkit Sun purchased, the MKS Toolkit, but obviously that doesn't work for open source, so they've chosen to use Cygwin. You'll need to make sure you have a couple of tools explicitly documented by Sun (ar.exe, cpio.exe, make.exe, file.exe, and m4.exe), but realistically you'll want the rest of the Cygwin developer tools just to have a complete environment. Cygwin afficionados will (hopefully) weigh in with what the best packages mix to grab are; I just grabbed everything that looked tasty at the time, including stuff that had nothing to do with building OpenJDK. Note that there is a big problem with the make.exe that comes with Cygwin, however...  (Update: Poonam's Weblog notes, "Along with the default installation, we need to install Devel, Interpreters and Utils packages.")
  • On my system, I put this in C:\Prg\cygwin.
  • CONTRIBUTION: There's a lot of MSYS developers who would love it if you could figure out how to make the build work with their toolchain, too.... and don't ask me how, because I have no real idea; I'm totally n00b when it comes to the differences between the two.
• GNU Make 3.78.1 to 3.80. NOT the 3.81 or later version, because apparently there's problems with DOS-like paths, like C:/ (which is obviously going to be a factor on Windows; see some of Kelly's problems with shells for examples of what he's had to wrestle with). Getting this turned out to be a pain, and required some searching; Kelly pointed out a patched make.exe on Cygwin that should work. Make sure this make appears before the regular Cygwin make in your shell prompt (later). (Update: Find the right one here, as well.)
  • On my system, I created a directory for the OpenJDK project, in C:\Prg\OpenJDK, and put GNU make in C:\Prg\OpenJDK\bin.
  • CONTRIBUTION: Either fix the damn PATH issues ("Dear Mr. Gates, Would you please tell your developers to use the right kind of slash, already?"), or else fix Cygwin's make to recognize C:/ (note the forward slash) paths. Not sure what else could be done here.
• Compiler. Sun originally was using the Microsoft Visual C++ 6.0 environment (which made sense at the time, since they were a company and could pay for it); with the open-source release... well... turns out that they're still using the Microsoft tools, only they've also added Microsoft Visual Studio 2003 to the list of supported compilers. (I'm using MSVS2003 myself.) This is a pain because both of those environments are commercial, and the Visual C++ 2005 Express (which is free) doesn't seem to work yet. People have suggested other compilers (Watcom's OpenC++, Cygwin's gcc, and so on), but right now that doesn't sound like a high priority for Sun. Of course, it's fair to suggest that if you're building for Windows, you probably have a MSVS installation somewhere, but still, it'd be nice....
  • On my system, I put MSVS2003 in C:\Prg\MSVS2003. Note that I *just* installed the Visual C++ bits, and left out the rest of the .NET stuff. (I do .NET in another VMWare image, so I don't need it here.) To make our life simpler, reister the environment variables globally. (This is an install option.)
  • CONTRIBUTION: Port the makefiles to use Visual C++ 2005 Express, or one of the other free compilers. I would think the easiest transition would be to VC2005Ex, but there may be some tools missing from the Express editions that the build needs; I haven't spent much time here researching what's working and not. This would likely be (much) harder for other compilers, owing to the differences in toolchains.
  • CONTRIBUTION: Port the makefiles to use Visual Studio 2008.
• DirectX SDK. Yes, you need the DirectX SDK, specifically the Summer 2004 9.X Update, for building some of the advanced graphics stuff. The Build README has it linked there, as well, and the link, as of this writing, is still good. Install it, then take note of the DXSDK environment variable--we'll need it later.
  • On my system, I put DirectX in C:\Prg\DirectX9SDK_062005.
• FreeType 2. This is something to do with fonts, and is needed to remove a commercial dependency that was holding up the OpenJDK from building about a year ago. Grab it, install it, and note the headers and lib directory. The FreeType download page has some links to pre-built stuff, but in the end, you just want freetype.dll, freetype.lib, and the various FreeType headers.
  • On my system, I put these in C:\Prg\OpenJDK\deps\freetype-igor (named after the helpful soul on build-dev who was kind enough to send me his pre-built FreeType bits). Note that underneath that directory, I have windows/freetype-i586/headers and /lib, which I'll use later for environment variables.
  • CONTRIBUTION: Put a "JDK-specific" bundle of FreeType somewhere on the Web for people to download and not have to build.
• A "bootstrap" JDK. Go grab JDK 1.6; you'll need this for building the Java bits. (I would hope this isn't a problem for you; if it is, you may want to quickly go to some other Web page. Any web page.)
  • On my system, this resides in C:\Prg\jdk1.6.0.
• Apache Ant. At least version 1.6.3, I'm using 1.7 without a problem.
  • On my system, this resides in C:\Prg\apache-ant-1.7.0.
• The "binary plugs". As much work has Sun has done to unencumber the JDK, there's still little bits and pieces that are commercial that they can't release the source to, so they put them into a "binary plugs" package and have you install it, then point to it in the build scripts, and copy those files over. They get versioned every time Sun releases a built JDK 7 bundle, but I don't think you need to grab this more than once; just the same, keep an eye on that as time goes on, and if you get weird build errors, check build-dev to see if the plugs changed.
  • On my system, this resides in C:\Prg\OpenJDK\deps\openjdk-binary-plugs. (The .jar file is an executable jar, so just "java -jar" it and tell it to install in C:\Prg\OpenJDK\deps; it adds the rest. It's done in 3 seconds, and consists of 1 file. Now you see why I wouldn't worry too much about this.)
• Mercurial. This is a distributed revision control system, and there's lots more to say about that at the Mercurial website. Its commands look somewhat similar to SVN, though definitely read "Distributed Revision Control with Mercurial" if you're going to be keeping up with the source trees as they go. You *want* the "forest" extension as part of your Mercurial binary, so grab the "Batteries Included" installer version. I went with the non-TortoiseHG version, and had to download all four of the released files off that page and install and uninstall and install and uninstall until I found one that worked (the "win32extrasp1.exe" version in the "dec" release list on Sourceforge).
  • On my system, Mercurial lives in C:\Prg\Mercurial. Put in on the PATH so you have access to "hg.exe".
  • CONTRIBUTION: Figure out what the differences are and post it someplace--how to get the "forest" extension installed and turned on in Mercurial was a pain; Google was of little to no help here. (Tag it as a comment to this blog entry, if you like, and I'll update the entry itself once I see it.)
  • Update: Daniel Fuchs blogs about how to get Mercurial's "forest" extension installed in your installation, in case you don't get the "Batteries Included" version:

    I simply cloned the forest repository in c:\Mercurial. In a cygwin terminal:

      cd c:/Mercurial
      hg clone http://www.terminus.org/hg/hgforest  hgforest
    

    Then I edited c:/Mercurial/Mercurial.ini and added the lines:

      [extensions]
      forest=c:/Mercurial/hgforest/forest.py
    

    as documented in the Mercurial Wiki.

• Optional: FindBugs. The build will start using FindBugs to do source-analysis to find bugs before they happen, so it's not a bad idea to have that as well.
  • On my system, this resides in C:\Prg\findbugs-1.2.1.

At this point, you should be ready to go.

2. Fetch the source.

Ivan's got it (mostly) right: just do this:

cd C:\Prg\OpenJDK

md jdk7

hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7

(Don't use MASTER as he does, I don't think that works--that was just for the experimental repositories.) Don't forget the trailing slash, either, or you'll get an error saying something along the lines of http://hg.openjdk.java.net/jdk7%5Ccorba is not a valid URL or somesuch.

If your Mercurial doesn't have the "forest" extension, "fclone" won't work; Ivan's got tips on how to pull down the sub-repositories by hand, but I get nervous doing that because what if the list changes and I wasn't paying attention?

Your network will go wild for about twenty minutes or so, maybe longer, pulling lots of stuff down from the URL above. The sources should now reside in C:\Prg\OpenJDK\jdk7. Go browse 'em for a bit, if you feel so inclined. Get a good rush going, because this next step can be the tricky one.

Update: Volker Simonis ran into some issues with using Mercurial and an HTTP proxy, and found it difficult to find assistance across the Web. In the interests of making it easier for others, he's allowed me to republish his experience here:

I just had a real hard time to get the forest extension working and finally found out that it was because the forest extension doesn't honor the "http_proxy" environment variable. So I thought I'll post it here in case anybody else will face the same problem in order to save him some time. (If you're only interested in the solution of the problem, you can skip the next paragraphs and jump right to the end of this post).

I installed Mercurial and the forest extension as described in various places, here on the list and on the Web - that was the easy part:) Afterwards I could do:

/share/software/OpenJDK> hg clone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
requesting all changes
adding changesets
adding manifests 
adding file changes
added 2 changesets with 26 changes to 26 files
26 files updated, 0 files merged, 0 files removed, 0 files unresolved

So everything worked fine! Note that I'm behind a firewall, but Mercurial correctly picked up my http proxy from the "http_proxy" environment variable!

But now, everytime I tried 'fclone', I got the following error:

/share/software/OpenJDK> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 
[.]
abort: error: Name or service not known

That was not amusing. First I thought I got something wrong during the installation of the forest extension. I than used the '--traceback' option to "hg" which told me that the error was in keepalive.py:

/share/software/OpenJDK> hg --traceback fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 
[.] 
Traceback (most recent call last):
...
File "/share/software/Python-2.5.1_bin/lib/python2.5/site-packages/mercurial/keepalive.py", 
line 328, in _start_transaction raise urllib2.URLError(err)
URLError: <urlopen error (-2, 'Name or service not known')>
abort: error: Name or service not known 

So I enabled the debugging output in keepalive.py and realized, that the first two connections to hg.openjdk.java.net where made trough the proxy, while the third one (the first that actually fetches files), wants to go directly to hg.openjdk.java.net, which badly fails:

/share/software/OpenJDK> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
keepalive.py - creating new connection to proxy:8080 (1078835788)
keepalive.py - STATUS: 200, OK
keepalive.py - re-using connection to proxy:8080 (1078835788)
keepalive.py - STATUS: 200, OK 
[.]
keepalive.py - creating new connection to hg.openjdk.java.net (1078970092)
abort: error: Name or service not known

The problem can be fixed by adding the proxy settings to your .hgrc file, like this:

[http_proxy]
host=proxy:8080

where you have to replace "proxy:8080" with the name and the port of your real proxy host!

Volker's original email came from the buid-dev list, and if you have further questions about Mercrial and HTTP proxies, I'd suggest that as a resource.

3. Set up your environment.

This is where you'll spend a fair amount of time, because getting this right can be ugly. There's some environment variables that tell the build script where to find things, and we have to point out those things, like compiler location and such. If you installed everything in the same places I did, then the following, which I put into C:\Prg\OpenJDK\build_shell.sh, should work for you:

#!/bin/sh

# "External" bits (outside of OpenJDK path structure)
#
export ALT_BOOTDIR=C:/Prg/jdk1.6.0
export ANT_HOME=c:/Prg/apache-ant-1.7.0
export FINDBUGS_HOME=/cygdrive/c/Prg/findbugs-1.2.1

# OpenJDK flag (to make FreeType check pass)
#
export OPENJDK=true

export OPENJDK_HOME=C:/Prg/OpenJDK
openjdkpath=$(cygpath --unix $OPENJDK_HOME)

# OpenJDK-related bits
# (ALT_JDK_IMPORT_PATH fixes a corba bug; remove it later)
#
export ALT_JDK_IMPORT_PATH=$(cygpath --unix C:/Prg/jdk1.6.0)
export ALT_BINARY_PLUGS_PATH=$OPENJDK_HOME/openjdk-binary-plugs
export ALT_UNICOWS_DLL_PATH=$OPENJDK_HOME/deps/unicows
export ALT_FREETYPE_LIB_PATH=$OPENJDK_HOME/deps/freetype-igor/windows/freetype-i586/lib
export ALT_FREETYPE_HEADERS_PATH=$OPENJDK_HOME/deps/freetype-igor/windows/freetype-i586/include/freetype2
. $openjdkpath/jdk7/jdk/make/jdk_generic_profile.sh

# Need GNU make in front of Cygwin's; this is the only practical way to do it
#
PATH=$openjdkpath/bin:$PATH
export PATH

# Let people know this is an OpenJDK-savvy prompt
#
export PS1='OpenJDK:\[\e]0;\w\a\]\[\e[32m\]\u@${COMPUTERNAME}:\[\e[33m\]\w\[\e[0m\]\n\$ '

Note the UNICOWS_DLL thing in the middle; this was necessary in earlier builds, and I don't know if it still is. For now I'm not messing with it; if you discover that you need it, the Build README has links. (Update: Kelly confirmed that this is no longer necessary in the OpenJDK build. Yay!)

Note that I set the COMPILER_VERSION flag to tell the build script which compiler I'm using--if that's not set, the build fails pretty quickly, complaining that "COMPILER_VERSION" cannot be empty. (Update: Kelly mentions, "I suspect the reason you are having the COMPILER_VERSION problem is that the makefiles are trying to run cl.exe to get the version, and if the PATH/LIB/INCLUDE env variables are not setup right, cl.exe fails. Several people have run into that problem.")

Note that OPENJDK must be set, or the build process thinks this is a commercial build, and an early sanity-check to see what version of FreeType is running will fail. (Specfically, Sun builds a tool just to see if the code compiles; if it fails to compile, chances are you forgot to set this flag. That's been my problem, each and every time I tried to rebuild the OpenJDK build space. Hopefully I never forget it again.)

Note that I call into jdk_generic_profile.sh to do some more setup work; this gets all the MSVS2003 stuff into the PATH as well.

Be very careful with which path you use; sometimes the build wants C:/Prg style paths, and sometimes it wants /cygdrive/c/Prg style paths. Don't assume the script above is perfect--I'm still testing it, and will update this entry as necessary as I find bugs in it.

(Update: Kelly mentions, "Be careful putting cygwin before VS2003 in the PATH, cygwin has a link.exe and so does VS2003, you need the one from VS2003.")

From a Cygwin bash prompt,

cd /cygdrive/c/Prg/OpenJDK

. ./build_shell.sh

cd jdk7

make sanity

It will churn, think, text will go flying by, and you will (hopefully) see "Sanity check passed". If not, look at the (voluminous) output to figure out what paths are wrong, and correct them. Note that certain paths may be reported as warnings and yet the buld will still succeed, that's OK, as far as I can tell.

And no, I don't know what all of those environment variables are for. Kelly might, but I suspect there's a lot of built-up cruft from over the years that they'd like to start paring down. Let's hope.

4. Type "make" and go away for a while.

Specifically, type "make help" to see the list of targets.

OpenJDK:Ted@XPJAVA:/cygdrive/c/Prg/OpenJDK/jdk7
$ make help
Makefile for the JDK builds (all the JDK).

--- Common Targets ---
all -- build the core JDK (default target)
help -- Print out help information
check -- Check make variable values for correctness
sanity -- Perform detailed sanity checks on system and settings
fastdebug_build -- build the core JDK in 'fastdebug' mode (-g -O)
debug_build -- build the core JDK in 'debug' mode (-g)
clean -- remove all built and imported files
clobber -- same as clean

--- Common Variables ---
ALT_OUTPUTDIR - Output directory
OUTPUTDIR=./build/windows-i586
ALT_PARALLEL_COMPILE_JOBS - Solaris/Linux parallel compile run count
PARALLEL_COMPILE_JOBS=2
ALT_SLASH_JAVA - Root of all build tools, e.g. /java or J:
SLASH_JAVA=J:
ALT_BOOTDIR - JDK used to boot the build
BOOTDIR=C:/Prg/jdk1.6.0
ALT_JDK_IMPORT_PATH - JDK used to import components of the build
JDK_IMPORT_PATH=c:/Prg/JDK16~1.0
ALT_COMPILER_PATH - Compiler install directory
COMPILER_PATH=C:/Prg/MSVS2003/Common7/Tools/../../Vc7/Bin/
ALT_CACERTS_FILE - Location of certificates file
CACERTS_FILE=/lib/security/cacerts
ALT_DEVTOOLS_PATH - Directory containing zip and gnumake
DEVTOOLS_PATH=/usr/bin/
ALT_DXSDK_PATH - Root directory of DirectX SDK
DXSDK_PATH=C:/Prg/DIRECT~1
ALT_MSDEVTOOLS_PATH - Root directory of VC++ tools (e.g. rc.exe)
MSDEVTOOLS_PATH=C:/Prg/MSVS2003/Common7/Tools/../../Vc7/Bin/
ALT_MSVCRT_DLL_PATH - Directory containing mscvrt.dll
MSVCRT_DLL_PATH=C:/WINDOWS/system32
WARNING: SLASH_JAVA does not exist, try make sanity
WARNING: CACERTS_FILE does not exist, try make sanity

--- Notes ---
- All builds use same output directory unless overridden with
ALT_OUTPUTDIR=<dir>, changing from product to fastdebug you may want
to use the clean target first.
- JDK_IMPORT_PATH must refer to a compatible build, not all past promoted
builds or previous release JDK builds will work.
- The fastest builds have been when the sources and the BOOTDIR are on
local disk.

--- Examples ---
make fastdebug_build
make ALT_OUTPUTDIR=/tmp/foobar all
make ALT_OUTPUTDIR=/tmp/foobar fastdebug_build
make ALT_OUTPUTDIR=/tmp/foobar all
make ALT_BOOTDIR=/opt/java/jdk1.5.0
make ALT_JDK_IMPORT_PATH=/opt/java/jdk1.6.0

OpenJDK:Ted@XPJAVA:/cygdrive/c/Prg/OpenJDK/jdk7
$

The one I want is "make fastdebug_build" or "make debug_build" (so I have debug symbols and can go spelunking). Do it.

Watch the stuff go flying by.

Get bored.

Get lunch.

Come back, it might be done.

If it's a successful build, you'll have "stuff" in the C:\Prg\OpenJDK\jdk7\build directory, corresponding to the kind of build you kicked off; for a "fastdebug" build, for example, there'll be a "windows-i586-fastdebug" directory in which you'll find a "j2sdk-image" directory in which there should be a complete JDK environment. Try running Java and see if it works (make sure your PATH is updated to point to the right place before you do!)

If not, it's debugging time. Note that the "build" directory is completely built from scratch, so if you get a partial build and want to start over from scratch, just "rd /s build" from the jdk7 directory. (It's easier than "make clean" or "make clobber", I've found.)

More About Builds

When building the JDK, you may want to build bits "underneath" the top-level directory, but doing this is a tad tricky. I asked about this on the build-dev list, and Kelly responded with a great email about the build process, specifically about launching "sub" makes within the system:

Due to history, a build directly from the jdk/make directories uses a default OUTPUTDIR of jdk/build/* but if FASTDEBUG=true, it's jdk/build/*-fastdebug, or if a plain debug build with just VARIANT=DBG it would be jdk/build/*-debug The variant builds leave results in a completely separate outputdir.

If you used the very top level makefile (which came from the now defunct control/make area) the default OUTPUTDIR is ./build/* (at the very top of the repositories). When this top level Makefile runs the jdk/make Makefiles, it passes in a ALT_OUTPUTDIR to refer to this top level build result area because it's default outputdir is not the same place.

I don't know the complete history as to why this was done this way, but my tendency is to try and get us back to a single default OUTPUTDIR for all the repositories. Someday...

This is what I do when I work on just the jdk repository:
cd jdk/make && gnumake

That primes the outputdir area, then I can drop down in:
cd jdk/make/java && gnumake

Or even drop in and clean an area and re-build it:
cd jdk/make/jpda && gnumake clean && gnumake

Or just repeat the entire build (incremental build)
cd jdk/make && gnumake

If I wanted the jdk image (j2sdk-image), I would need to:
cd jdk/make && gnumake image

But the output by default will go to jdk/build/* and a different directory if VARIANT=DBG or FASTDEBUG=true.

This should help having to go through the whole process for incremental updates. (Note that on my system, I had to call it "make" instead of "gnumake".)

Futures

As time goes on, I'll hopefully find the time to blog about how to find various little things in the JDK and make source-modifications to prove that they work, and use that as a springboard from which to let you start hacking on the JDK. In the meantime, please, if you run into trouble and find fixes to any of the above, comment or email me, and I'll correct this.

Contributions/Suggested TODOs

• Have the make system automatically capture the results of the build in a log file, for easier debugging. Granted, you can "make >| build.output", but that seems tedious when it could easily be captured automagically for you each time. ("make sanity" does this, capturing the results in build/windows-i586-fastdebug/sanityCheckMessages.txt and sanityCheckWarnings.txt.
• Documentation, documentation, documentation. This thing does a lot of recursive makes and invokes a lot of tools (some of them built as part of the build process itself), and it would be much easier to debug and understand if the process were better documented. Even a simple flowchart/tree of the various Make invocations and what each does (in practice) would be helpful when trying to track down issues.
• Add support for all output to be captured into a build log file. This can obviously be done at the command-line via "make |& tee build.log" or ">& build.log", but I think it'd be nice if it were somehow folded in as part of the build process so it happened automatically.

Updates

1. Kelly OHair sent me some updated information, such as the right link to use for the README file (from the HG repository instead of the SVN one).
2. Kelly also mentioned that the plugin and installers are not part of the OpenJDK build yet, so that's not something I could build even if I wanted to. Which is fine, for me.
3. Kelly confirmed that UNICOWS is not needed in OpenJDK.
4. Kelly mentions that link.exe on the PATH must be VS2003's, not Cygwin's.
5. Kelly mentions the COMPILER_VERSION problem might be PATH issues.
6. Kelly notes, "On the C:\ vs C:/ vs. /cyg*/ paths, I try and use the C:/ all the time everywhere, and anywhere in the Makefiles where I know I need C:\ or the /cyg*/ paths, I try and use cygpath or MKS dosname to force them into the right form. NMAKE.EXE only likes C:\ paths, and cygwin PATH only wants /cyg*/ paths, and Windows/Java/MKS never want /cyg*/ paths. :^( I wish we had a better solution on Windows for this shell/path mania."
7. Poonam's Weblog has a good page on building the OpenJDK on Windows with NetBeans, from which I've stolen... ahem, leveraged... some links. His webpage has a link for the UNICOWS download page, but that only includes the DLL, not the .LIB, which you will also need. (It's an import library for the DLL; you need both.) The only place I know to get the .LIB is from the Platform SDK, and you need an old one, circa November 2001 or so. I don't know if it's kosher to redistribute any other way (meaning, Microsoft won't let us, as far as I know).
8. Added Volker Simonis' experiences with HTTP proxies in Mercurial
9. Added the "sub" make build discussion from Kelly on build-dev
10. Added the link to Volker's blog about building on Suse Linux

From the cover of Dr. Dobb's Journal (Jan/2008):

PHP: The Power Behind Web 2.0

Only because you're saying "Web 2.0", mind you. *shudder* All those in favor of "alt.death.to.tim.oreilly.for.coining.that.phrase" newsgroup, say Aye....

You know, for years now, I've had just a terrible time making out Christmas lists.

The fact is, I'm fortunate enough to be in a position where if I really want something, I'm usually able to just go out and get it. If it's more expensive than what a single trip to the store armed with a credit card can obtain, then it's probably something I don't really need. (High-end gaming desktop machines do not fall into that category, no matter what my family might suggest.) So, it's rather difficult to come up with a list of things for friends and family to buy for me, particularly since impulse control is... well... almost entirely lacking in my case.

One year, I just said, "Make a charitable donation in my name to your favorite charity." I thought this was a nice solution, but it didn't go over incredibly well, probably because my family likes to see the whole "wow, thanks!" while we're all sitting underneath the tree. I ended up with the usual range of CDs, books, games, and so on. Now, mind you, I didn't return any of those things, but still, I felt vaguely guilty at my rampant consumerism when it's apparent with just a glance through any world-facing publication that there's a lot of people who could've used that money to survive for the year on what one CD cost.

I just figured out what I want for Christmas.

Those few of you who are non-geeks may not have heard of the "One Laptop Per Child" project: it's a charitable group that has created a (relatively) inexpensive laptop designed for use by underprivileged children to help close the gap between the technology "haves" and "have-nots". Now, in of itself, this is a Good Thing, but for a limited time, the foundation is sweetening the pot significantly:

From now through December 31, 2007, OLPC is offering a Give One Get One program in the United States and Canada. This is the first time the revolutionary XO laptop has been made available to the general public. For a donation of $399, one XO laptop will be sent to empower a child in a developing nation and one will be sent to the child in your life in recognition of your contribution. Up to $200 of your donation may be tax-deductible (your $399 donation minus the fair market value of the XO laptop you will be receiving).

For all U.S. donors who participate in the Give One Get One program, T-Mobile is offering one year of complimentary HotSpot access.

If you participate in Give One Get One today, you will receive your XO laptop(s) in early 2008. Your donated laptop will reach a child in Afghanistan, Cambodia, Haiti, Mongolia or Rwanda in the same early 2008 timeframe.

Aside from the fact that the T-Mobile access is a useful thing for a guy who's one the road close to 75% of his life, and aside from the fact that my family can take the "get one" laptop and give it to my 8-year-old as a starter laptop (or take it down to the local women's shelter and give it to a child there who probably would otherwise never set fingers on a working computer keyboard), the simple fact is, it's a nifty little gadget. Granted, I'm not going to give up my MacBookPro anytime soon over it, but it's got enough functionality to access wireless networks, and provides a Smalltalk-based environment and Python-based environment in which people can learn to program, and what's not to like about that?

Dear Santa--

I think I've been a good boy this year, but there are others in the world who've been better than me in circumstances that I can't even begin to imagine. So please, this year, make a donation to the One Laptop Per Child initiative, in either my name or yours.

Thanks, from both myself and someone neither of us has met yet,

Ted

I grieve with him. I can't imagine how the babysitters feel right now--at age 13 and 11, to have to make the hard choices any babysitter has to make in a the event of a fire, and to have to live with that choice for the rest of your life. To be the parents, to have to live with the "what-might-have-beens" and the "if-onlys".

And to have to learn, as Brian talks about, how to go on.

No parent should ever have to bury their child.

From wikipedia:

The Turing test is a proposal for a test of a machine's capability to demonstrate intelligence. Described by Alan Turing in the 1950 paper "Computing machinery and intelligence," it proceeds as follows: a human judge engages in a natural language conversation with one human and one machine, each of which try to appear human; if the judge cannot reliably tell which is which, then the machine is said to pass the test.

From C.NET: "Warning sounded over 'flirting robots'"

Those entering online dating forums risk having more than their hearts stolen.

A program that can mimic online flirtation and then extract personal information from its unsuspecting conversation partners is making the rounds in Russian chat forums, according to security software firm PC Tools.

The artificial intelligence of CyberLover's automated chats is good enough that victims have a tough time distinguishing the "bot" from a real potential suitor, PC Tools said. The software can work quickly too, establishing up to 10 relationships in 30 minutes, PC Tools said. It compiles a report on every person it meets complete with name, contact information, and photos.

"As a tool that can be used by hackers to conduct identity fraud, CyberLover demonstrates an unprecedented level of social engineering," PC Tools senior malware analyst Sergei Shevchenko said in a statement.

Among CyberLover's creepy features is its ability to offer a range of different profiles from "romantic lover" to "sexual predator." It can also lead victims to a "personal" Web site, which could be used to deliver malware, PC Tools said.

Daaaaaamnnnnnn... you can offer a range of different profiles. From "romantic lover" to "sexual predator".

Can the "Make You Happy All Night Long" Web Service be far behind?

I would love to get my hands on this code, though. Think about what the natural language parsing routines must be like, particularly considering they have to be able to parse smileys, too.

If they can mimic flirting, how far away are we from code that mimics online help agents? It'd be like voicemail, but worse, since you could never know if you were talking to a real person or not....

This is, without a doubt, the most accurate quote ever about the "fun" of writing a book:

Writing a book is an adventure. To begin with, it is a toy and an amusement; then it becomes a mistress, and then it becomes a master, and then a tyrant. The last phase is that just as you are about to be reconciled to your servitude, you kill the monster, and fling him out to the public. (Source: Winston Churchill)

Keep that in mind, all you who are considering authoring as a career or career supplement.

Were I to offer my own, it would be like so:

Writing a book is like having a child.

Trying is the best part, in some ways. You have this idea, this burning sensation in your heart, that just has to get out into the world. But you need a partner, a publisher who will help you bring your vision to life. You write proposals, you write tables of contents, you imagine the book cover in your mind. Then, YES! You get a publisher to agree. You sign the contract, fax it in, and you are on the way! We are authoring!

At first, it is wonderful and exciting and full of potential. You run into a few hangups, a few periods of nausea as you realize the magnitude of what you're really doing. You resolve to press on. As you continue, you begin to feel like you're in control again, but you start to get this sense like it's an albatross, a weight around your neck. Before long, you're dragging your feet, you can't seem to muster the energy to do anything, just get this thing done. The deadline approaches, the sheer horror of what's left to be done paralyzes you. You look your editor in the eye (literally or figuratively) and say, "I can't do this." The editor says, "Push". You whimper, "Don't make me do this, just cancel the contract." The editor says, "Push". You scream at them, "This is YOUR fault, you MADE me do this!" The editor says, "Push". Then, all of a sudden, it's done, it's out, it's on the shelf, and you take photos and show it off to all the friends, neighbors and family, who look at you a little sympathetically, and don't mention how awful you really look in that photo.

As the book is out in the world, you feel a sense of pride an joy at it. You imagine it profoundly changing the way people look at the world. You imagine it reaching bestseller lists. You're already practicing the speech for the Nobel. You're sitting in your study, you reach out and grab one of the free copies still sitting on your desk, and you open to a random page. Uh, oh. There's a typo, or a mistake, or something that clearly got past you and the technical reviewers and the copyeditors. Damn. Oh, well, one mistake can't make that much difference.

Then the reviews come in on Amazon. People like it. People post good reviews. One of them is not positive. You get angry: this is your baby they are attacking. How DARE they. You make plans to find large men with Italian names and track down that reviewer. You suddenly realize your overprotectiveness. You laugh at yourself weakly. You try to convince yourself that there's no pleasing some people.

Then someone comes up to you at a conference or interview or other gathering, and says, "Wow, you wrote that? I have that book on my shelf!" and suddenly it's all OK. It may not be perfect, but it's yours, and you love it all the same, warts and all.

Nearly a dozen books later, it's always the same.

Michael Nygard (author of the great book Release It!), writes that "[his] definition of 'done' continues to expand". Currently, his definition reads:

A feature is not "done" until all of the following can be said about it:

1. All unit tests are green.
2. The code is as simple as it can be.
3. It communicates clearly.
4. It compiles in the automated build from a clean checkout.
5. It has passed unit, functional, integration, stress, longevity, load, and resilience testing.
6. The customer has accepted the feature.
7. It is included in a release that has been branched in version control.
8. The feature's impact on capacity is well-understood.
9. Deployment instructions for the release are defined and do not include a "point of no return".
10. Rollback instructions for the release are defined and tested.
11. It has been deployed and verified.
12. It is generating revenue.

Until all of these are true, the feature is just unfinished inventory.

As much as I agree with the first 11, I'm not sure I agree with #12. Not because it's not important--too many software features are added with no positive result--but because it's too hard to measure the revenue a particular program, much less a particular software feature, is generating.

My guess is that this is also conflating the differences between "features" and "releases", since they aren't always one and the same, and that not all "features" will be ones mandated by the customer (making #6 somewhat irrelevant). Still, this is an important point to any and all development shops:

What do you call "done"?

For those who've seen me give presentations at the No Fluff Just Stuff symposia over the year who've also asked to get the demo code that I write during the presentations, I've put it all up online. Simply find the show you attended below, and the hyperlink downloads the associated .zip file:

• Boulder JUG
• Denver JUG
• Phoenix JUG
• Atlanta
• Austin
• Boston
• Chicago
• Columbus
• Dallas
• Denver
• Green Bay
• London
• Miami
• Milwaukee
• Minneapolis (1st show)
• Minneapolis (2nd show)
• Oklahoma City
• Omaha
• Orlando
• Phoenix
• Portland
• Princeton, NJ
• Raleigh, NC
• Reston, DC
• Seattle
• St Louis
• Toronto

Usual disclaimer applies, as with all demo code: I don't guarantee the code for any particular purpose except as a supplement to the lecture that was given at that time in that city, the code isn't guaranteed to do anything except soak up space on your hard drive, I can't be held responsible if you use it in production, and so on.

If any of the links are broken, email me and let me know, but I think they should all work.

... for Ruby, or PowerShell/.NET? I'm looking for something to make it easier to use WebDAV from a shell scripting language on Windows; Ruby and PowerShell are the two that come to mind as the easiest to use on Windows. For some reason, Google doesn't yield much by way of results, and I've got to believe there's better WebDAV support out there than what I'm finding.

(Yes, I could write one, but why bother, if one is out there that already exists? DRY!)

BTW, anybody who finds one and wants credit for it, I'll be happy to post here. If you're a commercial vendor and you send me a license to go with it, I'll post that, too, with some code and explanation on how I'm using it, though I doubt it's going to be all that different from how anybody else would use it.

Some of you knew me way back when I started the domain name "javageeks.com". Obviously, it's been a while since I updated that site, and in fact, if you go try to read some of the papers there, you'll get redirected to "neward.net", which has been on hiatus ever since the ISP there decided that my use of JSP was somehow trashing their server. (This is what happens when college students run an ISP in the spare time, I guess.)

Anyway, for a variety of reasons, I've since decided that my online presence should coalesce into one place, this domain (tedneward.com), and I've been slowly moving things over to this site in recognition of that fact. As a result, a bunch of the papers that I'd put up for reading on javageeks.com are now here on tedneward.com, in the "Writing" section of the website.

For those of you too tied to RSS to go visit the site itself, here's a list of the papers I've moved over thus far:

• Using the Boot Classpath
• Finding Loaded Classes
• Using Java Policy
• When is a Static Not Static?
• Multiple Java Homes
• X-Power: Using XML to Write Papers

It's not the complete list, since I'm also (trying to) move the authoring format over from my own home-grown XML+XSLT system into DocBook, but that means either developing an XSLT to transform my home-grown format into DocBook, or else just manually switching them over. I'm not convinced yet which is the faster way to go.

Part of it isn't just transmuting their form, though. Some of them could use with a touch of updating (such as the Java Policy one--there's been a few changes there since I wrote that paper, back in Java 1.2 days), so I think I'm going to have to labor through and transform and update them manually. Still, it could be worse: I could have written them in Lotus Ami Pro....

Now, I've had my issues with corporate management before (some of you may recall my run-in with Sun lawyers over the domain "javageeks.com"), and I've seen corporations behave badly with cease-and-desist letters (Hey, Microsoft... you and RedHat can come out of the corner now...), but this action on Sun's part to effectively muscle out the four project leads on the open-source OpenDS project definitely deserves some kind of award.

(For those of you who want the short version: four Sun employees, including Neil Wilson, the poster, have been working on the OpenDS project for the last n number of years. Sun decides to offshore its directory services development to Europe (WTF?!?), and pink-slips the four project owners. The project owners decide to continue working on the project, and decide that there should be a fifth owner, a Sun employee, in order to make sure Sun keeps a hand in a project it vested time & energy & money into. Sun decides instead to require that ALL the project owners be Sun employees. Project owners, a tad distrubed, circle back to discuss things. Sun management threaten to yank all four ex-employees' severance benefits if they fight this. Project owners basically decide to bail entirely, and not anger the Sun Gods any further, until after the benefits have been paid out. Project owner Wilson blogs the aforementioned story, which annoys a few dark corners of the blogosphere, including yours truly.)

In fact, in honor of the book "In Search of Stupidity", I hereby nominate Sun for a Stupidity Award, in this case, a two-parter: one, "For having the incredible temerity to threaten with termination the very core group of people who built the product into what it was", and two, "For having the absolute chutzpah to effectively backstab the very community-oriented efforts that Sun upper management are seeking to build as a way to make the company once again relevant in the computing industry".

I dunno if anybody at Sun reads my blog, but if you do, forward this over to Simon Phipps and Jonathan Schwartz, because if they want to be taken seriously as an open-source company, they need to find the morons who acted with this kind of heavy-handedness and defenestrate them at the earliest opportunity. An open letter of apology and response should be posted on one or both blogs. Then, they need to go back to Mr. Wilson, apologize profusely, and ask him and the other three--very nicely--if they'll agree to take up the reins of the OpenDS project again and allow Sun to demonstrate--by actions, not words--that they are a community-focused company.

Or, I suppose, they could do none of this, and prove--by actions, not words--that they don't give a sh*t. (Frankly, I could care less about directory services projects, but I would think Sun cares far more about its reputation at this point than the project itself.)

'Tis your move, Sun. Are you "profoundly involved in the open source world", or not?

Update: Simon Phipps writes, in a comment here (which may not be accessible to some RSS readers, so I will quote in full):

Ted: I have investigated this situation. It's very regerttable that Sun had to let some great engineers go as part of a staff reduction, unrelated to the governance issue, but sadly that happens in every company. The real issue I was concerned about here was the discussion of governance. I discovered that Neil did not tell the whole story.

The original governance of OpenDS said this:

<blockquote>The OpenDS project has single, overall Project Lead. This Project Lead, who is appointed by Sun Microsystems, is responsible for managing the entire project, and is the final arbiter of all decisions.</blockquote>

At some point, without any community discussion I can find, and without management discussion, while still Sun employees, the leaders of OpenDS changed the governance to say this:

<<The OpenDS project has single, overall Project Lead. This Project Lead, who is appointed and removed by a majority vote of the Project Owners, is responsible for managing the entire project, and is the final arbiter of all decisions in the development process.>>

You can see the change in the Subversion system for OpenDS here.

When Neil says:

<<On November 14, 2007, a member of executive management within Sun’s software division contacted one of the recently-laid-off OpenDS project owners and demanded that the owners approve a governance change that would grant Sun full control of the OpenDS project.>> that is misleading. What actually happened is the leaders were asked to revert their arbitrary change of the governance (they also removed the phrase "Project Owners will always attempt to reach consensus before making decisions") so that the matter could be openly discussed.

I'm continuing to investigate and monitor, but right now the evidence I see is that Sun has not acted in bad-faith towards the governance. I'd welcome evidence to the contrary if it exists, but that SVN dif seems pretty conclusive to me.

There are, I think, two issues here that are getting conflated, and I want to tease them apart. What is not an issue, as far as I'm concerned, was Sun's decision to let these engineers go in the first place. Business decisions that involve letting people go are always hard decisions to make, and I won't second-guess the thought process that led up to it. I'm willing to believe that if Sun management felt they could have kept them in place, they would have. (You may choose to disagree but that's your own value judgement to make.) That doesn't take away from the two issues I want to discuss here.

First, there is the issue of whether or not Sun's actions were in accordance with the project governance model for the OpenDS project, which Simon seems to focus on above. Frankly, I could care less about this aspect of things; whether that change was appropriate, necessary, legal, whatever, is not the issue I want to discuss or point out. Yes, there is the possibility that Neil-and-company's actions were regarded as being contrary to the charter of the project by the legitimately-selected Project Owner, and Sun felt it necessary to retaliate legally. Even in that scenario, it strikes me as a heavy-handed reaction.

(Quite footnote: on a whim, after posting my update, I checked the SVN revisions beyond what Simon posted. The revision change in question, from 1622 to 1739, occured on April 28th, 2007, which is close enough to have been part of this whole fiasco, but obviously I'm wasn't there, I can't know that for certain. Needless to say, if all these changes occurred without any community input, that's clearly not playing to the spirit of the open source community, IMHO.)

Secondly, there is the issue of Sun's threat to revoke their severance benefits. To me, this is unconscionable. Severance is a way of providing valua