DTrace, FreeBSD 9.0, and Erlang

While working on the DTrace probes for the Erlang R15 release (which I’ve blogged about earlier this month), I discovered that the build recipe for FreeBSD 9.0 was broken.  Oh no, my favorite OS doesn’t work, sound the alarm!

FreeBSD has supported kernel-space DTrace probes for a while.  However, support for user-space probes, USDT, has not arrived until FreeBSD 9.0RC1.  ”RC1” means “Release Candidate #1”.  As of this writing, RC2 is the newest release.  The official release of FreeBSD 9.0 (to be called FreeBSD 9.0-RELEASE) will not be ready for at least another month or perhaps longer; see the 9.0 wiki release schedule  for more details.

As I mentioned in the introduction to this article, USDT probes are a bit broken in FreeBSD 9.0RC2.  I’ve added two different kinds of probes to the Erlang virtual machine:

  1. Probes that are defined as part of the core virtual machine.  These probes fire in response to events within the virtual machine, e.g. spawning a new process or a garbage collection event.  These probes are defined in [erlang_dtrace.d](https://github.com/slfritchie/otp/blob/dtrace-review3/erts/emulator/beam/erlang_dtrace.d).
  2. A probe that can be triggered directly by Erlang code, e.g., `dtrace:p(42, "Hello, world!").` This probe is defined in [dtrace_user.d](https://github.com/slfritchie/otp/blob/dtrace-review3/lib/dtrace/c_src/dtrace_user.d) and by the Erlang module [dtrace.erl](https://github.com/slfritchie/otp/blob/dtrace-review3/lib/dtrace/src/dtrace.erl).

The probe that can be fired directly from Erlang code must be compiled using position-independent code (PIC) and assembled into a shared library.  The Erlang function dtrace:init() will load the shared library into the virtual machine and initialize the Erlang<->C glue necessary for an Erlang function to call a C function.

Both FreeBSD 9.0RC1 and FreeBSD 9.0RC2 cannot compile and link to PIC code.  So all of the probes in erlang_dtrace.d will work correctly (because they’re not part of a shared library), but the probe in dtrace_user.d will not work correctly.

There is a problem report (PR) for FreeBSD that discusses this PIC problem: kern/159046: [dtrace] [patch] dtrace library is linked with a wrong flags on -CURRENT.  I emailed an update to that ticket this afternoon, asking that Alex Samorukov’s fix be folded into the 9.0-RELEASE of FreeBSD.  I have no idea if it will happen in time or not … but I have my fingers crossed.

In the meantime, I have a fix for those who are using one of the FreeBSD release candidates where the PIC bug is still present.  You have a couple of options:

  1. If you're familiar with building & installing FreeBSD kernels and FreeBSD user-space programs, then following the advice in the FreeBSD problem report (linked above) will work just fine.
  2. Download PIC-compiled copies of the system libraries that are required for PIC-compiled trace code and install them manually.

If you choose option #2, then I have a step-by-step recipe for you:

  * Download the 5.5 KByte file [http://www.snookles.com/scott/misc/freebsd-9.0rcX.dtrace-pic-libs.tar.gz](http://www.snookles.com/scott/misc/freebsd-9.0rcX.dtrace-pic-libs.tar.gz) to your FreeBSD 9.0RCx's `/tmp` directory.
  * Run this command: `gunzip -c /tmp/freebsd-9.0rcX.dtrace-pic-libs.tar.gz | (cd / ; tar xvfp -)`

Good luck, and have fun with DTrace.

UPDATE 2011-Nov-29

It looks like the FreeBSD problem report has now been resolved, and the patch will be included in the 9.0-RELEASE distribution when it is released in the next month or three.  Very good news!