Meet Mr. DTrace - Part 4

Web April 30th, 2007

Eventually, we got the building environment working. Thanks to Ben and help from #gnusol, the missing piece is sunwbtool. We are going to build the Firefox first, then apply the patch to enable static probes in Javascript.

Using OpenSolaris binutils

Make sure you are using the OpenSolaris toolchain instead of GNU’s, otherwise the ld may crash with objects compiled with cc. Be careful to you $PATH, and check whether /usr/ccs/bin and $SUNWspro/bin is preferred over the generic /usr/bin.

And copy this configuration to your $HOME/.mozconfig, copied from Alex

. $topsrcdir/browser/config/mozconfig

ac_add_options –prefix=/opt/firefox
ac_add_options –disable-tests
ac_add_options –disable-debug
ac_add_options –disable-auto-deps
ac_add_options –disable-freetype2
ac_add_options –enable-official-branding
ac_add_options –enable-default-toolkit=gtk2
ac_add_options –enable-optimize=“-xO5 -xc99=no_lib”
ac_add_options –enable-static
ac_add_options –disable-shared
ac_add_options –enable-xft
ac_add_options –enable-svg

Building, building, building…

Run the following commands:

./configure
gmake 
#wait it to fail
cd config; gmake
cd ../js; gmake

If it works, we can apply the patch to add static probes:

cd js/src; patch -p2 < dtrace-js.diff
gmake jsdtrace.o
gmake

If you are in luck, you could get libmozjs.so. Now we need to preload it, using run-firefox.sh

libmozjs=/export/home/bookstack/work/firefox-1.5.0.7/trunk/js/src/libmozjs.so
export LD_PRELOAD=$libmozjs
export LD_LIBRARY_PATH=/usr/lib/firefox
export MOZ_PLUGIN_PATH=/usr/lib/firefox/plugins
# /usr/lib/firefox/firefox-bin -UILocale C -contentLocale c
/usr/lib/firefox/firefox-bin

Unfortunately, there is a core dump when loading the libmozjs.so whatever the dtrace support is added. I doubt that the Firefox in NexentaOS is built by GCC, and it is not binary compatible with the SunStudio, I might have to build the whole Firefox. Ooooh, that is painful, I even does not do that in my Gentoo Linux…

Another bad news is SunStudio could not parse glib’s head file since it does not support GCC extension, even with c99 enabled. I might ask some guru in mozdev mailing list about how to build Firefox with SunStudio 11.

Meet Mr. DTrace - Part 3

Web April 26th, 2007

Thanks to Ben Klang’s comments, I eventually make the SunStudio 11 works on NexentaOS!

Stick to the bleeding-edge

I migrated to NexentaOS Alpha-7 test2, since the SUNWhea is maintained in the unstable brach. Edited the /etc/apt/source.list to use unstable instead of testing, then upgraded the SUNWhea package, at the end “emerge” the SPROcc from the OpenSolaris DVD.

Test drive of DTrace probes provider

Copy the simple.c, myserv.d from Ticket #405, build it using SunStudio C Compiler:

cc -c simple.c
dtrace -G -32 -s myserv.d simple.o
cc -o simple myserv.o simple.o

Prepare the test.d like this:

#!/usr/sbin/dtrace -Zs

#pragma D option quiet

dtrace:::BEGIN
{
        printf(“Tracing… Hit Ctrl-C to end.\n);
}

myserv*:::loopstart-receive
{
        printf(“%d hit.\n, arg0);
}

dtrace:::END
{
        printf(” %-32s %-36s %8s\n, “FILE”, “FUNC”, “CALLS”);
}

Launch the simple with root privilege, execute the test.d with root privilege as well. The terminal would print out the value of counter i. Bravo!

Building the Mozilla Firefox

To build Mozilla Firefox, we need install the C++ compiler environment:

SPROcpl         Sun Studio 11 C++ Compiler
SPROscl         Sun Studio 11 Standard Class Library for C++
SPROsbld       Sun Studio 11 Linker Stab Library

To simplify the installation, –force option is added to emerge.py to install packages regardless the dependencies.

We also need to install libgtk2.0-dev and libidl-dev to meet the dependencies required by Firefox.

Fetch the source code using apt-get source firefox, edit the $HOME/.mozconfig,

. $topsrcdir/browser/config/mozconfig
ac_add_options –prefix=/opt/firefox
ac_add_options –enable-application=browser
ac_add_options –enable-optimize=”-xO2″

then invoke configure, make:

./configure
cd config ; make
cd ../js; make

CC reported an error at the linking time: /usr/ccs/bin/ar is not found. It seems that there is still some package missing, not quite sure it is binutils, I would ask for help from #gnusol tomorrow.

Meet Mr. DTrace - Part 2

Web April 24th, 2007

Since OpenSolaris Express Developer Edition asks for more than 768M memory, 8G disk, what a demanding monster! I could not fit it into VMWare on my laptop 700m. Therefore, before I order a new powerful desktop, I might stick to the NexentaOS right now.

SunStudio 11 is the native compiler in Solaris so it is supposed to work with DTrace static provider. Unfortunately there is no success story to install SunStudio 11 on NexentaOS so far according to the IRC discussion, I need to find my own way to work around this problem.

SunStudio is packaged in pkgcmd format, to make things even worse, NexentaOS drops the native pkgcmd, and replace it with a wrapper for Debian package. To automation the tedious unzip, copy, chown, chmod, ln and last but not the least, dependency resolving, I develop this python script

To use this script, you need to change your work directory to the packages, then you can list the available packages:

cd  /cdrom/sol_x11_x86/DeveloperTools/SunStudio/X86/kits/ide/packages
emerge.py -l .

We only care about the SunStudio C compiler, before we emerge it, we might take a look at the dependencies:

emerge.py -p SPROcc

calculating the dependency recursively ….
SUNWkvm  Core Architecture,  (Kvm)
SUNWhea  Header files
SUNWcsu  Core Solaris,  (Usr)
SUNWcsr  Core Solaris,  (Root)
SUNWtoo  Programming Tools
SPROlang                Components for Sun compilers and tools
SUNWesu  Extended System Utilities
SPROdwrfb              Dwarf Support Library binaries
SPROcc   
SUNWcar  Core Architecture,  (Root)

There are some packages missing, for example SUNWesu, since we are using APT package system, just ignore them, and install the package, emerge will recursively install all packages available in the current path:

emerge SPROcc

This procedure includes the following steps:

  • Calculate the dependencies
  • Unpack ${PKG}/archieves/none to the ${BASEDIR}
  • chmod, chown if necessary according to ${PKG}/pkgmap

After that, you need to add /opt/SUNWspro/bin to your ${PATH}, and compile the very first application, hello world. Oops, cc could not find stdio.h. It compiles simple C source code without library. It seems that the C Compiler is not well configured, I would figure it out later.

C++ Learning Note(5): Koenig Lookup

Development April 24th, 2007

This concept is well explained in the Wikipedia, and also in the codeproject. What confuses me is how this magic mentioned in Wikipedia is played:

While ADL makes it practical for free functions to be part of the interface of a class, it makes namespaces less strict and so can require the use of fully-qualified names when they would not otherwise be needed. For example, the C++ standard library makes extensive use of unqualified calls to std::swap to swap two values.

Here is my lousy approach to mimic the magic:

#include <iostream>
using namespace std;

void g();

void g() { cout << “g is outside NS” << endl; }

namespace NS {
        class A{};
        void f(A) { g(); cout << “f is called” << endl; }
        void g() { cout << “g is inside NS” << endl; }
}

int main()
{
        NS::A a;
        f(a);
}

with the following output:

g is outside NS
f is called

It looks good, suppose customized is missing, aka the Ln 6 is deleted. Compile, link, oops, gcc complains that the undefined reference to `g()’. Let’s move the g in the namespace before the f:

namespace NS {
        class A{};
        void g() { cout << “g is inside NS” << endl; }
        void f(A) { g(); cout << “f is called” << endl; }
}

It works, and the g inside NS namespace is called, no doubt, there is only g defined here. Let’s add a global g and check whether customized function is called, the full source code is:

#include <iostream>
using namespace std;

void g();

void g() { cout << “g is outside NS” << endl; }

namespace NS {
        class A{};
        void g() { cout << “g is inside NS” << endl; }
        void f(A) { g(); cout << “f is called” << endl; }
}

int main()
{
        NS::A a;
        f(a);
}

Wow, the output is:

g is inside NS
f is called

The global g is never called by NS::f! Is there something wrong? Please leave a feedback if you have an answer.

HOWTO install CELL Environment in Gentoo

Gentoo, HPC April 19th, 2007

The CELL BE SDK developed by IBM and Barcelona Supercomputing Center targets RedHat Fedora Core 6 host system. Using the rpm2targz and rpm, we can also install the CELL SDK in Gentoo Linux as well.

Dependency and Tools

Since I don’t want to use RPM as my package management, that is the motivation for me to use Gentoo, I need to resolve the dependencies manually, the following package are installed to meet the dependency requirement:

emerge glut libXmu libXext

gcc make perl rsync flex byacc are in the base system.

To extract rpm, we also need to install rpm2targz and rpm:

emerge rpm rpm2targz

Download the RPMs

The CELL SDK files are scattered in AlphaWorks and Bsc:

Download the CELLSDK21.iso from AlphaWorks, you might need to register before access.

Download all the x86-based file from Bsc, to make your life easier, you could use this script:

# ppu
wget -c  http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-binutils-2.17.50-8.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-gcc-4.1.1-10.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-gcc-c++-4.1.1-10.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-gcc-fortran-4.1.1-10.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-gdb-6.6-15.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-gcc-debuginfo-4.1.1-10.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-gdb-debuginfo-6.6-15.i686.rpm

# spu
wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-binutils-2.17.50-8.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-gcc-4.1.1-9.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-gcc-c++-4.1.1-9.i686.rpm

hwget -c ttp://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-gdb-6.6-12.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-newlib-1.15.0-7.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-gcc-debuginfo-4.1.1-9.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/spu-gdb-debuginfo-6.6-12.i686.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-sysroot-fc6-1.noarch.rpm

wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/ppu-sysroot64-fc6-1.noarch.rpm

# sysroot
wget -c http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk2.1/sysroot_image-2.1-8.noarch.rpm

Extract the RPMs

We need to convert all RPMs to tar, then untar it in the root of directory:

sudo mount CELLSDK21.iso /mnt/loop -o loop
for x in /mnt/loop/software/*i386.rpm
do
        rpm2tar $x
done

for x in *.rpm
do
        rpm2tar $x
done
BUILDDIR=`pwd`

cd /
for x in $BUILDDIR/*.tar
do
        sudo tar xvf $x
done

It would take a few hours depending on your network bandwidth. Unfortunately, the sysroot_image-2.1-8.noarch.rpm fails to covert to tar, using rpm2cpio as the rescure:

sudo rpm2cpio sysroot_image-2.1-8.noarch.rpm | cpio -i –make-directories

Test Drive

cd /opt/ibm/systemsim-cell/run/cell/linux
../run_gui

You MUST go to the specified path where .systemsim.tcl residents. Wait for the GUI window launches, then Click Go, if nothing is wrong, the Linux kernel is launched, and the sysroot_image is mapped as the guest OS system.

Warnning: The CELL simulator is extremely computing-extensive, you could click Stop to pause the simulator when it is idle.

Cleanup

We can unmerge the rpm right now if you want to a crusty system.

emerge -c rpm =db-3.2.9-r11 beecrypt =db-1.85-r3