Warren Falk

In an effort to save myself some time, I’m looking into setting up a share on my network dedicated to hosting precompiled gentoo packages (that I’ve compiled myself). This way when I unmerge a package and want to remerge it later, or I want to merge the same package on identically configured (same architecture and optimizations), I don’t have to go through all the trouble of recompiling everything.

I’m using a samba share because I already have one. If you want to use NFS or something else, then that is fine, but you’ll have to transpose this list as necessary. Also note that to use a samba share, you’ll want to have the smbfs module compiled and inserted, or compiled into the kernel. The basic process works as follows:

UPDATED: I’ve updated this now due to a problem with newer versions of portage getting ACCESS DENIED errors when creating the binaries. Because of the way the sandbox works, you can’t have symlinks inside the package directory (PKGDIR) that point outside the sandbox’s directory write list.

1. [Server A] Set up a centralized directory with read/write access for storing the binary packages. Share it with samba.
2. [Server B] Set up a centralized web server (apache in my case) that can point to the share. This could be the same box as “A”, but mine is not.
3. [Client] Mount the samba share somewhere. Mine is called “pub” because it was already created and I mount it in /mnt/pub, for instance.
4. [Client] cd into the share, and create a folder for your architecture. This should be something like i686-pc-linux-gnu-4.1.0/pentium4/O2. The first directory name (e.g. i686-pc-linux-gnu) is the compiler architecture (CHOST) being used (from `gcc-config -c`). The next directory name (e.g. pentium4) is the architecture. This comes from the -march in CFLAGS in /etc/make.conf. The next is the compiler optimization used (e.g. O2 for -O2) in the CFLAGS of /etc/make.conf.
5. [Client] Add PKGDIR variable to /etc/make.conf that points to the directory you created in the previous step
6. [Client] Add to /etc/make.conf: FEATURES=”buildpkg”
7. [Client] (optional, see caveat) Use quickpkg to create binary packages of everything you already have built. Caveat: if the package contains any files you’ve modified, these go into your binary package. So this isn’t really safe. The better thing to do is to re-emerge everything.
8. [Client] Add the following alias to your /etc/bash/bashrc aliases, alias emerge=”emerge –usepkg –getbinpkg”
9. [Client] Add to /etc/make.conf: PORTAGE_BINHOST=”http:///path/to/share/i686-pc-linux-gnu/pentium4/O2/” (Try not to miss the last slash as it results in a redirect from the server and an annoying message from emerge)

The basic idea, here, is that steps 1 and 2 create a central repository. Steps 3-8 ensure that binaries are cached for existing builds and new builds, and step 9 sets the system up to use the cached binaries if they are available.

Tips:

One thing I noticed is that I have some machines configured to use the -march k8 and some to use -march athlon64, but that these are aliases for the same thing and produce identical binaries. So, if you want to share binaries between these machines, you can symlink the k8 and athlon64 directories under the x86_64* dir.

Caveats:

1. I believe that different glibc versions might cause problems and this could potentially be remedied by adding a glibc version into the PKGDIR and PORTAGE_BINHOST paths.
2. If you upgrade gcc, gcc-config will not fix /etc/make for you to put the new binaries in a new folder, so you just have to remember to do that. I experimented with having make.conf detect the current version, but that caused problems with gcc-config.
3. If you share these among machines, it looks like the USE flags are compiled into the binaries so the two machines are forced to use the same USE flags or compile separately. Compiling separately might cause some confusing problems since the paths to the binary packages will be identical causing overwrites.

I’ve put the video card debacle on hold for now to compile KDE because it is convenient at this time to have my machine doing large compiles while I’m off doing other things. I have set up a script which runs the build, and then texts my cellphone when it is complete so that I don’t have to keep monitoring it. This is quite handy.

The first problem I run into is kde-base/superkaramba-3.5.2. I immediately tried 3.5.1 but it requires downgrading the kde-libs to 3.5.1 and I can’t do that because all the other packages depend on it. So I have to fix 3.5.2.

The failure message appeared to be that Python.h was missing in one of the compiles. However running make in the working directory didn’t reproduce that error. In fact, it succeeded fine. So I went and tried to run ebuild … install and there was the error, during the install phase, it is compiling. That led me down a wild goose chase. When I tried the whole thing over again step-by-step, I found that the compile appears to succeed, and portage things it succeeds, but it doesn’t. The configure script complains that it is missing Python.

Superkaramba can’t be compiled
because of missing Python libraries/headers.

Python is almost certainly installed, but let’s check:

# equery list python
[I--] [ ] dev-lang/python-2.4.2 (2.4)
[I--] [ ] dev-python/python-fchksum-1.7.1 (0)

Yep, it’s installed. So why does the configure script miss it? I’m looking at config.log and see that it finds Python 2.4 headers and libraries but apparently not the modules. I’m looking at this line:

configure:34568: result: header /usr/include/python2.4 library /usr/lib64 modules no

It is looking for the modules in the following locations:

configure: 34552: /usr/local/lib64/python2.4/copy.py
configure: 34552: /usr/lib64/python2.4/copy.py
configure: 34552: /usr/local/python2.4/copy.py
configure: 34552: /usr/lib64/python2.4/copy.py

Specifically it is looking for copy.py, and, of course, it isn’t in any of those locations, it is in /usr/lib/python2.4/copy.py. So I’m thinking that this has something to do with the fact that I’m on a 64bit system. Well, at least, that’s why it hasn’t been fixed yet.
It isn’t clear whether the configure script is looking in the wrong place, or the python installation isn’t quite right. My guess is to blame the configure script because it is searching the same location twice.

If I look at the configure script, I find that it in fact is hard coded to search the same location twice. Here’s the line:

python_libdirs=”$ac_python_dir/lib$kdelibsuff /usr/lib$kdelibsuff /usr/local /usr/lib$kdelibsuff $kde_extra_libs”

Notice that /usr/lib$kdelibsuff exists twice. Why would you want to search the same dir twice? I’m changed that to:

python_libdirs=”$ac_python_dir/lib$kdelibsuff /usr/lib$kdelibsuff /usr/local /usr/lib $kde_extra_libs”

…and reran configure. Look in config.log to find how portage is running configure. Mine was:

# ./configure –prefix=/usr \
–host=x86_64-pc-linux-gnu \
–mandir=/usr/share/man \
–infodir=/usr/share/info \
–datadir=/usr/share \
–sysconfdir=/etc \
–localstatedir=/var/lib \
–with-x \
–enable-mitshm \
–without-xinerama \
–with-qt-dir=/usr/qt/3 \
–enable-mt \
–with-qt-libraries=/usr/qt/3/lib64 \
–disable-dependency-tracking \
–disable-debug \
–without-debug \
–disable-final \
–without-arts \
–prefix=/usr/kde/3.5 \
–mandir=/usr/kde/3.5/share/man \
–infodir=/usr/kde/3.5/share/info \
–datadir=/usr/kde/3.5/share \
–sysconfdir=/usr/kde/3.5/etc \
–enable-libsuffix=64 \
–libdir=/usr/kde/3.5/lib64 \
–build=x86_64-pc-linux-gnu

This gives me:

Good – your configure finished. Start make now

And so I go back to portage to do run all the steps, but this time editing the file before compiling, and I find that the configure script is built from a file called acinclude.m4 where we want to make our change, and then I also see another file in the admin directory with the same line so I edit and change that one too just in case.

# cd /usr/portage/kde-base/superkaramba/
# ebuild superkaramba-3.5.2.ebuild clean
# ebuild superkaramba-3.5.2.ebuild unpack
# nano -w /var/tmp/portage/superkaramba-3.5.2/work/superkaramba-3.5.2/acinclude.m4
(made my change and saved)
# nano -w /var/tmp/portage/superkaramba-3.5.2/work/superkaramba-3.5.2/admin/acinclude.m4.in
(made my change and saved)
# ebuild superkaramba-3.5.2.ebuild compile

I remember, that the compile appeared to succeed, before, even though nothing was compiled, so this time, I am watching and I see the “Good – your configure finished…” message scroll by and it does, and portage is compiling stuff. I sigh with relief, cross my fingers and finish it off:

# ebuild superkaramba-3.5.2.ebuild install
# ebuild superkaramba-3.5.2.ebuild qmerge

All goes well.

[EDIT] … And I have no further problems compiling KDE 3.5.2

I got a great deal on the ATI card and I got it for gaming in Windows. Otherwise, I advise anyone interested in one day switching to linux to avoid ATI. Nvidia is so much nicer to the linux community.

Nevertheless, I’m going to try to get this working because ATI is what I have. The x850 is based on the radeon 300 chipset. The open-source drivers for linux just don’t work with it, from what I understand and so we need the proprietery (pronounced: “evil”) driver from ATI.

And I’m doing this all on an AMD64 machine with Gentoo, running X11 7.0 and gcc 4.1.0. Both of which, at this point, are not really stable.

I managed to get X11 compiled after fixing one of the ebuilds, but X doesn’t actually work. I first tried to install the stable version of the ATI drivers but the build failed.

Then I tried to install the testing (unstable) drivers. Those built, but didn’t work. I’m going to work on getting these to work rather than trying to get the stable version built. If I find myself in over my head, I’ll go back to the stable version and just try to get it built.

The error I’m getting with the development drivers is that when I run

X -configure

I get the following error:

dlopen: /usr/lib64/xorg/modules/drivers/fglrx_drv.so: undefined symbol: __glXActiveScreens
(EE) Failed to load /usr/lib64/xorg/modules/drivers/fglrx_drv.so
(EE) Failed to load module “fglrx” (loader failed, 7)

I’ll post my progress (if any) here as comments

This wasn’t as difficult as I thought it would be. The only problem I encountered buliding it successfully was in the previous post where mesa-progs failed to compile. I’m not even sure at this time why it is required, but I did get it working. Here’s what I did.

ebuild /usr/local/portage/x11-apps/mesa-progs/mesa-progs-6.4.2.ebuild unpack
cd /var/tmp/portage/mesa-progs-6.4.2/work/Mesa-6.4.2

Then I edit one of the Makefiles in there and add four lines.

nano -w progs/xdemos/Makefile

I add the following:

glxgears: glxgears.c
$(CC) $(CFLAGS) -I$(INCDIR) glxgears.c $(GLUT_LIB_DEPS) -o $@
glxinfo: glxinfo.c
$(CC) $(CFLAGS) -I$(INCDIR) glxinfo.c $(GLUT_LIB_DEPS) -o $@

Now I can compile, install and merge

ebuild /usr/local/portage/x11-apps/mesa-progs/mesa-progs-6.4.2.ebuild compile
ebuild /usr/local/portage/x11-apps/mesa-progs/mesa-progs-6.4.2.ebuild install
ebuild /usr/local/portage/x11-apps/mesa-progs/mesa-progs-6.4.2.ebuild qmerge

Done. This is roughly equivalent to running emerge mesa-progs except that it works.

…Well, it compiles. It doesn’t work. I can’t seem to get my ATI card to work. I am saving that for the next post, though.

GCC 4.1.0 is hard masked in Gentoo currently and as I understand it is unsupported. Modular X is not yet listed as stable but is in the testing phase. I can’t seem to get a cross compiling distcc setup without using GCC 4.1.0 because the crossdev package is unable to build the toolchain for 3.4.5 correctly (and selects 4.1.0 by default). So to make this work, I have to upgrade to GCC 4.1.0 on everything which up until modular X didn’t cause any problems, but within the last 4 packages of the 108 needed for modular X, mesa-progs failed.
The truth is that I don’t know if GCC 4.1.0 is to blame for the failure. I successfully emerged 98 of 102 packages, but failed on glxinfo in x11-apps/mesa-progs with the following:

x86_64-pc-linux-gnu-gcc -I../../include -Wall -march=athlon64 -O2 -pipe -m64 -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE -D_GNU_SOURCE -DPTHREADS -DUSE_EXTERNAL_DXTN_LIB=1 -DIN_DRI_DRIVER -DGLX_DIRECT_RENDERING -DHAVE_ALIAS -DUSE_X86_64_ASM -std=c99 -ffast-math glxinfo.c -L../../lib64 -lglut -lGLU -lGL -lm -o glxinfo
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XGetExtensionVersion’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XFreeDeviceList’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XQueryDeviceState’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XListInputDevices’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XFreeDeviceState’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XOpenDevice’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XmuLookupStandardColormap’
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.0/../../../../lib64/libglut.so: undefined reference to `XSelectExtensionEvent’
collect2: ld returned 1 exit status
distcc[10000] ERROR: compile glxinfo.c on localhost failed

I looked up those undefined references and found the last one in libXmu.so and the rest in libXi.so. I added -lXmu -lXi to the build command and glxinfo.o build correctly.I don’t know why GCC 4.1.0 might cause this problem. In fact, perhaps, it is an amd64 bug. Hopefully this helps someone. I will post something about how to fix it and build it with portage as soon as I read up on how to do that.

Since GCC 4.1.0 isn’t supported, it isn’t clear where I should post this bug since they might not yet consider it a bug. I imagine someone would like to know that the package is not yet forwards compatible, but don’t know who to tell or how to tell them.

Here’s a picture of me playing a really simple flash video game to entertain the boys. The game itself isn’t loads of fun, but Ethan and Isaac love to sit on my lap and watch me make the Fancy Pants Man jump over the angry penguin. Ethan gets really excited. He asks me to play the “Pantsy Fants” game, and Isaac asks me to jump over the “Anggy Penggin.”

[EDIT] The game can be found here for all that are interested.

I have a page dedicated to my ideas for The entertainment system to end all entertainment systems. Comments on it can be posted to this blog.

I have set up a page to describe my idea for an elegant representation of data to replace the unelegant standard XML simply because I can’t bear to work with it anymore than I have to. This should also fit nicely as a component in an application framework I’ve been conceptualizing for some time now too.

For now, all I have is a draft. The hope is that the draft will be clear about the basic concept of elegant data representation. Later, I will add technical specifications to the draft once they are complete. I would also like comments and questions on the draft. (They can be posted here as comments).

Lynn and I on Valentines day wearing our shirts. Lynn bought me the “geek” shirt for my birthday, and bought herself this shirt a week later and hid it until Valentines day. We wore the shirts to the restaurant and received many comments on them. We even had a guy ask us if he could take our picture with his camera phone. (This is not his picture, though.  this picture was taken in our living room before we left).

I love wordpress, but if you want to use wordpress as photo blogging software, it is somewhat lacking. Especially if you want it to be quick, easy, and something my wife can do. I have seen a few people tackle photos in different ways, but what I was looking for didn’t seem to exist.

My photo blogger plugin allows you to center a post around an image. You begin the post by uploading an image. The system will keep the full quality image, and will also downsample an image for display. The downsampled image is resized to fit completely within independent maximum width and height specifications which are configurable.

A post is automatically created for you with the image attached and the display image placed inline complete with the code necessary to allow users to click the image and see when it was taken, and download or view the full quality version of it.

You can edit the post in the regular wordpress editor to add any extra text etc.

In the future, I expect to add the ability to show a “gallery” or something like that in the Categories list which will find all images and list the thumbnails and post titles.

The plugin is available for download here: wf-blogphoto.php.txt [warrenfalk.com]

If you’re new to wordpress plugins, simply download it, remove the .txt from the end and place in your wp-content/plugins folder.