Building IRIX tardists

So, you've finally got an SGI of your own. You kind of like the system, you want to contribute something to the IRIX community in general and more specifically to the community that helped you making your first steps in the IRIX world: Nekochan. You're okay with building apps from source, and you don't care about having to tweak a bit here and there to get stuff to run properly.

Well, IRIX packages - tardists - don't grow on trees. It all boils down to a number of SGI enthousiasts who build software on their machines in their free time, and package it up for the rest of the community to enjoy. Now, compiling an application on IRIX isn't that painful (yes, there are exceptions of course... OpenOffice, anyone?) but getting a decent tardist from it is a totally different cup of tea.

Still not scared? Alright, get your seatbelts on then :-)


Getting the tools

If you're familiar with linux, BSD or any other UNIX variant, and you probably are if you've decided to try IRIX, you're certainly familiar with gcc - the GNU C Compiler suite - too. Well, let's give you the first kick in the balls: gcc just doesn't cut it. You can use gcc to learn the basics (and it's not a bad idea to do so, really) but if you want your package to end up on a nekoware mirror, you'll have to use MIPSPro.

MIPSPro is SGI's own compiler suite, and if you read up a bit on the machines SGI made, it's no surprise that gcc performs worse on such exotic high-performance hardware. On top of that, C++ machine code produced by gcc and MIPSPro cc don't like eachother much.

Anyway, enough already: get your hands on a MIPSPro 7.3 (or newer) media set. You can check eBay, or ask on the IRC channel if anyone knows a place where they sell some. The usual other tools you can find for yourself - gmake, autoconf etcetera. All in nekoware.

Last remark you probably heard tons of times before: nekoware requires at least IRIX 6.5.22, and a mips4 (R5000 or better) cpu for the main tree - there's a mips3 branch too for older machines.


Setting the variables

Next thing: setting global variables for development. I usually set these (nicked from joerg's relnotes file) and they work just fine:

export CC=c99
export CFLAGS='-O3 -mips4 -I/usr/nekoware/include -TARG:platform=IP27:proc=r10000'
export CXXFLAGS=$CFLAGS
export CPPFLAGS='-I/usr/nekoware/include -I/usr/include'
export CXX=cc
export F77=f77
export LDFLAGS='-L/usr/nekoware/lib -Wl,-rpath -Wl,/usr/nekoware/lib'
export PKG_CONFIG=/usr/nekoware/bin/pkg-config
export PKG_CONFIG_PATH='/usr/nekoware/lib/pkgconfig'
export PKG_CONFIG_LIBDIR='/usr/nekoware/lib'
export LD_LIBRARY_PATH='/usr/nekoware/lib'
export LD_LIBRARYN32_PATH='/usr/nekoware/lib'
export LD_LIBRARY64_PATH='/usr/nekoware/lib64'
export GNOME2_DIR='/usr/nekoware'
export GNUMAKE='/usr/nekoware/bin/make'

They're not hard to understand; since nekoware is in /usr/nekoware, you most definitely want to build and link against the libraries in there. You don't want to use the standard make - most software is much happier with GNU make.

Further on, there's the MIPSPro options. Defining CC and CXX will give you the MIPSPro compiler suite. As you can notice, the CFLAGS option indicates you want to build for mips4, being any MIPS CPU equal or newer than R5000, which is the oldest CPU version the main nekoware tree supports. There's additional optimizations being done for the R10000 platform (more specifically: for the IP27, which is an Origin200. Don't worry about it, it doesn't matter much), which is a common CPU in lots of O2, Octane, Origin etcetera - your build will work on R5000, but will get an additional speed bump on R10000 or better. If your app is very optimization-happy - mplayer, blender, stuff like that - you could provide different builds, optimized for R5K, R8K, R10K, R12K, R14K or R16K. You'd be one lucky bastard if you'd actually have an R16K to play with though ;-)


How to not shoot yourself in the foot

You'll probably have created a place in your homedir to put all source tarballs, build dirs etcetera together. Now, for the rest of the howto, let's suppose you're trying to build fooware version 1.2.3. Duplicate your source dir; this will make it easier for you to create a decent patch later on.

$ tar zxvf fooware-1.2.3.tar.gz
$ cp -Rv fooware-1.2.3 fooware-1.2.3-PATCHED
$ cd fooware-1.2.3-PATCHED

Next step is configuring the app, mostprobably. Nekoware lives in /usr/nekoware, so you'll want your files to end up there too. For pretty much all standard open source applications, this can be done by a simple switch:

$ ./configure --prefix=/usr/nekoware

You can, of course, add all switches you need to get the app to build. If you want to make it easy on yourself, write down the dependencies it checks. And then you can do your thing - edit sourcefiles, run gmake -j17 on your monster Origin3000, whatever it takes to make it build.


How to not shoot yourself even further in the foot

When it's done, you'll probably do

$ su
# gmake install
# exit

Done? Try your app now. If it's properly installed with the said prefix, it'll be in your path already. And if you're lucky, it'll even run like it's supposed to. Excellent! Generate a patch for your build:

$ cd ..
$ diff -urnp fooware-1.2.3 fooware-1.2.3-PATCHED | \
grep -v "^Only" > neko_fooware-1.2.3_irix.patch

That clean source dir comes in handy eh. Now, install the app to a different location to get a clean application tree. Let's assume you want to do that somewhere in /tmp - should be a safe place.

$ mkdir /tmp/build
$ cd fooware-1.2.3-PATCHED
$ gmake DESTDIR=/tmp/build install
$ cd ..

You'll have a full nekoware tree layout in /tmp/build now, populated with only files from your very fooware.


Package preparations

Create the extra nekoware dirs you'll use to store patches, sources, release notes and the distribution files swpkg will generate for you; then copy source and patch to their spot.

$ mkdir /tmp/build/usr/nekoware/patches
$ mkdir /tmp/build/usr/nekoware/src
$ mkdir /tmp/build/usr/nekoware/relnotes
$ mkdir /tmp/build/usr/nekoware/dist
$ cp neko_fooware-1.2.3_irix.patch /tmp/build/usr/nekoware/patches
$ cp fooware-1.2.3.tar.gz /tmp/build/usr/nekoware/src

You don't have a relnotes file yet, do you? Well, copy one over from another package and suit it to your needs. Stuff you might want to include:

  • The packager's name and contact address (yes! That's YOU!)
  • The app's name, version and description. You can spend a handful of lines to do this, if needed.
  • Some info on the machine you used to build
  • The version of your MIPSPro suite and your IRIX system
  • The environment variables you set - see before
  • Extra build notes: if just patching isn't enough, what else needs to be done?
  • Remarks about known glitches, etcetera
  • Known dependencies

When its done, copy this file to the appropriate dir.

$ cp neko_fooware.txt /tmp/build/usr/nekoware/relnotes

Let's get this party started

When all preparations are done, it's time to load the IRIX app that'll generate the distribution files and create the actual tardist package: swpkg. It'll need a place to store its build files, so creating a dir in /tmp is not a bad place for this.

$ mkdir /tmp/dist
$ swpkg

The UI is rather self-explaining. In the first tab, create your package tree. Give your main node a name - neko_fooware in this case. The description would be something like "FooWare 1.2.3 - a truly fubar kind of ware". Don't forget to assign your changes to the tree on the left, or they will be lost. The other options you can skip - the defaults are fine.

A standard nekoware package will contain the following sub-images:

  • neko_fooware.sw.eoe (binaries and files for execution only)
  • neko_fooware.sw.hdr (headers and development stuff)
  • neko_fooware.sw.lib (shared libraries)
  • neko_fooware.man.manpages (duh.)
  • neko_fooware.opt.src (the original, untampered source tarball)
  • neko_fooware.opt.patches (patches against the original source)
  • neko_fooware.opt.renotes (release notes for this package)
  • neko_fooware.opt.dist (distribution files, generated by swpkg)

It's of course possible that your application doesn't need patches, hasn't got executable binaries - etcetera. Use your common sense.


Tardist versions: beware of the dog

A word of caution: do not confuse the irix version number, which is assigned with swpkg on the first tab, with the application version. They have nothing to see with eachother. The irix version simply shows how "recent" the build is - if an application gets rebuilt (bugfixes, ...), the irix version gets bumped even if the application was built from the same source as the earlier package. In linux-rpm-land this would show itself in filenames like fooware-1.2.3-1.rpm, fooware-1.2.3-2.rpm and so on; the difference is that for the rpm, this number is reset for new versions of the package, and in irix it just moves on.

So - choose a number. If you're the first one to package fooware for nekoware, "1" is the most obvious choice. If you're not, you'd better check what irix version the last build had, and bump it with at least one number.

Also, remember that all version numbers inside the same tardist need to be equal! Even if nothing whatsoever changed in a sub-image since the last build, bump the number even if only one of the other sub-images requires it.


Package preparations, continued

Now, get the piece of paper where you wrote down the dependencies your fooware needed. Make sure to be specific - do you just need the shared libraries, or the binaries? Since apps are split up in the way described earlier, it makes a difference. Easiest way to make sure you get things completely right, is using 'ldd' on your binaries and/or libraries to see what it links into. Let's say fooware's binaries depend on the shared libs of somelib and thatapp, and the binaries of thestuff, all from nekoware. Write down the irix versions for each:

$ versions -n neko_somelib
$ versions -n neko_thatapp
$ versions -n neko_thestuff

Each 'versions' line will output a few lines, one for each sub-image that is installed with the corresponding version. Knowing that fooware's executables won't do much without their own libraries, the 'rules' section for fooware.sw.lib would look something like this:

replaces self
prereq (
   neko_fooware.sw.lib 1 maxint
   neko_somelib.sw.lib 2 maxint
   neko_thatapp.sw.eoe 7 maxint
   neko_thestuff.sw.hdr 1 maxint
)

I don't have to draw a picture here - the package replaces earlier packages with the same name, and the dependencies - prereqs - are listed, each with the minimum version they require.

There's also an "Install by default" checkbox for each sub-image. Usually sw.eoe, sw.lib, sw.hdr and man.manpages are installed by default. The opt.* images are for those who feel like rebuilding the tardist, those who want to find out how you did it - or, like my case long ago, for those who want to contribute a new version of an application but haven't got the slightest clue on where to start :-)

Before continuing to the next tab, save both dist files - neko_fooware.spec and neko_fooware.idb - in /tmp/build/usr/nekoware/dist so they're available for tagging.


Second & third tab: file tagging

Here's where you'll thank me on your bare knees for that clean build dir: every file that fooware installs, needs to be selected and assigned to a sub-image. Now imagine you'd have to track all of /usr/nekoware for files that belong to your fooware... oh yeah, that'd suck.

Import the full /tmp/build/ dir, and assign the files. Usually stuff in lib/ gets put in sw.lib, bin/ and share/ are sw.eoe domain, and include/ is for sw.hdr. I guess you can figure out the rest. You might want to remove the directory entries themselves - like /tmp/build/usr/nekoware itself; it'll be in your filelist, but it isn't an actual file from your application, which is considered sloppy.

Set "root" as owner and "sys" as group for all files, save the spec and idb file, and then exit swpkg. Yes, exit it. No, we're not done yet. You'll notice soon enough.


Interlude: fixing the paths

Now, the current tardist build files contain all files, have the right permissions, and are assigned to the right package sub-images. That's nice, but the paths are still wrong - we don't want our package to install its files in /tmp/build on all machines, do we? Time for some regex magic.

$ sed -i 's,/tmp/build,,g' /tmp/build/usr/nekoware/dist/neko_foobar.idb

Re-launch swpkg, and open spec and idb files from the build dir. Ta-dah, all paths are done.


Fourth tab: setting options

The fourth tab is a quickie. Ever heard of rqsall? You probably already cursed it to hell for taking so long after installing neko_whatever through swmgr. Well, every package needs to specify which files need to be requickstarted, and which ones don't.

In the given filelist, select all files that are no binaries: this means everything, excluding executables and libraries. To the left, check the "norqs" option, and apply it. You just shortened the rqsall run for your package by a nice bit.

Now, your distribution files are ready. Save the idb and spec file once more, then copy everything that isn't in /usr/nekoware yet, there:

$ su
# cp /tmp/build/usr/nekoware/src/* /usr/nekoware/src
# cp /tmp/build/usr/nekoware/patches/* /usr/nekoware/patches
# cp /tmp/build/usr/nekoware/relnotes/* /usr/nekoware/relnotes
# cp /tmp/build/usr/nekoware/dist/* /usr/nekoware/dist
# exit

Bingo. We now have two perfectly identical filelists for your tardist.


Last tab: test & build

The last tab is rather trivial. Replace /usr/dist by /tmp/dist in the option field, hit "test", and if it doesn't complain, "build". You can exit swpkg now; the rest is cleaning up. You've ended up with a couple of files in /tmp/dist - tar them up.

$ cd /tmp/dist
$ tar -cvf neko_fooware-1.2.3.tardist *

There you are - your tardist is done!


Testing, testing, testing...

Important last step is testing. To do this properly, at least uninstall your "work build" and try the tardist for yourself.

$ cd fooware-1.2.3-PATCHED
$ su
# gmake uninstall
# rm /usr/nekoware/dist/neko_fooware.*
# rm /usr/nekoware/src/fooware-1.2.3.tar.gz
# rm /usr/nekoware/relnotes/neko_fooware.txt
# rm /usr/nekoware/patches/neko_fooware-1.2.3_irix.patch
# inst -f /tmp/build/neko_fooware-1.2.3.tardist

Now run it again. Does it work? No complaints? No dependency screams? No hidden segfaults or other nastiness? Fine - you're done. Upload your tardist to the nekochan ftp, and create your first post in the "Development" section on the forum, under "Nekoware-current".

Congratulations... and may many more tardists follow!


Credits

I couldn't have written all this without having learned from the best - all the guys in #nekochan who helped me out for my own first tardist: thanks!

Last revision: v1.7, 2007-08-18. Feel free to send corrections/suggestions.