diff --git a/en_US.ISO8859-1/articles/portbuild/article.sgml b/en_US.ISO8859-1/articles/portbuild/article.sgml
index 1c8cec7efe..35798bac03 100644
--- a/en_US.ISO8859-1/articles/portbuild/article.sgml
+++ b/en_US.ISO8859-1/articles/portbuild/article.sgml
@@ -18,6 +18,7 @@
2004
2005
2006
+ 2007
The &os; Ports
Management Team
@@ -34,32 +35,55 @@
Introduction and Conventions
In order to provide pre-compiled binaries of third-party
- applications for &os;, the ports collection is regularly
+ applications for &os;, the Ports Collection is regularly
built on one of the Package Building Clusters.
- Currently, there are two such clusters:
- pointyhat.FreeBSD.org and
- dosirak.kr.FreeBSD.org.
+ Currently, the main cluster in use is at
+ .
Most of the package building magic occurs under the
/var/portbuild directory. Unless
otherwise specified, all paths will be relative to
this location. ${arch} will
be used to specify one of the package architectures
- (&i386;, alpha, &sparc64;, ia64, and amd64), and
+ (amd64, &i386;, ia64, and &sparc64;), and
${branch} will be used
- to specify the build branch (4, 5, 5-exp, 6, 6-exp, 7).
+ to specify the build branch (5, 5-exp, 6, 6-exp, 7).
+
+
+ Packages are no longer built for Release 4, nor
+ for the alpha architecture.
+
+
+ The scripts that control all of this live in
+ /var/portbuild/scripts/. These are the
+ checked-out copies from
+ /usr/ports/Tools/portbuild/scripts/.
+
+ Typically, incremental builds are done that use previous
+ packages as dependendencies; this takes less time, and puts less
+ load on the mirrors. Full builds are usually only done:
+
+
+ right after release time, for the
+ -STABLE branches
+
+ every month or so, for -CURRENT
+
+
+ for experimental builds
+
Build Client Management
- The &i386;, alpha, amd64, and two &sparc64; clients currently
- netboot from pointyhat; the other sparc64 client
- and ia64 clients are self-hosted. In all cases they set themselves
+ The &i386; clients currently
+ netboot from pointyhat; the other clients
+ are self-hosted. In all cases they set themselves
up at boot-time to prepare to build packages.
- In the latest round of portbuild updates,
+ Although connected nodes are supported,
disconnected cluster node support has
been added. A disconnected node is
one that does not mount the cluster master via NFS. It could be
@@ -71,8 +95,14 @@
The
ports-${arch}
- user can &man.ssh.1; as root onto
- each of the ${arch} nodes.
+ user can &man.ssh.1; as root
+ (not as ports-${arch})
+ onto
+ each of the ${arch} nodes
+ which are connected nodes. In general, this is not true of the
+ disconnected nodes. Use sudo and check the
+ portbuild.hostname.conf
+ for the user and access details.
The scripts/allgohans script can
be used to run a command on all of the
@@ -83,7 +113,7 @@
build cluster, and schedule which nodes build which ports.
This script is not very robust, and has a tendency to die.
It is best to start up this script on the build master
- (either pointyhat or dosirak)
+ (e.g. pointyhat)
after boot time using a &man.while.1; loop.
@@ -147,12 +177,6 @@
are used to perform the builds. Most useful are:
-
- dopackages.4 - Perform a
- 4.X build
-
-
-
dopackages.5 - Perform a
5.X build
@@ -186,6 +210,12 @@
+
+ As of early 2007, the 5-exp branch is just another 6-exp
+ branch in disguise. For instance, it will have
+ INDEX-6.
+
+
These are wrappers around dopackages,
and are all symlinked to dopackages.wrapper.
New branch wrapper scripts can be created by symlinking
@@ -278,7 +308,7 @@
was spent rebuilding things that were going to
fail anyway. Conversely, the other clusters
are slow enough that it would be a waste of time
- to try and build BROKEN ports.
+ to try and build BROKEN ports).
@@ -296,13 +326,6 @@
-
- -nodoccvs - Do not
- cvs update the
- doc tree during preprocessing. (obsolete)
-
-
-
-norestr - Do not attempt to build
RESTRICTED ports.
@@ -333,6 +356,26 @@
+ If the last build finished cleanly you don't need to delete
+ anything; if it was interrupted you just need to run
+ dosetupnodes on all clients for the
+ the relevant branch. errors/,
+ logs/, packages/, and so
+ forth, are cleaned by the scripts. If you are short of space,
+ you can also clean out ports/distfiles/.
+ Leave the latest/ directory alone; it is
+ a symlink for the webserver.
+
+
+ dosetupnodes is supposed to be run from
+ the dopackages script in the
+ -restart case, but it can be a good idea to
+ run it by hand and then verify that the clients all have the
+ expected job load. Sometimes,
+ dosetupnode cannot clean up a build and you
+ need to do it by hand. (This is a bug.)
+
+
Make sure the ${arch} build
is run as the ports-${arch} user
or it will complain loudly.
@@ -340,7 +383,7 @@
The actual package build itself occurs in two
identical phases. The reason for this is that sometimes
transient problems (e.g. NFS failures, FTP sites being
- unreachable, etc.) may halt the build. Doing things
+ unreachable, etc.) may halt a build. Doing things
in two phases is a workaround for these types of
problems.
@@ -367,6 +410,24 @@
Makefile with no SUBDIRs
in it. This is probably a bug.
+
+
+ Update the i386-6 tree and do a complete build
+
+ dopackages.6 i386 -nocvs -norestr -nofinish
+
+
+
+ Restart an interrupted amd64-5 build witout updating
+
+ dopackages.5 amd64 -nocvs -noportscvs -norestr -continue -noindex -noduds -nofinish
+
+
+
+ Post-process a completed sparc64-7 tree
+
+ dopackages.7 sparc64 -finish
+
@@ -458,39 +519,121 @@
Interrupting a Build
- Sending a HUP signal to the
- dopackages* shell processes or to any
- make process invoked by those scripts
- is usually sufficient to interrupt the build. The
+ Interrupting a build is a bit messy. First you need to
+ identify the tty in which it's running (either record the output
+ of &man.tty.1; when you start the build, or use ps x
+ to identify it. You need to make sure that nothing else important
+ is running in this tty, e.g. ps -t p1 or whatever.
+ Then either kill -HUP in there by e.g.
+ps -t p1 | awk '{print $1}' | xargs kill -HUP. Replace
+ p1 by whatever the tty is, of course.
+
+ The
package builds dispatched by make to
the client machines will clean themselves up after a
few minutes (check with ps x until they
- all go away). The following command usually does the trick:
+ all go away).
- &prompt.user; killall -HUP sh ssh make
+ If you don't kill &man.make.1;, then it will spawn more jobs.
+ If you don't kill dopackages, then it will restart
+ the entire build. If you don't kill the pdispatch
+ processes, they'll keep going (or respawn) until they've built their
+ package.
- Remove the
+ To free up resources, you will need to clean up by running
+ dosetupnode on each client machine. For example,
+ in &man.csh.1;:
+ &prompt.user; cd ~/loads; foreach i (*); /var/portbuild/scripts/dosetupnode i386 5-exp $i -norsync &; done
+
+ The -norsync says not to bother resyncing the
+ entire build data (ports tree, etc) on any remote machines, and it
+ will just clean up old chroots and then reset the
+ build queue for that machine.
+
+ If you forget to do this, then the old build
+ chroots won't be cleaned up for 24 hours, and no
+ new jobs will be dispatched in their place since
+ pointyhat thinks the job slot is still occupied.
+
+ To check, cat ~/loads/* to display the
+ status of client machines; the first column is the number of jobs
+ it thinks is running, and this should be roughly concordant
+ with the load average. loads is refreshed
+ every 2 minutes. If you do ps x | grep pdispatch
+ and it's less than the number of jobs that loads
+ thinks are in use, you're in trouble.
+
+ You may have problem with the umount
+ commands hanging. If so, you are going to have to use the
+ allgohans script to run an &man.ssh.1;
+ command across all clients for that buildenv. For example:
+ssh -l root gohan24 df
+
+ will get you a df, and
+
+allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/6-exp/ports"
+allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/6-exp/src"
+
+ are supposed to get rid of the hanging mounts. You will have to
+ keep doing them since there can be multiple mounts.
+
+
+ Ignore the following:
+
+umount: pointyhat.freebsd.org:/var/portbuild/i386/6-exp/ports: statfs: No such file or directory
+umount: pointyhat.freebsd.org:/var/portbuild/i386/6-exp/ports: unknown file system
+umount: Cleanup of /x/tmp/6-exp/chroot/53837/compat/linux/proc failed!
+/x/tmp/6-exp/chroot/53837/compat/linux/proc: not a file system root directory
+
+ The former 2 mean that that client didn't have those mounted;
+ the latter 2 are a bug.
+
+ You may also see messages about procfs.
+
+
+ After you have done all the above, remove the
${arch}/lock
- file before trying to restart the build.
+ file before trying to restart the build. If you don't,
+ dopackages will simply exit.
+
+ If you have to do a cvs update before
+ restarting, you may have to rebuild either duds,
+ INDEX, or both. If you are doing the latter
+ manually, you will also have to rebuild
+ packages/All/Makefile via the
+ makeparallel script.
Monitoring the Build
- The
+ You can use either
+ showrunning or python straslivy.py
+ to show the packages currently being built. The
scripts/stats ${branch}
- command counts the number of packages currently built.
+ command shows the number of packages already built.
Running cat /var/portbuild/*/loads/*
shows the client loads and number of concurrent builds in
- progress.
+ progress. The files that have been recently updated are the clients
+ that are online; the others are the offline clients.
+
+
+ The pdispatch command does the dispatching
+ of work onto the client, and post-processing.
+ ptimeout.host is a watchdog that kills a build
+ after timeouts. So, having 50 pdispatch
+ processes but only 4 &man.ssh.1; processes means 46
+ pdispatches are idle, waiting to get an
+ idle node.
+
Running tail -f ${arch}/${branch}/build.log
shows the overall build progress.
- If a build is failing, and it is not immediately obvious
- from the port build log as to why, you can preserve the
+ If a port build is failing, and it is not immediately obvious
+ from the log as to why, you can preserve the
WRKDIR for further analysis. To do this,
touch a file called .keep in the port's
directory. The next time the cluster tries to build this port,
@@ -503,6 +646,79 @@
/var/portbuild file system becomes full
then Bad Things happen.
+
+ The status of all current builds is generated twice an hour
+ and posted to
+ .
+ For each buildenv, the following is displayed:
+
+
+
+ cvs date is the contents of
+ cvsdone. This is why we recommend that you
+ update cvsdone for -exp
+ runs (see below).
+
+
+
+ date of latest log
+
+
+
+ number of lines in INDEX
+
+
+
+ the number of current build logs
+
+
+
+ the number of completed packages
+
+
+
+ the number of errors
+
+
+
+ the number of duds (shown as skipped)
+
+
+
+ missing shows the difference between
+ INDEX and the other columns. If you have
+ restarted a run after a cvs update, there
+ will likely be duplicates in the build log and error columns,
+ and this column will be meaningless. (The script is naive).
+
+
+
+ running and completed
+ are guesses based on a &man.grep.1; of build.log.
+
+
+
+
+
+
+ Dealing With Build Errors
+
+ The easiest way to track build failures is to receive
+ the emailed logs and sort them to a folder, so you can maintain a
+ running list of current failures and detect new ones easily.
+ To do this, add an email address to
+ ${branch}/portbuild.conf.
+ You can easily bounce the new ones to maintainers.
+
+ After a port appears broken on every build combination
+ multiple times, it is time to mark it BROKEN.
+ Two weeks' notification for the maintainers seems fair.
+
+
+ To avoid build errors with ports that need to be manually
+ fetched, put the distfiles into
+ ~ftp/pub/FreeBSD/distfiles.
+
@@ -637,15 +853,37 @@
bsd.port.mk), or to test large sweeping
upgrades. The current experimental patches branch is
6-exp on the &i386;
- architecture.
+ architecture. (For the moment, we are also using
+ 5-exp as a secondary branch, but
+ actually using the bits from RELENG_6.)
In general, an experimental patches build is run the same
- way as any other build. However, before running the
- dopackages script, you must apply the required
- patches to the ports tree. It is always a good idea to save
+ way as any other build, except that you should first update the
+ ports tree to the latest version and then apply your patches.
+ To do the former, you can use the following:
+
+ &prompt.user; cvs -R update -dP > update.out
+&prompt.user; date > cvsdone
+ This will most closely simulate what the dopackages
+ script does. (While cvsdone is merely
+ informative, it can be a help.)
+
+ You will need to edit update.out to look
+ for lines beginning with ^M, ^C,
+ or ^? and then deal with them.
+
+ It is always a good idea to save
original copies of all changed files, as well as a list of what
you are changing. You can then look back on this list when doing
- the final commit.
+ the final commit, to make sure you are committing exactly what you
+ tested.
+
+ Since the machine is shared, someone else may delete your
+ changes by mistake, so keep a copy of them in e.g. your home
+ directory on freefall. Don't use
+ tmp/; since pointyhat
+ itself runs some version of -CURRENT, you
+ can expect reboots (if nothing else, for updates).
In order to have a good control case with which to compare
failures, you should first do a package build of the branch on
@@ -733,7 +971,7 @@
&prompt.user; cd /var/portbuild/i386/6/ports
- Be sure to cvs update this tree to the same date as
+ Be sure to cvs update this tree to the same date as
the experimental patches tree.
The following command will set up the control branch for
@@ -764,7 +1002,9 @@
of their dependencies.
You can check the progress of this
- partial build the same way you would a regular build. Once all
+ partial build the same way you would a regular build.
+
+ Once all
the errors have been resolved, you can commit the package set.
After committing, it is customary to send a HEADS
UP email to