os/doc
3
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Removed the Makefile and SGML source for this (old and outdated) copy of

Browse source 
my "make world" tutorial.
This commit is contained in:
Nik Clayton 1998-07-24 13:32:05 +00:00
parent e43d54cdc3
commit 52f07f3229
Notes: svn2git 2020-12-08 03:00:23 +00:00 
svn path=/head/; revision=3168
2 changed files with 0 additions and 764 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
en/tutorials/upgrade/Makefile
View file

@ -1,6 +0,0 @@
# $Id: Makefile,v 1.1 1997-06-25 16:57:01 jfieber Exp $
DOCS= upgrade.docb
INDEXLINK= upgrade.html
.include "../../web.mk"

758
en/tutorials/upgrade/upgrade.docb
View file

@ -1,758 +0,0 @@
<!-- $Id: upgrade.docb,v 1.2 1997-09-14 03:53:16 jfieber Exp $ -->
<!DOCTYPE BOOK PUBLIC "-//Davenport//DTD DocBook V3.0//EN">
<book>
<bookinfo>
<bookbiblio>
<title/<quote>Making the world</quote> your own/
<authorgroup>
<author>
<firstname/Nik/
<surname/Clayton/
<affiliation>
<address><email/Nik.Clayton@iii.co.uk/</address>
</affiliation>
</author>
</authorgroup>
<pubdate>$Date: 1997-09-14 03:53:16 $</pubdate>
</bookbiblio>
</bookinfo>
<preface>
<title/Overview/
<para>This document assumes that you have downloaded a version of the
FreeBSD source code and placed it in <filename>/usr/src</filename>. This
might be the latest version from the -current development branch, or
perhaps you're just tracking -stable. Either way, you have the source
code and now need to update your system.</para>
<para>There are a number of steps to perform to do this, and a few
pitfalls to avoid along the way. This document takes you through
those necessary steps one by one.</para>
<warning>
<title>Take a backup</title>
<para>I cannot stress highly enough how important it is to take a backup
of your system <emphasis>before</emphasis> you do this. While remaking
the world is (as long as you follow these instructions) an easy task to
do, there will inevitably be times when you make mistakes, or when
mistakes made by others in the source tree render your system
unbootable.</para>
<para>Make sure you've taken a backup. And have a fixit floppy to
hand. I've never needed to use them, and, touch wood, I never will,
but it's always better to be safe than sorry.</para>
</warning>
<note>
<title>2.1.7 specific</title>
<para>This document was written and tested with FreeBSD
2.1.7-RELEASE. That was a while ago (at the time of writing
2.2.5-RELEASE is perhaps 30 days away. Most of the information pertains
to all versions of FreeBSD greater than 2.1. Where there are specific
differences between versions (and where I'm aware of them) I'll note
them. If you know of a difference between different versions that
impacts on this document, please let me know.</para>
</note>
</preface>
<chapter>
<title>Check <filename>/etc/make.conf</filename></title>
<para>Examine the file <filename>/etc/make.conf</filename>. This contains
some default defines for <command/make/, which will be used when you
rebuild the source. They're also used every time you use <command/make/,
so it's a good idea to make sure they're set to something sensible for
your system.</para>
<para>Everything is, by default, commented out. Uncomment those entries
that look useful. For a typical user (not a developer), you'll probably
want to uncomment the CFLAGS and NOPROFILE definitions. If your machine
has a floating point unit (386DX, 486DX, Pentium and up class machines)
then you can also uncomment the HAVE_FPU line.</para>
<!-- XXX the definitions above should be wrapped in appropriate DocBook
markup. Don't know what it is yet, though -->
</chapter>
<chapter>
<title/Get the system to single user mode/
<para>You want to compile the system in single user mode. Apart from the
obvious benefit of making things go slightly faster, re-making the system
will touch a lot of important system files, all the standard system
binaries, libraries, include files and so on. Try to change these on a
running system and you're asking for trouble.</para>
<para>As the superuser, you can execute
<informalexample>
<screen><prompt/#/ <userinput/shutdown now/</screen>
</informalexample>
from a running system, which will drop it to single user mode.</para>
<para>Alternatively, reboot the system, and at the boot prompt, enter the
<option>-s</option> flag. The system will then boot single user. At the
shell prompt you should then run
<informalexample>
<screen><prompt/#/ <userinput/fsck -p/
<prompt/#/ <userinput>mount -u /</userinput>
<prompt/#/ <userinput/mount -a -t ufs/
<prompt/#/ <userinput/swapon -a/</screen>
</informalexample>
which check the filesystems, remounts <filename>/</filename> read/write,
mounts all the other UFS filesystems referenced in
<filename>/etc/fstbab</filename> and then turns swapping on.</para>
<para>For extra speed, you can also do
<informalexample>
<screen><prompt/#/ <userinput/mount -u -o async -t ufs -a/</screen>
</informalexample>
which remounts all your UFS filesystems for asynchronous access. The
trade off is that if the power suddenly fails while the system is being
rebuilt you are more likely to suffer from filesystem corruption.</para>
</chapter>
<chapter>
<title/Recompile the source/
<para>In general, this is as simple as
<informalexample>
<screen><prompt/#/ <userinput>cd /usr/src</userinput>
<prompt/#/ <userinput>make world 2>&1 | tee /var/tmp/mw.out</userinput></screen>
</informalexample>
which will re-make the world, storing a copy of all the STDOUT and STDERR
messages in <filename>/var/tmp/mw.out</filename>. It's important to use
<filename>/var/tmp</filename>, as plain <filename>/tmp</filename> is
generally cleared out when you reboot, and you want this output to stay
around for a while.</para>
<note>
<title><filename>/bin/sh</filename> specific</title>
<para>The <quote>2>&1</quote> construct is specific to the
<filename>/bin/sh</filename> shell. Under <filename>/bin/csh</filename>
you could use
<informalexample>
<screen><prompt/#/ <userinput>make world |& tee /var/tmp/mw.out</userinput>
</screen>
</informalexample>
</para>
<para>Other shells have their own constructs to do the same
thing.</para>
<para>Alternatively, if you don't care to use shell redirection, you
could use <command>script</command> to save a copy of all the
output.</para>
<informalexample>
<screen><prompt/#/ <userinput>script /var/tmp/mw.out</userinput>
Script started, output file is /var/tmp/mw.out
<prompt/#/ <userinput/make world/
<emphasis>&hellip; compile, compile, compile &hellip;</emphasis>
<prompt/#/ <userinput/exit/
Script done, &hellip;
</screen>
</informalexample>
</note>
<para>Then go and make yourself several cups of tea. Remaking the world is
a long process. One of our servers, a 200Mhz P6 with fairly
run-of-the-mill SCSI disks, 64MB RAM and 256MB swap it takes a shade
under two hours to complete.</para>
<para>One of the 32MB (128MB swap), P133 machines takes about 5
hours.</para>
<para>The only caveat I am aware of is that (at least the last few times I
tried it with 2.1.5), <command/make world/ expected the <quote/dict/
distribution set to be installed. Otherwise it dies.</para>
<para>Which means, whenever I have to install a new machine, I generally
download the <quote/bin/, <quote/src/ and <quote/dict/ distributions, and
install them. I then make the world to get everything else.</para>
<para>This may have changed up to 2.1.7. I unfortunately do not have the
spare machines to test it.</para>
</chapter>
<chapter>
<title/Build and populate a new root directory somewhere safe/
<para>Remaking the world will not update certain directories (in
particular, <filename>/etc</filename>, <filename>/var</filename> and
<filename>/usr</filename>) with new or changed configuration files. This
is something you have to do by hand, eyeball, and judicious use of the
<command/diff/ command.</para>
<sect1>
<title>Backup your existing <filename>/etc</filename></title>
<para>Although, in theory, nothing's going to touch this directory
automatically, it's always better to be sure. So copy your existing
<filename>/etc</filename> directory somewhere safe. Something like
<informalexample>
<screen><prompt/#/ <userinput>cp -rp /etc /etc.old</userinput></screen>
</informalexample>
will do the trick (<option>-r</option> does a recursive copy,
<option>-p</option> preserves times, ownerships on files and
suchlike).</para>
</sect1>
<sect1>
<title/Build a dummy root/
<para>You need to build a dummy set of directories to install the new
<filename>/etc</filename> and other files into. I generally choose to
put this dummy dir in <filename>/var/tmp/root</filename>, and there are
a number of subdirectories required under this as well. So execute
<informalexample>
<screen><prompt/#/ <userinput>mkdir /var/tmp/root</userinput>
<prompt/#/ <userinput>mtree -deU -f /usr/src/etc/mtree/BSD.root.dist -p /var/tmp/root/</userinput>
<prompt/#/ <userinput>mtree -deU -f /usr/src/etc/mtree/BSD.var.dist -p /var/tmp/root/var/</userinput>
<prompt/#/ <userinput>mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist -p /var/tmp/root/usr/</userinput></screen>
</informalexample>
which will build the necessary directory structure.</para>
<para>A lot of these subdirs are extraneous, but you can ignore them
for the time being, they'll be removed in the next
step.</para>
</sect1>
<sect1>
<title/Install the updated files/
<para>Now that the directory tree has been built, you have to install
the new files from <filename>/usr/src/etc</filename> into it.
<informalexample>
<screen><prompt/#/ <userinput>cd /usr/src/etc</userinput>
<prompt/#/ <userinput>make DESTDIR=/var/tmp/root distribution</userinput></screen>
</informalexample>
This will leave several redundant empty directories scattered
around, cluttering up your <command/ls/ output. The simplest way
to get rid of them is to do
<informalexample>
<screen><prompt/#/ <userinput>cd /var/tmp/root</userinput>
<prompt/#/ <userinput>find -d . -type d | /usr/bin/perl -lne \
'opendir(D,$_);@f=readdir(D);rmdir if $#f != 1;closedir(D);'</userinput></screen>
</informalexample>
which does a depth first search, examines each directory, and if the
number of files in that directory is 2 ('1' is not a typo in the
script) i.e., '.' and '..' then it removes the
directory.</para>
</sect1>
</chapter>
<chapter>
<title>Merge in the changed files from
<filename>/var/tmp/root/*</filename></title>
<para><filename>/var/tmp/root</filename> now contains all the files that
should be placed in appropriate locations below
<filename>/</filename>. You now have to go through each of these files,
determining how they differ with your existing files. This is not a task
that can be automated (at the moment).</para>
<para>Note that some of the files that will have been installed in
<filename>/var/tmp/root</filename> have a leading '.'. Make sure you use
<command/ls -a/ to catch them.</para>
<para>The simplest way to do this is to use the <command/diff/ command to
compare the two files. Use either the <option>-c</option> for the context
output format, or <option>-u</option> for the unified output format. I
find it easier to read context diffs.</para>
<para>For example,
<informalexample>
<screen><prompt/#/ <userinput>diff -c /etc/shells /var/tmp/root/etc/shells</userinput></screen>
</informalexample>
will show you the differences between your
<filename>/etc/shells</filename> file and the new
<filename>/etc/shells</filename> file. Use these to decide whether to
merge in changes that you've made or whether to copy over your old
file.</para>
<para>When it comes to <filename>/var/tmp/root/dev</filename>, you should
just copy over the <filename/MAKEDEV/ file. You may need to examine and
update <filename/MAKEDEV.local/ if you previously had to customise it for
your local environment.</para>
<para>You will use those scripts a little later to update your
<filename>/dev</filename> directory.</para>
<para>Here is a (probably incomplete) list of files that you will
probably want to merge or copy by hand.
<informaltable>
<tgroup cols="2">
<thead>
<row>
<entry/Copy/
<entry/Merge/
</row>
</thead>
<tbody>
<row>
<entry/<emphasis/<filename/passwd///
<entry/<filename/inetd.conf//
</row>
<row>
<entry/<emphasis/<filename/master.passwd///
<entry/<emphasis/<filename/login.access///
</row>
<row>
<entry/<emphasis/<filename/pwd.db///
<entry/<filename/printcap//
</row>
<row>
<entry/<emphasis/<filename/spwd.db///
<entry/<filename/remote//
</row>
<row>
<entry/<emphasis/<filename/group///
<entry/<filename/services//
</row>
<row>
<entry/<filename/aliases/ (nb: run <command/newaliases/)/
<entry/<emphasis/<filename/shells///
</row>
<row>
<entry/<filename/crontab//
<entry/<emphasis/<filename/ttys///
</row>
<row>
<entry/<filename/csh.*//
<entry/<filename/fbtab//
</row>
<row>
<entry/<filename/dumpdates//
<entry/<filename/exports//
</row>
<row>
<entry/<emphasis/<filename/fstab///
<entry/<emphasis/<filename/sysconfig///
</row>
<row>
<entry/<filename/host//
<entry/<filename/rc.local//
</row>
<row/<entry/<filename/magic///
<row><entry><filename>namedb/*</filename></entry></row>
<row><entry><filename>ppp/*</filename></entry></row>
<row/<entry/<filename/profile///
<row/<entry/<filename/resolv.conf///
<row/<entry/<filename/ntp.*///
<row/<entry/<filename/start_if.*///
<row/<entry/<filename/XF86*///
</tbody>
</tgroup>
</informaltable></para>
<para>That is not an exhaustive list, and changes to FreeBSD in the future
may necessitate moving files from the <emphasis/Copy/ column to the
<emphasis/Merge/ column. But you get the idea.</para>
<para>Those filenames shown in <emphasis/emphasised type/ are vital to
the correct running of the system. Be extra sure that they are present
and correct before you reboot.</para>
<note>
<title><filename>/etc/rc.conf</filename> and
<filename>/etc/rc.network</filename></title>
<para>Starting with FreeBSD 2.2.2-RELEASE,
<filename/sysconfig/ has been renamed to <filename/rc.conf/, and
<filename/netstart/ has been renamed to <filename/rc.network/.</para>
</note>
</chapter>
<chapter>
<title>Update <filename>/dev</filename></title>
<para>For safety's sake, this is a multistep process. You should already
have copied in the <filename/MAKEDEV/ script to
<filename>/dev</filename>. Do the following,
<informalexample>
<screen><prompt/#/ <userinput>ls -la /dev > /var/tmp/dev1.out</userinput>
<prompt/#/ <userinput>ls -la /var/tmp/root/dev > /var/tmp/dev2.out</userinput>
</screen></informalexample></para>
<para>This gives you a reference for when things go wrong&hellip; Run a
quick diff over these two files to see if anything's missing. If you use
slices in your disk partitioning (which may not be necessary on a
<quote>dangerously dedicated</quote> disk) then these slices have almost
certainly not been made.</para>
<para>Note down the devices that exist in <filename/dev1.out/ and not
<filename/dev2.out/, and the necessary commands to remake them.</para>
<para>Now do,
<informalexample>
<screen><prompt/#/ <userinput>cd /dev</userinput>
<prompt/#/ <userinput>sh MAKEDEV all</userinput>
</screen>
</informalexample>
This will generate all the standard devices. You must now do whatever's
necessary to recreate devices that you noticed as missing in the previous
step. For my setup, that involved doing
<informalexample>
<screen><prompt/#/ <userinput>sh MAKEDEV sd0s1a</userinput>
<prompt/#/ <userinput>sh MAKEDEV sd1s1a</userinput>
</screen>
</informalexample>
to create the slice entries on my two disks. Your circumstances may
vary. If at all in doubt, make sure you have a handy boot and fixit
floppy, and a very recent backup of your system.</para>
</chapter>
<chapter>
<title/Set the timezone/
<para>If you didn't copy over the <filename/localtime/ file from your old
<filename>/etc</filename> (which is probably a good idea, you may as well
generate it fresh), run <command/tzsetup/ (in
<filename>/usr/sbin</filename>) to set your timezone.</para>
</chapter>
<chapter>
<title/Compiling a new kernel/
<para>To take full advantage of your new system you should recompile the
kernel. This is practically a necessity, as certain memory structures may
have changed, and programs like <command/ps/ and <command/top/ will fail
to work until the kernel and source code versions are the same.</para>
<para>Follow the handbook instructions for compiling a new kernel. If you
have previously built a custom kernel then carefully examine the
<filename/LINT/ config file to see if there are any new options which you
should take advantage of.</para>
<para>A previous version of this document suggested rebooting before
rebuilding the kernel. This is wrong because: </para>
<itemizedlist>
<listitem><para>Commands like <command/ps/, <command/ifconfig/ and
<command/sysctl/ may fail. This could leave your machine unable to
connect to the network.</para></listitem>
<listitem><para>Basic utilities like <command/mount/ could fail,
making it impossible to mount <filename>/</filename>,
<filename>/usr</filename> and so on. This is unlikely if you're
tracking a -stable candidate, but more likely if you're tracking
-current during a large merge.</para></listitem>
</itemizedlist>
<para>For these reasons, it is always best to rebuild and install a
new kernel before rebooting.</para>
</chapter>
<chapter>
<title/Rebooting/
<para>You're now done. After you've verified that everything appears to be
in the right place (pay particular attention to the <emphasis/emphasised/
files listed earlier), you can reboot the system. A simple
<command/fastboot/ should do it.</para>
</chapter>
<chapter>
<title>That's it</title>
<para>You should now have successfully upgraded your FreeBSD system.
Congratulations. It's likely that over the next few days you'll notice
little oddities that don't work as expected, or small upgrades you've
forgotten to do. Something I missed for several days was that
<filename>/etc/magic</filename> was missing. It was only when I went to
run <command/file/ that I realised. A quick <command/make install/ in
<filename>/usr/src/usr.bin/file</filename> sorted that one out.</para>
</chapter>
<chapter>
<title/Questions?/
<sect1>
<title/Do I need to re-make the world for every change?/
<para>There's no easy answer to this one, as it depends on the nature of
the change. For example, I've just run CVSup, and it's shown the
following files as being updated since I last ran it;</para>
<informalexample>
<screen><filename>src/games/cribbage/instr.c</filename>
<filename>src/games/sail/pl_main.c</filename>
<filename>src/release/sysinstall/config.c</filename>
<filename>src/release/sysinstall/media.c</filename>
<filename>src/share/mk/bsd.port.mk</filename></screen>
</informalexample>
<para>There's nothing in there that I'd re-make the world for. I'd go to
the appropriate sub-directories and <command/make all install/, and
that's about it. But if something major changed, like, say,
<filename>src/lib/libc/stdlib</filename> then I'd probably either
re-make the world, or at least those parts of it that are statically
linked (as well as anything else I might have added that's statically
linked).</para>
<para>At the end of the day, it's your call. You might be happy
re-making the world every fortnight say, and let changes accumulate
over that fortnight. Or you might want to re-make just those things
that have changed, and are confident you can spot all the
dependencies.</para>
<para>And, of course, this all depends on how often you want to upgrade,
and whether you are tracking -stable, a release candidate (2.2 at the
time of writing), or -current.</para>
<para>In any case, it's always worthwhile to subscribe to the relevant
mailing lists, depending on which version of FreeBSD you are staying up
to date with. Not only will this give you a <quote/heads up/ of
forthcoming changes, but it also means you'll see problems other people
might be having making the world, and lets you learn from their
problems.</para>
</sect1>
<sect1>
<title>My compile failed with lots of signal 12 (or other signal number)
errors</title>
<para>This is normally indicative of hardware problems. (Re)making the
world is an effective way to stress test your hardware, and will
frequently throw up memory problems. These normally manifest themselves
as the compiler mysteriously dieing on receipt of strange
signals.</para>
<para>A sure indicator of this is if you can restart the make and it
dies at a different point in the process.</para>
<para>In this instance there is little you can do except start swapping
around the components in your machine to determine which one is
failing.</para>
</sect1>
<sect1>
<title>Can I remove <filename>/usr/obj</filename> when I've
finished?</title>
<para>That depends on how you want to make the world on future
occasions.</para>
<para><filename>/usr/obj</filename> contains all the object files
that were produced during the compilation phase. Normally, one of the
first steps in the <quote/make world/ process is to remove this
directory and start afresh. In this case, keeping
<filename>/usr/obj</filename> around after you've finished makes
little sense, and will free up a large chunk of disk space (currently
about 150MB).</para>
<para>However, if you know what you're doing you can have <quote/make
world/ skip this step. This will make subsequent builds run much
faster, since most of sources will not need to be recompiled. The flip
side of this is that subtle dependency problems can creep in, causing
your build to fail in odd ways. This frequently generates noise on the
FreeBSD mailing lists, when one person complains that their build has
failed, not realising that it's because they've tried to cut
corners.</para>
<para>If you want to live dangerously then make the world, passing the
<quote/NOCLEAN/ definition to make, like this;
<informalexample>
<screen><prompt/#/ <userinput>make -DNOCLEAN world</userinput></screen>
</informalexample>
</para>
</sect1>
<sect1>
<title>My compile failed with a particular error, which I've now
fixed. Do I need to remake the world (and lose the result of the
previous build) or can I continue from where I left off?</title>
<para>This depends on how far through the process you got before you
found a problem.</para>
<para><emphasis>In general</emphasis> (and this is not a hard and fast
rule) the <quote>make world</quote> process builds new copies
essential tools (such as <command>gcc</command>, and
<command>make</command>) and the system libraries. These tools and
libraries are then installed. The new tools and libraries are then
used to rebuild themselves, and are installed again. The entire system
(now including regular user programs, such as <command>ls</command> or
<command>grep</command>) is then rebuilt with the new system
files.</para>
<para>If you're at the last state, and you know it (because you've
looked through the output that you were storing) then you can (fairly
safely) do
<informalexample>
<screen><emphasis>&hellip; fix the problem &hellip;</emphasis>
<prompt/#/ <userinput>cd /usr/src</userinput>
<prompt/#/ <userinput/make -DNOCLEAN all/
</screen>
</informalexample>
which will not undo the work of the previous <quote>make
world</quote>.</para>
<para>If you see the message
<screen>
--------------------------------------------------------------
Building everything..
--------------------------------------------------------------
</screen>
in the <quote>make world</quote> output then it's probably fairly safe
to do so.</para>
<para>If you don't see that message, or you're not sure, then it's always
better to be safe than sorry, and restart the build from
scratch.</para>
</sect1>
<sect1>
<title/Can I use one machine as a <emphasis/master/ to upgrade lots of
machines?/
<para>People often ask on the FreeBSD mailing lists whether they can do
all the compiling on one machine, and then use the results of that
compile to <command/make install/ on to other machines around the
network.</para>
<para>This is not something I've done. However, in a message to
questions@freebsd.org, Antonio Bemfica suggested the following
approach:</para>
<screen>
Date: Thu, 20 Feb 1997 14:05:01 -0400 (AST)
From: Antonio Bemfica &lt;bemfica@militzer.me.tuns.ca&gt;
To: freebsd-questions@freebsd.org
Message-ID: &lt;Pine.BSI.3.94.970220135725.245C-100000@militzer.me.tuns.ca&gt;
Josef Karthauser asked:
&gt; Has anybody got a good method for upgrading machines on a network
First make world, etc. on your main machine
Second, mount / and /usr from the remote machine:
main_machine% mount remote_machine:/ /mnt
main_machine% mount remote_machine:/usr /mnt/usr
Third, do a 'make install' with /mnt as the destination:
main_machine% make install DESTDIR=/mnt
Repeat for every other remote machine on your network. It works fine
for me.
Antonio
</screen>
<para>Which sounds interesting. Note that, of course, you will not
upgrade the target machines <filename>/etc</filename> directory (and
others as outlined above) by doing this.</para>
<note>
<title>2.2.2-RELEASE and up</title>
<para>My FreeBSD 2.2.2-RELEASE system shows a <quote>reinstall</quote>
target in <filename>/usr/src/Makefile</filename>. The comment for
this includes:</para>
<programlisting>
# reinstall
#
# If you have a build server, you can NFS mount the source and obj directories
# and do a 'make reinstall' on the *client* to install new binaries from the
# most recent server build.
</programlisting>
<para>I have no idea how well this works, or whether it is present in
earlier versions of FreeBSD. I mention it here for
completeness.</para>
</note>
</sect1>
</chapter>
<chapter>
<title>Contributors</title>
<para>The following people have contributed to this document in some form
or another. Either by directly suggesting alterations and improvements,
or by their messages to the FreeBSD mailing lists, from which I have
shamelessly cribbed information. My thanks to them.</para>
<itemizedlist>
<listitem>
<para>Kees Jan Koster, &lt;<ulink url="mailto:kjk1@ukc.ac.uk">kjk1@ukc.ac.uk</ulink>&gt;</para>
</listitem>
<listitem>
<para>A Joseph Kosy, &lt;<ulink url="mailto:koshy@india.hp.com">koshy@india.hp.com</ulink>&gt;</para>
</listitem>
<listitem>
<para>Greg Lehey, &lt;<ulink url="mailto:grog@lemis.com">grog@lemis.com</ulink>&gt;</para>
</listitem>
<listitem>
<para>Wes Peters, &lt;<ulink
url="mailto:softweyr@xmission.com">softweyr@xmission.com</ulink>&gt;</para>
</listitem>
<listitem>
<para>Joseph Stein, &lt;<ulink url="mailto:joes@joes.users.spiritone.com">joes@joes.users.spiritone.com</ulink>&gt;</para>
</listitem>
</itemizedlist>
</chapter>
</book>