<?xml version="1.0" encoding="iso-8859-1"?>
<!--
The FreeBSD Documentation Project
$FreeBSD$
-->
<chapter id="updating-upgrading">
<chapterinfo>
<authorgroup>
<author>
<firstname>Jim</firstname>
<surname>Mock</surname>
<contrib>Restructured, reorganized, and parts updated
by </contrib>
</author>
<!-- Mar 2000 -->
</authorgroup>
<authorgroup>
<author>
<firstname>Jordan</firstname>
<surname>Hubbard</surname>
<contrib>Original work by </contrib>
</author>
<author>
<firstname>Poul-Henning</firstname>
<surname>Kamp</surname>
</author>
<author>
<firstname>John</firstname>
<surname>Polstra</surname>
</author>
<author>
<firstname>Nik</firstname>
<surname>Clayton</surname>
</author>
</authorgroup>
<!-- with feedback from various others -->
</chapterinfo>
<title>Updating and Upgrading &os;</title>
<sect1 id="updating-upgrading-synopsis">
<title>Synopsis</title>
<para>&os; is under constant development between releases. Some
people prefer to use the officially released versions, while
others prefer to keep in sync with the latest developments.
However, even official releases are often updated with security
and other critical fixes. Regardless of the version used, &os;
provides all the necessary tools to keep the system updated, and
allows for easy upgrades between versions. This chapter
describes how to track the development system and the basic
tools for keeping a &os; system up-to-date.</para>
<para>After reading this chapter, you will know:</para>
<itemizedlist>
<listitem>
<para>Which utilities are available to update the system and
the Ports Collection.</para>
</listitem>
<listitem>
<para>How to keep a &os; system up-to-date with
<application>freebsd-update</application>,
<application>Subversion</application>, or
<application>CTM</application>.</para>
</listitem>
<listitem>
<para>How to compare the state of an installed system against
a known pristine copy.</para>
</listitem>
<listitem>
<para>How to keep the installed documentation up-to-date with
<application>Subversion</application> or documentation
ports<!--, and <application>Docsnap</application>-->.</para>
</listitem>
<listitem>
<para>The difference between the two development
branches: &os.stable; and &os.current;.</para>
</listitem>
<listitem>
<para>How to rebuild and reinstall the entire base
system.</para>
</listitem>
</itemizedlist>
<para>Before reading this chapter, you should:</para>
<itemizedlist>
<listitem>
<para>Properly set up the network connection (<xref
linkend="advanced-networking"/>).</para>
</listitem>
<listitem>
<para>Know how to install additional third-party
software (<xref linkend="ports"/>).</para>
</listitem>
</itemizedlist>
<note>
<para>Throughout this chapter, <command>svn</command> is used to
obtain and update &os; sources. To use it, first install the
<filename role="package">devel/subversion</filename> port or
package.</para>
</note>
</sect1>
<sect1 id="updating-upgrading-freebsdupdate">
<sect1info>
<authorgroup>
<author>
<firstname>Tom</firstname>
<surname>Rhodes</surname>
<contrib>Written by </contrib>
</author>
</authorgroup>
<authorgroup>
<author>
<firstname>Colin</firstname>
<surname>Percival</surname>
<contrib>Based on notes provided by </contrib>
</author>
</authorgroup>
</sect1info>
<title>&os; Update</title>
<indexterm><primary>Updating and Upgrading</primary></indexterm>
<indexterm>
<primary>freebsd-update</primary>
<see>updating-upgrading</see>
</indexterm>
<para>Applying security patches is an important part of
maintaining computer software, especially the operating system.
For the longest time on &os;, this process was not an easy one.
Patches had to be applied to the source code, the code rebuilt
into binaries, and then the binaries had to be
re-installed.</para>
<para>This is no longer the case as &os; now includes a utility
called <command>freebsd-update</command>. This utility
provides two separate functions. First, it allows for binary
security and errata updates to be applied to the &os; base
system without the build and install requirements. Second, the
utility supports minor and major release upgrades.</para>
<note>
<para>Binary updates are available for all architectures and
releases currently supported by the security team. Before
updating to a new release, its release announcement should be
reviewed as it contains important information pertinent to the
release. Release announcements are available from <ulink
url="http://www.FreeBSD.org/releases/"></ulink>.</para>
</note>
<para>If a <command>crontab</command> utilizing the features
of &man.freebsd-update.8; exists, it must be
disabled before the following operation is started.</para>
<sect2 id="freebsdupdate-config-file">
<title>The Configuration File</title>
<para>Some users may wish to tweak the default configuration
in <filename>/etc/freebsd-update.conf</filename>, allowing
better control of the process. The options are well
documented, but the following may require a bit more
explanation:</para>
<programlisting># Components of the base system which should be kept updated.
Components src world kernel</programlisting>
<para>This parameter controls which parts of &os; will be kept
up-to-date. The default is to update the source code, the
entire base system, and the kernel. Components are the same
as those available during installation. For instance, adding
<literal>world/games</literal> would allow game patches to be
applied. Using <literal>src/bin</literal> would allow the
source code in <filename class="directory">src/bin</filename>
to be updated.</para>
<para>The best option is to leave this at the default as
changing it to include specific items requires the user to
list every item to be updated. This could have disastrous
consequences as source code and binaries may become out of
sync.</para>
<programlisting># Paths which start with anything matching an entry in an IgnorePaths
# statement will be ignored.
IgnorePaths</programlisting>
<para>To leave specified directories, such as
<filename class="directory">/bin</filename> or
<filename class="directory">/sbin</filename>, untouched during
the update process, add their paths to this statement. This
option may be used to prevent
<command>freebsd-update</command> from overwriting local
modifications.</para>
<programlisting># Paths which start with anything matching an entry in an UpdateIfUnmodified
# statement will only be updated if the contents of the file have not been
# modified by the user (unless changes are merged; see below).
UpdateIfUnmodified /etc/ /var/ /root/ /.cshrc /.profile</programlisting>
<para>This option will only update unmodified configuration
files in the specified directories. Any changes made by the
user will invalidate the automatic updating of these files.
There is another option,
<literal>KeepModifiedMetadata</literal>, which will instruct
<command>freebsd-update</command> to save the changes during
the merge.</para>
<programlisting># When upgrading to a new &os; release, files which match MergeChanges
# will have any local changes merged into the version from the new release.
MergeChanges /etc/ /var/named/etc/</programlisting>
<para>List of directories with configuration files that
<command>freebsd-update</command> should attempt to merge.
The file merge process is a series of &man.diff.1; patches
similar to &man.mergemaster.8;, but with fewer options.
Merges are either accepted, open an editor, or
<command>freebsd-update</command> will abort. When in doubt,
backup <filename class="directory">/etc</filename> and just
accept the merges. See <xref linkend="mergemaster"/> for more
information about <command>mergemaster</command>.</para>
<programlisting># Directory in which to store downloaded updates and temporary
# files used by &os; Update.
# WorkDir /var/db/freebsd-update</programlisting>
<para>This directory is where all patches and temporary files
are placed. In cases where the user is doing a version
upgrade, this location should have a least a gigabyte of disk
space available.</para>
<programlisting># When upgrading between releases, should the list of Components be
# read strictly (StrictComponents yes) or merely as a list of components
# which *might* be installed of which &os; Update should figure out
# which actually are installed and upgrade those (StrictComponents no)?
# StrictComponents no</programlisting>
<para>When this option is set to <literal>yes</literal>,
<command>freebsd-update</command> will assume that the
<literal>Components</literal> list is complete and will not
attempt to make changes outside of the list. Effectively,
<command>freebsd-update</command> will attempt to update
every file which belongs to the <literal>Components</literal>
list.</para>
</sect2>
<sect2 id="freebsdupdate-security-patches">
<title>Security Patches</title>
<para>&os; security patches may be downloaded and installed
using the following command:</para>
<screen>&prompt.root; <userinput>freebsd-update fetch</userinput>
&prompt.root; <userinput>freebsd-update install</userinput></screen>
<para>If the update applied any kernel patches, the system will
need a reboot in order to boot into the patched kernel.
Otherwise, the system should be patched and
<command>freebsd-update</command> may be run as a nightly
&man.cron.8; job by adding this entry to
<filename>/etc/crontab</filename>:</para>
<programlisting>@daily root freebsd-update cron</programlisting>
<para>This entry states that <command>freebsd-update</command>
will run once every day. When run with <option>cron</option>,
<command>freebsd-update</command> will only check if updates
exist. If patches exist, they will automatically be
downloaded to the local disk but will not be applied. The
<username>root</username> user will be sent an email so that
they may be reviewed and manually installed.</para>
<para>If anything goes wrong, <command>freebsd-update</command>
has the ability to roll back the last set of changes with
the following command:</para>
<screen>&prompt.root; <userinput>freebsd-update rollback</userinput></screen>
<para>Once complete, the system should be restarted if the
kernel or any kernel modules were modified. This will allow
&os; to load the new binaries into memory.</para>
<para>Only the <filename>GENERIC</filename> kernel can be
automatically updated by <command>freebsd-update</command>.
If a custom kernel is installed, it will have to be rebuilt
and reinstalled after <command>freebsd-update</command>
finishes installing the rest of the updates. However,
<command>freebsd-update</command> will detect and update the
<filename>GENERIC</filename> kernel if
<filename class="directory">/boot/GENERIC</filename> exists,
even if it is not the current running kernel of the
system.</para>
<note>
<para>It is a good idea to always keep a copy of the
<filename>GENERIC</filename> kernel in
<filename class="directory">/boot/GENERIC</filename>. It
will be helpful in diagnosing a variety of problems, and in
performing version upgrades using
<command>freebsd-update</command> as described in
<xref linkend="freebsdupdate-upgrade"/>.</para>
</note>
<para>Unless the default configuration in
<filename>/etc/freebsd-update.conf</filename> has been
changed, <command>freebsd-update</command> will install the
updated kernel sources along with the rest of the updates.
Rebuilding and reinstalling a new custom kernel can then be
performed in the usual way.</para>
<note>
<para>The updates distributed by
<command>freebsd-update</command> do not always involve the
kernel. It is not necessary to rebuild a custom kernel if
the kernel sources have not been modified by the execution
of <command>freebsd-update install</command>.
However, <command>freebsd-update</command> will always
update <filename>/usr/src/sys/conf/newvers.sh</filename>.
The current patch level, as indicated by the
<literal>-p</literal> number reported by
<command>uname -r</command>, is obtained from this file.
Rebuilding a custom kernel, even if nothing else changed,
allows &man.uname.1; to accurately report the current
patch level of the system. This is particularly helpful
when maintaining multiple systems, as it allows for a quick
assessment of the updates installed in each one.</para>
</note>
</sect2>
<sect2 id="freebsdupdate-upgrade">
<title>Major and Minor Version Upgrades</title>
<para>Upgrades from one minor version of &os; to another, like
from &os; 9.0 to &os; 9.1, are called
<emphasis>minor version</emphasis> upgrades. Generally,
installed applications will continue to work without problems
after minor version upgrades.</para>
<para><emphasis>Major version</emphasis> upgrades occur when
&os; is upgraded from one major version to another, like from
&os; 8.X to &os; 9.X. Major version upgrades remove
old object files and libraries which will break most third
party applications. It is recommended that all installed
ports either be removed and re-installed or upgraded after a
major version upgrade using a utility such as
<filename role="package">ports-mgmt/portmaster</filename>. A
brute-force rebuild of all installed applications can be
accomplished with this command:</para>
<screen>&prompt.root; <userinput>portmaster -af</userinput></screen>
<para>This will ensure everything will be re-installed
correctly. Note that setting the
<makevar>BATCH</makevar> environment variable to
<literal>yes</literal> will answer <literal>yes</literal> to
any prompts during this process, removing the need for
manual intervention during the build process.</para>
<sect3 id="freebsd-update-custom-kernel">
<title>Dealing with Custom Kernels</title>
<para>If a custom kernel is in use, the upgrade process is
slightly more involved, and the procedure varies depending
on the version of &os;.</para>
<sect4 id="freebsd-update-custom-kernel-8x">
<title>Custom Kernels with &os; 8.X</title>
<para>A copy of the <filename>GENERIC</filename> kernel is
needed, and should be placed in <filename
class="directory">/boot/GENERIC</filename>. If the
<filename>GENERIC</filename> kernel is not present in the
system, it may be obtained using one of the following
methods:</para>
<itemizedlist>
<listitem>
<para>If a custom kernel has only been built once, the
kernel in <filename
class="directory">/boot/kernel.old</filename> is
actually <filename>GENERIC</filename>. Rename this
directory to <filename
class="directory">/boot/GENERIC</filename>.</para>
</listitem>
<listitem>
<para>Assuming physical access to the machine is
possible, a copy of the <filename>GENERIC</filename>
kernel can be installed from the installation media
using the following commands:</para>
<screen>&prompt.root; <userinput>mount /cdrom</userinput>
&prompt.root; <userinput>cd /cdrom/<replaceable>X.Y-RELEASE</replaceable>/kernels</userinput>
&prompt.root; <userinput>./install.sh GENERIC</userinput></screen>
<para>Replace <filename
class="directory"><replaceable>X.Y-RELEASE</replaceable></filename>
with the actual version of the release being used.
The <filename>GENERIC</filename> kernel will be
installed in <filename
class="directory">/boot/GENERIC</filename> by
default.</para>
</listitem>
<listitem>
<para>Failing all the above, the
<filename>GENERIC</filename> kernel may be rebuilt and
installed from source:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>env DESTDIR=/boot/GENERIC make kernel __MAKE_CONF=/dev/null SRCCONF=/dev/null</userinput>
&prompt.root; <userinput>mv /boot/GENERIC/boot/kernel/* /boot/GENERIC</userinput>
&prompt.root; <userinput>rm -rf /boot/GENERIC/boot</userinput></screen>
<para>For this kernel to be picked up as
<filename>GENERIC</filename> by
<command>freebsd-update</command>, the
<filename>GENERIC</filename> configuration file must
not have been modified in any way. It is also
suggested that it is built without any other special
options.</para>
</listitem>
</itemizedlist>
<para>Rebooting to the <filename>GENERIC</filename> kernel
is not required at this stage.</para>
</sect4>
<sect4 id="freebsd-update-custom-kernel-9x">
<title>Custom Kernels with &os; 9.X and Later</title>
<itemizedlist>
<listitem>
<para>If a custom kernel has only been built once, the
kernel in
<filename
class="directory">/boot/kernel.old</filename>
is actually the <literal>GENERIC</literal> kernel.
Rename this directory to <filename
class="directory">/boot/kernel</filename>.</para>
</listitem>
<listitem>
<para>If physical access to the machine is available, a
copy of the <literal>GENERIC</literal> kernel can be
installed from the installation media using these
commands:</para>
<screen>&prompt.root; <userinput>mount /cdrom</userinput>
&prompt.root; <userinput>cd /cdrom/usr/freebsd-dist</userinput>
&prompt.root; <userinput>tar -C/ -xvf kernel.txz boot/kernel/kernel</userinput></screen>
</listitem>
<listitem>
<para>If the options above cannot be used, the
<literal>GENERIC</literal> kernel may be rebuilt and
installed from source:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make kernel __MAKE_CONF=/dev/null SRCCONF=/dev/null</userinput></screen>
<para>For this kernel to be identified as the
<literal>GENERIC</literal> kernel by
<command>freebsd-update</command>, the
<filename>GENERIC</filename> configuration file must
not have been modified in any way. It is also
suggested that the kernel is built without any other
special options.</para>
</listitem>
</itemizedlist>
<para>Rebooting to the <filename>GENERIC</filename> kernel
is not required at this stage.</para>
</sect4>
</sect3>
<sect3 id="freebsdupdate-using">
<title>Performing the Upgrade</title>
<para>Major and minor version upgrades may be performed by
providing <command>freebsd-update</command> with a release
version target. The following command will update to
&os; 9.1:</para>
<screen>&prompt.root; <userinput>freebsd-update -r 9.1-RELEASE upgrade</userinput></screen>
<para>After the command has been received,
<command>freebsd-update</command> will evaluate the
configuration file and current system in an attempt to
gather the information necessary to perform the upgrade. A
screen listing will display which components have and have
not been detected. For example:</para>
<screen>Looking up update.FreeBSD.org mirrors... 1 mirrors found.
Fetching metadata signature for 9.0-RELEASE from update1.FreeBSD.org... done.
Fetching metadata index... done.
Inspecting system... done.
The following components of FreeBSD seem to be installed:
kernel/smp src/base src/bin src/contrib src/crypto src/etc src/games
src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue
src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin
world/base world/info world/lib32 world/manpages
The following components of FreeBSD do not seem to be installed:
kernel/generic world/catpages world/dict world/doc world/games
world/proflibs
Does this look reasonable (y/n)? y</screen>
<para>At this point, <command>freebsd-update</command> will
attempt to download all files required for the upgrade. In
some cases, the user may be prompted with questions
regarding what to install or how to proceed.</para>
<para>When using a custom kernel, the above step will produce
a warning similar to the following:</para>
<screen>WARNING: This system is running a "<replaceable>MYKERNEL</replaceable>" kernel, which is not a
kernel configuration distributed as part of FreeBSD 9.0-RELEASE.
This kernel will not be updated: you MUST update the kernel manually
before running "/usr/sbin/freebsd-update install"</screen>
<para>This warning may be safely ignored at this point. The
updated <filename>GENERIC</filename> kernel will be used as
an intermediate step in the upgrade process.</para>
<para>Once all the patches have been downloaded to the local
system, they will be applied. This process may take a
while, depending on the speed and workload of the machine.
Configuration files will then be merged. The merging
process requires some user intervention as a file may be
merged or an editor may appear on screen for a manual merge.
The results of every successful merge will be shown to the
user as the process continues. A failed or ignored merge
will cause the process to abort. Users may wish to make a
backup of <filename class="directory">/etc</filename> and
manually merge important files, such as
<filename>master.passwd</filename> or
<filename>group</filename> at a later time.</para>
<note>
<para>The system is not being altered yet as all patching
and merging is happening in another directory. Once all
patches have been applied successfully, all configuration
files have been merged and it seems the process will go
smoothly, the changes can be committed to disk by the
user using the following command:</para>
<screen>&prompt.root; <userinput>freebsd-update install</userinput></screen>
</note>
<para>The kernel and kernel modules will be patched first. At
this point, the machine must be rebooted. If the system is
running with a custom kernel, use &man.nextboot.8; to set
the kernel for the next boot to the updated
<filename class="directory">/boot/GENERIC</filename>:</para>
<screen>&prompt.root; <userinput>nextboot -k GENERIC</userinput></screen>
<warning>
<para>Before rebooting with the <filename>GENERIC</filename>
kernel, make sure it contains all the drivers required for
the system to boot properly and connect to the network,
if the machine being updated is accessed remotely. In
particular, if the running custom kernel contains built-in
functionality usually provided by kernel modules, make
sure to temporarily load these modules into the
<filename>GENERIC</filename> kernel using the
<filename>/boot/loader.conf</filename> facility.
It is recommended to disable non-essential services as
well as any disk and network mounts until the upgrade
process is complete.</para>
</warning>
<para>The machine should now be restarted with the updated
kernel:</para>
<screen>&prompt.root; <userinput>shutdown -r now</userinput></screen>
<para>Once the system has come back online, restart
<command>freebsd-update</command> using the following
command. The state of the process has been saved and thus,
<command>freebsd-update</command> will not start from the
beginning, but will remove all old shared libraries and
object files.</para>
<screen>&prompt.root; <userinput>freebsd-update install</userinput></screen>
<note>
<para>Depending upon whether any library version numbers
were bumped, there may only be two install phases instead
of three.</para>
</note>
</sect3>
<sect3 id="freebsdupdate-portsrebuild">
<title>Rebuilding Ports After a Major Version Upgrade</title>
<para>After a major version upgrade, all third party software
needs to be rebuilt and re-installed. This is required as
installed software may depend on libraries which have been
removed during the upgrade process. This process can be
automated using <filename
role="package">ports-mgmt/portmaster</filename>:</para>
<screen>&prompt.root; <userinput>portmaster -f</userinput></screen>
<para>Once this has completed, finish the upgrade process with
a final call to <command>freebsd-update</command> in order
to tie up all the loose ends in the upgrade process:</para>
<screen>&prompt.root; <userinput>freebsd-update install</userinput></screen>
<para>If the <filename>GENERIC</filename> kernel was
temporarily used, this is the time to build and install a
new custom kernel in the usual way.</para>
<para>Reboot the machine into the new &os; version. The
process is complete.</para>
</sect3>
</sect2>
<sect2 id="freebsdupdate-system-comparison">
<title>System State Comparison</title>
<para><command>freebsd-update</command> can be used to test the
state of the installed &os; version against a known good copy.
This option evaluates the current version of system utilities,
libraries, and configuration files. To begin the comparison,
issue the following command:</para>
<screen>&prompt.root; <userinput>freebsd-update IDS >> outfile.ids</userinput></screen>
<warning>
<para>While the command name is <acronym>IDS</acronym> it is
not a replacement for a real intrusion detection system such
as <filename role="package">security/snort</filename>. As
<command>freebsd-update</command> stores data on disk, the
possibility of tampering is evident. While this possibility
may be reduced using <varname>kern.securelevel</varname> and
by storing the <command>freebsd-update</command> data on a
read only file system when not in use, a better solution
would be to compare the system against a secure disk, such
as a <acronym>DVD</acronym> or securely stored external
<acronym>USB</acronym> disk device.</para>
</warning>
<para>The system will now be inspected, and a lengthy listing of
files, along with the &man.sha256.1; hash values for both the
known value in the release and the current installation, will
be sent to the specified
<filename>outfile.ids</filename> file.</para>
<para>The entries in the listing are extremely long, but the
output format may be easily parsed. For instance, to obtain a
list of all files which differ from those in the release,
issue the following command:</para>
<screen>&prompt.root; <userinput>cat outfile.ids | awk '{ print $1 }' | more</userinput>
/etc/master.passwd
/etc/motd
/etc/passwd
/etc/pf.conf</screen>
<para>This sample output has been truncated as many more files
exist. Some files have natural modifications. For example,
<filename>/etc/passwd</filename> has been modified because
users have been added to the system. Other files, such as
kernel modules, may differ as
<command>freebsd-update</command> may have updated them.
To exclude specific files or directories, add them to the
<literal>IDSIgnorePaths</literal> option in
<filename>/etc/freebsd-update.conf</filename>.</para>
<para>This system may be used as part of an elaborate upgrade
method, aside from the previously discussed version.</para>
</sect2>
</sect1>
<sect1 id="updating-upgrading-portsnap">
<sect1info>
<authorgroup>
<author>
<firstname>Tom</firstname>
<surname>Rhodes</surname>
<contrib>Written by </contrib>
</author>
</authorgroup>
<authorgroup>
<author>
<firstname>Colin</firstname>
<surname>Percival</surname>
<contrib>Based on notes provided by </contrib>
</author>
</authorgroup>
</sect1info>
<title>Portsnap: a Ports Collection Update Tool</title>
<indexterm><primary>Updating and Upgrading</primary></indexterm>
<indexterm>
<primary>Portsnap</primary>
<see>Updating and Upgrading</see>
</indexterm>
<para>The base system of &os; includes &man.portsnap.8; for
updating the Ports Collection. This utility connects to a
&os; site, verifies the secure key, and downloads a new copy of
the Ports Collection. The key is used to verify the integrity
of all downloaded files. To download the latest Ports
Collection files, issue the following command:</para>
<screen>&prompt.root; <userinput>portsnap fetch</userinput>
Looking up portsnap.FreeBSD.org mirrors... 9 mirrors found.
Fetching snapshot tag from geodns-1.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Tue May 22 02:12:15 CEST 2012 to Wed May 23 16:28:31 CEST 2012.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 3 metadata files... done.
Fetching 90 patches.....10....20....30....40....50....60....70....80....90. done.
Applying patches... done.
Fetching 133 new ports or files... done.</screen>
<para>What this example shows is that &man.portsnap.8; has found
and verified several patches to the current ports data. This
also indicates that the utility was run previously; if it was a
first time run, the collection would have simply been
downloaded.</para>
<para>When &man.portsnap.8; successfully completes a
<command>fetch</command> operation, the Ports Collection and
subsequent patches which exist on the local system have passed
verification. The first time <command>portsnap</command> is
executed, use <literal>extract</literal> to install the
downloaded files:</para>
<screen>&prompt.root; <userinput>portsnap extract</userinput>
/usr/ports/.cvsignore
/usr/ports/CHANGES
/usr/ports/COPYRIGHT
/usr/ports/GIDs
/usr/ports/KNOBS
/usr/ports/LEGAL
/usr/ports/MOVED
/usr/ports/Makefile
/usr/ports/Mk/bsd.apache.mk
/usr/ports/Mk/bsd.autotools.mk
/usr/ports/Mk/bsd.cmake.mk
<replaceable>...</replaceable></screen>
<para>To update an already installed Ports Collection, use
<command>portsnap update</command>:</para>
<screen>&prompt.root; <userinput>portsnap update</userinput></screen>
<para>The process is now complete, and applications may be
installed or upgraded using the updated Ports Collection.</para>
<para>When using <literal>fetch</literal>, the
<literal>extract</literal> or the <literal>update</literal>
operation may be run consecutively:</para>
<screen>&prompt.root; <userinput>portsnap fetch update</userinput></screen>
<para>This command downloads the latest version of the Ports
Collection and updates the local version under
<filename class="directory">/usr/ports</filename>.</para>
</sect1>
<sect1 id="updating-upgrading-documentation">
<title>Updating the Documentation Set</title>
<indexterm><primary>Updating and Upgrading</primary></indexterm>
<indexterm>
<primary>Documentation</primary>
<see>Updating and Upgrading</see>
</indexterm>
<para>Documentation is an integral part of the &os; operating
system. While an up-to-date version of the &os; Documentation
Set is always available on the <ulink
url="http://www.freebsd.org/doc/">&os; web site</ulink>,
some users might have slow or no permanent network connectivity.
There are several ways to update the local copy of documentation
with the latest &os; Documentation Set.</para>
<sect2 id="dsvn-doc">
<title>Using <application>Subversion</application> to Update the
Documentation</title>
<para>The &os; documentation sources can be obtained with
<application>svn</application>. This section
describes how to:</para>
<itemizedlist>
<listitem>
<para>Install the documentation toolchain, the tools that
are required to rebuild the &os; documentation from its
source.</para>
</listitem>
<listitem>
<para>Download a copy of the documentation source at
<filename class="directory">/usr/doc</filename>, using
<application>svn</application>.</para>
</listitem>
<listitem>
<para>Rebuild the &os; documentation from its source, and
install it under <filename
class="directory">/usr/share/doc</filename>.</para>
</listitem>
<listitem>
<para>Recognize some of the build options that are
supported by the build system of the documentation, such
as the options that build only some of the different
language translations of the documentation or the options
that select a specific output format.</para>
</listitem>
</itemizedlist>
</sect2>
<sect2 id="installing-documentation-toolchain">
<title>Installing <application>svn</application> and the
Documentation Toolchain</title>
<para>Rebuilding the &os; documentation from source requires a
collection of tools which are not part of the &os; base system
due to the amount of disk space these tools use. They are
also not useful to all &os; users, only those users that are
actively writing new documentation for &os; or are frequently
updating their documentation from source.</para>
<para>The required tools, including
<application>svn</application>, are available in the
<filename role="package">textproc/docproj</filename> meta-port
developed by the &os; Documentation Project.</para>
<note>
<para>When no &postscript; or PDF documentation required, one
might consider installing the <filename
role="package">textproc/docproj-nojadetex</filename> port
instead. This version of the documentation toolchain
includes everything except the
<application>teTeX</application> typesetting engine.
<application>teTeX</application> is a very large collection
of tools, so it may be quite sensible to omit its
installation if PDF output is not really necessary.</para>
</note>
</sect2>
<sect2 id="updating-documentation-sources">
<title>Updating the Documentation Sources</title>
<para>In this example, <application>svn</application> is used to
fetch a clean copy of the documentation sources from the
western US mirror using the HTTPS protocol:</para>
<screen>&prompt.root; <userinput>svn checkout <replaceable>https://svn0.us-west.FreeBSD.org</replaceable>/doc/head /usr/doc</userinput></screen>
<para>Select the closest mirror from the available <link
linkend="svn-mirrors">Subversion mirror sites</link>.</para>
<para>The initial download of the documentation sources may take
a while. Let it run until it completes.</para>
<para>Future updates of the documentation sources may be fetched
by running:</para>
<screen>&prompt.root; <userinput>svn update <filename class="directory">/usr/doc</filename></userinput></screen>
<para>After checking out the sources, an alternative way of
updating the documentation is supported by the
<filename>/usr/doc/Makefile</filename> by running the
following commands:</para>
<screen>&prompt.root; <userinput>cd /usr/doc</userinput>
&prompt.root; <userinput>make update</userinput></screen>
</sect2>
<sect2 id="updating-documentation-options">
<title>Tunable Options of the Documentation Sources</title>
<para>The updating and build system of the &os; documentation
set supports a few options that ease the process of updating
only parts of the documentation, or the build of specific
translations. These options can be set either as system-wide
options in <filename>/etc/make.conf</filename>, or as
command-line options passed to &man.make.1;.</para>
<para>The options include:</para>
<variablelist>
<varlistentry>
<term><makevar>DOC_LANG</makevar></term>
<listitem>
<para>The list of languages and encodings to build and
install, such as <literal>en_US.ISO8859-1</literal> for
English documentation.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><makevar>FORMATS</makevar></term>
<listitem>
<para>A single format or a list of output formats to be
built. Currently, <literal>html</literal>,
<literal>html-split</literal>, <literal>txt</literal>,
<literal>ps</literal>, <literal>pdf</literal>,
and <literal>rtf</literal> are supported.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><makevar>DOCDIR</makevar></term>
<listitem>
<para>Where to install the documentation. It defaults to
<filename
class="directory">/usr/share/doc</filename>.</para>
</listitem>
</varlistentry>
</variablelist>
<para>For more <command>make</command> variables supported as
system-wide options in &os;, refer to
&man.make.conf.5;.</para>
<para>For more <command>make</command> variables supported by
the build system of the &os; documentation, refer to the
<ulink url="&url.doc.langbase;/books/fdp-primer">&os;
Documentation Project Primer for New
Contributors</ulink>.</para>
</sect2>
<sect2 id="updating-installed-documentation">
<title>Installing the &os; Documentation from Source</title>
<para>Once an up-to-date snapshot of the documentation sources
has been fetched to <filename
class="directory">/usr/doc</filename>, everything is
ready for an update of the installed documentation.</para>
<para>A full update of all the languages defined in
<makevar>DOC_LANG</makevar> may be performed by typing:</para>
<screen>&prompt.root; <userinput>cd /usr/doc</userinput>
&prompt.root; <userinput>make install clean</userinput></screen>
<para>If an update of only a specific language is desired,
&man.make.1; can be invoked in a language specific
subdirectory of
<filename class="directory">/usr/doc</filename>:</para>
<screen>&prompt.root; <userinput>cd /usr/doc/en_US.ISO8859-1</userinput>
&prompt.root; <userinput>make update install clean</userinput></screen>
<para>The output formats that will be installed may be specified
by setting <makevar>FORMATS</makevar>:</para>
<screen>&prompt.root; <userinput>cd /usr/doc</userinput>
&prompt.root; <userinput>make FORMATS='html html-split' install clean</userinput></screen>
<para>For information on editing and submitting corrections to
the documentation, refer to the <ulink
url="&url.books.fdp-primer;">&os; Documentation
Project Primer for New Contributors</ulink>.</para>
</sect2>
<sect2 id="doc-ports">
<sect2info>
<authorgroup>
<author>
<firstname>Marc</firstname>
<surname>Fonvieille</surname>
<contrib>Based on the work of </contrib>
</author>
</authorgroup>
</sect2info>
<title>Using Documentation Ports</title>
<indexterm><primary>Updating and Upgrading</primary></indexterm>
<indexterm>
<primary>documentation package</primary>
<see>Updating and Upgrading</see>
</indexterm>
<para>The previous section presented a method for updating the
&os; documentation from sources. Source based updates may not
be feasible or practical for all &os; systems as building the
documentation sources requires the <emphasis>documentation
toolchain</emphasis>, a certain level of familiarity with
<application>svn</application> and source checkouts from a
repository, and a few manual steps to build the checked out
sources. This section describes an alternative method which
uses the Ports Collection and makes it possible to:</para>
<itemizedlist>
<listitem>
<para>Download and install pre-built snapshots of the
documentation, without having to locally build anything
or install the documentation toolchain.</para>
</listitem>
<listitem>
<para>Download the documentation sources and build them
through the ports framework, making the checkout and build
steps a bit easier.</para>
</listitem>
</itemizedlist>
<para>These two methods of updating the &os; documentation are
supported by a set of <emphasis>documentation
ports</emphasis>, updated by the &a.doceng; on a monthly
basis. These are listed in the &os; Ports Collection,
under the <ulink
url="http://www.freshports.org/docs/">docs</ulink>
category.</para>
<sect3 id="doc-ports-install-make">
<title>Building and Installing Documentation Ports</title>
<para>The documentation ports use the ports building framework
to make documentation builds easier. They automate the
process of checking out the documentation source, running
&man.make.1; with the appropriate environment settings and
command-line options, and they make the installation or
deinstallation of documentation as easy as the installation
of any other &os; port or package.</para>
<note>
<para>As an extra feature, when the documentation ports are
built locally, they record a dependency to the
<emphasis>documentation toolchain</emphasis> ports, so
that they are also automatically installed.</para>
</note>
<para>Organization of the documentation ports is as
follows:</para>
<itemizedlist>
<listitem>
<para>The <quote>master port</quote>, <filename
role="package">misc/freebsd-doc-en</filename>,
which installs all of the English documentation
ports.</para>
</listitem>
<listitem>
<para>The <quote>all in one port</quote>, <filename
role="package">misc/freebsd-doc-all</filename>,
builds and installs all documentation in all available
languages.</para>
</listitem>
<listitem>
<para>There is a <quote>slave port</quote> for each
translation, such as <filename
role="package">misc/freebsd-doc-hu</filename> for the
Hungarian-language documents.</para>
</listitem>
</itemizedlist>
<para>For example, to build and install the English
documentation in split <acronym>HTML</acronym> format,
similar to the format used on <ulink
url="http://www.FreeBSD.org"></ulink>, to
<filename
class="directory">/usr/local/share/doc/freebsd</filename>,
install the following port</para>
<screen>&prompt.root; <userinput>cd /usr/ports/misc/freebsd-doc-en</userinput>
&prompt.root; <userinput>make install clean</userinput></screen>
<sect4 id="doc-ports-options">
<title>Common Knobs and Options</title>
<para>There are many options for modifying the default
behavior of the documentation ports, including:</para>
<variablelist>
<varlistentry>
<term><makevar>WITH_HTML</makevar></term>
<listitem>
<para>Builds the HTML format with a single HTML file
per document. The formatted documentation is saved
to a file called <filename>article.html</filename>,
or <filename>book.html</filename>, as appropriate,
plus images.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><makevar>WITH_PDF</makevar></term>
<listitem>
<para>Builds the &adobe; Portable Document Format
(PDF). The formatted documentation is saved to a
file called <filename>article.pdf</filename> or
<filename>book.pdf</filename>, as
appropriate.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><makevar>DOCBASE</makevar></term>
<listitem>
<para>Specifies where to install the documentation.
It defaults to <filename
class="directory">/usr/local/share/doc/freebsd</filename>.</para>
<note>
<para>The default target directory differs from the
directory used <application>svn</application>.
This is because ports are usually installed within
<filename class="directory">/usr/local</filename>.
This can be overridden by using
<makevar>PREFIX</makevar>.</para>
</note>
</listitem>
</varlistentry>
</variablelist>
<para>This example uses variables to install the Hungarian
documentation as a PDF:</para>
<screen>&prompt.root; cd /usr/ports/misc/freebsd-doc-hu
&prompt.root; make -DWITH_PDF DOCBASE=share/doc/freebsd/hu install clean</screen>
</sect4>
</sect3>
<sect3 id="doc-ports-install-package">
<title>Using Documentation Packages</title>
<para>Building the documentation ports from source, as
described in the previous section, requires a local
installation of the documentation toolchain and a bit of
disk space for the build of the ports. When resources are
not available to install the documentation toolchain, or
because the build from sources would take too much disk
space, it is still possible to install pre-built snapshots
of the documentation ports.</para>
<para>The &a.doceng; prepares monthly snapshots of the &os;
documentation packages. These binary packages can be used
with any of the bundled package tools, like &man.pkg.add.1;,
&man.pkg.delete.1;, and so on.</para>
<note>
<para>When binary packages are used, the &os; documentation
will be installed in <emphasis>all</emphasis> available
formats for the given language.</para>
</note>
<para>For example, the following command will install the
latest pre-built package of the Hungarian
documentation:</para>
<screen>&prompt.root; <userinput>pkg_add -r hu-freebsd-doc</userinput></screen>
<note>
<para>Packages use a format that differs from the
corresponding port's name:
<literal><replaceable>lang</replaceable>-freebsd-doc</literal>,
where <replaceable>lang</replaceable> is the short format
of the language code, such as <literal>hu</literal> for
Hungarian, or <literal>zh_cn</literal> for Simplified
Chinese.</para>
</note>
</sect3>
<sect3 id="doc-ports-update">
<title>Updating Documentation Ports</title>
<para>Documentation ports can be updated like any other port.
For example, the following command updates the installed
Hungarian documentation using
<filename role="package">ports-mgmt/portmaster</filename>
by using packages only:</para>
<screen>&prompt.root; <userinput>portmaster -PP hu-freebsd-doc</userinput></screen>
</sect3>
</sect2>
</sect1>
<sect1 id="current-stable">
<title>Tracking a Development Branch</title>
<indexterm><primary>-CURRENT</primary></indexterm>
<indexterm><primary>-STABLE</primary></indexterm>
<para>There are two development branches to &os;: &os.current;
and &os.stable;. This section provides an explanation of each
and describes how to keep a system up-to-date with each
respective tree. &os.current; will be discussed first, then
&os.stable;.</para>
<sect2 id="current">
<title>Staying Current with &os;</title>
<para>&os.current; is the <quote>bleeding edge</quote> of &os;
development. &os.current; users are expected to have a high
degree of technical skill and should be capable of solving
difficult system problems on their own. If you are new to
&os;, track &os.stable; instead.</para>
<sect3>
<title>What Is &os.current;?</title>
<indexterm><primary>snapshot</primary></indexterm>
<para>&os.current; is the very latest source code for &os;.
This includes work in progress, experimental changes, and
transitional mechanisms that might or might not be present
in the next official release of the software. While many
&os; developers compile the &os.current; source code daily,
there are periods of time when the sources are not
buildable. These problems are resolved as quickly as
possible, but whether or not &os.current; brings disaster or
greatly desired functionality can be a matter of when the
source code was synced.</para>
</sect3>
<sect3>
<title>Who Needs &os.current;?</title>
<para>&os.current; is made available for three primary
interest groups:</para>
<orderedlist>
<listitem>
<para>Members of the &os; community who are actively
working on some part of the source tree and for whom
keeping <quote>current</quote> is an absolute
requirement.</para>
</listitem>
<listitem>
<para>Members of the &os; community who are active
testers, willing to spend time solving problems in order
to ensure that &os.current; remains as sane as possible.
These testers wish to make topical suggestions on
changes and the general direction of &os;, and submit
patches to implement them.</para>
</listitem>
<listitem>
<para>Those who merely wish to keep an eye on things, or
to use the current sources for reference purposes.
These people also make the occasional comment or
contribute code.</para>
</listitem>
</orderedlist>
</sect3>
<sect3>
<title>What Is &os.current; <emphasis>Not</emphasis>?</title>
<orderedlist>
<listitem>
<para>A fast-track to getting new features before the next
release. Pre-release features are not yet fully tested
and most likely contain bugs.</para>
</listitem>
<listitem>
<para>A quick way of getting bug fixes. Any given commit is
just as likely to introduce new bugs as to fix existing
ones.</para>
</listitem>
<listitem>
<para>In any way <quote>officially
supported</quote>.</para>
</listitem>
</orderedlist>
</sect3>
<sect3>
<title>Using &os.current;</title>
<indexterm>
<primary>-CURRENT</primary>
<secondary>using</secondary>
</indexterm>
<orderedlist>
<listitem>
<para>Join the &a.current.name; and the
&a.svn-src-head.name; lists. This is
<emphasis>essential</emphasis> in order to see the
comments that people are making about the current state
of the system and to receive important bulletins which
may be critical to the system's continued health.</para>
<para>The &a.svn-src-head.name; list records the commit
log entry for each change as it is made, along with any
pertinent information on possible side-effects.</para>
<para>To join these lists, go to &a.mailman.lists.link;,
click on the list to subscribe to, and follow the
instructions. In order to track changes to the whole
source tree, subscribe to the &a.svn-src-all.name;
list.</para>
</listitem>
<listitem>
<para>Grab the sources from a &os;
<link linkend="mirrors">mirror site</link> using
one of the following methods:</para>
<orderedlist>
<indexterm>
<primary>Subversion</primary>
</indexterm>
<indexterm>
<primary><command>cron</command></primary>
</indexterm>
<indexterm>
<primary>-CURRENT</primary>
<secondary>Syncing with
<application>Subversion</application>
</secondary>
</indexterm>
<indexterm>
<primary>-CURRENT</primary>
<secondary>Syncing with
<application>CTM</application>
</secondary>
</indexterm>
<listitem>
<para>Use <link linkend="svn">svn</link> to check out
the desired development or release branch. This is
the recommended method, providing access to &os;
development as it occurs. Checkout the -CURRENT
code from the <literal>head</literal> branch of one
of the <link linkend="svn-mirrors">Subversion mirror
sites</link>. Due to the size of the repository,
it is recommended that only desired subtrees be
checked out.</para>
</listitem>
<listitem>
<indexterm>
<primary>-CURRENT</primary>
<secondary>Syncing with CTM</secondary>
</indexterm>
<para>Use the <application><link
linkend="ctm">CTM</link></application> facility.
If you have bad connectivity such as high price
connections or only email access,
<application>CTM</application> is an option, but it
is not as reliable as <application><link
linkend="svn">Subversion</link></application>.
For this reason, <application><link
linkend="svn">Subversion</link></application>
is the recommended method for any system with
Internet connectivity.</para>
</listitem>
</orderedlist>
</listitem>
<listitem>
<para>If you plan to run, and not just look at the
sources, download <emphasis>all</emphasis> of
&os.current;, not just selected portions. Various parts
of the source depend on updates elsewhere, and trying to
compile just a subset is almost guaranteed to cause
problems.</para>
<indexterm>
<primary>-CURRENT</primary>
<secondary>compiling</secondary>
</indexterm>
<para>Before compiling &os.current;, read
<filename>/usr/src/Makefile</filename> very carefully.
<link linkend="makeworld">Install a new kernel and
rebuild the world</link> the first time through as part
of the upgrading process. Read the &a.current; and
<filename>/usr/src/UPDATING</filename> to stay
up-to-date on other bootstrapping procedures that
sometimes become necessary on the road to the next
release.</para>
</listitem>
<listitem>
<para>Be active! &os.current; users are encouraged to
submit their suggestions for enhancements or bug fixes.
Suggestions with accompanying code are received most
enthusiastically!</para>
</listitem>
</orderedlist>
</sect3>
</sect2>
<sect2 id="stable">
<title>Staying Stable with &os;</title>
<sect3>
<title>What Is &os.stable;?</title>
<indexterm><primary>-STABLE</primary></indexterm>
<para>&os.stable; is the development branch from which major
releases are made. Changes go into this branch at a
different pace, and with the general assumption that they
have first gone into &os.current; for testing. This is
<emphasis>still</emphasis> a development branch, however,
and this means that at any given time, the sources for
&os.stable; may or may not be suitable for any particular
purpose. It is simply another engineering development
track, not a resource for end-users.</para>
</sect3>
<sect3>
<title>Who Needs &os.stable;?</title>
<para>Those interested in tracking or contributing to the
FreeBSD development process, especially as it relates to the
next <quote>point</quote> release of FreeBSD, should
consider following &os.stable;.</para>
<para>While security fixes go into the &os.stable; branch, one
does not <emphasis>need</emphasis> to track &os.stable; to
receive security fixes. Every security advisory for &os;
explains how to fix the problem for the releases it
affects which are not yet EOL.
<footnote>
<para>For a complete description of the current security
policy for old releases of FreeBSD, refer to <ulink
url="&url.base;/security/">http://www.FreeBSD.org/security/</ulink>.</para></footnote>.</para>
<para>While the &os.stable; branch should compile and run at
all times, this cannot be guaranteed. While code is
developed in &os.current; before including it in
&os.stable;, more people run &os.stable; than &os.current;,
so it is inevitable that bugs and corner cases will
sometimes be found in &os.stable; that were not apparent in
&os.current;.</para>
<para>For these reasons, one should <emphasis>not</emphasis>
blindly track &os.stable;. It is particularly important not
to update any production servers to &os.stable; without
first thoroughly testing the code in a development/testing
environment.</para>
<para>Except for those users who have the resources to perform
testing, it is recommended that users instead run the most
recent release of FreeBSD, and use the binary update
mechanism to move from release to release.</para>
</sect3>
<sect3>
<title>Using &os.stable;</title>
<indexterm>
<primary>-STABLE</primary>
<secondary>using</secondary>
</indexterm>
<orderedlist>
<listitem>
<para>Join the &a.stable.name; list in order to stay
informed of build-dependencies that may appear in
&os.stable; or any other issues requiring special
attention. Developers will also make announcements in
this mailing list when they are contemplating some
controversial fix or update, giving the users a chance
to respond if they have any issues to raise concerning
the proposed change.</para>
<para>Join the relevant <application>svn</application>
list for the branch being tracked. For example, users
tracking the 9-STABLE branch should join the
&a.svn-src-stable-9.name; list. This list records the
commit log entry for each change as it is made, along
with any pertinent information on possible
side-effects.</para>
<para>To join these lists,
go to &a.mailman.lists.link;, click on the list to
subscribe to, and follow the instructions. In order to
track changes for the whole source tree, subscribe to
&a.svn-src-all.name;.</para>
</listitem>
<listitem>
<para>To install a new system running monthly
snapshots built from &os.stable;, refer to <ulink
url="&url.base;/snapshots/">Snapshots</ulink> for more
information. Alternatively, it is possible to install
the most recent &os.stable; release from the <link
linkend="mirrors">mirror sites</link> and follow the
instructions below to upgrade the system to the most
up-to-date &os.stable; source code.</para>
<para>Several methods are available to upgrade from a &os;
<link linkend="mirrors">mirror site</link> on a system
already running a previous release of &os;:</para>
<orderedlist>
<indexterm>
<primary>Subversion</primary>
</indexterm>
<indexterm>
<primary><command>cron</command></primary>
</indexterm>
<indexterm>
<primary>-STABLE</primary>
<secondary>syncing with
<application>Subversion</application></secondary>
</indexterm>
<listitem>
<para>Use <link linkend="svn">svn</link> to check out
the desired development or release branch. This is
the recommended method, providing access to &os;
development as it occurs. Branch names include
<literal>head</literal> for the current development
head, and branches identified in <ulink
url="&url.base;/releng/">the release engineering
page</ulink>, such as <literal>stable/9</literal>
or <literal>releng/9.0</literal>. URL prefixes for
<application>Subversion</application> checkout of
the base system are shown in <link
linkend="svn-mirrors">Subversion mirror
sites</link>.
Because of the size of the repository, it is
recommended that only desired subtrees be checked
out.</para>
</listitem>
<listitem>
<indexterm>
<primary>-STABLE</primary>
<secondary>syncing with CTM</secondary>
</indexterm>
<para>Consider using <application><link
linkend="ctm">CTM</link></application> if you do
not have a fast connection to the Internet.</para>
</listitem>
</orderedlist>
</listitem>
<listitem>
<indexterm>
<primary>-STABLE</primary>
<secondary>compiling</secondary>
</indexterm>
<para>Before compiling &os.stable;, read
<filename>/usr/src/Makefile</filename> carefully. <link
linkend="makeworld">Install a new kernel and rebuild
the world</link> the first time through as part of the
upgrading process. Read &a.stable; and
<filename>/usr/src/UPDATING</filename> to keep
up-to-date on other bootstrapping procedures that
sometimes become necessary on the road to the next
release.</para>
</listitem>
</orderedlist>
</sect3>
</sect2>
</sect1>
<sect1 id="synching">
<title>Synchronizing Source</title>
<para>There are various ways of using an Internet or email
connection to stay up-to-date with any given area, or all areas,
of the &os; project sources. The primary services are <link
linkend="svn">Subversion</link> and <link
linkend="ctm">CTM</link>.</para>
<warning>
<para>While it is possible to update only parts of the source
tree, the only supported update procedure is to update the
entire tree and recompile all the programs that run in user
space, such as those in <filename>/bin</filename> and
<filename>/sbin</filename>, and kernel sources. Updating only
part of the source tree, only the kernel, or only the userland
programs will often result in problems ranging from compile
errors to kernel panics or data corruption.</para>
</warning>
<indexterm>
<primary>Subversion</primary>
</indexterm>
<para><application>Subversion</application> uses the
<emphasis>pull</emphasis> model of updating sources. The user,
or a <command>cron</command> script, invokes the
<command>svn</command> program, and it brings files up-to-date.
<application>Subversion</application> is the preferred means of
updating local source trees. The updates are up-to-the-minute
and the user controls when they are downloaded. It is easy to
restrict updates to specific files or directories and the
requested updates are generated on the fly by the server.</para>
<indexterm>
<primary><application>CTM</application></primary>
</indexterm>
<para><application>CTM</application> does not interactively
compare the local sources with those on the master archive or
otherwise pull them across. Instead, a script which identifies
changes in files since its previous run is executed several
times a day on the master CTM machine. Any detected changes are
compressed, stamped with a sequence-number, and encoded for
transmission over email in printable ASCII only. Once received,
these <quote>CTM deltas</quote> can then be handed to the
&man.ctm.rmail.1; utility which will automatically decode,
verify, and apply the changes to the user's copy of the sources.
This process is more efficient than
<application>Subversion</application> and places less strain on
server resources since it is a <emphasis>push</emphasis>
rather than a <emphasis>pull</emphasis> model.</para>
<para>There are other trade-offs. If a user inadvertently
wipes out portions of the local archive,
<application>Subversion</application> will detect and rebuild
the damaged portions. <application>CTM</application> will not
do this, and if a user deletes some portion of the source tree
and does not have a backup, they will have to start from scratch
from the most recent CTM <quote>base delta</quote> and rebuild
it all with <application>CTM</application>.</para>
</sect1>
<sect1 id="makeworld">
<title>Rebuilding <quote>world</quote></title>
<indexterm>
<primary>Rebuilding <quote>world</quote></primary>
</indexterm>
<para>Once the local source tree is synchronized against a
particular version of &os; such as &os.stable; or &os.current;,
the source tree can be used to rebuild the system.</para>
<warning>
<title>Make a Backup</title>
<para>It cannot be stressed enough how important it is to make a
backup of the system <emphasis>before</emphasis> rebuilding
the system. While rebuilding the world is an easy task, there
will inevitably be times when mistakes in the source tree
render the system unbootable.</para>
<para>Create and verify a backup and have a bootable
installation media at hand. You will probably never have
to use it, but it is better to be safe than sorry!</para>
</warning>
<warning>
<title>Subscribe to the Right Mailing List</title>
<indexterm><primary>mailing list</primary></indexterm>
<para>The &os.stable; and &os.current; branches are, by their
nature, <emphasis>in development</emphasis>. People that
contribute to &os; are human, and mistakes occasionally
happen.</para>
<para>Sometimes these mistakes can be quite harmless, just
causing the system to print a new diagnostic warning. Or the
change may be catastrophic, and render the system unbootable
or destroy file systems.</para>
<para>When problems occur, a <quote>heads up</quote> is
posted to the appropriate mailing list, explaining the nature
of the problem and which systems it affects. An <quote>all
clear</quote> announcement is posted when the problem has
been solved.</para>
<para>Users who track &os.stable; or &os.current; and do
not read &a.stable; or &a.current; respectively, are asking
for trouble.</para>
</warning>
<warning>
<title>Do Not Use <command>make world</command></title>
<para>Some older documentation recommends using
<command>make world</command>. However, that command skips
some important steps and should only be used by experts. For
almost all circumstances <command>make world</command> is the
wrong thing to do, and the procedure described here should be
used instead.</para>
</warning>
<sect2 id="canonical-build">
<title>The Canonical Way to Update Your System</title>
<para>Before updating the system, read
<filename>/usr/src/UPDATING</filename> for any pre-buildworld
steps necessary for that version of the sources. Then, use
the procedure outlined here.</para>
<para>These upgrade steps assume an upgrade from an older &os;
version, consisting of an old compiler, old kernel,
old world, and old configuration files.
<quote>World</quote> includes the core system binaries,
libraries, and programming files. The compiler is part of
<quote>world</quote>, but has a few special concerns.</para>
<para>These steps also assume that the sources to a newer
version have already been obtained. If the sources are not
up-to-date, refer to <xref linkend="synching"/> for detailed
help about synchronizing to a newer version.</para>
<para>Updating the system from source is a more subtle process
than it might initially seem to be, and the &os; developers have
found it necessary over the years to change the recommended
approach fairly dramatically as new kinds of unavoidable
dependencies come to light. The rest of this section
describes the rationale behind the currently recommended
upgrade sequence.</para>
<para>Any successful update sequence must deal with the
following issues:</para>
<itemizedlist>
<listitem>
<para>The old compiler might have a bug and not be able to
compile the new kernel. So, the new kernel should be
built with the new compiler, meaning that the new compiler
must be built before the new kernel is built. This does
not necessarily mean that the new compiler must be
<emphasis>installed</emphasis> before building the new
kernel.</para>
</listitem>
<listitem>
<para>The new world might rely on new kernel features. So,
the new kernel must be installed before the new world is
installed.</para>
</listitem>
</itemizedlist>
<para>These first two issues are the basis for the
core <maketarget>buildworld</maketarget>,
<maketarget>buildkernel</maketarget>,
<maketarget>installkernel</maketarget>,
<maketarget>installworld</maketarget> sequence described in
the following paragraphs. Other reasons for using these
steps are listed below:</para>
<itemizedlist>
<listitem>
<para>The old world might not run correctly on the new
kernel, so the new world must be installed immediately
upon installing the new kernel.</para>
</listitem>
<listitem>
<para>Some configuration changes must be made before the new
world is installed, but others might break the old world.
Hence, two different configuration upgrade steps are
generally needed.</para>
</listitem>
<listitem>
<para>For the most part, the update process only replaces or
adds files and existing old files are not deleted. In a
few cases, this can cause problems. As a result, the
update procedure will sometimes specify certain files that
should be manually deleted at certain steps. This may or
may not be automated in the future.</para>
</listitem>
</itemizedlist>
<para>These concerns have led to the following recommended
sequence. Note that the detailed sequence for particular
updates may require additional steps, but this core process
should remain unchanged for some time:</para>
<orderedlist>
<listitem>
<para><command>make
<maketarget>buildworld</maketarget></command></para>
<para>This first compiles the new compiler and a few related
tools, then uses the new compiler to compile the rest of
the new world. The result ends up in
<filename class="directory">/usr/obj</filename>.</para>
</listitem>
<listitem>
<para><command>make
<maketarget>buildkernel</maketarget></command></para>
<para>This uses the <emphasis>new</emphasis> compiler
residing in <filename
class="directory">/usr/obj</filename> in order to
protect against compiler-kernel mismatches.</para>
</listitem>
<listitem>
<para><command>make
<maketarget>installkernel</maketarget></command></para>
<para>Place the new kernel and kernel modules onto the disk,
making it possible to boot with the newly updated
kernel.</para>
</listitem>
<listitem>
<para>Reboot into single user mode.</para>
<para>Single user mode minimizes problems from updating
software that is already running. It also minimizes any
problems from running the old world on a new
kernel.</para>
</listitem>
<listitem>
<para><command>mergemaster
<option>-p</option></command></para>
<para>This does some initial configuration file updates in
preparation for the new world. For instance, it may add
new user groups to the system, or new user names to the
password database. This is often necessary when new
groups or special system-user accounts have been added
since the last update, so that the
<maketarget>installworld</maketarget> step will be able to
use the newly installed system user or system group names
without problems.</para>
</listitem>
<listitem>
<para><command>make
<maketarget>installworld</maketarget></command></para>
<para>Copies the world
from <filename class="directory">/usr/obj</filename>. The
new kernel and new world are now installed on disk.</para>
</listitem>
<listitem>
<para><command>mergemaster</command></para>
<para>Repeated to update the remaining configuration files,
now that the new world is on disk.</para>
</listitem>
<listitem>
<para><command>make
<maketarget>delete-old</maketarget></command></para>
<para>This target deletes old (obsolete) files. This is important
because sometimes they cause problems if left on the disk, for
example the presence of the old <filename>utmp.h</filename>
causes problems in some ports when the new
<filename>utmpx.h</filename> is installed.</para>
</listitem>
<listitem>
<para>Reboot.</para>
<para>A full machine reboot is needed now to load the new
kernel and new world with new configuration files.</para>
</listitem>
<listitem>
<para><command>make <maketarget>delete-old-libs</maketarget></command></para>
<para>Remove any obsolete libraries to avoid conflicts with newer
ones. Make sure that all ports have been rebuilt
before old libraries are removed.</para>
</listitem>
</orderedlist>
<para>Upgrades from one release of the same &os; branch to a
more recent release of the same branch, such as from 9.0 to
9.1, may not need this procedure since it is less likely to
run into serious mismatches between compiler, kernel,
userland, and configuration files. The approach of
<command>make <maketarget>world</maketarget></command>
followed by building and installing a new kernel might work
well enough for minor updates.</para>
<para>When upgrading across major releases, people who do not
follow this procedure should expect some problems.</para>
<para>It is also worth noting that many upgrades may require
specific additional steps such as renaming or deleting
specific files prior to installworld. Read
<filename>/usr/src/UPDATING</filename> carefully, especially
at the end, where the currently recommended upgrade sequence
is explicitly spelled out.</para>
<para>This procedure has evolved over time as the developers
have found it impossible to completely prevent certain kinds
of mismatch problems. Hopefully, the current procedure will
remain stable for a long time.</para>
<para>To summarize, the currently recommended way of upgrading
&os; from sources is:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make buildworld</userinput>
&prompt.root; <userinput>make buildkernel</userinput>
&prompt.root; <userinput>make installkernel</userinput>
&prompt.root; <userinput>shutdown -r now</userinput></screen>
<note>
<para>There are a few rare cases when an extra run of
<command>mergemaster -p</command> is needed before the
<maketarget>buildworld</maketarget> step. These are
described in <filename>UPDATING</filename>. In general,
though, this step can safely be omitted when not updating
across one or more major &os; versions.</para>
</note>
<para>After <maketarget>installkernel</maketarget> finishes
successfully, boot into single user mode using <command>boot
-s</command> from the loader prompt. Then run:</para>
<screen>&prompt.root; <userinput>mount -u /</userinput>
&prompt.root; <userinput>mount -a -t ufs</userinput>
&prompt.root; <userinput>adjkerntz -i</userinput>
&prompt.root; <userinput>mergemaster -p</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld</userinput>
&prompt.root; <userinput>mergemaster</userinput>
&prompt.root; <userinput>make delete-old</userinput>
&prompt.root; <userinput>reboot</userinput>
&prompt.root; <userinput>make delete-old-libs</userinput></screen>
<warning>
<title>Read Further Explanations</title>
<para>The following sections clearly describe each step,
especially when using a custom kernel configuration.</para>
</warning>
</sect2>
<sect2 id="src-updating">
<title>Read <filename>/usr/src/UPDATING</filename></title>
<para>Before updating, read
<filename>/usr/src/UPDATING</filename>. This file contains
important information about potential problems and may specify
the order to run certain commands. If
<filename>UPDATING</filename> contradicts the procedure in
this section, <filename>UPDATING</filename> takes
precedence.</para>
<important>
<para>Reading <filename>UPDATING</filename> is not an
acceptable substitute for subscribing to the correct mailing
list. The two requirements are complementary, not
exclusive.</para>
</important>
</sect2>
<sect2 id="make-conf">
<title>Check <filename>/etc/make.conf</filename></title>
<indexterm>
<primary><filename>make.conf</filename></primary>
</indexterm>
<para>Available &man.make.1; options are shown in
&man.make.conf.5; and
<filename>/usr/share/examples/etc/make.conf</filename>. These
settings can be added to <filename>/etc/make.conf</filename>
to control the way &man.make.1; runs and how it builds
programs. Changes to some settings can have far-reaching and
potentially surprising effects. Read the comments in both
locations and keep in mind that the defaults have been chosen
for a combination of performance and safety.</para>
<para>Options set in <filename>/etc/make.conf</filename> take
effect every time &man.make.1; is used, including compiling
applications from the Ports Collection or user-written C
programs, or building the &os; operating system.</para>
</sect2>
<sect2 id="src-conf">
<title>Check <filename>/etc/src.conf</filename></title>
<indexterm>
<primary><filename>src.conf</filename></primary>
</indexterm>
<para><filename>/etc/src.conf</filename> controls the building
of the operating system from source code. Unlike
<filename>/etc/make.conf</filename>, the contents of
<filename>/etc/src.conf</filename> only take effect when the
&os; operating system itself is being built. Descriptions of
the many options available for this file are shown in
&man.src.conf.5;. Be cautious about disabling seemingly
unneeded kernel modules and build options. Sometimes there
are unexpected or subtle interactions.</para>
</sect2>
<sect2 id="updating-etc">
<title>Update the Files in <filename>/etc</filename></title>
<para><filename class="directory">/etc</filename> contains a
large part of the system's configuration information, as well
as scripts that are run at system startup. Some of these
scripts change between &os; versions.</para>
<para>Some of the configuration files are used in the day to
day running of the system, such as
<filename>/etc/group</filename>.</para>
<para>There have been occasions when the installation part of
<command>make installworld</command> expected certain
usernames or groups to exist. When performing an upgrade, it
is likely that these users or groups do not yet exist. In
some cases <command>make buildworld</command> will check to
see if these users or groups exist.</para>
<para>The solution is to run &man.mergemaster.8; in
pre-buildworld mode with <option>-p</option>. This compares
only those files that are essential for the success of
<maketarget>buildworld</maketarget> or
<maketarget>installworld</maketarget>.</para>
<tip>
<para>To check which files are owned by the group being
renamed or deleted:</para>
<screen>&prompt.root; <userinput>find / -group <replaceable>GID</replaceable> -print</userinput></screen>
<para>This command will show all files owned by group
<replaceable>GID</replaceable>, which can be either a group
name or a numeric group ID.</para>
</tip>
</sect2>
<sect2 id="makeworld-singleuser">
<title>Drop to Single User Mode</title>
<indexterm><primary>single-user mode</primary></indexterm>
<para>Consider compiling the system in single user mode.
Reinstalling the system touches a lot of important system
files, all the standard system binaries, libraries, and
include files. Changing these on a running system,
particularly one with active users, is asking for
trouble.</para>
<indexterm><primary>multi-user mode</primary></indexterm>
<para>Another method is to compile the system in multi-user
mode, and then drop into single user mode for the
installation. With this method, hold off on the following
steps until the build has completed. Drop to single user mode
in order to run <maketarget>installkernel</maketarget> or
<maketarget>installworld</maketarget>.</para>
<para>To enter single user mode from a running system:</para>
<screen>&prompt.root; <userinput>shutdown now</userinput></screen>
<para>Alternatively, reboot the system, and at the boot prompt,
select the <quote>single user</quote> option. Once at the
single user mode shell prompt, run:</para>
<screen>&prompt.root; <userinput>fsck -p</userinput>
&prompt.root; <userinput>mount -u /</userinput>
&prompt.root; <userinput>mount -a -t ufs</userinput>
&prompt.root; <userinput>swapon -a</userinput></screen>
<para>This checks the file systems, remounts
<filename>/</filename> read/write, mounts all the other UFS
file systems referenced in <filename>/etc/fstab</filename>,
and turns swapping on.</para>
<note>
<para>If the CMOS clock is set to local time and not to GMT
(this is true if the output of &man.date.1; does not show
the correct time and zone), run the following
command:</para>
<screen>&prompt.root; <userinput>adjkerntz -i</userinput></screen>
<para>This ensures that the local time-zone settings get set
up correctly.</para>
</note>
</sect2>
<sect2 id="cleaning-usr-obj">
<title>Remove <filename>/usr/obj</filename></title>
<para>As parts of the system are rebuilt, they are, by default,
placed in subdirectories of <filename>/usr/obj</filename>.
The directories shadow those under
<filename>/usr/src</filename>.</para>
<para>To speed up the <command>make buildworld</command>
process, and possibly save some dependency headaches,
remove this directory if it already exists.</para>
<para>Some files below <filename>/usr/obj</filename> may have
the immutable flag set which must be removed first using
&man.chflags.1;.</para>
<screen>&prompt.root; <userinput>cd /usr/obj</userinput>
&prompt.root; <userinput>chflags -R noschg *</userinput>
&prompt.root; <userinput>rm -rf *</userinput></screen>
</sect2>
<sect2 id="updating-upgrading-compilebase">
<title>Recompile the Base System</title>
<sect3>
<title>Saving the Output</title>
<para>It is a good idea to save the output from running
&man.make.1; to a file. If something goes wrong, a copy of
the error message can be posted to one of the &os; mailing
lists.</para>
<para>The easiest way to do this is to use &man.script.1;
with a parameter that specifies the name of the file to save
all output to. Run this command immediately before
rebuilding the world, and then type
<userinput>exit</userinput> when the process has
finished:</para>
<screen>&prompt.root; <userinput>script /var/tmp/mw.out</userinput>
Script started, output file is /var/tmp/mw.out
&prompt.root; <userinput>make TARGET</userinput>
<emphasis>… compile, compile, compile …</emphasis>
&prompt.root; <userinput>exit</userinput>
Script done, …</screen>
<para><emphasis>Do not</emphasis> save the output in <filename
class="directory">/tmp</filename> as this directory may be
cleared at next reboot. A better place to save the file is
<filename class="directory">/var/tmp</filename> or in
<username>root</username>'s home directory.</para>
</sect3>
<sect3 id="make-buildworld">
<title>Compile the Base System</title>
<para>While in <filename class="directory">/usr/src</filename>
type:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput></screen>
<indexterm><primary><command>make</command></primary></indexterm>
<para>To rebuild the world, use &man.make.1;. This command
reads instructions from the <filename>Makefile</filename>,
which describes how the programs that comprise &os; should
be built and the order in which they should be built.</para>
<para>The general format of the command is as follows:</para>
<screen>&prompt.root; <userinput>make -<replaceable>x</replaceable> -D<replaceable>VARIABLE</replaceable> <replaceable>target</replaceable></userinput></screen>
<para>In this example,
<option>-<replaceable>x</replaceable></option> is an option
passed to &man.make.1;. Refer to &man.make.1; for an
examples of available options.</para>
<para><option>-D<replaceable>VARIABLE</replaceable></option>
passes a variable to the <filename>Makefile</filename>. The
behavior of the <filename>Makefile</filename> is controlled
by these variables. These are the same variables as are set
in <filename>/etc/make.conf</filename>, and this provides
another way of setting them. For example:</para>
<screen>&prompt.root; <userinput>make -DNO_PROFILE <replaceable>target</replaceable></userinput></screen>
<para>is another way of specifying that profiled libraries
should not be built, and corresponds with the</para>
<programlisting>NO_PROFILE= true # Avoid compiling profiled libraries</programlisting>
<para>line in <filename>/etc/make.conf</filename>.</para>
<para><replaceable>target</replaceable> tells &man.make.1;
what to do. Each <filename>Makefile</filename> defines a
number of different <quote>targets</quote>, and the choice
of target determines what happens.</para>
<para>Some targets listed in the
<filename>Makefile</filename> are used by the build process
to break out the steps necessary to rebuild the system into
a number of sub-steps.</para>
<para>Most of the time, no parameters need to be passed to
&man.make.1; and the command looks like this:</para>
<screen>&prompt.root; <userinput>make <replaceable>target</replaceable></userinput></screen>
<para>Where <replaceable>target</replaceable> is one of many
build options. The first target should always be
<makevar>buildworld</makevar>.</para>
<para>As the names imply, <maketarget>buildworld</maketarget>
builds a complete new tree under
<filename>/usr/obj</filename> and
<maketarget>installworld</maketarget> installs this tree on
the current machine.</para>
<para>Having separate options is useful for two reasons.
First, it allows for a <quote>self hosted</quote> build that
does not affect any components of a running system. Because
of this, <maketarget>buildworld</maketarget> can be run on a
machine running in multi-user mode with no fear of
ill-effects. It is still recommended that
<maketarget>installworld</maketarget> be run in part in
single user mode, though.</para>
<para>Secondly, it allows NFS mounts to be used to upgrade
multiple machines on a network. If order to upgrade three
machines, <hostid>A</hostid>, <hostid>B</hostid> and
<hostid>C</hostid>, run <command>make buildworld</command>
and <command>make installworld</command> on
<hostid>A</hostid>. <hostid>B</hostid> and
<hostid>C</hostid> should then NFS mount
<filename>/usr/src</filename> and
<filename>/usr/obj</filename> from <hostid>A</hostid>, and
run <command>make installworld</command> to install the
results of the build on <hostid>B</hostid> and
<hostid>C</hostid>.</para>
<para>Although the <maketarget>world</maketarget> target still
exists, users are strongly encouraged not to use it.</para>
<para>Instead, run:</para>
<screen>&prompt.root; <userinput>make buildworld</userinput></screen>
<para>It is possible to specify <option>-j</option> which
will cause <command>make</command> to spawn several
simultaneous processes. This is most useful on multi-CPU
machines. However, since much of the compiling process is
I/O bound rather than CPU bound, it is also useful on single
CPU machines.</para>
<para>On a typical single-CPU machine, run:</para>
<screen>&prompt.root; <userinput>make -j4 buildworld</userinput></screen>
<para>&man.make.1; will then have up to 4 processes running at
any one time. Empirical evidence posted to the mailing
lists shows this generally gives the best performance
benefit.</para>
<para>On a multi-CPU machine using an SMP configured kernel,
try values between 6 and 10 and see how they speed things
up.</para>
</sect3>
<sect3>
<title>Timings</title>
<indexterm>
<primary>rebuilding <quote>world</quote></primary>
<secondary>timings</secondary>
</indexterm>
<para>Many factors influence the build time, but fairly recent
machines may only take a one or two hours to build the
&os.stable; tree, with no tricks or shortcuts used during
the process. A &os.current; tree will take somewhat
longer.</para>
</sect3>
</sect2>
<sect2 id="new-kernel">
<title>Compile and Install a New Kernel</title>
<indexterm>
<primary>kernel</primary>
<secondary>compiling</secondary>
</indexterm>
<para>To take full advantage of the new system, recompile the
kernel. This is practically a necessity, as certain memory
structures may have changed, and programs like &man.ps.1; and
&man.top.1; will fail to work until the kernel and source code
versions are the same.</para>
<para>The simplest, safest way to do this is to build and
install a kernel based on <filename>GENERIC</filename>. While
<filename>GENERIC</filename> may not have all the necessary
devices for the system, it should contain everything necessary
to boot the system back to single user mode. This is a good
test that the new system works properly. After booting from
<filename>GENERIC</filename> and verifying that the system
works, a new kernel can be built based on a custom kernel
configuration file.</para>
<para>On &os; it is important to
<link linkend="make-buildworld">build world</link> before
building a new kernel.</para>
<note>
<para>To build a custom kernel with an existing customized
configuration file, use
<literal>KERNCONF=<replaceable>MYKERNEL</replaceable></literal>:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make buildkernel KERNCONF=<replaceable>MYKERNEL</replaceable></userinput>
&prompt.root; <userinput>make installkernel KERNCONF=<replaceable>MYKERNEL</replaceable></userinput></screen>
</note>
<para>If <varname>kern.securelevel</varname> has been raised
above 1 <emphasis>and</emphasis> <literal>noschg</literal> or
similar flags have been set on the kernel binary, drop into
single user mode to use
<maketarget>installkernel</maketarget>. Otherwise, both these
commands can be run from multi user mode without problems.
See &man.init.8; for details about
<varname>kern.securelevel</varname> and &man.chflags.1; for
details about the various file flags.</para>
</sect2>
<sect2 id="new-kernel-singleuser">
<title>Reboot into Single User Mode</title>
<indexterm><primary>single-user mode</primary></indexterm>
<para>Reboot into single user mode to test that the new kernel
works using the instructions in <xref
linkend="makeworld-singleuser"/>.</para>
</sect2>
<sect2 id="make-installworld">
<title>Install the New System Binaries</title>
<para>Next, use <maketarget>installworld</maketarget> to install
the new system binaries:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld</userinput></screen>
<note>
<para>If variables were specified to
<command>make buildworld</command>, specify the same
variables to <command>make installworld</command>. However,
<option>-j</option> must never be used with
<maketarget>installworld</maketarget>.</para>
<para>For example, if you ran:</para>
<screen>&prompt.root; <userinput>make -DNO_PROFILE buildworld</userinput></screen>
<para>install the results with:</para>
<screen>&prompt.root; <userinput>make -DNO_PROFILE installworld</userinput></screen>
<para>otherwise, the command will try to install profiled
libraries that were not built during the
<command>make buildworld</command> phase.</para>
</note>
</sect2>
<sect2 id="post-installworld-updates">
<title>Update Files Not Updated by
<command>make installworld</command></title>
<para>Remaking the world will not update certain directories,
such as <filename class="directory">/etc</filename>,
<filename class="directory">/var</filename> and
<filename class="directory">/usr</filename>, with
new or changed configuration files.</para>
<para>The simplest way to update the files in these directories
is to use &man.mergemaster.8;. Be sure to first make a backup
of <filename>/etc</filename> in case anything goes
wrong.</para>
<sect3 id="mergemaster">
<sect3info>
<authorgroup>
<author>
<firstname>Tom</firstname>
<surname>Rhodes</surname>
<contrib>Contributed by </contrib>
</author>
</authorgroup>
</sect3info>
<title><command>mergemaster</command></title>
<indexterm>
<primary>
<command>mergemaster</command>
</primary>
</indexterm>
<para>&man.mergemaster.8; is a Bourne script to aid in
determining the differences between the configuration files
in <filename class="directory">/etc</filename>, and the
configuration files in the source tree
<filename class="directory">/usr/src/etc</filename>. This
is the recommended solution for keeping the system
configuration files up to date with those located in the
source tree.</para>
<para>To begin, type <command>mergemaster</command> and it
will build a temporary root environment, from
<filename>/</filename> down, and populate it with various
system configuration files. Those files are then compared
to the ones currently installed in the system. Files that
differ will be shown in &man.diff.1; format, with the
<option>+</option> sign representing added or modified
lines, and <option>-</option> representing lines that will
be either removed completely, or replaced with a new file.
Refer to &man.diff.1; for more information about the
&man.diff.1; syntax and how file differences are
shown.</para>
<para>&man.mergemaster.8; will then display each file that
differs, and present the options of either deleting the new
file, referred to as the temporary file, installing the
temporary file in its unmodified state, merging the
temporary file with the currently installed file, or viewing
the &man.diff.1; results again.</para>
<para>Choosing to delete the temporary file will tell
&man.mergemaster.8; to keep the current file unchanged and
to delete the new version. This option is not recommended,
unless there is no reason to change the current file. To
get help at any time, type <keycap>?</keycap> at the
&man.mergemaster.8; prompt. If the user chooses to skip a
file, it will be presented again after all other files have
been dealt with.</para>
<para>Choosing to install the unmodified temporary file will
replace the current file with the new one. For most
unmodified files, this is the best option.</para>
<para>Choosing to merge the file will present a text editor,
and the contents of both files. The files can be merged
by reviewing both files side by side on the screen, and
choosing parts from both to create a finished product. When
the files are compared side by side, <keycap>l</keycap>
selects the left contents and <keycap>r</keycap> selects
contents from the right. The final output will be a file
consisting of both parts, which can then be installed. This
option is customarily used for files where settings have
been modified by the user.</para>
<para>Choosing to view the &man.diff.1; results again will
display the file differences just like &man.mergemaster.8;
did before prompting an option.</para>
<para>After &man.mergemaster.8; is done with the system files,
it will prompt for other options. &man.mergemaster.8; may
prompt to rebuild the password file and will finish up with
an option to remove left-over temporary files.</para>
</sect3>
<sect3>
<title>Manual Update</title>
<para>To perform the update manually instead, do not just copy
over the files from
<filename class="directory">/usr/src/etc</filename> to
<filename class="directory">/etc</filename> and expect it to
work. Some files must be <quote>installed</quote> first as
<filename class="directory">/usr/src/etc</filename>
<emphasis>is not</emphasis> a copy of what
<filename class="directory">/etc</filename> should look
like. In addition, some files that should be in
<filename>/etc</filename> are not in
<filename>/usr/src/etc</filename>.</para>
<para>If you are using &man.mergemaster.8; (as recommended),
you can skip forward to the
<link linkend="updating-upgrading-rebooting">next
section</link>.</para>
<para>The simplest way to merge files by hand is to install
the files into a new directory, and then work through them
looking for differences.</para>
<warning>
<title>Backup Your Existing
<filename>/etc</filename></title>
<para>It is recommended to first copy the existing
<filename class="directory">/etc</filename> somewhere
safe, like so:</para>
<screen>&prompt.root; <userinput>cp -Rp /etc /etc.old</userinput></screen>
<para>where <option>-R</option> does a recursive copy and
<option>-p</option> preserves times and the ownerships on
files.</para>
</warning>
<para>Build a temporary set of directories into which the new
<filename class="directory">/etc</filename> and other files
can be installed:</para>
<screen>&prompt.root; <userinput>mkdir /var/tmp/root</userinput>
&prompt.root; <userinput>cd /usr/src/etc</userinput>
&prompt.root; <userinput>make DESTDIR=/var/tmp/root distrib-dirs distribution</userinput></screen>
<para>This will build the necessary directory structure and
install the files. A lot of the subdirectories that have
been created under <filename
class="directory">/var/tmp/root</filename> are empty and
should be deleted. The simplest way to do this is
to:</para>
<screen>&prompt.root; <userinput>cd /var/tmp/root</userinput>
&prompt.root; <userinput>find -d . -type d | xargs rmdir 2>/dev/null</userinput></screen>
<para>This will remove all empty directories while redirecting
standard error to <filename>/dev/null</filename> to prevent
the warnings about the directories that are not
empty.</para>
<para><filename class="directory">/var/tmp/root</filename> now
contains all the files that should be placed in appropriate
locations below <filename class="directory">/</filename>.
Go through each of these files, determining how they differ
from the system's existing files.</para>
<para>Some of the files installed into <filename
class="directory">/var/tmp/root</filename> have a
leading <quote>.</quote>. Make sure to use <command>ls
-a</command> in order to catch them.</para>
<para>The simplest way to compare files is to use
&man.diff.1;:</para>
<screen>&prompt.root; <userinput>diff /etc/shells /var/tmp/root/etc/shells</userinput></screen>
<para>This command will show the differences between the
existing <filename>/etc/shells</filename> and the new
<filename>/var/tmp/root/etc/shells</filename>. Review the
differences to decide whether to merge in custom changes
or to replace the existing file with the new one.</para>
<tip>
<title>Name the New Root Directory
(<filename class="directory">/var/tmp/root</filename>)
with a Time Stamp, so You Can Easily Compare Differences
Between Versions</title>
<para>Frequently rebuilding world entails frequently
updating <filename class="directory">/etc</filename>
as well, which can be a bit of a chore.</para>
<para>To speed up this process, use the following
procedure to keep a copy of the last set of changed files
that were merged into <filename
class="directory">/etc</filename>.</para>
<procedure>
<step>
<para>Make the world as normal. When updating
<filename class="directory">/etc</filename> and the
other directories, give the target directory a name
based on the current date:</para>
<screen>&prompt.root; <userinput>mkdir /var/tmp/root-20130214</userinput>
&prompt.root; <userinput>cd /usr/src/etc</userinput>
&prompt.root; <userinput>make DESTDIR=/var/tmp/root-20130214 \
distrib-dirs distribution</userinput></screen>
</step>
<step>
<para>Merge in the changes from this directory as
outlined above. <emphasis>Do not</emphasis> remove
the <filename>/var/tmp/root-20130214</filename>
directory when you have finished.</para>
</step>
<step>
<para>After downloading the latest version of the
source and remaking it, follow step 1. Create a new
directory, which reflects the new date. This example
uses
<filename>/var/tmp/root-20130221</filename>.</para>
</step>
<step>
<para>Use &man.diff.1; to see the differences that have
been made in the intervening week by creating a
recursive diff between the two directories:</para>
<screen>&prompt.root; <userinput>cd /var/tmp</userinput>
&prompt.root; <userinput>diff -r root-20130214 root-20130221</userinput></screen>
<para>Typically, this will be a much smaller set of
differences than those between <filename
class="directory">/var/tmp/root-20130221/etc</filename>
and <filename class="directory">/etc</filename>.
Because the set of differences is smaller, it is
easier to migrate those changes across into
<filename class="directory">/etc</filename>.</para>
</step>
<step>
<para>When finished, remove the older of the two
<filename class="directory">/var/tmp/root-*</filename>
directories:</para>
<screen>&prompt.root; <userinput>rm -rf /var/tmp/root-20130214</userinput></screen>
</step>
<step>
<para>Repeat this process whenever merging
in changes to <filename
class="directory">/etc</filename>.</para>
</step>
</procedure>
<para>Use &man.date.1; to automate the generation of the
directory names:</para>
<screen>&prompt.root; <userinput>mkdir /var/tmp/root-`date "+%Y%m%d"`</userinput></screen>
</tip>
</sect3>
</sect2>
<sect2 id="make-delete-old">
<sect2info>
<authorgroup>
<author>
<firstname>Anton</firstname>
<surname>Shterenlikht</surname>
<contrib>Based on notes provided by </contrib>
</author>
</authorgroup>
</sect2info>
<title>Deleting Obsolete Files and Directories</title>
<indexterm>
<primary>Deleting obsolete files and directories</primary>
</indexterm>
<para>As a part of the &os; development lifecycle, files and their
contents occasionally become obsolete. This may be because
functionality is implemented elsewhere, the version number of
the library has changed, or it was removed from the system
entirely. This includes old files, libraries, and directories,
which should be removed when updating the system. The benefit
is that the system is not cluttered with old files which take up
unnecessary space on the storage and backup media.
Additionally, if the old library has a security or stability
issue, the system should be updated to the newer library to keep
it safe and to prevent crashes caused by the old library.
Files, directories, and libraries which are considered obsolete
are listed in <filename>/usr/src/ObsoleteFiles.inc</filename>.
The following instructions should be used to remove obsolete
files during the system upgrade process.</para>
<para>After the <command>make
<maketarget>installworld</maketarget></command>
and the subsequent <command>mergemaster</command> have finished
successfully, check for obsolete files and libraries as
follows:</para>
<screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make check-old</userinput></screen>
<para>If any obsolete files are found, they can be deleted using
the following command:</para>
<screen>&prompt.root; <userinput>make delete-old</userinput></screen>
<tip>
<para>Refer to <filename>/usr/src/Makefile</filename>
for more targets of interest.</para>
</tip>
<para>A prompt is displayed before deleting each obsolete file.
To skip the prompt and let the system remove these files
automatically, use
<makevar>BATCH_DELETE_OLD_FILES</makevar>:</para>
<screen>&prompt.root; <userinput>make -DBATCH_DELETE_OLD_FILES delete-old</userinput></screen>
<para>The same goal can be achieved by piping these commands
through <command>yes</command>:</para>
<screen>&prompt.root; <userinput>yes|make delete-old</userinput></screen>
</sect2>
<sect2 id="updating-upgrading-rebooting">
<title>Rebooting</title>
<para>Verify that everything appears to be in the right place,
then reboot the system using &man.shutdown.8;:</para>
<screen>&prompt.root; <userinput>shutdown -r now</userinput></screen>
</sect2>
<sect2 id="make-delete-old-libs">
<title>Deleting obsolete libraries</title>
<warning>
<title>Warning</title>
<para>Deleting obsolete files will break applications that
still depend on those obsolete files. This is especially true
for old libraries. In most cases, the programs, ports, or
libraries that used the old library need to be recompiled
before <command>make
<maketarget>delete-old-libs</maketarget></command> is
executed.</para>
</warning>
<para>Utilities for checking shared library dependencies are
available from the Ports Collection in
<filename role="package">sysutils/libchk</filename> or <filename
role="package">sysutils/bsdadminscripts</filename>.</para>
<para>Obsolete shared libraries can conflict with newer libraries,
causing messages like these:</para>
<screen>/usr/bin/ld: warning: libz.so.4, needed by /usr/local/lib/libtiff.so, may conflict with libz.so.5
/usr/bin/ld: warning: librpcsvc.so.4, needed by /usr/local/lib/libXext.so, may conflict with librpcsvc.so.5</screen>
<para>To solve these problems, determine which port installed the
library:</para>
<screen>&prompt.root; <userinput>pkg_info -W /usr/local/lib/libtiff.so</userinput>
/usr/local/lib/libtiff.so was installed by package tiff-3.9.4
&prompt.root; <userinput>pkg_info -W /usr/local/lib/libXext.so</userinput>
/usr/local/lib/libXext.so was installed by package libXext-1.1.1,1</screen>
<para>Then deinstall, rebuild and reinstall the port. <filename
role="package">ports-mgmt/portmaster</filename> can be used to
automate this process. After all ports are rebuilt and no
longer use the old libraries, delete the old libraries using the
following command:</para>
<screen>&prompt.root; <userinput>make delete-old-libs</userinput></screen>
<para>You should now have successfully upgraded the &os;
system. Congratulations.</para>
<para>If things went slightly wrong, it is easy to rebuild a
particular piece of the system. For example, if
<filename>/etc/magic</filename> was accidentally deleted as
part of the upgrade or merge of <filename
class="directory">/etc</filename>, &man.file.1; will stop
working. To fix this, run:</para>
<screen>&prompt.root; <userinput>cd /usr/src/usr.bin/file</userinput>
&prompt.root; <userinput>make all install</userinput></screen>
</sect2>
<sect2 id="updating-questions">
<title>Questions</title>
<qandaset>
<qandaentry>
<question>
<para>Do I need to re-make the world for every
change?</para>
</question>
<answer>
<para>There is no easy answer, as it depends on the nature
of the change. For example, if running
<application>svn</application> only shows the following
files as being updated:</para>
<screen><filename>src/games/cribbage/instr.c</filename>
<filename>src/games/sail/pl_main.c</filename>
<filename>src/release/sysinstall/config.c</filename>
<filename>src/release/sysinstall/media.c</filename>
<filename>src/share/mk/bsd.port.mk</filename></screen>
<para>it probably is not worth rebuilding the entire
world. Instead, go into the appropriate sub-directories
and run <command>make all install</command>. But if
something major changed, such as
<filename>src/lib/libc/stdlib</filename>, either
re-make world, or at least those parts of it that are
statically linked.</para>
<para>At the end of the day, it is your call. Some users
re-make the world every fortnight and let changes
accumulate over that fortnight. Others only re-make
those things that have changed and are careful to spot
all the dependencies.</para>
<para>It all depends on how often a user wants to upgrade
and whether they are tracking &os.stable; or
&os.current;.</para>
</answer>
</qandaentry>
<qandaentry>
<question>
<para>My compile failed with lots of signal 11 (or other
signal number) errors. What happened?</para>
</question>
<answer>
<indexterm><primary>signal 11</primary></indexterm>
<para>This normally indicates hardware problems.
(Re)making world is an effective way to stress test
hardware, and will frequently throw up memory
problems which normally manifest themselves as the
compiler mysteriously aborts.</para>
<para>A sure indicator of this occurs when
<application>make</application> is restarted and it
dies at a different point in the process.</para>
<para>To resolve this error, start swapping around the
components in the machine to determine which one is
failing.</para>
</answer>
</qandaentry>
<qandaentry>
<question>
<para>Can <filename class="directory">/usr/obj</filename>
be removed when finished?</para>
</question>
<answer>
<para>The short answer is yes.</para>
<para><filename class="directory">/usr/obj</filename>
contains all the object files that were produced during
the compilation phase. Normally, one of the first steps
in the <command>make buildworld</command> process is to
remove this directory and start afresh. Keeping
<filename class="directory">/usr/obj</filename> around
when finished makes little sense, and its removal frees
up a approximately 2 GB of disk space.</para>
<para>Advances users can instruct
<command>make buildworld</command> to skip this step.
This speeds up subsequent builds, since most of the
sources will not need to be recompiled. The flip side
is that subtle dependency problems can creep in, causing
the build to fail in odd ways. This frequently
generates noise on the &os; mailing lists, when one
person complains that their build has failed, not
realizing that it is because they have tried to cut
corners.</para>
</answer>
</qandaentry>
<qandaentry>
<question>
<para>Can interrupted builds be resumed?</para>
</question>
<answer>
<para>This depends on how far into the process the
problem occurs.</para>
<para>In general, <command>make buildworld</command>
builds new copies of essential tools, such as
&man.gcc.1; and &man.make.1;, and the system libraries.
These tools and libraries are then installed, used to
rebuild themselves, and are installed again. The entire
system, including regular user programs such as
&man.ls.1; or &man.grep.1;, is then rebuilt with the new
system files.</para>
<para>During the last stage, it is fairly safe to:</para>
<screen><emphasis>… fix the problem …</emphasis>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make -DNO_CLEAN all</userinput></screen>
<para>This will not undo the work of the previous
<command>make buildworld</command>.</para>
<para>If you see the message:</para>
<screen>--------------------------------------------------------------
Building everything..
--------------------------------------------------------------</screen>
<para>in the <command>make buildworld</command> output,
it is probably fairly safe to do so.</para>
<para>If that message is not displayed, or you are not
sure, it is always better to be safe than sorry, and
restart the build from scratch.</para>
</answer>
</qandaentry>
<qandaentry>
<question>
<para>How can I speed up making the world?</para>
</question>
<answer>
<itemizedlist>
<listitem>
<para>Run it in single user mode.</para>
</listitem>
<listitem>
<para>Put <filename
class="directory">/usr/src</filename> and
<filename class="directory">/usr/obj</filename>
on separate file systems held on separate disks. If
possible, put these disks on separate disk
controllers.</para>
</listitem>
<listitem>
<para>Alternately, put these file systems across
multiple disks using &man.ccd.4;.</para>
</listitem>
<listitem>
<para>Turn off profiling by setting
<quote>NO_PROFILE=true</quote> in
<filename>/etc/make.conf</filename>.</para>
</listitem>
<listitem>
<para>Pass
<option>-j<replaceable>n</replaceable></option>
to &man.make.1; to run multiple processes in
parallel. This usually helps on both single and
multi processor machines.</para>
</listitem>
<listitem>
<para>The file system holding
<filename class="directory">/usr/src</filename> can
be mounted or remounted with
<option>noatime</option>.
This prevents the file system from recording the
file access time which is probably not
needed.</para>
<screen>&prompt.root; <userinput>mount -u -o noatime /usr/src</userinput></screen>
<warning>
<para>This example assumes <filename
class="directory">/usr/src</filename> is on its
own file system. If it is part of
<filename class="directory">/usr</filename>, then
use that file system mount point instead.</para>
</warning>
</listitem>
<listitem>
<para>The file system holding <filename
class="directory">/usr/obj</filename> can be
mounted or remounted with <option>async</option>
so that disk writes happen asynchronously. The
write completes immediately, and the data is written
to the disk a few seconds later. This allows writes
to be clustered together, and can provide a dramatic
performance boost.</para>
<warning>
<para>Keep in mind that this option makes the file
system more fragile. With this option, there is
an increased chance that, should power fail, the
file system will be in an unrecoverable state when
the machine restarts.</para>
<para>If <filename
class="directory">/usr/obj</filename> is the
only directory on this file system, this is not a
problem. If you have other, valuable data on the
same file system, ensure that there are verified
backups before enabling this option.</para>
</warning>
<screen>&prompt.root; <userinput>mount -u -o async /usr/obj</userinput></screen>
<warning>
<para>If <filename
class="directory">/usr/obj</filename> is
not on its own file system, replace it in the
example with the name of the appropriate mount
point.</para>
</warning>
</listitem>
</itemizedlist>
</answer>
</qandaentry>
<qandaentry>
<question>
<para>What do I do if something goes wrong?</para>
</question>
<answer>
<para>Make absolutely sure that the environment has no
extraneous cruft from earlier builds:</para>
<screen>&prompt.root; <userinput>chflags -R noschg /usr/obj/usr</userinput>
&prompt.root; <userinput>rm -rf /usr/obj/usr</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make cleandir</userinput>
&prompt.root; <userinput>make cleandir</userinput></screen>
<para>Yes, <command>make cleandir</command> really should
be run twice.</para>
<para>Then, restart the whole process, starting
with <command>make buildworld</command>.</para>
<para>If problems persist, send the error and the
output of <command>uname -a</command> to &a.questions;.
Be prepared to answer other questions about the
setup!</para>
</answer>
</qandaentry>
</qandaset>
</sect2>
</sect1>
<sect1 id="small-lan">
<sect1info>
<authorgroup>
<author>
<firstname>Mike</firstname>
<surname>Meyer</surname>
<contrib>Contributed by </contrib>
</author>
</authorgroup>
</sect1info>
<title>Tracking for Multiple Machines</title>
<indexterm>
<primary>NFS</primary>
<secondary>installing multiple machines</secondary>
</indexterm>
<para>When multiple machines need to track the same source tree,
it is a waste of disk space, network bandwidth, and CPU cycles
to have each system download the sources and rebuild everything.
The solution is to have one machine do most of the work, while
the rest of the machines mount that work via NFS. This section
outlines a method of doing so.</para>
<sect2 id="small-lan-preliminaries">
<title>Preliminaries</title>
<para>First, identify a set of machines which will run the
same set of binaries, known as a <emphasis>build
set</emphasis>. Each machine can have a custom kernel, but
will run the same userland binaries. From that set, choose a
machine to be the <emphasis>build machine</emphasis> that the
world and kernel are built on. Ideally, this is a fast
machine that has sufficient spare CPU to run
<command>make buildworld</command> and
<command>make buildkernel</command>. Select a machine to be
the <emphasis>test machine</emphasis>, which will test
software updates before they are put into production. This
<emphasis>must</emphasis> be a machine that can afford to be
down for an extended period of time. It can be the build
machine, but need not be.</para>
<para>All the machines in this build set need to mount
<filename class="directory">/usr/obj</filename> and
<filename class="directory">/usr/src</filename> from the same
machine, and at the same point. Ideally, those directories
are on two different drives on the build machine, but they can
be NFS mounted on that machine as well. For multiple
build sets, <filename class="directory">/usr/src</filename>
should be on one build machine, and NFS mounted on the
rest.</para>
<para>Finally, ensure that <filename>/etc/make.conf</filename>
and <filename>/etc/src.conf</filename> on all the machines in
the build set agree with the build machine. That means that
the build machine must build all the parts of the base system
that any machine in the build set is going to install. Also,
each build machine should have its kernel name set with
<makevar>KERNCONF</makevar> in
<filename>/etc/make.conf</filename>, and the build machine
should list them all in <makevar>KERNCONF</makevar>, listing
its own kernel first. The build machine must have the kernel
configuration files for each machine in
<filename
class="directory">/usr/src/sys/<replaceable>arch</replaceable>/conf</filename>
if it is going to build their kernels.</para>
</sect2>
<sect2 id="small-lan-base-system">
<title>The Base System</title>
<para>On the build machine, build the kernel and world as
described in <xref linkend="make-buildworld"/>, but do
not install anything. After the build has finished, go to the
test machine, and install the built kernel. If this machine
mounts <filename class="directory">/usr/src</filename> and
<filename class="directory">/usr/obj</filename> via NFS,
enable the network and mount these directories after rebooting
to single user mode. The easiest way to do this is to boot to
multi-user, then run <command>shutdown now</command> to go to
single user mode. Once there, install the new kernel and
world and run <command>mergemaster</command> as usual. When
done, reboot to return to normal multi-user operations for
this machine.</para>
<para>After verifying that everything on the test machine is
working properly, use the same procedure to install the new
software on each of the other machines in the build
set.</para>
</sect2>
<sect2 id="small-lan-ports">
<title>Ports</title>
<para>The same ideas can be used for the ports tree. The first
critical step is to mount <filename
class="directory">/usr/ports</filename> from the same
machine to all the machines in the build set. Then, configure
<filename>/etc/make.conf</filename> properly to share
distfiles. Set <makevar>DISTDIR</makevar> to a common shared
directory that is writable by whichever user
<username>root</username> is mapped to by the NFS mounts.
Each machine should set <makevar>WRKDIRPREFIX</makevar> to a
local build directory. Finally, if the system is to build and
distribute packages, set <makevar>PACKAGES</makevar> to a
directory similar to <makevar>DISTDIR</makevar>.</para>
</sect2>
</sect1>
</chapter>