<!--
     The FreeBSD Documentation Project

     $FreeBSD$
-->

<chapter id="cutting-edge">
  <chapterinfo>
    <authorgroup>
      <author>
	<firstname>Jim</firstname>
	<surname>Mock</surname>
	<contrib>Restructured, reorganized, and parts updated by </contrib>
      </author>
      <!-- Mar 2000 -->
    </authorgroup>
    <authorgroup>
      <author>
	<firstname>Jordan</firstname>
	<surname>Hubbard</surname>
	<contrib>Original work by </contrib>
      </author>
      <author>
	<firstname>Poul-Henning</firstname>
	<surname>Kamp</surname>
      </author>
      <author>
	<firstname>John</firstname>
	<surname>Polstra</surname>
      </author>
      <author>
	<firstname>Nik</firstname>
	<surname>Clayton</surname>
      </author>
    </authorgroup>
    <!-- with feedback from various others -->
  </chapterinfo>

  <title>The Cutting Edge</title>

  <sect1 id="cutting-edge-synopsis">
    <title>Synopsis</title>

    <para>&os; is under constant development between releases.  For
      people who want to be on the cutting edge, there are several easy
      mechanisms for keeping your system in sync with the latest
      developments.  Be warned&mdash;the cutting edge is not for everyone!
      This chapter will help you decide if you want to track the
      development system, or stick with one of the released
      versions.</para>

    <para>After reading this chapter, you will know:</para>

    <itemizedlist>
      <listitem><para>The difference between the two development
      branches: &os.stable; and &os.current;.</para>
      </listitem>
      <listitem><para>How to keep your system up to date with
	  <application>CVSup</application>,
	  <application>CVS</application>, or
	  <application>CTM</application>.</para>
      </listitem>
      <listitem><para>How to rebuild and reinstall the entire base
      system with <command>make buildworld</command> (etc).</para>
      </listitem>

    </itemizedlist>

    <para>Before reading this chapter, you should:</para>

    <itemizedlist>
      <listitem><para>Properly set up your network connection (<xref
      linkend="advanced-networking">).</para>
      </listitem>
      <listitem><para>Know how to install additional third-party
        software (<xref linkend="ports">).</para></listitem>
    </itemizedlist>
  </sect1>
  
  <sect1 id="current-stable">
    <title>&os.current; vs. &os.stable;</title>
    <indexterm><primary>-CURRENT</primary></indexterm>
    <indexterm><primary>-STABLE</primary></indexterm>

    <para>There are two development branches to FreeBSD: &os.current; and
      &os.stable;.  This section will explain a bit about each and describe
      how to keep your system up-to-date with each respective tree.
      &os.current; will be discussed first, then &os.stable;.</para>

    <sect2 id="current">
      <title>Staying Current with &os;</title>

      <para>As you read this, keep in mind that &os.current; is the
	<quote>bleeding edge</quote> of &os; development.
	&os.current; users are expected to have a high degree of
	technical skill, and should be capable of solving difficult
	system problems on their own.  If you are new to &os;, think
	twice before installing it.  </para>

      <sect3>
	<title>What Is &os.current;?</title>
	<indexterm><primary>snapshot</primary></indexterm>

	<para>&os.current; is the latest working sources for &os;.
	  This includes work in progress, experimental changes, and
	  transitional mechanisms that might or might not be present
	  in the next official release of the software.  While many
	  &os; developers compile the &os.current; source code daily,
	  there are periods of time when the sources are not
	  buildable.  These problems are resolved as expeditiously as
	  possible, but whether or not &os.current; brings disaster or
	  greatly desired functionality can be a matter of which exact
	  moment you grabbed the source code in!</para>
      </sect3>

      <sect3>
	<title>Who Needs &os.current;?</title>

	<para>&os.current; is made available for 3 primary
	  interest groups:</para>

	<orderedlist>
	  <listitem>
	    <para>Members of the &os; community who are actively working
	      on some part of the source tree and for whom keeping
	      <quote>current</quote> is an absolute
	      requirement.</para>
	  </listitem>

	  <listitem>
	    <para>Members of the &os; community who are active testers,
	      willing to spend time solving problems in order to
	      ensure that &os.current; remains as sane as possible.
	      These are also people who wish to make topical
	      suggestions on changes and the general direction of
	      &os;, and submit patches to implement them.</para>
	  </listitem>

	  <listitem>
	    <para>Those who merely wish to keep an eye on things, or
	      to use the current sources for reference purposes
	      (e.g. for <emphasis>reading</emphasis>, not running).
	      These people also make the occasional comment or
	      contribute code.</para>
	  </listitem>
	</orderedlist>
      </sect3>

      <sect3>
	<title>What Is &os.current; <emphasis>Not</emphasis>?</title>

	<orderedlist>
	  <listitem>
	    <para>A fast-track to getting pre-release bits because you
	      heard there is some cool new feature in there and you
	      want to be the first on your block to have it.  Being
	      the first on the block to get the new feature means that
	      you are the first on the block to get the new
	      bugs.</para>
	  </listitem>

	  <listitem>
	    <para>A quick way of getting bug fixes.  Any given version
	      of &os.current; is just as likely to introduce new bugs
	      as to fix existing ones.</para>
	  </listitem>

	  <listitem>
	    <para>In any way <quote>officially supported</quote>.  We
	      do our best to help people genuinely in one of the 3
	      <quote>legitimate</quote> &os.current; groups, but we
	      simply <emphasis>do not have the time</emphasis> to
	      provide tech support.  This is not because we are mean
	      and nasty people who do not like helping people out (we
	      would not even be doing &os; if we were).  We simply
	      cannot answer hundreds messages a day
	      <emphasis>and</emphasis> work on FreeBSD!  Given the
	      choice between improving &os; and answering lots of
	      questions on experimental code, the developers opt for
	      the former.</para>
	  </listitem>
	</orderedlist>
      </sect3>

      <sect3>
	<title>Using &os.current;</title>
      
	<indexterm>
	  <primary>-CURRENT</primary>
	  <secondary>using</secondary>
	</indexterm>
	<orderedlist>
	  <listitem>
	    <para>Join the &a.current.name; and the &a.cvsall.name; lists.  This is not
	      just a good idea, it is <emphasis>essential</emphasis>.  If
	      you are not on the <emphasis>&a.current.name;</emphasis> list,
	      you will not see the comments that people are
	      making about the current state of the system and thus will
	      probably end up stumbling over a lot of problems that others
	      have already found and solved.  Even more importantly, you
	      will miss out on important bulletins which may be critical
	      to your system's continued health.</para>

	    <para>The &a.cvsall.name; list will allow you to see the
	      commit log entry for each change as it is made along with
	      any pertinent information on possible side-effects.</para>

	    <para>To join these lists, or one of the others available
	      go to &a.mailman.lists.link; and click on the list that
	      you wish to subscribe to.  Instructions on the rest of
	      the procedure are available there.</para>
	  </listitem>

	  <listitem>
	    <para>Grab the sources from a &os; <link linkend="mirrors">mirror
	      site</link>.  You can do this in one of two ways:</para>

	    <orderedlist>
              <indexterm>
                <primary><command>cvsup</command></primary>
              </indexterm>
              <indexterm>
                <primary><command>cron</command></primary>
              </indexterm>
              <indexterm>
	        <primary>-CURRENT</primary>
		<secondary>Syncing with <application>CVSup</application></secondary>
	      </indexterm>

	      <listitem>
		<para>Use the <link linkend="cvsup">cvsup</link> program
                  with the <filename>supfile</filename> named <filename>standard-supfile</filename>
                  available from <filename>/usr/share/examples/cvsup</filename>.
		  This is the most recommended
		  method, since it allows you to grab the entire
		  collection once and then only what has changed from then
		  on.  Many people run <command>cvsup</command> from 
		  <command>cron</command> and keep their
 		  sources up-to-date automatically.  You have to 
 		  customize the sample <filename>supfile</filename> above, and configure 
 		  <link linkend="cvsup">cvsup</link> for your environment.</para>
	      </listitem>

              <indexterm>
	        <primary>-CURRENT</primary>
		<secondary>Syncing with CTM</secondary>
	      </indexterm>
	      <listitem>
		<para>Use the <application><link
		  linkend="ctm">CTM</link></application> facility.  If you
		  have very bad connectivity (high price connections or
		  only email access) <application>CTM</application> is an option.
		  However, it is a lot of hassle and can give you broken files.
		  This leads to it being rarely used, which again increases
		  the chance of it not working for fairly long periods of
		  time.  We recommend using
		  <application><link linkend="cvsup">CVSup</link></application>
		  for anybody with a 9600&nbsp;bps modem or faster connection.
		</para>
	      </listitem>
	    </orderedlist>
	  </listitem>

	  <listitem>
	    <para>If you are grabbing the sources to run, and not just
	      look at, then grab <emphasis>all</emphasis> of &os.current;, not
	      just selected portions.  The reason for this is that various
	      parts of the source depend on updates elsewhere, and trying
	      to compile just a subset is almost guaranteed to get you
	      into trouble.</para>

	  <indexterm>
	    <primary>-CURRENT</primary>
	    <secondary>compiling</secondary>
	  </indexterm>
	    <para>Before compiling &os.current;, read the
	      <filename>Makefile</filename> in <filename>/usr/src</filename>
	      carefully.  You should at least <link
	      linkend="makeworld">install a new kernel and rebuild the world</link> the first time through
	      as part of the upgrading process.  Reading the &a.current;
	      and <filename>/usr/src/UPDATING</filename> will keep you up-to-date on other bootstrapping procedures
	      that sometimes become necessary as we move toward the next
	      release.</para>
	  </listitem>
	      
	  <listitem>
	    <para>Be active! If you are running &os.current;, we want
	      to know what you have to say about it, especially if you
	      have suggestions for enhancements or bug fixes.  Suggestions
	      with accompanying code are received most
	      enthusiastically!</para>
	  </listitem>
	</orderedlist>
      </sect3>
    </sect2>
  
    <sect2 id="stable">
      <title>Staying Stable with &os;</title>

      <sect3>
	<title>What Is &os.stable;?</title>
	<indexterm><primary>-STABLE</primary></indexterm>

	<para>&os.stable; is our development branch from which major releases
	  are made.  Changes go into this branch at a different pace, and
	  with the general assumption that they have first gone into
	  &os.current; for testing.  This is <emphasis>still</emphasis>
	  a development branch, however, and this means that at any given time,
	  the sources for &os.stable; may or may not be suitable for any
	  particular purpose.  It is simply another engineering development
	  track, not a resource for end-users.</para>
      </sect3>

      <sect3>
	<title>Who Needs &os.stable;?</title>

	<para>If you are interested in tracking or contributing to the
          FreeBSD development process, especially as it relates to the
	  next <quote>point</quote> release of FreeBSD, then you should
	  consider following &os.stable;.</para>

	<para>While it is true that security fixes also go into the
	  &os.stable; branch, you do not <emphasis>need</emphasis> to
	  track &os.stable; to do this. Every security advisory for
	  FreeBSD explains how to fix the problem for the releases it
	  affects
	  <footnote><para>That is not quite true.  We can not continue to
  	    support old releases of FreeBSD forever, although we do
  	    support them for many years.  For a complete description
  	    of the current security policy for old releases of
  	    FreeBSD, please see <ulink
  	    url="&url.base;/security/">http://www.FreeBSD.org/security/</ulink>.</para>
	  </footnote>
	  , and tracking an entire development branch just
	  for security reasons is likely to bring in a lot of unwanted
	  changes as well.</para>

	<para>Although we endeavor to ensure that the &os.stable; branch
	  compiles and runs at all times, this cannot be guaranteed.  In
	  addition, while code is developed in &os.current; before including
	  it in &os.stable;, more people run &os.stable; than &os.current;, so
	  it is inevitable that bugs and corner cases will sometimes be found
	  in &os.stable; that were not apparent in &os.current;.</para>

	<para>For these reasons, we do <emphasis>not</emphasis> recommend that
	  you blindly track &os.stable;, and it is particularly important that
	  you do not update any production servers to &os.stable; without
	  first thoroughly testing the code in your development
	  environment.</para>

	<para>If you do not have the resources to do this then we recommend
	  that you run the most recent release of FreeBSD, and use the binary
	  update mechanism to move from release to release.</para>
      </sect3>

      <sect3>
	<title>Using &os.stable;</title>
      
	<indexterm>
	  <primary>-STABLE</primary>
	  <secondary>using</secondary>
	</indexterm>
	<orderedlist>
	  <listitem>
	    <para>Join the &a.stable.name; list.  This will keep you informed of
	      build-dependencies that may appear in &os.stable;
	      or any other issues requiring
	      special attention.  Developers will also make announcements
	      in this mailing list when they are contemplating some
	      controversial fix or update, giving the users a chance to
	      respond if they have any issues to raise concerning the
	      proposed change.</para>

	    <para>The &a.cvsall.name; list will allow you to see the
	      commit log entry for each change as it is made along with
	      any pertinent information on possible side-effects.</para>

	    <para>To join these lists, or one of the others available
	      go to &a.mailman.lists.link; and click on the list that
	      you wish to subscribe to.  Instructions on the rest of
	      the procedure are available there.</para>
	  </listitem>

	  <listitem>
	    <para>If you are installing a new system and want it to be as
	      stable as possible, you can simply grab the latest dated
	      branch snapshot from <ulink
	      url="ftp://snapshots.jp.FreeBSD.org/pub/FreeBSD/snapshots/"></ulink>
	      and install it like any other release.  Or you can
	      install the most recent &os.stable; release from the
	      <link linkend="mirrors">mirror sites</link> and follow
	      the instructions below to upgrade your system to the
	      most up to date &os.stable; source code.</para>

	    <para>If you are already running a previous release of &os;
	      and wish to upgrade via sources then you can easily do so
	      from &os; <link linkend="mirrors">mirror site</link>.  This can
	      be done in one of two ways:</para>

	    <orderedlist>
	      <indexterm>
		<primary><command>cvsup</command></primary>
	      </indexterm>
	      <indexterm>
		<primary><command>cron</command></primary>
	      </indexterm>
	      <indexterm>
	        <primary>-STABLE</primary>
		<secondary>syncing with <application>CVSup</application></secondary>
	      </indexterm>
	      <listitem>
		<para>Use the <link linkend="cvsup">cvsup</link> program
                  with the <filename>supfile</filename> named <filename>stable-supfile</filename>
                  from the directory
                  <filename>/usr/share/examples/cvsup</filename>.
		  This is the most recommended
		  method, since it allows you to grab the entire
		  collection once and then only what has changed from then
		  on.  Many people run <command>cvsup</command> from 
		  <command>cron</command> to keep their
		  sources up-to-date automatically.  You have to
		  customize the sample <filename>supfile</filename> above,
		  and configure <link linkend="cvsup">cvsup</link> for your
		  environment.</para>
	      </listitem>

	      <indexterm>
	        <primary>-STABLE</primary>
		<secondary>syncing with CTM</secondary>
	      </indexterm>
	      <listitem>
		<para>Use the <application><link
		  linkend="ctm">CTM</link></application> facility.  If
		  you do not have a fast and inexpensive connection to
		  the Internet, this is the method you should consider
		  using.
		  </para>
	      </listitem>
	   </orderedlist>
	 </listitem>

	  <listitem>
	    <para>Essentially, if you need rapid on-demand access to the
	      source and communications bandwidth is not a consideration,
	      use <command>cvsup</command> or <command>ftp</command>.
	      Otherwise, use <application>CTM</application>.</para>
	  </listitem>

	  <indexterm>
	    <primary>-STABLE</primary>
	    <secondary>compiling</secondary>
	  </indexterm>
	  <listitem>
	    <para>Before compiling &os.stable;, read the
	    <filename>Makefile</filename> in <filename>/usr/src</filename>
	    carefully.  You should at least <link
	    linkend="makeworld">install a new kernel and rebuild the world</link> the first time through
	    as part of the upgrading process.  Reading the &a.stable; and <filename>/usr/src/UPDATING</filename> will
	    keep you up-to-date on other bootstrapping procedures that
	    sometimes become necessary as we move toward the next
	    release.</para>
	  </listitem>
	</orderedlist>
      </sect3>
    </sect2>
  </sect1>

  <sect1 id="synching">
    <title>Synchronizing Your Source</title>
    
    <para>There are various ways of using an Internet (or email)
      connection to stay up-to-date with any given area of the &os;
      project sources, or all areas, depending on what interests you.  The
      primary services we offer are <link linkend="anoncvs">Anonymous
      CVS</link>, <link linkend="cvsup">CVSup</link>, and <link
      linkend="ctm">CTM</link>.</para>

    <warning>
      <para>While it is possible to update only parts of your source tree,
        the only supported update procedure is to update the entire tree
        and recompile both userland (i.e., all the programs that run in
        user space, such as those in <filename>/bin</filename> and
        <filename>/sbin</filename>) and kernel sources.  Updating only part
        of your source tree, only the kernel, or only userland will often
        result in problems.  These problems may range from compile errors
        to kernel panics or data corruption.</para>
    </warning>

    <indexterm>
      <primary>CVS</primary>
      <secondary>anonymous</secondary>
    </indexterm>

    <para><application>Anonymous CVS</application> and
      <application>CVSup</application> use the <emphasis>pull</emphasis>
      model of updating sources.  In the case of
      <application>CVSup</application> the user (or a 
      <command>cron</command> script) invokes
      the <command>cvsup</command> program, and it interacts with a
      <command>cvsupd</command> server somewhere to bring your files
      up-to-date.  The updates you receive are up-to-the-minute and you
      get them when, and only when, you want them.  You can easily
      restrict your updates to the specific files or directories that are
      of interest to you.  Updates are generated on the fly by the server,
      according to what you have and what you want to have.
      <application>Anonymous CVS</application> is quite a bit more
      simplistic than <application>CVSup</application> in that it is just an extension to
      <application>CVS</application> which allows it to pull changes
      directly from a remote CVS repository.
      <application>CVSup</application> can do this far more efficiently,
      but <application>Anonymous CVS</application> is easier to
      use.</para>

    <indexterm>
      <primary><application>CTM</application></primary>
    </indexterm>
    <para><application>CTM</application>, on the other hand, does not
      interactively compare the sources you have with those on the master
      archive or otherwise pull them across.  Instead, a script which
      identifies changes in files since its previous run is executed
      several times a day on the master CTM machine, any detected changes
      being compressed, stamped with a sequence-number and encoded for
      transmission over email (in printable ASCII only).  Once received,
      these <quote>CTM deltas</quote> can then be handed to the
      &man.ctm.rmail.1; utility which will automatically decode, verify
      and apply the changes to the user's copy of the sources.  This
      process is far more efficient than <application>CVSup</application>,
      and places less strain on our server resources since it is a
      <emphasis>push</emphasis> rather than a <emphasis>pull</emphasis>
      model.</para>

    <para>There are other trade-offs, of course.  If you inadvertently
      wipe out portions of your archive, <application>CVSup</application>
      will detect and rebuild the damaged portions for you.
      <application>CTM</application> will not do this, and if you wipe some
      portion of your source tree out (and do not have it backed up) then
      you will have to start from scratch (from the most recent CVS
      <quote>base delta</quote>) and rebuild it all with <application>CTM</application> or, with
      <application>Anonymous CVS</application>, simply delete the bad bits and resync.</para>
  </sect1>

  <sect1 id="makeworld">
    <title>Rebuilding <quote>world</quote></title>

    <indexterm>
      <primary>Rebuilding <quote>world</quote></primary>
    </indexterm>
    <para>Once you have synchronized your local source tree against a
      particular version of &os; (&os.stable;, &os.current;, and so on)
      you can then use the source
      tree to rebuild the system.</para>

    <warning>
      <title>Take a Backup</title>

      <para>It cannot be stressed enough how important it is to take a
	backup of your system <emphasis>before</emphasis> you do this.
	While rebuilding the world is (as long as you follow these
	instructions) an easy task to do, there will inevitably be times
	when you make mistakes, or when mistakes made by others in the
	source tree render your system unbootable.</para>

      <para>Make sure you have taken a backup.  And have a fixit floppy or
	bootable CD at
	hand.  You will probably never have to use it, but it is better to be
	safe than sorry!</para>
    </warning>

    <warning>
      <title>Subscribe to the Right Mailing List</title>

      <indexterm><primary>mailing list</primary></indexterm>
      <para>The &os.stable; and &os.current; branches are, by their
	nature, <emphasis>in development</emphasis>.  People that
	contribute to &os; are human, and mistakes occasionally
	happen.</para>

      <para>Sometimes these mistakes can be quite harmless, just causing
	your system to print a new diagnostic warning.  Or the change may
	be catastrophic, and render your system unbootable or destroy your
	file systems (or worse).</para>

      <para>If problems like these occur, a <quote>heads up</quote> is
        posted to the appropriate mailing list, explaining the nature of
	the problem and which systems it affects.  And an <quote>all
	clear</quote> announcement is posted when the problem has been
	solved.</para>

      <para>If you try to track &os.stable; or &os.current; and do
	not read the &a.stable; or the
	&a.current; respectively, then you are
	asking for trouble.</para>
    </warning>

    <warning>
      <title>Do not use <command>make world</command></title>

      <para>A lot of older documentation recommends using
	<command>make world</command> for this.  Doing that skips
	some important steps and should only be used if you are
	sure of what you are doing.  For almost all circumstances
	<command>make world</command> is the wrong thing to do, and
	the procedure described here should be used instead.</para>
    </warning>

    <sect2>
      <title>The Canonical Way to Update Your System</title>

      <para>To update your system, you should check
	<filename>/usr/src/UPDATING</filename> for any pre-buildworld steps
	necessary for your version of the sources and then use the following
	procedure:</para>

      <screen>&prompt.root; <userinput>make buildworld</userinput>
&prompt.root; <userinput>make buildkernel</userinput>
&prompt.root; <userinput>make installkernel</userinput>
&prompt.root; <userinput>reboot</userinput></screen>

      <note>
	<para>There are a few rare cases when an extra run of
	  <command>mergemaster -p</command> is needed before the
	  <maketarget>buildworld</maketarget> step.  These are
	  described in <filename>UPDATING</filename>.  In general,
	  though, you can safely omit this step if you are not
	  updating across one or more major &os; versions.</para>
      </note>

      <para>After <maketarget>installkernel</maketarget> finishes
	successfully, you should boot in single user mode
	(i.e.&nbsp;using <command>boot -s</command> from the loader
	prompt).  Then run:</para>

      <screen>&prompt.root; <userinput>mergemaster -p</userinput>
&prompt.root; <userinput>make installworld</userinput>
&prompt.root; <userinput>mergemaster</userinput>
&prompt.root; <userinput>reboot</userinput></screen>

      <warning>
	<title>Read Further Explanations</title>

	<para>The sequence described above is only a short resume to
	  help you getting started.  You should however read the
	  following sections to clearly understand each step, especially
	  if you want to use a custom kernel configuration.</para>
      </warning>
    </sect2>

    <sect2>
      <title>Read <filename>/usr/src/UPDATING</filename></title>

      <para>Before you do anything else, read
	<filename>/usr/src/UPDATING</filename> (or the equivalent file
	wherever you have a copy of the source code).  This file should
	contain important information about problems you might encounter, or
	specify the order in which you might have to run certain commands.
	If <filename>UPDATING</filename> contradicts something you read here,
	<filename>UPDATING</filename> takes precedence.</para>

      <important>
	<para>Reading <filename>UPDATING</filename> is not an acceptable
	  substitute for subscribing to the correct mailing list, as described 
	  previously.  The two requirements are complementary, not
	  exclusive.</para>
      </important>
    </sect2>

    <sect2>
      <title>Check <filename>/etc/make.conf</filename></title>
      <indexterm>
        <primary><filename>make.conf</filename></primary>
      </indexterm>

      <para>Examine the files
	<filename>/usr/share/examples/etc/make.conf</filename>
	(called <filename>/etc/defaults/make.conf</filename> in &os;&nbsp;4.X) and
	<filename>/etc/make.conf</filename>.  The first contains some
	default defines &ndash; most of which are commented out.  To
	make use of them when you rebuild your system from source, add
	them to <filename>/etc/make.conf</filename>.  Keep in mind that
	anything you add to <filename>/etc/make.conf</filename> is also
	used every time you run <command>make</command>, so it is a good
	idea to set them to something sensible for your system.</para>

      <para>A typical user will probably want to copy the
	<makevar>CFLAGS</makevar> and
	<makevar>NOPROFILE</makevar> lines found in
	<filename>/usr/share/examples/etc/make.conf</filename>
	(or in <filename>/etc/defaults/make.conf</filename> on &os;&nbsp;4.X) to
	<filename>/etc/make.conf</filename> and uncomment them.</para>

      <para>Examine the other definitions (<makevar>COPTFLAGS</makevar>, 
	<makevar>NOPORTDOCS</makevar> and so
	on) and decide if they are relevant to you.</para>
    </sect2>

    <sect2>
      <title>Update the Files in <filename>/etc</filename></title>

      <para>The <filename>/etc</filename> directory contains a large part
	of your system's configuration information, as well as scripts
	that are run at system startup.  Some of these scripts change from
	version to version of FreeBSD.</para>

      <para>Some of the configuration files are also used in the day to
	day running of the system.  In particular,
	<filename>/etc/group</filename>.</para>

      <para>There have been occasions when the installation part of
        <command>make installworld</command> has expected certain usernames or groups
	to exist.  When performing an upgrade it is likely that these
	users or groups did not exist. This caused problems when upgrading.
	In some cases <command>make buildworld</command> will check to see if
	these users or groups exist.</para>

      <para>A recent example of this is when the
	<username>smmsp</username> user was added.  Users had the
	installation process fail for them when
	&man.mtree.8; was trying to create
	<filename>/var/spool/clientmqueue</filename>.</para>

      <para>The solution is to examine
	<filename>/usr/src/etc/group</filename> and compare its list of
	groups with your own.  If there are any groups in the new file that
	are not in your file then copy them over.  Similarly, you should
	rename any groups in <filename>/etc/group</filename> which have
	the same GID but a different name to those in
	<filename>/usr/src/etc/group</filename>.</para>

      <para>Since 4.6-RELEASE you can run &man.mergemaster.8; in
	pre-buildworld mode by providing the <option>-p</option> option.
	This will compare only those files that are essential for the success
	of <maketarget>buildworld</maketarget> or
	<maketarget>installworld</maketarget>.  If your old version of
	<command>mergemaster</command> does not support <option>-p</option>,
	use the new version in the source tree when running for the first
	time:</para>

      <screen>&prompt.root; <userinput>cd /usr/src/usr.sbin/mergemaster</userinput>
&prompt.root; <userinput>./mergemaster.sh -p</userinput></screen>

      <tip>
	<para>If you are feeling particularly paranoid, you can check your
	  system to see which files are owned by the group you are
	  renaming or deleting:</para>

	<screen>&prompt.root; <userinput>find / -group <replaceable>GID</replaceable> -print</userinput></screen>

	<para>will show all files owned by group
	  <replaceable>GID</replaceable> (which can be either a group name
	  or a numeric group ID).</para>
      </tip>
    </sect2>

    <sect2 id="makeworld-singleuser">
      <title>Drop to Single User Mode</title>
      <indexterm><primary>single-user mode</primary></indexterm>

      <para>You may want to compile the system in single user mode.  Apart
	from the obvious benefit of making things go slightly faster,
	reinstalling the system will touch a lot of important system
	files, all the standard system binaries, libraries, include files
	and so on.  Changing these on a running system (particularly if
	you have active users on the system at the time) is asking for
	trouble.</para>

      <indexterm><primary>multi-user mode</primary></indexterm>
      <para>Another method is to compile the system in multi-user mode, and
        then drop into single user mode for the installation.  If you would
        like to do it this way, simply hold off on the following steps until
        the build has completed.  You can postpone dropping to single user
	mode until you have to <maketarget>installkernel</maketarget> or
	<maketarget>installworld</maketarget>.</para>

      <para>As the superuser, you can execute:</para>

      <screen>&prompt.root; <userinput>shutdown now</userinput></screen>

      <para>from a running system, which will drop it to single user
	mode.</para>

      <para>Alternatively, reboot the system, and at the boot prompt,
        enter the <option>-s</option> flag.  The system will then boot
	single user.  At the shell prompt you should then run:</para>

      <screen>&prompt.root; <userinput>fsck -p</userinput>
&prompt.root; <userinput>mount -u /</userinput>
&prompt.root; <userinput>mount -a -t ufs</userinput>
&prompt.root; <userinput>swapon -a</userinput></screen>

      <para>This checks the file systems, remounts <filename>/</filename>
	read/write, mounts all the other UFS file systems referenced in
	<filename>/etc/fstab</filename> and then turns swapping on.</para>


        <note>
          <para>If your CMOS clock is set to local time and not to GMT
	    (this is true if the output of the &man.date.1; command
	    does not show the correct time and zone),
            you may also need to run the following command:</para>
<screen>&prompt.root; <userinput>adjkerntz -i</userinput></screen>

          <para>This will make sure that your local time-zone settings
            get set up correctly &mdash; without this, you may later run into some
            problems.
          </para>
        </note>

    </sect2>

    <sect2>
      <title>Remove <filename>/usr/obj</filename></title>

      <para>As parts of the system are rebuilt they are placed in
	directories which (by default) go under
	<filename>/usr/obj</filename>.  The directories shadow those under
	<filename>/usr/src</filename>.</para>

      <para>You can speed up the <command>make buildworld</command> process, and
	possibly save yourself some dependency headaches by removing this
	directory as well.</para>

      <para>Some files below <filename>/usr/obj</filename> may have the
	immutable flag set (see &man.chflags.1; for more information)
	which must be removed first.</para>

      <screen>&prompt.root; <userinput>cd /usr/obj</userinput>
&prompt.root; <userinput>chflags -R noschg *</userinput>
&prompt.root; <userinput>rm -rf *</userinput></screen>
    </sect2>

    <sect2>
      <title>Recompile the Source</title>

      <sect3>
	<title>Saving the Output</title>

	<para>It is a good idea to save the output you get from running
	  &man.make.1; to another file.  If something goes wrong you will
	  have a copy of the error message.  While this might not help you
	  in diagnosing what has gone wrong, it can help others if you post
	  your problem to one of the &os; mailing lists.</para>

	<para>The easiest way to do this is to use the &man.script.1;
	  command, with a parameter that specifies the name of the file to
	  save all output to.  You would do this immediately before
	  rebuilding the world, and then type <userinput>exit</userinput>
	  when the process has finished.</para>

	<screen>&prompt.root; <userinput>script /var/tmp/mw.out</userinput>
Script started, output file is /var/tmp/mw.out	 
&prompt.root; <userinput>make TARGET</userinput>
<emphasis>&hellip; compile, compile, compile &hellip;</emphasis>	  
&prompt.root; <userinput>exit</userinput>
Script done, &hellip;</screen>

	<para>If you do this, <emphasis>do not</emphasis> save the output
	  in <filename>/tmp</filename>.  This directory may be cleared
	  next time you reboot.  A better place to store it is in
	  <filename>/var/tmp</filename> (as in the previous example) or
	  in <username>root</username>'s home directory.</para>
      </sect3>

      <sect3 id="make-buildworld">
	<title>Compile the Base System</title>

	<para>You must be in the <filename>/usr/src</filename>
	  directory:</para>

	<screen>&prompt.root; <userinput>cd /usr/src</userinput></screen>

	<para>(unless, of course, your source code is elsewhere, in which
	  case change to that directory instead).</para>
	<indexterm><primary><command>make</command></primary></indexterm>

	<para>To rebuild the world you use the &man.make.1; command.  This
          command reads instructions from the <filename>Makefile</filename>,
          which describes how the programs that comprise &os; should be
          rebuilt, the order in which they should be built, and so on.</para>

	<para>The general format of the command line you will type is as
	  follows:</para>

	<screen>&prompt.root; <userinput>make -<replaceable>x</replaceable> -D<replaceable>VARIABLE</replaceable> <replaceable>target</replaceable></userinput></screen>

	<para>In this example, <option>-<replaceable>x</replaceable></option>
	  is an option that you would pass to &man.make.1;.  See the
	  &man.make.1; manual page for an example of the options you can
	  pass.</para>

	<para><option>-D<replaceable>VARIABLE</replaceable></option>
	  passes a variable to the <filename>Makefile</filename>.  The
	  behavior of the <filename>Makefile</filename> is controlled by
	  these variables.  These are the same variables as are set in
	  <filename>/etc/make.conf</filename>, and this provides another
	  way of setting them.</para>

	<screen>&prompt.root; <userinput>make -DNOPROFILE <replaceable>target</replaceable></userinput></screen>

	<para>is another way of specifying that profiled libraries should
	  not be built, and corresponds with the</para>

	<programlisting>NOPROFILE=    true 	#    Avoid compiling profiled libraries</programlisting>

	<para>line in <filename>/etc/make.conf</filename>.</para>

	<para><replaceable>target</replaceable> tells &man.make.1; what
	  you want to do.  Each <filename>Makefile</filename> defines a
	  number of different <quote>targets</quote>, and your choice of
	  target determines what happens.</para>

	<para>Some targets are listed in the
	  <filename>Makefile</filename>, but are not meant for you to run.
	  Instead, they are used by the build process to break out the
	  steps necessary to rebuild the system into a number of
	  sub-steps.</para>

	<para>Most of the time you will not need to pass any parameters to
	    &man.make.1;, and so your command like will look like
	    this:</para>

	<screen>&prompt.root; <userinput>make <replaceable>target</replaceable></userinput></screen>

	<para>Beginning with version 2.2.5 of &os; (actually, it was
	  first created on the &os.current; branch, and then retrofitted to
	  &os.stable; midway between 2.2.2 and 2.2.5) the
	  <maketarget>world</maketarget> target has been split in
	  two: <maketarget>buildworld</maketarget> and
	  <maketarget>installworld</maketarget>.  Beginning with version
	  5.3 of &os; the <maketarget>world</maketarget> target will be changed
	  so it will not work at all by default because it is actually
	  dangerous for most users.</para>

	<para>As the names imply, <maketarget>buildworld</maketarget>
	  builds a complete new tree under <filename>/usr/obj</filename>,
	  and <maketarget>installworld</maketarget> installs this tree on
	  the current machine.</para>

	<para>This is very useful for 2 reasons.  First, it allows you
	  to do the build safe in the knowledge that no components of
	  your running system will be affected.  The build is
	  <quote>self hosted</quote>.  Because of this, you can safely
	  run <maketarget>buildworld</maketarget> on a machine running
	  in multi-user mode with no fear of ill-effects.  It is still
	  recommended that you run the
	  <maketarget>installworld</maketarget> part in single user
	  mode, though.</para>

	<para>Secondly, it allows you to use NFS mounts to upgrade 
	  multiple machines on your network.  If you have three machines,
	  <hostid>A</hostid>, <hostid>B</hostid> and <hostid>C</hostid> that you want to upgrade, run <command>make
	  buildworld</command> and <command>make installworld</command> on
	  <hostid>A</hostid>.  <hostid>B</hostid> and <hostid>C</hostid> should then NFS mount <filename>/usr/src</filename>
	  and <filename>/usr/obj</filename> from <hostid>A</hostid>, and you can then run
	  <command>make installworld</command> to install the results of 
	  the build on <hostid>B</hostid> and <hostid>C</hostid>.</para>

	<para>Although the <maketarget>world</maketarget> target still exists,
	  you are strongly encouraged not to use it.</para>

	<para>Run</para>

	<screen>&prompt.root; <userinput>make buildworld</userinput></screen>
 
        <para>It is now possible to specify a <option>-j</option> option to
          <command>make</command> which will cause it to spawn several
          simultaneous processes.  This is most useful on multi-CPU machines.
          However, since much of the compiling process is IO bound rather
          than CPU bound it is also useful on single CPU machines.</para>

	<para>On a typical single-CPU machine you would run:</para>
	  
	  <screen>&prompt.root; <userinput>make -j4 buildworld</userinput></screen>

	<para>&man.make.1; will then have up to 4 processes running at any one
	  time.  Empirical evidence posted to the mailing lists shows this
	  generally gives the best performance benefit.</para>

	<para>If you have a multi-CPU machine and you are using an SMP
	  configured kernel try values between 6 and 10 and see how they speed
	  things up.</para>

	<para>Be aware that this is still somewhat experimental, and commits
	  to the source tree may occasionally break this feature.  If the
	  world fails to compile using this parameter try again without it
	  before you report any problems.</para>
      </sect3>
      
      <sect3>
	<title>Timings</title>
	<indexterm>
	  <primary>rebuilding <quote>world</quote></primary>
	  <secondary>timings</secondary>
	</indexterm>

        <para>Many factors influence the build time, but currently a 500&nbsp;MHz
          &pentium;&nbsp;III with 128&nbsp;MB of RAM takes about 2&nbsp;hours to build
          the &os.stable; tree, with no tricks or shortcuts used during the
          process.  A &os.current; tree will take somewhat longer.</para>
      </sect3>
    </sect2>
    
    <sect2>
      <title>Compile and Install a New Kernel</title>
      <indexterm>
        <primary>kernel</primary>
	<secondary>compiling</secondary>
      </indexterm>

      <para>To take full advantage of your new system you should recompile the
	kernel.  This is practically a necessity, as certain memory structures
	may have changed, and programs like &man.ps.1; and &man.top.1; will
	fail to work until the kernel and source code versions are the
	same.</para>

      <para>The simplest, safest way to do this is to build and install a
	kernel based on <filename>GENERIC</filename>.  While
	<filename>GENERIC</filename> may not have all the necessary devices
	for your system, it should contain everything necessary to boot your
	system back to single user mode.  This is a good test that the new
	system works properly.  After booting from
	<filename>GENERIC</filename> and verifying that your system works you
	can then build a new kernel based on your normal kernel	configuration
	file.</para>

      <para>On modern versions of FreeBSD it is important to <link
        linkend="make-buildworld">build world</link> before building a
        new kernel.</para>

      <note><para>If you want to build a custom kernel, and already have a configuration
	file, just use <literal>KERNCONF=<replaceable>MYKERNEL</replaceable></literal>
	like this:</para>

      <screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make buildkernel KERNCONF=<replaceable>MYKERNEL</replaceable></userinput>
&prompt.root; <userinput>make installkernel KERNCONF=<replaceable>MYKERNEL</replaceable></userinput></screen>

	<para>In FreeBSD&nbsp;4.2 and older you must replace
	  <literal>KERNCONF=</literal> with <literal>KERNEL=</literal>.
	  4.2-STABLE that was fetched before Feb 2nd, 2001 does not
	  recognize <literal>KERNCONF=</literal>.</para>
      </note>

      <para>Note that if you have raised <literal>kern.securelevel</literal>
	above 1 <emphasis>and</emphasis> you have set either the
	<literal>noschg</literal> or similar flags to your kernel binary, you
	might find it necessary to drop into single user mode to use
	<maketarget>installkernel</maketarget>.  Otherwise you should be able
	to run both these commands from multi user mode without
	problems.  See &man.init.8; for details about
	<literal>kern.securelevel</literal> and &man.chflags.1; for details
	about the various file flags.</para>
      <para>If you are upgrading to a version of &os; below 4.0 you should
	use the old kernel build procedure.  However, it is recommended
	that you use the new version of &man.config.8;, using a command line
	like this.</para>
      
      <screen>&prompt.root; <userinput>/usr/obj/usr/src/usr.sbin/config/config <replaceable>KERNELNAME</replaceable></userinput></screen>
    </sect2>

    <sect2>
      <title>Reboot into Single User Mode</title>
      <indexterm><primary>single-user mode</primary></indexterm>

      <para>You should reboot into single user mode to test the new kernel
	works.  Do this by following the instructions in
	<xref linkend="makeworld-singleuser">.</para>
    </sect2>

    <sect2>
      <title>Install the New System Binaries</title>

      <para>If you were building a version of &os; recent enough to have
	used <command>make buildworld</command> then you should now use
	<maketarget>installworld</maketarget> to install the new system
	binaries.</para>

      <para>Run</para>

      <screen>&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld</userinput></screen>

      <note>
	<para>If you specified variables on the <command>make
	    buildworld</command> command line, you must specify the same
	  variables in the <command>make installworld</command> command
	  line.  This does not necessarily hold true for other options;
	  for example, <option>-j</option> must never be used with
	  <maketarget>installworld</maketarget>.</para>
	
	<para>For example, if you ran:</para>

	<screen>&prompt.root; <userinput>make -DNOPROFILE buildworld</userinput></screen>

	<para>you must install the results with:</para>

	<screen>&prompt.root; <userinput>make -DNOPROFILE installworld</userinput></screen>

	<para>otherwise it would try to install profiled libraries that
	  had not been built during the <command>make buildworld</command>
	  phase.</para>
      </note>
    </sect2>

    <sect2>
      <title>Update Files Not Updated by <command>make installworld</command></title>
      
      <para>Remaking the world will not update certain directories (in
	particular, <filename>/etc</filename>, <filename>/var</filename> and
	<filename>/usr</filename>) with new or changed configuration files.</para>

      <para>The simplest way to update these files is to use
        &man.mergemaster.8;, though it is possible to do it manually
        if you would prefer to do that.  Regardless of which way you
	choose, be sure to make a backup of <filename>/etc</filename> in
	case anything goes wrong.</para>

    <sect3 id="mergemaster">
      <sect3info>
	<authorgroup>
	  <author>
	    <firstname>Tom</firstname>
	    <surname>Rhodes</surname>
	    <contrib>Contributed by </contrib>
	  </author>
	</authorgroup>
      </sect3info>
      <title><command>mergemaster</command></title>
        <indexterm><primary><command>mergemaster</command></primary></indexterm>

      <para>The &man.mergemaster.8; utility is a Bourne script that will
        aid you in determining the differences between your configuration files
        in <filename>/etc</filename>, and the configuration files in
        the source tree <filename>/usr/src/etc</filename>. This is
        the recommended solution for keeping the system configuration files up to date
        with those located in the source tree.</para>

      <para><command>mergemaster</command> was integrated into the FreeBSD base
        system between 3.3-RELEASE and 3.4-RELEASE, which means it is
        present in all -STABLE and -CURRENT systems since 3.3.</para>

      <para>To begin simply type <command>mergemaster</command> at your prompt, and
        watch it start going.  <command>mergemaster</command> will then build a
        temporary root environment, from <filename>/</filename> down, and populate
        it with various system configuration files.  Those files are then compared
        to the ones currently installed in your system.  At this point, files that
        differ will be shown in &man.diff.1; format, with the <option>+</option> sign
        representing added or modified lines, and <option>-</option> representing
        lines that will be either removed completely, or replaced with a new line.
        See the &man.diff.1; manual page for more information about the &man.diff.1;
        syntax and how file differences are shown.</para>

      <para>&man.mergemaster.8; will then show you each file that displays variances,
        and at this point you will have the option of either deleting the new file (referred
        to as the temporary file), installing the temporary file in its unmodified state,
        merging the temporary file with the currently installed file, or viewing the
        &man.diff.1; results again.</para>

      <para>Choosing to delete the temporary file will tell &man.mergemaster.8; that we
        wish to keep our current file unchanged, and to delete the new version.
        This option is not recommended, unless you see no
        reason to change the current file.  You can get help at any time by
        typing <keycap>?</keycap> at the &man.mergemaster.8; prompt.  If the user
        chooses to skip a file, it will be presented again after all other files
        have been dealt with.</para>

      <para>Choosing to install the unmodified temporary file will replace the
        current file with the new one.  For most unmodified files, this is the best
        option.</para>

      <para>Choosing to merge the file will present you with a text editor,
        and the contents of both files.  You can now merge them by
        reviewing both files side by side on the screen, and choosing parts from
        both to create a finished product.  When the files are compared side by side,
        the <keycap>l</keycap> key will select the left contents and the
        <keycap>r</keycap> key will select contents from your right.
        The final output will be a file consisting of both parts, which can then be
        installed.  This option is customarily used for files where settings have been
        modified by the user.</para>

      <para>Choosing to view the &man.diff.1; results again will show you the file differences
        just like &man.mergemaster.8; did before prompting you for an option.</para>

      <para>After &man.mergemaster.8; is done with the system files you will be
        prompted for other options.  &man.mergemaster.8; may ask if you want to rebuild
        the password file and/or run &man.MAKEDEV.8; if you run a FreeBSD version prior to 5.0, and will finish up with an option to
        remove left-over temporary files.</para>
      </sect3>

      <sect3>
	<title>Manual Update</title>

      <para>If you wish to do the update manually, however,
        you cannot just copy over the files from
	<filename>/usr/src/etc</filename> to <filename>/etc</filename> and
	have it work.  Some of these files must be <quote>installed</quote>
	first.  This is because the <filename>/usr/src/etc</filename>
	directory <emphasis>is not</emphasis> a copy of what your
	<filename>/etc</filename> directory should look like.  In addition,
	there are files that should be in <filename>/etc</filename> that are
	not in <filename>/usr/src/etc</filename>.</para>

      <para>If you are using &man.mergemaster.8; (as recommended),
        you can skip forward to the <link linkend="update-dev">next
	section</link>.</para>

      <para>The simplest way to do this by hand is to install the
	files into a new directory, and then work through them looking
	for differences.</para>
    
      <warning>
	<title>Backup Your Existing <filename>/etc</filename></title>

	<para>Although, in theory, nothing is going to touch this directory
	  automatically, it is always better to be sure.  So copy your
	  existing <filename>/etc</filename> directory somewhere safe.
	  Something like:</para>

	<screen>&prompt.root; <userinput>cp -Rp /etc /etc.old</userinput></screen>

	<para><option>-R</option> does a recursive copy, <option>-p</option>
	  preserves times, ownerships on files and suchlike.</para>
      </warning>
      
      <para>You need to build a dummy set of directories to install the new
	<filename>/etc</filename> and other files into.
	<filename>/var/tmp/root</filename> is a reasonable choice, and
	there are a number of subdirectories required under this as
	well.</para>

      <screen>&prompt.root; <userinput>mkdir /var/tmp/root</userinput>
&prompt.root; <userinput>cd /usr/src/etc</userinput>
&prompt.root; <userinput>make DESTDIR=/var/tmp/root distrib-dirs distribution</userinput></screen>

      <para>This will build the necessary directory structure and install the
	files.  A lot of the subdirectories that have been created under
	<filename>/var/tmp/root</filename> are empty and should be deleted.
	The simplest way to do this is to:</para>
      
      <screen>&prompt.root; <userinput>cd /var/tmp/root</userinput>
&prompt.root; <userinput>find -d . -type d | xargs rmdir 2&gt;/dev/null</userinput></screen>
      
      <para>This will remove all empty directories.  (Standard error is
        redirected to <filename>/dev/null</filename> to prevent the warnings
        about the directories that are not empty.)</para>

      <para><filename>/var/tmp/root</filename> now contains all the files that
	should be placed in appropriate locations below
	<filename>/</filename>.  You now have to go through each of these
	files, determining how they differ with your existing files.</para>
    
      <para>Note that some of the files that will have been installed in
	<filename>/var/tmp/root</filename> have a leading <quote>.</quote>.  At the
	time of writing the only files like this are shell startup files in
	<filename>/var/tmp/root/</filename> and
	<filename>/var/tmp/root/root/</filename>, although there may be others
	(depending on when you are reading this).  Make sure you use
	<command>ls -a</command> to catch them.</para>
    
      <para>The simplest way to do this is to use &man.diff.1; to compare the
	two files:</para>
    
      <screen>&prompt.root; <userinput>diff /etc/shells /var/tmp/root/etc/shells</userinput></screen>
      
      <para>This will show you the differences between your
	<filename>/etc/shells</filename> file and the new
	<filename>/var/tmp/root/etc/shells</filename> file.  Use these to decide whether to
	merge in changes that you have made or whether to copy over your old
	file.</para>
    
      <tip>
	<title>Name the New Root Directory
	  (<filename>/var/tmp/root</filename>) with a Time Stamp, so You Can
	  Easily Compare Differences Between Versions</title>

	<para>Frequently rebuilding the world means that you have to update
	<filename>/etc</filename> frequently as well, which can be a bit of
	  a chore.</para>

	<para>You can speed this process up by keeping a copy of the last set
	  of changed files that you merged into <filename>/etc</filename>.
	  The following procedure gives one idea of how to do this.</para>

	<procedure>
	  <step>
	    <para>Make the world as normal.  When you want to update
	      <filename>/etc</filename> and the other directories, give the
	      target directory a name based on the current date.  If you were
	      doing this on the 14th of February 1998 you could do the
	      following:</para>
	  
	    <screen>&prompt.root; <userinput>mkdir /var/tmp/root-19980214</userinput>
&prompt.root; <userinput>cd /usr/src/etc</userinput>
&prompt.root; <userinput>make DESTDIR=/var/tmp/root-19980214 \
    distrib-dirs distribution</userinput></screen>
	  </step>
	  
	  <step>
	    <para>Merge in the changes from this directory as outlined
	      above.</para>
	    
	    <para><emphasis>Do not</emphasis> remove the
	      <filename>/var/tmp/root-19980214</filename> directory when you
	      have finished.</para>
	  </step>
	  
	  <step>
	    <para>When you have downloaded the latest version of the source
	      and remade it, follow step 1.  This will give you a new
	      directory, which might be called
	      <filename>/var/tmp/root-19980221</filename> (if you wait a week
	      between doing updates).</para>
	  </step>
	  
	  <step>
	    <para>You can now see the differences that have been made in the
	      intervening week using &man.diff.1; to create a recursive diff
	      between the two directories:</para>
	      
	    <screen>&prompt.root; <userinput>cd /var/tmp</userinput>
&prompt.root; <userinput>diff -r root-19980214 root-19980221</userinput></screen>
	  
	    <para>Typically, this will be a much smaller set of differences
	      than those between
	      <filename>/var/tmp/root-19980221/etc</filename> and
	      <filename>/etc</filename>.  Because the set of differences is
	      smaller, it is easier to migrate those changes across into your
	      <filename>/etc</filename> directory.</para>
	  </step>
	  
	  <step>
	    <para>You can now remove the older of the two
	      <filename>/var/tmp/root-*</filename> directories:</para>
	      
	    <screen>&prompt.root; <userinput>rm -rf /var/tmp/root-19980214</userinput></screen>
	  </step>
	  
	  <step>
	    <para>Repeat this process every time you need to merge in changes
	      to <filename>/etc</filename>.</para>
	  </step>
	</procedure>

	<para>You can use &man.date.1; to automate the generation of the
	  directory names:</para>
	  
	<screen>&prompt.root; <userinput>mkdir /var/tmp/root-`date "+%Y%m%d"`</userinput></screen>
      </tip>
      </sect3>
    </sect2>
  
    <sect2 id="update-dev">
      <title>Update <filename>/dev</filename></title>
      
      <note>
        <indexterm><primary>DEVFS</primary></indexterm>
	<para>If you are running FreeBSD&nbsp;5.0 or later you can safely
	  skip this section.  These versions use &man.devfs.5; to
	  allocate device nodes transparently for the user.</para>
      </note>

      <para>In most cases, the &man.mergemaster.8; tool will realize when
        it is necessary to update the device nodes, and offer to complete it
        automatically.  These instructions tell how to update the device
        nodes manually.</para>
      
      <para>For safety's sake, this is a multi-step process.</para>

      <procedure>
	<step>
	  <para>Copy <filename>/var/tmp/root/dev/MAKEDEV</filename> to
	    <filename>/dev</filename>:</para>

	  <screen>&prompt.root; <userinput>cp /var/tmp/root/dev/MAKEDEV /dev</userinput></screen>
	  <indexterm>
	    <primary><filename>MAKEDEV</filename></primary>
	  </indexterm>

          <para>If you used &man.mergemaster.8; to
            update <filename>/etc</filename>, then your
            <filename>MAKEDEV</filename> script should have been updated
            already, though it cannot hurt to check (with &man.diff.1;)
            and copy it manually if necessary.</para>
	</step>

	<step>
	  <para>Now, take a snapshot of your current
	    <filename>/dev</filename>.  This snapshot needs to contain the
	    permissions, ownerships, major and minor numbers of each filename,
	    but it should not contain the time stamps.  The easiest way to do
	    this is to use &man.awk.1; to strip out some of the
	    information:</para>

	  <screen>&prompt.root; <userinput>cd /dev</userinput>
&prompt.root; <userinput>ls -l | awk '{print $1, $2, $3, $4, $5, $6, $NF}' > /var/tmp/dev.out</userinput></screen>
	</step>

	<step>
	  <para>Remake all the device nodes:</para>
	    
	    <screen>&prompt.root; <userinput>sh MAKEDEV all</userinput></screen>
	</step>

	<step>
	  <para>Write another snapshot of the directory, this time to
	    <filename>/var/tmp/dev2.out</filename>.  Now look through these
	    two files for any device node that you missed creating.  There should
	    not be any, but it is better to be safe than sorry.</para>

	  <screen>&prompt.root; <userinput>diff /var/tmp/dev.out /var/tmp/dev2.out</userinput></screen>

	  <para>You are most likely to notice disk slice discrepancies which
	    will involve commands such as:</para>
	  
	    <screen>&prompt.root; <userinput>sh MAKEDEV sd0s1</userinput></screen>

	  <para>to recreate the slice entries.  Your precise circumstances may
	    vary.</para>
	</step>
      </procedure>
    </sect2>
    
    <sect2>
      <title>Update <filename>/stand</filename></title>
      
      <note>
	<para>This step is included only for completeness.  It can safely be
	  omitted.  If you are using FreeBSD&nbsp;5.2 or later, the
          <filename>/rescue</filename> directory is automatically updated
          for the user with current, statically compiled binaries during
	  <command>make installworld</command>, thus obsoleting the need
	  to update <filename>/stand</filename> (which does not exist at
	  all on FreeBSD&nbsp;5.2 and later).</para>
      </note>
      
      <para>For the sake of completeness, you may want to update the files in
	<filename>/stand</filename> as well.  These files consist of hard
	links to the <filename>/stand/sysinstall</filename> binary.  This
	binary should be statically linked, so that it can work when no other
	file systems (and in particular <filename>/usr</filename>) have been
	mounted.</para>

      <screen>&prompt.root; <userinput>cd /usr/src/release/sysinstall</userinput>
&prompt.root; <userinput>make all install</userinput></screen>
    </sect2>
    
    <sect2>
      <title>Rebooting</title>
      
      <para>You are now done.  After you have verified that everything appears
	to be in the right place you can reboot the system.  A simple
	&man.shutdown.8; should do it:</para>

      <screen>&prompt.root; <userinput>shutdown -r now</userinput></screen>
    </sect2>

    <sect2>
      <title>Finished</title>
      
      <para>You should now have successfully upgraded your &os; system.
	Congratulations.</para>
      
      <para>If things went slightly wrong, it is easy to rebuild a particular
        piece of the system.  For example, if you accidentally deleted
        <filename>/etc/magic</filename> as part of the upgrade or merge of
        <filename>/etc</filename>, the &man.file.1; command will stop working.
        In this case, the fix would be to run:</para>

	<screen>&prompt.root; <userinput>cd /usr/src/usr.bin/file</userinput>
&prompt.root; <userinput>make all install</userinput></screen>
    </sect2>    
    
    <sect2>
      <title>Questions</title>

      <qandaset>
	<qandaentry>
	  <question>
	    <para>Do I need to re-make the world for every change?</para>
	  </question>

	  <answer>
            <para>There is no easy answer to this one, as it depends on the
	      nature of the change.  For example, if you just ran <application>CVSup</application>, and
	      it has shown the following files as being updated:</para>
      
	    <screen><filename>src/games/cribbage/instr.c</filename>
<filename>src/games/sail/pl_main.c</filename>
<filename>src/release/sysinstall/config.c</filename>
<filename>src/release/sysinstall/media.c</filename>
<filename>src/share/mk/bsd.port.mk</filename></screen>

	    <para>it probably is not worth rebuilding the entire world.
	      You could just go to the appropriate sub-directories and
	      <command>make all install</command>, and that's about it.  But
	      if something major changed, for example
	      <filename>src/lib/libc/stdlib</filename> then you should either
	      re-make the world, or at least those parts of it that are
	      statically linked (as well as anything else you might have added
	      that is statically linked).</para>
      
	    <para>At the end of the day, it is your call.  You might be happy
	      re-making the world every fortnight say, and let changes
	      accumulate over that fortnight.  Or you might want to re-make
	      just those things that have changed, and be confident you can
	      spot all the dependencies.</para>
      
	    <para>And, of course, this all depends on how often you want to
	      upgrade, and whether you are tracking &os.stable; or
	      &os.current;.</para>
	  </answer>
	</qandaentry>

	<qandaentry>
	  <question>
	    <para>My compile failed with lots of signal 11 (or other signal
	      number) errors.  What has happened?</para>
	  </question>
    <indexterm><primary>signal 11</primary></indexterm>

	  <answer>

	    <para>This is normally indicative of hardware problems.
	      (Re)making the world is an effective way to stress test your
	      hardware, and will frequently throw up memory problems.  These
	      normally manifest themselves as the compiler mysteriously dying
	      on receipt of strange signals.</para>
      
	    <para>A sure indicator of this is if you can restart the make and
	      it dies at a different point in the process.</para>
      
	    <para>In this instance there is little you can do except start
	      swapping around the components in your machine to determine
	      which one is failing.</para>
	  </answer>
	</qandaentry>

	<qandaentry>
	  <question>
	    <para>Can I remove <filename>/usr/obj</filename> when I have
	      finished?</para>
	  </question>
	  
	  <answer>
	    <para>The short answer is yes.</para>
      
	    <para><filename>/usr/obj</filename> contains all the object files
	      that were produced during the compilation phase.  Normally, one
	      of the first steps in the <command>make buildworld</command> process is to
	      remove this directory and start afresh.  In this case, keeping
	      <filename>/usr/obj</filename> around after you have finished
	      makes little sense, and will free up a large chunk of disk space
	      (currently about 340&nbsp;MB).</para>
      
	    <para>However, if you know what you are doing you can have
	      <command>make buildworld</command> skip this step.  This will make subsequent
	      builds run much faster, since most of sources will not need to
	      be recompiled.  The flip side of this is that subtle dependency
	      problems can creep in, causing your build to fail in odd ways.
	      This frequently generates noise on the &os; mailing lists,
	      when one person complains that their build has failed, not
	      realizing that it is because they have tried to cut
	      corners.</para>
	  </answer>
	</qandaentry>

	<qandaentry>
	  <question>
	    <para>Can interrupted builds be resumed?</para>
	  </question>

	  <answer>
	    <para>This depends on how far through the process you got before
	      you found a problem.</para>

	    <para><emphasis>In general</emphasis> (and this is not a hard and
	      fast rule) the <command>make buildworld</command> process builds new
	      copies of essential tools (such as &man.gcc.1;, and
	      &man.make.1;) and the system libraries.  These tools and
	      libraries are then installed.  The new tools and libraries are
	      then used to rebuild themselves, and are installed again. The
	      entire system (now including regular user programs, such as
		&man.ls.1; or &man.grep.1;) is then rebuilt with the new
	      system files.</para>

	    <para>If you are at the last stage, and you know it (because you
	      have looked through the output that you were storing) then you
	      can (fairly safely) do:</para>

	    <screen><emphasis>&hellip; fix the problem &hellip;</emphasis>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make -DNOCLEAN all</userinput></screen>

	    <para>This will not undo the work of the previous
	      <command>make buildworld</command>.</para>

	    <para>If you see the message:</para>

	      <screen>--------------------------------------------------------------
Building everything..
--------------------------------------------------------------</screen>

	    <para>in the <command>make buildworld</command> output then it is
	      probably fairly safe to do so.</para>
	    
	    <para>If you do not see that message, or you are not sure, then it
	      is always better to be safe than sorry, and restart the build
	      from scratch.</para>
	  </answer>
	</qandaentry>

	<qandaentry>
	  <question>
	    <para>How can I speed up making the world?</para>
          </question>

          <answer>
	    <itemizedlist>
	      <listitem>
		<para>Run in single user mode.</para>
	      </listitem>
	      
	      <listitem>
		<para>Put the <filename>/usr/src</filename> and
		  <filename>/usr/obj</filename> directories on separate
		  file systems held on separate disks.  If possible, put these
		  disks on separate disk controllers.</para>
	      </listitem>
	      
	      <listitem>
		<para>Better still, put these file systems across multiple
		  disks using the &man.ccd.4; (concatenated disk
		  driver) device.</para>
	      </listitem>
	      
	      <listitem>
		<para>Turn off profiling (set <quote>NOPROFILE=true</quote> in
		  <filename>/etc/make.conf</filename>).  You almost certainly
		  do not need it.</para>
	      </listitem>
	      
	      <listitem>
		<para>Also in <filename>/etc/make.conf</filename>, set
		  <makevar>CFLAGS</makevar> to something like <option>-O
		  -pipe</option>.  The optimization <option>-O2</option> is much
		  slower, and the optimization difference between
		  <option>-O</option> and <option>-O2</option> is normally
		  negligible.  <option>-pipe</option> lets the compiler use
		  pipes rather than temporary files for communication, which
		  saves disk access (at the expense of memory).</para>
	      </listitem>
	      
	      <listitem>
		<para>Pass the <option>-j<replaceable>n</replaceable></option> option to &man.make.1; to
		  run multiple processes in parallel.  This usually helps 
		  regardless of whether you have a single or a multi processor
		  machine.</para>
	      </listitem>
	      
	      <listitem><para>The file system holding
		  <filename>/usr/src</filename> can be mounted (or remounted)
		  with the <option>noatime</option> option.  This prevents the
		  file system from recording the file access time.  You probably
		  do not need this information anyway.</para>
		  
		  <screen>&prompt.root; <userinput>mount -u -o noatime /usr/src</userinput></screen>
		  
		  <warning>
		    <para>The example assumes <filename>/usr/src</filename> is
		      on its own file system.  If it is not (if it is a part of
		      <filename>/usr</filename> for example) then you will
		      need to use that file system mount point, and not
		      <filename>/usr/src</filename>.</para>
		  </warning>
	      </listitem>
	      
	      <listitem>
		<para>The file system holding <filename>/usr/obj</filename> can
		  be mounted (or remounted) with the <option>async</option>
		  option.  This causes disk writes to happen asynchronously.
		  In other words, the write completes immediately, and the
		  data is written to the disk a few seconds later.  This
		  allows writes to be clustered together, and can be a
		  dramatic performance boost.</para>

		<warning>
		  <para>Keep in mind that this option makes your file system
		    more fragile.  With this option there is an increased
		    chance that, should power fail, the file system will be in
		    an unrecoverable state when the machine restarts.</para>
	   
		  <para>If <filename>/usr/obj</filename> is the only thing on
		    this file system then it is not a problem.  If you have
		    other, valuable data on the same file system then ensure
		    your backups are fresh before you enable this
		    option.</para>
		</warning>
		
		<screen>&prompt.root; <userinput>mount -u -o async /usr/obj</userinput></screen>
		
		<warning>
		  <para>As above, if <filename>/usr/obj</filename> is not on
		    its own file system, replace it in the example with the
		    name of the appropriate mount point.</para>
		</warning>
	      </listitem>
	    </itemizedlist>
	  </answer>
	</qandaentry>

        <qandaentry>
          <question>
            <para>What do I do if something goes wrong?</para>
          </question>

          <answer>
            <para>Make absolutely sure your environment has no
              extraneous cruft from earlier builds.  This is simple
              enough.</para>

            <screen>&prompt.root; <userinput>chflags -R noschg /usr/obj/usr</userinput>
&prompt.root; <userinput>rm -rf /usr/obj/usr</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make cleandir</userinput>
&prompt.root; <userinput>make cleandir</userinput></screen>

            <para>Yes, <command>make cleandir</command> really should
              be run twice.</para>

            <para>Then restart the whole process, starting
              with <command>make buildworld</command>.</para>

            <para>If you still have problems, send the error and the
              output of <command>uname -a</command> to &a.questions;.
              Be prepared to answer other questions about your
              setup!</para>
          </answer>
        </qandaentry>
      </qandaset>
    </sect2>
  </sect1>

  <sect1 id="small-lan">
    <sect1info>
      <authorgroup>
	<author>
	  <firstname>Mike</firstname>
	  <surname>Meyer</surname>
	  <contrib>Contributed by </contrib>
	</author>
      </authorgroup>
    </sect1info>
    <title>Tracking for Multiple Machines</title>
    <indexterm>
      <primary>NFS</primary>
      <secondary>installing multiple machines</secondary>
    </indexterm>
    
    <para>If you have multiple machines that you want to track the
      same source tree, then having all of them download sources and
      rebuild everything seems like a waste of resources: disk space,
      network bandwidth, and CPU cycles.  It is, and the solution is
      to have one machine do most of the work, while the rest of the
      machines mount that work via NFS.  This section outlines a
      method of doing so.</para>

    <sect2 id="small-lan-preliminaries">
      <title>Preliminaries</title>

      <para>First, identify a set of machines that is going to run
	the same set of binaries, which we will call a
	<emphasis>build set</emphasis>.  Each machine can have a
	custom kernel, but they will be running the same userland
	binaries.  From that set, choose a machine to be the
	<emphasis>build machine</emphasis>.  It is going to be the
	machine that the world and kernel are built on. Ideally, it
	should be a fast machine that has sufficient spare CPU to
	run <command>make buildworld</command> and
	<command>make buildkernel</command>.  You will also want to
	choose a machine to be the <emphasis>test
	machine</emphasis>, which will test software updates before they
	are put into production.  This <emphasis>must</emphasis> be a
	machine that you can afford to have down for an extended
	period of time.  It can be the build machine, but need not be.</para>

      <para>All the machines in this build set need to mount
	<filename>/usr/obj</filename> and
	<filename>/usr/src</filename> from the same machine, and at
	the same point.  Ideally, those are on two different drives
	on the build machine, but they can be NFS mounted on that machine
	as well.  If you have multiple build sets,
	<filename>/usr/src</filename> should be on one build machine, and
	NFS mounted on the rest.</para>
	  
      <para>Finally make sure that
	<filename>/etc/make.conf</filename> on all the machines in
	the build set agrees with the build machine.  That means that
	the build machine must build all the parts of the base
	system that any machine in the build set is going to
	install.  Also, each build machine should have its kernel
	name set with <makevar>KERNCONF</makevar> in
	<filename>/etc/make.conf</filename>, and the build machine
	should list them all in <makevar>KERNCONF</makevar>, listing
	its own kernel first.  The build machine must have the kernel
	configuration files for each machine in
	<filename>/usr/src/sys/<replaceable>arch</replaceable>/conf</filename>
	if it is going to build their kernels.</para>
    </sect2>

    <sect2>
      <title>The Base System</title>

      <para>Now that all that is done, you are ready to build
	everything.  Build the kernel and world as described in <xref
	linkend="make-buildworld"> on the build machine,
	but do not install anything.  After the build has finished, go
	to the test machine, and install the kernel you just
	built.  If this machine mounts <filename>/usr/src</filename>
	and <filename>/usr/obj</filename> via NFS, when you reboot
	to single user you will need to enable the network and mount
	them.  The easiest way to do this is to boot to multi-user,
	then run <command>shutdown now</command> to go to single user
	mode.  Once there, you can install the new kernel and world and run
	<command>mergemaster</command> just as you normally would.  When
	done, reboot to return to normal multi-user operations for this
	machine.</para>

      <para>After you are certain that everything on the test
	machine is working properly, use the same procedure to
	install the new software on each of the other machines in
	the build set.</para>
    </sect2>

    <sect2>
      <title>Ports</title>

      <para>The same ideas can be used for the ports tree.  The first
	critical step is mounting <filename>/usr/ports</filename> from
	the same machine to all the machines in the build set.  You can
	then set up <filename>/etc/make.conf</filename> properly to share
	distfiles.  You should set <makevar>DISTDIR</makevar> to a
	common shared directory that is writable by whichever user
	<username>root</username> is mapped to by your NFS mounts.  Each
	machine should set <makevar>WRKDIRPREFIX</makevar> to a
	local build directory.  Finally, if you are going to be
	building and distributing packages, you should set
	<makevar>PACKAGES</makevar> to a directory similar to
	<makevar>DISTDIR</makevar>.</para> 
    </sect2>
  </sect1>
</chapter>
<!-- 
     Local Variables:
     mode: sgml
     sgml-declaration: "../chapter.decl"
     sgml-indent-data: t
     sgml-omittag: nil
     sgml-always-quote-attributes: t
     sgml-parent-document: ("../book.sgml" "part" "chapter")
     End:
-->

