896 lines
35 KiB
Text
896 lines
35 KiB
Text
<!--
|
|
The FreeBSD Documentation Project
|
|
-->
|
|
|
|
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
|
|
<!ENTITY branches.ascii SYSTEM "branches.ascii">
|
|
<!ENTITY % authors PUBLIC "-//FreeBSD//ENTITIES DocBook Author Entities//EN">
|
|
%authors;
|
|
<!ENTITY % teams PUBLIC "-//FreeBSD//ENTITIES DocBook Team Entities//EN">
|
|
%teams;
|
|
<!ENTITY % mailing-lists PUBLIC "-//FreeBSD//ENTITIES DocBook Mailing List Entities//EN">
|
|
%mailing-lists;
|
|
<!ENTITY % man PUBLIC "-//FreeBSD//ENTITIES DocBook Manual Page Entities//EN">
|
|
%man;
|
|
<!ENTITY % freebsd PUBLIC "-//FreeBSD//ENTITIES DocBook Miscellaneous FreeBSD Entities//EN">
|
|
%freebsd;
|
|
<!ENTITY art.re.pkgs '<ulink url="../releng-packages/article.html">The Release Engineering of Third Party Packages</ulink>'>
|
|
]>
|
|
|
|
<article>
|
|
<title>FreeBSD Release Engineering</title>
|
|
<articleinfo>
|
|
|
|
<!-- This paper was presented at BSDCon Europe in Brighton, UK on
|
|
November 11, 2001 -->
|
|
<confgroup>
|
|
<confdates>November 2001</confdates>
|
|
<conftitle>BSDCon Europe</conftitle>
|
|
</confgroup>
|
|
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Murray</firstname>
|
|
<surname>Stokely</surname>
|
|
<authorblurb>
|
|
<para>I've been involved in the development of FreeBSD based products
|
|
since 1997 at Walnut Creek CDROM, BSDi, and now Wind River Systems.
|
|
FreeBSD 4.4 was the first official release of FreeBSD that I played
|
|
a significant part in.</para>
|
|
</authorblurb>
|
|
<affiliation>
|
|
<address><email>murray@FreeBSD.org</email>
|
|
<otheraddr><ulink
|
|
url="http://www.FreeBSD.org/~murray">http://www.FreeBSD.org/~murray</ulink></otheraddr>
|
|
</address>
|
|
</affiliation>
|
|
</author>
|
|
</authorgroup>
|
|
|
|
<pubdate>$FreeBSD$</pubdate>
|
|
<abstract>
|
|
<para>This paper describes the approach used by the FreeBSD
|
|
release engineering team to make production quality releases
|
|
of the FreeBSD Operating System. It details the methodology
|
|
used for the release of FreeBSD 4.4 and describes the tools
|
|
available for those interested in producing customized FreeBSD
|
|
releases for corporate rollouts or commercial
|
|
productization.</para>
|
|
</abstract>
|
|
|
|
</articleinfo>
|
|
|
|
<!-- Introduction -->
|
|
<sect1 id="introduction">
|
|
<title>Introduction</title>
|
|
|
|
<para>The development of FreeBSD is a very open process. FreeBSD is
|
|
comprised of contributions from thousands of people around the
|
|
world. The FreeBSD Project provides anonymous
|
|
<acronym>CVS</acronym>[1] access to the general public so that
|
|
others can have access to log messages, diffs (patches) between
|
|
development branches, and other productivity enhancements that
|
|
formal source code management provides. This has been a huge help
|
|
in attracting more talented developers to FreeBSD. However, I
|
|
think everyone would agree that chaos would soon manifest if write
|
|
access was opened up to everyone on the Internet. Therefore only
|
|
a <quote>select</quote> group of nearly 300 people are given write
|
|
access to the <acronym>CVS</acronym> repository. These
|
|
<emphasis>committers[6]</emphasis> are responsible for the bulk of
|
|
FreeBSD development. An elected <emphasis>core-team[7]</emphasis>
|
|
of very senior developers provides some level of direction over
|
|
the project.</para>
|
|
|
|
<para>The rapid pace of <systemitem
|
|
class="osname">FreeBSD</systemitem> development leaves little time
|
|
for polishing the development system into a production quality
|
|
release. To solve this dilemma, development continues on two
|
|
parallel tracks. The main development branch is the
|
|
<emphasis>HEAD</emphasis> or <emphasis>trunk</emphasis> of our CVS
|
|
tree, known as <quote>FreeBSD-CURRENT</quote> or
|
|
<quote>-CURRENT</quote> for short.</para>
|
|
|
|
<para>A more stable branch is maintained, known as
|
|
<quote>FreeBSD-STABLE</quote> or <quote>-STABLE</quote> for short.
|
|
Both branches live in a master CVS repository in California and
|
|
are replicated via <application
|
|
class="software">CVSup</application>[2] to mirrors all over the
|
|
world. FreeBSD-CURRENT[8] is the <quote>bleeding-edge</quote> of
|
|
FreeBSD development where all new changes first enter the system.
|
|
FreeBSD-STABLE is the development branch from which major releases
|
|
are made. Changes go into this branch at a different pace, and
|
|
with general assumption that they have first gone into
|
|
FreeBSD-CURRENT and have been thoroughly tested by our user
|
|
community.</para>
|
|
|
|
<para>In the interim period between releases, nightly snapshots are
|
|
built automatically by the FreeBSD Project build machines and made
|
|
available for download from <systemitem
|
|
class="resource">ftp://stable.FreeBSD.org/</systemitem>. The
|
|
widespread availability of binary release snapshots, and the
|
|
tendency of our user community to keep up with -STABLE development
|
|
with CVSup and <quote><command>make</command>
|
|
<MakeTarget>world</MakeTarget></quote>[8] helps to keep
|
|
FreeBSD-STABLE in a very reliable condition even before the
|
|
quality assurance activities ramp up pending a major
|
|
release.</para>
|
|
|
|
<para>Bug reports and feature requests are continuously submitted by
|
|
users throughout the release cycle. Problems reports are entered into our
|
|
<application class="software">GNATS</application>[9] database
|
|
through email, the &man.send-pr.1; application, or via the web
|
|
interface provided at <ulink
|
|
url="http://www.FreeBSD.org/send-pr.html"></ulink>
|
|
In addition to the multitude of different technical mailing lists
|
|
about FreeBSD, the &a.qa; provides a forum for discussing the finer
|
|
points of <quote>release-polishing</quote>.</para>
|
|
|
|
<para>To service our most conservative users, individual release
|
|
branches were introduced with FreeBSD 4.3.
|
|
These release branches are created shortly before a final release
|
|
is made. After the release goes out, only the most critical
|
|
security fixes and additions are merged onto the release branch.
|
|
In addition to source updates via CVS, binary patchkits are
|
|
available to keep systems on the <emphasis>RELENG_4_3 </emphasis>
|
|
and <emphasis>RELENG_4_4</emphasis> branches updated.</para>
|
|
|
|
<para><xref linkend="release-proc"> discusses the
|
|
different phases of the release engineering process leading up to
|
|
the actual system build and <xref linkend="release-build">
|
|
describes the actual build process. <xref
|
|
linkend="extensibility"> describes how the base
|
|
release may be extended by third parties and <xref
|
|
linkend="lessons-learned"> details some of the
|
|
lessons learned through the release of FreeBSD 4.4. Finally,
|
|
<xref linkend="future"> presents future directions
|
|
of development.</para>
|
|
</sect1>
|
|
|
|
<!-- Release Process -->
|
|
<sect1 id="release-proc">
|
|
<title>Release Process</title>
|
|
|
|
<para>New releases of FreeBSD are released from the -STABLE branch
|
|
at approximately four month intervals. The FreeBSD release
|
|
process begins to ramp up 45 days before the anticipated release
|
|
date when the release engineer sends an email to the development
|
|
mailing lists to remind developers that they only have 15 days to
|
|
integrate new changes before the code freeze. During this time,
|
|
many developers perform what have become know as <quote>MFC
|
|
sweeps</quote>. <acronym>MFC</acronym> stands for <quote>Merge
|
|
From CURRENT</quote> and it describes the process of merging a
|
|
tested change from our -CURRENT development branch to our -STABLE
|
|
branch.</para>
|
|
|
|
<sect2>
|
|
<title>Code Review</title>
|
|
|
|
<para>Thirty days before the anticipated release, the source
|
|
repository enters a <quote>code slush</quote>. During this
|
|
time, all commits to the -STABLE branch must be approved by the
|
|
&a.re;. The kinds of changes that are allowed during this 15 day
|
|
period include:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Bug fixes.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Documentation updates.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Security-related fixes of any kind.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Minor changes to device drivers, such as adding new Device
|
|
IDs.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Any additional change that the release engineering team feels
|
|
is justified, given the potential risk.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>After the first 15 days of the code slush, a
|
|
<emphasis>release candidate</emphasis> is released for
|
|
widespread testing and the code enters a <quote>code
|
|
freeze</quote> where it becomes much harder to justify new
|
|
changes to the system unless a serious bug-fix or security issue
|
|
is involved. During the code freeze, at least one release
|
|
candidate is released per week, until the final release is
|
|
ready. During the days leading to the final release, the
|
|
release engineering team is in constant communication with the
|
|
security-officer team, the documentation maintainers, and the
|
|
port maintainers, to ensure that all of the
|
|
different components required for a successful release are
|
|
available.</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Final Release Checklist</title>
|
|
|
|
<para>When several release candidates have been made available for
|
|
widespread testing and all major issues have been resolved, the
|
|
final release <quote>polishing</quote> can begin.</para>
|
|
|
|
<sect3>
|
|
<title>Creating the Release Branch</title>
|
|
|
|
<para>As described in the introduction, the <literal>RELENG_X_Y</literal> release
|
|
branch is a relatively new addition to our release engineering
|
|
methodology. The first step in creating this branch is to
|
|
ensure that you are working with the newest version of the
|
|
<literal>RELENG_X</literal> sources that you want to branch
|
|
<emphasis>from</emphasis>.</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs update -rRELENG_4 -P -d</userinput></screen>
|
|
|
|
<para>The next step is to create a branch point
|
|
<emphasis>tag</emphasis>, so that diffs against the start of
|
|
the branch are easier with CVS:</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs rtag -rRELENG_4 RELENG_4_4_BP src</userinput></screen>
|
|
|
|
<para>And then a new branch tag is created with:</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs rtag -b -rRELENG_4_4_BP RELENG_4_4 src</userinput></screen>
|
|
|
|
<note>
|
|
<para><emphasis>The <literal>RELENG_*</literal> tags are
|
|
restricted for use by the CVS-meisters and release
|
|
engineers.</emphasis></para>
|
|
</note>
|
|
|
|
<sidebar>
|
|
<para>A <quote><emphasis>tag</emphasis></quote> is CVS
|
|
vernacular for a label that identifies the source at a specific point
|
|
in time. By tagging the tree, we ensure that future release builders
|
|
will always be able to use the same source we used to create the
|
|
official FreeBSD Project releases.</para>
|
|
</sidebar>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches" align="center">
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<literallayout>
|
|
&branches.ascii;
|
|
</literallayout>
|
|
</textobject>
|
|
|
|
<textobject>
|
|
<phrase>FreeBSD Development Branches</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
</sect3>
|
|
|
|
<sect3 id="versionbump">
|
|
<title>Bumping up the Version Number</title>
|
|
|
|
<para>Before the final release can be tagged, built, and
|
|
released, the following files need to be modified to reflect
|
|
the correct version of FreeBSD:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/share/sgml/freebsd.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/Makefile.inc</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/UPDATING</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/gnu/usr.bin/groff/tmac/mdoc.local</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/Makefile</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/share/sgml/release.dsl</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/examples/Makefile.relnotesng</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/sgml/release.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/conf/newvers.sh</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/sys/param.h</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/releases/*</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/docs.sgml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/cgi/ports.cgi</filename></para>
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
<para>The release notes and errata files also need to be adjusted for the
|
|
new release (on the release branch) and truncated appropriately
|
|
(on the stable/current branch):</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/relnotes/common/new.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/errata/article.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para><application>Sysinstall</application> should be updated to note
|
|
the number of available ports and the amount of disk space required
|
|
for the Ports Collection. This information is currently kept in
|
|
<filename>src/release/sysinstall/dist.c</filename>.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Creating Release Tags</title>
|
|
|
|
<para>When the final release is ready, the following command
|
|
will create the <literal>RELENG_4_4_0_RELEASE</literal>
|
|
tag.</para>
|
|
|
|
<screen>
|
|
/usr/src&prompt.root; <userinput>cvs rtag -rRELENG_4_4 RELENG_4_4_0_RELEASE src</userinput>
|
|
</screen>
|
|
|
|
<para>The Documentation and Ports managers are responsible for
|
|
tagging the respective trees with the <literal>RELEASE_4_4_0</literal>
|
|
tag.</para>
|
|
|
|
<para>Occasionally, a last minute fix may be required
|
|
<emphasis>after</emphasis> the final tags have been created.
|
|
In practice this isn't a problem, since <acronym>CVS</acronym>
|
|
allows tags to be manipulated with <command>cvs
|
|
tag -d <replaceable>tagname filename</replaceable></command>.
|
|
It is very important that any last minute changes be tagged
|
|
appropriately as part of the release. FreeBSD releases must
|
|
always be reproduceable. Local hacks in the release
|
|
engineer's environment are not acceptable.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Building -->
|
|
<sect1 id="release-build">
|
|
<title>Release Building</title>
|
|
|
|
<para>FreeBSD <quote>releases</quote> can be built by anyone with a
|
|
fast machine and access to a source repository. (That should be
|
|
everyone, since we offer anonymous CVS! See The Handbook for
|
|
details.). The <emphasis>only</emphasis> special requirement is
|
|
that the <devicename>vn</devicename> (<emphasis>On -CURRENT, this
|
|
device has been replaced by the new <devicename>md</devicename>
|
|
memory disk driver </emphasis>.) device must be available. If the
|
|
device is not loaded into your kernel, then the kernel module
|
|
should be automatically loaded when &man.vnconfig.8; is executed
|
|
during the boot media creation phase. All of the tools necessary
|
|
to build a release are available from the CVS repository in
|
|
<filename>src/release</filename>. These tools aim to provide a
|
|
consistent way to build FreeBSD releases. A complete release can
|
|
actually be built with only a single command, including the
|
|
creation of <acronym>ISO</acronym> images suitable for burning to
|
|
CDROM, installation floppies, and an FTP install directory. This
|
|
command is aptly named <quote><command>make
|
|
release</command></quote>.</para>
|
|
|
|
<sect2>
|
|
<title><quote>make release</quote></title>
|
|
|
|
<para>To successfully build a release, you must first populate
|
|
<filename>/usr/obj</filename> by running <quote><command>make
|
|
world</command></quote> or simply
|
|
<quote><command>make
|
|
buildworld</command></quote>. The release
|
|
target requires several variables be set properly to build a
|
|
release:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><makevar>CHROOTDIR</makevar> - The directory to be used as the
|
|
chroot environment for the entire release build.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><makevar>BUILDNAME</makevar> - The name of the release to be
|
|
built.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><makevar>CVSROOT</makevar> - The location of a CVS Repository.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><makevar>RELEASETAG</makevar> - The CVS tag corresponding to the
|
|
release you would like to build.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>If you do not already have access to a local CVS
|
|
repository, then you may mirror one with <ulink
|
|
url="http://www.FreeBSD.org/handbook/synching.html#CVSUP">CVSup</ulink>.
|
|
The supplied supfile,
|
|
<filename>/usr/share/examples/cvsup/cvs-supfile</filename>, is
|
|
a useful starting point for mirroring the CVS
|
|
repository.</para>
|
|
|
|
<para>If <makevar>RELEASETAG</makevar> is omitted, then the
|
|
release will be built from the HEAD (a.k.a. -CURRENT) branch.
|
|
Releases built from this branch are normally referred to as
|
|
<quote>-CURRENT snapshots</quote>.</para>
|
|
|
|
<para>There are many other variables available to customize the
|
|
release build. Most of these variables are documented at the
|
|
top of <filename>src/release/Makefile</filename>. The exact
|
|
command used to build the official FreeBSD 4.4 (x86) release
|
|
was:</para>
|
|
|
|
<screen><command>make <literal>release CHROOTDIR=/local3/release \
|
|
BUILDNAME=4.4-RELEASE \
|
|
CVSROOT=/host/cvs/usr/home/ncvs \
|
|
RELEASETAG=RELENG_4_4_0_RELEASE</literal>
|
|
</command>
|
|
</screen>
|
|
|
|
<para>The release Makefile can be broken down into several distinct
|
|
steps.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Creation of a sanitized system environment in a separate
|
|
directory hierarchy with <quote><command>make
|
|
<literal>installworld</literal></command></quote>.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Checkout from CVS of a clean version of the system source,
|
|
documentation, and ports into the release build hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Population of <filename>/etc</filename> and
|
|
<filename>/dev</filename> in the chrooted
|
|
environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>chroot into the release build hierarchy, to make it harder for
|
|
the outside environment to taint this build.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><quote><command>make world</command></quote>
|
|
in the chrooted environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build of Kerberos-related binaries.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build <quote><filename>GENERIC</filename></quote> kernel.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Creation of a staging directory tree where the binary
|
|
distributions will be built and packaged.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the documentation toolchain needed to
|
|
convert the documentation source (SGML) into HTML and text documents
|
|
that will accompany the release.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the actual documentation
|
|
(user manuals, tutorials, release notes, hardware compatibility lists,
|
|
and so on.)</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build of the <quote>crunched</quote> binaries used for
|
|
installation floppies.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Package up distribution tarballs of the binaries and sources.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Create the boot media and a <quote>fixit</quote> floppy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Create FTP installation hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>(optionally)</emphasis> Create ISO images for
|
|
CDROM/DVD media.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>For more information about the release build infrastructure,
|
|
please see &man.release.7;.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Building XFree86</title>
|
|
|
|
<para>XFree86 is an important component for many desktop users.
|
|
The easiest way to build XFree86 is to use the
|
|
<filename>src/release/scripts/X11/build_x.sh</filename> script.
|
|
This script requires that XFree86 and Tcl/Tk already be
|
|
installed on the build host. After compiling the necessary X
|
|
servers, the script will package all of the files into tarballs
|
|
that &man.sysinstall.8; expects to find in the
|
|
<filename>XF86336</filename> directory of the installation
|
|
media.</para>
|
|
|
|
<note><para>It is important to remove any site-specific settings
|
|
from <filename>/etc/make.conf</filename>. For example, it would
|
|
be unwise to distribute binaries that were built on a system
|
|
with <varname>CPUTYPE</varname> set to a specific
|
|
processor.</para></note>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Contributed Software (<quote>ports</quote>)</title>
|
|
|
|
<para>The <ulink url="http://www.FreeBSD.org/ports">FreeBSD Ports
|
|
collection</ulink> is a collection of over &os.numports;
|
|
third-party software packages available for FreeBSD. The &a.portmgr;
|
|
is responsible for maintaining a consistent ports tree that can be used
|
|
to create the binary packages that accompany official FreeBSD
|
|
releases.</para>
|
|
|
|
<para>The release engineering activities for our collection of
|
|
third-party packages is beyond the scope of this document. A
|
|
separate article, &art.re.pkgs;, covers this topic
|
|
in depth.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Release ISOs</title>
|
|
|
|
<para>Starting with FreeBSD 4.4, the FreeBSD Project decided to
|
|
release all four ISO images that were previously sold on the
|
|
<emphasis>BSDi/Wind River Systems/FreeBSD Mall</emphasis>
|
|
<quote>official</quote> CDROM distributions. Each of the four
|
|
discs must contain a <filename>README.TXT</filename> file that
|
|
explains the contents of the disc, a
|
|
<filename>CDROM.INF</filename> file that provides meta-data for
|
|
the disc so that &man.sysinstall.8; can validate and use the
|
|
contents, and a <filename>filename.txt</filename> file that
|
|
provides a manifest for the disc. This
|
|
<emphasis>manifest</emphasis> can be created with a simple
|
|
command:</para>
|
|
|
|
<screen>/stage/cdrom&prompt.root; <userinput>find . -type f | sed -e 's/^\.\///' | sort > filename.txt</userinput></screen>
|
|
|
|
<para>The specific requirements of each CD are outlined below.</para>
|
|
|
|
<sect3>
|
|
<title>Disc 1</title>
|
|
|
|
<para>The first disc is almost completely created by
|
|
<quote><command>make
|
|
<literal>release</literal></command></quote>. The only changes
|
|
that should be made to the disc1 directory are the addition of
|
|
a <filename>tools</filename> directory, <application
|
|
class="software">XFree86</application>, and as many popular
|
|
third party software packages as will fit on the disc. The
|
|
<filename>tools</filename> directory contains software that allow users to create
|
|
installation floppies from other operating systems. This disc
|
|
should be made bootable so that users of modern PCs do not
|
|
need to create installation floppy disks.</para>
|
|
|
|
<para>If an alternate version of XFree86 is to be provided, then
|
|
&man.sysinstall.8; must be updated to reflect the new location
|
|
and installation instructions. The relevant code is contained
|
|
in <filename>src/release/sysinstall</filename> on -STABLE or
|
|
<filename>src/usr.sbin/sysinstall</filename> on
|
|
-CURRENT. Specifically, the files <filename>dist.c</filename>,
|
|
<filename>menus.c</filename>, and
|
|
<filename>config.c</filename> will need to be updated.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Disc 2</title>
|
|
|
|
<para>The second disc is also largely created by <quote>make
|
|
release</quote>. This disc contains a <quote>live
|
|
filesystem</quote> that can be used from &man.sysinstall.8; to
|
|
troubleshoot a FreeBSD installation. This disc should be
|
|
bootable and should also contain a compressed copy of the CVS
|
|
repository in the <filename>CVSROOT</filename> directory and
|
|
commercial software demos in the <filename>commerce</filename>
|
|
directory.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Discs 3 and 4</title>
|
|
|
|
<para>The remaining two discs contain additional software
|
|
packages for FreeBSD. The packages should be clustered so that
|
|
a package and all of its <emphasis>dependencies</emphasis> are
|
|
included on the same disc. More information about the
|
|
creation of these discs is provided in the &art.re.pkgs;
|
|
article.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Extensibility -->
|
|
<sect1 id="extensibility">
|
|
<title>Extensibility</title>
|
|
|
|
<para>Although FreeBSD forms a complete operating system, there is
|
|
nothing that forces you to use the system exactly as we have
|
|
packaged it up for distribution. We have tried to design the
|
|
system to be as extensible as possible so that it can serve as a
|
|
platform that other commercial products can be built on top
|
|
of. The only <quote>rule</quote> we have about this is that if you
|
|
are going to distribute FreeBSD with non-trivial changes, we
|
|
encourage you to document your enhancements! The FreeBSD community
|
|
can only help support users of the software we provide. We
|
|
certainly encourage innovation in the form of advanced
|
|
installation and administration tools, for example, but we can't
|
|
be expected to answer questions about it.</para>
|
|
|
|
<sect2>
|
|
<title>Creating Customized Boot floppies</title>
|
|
|
|
<para>Many sites have complex requirements that may require
|
|
additional kernel modules or userland tools be added to the
|
|
installation floppies. The <quote>quick and dirty</quote> way
|
|
to accomplish this would be to modify the staging directory of
|
|
an existing <quote>make release</quote> build hierarchy:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Apply patches or add additional files inside the chroot
|
|
release build directory.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>rm
|
|
${CHROOTDIR}/usr/obj/usr/src/release/release.[48]</command></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>rebuild &man.sysinstall.8;, the kernel, or whatever
|
|
parts of the system your change affected.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>chroot ${CHROOTDIR} ./mk release.4
|
|
</command></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>chroot ${CHROOTDIR} ./mk release.8
|
|
</command></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>New release floppies will be located in
|
|
<filename>${CHROOTDIR}/R/stage/floppies</filename>.</para>
|
|
|
|
<para>Alternatively, the
|
|
<quote><filename>boot.flp</filename></quote> make
|
|
target can be called, or the filesystem
|
|
creating script,
|
|
<filename>src/release/scripts/doFS.sh</filename>, may be invoked
|
|
directly.</para>
|
|
|
|
<para>Local patches may also be supplied to the release build by
|
|
defining the <makevar>LOCAL_PATCH</makevar> variable in <quote>make
|
|
release</quote>.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Scripting <command>sysinstall</command></title>
|
|
|
|
<para>The FreeBSD system installation and configuration tool,
|
|
&man.sysinstall.8;, can be scripted to provide automated installs
|
|
for large sites. This functionality can be used in conjunction
|
|
with Intel's PXE[13] to bootstrap systems from the network, or
|
|
via custom boot floppies with a sysinstall script. An example
|
|
sysinstall script is available in the CVS tree as
|
|
<filename>src/release/sysinstall/install.cfg</filename>.</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Lessons Learned -->
|
|
<sect1 id="lessons-learned">
|
|
<title>Lessons Learned from FreeBSD 4.4</title>
|
|
|
|
<para>The release engineering process for 4.4 formally began on
|
|
August 1st, 2001. After that date all commits to the
|
|
<literal>RELENG_4</literal> branch of FreeBSD had to be explicitly
|
|
approved by the &a.re;. The first
|
|
release candidate for the x86 architecture was released on August
|
|
16, followed by 4 more release candidates leading up to the final
|
|
release on September 18th. The security officer was very involved
|
|
in the last week of the process as several security issues were
|
|
found in the earlier release candidates. A total of over
|
|
<emphasis>500</emphasis> emails were sent to the &a.re; in
|
|
little over a month.</para>
|
|
|
|
<para>Our user community has made it very clear that the security
|
|
and stability of a FreeBSD release should not be sacrificed for
|
|
any self-imposed deadlines or target release dates. The FreeBSD
|
|
Project has grown tremendously over its lifetime and the need for
|
|
standardized release engineering procedures has never been more
|
|
apparent. This will become even more important as FreeBSD is
|
|
ported to new platforms.</para>
|
|
</sect1>
|
|
|
|
<!-- Future Directions -->
|
|
<sect1 id="future">
|
|
<title>Future Directions</title>
|
|
|
|
<para>It is imperative for our release engineering activities to
|
|
scale with our growing userbase. Along these lines we are working
|
|
very hard to document the procedures involved in producing FreeBSD
|
|
releases.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><emphasis>Parallelism</emphasis> - Certain portions of the
|
|
release build are actually <quote>embarrassingly
|
|
parallel</quote>. Most of the tasks are very I/O intensive,
|
|
so multiple high-speed disk drives is actually important that
|
|
multiple processors in speeding up the <quote>make
|
|
release</quote> process. If multiple disks are used for
|
|
different hierarchies in the <emphasis>chroot</emphasis>
|
|
environment, then the CVS checkout of the ports and doc trees
|
|
can be happening simultaneously to the <quote>make
|
|
<literal>world</literal></quote> on another disk. Using a
|
|
<acronym>RAID</acronym> solution (hardware or software) can
|
|
significantly decrease the overall build time.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Cross-building releases</emphasis> - Building
|
|
IA-64 or Alpha release on x86 hardware? <quote>make
|
|
TARGET=ia64 release</quote>.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Regression Testing</emphasis> - We need better
|
|
automated correctness testing for FreeBSD.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Installation Tools</emphasis> - Our installation
|
|
program has long since outlived its intended life span.
|
|
Several projects are under development to provide a more
|
|
advanced installation mechanism. One of the most promising is
|
|
the libh project[5] which aims to provided an intelligent new
|
|
package framework and GUI installation program.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</sect1>
|
|
|
|
<!-- Acknowledgements -->
|
|
<sect1 id="ackno">
|
|
<title>Acknowledgements</title>
|
|
|
|
<para>I would like to thank Jordan Hubbard for giving me the
|
|
opportunity to take on some of the release engineering
|
|
responsibilities for FreeBSD 4.4 and also for all of his work
|
|
throughout the years making FreeBSD what it is today. Of course
|
|
the release wouldn't have been possible without all of the
|
|
release-related work done by &a.asami;, &a.steve;, &a.bmah;, &a.nik;,
|
|
&a.obrien;, &a.kris;, &a.jhb; and the rest of the FreeBSD development
|
|
community. I would also like to thank &a.rgrimes;, &a.phk;, and others
|
|
who worked on the release engineering tools in the very early days
|
|
of FreeBSD. This article was influenced by release engineering
|
|
documents from the CSRG[14], the NetBSD Project[11], and John
|
|
Baldwin's proposed release engineering process notes[12].</para>
|
|
</sect1>
|
|
|
|
<!-- Reference / Biblio Section -->
|
|
<sect1 id="biblio">
|
|
<title>References</title>
|
|
<para>[1] CVS - Concurrent Versions System
|
|
<ulink url="http://www.cvshome.org"></ulink></para>
|
|
|
|
<para>[2] CVSup - The CVS-Optimized General Purpose Network File Distribution
|
|
System <ulink url="http://www.polstra.com/projects/freeware/CVSup"></ulink>
|
|
</para>
|
|
|
|
<para>[3] <ulink url="http://bento.FreeBSD.org"></ulink></para>
|
|
|
|
<para>[4] FreeBSD Ports Collection
|
|
<ulink url="http://www.FreeBSD.org/ports"></ulink></para>
|
|
|
|
<para>[5] The libh Project
|
|
<ulink url="http://www.FreeBSD.org/projects/libh.html"></ulink></para>
|
|
|
|
<para>[6] FreeBSD Committers <ulink
|
|
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributors/staff-committers.html"></ulink>
|
|
</para>
|
|
|
|
<para>[7] FreeBSD Core-Team
|
|
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributors/staff-core.html"></ulink></para>
|
|
|
|
<para>[8] FreeBSD Handbook
|
|
<ulink url="http://www.FreeBSD.org/handbook"></ulink>
|
|
</para>
|
|
|
|
<para>[9] GNATS: The GNU Bug Tracking System
|
|
<ulink url="http://www.gnu.org/software/gnats"></ulink>
|
|
</para>
|
|
|
|
<para>[10] FreeBSD PR Statistics
|
|
<ulink url="http://www.FreeBSD.org/prstats/index.html"></ulink></para>
|
|
|
|
<para>[11] NetBSD Developer Documentation: Release Engineering
|
|
<ulink url="http://www.NetBSD.org/developers/releng/index.html"></ulink>
|
|
</para>
|
|
|
|
<para>[12] John Baldwin's FreeBSD Release Engineering Proposal
|
|
<ulink url="http://people.FreeBSD.org/~jhb/docs/releng.txt"></ulink>
|
|
</para>
|
|
|
|
<para>[13] PXE Jumpstart Guide
|
|
<ulink
|
|
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/pxe/index.html"></ulink>
|
|
</para>
|
|
|
|
<para>[14] Marshall Kirk McKusick, Michael J. Karels, and Keith Bostic:
|
|
<ulink url="http://docs.freebsd.org/doc/papers/releng.html">
|
|
<emphasis>The Release Engineering of 4.3BSD</emphasis></ulink>
|
|
</para>
|
|
</sect1>
|
|
</article>
|