1148 lines
43 KiB
Text
1148 lines
43 KiB
Text
<!--
|
|
The FreeBSD Documentation Project
|
|
-->
|
|
|
|
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
|
|
<!ENTITY % articles.ent PUBLIC "-//FreeBSD//ENTITIES DocBook FreeBSD Articles Entity Set//EN">
|
|
%articles.ent;
|
|
<!ENTITY art.re.pkgs '<ulink url="&url.articles.releng-packages;/article.html">The Release Engineering of Third Party Packages</ulink>'>
|
|
]>
|
|
|
|
<article>
|
|
<title>FreeBSD Release Engineering</title>
|
|
<articleinfo>
|
|
|
|
<!-- This paper was presented at BSDCon Europe in Brighton, UK on
|
|
November 11, 2001 -->
|
|
<confgroup>
|
|
<confdates>November 2001</confdates>
|
|
<conftitle>BSDCon Europe</conftitle>
|
|
</confgroup>
|
|
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Murray</firstname>
|
|
<surname>Stokely</surname>
|
|
<authorblurb>
|
|
<para>I've been involved in the development of FreeBSD based products
|
|
since 1997 at Walnut Creek CDROM, BSDi, and now Wind River Systems.
|
|
FreeBSD 4.4 was the first official release of FreeBSD that I played
|
|
a significant part in.</para>
|
|
</authorblurb>
|
|
<affiliation>
|
|
<address><email>murray@FreeBSD.org</email>
|
|
<otheraddr><ulink url="http://www.FreeBSD.org/~murray/"></ulink></otheraddr>
|
|
</address>
|
|
</affiliation>
|
|
</author>
|
|
</authorgroup>
|
|
|
|
<pubdate>$FreeBSD$</pubdate>
|
|
|
|
<legalnotice id="trademarks" role="trademarks">
|
|
&tm-attrib.freebsd;
|
|
&tm-attrib.cvsup;
|
|
&tm-attrib.intel;
|
|
&tm-attrib.xfree86;
|
|
&tm-attrib.general;
|
|
</legalnotice>
|
|
|
|
<abstract>
|
|
<para>This paper describes the approach used by the FreeBSD
|
|
release engineering team to make production quality releases
|
|
of the FreeBSD Operating System. It details the methodology
|
|
used for the official FreeBSD releases and describes the tools
|
|
available for those interested in producing customized FreeBSD
|
|
releases for corporate rollouts or commercial
|
|
productization.</para>
|
|
</abstract>
|
|
|
|
</articleinfo>
|
|
|
|
<!-- Introduction -->
|
|
<sect1 id="introduction">
|
|
<title>Introduction</title>
|
|
|
|
<para>The development of FreeBSD is a very open process. FreeBSD is
|
|
comprised of contributions from thousands of people around the
|
|
world. The FreeBSD Project provides anonymous
|
|
<acronym>CVS</acronym>[1] access to the general public so that
|
|
others can have access to log messages, diffs (patches) between
|
|
development branches, and other productivity enhancements that
|
|
formal source code management provides. This has been a huge help
|
|
in attracting more talented developers to FreeBSD. However, I
|
|
think everyone would agree that chaos would soon manifest if write
|
|
access was opened up to everyone on the Internet. Therefore only
|
|
a <quote>select</quote> group of nearly 300 people are given write
|
|
access to the <acronym>CVS</acronym> repository. These
|
|
<emphasis>committers[5]</emphasis> are responsible for the bulk of
|
|
FreeBSD development. An elected <emphasis>core-team[6]</emphasis>
|
|
of very senior developers provides some level of direction over
|
|
the project.</para>
|
|
|
|
<para>The rapid pace of <systemitem
|
|
class="osname">FreeBSD</systemitem> development leaves little time
|
|
for polishing the development system into a production quality
|
|
release. To solve this dilemma, development continues on two
|
|
parallel tracks. The main development branch is the
|
|
<emphasis>HEAD</emphasis> or <emphasis>trunk</emphasis> of our CVS
|
|
tree, known as <quote>FreeBSD-CURRENT</quote> or
|
|
<quote>-CURRENT</quote> for short.</para>
|
|
|
|
<para>A more stable branch is maintained, known as
|
|
<quote>FreeBSD-STABLE</quote> or <quote>-STABLE</quote> for short.
|
|
Both branches live in a master CVS repository in California and
|
|
are replicated via <application
|
|
class="software">CVSup</application>[2] to mirrors all over the
|
|
world. FreeBSD-CURRENT[7] is the <quote>bleeding-edge</quote> of
|
|
FreeBSD development where all new changes first enter the system.
|
|
FreeBSD-STABLE is the development branch from which major releases
|
|
are made. Changes go into this branch at a different pace, and
|
|
with the general assumption that they have first gone into
|
|
FreeBSD-CURRENT and have been thoroughly tested by our user
|
|
community.</para>
|
|
|
|
<para>In the interim period between releases, monthly snapshots are
|
|
built automatically by the FreeBSD Project build machines and made
|
|
available for download from <systemitem
|
|
class="resource">ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/</systemitem>.
|
|
The widespread availability of binary release snapshots, and the
|
|
tendency of our user community to keep up with -STABLE development
|
|
with CVSup and <quote><command>make</command>
|
|
<maketarget>world</maketarget></quote>[7] helps to keep
|
|
FreeBSD-STABLE in a very reliable condition even before the
|
|
quality assurance activities ramp up pending a major
|
|
release.</para>
|
|
|
|
<para>Bug reports and feature requests are continuously submitted by
|
|
users throughout the release cycle. Problems reports are entered into our
|
|
<application class="software">GNATS</application>[8] database
|
|
through email, the &man.send-pr.1; application, or via the web
|
|
interface provided at <ulink
|
|
url="http://www.FreeBSD.org/send-pr.html"></ulink>.
|
|
In addition to the multitude of different technical mailing lists
|
|
about FreeBSD, the &a.qa; provides a forum for discussing the finer
|
|
points of <quote>release-polishing</quote>.</para>
|
|
|
|
<para>To service our most conservative users, individual release
|
|
branches were introduced with FreeBSD 4.3.
|
|
These release branches are created shortly before a final release
|
|
is made. After the release goes out, only the most critical
|
|
security fixes and additions are merged onto the release branch.
|
|
In addition to source updates via CVS, binary patchkits are
|
|
available to keep systems on the
|
|
<emphasis>RELENG_<replaceable>X</replaceable>_<replaceable>Y</replaceable></emphasis>
|
|
branches updated.</para>
|
|
|
|
<sect2>
|
|
<title>What this article describes</title>
|
|
|
|
<para>The following sections of this article describe:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><xref linkend="release-proc"></term>
|
|
|
|
<listitem>
|
|
<para>The different phases of the release engineering process
|
|
leading up to the actual system build.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="release-build"></term>
|
|
|
|
<listitem>
|
|
<para>The actual build process.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="extensibility"></term>
|
|
|
|
<listitem>
|
|
<para>How the base release may be extended by third parties.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="lessons-learned"></term>
|
|
|
|
<listitem>
|
|
<para>Some of the lessons learned through the release of &os; 4.4.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="future"></term>
|
|
|
|
<listitem>
|
|
<para>Future directions of development.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Process -->
|
|
<sect1 id="release-proc">
|
|
<title>Release Process</title>
|
|
|
|
<para>New releases of FreeBSD are released from the -STABLE branch
|
|
at approximately four month intervals. The FreeBSD release
|
|
process begins to ramp up 45 days before the anticipated release
|
|
date when the release engineer sends an email to the development
|
|
mailing lists to remind developers that they only have 15 days to
|
|
integrate new changes before the code freeze. During this time,
|
|
many developers perform what have become known as <quote>MFC
|
|
sweeps</quote>. <acronym>MFC</acronym> stands for <quote>Merge
|
|
From CURRENT</quote> and it describes the process of merging a
|
|
tested change from our -CURRENT development branch to our -STABLE
|
|
branch.</para>
|
|
|
|
<sect2>
|
|
<title>Code Review</title>
|
|
|
|
<para>Thirty days before the anticipated release, the source
|
|
repository enters a <quote>code slush</quote>. During this
|
|
time, all commits to the -STABLE branch must be approved by the
|
|
&a.re;. The kinds of changes that are allowed during this 15 day
|
|
period include:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Bug fixes.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Documentation updates.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Security-related fixes of any kind.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Minor changes to device drivers, such as adding new Device
|
|
IDs.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Any additional change that the release engineering team feels
|
|
is justified, given the potential risk.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>After the first 15 days of the code slush, a
|
|
<emphasis>release candidate</emphasis> is released for
|
|
widespread testing and the code enters a <quote>code
|
|
freeze</quote> where it becomes much harder to justify new
|
|
changes to the system unless a serious bug-fix or security issue
|
|
is involved. During the code freeze, at least one release
|
|
candidate is released per week, until the final release is
|
|
ready. During the days leading to the final release, the
|
|
release engineering team is in constant communication with the
|
|
security-officer team, the documentation maintainers, and the
|
|
port maintainers, to ensure that all of the
|
|
different components required for a successful release are
|
|
available.</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Final Release Checklist</title>
|
|
|
|
<para>When several release candidates have been made available for
|
|
widespread testing and all major issues have been resolved, the
|
|
final release <quote>polishing</quote> can begin.</para>
|
|
|
|
<sect3 id="rel-branch">
|
|
<title>Creating the Release Branch</title>
|
|
|
|
<para>As described in the introduction, the
|
|
<literal>RELENG_<replaceable>X</replaceable>_<replaceable>Y</replaceable></literal>
|
|
release branch is a relatively new addition to our release
|
|
engineering
|
|
methodology. The first step in creating this branch is to
|
|
ensure that you are working with the newest version of the
|
|
<literal>RELENG_<replaceable>X</replaceable></literal> sources
|
|
that you want to branch <emphasis>from</emphasis>.</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs update -rRELENG_4 -P -d</userinput></screen>
|
|
|
|
<para>The next step is to create a branch point
|
|
<emphasis>tag</emphasis>, so that diffs against the start of
|
|
the branch are easier with CVS:</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs rtag -rRELENG_4 RELENG_4_8_BP src</userinput></screen>
|
|
|
|
<para>And then a new branch tag is created with:</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs rtag -b -rRELENG_4_8_BP RELENG_4_8 src</userinput></screen>
|
|
|
|
<note>
|
|
<para><emphasis>The
|
|
<literal>RELENG_<replaceable>*</replaceable></literal> tags
|
|
are restricted for use by the CVS-meisters and release
|
|
engineers.</emphasis></para>
|
|
</note>
|
|
|
|
<sidebar>
|
|
<para>A <quote><emphasis>tag</emphasis></quote> is CVS
|
|
vernacular for a label that identifies the source at a specific point
|
|
in time. By tagging the tree, we ensure that future release builders
|
|
will always be able to use the same source we used to create the
|
|
official FreeBSD Project releases.</para>
|
|
</sidebar>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-head" align="center">
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>FreeBSD Development Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng3" align="center">
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>FreeBSD 3.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng4" align="center">
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>FreeBSD 4.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng5" align="center">
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>FreeBSD 5.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng6" align="center">
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>FreeBSD 6.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
</sect3>
|
|
|
|
<sect3 id="versionbump">
|
|
<title>Bumping up the Version Number</title>
|
|
|
|
<para>Before the final release can be tagged, built, and
|
|
released, the following files need to be modified to reflect
|
|
the correct version of FreeBSD:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/porters-handbook/book.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/share/sgml/freebsd.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/Makefile.inc1</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/UPDATING</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/gnu/usr.bin/groff/tmac/mdoc.local</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/Makefile</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/share/sgml/release.dsl</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/examples/Makefile.relnotesng</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/sgml/release.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/share/examples/cvsup/standard-supfile</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/conf/newvers.sh</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/sys/param.h</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/usr.sbin/pkg_install/add/main.c</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/docs/man.sgml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/cgi/ports.cgi</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>ports/Tools/scripts/release/config</filename></para>
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
<para>The release notes and errata files also need to be adjusted for the
|
|
new release (on the release branch) and truncated appropriately
|
|
(on the stable/current branch):</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/relnotes/common/new.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/errata/article.sgml
|
|
</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para><application>Sysinstall</application> should be updated to note
|
|
the number of available ports and the amount of disk space required
|
|
for the Ports Collection[4]. This information is currently kept in
|
|
<filename>src/usr.sbin/sysinstall/dist.c</filename>.</para>
|
|
|
|
<para>After the release has been built, a number of file should
|
|
be updated to announce the release to the world.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>doc/share/images/articles/releng/branches-releng<replaceable>X</replaceable>.pic</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/share/sgml/advisories.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/share/sgml/includes.release.sgml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/share/sgml/includes.release.xsl</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/releases/*</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/releng/index.sgml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/news/news.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/search/web.atoz</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/share/misc/bsd-family-tree</filename></para>
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
</sect3>
|
|
|
|
<sect3 id="versionbump-major">
|
|
<title>Preparing a new major release branch
|
|
(RELENG_<replaceable>X</replaceable>)</title>
|
|
|
|
<para>When a new major release branch, such as
|
|
<literal>RELENG_6</literal> is branched from HEAD, some
|
|
additional files must be updated before releases can be made
|
|
from this new branch.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/share/examples/cvsup/stable-supfile</filename>
|
|
- must be updated to point to the new -STABLE branch, when
|
|
applicable.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Creating Release Tags</title>
|
|
|
|
<para>When the final release is ready, the following command
|
|
will create the <literal>RELENG_4_8_0_RELEASE</literal>
|
|
tag.</para>
|
|
|
|
<screen>/usr/src&prompt.root; <userinput>cvs rtag -rRELENG_4_8 RELENG_4_8_0_RELEASE src</userinput></screen>
|
|
|
|
<para>The Documentation and Ports managers are responsible for
|
|
tagging the respective trees with the <literal>RELEASE_4_8_0</literal>
|
|
tag.</para>
|
|
|
|
<para>Occasionally, a last minute fix may be required
|
|
<emphasis>after</emphasis> the final tags have been created.
|
|
In practice this is not a problem, since <acronym>CVS</acronym>
|
|
allows tags to be manipulated with <command>cvs
|
|
tag -d <replaceable>tagname filename</replaceable></command>.
|
|
It is very important that any last minute changes be tagged
|
|
appropriately as part of the release. FreeBSD releases must
|
|
always be reproducible. Local hacks in the release
|
|
engineer's environment are not acceptable.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Building -->
|
|
<sect1 id="release-build">
|
|
<title>Release Building</title>
|
|
|
|
<para>FreeBSD <quote>releases</quote> can be built by anyone with a
|
|
fast machine and access to a source repository. (That should be
|
|
everyone, since we offer anonymous CVS! See The Handbook for
|
|
details.) The <emphasis>only</emphasis> special requirement is
|
|
that the &man.vn.4; device must be available. (On -CURRENT, this
|
|
device has been replaced by the new &man.md.4;
|
|
memory disk driver.) If the
|
|
device is not loaded into your kernel, then the kernel module
|
|
should be automatically loaded when &man.vnconfig.8; is executed
|
|
during the boot media creation phase. All of the tools necessary
|
|
to build a release are available from the CVS repository in
|
|
<filename>src/release</filename>. These tools aim to provide a
|
|
consistent way to build FreeBSD releases. A complete release can
|
|
actually be built with only a single command, including the
|
|
creation of <acronym>ISO</acronym> images suitable for burning to
|
|
CDROM, installation floppies, and an FTP install directory. This
|
|
command is aptly named <command>make
|
|
release</command>.</para>
|
|
|
|
<sect2>
|
|
<title><command>make release</command></title>
|
|
|
|
<para>To successfully build a release, you must first populate
|
|
<filename>/usr/obj</filename> by running <command>make
|
|
world</command> or simply
|
|
<command>make
|
|
buildworld</command>. The release
|
|
target requires several variables be set properly to build a
|
|
release:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><makevar>CHROOTDIR</makevar> - The directory to be used as the
|
|
chroot environment for the entire release build.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><makevar>BUILDNAME</makevar> - The name of the release to be
|
|
built.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><makevar>CVSROOT</makevar> - The location of a CVS Repository.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><makevar>RELEASETAG</makevar> - The CVS tag corresponding to the
|
|
release you would like to build.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>If you do not already have access to a local CVS
|
|
repository, then you may mirror one with <ulink
|
|
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/synching.html#CVSUP">CVSup</ulink>.
|
|
The supplied supfile,
|
|
<filename>/usr/share/examples/cvsup/cvs-supfile</filename>, is
|
|
a useful starting point for mirroring the CVS
|
|
repository.</para>
|
|
|
|
<para>If <makevar>RELEASETAG</makevar> is omitted, then the
|
|
release will be built from the <literal>HEAD</literal> (a.k.a. -CURRENT) branch.
|
|
Releases built from this branch are normally referred to as
|
|
<quote>-CURRENT snapshots</quote>.</para>
|
|
|
|
<para>There are many other variables available to customize the
|
|
release build. Most of these variables are documented at the
|
|
top of <filename>src/release/Makefile</filename>. The exact
|
|
command used to build the official FreeBSD 4.7 (x86) release
|
|
was:</para>
|
|
|
|
<screen><command>make <literal>release CHROOTDIR=/local3/release \
|
|
BUILDNAME=4.7-RELEASE \
|
|
CVSROOT=/host/cvs/usr/home/ncvs \
|
|
RELEASETAG=RELENG_4_7_0_RELEASE</literal>
|
|
</command>
|
|
</screen>
|
|
|
|
<para>The release <filename>Makefile</filename> can be broken down into several distinct
|
|
steps.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Creation of a sanitized system environment in a separate
|
|
directory hierarchy with <quote><command>make
|
|
<literal>installworld</literal></command></quote>.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Checkout from CVS of a clean version of the system source,
|
|
documentation, and ports into the release build hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Population of <filename>/etc</filename> and
|
|
<filename>/dev</filename> in the chrooted
|
|
environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>chroot into the release build hierarchy, to make it harder for
|
|
the outside environment to taint this build.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>make world</command>
|
|
in the chrooted environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build of Kerberos-related binaries.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build <filename>GENERIC</filename> kernel.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Creation of a staging directory tree where the binary
|
|
distributions will be built and packaged.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the documentation toolchain needed to
|
|
convert the documentation source (SGML) into HTML and text documents
|
|
that will accompany the release.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the actual documentation
|
|
(user manuals, tutorials, release notes, hardware compatibility lists,
|
|
and so on.)</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build of the <quote>crunched</quote> binaries used for
|
|
installation floppies.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Package up distribution tarballs of the binaries and sources.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Create the boot media and a <quote>fixit</quote> floppy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Create FTP installation hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>(optionally)</emphasis> Create ISO images for
|
|
CDROM/DVD media.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>For more information about the release build infrastructure,
|
|
please see &man.release.7;.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Building <application>&xfree86;</application></title>
|
|
|
|
<para><application>&xfree86;</application> is an important component for many desktop users.
|
|
Prior to FreeBSD 4.6-RELEASE, releases used &xfree86;
|
|
3.<replaceable>X</replaceable> by default.
|
|
The easiest way to build these versions is to use the
|
|
<filename>src/release/scripts/X11/build_x.sh</filename> script.
|
|
This script requires that &xfree86; and Tcl/Tk already be
|
|
installed on the build host. After compiling the necessary X
|
|
servers, the script will package all of the files into tarballs
|
|
that &man.sysinstall.8; expects to find in the
|
|
<filename>XF86336</filename> directory of the installation
|
|
media.</para>
|
|
|
|
<para>Beginning with FreeBSD 4.6-RELEASE, &man.sysinstall.8;
|
|
installs &xfree86; 4.<replaceable>X</replaceable> by default, as a
|
|
set of <quote>normal</quote> packages. These can either be the
|
|
packages generated by the package-building cluster or packages
|
|
built from an appropriately tagged ports tree.</para>
|
|
|
|
<para>Beginning with FreeBSD 5.3-RELEASE, &man.sysinstall.8;
|
|
installs &xorg; packages instead of &xfree86; packages by
|
|
default.</para>
|
|
|
|
<note><para>It is important to remove any site-specific settings
|
|
from <filename>/etc/make.conf</filename>. For example, it would
|
|
be unwise to distribute binaries that were built on a system
|
|
with <varname>CPUTYPE</varname> set to a specific
|
|
processor.</para></note>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Contributed Software (<quote>ports</quote>)</title>
|
|
|
|
<para>The <ulink url="http://www.FreeBSD.org/ports">FreeBSD Ports
|
|
collection</ulink> is a collection of over &os.numports;
|
|
third-party software packages available for FreeBSD. The &a.portmgr;
|
|
is responsible for maintaining a consistent ports tree that can be used
|
|
to create the binary packages that accompany official FreeBSD
|
|
releases.</para>
|
|
|
|
<para>The release engineering activities for our collection of
|
|
third-party packages is beyond the scope of this document. A
|
|
separate article, &art.re.pkgs;, covers this topic
|
|
in depth.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Release ISOs</title>
|
|
|
|
<para>Starting with FreeBSD 4.4, the FreeBSD Project decided to
|
|
release all four ISO images that were previously sold on the
|
|
<emphasis>BSDi/Wind River Systems/FreeBSD Mall</emphasis>
|
|
<quote>official</quote> CDROM distributions. Each of the four
|
|
discs must contain a <filename>README.TXT</filename> file that
|
|
explains the contents of the disc, a
|
|
<filename>CDROM.INF</filename> file that provides meta-data for
|
|
the disc so that &man.sysinstall.8; can validate and use the
|
|
contents, and a <filename>filename.txt</filename> file that
|
|
provides a manifest for the disc. This
|
|
<emphasis>manifest</emphasis> can be created with a simple
|
|
command:</para>
|
|
|
|
<screen>/stage/cdrom&prompt.root; <userinput>find . -type f | sed -e 's/^\.\///' | sort > filename.txt</userinput></screen>
|
|
|
|
<para>The specific requirements of each CD are outlined below.</para>
|
|
|
|
<sect3>
|
|
<title>Disc 1</title>
|
|
|
|
<para>The first disc is almost completely created by
|
|
<command>make
|
|
release</command>. The only changes
|
|
that should be made to the <filename>disc1</filename> directory are the addition of
|
|
a <filename>tools</filename> directory, <application
|
|
class="software">&xfree86;</application>, and as many popular
|
|
third party software packages as will fit on the disc. The
|
|
<filename>tools</filename> directory contains software that allow users to create
|
|
installation floppies from other operating systems. This disc
|
|
should be made bootable so that users of modern PCs do not
|
|
need to create installation floppy disks.</para>
|
|
|
|
<para>If an alternate version of &xfree86; is to be provided, then
|
|
&man.sysinstall.8; must be updated to reflect the new location
|
|
and installation instructions. The relevant code is contained
|
|
in <filename>src/usr.sbin/sysinstall</filename>.
|
|
Specifically, the files <filename>dist.c</filename>,
|
|
<filename>menus.c</filename>, and
|
|
<filename>config.c</filename> will need to be updated.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Disc 2</title>
|
|
|
|
<para>The second disc is also largely created by <command>make
|
|
release</command>. This disc contains a <quote>live
|
|
filesystem</quote> that can be used from &man.sysinstall.8; to
|
|
troubleshoot a FreeBSD installation. This disc should be
|
|
bootable and should also contain a compressed copy of the CVS
|
|
repository in the <filename>CVSROOT</filename> directory and
|
|
commercial software demos in the <filename>commerce</filename>
|
|
directory.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Discs 3 and 4</title>
|
|
|
|
<para>The remaining two discs contain additional software
|
|
packages for FreeBSD. The packages should be clustered so that
|
|
a package and all of its <emphasis>dependencies</emphasis> are
|
|
included on the same disc. More information about the
|
|
creation of these discs is provided in the &art.re.pkgs;
|
|
article.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Multi-volume support</title>
|
|
|
|
<para><application>Sysinstall</application> supports multiple
|
|
volume package installations. This requires that each disc
|
|
have an <filename>INDEX</filename> file containing all of the
|
|
packages on all volumes of a set, along with an extra field
|
|
that indicates which volume that particular package is on.
|
|
Each volume in the set must also have the
|
|
<literal>CD_VOLUME</literal> variable set in the
|
|
<filename>cdrom.inf</filename> file so that sysinstall can
|
|
tell which volume is which. When a user attempts to install a
|
|
package that is not on the current disc, sysinstall will
|
|
prompt the user to insert the appropriate one.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Distribution -->
|
|
<sect1 id="distribution">
|
|
<title>Distribution</title>
|
|
|
|
<sect2 id="dist-ftp">
|
|
<title>FTP Sites</title>
|
|
|
|
<para>When the release has been thoroughly tested and packaged for
|
|
distribution, the master FTP site must be updated. The official
|
|
FreeBSD public FTP sites are all mirrors of a master server that
|
|
is open only to other FTP sites. This site is known as
|
|
<hostid>ftp-master</hostid>. When the release is ready, the
|
|
following files must be modified on <hostid>ftp-master</hostid>:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/<replaceable>X.Y</replaceable>-RELEASE/</filename></term>
|
|
<listitem>
|
|
<para>The installable FTP directory as output from <command>make
|
|
release</command>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/ports/<replaceable>arch</replaceable>/packages-<replaceable>X.Y</replaceable>-release/</filename></term>
|
|
<listitem><para>The complete package build for this
|
|
release.</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/<replaceable>X.Y</replaceable>-RELEASE/tools</filename></term>
|
|
<listitem><para>A symlink to
|
|
<filename>../../../tools</filename>.</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/<replaceable>X.Y</replaceable>-RELEASE/packages</filename></term>
|
|
<listitem><para>A symlink to
|
|
<filename>../../../ports/<replaceable>arch</replaceable>/packages-<replaceable>X.Y</replaceable>-release</filename>.</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/ISO-IMAGES/<replaceable>X.Y</replaceable>/<replaceable>X.Y</replaceable>-RELEASE-<replaceable>arch</replaceable>-*.iso</filename></term>
|
|
<listitem><para>The ISO images. The <quote>*</quote> is
|
|
<filename>disc1</filename>, <filename>disc2</filename>, etc.
|
|
Only if there is a <filename>disc1</filename> and there is an
|
|
alternative first installation CD (for example a
|
|
stripped-down install with no windowing system) there may
|
|
be a <filename>mini</filename> as well.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
<para>For more information about the distribution mirror
|
|
architecture of the FreeBSD FTP sites, please see the <ulink
|
|
url="&url.articles.hubs;/">Mirroring FreeBSD</ulink> article.</para>
|
|
|
|
<para>It may take many hours to two days after updating
|
|
<hostid>ftp-master</hostid> before a majority of the Tier-1 FTP
|
|
sites have the new software depending on whether or not a package
|
|
set got loaded at the same time. It is imperative that the release
|
|
engineers coordinate with the &a.mirror-announce; before announcing the general
|
|
availability of new software on the FTP sites. Ideally
|
|
the release package set should be loaded at least four
|
|
days prior to release day. The release bits should be
|
|
loaded between 24 and 48 hours before the planned release
|
|
time with <quote>other</quote> file permissions turned off.
|
|
This will allow the mirror sites to download it but the
|
|
general public will not be able to download it from the mirror
|
|
sites. Mail should be sent to &a.mirror-announce; at the time
|
|
the release bits get posted saying the release has been staged
|
|
and giving the time that the mirror sites should begin allowing
|
|
access. Be sure to include a time zone with the
|
|
time, for example make it relative to GMT.</para>
|
|
</sect2>
|
|
|
|
<sect2 id="dist-cdrom">
|
|
<title>CD-ROM Replication</title>
|
|
|
|
<para>Coming soon: Tips for sending FreeBSD ISOs to a replicator
|
|
and quality assurance measures to be taken.</para>
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
<!-- Extensibility -->
|
|
<sect1 id="extensibility">
|
|
<title>Extensibility</title>
|
|
|
|
<para>Although FreeBSD forms a complete operating system, there is
|
|
nothing that forces you to use the system exactly as we have
|
|
packaged it up for distribution. We have tried to design the
|
|
system to be as extensible as possible so that it can serve as a
|
|
platform that other commercial products can be built on top
|
|
of. The only <quote>rule</quote> we have about this is that if you
|
|
are going to distribute FreeBSD with non-trivial changes, we
|
|
encourage you to document your enhancements! The FreeBSD community
|
|
can only help support users of the software we provide. We
|
|
certainly encourage innovation in the form of advanced
|
|
installation and administration tools, for example, but we cannot
|
|
be expected to answer questions about it.</para>
|
|
|
|
<sect2>
|
|
<title>Creating Customized Boot floppies</title>
|
|
|
|
<para>Many sites have complex requirements that may require
|
|
additional kernel modules or userland tools be added to the
|
|
installation floppies. The <quote>quick and dirty</quote> way
|
|
to accomplish this would be to modify the staging directory of
|
|
an existing <command>make release</command> build hierarchy:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Apply patches or add additional files inside the chroot
|
|
release build directory.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>rm
|
|
${CHROOTDIR}/usr/obj/usr/src/release/release.[59]</command></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>rebuild &man.sysinstall.8;, the kernel, or whatever
|
|
parts of the system your change affected.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>chroot ${CHROOTDIR} ./mk floppies
|
|
</command></para>
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
<para>New release floppies will be located in
|
|
<filename>${CHROOTDIR}/R/stage/floppies</filename>.</para>
|
|
|
|
<para>Alternatively, the
|
|
<filename>boot.flp</filename> make
|
|
target can be called, or the filesystem
|
|
creating script,
|
|
<filename>src/release/scripts/doFS.sh</filename>, may be invoked
|
|
directly.</para>
|
|
|
|
<para>Local patches may also be supplied to the release build by
|
|
defining the <makevar>LOCAL_PATCH</makevar> variable in <command>make
|
|
release</command>.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Scripting <command>sysinstall</command></title>
|
|
|
|
<para>The FreeBSD system installation and configuration tool,
|
|
&man.sysinstall.8;, can be scripted to provide automated installs
|
|
for large sites. This functionality can be used in conjunction
|
|
with &intel; PXE[12] to bootstrap systems from the network, or
|
|
via custom boot floppies with a sysinstall script. An example
|
|
sysinstall script is available in the CVS tree as
|
|
<filename>src/usr.sbin/sysinstall/install.cfg</filename>.</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Lessons Learned -->
|
|
<sect1 id="lessons-learned">
|
|
<title>Lessons Learned from FreeBSD 4.4</title>
|
|
|
|
<para>The release engineering process for 4.4 formally began on
|
|
August 1st, 2001. After that date all commits to the
|
|
<literal>RELENG_4</literal> branch of FreeBSD had to be explicitly
|
|
approved by the &a.re;. The first
|
|
release candidate for the x86 architecture was released on August
|
|
16, followed by 4 more release candidates leading up to the final
|
|
release on September 18th. The security officer was very involved
|
|
in the last week of the process as several security issues were
|
|
found in the earlier release candidates. A total of over
|
|
<emphasis>500</emphasis> emails were sent to the &a.re; in
|
|
little over a month.</para>
|
|
|
|
<para>Our user community has made it very clear that the security
|
|
and stability of a FreeBSD release should not be sacrificed for
|
|
any self-imposed deadlines or target release dates. The FreeBSD
|
|
Project has grown tremendously over its lifetime and the need for
|
|
standardized release engineering procedures has never been more
|
|
apparent. This will become even more important as FreeBSD is
|
|
ported to new platforms.</para>
|
|
</sect1>
|
|
|
|
<!-- Future Directions -->
|
|
<sect1 id="future">
|
|
<title>Future Directions</title>
|
|
|
|
<para>It is imperative for our release engineering activities to
|
|
scale with our growing userbase. Along these lines we are working
|
|
very hard to document the procedures involved in producing FreeBSD
|
|
releases.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><emphasis>Parallelism</emphasis> - Certain portions of the
|
|
release build are actually <quote>embarrassingly
|
|
parallel</quote>. Most of the tasks are very I/O intensive,
|
|
so having multiple high-speed disk drives is actually more important than
|
|
using multiple processors in speeding up the <command>make
|
|
release</command> process. If multiple disks are used for
|
|
different hierarchies in the &man.chroot.2;
|
|
environment, then the CVS checkout of the <filename>ports</filename> and <filename>doc</filename> trees
|
|
can be happening simultaneously as the <command>make
|
|
world</command> on another disk. Using a
|
|
<acronym>RAID</acronym> solution (hardware or software) can
|
|
significantly decrease the overall build time.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Cross-building releases</emphasis> - Building
|
|
IA-64 or Alpha release on x86 hardware? <command>make
|
|
TARGET=ia64 release</command>.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Regression Testing</emphasis> - We need better
|
|
automated correctness testing for FreeBSD.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Installation Tools</emphasis> - Our installation
|
|
program has long since outlived its intended life span.
|
|
Several projects are under development to provide a more
|
|
advanced installation mechanism. The libh project was one
|
|
such project that aimed to provide an intelligent new package
|
|
framework and GUI installation program.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</sect1>
|
|
|
|
<!-- Acknowledgements -->
|
|
<sect1 id="ackno">
|
|
<title>Acknowledgements</title>
|
|
|
|
<para>I would like to thank Jordan Hubbard for giving me the
|
|
opportunity to take on some of the release engineering
|
|
responsibilities for FreeBSD 4.4 and also for all of his work
|
|
throughout the years making FreeBSD what it is today. Of course
|
|
the release would not have been possible without all of the
|
|
release-related work done by &a.asami;, &a.steve;, &a.bmah;, &a.nik;,
|
|
&a.obrien;, &a.kris;, &a.jhb; and the rest of the FreeBSD development
|
|
community. I would also like to thank &a.rgrimes;, &a.phk;, and others
|
|
who worked on the release engineering tools in the very early days
|
|
of FreeBSD. This article was influenced by release engineering
|
|
documents from the CSRG[13], the NetBSD Project[10], and John
|
|
Baldwin's proposed release engineering process notes[11].</para>
|
|
</sect1>
|
|
|
|
<!-- Reference / Biblio Section -->
|
|
<sect1 id="biblio">
|
|
<title>References</title>
|
|
<para>[1] CVS - Concurrent Versions System
|
|
<ulink url="http://www.cvshome.org"></ulink></para>
|
|
|
|
<para>[2] CVSup - The CVS-Optimized General Purpose Network File Distribution
|
|
System <ulink url="http://www.polstra.com/projects/freeware/CVSup"></ulink>
|
|
</para>
|
|
|
|
<para>[3] <ulink url="http://pointyhat.FreeBSD.org"></ulink></para>
|
|
|
|
<para>[4] FreeBSD Ports Collection
|
|
<ulink url="http://www.FreeBSD.org/ports"></ulink></para>
|
|
|
|
<para>[5] FreeBSD Committers <ulink
|
|
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributors/staff-committers.html"></ulink>
|
|
</para>
|
|
|
|
<para>[6] FreeBSD Core-Team
|
|
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributors/staff-core.html"></ulink></para>
|
|
|
|
<para>[7] FreeBSD Handbook
|
|
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook"></ulink>
|
|
</para>
|
|
|
|
<para>[8] GNATS: The GNU Bug Tracking System
|
|
<ulink url="http://www.gnu.org/software/gnats"></ulink>
|
|
</para>
|
|
|
|
<para>[9] FreeBSD PR Statistics
|
|
<ulink url="http://www.FreeBSD.org/prstats/index.html"></ulink></para>
|
|
|
|
<para>[10] NetBSD Developer Documentation: Release Engineering
|
|
<ulink url="http://www.NetBSD.org/developers/releng/index.html"></ulink>
|
|
</para>
|
|
|
|
<para>[11] John Baldwin's FreeBSD Release Engineering Proposal
|
|
<ulink url="http://people.FreeBSD.org/~jhb/docs/releng.txt"></ulink>
|
|
</para>
|
|
|
|
<para>[12] PXE Jumpstart Guide
|
|
<ulink
|
|
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/pxe/index.html"></ulink>
|
|
</para>
|
|
|
|
<para>[13] Marshall Kirk McKusick, Michael J. Karels, and Keith Bostic:
|
|
<ulink url="http://docs.FreeBSD.org/44doc/papers/releng.html">
|
|
<emphasis>The Release Engineering of 4.3BSD</emphasis></ulink>
|
|
</para>
|
|
</sect1>
|
|
</article>
|