342 lines
15 KiB
Text
342 lines
15 KiB
Text
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN" [
|
||
|
|
<!ENTITY base CDATA "..">
|
||
|
|
<!ENTITY email 'releng'>
|
||
|
|
<!ENTITY date "$FreeBSD$">
|
||
|
|
<!ENTITY title "FreeBSD Release Engineering">
|
||
|
|
<!ENTITY % includes SYSTEM "../includes.sgml"> %includes;
|
||
|
|
]>
|
||
|
|
|
||
|
|
<html>
|
||
|
|
&header;
|
||
|
|
|
||
|
|
<p>This page documents the process used by the release engineering
|
||
|
|
team to produce official FreeBSD releases. Comments, suggestions,
|
||
|
|
and additional material are welcome. This is, as always, a work in
|
||
|
|
progress.</p>
|
||
|
|
|
||
|
|
<h1>Release Process</h1>
|
||
|
|
|
||
|
|
<p>New releases of FreeBSD are released from the -STABLE branch at
|
||
|
|
approximately four month intervals. The FreeBSD release process
|
||
|
|
begins to ramp up 45 days before the anticipated release date when
|
||
|
|
the release engineer sends an email to the development mailing
|
||
|
|
lists to remind developers that they only have 15 days to
|
||
|
|
integrate new changes before the code freeze. During this time,
|
||
|
|
many developers perform what have become known as ``MFC sweeps''.
|
||
|
|
MFC stands for ``Merge From CURRENT'' and it describes the process
|
||
|
|
of merging a tested change from our -CURRENT development branch to
|
||
|
|
our -STABLE branch.</p>
|
||
|
|
|
||
|
|
<center><img src="../gifs/branches.png" alt="Release Branch
|
||
|
|
Diagram"></center>
|
||
|
|
|
||
|
|
<p>To service our most conservative users, individual release
|
||
|
|
branches were introduced with FreeBSD 4.3. These release branches
|
||
|
|
are created shortly before a final release is made, and after the
|
||
|
|
release goes out only the most critical security fixes are merged
|
||
|
|
onto the release branch. In addition to source updates via CVS,
|
||
|
|
binary patchkits are available to keep systems on the
|
||
|
|
<tt>RELENG_4_3</tt> and <tt>RELENG_4_4</tt> branches updated.</p>
|
||
|
|
|
||
|
|
<h2>Code Review</h2>
|
||
|
|
|
||
|
|
<p>Thirty days before the anticipated release, the source repository
|
||
|
|
enters a ``code slush''. During this time, all commits to the
|
||
|
|
-STABLE branch must be approved by the release engineer (<a
|
||
|
|
href="mailto:re@FreeBSD.org">re@FreeBSD.org</a>). The kinds of
|
||
|
|
changes that are allowed during this 15 day period include :</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li>Bug-fixes.</li>
|
||
|
|
<li>Documentation updates.</li>
|
||
|
|
<li>Security-related fixes of any kind.</li>
|
||
|
|
<li><em>Minor</em> changes to device drivers, such as adding new
|
||
|
|
device IDs.</li>
|
||
|
|
|
||
|
|
<li>Any additional change that the release engineering team feels
|
||
|
|
is justified given the potential risk.</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<p>After the first 15 days of the code slush, a release candidate is
|
||
|
|
released for widespread testing and the code enters a ``code
|
||
|
|
freeze'' where it becomes much harder to justify new changes to
|
||
|
|
the system unless a serious bug-fix or security issue is involved.
|
||
|
|
During the code freeze, at least one release candidate is released
|
||
|
|
per week until the final release is ready. During the days
|
||
|
|
leading up to the final release, the release engineering team is
|
||
|
|
in constant communication with the security-officer team, the
|
||
|
|
documentation maintainers, and the ports managers, to make sure
|
||
|
|
that all of the different components required for a successful
|
||
|
|
release are available.</p>
|
||
|
|
|
||
|
|
<h2>Final Release Checklist</h2>
|
||
|
|
|
||
|
|
<p>When several release candidates have been made available for
|
||
|
|
widespread testing and all major issues have been resolved, the
|
||
|
|
final release ``polishing'' can begin.</p>
|
||
|
|
|
||
|
|
<h3>Creating the Release Branch</h3>
|
||
|
|
|
||
|
|
<p>As described in the introduction, the <tt>RELENG_X_Y</tt> release
|
||
|
|
branch is a relatively new addition to our release engineering
|
||
|
|
methodology. The first step in creating this branch is to ensure
|
||
|
|
that you are working with the newest version of the
|
||
|
|
<tt>RELENG_X</tt> sources that you want to branch
|
||
|
|
<em>from</em>.</p>
|
||
|
|
|
||
|
|
<tt>/usr/src# cvs up -rRELENG_4 -P -d</tt>
|
||
|
|
|
||
|
|
<p>The next step is to create a branch point tag. A ``tag'' is CVS
|
||
|
|
vernacular for a label that identifies the source at a specific
|
||
|
|
point in time. By tagging the tree, we ensure that future release
|
||
|
|
builders will always be able to use the same source we used to
|
||
|
|
create the official FreeBSD Project releases. The branch point
|
||
|
|
tag makes it easier to produce diffs against the start of the
|
||
|
|
branch :</p>
|
||
|
|
|
||
|
|
<tt>/usr/src# cvs rtag -rRELENG_4 RELENG_4_4_BP src</tt>
|
||
|
|
|
||
|
|
<p>And then a new branch tag is created with :</p>
|
||
|
|
|
||
|
|
<tt>/usr/src# cvs rtag -b -rRELENG_4_4_BP RELENG_4_4 src</tt>
|
||
|
|
|
||
|
|
<p>Note: The <tt>RELENG_*</tt> tags are restricted for use by the
|
||
|
|
CVS-meisters and release engineers.</p>
|
||
|
|
|
||
|
|
<h3>Bumping up the Version Number</h3>
|
||
|
|
|
||
|
|
<p>Before the final release can be tagged, built, and released, the
|
||
|
|
following files need to be modified to reflect the correct version
|
||
|
|
of FreeBSD :</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li><tt>src/sys/conf/newvers.sh</tt></li>
|
||
|
|
<li><tt>src/sys/sys/param.h</tt></li>
|
||
|
|
<li><tt>src/release/doc/share/sgml/release.ent</tt></li>
|
||
|
|
<li><tt>src/gnu/usr.bin/groff/tmac/mdoc.local</tt></li>
|
||
|
|
<li><tt>doc/share/sgml/freebsd.ent</tt></li>
|
||
|
|
<li><tt>doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml</tt></li>
|
||
|
|
<li><tt>www/en/releases/*</tt></li>
|
||
|
|
<li><tt>src/UPDATING</tt></li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<h3>Creating the Release Tags</h3>
|
||
|
|
|
||
|
|
<p>When the final release is ready, the following command will create the
|
||
|
|
<tt>RELENG_4_4_0_RELEASE</tt> tag.</p>
|
||
|
|
|
||
|
|
<tt>/usr/src# cvs rtag -rRELENG_4_4 RELENG_4_4_0_RELEASE src</tt>
|
||
|
|
|
||
|
|
<p>The Documentation and Ports managers are responsible for tagging
|
||
|
|
the respective trees with the <tt>RELEASE_4_4_0</tt> tag.</p>
|
||
|
|
|
||
|
|
<p>Occasionally, a last minute fix may be required <em>after</em>
|
||
|
|
the final tags have been created. In practice this isn't a
|
||
|
|
problem, since CVS allows tags to be manipulated with <tt>cvs tag
|
||
|
|
-d <em>tagname</em> <em>filename</em></tt>. It is very important
|
||
|
|
that any last minute changes be tagged appropriately as part of
|
||
|
|
the release. FreeBSD releases must always be reproduceable.
|
||
|
|
Local hacks in the release engineer's environment are not
|
||
|
|
acceptable.</p>
|
||
|
|
|
||
|
|
<h1>Release Building</h1>
|
||
|
|
|
||
|
|
<p>FreeBSD releases can be built by anyone with a fast machine and
|
||
|
|
access to a source repository. <em>(That should be everyone,
|
||
|
|
since we offer anonymous CVS! See <a
|
||
|
|
href="http://www.FreeBSD.org/handbook">The Handbook</a> for
|
||
|
|
details.)</em>. The only special requirement is that the
|
||
|
|
<tt>vn</tt> <em>(On -CURRENT, this device has been replaced by the
|
||
|
|
new <tt>md</tt> memory disk driver.)</em> device must be
|
||
|
|
available. If the device is not loaded into your kernel, then the
|
||
|
|
kernel module should be automatically loaded when
|
||
|
|
<tt>vnconfig</tt> is executed during the boot media creation
|
||
|
|
phase. All of the tools necessary to build a release are
|
||
|
|
available from the CVS repository in <tt>src/release</tt>. These
|
||
|
|
tools aim to provide a consistent way to build FreeBSD releases.
|
||
|
|
A complete release can actually be built with only a single
|
||
|
|
command, including the creation of ISO images suitable for burning
|
||
|
|
to CDROM, installation floppies, and an FTP install directory.
|
||
|
|
This command is aptly named ``make release''.</p>
|
||
|
|
|
||
|
|
<h2>``make release''</h2>
|
||
|
|
|
||
|
|
<p>To successfully build a release, you must first populate
|
||
|
|
<tt>/usr/obj</tt> by running ``make world'' or simply ``make
|
||
|
|
buildworld''. The release target requires several variables be
|
||
|
|
set properly to build a release :</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li><strong><tt>CHROOTDIR</tt></strong> - The directory to be used
|
||
|
|
as the chroot environment for the entire release build.</li>
|
||
|
|
<li><strong><tt>BUILDNAME</tt></strong> - The name of the release
|
||
|
|
to be built.</li>
|
||
|
|
<li><strong><tt>CVSROOT</tt></strong> - The location of a CVS
|
||
|
|
repository.</li>
|
||
|
|
<li><strong><tt>RELEASETAG</tt></strong> - The CVS tag
|
||
|
|
corresponding to the release you would like to build.</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<p>There are many other variables available to customize the release
|
||
|
|
build. Most of these variables are documented at the top of
|
||
|
|
<tt>src/release/Makefile</tt>. The exact command used to build
|
||
|
|
the official FreeBSD 4.4 (x86) release was :</p>
|
||
|
|
|
||
|
|
<tt>make release CHROOTDIR=/local3/release BUILDNAME=4.4-RELEASE CVSROOT=/host/cvs/usr/home/ncvs RELEASETAG=RELENG_4_4_0_RELEASE</tt>
|
||
|
|
|
||
|
|
<p>The release Makefile can be broken down into several distinct
|
||
|
|
steps.</p>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li>Creation of a sanitized system environment in a separate
|
||
|
|
directory hierarchy with ``make installworld''.</li>
|
||
|
|
<li>Checkout from CVS of a clean version of the system source,
|
||
|
|
documentation, and ports into the release build hierarchy.</li>
|
||
|
|
<li>Population of <tt>/etc</tt> and <tt>/dev</tt> in the chrooted
|
||
|
|
environment.</li>
|
||
|
|
<li>chroot into the release build hierarchy, to make it harder for
|
||
|
|
the outside environment to taint this build.</li>
|
||
|
|
<li>``make world'' in the chrooted environment.</li>
|
||
|
|
<li>Build of Kerberos-related binaries.</li>
|
||
|
|
<li>Build 'GENERIC' kernel.</li>
|
||
|
|
<li>Creation of a staging directory tree where the binary
|
||
|
|
distributions will be built and packaged.</li>
|
||
|
|
<li>Build and installation of the documentation toolchain needed
|
||
|
|
to convert the documentation source (SGML) into HTML, and text
|
||
|
|
documents that will accompany the release.</li>
|
||
|
|
<li>Build and installation of the actual documentation (user
|
||
|
|
manuals, tutorials, release notes, hardware compatibility lists,
|
||
|
|
etc...)</li>
|
||
|
|
<li>Build of the ``crunched'' binaries used for installation
|
||
|
|
floppies.</li>
|
||
|
|
<li>Package up distribution tarballs of the binaries and
|
||
|
|
sources.</li>
|
||
|
|
<li>Create the boot media and a fixit floppy.</li>
|
||
|
|
<li>Create FTP installation hierarchy.</li>
|
||
|
|
<li>(optionally) Create ISO images for CDROM/DVD media.</li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
<h2>Contributed Software (``ports'')</h2>
|
||
|
|
|
||
|
|
<p>The <a href="http://www.FreeBSD.org/ports">FreeBSD Ports
|
||
|
|
collection</a> is a collection of over 6,000 third-party software
|
||
|
|
packages available for FreeBSD. The ports team (<a
|
||
|
|
href="mailto:portmgr@FreeBSD.org">portmgr@FreeBSD.org</a>)
|
||
|
|
responsible for maintaining a consistent ports tree that can be
|
||
|
|
used to create the binary packages that accompany a given FreeBSD
|
||
|
|
release.</p>
|
||
|
|
|
||
|
|
<h3>The Ports Cluster</h3>
|
||
|
|
|
||
|
|
<p>In order to provide a consistent set of third-party packages for
|
||
|
|
FreeBSD releases, every port is built in a separate chroot
|
||
|
|
environment, starting with an empty <tt>/usr/local</tt> and
|
||
|
|
<tt>/usr/X11R6</tt>. The requisite dependencies are installed as
|
||
|
|
packages before the build proceeds. This enforces <em>
|
||
|
|
consistency</em> in the package build process. By starting the
|
||
|
|
package build in a pristine environment, we can assure that the
|
||
|
|
package metadata (such as required dependencies) is accurate, and
|
||
|
|
so we will never generate packages that might work on some systems
|
||
|
|
and not on others depending on what software was previously
|
||
|
|
installed.</p>
|
||
|
|
|
||
|
|
<p>The ``Ports Cluster'' for the x86 architecture currently consists
|
||
|
|
of a master node (Dual Pentium III 733Mhz) and 8 slave nodes
|
||
|
|
(Pentium III 800Mhz) to do the actual package builds. With this
|
||
|
|
configuration, a complete package build takes over 24 hours.
|
||
|
|
These machines are co-located with the other FreeBSD Project
|
||
|
|
equipment at Yahoo's corner of Exodus in Santa Clara, CA.</p>
|
||
|
|
|
||
|
|
<p>The ``Ports Cluster'' for the Alpha architecture consists of 7
|
||
|
|
PWS 500A machines donated by Compaq and also co-located with
|
||
|
|
Yahoo's facilities.</p>
|
||
|
|
|
||
|
|
<h3>The Package Split</h3>
|
||
|
|
|
||
|
|
<p>For FreeBSD 4.4 over 4.1 gigabytes of packages were created.
|
||
|
|
This causes a problem for CDROM distributions because we would
|
||
|
|
like to ship as many packages as possible without making the user
|
||
|
|
insert another disc to satisfy dependencies. The solution is to
|
||
|
|
create ``clusters'' of like packages with similar dependencies
|
||
|
|
onto specific discs. The package split is performed by the <a
|
||
|
|
href="mailto:portmgr@FreeBSD.org">portmgr</a> team in coordination
|
||
|
|
with the wishes of the general user community with respect to
|
||
|
|
which packages get to appear on the first CD.</p>
|
||
|
|
|
||
|
|
<h2>Release ISOs</h2>
|
||
|
|
|
||
|
|
<p>Starting with FreeBSD 4.4, the FreeBSD Project decided to release
|
||
|
|
all four ISO images that were previously sold on the BSDi/Wind
|
||
|
|
River Systems ``official'' CDROM distributions. Each of the four
|
||
|
|
discs must contain a <tt>README.TXT</tt> file that explains the
|
||
|
|
contents of the disc, a <tt>CDROM.INF</tt> file that provides
|
||
|
|
meta-data for the disc so that sysinstall can validate and use the
|
||
|
|
contents, and a <tt>filename.txt</tt> file that provides a
|
||
|
|
manifest for the disc. This manifest can be created with a simple
|
||
|
|
command :</p>
|
||
|
|
|
||
|
|
<tt>/stage/cdrom# find . -type f | sed -e 's/\^.\///' | sort > filename.txt</tt>
|
||
|
|
|
||
|
|
<p>The specific requirements of each CD is outlined below.</p>
|
||
|
|
|
||
|
|
<h3>Disc #1</h3>
|
||
|
|
|
||
|
|
<p>The first disc is almost completely created by ``make release''.
|
||
|
|
The only changes that should be made to the <tt>disc1</tt>
|
||
|
|
directory are the addition of a 'tools' directory, XFree86, and as
|
||
|
|
many popular third party software packages as will fit on the
|
||
|
|
disc. The 'tools' directory contains software that allow users to
|
||
|
|
create installation floppies from other operating systems. This
|
||
|
|
disc should be made bootable so that users of modern PCs do not
|
||
|
|
need to create installation floppy disks.</p>
|
||
|
|
|
||
|
|
<p>If an alternate version of XFree86 is to be provided, then
|
||
|
|
sysinstall must be updated to reflect the new location and
|
||
|
|
installation instructions. The relevant code is contained in
|
||
|
|
<tt>src/release/sysinstall</tt> on -STABLE or
|
||
|
|
<tt>src/usr.sbin/sysinstall</tt> on -CURRENT. Specifically, the
|
||
|
|
files <tt>dist.c</tt>, <tt>menus.c</tt>, and <tt>config.c</tt>
|
||
|
|
will need to be updated.</p>
|
||
|
|
|
||
|
|
<h3>Disc #2</h3>
|
||
|
|
|
||
|
|
<p>The second disc is also largely created by ``make release''.
|
||
|
|
This disc contains a ``live filesystem'' that can be used from
|
||
|
|
sysinstall to troubleshoot a FreeBSD installation. This disc
|
||
|
|
should be bootable and should also contain a compressed copy of
|
||
|
|
the CVS repository in the <tt>CVSROOT</tt> directory and
|
||
|
|
commercial software demos in the <tt>commerce</tt> directory.</p>
|
||
|
|
|
||
|
|
<h3>Discs #3 and 4</h3>
|
||
|
|
|
||
|
|
<p>The remaining two discs contains additional software packages for
|
||
|
|
FreeBSD. The packages should be clustered so that a package and
|
||
|
|
all of its dependencies are included on the same disc.</p>
|
||
|
|
|
||
|
|
<h1>Further Reading</h1>
|
||
|
|
|
||
|
|
<ul>
|
||
|
|
<li><a
|
||
|
|
href="http://people.FreeBSD.org/~murray/papers/releng.pdf">The
|
||
|
|
Release Engineering of FreeBSD 4.4</a>, BSDCon Europe 2001
|
||
|
|
Proceedings. [ <a
|
||
|
|
href="http://people.FreeBSD.org/~murray/papers/releng.pdf">PDF</a>
|
||
|
|
] [ <a
|
||
|
|
href="http://people.FreeBSD.org/~murray/papers/releng.ps">PostScript</a>
|
||
|
|
] [ <a
|
||
|
|
href="http://people.FreeBSD.org/~murray/#presentations">Slides</a>
|
||
|
|
]</li>
|
||
|
|
|
||
|
|
<li><a
|
||
|
|
href="http://www.netbsd.org/developers/releng/index.html">NetBSD
|
||
|
|
Developer Documentation : Release Engineering</a></li>
|
||
|
|
|
||
|
|
<li><a href="http://people.FreeBSD.org/~jhb/docs/releng.txt">John
|
||
|
|
Baldwin's Release Engineering Proposal</a></li>
|
||
|
|
</ul>
|
||
|
|
|
||
|
|
&footer;
|
||
|
|
|
||
|
|
</body>
|
||
|
|
</html>
|