1072 lines
41 KiB
XML
1072 lines
41 KiB
XML
<?xml version="1.0" encoding="iso-8859-1"?>
|
|
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN"
|
|
"http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd" [
|
|
<!ENTITY art.re.pkgs '<link xmlns="http://docbook.org/ns/docbook" xlink:href="{{{url.articles.releng-packages}}}/article.html">The Release Engineering of Third Party Packages</link>'>
|
|
]>
|
|
<article xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en">
|
|
|
|
<info><title>&os; Release Engineering</title>
|
|
|
|
|
|
|
|
<confgroup>
|
|
<confdates>November 2001</confdates>
|
|
<conftitle>BSDCon Europe</conftitle>
|
|
</confgroup>
|
|
|
|
<authorgroup>
|
|
<author><personname><firstname>Murray</firstname><surname>Stokely</surname></personname><personblurb>
|
|
<para>I've been involved in the development of &os; based products
|
|
since 1997 at Walnut Creek CDROM, BSDi, and now Wind River Systems.
|
|
&os; 4.4 was the first official release of &os; that I played
|
|
a significant part in.</para>
|
|
</personblurb><affiliation>
|
|
<address><email>murray@FreeBSD.org</email>
|
|
<otheraddr xlink:href="http://www.FreeBSD.org/~murray/">http://www.FreeBSD.org/~murray/</otheraddr>
|
|
</address>
|
|
</affiliation></author>
|
|
</authorgroup>
|
|
|
|
<legalnotice xml:id="trademarks" role="trademarks">
|
|
&tm-attrib.freebsd;
|
|
&tm-attrib.cvsup;
|
|
&tm-attrib.intel;
|
|
&tm-attrib.general;
|
|
</legalnotice>
|
|
|
|
<pubdate>$FreeBSD$</pubdate>
|
|
|
|
<abstract>
|
|
<para>
|
|
<warning>
|
|
<para>2013/02/26: This document is partially outdated and does not
|
|
accurately describe the current release procedures of the
|
|
&os; Release Engineering team. The &os; Release
|
|
Engineering team is currently reviewing this document and
|
|
will publish updated content soon.
|
|
</para>
|
|
</warning>
|
|
</para>
|
|
<para>This paper describes the approach used by the &os;
|
|
release engineering team to make production quality releases
|
|
of the &os; Operating System. It details the methodology
|
|
used for the official &os; releases and describes the tools
|
|
available for those interested in producing customized &os;
|
|
releases for corporate rollouts or commercial
|
|
productization.</para>
|
|
</abstract>
|
|
|
|
</info>
|
|
|
|
<!-- Introduction -->
|
|
<sect1 xml:id="introduction">
|
|
<title>Introduction</title>
|
|
|
|
<para>The development of &os; is a very open process. &os; is
|
|
comprised of contributions from thousands of people around the
|
|
world. The &os; Project provides
|
|
Subversion
|
|
<footnote>
|
|
<simpara>
|
|
Subversion, <uri xlink:href="http://subversion.apache.org">http://subversion.apache.org</uri>
|
|
</simpara>
|
|
</footnote>
|
|
access to the general public so that
|
|
others can have access to log messages, diffs (patches) between
|
|
development branches, and other productivity enhancements that
|
|
formal source code management provides. This has been a huge help
|
|
in attracting more talented developers to &os;. However, I
|
|
think everyone would agree that chaos would soon manifest if write
|
|
access to the main repository was opened up to everyone on the Internet.
|
|
Therefore only a <quote>select</quote> group of nearly 300 people are
|
|
given write access to the Subversion repository. These
|
|
<link xlink:href="&url.articles.contributors;/article.html#staff-committers">committers</link>
|
|
<footnote>
|
|
<simpara>
|
|
<link xlink:href="&url.articles.contributors;/article.html#staff-committers">FreeBSD committers</link>
|
|
</simpara>
|
|
</footnote>
|
|
are usually the people who do the bulk of &os; development. An elected
|
|
<link xlink:href="&url.base;/administration.html#t-core">Core Team</link>
|
|
<footnote>
|
|
<simpara>
|
|
<link xlink:href="&url.base;/administration.html#t-core">&os; Core Team</link>
|
|
</simpara>
|
|
</footnote>
|
|
of developers provide some level of direction over the project.</para>
|
|
|
|
<para>The rapid pace of <systemitem>&os;</systemitem>
|
|
development makes the main development branch unsuitable for the
|
|
everyday use by the general public. In particular, stabilizing
|
|
efforts are required for polishing the development system into a
|
|
production quality release. To solve this conflict, development
|
|
continues on several parallel tracks. The main development branch
|
|
is the <emphasis>HEAD</emphasis> or <emphasis>trunk</emphasis> of
|
|
our Subversion tree, known as <quote>&os;-CURRENT</quote> or
|
|
<quote>-CURRENT</quote> for short.</para>
|
|
|
|
<para>A set of more stable branches are maintained, known as
|
|
<quote>&os;-STABLE</quote> or <quote>-STABLE</quote> for short.
|
|
All branches live in a master Subversion repository maintained by the
|
|
&os; Project. &os;-CURRENT is the <quote>bleeding-edge</quote> of
|
|
&os; development where all new changes first enter the system.
|
|
&os;-STABLE is the development branch from which major releases
|
|
are made. Changes go into this branch at a different pace, and
|
|
with the general assumption that they have first gone into
|
|
&os;-CURRENT and have been thoroughly tested by our user
|
|
community.</para>
|
|
|
|
<para>The term <emphasis>stable</emphasis> in the name of the branch
|
|
refers to the presumed Application Binary Interface stability,
|
|
which is promised by the project. This means that a user
|
|
application compiled on an older version of the system from the
|
|
same branch works on a newer system from the same branch. The
|
|
ABI stability has improved greatly from the compared to previous
|
|
releases. In most cases, binaries from the older
|
|
<emphasis>STABLE</emphasis> systems run unmodified on newer systems,
|
|
including <emphasis>HEAD</emphasis>, assuming that the system
|
|
management interfaces are not used.</para>
|
|
|
|
<para>In the interim period between releases, weekly snapshots are
|
|
built automatically by the &os; Project build machines and made
|
|
available for download from <systemitem>ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/</systemitem>.
|
|
The widespread availability of binary release snapshots, and the
|
|
tendency of our user community to keep up with -STABLE development
|
|
with Subversion and <quote><command>make</command>
|
|
<buildtarget>buildworld</buildtarget></quote>
|
|
<footnote>
|
|
<simpara>
|
|
<link xlink:href="&url.books.handbook;/makeworld.html">Rebuilding "world"</link>
|
|
</simpara>
|
|
</footnote>
|
|
helps to keep
|
|
&os;-STABLE in a very reliable condition even before the
|
|
quality assurance activities ramp up pending a major
|
|
release.</para>
|
|
|
|
<para>In addition to installation ISO snapshots, weekly virtual
|
|
machine images are also provided for use with
|
|
<application>VirtualBox</application>,
|
|
<application>qemu</application>, or other popular emulation
|
|
software. The virtual machine images can be downloaded from
|
|
<systemitem>ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/VM-IMAGES/</systemitem>.</para>
|
|
|
|
<para>The virtual machine images are approximately 150MB &man.xz.1;
|
|
compressed, and contain a 10GB sparse filesystem when attached to
|
|
a virtual machine.</para>
|
|
|
|
<para>Bug reports and feature requests are continuously submitted by
|
|
users throughout the release cycle. Problems reports are entered into our
|
|
<application>GNATS</application> database
|
|
<footnote>
|
|
<simpara>
|
|
GNATS: The GNU Bug Tracking System
|
|
<uri xlink:href="http://www.gnu.org/software/gnats">http://www.gnu.org/software/gnats</uri>
|
|
</simpara>
|
|
</footnote>
|
|
through email, the &man.send-pr.1; application, or via the web
|
|
interface provided at <uri xlink:href="http://www.FreeBSD.org/send-pr.html">http://www.FreeBSD.org/send-pr.html</uri>.</para>
|
|
|
|
<para>To service our most conservative users, individual release
|
|
branches were introduced with &os; 4.3.
|
|
These release branches are created shortly before a final release
|
|
is made. After the release goes out, only the most critical
|
|
security fixes and additions are merged onto the release branch.
|
|
In addition to source updates via Subversion, binary patchkits are
|
|
available to keep systems on the
|
|
<emphasis>releng/<replaceable>X</replaceable>.<replaceable>Y</replaceable></emphasis>
|
|
branches updated.</para>
|
|
|
|
<sect2>
|
|
<title>What this article describes</title>
|
|
|
|
<para>The following sections of this article describe:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><xref linkend="release-proc"/></term>
|
|
|
|
<listitem>
|
|
<para>The different phases of the release engineering process
|
|
leading up to the actual system build.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="release-build"/></term>
|
|
|
|
<listitem>
|
|
<para>The actual build process.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="extensibility"/></term>
|
|
|
|
<listitem>
|
|
<para>How the base release may be extended by third parties.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="lessons-learned"/></term>
|
|
|
|
<listitem>
|
|
<para>Some of the lessons learned through the release of &os; 4.4.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="future"/></term>
|
|
|
|
<listitem>
|
|
<para>Future directions of development.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Process -->
|
|
<sect1 xml:id="release-proc">
|
|
<title>Release Process</title>
|
|
|
|
<para>New releases of &os; are released from the -STABLE branch
|
|
at approximately four month intervals. The &os; release
|
|
process begins to ramp up 70-80 days before the anticipated release
|
|
date when the release engineer sends an email to the development
|
|
mailing lists to remind developers that they only have 15 days to
|
|
integrate new changes before the code freeze. During this time,
|
|
many developers perform what have become known as <quote>MFC
|
|
sweeps</quote>.</para>
|
|
|
|
<para><acronym>MFC</acronym> stands for <quote>Merge From
|
|
CURRENT</quote> and it describes the process of merging a tested
|
|
change from our -CURRENT development branch to our -STABLE branch.
|
|
Project policy requires any change to be first applied to
|
|
trunk, and merged to the -STABLE branches after sufficient
|
|
external testing was done by -CURRENT users (developers are
|
|
expected to extensively test the change before committing to
|
|
-CURRENT, but it is impossible for a person to exercise all usages
|
|
of the general-purpose operating system). Minimal MFC period is 3
|
|
days, which is typically used only for trivial or critical
|
|
bugfixes.</para>
|
|
|
|
<sect2>
|
|
<title>Code Review</title>
|
|
|
|
<para>Sixty days before the anticipated release, the source
|
|
repository enters a <quote>code freeze</quote>. During this
|
|
time, all commits to the -STABLE branch must be approved by
|
|
&a.re;. The approval process is technically enforced by a
|
|
pre-commit hook. The kinds of changes that are allowed during
|
|
this period include:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Bug fixes.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Documentation updates.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Security-related fixes of any kind.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Minor changes to device drivers, such as adding new Device
|
|
IDs.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Driver updates from the vendors.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Any additional change that the release engineering team feels
|
|
is justified, given the potential risk.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Shortly after the code freeze is started, a
|
|
<emphasis>BETA1</emphasis> image is built and released for
|
|
widespread testing. During the code freeze, at least one beta
|
|
image or release candidate is released every two weeks until the
|
|
final release is ready. During the days preceeding the final
|
|
release, the release engineering team is in constant
|
|
communication with the security-officer team, the documentation
|
|
maintainers, and the port maintainers to ensure that all of the
|
|
different components required for a successful release are
|
|
available.</para>
|
|
|
|
<para>After the quality of the BETA images is satisfying enough,
|
|
and no large and potentially risky changes are planned, the
|
|
release branch is created and <emphasis>Release
|
|
Candidate</emphasis> (RC) images are built from the release
|
|
branch, instead of the BETA images from the STABLE branch.
|
|
Also, the freeze on the STABLE branch is lifted and release
|
|
branch enters a <quote>hard code freeze</quote> where it becomes
|
|
much harder to justify new changes to the system unless a
|
|
serious bug-fix or security issue is involved.</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Final Release Checklist</title>
|
|
|
|
<para>When several BETA images have been made available for
|
|
widespread testing and all major issues have been resolved, the
|
|
final release <quote>polishing</quote> can begin.</para>
|
|
|
|
<sect3 xml:id="rel-branch">
|
|
<title>Creating the Release Branch</title>
|
|
|
|
<note>
|
|
<para>In all examples below, <literal>$FSVN</literal>
|
|
refers to the location of the &os; Subversion repository,
|
|
<literal>svn+ssh://svn.FreeBSD.org/base/</literal>.</para>
|
|
</note>
|
|
|
|
<para>The layout of &os; branches in Subversion is
|
|
described in the <link xlink:href="&url.articles.committers-guide;/subversion-primer.html#subversion-primer-base-layout">Committer's Guide</link>.
|
|
The first step in creating a branch is to
|
|
identify the revision of the
|
|
<literal>stable/X</literal> sources
|
|
that you want to branch <emphasis>from</emphasis>.</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn log -v $FSVN/stable/9</userinput></screen>
|
|
|
|
<para>The next step is to create the <emphasis>release branch</emphasis>
|
|
</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn cp $FSVN/stable/9@REVISION $FSVN/releng/9.2</userinput></screen>
|
|
|
|
<para>This branch can be checked out:</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn co $FSVN/releng/9.2 src</userinput></screen>
|
|
|
|
<note>
|
|
<para>Creating the <literal>releng</literal> branch and
|
|
<literal>release</literal> tags is done by the <link xlink:href="&url.base;/administration.html#t-re">Release
|
|
Engineering Team</link>.
|
|
</para>
|
|
</note>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-head" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; Development Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng3" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 3.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng4" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 4.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng5" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 5.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng6" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 6.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng7" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 7.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng8" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 8.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng9" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 9.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
</sect3>
|
|
|
|
<sect3 xml:id="versionbump">
|
|
<title>Bumping up the Version Number</title>
|
|
|
|
<para>Before the final release can be tagged, built, and
|
|
released, the following files need to be modified to reflect
|
|
the correct version of &os;:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.xml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/porters-handbook/book.xml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/share/xml/freebsd.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/Makefile.inc1</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/UPDATING</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/gnu/usr.bin/groff/tmac/mdoc.local</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/Makefile</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/share/xml/release.dsl</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/examples/Makefile.relnotesng</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/xml/release.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/share/examples/cvsup/standard-supfile</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/conf/newvers.sh</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/sys/param.h</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/usr.sbin/pkg_install/add/main.c</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/docs/man.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>www/en/cgi/ports.cgi</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>ports/Tools/scripts/release/config</filename></para>
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
<para>The release notes and errata files also need to be adjusted for the
|
|
new release (on the release branch) and truncated appropriately
|
|
(on the stable/current branch):</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/relnotes/common/new.xml
|
|
</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/errata/article.xml
|
|
</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para><application>Sysinstall</application> should be updated to note
|
|
the number of available ports and the amount of disk space required
|
|
for the Ports Collection.
|
|
<footnote>
|
|
<simpara>
|
|
&os; Ports Collection
|
|
<uri xlink:href="http://www.FreeBSD.org/ports">http://www.FreeBSD.org/ports</uri>
|
|
</simpara>
|
|
</footnote>
|
|
This information is currently kept in
|
|
<filename>src/usr.sbin/sysinstall/dist.c</filename>.</para>
|
|
|
|
<para>After the release has been built, a number of files should
|
|
be updated to announce the release to the world. These files
|
|
are relative to <literal>head/</literal> within the
|
|
<literal>doc/</literal> subversion tree.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>share/images/articles/releng/branches-relengX.pic</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>head/share/xml/release.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>en_US.ISO8859-1/htdocs/releases/*</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>en_US.ISO8859-1/htdocs/releng/index.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>share/xml/news.xml</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Additionally, update the <quote>BSD Family Tree</quote>
|
|
file:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/share/misc/bsd-family-tree</filename></para>
|
|
</listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Creating the Release Tag</title>
|
|
|
|
<para>When the final release is ready, the following command
|
|
will create the <literal>release/9.2.0</literal>
|
|
tag.</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn cp $FSVN/releng/9.2 $FSVN/release/9.2.0</userinput></screen>
|
|
|
|
<para>The Documentation and Ports managers are responsible for
|
|
tagging their respective trees with the <literal>tags/RELEASE_9_2_0</literal>
|
|
tag.</para>
|
|
|
|
<sidebar>
|
|
<para>When the Subversion <command>svn cp</command> command
|
|
is used to create a <emphasis>release tag</emphasis>,
|
|
this identifies the source at a specific point in time.
|
|
By creating tags, we ensure that future release builders
|
|
will always be able to use the exact same source we used to create the
|
|
official &os; Project releases.</para>
|
|
</sidebar>
|
|
|
|
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Building -->
|
|
<sect1 xml:id="release-build">
|
|
<title>Release Building</title>
|
|
|
|
<para>&os; <quote>releases</quote> can be built by anyone with a
|
|
fast machine and access to a source repository. (That should be
|
|
everyone, since we offer Subversion access !
|
|
See the
|
|
<link xlink:href="&url.books.handbook;/svn.html">Subversion section
|
|
in the Handbook</link> for
|
|
details.) The <emphasis>only</emphasis> special requirement is
|
|
that the &man.md.4; device must be available. If the
|
|
device is not loaded into your kernel, then the kernel module
|
|
should be automatically loaded when &man.mdconfig.8; is executed
|
|
during the boot media creation phase. All of the tools necessary
|
|
to build a release are available from the Subversion repository in
|
|
<filename>src/release</filename>. These tools aim to provide a
|
|
consistent way to build &os; releases. A complete release can
|
|
actually be built with only a single command, including the
|
|
creation of <acronym>ISO</acronym> images suitable for burning to
|
|
CDROM or DVD, and an FTP install directory. &man.release.7; fully
|
|
documents the <command>src/release/generate-release.sh</command>
|
|
script which is used to build a release. <command>generate-release.sh</command>
|
|
is a wrapper around the Makefile target: <command>make release</command>.</para>
|
|
|
|
<sect2>
|
|
<title>Building a Release</title>
|
|
|
|
<para>&man.release.7; documents the exact commands required to
|
|
build a &os; release. The following sequences of commands can build
|
|
an 9.2.0 release:</para>
|
|
|
|
<screen>&prompt.root; <userinput>cd /usr/src/release</userinput></screen>
|
|
<screen>&prompt.root; <userinput>sh generate-release.sh release/9.2.0 /local3/release</userinput></screen>
|
|
|
|
<para>After running these commands, all prepared release
|
|
files are available in <filename>/local3/release/R</filename>
|
|
directory.</para>
|
|
|
|
<para>The release <filename>Makefile</filename> can be broken down into several distinct
|
|
steps.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Creation of a sanitized system environment in a separate
|
|
directory hierarchy with <quote><command>make
|
|
installworld</command></quote>.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Checkout from Subversion of a clean version of the system source,
|
|
documentation, and ports into the release build hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Population of <filename>/etc</filename> and
|
|
<filename>/dev</filename> in the chrooted
|
|
environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>chroot into the release build hierarchy, to make it harder for
|
|
the outside environment to taint this build.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>make world</command>
|
|
in the chrooted environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build of Kerberos-related binaries.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build <filename>GENERIC</filename> kernel.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Creation of a staging directory tree where the binary
|
|
distributions will be built and packaged.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the documentation toolchain needed to
|
|
convert the documentation source (SGML) into HTML and text documents
|
|
that will accompany the release.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the actual documentation
|
|
(user manuals, tutorials, release notes, hardware compatibility lists,
|
|
and so on.)</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Package up distribution tarballs of the binaries and sources.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Create FTP installation hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>(optionally)</emphasis> Create ISO images for
|
|
CDROM/DVD media.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>For more information about the release build infrastructure,
|
|
please see &man.release.7;.</para>
|
|
|
|
<note><para>It is important to remove any site-specific settings
|
|
from <filename>/etc/make.conf</filename>. For example, it would
|
|
be unwise to distribute binaries that were built on a system
|
|
with <varname>CPUTYPE</varname> set to a specific
|
|
processor.</para></note>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Contributed Software (<quote>ports</quote>)</title>
|
|
|
|
<para>The <link xlink:href="http://www.FreeBSD.org/ports">&os; Ports
|
|
collection</link> is a collection of over &os.numports;
|
|
third-party software packages available for &os;. The &a.portmgr;
|
|
is responsible for maintaining a consistent ports tree that can be used
|
|
to create the binary packages that accompany official &os;
|
|
releases.</para>
|
|
|
|
<para>The release engineering activities for our collection of
|
|
third-party packages is beyond the scope of this document. A
|
|
separate article, &art.re.pkgs;, covers this topic
|
|
in depth.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Release ISOs</title>
|
|
|
|
<para>Starting with &os; 4.4, the &os; Project decided to
|
|
release all four ISO images that were previously sold on the
|
|
<emphasis>BSDi/Wind River Systems/FreeBSD Mall</emphasis>
|
|
<quote>official</quote> CDROM distributions. Each of the four
|
|
discs must contain a <filename>README.TXT</filename> file that
|
|
explains the contents of the disc, a
|
|
<filename>CDROM.INF</filename> file that provides meta-data for
|
|
the disc so that &man.sysinstall.8; can validate and use the
|
|
contents, and a <filename>filename.txt</filename> file that
|
|
provides a manifest for the disc. This
|
|
<emphasis>manifest</emphasis> can be created with a simple
|
|
command:</para>
|
|
|
|
<screen>/stage/cdrom&prompt.root; <userinput>find . -type f | sed -e 's/^\.\///' | sort > filename.txt</userinput></screen>
|
|
|
|
<para>The specific requirements of each CD are outlined below.</para>
|
|
|
|
<sect3>
|
|
<title>Disc 1</title>
|
|
|
|
<para>The first disc is almost completely created by
|
|
<command>make
|
|
release</command>. The only changes
|
|
that should be made to the <filename>disc1</filename> directory are the addition of
|
|
a <filename>tools</filename> directory, and as many popular
|
|
third party software packages as will fit on the disc. The
|
|
<filename>tools</filename> directory contains software that allow users to create
|
|
installation floppies from other operating systems. This disc
|
|
should be made bootable so that users of modern PCs do not
|
|
need to create installation floppy disks.</para>
|
|
|
|
<para>If a custom kernel of &os; is to be included, then
|
|
&man.sysinstall.8; and &man.release.7; must be updated to
|
|
include installation instructions. The relevant code is contained
|
|
in <filename>src/release</filename> and <filename>src/usr.sbin/sysinstall</filename>.
|
|
Specifically, the file <filename>src/release/Makefile</filename>, and
|
|
<filename>dist.c</filename>, <filename>dist.h</filename>,
|
|
<filename>menus.c</filename>, <filename>install.c</filename>, and
|
|
<filename>Makefile</filename> will need to be updated under
|
|
<filename>src/usr.sbin/sysinstall</filename>. Optionally, you may choose
|
|
to update <filename>sysinstall.8</filename>.</para>
|
|
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Disc 2</title>
|
|
|
|
<para>The second disc is also largely created by <command>make
|
|
release</command>. This disc contains a <quote>live
|
|
filesystem</quote> that can be used from &man.sysinstall.8; to
|
|
troubleshoot a &os; installation. This disc should be
|
|
bootable and should also contain a compressed copy of the CVS
|
|
repository in the <filename>CVSROOT</filename> directory and
|
|
commercial software demos in the <filename>commerce</filename>
|
|
directory.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Discs 3 and 4</title>
|
|
|
|
<para>The remaining two discs contain additional software
|
|
packages for &os;. The packages should be clustered so that
|
|
a package and all of its <emphasis>dependencies</emphasis> are
|
|
included on the same disc. More information about the
|
|
creation of these discs is provided in the &art.re.pkgs;
|
|
article.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Multi-volume support</title>
|
|
|
|
<para><application>Sysinstall</application> supports multiple
|
|
volume package installations. This requires that each disc
|
|
have an <filename>INDEX</filename> file containing all of the
|
|
packages on all volumes of a set, along with an extra field
|
|
that indicates which volume that particular package is on.
|
|
Each volume in the set must also have the
|
|
<literal>CD_VOLUME</literal> variable set in the
|
|
<filename>cdrom.inf</filename> file so that sysinstall can
|
|
tell which volume is which. When a user attempts to install a
|
|
package that is not on the current disc, sysinstall will
|
|
prompt the user to insert the appropriate one.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Distribution -->
|
|
<sect1 xml:id="distribution">
|
|
<title>Distribution</title>
|
|
|
|
<sect2 xml:id="dist-ftp">
|
|
<title>FTP Sites</title>
|
|
|
|
<para>When the release has been thoroughly tested and packaged for
|
|
distribution, the master FTP site must be updated. The official
|
|
&os; public FTP sites are all mirrors of a master server that
|
|
is open only to other FTP sites. This site is known as
|
|
<systemitem>ftp-master</systemitem>. When the release is ready, the
|
|
following files must be modified on <systemitem>ftp-master</systemitem>:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/arch/X.Y-RELEASE/</filename></term>
|
|
<listitem>
|
|
<para>The installable FTP directory as output from <command>make
|
|
release</command>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/ports/arch/packages-X.Y-release/</filename></term>
|
|
<listitem><para>The complete package build for this
|
|
release.</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/arch/X.Y-RELEASE/tools</filename></term>
|
|
<listitem><para>A symlink to
|
|
<filename>../../../tools</filename>.</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/arch/X.Y-RELEASE/packages</filename></term>
|
|
<listitem><para>A symlink to
|
|
<filename>../../../ports/arch/packages-X.Y-release</filename>.</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/arch/ISO-IMAGES/X.Y/X.Y-RELEASE-arch-*.iso</filename></term>
|
|
<listitem><para>The ISO images. The <quote>*</quote> is
|
|
<filename>disc1</filename>, <filename>disc2</filename>, etc.
|
|
Only if there is a <filename>disc1</filename> and there is an
|
|
alternative first installation CD (for example a
|
|
stripped-down install with no windowing system) there may
|
|
be a <filename>mini</filename> as well.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
<para>For more information about the distribution mirror
|
|
architecture of the &os; FTP sites, please see the <link xlink:href="&url.articles.hubs;/">Mirroring &os;</link> article.</para>
|
|
|
|
<para>It may take many hours to two days after updating
|
|
<systemitem>ftp-master</systemitem> before a majority of the Tier-1 FTP
|
|
sites have the new software depending on whether or not a package
|
|
set got loaded at the same time. It is imperative that the release
|
|
engineers coordinate with the &a.mirror-announce; before announcing the general
|
|
availability of new software on the FTP sites. Ideally
|
|
the release package set should be loaded at least four
|
|
days prior to release day. The release bits should be
|
|
loaded between 24 and 48 hours before the planned release
|
|
time with <quote>other</quote> file permissions turned off.
|
|
This will allow the mirror sites to download it but the
|
|
general public will not be able to download it from the mirror
|
|
sites. Mail should be sent to &a.mirror-announce; at the time
|
|
the release bits get posted saying the release has been staged
|
|
and giving the time that the mirror sites should begin allowing
|
|
access. Be sure to include a time zone with the
|
|
time, for example make it relative to GMT.</para>
|
|
</sect2>
|
|
|
|
<sect2 xml:id="dist-cdrom">
|
|
<title>CD-ROM Replication</title>
|
|
|
|
<para>Coming soon: Tips for sending &os; ISOs to a replicator
|
|
and quality assurance measures to be taken.</para>
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
<!-- Extensibility -->
|
|
<sect1 xml:id="extensibility">
|
|
<title>Extensibility</title>
|
|
|
|
<para>Although &os; forms a complete operating system, there is
|
|
nothing that forces you to use the system exactly as we have
|
|
packaged it up for distribution. We have tried to design the
|
|
system to be as extensible as possible so that it can serve as a
|
|
platform that other commercial products can be built on top
|
|
of. The only <quote>rule</quote> we have about this is that if you
|
|
are going to distribute &os; with non-trivial changes, we
|
|
encourage you to document your enhancements! The &os; community
|
|
can only help support users of the software we provide. We
|
|
certainly encourage innovation in the form of advanced
|
|
installation and administration tools, for example, but we cannot
|
|
be expected to answer questions about it.</para>
|
|
|
|
<sect2>
|
|
<title>Scripting <command>sysinstall</command></title>
|
|
|
|
<para>The &os; system installation and configuration tool,
|
|
&man.sysinstall.8;, can be scripted to provide automated installs
|
|
for large sites. This functionality can be used in conjunction
|
|
with &intel; PXE
|
|
<footnote>
|
|
<simpara>
|
|
<uri xlink:href="&url.books.handbook;/network-pxe-nfs.html">&url.books.handbook;/network-pxe-nfs.html</uri>
|
|
</simpara>
|
|
</footnote>
|
|
to bootstrap systems from the network.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Lessons Learned -->
|
|
<sect1 xml:id="lessons-learned">
|
|
<title>Lessons Learned from &os; 4.4</title>
|
|
|
|
<para>The release engineering process for 4.4 formally began on
|
|
August 1st, 2001. After that date all commits to the
|
|
<literal>RELENG_4</literal> branch of &os; had to be explicitly
|
|
approved by the &a.re;. The first
|
|
release candidate for the x86 architecture was released on August
|
|
16, followed by 4 more release candidates leading up to the final
|
|
release on September 18th. The security officer was very involved
|
|
in the last week of the process as several security issues were
|
|
found in the earlier release candidates. A total of over
|
|
<emphasis>500</emphasis> emails were sent to the &a.re; in
|
|
little over a month.</para>
|
|
|
|
<para>Our user community has made it very clear that the security
|
|
and stability of a &os; release should not be sacrificed for
|
|
any self-imposed deadlines or target release dates. The &os;
|
|
Project has grown tremendously over its lifetime and the need for
|
|
standardized release engineering procedures has never been more
|
|
apparent. This will become even more important as &os; is
|
|
ported to new platforms.</para>
|
|
</sect1>
|
|
|
|
<!-- Future Directions -->
|
|
<sect1 xml:id="future">
|
|
<title>Future Directions</title>
|
|
|
|
<para>It is imperative for our release engineering activities to
|
|
scale with our growing userbase. Along these lines we are working
|
|
very hard to document the procedures involved in producing &os;
|
|
releases.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><emphasis>Parallelism</emphasis> - Certain portions of the
|
|
release build are actually <quote>embarrassingly
|
|
parallel</quote>. Most of the tasks are very I/O intensive,
|
|
so having multiple high-speed disk drives is actually more important than
|
|
using multiple processors in speeding up the <command>make
|
|
release</command> process. If multiple disks are used for
|
|
different hierarchies in the &man.chroot.2;
|
|
environment, then the CVS checkout of the <filename>ports</filename> and <filename>doc</filename> trees
|
|
can be happening simultaneously as the <command>make
|
|
world</command> on another disk. Using a
|
|
<acronym>RAID</acronym> solution (hardware or software) can
|
|
significantly decrease the overall build time.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Cross-building releases</emphasis> - Building
|
|
IA-64 or Alpha release on x86 hardware? <command>make
|
|
TARGET=ia64 release</command>.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Regression Testing</emphasis> - We need better
|
|
automated correctness testing for &os;.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Installation Tools</emphasis> - Our installation
|
|
program has long since outlived its intended life span.
|
|
Several projects are under development to provide a more
|
|
advanced installation mechanism. The libh project was one
|
|
such project that aimed to provide an intelligent new package
|
|
framework and GUI installation program.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</sect1>
|
|
|
|
<!-- Acknowledgements -->
|
|
<sect1 xml:id="ackno">
|
|
<title>Acknowledgements</title>
|
|
|
|
<para>I would like to thank Jordan Hubbard for giving me the
|
|
opportunity to take on some of the release engineering
|
|
responsibilities for &os; 4.4 and also for all of his work
|
|
throughout the years making &os; what it is today. Of course
|
|
the release would not have been possible without all of the
|
|
release-related work done by &a.asami.email;, &a.steve.email;, &a.bmah.email;, &a.nik.email;,
|
|
&a.obrien.email;, &a.kris.email;, &a.jhb.email; and the rest of the &os; development
|
|
community. I would also like to thank &a.rgrimes.email;, &a.phk.email;, and others
|
|
who worked on the release engineering tools in the very early days
|
|
of &os;. This article was influenced by release engineering
|
|
documents from the CSRG
|
|
<footnote>
|
|
<simpara>
|
|
Marshall Kirk McKusick, Michael J. Karels, and Keith Bostic:
|
|
<link xlink:href="http://docs.FreeBSD.org/44doc/papers/releng.html">
|
|
<emphasis>The Release Engineering of 4.3BSD</emphasis></link>
|
|
</simpara>
|
|
</footnote>
|
|
,
|
|
the NetBSD Project ,
|
|
<footnote>
|
|
<simpara>
|
|
NetBSD Developer Documentation: Release Engineering
|
|
<uri xlink:href="http://www.NetBSD.org/developers/releng/index.html">http://www.NetBSD.org/developers/releng/index.html</uri>
|
|
</simpara>
|
|
</footnote>
|
|
, and John
|
|
Baldwin's proposed release engineering process notes.
|
|
<footnote>
|
|
<simpara>
|
|
John Baldwin's &os; Release Engineering Proposal
|
|
<uri xlink:href="http://people.FreeBSD.org/~jhb/docs/releng.txt">http://people.FreeBSD.org/~jhb/docs/releng.txt</uri>
|
|
</simpara>
|
|
</footnote>
|
|
</para>
|
|
</sect1>
|
|
|
|
</article>
|