452b46c55f
Apparently, igor does not like the occurance of the second <para> in the <para><note><para> sequence in the abstract. Replacing it with something else makes the file not pass the DTD checks. Err on the side of letting the file compile, leaving a couple of igor checks unresolved. Overall, the docbook xml source in this file should be much cleaner now.
1061 lines
38 KiB
XML
1061 lines
38 KiB
XML
<?xml version="1.0" encoding="iso-8859-1"?>
|
|
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN"
|
|
"http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd" [
|
|
]>
|
|
<article xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
|
|
xml:lang="en">
|
|
|
|
<info>
|
|
<title>&os; Release Engineering</title>
|
|
|
|
<confgroup>
|
|
<confdates>November 2001</confdates>
|
|
<conftitle>BSDCon Europe</conftitle>
|
|
</confgroup>
|
|
|
|
<authorgroup>
|
|
<author>
|
|
<personname>
|
|
<firstname>Murray</firstname>
|
|
<surname>Stokely</surname>
|
|
</personname>
|
|
<personblurb>
|
|
<para>I've been involved in the development of &os; based
|
|
products since 1997 at Walnut Creek CDROM, BSDi, and now
|
|
Wind River Systems. &os; 4.4 was the first official
|
|
release of &os; that I played a significant part
|
|
in.</para>
|
|
</personblurb>
|
|
<affiliation>
|
|
<address>
|
|
<email>murray@FreeBSD.org</email>
|
|
<otheraddr
|
|
xlink:href="https://people.FreeBSD.org/~murray/">https://people.FreeBSD.org/~murray/</otheraddr>
|
|
</address>
|
|
</affiliation>
|
|
</author>
|
|
</authorgroup>
|
|
|
|
<legalnotice xml:id="trademarks" role="trademarks">
|
|
&tm-attrib.freebsd;
|
|
&tm-attrib.intel;
|
|
&tm-attrib.general;
|
|
</legalnotice>
|
|
|
|
<pubdate>$FreeBSD$</pubdate>
|
|
|
|
<abstract>
|
|
<para>
|
|
<note>
|
|
<para>This document is outdated and does not accurately
|
|
describe the current release procedures of the &os;
|
|
Release Engineering team. It is retained for historical
|
|
purposes. The current procedures used by the &os; Release
|
|
Engineering team are available in the <link
|
|
xlink:href="&url.articles.freebsd-releng;/article.html">&os;
|
|
Release Engineering</link> article.</para></note></para>
|
|
|
|
<para>This paper describes the approach used by the &os;
|
|
release engineering team to make production quality
|
|
releases of the &os; Operating System. It details the
|
|
methodology used for the official &os; releases and
|
|
describes the tools available for those interested in
|
|
producing customized &os; releases for corporate rollouts
|
|
or commercial productization.</para>
|
|
</abstract>
|
|
</info>
|
|
|
|
<!-- Introduction -->
|
|
<sect1 xml:id="introduction">
|
|
<title>Introduction</title>
|
|
|
|
<para>The development of &os; is a very open process. &os; is
|
|
comprised of contributions from thousands of people around the
|
|
world. The &os; Project provides Subversion <footnote>
|
|
<simpara>Subversion, <uri
|
|
xlink:href="http://subversion.apache.org">http://subversion.apache.org</uri>
|
|
</simpara></footnote> access to the general public so that
|
|
others can have access to log messages, diffs (patches)
|
|
between development branches, and other productivity
|
|
enhancements that formal source code management provides.
|
|
This has been a huge help in attracting more talented
|
|
developers to &os;. However, I think everyone would agree
|
|
that chaos would soon manifest if write access to the main
|
|
repository was opened up to everyone on the Internet.
|
|
Therefore only a <quote>select</quote> group of nearly 300
|
|
people are given write access to the Subversion repository.
|
|
These <link
|
|
xlink:href="&url.articles.contributors;/article.html#staff-committers">committers</link>
|
|
<footnote>
|
|
<simpara><link
|
|
xlink:href="&url.articles.contributors;/article.html#staff-committers">FreeBSD
|
|
committers</link>
|
|
</simpara>
|
|
</footnote>
|
|
are usually the people who do the bulk of &os; development.
|
|
An elected <link
|
|
xlink:href="&url.base;/administration.html#t-core">Core
|
|
Team</link>
|
|
<footnote>
|
|
<simpara><link
|
|
xlink:href="&url.base;/administration.html#t-core">&os;
|
|
Core Team</link></simpara>
|
|
</footnote>
|
|
of developers provide some level of direction over the
|
|
project.</para>
|
|
|
|
<para>The rapid pace of <systemitem>&os;</systemitem>
|
|
development makes the main development branch unsuitable for
|
|
the everyday use by the general public. In particular,
|
|
stabilizing efforts are required for polishing the development
|
|
system into a production quality release. To solve this
|
|
conflict, development continues on several parallel tracks.
|
|
The main development branch is the <emphasis>HEAD</emphasis>
|
|
or <emphasis>trunk</emphasis> of our Subversion tree, known as
|
|
<quote>&os;-CURRENT</quote> or <quote>-CURRENT</quote> for
|
|
short.</para>
|
|
|
|
<para>A set of more stable branches are maintained, known as
|
|
<quote>&os;-STABLE</quote> or <quote>-STABLE</quote> for
|
|
short. All branches live in a master Subversion repository
|
|
maintained by the &os; Project. &os;-CURRENT is the
|
|
<quote>bleeding-edge</quote> of &os; development where all new
|
|
changes first enter the system. &os;-STABLE is the
|
|
development branch from which major releases are made.
|
|
Changes go into this branch at a different pace, and with the
|
|
general assumption that they have first gone into &os;-CURRENT
|
|
and have been thoroughly tested by our user community.</para>
|
|
|
|
<para>The term <emphasis>stable</emphasis> in the name of the
|
|
branch refers to the presumed Application Binary Interface
|
|
stability, which is promised by the project. This means that
|
|
a user application compiled on an older version of the system
|
|
from the same branch works on a newer system from the same
|
|
branch. The ABI stability has improved greatly from the
|
|
compared to previous releases. In most cases, binaries from
|
|
the older <emphasis>STABLE</emphasis> systems run unmodified
|
|
on newer systems, including <emphasis>HEAD</emphasis>,
|
|
assuming that the system management interfaces are not
|
|
used.</para>
|
|
|
|
<para>In the interim period between releases, weekly snapshots
|
|
are built automatically by the &os; Project build machines and
|
|
made available for download from
|
|
<systemitem>ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/</systemitem>.
|
|
The widespread availability of binary release snapshots, and
|
|
the tendency of our user community to keep up with -STABLE
|
|
development with Subversion and <quote><command>make</command>
|
|
<buildtarget>buildworld</buildtarget></quote> <footnote>
|
|
<simpara><link
|
|
xlink:href="&url.books.handbook;/makeworld.html">Rebuilding
|
|
"world"</link></simpara></footnote> helps to keep
|
|
&os;-STABLE in a very reliable condition even before the
|
|
quality assurance activities ramp up pending a major
|
|
release.</para>
|
|
|
|
<para>In addition to installation ISO snapshots, weekly virtual
|
|
machine images are also provided for use with
|
|
<application>VirtualBox</application>,
|
|
<application>qemu</application>, or other popular emulation
|
|
software. The virtual machine images can be downloaded from
|
|
<systemitem>ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/VM-IMAGES/</systemitem>.</para>
|
|
|
|
<para>The virtual machine images are approximately 150MB
|
|
&man.xz.1; compressed, and contain a 10GB sparse filesystem
|
|
when attached to a virtual machine.</para>
|
|
|
|
<para>Bug reports and feature requests are continuously
|
|
submitted by users throughout the release cycle. Problems
|
|
reports are entered into our
|
|
<application>Bugzilla</application> database through the web
|
|
interface provided at <uri
|
|
xlink:href="https://www.freebsd.org/support/bugreports.html">https://www.freebsd.org/support/bugreports.html</uri>.</para>
|
|
|
|
<para>To service our most conservative users, individual release
|
|
branches were introduced with &os; 4.3. These release
|
|
branches are created shortly before a final release is made.
|
|
After the release goes out, only the most critical security
|
|
fixes and additions are merged onto the release branch. In
|
|
addition to source updates via Subversion, binary patchkits
|
|
are available to keep systems on the
|
|
<emphasis>releng/<replaceable>X</replaceable>.<replaceable>Y</replaceable></emphasis>
|
|
branches updated.</para>
|
|
|
|
<sect2>
|
|
<title>What This Article Describes</title>
|
|
|
|
<para>The following sections of this article describe:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><xref linkend="release-proc"/></term>
|
|
|
|
<listitem>
|
|
<para>The different phases of the release engineering
|
|
process leading up to the actual system build.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="release-build"/></term>
|
|
|
|
<listitem>
|
|
<para>The actual build process.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="extensibility"/></term>
|
|
|
|
<listitem>
|
|
<para>How the base release may be extended by third
|
|
parties.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="lessons-learned"/></term>
|
|
|
|
<listitem>
|
|
<para>Some of the lessons learned through the release of
|
|
&os; 4.4.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><xref linkend="future"/></term>
|
|
|
|
<listitem>
|
|
<para>Future directions of development.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Process -->
|
|
<sect1 xml:id="release-proc">
|
|
<title>Release Process</title>
|
|
|
|
<para>New releases of &os; are released from the -STABLE branch
|
|
at approximately four month intervals. The &os; release
|
|
process begins to ramp up 70-80 days before the anticipated
|
|
release date when the release engineer sends an email to the
|
|
development mailing lists to remind developers that they only
|
|
have 15 days to integrate new changes before the code freeze.
|
|
During this time, many developers perform what have become
|
|
known as <quote>MFC sweeps</quote>.</para>
|
|
|
|
<para><acronym>MFC</acronym> stands for <quote>Merge From
|
|
CURRENT</quote> and it describes the process of merging a
|
|
tested change from our -CURRENT development branch to our
|
|
-STABLE branch. Project policy requires any change to be
|
|
first applied to trunk, and merged to the -STABLE branches
|
|
after sufficient external testing was done by -CURRENT users
|
|
(developers are expected to extensively test the change before
|
|
committing to -CURRENT, but it is impossible for a person to
|
|
exercise all usages of the general-purpose operating system).
|
|
Minimal MFC period is 3 days, which is typically used only for
|
|
trivial or critical bugfixes.</para>
|
|
|
|
<sect2>
|
|
<title>Code Review</title>
|
|
|
|
<para>Sixty days before the anticipated release, the source
|
|
repository enters a <quote>code freeze</quote>. During this
|
|
time, all commits to the -STABLE branch must be approved by
|
|
&a.re;. The approval process is technically enforced by a
|
|
pre-commit hook. The kinds of changes that are allowed
|
|
during this period include:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Bug fixes.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Documentation updates.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Security-related fixes of any kind.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Minor changes to device drivers, such as adding new
|
|
Device IDs.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Driver updates from the vendors.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Any additional change that the release engineering
|
|
team feels is justified, given the potential
|
|
risk.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Shortly after the code freeze is started, a
|
|
<emphasis>BETA1</emphasis> image is built and released for
|
|
widespread testing. During the code freeze, at least one
|
|
beta image or release candidate is released every two weeks
|
|
until the final release is ready. During the days preceding
|
|
the final release, the release engineering team is in
|
|
constant communication with the security-officer team, the
|
|
documentation maintainers, and the port maintainers to
|
|
ensure that all of the different components required for a
|
|
successful release are available.</para>
|
|
|
|
<para>After the quality of the BETA images is satisfying
|
|
enough, and no large and potentially risky changes are
|
|
planned, the release branch is created and <emphasis>Release
|
|
Candidate</emphasis> (RC) images are built from the
|
|
release branch, instead of the BETA images from the STABLE
|
|
branch. Also, the freeze on the STABLE branch is lifted and
|
|
release branch enters a <quote>hard code freeze</quote>
|
|
where it becomes much harder to justify new changes to the
|
|
system unless a serious bug-fix or security issue is
|
|
involved.</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Final Release Checklist</title>
|
|
|
|
<para>When several BETA images have been made available for
|
|
widespread testing and all major issues have been resolved,
|
|
the final release <quote>polishing</quote> can begin.</para>
|
|
|
|
<sect3 xml:id="rel-branch">
|
|
<title>Creating the Release Branch</title>
|
|
|
|
<note>
|
|
<para>In all examples below,
|
|
<literal>$FSVN</literal> refers to the location
|
|
of the &os; Subversion repository,
|
|
<literal>svn+ssh://svn.FreeBSD.org/base/</literal>.</para>
|
|
</note>
|
|
|
|
<para>The layout of &os; branches in Subversion is described
|
|
in the <link
|
|
xlink:href="&url.articles.committers-guide;/subversion-primer.html#subversion-primer-base-layout">Committer's
|
|
Guide</link>. The first step in creating a branch is to
|
|
identify the revision of the
|
|
<literal>stable/<replaceable>X</replaceable></literal>
|
|
sources that you want to branch
|
|
<emphasis>from</emphasis>.</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn log -v $FSVN/stable/9</userinput></screen>
|
|
|
|
<para>The next step is to create the <emphasis>release
|
|
branch</emphasis></para>
|
|
|
|
<screen>&prompt.root; <userinput>svn cp $FSVN/stable/9@REVISION $FSVN/releng/9.2</userinput></screen>
|
|
|
|
<para>This branch can be checked out:</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn co $FSVN/releng/9.2 src</userinput></screen>
|
|
|
|
<note>
|
|
<para>Creating the <literal>releng</literal> branch and
|
|
<literal>release</literal> tags is done by the <link
|
|
xlink:href="&url.base;/administration.html#t-re">Release
|
|
Engineering Team</link>.</para>
|
|
</note>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-head" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; Development Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng3" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 3.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng4" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 4.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng5" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 5.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng6" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 6.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng7" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 7.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng8" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 8.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="branches-releng9" align="center"/>
|
|
</imageobject>
|
|
|
|
<textobject>
|
|
<phrase>&os; 9.x STABLE Branch</phrase>
|
|
</textobject>
|
|
</mediaobject>
|
|
</sect3>
|
|
|
|
<sect3 xml:id="versionbump">
|
|
<title>Bumping up the Version Number</title>
|
|
|
|
<para>Before the final release can be tagged, built, and
|
|
released, the following files need to be modified to reflect
|
|
the correct version of &os;:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/books/porters-handbook/book.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/htdocs/cgi/ports.cgi</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>ports/Tools/scripts/release/config</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/share/xml/freebsd.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/Makefile.inc1</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/UPDATING</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/gnu/usr.bin/groff/tmac/mdoc.local</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/Makefile</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/share/xml/release.dsl</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/examples/Makefile.relnotesng</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/share/xml/release.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/conf/newvers.sh</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/sys/sys/param.h</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/usr.sbin/pkg_install/add/main.c</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>doc/en_US.ISO8859-1/htdocs/search/opensearch/man.xml</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>The release notes and errata files also need to be
|
|
adjusted for the new release (on the release branch) and
|
|
truncated appropriately (on the stable/current branch):</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/relnotes/common/new.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>src/release/doc/en_US.ISO8859-1/errata/article.xml</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para><application>Sysinstall</application> should be updated to
|
|
note the number of available ports and the amount of disk
|
|
space required for the Ports Collection.
|
|
<footnote>
|
|
<simpara>&os; Ports Collection <uri
|
|
xlink:href="https://www.FreeBSD.org/ports">https://www.FreeBSD.org/ports</uri>
|
|
</simpara>
|
|
</footnote>
|
|
This information is currently kept in
|
|
<filename>src/usr.sbin/bsdinstall/dist.c</filename>.</para>
|
|
|
|
<para>After the release has been built, a number of files should
|
|
be updated to announce the release to the world. These files
|
|
are relative to <literal>head/</literal> within the
|
|
<literal>doc/</literal> subversion tree.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>share/images/articles/releng/branches-releng<replaceable>X</replaceable>.pic</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>head/share/xml/release.ent</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>en_US.ISO8859-1/htdocs/releases/*</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>en_US.ISO8859-1/htdocs/releng/index.xml</filename></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>share/xml/news.xml</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Additionally, update the <quote>BSD Family Tree</quote>
|
|
file:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>src/share/misc/bsd-family-tree</filename></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Creating the Release Tag</title>
|
|
|
|
<para>When the final release is ready, the following command
|
|
will create the <literal>release/9.2.0</literal> tag.</para>
|
|
|
|
<screen>&prompt.root; <userinput>svn cp $FSVN/releng/9.2 $FSVN/release/9.2.0</userinput></screen>
|
|
|
|
<para>The Documentation and Ports managers are responsible for
|
|
tagging their respective trees with the
|
|
<literal>tags/RELEASE_9_2_0</literal> tag.</para>
|
|
|
|
<sidebar>
|
|
<para>When the Subversion <command>svn cp</command> command is
|
|
used to create a <emphasis>release tag</emphasis>, this
|
|
identifies the source at a specific point in time. By
|
|
creating tags, we ensure that future release builders will
|
|
always be able to use the exact same source we used to
|
|
create the official &os; Project releases.</para>
|
|
</sidebar>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Release Building -->
|
|
<sect1 xml:id="release-build">
|
|
<title>Release Building</title>
|
|
|
|
<para>&os; <quote>releases</quote> can be built by anyone with a
|
|
fast machine and access to a source repository. (That should be
|
|
everyone, since we offer Subversion access! See the <link
|
|
xlink:href="&url.books.handbook;/svn.html">Subversion section in
|
|
the Handbook</link> for details.) The <emphasis>only</emphasis>
|
|
special requirement is that the &man.md.4; device must be
|
|
available. If the device is not loaded into your kernel, then the
|
|
kernel module should be automatically loaded when &man.mdconfig.8;
|
|
is executed during the boot media creation phase. All of the
|
|
tools necessary to build a release are available from the
|
|
Subversion repository in <filename>src/release</filename>. These
|
|
tools aim to provide a consistent way to build &os; releases. A
|
|
complete release can actually be built with only a single command,
|
|
including the creation of <acronym>ISO</acronym> images suitable
|
|
for burning to CDROM or DVD, and an FTP install directory.
|
|
&man.release.7; fully documents the
|
|
<command>src/release/generate-release.sh</command> script which is
|
|
used to build a release. <command>generate-release.sh</command>
|
|
is a wrapper around the Makefile target: <command>make
|
|
release</command>.</para>
|
|
|
|
<sect2>
|
|
<title>Building a Release</title>
|
|
|
|
<para>&man.release.7; documents the exact commands required to
|
|
build a &os; release. The following sequences of commands can
|
|
build an 9.2.0 release:</para>
|
|
|
|
<screen>&prompt.root; <userinput>cd /usr/src/release</userinput>
|
|
&prompt.root; <userinput>sh generate-release.sh release/9.2.0 /local3/release</userinput></screen>
|
|
|
|
<para>After running these commands, all prepared release files are
|
|
available in <filename>/local3/release/R</filename>
|
|
directory.</para>
|
|
|
|
<para>The release <filename>Makefile</filename> can be broken down
|
|
into several distinct steps.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Creation of a sanitized system environment in a separate
|
|
directory hierarchy with <quote><command>make
|
|
installworld</command></quote>.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Checkout from Subversion of a clean version of the
|
|
system source, documentation, and ports into the release
|
|
build hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Population of <filename>/etc</filename> and
|
|
<filename>/dev</filename> in the chrooted
|
|
environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>chroot into the release build hierarchy, to make it
|
|
harder for the outside environment to taint this
|
|
build.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><command>make world</command> in the chrooted
|
|
environment.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build of Kerberos-related binaries.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build <filename>GENERIC</filename> kernel.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Creation of a staging directory tree where the binary
|
|
distributions will be built and packaged.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the documentation toolchain
|
|
needed to convert the documentation source (SGML) into HTML
|
|
and text documents that will accompany the release.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Build and installation of the actual documentation (user
|
|
manuals, tutorials, release notes, hardware compatibility
|
|
lists, and so on.)</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Package up distribution tarballs of the binaries and
|
|
sources.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Create FTP installation hierarchy.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>(optionally)</emphasis> Create ISO images for
|
|
CDROM/DVD media.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>For more information about the release build infrastructure,
|
|
please see &man.release.7;.</para>
|
|
|
|
<note>
|
|
<para>It is important to remove any site-specific settings from
|
|
<filename>/etc/make.conf</filename>. For example, it would be
|
|
unwise to distribute binaries that were built on a system with
|
|
<varname>CPUTYPE</varname> set to a specific
|
|
processor.</para>
|
|
</note>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Contributed Software (<quote>ports</quote>)</title>
|
|
|
|
<para>The <link xlink:href="https://www.FreeBSD.org/ports">&os;
|
|
Ports collection</link> is a collection of over &os.numports;
|
|
third-party software packages available for &os;. The
|
|
&a.portmgr; is responsible for maintaining a consistent ports
|
|
tree that can be used to create the binary packages that
|
|
accompany official &os; releases.</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Release ISOs</title>
|
|
|
|
<para>Starting with &os; 4.4, the &os; Project decided to
|
|
release all four ISO images that were previously sold on the
|
|
<emphasis>BSDi/Wind River Systems/FreeBSD Mall</emphasis>
|
|
<quote>official</quote> CDROM distributions. Each of the four
|
|
discs must contain a <filename>README.TXT</filename> file that
|
|
explains the contents of the disc, a
|
|
<filename>CDROM.INF</filename> file that provides meta-data for
|
|
the disc so that &man.bsdinstall.8; can validate and use the
|
|
contents, and a <filename>filename.txt</filename> file that
|
|
provides a manifest for the disc. This
|
|
<emphasis>manifest</emphasis> can be created with a simple
|
|
command:</para>
|
|
|
|
<screen>/stage/cdrom&prompt.root; <userinput>find . -type f | sed -e 's/^\.\///' | sort > filename.txt</userinput></screen>
|
|
|
|
<para>The specific requirements of each CD are outlined
|
|
below.</para>
|
|
|
|
<sect3>
|
|
<title>Disc 1</title>
|
|
|
|
<para>The first disc is almost completely created by
|
|
<command>make release</command>. The only changes that should
|
|
be made to the <filename>disc1</filename> directory are the
|
|
addition of a <filename>tools</filename> directory, and as
|
|
many popular third party software packages as will fit on the
|
|
disc. The <filename>tools</filename> directory contains
|
|
software that allow users to create installation floppies from
|
|
other operating systems. This disc should be made bootable so
|
|
that users of modern PCs do not need to create installation
|
|
floppy disks.</para>
|
|
|
|
<para>If a custom kernel of &os; is to be included, then
|
|
&man.bsdinstall.8; and &man.release.7; must be updated to
|
|
include installation instructions. The relevant code is
|
|
contained in <filename>src/release</filename> and
|
|
<filename>src/usr.sbin/bsdinstall</filename>. Specifically,
|
|
the file <filename>src/release/Makefile</filename>, and
|
|
<filename>dist.c</filename>, <filename>dist.h</filename>,
|
|
<filename>menus.c</filename>, <filename>install.c</filename>,
|
|
and <filename>Makefile</filename> will need to be updated
|
|
under <filename>src/usr.sbin/bsdinstall</filename>.
|
|
Optionally, you may choose to update
|
|
<filename>bsdinstall.8</filename>.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Disc 2</title>
|
|
|
|
<para>The second disc is also largely created by <command>make
|
|
release</command>. This disc contains a <quote>live
|
|
filesystem</quote> that can be used from &man.bsdinstall.8;
|
|
to troubleshoot a &os; installation. This disc should be
|
|
bootable and should also contain a compressed copy of the CVS
|
|
repository in the <filename>CVSROOT</filename> directory and
|
|
commercial software demos in the <filename>commerce</filename>
|
|
directory.</para>
|
|
</sect3>
|
|
|
|
<sect3>
|
|
<title>Multi-volume Support</title>
|
|
|
|
<para><application>Sysinstall</application> supports multiple
|
|
volume package installations. This requires that each disc
|
|
have an <filename>INDEX</filename> file containing all of the
|
|
packages on all volumes of a set, along with an extra field
|
|
that indicates which volume that particular package is on.
|
|
Each volume in the set must also have the
|
|
<literal>CD_VOLUME</literal> variable set in the
|
|
<filename>cdrom.inf</filename> file so that bsdinstall can
|
|
tell which volume is which. When a user attempts to install a
|
|
package that is not on the current disc, bsdinstall will
|
|
prompt the user to insert the appropriate one.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Distribution -->
|
|
<sect1 xml:id="distribution">
|
|
<title>Distribution</title>
|
|
|
|
<sect2 xml:id="dist-ftp">
|
|
<title>FTP Sites</title>
|
|
|
|
<para>When the release has been thoroughly tested and packaged for
|
|
distribution, the master FTP site must be updated. The official
|
|
&os; public FTP sites are all mirrors of a master server that is
|
|
open only to other FTP sites. This site is known as
|
|
<systemitem>ftp-master</systemitem>. When the release is ready,
|
|
the following files must be modified on
|
|
<systemitem>ftp-master</systemitem>:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/<replaceable>X.Y</replaceable>-RELEASE/</filename></term>
|
|
<listitem>
|
|
<para>The installable FTP directory as output from
|
|
<command>make release</command>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/ports/<replaceable>arch</replaceable>/packages-<replaceable>X.Y</replaceable>-release/</filename></term>
|
|
<listitem>
|
|
<para>The complete package build for this
|
|
release.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/<replaceable>X.Y</replaceable>-RELEASE/tools</filename></term>
|
|
<listitem>
|
|
<para>A symlink to
|
|
<filename>../../../tools</filename>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/<replaceable>X.Y</replaceable>-RELEASE/packages</filename></term>
|
|
<listitem>
|
|
<para>A symlink to
|
|
<filename>../../../ports/<replaceable>arch</replaceable>/packages-<replaceable>X.Y</replaceable>-release</filename>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><filename>/pub/FreeBSD/releases/<replaceable>arch</replaceable>/ISO-IMAGES/<replaceable>X.Y</replaceable>/<replaceable>X.Y</replaceable>-RELEASE-<replaceable>arch</replaceable>-*.iso</filename></term>
|
|
<listitem>
|
|
<para>The ISO images. The <quote>*</quote> is
|
|
<filename>disc1</filename>, <filename>disc2</filename>,
|
|
etc. Only if there is a <filename>disc1</filename> and
|
|
there is an alternative first installation CD (for example
|
|
a stripped-down install with no windowing system) there
|
|
may be a <filename>mini</filename> as well.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
<para>For more information about the distribution mirror
|
|
architecture of the &os; FTP sites, please see the <link
|
|
xlink:href="&url.articles.hubs;/">Mirroring &os;</link>
|
|
article.</para>
|
|
|
|
<para>It may take many hours to two days after updating
|
|
<systemitem>ftp-master</systemitem> before a majority of the
|
|
Tier-1 FTP sites have the new software depending on whether or
|
|
not a package set got loaded at the same time. It is imperative
|
|
that the release engineers coordinate with the
|
|
&a.mirror-announce; before announcing the general availability
|
|
of new software on the FTP sites. Ideally the release package
|
|
set should be loaded at least four days prior to release day.
|
|
The release bits should be loaded between 24 and 48 hours before
|
|
the planned release time with <quote>other</quote> file
|
|
permissions turned off. This will allow the mirror sites to
|
|
download it but the general public will not be able to download
|
|
it from the mirror sites. Mail should be sent to
|
|
&a.mirror-announce; at the time the release bits get posted
|
|
saying the release has been staged and giving the time that the
|
|
mirror sites should begin allowing access. Be sure to include a
|
|
time zone with the time, for example make it relative to
|
|
GMT.</para>
|
|
</sect2>
|
|
|
|
<sect2 xml:id="dist-cdrom">
|
|
<title>CD-ROM Replication</title>
|
|
|
|
<para>Coming soon: Tips for sending &os; ISOs to a replicator
|
|
and quality assurance measures to be taken.</para>
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
<!-- Extensibility -->
|
|
<sect1 xml:id="extensibility">
|
|
<title>Extensibility</title>
|
|
|
|
<para>Although &os; forms a complete operating system, there is
|
|
nothing that forces you to use the system exactly as we have
|
|
packaged it up for distribution. We have tried to design the
|
|
system to be as extensible as possible so that it can serve as a
|
|
platform that other commercial products can be built on top of.
|
|
The only <quote>rule</quote> we have about this is that if you are
|
|
going to distribute &os; with non-trivial changes, we encourage
|
|
you to document your enhancements! The &os; community can only
|
|
help support users of the software we provide. We certainly
|
|
encourage innovation in the form of advanced installation and
|
|
administration tools, for example, but we cannot be expected to
|
|
answer questions about it.</para>
|
|
|
|
<sect2>
|
|
<title>Scripting <command>bsdinstall</command></title>
|
|
|
|
<para>The &os; system installation and configuration tool,
|
|
&man.bsdinstall.8;, can be scripted to provide automated
|
|
installs for large sites. This functionality can be used in
|
|
conjunction with &intel; PXE
|
|
<footnote>
|
|
<simpara><uri
|
|
xlink:href="&url.books.handbook;/network-pxe-nfs.html">&url.books.handbook;/network-pxe-nfs.html</uri>
|
|
</simpara>
|
|
</footnote>
|
|
to bootstrap systems from the network.</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- Lessons Learned -->
|
|
<sect1 xml:id="lessons-learned">
|
|
<title>Lessons Learned from &os; 4.4</title>
|
|
|
|
<para>The release engineering process for 4.4 formally began on
|
|
August 1st, 2001. After that date all commits to the
|
|
<literal>RELENG_4</literal> branch of &os; had to be explicitly
|
|
approved by the &a.re;. The first release candidate for the x86
|
|
architecture was released on August 16, followed by 4 more release
|
|
candidates leading up to the final release on September 18th. The
|
|
security officer was very involved in the last week of the process
|
|
as several security issues were found in the earlier release
|
|
candidates. A total of over <emphasis>500</emphasis> emails were
|
|
sent to the &a.re; in little over a month.</para>
|
|
|
|
<para>Our user community has made it very clear that the security
|
|
and stability of a &os; release should not be sacrificed for any
|
|
self-imposed deadlines or target release dates. The &os; Project
|
|
has grown tremendously over its lifetime and the need for
|
|
standardized release engineering procedures has never been more
|
|
apparent. This will become even more important as &os; is ported
|
|
to new platforms.</para>
|
|
</sect1>
|
|
|
|
<!-- Future Directions -->
|
|
<sect1 xml:id="future">
|
|
<title>Future Directions</title>
|
|
|
|
<para>It is imperative for our release engineering activities to
|
|
scale with our growing userbase. Along these lines we are working
|
|
very hard to document the procedures involved in producing &os;
|
|
releases.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><emphasis>Parallelism</emphasis> - Certain portions of the
|
|
release build are actually <quote>embarrassingly
|
|
parallel</quote>. Most of the tasks are very
|
|
I/O intensive, so having multiple high-speed disk drives
|
|
is actually more important than using multiple processors in
|
|
speeding up the <command>make release</command> process. If
|
|
multiple disks are used for different hierarchies in the
|
|
&man.chroot.2; environment, then the CVS checkout of the
|
|
<filename>ports</filename> and <filename>doc</filename> trees
|
|
can be happening simultaneously as the <command>make
|
|
world</command> on another disk. Using a
|
|
<acronym>RAID</acronym> solution (hardware or software) can
|
|
significantly decrease the overall build time.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Cross-building releases</emphasis> - Building
|
|
IA-64 or Alpha release on x86 hardware? <command>make
|
|
TARGET=ia64 release</command>.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Regression Testing</emphasis> - We need better
|
|
automated correctness testing for &os;.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis>Installation Tools</emphasis> - Our installation
|
|
program has long since outlived its intended life span.
|
|
Several projects are under development to provide a more
|
|
advanced installation mechanism. The libh project was one
|
|
such project that aimed to provide an intelligent new package
|
|
framework and GUI installation program.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</sect1>
|
|
|
|
<!-- Acknowledgements -->
|
|
<sect1 xml:id="ackno">
|
|
<title>Acknowledgements</title>
|
|
|
|
<para>I would like to thank Jordan Hubbard for giving me the
|
|
opportunity to take on some of the release engineering
|
|
responsibilities for &os; 4.4 and also for all of his work
|
|
throughout the years making &os; what it is today. Of course the
|
|
release would not have been possible without all of the
|
|
release-related work done by &a.asami.email;, &a.steve.email;,
|
|
&a.bmah.email;, &a.nik.email;, &a.obrien.email;, &a.kris.email;,
|
|
&a.jhb.email; and the rest of the &os; development community. I
|
|
would also like to thank &a.rgrimes.email;, &a.phk.email;, and
|
|
others who worked on the release engineering tools in the very
|
|
early days of &os;. This article was influenced by release
|
|
engineering documents from the CSRG
|
|
<footnote>
|
|
<simpara>Marshall Kirk McKusick, Michael J. Karels, and Keith
|
|
Bostic: <link
|
|
xlink:href="http://docs.FreeBSD.org/44doc/papers/releng.html">
|
|
<emphasis>The Release Engineering of
|
|
4.3BSD</emphasis></link>
|
|
</simpara>
|
|
</footnote>
|
|
,
|
|
the NetBSD Project,
|
|
<footnote>
|
|
<simpara>NetBSD Developer Documentation: Release Engineering
|
|
<uri
|
|
xlink:href="http://www.NetBSD.org/developers/releng/index.html">http://www.NetBSD.org/developers/releng/index.html</uri>
|
|
</simpara>
|
|
</footnote>
|
|
, and John Baldwin's proposed release engineering process notes.
|
|
<footnote>
|
|
<simpara>John Baldwin's &os; Release Engineering Proposal <uri
|
|
xlink:href="https://people.FreeBSD.org/~jhb/docs/releng.txt">https://people.FreeBSD.org/~jhb/docs/releng.txt</uri>
|
|
</simpara>
|
|
</footnote></para></sect1></article>
|