os/doc
3
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Take an initial pass at updating the content in this document

Browse source 
to more accurately reflect the current procedures of the FreeBSD
Release Engineering team.

Reviewed by: delphij nwhitehorn
This commit is contained in:
Craig Rodrigues 2013-03-18 23:18:12 +00:00
parent f0a3748c4e
commit b020963ecc
Notes: svn2git 2020-12-08 03:00:23 +00:00 
svn path=/head/; revision=41264
1 changed files with 156 additions and 263 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

419
en_US.ISO8859-1/articles/releng/article.xml
View file

@ -11,7 +11,9 @@
<articleinfo>
<!-- This paper was presented at BSDCon Europe in Brighton, UK on
November 11, 2001 -->
November 11, 2001. -->
<!-- The content in this paper was updated in March 2013 to
reflect the current FreeBSD Release process. -->
<confgroup>
<confdates>November 2001</confdates>
<conftitle>BSDCon Europe</conftitle>
@ -74,8 +76,14 @@
<para>The development of &os; is a very open process. &os; is
comprised of contributions from thousands of people around the
world. The &os; Project provides anonymous
<acronym>CVS</acronym>[1] access to the general public so that
world. The &os; Project provides
Subversion
<footnote>
<simpara>
Subversion, <ulink url="http://subversion.apache.org"></ulink>
</simpara>
</footnote>
access to the general public so that
others can have access to log messages, diffs (patches) between
development branches, and other productivity enhancements that
formal source code management provides. This has been a huge help
@ -83,9 +91,20 @@
think everyone would agree that chaos would soon manifest if write
access was opened up to everyone on the Internet. Therefore only
a <quote>select</quote> group of nearly 300 people are given write
access to the <acronym>CVS</acronym> repository. These
<emphasis>committers[5]</emphasis> are responsible for the bulk of
&os; development. An elected <emphasis>core-team[6]</emphasis>
access to the Subversion repository. These
<ulink url="&url.articles.contributors;/article.html#staff-committers">committers</ulink>
<footnote>
<simpara>
<ulink url="&url.articles.contributors;/article.html#staff-committers">FreeBSD committers</ulink>
</simpara>
</footnote>
are responsible for the bulk of &os; development. An elected
<ulink url="&url.base;/administration.html#t-core">Core Team</ulink>
<footnote>
<simpara>
<ulink url="&url.base;/administration.html#t-core">&os; Core Team</ulink>
</simpara>
</footnote>
of very senior developers provides some level of direction over
the project.</para>
@ -94,16 +113,15 @@
for polishing the development system into a production quality
release. To solve this dilemma, development continues on two
parallel tracks. The main development branch is the
<emphasis>HEAD</emphasis> or <emphasis>trunk</emphasis> of our CVS
<emphasis>HEAD</emphasis> or <emphasis>trunk</emphasis> of our Subversion
tree, known as <quote>&os;-CURRENT</quote> or
<quote>-CURRENT</quote> for short.</para>
<para>A more stable branch is maintained, known as
<quote>&os;-STABLE</quote> or <quote>-STABLE</quote> for short.
Both branches live in a master CVS repository in California and
are replicated via <application
class="software">CVSup</application>[2] to mirrors all over the
world. &os;-CURRENT[7] is the <quote>bleeding-edge</quote> of
Both branches live in a master Subversion repository on a machine
maintained by the &os; Project.
&os;-CURRENT is the <quote>bleeding-edge</quote> of
&os; development where all new changes first enter the system.
&os;-STABLE is the development branch from which major releases
are made. Changes go into this branch at a different pace, and
@ -117,52 +135,64 @@
class="resource">ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/</systemitem>.
The widespread availability of binary release snapshots, and the
tendency of our user community to keep up with -STABLE development
with CVSup and <quote><command>make</command>
<maketarget>world</maketarget></quote>[7] helps to keep
with Subversion and <quote><command>make</command>
<maketarget>buildworld</maketarget></quote>
<footnote>
<simpara>
<ulink url="&url.books.handbook;/makeworld.html">Rebuilding "world"</ulink>
</simpara>
</footnote>
helps to keep
&os;-STABLE in a very reliable condition even before the
quality assurance activities ramp up pending a major
release.</para>
<para>Bug reports and feature requests are continuously submitted by
users throughout the release cycle. Problems reports are entered into our
<application class="software">GNATS</application>[8] database
<application class="software">GNATS</application> database
<footnote>
<simpara>
GNATS: The GNU Bug Tracking System
<ulink url="http://www.gnu.org/software/gnats"></ulink>
</simpara>
</footnote>
through email, the &man.send-pr.1; application, or via the web
interface provided at <ulink
url="http://www.FreeBSD.org/send-pr.html"></ulink>.</para>
<para>To service our most conservative users, individual release
branches were introduced with &os;&nbsp;4.3.
These release branches are created shortly before a final release
is made. After the release goes out, only the most critical
security fixes and additions are merged onto the release branch.
In addition to source updates via CVS, binary patchkits are
In addition to source updates via Subversion, binary patchkits are
available to keep systems on the
<emphasis>RELENG_<replaceable>X</replaceable>_<replaceable>Y</replaceable></emphasis>
branches updated.</para>
<sect2>
<title>What this article describes</title>
<para>The following sections of this article describe:</para>
<variablelist>
<varlistentry>
<term><xref linkend="release-proc"/></term>
<listitem>
<para>The different phases of the release engineering process
leading up to the actual system build.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><xref linkend="release-build"/></term>
<listitem>
<para>The actual build process.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><xref linkend="extensibility"/></term>
@ -170,7 +200,7 @@
<para>How the base release may be extended by third parties.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><xref linkend="lessons-learned"/></term>
@ -264,41 +294,37 @@
<sect3 id="rel-branch">
<title>Creating the Release Branch</title>
<para>As described in the introduction, the
<literal>RELENG_<replaceable>X</replaceable>_<replaceable>Y</replaceable></literal>
release branch is a relatively new addition to our release
engineering
methodology. The first step in creating this branch is to
ensure that you are working with the newest version of the
<literal>RELENG_<replaceable>X</replaceable></literal> sources
that you want to branch <emphasis>from</emphasis>.</para>
<screen>/usr/src&prompt.root; <userinput>cvs update -rRELENG_4 -P -d</userinput></screen>
<para>The next step is to create a branch point
<emphasis>tag</emphasis>, so that diffs against the start of
the branch are easier with CVS:</para>
<screen>/usr/src&prompt.root; <userinput>cvs rtag -rRELENG_4 RELENG_4_8_BP src</userinput></screen>
<para>And then a new branch tag is created with:</para>
<screen>/usr/src&prompt.root; <userinput>cvs rtag -b -rRELENG_4_8_BP RELENG_4_8 src</userinput></screen>
<note>
<para><emphasis>The
<literal>RELENG_<replaceable>*</replaceable></literal> tags
are restricted for use by the CVS-meisters and release
engineers.</emphasis></para>
<para>In all examples below, <literal>&dollar;FSVN</literal>
refers to the location of the &os; Subversion repository,
<literal>svn+ssh://svn.freebsd.org/base/</literal>.</para>
</note>
<sidebar>
<para>A <quote><emphasis>tag</emphasis></quote> is CVS
vernacular for a label that identifies the source at a specific point
in time. By tagging the tree, we ensure that future release builders
will always be able to use the same source we used to create the
official &os; Project releases.</para>
</sidebar>
<para>The layout of &os; branches in Subversion is
described in the <ulink url="&url.articles.committers-guide;/subversion-primer.html#subversion-primer-base-layout">Committer's Guide</ulink>.
The first step in creating a branch is to
identify the revision of the
<literal>stable/<replaceable>X</replaceable></literal> sources
that you want to branch <emphasis>from</emphasis>.</para>
<screen>&prompt.root; <userinput>svn log -v $FSVN/stable/9</userinput></screen>
<para>The next step is to create the <emphasis>release branch</emphasis>
</para>
<screen>&prompt.root; <userinput>svn cp $FSVN/stable/9@REVISION $FSVN/releng/9.2</userinput></screen>
<para>This branch can be checked out:</para>
<screen>&prompt.root; <userinput>svn co $FSVN/releng/9.2 src</userinput></screen>
<note>
<para>Creating <literal>releng</literal> branch and <literal>release</literal>
tags are restricted to
<ulink url="&url.base;/administration.html#t-subversion">Subversion administrators</ulink>
and the <ulink url="&url.base;/administration.html#t-re">Release Engineering Team</ulink>.
</para>
</note>
<mediaobject>
<imageobject>
@ -479,7 +505,14 @@
<para><application>Sysinstall</application> should be updated to note
the number of available ports and the amount of disk space required
for the Ports Collection[4]. This information is currently kept in
for the Ports Collection.
<footnote>
<simpara>
&os; Ports Collection
<ulink url="http://www.FreeBSD.org/ports"></ulink>
</simpara>
</footnote>
This information is currently kept in
<filename>src/usr.sbin/sysinstall/dist.c</filename>.</para>
<para>After the release has been built, a number of file should
@ -526,47 +559,29 @@
</sect3>
<sect3 id="versionbump-major">
<title>Preparing a new major release branch
(RELENG_<replaceable>X</replaceable>)</title>
<para>When a new major release branch, such as
<literal>RELENG_6</literal> is branched from HEAD, some
additional files must be updated before releases can be made
from this new branch.</para>
<itemizedlist>
<listitem>
<para><filename>src/share/examples/cvsup/stable-supfile</filename>
- must be updated to point to the new -STABLE branch, when
applicable.</para>
</listitem>
</itemizedlist>
</sect3>
<sect3>
<title>Creating Release Tags</title>
<title>Creating the Release Tag</title>
<para>When the final release is ready, the following command
will create the <literal>RELENG_4_8_0_RELEASE</literal>
will create the <literal>release/9.2.0</literal>
tag.</para>
<screen>/usr/src&prompt.root; <userinput>cvs rtag -rRELENG_4_8 RELENG_4_8_0_RELEASE src</userinput></screen>
<screen>&prompt.root; <userinput>svn cp $FSVN/releng/9.2 $FSVN/release/9.2.0</userinput></screen>
<para>The Documentation and Ports managers are responsible for
tagging the respective trees with the <literal>RELEASE_4_8_0</literal>
tagging their respective trees with the <literal>tags/RELEASE_9_2_0</literal>
tag.</para>
<para>Occasionally, a last minute fix may be required
<emphasis>after</emphasis> the final tags have been created.
In practice this is not a problem, since <acronym>CVS</acronym>
allows tags to be manipulated with <command>cvs
tag -d <replaceable>tagname filename</replaceable></command>.
It is very important that any last minute changes be tagged
appropriately as part of the release. &os; releases must
always be reproducible. Local hacks in the release
engineer's environment are not acceptable.</para>
<sidebar>
<para>When the Subversion <command>svn cp</command> command
is used to create a <emphasis>release tag</emphasis>,
this identifies the source at a specific point in time.
By creating tags, we ensure that future release builders
will always be able to use the exact same source we used to create the
official &os; Project releases.</para>
</sidebar>
</sect3>
</sect2>
</sect1>
@ -577,79 +592,38 @@ applicable.</para>
<para>&os; <quote>releases</quote> can be built by anyone with a
fast machine and access to a source repository. (That should be
everyone, since we offer anonymous CVS! See The Handbook for
everyone, since we offer Subversion access !
See the
<ulink url="&url.books.handbook;/svn.html">Subversion section
in the Handbook</ulink> for
details.) The <emphasis>only</emphasis> special requirement is
that the &man.md.4; device must be available. If the
device is not loaded into your kernel, then the kernel module
should be automatically loaded when &man.mdconfig.8; is executed
during the boot media creation phase. All of the tools necessary
to build a release are available from the CVS repository in
to build a release are available from the Subversion repository in
<filename>src/release</filename>. These tools aim to provide a
consistent way to build &os; releases. A complete release can
actually be built with only a single command, including the
creation of <acronym>ISO</acronym> images suitable for burning to
CDROM, installation floppies, and an FTP install directory. This
command is aptly named <command>make
release</command>.</para>
CDROM or DVD, and an FTP install directory. &man.release.7; fully
documents the <command>src/release/generate-release.sh</command>
script which is used to build a release. <command>generate-release.sh</command>
is a wrapper around the Makefile target: <command>make release</command>.</para>
<sect2>
<title><command>make release</command></title>
<title>Building a Release</title>
<para>To successfully build a release, you must first populate
<filename>/usr/obj</filename> by running <command>make
world</command> or simply
<command>make
buildworld</command>. The release
target requires several variables be set properly to build a
release:</para>
<para>&man.release.7; documents the exact commands required to
build a &os; release. The following sequences of commands can build
an 9.2.0 release:</para>
<itemizedlist>
<listitem>
<para><makevar>CHROOTDIR</makevar> - The directory to be used as the
chroot environment for the entire release build.</para>
</listitem>
<screen>&prompt.root; <userinput>cd /usr/src/release</userinput></screen>
<screen>&prompt.root; <userinput>sh generate-release.sh release/9.2.0 /local3/release</userinput></screen>
<listitem>
<para><makevar>BUILDNAME</makevar> - The name of the release to be
built.</para>
</listitem>
<listitem>
<para><makevar>CVSROOT</makevar> - The location of a CVS Repository.
</para>
</listitem>
<listitem>
<para><makevar>RELEASETAG</makevar> - The CVS tag corresponding to the
release you would like to build.</para>
</listitem>
</itemizedlist>
<para>If you do not already have access to a local CVS
repository, then you may mirror one with <ulink
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/synching.html#CVSUP">CVSup</ulink>.
The supplied supfile,
<filename>/usr/share/examples/cvsup/cvs-supfile</filename>, is
a useful starting point for mirroring the CVS
repository.</para>
<para>If <makevar>RELEASETAG</makevar> is omitted, then the
release will be built from the <literal>HEAD</literal> (aka -CURRENT) branch.
Releases built from this branch are normally referred to as
<quote>-CURRENT snapshots</quote>.</para>
<para>There are many other variables available to customize the
release build. Most of these variables are documented at the
top of <filename>src/release/Makefile</filename>. The exact
command used to build the official &os;&nbsp;4.7 (x86) release
was:</para>
<screen><command>make <literal>release CHROOTDIR=/local3/release \
BUILDNAME=4.7-RELEASE \
CVSROOT=/host/cvs/usr/home/ncvs \
RELEASETAG=RELENG_4_7_0_RELEASE</literal>
</command>
</screen>
<para>After running these commands, all prepared release
files are available in <filename>/local3/release/R</filename>
directory.</para>
<para>The release <filename>Makefile</filename> can be broken down into several distinct
steps.</para>
@ -663,7 +637,7 @@ applicable.</para>
</listitem>
<listitem>
<para>Checkout from CVS of a clean version of the system source,
<para>Checkout from Subversion of a clean version of the system source,
documentation, and ports into the release build hierarchy.</para>
</listitem>
@ -708,20 +682,11 @@ applicable.</para>
and so on.)</para>
</listitem>
<listitem>
<para>Build of the <quote>crunched</quote> binaries used for
installation floppies.</para>
</listitem>
<listitem>
<para>Package up distribution tarballs of the binaries and sources.
</para>
</listitem>
<listitem>
<para>Create the boot media and a <quote>fixit</quote> floppy.</para>
</listitem>
<listitem>
<para>Create FTP installation hierarchy.</para>
</listitem>
@ -951,64 +916,20 @@ applicable.</para>
installation and administration tools, for example, but we cannot
be expected to answer questions about it.</para>
<sect2>
<title>Creating Customized Boot floppies</title>
<para>Many sites have complex requirements that may require
additional kernel modules or userland tools be added to the
installation floppies. The <quote>quick and dirty</quote> way
to accomplish this would be to modify the staging directory of
an existing <command>make release</command> build hierarchy:</para>
<itemizedlist>
<listitem>
<para>Apply patches or add additional files inside the chroot
release build directory.</para>
</listitem>
<listitem>
<para><command>rm
${CHROOTDIR}/usr/obj/usr/src/release/release.[59]</command></para>
</listitem>
<listitem>
<para>rebuild &man.sysinstall.8;, the kernel, or whatever
parts of the system your change affected.</para>
</listitem>
<listitem>
<para><command>chroot ${CHROOTDIR} ./mk floppies
</command></para>
</listitem>
</itemizedlist>
<para>New release floppies will be located in
<filename>${CHROOTDIR}/R/stage/floppies</filename>.</para>
<para>Alternatively, the
<filename>boot.flp</filename> make
target can be called, or the filesystem
creating script,
<filename>src/release/scripts/doFS.sh</filename>, may be invoked
directly.</para>
<para>Local patches may also be supplied to the release build by
defining the <makevar>LOCAL_PATCH</makevar> variable in <command>make
release</command>.
</para>
</sect2>
<sect2>
<title>Scripting <command>sysinstall</command></title>
<para>The &os; system installation and configuration tool,
&man.sysinstall.8;, can be scripted to provide automated installs
for large sites. This functionality can be used in conjunction
with &intel; PXE[12] to bootstrap systems from the network, or
via custom boot floppies with a sysinstall script. An example
sysinstall script is available in the CVS tree as
<filename>src/usr.sbin/sysinstall/install.cfg</filename>.</para>
with &intel; PXE
<footnote>
<simpara>
<ulink url="&url.books.handbook;/network-pxe-nfs.html"></ulink>
</simpara>
</footnote>
to bootstrap systems from the network.
</para>
</sect2>
</sect1>
@ -1100,59 +1021,31 @@ applicable.</para>
community. I would also like to thank &a.rgrimes;, &a.phk;, and others
who worked on the release engineering tools in the very early days
of &os;. This article was influenced by release engineering
documents from the CSRG[13], the NetBSD Project[10], and John
Baldwin's proposed release engineering process notes[11].</para>
</sect1>
<!-- Reference / Biblio Section -->
<sect1 id="biblio">
<title>References</title>
<para>[1] CVS - Concurrent Versions System
<ulink url="http://www.cvshome.org"></ulink></para>
<para>[2] CVSup - The CVS-Optimized General Purpose Network File Distribution
System <ulink url="http://www.polstra.com/projects/freeware/CVSup"></ulink>
</para>
<para>[3] <ulink url="http://pointyhat.FreeBSD.org"></ulink></para>
<para>[4] &os; Ports Collection
<ulink url="http://www.FreeBSD.org/ports"></ulink></para>
<para>[5] &os; Committers <ulink
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributors/staff-committers.html"></ulink>
</para>
<para>[6] &os; Core Team
<ulink url="&url.base;/administration.html#t-core"></ulink></para>
<para>[7] &os; Handbook
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook"></ulink>
</para>
<para>[8] GNATS: The GNU Bug Tracking System
<ulink url="http://www.gnu.org/software/gnats"></ulink>
</para>
<para>[9] &os; PR Statistics
<ulink url="http://www.FreeBSD.org/prstats/index.html"></ulink></para>
<para>[10] NetBSD Developer Documentation: Release Engineering
<ulink url="http://www.NetBSD.org/developers/releng/index.html"></ulink>
</para>
<para>[11] John Baldwin's &os; Release Engineering Proposal
<ulink url="http://people.FreeBSD.org/~jhb/docs/releng.txt"></ulink>
</para>
<para>[12] PXE Jumpstart Guide
<ulink
url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/pxe/index.html"></ulink>
</para>
<para>[13] Marshall Kirk McKusick, Michael J. Karels, and Keith Bostic:
<ulink url="http://docs.FreeBSD.org/44doc/papers/releng.html">
<emphasis>The Release Engineering of 4.3BSD</emphasis></ulink>
documents from the CSRG
<footnote>
<simpara>
Marshall Kirk McKusick, Michael J. Karels, and Keith Bostic:
<ulink url="http://docs.FreeBSD.org/44doc/papers/releng.html">
<emphasis>The Release Engineering of 4.3BSD</emphasis></ulink>
</simpara>
</footnote>
,
the NetBSD Project ,
<footnote>
<simpara>
NetBSD Developer Documentation: Release Engineering
<ulink url="http://www.NetBSD.org/developers/releng/index.html"></ulink>
</simpara>
</footnote>
, and John
Baldwin's proposed release engineering process notes.
<footnote>
<simpara>
John Baldwin's &os; Release Engineering Proposal
<ulink url="http://people.FreeBSD.org/~jhb/docs/releng.txt"></ulink>
</simpara>
</footnote>
</para>
</sect1>
</article>