os/doc
3
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Start scrubbing (or flagging) all obsolete information.

Browse source 
While here, continue moving from using /var/portbuild to /a/portbuild
for server-based instructions, and be more pedantic about labeling
things server-side vs. client-side.
This commit is contained in:
Mark Linimon 2013-02-15 14:49:56 +00:00
parent ddd400264a
commit b91e74bc05
Notes: svn2git 2020-12-08 03:00:23 +00:00 
svn path=/head/; revision=40973
1 changed files with 71 additions and 62 deletions

133
en_US.ISO8859-1/articles/portbuild/article.xml
View file

@ -77,7 +77,7 @@
</para>
<note>
<para>Packages are no longer built for branches 4, 5, or 6, nor
<para>FreeBSD no longer builds packages for branches 4, 5, or 6, nor
for the alpha architecture.</para>
</note>
@ -113,7 +113,7 @@
</sect2>
<sect2 id="codebase-notes">
<title>Notes on the codebase</title>
<title>Historical notes on the codebase</title>
<para>Until mid-2010, the scripts were completely specific to
<hostid>pointyhat.FreeBSD.org</hostid> as the head (dispatch) node. During
@ -149,18 +149,6 @@
</listitem>
</itemizedlist>
<para>This document was originally written before these changes
were made. Where things such as script invocations have changed,
they were denoted as <literal>new codebase:</literal> as opposed
to <literal>old codebase:</literal>.</para>
<note>
<para>Up until November 2012, <hostid>pointyhat</hostid> had still
been running the old codebase. That installation has now been
permanently offlined. Therefore, all the instructions having
to do with the old codebase have been removed.</para>
</note>
<note>
<para>Also during this process, the codebase was migrated to the
<ulink url="http://svnweb.freebsd.org/base/projects/portbuild/scripts/">
@ -175,11 +163,11 @@
<sect1 id="management">
<title>Build Client Management</title>
<para>The &i386; clients co-located with <hostid>pointyhat</hostid>
netboot from it (<replaceable>connected</replaceable> nodes); all
other clients (<replaceable>disconnected</replaceable> nodes)
are either self-hosted or netboot from some other
<literal>pxe</literal> host.
<para>You may set up clients to either netboot from the master
(<replaceable>connected</replaceable> nodes)
or have them either self-hosted or netboot from some other
<literal>pxe</literal> host
(<replaceable>disconnected</replaceable> nodes).
In all cases they set themselves
up at boot-time to prepare to build packages.</para>
@ -204,29 +192,31 @@
<sect1 id="setup">
<title>Jail Build Environment Setup</title>
<para>Package builds are performed in a
<para>Package builds are performed by the clients in a
<literal>jail</literal> populated by the
<filename>portbuild</filename> script using the
<filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename>
file.</para>
<para>The <command>makeworld</command> command builds a world from the
<para>On the server, use the
<command>makeworld</command> command to build a world from the
<filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/src/</filename>
tree and installs it into
tree and install it into
<filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename>.
The tree will
be updated first unless <literal>-novcs</literal> is
specified. It should be run as <username>root</username>:</para>
specified.</para>
<screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/makeworld <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-novcs]</userinput></screen>
<para>The <filename>bindist.tar</filename> tarball is created from the
<para>Similiarly on the server, the
<filename>bindist.tar</filename> tarball is created from the
previously installed world by the <command>mkbindist</command>
script. It should be also be run as <username>root</username>:</para>
script.</para>
<screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/mkbindist <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable></userinput></screen>
<para>The per-machine tarballs are located in
<para>The per-machine tarballs are located on the server in
<filename><replaceable>${arch}</replaceable>/clients</filename>.</para>
<para>The <filename>bindist.tar</filename> file is extracted
@ -237,6 +227,16 @@
<para>For both commands above, if
<replaceable>${buildid}</replaceable> is
<literal>latest</literal>, it may be omitted.</para>
<note>
<para>Currently the above two scripts must be run as
<username>root</username>; otherwise, the install scripts
lack sufficient permissions. This is undesirable for
security reasons. Work is in progress in -HEAD to allow
users to do installations; once that is committed, the
intention is to use that and run these two commands as
<username>srcbuild</username>.</para>
</note>
</sect1>
<sect1 id="customizing">
@ -1018,12 +1018,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<screen>&prompt.user; <userinput>cd <replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput>
&prompt.user; <userinput>find distfiles > distfiles-<replaceable>${release}</replaceable></userinput></screen>
<!-- XXX MCL apparently obsolete -->
<para>This inventory file typically lives in
<filename>i386/<replaceable>${branch}</replaceable></filename>
on the cluster master.</para>
<para>This is useful to aid in periodically cleaning out
<para>You should use that output to periodically clean out
the distfiles from <hostid>ftp-master</hostid>. When space
gets tight, distfiles from recent releases can be kept while
others can be thrown away.</para>
@ -1047,6 +1042,16 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<sect1 id="uploading">
<title>Uploading Packages</title>
<note>
<para>For FreeBSD.org as of 2013, the instructions
about uploading to <hostid>ftp-master</hostid> are obsolete.
In the future, <hostid>ftp-master</hostid> will pull
from <hostid>pointyhat</hostid>, using a mechanism yet
to be implemented. However, the instructions about
<makevar>RESTRICTED</makevar> and <makevar>NO_CDROM</makevar>
must still be <emphasis>carefully</emphasis> followed.</para>
</note>
<para>Once a build has completed, packages and/or distfiles
can be transferred to <hostid>ftp-master</hostid> for
propagation to the FTP mirror network. If the build was
@ -1124,7 +1129,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
a new release), copy packages to the staging area on
<hostid>ftp-master</hostid> with something like the following:</para>
<screen>&prompt.root; <userinput>cd /var/portbuild/<replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput>
<screen>&prompt.root; <userinput>cd /a/portbuild/<replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput>
&prompt.root; <userinput>tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/<replaceable>${arch}</replaceable>/tmp/<replaceable>${subdir}</replaceable></userinput></screen>
<para>Then log into <hostid>ftp-master</hostid>, verify that
@ -1152,7 +1157,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<para>Distfiles should be transferred with the
<command>cpdistfiles</command> script:</para>
<screen>&prompt.root; <userinput>/var/portbuild/scripts/cpdistfiles <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-yesreally] | tee log2</userinput></screen>
<screen>&prompt.root; <userinput>/a/portbuild/scripts/cpdistfiles <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-yesreally] | tee log2</userinput></screen>
<para>Doing it by hand is deprecated.</para>
</sect1>
@ -1160,6 +1165,11 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<sect1 id="expbuilds">
<title>Experimental Patches Builds</title>
<note>
<para>Most of the information in this section is obsolete
as of 2013.</para>
</note>
<para>Experimental patches builds are run from time to time to
new features or bugfixes to the ports infrastructure (i.e.
<filename>bsd.port.mk</filename>), or to test large sweeping
@ -1210,12 +1220,6 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
build. This will ensure an apples-to-apples comparison
later.</para>
<!-- XXX MCL currently there is only one build cluster
<note><para>One build cluster can do the control build while the other
does the experimental patches build. This can be a great
time-saver.</para></note>
-->
<para>Once the build finishes, compare the control build failures
to those of the experimental patches build. Use the following
commands to facilitate this (this assumes the <literal>8</literal>
@ -1300,9 +1304,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<!-- XXX MCL fix -->
<para>The following command will set up the control branch for
the partial build (old codebase):</para>
the partial build:</para>
<screen>&prompt.user; <userinput>/var/portbuild/scripts/dopackages.8 -noportsvcs -nobuild -novcs -nofinish</userinput></screen>
<screen>&prompt.user; <userinput>/a/portbuild/scripts/dopackages.wrapper 8 -noportsvcs -nobuild -novcs -nofinish</userinput></screen>
<!-- XXX MCL obsolete -->
<para>The builds must be performed from the
@ -1317,7 +1321,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<note>
<para>&lt;#&gt; is the concurrency of the build to
attempt. It is usually the sum of the weights listed in
<filename>/var/portbuild/i386/mlist</filename> unless you have a
<filename>/a/portbuild/i386/mlist</filename> unless you have a
reason to run a heavier or lighter build.</para>
<para>The list of packages to build should be a list of package
@ -1350,17 +1354,15 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<para>Before following these steps, please coordinate with
<literal>portmgr</literal>.</para>
<note>
<para>Due to some generous donations, <literal>portmgr</literal> is
no longer looking for the loan of &i386; or <literal>amd64</literal>
systems. However, we are still interested in borrowing tier-2
systems.</para>
</note>
<sect2 id="node-requirements">
<title>Node requirements</title>
<para><literal>portmgr</literal> is still working on characterizing
<note>
<para>This section is only of interest when considering
tier-2 architectures.</para>
</note>
<para>Here are the requirement for
what a node needs to be generally useful.</para>
<itemizedlist>
@ -1440,7 +1442,8 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<filename>/usr2/</filename>.)</para>
<note>
<para>The filename <filename>chroot</filename> is a
historical remnant.</para>
historical remnant. The <command>chroot</command>
command is no longer used.</para>
</note>
</step>
</procedure>
@ -1481,8 +1484,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<step>
<para>Generate a kernel config file. Include
<filename>GENERIC</filename> (or, if you are using more than
3.5G on &i386;, <filename>PAE</filename>).</para>
<filename>GENERIC</filename> (or, if on &i386, and
you are using more than
3.5G, <filename>PAE</filename>).</para>
<para>Required options:</para>
@ -1497,9 +1501,14 @@ options SHMMAXPGS=65536
options SEMMNI=40
options SEMMNS=240
options SEMUME=40
options SEMMNU=120
options SEMMNU=120</programlisting>
options ALT_BREAK_TO_DEBUGGER</programlisting>
<para>If you are interested in debugging general
problems, you may wish to use the following.
However, for unattended operations, it is best
to leave it out:</para>
<programlisting>options ALT_BREAK_TO_DEBUGGER</programlisting>
<para>For <filename>PAE</filename>, it is not currently possible
to load modules. Therefore, if you are running an architecture
@ -1916,7 +1925,7 @@ portbuild ALL=(ALL) NOPASSWD: ALL</programlisting>
<step>
<para>Create
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/clients/bindist-<replaceable>${hostname}</replaceable>.tar</filename>.</para>
<filename>/a/portbuild/<replaceable>${arch}</replaceable>/clients/bindist-<replaceable>${hostname}</replaceable>.tar</filename>.</para>
<itemizedlist>
<listitem>
@ -1955,16 +1964,16 @@ MASTER_SITE_OVERRIDE= \
<para>Hint: you will need one of these for each machine;
however, if you have multiple machines at one site, you
should create a site-specific one (e.g., in
<filename>/var/portbuild/conf/clients/</filename>)
<filename>/a/portbuild/conf/clients/</filename>)
and symlink to it.</para>
</step>
<step>
<para>Create
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/portbuild-<replaceable>${hostname}</replaceable></filename>
<filename>/a/portbuild/<replaceable>${arch}</replaceable>/portbuild-<replaceable>${hostname}</replaceable></filename>
using one of the existing ones as a guide. This
file contains overrides to
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/portbuild.conf</filename>.</para>
<filename>/a/portbuild/<replaceable>${arch}</replaceable>/portbuild.conf</filename>.</para>
<para>Suggested values:</para>
@ -2030,7 +2039,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlisting>
<step>
<para>Populate the client's copy of
<filename>/var/portbuild/scripts/</filename> by something like
<userinput>/var/portbuild/scripts/dosetupnode <replaceable>arch</replaceable> <replaceable>major</replaceable> latest <replaceable>hostname</replaceable></userinput>.
<userinput>/a/portbuild/scripts/dosetupnode <replaceable>arch</replaceable> <replaceable>major</replaceable> latest <replaceable>hostname</replaceable></userinput>.
Verify that you now have files in that directory.</para>
</step>
@ -2075,7 +2084,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlisting>
<para>Once you are sure that the client is working, tell
<application>pollmachine</application> about it by adding
it to
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/mlist</filename>.</para>
<filename>/a/portbuild/<replaceable>${arch}</replaceable>/mlist</filename>.</para>
</step>
</procedure>
</sect2>