Start scrubbing (or flagging) all obsolete information.

While here, continue moving from using /var/portbuild to /a/portbuild
for server-based instructions, and be more pedantic about labeling
things server-side vs. client-side.
This commit is contained in:
Mark Linimon 2013-02-15 14:49:56 +00:00
parent ddd400264a
commit b91e74bc05
Notes: svn2git 2020-12-08 03:00:23 +00:00
svn path=/head/; revision=40973

View file

@ -77,7 +77,7 @@
</para> </para>
<note> <note>
<para>Packages are no longer built for branches 4, 5, or 6, nor <para>FreeBSD no longer builds packages for branches 4, 5, or 6, nor
for the alpha architecture.</para> for the alpha architecture.</para>
</note> </note>
@ -113,7 +113,7 @@
</sect2> </sect2>
<sect2 id="codebase-notes"> <sect2 id="codebase-notes">
<title>Notes on the codebase</title> <title>Historical notes on the codebase</title>
<para>Until mid-2010, the scripts were completely specific to <para>Until mid-2010, the scripts were completely specific to
<hostid>pointyhat.FreeBSD.org</hostid> as the head (dispatch) node. During <hostid>pointyhat.FreeBSD.org</hostid> as the head (dispatch) node. During
@ -149,18 +149,6 @@
</listitem> </listitem>
</itemizedlist> </itemizedlist>
<para>This document was originally written before these changes
were made. Where things such as script invocations have changed,
they were denoted as <literal>new codebase:</literal> as opposed
to <literal>old codebase:</literal>.</para>
<note>
<para>Up until November 2012, <hostid>pointyhat</hostid> had still
been running the old codebase. That installation has now been
permanently offlined. Therefore, all the instructions having
to do with the old codebase have been removed.</para>
</note>
<note> <note>
<para>Also during this process, the codebase was migrated to the <para>Also during this process, the codebase was migrated to the
<ulink url="http://svnweb.freebsd.org/base/projects/portbuild/scripts/"> <ulink url="http://svnweb.freebsd.org/base/projects/portbuild/scripts/">
@ -175,11 +163,11 @@
<sect1 id="management"> <sect1 id="management">
<title>Build Client Management</title> <title>Build Client Management</title>
<para>The &i386; clients co-located with <hostid>pointyhat</hostid> <para>You may set up clients to either netboot from the master
netboot from it (<replaceable>connected</replaceable> nodes); all (<replaceable>connected</replaceable> nodes)
other clients (<replaceable>disconnected</replaceable> nodes) or have them either self-hosted or netboot from some other
are either self-hosted or netboot from some other <literal>pxe</literal> host
<literal>pxe</literal> host. (<replaceable>disconnected</replaceable> nodes).
In all cases they set themselves In all cases they set themselves
up at boot-time to prepare to build packages.</para> up at boot-time to prepare to build packages.</para>
@ -204,29 +192,31 @@
<sect1 id="setup"> <sect1 id="setup">
<title>Jail Build Environment Setup</title> <title>Jail Build Environment Setup</title>
<para>Package builds are performed in a <para>Package builds are performed by the clients in a
<literal>jail</literal> populated by the <literal>jail</literal> populated by the
<filename>portbuild</filename> script using the <filename>portbuild</filename> script using the
<filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename> <filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename>
file.</para> file.</para>
<para>The <command>makeworld</command> command builds a world from the <para>On the server, use the
<command>makeworld</command> command to build a world from the
<filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/src/</filename> <filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/src/</filename>
tree and installs it into tree and install it into
<filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename>. <filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename>.
The tree will The tree will
be updated first unless <literal>-novcs</literal> is be updated first unless <literal>-novcs</literal> is
specified. It should be run as <username>root</username>:</para> specified.</para>
<screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/makeworld <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-novcs]</userinput></screen> <screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/makeworld <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-novcs]</userinput></screen>
<para>The <filename>bindist.tar</filename> tarball is created from the <para>Similiarly on the server, the
<filename>bindist.tar</filename> tarball is created from the
previously installed world by the <command>mkbindist</command> previously installed world by the <command>mkbindist</command>
script. It should be also be run as <username>root</username>:</para> script.</para>
<screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/mkbindist <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable></userinput></screen> <screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/mkbindist <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable></userinput></screen>
<para>The per-machine tarballs are located in <para>The per-machine tarballs are located on the server in
<filename><replaceable>${arch}</replaceable>/clients</filename>.</para> <filename><replaceable>${arch}</replaceable>/clients</filename>.</para>
<para>The <filename>bindist.tar</filename> file is extracted <para>The <filename>bindist.tar</filename> file is extracted
@ -237,6 +227,16 @@
<para>For both commands above, if <para>For both commands above, if
<replaceable>${buildid}</replaceable> is <replaceable>${buildid}</replaceable> is
<literal>latest</literal>, it may be omitted.</para> <literal>latest</literal>, it may be omitted.</para>
<note>
<para>Currently the above two scripts must be run as
<username>root</username>; otherwise, the install scripts
lack sufficient permissions. This is undesirable for
security reasons. Work is in progress in -HEAD to allow
users to do installations; once that is committed, the
intention is to use that and run these two commands as
<username>srcbuild</username>.</para>
</note>
</sect1> </sect1>
<sect1 id="customizing"> <sect1 id="customizing">
@ -1018,12 +1018,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<screen>&prompt.user; <userinput>cd <replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput> <screen>&prompt.user; <userinput>cd <replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput>
&prompt.user; <userinput>find distfiles > distfiles-<replaceable>${release}</replaceable></userinput></screen> &prompt.user; <userinput>find distfiles > distfiles-<replaceable>${release}</replaceable></userinput></screen>
<!-- XXX MCL apparently obsolete --> <para>You should use that output to periodically clean out
<para>This inventory file typically lives in
<filename>i386/<replaceable>${branch}</replaceable></filename>
on the cluster master.</para>
<para>This is useful to aid in periodically cleaning out
the distfiles from <hostid>ftp-master</hostid>. When space the distfiles from <hostid>ftp-master</hostid>. When space
gets tight, distfiles from recent releases can be kept while gets tight, distfiles from recent releases can be kept while
others can be thrown away.</para> others can be thrown away.</para>
@ -1047,6 +1042,16 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<sect1 id="uploading"> <sect1 id="uploading">
<title>Uploading Packages</title> <title>Uploading Packages</title>
<note>
<para>For FreeBSD.org as of 2013, the instructions
about uploading to <hostid>ftp-master</hostid> are obsolete.
In the future, <hostid>ftp-master</hostid> will pull
from <hostid>pointyhat</hostid>, using a mechanism yet
to be implemented. However, the instructions about
<makevar>RESTRICTED</makevar> and <makevar>NO_CDROM</makevar>
must still be <emphasis>carefully</emphasis> followed.</para>
</note>
<para>Once a build has completed, packages and/or distfiles <para>Once a build has completed, packages and/or distfiles
can be transferred to <hostid>ftp-master</hostid> for can be transferred to <hostid>ftp-master</hostid> for
propagation to the FTP mirror network. If the build was propagation to the FTP mirror network. If the build was
@ -1124,7 +1129,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
a new release), copy packages to the staging area on a new release), copy packages to the staging area on
<hostid>ftp-master</hostid> with something like the following:</para> <hostid>ftp-master</hostid> with something like the following:</para>
<screen>&prompt.root; <userinput>cd /var/portbuild/<replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput> <screen>&prompt.root; <userinput>cd /a/portbuild/<replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput>
&prompt.root; <userinput>tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/<replaceable>${arch}</replaceable>/tmp/<replaceable>${subdir}</replaceable></userinput></screen> &prompt.root; <userinput>tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/<replaceable>${arch}</replaceable>/tmp/<replaceable>${subdir}</replaceable></userinput></screen>
<para>Then log into <hostid>ftp-master</hostid>, verify that <para>Then log into <hostid>ftp-master</hostid>, verify that
@ -1152,7 +1157,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<para>Distfiles should be transferred with the <para>Distfiles should be transferred with the
<command>cpdistfiles</command> script:</para> <command>cpdistfiles</command> script:</para>
<screen>&prompt.root; <userinput>/var/portbuild/scripts/cpdistfiles <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-yesreally] | tee log2</userinput></screen> <screen>&prompt.root; <userinput>/a/portbuild/scripts/cpdistfiles <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-yesreally] | tee log2</userinput></screen>
<para>Doing it by hand is deprecated.</para> <para>Doing it by hand is deprecated.</para>
</sect1> </sect1>
@ -1160,6 +1165,11 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<sect1 id="expbuilds"> <sect1 id="expbuilds">
<title>Experimental Patches Builds</title> <title>Experimental Patches Builds</title>
<note>
<para>Most of the information in this section is obsolete
as of 2013.</para>
</note>
<para>Experimental patches builds are run from time to time to <para>Experimental patches builds are run from time to time to
new features or bugfixes to the ports infrastructure (i.e. new features or bugfixes to the ports infrastructure (i.e.
<filename>bsd.port.mk</filename>), or to test large sweeping <filename>bsd.port.mk</filename>), or to test large sweeping
@ -1210,12 +1220,6 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
build. This will ensure an apples-to-apples comparison build. This will ensure an apples-to-apples comparison
later.</para> later.</para>
<!-- XXX MCL currently there is only one build cluster
<note><para>One build cluster can do the control build while the other
does the experimental patches build. This can be a great
time-saver.</para></note>
-->
<para>Once the build finishes, compare the control build failures <para>Once the build finishes, compare the control build failures
to those of the experimental patches build. Use the following to those of the experimental patches build. Use the following
commands to facilitate this (this assumes the <literal>8</literal> commands to facilitate this (this assumes the <literal>8</literal>
@ -1300,9 +1304,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<!-- XXX MCL fix --> <!-- XXX MCL fix -->
<para>The following command will set up the control branch for <para>The following command will set up the control branch for
the partial build (old codebase):</para> the partial build:</para>
<screen>&prompt.user; <userinput>/var/portbuild/scripts/dopackages.8 -noportsvcs -nobuild -novcs -nofinish</userinput></screen> <screen>&prompt.user; <userinput>/a/portbuild/scripts/dopackages.wrapper 8 -noportsvcs -nobuild -novcs -nofinish</userinput></screen>
<!-- XXX MCL obsolete --> <!-- XXX MCL obsolete -->
<para>The builds must be performed from the <para>The builds must be performed from the
@ -1317,7 +1321,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<note> <note>
<para>&lt;#&gt; is the concurrency of the build to <para>&lt;#&gt; is the concurrency of the build to
attempt. It is usually the sum of the weights listed in attempt. It is usually the sum of the weights listed in
<filename>/var/portbuild/i386/mlist</filename> unless you have a <filename>/a/portbuild/i386/mlist</filename> unless you have a
reason to run a heavier or lighter build.</para> reason to run a heavier or lighter build.</para>
<para>The list of packages to build should be a list of package <para>The list of packages to build should be a list of package
@ -1350,17 +1354,15 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<para>Before following these steps, please coordinate with <para>Before following these steps, please coordinate with
<literal>portmgr</literal>.</para> <literal>portmgr</literal>.</para>
<note>
<para>Due to some generous donations, <literal>portmgr</literal> is
no longer looking for the loan of &i386; or <literal>amd64</literal>
systems. However, we are still interested in borrowing tier-2
systems.</para>
</note>
<sect2 id="node-requirements"> <sect2 id="node-requirements">
<title>Node requirements</title> <title>Node requirements</title>
<para><literal>portmgr</literal> is still working on characterizing <note>
<para>This section is only of interest when considering
tier-2 architectures.</para>
</note>
<para>Here are the requirement for
what a node needs to be generally useful.</para> what a node needs to be generally useful.</para>
<itemizedlist> <itemizedlist>
@ -1440,7 +1442,8 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<filename>/usr2/</filename>.)</para> <filename>/usr2/</filename>.)</para>
<note> <note>
<para>The filename <filename>chroot</filename> is a <para>The filename <filename>chroot</filename> is a
historical remnant.</para> historical remnant. The <command>chroot</command>
command is no longer used.</para>
</note> </note>
</step> </step>
</procedure> </procedure>
@ -1481,8 +1484,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
<step> <step>
<para>Generate a kernel config file. Include <para>Generate a kernel config file. Include
<filename>GENERIC</filename> (or, if you are using more than <filename>GENERIC</filename> (or, if on &i386, and
3.5G on &i386;, <filename>PAE</filename>).</para> you are using more than
3.5G, <filename>PAE</filename>).</para>
<para>Required options:</para> <para>Required options:</para>
@ -1497,9 +1501,14 @@ options SHMMAXPGS=65536
options SEMMNI=40 options SEMMNI=40
options SEMMNS=240 options SEMMNS=240
options SEMUME=40 options SEMUME=40
options SEMMNU=120 options SEMMNU=120</programlisting>
options ALT_BREAK_TO_DEBUGGER</programlisting> <para>If you are interested in debugging general
problems, you may wish to use the following.
However, for unattended operations, it is best
to leave it out:</para>
<programlisting>options ALT_BREAK_TO_DEBUGGER</programlisting>
<para>For <filename>PAE</filename>, it is not currently possible <para>For <filename>PAE</filename>, it is not currently possible
to load modules. Therefore, if you are running an architecture to load modules. Therefore, if you are running an architecture
@ -1916,7 +1925,7 @@ portbuild ALL=(ALL) NOPASSWD: ALL</programlisting>
<step> <step>
<para>Create <para>Create
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/clients/bindist-<replaceable>${hostname}</replaceable>.tar</filename>.</para> <filename>/a/portbuild/<replaceable>${arch}</replaceable>/clients/bindist-<replaceable>${hostname}</replaceable>.tar</filename>.</para>
<itemizedlist> <itemizedlist>
<listitem> <listitem>
@ -1955,16 +1964,16 @@ MASTER_SITE_OVERRIDE= \
<para>Hint: you will need one of these for each machine; <para>Hint: you will need one of these for each machine;
however, if you have multiple machines at one site, you however, if you have multiple machines at one site, you
should create a site-specific one (e.g., in should create a site-specific one (e.g., in
<filename>/var/portbuild/conf/clients/</filename>) <filename>/a/portbuild/conf/clients/</filename>)
and symlink to it.</para> and symlink to it.</para>
</step> </step>
<step> <step>
<para>Create <para>Create
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/portbuild-<replaceable>${hostname}</replaceable></filename> <filename>/a/portbuild/<replaceable>${arch}</replaceable>/portbuild-<replaceable>${hostname}</replaceable></filename>
using one of the existing ones as a guide. This using one of the existing ones as a guide. This
file contains overrides to file contains overrides to
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/portbuild.conf</filename>.</para> <filename>/a/portbuild/<replaceable>${arch}</replaceable>/portbuild.conf</filename>.</para>
<para>Suggested values:</para> <para>Suggested values:</para>
@ -2030,7 +2039,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlisting>
<step> <step>
<para>Populate the client's copy of <para>Populate the client's copy of
<filename>/var/portbuild/scripts/</filename> by something like <filename>/var/portbuild/scripts/</filename> by something like
<userinput>/var/portbuild/scripts/dosetupnode <replaceable>arch</replaceable> <replaceable>major</replaceable> latest <replaceable>hostname</replaceable></userinput>. <userinput>/a/portbuild/scripts/dosetupnode <replaceable>arch</replaceable> <replaceable>major</replaceable> latest <replaceable>hostname</replaceable></userinput>.
Verify that you now have files in that directory.</para> Verify that you now have files in that directory.</para>
</step> </step>
@ -2075,7 +2084,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlisting>
<para>Once you are sure that the client is working, tell <para>Once you are sure that the client is working, tell
<application>pollmachine</application> about it by adding <application>pollmachine</application> about it by adding
it to it to
<filename>/var/portbuild/<replaceable>${arch}</replaceable>/mlist</filename>.</para> <filename>/a/portbuild/<replaceable>${arch}</replaceable>/mlist</filename>.</para>
</step> </step>
</procedure> </procedure>
</sect2> </sect2>