Add an edited version of Gabor Pali <pgj@FreeBSD.org>'s contribution on

port staging support.

en_US.ISO8859-1/books/porters-handbook/book.xml

@@ -271,6 +271,7 @@ lib/X11/oneko/mouse.xpm
 	  following lines to the <filename>Makefile</filename>:</para>
 
 	<programlisting>PLIST_FILES=	bin/oneko \
+		man/man1/oneko.1.gz \
 		lib/X11/app-defaults/Oneko \
 		lib/X11/oneko/cat1.xpm \
 		lib/X11/oneko/cat2.xpm \
@@ -335,76 +336,79 @@ PLIST_DIRS=	lib/X11/oneko</programlisting>
       <itemizedlist>
 	<listitem>
 	  <para><filename>pkg-plist</filename> does not contain
-	    anything not installed by your port</para>
+	    anything not installed by the port.</para>
 	</listitem>
 
 	<listitem>
 	  <para><filename>pkg-plist</filename> contains everything
-	    that is installed by your port</para>
+	    that is installed by the port.</para>
 	</listitem>
 
 	<listitem>
-	  <para>Your port can be installed multiple times using the
+	  <para>The port can be installed using the
-	    <maketarget>reinstall</maketarget> target</para>
+	    <maketarget>install</maketarget> target.  This verifies
+	    that the install script works correctly.</para>
 	</listitem>
 
 	<listitem>
-	  <para>Your port <link linkend="plist-cleaning">cleans
+	  <para>The port can be deinstalled properly using the
-	      up</link> after itself upon deinstall</para>
+	    <maketarget>deinstall</maketarget> target.  This verifies
+	    that the deinstall script works correctly.</para>
+	</listitem>
+
+	<listitem>
+	  <para>Make sure that <command>make package</command> can be
+	    run as a normal user (that is, not as
+	    <username>root</username>).  If that fails,
+	    <literal>NEED_ROOT=yes</literal> must be added to the
+	    port <filename>Makefile</filename>.</para>
 	</listitem>
       </itemizedlist>
 
       <procedure>
 	<title>Recommended Test Ordering</title>
 
+	<step>
+	  <para><command>make stage</command></para>
+	</step>
+
+	<step>
+	  <para><command>make check-orphans</command></para>
+	</step>
+
+	<step>
+	  <para><command>make package</command></para>
+	</step>
+
 	<step>
 	  <para><command>make install</command></para>
 	</step>
 
-	<step>
-	  <para><command>make package</command></para>
-	</step>
-
 	<step>
 	  <para><command>make deinstall</command></para>
 	</step>
 
 	<step>
-	  <para><command>pkg_add
+	  <para><command>pkg_add <replaceable>package-name</replaceable></command></para>
-	      <replaceable>package-name</replaceable></command></para>
+
+	  <para>Or, for users of <emphasis>pkg</emphasis>:</para>
+
+	  <para><command>pkg add <replaceable>package-name</replaceable></command></para>
 	</step>
 
 	<step>
-	  <para><command>make deinstall</command></para>
+	  <para><command>make package</command> (as user)</para>
-	</step>
-
-	<step>
-	  <para><command>make reinstall</command></para>
-	</step>
-
-	<step>
-	  <para><command>make package</command></para>
-	</step>
-
-	<step>
-	  <para><command>make readme</command></para>
 	</step>
       </procedure>
 
-      <para>Make sure that there are not any warnings issued in any of
+      <para>Make certain no warnings are shown in any of
-	the <maketarget>package</maketarget> and
+	the stages.</para>
-	<maketarget>deinstall</maketarget> stages.  After step 3,
-	check to see if all the new directories are correctly deleted.
-	Also, try using the software after step 4, to ensure that it
-	works correctly when installed from a package.</para>
 
-      <para>The most thorough way to automate these steps is via
+      <para>Thorough automated testing can be done with <filename role="package">ports-mgmt/tinderbox</filename> or
-	installing the <application>ports tinderbox</application>.
+	<filename role="package">ports-mgmt/poudriere</filename> from the Ports Collection.
-	This maintains <literal>jails</literal> in which you can
+	These applications maintain
-	test all of the above steps without changing the state of
+	<literal>jails</literal> where all of the steps shown above
-	your running system.  Please see
+	can be tested without affecting the state of the host system.</para>
-	<filename>ports/ports-mgmt/tinderbox</filename> for more
-	information.</para>
     </sect1>
 
     <sect1 id="porting-portlint">
@@ -572,11 +576,12 @@ PLIST_DIRS=	lib/X11/oneko</programlisting>
 	</step>
 
 	<step>
-	  <para>The <maketarget>stage</maketarget> target is run.
+	  <para>The <maketarget>stage</maketarget> target is run.  This
-	    This puts the final set of built files into a temporary
+	    puts the final set of built files into a temporary directory
-	    directory (<makevar>STAGEDIR</makevar>).  The hierarchy of
+	    (<makevar>STAGEDIR</makevar>, see <xref
-	    this directory mirrors that of the system on which the
+	      linkend="staging"/>).  The hierarchy of this directory
-	    package will be installed.</para>
+	    mirrors that of the system on which the package will be
+	    installed.</para>
 	</step>
 
 	<step>
@@ -4979,10 +4984,18 @@ PORTVERSION=	1.0</programlisting>
       <sect2 id="install-macros">
 	<title><makevar>INSTALL_*</makevar> Macros</title>
 
-	<para>Do use the macros provided in
+	<para>Use the macros provided in
-	  <filename>bsd.port.mk</filename> to ensure correct modes and
+	  <filename>bsd.port.mk</filename> to ensure correct modes of
-	  ownership of files in your own
+	  files in the port's <maketarget>*-install</maketarget> targets.
-	  <maketarget>*-install</maketarget> targets.</para>
+	  Set ownership directly in
+	  <filename>pkg-plist</filename> with the corresponding
+	  entries, such as <literal>@owner
+	  <replaceable>owner</replaceable></literal> and <literal>@group
+	  <replaceable>group</replaceable></literal>.  These
+	  operators work until being overridden, or until the end of
+	  <filename>pkg-plist</filename>, so do not forget to reset them
+	  after they are no longer needed.  The default
+	  ownership is <literal>root:wheel</literal>.</para>
 
 	<itemizedlist>
 	  <listitem>
@@ -5056,11 +5069,15 @@ PORTVERSION=	1.0</programlisting>
       <sect2 id="install-copytree">
 	<title>Installing a Whole Tree of Files</title>
 
-	<para>Sometimes, there is a need to install a big number of
+	<para>Sometimes, a large number of
-	  files, preserving their hierarchical organization, i.e.,
+	  files must be installed while preserving their hierarchical organization.  For example,
 	  copying over a whole directory tree from
 	  <makevar>WRKSRC</makevar> to a target directory under
-	  <makevar>PREFIX</makevar>.</para>
+	  <makevar>PREFIX</makevar>.  Note that
+	  <makevar>PREFIX</makevar>, <makevar>EXAMPLESDIR</makevar>,
+	  <makevar>DATADIR</makevar>, and other path varialbes must always be
+	  prepended with <makevar>STAGEDIR</makevar> to respect
+	  staging (see <xref linkend="staging"/>).</para>
 
 	<para>Two macros exist for this situation.  The advantage of
 	  using these macros instead of <command>cp</command> is that
@@ -5103,9 +5120,15 @@ PORTVERSION=	1.0</programlisting>
 	(cd ${WRKSRC}/examples &amp;&amp; \
 	${COPYTREE_SHARE} . ${STAGEDIR}${EXAMPLESDIR} "! -name Makefile")</programlisting>
 
-	<para>Note that these macros does not add the installed files
+	<para>These macros do not add the installed files
-	  to <filename>pkg-plist</filename>.  You still need to list
+	  to <filename>pkg-plist</filename>.  They must be added manually.
-	  them.</para>
+	  For optional documentation
+	  (<makevar>PORTDOCS</makevar>, see <xref
+	    linkend="install-documentation"/>) and examples
+	  (<makevar>PORTEXAMPLES</makevar>), the
+	  <literal>%%PORTDOCS%%</literal> or
+	  <literal>%%PORTEXAMPLES%%</literal> prefixes must be
+	  prepended in <filename>pkg-plist</filename>.</para>
       </sect2>
 
       <sect2 id="install-documentation">
@@ -5126,9 +5149,11 @@ PORTVERSION=	1.0</programlisting>
 	  <makevar>PKGNAME</makevar>.</para>
 
 	<para>Since only the files listed in
-	  <filename>pkg-plist</filename> are installed it is safe to
+	  <filename>pkg-plist</filename> are installed, it is safe to
-	  always install documentation to
+	  always install documentation to <makevar>STAGEDIR</makevar>
-	  <makevar>STAGEDIR</makevar>.</para>
+	  (see <xref linkend="staging"/>).  Hence <literal>.if</literal>
+	  blocks are only needed when the installed files are
+	  large enough to cause significant I/O overhead.</para>
 
 	<programlisting>post-install:
 	  ${MKDIR} ${STAGEDIR}${DOCSDIR}
@@ -5271,6 +5296,91 @@ PORTVERSION=	1.0</programlisting>
       when you create a port.  This section explains the most common
       of those.</para>
 
+    <sect1 id="staging">
+      <title>Staging</title>
+
+      <para><filename>bsd.port.mk</filename> expects ports to work with
+	a <quote>stage directory</quote>.  This means that a
+	port should not install files directly to the regular
+	destination directories (that is, under
+	<makevar>PREFIX</makevar>, for example) but instead into a
+	separate directory from which the package is then built.  In
+	many cases, this does not require root privileges, making it
+	possible to build packages as an unprivileged user.  With
+	staging, the port is built and installed into the stage
+	directory, <makevar>STAGEDIR</makevar>.  A package is created
+	from the stage directory and then installed on the system.  Automake tools refer to
+	this concept as <makevar>DESTDIR</makevar>, but in &os;,
+	<makevar>DESTDIR</makevar> has a different meaning (see <xref
+	linkend="porting-prefix"/>).</para>
+
+      <para>When a port still requires system-wide privileges in order
+	to run the <maketarget>package</maketarget> target, this
+	line must be added to the
+	<filename>Makefile</filename>:</para>
+
+      <programlisting>NEED_ROOT=	yes</programlisting>
+
+      <para>Meta ports, or ports that do not install files themselves
+	but only depend on other ports, should avoid
+	needlessly extracting the &man.mtree.8; to the stage directory.
+	This is the basic directory layout
+	of the package, and these empty directories will be seens as
+	orphans.  To prevent &man.mtree.8; extraction, add this line:</para>
+
+      <programlisting>NO_MTREE=	yes</programlisting>
+
+      <para>Staging is enabled by prepending the
+	<makevar>STAGEDIR</makevar> variable to paths used in
+	the <maketarget>pre-install</maketarget>,
+	<maketarget>do-install</maketarget>, and
+	<maketarget>post-install</maketarget> targets (see the examples
+	through the book).  Typically, this includes
+	<makevar>PREFIX</makevar>, <makevar>ETCDIR</makevar>,
+	<makevar>DATADIR</makevar>, <makevar>EXAMPLESDIR</makevar>,
+	<makevar>MANPREFIX</makevar>, <makevar>DOCSDIR</makevar>, and so on.
+	Directories should be created as part of the
+	<maketarget>post-install</maketarget> target.  Avoid using
+	absolute paths whenever possible.</para>
+
+      <para>When creating a symlink, <makevar>STAGEDIR</makevar> should
+	be prepended to the target path only.  For example:</para>
+
+      <programlisting>${LN} -sf libfoo.so.42 ${STAGEDIR}${PREFIX}/lib/libfoo.so</programlisting>
+
+      <para>The source path
+	<filename>${PREFIX}/lib/libfoo.so.42</filename> looks
+	fine but could, in fact, be incorrect.  Absolute paths can
+	point to a wrong location, like when a remote
+	file system has been mounted with <acronym>NFS</acronym> under a non-root mount point.
+	Relative paths are less fragile, and often much shorter.</para>
+
+      <para>Ports that install kernel modules must prepend the
+	<makevar>STAGEDIR</makevar> variable to
+	their default destination, <filename
+	  class="directory">/boot/modules</filename>.  Then the
+	<maketarget>pre-install</maketarget> target can be used to handle
+	the creation of this directory:</para>
+
+      <programlisting>pre-install:
+	${MKDIR} ${STAGEDIR}/boot/modules</programlisting>
+
+      <para>&man.kldxref.8; should not be run when installing to
+	the temporary staging directory.  To prevent this, define
+	the<makevar>NO_XREF</makevar> variable and add it
+	to <makevar>MAKE_ENV</makevar> in the port's
+	<filename>Makefile</filename>:</para>
+
+      <programlisting>MAKE_ENV+=	KMODDIR=/boot/modules NO_XREF=yes</programlisting>
+      <para>The hints files for the kernel loader should
+	be regenerated only when the package is installed or
+	deinstalled.  Do this by invoking &man.kldxref.8; from
+	<filename>pkg-plist</filename>:</para>
+
+      <programlisting>@unexec kldxref /boot/modules
+@exec kldxref /boot/modules</programlisting>
+    </sect1>
+
     <sect1 id="porting-shlibs">
       <title>Shared Libraries</title>
 
@@ -6219,6 +6329,18 @@ PLIST_SUB+=		NLS="@comment "
 
 	<programlisting>p5-IO-Tee&gt;=0.64:${PORTSDIR}/devel/p5-IO-Tee</programlisting>
       </example>
+
+      <para>For Perl ports that install manual pages, the macro
+	<makevar>PERL5_MAN<replaceable>x</replaceable></makevar> (where
+	<replaceable>x</replaceable> ranges from <literal>1</literal> to
+	<literal>9</literal>) can be used inside
+	<filename>pkg-plist</filename>.  For example,</para>
+
+      <programlisting>lib/perl5/5.14/man/man3/AnyEvent::I3.3.gz</programlisting>
+
+      <para>can be replaced with</para>
+
+      <programlisting>%%PERL5_MAN3%%/AnyEvent::I3.3.gz</programlisting>
     </sect1>
 
     <sect1 id="using-x11">
@@ -7829,6 +7951,33 @@ _DOCSDIR=	.
 
       <para>A complete list of available variables can be found in
 	<filename>/usr/ports/Mk/bsd.python.mk</filename>.</para>
+
+      <para>Some Python applications claim to have
+	<makevar>DESTDIR</makevar> support (which would be required for
+	staging) but it is broken (Mailman up to 2.1.16, for instance).
+	This can be worked around by recompiling the scripts.  This can
+	be done, for example, in the <maketarget>post-build</maketarget>
+	target.  Assuming the Python scripts are supposed to reside in
+	<makevar>PYTHONPREFIX_SITELIBDIR</makevar> after
+	installation, this solution can be applied:</para>
+
+      <programlisting>(cd ${STAGEDIR}${PREFIX} \
+  &amp;&amp; ${PYTHON_CMD} ${PYTHON_LIBDIR}/compileall.py \
+   -d ${PREFIX} -f ${PYTHONPREFIX_SITELIBDIR:S;${PREFIX}/;;})
+</programlisting>
+
+      <para>This recompiles the sources with a path relative to the
+	stage directory, and prepends the value of
+	<makevar>PREFIX</makevar> to the file name recorded in the
+	byte-compiled output file by <literal>-d</literal>.
+	<literal>-f</literal> is required to force recompilation,
+	and the <literal>:S;${PREFIX}/;;</literal> strips prefixes from
+	the value of the <makevar>PYTHONPREFIX_SITELIBDIR</makevar>
+	variable to make it relative to
+	<makevar>PREFIX</makevar>.</para>
+
+      <para>Python&nbsp;2.7 or newer is required for this.  It does not work
+	with Python&nbsp;2.6.</para>
     </sect1>
 
     <sect1 id="using-tcl">
@@ -10236,10 +10385,9 @@ etc/orbit.conf.sample
 	which generate docs with
 	<application>Javadoc</application>).</para>
 
-      <para>Maintainers who prefer dynamic package lists are
+      <para>Note that the <maketarget>makeplist</maketarget> target can
-	encouraged to add a new target to their port which generates
+	be used for ports that support staging to display the package
-	the <filename>pkg-plist</filename> file so that users may
+	list.</para>
-	examine the contents.</para>
     </sect1>
 
     <sect1 id="plist-autoplist">
@@ -10625,7 +10773,12 @@ as .putsy.conf and edit it.</programlisting>
 	<makevar>PREFIX</makevar>, the package creation process will
 	complain that it cannot find the files.</para>
 
-      <para>This test will not find hard-coded paths inside the
+      <para>In addition, it is worth checking the same with the
+	stage directory support (see <xref linkend="staging"/>):</para>
+
+      <screen>&prompt.root; <userinput>make stage &amp;&amp; make check-orphans &amp;&amp; make package</userinput></screen>
+
+      <para>These tests will not find hard-coded paths inside the
 	port's files, nor will it verify that
 	<makevar>LOCALBASE</makevar> is being used to correctly refer
 	to files from other ports.  The temporarily-installed port in