Completely update the Diskless Operation section.

Submitted by:	Jean-Francois Dockes <jean-francois.dockes@wanadoo.fr>
This commit is contained in:
Chern Lee 2001-10-13 01:39:02 +00:00
parent 0b61dde25e
commit 2b432f7183
Notes: svn2git 2020-12-08 03:00:23 +00:00
svn path=/head/; revision=10926

View file

@ -1097,256 +1097,490 @@ Exports list on foobar:
<sect1info>
<authorgroup>
<author>
<firstname>Martin</firstname>
<surname>Renters</surname>
<contrib>Contributed by </contrib>
<firstname>Jean-Fran&ccedil;ois</firstname>
<surname>Dock&egrave;s</surname>
<contrib>Updated by </contrib>
</author>
</authorgroup>
</sect1info>
<title>Diskless Operation</title>
<indexterm><primary>diskless workstation</primary></indexterm>
<para><filename>netboot.com</filename>/<filename>netboot.rom</filename>
allow you to boot your FreeBSD machine over the network and run FreeBSD
without having a disk on your client. Under 2.0 it is now possible to
have local swap. Swapping over NFS is also still supported.</para>
<para>Supported Ethernet cards include: Western Digital/SMC 8003, 8013,
8216 and compatibles; NE1000/NE2000 and compatibles (requires
recompile)</para>
<para>A FreeBSD machine can boot over the network and operate without a
local disk, using file systems mounted from an NFS server. No system
modification is necessary, beyond standard configuration files.
Such a system is easy to set up because all the necessary elements
are readily available:</para>
<itemizedlist>
<listitem>
<para>There are at least two possible methods to load the kernel over
the network:</para>
<itemizedlist>
<listitem>
<para><emphasis>PXE</emphasis>: Intel's Preboot Execution
Environment system is a form of smart boot ROM built into some
networking cards or motherboards. See &man.pxeboot.8; for more
details.</para>
</listitem>
<listitem>
<para><emphasis>The <application>etherboot</application>
port</emphasis> (<filename>/usr/ports/net/etherboot</filename>)
produces ROM-able code to boot kernels over the network. The code
can be either burnt into a boot PROM on a network card, or loaded
from a local floppy (or hard) disk drive, or from a running
MS-DOS system. Many network cards are supported.</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>A sample script
(<filename>/usr/local/share/examples/clone_root</filename>) eases
the creation and maintenance of the workstation's root filesystem
on the server. The script will probably require a little
customization but it will get you started very quickly</para>
</listitem>
<listitem>
<para>Standard system startup files exist in <filename>/etc</filename>
to detect and support a diskless system startup.</para>
</listitem>
<listitem>
<para>Swapping, if needed, can be done either to an NFS file or to
a local disk</para>
</listitem>
</itemizedlist>
<para>There are many ways to set up diskless workstations. Many
elements are involved, and most can be customized to suit local
taste. The following will describe the setup of a complete system,
emphasising simplicity and compatibility with the
standard FreeBSD startup scripts. The system described has the
following characteristics:</para>
<itemizedlist>
<listitem>
<para>The diskless workstations use a shared
read-only <filename>root</filename> filesystem, and a shared
read-only <filename>/usr</filename>.</para>
<para>The <filename>root</filename> file system is a copy of a
standard FreeBSD root (typically the server's), with some
configuration files overriden by ones specific to diskless
operation or, possibly, to the workstation they belong to.</para>
<para>The parts of the <filename>root</filename> which have to be
writable are overlayed with &man.mfs.8; filesystems. Any changes
will be lost when the system reboots.</para>
</listitem>
<listitem>
<para>The kernel is loaded by <application>etherboot
</application>, using DHCP (or BOOTP) and TFTP.</para>
</listitem>
</itemizedlist>
<caution><para>As described, this system is insecure. It should
live in a protected area of a network, and be untrusted by
other hosts.</para>
</caution>
<sect2>
<title>Setup Instructions</title>
<procedure>
<step>
<para>Find a machine that will be your server. This machine will
require enough disk space to hold the FreeBSD 2.0 binaries and
have bootp, tftp and NFS services available. Tested
machines:</para>
<itemizedlist>
<indexterm><primary>HP-UX</primary></indexterm>
<listitem>
<para>HP9000/8xx running HP-UX 9.04 or later (pre 9.04 does not
work)</para>
</listitem>
<indexterm><primary>Solaris</primary></indexterm>
<listitem>
<para>Sun/Solaris 2.3. (you may need to get bootp)</para>
</listitem>
</itemizedlist>
</step>
<step>
<para>Set up a bootp server to provide the client with IP address, gateway,
netmask.</para>
<programlisting>diskless:\
:ht=ether:\
:ha=0000c01f848a:\
:sm=255.255.255.0:\
:hn:\
:ds=192.1.2.3:\
:ip=192.1.2.4:\
:gw=192.1.2.5:\
:vm=rfc1048:</programlisting>
</step>
<sect3>
<title>Configuring DHCP/BOOTP</title>
<indexterm><primary>TFTP</primary></indexterm>
<indexterm><primary>bootp</primary></indexterm>
<step>
<para>Set up a TFTP server (on same machine as bootp server) to
provide booting information to client. The name of this file is
<filename>cfg.<replaceable>X.X.X.X</replaceable></filename> (or
<filename>/tftpboot/cfg.<replaceable>X.X.X.X</replaceable></filename>,
it will try both) where <replaceable>X.X.X.X</replaceable> is the
IP address of the client. The contents of this file can be any
valid netboot commands. Under 2.0, netboot has the following
commands:</para>
<para>There are two protocols that are commonly used to boot a
workstation that retrieves its configuration over the network: BOOTP
and DHCP. They are used at several points in the workstation
bootstrap:</para>
<itemizedlist>
<listitem><para><application>etherboot</application> uses
DHCP (by default) or BOOTP (needs a configuration option) to
find the kernel. (PXE uses DHCP).</para>
</listitem>
<listitem><para>The kernel uses BOOTP to locate the NFS
root.</para>
</listitem>
</itemizedlist>
<informaltable frame="none">
<tgroup cols="2">
<tbody>
<row>
<entry>help</entry>
<entry>print help list</entry>
</row>
<row>
<entry>ip
<option><replaceable>X.X.X.X</replaceable></option></entry>
<entry>print/set client's IP address</entry>
</row>
<row>
<entry>server
<option><replaceable>X.X.X.X</replaceable></option></entry>
<entry>print/set bootp/tftp server address</entry>
</row>
<row>
<entry>netmask
<option><replaceable>X.X.X.X</replaceable></option></entry>
<entry>print/set netmask</entry>
</row>
<para>It is possible to configure a system to use only BOOTP.
The &man.bootpd.8; server program is included in the
base FreeBSD system.</para>
<row>
<entry>hostname <replaceable>name</replaceable></entry>
<entry>print/set hostname</entry>
</row>
<row>
<entry>kernel
<option><replaceable>name</replaceable></option></entry>
<entry>print/set kernel name</entry>
</row>
<row>
<entry>rootfs
<option><replaceable>ip:/fs</replaceable></option></entry>
<entry>print/set root filesystem</entry>
</row>
<row>
<entry>swapfs
<option><replaceable>ip:/fs</replaceable></option></entry>
<entry>print/set swap filesystem</entry>
</row>
<row>
<entry>swapsize
<option><replaceable>size</replaceable></option></entry>
<entry>set diskless swapsize in KBytes</entry>
</row>
<row>
<entry>diskboot</entry>
<entry>boot from disk</entry>
</row>
<row>
<entry>autoboot</entry>
<entry>continue boot process</entry>
</row>
<row>
<entry>trans
<option>on</option>|<option>off</option></entry>
<entry>turn transceiver on|off</entry>
</row>
<row>
<entry>flags
<option>b</option><option>c</option><option>d</option><option>h</option><option>s</option><option>v</option></entry>
<entry>set boot flags</entry>
</row>
</tbody>
</tgroup>
</informaltable>
<para>A typical completely diskless config file might contain:</para>
<para>However, DHCP has a number of advantages over BOOTP (nicer
configuration files, possibility of using PXE, plus many others
not directly related to diskless operation), and we shall describe
both a pure BOOTP, and a BOOTP+DHCP configuration, with an
emphasis on the latter, which will use the ISC DHCP software
package.</para>
<programlisting>rootfs 192.1.2.3:/rootfs/myclient
swapfs 192.1.2.3:/swapfs
swapsize 20000
hostname myclient.mydomain</programlisting>
<sect4>
<title>Configuration using ISC DHCP</title>
<para>A config file for a machine with local swap might contain:</para>
<para>The <application>isc-dhcp</application> server can answer
both BOOTP and DHCP requests.</para>
<programlisting>rootfs 192.1.2.3:/rootfs/myclient
hostname myclient.mydomain</programlisting>
</step>
<para>As of release 4.4, <application>isc-dhcp</application> is not part of the base
system. You will first need to install the
<filename>/usr/ports/net/isc-dhcp3</filename> port or the
corresponding package. Please refer to <xref linkend="ports">
for general information about ports and packages.</para>
<step>
<para>Ensure that your NFS server has exported the root (and swap if
applicable) filesystems to your client, and that the client has
root access to these filesystems A typical
<filename>/etc/exports</filename> file on FreeBSD might look
like:</para>
<programlisting>/rootfs/myclient -maproot=0:0 myclient.mydomain
/swapfs -maproot=0:0 myclient.mydomain</programlisting>
<para>Once <application>isc-dhcp</application> is installed, it
needs a configuration file to run, (normally named
<filename>/usr/local/etc/dhcpd.conf</filename>). Here follows
a commented example:</para>
<para>And on HP-UX:</para>
<programlisting>
default-lease-time 600;
max-lease-time 7200;
authoritative;
<programlisting>/rootfs/myclient -root=myclient.mydomain
/swapfs -root=myclient.mydomain</programlisting>
</step>
<indexterm>
<primary>NFS</primary>
<secondary>swapping over</secondary>
</indexterm>
<step>
<para>If you are swapping over NFS (completely diskless
configuration) create a swap file for your client using
<command>dd</command>. If your <command>swapfs</command>
command has the arguments <filename>/swapfs</filename> and
the size 20000 as in the example above, the swapfile for
<hostid>myclient</hostid> will be called
<filename>/swapfs/swap.<replaceable>X.X.X.X</replaceable></filename>
where <replaceable>X.X.X.X</replaceable> is the client's IP
address, e.g.:</para>
<screen>&prompt.root; <userinput>dd if=/dev/zero of=/swapfs/swap.192.1.2.4 bs=1k count=20000</userinput></screen>
<para>Also, the client's swap space might contain sensitive
information once swapping starts, so make sure to restrict read
and write access to this file to prevent unauthorized
access:</para>
<screen>&prompt.root; <userinput>chmod 0600 /swapfs/swap.192.1.2.4</userinput></screen>
</step>
option domain-name "mydom.com";
option domain-name-servers 192.168.4.1;
option routers 192.168.4.1;
<step>
<para>Unpack the root filesystem in the directory the client will
use for its root filesystem (<filename>/rootfs/myclient</filename>
in the example above).</para>
<itemizedlist>
<listitem>
<para>On HP-UX systems: The server should be running HP-UX 9.04
or later for HP9000/800 series machines. Prior versions do not
allow the creation of device files over NFS.</para>
</listitem>
subnet 192.168.4.0 netmask 255.255.255.0 {
use-host-decl-names on; <co id="co-dhcp-host-name">
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.4.255;
host margaux {
hardware ethernet 01:23:45:67:89:ab;
fixed-address margaux.mydom.com;
next-server 192.168.4.4;<co id="co-dhcp-next-server">
filename "/tftpboot/kernel.diskless";<co id="co-dhcp-filename">
option root-path "192.168.4.4:/data/misc/diskless";<co id="co-dhcp-root-path">
}
}
</programlisting>
<calloutlist>
<callout arearefs="co-dhcp-host-name"><para>This option tells
<command>dhcpd</command> to send the value in the
<literal>host</literal> declarations as the hostname for the
diskless host. An alternate way would be to add an
<literal>option host-name
<replaceable>margaux</replaceable></literal> inside the
host declarations.</para>
</callout>
<listitem>
<para>When extracting <filename>/dev</filename> in
<filename>/rootfs/myclient</filename>, beware that some
systems (HPUX) will not create device files that FreeBSD is
happy with. You may have to go to single user mode on the
first bootup (press control-c during the bootup phase), cd
<filename>/dev</filename> and do a <command>sh ./MAKEDEV
all</command> from the client to fix this.</para>
</listitem>
</itemizedlist>
</step>
<callout arearefs="co-dhcp-next-server"><para>The
<literal>next-server</literal> directive designates
the TFTP server (the default is to use the same host as the
DHCP server).</para>
</callout>
<callout arearefs="co-dhcp-filename"><para>The
<literal>filename</literal> directive defines the file that
<application>etherboot</application> will load as a
kernel.
<note><para>PXE appears to prefer a relative file
name, and it loads <command>pxeboot</command>, not the
kernel (<literal>option filename
"pxeboot"</literal>).</para>
</note>
</para>
</callout>
<callout arearefs="co-dhcp-root-path"><para>The
<literal>root-path</literal> option defines the path to
the root filesystem, in usual NFS notation</para>
</callout>
</calloutlist>
<step>
<para>Run <command>netboot.com</command> on the client or make an
EPROM from the <filename>netboot.rom</filename> file</para>
</step>
</procedure>
</sect2>
<sect2>
<title>Using Shared <filename>/</filename> and <filename>/usr</filename>
Filesystems</title>
</sect4>
<sect4>
<title>Configuration using BOOTP</title>
<para>Here follows an equivalent <command>bootpd</command>
configuration. This would be found in
<filename>/etc/bootptab</filename>.</para>
<para>Please note that <application>etherboot</application> must
be compiled with the non-default option
<literal>NO_DHCP_SUPPORT</literal> in order to use BOOTP, and that PXE
<emphasis>needs</emphasis> DHCP. The only obvious advantage of
<application>bootpd</application> is that it exists in the base system.</para>
<programlisting>
.def100:\
:hn:ht=1:sa=192.168.4.4:vm=rfc1048:\
:sm=255.255.255.0:\
:ds=192.168.4.1:\
:gw=192.168.4.1:\
:hd="/tftpboot":\
:bf="/kernel.diskless":\
:rp="192.168.4.4:/data/misc/diskless":
margaux:ha=0123456789ab:tc=.def100
</programlisting>
</sect4>
</sect3>
<sect3>
<title>Preparing a boot program with
<application>Etherboot</application></title>
<para><ulink url="http://etherboot.sourceforge.net">Etherboot's Web
site</ulink> contains
<ulink url="http://etherboot.sourceforge.net/doc/html/userman.html">
extensive documentation</ulink> mainly intended for Linux
systems, but nonetheless containing useful information. The following
will just outline how you would use
<application>etherboot</application> on a FreeBSD system.</para>
<para>You must first install - and possibly compile - the
<application>etherboot</application> package. The
<application>etherboot</application> port can normally be found in
<filename>/usr/ports/net/etherboot</filename>. If the ports tree is
installed on your system, just typing <literal>make</literal> in
this directory should take care of everything. Else refer to
<xref linkend="ports"> for information about ports and
packages.</para>
<para>For our setup, we shall use a boot floppy. For other methods
(PROM, or dos program), please refer to the
<application>etherboot</application> documentation.</para>
<para>To make a boot floppy, insert a floppy in the drive on the
machine where you installed <application>etherboot</application>,
then change your current directory to the <filename>src</filename>
directory in the <application>etherboot</application> tree and
type:</para>
<screen>
&prompt.root; <userinput>gmake bin32/<replaceable>devicetype</replaceable>.fd0</userinput>
</screen>
<para><replaceable>devicetype</replaceable> depends on the type of
the ethernet card in the diskless workstation. Refer to the
<filename>NIC</filename> file in the same directory to determine the
right <replaceable>devicetype</replaceable>.</para>
</sect3>
<sect3>
<title>Configuring the TFTP and NFS servers</title>
<para>You need to enable <command>tftpd</command> on the TFTP
server:</para>
<procedure>
<step>
<para>Create a directory from which <command>tftpd</command>
will serve the files, ie: <filename>/tftpboot</filename></para>
</step>
<step>
<para>Add this line to your
<filename>/etc/inetd.conf</filename>:</para>
<programlisting>tftp dgram udp wait nobody /usr/libexec/tftpd tftpd /tftpboot</programlisting>
<note><para>It appears that at least some PXE versions wants
the TCP version of TFTP. In this case, add a second line,
replacing <literal>dgram udp</literal> with <literal>stream
tcp</literal></para>
</note>
</step>
<step>
<para>Tell <command>inetd</command> to reread its configuration
file:</para>
<screen>&prompt.root; <userinput>kill -HUP `cat /var/run/inetd.pid`</userinput></screen>
</step>
</procedure>
<para>You can place the tftpboot directory anywhere on the server, but,
of course, the actual location, the value in
<filename>inetd.conf</filename>, and the value in
<filename>dhcpd.conf</filename> must be consistent !</para>
<para>You also need to enable NFS service and export the
appropriate filesystem on the NFS server</para>
<procedure>
<step>
<para>Add this to <filename>/etc/rc.conf</filename>:</para>
<programlisting>nfs_server_enable="YES"</programlisting>
</step>
<step>
<para>Export the filesystem where the diskless root directory
is located by adding the following to
<filename>/etc/exports</filename> (adjust the volume mount
point and workstation name!):</para>
<programlisting><replaceable>/data/misc</replaceable> -alldirs -ro <replaceable>margaux</replaceable></programlisting>
</step>
<step>
<para>Tell <command>mountd</command> to reread its configuration
file. If you actually needed to configure NFS service at step
1, you probably want to reboot instead.</para>
<screen>&prompt.root; <userinput>kill -HUP `cat /var/run/mountd.pid`</userinput></screen>
</step>
</procedure>
</sect3>
<sect3>
<title>Building a diskless kernel</title>
<para>Create a kernel configuration file for the diskless client
with the following options (in addition to the usual
ones):</para>
<programlisting>
options BOOTP # Use BOOTP to obtain IP address/hostname
options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info
options BOOTP_COMPAT # Workaround for broken bootp daemons.
</programlisting>
<para>You may also want to use <literal>BOOTP_NFSV3</literal> and
<literal>BOOTP_WIRED_TO</literal> (refer to <filename>LINT</filename>).</para>
<para>Build the kernel (See <xref linkend="kernelconfig">),
and copy it to the tftp directory, under the name listed
in <filename>dhcpd.conf</filename></para>
</sect3>
<sect3>
<title>Preparing the root filesystem</title>
<para>You need to create a root filesystem for the diskless
workstations, in the location listed as
<literal>root-path</literal> in
<filename>dhcpd.conf</filename>.</para>
<para>The easiest way to do this is to use the
<filename>/usr/share/examples/diskless/clone_root</filename>
shell script. This script needs customization, at least to adjust the
place where the filesystem will be created (the
<literal>DEST</literal> variable).
<para>Refer to the comments at the top of the script for
instructions. They explain how the base filesystem is built,
and how files may be selectively overriden by versions specific
to diskless operation, to a subnetwork, or to an individual
workstation. They also give examples for the diskless
<filename>/etc/fstab</filename> and <filename>
/etc/rc.conf</filename></para>
<para>The README files in
<filename>/usr/share/examples/diskless</filename> contain a lot
of interesting background information, but, together with the
other examples in the <filename>diskless</filename> directory,
they actually document a configuration method which is distinct
from the one used by <filename>clone_root</filename> and
<filename>/etc/rc.diskless[12]</filename>, which is a little
confusing. Use them for reference only, except if you prefer
the method that they describe, in which case you will need
customized <filename>rc</filename> scripts</para>
<para>As of FreeBSD version 4.4-RELEASE, there is a small
incompatibility between the <filename>clone_root</filename>
script and the <filename>/etc/rc.diskless1</filename>
script. Please refer to PR
<ulink url="http://www.freebsd.org/cgi/query-pr.cgi?pr=31200">
conf/31200</ulink> for the small adjustment needed in
<filename>clone_root</filename>.
Also see PR
<ulink url="http://www.freebsd.org/cgi/query-pr.cgi?pr=29870">conf/29870</ulink>
about a small adjustment needed in
<filename>/etc/rc.diskless2</filename>.
</para>
</sect3>
<sect3>
<title>Configuring swap</title>
<para>If needed, it is possible to do swapping over NFS, to a file
on the server. The exact <filename>bootptab</filename> or
<filename>dhcpd.conf</filename> options are a little mysterious
and poorly documented. Anyway, here is what worked for me, using
isc-dhcp 3.0rc11.</para>
<procedure>
<step><para>Add the following lines to
<filename>dhcpd.conf</filename>:</para>
<programlisting>
# Global section
option swap-path code 128 = string;
option swap-size code 129 = integer 32;
host margaux {
... # Standard lines, see above
option swap-path <replaceable>"192.168.4.4:/netswapvolume/netswap"</replaceable>;
option swap-size <replaceable>64000</replaceable>;
}
</programlisting>
<para>The idea is that, at least for a FreeBSD client,
DHCP/BOOTP option code 128 is the path to the NFS swap file,
and option code 129 is the swap size in kilobytes. Older
versions of <command>dhcpd</command> allowed a syntax of
<literal>option option-128 "...</literal>, which does not
seem to work any more.</para>
<para><filename>/etc/bootptab</filename> would use the
following syntax instead:
<literal>T128="192.168.4.4:/netswapvolume/netswap":T129=64000
</literal></para>
</step>
<step>
<para>On the NFS swap file server, create the swap
file(s)</para>
<screen>
&prompt.root; <userinput>mkdir <replaceable>/netswapvolume/netswap</replaceable></userinput>
&prompt.root; <userinput>cd <replaceable>/netswapvolume/netswap</replaceable></userinput>
&prompt.root; <userinput>dd if=/dev/zero bs=1024 count=<replaceable>64000</replaceable> of=swap.<replaceable>192.168.4.6</replaceable></userinput>
&prompt.root; <userinput>chmod 0600 swap.<replaceable>192.168.4.6</replaceable></userinput>
</screen>
<para><replaceable>192.168.4.6</replaceable> is the IP address
for the diskless client</para>
</step>
<step>
<para>On the NFS swap file server, add the following line to
<filename>/etc/exports</filename></para>
<programlisting>
<replaceable>/netswapvolume</replaceable> -maproot=0:10 -alldirs <replaceable>margaux</replaceable>
</programlisting>
<para>Then tell mountd to reread the exports file, as above.</para>
</step>
</procedure>
</sect3>
<sect3>
<title>Miscellaneous issues</title>
<sect4>
<title>Running with a read-only <filename>/usr</filename></title
<para>If the diskless workstation is configured to run X, you
will have to adjust the xdm configuration file, which puts
the error log on <filename>/usr</filename> by default.</para>
</sect4>
<sect4>
<title>Using a non-FreeBSD server</title>
<para>When the server for the root filesystem is not running FreeBSD,
you will have to create the root file system on a
FreeBSD machine, then copy it to its destination, using
<command>tar</command> or <command>cpio</command>.</para>
<para>In this situation, it seems that there are sometimes
problems with the special files in <filename>/dev</filename>,
because of differing minor/major number integer sizes. It seems
that a solution to this problem is to run MAKEDEV on a FreeBSD
machine, in a directory mounted through NFS from the final
server.</para>
</sect4>
</sect3>
<para>Although this is not an officially sanctioned or supported way
of doing this, some people report that it works quite well. If
anyone has any suggestions on how to do this cleanly, please tell
&a.doc;.</para>
</sect2>
<sect2>
<title>Compiling Netboot for Specific Setups</title>
<para>Netboot can be compiled to support NE1000/2000 cards by changing
the configuration in
<filename>/sys/i386/boot/netboot/Makefile</filename>. See the
comments at the top of this file.</para>
</sect2>
</sect1>