<?xml version="1.0" encoding="iso-8859-1"?>
<!--
The FreeBSD Documentation Project
$FreeBSD$
-->
<chapter id="jails">
<chapterinfo>
<authorgroup>
<author>
<firstname>Matteo</firstname>
<surname>Riondato</surname>
<contrib>Contributed by </contrib>
</author>
</authorgroup>
</chapterinfo>
<title>Jails</title>
<indexterm><primary>jails</primary></indexterm>
<sect1 id="jails-synopsis">
<title>Synopsis</title>
<para>This chapter will provide an explanation of what &os; jails
are and how to use them. Jails, sometimes referred to as an
enhanced replacement of
<emphasis>chroot environments</emphasis>, are a very powerful
tool for system administrators, but their basic usage can also
be useful for advanced users.</para>
<important>
<para>Jails are a powerful tool, but they are not a security
panacea. It is particularly important to note that while it
is not possible for a jailed process to break out on its own,
there are several ways in which an unprivileged user outside
the jail can cooperate with a privileged user inside the jail
and thereby obtain elevated privileges in the host
environment.</para>
<para>Most of these attacks can be mitigated by ensuring that
the jail root is not accessible to unprivileged users in the
host environment. Regardless, as a general rule, untrusted
users with privileged access to a jail should not be given
access to the host environment.</para>
</important>
<para>After reading this chapter, you will know:</para>
<itemizedlist>
<listitem>
<para>What a jail is, and what purpose it may serve in &os;
installations.</para>
</listitem>
<listitem>
<para>How to build, start, and stop a jail.</para>
</listitem>
<listitem>
<para>The basics of jail administration, both from inside
and outside the jail.</para>
</listitem>
</itemizedlist>
<para>Other sources of useful information about jails are:</para>
<itemizedlist>
<listitem>
<para>The &man.jail.8; manual page. This is the full
reference of the <command>jail</command> utility — the
administrative tool which can be used in &os; to start,
stop, and control &os; jails.</para>
</listitem>
<listitem>
<para>The mailing lists and their archives. The archives of
the &a.questions; and other mailing lists hosted by the
&a.mailman.lists; already contain a wealth of material for
jails. It should always be engaging to search the archives,
or post a new question to the &a.questions.name; mailing
list.</para>
</listitem>
</itemizedlist>
</sect1>
<sect1 id="jails-terms">
<title>Terms Related to Jails</title>
<para>To facilitate better understanding of parts of the &os;
system related to jails, their internals and the way they
interact with the rest of &os;, the following terms are used
further in this chapter:</para>
<variablelist>
<varlistentry>
<term>&man.chroot.8; (command)</term>
<listitem>
<para>Utility, which uses &man.chroot.2; &os; system call to
change the root directory of a process and all its
descendants.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>&man.chroot.2; (environment)</term>
<listitem>
<para>The environment of processes running in a
<quote>chroot</quote>. This includes resources such as
the part of the file system which is visible, user and
group IDs which are available, network interfaces and
other IPC mechanisms, etc.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>&man.jail.8; (command)</term>
<listitem>
<para>The system administration utility which allows
launching of processes within a jail environment.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>host (system, process, user, etc.)</term>
<listitem>
<para>The controlling system of a jail environment. The
host system has access to all the hardware resources
available, and can control processes both outside of and
inside a jail environment. One of the important
differences of the host system from a jail is that the
limitations which apply to superuser processes inside a
jail are not enforced for processes of the host
system.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>hosted (system, process, user, etc.)</term>
<listitem>
<para>A process, user or other entity, whose access to
resources is restricted by a &os; jail.</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="jails-intro">
<title>Introduction</title>
<para>Since system administration is a difficult and perplexing
task, many powerful tools were developed to make life easier for
the administrator. These tools mostly provide enhancements of
some sort to the way systems are installed, configured and
maintained. Part of the tasks which an administrator is
expected to do is to properly configure the security of a
system, so that it can continue serving its real purpose,
without allowing security violations.</para>
<para>One of the tools which can be used to enhance the security
of a &os; system are <emphasis>jails</emphasis>. Jails were
introduced in &os; 4.X by &a.phk;, but were greatly
improved in &os; 5.X to make them a powerful and flexible
subsystem. Their development still goes on, enhancing their
usefulness, performance, reliability, and security.</para>
<sect2 id="jails-what">
<title>What is a Jail</title>
<para>BSD-like operating systems have had &man.chroot.2; since
the time of 4.2BSD. The &man.chroot.8; utility can be used to
change the root directory of a set of processes, creating a
safe environment, separate from the rest of the system.
Processes created in the chrooted environment can not access
files or resources outside of it. For that reason,
compromising a service running in a chrooted environment
should not allow the attacker to compromise the entire system.
The &man.chroot.8; utility is good for easy tasks which do not
require much flexibility or complex, advanced features. Since
the inception of the chroot concept, however, many ways have
been found to escape from a chrooted environment and, although
they have been fixed in modern versions of the &os; kernel, it
was clear that &man.chroot.2; was not the ideal solution for
securing services. A new subsystem had to be
implemented.</para>
<para>This is one of the main reasons why
<emphasis>jails</emphasis> were developed.</para>
<para>Jails improve on the concept of the traditional
&man.chroot.2; environment in several ways. In a traditional
&man.chroot.2; environment, processes are only limited in the
part of the file system they can access. The rest of the
system resources (like the set of system users, the running
processes, or the networking subsystem) are shared by the
chrooted processes and the processes of the host system.
Jails expand this model by virtualizing not only access to the
file system, but also the set of users, the networking
subsystem of the &os; kernel and a few other things. A more
complete set of fine-grained controls available for tuning the
access of a jailed environment is described in
<xref linkend="jails-tuning"/>.</para>
<para>A jail is characterized by four elements:</para>
<itemizedlist>
<listitem>
<para>A directory subtree — the starting point from
which a jail is entered. Once inside the jail, a process
is not permitted to escape outside of this subtree.
Traditional security issues which plagued the original
&man.chroot.2; design will not affect &os; jails.</para>
</listitem>
<listitem>
<para>A hostname — the hostname which will be used
within the jail. Jails are mainly used for hosting
network services, therefore having a descriptive hostname
for each jail can really help the system
administrator.</para>
</listitem>
<listitem>
<para>An <acronym>IP</acronym> address — this will be
assigned to the jail and cannot be changed in any way
during the jail's life span. The IP address of a jail is
usually an alias address for an existing network
interface, but this is not strictly necessary.</para>
</listitem>
<listitem>
<para>A command — the path name of an executable to
run inside the jail. The path is relative to the
root directory of the jail environment.</para>
</listitem>
</itemizedlist>
<para>Apart from these, jails can have their own set of users
and their own <username>root</username> user. Naturally, the
powers of the <username>root</username> user are limited
within the jail environment and, from the point of view of the
host system, the jail <username>root</username> user is not an
omnipotent user. In addition, the <username>root</username>
user of a jail is not allowed to perform critical operations
to the system outside of the associated &man.jail.8;
environment. More information about capabilities and
restrictions of the <username>root</username> user will be
discussed in
<xref linkend="jails-tuning"/> below.</para>
</sect2>
</sect1>
<sect1 id="jails-build">
<title>Creating and Controlling Jails</title>
<para>Some administrators divide jails into the following two
types: <quote>complete</quote> jails, which resemble a real &os;
system, and <quote>service</quote> jails, dedicated to one
application or service, possibly running with privileges. This
is only a conceptual division and the process of building a jail
is not affected by it. The &man.jail.8; manual page is quite
clear about the procedure for building a jail:</para>
<screen>&prompt.root; <userinput>setenv D <replaceable>/here/is/the/jail</replaceable></userinput>
&prompt.root; <userinput>mkdir -p $D</userinput> <co id="jailpath"/>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make buildworld</userinput> <co id="jailbuildworld"/>
&prompt.root; <userinput>make installworld DESTDIR=$D</userinput> <co id="jailinstallworld"/>
&prompt.root; <userinput>make distribution DESTDIR=$D</userinput> <co id="jaildistrib"/>
&prompt.root; <userinput>mount -t devfs devfs $D/dev</userinput> <co id="jaildevfs"/></screen>
<calloutlist>
<callout arearefs="jailpath">
<para>Selecting a location for a jail is the best starting
point. This is where the jail will physically reside within
the file system of the jail's host. A good choice can be
<filename
class="directory">/usr/jail/<replaceable>jailname</replaceable></filename>,
where <replaceable>jailname</replaceable> is the hostname
identifying the jail. The
<filename class="directory">/usr/</filename> file system
usually has enough space for the jail file system, which for
<quote>complete</quote> jails is, essentially, a replication
of every file present in a default installation of the &os;
base system.</para>
</callout>
<callout arearefs="jailbuildworld">
<para>If you have already rebuilt your userland using
<command>make world</command> or
<command>make buildworld</command>, you can skip this step
and install your existing userland into the new jail.</para>
</callout>
<callout arearefs="jailinstallworld">
<para>This command will populate the directory subtree chosen
as jail's physical location on the file system with the
necessary binaries, libraries, manual pages and so
on.</para>
</callout>
<callout arearefs="jaildistrib">
<para>The <maketarget>distribution</maketarget> target for
<application>make</application> installs every needed
configuration file. In simple words, it installs every
installable file of
<filename class="directory">/usr/src/etc/</filename> to the
<filename class="directory">/etc</filename> directory of the
jail environment:
<filename class="directory">$D/etc/</filename>.</para>
</callout>
<callout arearefs="jaildevfs">
<para>Mounting the &man.devfs.8; file system inside a jail is
not required. On the other hand, any, or almost any
application requires access to at least one device,
depending on the purpose of the given application. It is
very important to control access to devices from inside a
jail, as improper settings could permit an attacker to do
nasty things in the jail. Control over &man.devfs.8; is
managed through rulesets which are described in the
&man.devfs.8; and &man.devfs.conf.5; manual pages.</para>
</callout>
</calloutlist>
<para>Once a jail is installed, it can be started by using the
&man.jail.8; utility. The &man.jail.8; utility takes four
mandatory arguments which are described in the
<xref linkend="jails-what"/>. Other arguments may be specified
too, e.g., to run the jailed process with the credentials of a
specific user. The
<option><replaceable>command</replaceable></option> argument
depends on the type of the jail; for a
<emphasis>virtual system</emphasis>,
<filename>/etc/rc</filename> is a good choice, since it will
replicate the startup sequence of a real &os; system. For a
<emphasis>service</emphasis> jail, it depends on the service or
application that will run within the jail.</para>
<para>Jails are often started at boot time and the &os;
<filename>rc</filename> mechanism provides an easy way to do
this.</para>
<procedure>
<step>
<para>A list of the jails which are enabled to start at boot
time should be added to the &man.rc.conf.5; file:</para>
<programlisting>jail_enable="YES" # Set to NO to disable starting of any jails
jail_list="<replaceable>www</replaceable>" # Space separated list of names of jails</programlisting>
<note>
<para>Jail names in <varname>jail_list</varname> should
contain alphanumeric characters only.</para>
</note>
</step>
<step>
<para>For each jail listed in <varname>jail_list</varname>, a
group of &man.rc.conf.5; settings, which describe the
particular jail, should be added:</para>
<programlisting>jail_<replaceable>www</replaceable>_rootdir="/usr/jail/www" # jail's root directory
jail_<replaceable>www</replaceable>_hostname="<replaceable>www</replaceable>.example.org" # jail's hostname
jail_<replaceable>www</replaceable>_ip="192.168.0.10" # jail's IP address
jail_<replaceable>www</replaceable>_devfs_enable="YES" # mount devfs in the jail
jail_<replaceable>www</replaceable>_devfs_ruleset="<replaceable>www_ruleset</replaceable>" # devfs ruleset to apply to jail</programlisting>
<para>The default startup of jails configured in
&man.rc.conf.5;, will run the <filename>/etc/rc</filename>
script of the jail, which assumes the jail is a complete
virtual system. For service jails, the default startup
command of the jail should be changed, by setting the
<varname>jail_<replaceable>jailname</replaceable>_exec_start</varname>
option appropriately.</para>
<note>
<para>For a full list of available options, please see the
&man.rc.conf.5; manual page.</para>
</note>
</step>
</procedure>
<para>&man.service.8; can be used to
start or stop a jail by hand, if an entry for it exists in
<filename>rc.conf</filename>:</para>
<screen>&prompt.root; <userinput>service jail start <replaceable>www</replaceable></userinput>
&prompt.root; <userinput>service jail stop <replaceable>www</replaceable></userinput></screen>
<para>A clean way to shut down a &man.jail.8; is not available at
the moment. This is because commands normally used to
accomplish a clean system shutdown cannot be used inside a jail.
The best way to shut down a jail is to run the following command
from within the jail itself or using the &man.jexec.8; utility
from outside the jail:</para>
<screen>&prompt.root; <userinput>sh /etc/rc.shutdown</userinput></screen>
<para>More information about this can be found in the &man.jail.8;
manual page.</para>
</sect1>
<sect1 id="jails-tuning">
<title>Fine Tuning and Administration</title>
<para>There are several options which can be set for any jail, and
various ways of combining a host &os; system with jails, to
produce higher level applications. This section
presents:</para>
<itemizedlist>
<listitem>
<para>Some of the options available for tuning the behavior
and security restrictions implemented by a jail
installation.</para>
</listitem>
<listitem>
<para>Some of the high-level applications for jail management,
which are available through the &os; Ports Collection, and
can be used to implement overall jail-based
solutions.</para>
</listitem>
</itemizedlist>
<sect2 id="jails-tuning-utilities">
<title>System Tools for Jail Tuning in &os;</title>
<para>Fine tuning of a jail's configuration is mostly done by
setting &man.sysctl.8; variables. A special subtree of sysctl
exists as a basis for organizing all the relevant options: the
<varname>security.jail.*</varname> hierarchy of &os; kernel
options. Here is a list of the main jail-related sysctls,
complete with their default value. Names should be
self-explanatory, but for more information about them, please
refer to the &man.jail.8; and &man.sysctl.8; manual
pages.</para>
<itemizedlist>
<listitem>
<para><varname>security.jail.set_hostname_allowed:
1</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.socket_unixiproute_only:
1</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.sysvipc_allowed:
0</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.enforce_statfs:
2</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.allow_raw_sockets:
0</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.chflags_allowed:
0</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.jailed: 0</varname></para>
</listitem>
</itemizedlist>
<para>These variables can be used by the system administrator of
the <emphasis>host system</emphasis> to add or remove some of
the limitations imposed by default on the
<username>root</username> user. Note that there are some
limitations which cannot be removed. The
<username>root</username> user is not allowed to mount or
unmount file systems from within a &man.jail.8;. The
<username>root</username> inside a jail may not load or unload
&man.devfs.8; rulesets, set firewall rules, or do many other
administrative tasks which require modifications of in-kernel
data, such as setting the <varname>securelevel</varname> of
the kernel.</para>
<para>The base system of &os; contains a basic set of tools for
viewing information about the active jails, and attaching to a
jail to run administrative commands. The &man.jls.8; and
&man.jexec.8; commands are part of the base &os; system, and
can be used to perform the following simple tasks:</para>
<itemizedlist>
<listitem>
<para>Print a list of active jails and their corresponding
jail identifier (<acronym>JID</acronym>),
<acronym>IP</acronym> address, hostname and path.</para>
</listitem>
<listitem>
<para>Attach to a running jail, from its host system, and
run a command inside the jail or perform administrative
tasks inside the jail itself. This is especially useful
when the <username>root</username> user wants to cleanly
shut down a jail. The &man.jexec.8; utility can also be
used to start a shell in a jail to do administration in
it; for example:</para>
<screen>&prompt.root; <userinput>jexec <replaceable>1</replaceable> tcsh</userinput></screen>
</listitem>
</itemizedlist>
</sect2>
<sect2 id="jails-tuning-admintools">
<title>High-Level Administrative Tools in the &os; Ports
Collection</title>
<para>Among the many third-party utilities for jail
administration, one of the most complete and useful is
<filename role="package">sysutils/jailutils</filename>. It is
a set of small applications that contribute to &man.jail.8;
management. Please refer to its web page for more
information.</para>
</sect2>
</sect1>
<sect1 id="jails-application">
<title>Application of Jails</title>
<sect2 id="jails-service-jails">
<sect2info>
<authorgroup>
<author>
<firstname>Daniel</firstname>
<surname>Gerzo</surname>
<contrib>Contributed by </contrib>
<!-- 15. May 2007 -->
</author>
</authorgroup>
</sect2info>
<title>Service Jails</title>
<para>This section is based upon an idea originally presented by
&a.simon; at <ulink
url="http://simon.nitro.dk/service-jails.html"></ulink>, and
an updated article written by Ken Tom
<email>locals@gmail.com</email>. This section illustrates how
to set up a &os; system that adds an additional layer of
security, using the &man.jail.8; feature. It is also assumed
that the given system is at least running RELENG_6_0 and the
information provided earlier in this chapter has been well
understood.</para>
<sect3 id="jails-service-jails-design">
<title>Design</title>
<para>One of the major problems with jails is the management
of their upgrade process. This tends to be a problem
because every jail has to be rebuilt from scratch whenever
it is updated. This is usually not a problem for a single
jail, since the update process is fairly simple, but can be
quite time consuming and tedious if a lot of jails are
created.</para>
<warning>
<para>This setup requires advanced experience with &os; and
usage of its features. If the presented steps below look
too complicated, it is advised to take a look at a simpler
system such as
<filename role="package">sysutils/ezjail</filename>, which
provides an easier method of administering &os; jails and
is not as sophisticated as this setup.</para>
</warning>
<para>This idea has been presented to resolve such issues by
sharing as much as is possible between jails, in a safe way
— using read-only &man.mount.nullfs.8; mounts, so that
updating will be simpler, and putting single services into
individual jails will become more attractive. Additionally,
it provides a simple way to add or remove jails as well as a
way to upgrade them.</para>
<note>
<para>Examples of services in this context are: an
<acronym>HTTP</acronym> server, a <acronym>DNS</acronym>
server, a <acronym>SMTP</acronym> server, and so
forth.</para>
</note>
<para>The goals of the setup described in this section
are:</para>
<itemizedlist>
<listitem>
<para>Create a simple and easy to understand jail
structure. This implies <emphasis>not</emphasis> having
to run a full installworld on each and every
jail.</para>
</listitem>
<listitem>
<para>Make it easy to add new jails or remove existing
ones.</para>
</listitem>
<listitem>
<para>Make it easy to update or upgrade existing
jails.</para>
</listitem>
<listitem>
<para>Make it possible to run a customized &os;
branch.</para>
</listitem>
<listitem>
<para>Be paranoid about security, reducing as much as
possible the possibility of compromise.</para>
</listitem>
<listitem>
<para>Save space and inodes, as much as possible.</para>
</listitem>
</itemizedlist>
<para>As it has been already mentioned, this design relies
heavily on having a single master template which is
read-only (known as <application>nullfs</application>)
mounted into each jail and one read-write device per jail.
A device can be a separate physical disc, a partition, or a
vnode backed &man.md.4; device. In this example, we will
use read-write <application>nullfs</application>
mounts.</para>
<para>The file system layout is described in the following
list:</para>
<itemizedlist>
<listitem>
<para>Each jail will be mounted under the
<filename class="directory">/home/j</filename>
directory.</para>
</listitem>
<listitem>
<para><filename class="directory">/home/j/mroot</filename>
is the template for each jail and the read-only
partition for all of the jails.</para>
</listitem>
<listitem>
<para>A blank directory will be created for each jail
under the <filename class="directory">/home/j</filename>
directory.</para>
</listitem>
<listitem>
<para>Each jail will have a
<filename class="directory">/s</filename> directory,
that will be linked to the read-write portion of the
system.</para>
</listitem>
<listitem>
<para>Each jail shall have its own read-write system that
is based upon <filename
class="directory">/home/j/skel</filename>.</para>
</listitem>
<listitem>
<para>Each jailspace (read-write portion of each jail)
shall be created in <filename
class="directory">/home/js</filename>.</para>
</listitem>
</itemizedlist>
<note>
<para>This assumes that the jails are based under the
<filename class="directory">/home</filename> partition.
This can, of course, be changed to anything else, but this
change will have to be reflected in each of the examples
below.</para>
</note>
<!-- Insert an image or drawing here to illustrate the example. -->
</sect3>
<sect3 id="jails-service-jails-template">
<title>Creating the Template</title>
<para>This section will describe the steps needed to create
the master template that will be the read-only portion for
the jails to use.</para>
<para>It is always a good idea to update the &os; system to
the latest -RELEASE branch. Check the corresponding
Handbook <ulink
url="&url.books.handbook;/makeworld.html">Chapter</ulink>
to accomplish this task. In the case the update is not
feasible, the buildworld will be required in order to be
able to proceed. Additionally, the
<filename role="package">sysutils/cpdup</filename> package
will be required. We will use the &man.portsnap.8; utility
to download the &os; Ports Collection. The Handbook
<ulink url="&url.books.handbook;/portsnap.html">Portsnap
Chapter</ulink> is always good reading for
newcomers.</para>
<procedure>
<step>
<para>First, create a directory structure for the
read-only file system which will contain the &os;
binaries for our jails, then change directory to the
&os; source tree and install the read-only file system
to the jail template:</para>
<screen>&prompt.root; <userinput>mkdir /home/j /home/j/mroot</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot</userinput></screen>
</step>
<step>
<para>Next, prepare a &os; Ports Collection for the jails
as well as a &os; source tree, which is required for
<application>mergemaster</application>:</para>
<screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
&prompt.root; <userinput>mkdir usr/ports</userinput>
&prompt.root; <userinput>portsnap -p /home/j/mroot/usr/ports fetch extract</userinput>
&prompt.root; <userinput>cpdup /usr/src /home/j/mroot/usr/src</userinput></screen>
</step>
<step>
<para>Create a skeleton for the read-write portion of the
system:</para>
<screen>&prompt.root; <userinput>mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles</userinput>
&prompt.root; <userinput>mv etc /home/j/skel</userinput>
&prompt.root; <userinput>mv usr/local /home/j/skel/usr-local</userinput>
&prompt.root; <userinput>mv tmp /home/j/skel</userinput>
&prompt.root; <userinput>mv var /home/j/skel</userinput>
&prompt.root; <userinput>mv root /home/j/skel</userinput></screen>
</step>
<step>
<para>Use <application>mergemaster</application> to
install missing configuration files. Then get rid of
the extra directories that
<application>mergemaster</application> creates:</para>
<screen>&prompt.root; <userinput>mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i</userinput>
&prompt.root; <userinput>cd /home/j/skel</userinput>
&prompt.root; <userinput>rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev</userinput></screen>
</step>
<step>
<para>Now, symlink the read-write file system to the
read-only file system. Please make sure that the
symlinks are created in the correct
<filename class="directory">s/</filename> locations.
Real directories or the creation of directories in the
wrong locations will cause the installation to
fail.</para>
<screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
&prompt.root; <userinput>mkdir s</userinput>
&prompt.root; <userinput>ln -s s/etc etc</userinput>
&prompt.root; <userinput>ln -s s/home home</userinput>
&prompt.root; <userinput>ln -s s/root root</userinput>
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
&prompt.root; <userinput>ln -s ../../s/distfiles usr/ports/distfiles</userinput>
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
</step>
<step>
<para>As a last step, create a generic
<filename>/home/j/skel/etc/make.conf</filename> with its
contents as shown below:</para>
<programlisting>WRKDIRPREFIX?= /s/portbuild</programlisting>
<para>Having <literal>WRKDIRPREFIX</literal> set up this
way will make it possible to compile &os; ports inside
each jail. Remember that the ports directory is part of
the read-only system. The custom path for
<literal>WRKDIRPREFIX</literal> allows builds to be done
in the read-write portion of every jail.</para>
</step>
</procedure>
</sect3>
<sect3 id="jails-service-jails-creating">
<title>Creating Jails</title>
<para>Now that we have a complete &os; jail template, we can
setup and configure the jails in
<filename>/etc/rc.conf</filename>. This example
demonstrates the creation of 3 jails: <quote>NS</quote>,
<quote>MAIL</quote> and <quote>WWW</quote>.</para>
<procedure>
<step>
<para>Put the following lines into the
<filename>/etc/fstab</filename> file, so that the
read-only template for the jails and the read-write
space will be available in the respective jails:</para>
<programlisting>/home/j/mroot /home/j/ns nullfs ro 0 0
/home/j/mroot /home/j/mail nullfs ro 0 0
/home/j/mroot /home/j/www nullfs ro 0 0
/home/js/ns /home/j/ns/s nullfs rw 0 0
/home/js/mail /home/j/mail/s nullfs rw 0 0
/home/js/www /home/j/www/s nullfs rw 0 0</programlisting>
<note>
<para>Partitions marked with a 0 pass number are not
checked by &man.fsck.8; during boot, and partitions
marked with a 0 dump number are not backed up by
&man.dump.8;. We do not want
<application>fsck</application> to check
<application>nullfs</application> mounts or
<application>dump</application> to back up the
read-only nullfs mounts of the jails. This is why
they are marked with <quote>0 0</quote> in the
last two columns of each <filename>fstab</filename>
entry above.</para>
</note>
</step>
<step>
<para>Configure the jails in
<filename>/etc/rc.conf</filename>:</para>
<programlisting>jail_enable="YES"
jail_set_hostname_allow="NO"
jail_list="ns mail www"
jail_ns_hostname="ns.example.org"
jail_ns_ip="192.168.3.17"
jail_ns_rootdir="/usr/home/j/ns"
jail_ns_devfs_enable="YES"
jail_mail_hostname="mail.example.org"
jail_mail_ip="192.168.3.18"
jail_mail_rootdir="/usr/home/j/mail"
jail_mail_devfs_enable="YES"
jail_www_hostname="www.example.org"
jail_www_ip="62.123.43.14"
jail_www_rootdir="/usr/home/j/www"
jail_www_devfs_enable="YES"</programlisting>
<warning>
<para>The reason why the
<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
variable is set to
<filename class="directory">/usr/home</filename>
instead of
<filename class="directory">/home</filename> is that
the physical path of the
<filename class="directory">/home</filename> directory
on a default &os; installation is
<filename class="directory">/usr/home</filename>. The
<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
variable must <emphasis>not</emphasis> be set to a
path which includes a symbolic link, otherwise the
jails will refuse to start. Use the &man.realpath.1;
utility to determine a value which should be set to
this variable. Please see the &os;-SA-07:01.jail
Security Advisory for more information.</para>
</warning>
</step>
<step>
<para>Create the required mount points for the read-only
file system of each jail:</para>
<screen>&prompt.root; <userinput>mkdir /home/j/ns /home/j/mail /home/j/www</userinput></screen>
</step>
<step>
<para>Install the read-write template into each jail.
Note the use of
<filename role="package">sysutils/cpdup</filename>,
which helps to ensure that a correct copy is done of
each directory:</para>
<!-- keramida: Why is cpdup required here? Doesn't cpio(1)
already include adequate functionality for performing this
job *and* have the advantage of being part of the base
system of FreeBSD? -->
<screen>&prompt.root; <userinput>mkdir /home/js</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/ns</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/mail</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/www</userinput></screen>
</step>
<step>
<para>In this phase, the jails are built and prepared to
run. First, mount the required file systems for each
jail, and then start them using the jail rc
script.</para>
<screen>&prompt.root; <userinput>mount -a</userinput>
&prompt.root; <userinput>service jail start</userinput></screen>
</step>
</procedure>
<para>The jails should be running now. To check if they have
started correctly, use the &man.jls.8; command. Its output
should be similar to the following:</para>
<screen>&prompt.root; <userinput>jls</userinput>
JID IP Address Hostname Path
3 192.168.3.17 ns.example.org /home/j/ns
2 192.168.3.18 mail.example.org /home/j/mail
1 62.123.43.14 www.example.org /home/j/www</screen>
<para>At this point, it should be possible to log onto each
jail, add new users or configure daemons. The
<literal>JID</literal> column indicates the jail
identification number of each running jail. Use the
following command in order to perform administrative tasks
in the jail whose <literal>JID</literal> is 3:</para>
<screen>&prompt.root; <userinput>jexec 3 tcsh</userinput></screen>
</sect3>
<sect3 id="jails-service-jails-upgrading">
<title>Upgrading</title>
<para>In time, there will be a need to upgrade the system to a
newer version of &os;, either because of a security issue,
or because new features have been implemented which are
useful for the existing jails. The design of this setup
provides an easy way to upgrade existing jails.
Additionally, it minimizes their downtime, as the jails will
be brought down only in the very last minute. Also, it
provides a way to roll back to the older versions should any
problems occur.</para>
<procedure>
<step>
<para>The first step is to upgrade the host system in the
usual manner. Then create a new temporary read-only
template in <filename
class="directory">/home/j/mroot2</filename>.</para>
<screen>&prompt.root; <userinput>mkdir /home/j/mroot2</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot2</userinput>
&prompt.root; <userinput>cd /home/j/mroot2</userinput>
&prompt.root; <userinput>cpdup /usr/src usr/src</userinput>
&prompt.root; <userinput>mkdir s</userinput></screen>
<para>The <maketarget>installworld</maketarget> run
creates a few unnecessary directories, which should be
removed:</para>
<screen>&prompt.root; <userinput>chflags -R 0 var</userinput>
&prompt.root; <userinput>rm -R etc var root usr/local tmp</userinput></screen>
</step>
<step>
<para>Recreate the read-write symlinks for the master file
system:</para>
<screen>&prompt.root; <userinput>ln -s s/etc etc</userinput>
&prompt.root; <userinput>ln -s s/root root</userinput>
&prompt.root; <userinput>ln -s s/home home</userinput>
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
</step>
<step>
<para>The right time to stop the jails is now:</para>
<screen>&prompt.root; <userinput>service jail stop</userinput></screen>
</step>
<step>
<para>Unmount the original file systems:</para>
<!-- keramida: Shouldn't we suggest a short script-based
loop here, instead of tediously copying the same commands
multiple times? -->
<screen>&prompt.root; <userinput>umount /home/j/ns/s</userinput>
&prompt.root; <userinput>umount /home/j/ns</userinput>
&prompt.root; <userinput>umount /home/j/mail/s</userinput>
&prompt.root; <userinput>umount /home/j/mail</userinput>
&prompt.root; <userinput>umount /home/j/www/s</userinput>
&prompt.root; <userinput>umount /home/j/www</userinput></screen>
<note>
<para>The read-write systems are attached to the
read-only system
(<filename class="directory">/s</filename>) and must
be unmounted first.</para>
</note>
</step>
<step>
<para>Move the old read-only file system and replace it
with the new one. This will serve as a backup and
archive of the old read-only file system should
something go wrong. The naming convention used here
corresponds to when a new read-only file system has been
created. Move the original &os; Ports Collection over
to the new file system to save some space and
inodes:</para>
<screen>&prompt.root; <userinput>cd /home/j</userinput>
&prompt.root; <userinput>mv mroot mroot.20060601</userinput>
&prompt.root; <userinput>mv mroot2 mroot</userinput>
&prompt.root; <userinput>mv mroot.20060601/usr/ports mroot/usr</userinput></screen>
</step>
<step>
<para>At this point the new read-only template is ready,
so the only remaining task is to remount the file
systems and start the jails:</para>
<screen>&prompt.root; <userinput>mount -a</userinput>
&prompt.root; <userinput>service jail start</userinput></screen>
</step>
</procedure>
<para>Use &man.jls.8; to check if the jails started correctly.
Do not forget to run mergemaster in each jail. The
configuration files will need to be updated as well as the
rc.d scripts.</para>
</sect3>
</sect2>
</sect1>
</chapter>