1038 lines
39 KiB
XML
1038 lines
39 KiB
XML
<?xml version="1.0" encoding="iso-8859-1"?>
|
|
<!--
|
|
The FreeBSD Documentation Project
|
|
|
|
$FreeBSD$
|
|
-->
|
|
<chapter id="jails">
|
|
<chapterinfo>
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Matteo</firstname>
|
|
<surname>Riondato</surname>
|
|
<contrib>Contributed by </contrib>
|
|
</author>
|
|
</authorgroup>
|
|
</chapterinfo>
|
|
|
|
<title>Jails</title>
|
|
|
|
<indexterm><primary>jails</primary></indexterm>
|
|
|
|
<sect1 id="jails-synopsis">
|
|
<title>Synopsis</title>
|
|
|
|
<para>This chapter will provide an explanation of what &os; jails
|
|
are and how to use them. Jails, sometimes referred to as an
|
|
enhanced replacement of
|
|
<emphasis>chroot environments</emphasis>, are a very powerful
|
|
tool for system administrators, but their basic usage can also
|
|
be useful for advanced users.</para>
|
|
|
|
<important>
|
|
<para>Jails are a powerful tool, but they are not a security
|
|
panacea. It is particularly important to note that while it
|
|
is not possible for a jailed process to break out on its own,
|
|
there are several ways in which an unprivileged user outside
|
|
the jail can cooperate with a privileged user inside the jail
|
|
and thereby obtain elevated privileges in the host
|
|
environment.</para>
|
|
|
|
<para>Most of these attacks can be mitigated by ensuring that
|
|
the jail root is not accessible to unprivileged users in the
|
|
host environment. Regardless, as a general rule, untrusted
|
|
users with privileged access to a jail should not be given
|
|
access to the host environment.</para>
|
|
</important>
|
|
|
|
<para>After reading this chapter, you will know:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>What a jail is, and what purpose it may serve in &os;
|
|
installations.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>How to build, start, and stop a jail.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The basics of jail administration, both from inside
|
|
and outside the jail.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Other sources of useful information about jails are:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>The &man.jail.8; manual page. This is the full
|
|
reference of the <command>jail</command> utility — the
|
|
administrative tool which can be used in &os; to start,
|
|
stop, and control &os; jails.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The mailing lists and their archives. The archives of
|
|
the &a.questions; and other mailing lists hosted by the
|
|
&a.mailman.lists; already contain a wealth of material for
|
|
jails. It should always be engaging to search the archives,
|
|
or post a new question to the &a.questions.name; mailing
|
|
list.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect1>
|
|
|
|
<sect1 id="jails-terms">
|
|
<title>Terms Related to Jails</title>
|
|
|
|
<para>To facilitate better understanding of parts of the &os;
|
|
system related to jails, their internals and the way they
|
|
interact with the rest of &os;, the following terms are used
|
|
further in this chapter:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>&man.chroot.8; (command)</term>
|
|
<listitem>
|
|
<para>Utility, which uses &man.chroot.2; &os; system call to
|
|
change the root directory of a process and all its
|
|
descendants.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>&man.chroot.2; (environment)</term>
|
|
<listitem>
|
|
<para>The environment of processes running in a
|
|
<quote>chroot</quote>. This includes resources such as
|
|
the part of the file system which is visible, user and
|
|
group IDs which are available, network interfaces and
|
|
other IPC mechanisms, etc.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>&man.jail.8; (command)</term>
|
|
<listitem>
|
|
<para>The system administration utility which allows
|
|
launching of processes within a jail environment.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>host (system, process, user, etc.)</term>
|
|
<listitem>
|
|
<para>The controlling system of a jail environment. The
|
|
host system has access to all the hardware resources
|
|
available, and can control processes both outside of and
|
|
inside a jail environment. One of the important
|
|
differences of the host system from a jail is that the
|
|
limitations which apply to superuser processes inside a
|
|
jail are not enforced for processes of the host
|
|
system.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>hosted (system, process, user, etc.)</term>
|
|
<listitem>
|
|
<para>A process, user or other entity, whose access to
|
|
resources is restricted by a &os; jail.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</sect1>
|
|
|
|
<sect1 id="jails-intro">
|
|
<title>Introduction</title>
|
|
|
|
<para>Since system administration is a difficult and perplexing
|
|
task, many powerful tools were developed to make life easier for
|
|
the administrator. These tools mostly provide enhancements of
|
|
some sort to the way systems are installed, configured and
|
|
maintained. Part of the tasks which an administrator is
|
|
expected to do is to properly configure the security of a
|
|
system, so that it can continue serving its real purpose,
|
|
without allowing security violations.</para>
|
|
|
|
<para>One of the tools which can be used to enhance the security
|
|
of a &os; system are <emphasis>jails</emphasis>. Jails were
|
|
introduced in &os; 4.X by &a.phk;, but were greatly
|
|
improved in &os; 5.X to make them a powerful and flexible
|
|
subsystem. Their development still goes on, enhancing their
|
|
usefulness, performance, reliability, and security.</para>
|
|
|
|
<sect2 id="jails-what">
|
|
<title>What is a Jail</title>
|
|
|
|
<para>BSD-like operating systems have had &man.chroot.2; since
|
|
the time of 4.2BSD. The &man.chroot.8; utility can be used to
|
|
change the root directory of a set of processes, creating a
|
|
safe environment, separate from the rest of the system.
|
|
Processes created in the chrooted environment can not access
|
|
files or resources outside of it. For that reason,
|
|
compromising a service running in a chrooted environment
|
|
should not allow the attacker to compromise the entire system.
|
|
The &man.chroot.8; utility is good for easy tasks which do not
|
|
require much flexibility or complex, advanced features. Since
|
|
the inception of the chroot concept, however, many ways have
|
|
been found to escape from a chrooted environment and, although
|
|
they have been fixed in modern versions of the &os; kernel, it
|
|
was clear that &man.chroot.2; was not the ideal solution for
|
|
securing services. A new subsystem had to be
|
|
implemented.</para>
|
|
|
|
<para>This is one of the main reasons why
|
|
<emphasis>jails</emphasis> were developed.</para>
|
|
|
|
<para>Jails improve on the concept of the traditional
|
|
&man.chroot.2; environment in several ways. In a traditional
|
|
&man.chroot.2; environment, processes are only limited in the
|
|
part of the file system they can access. The rest of the
|
|
system resources (like the set of system users, the running
|
|
processes, or the networking subsystem) are shared by the
|
|
chrooted processes and the processes of the host system.
|
|
Jails expand this model by virtualizing not only access to the
|
|
file system, but also the set of users, the networking
|
|
subsystem of the &os; kernel and a few other things. A more
|
|
complete set of fine-grained controls available for tuning the
|
|
access of a jailed environment is described in
|
|
<xref linkend="jails-tuning"/>.</para>
|
|
|
|
<para>A jail is characterized by four elements:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>A directory subtree — the starting point from
|
|
which a jail is entered. Once inside the jail, a process
|
|
is not permitted to escape outside of this subtree.
|
|
Traditional security issues which plagued the original
|
|
&man.chroot.2; design will not affect &os; jails.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A hostname — the hostname which will be used
|
|
within the jail. Jails are mainly used for hosting
|
|
network services, therefore having a descriptive hostname
|
|
for each jail can really help the system
|
|
administrator.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>An <acronym>IP</acronym> address — this will be
|
|
assigned to the jail and cannot be changed in any way
|
|
during the jail's life span. The IP address of a jail is
|
|
usually an alias address for an existing network
|
|
interface, but this is not strictly necessary.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A command — the path name of an executable to
|
|
run inside the jail. The path is relative to the
|
|
root directory of the jail environment.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Apart from these, jails can have their own set of users
|
|
and their own <username>root</username> user. Naturally, the
|
|
powers of the <username>root</username> user are limited
|
|
within the jail environment and, from the point of view of the
|
|
host system, the jail <username>root</username> user is not an
|
|
omnipotent user. In addition, the <username>root</username>
|
|
user of a jail is not allowed to perform critical operations
|
|
to the system outside of the associated &man.jail.8;
|
|
environment. More information about capabilities and
|
|
restrictions of the <username>root</username> user will be
|
|
discussed in
|
|
<xref linkend="jails-tuning"/> below.</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="jails-build">
|
|
<title>Creating and Controlling Jails</title>
|
|
|
|
<para>Some administrators divide jails into the following two
|
|
types: <quote>complete</quote> jails, which resemble a real &os;
|
|
system, and <quote>service</quote> jails, dedicated to one
|
|
application or service, possibly running with privileges. This
|
|
is only a conceptual division and the process of building a jail
|
|
is not affected by it. The &man.jail.8; manual page is quite
|
|
clear about the procedure for building a jail:</para>
|
|
|
|
<screen>&prompt.root; <userinput>setenv D <replaceable>/here/is/the/jail</replaceable></userinput>
|
|
&prompt.root; <userinput>mkdir -p $D</userinput> <co id="jailpath"/>
|
|
&prompt.root; <userinput>cd /usr/src</userinput>
|
|
&prompt.root; <userinput>make buildworld</userinput> <co id="jailbuildworld"/>
|
|
&prompt.root; <userinput>make installworld DESTDIR=$D</userinput> <co id="jailinstallworld"/>
|
|
&prompt.root; <userinput>make distribution DESTDIR=$D</userinput> <co id="jaildistrib"/>
|
|
&prompt.root; <userinput>mount -t devfs devfs $D/dev</userinput> <co id="jaildevfs"/></screen>
|
|
|
|
<calloutlist>
|
|
<callout arearefs="jailpath">
|
|
<para>Selecting a location for a jail is the best starting
|
|
point. This is where the jail will physically reside within
|
|
the file system of the jail's host. A good choice can be
|
|
<filename
|
|
class="directory">/usr/jail/<replaceable>jailname</replaceable></filename>,
|
|
where <replaceable>jailname</replaceable> is the hostname
|
|
identifying the jail. The
|
|
<filename class="directory">/usr/</filename> file system
|
|
usually has enough space for the jail file system, which for
|
|
<quote>complete</quote> jails is, essentially, a replication
|
|
of every file present in a default installation of the &os;
|
|
base system.</para>
|
|
</callout>
|
|
|
|
<callout arearefs="jailbuildworld">
|
|
<para>If you have already rebuilt your userland using
|
|
<command>make world</command> or
|
|
<command>make buildworld</command>, you can skip this step
|
|
and install your existing userland into the new jail.</para>
|
|
</callout>
|
|
|
|
<callout arearefs="jailinstallworld">
|
|
<para>This command will populate the directory subtree chosen
|
|
as jail's physical location on the file system with the
|
|
necessary binaries, libraries, manual pages and so
|
|
on.</para>
|
|
</callout>
|
|
|
|
<callout arearefs="jaildistrib">
|
|
<para>The <maketarget>distribution</maketarget> target for
|
|
<application>make</application> installs every needed
|
|
configuration file. In simple words, it installs every
|
|
installable file of
|
|
<filename class="directory">/usr/src/etc/</filename> to the
|
|
<filename class="directory">/etc</filename> directory of the
|
|
jail environment:
|
|
<filename class="directory">$D/etc/</filename>.</para>
|
|
</callout>
|
|
|
|
<callout arearefs="jaildevfs">
|
|
<para>Mounting the &man.devfs.8; file system inside a jail is
|
|
not required. On the other hand, any, or almost any
|
|
application requires access to at least one device,
|
|
depending on the purpose of the given application. It is
|
|
very important to control access to devices from inside a
|
|
jail, as improper settings could permit an attacker to do
|
|
nasty things in the jail. Control over &man.devfs.8; is
|
|
managed through rulesets which are described in the
|
|
&man.devfs.8; and &man.devfs.conf.5; manual pages.</para>
|
|
</callout>
|
|
</calloutlist>
|
|
|
|
<para>Once a jail is installed, it can be started by using the
|
|
&man.jail.8; utility. The &man.jail.8; utility takes four
|
|
mandatory arguments which are described in the
|
|
<xref linkend="jails-what"/>. Other arguments may be specified
|
|
too, e.g., to run the jailed process with the credentials of a
|
|
specific user. The
|
|
<option><replaceable>command</replaceable></option> argument
|
|
depends on the type of the jail; for a
|
|
<emphasis>virtual system</emphasis>,
|
|
<filename>/etc/rc</filename> is a good choice, since it will
|
|
replicate the startup sequence of a real &os; system. For a
|
|
<emphasis>service</emphasis> jail, it depends on the service or
|
|
application that will run within the jail.</para>
|
|
|
|
<para>Jails are often started at boot time and the &os;
|
|
<filename>rc</filename> mechanism provides an easy way to do
|
|
this.</para>
|
|
|
|
<procedure>
|
|
<step>
|
|
<para>A list of the jails which are enabled to start at boot
|
|
time should be added to the &man.rc.conf.5; file:</para>
|
|
|
|
<programlisting>jail_enable="YES" # Set to NO to disable starting of any jails
|
|
jail_list="<replaceable>www</replaceable>" # Space separated list of names of jails</programlisting>
|
|
|
|
<note>
|
|
<para>Jail names in <varname>jail_list</varname> should
|
|
contain alphanumeric characters only.</para>
|
|
</note>
|
|
</step>
|
|
|
|
<step>
|
|
<para>For each jail listed in <varname>jail_list</varname>, a
|
|
group of &man.rc.conf.5; settings, which describe the
|
|
particular jail, should be added:</para>
|
|
|
|
<programlisting>jail_<replaceable>www</replaceable>_rootdir="/usr/jail/www" # jail's root directory
|
|
jail_<replaceable>www</replaceable>_hostname="<replaceable>www</replaceable>.example.org" # jail's hostname
|
|
jail_<replaceable>www</replaceable>_ip="192.168.0.10" # jail's IP address
|
|
jail_<replaceable>www</replaceable>_devfs_enable="YES" # mount devfs in the jail
|
|
jail_<replaceable>www</replaceable>_devfs_ruleset="<replaceable>www_ruleset</replaceable>" # devfs ruleset to apply to jail</programlisting>
|
|
|
|
<para>The default startup of jails configured in
|
|
&man.rc.conf.5;, will run the <filename>/etc/rc</filename>
|
|
script of the jail, which assumes the jail is a complete
|
|
virtual system. For service jails, the default startup
|
|
command of the jail should be changed, by setting the
|
|
<varname>jail_<replaceable>jailname</replaceable>_exec_start</varname>
|
|
option appropriately.</para>
|
|
|
|
<note>
|
|
<para>For a full list of available options, please see the
|
|
&man.rc.conf.5; manual page.</para>
|
|
</note>
|
|
</step>
|
|
</procedure>
|
|
|
|
<para>&man.service.8; can be used to
|
|
start or stop a jail by hand, if an entry for it exists in
|
|
<filename>rc.conf</filename>:</para>
|
|
|
|
<screen>&prompt.root; <userinput>service jail start <replaceable>www</replaceable></userinput>
|
|
&prompt.root; <userinput>service jail stop <replaceable>www</replaceable></userinput></screen>
|
|
|
|
<para>A clean way to shut down a &man.jail.8; is not available at
|
|
the moment. This is because commands normally used to
|
|
accomplish a clean system shutdown cannot be used inside a jail.
|
|
The best way to shut down a jail is to run the following command
|
|
from within the jail itself or using the &man.jexec.8; utility
|
|
from outside the jail:</para>
|
|
|
|
<screen>&prompt.root; <userinput>sh /etc/rc.shutdown</userinput></screen>
|
|
|
|
<para>More information about this can be found in the &man.jail.8;
|
|
manual page.</para>
|
|
</sect1>
|
|
|
|
<sect1 id="jails-tuning">
|
|
<title>Fine Tuning and Administration</title>
|
|
|
|
<para>There are several options which can be set for any jail, and
|
|
various ways of combining a host &os; system with jails, to
|
|
produce higher level applications. This section
|
|
presents:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Some of the options available for tuning the behavior
|
|
and security restrictions implemented by a jail
|
|
installation.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Some of the high-level applications for jail management,
|
|
which are available through the &os; Ports Collection, and
|
|
can be used to implement overall jail-based
|
|
solutions.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<sect2 id="jails-tuning-utilities">
|
|
<title>System Tools for Jail Tuning in &os;</title>
|
|
|
|
<para>Fine tuning of a jail's configuration is mostly done by
|
|
setting &man.sysctl.8; variables. A special subtree of sysctl
|
|
exists as a basis for organizing all the relevant options: the
|
|
<varname>security.jail.*</varname> hierarchy of &os; kernel
|
|
options. Here is a list of the main jail-related sysctls,
|
|
complete with their default value. Names should be
|
|
self-explanatory, but for more information about them, please
|
|
refer to the &man.jail.8; and &man.sysctl.8; manual
|
|
pages.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><varname>security.jail.set_hostname_allowed:
|
|
1</varname></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><varname>security.jail.socket_unixiproute_only:
|
|
1</varname></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><varname>security.jail.sysvipc_allowed:
|
|
0</varname></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><varname>security.jail.enforce_statfs:
|
|
2</varname></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><varname>security.jail.allow_raw_sockets:
|
|
0</varname></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><varname>security.jail.chflags_allowed:
|
|
0</varname></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><varname>security.jail.jailed: 0</varname></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>These variables can be used by the system administrator of
|
|
the <emphasis>host system</emphasis> to add or remove some of
|
|
the limitations imposed by default on the
|
|
<username>root</username> user. Note that there are some
|
|
limitations which cannot be removed. The
|
|
<username>root</username> user is not allowed to mount or
|
|
unmount file systems from within a &man.jail.8;. The
|
|
<username>root</username> inside a jail may not load or unload
|
|
&man.devfs.8; rulesets, set firewall rules, or do many other
|
|
administrative tasks which require modifications of in-kernel
|
|
data, such as setting the <varname>securelevel</varname> of
|
|
the kernel.</para>
|
|
|
|
<para>The base system of &os; contains a basic set of tools for
|
|
viewing information about the active jails, and attaching to a
|
|
jail to run administrative commands. The &man.jls.8; and
|
|
&man.jexec.8; commands are part of the base &os; system, and
|
|
can be used to perform the following simple tasks:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Print a list of active jails and their corresponding
|
|
jail identifier (<acronym>JID</acronym>),
|
|
<acronym>IP</acronym> address, hostname and path.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Attach to a running jail, from its host system, and
|
|
run a command inside the jail or perform administrative
|
|
tasks inside the jail itself. This is especially useful
|
|
when the <username>root</username> user wants to cleanly
|
|
shut down a jail. The &man.jexec.8; utility can also be
|
|
used to start a shell in a jail to do administration in
|
|
it; for example:</para>
|
|
|
|
<screen>&prompt.root; <userinput>jexec <replaceable>1</replaceable> tcsh</userinput></screen>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect2>
|
|
|
|
<sect2 id="jails-tuning-admintools">
|
|
<title>High-Level Administrative Tools in the &os; Ports
|
|
Collection</title>
|
|
|
|
<para>Among the many third-party utilities for jail
|
|
administration, one of the most complete and useful is
|
|
<filename role="package">sysutils/jailutils</filename>. It is
|
|
a set of small applications that contribute to &man.jail.8;
|
|
management. Please refer to its web page for more
|
|
information.</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="jails-application">
|
|
<title>Application of Jails</title>
|
|
|
|
<sect2 id="jails-service-jails">
|
|
<sect2info>
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Daniel</firstname>
|
|
<surname>Gerzo</surname>
|
|
<contrib>Contributed by </contrib>
|
|
<!-- 15. May 2007 -->
|
|
</author>
|
|
</authorgroup>
|
|
</sect2info>
|
|
|
|
<title>Service Jails</title>
|
|
|
|
<para>This section is based upon an idea originally presented by
|
|
&a.simon; at <ulink
|
|
url="http://simon.nitro.dk/service-jails.html"></ulink>, and
|
|
an updated article written by Ken Tom
|
|
<email>locals@gmail.com</email>. This section illustrates how
|
|
to set up a &os; system that adds an additional layer of
|
|
security, using the &man.jail.8; feature. It is also assumed
|
|
that the given system is at least running RELENG_6_0 and the
|
|
information provided earlier in this chapter has been well
|
|
understood.</para>
|
|
|
|
<sect3 id="jails-service-jails-design">
|
|
<title>Design</title>
|
|
|
|
<para>One of the major problems with jails is the management
|
|
of their upgrade process. This tends to be a problem
|
|
because every jail has to be rebuilt from scratch whenever
|
|
it is updated. This is usually not a problem for a single
|
|
jail, since the update process is fairly simple, but can be
|
|
quite time consuming and tedious if a lot of jails are
|
|
created.</para>
|
|
|
|
<warning>
|
|
<para>This setup requires advanced experience with &os; and
|
|
usage of its features. If the presented steps below look
|
|
too complicated, it is advised to take a look at a simpler
|
|
system such as
|
|
<filename role="package">sysutils/ezjail</filename>, which
|
|
provides an easier method of administering &os; jails and
|
|
is not as sophisticated as this setup.</para>
|
|
</warning>
|
|
|
|
<para>This idea has been presented to resolve such issues by
|
|
sharing as much as is possible between jails, in a safe way
|
|
— using read-only &man.mount.nullfs.8; mounts, so that
|
|
updating will be simpler, and putting single services into
|
|
individual jails will become more attractive. Additionally,
|
|
it provides a simple way to add or remove jails as well as a
|
|
way to upgrade them.</para>
|
|
|
|
<note>
|
|
<para>Examples of services in this context are: an
|
|
<acronym>HTTP</acronym> server, a <acronym>DNS</acronym>
|
|
server, a <acronym>SMTP</acronym> server, and so
|
|
forth.</para>
|
|
</note>
|
|
|
|
<para>The goals of the setup described in this section
|
|
are:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Create a simple and easy to understand jail
|
|
structure. This implies <emphasis>not</emphasis> having
|
|
to run a full installworld on each and every
|
|
jail.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Make it easy to add new jails or remove existing
|
|
ones.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Make it easy to update or upgrade existing
|
|
jails.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Make it possible to run a customized &os;
|
|
branch.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Be paranoid about security, reducing as much as
|
|
possible the possibility of compromise.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Save space and inodes, as much as possible.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>As it has been already mentioned, this design relies
|
|
heavily on having a single master template which is
|
|
read-only (known as <application>nullfs</application>)
|
|
mounted into each jail and one read-write device per jail.
|
|
A device can be a separate physical disc, a partition, or a
|
|
vnode backed &man.md.4; device. In this example, we will
|
|
use read-write <application>nullfs</application>
|
|
mounts.</para>
|
|
|
|
<para>The file system layout is described in the following
|
|
list:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Each jail will be mounted under the
|
|
<filename class="directory">/home/j</filename>
|
|
directory.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename class="directory">/home/j/mroot</filename>
|
|
is the template for each jail and the read-only
|
|
partition for all of the jails.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A blank directory will be created for each jail
|
|
under the <filename class="directory">/home/j</filename>
|
|
directory.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Each jail will have a
|
|
<filename class="directory">/s</filename> directory,
|
|
that will be linked to the read-write portion of the
|
|
system.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Each jail shall have its own read-write system that
|
|
is based upon <filename
|
|
class="directory">/home/j/skel</filename>.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Each jailspace (read-write portion of each jail)
|
|
shall be created in <filename
|
|
class="directory">/home/js</filename>.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<note>
|
|
<para>This assumes that the jails are based under the
|
|
<filename class="directory">/home</filename> partition.
|
|
This can, of course, be changed to anything else, but this
|
|
change will have to be reflected in each of the examples
|
|
below.</para>
|
|
</note>
|
|
<!-- Insert an image or drawing here to illustrate the example. -->
|
|
</sect3>
|
|
|
|
<sect3 id="jails-service-jails-template">
|
|
<title>Creating the Template</title>
|
|
|
|
<para>This section will describe the steps needed to create
|
|
the master template that will be the read-only portion for
|
|
the jails to use.</para>
|
|
|
|
<para>It is always a good idea to update the &os; system to
|
|
the latest -RELEASE branch. Check the corresponding
|
|
Handbook <ulink
|
|
url="&url.books.handbook;/makeworld.html">Chapter</ulink>
|
|
to accomplish this task. In the case the update is not
|
|
feasible, the buildworld will be required in order to be
|
|
able to proceed. Additionally, the
|
|
<filename role="package">sysutils/cpdup</filename> package
|
|
will be required. We will use the &man.portsnap.8; utility
|
|
to download the &os; Ports Collection. The Handbook
|
|
<ulink url="&url.books.handbook;/portsnap.html">Portsnap
|
|
Chapter</ulink> is always good reading for
|
|
newcomers.</para>
|
|
|
|
<procedure>
|
|
<step>
|
|
<para>First, create a directory structure for the
|
|
read-only file system which will contain the &os;
|
|
binaries for our jails, then change directory to the
|
|
&os; source tree and install the read-only file system
|
|
to the jail template:</para>
|
|
|
|
<screen>&prompt.root; <userinput>mkdir /home/j /home/j/mroot</userinput>
|
|
&prompt.root; <userinput>cd /usr/src</userinput>
|
|
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Next, prepare a &os; Ports Collection for the jails
|
|
as well as a &os; source tree, which is required for
|
|
<application>mergemaster</application>:</para>
|
|
|
|
<screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
|
|
&prompt.root; <userinput>mkdir usr/ports</userinput>
|
|
&prompt.root; <userinput>portsnap -p /home/j/mroot/usr/ports fetch extract</userinput>
|
|
&prompt.root; <userinput>cpdup /usr/src /home/j/mroot/usr/src</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Create a skeleton for the read-write portion of the
|
|
system:</para>
|
|
|
|
<screen>&prompt.root; <userinput>mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles</userinput>
|
|
&prompt.root; <userinput>mv etc /home/j/skel</userinput>
|
|
&prompt.root; <userinput>mv usr/local /home/j/skel/usr-local</userinput>
|
|
&prompt.root; <userinput>mv tmp /home/j/skel</userinput>
|
|
&prompt.root; <userinput>mv var /home/j/skel</userinput>
|
|
&prompt.root; <userinput>mv root /home/j/skel</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Use <application>mergemaster</application> to
|
|
install missing configuration files. Then get rid of
|
|
the extra directories that
|
|
<application>mergemaster</application> creates:</para>
|
|
|
|
<screen>&prompt.root; <userinput>mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i</userinput>
|
|
&prompt.root; <userinput>cd /home/j/skel</userinput>
|
|
&prompt.root; <userinput>rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Now, symlink the read-write file system to the
|
|
read-only file system. Please make sure that the
|
|
symlinks are created in the correct
|
|
<filename class="directory">s/</filename> locations.
|
|
Real directories or the creation of directories in the
|
|
wrong locations will cause the installation to
|
|
fail.</para>
|
|
|
|
<screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
|
|
&prompt.root; <userinput>mkdir s</userinput>
|
|
&prompt.root; <userinput>ln -s s/etc etc</userinput>
|
|
&prompt.root; <userinput>ln -s s/home home</userinput>
|
|
&prompt.root; <userinput>ln -s s/root root</userinput>
|
|
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
|
|
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
|
|
&prompt.root; <userinput>ln -s ../../s/distfiles usr/ports/distfiles</userinput>
|
|
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
|
|
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>As a last step, create a generic
|
|
<filename>/home/j/skel/etc/make.conf</filename> with its
|
|
contents as shown below:</para>
|
|
|
|
<programlisting>WRKDIRPREFIX?= /s/portbuild</programlisting>
|
|
|
|
<para>Having <literal>WRKDIRPREFIX</literal> set up this
|
|
way will make it possible to compile &os; ports inside
|
|
each jail. Remember that the ports directory is part of
|
|
the read-only system. The custom path for
|
|
<literal>WRKDIRPREFIX</literal> allows builds to be done
|
|
in the read-write portion of every jail.</para>
|
|
</step>
|
|
</procedure>
|
|
</sect3>
|
|
|
|
<sect3 id="jails-service-jails-creating">
|
|
<title>Creating Jails</title>
|
|
|
|
<para>Now that we have a complete &os; jail template, we can
|
|
setup and configure the jails in
|
|
<filename>/etc/rc.conf</filename>. This example
|
|
demonstrates the creation of 3 jails: <quote>NS</quote>,
|
|
<quote>MAIL</quote> and <quote>WWW</quote>.</para>
|
|
|
|
<procedure>
|
|
<step>
|
|
<para>Put the following lines into the
|
|
<filename>/etc/fstab</filename> file, so that the
|
|
read-only template for the jails and the read-write
|
|
space will be available in the respective jails:</para>
|
|
|
|
<programlisting>/home/j/mroot /home/j/ns nullfs ro 0 0
|
|
/home/j/mroot /home/j/mail nullfs ro 0 0
|
|
/home/j/mroot /home/j/www nullfs ro 0 0
|
|
/home/js/ns /home/j/ns/s nullfs rw 0 0
|
|
/home/js/mail /home/j/mail/s nullfs rw 0 0
|
|
/home/js/www /home/j/www/s nullfs rw 0 0</programlisting>
|
|
|
|
<note>
|
|
<para>Partitions marked with a 0 pass number are not
|
|
checked by &man.fsck.8; during boot, and partitions
|
|
marked with a 0 dump number are not backed up by
|
|
&man.dump.8;. We do not want
|
|
<application>fsck</application> to check
|
|
<application>nullfs</application> mounts or
|
|
<application>dump</application> to back up the
|
|
read-only nullfs mounts of the jails. This is why
|
|
they are marked with <quote>0 0</quote> in the
|
|
last two columns of each <filename>fstab</filename>
|
|
entry above.</para>
|
|
</note>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Configure the jails in
|
|
<filename>/etc/rc.conf</filename>:</para>
|
|
|
|
<programlisting>jail_enable="YES"
|
|
jail_set_hostname_allow="NO"
|
|
jail_list="ns mail www"
|
|
jail_ns_hostname="ns.example.org"
|
|
jail_ns_ip="192.168.3.17"
|
|
jail_ns_rootdir="/usr/home/j/ns"
|
|
jail_ns_devfs_enable="YES"
|
|
jail_mail_hostname="mail.example.org"
|
|
jail_mail_ip="192.168.3.18"
|
|
jail_mail_rootdir="/usr/home/j/mail"
|
|
jail_mail_devfs_enable="YES"
|
|
jail_www_hostname="www.example.org"
|
|
jail_www_ip="62.123.43.14"
|
|
jail_www_rootdir="/usr/home/j/www"
|
|
jail_www_devfs_enable="YES"</programlisting>
|
|
|
|
<warning>
|
|
<para>The reason why the
|
|
<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
|
|
variable is set to
|
|
<filename class="directory">/usr/home</filename>
|
|
instead of
|
|
<filename class="directory">/home</filename> is that
|
|
the physical path of the
|
|
<filename class="directory">/home</filename> directory
|
|
on a default &os; installation is
|
|
<filename class="directory">/usr/home</filename>. The
|
|
<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
|
|
variable must <emphasis>not</emphasis> be set to a
|
|
path which includes a symbolic link, otherwise the
|
|
jails will refuse to start. Use the &man.realpath.1;
|
|
utility to determine a value which should be set to
|
|
this variable. Please see the &os;-SA-07:01.jail
|
|
Security Advisory for more information.</para>
|
|
</warning>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Create the required mount points for the read-only
|
|
file system of each jail:</para>
|
|
|
|
<screen>&prompt.root; <userinput>mkdir /home/j/ns /home/j/mail /home/j/www</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Install the read-write template into each jail.
|
|
Note the use of
|
|
<filename role="package">sysutils/cpdup</filename>,
|
|
which helps to ensure that a correct copy is done of
|
|
each directory:</para>
|
|
<!-- keramida: Why is cpdup required here? Doesn't cpio(1)
|
|
already include adequate functionality for performing this
|
|
job *and* have the advantage of being part of the base
|
|
system of FreeBSD? -->
|
|
|
|
<screen>&prompt.root; <userinput>mkdir /home/js</userinput>
|
|
&prompt.root; <userinput>cpdup /home/j/skel /home/js/ns</userinput>
|
|
&prompt.root; <userinput>cpdup /home/j/skel /home/js/mail</userinput>
|
|
&prompt.root; <userinput>cpdup /home/j/skel /home/js/www</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>In this phase, the jails are built and prepared to
|
|
run. First, mount the required file systems for each
|
|
jail, and then start them using the jail rc
|
|
script.</para>
|
|
|
|
<screen>&prompt.root; <userinput>mount -a</userinput>
|
|
&prompt.root; <userinput>service jail start</userinput></screen>
|
|
</step>
|
|
</procedure>
|
|
|
|
<para>The jails should be running now. To check if they have
|
|
started correctly, use the &man.jls.8; command. Its output
|
|
should be similar to the following:</para>
|
|
|
|
<screen>&prompt.root; <userinput>jls</userinput>
|
|
JID IP Address Hostname Path
|
|
3 192.168.3.17 ns.example.org /home/j/ns
|
|
2 192.168.3.18 mail.example.org /home/j/mail
|
|
1 62.123.43.14 www.example.org /home/j/www</screen>
|
|
|
|
<para>At this point, it should be possible to log onto each
|
|
jail, add new users or configure daemons. The
|
|
<literal>JID</literal> column indicates the jail
|
|
identification number of each running jail. Use the
|
|
following command in order to perform administrative tasks
|
|
in the jail whose <literal>JID</literal> is 3:</para>
|
|
|
|
<screen>&prompt.root; <userinput>jexec 3 tcsh</userinput></screen>
|
|
</sect3>
|
|
|
|
<sect3 id="jails-service-jails-upgrading">
|
|
<title>Upgrading</title>
|
|
|
|
<para>In time, there will be a need to upgrade the system to a
|
|
newer version of &os;, either because of a security issue,
|
|
or because new features have been implemented which are
|
|
useful for the existing jails. The design of this setup
|
|
provides an easy way to upgrade existing jails.
|
|
Additionally, it minimizes their downtime, as the jails will
|
|
be brought down only in the very last minute. Also, it
|
|
provides a way to roll back to the older versions should any
|
|
problems occur.</para>
|
|
|
|
<procedure>
|
|
<step>
|
|
<para>The first step is to upgrade the host system in the
|
|
usual manner. Then create a new temporary read-only
|
|
template in <filename
|
|
class="directory">/home/j/mroot2</filename>.</para>
|
|
|
|
<screen>&prompt.root; <userinput>mkdir /home/j/mroot2</userinput>
|
|
&prompt.root; <userinput>cd /usr/src</userinput>
|
|
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot2</userinput>
|
|
&prompt.root; <userinput>cd /home/j/mroot2</userinput>
|
|
&prompt.root; <userinput>cpdup /usr/src usr/src</userinput>
|
|
&prompt.root; <userinput>mkdir s</userinput></screen>
|
|
|
|
<para>The <maketarget>installworld</maketarget> run
|
|
creates a few unnecessary directories, which should be
|
|
removed:</para>
|
|
|
|
<screen>&prompt.root; <userinput>chflags -R 0 var</userinput>
|
|
&prompt.root; <userinput>rm -R etc var root usr/local tmp</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Recreate the read-write symlinks for the master file
|
|
system:</para>
|
|
|
|
<screen>&prompt.root; <userinput>ln -s s/etc etc</userinput>
|
|
&prompt.root; <userinput>ln -s s/root root</userinput>
|
|
&prompt.root; <userinput>ln -s s/home home</userinput>
|
|
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
|
|
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
|
|
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
|
|
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>The right time to stop the jails is now:</para>
|
|
|
|
<screen>&prompt.root; <userinput>service jail stop</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Unmount the original file systems:</para>
|
|
<!-- keramida: Shouldn't we suggest a short script-based
|
|
loop here, instead of tediously copying the same commands
|
|
multiple times? -->
|
|
|
|
<screen>&prompt.root; <userinput>umount /home/j/ns/s</userinput>
|
|
&prompt.root; <userinput>umount /home/j/ns</userinput>
|
|
&prompt.root; <userinput>umount /home/j/mail/s</userinput>
|
|
&prompt.root; <userinput>umount /home/j/mail</userinput>
|
|
&prompt.root; <userinput>umount /home/j/www/s</userinput>
|
|
&prompt.root; <userinput>umount /home/j/www</userinput></screen>
|
|
|
|
<note>
|
|
<para>The read-write systems are attached to the
|
|
read-only system
|
|
(<filename class="directory">/s</filename>) and must
|
|
be unmounted first.</para>
|
|
</note>
|
|
</step>
|
|
|
|
<step>
|
|
<para>Move the old read-only file system and replace it
|
|
with the new one. This will serve as a backup and
|
|
archive of the old read-only file system should
|
|
something go wrong. The naming convention used here
|
|
corresponds to when a new read-only file system has been
|
|
created. Move the original &os; Ports Collection over
|
|
to the new file system to save some space and
|
|
inodes:</para>
|
|
|
|
<screen>&prompt.root; <userinput>cd /home/j</userinput>
|
|
&prompt.root; <userinput>mv mroot mroot.20060601</userinput>
|
|
&prompt.root; <userinput>mv mroot2 mroot</userinput>
|
|
&prompt.root; <userinput>mv mroot.20060601/usr/ports mroot/usr</userinput></screen>
|
|
</step>
|
|
|
|
<step>
|
|
<para>At this point the new read-only template is ready,
|
|
so the only remaining task is to remount the file
|
|
systems and start the jails:</para>
|
|
|
|
<screen>&prompt.root; <userinput>mount -a</userinput>
|
|
&prompt.root; <userinput>service jail start</userinput></screen>
|
|
</step>
|
|
</procedure>
|
|
|
|
<para>Use &man.jls.8; to check if the jails started correctly.
|
|
Do not forget to run mergemaster in each jail. The
|
|
configuration files will need to be updated as well as the
|
|
rc.d scripts.</para>
|
|
</sect3>
|
|
</sect2>
|
|
</sect1>
|
|
</chapter>
|