<?xml version="1.0" encoding="iso-8859-2"?> <!-- The FreeBSD Documentation Project $FreeBSD$ --> <chapter id="GEOM"> <chapterinfo> <authorgroup> <author> <firstname>Tom</firstname> <surname>Rhodes</surname> <contrib>Written by </contrib> </author> </authorgroup> </chapterinfo> <title>GEOM: Modular Disk Transformation Framework</title> <sect1 id="GEOM-synopsis"> <title>Synopsis</title> <indexterm> <primary>GEOM</primary> </indexterm> <indexterm> <primary>GEOM Disk Framework</primary> <see>GEOM</see> </indexterm> <para>This chapter covers the use of disks under the GEOM framework in &os;. This includes the major <acronym role="Redundant Array of Inexpensive Disks">RAID</acronym> control utilities which use the framework for configuration. This chapter will not go into in depth discussion on how GEOM handles or controls I/O, the underlying subsystem, or code. This information is provided through the &man.geom.4; manual page and its various SEE ALSO references. This chapter is also not a definitive guide to <acronym>RAID</acronym> configurations. Only GEOM-supported <acronym>RAID</acronym> classifications will be discussed.</para> <para>After reading this chapter, you will know:</para> <itemizedlist> <listitem> <para>What type of <acronym>RAID</acronym> support is available through GEOM.</para> </listitem> <listitem> <para>How to use the base utilities to configure, maintain, and manipulate the various <acronym>RAID</acronym> levels.</para> </listitem> <listitem> <para>How to mirror, stripe, encrypt, and remotely connect disk devices through GEOM.</para> </listitem> <listitem> <para>How to troubleshoot disks attached to the GEOM framework.</para> </listitem> </itemizedlist> <para>Before reading this chapter, you should:</para> <itemizedlist> <listitem> <para>Understand how &os; treats disk devices (<xref linkend="disks"/>).</para> </listitem> <listitem> <para>Know how to configure and install a new &os; kernel (<xref linkend="kernelconfig"/>).</para> </listitem> </itemizedlist> </sect1> <sect1 id="GEOM-intro"> <title>GEOM Introduction</title> <para>GEOM permits access and control to classes — Master Boot Records, <acronym>BSD</acronym> labels, etc — through the use of providers, or the special files in <filename class="directory">/dev</filename>. Supporting various software <acronym>RAID</acronym> configurations, GEOM will transparently provide access to the operating system and operating system utilities.</para> </sect1> <sect1 id="GEOM-striping"> <sect1info> <authorgroup> <author> <firstname>Tom</firstname> <surname>Rhodes</surname> <contrib>Written by </contrib> </author> <author> <firstname>Murray</firstname> <surname>Stokely</surname> </author> </authorgroup> </sect1info> <title>RAID0 - Striping</title> <indexterm> <primary>GEOM</primary> </indexterm> <indexterm> <primary>Striping</primary> </indexterm> <para>Striping is a method used to combine several disk drives into a single volume. In many cases, this is done through the use of hardware controllers. The GEOM disk subsystem provides software support for <acronym>RAID</acronym>0, also known as disk striping.</para> <para>In a <acronym>RAID</acronym>0 system, data are split up in blocks that get written across all the drives in the array. Instead of having to wait on the system to write 256k to one disk, a <acronym>RAID</acronym>0 system can simultaneously write 64k to each of four different disks, offering superior I/O performance. This performance can be enhanced further by using multiple disk controllers.</para> <para>Each disk in a <acronym>RAID</acronym>0 stripe must be of the same size, since I/O requests are interleaved to read or write to multiple disks in parallel.</para> <mediaobject> <imageobject> <imagedata fileref="geom/striping" align="center"/> </imageobject> <textobject> <phrase>Disk Striping Illustration</phrase> </textobject> </mediaobject> <procedure> <title>Creating a stripe of unformatted ATA disks</title> <step><para>Load the <filename>geom_stripe</filename> module:</para> <screen>&prompt.root; <userinput>kldload geom_stripe.ko</userinput></screen> </step> <step><para>Ensure that a suitable mount point exists. If this volume will become a root partition, then temporarily use another mount point such as <filename class="directory">/mnt</filename>:</para> <screen>&prompt.root; <userinput>mkdir /mnt</userinput></screen> </step> <step><para>Determine the device names for the disks which will be striped, and create the new stripe device. For example, the following command could be used to stripe two unused, unpartitioned <acronym>ATA</acronym> disks: <filename>/dev/ad2</filename> and <filename>/dev/ad3</filename>.</para> <screen>&prompt.root; <userinput>gstripe label -v st0 /dev/ad2 /dev/ad3</userinput></screen> <!-- <para>A message should be returned explaining that meta data has been stored on the devices. XXX: What message? Put it inside the screen output above. --> </step> <step><para>A partition table must be created on the new volume with the following command:</para> <screen>&prompt.root; <userinput>bsdlabel -wB /dev/stripe/st0</userinput></screen> </step> <step><para>This process should have created two other devices in the <filename class="directory">/dev/stripe</filename> directory in addition to the <devicename>st0</devicename> device. Those include <devicename>st0a</devicename> and <devicename>st0c</devicename>. A file system must now be created on the <devicename>st0a</devicename> device using the following <command>newfs</command> command:</para> <screen>&prompt.root; <userinput>newfs -U /dev/stripe/st0a</userinput></screen> <para>Many numbers will glide across the screen, and after a few seconds, the process will be complete. The volume has been created and is ready to be mounted.</para> </step> </procedure> <para>The following command can be used to manually mount a newly created disk stripe:</para> <screen>&prompt.root; <userinput>mount /dev/stripe/st0a /mnt</userinput></screen> <para>To mount this striped file system automatically during the boot process, place the volume information in <filename>/etc/fstab</filename> file:</para> <screen>&prompt.root; <userinput>echo "/dev/stripe/st0a /mnt ufs rw 2 2" \</userinput> <userinput>>> /etc/fstab</userinput></screen> <para>The <filename>geom_stripe</filename> module must also be automatically loaded during system initialization, by adding a line to <filename>/boot/loader.conf</filename>:</para> <screen>&prompt.root; <userinput>echo 'geom_stripe_load="YES"' >> /boot/loader.conf</userinput></screen> </sect1> <sect1 id="GEOM-mirror"> <title>RAID1 - Mirroring</title> <indexterm> <primary>GEOM</primary> </indexterm> <indexterm> <primary>Disk Mirroring</primary> </indexterm> <para>Mirroring is a technology used by many corporations and home users to back up data without interruption. When a mirror exists, it simply means that diskB replicates diskA. Or, perhaps diskC+D replicates diskA+B. Regardless of the disk configuration, the important aspect is that information on one disk or partition is being replicated. Later, that information could be more easily restored, backed up without causing service or access interruption, and even be physically stored in a data safe.</para> <para>To begin, ensure the system has two disk drives of equal size, this exercise assumes they are direct access (&man.da.4;) <acronym>SCSI</acronym> disks.</para> <para>Begin by installing &os; on the first disk with only two partitions. One should be a swap partition, double the <acronym>RAM</acronym> size and all remaining space devoted to the root (<filename class="directory">/</filename>) file system. It is possible to have separate partitions for other mount points; however, this will increase the difficulty level ten fold due to manual alteration of the &man.bsdlabel.8; and &man.fdisk.8; settings.</para> <para>Reboot and wait for the system to fully initialize. Once this process has completed, log in as the <username>root</username> user.</para> <para>Create the <filename>/dev/mirror/gm</filename> device and link it with <filename>/dev/da1</filename>:</para> <screen>&prompt.root; <userinput>gmirror label -vnb round-robin gm0 /dev/da1</userinput></screen> <para>The system should respond with:</para> <screen> Metadata value stored on /dev/da1. Done.</screen> <para>Initialize GEOM, this will load the <filename>/boot/kernel/geom_mirror.ko</filename> kernel module:</para> <screen>&prompt.root; <userinput>gmirror load</userinput></screen> <note> <para>This command should have created the <devicename>gm0</devicename>, device node under the <filename class="directory">/dev/mirror</filename> directory.</para> </note> <para>Install a generic <command>fdisk</command> label and boot code to newly created <devicename>gm0</devicename> device:</para> <screen>&prompt.root; <userinput>fdisk -vBI /dev/mirror/gm0</userinput></screen> <para>Now install generic <command>bsdlabel</command> information:</para> <screen>&prompt.root; <userinput>bsdlabel -wB /dev/mirror/gm0s1</userinput></screen> <note> <para>If multiple slices and partitions exist, the flags for the previous two commands will require alteration. They must match the slice and partition size of the other disk.</para> </note> <para>Use the &man.newfs.8; utility to create a default file system on the <devicename>gm0s1a</devicename> device node:</para> <screen>&prompt.root; <userinput>newfs -U /dev/mirror/gm0s1a</userinput></screen> <para>This should have caused the system to spit out some information and a bunch of numbers. This is good. Examine the screen for any error messages and mount the device to the <filename class="directory">/mnt</filename> mount point:</para> <screen>&prompt.root; <userinput>mount /dev/mirror/gm0s1a /mnt</userinput></screen> <para>Now move all data from the boot disk over to this new file system. This example uses the &man.dump.8; and &man.restore.8; commands; however, &man.dd.1; would also work with this scenario.</para> <screen>&prompt.root; <userinput>dump -L -0 -f- / |(cd /mnt && restore -r -v -f-)</userinput></screen> <para>This must be done for each file system. Simply place the appropriate file system in the correct location when running the aforementioned command.</para> <para>Now edit the replicated <filename>/mnt/etc/fstab</filename> file and remove or comment out the swap file <footnote> <para>It should be noted that commenting out the swap file entry in <filename>fstab</filename> will most likely require you to re-establish a different way of enabling swap space. Please refer to <xref linkend="adding-swap-space"/> for more information.</para> </footnote>. Change the other file system information to use the new disk. See the following example:</para> <programlisting># Device Mountpoint FStype Options Dump Pass# #/dev/da0s2b none swap sw 0 0 /dev/mirror/gm0s1a / ufs rw 1 1</programlisting> <para>Now create a <filename>boot.conf</filename> file on both the current and new root partitions. This file will <quote>help</quote> the system <acronym>BIOS</acronym> boot the correct drive:</para> <screen>&prompt.root; <userinput>echo "1:da(1,a)/boot/loader" > /boot.config</userinput></screen> <screen>&prompt.root; <userinput>echo "1:da(1,a)/boot/loader" > /mnt/boot.config</userinput></screen> <note> <para>We have placed it on both root partitions to ensure proper boot up. If for some reason the system cannot read from the new root partition, a failsafe is available.</para> </note> <para>Now add the following line to the new <filename>/boot/loader.conf</filename>:</para> <screen>&prompt.root; <userinput>echo 'geom_mirror_load="YES"' >> /mnt/boot/loader.conf</userinput></screen> <para>This will instruct &man.loader.8; utility to load the <filename>geom_mirror.ko</filename> module during system initialization.</para> <para>Reboot the system:</para> <screen>&prompt.root; <userinput>shutdown -r now</userinput></screen> <para>If all has gone well, the system should have booted from the <devicename>gm0s1a</devicename> device and a <command>login</command> prompt should be waiting. If something went wrong, see review the forthcoming troubleshooting section. Now add the <devicename>da0</devicename> disk to <devicename>gm0</devicename> device:</para> <screen>&prompt.root; <userinput>gmirror configure -a gm0</userinput> &prompt.root; <userinput>gmirror insert gm0 /dev/da0</userinput></screen> <para>The <option>-a</option> flag tells &man.gmirror.8; to use automatic synchronization; i.e., mirror the disk writes automatically. The manual page explains how to rebuild and replace disks, although it uses <devicename>data</devicename> in place of <devicename>gm0</devicename>.</para> <sect2> <title>Troubleshooting</title> <sect3> <title>System refuses to boot</title> <para>If the system boots up to a prompt similar to:</para> <programlisting>ffs_mountroot: can't find rootvp Root mount failed: 6 mountroot></programlisting> <para>Reboot the machine using the power or reset button. At the boot menu, select option six (6). This will drop the system to a &man.loader.8; prompt. Load the kernel module manually:</para> <screen>OK? <userinput>load geom_mirror.ko</userinput> OK? <userinput>boot</userinput></screen> <para>If this works then for whatever reason the module was not being loaded properly. Place:</para> <programlisting>options GEOM_MIRROR</programlisting> <para>in the kernel configuration file, rebuild and reinstall. That should remedy this issue.</para> </sect3> </sect2> </sect1> <sect1 id="geom-ggate"> <title>GEOM Gate Network Devices</title> <para>GEOM supports the remote use of devices, such as disks, CD-ROMs, files, etc. through the use of the gate utilities. This is similar to <acronym>NFS</acronym>.</para> <para>To begin, an exports file must be created. This file specifies who is permitted to access the exported resources and what level of access they are offered. For example, to export the forth slice on the first <acronym>SCSI</acronym> disk, the following <filename>/etc/gg.exports</filename> is more than adequate:</para> <programlisting>192.168.1.0/24 RW /dev/da0s4d</programlisting> <para>It will allow all hosts inside the private network access the file system on the <devicename>da0s4d</devicename> partition.</para> <para>To export this device, ensure it is not currently mounted, and start the &man.ggated.8; server daemon:</para> <screen>&prompt.root; <userinput>ggated</userinput></screen> <para>Now to <command>mount</command> the device on the client machine, issue the following commands:</para> <screen>&prompt.root; <userinput>ggatec create -o rw 192.168.1.1 /dev/da0s4d</userinput></screen> <screen>ggate0</screen> <screen>&prompt.root; <userinput>mount /dev/ggate0 /mnt</userinput></screen> <para>From here on, the device may be accessed through the <filename class="directory">/mnt</filename> mount point.</para> <note> <para>It should be pointed out that this will fail if the device is currently mounted on either the server machine or any other machine on the network.</para> </note> <para>When the device is no longer needed, it may be safely unmounted with the &man.umount.8; command, similar to any other disk device.</para> </sect1> </chapter>