os/doc
3
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

- Add gjournal-desktop aticle

Browse source 
Submitted by:	Manolis Kiagias <sonicy@otenet.gr>
This commit is contained in:
Gabor Kovesdan 2008-04-28 15:44:35 +00:00
parent c58aeefaa7
commit 0db8aca859
Notes: svn2git 2020-12-08 03:00:23 +00:00 
svn path=/head/; revision=31928
5 changed files with 694 additions and 0 deletions
en_US.ISO8859-1/articles
Makefile
gjournal-desktop
Makefilearticle.sgml
share/images/articles/gjournal-desktop
disklabel1.pngdisklabel2.png

1
en_US.ISO8859-1/articles/Makefile
View file

@ -24,6 +24,7 @@ SUBDIR+= fonts
SUBDIR+= formatting-media
SUBDIR+= freebsd-questions
SUBDIR+= geom-class
SUBDIR+= gjournal-desktop
SUBDIR+= hats
SUBDIR+= hubs
SUBDIR+= ipsec-must

19
en_US.ISO8859-1/articles/gjournal-desktop/Makefile Normal file
View file

@ -0,0 +1,19 @@
#
# $FreeBSD$
#
# Article: Implementing UFS journaling on a desktop PC
DOC?= article
FORMATS?= html
INSTALL_COMPRESSED?=gz
INSTALL_ONLY_COMPRESSED?=
SRCS= article.sgml
IMAGES_EN= disklabel1.png
IMAGES_EN+= disklabel2.png
URL_RELPREFIX?= ../../../..
DOC_PREFIX?= ${.CURDIR}/../../..
.include "${DOC_PREFIX}/share/mk/doc.project.mk"

674
en_US.ISO8859-1/articles/gjournal-desktop/article.sgml Normal file
View file

@ -0,0 +1,674 @@
<!--
Implementing UFS Journaling on a desktop PC
The FreeBSD Documentation Project
-->
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
<!ENTITY % articles.ent PUBLIC "-//FreeBSD//ENTITIES DocBook FreeBSD Articles Entity Set//EN">
%articles.ent;
]>
<article lang="en">
<articleinfo>
<title>Implementing UFS journaling on a desktop PC</title>
<author>
<firstname>Manolis</firstname>
<surname>Kiagias</surname>
<affiliation>
<address><email>sonicy@otenet.gr</email></address>
</affiliation>
</author>
<copyright>
<year>2008</year>
<holder role="mailto:sonicy@otenet.gr">Manolis Kiagias</holder>
</copyright>
<releaseinfo>$FreeBSD$</releaseinfo>
<legalnotice id="trademarks" role="trademarks">
&tm-attrib.freebsd;
&tm-attrib.general;
</legalnotice>
<abstract>
<para>A journaling filesystem uses a log to record all transactions
that take place in the filesystem, and preserves its integrity in the
event of a system crash or power failure. Although it is still
possible to lose unsaved changes to files, journaling almost
completely eliminates the possibility of filesystem corruption caused
by an unclean shutdown. It also shortens to a minimum the time
required for after-failure filesystem checking. Although the UFS
filesystem employed by &os; does not implement journaling itself,
the new journal class of the GEOM framework in &os;&nbsp;7.X can be
used to provide filesystem independent journaling. This article
explains how to implement UFS journaling on a typical desktop PC
scenario.</para>
</abstract>
</articleinfo>
<sect1 id="introduction">
<title>Introduction</title>
<para>While professional servers are usually well protected from
unforeseen shutdowns, the typical desktop is at the mercy of power
failures, accidental resets, and other user related incidents that can
lead to unclean shutdowns. Soft updates usually protect the filesystem
efficiently in such cases, although most of the times a lengthy
background check is required. On rare occasions, filesystem corruption
reaches a point where user intervention is required and data may be
lost.</para>
<para>The new journaling capability provided by GEOM can greatly assist
in such scenarios, by virtually eliminating the time required for
filesystem checking, and ensuring that the filesystem is quickly
restored to a consistent state.</para>
<para>This article describes a procedure for implementing UFS journaling
on a typical desktop PC scenario (one hard disk used for both operating
system and data). It should be followed during a fresh installation of
&os;. The steps are simple enough and do not require overly complex
interaction with the command line.</para>
<para>After reading this article, you will know:</para>
<itemizedlist>
<listitem>
<para>How to reserve space for journaling during a new installation of
&os;.</para>
</listitem>
<listitem>
<para>How to load and enable the <literal>geom_journal</literal>
module (or build support for it in your custom kernel).</para>
</listitem>
<listitem>
<para>How to convert your existing filesystems to utilize journaling,
and what options to use in <filename>/etc/fstab</filename> to mount
them.</para>
</listitem>
<listitem>
<para>How to implement journaling in new (empty) partitions.</para>
</listitem>
<listitem>
<para>How to troubleshoot common problems associated with
journaling.</para>
</listitem>
</itemizedlist>
<para>Before reading this article, you should be able to:</para>
<itemizedlist>
<listitem>
<para>Understand basic &unix; and &os; concepts.</para>
</listitem>
<listitem>
<para>Be familiar with the installation procedure of &os; and the
<application>sysinstall</application> utility.</para>
</listitem>
</itemizedlist>
<warning>
<para>The procedure described here is intended for preparing a new
installation where no actual user data is stored on the disk yet.
While it is possible to modify and extend this procedure for
systems already in production, you should <emphasis>backup</emphasis>
all important data before doing so. Messing around with disks and
partitions at a low level can lead to fatal mistakes and data
loss.</para>
</warning>
</sect1>
<sect1 id="understanding-journaling">
<title>Understanding journaling in &os;</title>
<para>The journaling provided by GEOM in &os;&nbsp;7.X is notfilesystem
specific (unlike for example the ext3 filesystem in &linux;) but is
functioning at the block level. Though this means it can be applied
to different filesystems, for &os;&nbsp;7.0-RELEASE, it can only be used
on UFS2.</para>
<para>This functionality is provided by loading the
<filename>geom_journal.ko</filename> module into the kernel (or
building it into a custom kernel) and using the
<command>gjournal</command> command to configure the filesystems.
In general, you would like to journal large filesystems, like
<filename>/usr</filename>. You will need however (see the following
section) to reserve some free disk space.</para>
<para>When a filesystem is journaled, some disk space is needed to keep
the journal itself. The disk space that holds the actual data is
referred to as the <emphasis>data provider</emphasis>, while the one
that holds the journal is referred to as the
<emphasis>journal provider</emphasis>. The data and journal providers
need to be on different partitions when journaling an existing
(non empty) partition. When journaling a new partition, you have the
option to use a single provider for both data and journal. In any case,
the <command>gjournal</command> command combines both providers to
create the final journaled filesystem. For example:</para>
<itemizedlist>
<listitem>
<para>You wish to journal your <filename>/usr</filename> filesystem,
stored in <filename role="device">/dev/ad0s1f</filename> (which
already contains data).</para>
</listitem>
<listitem>
<para>You reserved some free disk space in a partition in
<filename role="device">/dev/ad0s1g</filename>.</para>
</listitem>
<listitem>
<para>Using <command>gjournal</command>, a new <filename
role="device">/dev/ad0s1f.journal</filename> device is created
where <filename role="device">/dev/ad0s1f</filename> is the data
provider, and <filename role="device">/dev/ad0s1g</filename> is
the journal provider. This new device is then used for all
subsequent file operations.</para>
</listitem>
</itemizedlist>
<para>The amount of disk space you need to reserve for the journal
provider depends on the usage load of the filesystem and not on the
size of the data provider. For example on a typical office desktop,
a 1Gb journal provider for the <filename>/usr</filename> filesystem
will suffice, while a machine that deals with heavy disk I/O (i.e.
video editing) may need more. A kernel panic will occur if the
journal space is exhausted before it has a chance to be
committed.</para>
<para>For more information about journaling, please read the manual
page of &man.gjournal.8;.</para>
</sect1>
<sect1 id="reserve-space">
<title>Steps during the installation of &os;</title>
<sect2>
<title>Reserving space for journaling</title>
<para>A typical desktop machine usually has one hard disk that stores
both the OS and user data. Arguably, the default partitioning scheme
selected by <application>sysinstall</application> is more or less
suitable: A desktop machine does not need a large
<filename>/var</filename> partition, while <filename>/usr</filename>
is allocated the bulk of the disk space, since user data and a lot of
packages are installed into its sub-directories.</para>
<para>The default partitioning (the one obtained by pressing
<keycap>A</keycap> at the disklabel editor) does not leave any
unallocated space. Each partition that will be journaled, requires
another partition for the journal. Since the <filename>/usr</filename>
partition is the largest, it makes sense to shrink this partition
slightly, to obtain the space required for journaling.</para>
<para>In our example, an 80Gb disk is used. The following screenshot
shows the default partitions created by the disklabel editor during
installation:</para>
<mediaobject>
<imageobject>
<imagedata fileref="disklabel1.png">
</imageobject>
</mediaobject>
<para>If this is more or less what you need, it is very easy to adjust
for journaling. Simply use the <keycap>up</keycap> and
<keycap>down</keycap> arrow keys to move the highlight to the
<filename>/usr</filename> partition and press <keycap>D</keycap> to
delete it.</para>
<para>Now, move the highlight to the disk name at the top of the screen
and press <keycap>C</keycap> to create a new partition for
<filename>/usr</filename>. This new partition should be smaller by
1Gb (if you intend to journal <filename>/usr</filename> only), or
2Gb (if you intend to journal both <filename>/usr</filename> and
<filename>/var</filename>). From the pop-up that appears, opt to
create a filesystem, and type <filename>/usr</filename> as the mount
point.</para>
<note>
<para>Should you journal the <filename>/var</filename> partition?
Normally, journaling makes sense on quite large partitions. You may
decide not to journal <filename>/var</filename>, although doing so
on a typical desktop will cause no harm. If the filesystem is
lightly used (quite probable for a desktop) you may wish to
allocate less disk space for its journal.</para>
<para>In our example, we journal both <filename>/usr</filename> and
<filename>/var</filename>. You may of course adjust the procedure
to your own needs.</para>
</note>
<para>To keep things as easy going as possible, we are going to use
<application>sysinstall</application> to create the partitions
required for journaling. However, during installation,
<application>sysinstall</application> insists on asking a mount point
for each partition you create. At this point, you do not have any
mount points for the partitions that will hold the journals, and in
reality you <emphasis>do not even need them</emphasis>. These are
not partitions that we are ever going to mount somewhere.</para>
<para>To avoid these problems with
<application>sysinstall</application>, we are going to create the
journal partitions as swap space. Swap is never mounted, and
<application>sysinstall</application> has no problem creating as many
swap partitions as needed. After the first reboot,
<filename>/etc/fstab</filename> will have to be edited, and the extra
swap space entries removed.</para>
<para>To create the swap, again use the <keycap>up</keycap> and
<keycap>down</keycap> arrow keys to move the highlight to the top of
the disklabel editor screen, so that the disk name itself is
highlighted. Then press <keycap>N</keycap>, enter the desired size
(<replaceable>1024M</replaceable>), and select
<quote>swap space</quote> from the pop-up menu that appears. Repeat
for every journal you wish to create. In our example, we create two
partitions to provide for the journals of <filename>/usr</filename> and
<filename>/var</filename>. The final result is shown in the following
screenshot:</para>
<mediaobject>
<imageobject>
<imagedata fileref="disklabel2.png">
</imageobject>
</mediaobject>
<para>When you have completed creating the partitions, we suggest you
write down the partition names, and mount points, so you can easily
refer to this information during the configuration phase. This will
help alleviate mistakes that may damage your installation. The
following table shows our notes for the sample configuration:</para>
<table pgwide="1">
<title>Partitions and journals</title>
<tgroup cols="3">
<thead>
<row>
<entry>Partition</entry>
<entry>Mount point</entry>
<entry>Journal</entry>
</row>
</thead>
<tbody>
<row>
<entry>ad0s1d</entry>
<entry>/var</entry>
<entry>ad0s1h</entry>
</row>
<row>
<entry>ad0s1f</entry>
<entry>/usr</entry>
<entry>ad0s1g</entry>
</row>
</tbody>
</table>
<para>Continue the installation as you would normally do. We would
however suggest you postpone installation of add-on software
(packages) until you have completely setup journaling.</para>
</sect2>
<sect2 id="first-boot">
<title>Booting for the first time</title>
<para>Your system will come up normally, but you will need to edit
<filename>/etc/fstab</filename> and remove the extra swap partitions
you created for the journals. Normally, the swap partition you will
actually use is the one with the <quote>b</quote> suffix (i.e.
ad0s1b in our example). Remove all other swap space entries and
reboot so that &os; will stop using them.</para>
<para>When the system comes up again, we will be ready to configure
journaling.</para>
</sect2>
<sect1 id="configure-journal">
<title>Setting up journaling</title>
<sect2 id="running-gjournal">
<title>Executing gjournal</title>
<para>Having prepared all the required partitions, it is quite easy
to configure journaling. We will need to switch to single user
mode, so login as root and type:</para>
<screen>&prompt.root; shutdown now</screen>
<para>Press enter to get the default shell. We will need to unmount
the partitions that will be journaled, in our example
<filename>/usr </filename> and <filename>/var</filename>:</para>
<screen>&prompt.root; umount /usr /var</screen>
<para>Load the module required for journaling:</para>
<screen>&prompt.root; gjournal load</screen>
<para>Now, use your notes to determine which partition will be used
for each journal. In our example, <filename>/usr</filename> is
<filename role="device">ad0s1f</filename> and its journal will be
<filename role="device">ad0s1g</filename>, while
<filename>/var</filename> is <filename
role="device">ad0s1d</filename> and will
be journaled to <filename role="device">ad0s1h</filename>.
The following commands are required:</para>
<screen>&prompt.root; gjournal label ad0s1f ad0s1g
GEOM_JOURNAL: Journal 2948326772: ad0s1f contains data.
GEOM_JOURNAL: Journal 2948326772: ad0s1g contains journal.
&prompt.root; gjournal label ad0s1d ad0s1h
GEOM_JOURNAL: Journal 3193218002: ad0s1d contains data.
GEOM_JOURNAL: Journal 3193218002: ad0s1h contains journal.</screen>
<note>
<para>If the last sector of either partition is used,
<command>gjournal</command> will return an error. You will have
to run the command using the <option>-f</option> flag to force an
overwrite, i.e.:</para>
<screen>&prompt.root; gjournal label -f ad0s1d ad0s1h</screen>
<para>Since this is a new installation, it is highly unlikely that
anything will be actually overwritten.</para></note>
<para>At this point, two new devices are created, namely
<filename role="device">ad0s1d.journal</filename> and
<filename role="device">ad0s1f.journal</filename>. These represent
the <filename>/var</filename> and <filename>/usr</filename>
partitions we have to mount. Before mounting, we must however set
the journal flag on them and clear the soft updates flag:</para>
<screen>&prompt.root; tunefs -J enable -n disable ad0s1d.journal
tunefs: gjournal set
tunefs: soft updates cleared
&prompt.root; tunefs -J enable -n disable ad0s1f.journal
tunefs: gjournal set
tunefs: soft updates cleared</screen>
<para>Now, mount the new devices manually at their respective places
(note that we can now use the <quote>async</quote> mount
option):</para>
<screen>&prompt.root; mount -o async /dev/ad0s1d.journal /var
&prompt.root; mount -o async /dev/ad0s1f.journal /usr</screen>
<para>Edit <filename>/etc/fstab</filename> and update the entries for
<filename>/usr</filename> and <filename>/var</filename>:</para>
<programlisting>/dev/ad0s1f.journal /usr ufs rw,async 2 2
/dev/ad0s1d.journal /var ufs rw,async 2 2</programlisting>
<warning>
<para>Make sure the above entries are correct, or you will have
trouble starting up normally after you reboot!</para>
</warning>
<para>Finally, edit <filename>/boot/loader.conf</filename> and add the
following line so the &man.gjournal.8; module is loaded at every
boot:</para>
<programlisting>geom_journal_load="YES"</programlisting>
<para>Congratulations! Your system is now set for journaling. You can
either type <quote>exit</quote> to return to multi-user mode, or
reboot to test your configuration (recommended). During the boot you
will see messages like the following:</para>
<screen>ad0: 76293MB XEC XE800JD-00HBC0 08.02D08 at ata0-master SATA150
GEOM_JOURNAL: Journal 2948326772: ad0s1g contains journal.
GEOM_JOURNAL: Journal 3193218002: ad0s1h contains journal.
GEOM_JOURNAL: Journal 3193218002: ad0s1d contains data.
GEOM_JOURNAL: Journal ad0s1d clean.
GEOM_JOURNAL: Journal 2948326772: ad0s1f contains data.
GEOM_JOURNAL: Journal ad0s1f clean.</screen>
<para>After an unclean shutdown, the messages will vary slightly,
i.e.:</para>
<screen>GEOM_JOURNAL: Journal ad0s1d consistent.</screen>
<para>This usually means that &man.gjournal.8; used the information in
the journal provider to return the filesystem to a consistent
state.</para>
</sect2>
<sect2 id="gjournal-new">
<title>Journaling newly created partitions</title>
<para>While the above procedure is necessary for journaling partitions
that already contain data, journaling an empty partition is somewhat
easier, since both the data and the journal provider can be stored
in the same partition. For example, assume a new disk was installed,
and a new partition <filename role="device">/dev/ads1s1d</filename>
was created. Creating the journal would be as simple as:</para>
<screen>&prompt.root; gjournal label ad1s1d</screen>
<para>The journal size will be 1GB by default. You may adjust it by
using the <option>-s</option> option. The value can be given in
bytes, or appended by <literal>K</literal>, <literal>M</literal> or
<literal>G</literal> to denote Kilobytes, Megabytes or Gigabytes
respectively. Note that <command>gjournal</command> will not allow
you to create unsuitably small journal sizes.</para>
<para>For example, to create a 2GB journal, you could use the following
command:</para>
<screen>&prompt.root; gjournal label -s 2G ad1s1d</screen>
<para>You can then create a filesystem on your new partition, and
enable journaling using the <option>-J</option> option:</para>
<screen>&prompt.root; newfs -J /dev/ad1s1d.journal</screen>
</sect2>
<sect2 id="configure-kernel">
<title>Building journaling into your custom kernel</title>
<para>If you do not wish to load <literal>geom_journal</literal> as a
module, you can build its functions right into your kernel. Edit your
custom kernel configuration file, and make sure it includes these two
lines:</para>
<programlisting>options UFS_GJOURNAL # Note: This is already in GENERIC
options GEOM_JOURNAL # You will have to add this one</programlisting>
<para>Rebuild and reinstall your kernel following the relevant
<ulink url="&url.books.handbook;/kernelconfig.html">instructions in
the &os;&nbsp;Handbook.</ulink></para>
<para>Do not forget to remove the relevant <quote>load</quote> entry
from <filename>/boot/loader.conf</filename> if you have previously
used it.</para>
</sect2>
</sect1>
<sect1 id="troubleshooting-gjournal">
<title>Troubleshooting journaling</title>
<para>The following section covers frequently asked questions regarding
problems related to journaling.</para>
<qandaset>
<qandaentry>
<question id="kernel-panic">
<para>I am getting kernel panics during periods of high disk
activity. How is this related to journaling?</para>
</question>
<answer>
<para>The journal probably fills up before it has a chance to get
committed (flushed) to disk. Keep in mind the size of the
journal depends on the usage load, and not the size of the data
provider. If your disk activity is high, you need a larger
partition for the journal.</para>
</answer>
</qandaentry>
<qandaentry>
<question id="unable-boot">
<para>I made some mistake during configuration, and I cannot boot
normally now. Can this be fixed some way?</para>
</question>
<answer>
<para>You either forgot (or misspelled) the entry in
<filename>/boot/loader.conf</filename>, or there are errors in
your <filename>/etc/fstab</filename> file. These are usually easy
to fix. Press enter to get to the default single user shell. Then
locate the root of the problem:</para>
<screen>&prompt.root; cat /boot/loader.conf</screen>
<para>If the <literal>geom_journal_load</literal> entry is missing
or misspelled, the journaled devices are never created. Load the
module manually, mount all partitions, and continue with
multi-user boot:</para>
<screen>&prompt.root; gjournal load
GEOM_JOURNAL: Journal 2948326772: ad0s1g contains journal.
GEOM_JOURNAL: Journal 3193218002: ad0s1h contains journal.
GEOM_JOURNAL: Journal 3193218002: ad0s1d contains data.
GEOM_JOURNAL: Journal ad0s1d clean.
GEOM_JOURNAL: Journal 2948326772: ad0s1f contains data.
GEOM_JOURNAL: Journal ad0s1f clean.
&prompt.root; mount -a
&prompt.root; exit
(boot continues)</screen>
<para>If, on the other hand, this entry is correct, have a look at
<filename>/etc/fstab</filename>. You will probably find a
misspelled or missing entry. In this case, mount all remaining
partitions by hand and continue with the multi-user boot.</para>
</answer>
</qandaentry>
<qandaentry>
<question id="remove-journaling">
<para>Can I remove journaling and return to my standard filesystem
with soft updates?</para>
</question>
<answer>
<para>Sure. Use the following procedure, which reverses the
changes. The partitions you created for the journal providers
can then be used for other purposes, if you so wish.<para>
<para>Login as root and switch to single user mode:</para>
<screen>&prompt.root; shutdown now</screen>
<para>Synchronize the journals:</para>
<screen>&prompt.root; gjournal sync</screen>
<para>Unmount the journaled partitions:</para>
<screen>&prompt.root; umount /usr /var</screen>
<para>Stop the journaling providers:</para>
<screen>&prompt.root; gjournal stop ad0s1d.journal
&prompt.root; gjournal stop ad0s1f.journal</screen>
<para>Clear journaling metadata from all the devices used:</para>
<screen>&prompt.root; gjournal clear ad0s1d
&prompt.root; gjournal clear ad0s1f
&prompt.root; gjournal clear ad0s1g
&prompt.root; gjournal clear ad0s1h</screen>
<para>Clear the filesystem journaling flag, and restore the soft
updates flag:</para>
<screen>&prompt.root; tunefs -J disable -n enable ad0s1d
tunefs: gjournal cleared
tunefs: soft updates set
&prompt.root; tunefs-J disable -n enable ad0s1f
tunefs: gjournal cleared
tunefs: soft updates set</screen>
<para>Remount the old devices by hand:</para>
<screen>&prompt.root; mount -o rw /dev/ad0s1d /var
&prompt.root; mount -o rw /dev/ad0s1f /usr</screen>
<para>Edit <filename>/etc/fstab</filename> and restore it to its
original state:</para>
<programlisting>/dev/ad0s1f /usr ufs rw 2 2
/dev/ad0s1d /var ufs rw 2 2</programlisting>
<para>Finally, edit <filename>/boot/loader.conf</filename>, remove
the entry that loads the <literal>geom_journal</literal> module
and reboot.</para>
</answer>
</qandaentry>
</qandaset>
</sect1>
<sect1 id="further-reading">
<title>Further reading</title>
<para>Journaling is a fairly new feature of &os;, and as such, it is
not very well documented yet. You may however find the following
additional references useful:</para>
<itemizedlist>
<listitem>
<para>A <ulink url="&url.books.handbook;/geom-gjournal.html">new section on journaling</ulink>
is now part of the &os;&nbsp;Handbook.</para>
</listitem>
<listitem>
<para><ulink url="http://lists.freebsd.org/pipermail/freebsd-current/2006-June/064043.html">This post</ulink>
in &a.current.name; by &man.gjournal.8;'s developer, &a.pjd;.</para>
</listitem>
<listitem>
<para><ulink url="http://lists.freebsd.org/pipermail/freebsd-questions/2008-April/173501.html">This post</ulink>
in &a.questions.name; by Ivan Voras.</para>
</listitem>
<listitem>
<para>The manual pages of &man.gjournal.8; and &man.geom.8;.</para>
</listitem>
</itemizedlist>
</sect1>
</article>
<!--
Local Variables:
mode: sgml
sgml-indent-data: t
sgml-omittag: nil
sgml-always-quote-attributes: t
fill-column: 78
indent-tabs-mode: nil
End:
-->

BIN
share/images/articles/gjournal-desktop/disklabel1.png Normal file
View file

Binary file not shown.
Side by side

After

(image error) Size: 73 KiB

BIN
share/images/articles/gjournal-desktop/disklabel2.png Normal file
View file

Binary file not shown.
Side by side

After

(image error) Size: 78 KiB