Migrate doc to Hugo/AsciiDoctor
I'm very pleased to announce the release of our new website and documentation using the new toolchain with Hugo and AsciiDoctor. To get more information about the new toolchain please read the FreeBSD Documentation Project Primer[1], Hugo docs[2] and AsciiDoctor docs[3]. Acknowledgment: Benedict Reuschling <bcr@> Glen Barber <gjb@> Hiroki Sato <hrs@> Li-Wen Hsu <lwhsu@> Sean Chittenden <seanc@> The FreeBSD Foundation [1] https://docs.FreeBSD.org/en/books/fdp-primer/ [2] https://gohugo.io/documentation/ [3] https://docs.asciidoctor.org/home/ Approved by: doceng, core
This commit is contained in:
Sergio Carlavilla Delgado
parent
0cff342f42
commit
989d921f5d
14375 changed files with 1277937 additions and 3448703 deletions
7
documentation/content/en/books/_index.adoc
Normal file
7
documentation/content/en/books/_index.adoc
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
---
|
||||
title: Books
|
||||
---
|
||||
|
||||
= Books
|
||||
|
||||
{{< list-books-directories >}}
|
||||
100
documentation/content/en/books/arch-handbook/_index.adoc
Normal file
100
documentation/content/en/books/arch-handbook/_index.adoc
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
---
|
||||
title: FreeBSD Architecture Handbook
|
||||
authors:
|
||||
- author: The FreeBSD Documentation Project
|
||||
copyright: Copyright © 2000-2006, 2012-2013 The FreeBSD Documentation Project
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["freebsd", "apple", "microsoft", "unix", "general"]
|
||||
---
|
||||
|
||||
= FreeBSD Architecture Handbook
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 2
|
||||
:icons: font
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:partnums:
|
||||
:chapter-signifier: Chapter
|
||||
:part-signifier: Part
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:book: true
|
||||
:pdf: false
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
:chapters-path: content/en/books/arch-handbook/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
[.abstract-title]
|
||||
Abstract
|
||||
|
||||
Welcome to the FreeBSD Architecture Handbook. This manual is a _work in progress_ and is the work of many individuals. Many sections do not yet exist and some of those that do exist need to be updated. If you are interested in helping with this project, send email to the {freebsd-doc}.
|
||||
|
||||
The latest version of this document is always available from the link:https://www.FreeBSD.org/[FreeBSD World Wide Web server]. It may also be downloaded in a variety of formats and compression options from the https://download.freebsd.org/ftp/doc/[FreeBSD FTP server] or one of the numerous link:{handbook}#mirrors-ftp/[mirror sites].
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
// Section one
|
||||
[[kernel]]
|
||||
= Kernel
|
||||
|
||||
include::{chapters-path}boot/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}locking/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}kobj/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}jail/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}sysinit/chapter.adoc[leveloffset=+1], lines=7..21;32..-1]
|
||||
include::{chapters-path}mac/chapter.adoc[leveloffset=+1, lines=12..26;37..-1]
|
||||
include::{chapters-path}vm/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}smp/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
|
||||
// Section two
|
||||
[[devicedrivers]]
|
||||
= Device Drivers
|
||||
|
||||
include::{chapters-path}driverbasics/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}isa/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}pci/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}scsi/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}usb/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}newbus/chapter.adoc[leveloffset=+1, lines=12..26;37..-1]
|
||||
include::{chapters-path}sound/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
include::{chapters-path}pccard/chapter.adoc[leveloffset=+1, lines=7..21;32..-1]
|
||||
|
||||
// Section three
|
||||
[[appendices]]
|
||||
= Appendices
|
||||
|
||||
include::{chapters-path}bibliography/chapter.adoc[leveloffset=+1, lines=6..19;28..-1]
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
---
|
||||
title: Bibliography
|
||||
prev: books/arch-handbook/partiii
|
||||
---
|
||||
|
||||
[appendix]
|
||||
[[bibliography]]
|
||||
= Bibliography
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums!:
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
[1] _Marshall Kirk McKusick, Keith Bostic, Michael J Karels, and John S Quarterman._ Copyright © 1996 Addison-Wesley Publishing Company, Inc.. 0-201-54979-4. Addison-Wesley Publishing Company, Inc.. The Design and Implementation of the 4.4 BSD Operating System. 1-2.
|
||||
1319
documentation/content/en/books/arch-handbook/boot/chapter.adoc
Normal file
1319
documentation/content/en/books/arch-handbook/boot/chapter.adoc
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,17 @@
|
|||
boot/chapter.adoc
|
||||
locking/chapter.adoc
|
||||
kobj/chapter.adoc
|
||||
jail/chapter.adoc
|
||||
sysinit/chapter.adoc
|
||||
mac/chapter.adoc
|
||||
vm/chapter.adoc
|
||||
smp/chapter.adoc
|
||||
driverbasics/chapter.adoc
|
||||
isa/chapter.adoc
|
||||
pci/chapter.adoc
|
||||
scsi/chapter.adoc
|
||||
usb/chapter.adoc
|
||||
newbus/chapter.adoc
|
||||
sound/chapter.adoc
|
||||
pccard/chapter.adoc
|
||||
bibliography/chapter.adoc
|
||||
|
|
@ -0,0 +1,328 @@
|
|||
---
|
||||
title: Chapter 9. Writing FreeBSD Device Drivers
|
||||
prev: books/arch-handbook/partii
|
||||
next: books/arch-handbook/isa
|
||||
---
|
||||
|
||||
[[driverbasics]]
|
||||
= Writing FreeBSD Device Drivers
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 9
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[driverbasics-intro]]
|
||||
== Introduction
|
||||
|
||||
This chapter provides a brief introduction to writing device drivers for FreeBSD. A device in this context is a term used mostly for hardware-related stuff that belongs to the system, like disks, printers, or a graphics display with its keyboard. A device driver is the software component of the operating system that controls a specific device. There are also so-called pseudo-devices where a device driver emulates the behavior of a device in software without any particular underlying hardware. Device drivers can be compiled into the system statically or loaded on demand through the dynamic kernel linker facility `kld'.
|
||||
|
||||
Most devices in a UNIX(R)-like operating system are accessed through device-nodes, sometimes also called special files. These files are usually located under the directory [.filename]#/dev# in the filesystem hierarchy.
|
||||
|
||||
Device drivers can roughly be broken down into two categories; character and network device drivers.
|
||||
|
||||
[[driverbasics-kld]]
|
||||
== Dynamic Kernel Linker Facility - KLD
|
||||
|
||||
The kld interface allows system administrators to dynamically add and remove functionality from a running system. This allows device driver writers to load their new changes into a running kernel without constantly rebooting to test changes.
|
||||
|
||||
The kld interface is used through:
|
||||
|
||||
* `kldload` - loads a new kernel module
|
||||
* `kldunload` - unloads a kernel module
|
||||
* `kldstat` - lists loaded modules
|
||||
|
||||
Skeleton Layout of a kernel module
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* KLD Skeleton
|
||||
* Inspired by Andrew Reiter's Daemonnews article
|
||||
*/
|
||||
|
||||
#include <sys/types.h>
|
||||
#include <sys/module.h>
|
||||
#include <sys/systm.h> /* uprintf */
|
||||
#include <sys/errno.h>
|
||||
#include <sys/param.h> /* defines used in kernel.h */
|
||||
#include <sys/kernel.h> /* types used in module initialization */
|
||||
|
||||
/*
|
||||
* Load handler that deals with the loading and unloading of a KLD.
|
||||
*/
|
||||
|
||||
static int
|
||||
skel_loader(struct module *m, int what, void *arg)
|
||||
{
|
||||
int err = 0;
|
||||
|
||||
switch (what) {
|
||||
case MOD_LOAD: /* kldload */
|
||||
uprintf("Skeleton KLD loaded.\n");
|
||||
break;
|
||||
case MOD_UNLOAD:
|
||||
uprintf("Skeleton KLD unloaded.\n");
|
||||
break;
|
||||
default:
|
||||
err = EOPNOTSUPP;
|
||||
break;
|
||||
}
|
||||
return(err);
|
||||
}
|
||||
|
||||
/* Declare this module to the rest of the kernel */
|
||||
|
||||
static moduledata_t skel_mod = {
|
||||
"skel",
|
||||
skel_loader,
|
||||
NULL
|
||||
};
|
||||
|
||||
DECLARE_MODULE(skeleton, skel_mod, SI_SUB_KLD, SI_ORDER_ANY);
|
||||
....
|
||||
|
||||
=== Makefile
|
||||
|
||||
FreeBSD provides a system makefile to simplify compiling a kernel module.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
SRCS=skeleton.c
|
||||
KMOD=skeleton
|
||||
|
||||
.include <bsd.kmod.mk>
|
||||
....
|
||||
|
||||
Running `make` with this makefile will create a file [.filename]#skeleton.ko# that can be loaded into the kernel by typing:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload -v ./skeleton.ko
|
||||
....
|
||||
|
||||
[[driverbasics-char]]
|
||||
== Character Devices
|
||||
|
||||
A character device driver is one that transfers data directly to and from a user process. This is the most common type of device driver and there are plenty of simple examples in the source tree.
|
||||
|
||||
This simple example pseudo-device remembers whatever values are written to it and can then echo them back when read.
|
||||
|
||||
.Example of a Sample Echo Pseudo-Device Driver for FreeBSD 10.X - 12.X
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* Simple Echo pseudo-device KLD
|
||||
*
|
||||
* Murray Stokely
|
||||
* Søren (Xride) Straarup
|
||||
* Eitan Adler
|
||||
*/
|
||||
|
||||
#include <sys/types.h>
|
||||
#include <sys/module.h>
|
||||
#include <sys/systm.h> /* uprintf */
|
||||
#include <sys/param.h> /* defines used in kernel.h */
|
||||
#include <sys/kernel.h> /* types used in module initialization */
|
||||
#include <sys/conf.h> /* cdevsw struct */
|
||||
#include <sys/uio.h> /* uio struct */
|
||||
#include <sys/malloc.h>
|
||||
|
||||
#define BUFFERSIZE 255
|
||||
|
||||
/* Function prototypes */
|
||||
static d_open_t echo_open;
|
||||
static d_close_t echo_close;
|
||||
static d_read_t echo_read;
|
||||
static d_write_t echo_write;
|
||||
|
||||
/* Character device entry points */
|
||||
static struct cdevsw echo_cdevsw = {
|
||||
.d_version = D_VERSION,
|
||||
.d_open = echo_open,
|
||||
.d_close = echo_close,
|
||||
.d_read = echo_read,
|
||||
.d_write = echo_write,
|
||||
.d_name = "echo",
|
||||
};
|
||||
|
||||
struct s_echo {
|
||||
char msg[BUFFERSIZE + 1];
|
||||
int len;
|
||||
};
|
||||
|
||||
/* vars */
|
||||
static struct cdev *echo_dev;
|
||||
static struct s_echo *echomsg;
|
||||
|
||||
MALLOC_DECLARE(M_ECHOBUF);
|
||||
MALLOC_DEFINE(M_ECHOBUF, "echobuffer", "buffer for echo module");
|
||||
|
||||
/*
|
||||
* This function is called by the kld[un]load(2) system calls to
|
||||
* determine what actions to take when a module is loaded or unloaded.
|
||||
*/
|
||||
static int
|
||||
echo_loader(struct module *m __unused, int what, void *arg __unused)
|
||||
{
|
||||
int error = 0;
|
||||
|
||||
switch (what) {
|
||||
case MOD_LOAD: /* kldload */
|
||||
error = make_dev_p(MAKEDEV_CHECKNAME | MAKEDEV_WAITOK,
|
||||
&echo_dev,
|
||||
&echo_cdevsw,
|
||||
0,
|
||||
UID_ROOT,
|
||||
GID_WHEEL,
|
||||
0600,
|
||||
"echo");
|
||||
if (error != 0)
|
||||
break;
|
||||
|
||||
echomsg = malloc(sizeof(*echomsg), M_ECHOBUF, M_WAITOK |
|
||||
M_ZERO);
|
||||
printf("Echo device loaded.\n");
|
||||
break;
|
||||
case MOD_UNLOAD:
|
||||
destroy_dev(echo_dev);
|
||||
free(echomsg, M_ECHOBUF);
|
||||
printf("Echo device unloaded.\n");
|
||||
break;
|
||||
default:
|
||||
error = EOPNOTSUPP;
|
||||
break;
|
||||
}
|
||||
return (error);
|
||||
}
|
||||
|
||||
static int
|
||||
echo_open(struct cdev *dev __unused, int oflags __unused, int devtype __unused,
|
||||
struct thread *td __unused)
|
||||
{
|
||||
int error = 0;
|
||||
|
||||
uprintf("Opened device \"echo\" successfully.\n");
|
||||
return (error);
|
||||
}
|
||||
|
||||
static int
|
||||
echo_close(struct cdev *dev __unused, int fflag __unused, int devtype __unused,
|
||||
struct thread *td __unused)
|
||||
{
|
||||
|
||||
uprintf("Closing device \"echo\".\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
/*
|
||||
* The read function just takes the buf that was saved via
|
||||
* echo_write() and returns it to userland for accessing.
|
||||
* uio(9)
|
||||
*/
|
||||
static int
|
||||
echo_read(struct cdev *dev __unused, struct uio *uio, int ioflag __unused)
|
||||
{
|
||||
size_t amt;
|
||||
int error;
|
||||
|
||||
/*
|
||||
* How big is this read operation? Either as big as the user wants,
|
||||
* or as big as the remaining data. Note that the 'len' does not
|
||||
* include the trailing null character.
|
||||
*/
|
||||
amt = MIN(uio->uio_resid, uio->uio_offset >= echomsg->len + 1 ? 0 :
|
||||
echomsg->len + 1 - uio->uio_offset);
|
||||
|
||||
if ((error = uiomove(echomsg->msg, amt, uio)) != 0)
|
||||
uprintf("uiomove failed!\n");
|
||||
|
||||
return (error);
|
||||
}
|
||||
|
||||
/*
|
||||
* echo_write takes in a character string and saves it
|
||||
* to buf for later accessing.
|
||||
*/
|
||||
static int
|
||||
echo_write(struct cdev *dev __unused, struct uio *uio, int ioflag __unused)
|
||||
{
|
||||
size_t amt;
|
||||
int error;
|
||||
|
||||
/*
|
||||
* We either write from the beginning or are appending -- do
|
||||
* not allow random access.
|
||||
*/
|
||||
if (uio->uio_offset != 0 && (uio->uio_offset != echomsg->len))
|
||||
return (EINVAL);
|
||||
|
||||
/* This is a new message, reset length */
|
||||
if (uio->uio_offset == 0)
|
||||
echomsg->len = 0;
|
||||
|
||||
/* Copy the string in from user memory to kernel memory */
|
||||
amt = MIN(uio->uio_resid, (BUFFERSIZE - echomsg->len));
|
||||
|
||||
error = uiomove(echomsg->msg + uio->uio_offset, amt, uio);
|
||||
|
||||
/* Now we need to null terminate and record the length */
|
||||
echomsg->len = uio->uio_offset;
|
||||
echomsg->msg[echomsg->len] = 0;
|
||||
|
||||
if (error != 0)
|
||||
uprintf("Write failed: bad address!\n");
|
||||
return (error);
|
||||
}
|
||||
|
||||
DEV_MODULE(echo, echo_loader, NULL);
|
||||
....
|
||||
====
|
||||
|
||||
With this driver loaded try:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# echo -n "Test Data" > /dev/echo
|
||||
# cat /dev/echo
|
||||
Opened device "echo" successfully.
|
||||
Test Data
|
||||
Closing device "echo".
|
||||
....
|
||||
|
||||
Real hardware devices are described in the next chapter.
|
||||
|
||||
[[driverbasics-block]]
|
||||
== Block Devices (Are Gone)
|
||||
|
||||
Other UNIX(R) systems may support a second type of disk device known as block devices. Block devices are disk devices for which the kernel provides caching. This caching makes block-devices almost unusable, or at least dangerously unreliable. The caching will reorder the sequence of write operations, depriving the application of the ability to know the exact disk contents at any one instant in time.
|
||||
|
||||
This makes predictable and reliable crash recovery of on-disk data structures (filesystems, databases, etc.) impossible. Since writes may be delayed, there is no way the kernel can report to the application which particular write operation encountered a write error, this further compounds the consistency problem.
|
||||
|
||||
For this reason, no serious applications rely on block devices, and in fact, almost all applications which access disks directly take great pains to specify that character (or "raw") devices should always be used. As the implementation of the aliasing of each disk (partition) to two devices with different semantics significantly complicated the relevant kernel code, FreeBSD dropped support for cached disk devices as part of the modernization of the disk I/O infrastructure.
|
||||
|
||||
[[driverbasics-net]]
|
||||
== Network Drivers
|
||||
|
||||
Drivers for network devices do not use device nodes in order to be accessed. Their selection is based on other decisions made inside the kernel and instead of calling open(), use of a network device is generally introduced by using the system call socket(2).
|
||||
|
||||
For more information see ifnet(9), the source of the loopback device, and Bill Paul's network drivers.
|
||||
1103
documentation/content/en/books/arch-handbook/isa/chapter.adoc
Normal file
1103
documentation/content/en/books/arch-handbook/isa/chapter.adoc
Normal file
File diff suppressed because it is too large
Load diff
510
documentation/content/en/books/arch-handbook/jail/chapter.adoc
Normal file
510
documentation/content/en/books/arch-handbook/jail/chapter.adoc
Normal file
|
|
@ -0,0 +1,510 @@
|
|||
---
|
||||
title: Chapter 4. The Jail Subsystem
|
||||
prev: books/arch-handbook/kobj
|
||||
next: books/arch-handbook/sysinit
|
||||
---
|
||||
|
||||
[[jail]]
|
||||
= The Jail Subsystem
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 4
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
On most UNIX(R) systems, `root` has omnipotent power. This promotes insecurity. If an attacker gained `root` on a system, he would have every function at his fingertips. In FreeBSD there are sysctls which dilute the power of `root`, in order to minimize the damage caused by an attacker. Specifically, one of these functions is called `secure levels`. Similarly, another function which is present from FreeBSD 4.0 and onward, is a utility called man:jail[8]. Jail chroots an environment and sets certain restrictions on processes which are forked within the jail. For example, a jailed process cannot affect processes outside the jail, utilize certain system calls, or inflict any damage on the host environment.
|
||||
|
||||
Jail is becoming the new security model. People are running potentially vulnerable servers such as Apache, BIND, and sendmail within jails, so that if an attacker gains `root` within the jail, it is only an annoyance, and not a devastation. This article mainly focuses on the internals (source code) of jail. For information on how to set up a jail see the link:{handbook}#jails/[handbook entry on jails].
|
||||
|
||||
[[jail-arch]]
|
||||
== Architecture
|
||||
|
||||
Jail consists of two realms: the userland program, man:jail[8], and the code implemented within the kernel: the man:jail[2] system call and associated restrictions. I will be discussing the userland program and then how jail is implemented within the kernel.
|
||||
|
||||
=== Userland Code
|
||||
|
||||
The source for the userland jail is located in [.filename]#/usr/src/usr.sbin/jail#, consisting of one file, [.filename]#jail.c#. The program takes these arguments: the path of the jail, hostname, IP address, and the command to be executed.
|
||||
|
||||
==== Data Structures
|
||||
|
||||
In [.filename]#jail.c#, the first thing I would note is the declaration of an important structure `struct jail j;` which was included from [.filename]#/usr/include/sys/jail.h#.
|
||||
|
||||
The definition of the `jail` structure is:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/include/sys/jail.h:
|
||||
|
||||
struct jail {
|
||||
u_int32_t version;
|
||||
char *path;
|
||||
char *hostname;
|
||||
u_int32_t ip_number;
|
||||
};
|
||||
....
|
||||
|
||||
As you can see, there is an entry for each of the arguments passed to the man:jail[8] program, and indeed, they are set during its execution.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/usr.sbin/jail/jail.c
|
||||
char path[PATH_MAX];
|
||||
...
|
||||
if (realpath(argv[0], path) == NULL)
|
||||
err(1, "realpath: %s", argv[0]);
|
||||
if (chdir(path) != 0)
|
||||
err(1, "chdir: %s", path);
|
||||
memset(&j, 0, sizeof(j));
|
||||
j.version = 0;
|
||||
j.path = path;
|
||||
j.hostname = argv[1];
|
||||
....
|
||||
|
||||
==== Networking
|
||||
|
||||
One of the arguments passed to the man:jail[8] program is an IP address with which the jail can be accessed over the network. man:jail[8] translates the IP address given into host byte order and then stores it in `j` (the `jail` structure).
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/usr.sbin/jail/jail.c:
|
||||
struct in_addr in;
|
||||
...
|
||||
if (inet_aton(argv[2], &in) == 0)
|
||||
errx(1, "Could not make sense of ip-number: %s", argv[2]);
|
||||
j.ip_number = ntohl(in.s_addr);
|
||||
....
|
||||
|
||||
The man:inet_aton[3] function "interprets the specified character string as an Internet address, placing the address into the structure provided." The `ip_number` member in the `jail` structure is set only when the IP address placed onto the `in` structure by man:inet_aton[3] is translated into host byte order by man:ntohl[3].
|
||||
|
||||
==== Jailing the Process
|
||||
|
||||
Finally, the userland program jails the process. Jail now becomes an imprisoned process itself and then executes the command given using man:execv[3].
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/usr.sbin/jail/jail.c
|
||||
i = jail(&j);
|
||||
...
|
||||
if (execv(argv[3], argv + 3) != 0)
|
||||
err(1, "execv: %s", argv[3]);
|
||||
....
|
||||
|
||||
As you can see, the `jail()` function is called, and its argument is the `jail` structure which has been filled with the arguments given to the program. Finally, the program you specify is executed. I will now discuss how jail is implemented within the kernel.
|
||||
|
||||
=== Kernel Space
|
||||
|
||||
We will now be looking at the file [.filename]#/usr/src/sys/kern/kern_jail.c#. This is the file where the man:jail[2] system call, appropriate sysctls, and networking functions are defined.
|
||||
|
||||
==== Sysctls
|
||||
|
||||
In [.filename]#kern_jail.c#, the following sysctls are defined:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c:
|
||||
int jail_set_hostname_allowed = 1;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, set_hostname_allowed, CTLFLAG_RW,
|
||||
&jail_set_hostname_allowed, 0,
|
||||
"Processes in jail can set their hostnames");
|
||||
|
||||
int jail_socket_unixiproute_only = 1;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, socket_unixiproute_only, CTLFLAG_RW,
|
||||
&jail_socket_unixiproute_only, 0,
|
||||
"Processes in jail are limited to creating UNIX/IPv4/route sockets only");
|
||||
|
||||
int jail_sysvipc_allowed = 0;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, sysvipc_allowed, CTLFLAG_RW,
|
||||
&jail_sysvipc_allowed, 0,
|
||||
"Processes in jail can use System V IPC primitives");
|
||||
|
||||
static int jail_enforce_statfs = 2;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, enforce_statfs, CTLFLAG_RW,
|
||||
&jail_enforce_statfs, 0,
|
||||
"Processes in jail cannot see all mounted file systems");
|
||||
|
||||
int jail_allow_raw_sockets = 0;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, allow_raw_sockets, CTLFLAG_RW,
|
||||
&jail_allow_raw_sockets, 0,
|
||||
"Prison root can create raw sockets");
|
||||
|
||||
int jail_chflags_allowed = 0;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, chflags_allowed, CTLFLAG_RW,
|
||||
&jail_chflags_allowed, 0,
|
||||
"Processes in jail can alter system file flags");
|
||||
|
||||
int jail_mount_allowed = 0;
|
||||
SYSCTL_INT(_security_jail, OID_AUTO, mount_allowed, CTLFLAG_RW,
|
||||
&jail_mount_allowed, 0,
|
||||
"Processes in jail can mount/unmount jail-friendly file systems");
|
||||
....
|
||||
|
||||
Each of these sysctls can be accessed by the user through the man:sysctl[8] program. Throughout the kernel, these specific sysctls are recognized by their name. For example, the name of the first sysctl is `security.jail.set_hostname_allowed`.
|
||||
|
||||
==== man:jail[2] System Call
|
||||
|
||||
Like all system calls, the man:jail[2] system call takes two arguments, `struct thread *td` and `struct jail_args *uap`. `td` is a pointer to the `thread` structure which describes the calling thread. In this context, `uap` is a pointer to the structure in which a pointer to the `jail` structure passed by the userland [.filename]#jail.c# is contained. When I described the userland program before, you saw that the man:jail[2] system call was given a `jail` structure as its own argument.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c:
|
||||
/*
|
||||
* struct jail_args {
|
||||
* struct jail *jail;
|
||||
* };
|
||||
*/
|
||||
int
|
||||
jail(struct thread *td, struct jail_args *uap)
|
||||
....
|
||||
|
||||
Therefore, `uap->jail` can be used to access the `jail` structure which was passed to the system call. Next, the system call copies the `jail` structure into kernel space using the man:copyin[9] function. man:copyin[9] takes three arguments: the address of the data which is to be copied into kernel space, `uap->jail`, where to store it, `j` and the size of the storage. The `jail` structure pointed by `uap->jail` is copied into kernel space and is stored in another `jail` structure, `j`.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c:
|
||||
error = copyin(uap->jail, &j, sizeof(j));
|
||||
....
|
||||
|
||||
There is another important structure defined in [.filename]#jail.h#. It is the `prison` structure. The `prison` structure is used exclusively within kernel space. Here is the definition of the `prison` structure.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/include/sys/jail.h:
|
||||
struct prison {
|
||||
LIST_ENTRY(prison) pr_list; /* (a) all prisons */
|
||||
int pr_id; /* (c) prison id */
|
||||
int pr_ref; /* (p) refcount */
|
||||
char pr_path[MAXPATHLEN]; /* (c) chroot path */
|
||||
struct vnode *pr_root; /* (c) vnode to rdir */
|
||||
char pr_host[MAXHOSTNAMELEN]; /* (p) jail hostname */
|
||||
u_int32_t pr_ip; /* (c) ip addr host */
|
||||
void *pr_linux; /* (p) linux abi */
|
||||
int pr_securelevel; /* (p) securelevel */
|
||||
struct task pr_task; /* (d) destroy task */
|
||||
struct mtx pr_mtx;
|
||||
void **pr_slots; /* (p) additional data */
|
||||
};
|
||||
....
|
||||
|
||||
The man:jail[2] system call then allocates memory for a `prison` structure and copies data between the `jail` and `prison` structure.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c:
|
||||
MALLOC(pr, struct prison *, sizeof(*pr), M_PRISON, M_WAITOK | M_ZERO);
|
||||
...
|
||||
error = copyinstr(j.path, &pr->pr_path, sizeof(pr->pr_path), 0);
|
||||
if (error)
|
||||
goto e_killmtx;
|
||||
...
|
||||
error = copyinstr(j.hostname, &pr->pr_host, sizeof(pr->pr_host), 0);
|
||||
if (error)
|
||||
goto e_dropvnref;
|
||||
pr->pr_ip = j.ip_number;
|
||||
....
|
||||
|
||||
Next, we will discuss another important system call man:jail_attach[2], which implements the function to put a process into the jail.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c:
|
||||
/*
|
||||
* struct jail_attach_args {
|
||||
* int jid;
|
||||
* };
|
||||
*/
|
||||
int
|
||||
jail_attach(struct thread *td, struct jail_attach_args *uap)
|
||||
....
|
||||
|
||||
This system call makes the changes that can distinguish a jailed process from those unjailed ones. To understand what man:jail_attach[2] does for us, certain background information is needed.
|
||||
|
||||
On FreeBSD, each kernel visible thread is identified by its `thread` structure, while the processes are described by their `proc` structures. You can find the definitions of the `thread` and `proc` structure in [.filename]#/usr/include/sys/proc.h#. For example, the `td` argument in any system call is actually a pointer to the calling thread's `thread` structure, as stated before. The `td_proc` member in the `thread` structure pointed by `td` is a pointer to the `proc` structure which represents the process that contains the thread represented by `td`. The `proc` structure contains members which can describe the owner's identity(`p_ucred`), the process resource limits(`p_limit`), and so on. In the `ucred` structure pointed by `p_ucred` member in the `proc` structure, there is a pointer to the `prison` structure(`cr_prison`).
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/include/sys/proc.h:
|
||||
struct thread {
|
||||
...
|
||||
struct proc *td_proc;
|
||||
...
|
||||
};
|
||||
struct proc {
|
||||
...
|
||||
struct ucred *p_ucred;
|
||||
...
|
||||
};
|
||||
/usr/include/sys/ucred.h
|
||||
struct ucred {
|
||||
...
|
||||
struct prison *cr_prison;
|
||||
...
|
||||
};
|
||||
....
|
||||
|
||||
In [.filename]#kern_jail.c#, the function `jail()` then calls function `jail_attach()` with a given `jid`. And `jail_attach()` calls function `change_root()` to change the root directory of the calling process. The `jail_attach()` then creates a new `ucred` structure, and attaches the newly created `ucred` structure to the calling process after it has successfully attached the `prison` structure to the `ucred` structure. From then on, the calling process is recognized as jailed. When the kernel routine `jailed()` is called in the kernel with the newly created `ucred` structure as its argument, it returns 1 to tell that the credential is connected with a jail. The public ancestor process of all the process forked within the jail, is the process which runs man:jail[8], as it calls the man:jail[2] system call. When a program is executed through man:execve[2], it inherits the jailed property of its parent's `ucred` structure, therefore it has a jailed `ucred` structure.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c
|
||||
int
|
||||
jail(struct thread *td, struct jail_args *uap)
|
||||
{
|
||||
...
|
||||
struct jail_attach_args jaa;
|
||||
...
|
||||
error = jail_attach(td, &jaa);
|
||||
if (error)
|
||||
goto e_dropprref;
|
||||
...
|
||||
}
|
||||
|
||||
int
|
||||
jail_attach(struct thread *td, struct jail_attach_args *uap)
|
||||
{
|
||||
struct proc *p;
|
||||
struct ucred *newcred, *oldcred;
|
||||
struct prison *pr;
|
||||
...
|
||||
p = td->td_proc;
|
||||
...
|
||||
pr = prison_find(uap->jid);
|
||||
...
|
||||
change_root(pr->pr_root, td);
|
||||
...
|
||||
newcred->cr_prison = pr;
|
||||
p->p_ucred = newcred;
|
||||
...
|
||||
}
|
||||
....
|
||||
|
||||
When a process is forked from its parent process, the man:fork[2] system call uses `crhold()` to maintain the credential for the newly forked process. It inherently keep the newly forked child's credential consistent with its parent, so the child process is also jailed.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_fork.c:
|
||||
p2->p_ucred = crhold(td->td_ucred);
|
||||
...
|
||||
td2->td_ucred = crhold(p2->p_ucred);
|
||||
....
|
||||
|
||||
[[jail-restrictions]]
|
||||
== Restrictions
|
||||
|
||||
Throughout the kernel there are access restrictions relating to jailed processes. Usually, these restrictions only check whether the process is jailed, and if so, returns an error. For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
if (jailed(td->td_ucred))
|
||||
return (EPERM);
|
||||
....
|
||||
|
||||
=== SysV IPC
|
||||
|
||||
System V IPC is based on messages. Processes can send each other these messages which tell them how to act. The functions which deal with messages are: man:msgctl[3], man:msgget[3], man:msgsnd[3] and man:msgrcv[3]. Earlier, I mentioned that there were certain sysctls you could turn on or off in order to affect the behavior of jail. One of these sysctls was `security.jail.sysvipc_allowed`. By default, this sysctl is set to 0. If it were set to 1, it would defeat the whole purpose of having a jail; privileged users from the jail would be able to affect processes outside the jailed environment. The difference between a message and a signal is that the message only consists of the signal number.
|
||||
|
||||
[.filename]#/usr/src/sys/kern/sysv_msg.c#:
|
||||
|
||||
* `msgget(key, msgflg)`: `msgget` returns (and possibly creates) a message descriptor that designates a message queue for use in other functions.
|
||||
* `msgctl(msgid, cmd, buf)`: Using this function, a process can query the status of a message descriptor.
|
||||
* `msgsnd(msgid, msgp, msgsz, msgflg)`: `msgsnd` sends a message to a process.
|
||||
* `msgrcv(msgid, msgp, msgsz, msgtyp, msgflg)`: a process receives messages using this function
|
||||
|
||||
In each of the system calls corresponding to these functions, there is this conditional:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/sysv_msg.c:
|
||||
if (!jail_sysvipc_allowed && jailed(td->td_ucred))
|
||||
return (ENOSYS);
|
||||
....
|
||||
|
||||
Semaphore system calls allow processes to synchronize execution by doing a set of operations atomically on a set of semaphores. Basically semaphores provide another way for processes lock resources. However, process waiting on a semaphore, that is being used, will sleep until the resources are relinquished. The following semaphore system calls are blocked inside a jail: man:semget[2], man:semctl[2] and man:semop[2].
|
||||
|
||||
[.filename]#/usr/src/sys/kern/sysv_sem.c#:
|
||||
|
||||
* `semctl(semid, semnum, cmd, ...)`: `semctl` does the specified `cmd` on the semaphore queue indicated by `semid`.
|
||||
* `semget(key, nsems, flag)`: `semget` creates an array of semaphores, corresponding to `key`.
|
||||
+
|
||||
`key and flag take on the same meaning as they do in msgget.`
|
||||
* `semop(semid, array, nops)`: `semop` performs a group of operations indicated by `array`, to the set of semaphores identified by `semid`.
|
||||
|
||||
System V IPC allows for processes to share memory. Processes can communicate directly with each other by sharing parts of their virtual address space and then reading and writing data stored in the shared memory. These system calls are blocked within a jailed environment: man:shmdt[2], man:shmat[2], man:shmctl[2] and man:shmget[2].
|
||||
|
||||
[.filename]#/usr/src/sys/kern/sysv_shm.c#:
|
||||
|
||||
* `shmctl(shmid, cmd, buf)`: `shmctl` does various control operations on the shared memory region identified by `shmid`.
|
||||
* `shmget(key, size, flag)`: `shmget` accesses or creates a shared memory region of `size` bytes.
|
||||
* `shmat(shmid, addr, flag)`: `shmat` attaches a shared memory region identified by `shmid` to the address space of a process.
|
||||
* `shmdt(addr)`: `shmdt` detaches the shared memory region previously attached at `addr`.
|
||||
|
||||
=== Sockets
|
||||
|
||||
Jail treats the man:socket[2] system call and related lower-level socket functions in a special manner. In order to determine whether a certain socket is allowed to be created, it first checks to see if the sysctl `security.jail.socket_unixiproute_only` is set. If set, sockets are only allowed to be created if the family specified is either `PF_LOCAL`, `PF_INET` or `PF_ROUTE`. Otherwise, it returns an error.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/uipc_socket.c:
|
||||
int
|
||||
socreate(int dom, struct socket **aso, int type, int proto,
|
||||
struct ucred *cred, struct thread *td)
|
||||
{
|
||||
struct protosw *prp;
|
||||
...
|
||||
if (jailed(cred) && jail_socket_unixiproute_only &&
|
||||
prp->pr_domain->dom_family != PF_LOCAL &&
|
||||
prp->pr_domain->dom_family != PF_INET &&
|
||||
prp->pr_domain->dom_family != PF_ROUTE) {
|
||||
return (EPROTONOSUPPORT);
|
||||
}
|
||||
...
|
||||
}
|
||||
....
|
||||
|
||||
=== Berkeley Packet Filter
|
||||
|
||||
The Berkeley Packet Filter provides a raw interface to data link layers in a protocol independent fashion. BPF is now controlled by the man:devfs[8] whether it can be used in a jailed environment.
|
||||
|
||||
=== Protocols
|
||||
|
||||
There are certain protocols which are very common, such as TCP, UDP, IP and ICMP. IP and ICMP are on the same level: the network layer 2. There are certain precautions which are taken in order to prevent a jailed process from binding a protocol to a certain address only if the `nam` parameter is set. `nam` is a pointer to a `sockaddr` structure, which describes the address on which to bind the service. A more exact definition is that `sockaddr` "may be used as a template for referring to the identifying tag and length of each address". In the function `in_pcbbind_setup()`, `sin` is a pointer to a `sockaddr_in` structure, which contains the port, address, length and domain family of the socket which is to be bound. Basically, this disallows any processes from jail to be able to specify the address that does not belong to the jail in which the calling process exists.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/netinet/in_pcb.c:
|
||||
int
|
||||
in_pcbbind_setup(struct inpcb *inp, struct sockaddr *nam, in_addr_t *laddrp,
|
||||
u_short *lportp, struct ucred *cred)
|
||||
{
|
||||
...
|
||||
struct sockaddr_in *sin;
|
||||
...
|
||||
if (nam) {
|
||||
sin = (struct sockaddr_in *)nam;
|
||||
...
|
||||
if (sin->sin_addr.s_addr != INADDR_ANY)
|
||||
if (prison_ip(cred, 0, &sin->sin_addr.s_addr))
|
||||
return(EINVAL);
|
||||
...
|
||||
if (lport) {
|
||||
...
|
||||
if (prison && prison_ip(cred, 0, &sin->sin_addr.s_addr))
|
||||
return (EADDRNOTAVAIL);
|
||||
...
|
||||
}
|
||||
}
|
||||
if (lport == 0) {
|
||||
...
|
||||
if (laddr.s_addr != INADDR_ANY)
|
||||
if (prison_ip(cred, 0, &laddr.s_addr))
|
||||
return (EINVAL);
|
||||
...
|
||||
}
|
||||
...
|
||||
if (prison_ip(cred, 0, &laddr.s_addr))
|
||||
return (EINVAL);
|
||||
...
|
||||
}
|
||||
....
|
||||
|
||||
You might be wondering what function `prison_ip()` does. `prison_ip()` is given three arguments, a pointer to the credential(represented by `cred`), any flags, and an IP address. It returns 1 if the IP address does NOT belong to the jail or 0 otherwise. As you can see from the code, if it is indeed an IP address not belonging to the jail, the protocol is not allowed to bind to that address.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/kern/kern_jail.c:
|
||||
int
|
||||
prison_ip(struct ucred *cred, int flag, u_int32_t *ip)
|
||||
{
|
||||
u_int32_t tmp;
|
||||
|
||||
if (!jailed(cred))
|
||||
return (0);
|
||||
if (flag)
|
||||
tmp = *ip;
|
||||
else
|
||||
tmp = ntohl(*ip);
|
||||
if (tmp == INADDR_ANY) {
|
||||
if (flag)
|
||||
*ip = cred->cr_prison->pr_ip;
|
||||
else
|
||||
*ip = htonl(cred->cr_prison->pr_ip);
|
||||
return (0);
|
||||
}
|
||||
if (tmp == INADDR_LOOPBACK) {
|
||||
if (flag)
|
||||
*ip = cred->cr_prison->pr_ip;
|
||||
else
|
||||
*ip = htonl(cred->cr_prison->pr_ip);
|
||||
return (0);
|
||||
}
|
||||
if (cred->cr_prison->pr_ip != tmp)
|
||||
return (1);
|
||||
return (0);
|
||||
}
|
||||
....
|
||||
|
||||
=== Filesystem
|
||||
|
||||
Even `root` users within the jail are not allowed to unset or modify any file flags, such as immutable, append-only, and undeleteable flags, if the securelevel is greater than 0.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/usr/src/sys/ufs/ufs/ufs_vnops.c:
|
||||
static int
|
||||
ufs_setattr(ap)
|
||||
...
|
||||
{
|
||||
...
|
||||
if (!priv_check_cred(cred, PRIV_VFS_SYSFLAGS, 0)) {
|
||||
if (ip->i_flags
|
||||
& (SF_NOUNLINK | SF_IMMUTABLE | SF_APPEND)) {
|
||||
error = securelevel_gt(cred, 0);
|
||||
if (error)
|
||||
return (error);
|
||||
}
|
||||
...
|
||||
}
|
||||
}
|
||||
/usr/src/sys/kern/kern_priv.c
|
||||
int
|
||||
priv_check_cred(struct ucred *cred, int priv, int flags)
|
||||
{
|
||||
...
|
||||
error = prison_priv_check(cred, priv);
|
||||
if (error)
|
||||
return (error);
|
||||
...
|
||||
}
|
||||
/usr/src/sys/kern/kern_jail.c
|
||||
int
|
||||
prison_priv_check(struct ucred *cred, int priv)
|
||||
{
|
||||
...
|
||||
switch (priv) {
|
||||
...
|
||||
case PRIV_VFS_SYSFLAGS:
|
||||
if (jail_chflags_allowed)
|
||||
return (0);
|
||||
else
|
||||
return (EPERM);
|
||||
...
|
||||
}
|
||||
...
|
||||
}
|
||||
....
|
||||
226
documentation/content/en/books/arch-handbook/kobj/chapter.adoc
Normal file
226
documentation/content/en/books/arch-handbook/kobj/chapter.adoc
Normal file
|
|
@ -0,0 +1,226 @@
|
|||
---
|
||||
title: Chapter 3. Kernel Objects
|
||||
prev: books/arch-handbook/locking
|
||||
next: books/arch-handbook/jail
|
||||
---
|
||||
|
||||
[[kernel-objects]]
|
||||
= Kernel Objects
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 3
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
Kernel Objects, or _Kobj_ provides an object-oriented C programming system for the kernel. As such the data being operated on carries the description of how to operate on it. This allows operations to be added and removed from an interface at run time and without breaking binary compatibility.
|
||||
|
||||
[[kernel-objects-term]]
|
||||
== Terminology
|
||||
|
||||
Object::
|
||||
A set of data - data structure - data allocation.
|
||||
|
||||
Method::
|
||||
An operation - function.
|
||||
|
||||
Class::
|
||||
One or more methods.
|
||||
|
||||
Interface::
|
||||
A standard set of one or more methods.
|
||||
|
||||
[[kernel-objects-operation]]
|
||||
== Kobj Operation
|
||||
|
||||
Kobj works by generating descriptions of methods. Each description holds a unique id as well as a default function. The description's address is used to uniquely identify the method within a class' method table.
|
||||
|
||||
A class is built by creating a method table associating one or more functions with method descriptions. Before use the class is compiled. The compilation allocates a cache and associates it with the class. A unique id is assigned to each method description within the method table of the class if not already done so by another referencing class compilation. For every method to be used a function is generated by script to qualify arguments and automatically reference the method description for a lookup. The generated function looks up the method by using the unique id associated with the method description as a hash into the cache associated with the object's class. If the method is not cached the generated function proceeds to use the class' table to find the method. If the method is found then the associated function within the class is used; otherwise, the default function associated with the method description is used.
|
||||
|
||||
These indirections can be visualized as the following:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
object->cache<->class
|
||||
....
|
||||
|
||||
[[kernel-objects-using]]
|
||||
== Using Kobj
|
||||
|
||||
=== Structures
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct kobj_method
|
||||
....
|
||||
|
||||
=== Functions
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
void kobj_class_compile(kobj_class_t cls);
|
||||
void kobj_class_compile_static(kobj_class_t cls, kobj_ops_t ops);
|
||||
void kobj_class_free(kobj_class_t cls);
|
||||
kobj_t kobj_create(kobj_class_t cls, struct malloc_type *mtype, int mflags);
|
||||
void kobj_init(kobj_t obj, kobj_class_t cls);
|
||||
void kobj_delete(kobj_t obj, struct malloc_type *mtype);
|
||||
....
|
||||
|
||||
=== Macros
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
KOBJ_CLASS_FIELDS
|
||||
KOBJ_FIELDS
|
||||
DEFINE_CLASS(name, methods, size)
|
||||
KOBJMETHOD(NAME, FUNC)
|
||||
....
|
||||
|
||||
=== Headers
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
<sys/param.h>
|
||||
<sys/kobj.h>
|
||||
....
|
||||
|
||||
=== Creating an Interface Template
|
||||
|
||||
The first step in using Kobj is to create an Interface. Creating the interface involves creating a template that the script [.filename]#src/sys/kern/makeobjops.pl# can use to generate the header and code for the method declarations and method lookup functions.
|
||||
|
||||
Within this template the following keywords are used: `#include`, `INTERFACE`, `CODE`, `METHOD`, `STATICMETHOD`, and `DEFAULT`.
|
||||
|
||||
The `#include` statement and what follows it is copied verbatim to the head of the generated code file.
|
||||
|
||||
For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#include <sys/foo.h>
|
||||
....
|
||||
|
||||
The `INTERFACE` keyword is used to define the interface name. This name is concatenated with each method name as [interface name]_[method name]. Its syntax is INTERFACE [interface name];.
|
||||
|
||||
For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
INTERFACE foo;
|
||||
....
|
||||
|
||||
The `CODE` keyword copies its arguments verbatim into the code file. Its syntax is `CODE { [whatever] };`
|
||||
|
||||
For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
CODE {
|
||||
struct foo * foo_alloc_null(struct bar *)
|
||||
{
|
||||
return NULL;
|
||||
}
|
||||
};
|
||||
....
|
||||
|
||||
The `METHOD` keyword describes a method. Its syntax is `METHOD [return type] [method name] { [object [, arguments]] };`
|
||||
|
||||
For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
METHOD int bar {
|
||||
struct object *;
|
||||
struct foo *;
|
||||
struct bar;
|
||||
};
|
||||
....
|
||||
|
||||
The `DEFAULT` keyword may follow the `METHOD` keyword. It extends the `METHOD` key word to include the default function for method. The extended syntax is `METHOD [return type] [method name] { [object; [other arguments]] }DEFAULT [default function];`
|
||||
|
||||
For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
METHOD int bar {
|
||||
struct object *;
|
||||
struct foo *;
|
||||
int bar;
|
||||
} DEFAULT foo_hack;
|
||||
....
|
||||
|
||||
The `STATICMETHOD` keyword is used like the `METHOD` keyword except the kobj data is not at the head of the object structure so casting to kobj_t would be incorrect. Instead `STATICMETHOD` relies on the Kobj data being referenced as 'ops'. This is also useful for calling methods directly out of a class's method table.
|
||||
|
||||
Other complete examples:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
src/sys/kern/bus_if.m
|
||||
src/sys/kern/device_if.m
|
||||
....
|
||||
|
||||
=== Creating a Class
|
||||
|
||||
The second step in using Kobj is to create a class. A class consists of a name, a table of methods, and the size of objects if Kobj's object handling facilities are used. To create the class use the macro `DEFINE_CLASS()`. To create the method table create an array of kobj_method_t terminated by a NULL entry. Each non-NULL entry may be created using the macro `KOBJMETHOD()`.
|
||||
|
||||
For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
DEFINE_CLASS(fooclass, foomethods, sizeof(struct foodata));
|
||||
|
||||
kobj_method_t foomethods[] = {
|
||||
KOBJMETHOD(bar_doo, foo_doo),
|
||||
KOBJMETHOD(bar_foo, foo_foo),
|
||||
{ NULL, NULL}
|
||||
};
|
||||
....
|
||||
|
||||
The class must be "compiled". Depending on the state of the system at the time that the class is to be initialized a statically allocated cache, "ops table" have to be used. This can be accomplished by declaring a `struct kobj_ops` and using `kobj_class_compile_static();` otherwise, `kobj_class_compile()` should be used.
|
||||
|
||||
=== Creating an Object
|
||||
|
||||
The third step in using Kobj involves how to define the object. Kobj object creation routines assume that Kobj data is at the head of an object. If this in not appropriate you will have to allocate the object yourself and then use `kobj_init()` on the Kobj portion of it; otherwise, you may use `kobj_create()` to allocate and initialize the Kobj portion of the object automatically. `kobj_init()` may also be used to change the class that an object uses.
|
||||
|
||||
To integrate Kobj into the object you should use the macro KOBJ_FIELDS.
|
||||
|
||||
For example
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct foo_data {
|
||||
KOBJ_FIELDS;
|
||||
foo_foo;
|
||||
foo_bar;
|
||||
};
|
||||
....
|
||||
|
||||
=== Calling Methods
|
||||
|
||||
The last step in using Kobj is to simply use the generated functions to use the desired method within the object's class. This is as simple as using the interface name and the method name with a few modifications. The interface name should be concatenated with the method name using a '_' between them, all in upper case.
|
||||
|
||||
For example, if the interface name was foo and the method was bar then the call would be:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
[return value = ] FOO_BAR(object [, other parameters]);
|
||||
....
|
||||
|
||||
=== Cleaning Up
|
||||
|
||||
When an object allocated through `kobj_create()` is no longer needed `kobj_delete()` may be called on it, and when a class is no longer being used `kobj_class_free()` may be called on it.
|
||||
|
|
@ -0,0 +1,126 @@
|
|||
---
|
||||
title: Chapter 2. Locking Notes
|
||||
prev: books/arch-handbook/boot
|
||||
next: books/arch-handbook/kobj
|
||||
---
|
||||
|
||||
[[locking]]
|
||||
= Locking Notes
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 2
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
_This chapter is maintained by the FreeBSD SMP Next Generation Project._
|
||||
|
||||
This document outlines the locking used in the FreeBSD kernel to permit effective multi-processing within the kernel. Locking can be achieved via several means. Data structures can be protected by mutexes or man:lockmgr[9] locks. A few variables are protected simply by always using atomic operations to access them.
|
||||
|
||||
[[locking-mutexes]]
|
||||
== Mutexes
|
||||
|
||||
A mutex is simply a lock used to guarantee mutual exclusion. Specifically, a mutex may only be owned by one entity at a time. If another entity wishes to obtain a mutex that is already owned, it must wait until the mutex is released. In the FreeBSD kernel, mutexes are owned by processes.
|
||||
|
||||
Mutexes may be recursively acquired, but they are intended to be held for a short period of time. Specifically, one may not sleep while holding a mutex. If you need to hold a lock across a sleep, use a man:lockmgr[9] lock.
|
||||
|
||||
Each mutex has several properties of interest:
|
||||
|
||||
Variable Name::
|
||||
The name of the struct mtx variable in the kernel source.
|
||||
|
||||
Logical Name::
|
||||
The name of the mutex assigned to it by `mtx_init`. This name is displayed in KTR trace messages and witness errors and warnings and is used to distinguish mutexes in the witness code.
|
||||
|
||||
Type::
|
||||
The type of the mutex in terms of the `MTX_*` flags. The meaning for each flag is related to its meaning as documented in man:mutex[9].
|
||||
|
||||
`MTX_DEF`:::
|
||||
A sleep mutex
|
||||
|
||||
`MTX_SPIN`:::
|
||||
A spin mutex
|
||||
|
||||
`MTX_RECURSE`:::
|
||||
This mutex is allowed to recurse.
|
||||
|
||||
Protectees::
|
||||
A list of data structures or data structure members that this entry protects. For data structure members, the name will be in the form of `structure name`.`member name`.
|
||||
|
||||
Dependent Functions::
|
||||
Functions that can only be called if this mutex is held.
|
||||
|
||||
.Mutex List
|
||||
[cols="15%,10%,10%,55%,20%", frame="all", options="header"]
|
||||
|===
|
||||
| Variable Name
|
||||
| Logical Name
|
||||
| Type
|
||||
| Protectees
|
||||
| Dependent Functions
|
||||
|
||||
|sched_lock
|
||||
|"sched lock"
|
||||
|`MTX_SPIN` \| `MTX_RECURSE`
|
||||
|`_gmonparam`, `cnt.v_swtch`, `cp_time`, `curpriority`, `mtx`.`mtx_blocked`, `mtx`.`mtx_contested`, `proc`.`p_procq`, `proc`.`p_slpq`, `proc`.`p_sflag`, `proc`.`p_stat`, `proc`.`p_estcpu`, `proc`.`p_cpticks` `proc`.`p_pctcpu`, `proc`.`p_wchan`, `proc`.`p_wmesg`, `proc`.`p_swtime`, `proc`.`p_slptime`, `proc`.`p_runtime`, `proc`.`p_uu`, `proc`.`p_su`, `proc`.`p_iu`, `proc`.`p_uticks`, `proc`.`p_sticks`, `proc`.`p_iticks`, `proc`.`p_oncpu`, `proc`.`p_lastcpu`, `proc`.`p_rqindex`, `proc`.`p_heldmtx`, `proc`.`p_blocked`, `proc`.`p_mtxname`, `proc`.`p_contested`, `proc`.`p_priority`, `proc`.`p_usrpri`, `proc`.`p_nativepri`, `proc`.`p_nice`, `proc`.`p_rtprio`, `pscnt`, `slpque`, `itqueuebits`, `itqueues`, `rtqueuebits`, `rtqueues`, `queuebits`, `queues`, `idqueuebits`, `idqueues`, `switchtime`, `switchticks`
|
||||
|`setrunqueue`, `remrunqueue`, `mi_switch`, `chooseproc`, `schedclock`, `resetpriority`, `updatepri`, `maybe_resched`, `cpu_switch`, `cpu_throw`, `need_resched`, `resched_wanted`, `clear_resched`, `aston`, `astoff`, `astpending`, `calcru`, `proc_compare`
|
||||
|
||||
|vm86pcb_lock
|
||||
|"vm86pcb lock"
|
||||
|`MTX_DEF`
|
||||
|`vm86pcb`
|
||||
|`vm86_bioscall`
|
||||
|
||||
|Giant
|
||||
|"Giant"
|
||||
|`MTX_DEF` \| `MTX_RECURSE`
|
||||
|nearly everything
|
||||
|lots
|
||||
|
||||
|callout_lock
|
||||
|"callout lock"
|
||||
|`MTX_SPIN` \| `MTX_RECURSE`
|
||||
|`callfree`, `callwheel`, `nextsoftcheck`, `proc`.`p_itcallout`, `proc`.`p_slpcallout`, `softticks`, `ticks`
|
||||
|
|
||||
|===
|
||||
|
||||
[[locking-sx]]
|
||||
== Shared Exclusive Locks
|
||||
|
||||
These locks provide basic reader-writer type functionality and may be held by a sleeping process. Currently they are backed by man:lockmgr[9].
|
||||
|
||||
.Shared Exclusive Lock List
|
||||
[cols="20%,80%", options="header"]
|
||||
|===
|
||||
| Variable Name
|
||||
| Protectees
|
||||
|
||||
|`allproc_lock`
|
||||
|`allproc` `zombproc` `pidhashtbl` `proc`.`p_list` `proc`.`p_hash` `nextpid`
|
||||
|
||||
|`proctree_lock`
|
||||
|`proc`.`p_children` `proc`.`p_sibling`
|
||||
|===
|
||||
|
||||
[[locking-atomic]]
|
||||
== Atomically Protected Variables
|
||||
|
||||
An atomically protected variable is a special variable that is not protected by an explicit lock. Instead, all data accesses to the variables use special atomic operations as described in man:atomic[9]. Very few variables are treated this way, although other synchronization primitives such as mutexes are implemented with atomically protected variables.
|
||||
|
||||
* `mtx`.`mtx_lock`
|
||||
5756
documentation/content/en/books/arch-handbook/mac/chapter.adoc
Normal file
5756
documentation/content/en/books/arch-handbook/mac/chapter.adoc
Normal file
File diff suppressed because it is too large
Load diff
194
documentation/content/en/books/arch-handbook/newbus/chapter.adoc
Normal file
194
documentation/content/en/books/arch-handbook/newbus/chapter.adoc
Normal file
|
|
@ -0,0 +1,194 @@
|
|||
---
|
||||
title: Chapter 14. Newbus
|
||||
authors:
|
||||
- author: Jeroen Ruigrok van der Werven (asmodai)
|
||||
email: asmodai@FreeBSD.org
|
||||
- author: Hiten Pandya
|
||||
email: hiten@uk.FreeBSD.org
|
||||
prev: books/arch-handbook/usb
|
||||
next: books/arch-handbook/sound
|
||||
---
|
||||
|
||||
[[newbus]]
|
||||
= Newbus
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 14
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
_Special thanks to Matthew N. Dodd, Warner Losh, Bill Paul, Doug Rabson, Mike Smith, Peter Wemm and Scott Long_.
|
||||
|
||||
This chapter explains the Newbus device framework in detail.
|
||||
|
||||
[[newbus-devdrivers]]
|
||||
== Device Drivers
|
||||
|
||||
=== Purpose of a Device Driver
|
||||
|
||||
A device driver is a software component which provides the interface between the kernel's generic view of a peripheral (e.g., disk, network adapter) and the actual implementation of the peripheral. The _device driver interface (DDI)_ is the defined interface between the kernel and the device driver component.
|
||||
|
||||
=== Types of Device Drivers
|
||||
|
||||
There used to be days in UNIX(R), and thus FreeBSD, in which there were four types of devices defined:
|
||||
|
||||
* block device drivers
|
||||
* character device drivers
|
||||
* network device drivers
|
||||
* pseudo-device drivers
|
||||
|
||||
_Block devices_ performed in a way that used fixed size blocks [of data]. This type of driver depended on the so-called _buffer cache_, which had cached accessed blocks of data in a dedicated part of memory. Often this buffer cache was based on write-behind, which meant that when data was modified in memory it got synced to disk whenever the system did its periodical disk flushing, thus optimizing writes.
|
||||
|
||||
=== Character Devices
|
||||
|
||||
However, in the versions of FreeBSD 4.0 and onward the distinction between block and character devices became non-existent.
|
||||
|
||||
[[newbus-overview]]
|
||||
== Overview of Newbus
|
||||
|
||||
_Newbus_ is the implementation of a new bus architecture based on abstraction layers which saw its introduction in FreeBSD 3.0 when the Alpha port was imported into the source tree. It was not until 4.0 before it became the default system to use for device drivers. Its goals are to provide a more object-oriented means of interconnecting the various busses and devices which a host system provides to the _Operating System_.
|
||||
|
||||
Its main features include amongst others:
|
||||
|
||||
* dynamic attaching
|
||||
* easy modularization of drivers
|
||||
* pseudo-busses
|
||||
|
||||
One of the most prominent changes is the migration from the flat and ad-hoc system to a device tree layout.
|
||||
|
||||
At the top level resides the _"root"_ device which is the parent to hang all other devices on. For each architecture, there is typically a single child of "root" which has such things as _host-to-PCI bridges_, etc. attached to it. For x86, this "root" device is the _"nexus"_ device. For Alpha, various different models of Alpha have different top-level devices corresponding to the different hardware chipsets, including _lca_, _apecs_, _cia_ and _tsunami_.
|
||||
|
||||
A device in the Newbus context represents a single hardware entity in the system. For instance each PCI device is represented by a Newbus device. Any device in the system can have children; a device which has children is often called a _"bus"_. Examples of common busses in the system are ISA and PCI, which manage lists of devices attached to ISA and PCI busses respectively.
|
||||
|
||||
Often, a connection between different kinds of bus is represented by a _"bridge"_ device, which normally has one child for the attached bus. An example of this is a _PCI-to-PCI bridge_ which is represented by a device _[.filename]#pcibN#_ on the parent PCI bus and has a child _[.filename]#pciN#_ for the attached bus. This layout simplifies the implementation of the PCI bus tree, allowing common code to be used for both top-level and bridged busses.
|
||||
|
||||
Each device in the Newbus architecture asks its parent to map its resources. The parent then asks its own parent until the nexus is reached. So, basically the nexus is the only part of the Newbus system which knows about all resources.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
An ISA device might want to map its IO port at `0x230`, so it asks its parent, in this case the ISA bus. The ISA bus hands it over to the PCI-to-ISA bridge which in its turn asks the PCI bus, which reaches the host-to-PCI bridge and finally the nexus. The beauty of this transition upwards is that there is room to translate the requests. For example, the `0x230` IO port request might become memory-mapped at `0xb0000230` on a MIPS box by the PCI bridge.
|
||||
====
|
||||
|
||||
Resource allocation can be controlled at any place in the device tree. For instance on many Alpha platforms, ISA interrupts are managed separately from PCI interrupts and resource allocations for ISA interrupts are managed by the Alpha's ISA bus device. On IA-32, ISA and PCI interrupts are both managed by the top-level nexus device. For both ports, memory and port address space is managed by a single entity - nexus for IA-32 and the relevant chipset driver on Alpha (e.g., CIA or tsunami).
|
||||
|
||||
In order to normalize access to memory and port mapped resources, Newbus integrates the `bus_space` APIs from NetBSD. These provide a single API to replace inb/outb and direct memory reads/writes. The advantage of this is that a single driver can easily use either memory-mapped registers or port-mapped registers (some hardware supports both).
|
||||
|
||||
This support is integrated into the resource allocation mechanism. When a resource is allocated, a driver can retrieve the associated `bus_space_tag_t` and `bus_space_handle_t` from the resource.
|
||||
|
||||
Newbus also allows for definitions of interface methods in files dedicated to this purpose. These are the [.filename]#.m# files that are found under the [.filename]#src/sys# hierarchy.
|
||||
|
||||
The core of the Newbus system is an extensible "object-based programming" model. Each device in the system has a table of methods which it supports. The system and other devices uses those methods to control the device and request services. The different methods supported by a device are defined by a number of "interfaces". An "interface" is simply a group of related methods which can be implemented by a device.
|
||||
|
||||
In the Newbus system, the methods for a device are provided by the various device drivers in the system. When a device is attached to a driver during _auto-configuration_, it uses the method table declared by the driver. A device can later _detach_ from its driver and _re-attach_ to a new driver with a new method table. This allows dynamic replacement of drivers which can be useful for driver development.
|
||||
|
||||
The interfaces are described by an interface definition language similar to the language used to define vnode operations for file systems. The interface would be stored in a methods file (which would normally be named [.filename]#foo_if.m#).
|
||||
|
||||
.Newbus Methods
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
# Foo subsystem/driver (a comment...)
|
||||
|
||||
INTERFACE foo
|
||||
|
||||
METHOD int doit {
|
||||
device_t dev;
|
||||
};
|
||||
|
||||
# DEFAULT is the method that will be used, if a method was not
|
||||
# provided via: DEVMETHOD()
|
||||
|
||||
METHOD void doit_to_child {
|
||||
device_t dev;
|
||||
driver_t child;
|
||||
} DEFAULT doit_generic_to_child;
|
||||
....
|
||||
====
|
||||
|
||||
When this interface is compiled, it generates a header file "[.filename]#foo_if.h#" which contains function declarations:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int FOO_DOIT(device_t dev);
|
||||
int FOO_DOIT_TO_CHILD(device_t dev, device_t child);
|
||||
....
|
||||
|
||||
A source file, "[.filename]#foo_if.c#" is also created to accompany the automatically generated header file; it contains implementations of those functions which look up the location of the relevant functions in the object's method table and call that function.
|
||||
|
||||
The system defines two main interfaces. The first fundamental interface is called _"device"_ and includes methods which are relevant to all devices. Methods in the _"device"_ interface include _"probe"_, _"attach"_ and _"detach"_ to control detection of hardware and _"shutdown"_, _"suspend"_ and _"resume"_ for critical event notification.
|
||||
|
||||
The second, more complex interface is _"bus"_. This interface contains methods suitable for devices which have children, including methods to access bus specific per-device information footnote:[man:bus_generic_read_ivar[9] and man:bus_generic_write_ivar[9]], event notification (`_child_detached_`, `_driver_added_`) and resource management (`_alloc_resource_`, `_activate_resource_`, `_deactivate_resource_`, `_release_resource_`).
|
||||
|
||||
Many methods in the "bus" interface are performing services for some child of the bus device. These methods would normally use the first two arguments to specify the bus providing the service and the child device which is requesting the service. To simplify driver code, many of these methods have accessor functions which lookup the parent and call a method on the parent. For instance the method `BUS_TEARDOWN_INTR(device_t dev, device_t child, ...)` can be called using the function `bus_teardown_intr(device_t child, ...)`.
|
||||
|
||||
Some bus types in the system define additional interfaces to provide access to bus-specific functionality. For instance, the PCI bus driver defines the "pci" interface which has two methods `_read_config_` and `_write_config_` for accessing the configuration registers of a PCI device.
|
||||
|
||||
[[newbus-api]]
|
||||
== Newbus API
|
||||
|
||||
As the Newbus API is huge, this section makes some effort at documenting it. More information to come in the next revision of this document.
|
||||
|
||||
=== Important Locations in the Source Hierarchy
|
||||
|
||||
[.filename]#src/sys/[arch]/[arch]# - Kernel code for a specific machine architecture resides in this directory. For example, the `i386` architecture, or the `SPARC64` architecture.
|
||||
|
||||
[.filename]#src/sys/dev/[bus]# - device support for a specific `[bus]` resides in this directory.
|
||||
|
||||
[.filename]#src/sys/dev/pci# - PCI bus support code resides in this directory.
|
||||
|
||||
[.filename]#src/sys/[isa|pci]# - PCI/ISA device drivers reside in this directory. The PCI/ISA bus support code used to exist in this directory in FreeBSD version `4.0`.
|
||||
|
||||
=== Important Structures and Type Definitions
|
||||
|
||||
`devclass_t` - This is a type definition of a pointer to a `struct devclass`.
|
||||
|
||||
`device_method_t` - This is the same as `kobj_method_t` (see [.filename]#src/sys/kobj.h#).
|
||||
|
||||
`device_t` - This is a type definition of a pointer to a `struct device`. `device_t` represents a device in the system. It is a kernel object. See [.filename]#src/sys/sys/bus_private.h# for implementation details.
|
||||
|
||||
`driver_t` - This is a type definition which references `struct driver`. The `driver` struct is a class of the `device` kernel object; it also holds data private to the driver.
|
||||
|
||||
*_driver_t_ Implementation*
|
||||
[.programlisting]
|
||||
....
|
||||
struct driver {
|
||||
KOBJ_CLASS_FIELDS;
|
||||
void *priv; /* driver private data */
|
||||
};
|
||||
....
|
||||
|
||||
A `device_state_t` type, which is an enumeration, `device_state`. It contains the possible states of a Newbus device before and after the autoconfiguration process.
|
||||
|
||||
*Device States _device_state_t*
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* src/sys/sys/bus.h
|
||||
*/
|
||||
typedef enum device_state {
|
||||
DS_NOTPRESENT, /* not probed or probe failed */
|
||||
DS_ALIVE, /* probe succeeded */
|
||||
DS_ATTACHED, /* attach method called */
|
||||
DS_BUSY /* device is open */
|
||||
} device_state_t;
|
||||
....
|
||||
|
||||
190
documentation/content/en/books/arch-handbook/pccard/chapter.adoc
Normal file
190
documentation/content/en/books/arch-handbook/pccard/chapter.adoc
Normal file
|
|
@ -0,0 +1,190 @@
|
|||
---
|
||||
title: Chapter 16. PC Card
|
||||
prev: books/arch-handbook/sound
|
||||
next: books/arch-handbook/partiii
|
||||
---
|
||||
|
||||
[[pccard]]
|
||||
= PC Card
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 16
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
This chapter will talk about the FreeBSD mechanisms for writing a device driver for a PC Card or CardBus device. However, at present it just documents how to add a new device to an existing pccard driver.
|
||||
|
||||
[[pccard-adddev]]
|
||||
== Adding a Device
|
||||
|
||||
Device drivers know what devices they support. There is a table of supported devices in the kernel that drivers use to attach to a device.
|
||||
|
||||
[[pccard-overview]]
|
||||
=== Overview
|
||||
|
||||
PC Cards are identified in one of two ways, both based on the _Card Information Structure_ (CIS) stored on the card. The first method is to use numeric manufacturer and product numbers. The second method is to use the human readable strings that are also contained in the CIS. The PC Card bus uses a centralized database and some macros to facilitate a design pattern to help the driver writer match devices to his driver.
|
||||
|
||||
Original equipment manufacturers (OEMs) often develop a reference design for a PC Card product, then sell this design to other companies to market. Those companies refine the design, market the product to their target audience or geographic area, and put their own name plate onto the card. The refinements to the physical card are typically very minor, if any changes are made at all. To strengthen their brand, these vendors place their company name in the human readable strings in the CIS space, but leave the manufacturer and product IDs unchanged.
|
||||
|
||||
Due to this practice, FreeBSD drivers usually rely on numeric IDs for device identification. Using numeric IDs and a centralized database complicates adding IDs and support for cards to the system. One must carefully check to see who really made the card, especially when it appears that the vendor who made the card might already have a different manufacturer ID listed in the central database. Linksys, D-Link, and NetGear are a number of US manufacturers of LAN hardware that often sell the same design. These same designs can be sold in Japan under names such as Buffalo and Corega. Often, these devices will all have the same manufacturer and product IDs.
|
||||
|
||||
The PC Card bus code keeps a central database of card information, but not which driver is associated with them, in [.filename]#/sys/dev/pccard/pccarddevs#. It also provides a set of macros that allow one to easily construct simple entries in the table the driver uses to claim devices.
|
||||
|
||||
Finally, some really low end devices do not contain manufacturer identification at all. These devices must be detected by matching the human readable CIS strings. While it would be nice if we did not need this method as a fallback, it is necessary for some very low end CD-ROM players and Ethernet cards. This method should generally be avoided, but a number of devices are listed in this section because they were added prior to the recognition of the OEM nature of the PC Card business. When adding new devices, prefer using the numeric method.
|
||||
|
||||
[[pccard-pccarddevs]]
|
||||
=== Format of [.filename]#pccarddevs#
|
||||
|
||||
There are four sections in the [.filename]#pccarddevs# files. The first section lists the manufacturer numbers for vendors that use them. This section is sorted in numerical order. The next section has all of the products that are used by these vendors, along with their product ID numbers and a description string. The description string typically is not used (instead we set the device's description based on the human readable CIS, even if we match on the numeric version). These two sections are then repeated for devices that use the string matching method. Finally, C-style comments enclosed in `/*` and `*/` characters are allowed anywhere in the file.
|
||||
|
||||
The first section of the file contains the vendor IDs. Please keep this list sorted in numeric order. Also, please coordinate changes to this file because we share it with NetBSD to help facilitate a common clearing house for this information. For example, here are the first few vendor IDs:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
vendor FUJITSU 0x0004 Fujitsu Corporation
|
||||
vendor NETGEAR_2 0x000b Netgear
|
||||
vendor PANASONIC 0x0032 Matsushita Electric Industrial Co.
|
||||
vendor SANDISK 0x0045 Sandisk Corporation
|
||||
....
|
||||
|
||||
Chances are very good that the `NETGEAR_2` entry is really an OEM that NETGEAR purchased cards from and the author of support for those cards was unaware at the time that Netgear was using someone else's ID. These entries are fairly straightforward. The vendor keyword denotes the kind of line that this is, followed by the name of the vendor. This name will be repeated later in [.filename]#pccarddevs#, as well as used in the driver's match tables, so keep it short and a valid C identifier. A numeric ID in hex identifies the manufacturer. Do not add IDs of the form `0xffffffff` or `0xffff` because these are reserved IDs (the former is "no ID set" while the latter is sometimes seen in extremely poor quality cards to try to indicate "none"). Finally there is a string description of the company that makes the card. This string is not used in FreeBSD for anything but commentary purposes.
|
||||
|
||||
The second section of the file contains the products. As shown in this example, the format is similar to the vendor lines:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/* Allied Telesis K.K. */
|
||||
product ALLIEDTELESIS LA_PCM 0x0002 Allied Telesis LA-PCM
|
||||
|
||||
/* Archos */
|
||||
product ARCHOS ARC_ATAPI 0x0043 MiniCD
|
||||
....
|
||||
|
||||
The `product` keyword is followed by the vendor name, repeated from above. This is followed by the product name, which is used by the driver and should be a valid C identifier, but may also start with a number. As with the vendors, the hex product ID for this card follows the same convention for `0xffffffff` and `0xffff`. Finally, there is a string description of the device itself. This string typically is not used in FreeBSD, since FreeBSD's pccard bus driver will construct a string from the human readable CIS entries, but it can be used in the rare cases where this is somehow insufficient. The products are in alphabetical order by manufacturer, then numerical order by product ID. They have a C comment before each manufacturer's entries and there is a blank line between entries.
|
||||
|
||||
The third section is like the previous vendor section, but with all of the manufacturer numeric IDs set to `-1`, meaning "match anything found" in the FreeBSD pccard bus code. Since these are C identifiers, their names must be unique. Otherwise the format is identical to the first section of the file.
|
||||
|
||||
The final section contains the entries for those cards that must be identified by string entries. This section's format is a little different from the generic section:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
product ADDTRON AWP100 { "Addtron", "AWP-100&spWireless&spPCMCIA", "Version&sp01.02", NULL }
|
||||
product ALLIEDTELESIS WR211PCM { "Allied&spTelesis&spK.K.", "WR211PCM", NULL, NULL } Allied Telesis WR211PCM
|
||||
....
|
||||
|
||||
The familiar `product` keyword is followed by the vendor name and the card name, just as in the second section of the file. Here the format deviates from that used earlier. There is a {} grouping, followed by a number of strings. These strings correspond to the vendor, product, and extra information that is defined in a CIS_INFO tuple. These strings are filtered by the program that generates [.filename]#pccarddevs.h# to replace &sp with a real space. NULL strings mean that the corresponding part of the entry should be ignored. The example shown here contains a bad entry. It should not contain the version number unless that is critical for the operation of the card. Sometimes vendors will have many different versions of the card in the field that all work, in which case that information only makes it harder for someone with a similar card to use it with FreeBSD. Sometimes it is necessary when a vendor wishes to sell many different parts under the same brand due to market considerations (availability, price, and so forth). Then it can be critical to disambiguating the card in those rare cases where the vendor kept the same manufacturer/product pair. Regular expression matching is not available at this time.
|
||||
|
||||
[[pccard-probe]]
|
||||
=== Sample Probe Routine
|
||||
|
||||
To understand how to add a device to the list of supported devices, one must understand the probe and/or match routines that many drivers have. It is complicated a little in FreeBSD 5.x because there is a compatibility layer for OLDCARD present as well. Since only the window-dressing is different, an idealized version will be presented here.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static const struct pccard_product wi_pccard_products[] = {
|
||||
PCMCIA_CARD(3COM, 3CRWE737A, 0),
|
||||
PCMCIA_CARD(BUFFALO, WLI_PCM_S11, 0),
|
||||
PCMCIA_CARD(BUFFALO, WLI_CF_S11G, 0),
|
||||
PCMCIA_CARD(TDK, LAK_CD011WL, 0),
|
||||
{ NULL }
|
||||
};
|
||||
|
||||
static int
|
||||
wi_pccard_probe(dev)
|
||||
device_t dev;
|
||||
{
|
||||
const struct pccard_product *pp;
|
||||
|
||||
if ((pp = pccard_product_lookup(dev, wi_pccard_products,
|
||||
sizeof(wi_pccard_products[0]), NULL)) != NULL) {
|
||||
if (pp->pp_name != NULL)
|
||||
device_set_desc(dev, pp->pp_name);
|
||||
return (0);
|
||||
}
|
||||
return (ENXIO);
|
||||
}
|
||||
....
|
||||
|
||||
Here we have a simple pccard probe routine that matches a few devices. As stated above, the name may vary (if it is not `foo_pccard_probe()` it will be `foo_pccard_match()`). The function `pccard_product_lookup()` is a generalized function that walks the table and returns a pointer to the first entry that it matches. Some drivers may use this mechanism to convey additional information about some cards to the rest of the driver, so there may be some variance in the table. The only requirement is that each row of the table must have a `struct pccard_product` as the first element.
|
||||
|
||||
Looking at the table `wi_pccard_products`, one notices that all the entries are of the form `PCMCIA_CARD(_foo_, _bar_, _baz_)`. The _foo_ part is the manufacturer ID from [.filename]#pccarddevs#. The _bar_ part is the product ID. _baz_ is the expected function number for this card. Many pccards can have multiple functions, and some way to disambiguate function 1 from function 0 is needed. You may see `PCMCIA_CARD_D`, which includes the device description from [.filename]#pccarddevs#. You may also see `PCMCIA_CARD2` and `PCMCIA_CARD2_D` which are used when you need to match both CIS strings and manufacturer numbers, in the "use the default description" and "take the description from pccarddevs" flavors.
|
||||
|
||||
[[pccard-add]]
|
||||
=== Putting it All Together
|
||||
|
||||
To add a new device, one must first obtain the identification information from the device. The easiest way to do this is to insert the device into a PC Card or CF slot and issue `devinfo -v`. Sample output:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
cbb1 pnpinfo vendor=0x104c device=0xac51 subvendor=0x1265 subdevice=0x0300 class=0x060700 at slot=10 function=1
|
||||
cardbus1
|
||||
pccard1
|
||||
unknown pnpinfo manufacturer=0x026f product=0x030c cisvendor="BUFFALO" cisproduct="WLI2-CF-S11" function_type=6 at function=0
|
||||
....
|
||||
|
||||
`manufacturer` and `product` are the numeric IDs for this product, while `cisvendor` and `cisproduct` are the product description strings from the CIS.
|
||||
|
||||
Since we first want to prefer the numeric option, first try to construct an entry based on that. The above card has been slightly fictionalized for the purpose of this example. The vendor is BUFFALO, which we see already has an entry:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
vendor BUFFALO 0x026f BUFFALO (Melco Corporation)
|
||||
....
|
||||
|
||||
But there is no entry for this particular card. Instead we find:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/* BUFFALO */
|
||||
product BUFFALO WLI_PCM_S11 0x0305 BUFFALO AirStation 11Mbps WLAN
|
||||
product BUFFALO LPC_CF_CLT 0x0307 BUFFALO LPC-CF-CLT
|
||||
product BUFFALO LPC3_CLT 0x030a BUFFALO LPC3-CLT Ethernet Adapter
|
||||
product BUFFALO WLI_CF_S11G 0x030b BUFFALO AirStation 11Mbps CF WLAN
|
||||
....
|
||||
|
||||
To add the device, we can just add this entry to [.filename]#pccarddevs#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
product BUFFALO WLI2_CF_S11G 0x030c BUFFALO AirStation ultra 802.11b CF
|
||||
....
|
||||
|
||||
Once these steps are complete, the card can be added to the driver. That is a simple operation of adding one line:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static const struct pccard_product wi_pccard_products[] = {
|
||||
PCMCIA_CARD(3COM, 3CRWE737A, 0),
|
||||
PCMCIA_CARD(BUFFALO, WLI_PCM_S11, 0),
|
||||
PCMCIA_CARD(BUFFALO, WLI_CF_S11G, 0),
|
||||
+ PCMCIA_CARD(BUFFALO, WLI_CF2_S11G, 0),
|
||||
PCMCIA_CARD(TDK, LAK_CD011WL, 0),
|
||||
{ NULL }
|
||||
};
|
||||
....
|
||||
|
||||
Note that I have included a '`+`' in the line before the line that I added, but that is simply to highlight the line. Do not add it to the actual driver. Once you have added the line, you can recompile your kernel or module and test it. If the device is recognized and works, please submit a patch. If it does not work, please figure out what is needed to make it work and submit a patch. If the device is not recognized at all, you have done something wrong and should recheck each step.
|
||||
|
||||
If you are a FreeBSD src committer, and everything appears to be working, then you can commit the changes to the tree. However, there are some minor tricky things to be considered. [.filename]#pccarddevs# must be committed to the tree first. Then [.filename]#pccarddevs.h# must be regenerated and committed as a second step, ensuring that the right $FreeBSD$ tag is in the latter file. Finally, commit the additions to the driver.
|
||||
|
||||
[[pccard-pr]]
|
||||
=== Submitting a New Device
|
||||
|
||||
Please do not send entries for new devices to the author directly. Instead, submit them as a PR and send the author the PR number for his records. This ensures that entries are not lost. When submitting a PR, it is unnecessary to include the [.filename]#pccardevs.h# diffs in the patch, since those will be regenerated. It is necessary to include a description of the device, as well as the patches to the client driver. If you do not know the name, use OEM99 as the name, and the author will adjust OEM99 accordingly after investigation. Committers should not commit OEM99, but instead find the highest OEM entry and commit one more than that.
|
||||
405
documentation/content/en/books/arch-handbook/pci/chapter.adoc
Normal file
405
documentation/content/en/books/arch-handbook/pci/chapter.adoc
Normal file
|
|
@ -0,0 +1,405 @@
|
|||
---
|
||||
title: Chapter 11. PCI Devices
|
||||
prev: books/arch-handbook/isa
|
||||
next: books/arch-handbook/scsi
|
||||
---
|
||||
|
||||
[[pci]]
|
||||
= PCI Devices
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 11
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
This chapter will talk about the FreeBSD mechanisms for writing a device driver for a device on a PCI bus.
|
||||
|
||||
[[pci-probe]]
|
||||
== Probe and Attach
|
||||
|
||||
Information here about how the PCI bus code iterates through the unattached devices and see if a newly loaded kld will attach to any of them.
|
||||
|
||||
=== Sample Driver Source ([.filename]#mypci.c#)
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* Simple KLD to play with the PCI functions.
|
||||
*
|
||||
* Murray Stokely
|
||||
*/
|
||||
|
||||
#include <sys/param.h> /* defines used in kernel.h */
|
||||
#include <sys/module.h>
|
||||
#include <sys/systm.h>
|
||||
#include <sys/errno.h>
|
||||
#include <sys/kernel.h> /* types used in module initialization */
|
||||
#include <sys/conf.h> /* cdevsw struct */
|
||||
#include <sys/uio.h> /* uio struct */
|
||||
#include <sys/malloc.h>
|
||||
#include <sys/bus.h> /* structs, prototypes for pci bus stuff and DEVMETHOD macros! */
|
||||
|
||||
#include <machine/bus.h>
|
||||
#include <sys/rman.h>
|
||||
#include <machine/resource.h>
|
||||
|
||||
#include <dev/pci/pcivar.h> /* For pci_get macros! */
|
||||
#include <dev/pci/pcireg.h>
|
||||
|
||||
/* The softc holds our per-instance data. */
|
||||
struct mypci_softc {
|
||||
device_t my_dev;
|
||||
struct cdev *my_cdev;
|
||||
};
|
||||
|
||||
/* Function prototypes */
|
||||
static d_open_t mypci_open;
|
||||
static d_close_t mypci_close;
|
||||
static d_read_t mypci_read;
|
||||
static d_write_t mypci_write;
|
||||
|
||||
/* Character device entry points */
|
||||
|
||||
static struct cdevsw mypci_cdevsw = {
|
||||
.d_version = D_VERSION,
|
||||
.d_open = mypci_open,
|
||||
.d_close = mypci_close,
|
||||
.d_read = mypci_read,
|
||||
.d_write = mypci_write,
|
||||
.d_name = "mypci",
|
||||
};
|
||||
|
||||
/*
|
||||
* In the cdevsw routines, we find our softc by using the si_drv1 member
|
||||
* of struct cdev. We set this variable to point to our softc in our
|
||||
* attach routine when we create the /dev entry.
|
||||
*/
|
||||
|
||||
int
|
||||
mypci_open(struct cdev *dev, int oflags, int devtype, struct thread *td)
|
||||
{
|
||||
struct mypci_softc *sc;
|
||||
|
||||
/* Look up our softc. */
|
||||
sc = dev->si_drv1;
|
||||
device_printf(sc->my_dev, "Opened successfully.\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
int
|
||||
mypci_close(struct cdev *dev, int fflag, int devtype, struct thread *td)
|
||||
{
|
||||
struct mypci_softc *sc;
|
||||
|
||||
/* Look up our softc. */
|
||||
sc = dev->si_drv1;
|
||||
device_printf(sc->my_dev, "Closed.\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
int
|
||||
mypci_read(struct cdev *dev, struct uio *uio, int ioflag)
|
||||
{
|
||||
struct mypci_softc *sc;
|
||||
|
||||
/* Look up our softc. */
|
||||
sc = dev->si_drv1;
|
||||
device_printf(sc->my_dev, "Asked to read %zd bytes.\n", uio->uio_resid);
|
||||
return (0);
|
||||
}
|
||||
|
||||
int
|
||||
mypci_write(struct cdev *dev, struct uio *uio, int ioflag)
|
||||
{
|
||||
struct mypci_softc *sc;
|
||||
|
||||
/* Look up our softc. */
|
||||
sc = dev->si_drv1;
|
||||
device_printf(sc->my_dev, "Asked to write %zd bytes.\n", uio->uio_resid);
|
||||
return (0);
|
||||
}
|
||||
|
||||
/* PCI Support Functions */
|
||||
|
||||
/*
|
||||
* Compare the device ID of this device against the IDs that this driver
|
||||
* supports. If there is a match, set the description and return success.
|
||||
*/
|
||||
static int
|
||||
mypci_probe(device_t dev)
|
||||
{
|
||||
|
||||
device_printf(dev, "MyPCI Probe\nVendor ID : 0x%x\nDevice ID : 0x%x\n",
|
||||
pci_get_vendor(dev), pci_get_device(dev));
|
||||
|
||||
if (pci_get_vendor(dev) == 0x11c1) {
|
||||
printf("We've got the Winmodem, probe successful!\n");
|
||||
device_set_desc(dev, "WinModem");
|
||||
return (BUS_PROBE_DEFAULT);
|
||||
}
|
||||
return (ENXIO);
|
||||
}
|
||||
|
||||
/* Attach function is only called if the probe is successful. */
|
||||
|
||||
static int
|
||||
mypci_attach(device_t dev)
|
||||
{
|
||||
struct mypci_softc *sc;
|
||||
|
||||
printf("MyPCI Attach for : deviceID : 0x%x\n", pci_get_devid(dev));
|
||||
|
||||
/* Look up our softc and initialize its fields. */
|
||||
sc = device_get_softc(dev);
|
||||
sc->my_dev = dev;
|
||||
|
||||
/*
|
||||
* Create a /dev entry for this device. The kernel will assign us
|
||||
* a major number automatically. We use the unit number of this
|
||||
* device as the minor number and name the character device
|
||||
* "mypci<unit>".
|
||||
*/
|
||||
sc->my_cdev = make_dev(&mypci_cdevsw, device_get_unit(dev),
|
||||
UID_ROOT, GID_WHEEL, 0600, "mypci%u", device_get_unit(dev));
|
||||
sc->my_cdev->si_drv1 = sc;
|
||||
printf("Mypci device loaded.\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
/* Detach device. */
|
||||
|
||||
static int
|
||||
mypci_detach(device_t dev)
|
||||
{
|
||||
struct mypci_softc *sc;
|
||||
|
||||
/* Teardown the state in our softc created in our attach routine. */
|
||||
sc = device_get_softc(dev);
|
||||
destroy_dev(sc->my_cdev);
|
||||
printf("Mypci detach!\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
/* Called during system shutdown after sync. */
|
||||
|
||||
static int
|
||||
mypci_shutdown(device_t dev)
|
||||
{
|
||||
|
||||
printf("Mypci shutdown!\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
/*
|
||||
* Device suspend routine.
|
||||
*/
|
||||
static int
|
||||
mypci_suspend(device_t dev)
|
||||
{
|
||||
|
||||
printf("Mypci suspend!\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
/*
|
||||
* Device resume routine.
|
||||
*/
|
||||
static int
|
||||
mypci_resume(device_t dev)
|
||||
{
|
||||
|
||||
printf("Mypci resume!\n");
|
||||
return (0);
|
||||
}
|
||||
|
||||
static device_method_t mypci_methods[] = {
|
||||
/* Device interface */
|
||||
DEVMETHOD(device_probe, mypci_probe),
|
||||
DEVMETHOD(device_attach, mypci_attach),
|
||||
DEVMETHOD(device_detach, mypci_detach),
|
||||
DEVMETHOD(device_shutdown, mypci_shutdown),
|
||||
DEVMETHOD(device_suspend, mypci_suspend),
|
||||
DEVMETHOD(device_resume, mypci_resume),
|
||||
|
||||
DEVMETHOD_END
|
||||
};
|
||||
|
||||
static devclass_t mypci_devclass;
|
||||
|
||||
DEFINE_CLASS_0(mypci, mypci_driver, mypci_methods, sizeof(struct mypci_softc));
|
||||
DRIVER_MODULE(mypci, pci, mypci_driver, mypci_devclass, 0, 0);
|
||||
....
|
||||
|
||||
=== [.filename]#Makefile# for Sample Driver
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# Makefile for mypci driver
|
||||
|
||||
KMOD= mypci
|
||||
SRCS= mypci.c
|
||||
SRCS+= device_if.h bus_if.h pci_if.h
|
||||
|
||||
.include <bsd.kmod.mk>
|
||||
....
|
||||
|
||||
If you place the above source file and [.filename]#Makefile# into a directory, you may run `make` to compile the sample driver. Additionally, you may run `make load` to load the driver into the currently running kernel and `make unload` to unload the driver after it is loaded.
|
||||
|
||||
=== Additional Resources
|
||||
|
||||
* http://www.pcisig.org/[PCI Special Interest Group]
|
||||
* PCI System Architecture, Fourth Edition by Tom Shanley, et al.
|
||||
|
||||
[[pci-bus]]
|
||||
== Bus Resources
|
||||
|
||||
FreeBSD provides an object-oriented mechanism for requesting resources from a parent bus. Almost all devices will be a child member of some sort of bus (PCI, ISA, USB, SCSI, etc) and these devices need to acquire resources from their parent bus (such as memory segments, interrupt lines, or DMA channels).
|
||||
|
||||
=== Base Address Registers
|
||||
|
||||
To do anything particularly useful with a PCI device you will need to obtain the _Base Address Registers_ (BARs) from the PCI Configuration space. The PCI-specific details of obtaining the BAR are abstracted in the `bus_alloc_resource()` function.
|
||||
|
||||
For example, a typical driver might have something similar to this in the `attach()` function:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sc->bar0id = PCIR_BAR(0);
|
||||
sc->bar0res = bus_alloc_resource(dev, SYS_RES_MEMORY, &sc->bar0id,
|
||||
0, ~0, 1, RF_ACTIVE);
|
||||
if (sc->bar0res == NULL) {
|
||||
printf("Memory allocation of PCI base register 0 failed!\n");
|
||||
error = ENXIO;
|
||||
goto fail1;
|
||||
}
|
||||
|
||||
sc->bar1id = PCIR_BAR(1);
|
||||
sc->bar1res = bus_alloc_resource(dev, SYS_RES_MEMORY, &sc->bar1id,
|
||||
0, ~0, 1, RF_ACTIVE);
|
||||
if (sc->bar1res == NULL) {
|
||||
printf("Memory allocation of PCI base register 1 failed!\n");
|
||||
error = ENXIO;
|
||||
goto fail2;
|
||||
}
|
||||
sc->bar0_bt = rman_get_bustag(sc->bar0res);
|
||||
sc->bar0_bh = rman_get_bushandle(sc->bar0res);
|
||||
sc->bar1_bt = rman_get_bustag(sc->bar1res);
|
||||
sc->bar1_bh = rman_get_bushandle(sc->bar1res);
|
||||
....
|
||||
|
||||
Handles for each base address register are kept in the `softc` structure so that they can be used to write to the device later.
|
||||
|
||||
These handles can then be used to read or write from the device registers with the `bus_space_*` functions. For example, a driver might contain a shorthand function to read from a board specific register like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
uint16_t
|
||||
board_read(struct ni_softc *sc, uint16_t address)
|
||||
{
|
||||
return bus_space_read_2(sc->bar1_bt, sc->bar1_bh, address);
|
||||
}
|
||||
....
|
||||
|
||||
Similarly, one could write to the registers with:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
void
|
||||
board_write(struct ni_softc *sc, uint16_t address, uint16_t value)
|
||||
{
|
||||
bus_space_write_2(sc->bar1_bt, sc->bar1_bh, address, value);
|
||||
}
|
||||
....
|
||||
|
||||
These functions exist in 8bit, 16bit, and 32bit versions and you should use `bus_space_{read|write}_{1|2|4}` accordingly.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
In FreeBSD 7.0 and later, you can use the `bus_*` functions instead of `bus_space_*`. The `bus_*` functions take a struct resource * pointer instead of a bus tag and handle. Thus, you could drop the bus tag and bus handle members from the `softc` and rewrite the `board_read()` function as:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
uint16_t
|
||||
board_read(struct ni_softc *sc, uint16_t address)
|
||||
{
|
||||
return (bus_read(sc->bar1res, address));
|
||||
}
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
=== Interrupts
|
||||
|
||||
Interrupts are allocated from the object-oriented bus code in a way similar to the memory resources. First an IRQ resource must be allocated from the parent bus, and then the interrupt handler must be set up to deal with this IRQ.
|
||||
|
||||
Again, a sample from a device `attach()` function says more than words.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/* Get the IRQ resource */
|
||||
|
||||
sc->irqid = 0x0;
|
||||
sc->irqres = bus_alloc_resource(dev, SYS_RES_IRQ, &(sc->irqid),
|
||||
0, ~0, 1, RF_SHAREABLE | RF_ACTIVE);
|
||||
if (sc->irqres == NULL) {
|
||||
printf("IRQ allocation failed!\n");
|
||||
error = ENXIO;
|
||||
goto fail3;
|
||||
}
|
||||
|
||||
/* Now we should set up the interrupt handler */
|
||||
|
||||
error = bus_setup_intr(dev, sc->irqres, INTR_TYPE_MISC,
|
||||
my_handler, sc, &(sc->handler));
|
||||
if (error) {
|
||||
printf("Couldn't set up irq\n");
|
||||
goto fail4;
|
||||
}
|
||||
....
|
||||
|
||||
Some care must be taken in the detach routine of the driver. You must quiesce the device's interrupt stream, and remove the interrupt handler. Once `bus_teardown_intr()` has returned, you know that your interrupt handler will no longer be called and that all threads that might have been executing this interrupt handler have returned. Since this function can sleep, you must not hold any mutexes when calling this function.
|
||||
|
||||
=== DMA
|
||||
|
||||
This section is obsolete, and present only for historical reasons. The proper methods for dealing with these issues is to use the `bus_space_dma*()` functions instead. This paragraph can be removed when this section is updated to reflect that usage. However, at the moment, the API is in a bit of flux, so once that settles down, it would be good to update this section to reflect that.
|
||||
|
||||
On the PC, peripherals that want to do bus-mastering DMA must deal with physical addresses. This is a problem since FreeBSD uses virtual memory and deals almost exclusively with virtual addresses. Fortunately, there is a function, `vtophys()` to help.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#include <vm/vm.h>
|
||||
#include <vm/pmap.h>
|
||||
|
||||
#define vtophys(virtual_address) (...)
|
||||
....
|
||||
|
||||
The solution is a bit different on the alpha however, and what we really want is a function called `vtobus()`.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#if defined(__alpha__)
|
||||
#define vtobus(va) alpha_XXX_dmamap((vm_offset_t)va)
|
||||
#else
|
||||
#define vtobus(va) vtophys(va)
|
||||
#endif
|
||||
....
|
||||
|
||||
=== Deallocating Resources
|
||||
|
||||
It is very important to deallocate all of the resources that were allocated during `attach()`. Care must be taken to deallocate the correct stuff even on a failure condition so that the system will remain usable while your driver dies.
|
||||
1355
documentation/content/en/books/arch-handbook/scsi/chapter.adoc
Normal file
1355
documentation/content/en/books/arch-handbook/scsi/chapter.adoc
Normal file
File diff suppressed because it is too large
Load diff
341
documentation/content/en/books/arch-handbook/smp/chapter.adoc
Normal file
341
documentation/content/en/books/arch-handbook/smp/chapter.adoc
Normal file
|
|
@ -0,0 +1,341 @@
|
|||
---
|
||||
title: Chapter 8. SMPng Design Document
|
||||
prev: books/arch-handbook/vm
|
||||
next: books/arch-handbook/partii
|
||||
---
|
||||
|
||||
[[smp]]
|
||||
= SMPng Design Document
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 8
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[smp-intro]]
|
||||
== Introduction
|
||||
|
||||
This document presents the current design and implementation of the SMPng Architecture. First, the basic primitives and tools are introduced. Next, a general architecture for the FreeBSD kernel's synchronization and execution model is laid out. Then, locking strategies for specific subsystems are discussed, documenting the approaches taken to introduce fine-grained synchronization and parallelism for each subsystem. Finally, detailed implementation notes are provided to motivate design choices, and make the reader aware of important implications involving the use of specific primitives.
|
||||
|
||||
This document is a work-in-progress, and will be updated to reflect on-going design and implementation activities associated with the SMPng Project. Many sections currently exist only in outline form, but will be fleshed out as work proceeds. Updates or suggestions regarding the document may be directed to the document editors.
|
||||
|
||||
The goal of SMPng is to allow concurrency in the kernel. The kernel is basically one rather large and complex program. To make the kernel multi-threaded we use some of the same tools used to make other programs multi-threaded. These include mutexes, shared/exclusive locks, semaphores, and condition variables. For the definitions of these and other SMP-related terms, please see the <<smp-glossary>> section of this article.
|
||||
|
||||
[[smp-lock-fundamentals]]
|
||||
== Basic Tools and Locking Fundamentals
|
||||
|
||||
=== Atomic Instructions and Memory Barriers
|
||||
|
||||
There are several existing treatments of memory barriers and atomic instructions, so this section will not include a lot of detail. To put it simply, one can not go around reading variables without a lock if a lock is used to protect writes to that variable. This becomes obvious when you consider that memory barriers simply determine relative order of memory operations; they do not make any guarantee about timing of memory operations. That is, a memory barrier does not force the contents of a CPU's local cache or store buffer to flush. Instead, the memory barrier at lock release simply ensures that all writes to the protected data will be visible to other CPU's or devices if the write to release the lock is visible. The CPU is free to keep that data in its cache or store buffer as long as it wants. However, if another CPU performs an atomic instruction on the same datum, the first CPU must guarantee that the updated value is made visible to the second CPU along with any other operations that memory barriers may require.
|
||||
|
||||
For example, assuming a simple model where data is considered visible when it is in main memory (or a global cache), when an atomic instruction is triggered on one CPU, other CPU's store buffers and caches must flush any writes to that same cache line along with any pending operations behind a memory barrier.
|
||||
|
||||
This requires one to take special care when using an item protected by atomic instructions. For example, in the sleep mutex implementation, we have to use an `atomic_cmpset` rather than an `atomic_set` to turn on the `MTX_CONTESTED` bit. The reason is that we read the value of `mtx_lock` into a variable and then make a decision based on that read. However, the value we read may be stale, or it may change while we are making our decision. Thus, when the `atomic_set` executed, it may end up setting the bit on another value than the one we made the decision on. Thus, we have to use an `atomic_cmpset` to set the value only if the value we made the decision on is up-to-date and valid.
|
||||
|
||||
Finally, atomic instructions only allow one item to be updated or read. If one needs to atomically update several items, then a lock must be used instead. For example, if two counters must be read and have values that are consistent relative to each other, then those counters must be protected by a lock rather than by separate atomic instructions.
|
||||
|
||||
=== Read Locks Versus Write Locks
|
||||
|
||||
Read locks do not need to be as strong as write locks. Both types of locks need to ensure that the data they are accessing is not stale. However, only write access requires exclusive access. Multiple threads can safely read a value. Using different types of locks for reads and writes can be implemented in a number of ways.
|
||||
|
||||
First, sx locks can be used in this manner by using an exclusive lock when writing and a shared lock when reading. This method is quite straightforward.
|
||||
|
||||
A second method is a bit more obscure. You can protect a datum with multiple locks. Then for reading that data you simply need to have a read lock of one of the locks. However, to write to the data, you need to have a write lock of all of the locks. This can make writing rather expensive but can be useful when data is accessed in various ways. For example, the parent process pointer is protected by both the `proctree_lock` sx lock and the per-process mutex. Sometimes the proc lock is easier as we are just checking to see who a parent of a process is that we already have locked. However, other places such as `inferior` need to walk the tree of processes via parent pointers and locking each process would be prohibitive as well as a pain to guarantee that the condition you are checking remains valid for both the check and the actions taken as a result of the check.
|
||||
|
||||
=== Locking Conditions and Results
|
||||
|
||||
If you need a lock to check the state of a variable so that you can take an action based on the state you read, you can not just hold the lock while reading the variable and then drop the lock before you act on the value you read. Once you drop the lock, the variable can change rendering your decision invalid. Thus, you must hold the lock both while reading the variable and while performing the action as a result of the test.
|
||||
|
||||
[[smp-design]]
|
||||
== General Architecture and Design
|
||||
|
||||
=== Interrupt Handling
|
||||
|
||||
Following the pattern of several other multi-threaded UNIX(R) kernels, FreeBSD deals with interrupt handlers by giving them their own thread context. Providing a context for interrupt handlers allows them to block on locks. To help avoid latency, however, interrupt threads run at real-time kernel priority. Thus, interrupt handlers should not execute for very long to avoid starving other kernel threads. In addition, since multiple handlers may share an interrupt thread, interrupt handlers should not sleep or use a sleepable lock to avoid starving another interrupt handler.
|
||||
|
||||
The interrupt threads currently in FreeBSD are referred to as heavyweight interrupt threads. They are called this because switching to an interrupt thread involves a full context switch. In the initial implementation, the kernel was not preemptive and thus interrupts that interrupted a kernel thread would have to wait until the kernel thread blocked or returned to userland before they would have an opportunity to run.
|
||||
|
||||
To deal with the latency problems, the kernel in FreeBSD has been made preemptive. Currently, we only preempt a kernel thread when we release a sleep mutex or when an interrupt comes in. However, the plan is to make the FreeBSD kernel fully preemptive as described below.
|
||||
|
||||
Not all interrupt handlers execute in a thread context. Instead, some handlers execute directly in primary interrupt context. These interrupt handlers are currently misnamed "fast" interrupt handlers since the `INTR_FAST` flag used in earlier versions of the kernel is used to mark these handlers. The only interrupts which currently use these types of interrupt handlers are clock interrupts and serial I/O device interrupts. Since these handlers do not have their own context, they may not acquire blocking locks and thus may only use spin mutexes.
|
||||
|
||||
Finally, there is one optional optimization that can be added in MD code called lightweight context switches. Since an interrupt thread executes in a kernel context, it can borrow the vmspace of any process. Thus, in a lightweight context switch, the switch to the interrupt thread does not switch vmspaces but borrows the vmspace of the interrupted thread. In order to ensure that the vmspace of the interrupted thread does not disappear out from under us, the interrupted thread is not allowed to execute until the interrupt thread is no longer borrowing its vmspace. This can happen when the interrupt thread either blocks or finishes. If an interrupt thread blocks, then it will use its own context when it is made runnable again. Thus, it can release the interrupted thread.
|
||||
|
||||
The cons of this optimization are that they are very machine specific and complex and thus only worth the effort if their is a large performance improvement. At this point it is probably too early to tell, and in fact, will probably hurt performance as almost all interrupt handlers will immediately block on Giant and require a thread fix-up when they block. Also, an alternative method of interrupt handling has been proposed by Mike Smith that works like so:
|
||||
|
||||
. Each interrupt handler has two parts: a predicate which runs in primary interrupt context and a handler which runs in its own thread context.
|
||||
. If an interrupt handler has a predicate, then when an interrupt is triggered, the predicate is run. If the predicate returns true then the interrupt is assumed to be fully handled and the kernel returns from the interrupt. If the predicate returns false or there is no predicate, then the threaded handler is scheduled to run.
|
||||
|
||||
Fitting light weight context switches into this scheme might prove rather complicated. Since we may want to change to this scheme at some point in the future, it is probably best to defer work on light weight context switches until we have settled on the final interrupt handling architecture and determined how light weight context switches might or might not fit into it.
|
||||
|
||||
=== Kernel Preemption and Critical Sections
|
||||
|
||||
==== Kernel Preemption in a Nutshell
|
||||
|
||||
Kernel preemption is fairly simple. The basic idea is that a CPU should always be doing the highest priority work available. Well, that is the ideal at least. There are a couple of cases where the expense of achieving the ideal is not worth being perfect.
|
||||
|
||||
Implementing full kernel preemption is very straightforward: when you schedule a thread to be executed by putting it on a run queue, you check to see if its priority is higher than the currently executing thread. If so, you initiate a context switch to that thread.
|
||||
|
||||
While locks can protect most data in the case of a preemption, not all of the kernel is preemption safe. For example, if a thread holding a spin mutex preempted and the new thread attempts to grab the same spin mutex, the new thread may spin forever as the interrupted thread may never get a chance to execute. Also, some code such as the code to assign an address space number for a process during `exec` on the Alpha needs to not be preempted as it supports the actual context switch code. Preemption is disabled for these code sections by using a critical section.
|
||||
|
||||
==== Critical Sections
|
||||
|
||||
The responsibility of the critical section API is to prevent context switches inside of a critical section. With a fully preemptive kernel, every `setrunqueue` of a thread other than the current thread is a preemption point. One implementation is for `critical_enter` to set a per-thread flag that is cleared by its counterpart. If `setrunqueue` is called with this flag set, it does not preempt regardless of the priority of the new thread relative to the current thread. However, since critical sections are used in spin mutexes to prevent context switches and multiple spin mutexes can be acquired, the critical section API must support nesting. For this reason the current implementation uses a nesting count instead of a single per-thread flag.
|
||||
|
||||
In order to minimize latency, preemptions inside of a critical section are deferred rather than dropped. If a thread that would normally be preempted to is made runnable while the current thread is in a critical section, then a per-thread flag is set to indicate that there is a pending preemption. When the outermost critical section is exited, the flag is checked. If the flag is set, then the current thread is preempted to allow the higher priority thread to run.
|
||||
|
||||
Interrupts pose a problem with regards to spin mutexes. If a low-level interrupt handler needs a lock, it needs to not interrupt any code needing that lock to avoid possible data structure corruption. Currently, providing this mechanism is piggybacked onto critical section API by means of the `cpu_critical_enter` and `cpu_critical_exit` functions. Currently this API disables and re-enables interrupts on all of FreeBSD's current platforms. This approach may not be purely optimal, but it is simple to understand and simple to get right. Theoretically, this second API need only be used for spin mutexes that are used in primary interrupt context. However, to make the code simpler, it is used for all spin mutexes and even all critical sections. It may be desirable to split out the MD API from the MI API and only use it in conjunction with the MI API in the spin mutex implementation. If this approach is taken, then the MD API likely would need a rename to show that it is a separate API.
|
||||
|
||||
==== Design Tradeoffs
|
||||
|
||||
As mentioned earlier, a couple of trade-offs have been made to sacrifice cases where perfect preemption may not always provide the best performance.
|
||||
|
||||
The first trade-off is that the preemption code does not take other CPUs into account. Suppose we have a two CPU's A and B with the priority of A's thread as 4 and the priority of B's thread as 2. If CPU B makes a thread with priority 1 runnable, then in theory, we want CPU A to switch to the new thread so that we will be running the two highest priority runnable threads. However, the cost of determining which CPU to enforce a preemption on as well as actually signaling that CPU via an IPI along with the synchronization that would be required would be enormous. Thus, the current code would instead force CPU B to switch to the higher priority thread. Note that this still puts the system in a better position as CPU B is executing a thread of priority 1 rather than a thread of priority 2.
|
||||
|
||||
The second trade-off limits immediate kernel preemption to real-time priority kernel threads. In the simple case of preemption defined above, a thread is always preempted immediately (or as soon as a critical section is exited) if a higher priority thread is made runnable. However, many threads executing in the kernel only execute in a kernel context for a short time before either blocking or returning to userland. Thus, if the kernel preempts these threads to run another non-realtime kernel thread, the kernel may switch out the executing thread just before it is about to sleep or execute. The cache on the CPU must then adjust to the new thread. When the kernel returns to the preempted thread, it must refill all the cache information that was lost. In addition, two extra context switches are performed that could be avoided if the kernel deferred the preemption until the first thread blocked or returned to userland. Thus, by default, the preemption code will only preempt immediately if the higher priority thread is a real-time priority thread.
|
||||
|
||||
Turning on full kernel preemption for all kernel threads has value as a debugging aid since it exposes more race conditions. It is especially useful on UP systems were many races are hard to simulate otherwise. Thus, there is a kernel option `FULL_PREEMPTION` to enable preemption for all kernel threads that can be used for debugging purposes.
|
||||
|
||||
=== Thread Migration
|
||||
|
||||
Simply put, a thread migrates when it moves from one CPU to another. In a non-preemptive kernel this can only happen at well-defined points such as when calling `msleep` or returning to userland. However, in the preemptive kernel, an interrupt can force a preemption and possible migration at any time. This can have negative affects on per-CPU data since with the exception of `curthread` and `curpcb` the data can change whenever you migrate. Since you can potentially migrate at any time this renders unprotected per-CPU data access rather useless. Thus it is desirable to be able to disable migration for sections of code that need per-CPU data to be stable.
|
||||
|
||||
Critical sections currently prevent migration since they do not allow context switches. However, this may be too strong of a requirement to enforce in some cases since a critical section also effectively blocks interrupt threads on the current processor. As a result, another API has been provided to allow the current thread to indicate that if it preempted it should not migrate to another CPU.
|
||||
|
||||
This API is known as thread pinning and is provided by the scheduler. The API consists of two functions: `sched_pin` and `sched_unpin`. These functions manage a per-thread nesting count `td_pinned`. A thread is pinned when its nesting count is greater than zero and a thread starts off unpinned with a nesting count of zero. Each scheduler implementation is required to ensure that pinned threads are only executed on the CPU that they were executing on when the `sched_pin` was first called. Since the nesting count is only written to by the thread itself and is only read by other threads when the pinned thread is not executing but while `sched_lock` is held, then `td_pinned` does not need any locking. The `sched_pin` function increments the nesting count and `sched_unpin` decrements the nesting count. Note that these functions only operate on the current thread and bind the current thread to the CPU it is executing on at the time. To bind an arbitrary thread to a specific CPU, the `sched_bind` and `sched_unbind` functions should be used instead.
|
||||
|
||||
=== Callouts
|
||||
|
||||
The `timeout` kernel facility permits kernel services to register functions for execution as part of the `softclock` software interrupt. Events are scheduled based on a desired number of clock ticks, and callbacks to the consumer-provided function will occur at approximately the right time.
|
||||
|
||||
The global list of pending timeout events is protected by a global spin mutex, `callout_lock`; all access to the timeout list must be performed with this mutex held. When `softclock` is woken up, it scans the list of pending timeouts for those that should fire. In order to avoid lock order reversal, the `softclock` thread will release the `callout_lock` mutex when invoking the provided `timeout` callback function. If the `CALLOUT_MPSAFE` flag was not set during registration, then Giant will be grabbed before invoking the callout, and then released afterwards. The `callout_lock` mutex will be re-grabbed before proceeding. The `softclock` code is careful to leave the list in a consistent state while releasing the mutex. If `DIAGNOSTIC` is enabled, then the time taken to execute each function is measured, and a warning is generated if it exceeds a threshold.
|
||||
|
||||
[[smp-lock-strategies]]
|
||||
== Specific Locking Strategies
|
||||
|
||||
=== Credentials
|
||||
|
||||
`struct ucred` is the kernel's internal credential structure, and is generally used as the basis for process-driven access control within the kernel. BSD-derived systems use a "copy-on-write" model for credential data: multiple references may exist for a credential structure, and when a change needs to be made, the structure is duplicated, modified, and then the reference replaced. Due to wide-spread caching of the credential to implement access control on open, this results in substantial memory savings. With a move to fine-grained SMP, this model also saves substantially on locking operations by requiring that modification only occur on an unshared credential, avoiding the need for explicit synchronization when consuming a known-shared credential.
|
||||
|
||||
Credential structures with a single reference are considered mutable; shared credential structures must not be modified or a race condition is risked. A mutex, `cr_mtxp` protects the reference count of `struct ucred` so as to maintain consistency. Any use of the structure requires a valid reference for the duration of the use, or the structure may be released out from under the illegitimate consumer.
|
||||
|
||||
The `struct ucred` mutex is a leaf mutex and is implemented via a mutex pool for performance reasons.
|
||||
|
||||
Usually, credentials are used in a read-only manner for access control decisions, and in this case `td_ucred` is generally preferred because it requires no locking. When a process' credential is updated the `proc` lock must be held across the check and update operations thus avoid races. The process credential `p_ucred` must be used for check and update operations to prevent time-of-check, time-of-use races.
|
||||
|
||||
If system call invocations will perform access control after an update to the process credential, the value of `td_ucred` must also be refreshed to the current process value. This will prevent use of a stale credential following a change. The kernel automatically refreshes the `td_ucred` pointer in the thread structure from the process `p_ucred` whenever a process enters the kernel, permitting use of a fresh credential for kernel access control.
|
||||
|
||||
=== File Descriptors and File Descriptor Tables
|
||||
|
||||
Details to follow.
|
||||
|
||||
=== Jail Structures
|
||||
|
||||
`struct prison` stores administrative details pertinent to the maintenance of jails created using the man:jail[2] API. This includes the per-jail hostname, IP address, and related settings. This structure is reference-counted since pointers to instances of the structure are shared by many credential structures. A single mutex, `pr_mtx` protects read and write access to the reference count and all mutable variables inside the struct jail. Some variables are set only when the jail is created, and a valid reference to the `struct prison` is sufficient to read these values. The precise locking of each entry is documented via comments in [.filename]#sys/jail.h#.
|
||||
|
||||
=== MAC Framework
|
||||
|
||||
The TrustedBSD MAC Framework maintains data in a variety of kernel objects, in the form of `struct label`. In general, labels in kernel objects are protected by the same lock as the remainder of the kernel object. For example, the `v_label` label in `struct vnode` is protected by the vnode lock on the vnode.
|
||||
|
||||
In addition to labels maintained in standard kernel objects, the MAC Framework also maintains a list of registered and active policies. The policy list is protected by a global mutex (`mac_policy_list_lock`) and a busy count (also protected by the mutex). Since many access control checks may occur in parallel, entry to the framework for a read-only access to the policy list requires holding the mutex while incrementing (and later decrementing) the busy count. The mutex need not be held for the duration of the MAC entry operation--some operations, such as label operations on file system objects--are long-lived. To modify the policy list, such as during policy registration and de-registration, the mutex must be held and the reference count must be zero, to prevent modification of the list while it is in use.
|
||||
|
||||
A condition variable, `mac_policy_list_not_busy`, is available to threads that need to wait for the list to become unbusy, but this condition variable must only be waited on if the caller is holding no other locks, or a lock order violation may be possible. The busy count, in effect, acts as a form of shared/exclusive lock over access to the framework: the difference is that, unlike with an sx lock, consumers waiting for the list to become unbusy may be starved, rather than permitting lock order problems with regards to the busy count and other locks that may be held on entry to (or inside) the MAC Framework.
|
||||
|
||||
=== Modules
|
||||
|
||||
For the module subsystem there exists a single lock that is used to protect the shared data. This lock is a shared/exclusive (SX) lock and has a good chance of needing to be acquired (shared or exclusively), therefore there are a few macros that have been added to make access to the lock more easy. These macros can be located in [.filename]#sys/module.h# and are quite basic in terms of usage. The main structures protected under this lock are the `module_t` structures (when shared) and the global `modulelist_t` structure, modules. One should review the related source code in [.filename]#kern/kern_module.c# to further understand the locking strategy.
|
||||
|
||||
=== Newbus Device Tree
|
||||
|
||||
The newbus system will have one sx lock. Readers will hold a shared (read) lock (man:sx_slock[9]) and writers will hold an exclusive (write) lock (man:sx_xlock[9]). Internal functions will not do locking at all. Externally visible ones will lock as needed. Those items that do not matter if the race is won or lost will not be locked, since they tend to be read all over the place (e.g., man:device_get_softc[9]). There will be relatively few changes to the newbus data structures, so a single lock should be sufficient and not impose a performance penalty.
|
||||
|
||||
=== Pipes
|
||||
|
||||
...
|
||||
|
||||
=== Processes and Threads
|
||||
|
||||
- process hierarchy
|
||||
|
||||
- proc locks, references
|
||||
|
||||
- thread-specific copies of proc entries to freeze during system calls, including td_ucred
|
||||
|
||||
- inter-process operations
|
||||
|
||||
- process groups and sessions
|
||||
|
||||
=== Scheduler
|
||||
|
||||
Lots of references to `sched_lock` and notes pointing at specific primitives and related magic elsewhere in the document.
|
||||
|
||||
=== Select and Poll
|
||||
|
||||
The `select` and `poll` functions permit threads to block waiting on events on file descriptors--most frequently, whether or not the file descriptors are readable or writable.
|
||||
|
||||
...
|
||||
|
||||
=== SIGIO
|
||||
|
||||
The SIGIO service permits processes to request the delivery of a SIGIO signal to its process group when the read/write status of specified file descriptors changes. At most one process or process group is permitted to register for SIGIO from any given kernel object, and that process or group is referred to as the owner. Each object supporting SIGIO registration contains pointer field that is `NULL` if the object is not registered, or points to a `struct sigio` describing the registration. This field is protected by a global mutex, `sigio_lock`. Callers to SIGIO maintenance functions must pass in this field "by reference" so that local register copies of the field are not made when unprotected by the lock.
|
||||
|
||||
One `struct sigio` is allocated for each registered object associated with any process or process group, and contains back-pointers to the object, owner, signal information, a credential, and the general disposition of the registration. Each process or progress group contains a list of registered `struct sigio` structures, `p_sigiolst` for processes, and `pg_sigiolst` for process groups. These lists are protected by the process or process group locks respectively. Most fields in each `struct sigio` are constant for the duration of the registration, with the exception of the `sio_pgsigio` field which links the `struct sigio` into the process or process group list. Developers implementing new kernel objects supporting SIGIO will, in general, want to avoid holding structure locks while invoking SIGIO supporting functions, such as `fsetown` or `funsetown` to avoid defining a lock order between structure locks and the global SIGIO lock. This is generally possible through use of an elevated reference count on the structure, such as reliance on a file descriptor reference to a pipe during a pipe operation.
|
||||
|
||||
=== Sysctl
|
||||
|
||||
The `sysctl` MIB service is invoked from both within the kernel and from userland applications using a system call. At least two issues are raised in locking: first, the protection of the structures maintaining the namespace, and second, interactions with kernel variables and functions that are accessed by the sysctl interface. Since sysctl permits the direct export (and modification) of kernel statistics and configuration parameters, the sysctl mechanism must become aware of appropriate locking semantics for those variables. Currently, sysctl makes use of a single global sx lock to serialize use of `sysctl`; however, it is assumed to operate under Giant and other protections are not provided. The remainder of this section speculates on locking and semantic changes to sysctl.
|
||||
|
||||
- Need to change the order of operations for sysctl's that update values from read old, copyin and copyout, write new to copyin, lock, read old and write new, unlock, copyout. Normal sysctl's that just copyout the old value and set a new value that they copyin may still be able to follow the old model. However, it may be cleaner to use the second model for all of the sysctl handlers to avoid lock operations.
|
||||
|
||||
- To allow for the common case, a sysctl could embed a pointer to a mutex in the SYSCTL_FOO macros and in the struct. This would work for most sysctl's. For values protected by sx locks, spin mutexes, or other locking strategies besides a single sleep mutex, SYSCTL_PROC nodes could be used to get the locking right.
|
||||
|
||||
=== Taskqueue
|
||||
|
||||
The taskqueue's interface has two basic locks associated with it in order to protect the related shared data. The `taskqueue_queues_mutex` is meant to serve as a lock to protect the `taskqueue_queues` TAILQ. The other mutex lock associated with this system is the one in the `struct taskqueue` data structure. The use of the synchronization primitive here is to protect the integrity of the data in the `struct taskqueue`. It should be noted that there are no separate macros to assist the user in locking down his/her own work since these locks are most likely not going to be used outside of [.filename]#kern/subr_taskqueue.c#.
|
||||
|
||||
[[smp-implementation-notes]]
|
||||
== Implementation Notes
|
||||
|
||||
=== Sleep Queues
|
||||
|
||||
A sleep queue is a structure that holds the list of threads asleep on a wait channel. Each thread that is not asleep on a wait channel carries a sleep queue structure around with it. When a thread blocks on a wait channel, it donates its sleep queue structure to that wait channel. Sleep queues associated with a wait channel are stored in a hash table.
|
||||
|
||||
The sleep queue hash table holds sleep queues for wait channels that have at least one blocked thread. Each entry in the hash table is called a sleepqueue chain. The chain contains a linked list of sleep queues and a spin mutex. The spin mutex protects the list of sleep queues as well as the contents of the sleep queue structures on the list. Only one sleep queue is associated with a given wait channel. If multiple threads block on a wait channel than the sleep queues associated with all but the first thread are stored on a list of free sleep queues in the master sleep queue. When a thread is removed from the sleep queue it is given one of the sleep queue structures from the master queue's free list if it is not the only thread asleep on the queue. The last thread is given the master sleep queue when it is resumed. Since threads may be removed from the sleep queue in a different order than they are added, a thread may depart from a sleep queue with a different sleep queue structure than the one it arrived with.
|
||||
|
||||
The `sleepq_lock` function locks the spin mutex of the sleep queue chain that maps to a specific wait channel. The `sleepq_lookup` function looks in the hash table for the master sleep queue associated with a given wait channel. If no master sleep queue is found, it returns `NULL`. The `sleepq_release` function unlocks the spin mutex associated with a given wait channel.
|
||||
|
||||
A thread is added to a sleep queue via the `sleepq_add`. This function accepts the wait channel, a pointer to the mutex that protects the wait channel, a wait message description string, and a mask of flags. The sleep queue chain should be locked via `sleepq_lock` before this function is called. If no mutex protects the wait channel (or it is protected by Giant), then the mutex pointer argument should be `NULL`. The flags argument contains a type field that indicates the kind of sleep queue that the thread is being added to and a flag to indicate if the sleep is interruptible (`SLEEPQ_INTERRUPTIBLE`). Currently there are only two types of sleep queues: traditional sleep queues managed via the `msleep` and `wakeup` functions (`SLEEPQ_MSLEEP`) and condition variable sleep queues (`SLEEPQ_CONDVAR`). The sleep queue type and lock pointer argument are used solely for internal assertion checking. Code that calls `sleepq_add` should explicitly unlock any interlock protecting the wait channel after the associated sleepqueue chain has been locked via `sleepq_lock` and before blocking on the sleep queue via one of the waiting functions.
|
||||
|
||||
A timeout for a sleep is set by invoking `sleepq_set_timeout`. The function accepts the wait channel and the timeout time as a relative tick count as its arguments. If a sleep should be interrupted by arriving signals, the `sleepq_catch_signals` function should be called as well. This function accepts the wait channel as its only parameter. If there is already a signal pending for this thread, then `sleepq_catch_signals` will return a signal number; otherwise, it will return 0.
|
||||
|
||||
Once a thread has been added to a sleep queue, it blocks using one of the `sleepq_wait` functions. There are four wait functions depending on whether or not the caller wishes to use a timeout or have the sleep aborted by caught signals or an interrupt from the userland thread scheduler. The `sleepq_wait` function simply waits until the current thread is explicitly resumed by one of the wakeup functions. The `sleepq_timedwait` function waits until either the thread is explicitly resumed or the timeout set by an earlier call to `sleepq_set_timeout` expires. The `sleepq_wait_sig` function waits until either the thread is explicitly resumed or its sleep is aborted. The `sleepq_timedwait_sig` function waits until either the thread is explicitly resumed, the timeout set by an earlier call to `sleepq_set_timeout` expires, or the thread's sleep is aborted. All of the wait functions accept the wait channel as their first parameter. In addition, the `sleepq_timedwait_sig` function accepts a second boolean parameter to indicate if the earlier call to `sleepq_catch_signals` found a pending signal.
|
||||
|
||||
If the thread is explicitly resumed or is aborted by a signal, then a value of zero is returned by the wait function to indicate a successful sleep. If the thread is resumed by either a timeout or an interrupt from the userland thread scheduler then an appropriate errno value is returned instead. Note that since `sleepq_wait` can only return 0 it does not return anything and the caller should assume a successful sleep. Also, if a thread's sleep times out and is aborted simultaneously then `sleepq_timedwait_sig` will return an error indicating that a timeout occurred. If an error value of 0 is returned and either `sleepq_wait_sig` or `sleepq_timedwait_sig` was used to block, then the function `sleepq_calc_signal_retval` should be called to check for any pending signals and calculate an appropriate return value if any are found. The signal number returned by the earlier call to `sleepq_catch_signals` should be passed as the sole argument to `sleepq_calc_signal_retval`.
|
||||
|
||||
Threads asleep on a wait channel are explicitly resumed by the `sleepq_broadcast` and `sleepq_signal` functions. Both functions accept the wait channel from which to resume threads, a priority to raise resumed threads to, and a flags argument to indicate which type of sleep queue is being resumed. The priority argument is treated as a minimum priority. If a thread being resumed already has a higher priority (numerically lower) than the priority argument then its priority is not adjusted. The flags argument is used for internal assertions to ensure that sleep queues are not being treated as the wrong type. For example, the condition variable functions should not resume threads on a traditional sleep queue. The `sleepq_broadcast` function resumes all threads that are blocked on the specified wait channel while `sleepq_signal` only resumes the highest priority thread blocked on the wait channel. The sleep queue chain should first be locked via the `sleepq_lock` function before calling these functions.
|
||||
|
||||
A sleeping thread may have its sleep interrupted by calling the `sleepq_abort` function. This function must be called with `sched_lock` held and the thread must be queued on a sleep queue. A thread may also be removed from a specific sleep queue via the `sleepq_remove` function. This function accepts both a thread and a wait channel as an argument and only awakens the thread if it is on the sleep queue for the specified wait channel. If the thread is not on a sleep queue or it is on a sleep queue for a different wait channel, then this function does nothing.
|
||||
|
||||
=== Turnstiles
|
||||
|
||||
- Compare/contrast with sleep queues.
|
||||
|
||||
- Lookup/wait/release. - Describe TDF_TSNOBLOCK race.
|
||||
|
||||
- Priority propagation.
|
||||
|
||||
=== Details of the Mutex Implementation
|
||||
|
||||
- Should we require mutexes to be owned for mtx_destroy() since we can not safely assert that they are unowned by anyone else otherwise?
|
||||
|
||||
==== Spin Mutexes
|
||||
|
||||
- Use a critical section...
|
||||
|
||||
==== Sleep Mutexes
|
||||
|
||||
- Describe the races with contested mutexes
|
||||
|
||||
- Why it is safe to read mtx_lock of a contested mutex when holding the turnstile chain lock.
|
||||
|
||||
=== Witness
|
||||
|
||||
- What does it do
|
||||
|
||||
- How does it work
|
||||
|
||||
[[smp-misc]]
|
||||
== Miscellaneous Topics
|
||||
|
||||
=== Interrupt Source and ICU Abstractions
|
||||
|
||||
- struct isrc
|
||||
|
||||
- pic drivers
|
||||
|
||||
=== Other Random Questions/Topics
|
||||
|
||||
- Should we pass an interlock into `sema_wait`?
|
||||
|
||||
- Should we have non-sleepable sx locks?
|
||||
|
||||
- Add some info about proper use of reference counts.
|
||||
|
||||
:sectnums!:
|
||||
|
||||
[glossary]
|
||||
[[smp-glossary]]
|
||||
== Glossary
|
||||
|
||||
[.glosslist]
|
||||
atomic::
|
||||
An operation is atomic if all of its effects are visible to other CPUs together when the proper access protocol is followed. In the degenerate case are atomic instructions provided directly by machine architectures. At a higher level, if several members of a structure are protected by a lock, then a set of operations are atomic if they are all performed while holding the lock without releasing the lock in between any of the operations.
|
||||
+
|
||||
See Also operation.
|
||||
|
||||
block::
|
||||
A thread is blocked when it is waiting on a lock, resource, or condition. Unfortunately this term is a bit overloaded as a result.
|
||||
+
|
||||
See Also sleep.
|
||||
|
||||
critical section::
|
||||
A section of code that is not allowed to be preempted. A critical section is entered and exited using the man:critical_enter[9] API.
|
||||
|
||||
MD::
|
||||
Machine dependent.
|
||||
+
|
||||
See Also MI.
|
||||
|
||||
memory operation::
|
||||
A memory operation reads and/or writes to a memory location.
|
||||
|
||||
MI::
|
||||
Machine independent.
|
||||
+
|
||||
See Also MD.
|
||||
|
||||
operation::
|
||||
See memory operation.
|
||||
|
||||
primary interrupt context::
|
||||
Primary interrupt context refers to the code that runs when an interrupt occurs. This code can either run an interrupt handler directly or schedule an asynchronous interrupt thread to execute the interrupt handlers for a given interrupt source.
|
||||
|
||||
realtime kernel thread::
|
||||
A high priority kernel thread. Currently, the only realtime priority kernel threads are interrupt threads.
|
||||
+
|
||||
See Also thread.
|
||||
|
||||
sleep::
|
||||
A thread is asleep when it is blocked on a condition variable or a sleep queue via msleep or tsleep.
|
||||
+
|
||||
See Also block.
|
||||
|
||||
sleepable lock::
|
||||
A sleepable lock is a lock that can be held by a thread which is asleep. Lockmgr locks and sx locks are currently the only sleepable locks in FreeBSD. Eventually, some sx locks such as the allproc and proctree locks may become non-sleepable locks.
|
||||
+
|
||||
See Also sleep.
|
||||
|
||||
thread::
|
||||
A kernel thread represented by a struct thread. Threads own locks and hold a single execution context.
|
||||
|
||||
wait channel::
|
||||
A kernel virtual address that threads may sleep on.
|
||||
|
||||
:sectnums:
|
||||
332
documentation/content/en/books/arch-handbook/sound/chapter.adoc
Normal file
332
documentation/content/en/books/arch-handbook/sound/chapter.adoc
Normal file
|
|
@ -0,0 +1,332 @@
|
|||
---
|
||||
title: Chapter 15. Sound Subsystem
|
||||
prev: books/arch-handbook/newbus
|
||||
next: books/arch-handbook/pccard
|
||||
---
|
||||
|
||||
[[oss]]
|
||||
= Sound Subsystem
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 15
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[oss-intro]]
|
||||
== Introduction
|
||||
|
||||
The FreeBSD sound subsystem cleanly separates generic sound handling issues from device-specific ones. This makes it easier to add support for new hardware.
|
||||
|
||||
The man:pcm[4] framework is the central piece of the sound subsystem. It mainly implements the following elements:
|
||||
|
||||
* A system call interface (read, write, ioctls) to digitized sound and mixer functions. The ioctl command set is compatible with the legacy _OSS_ or _Voxware_ interface, allowing common multimedia applications to be ported without modification.
|
||||
* Common code for processing sound data (format conversions, virtual channels).
|
||||
* A uniform software interface to hardware-specific audio interface modules.
|
||||
* Additional support for some common hardware interfaces (ac97), or shared hardware-specific code (ex: ISA DMA routines).
|
||||
|
||||
The support for specific sound cards is implemented by hardware-specific drivers, which provide channel and mixer interfaces to plug into the generic [.filename]#pcm# code.
|
||||
|
||||
In this chapter, the term [.filename]#pcm# will refer to the central, common part of the sound driver, as opposed to the hardware-specific modules.
|
||||
|
||||
The prospective driver writer will of course want to start from an existing module and use the code as the ultimate reference. But, while the sound code is nice and clean, it is also mostly devoid of comments. This document tries to give an overview of the framework interface and answer some questions that may arise while adapting the existing code.
|
||||
|
||||
As an alternative, or in addition to starting from a working example, you can find a commented driver template at https://people.FreeBSD.org/~cg/template.c[ https://people.FreeBSD.org/~cg/template.c]
|
||||
|
||||
[[oss-files]]
|
||||
== Files
|
||||
|
||||
All the relevant code lives in [.filename]#/usr/src/sys/dev/sound/#, except for the public ioctl interface definitions, found in [.filename]#/usr/src/sys/sys/soundcard.h#
|
||||
|
||||
Under [.filename]#/usr/src/sys/dev/sound/#, the [.filename]#pcm/# directory holds the central code, while the [.filename]#pci/#, [.filename]#isa/# and [.filename]#usb/# directories have the drivers for PCI and ISA boards, and for USB audio devices.
|
||||
|
||||
[[pcm-probe-and-attach]]
|
||||
== Probing, Attaching, etc.
|
||||
|
||||
Sound drivers probe and attach in almost the same way as any hardware driver module. You might want to look at the <<isa-driver,ISA>> or <<pci,PCI>> specific sections of the handbook for more information.
|
||||
|
||||
However, sound drivers differ in some ways:
|
||||
|
||||
* They declare themselves as [.filename]#pcm# class devices, with a `struct snddev_info` device private structure:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
static driver_t xxx_driver = {
|
||||
"pcm",
|
||||
xxx_methods,
|
||||
sizeof(struct snddev_info)
|
||||
};
|
||||
|
||||
DRIVER_MODULE(snd_xxxpci, pci, xxx_driver, pcm_devclass, 0, 0);
|
||||
MODULE_DEPEND(snd_xxxpci, snd_pcm, PCM_MINVER, PCM_PREFVER,PCM_MAXVER);
|
||||
....
|
||||
+
|
||||
Most sound drivers need to store additional private information about their device. A private data structure is usually allocated in the attach routine. Its address is passed to [.filename]#pcm# by the calls to `pcm_register()` and `mixer_init()`. [.filename]#pcm# later passes back this address as a parameter in calls to the sound driver interfaces.
|
||||
* The sound driver attach routine should declare its MIXER or AC97 interface to [.filename]#pcm# by calling `mixer_init()`. For a MIXER interface, this causes in turn a call to <<xxxmixer-init,`xxxmixer_init()`>>.
|
||||
* The sound driver attach routine declares its general CHANNEL configuration to [.filename]#pcm# by calling `pcm_register(dev, sc, nplay, nrec)`, where `sc` is the address for the device data structure, used in further calls from [.filename]#pcm#, and `nplay` and `nrec` are the number of play and record channels.
|
||||
* The sound driver attach routine declares each of its channel objects by calls to `pcm_addchan()`. This sets up the channel glue in [.filename]#pcm# and causes in turn a call to <<xxxchannel-init,`xxxchannel_init()`>>.
|
||||
* The sound driver detach routine should call `pcm_unregister()` before releasing its resources.
|
||||
|
||||
There are two possible methods to handle non-PnP devices:
|
||||
|
||||
* Use a `device_identify()` method (example: [.filename]#sound/isa/es1888.c#). The `device_identify()` method probes for the hardware at known addresses and, if it finds a supported device, creates a new pcm device which is then passed to probe/attach.
|
||||
* Use a custom kernel configuration with appropriate hints for pcm devices (example: [.filename]#sound/isa/mss.c#).
|
||||
|
||||
[.filename]#pcm# drivers should implement `device_suspend`, `device_resume` and `device_shutdown` routines, so that power management and module unloading function correctly.
|
||||
|
||||
[[oss-interfaces]]
|
||||
== Interfaces
|
||||
|
||||
The interface between the [.filename]#pcm# core and the sound drivers is defined in terms of <<kernel-objects,kernel objects>>.
|
||||
|
||||
There are two main interfaces that a sound driver will usually provide: _CHANNEL_ and either _MIXER_ or _AC97_.
|
||||
|
||||
The _AC97_ interface is a very small hardware access (register read/write) interface, implemented by drivers for hardware with an AC97 codec. In this case, the actual MIXER interface is provided by the shared AC97 code in [.filename]#pcm#.
|
||||
|
||||
=== The CHANNEL Interface
|
||||
|
||||
==== Common Notes for Function Parameters
|
||||
|
||||
Sound drivers usually have a private data structure to describe their device, and one structure for each play and record data channel that it supports.
|
||||
|
||||
For all CHANNEL interface functions, the first parameter is an opaque pointer.
|
||||
|
||||
The second parameter is a pointer to the private channel data structure, except for `channel_init()` which has a pointer to the private device structure (and returns the channel pointer for further use by [.filename]#pcm#).
|
||||
|
||||
==== Overview of Data Transfer Operations
|
||||
|
||||
For sound data transfers, the [.filename]#pcm# core and the sound drivers communicate through a shared memory area, described by a `struct snd_dbuf`.
|
||||
|
||||
`struct snd_dbuf` is private to [.filename]#pcm#, and sound drivers obtain values of interest by calls to accessor functions (`sndbuf_getxxx()`).
|
||||
|
||||
The shared memory area has a size of `sndbuf_getsize()` and is divided into fixed size blocks of `sndbuf_getblksz()` bytes.
|
||||
|
||||
When playing, the general transfer mechanism is as follows (reverse the idea for recording):
|
||||
|
||||
* [.filename]#pcm# initially fills up the buffer, then calls the sound driver's <<channel-trigger,`xxxchannel_trigger()`>> function with a parameter of PCMTRIG_START.
|
||||
* The sound driver then arranges to repeatedly transfer the whole memory area (`sndbuf_getbuf()`, `sndbuf_getsize()`) to the device, in blocks of `sndbuf_getblksz()` bytes. It calls back the `chn_intr()`[.filename]#pcm# function for each transferred block (this will typically happen at interrupt time).
|
||||
* `chn_intr()` arranges to copy new data to the area that was transferred to the device (now free), and make appropriate updates to the `snd_dbuf` structure.
|
||||
|
||||
[[xxxchannel-init]]
|
||||
==== channel_init
|
||||
|
||||
`xxxchannel_init()` is called to initialize each of the play or record channels. The calls are initiated from the sound driver attach routine. (See the <<pcm-probe-and-attach,probe and attach section>>).
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static void *
|
||||
xxxchannel_init(kobj_t obj, void *data,
|
||||
struct snd_dbuf *b, struct pcm_channel *c, int dir) <.>
|
||||
{
|
||||
struct xxx_info *sc = data;
|
||||
struct xxx_chinfo *ch;
|
||||
...
|
||||
return ch; <.>
|
||||
}
|
||||
....
|
||||
|
||||
<.> `b` is the address for the channel `struct snd_dbuf`. It should be initialized in the function by calling `sndbuf_alloc()`. The buffer size to use is normally a small multiple of the 'typical' unit transfer size for your device.`c` is the [.filename]#pcm# channel control structure pointer. This is an opaque object. The function should store it in the local channel structure, to be used in later calls to [.filename]#pcm# (ie: `chn_intr(c)`).`dir` indicates the channel direction (`PCMDIR_PLAY` or `PCMDIR_REC`).
|
||||
|
||||
<.> The function should return a pointer to the private area used to control this channel. This will be passed as a parameter to other channel interface calls.
|
||||
|
||||
==== channel_setformat
|
||||
|
||||
`xxxchannel_setformat()` should set up the hardware for the specified channel for the specified sound format.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxchannel_setformat(kobj_t obj, void *data, u_int32_t format) <.>
|
||||
{
|
||||
struct xxx_chinfo *ch = data;
|
||||
...
|
||||
return 0;
|
||||
}
|
||||
....
|
||||
|
||||
<.> `format` is specified as an `AFMT_XXX value` ([.filename]#soundcard.h#).
|
||||
|
||||
==== channel_setspeed
|
||||
|
||||
`xxxchannel_setspeed()` sets up the channel hardware for the specified sampling speed, and returns the possibly adjusted speed.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxchannel_setspeed(kobj_t obj, void *data, u_int32_t speed)
|
||||
{
|
||||
struct xxx_chinfo *ch = data;
|
||||
...
|
||||
return speed;
|
||||
}
|
||||
....
|
||||
|
||||
==== channel_setblocksize
|
||||
|
||||
`xxxchannel_setblocksize()` sets the block size, which is the size of unit transactions between [.filename]#pcm# and the sound driver, and between the sound driver and the device. Typically, this would be the number of bytes transferred before an interrupt occurs. During a transfer, the sound driver should call [.filename]#pcm#'s `chn_intr()` every time this size has been transferred.
|
||||
|
||||
Most sound drivers only take note of the block size here, to be used when an actual transfer will be started.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxchannel_setblocksize(kobj_t obj, void *data, u_int32_t blocksize)
|
||||
{
|
||||
struct xxx_chinfo *ch = data;
|
||||
...
|
||||
return blocksize; <.>
|
||||
}
|
||||
....
|
||||
|
||||
<.> The function returns the possibly adjusted block size. In case the block size is indeed changed, `sndbuf_resize()` should be called to adjust the buffer.
|
||||
|
||||
[[channel-trigger]]
|
||||
==== channel_trigger
|
||||
|
||||
`xxxchannel_trigger()` is called by [.filename]#pcm# to control data transfer operations in the driver.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxchannel_trigger(kobj_t obj, void *data, int go) <.>
|
||||
{
|
||||
struct xxx_chinfo *ch = data;
|
||||
...
|
||||
return 0;
|
||||
}
|
||||
....
|
||||
|
||||
<.> `go` defines the action for the current call. The possible values are:
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
If the driver uses ISA DMA, `sndbuf_isadma()` should be called before performing actions on the device, and will take care of the DMA chip side of things.
|
||||
====
|
||||
|
||||
==== channel_getptr
|
||||
|
||||
`xxxchannel_getptr()` returns the current offset in the transfer buffer. This will typically be called by `chn_intr()`, and this is how [.filename]#pcm# knows where it can transfer new data.
|
||||
|
||||
==== channel_free
|
||||
|
||||
`xxxchannel_free()` is called to free up channel resources, for example when the driver is unloaded, and should be implemented if the channel data structures are dynamically allocated or if `sndbuf_alloc()` was not used for buffer allocation.
|
||||
|
||||
==== channel_getcaps
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct pcmchan_caps *
|
||||
xxxchannel_getcaps(kobj_t obj, void *data)
|
||||
{
|
||||
return &xxx_caps; <.>
|
||||
}
|
||||
....
|
||||
|
||||
<.> The routine returns a pointer to a (usually statically-defined) `pcmchan_caps` structure (defined in [.filename]#sound/pcm/channel.h#. The structure holds the minimum and maximum sampling frequencies, and the accepted sound formats. Look at any sound driver for an example.
|
||||
|
||||
==== More Functions
|
||||
|
||||
`channel_reset()`, `channel_resetdone()`, and `channel_notify()` are for special purposes and should not be implemented in a driver without discussing it on the {freebsd-multimedia}.
|
||||
|
||||
`channel_setdir()` is deprecated.
|
||||
|
||||
=== The MIXER Interface
|
||||
|
||||
[[xxxmixer-init]]
|
||||
==== mixer_init
|
||||
|
||||
`xxxmixer_init()` initializes the hardware and tells [.filename]#pcm# what mixer devices are available for playing and recording
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxmixer_init(struct snd_mixer *m)
|
||||
{
|
||||
struct xxx_info *sc = mix_getdevinfo(m);
|
||||
u_int32_t v;
|
||||
|
||||
[Initialize hardware]
|
||||
|
||||
[Set appropriate bits in v for play mixers] <.>
|
||||
mix_setdevs(m, v);
|
||||
[Set appropriate bits in v for record mixers]
|
||||
mix_setrecdevs(m, v)
|
||||
|
||||
return 0;
|
||||
}
|
||||
....
|
||||
|
||||
<.> Set bits in an integer value and call `mix_setdevs()` and `mix_setrecdevs()` to tell [.filename]#pcm# what devices exist.
|
||||
|
||||
Mixer bits definitions can be found in [.filename]#soundcard.h# (`SOUND_MASK_XXX` values and `SOUND_MIXER_XXX` bit shifts).
|
||||
|
||||
==== mixer_set
|
||||
|
||||
`xxxmixer_set()` sets the volume level for one mixer device.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxmixer_set(struct snd_mixer *m, unsigned dev,
|
||||
unsigned left, unsigned right) <.>
|
||||
{
|
||||
struct sc_info *sc = mix_getdevinfo(m);
|
||||
[set volume level]
|
||||
return left | (right << 8); <.>
|
||||
}
|
||||
....
|
||||
|
||||
<.> The device is specified as a `SOUND_MIXER_XXX` value. The volume values are specified in range [0-100]. A value of zero should mute the device.
|
||||
<.> As the hardware levels probably will not match the input scale, and some rounding will occur, the routine returns the actual level values (in range 0-100) as shown.
|
||||
|
||||
==== mixer_setrecsrc
|
||||
|
||||
`xxxmixer_setrecsrc()` sets the recording source device.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
static int
|
||||
xxxmixer_setrecsrc(struct snd_mixer *m, u_int32_t src) <.>
|
||||
{
|
||||
struct xxx_info *sc = mix_getdevinfo(m);
|
||||
|
||||
[look for non zero bit(s) in src, set up hardware]
|
||||
|
||||
[update src to reflect actual action]
|
||||
return src; <.>
|
||||
}
|
||||
....
|
||||
|
||||
<.> The desired recording devices are specified as a bit field
|
||||
<.> The actual devices set for recording are returned. Some drivers can only set one device for recording. The function should return -1 if an error occurs.
|
||||
|
||||
==== mixer_uninit, mixer_reinit
|
||||
|
||||
`xxxmixer_uninit()` should ensure that all sound is muted and if possible mixer hardware should be powered down.
|
||||
|
||||
`xxxmixer_reinit()` should ensure that the mixer hardware is powered up and any settings not controlled by `mixer_set()` or `mixer_setrecsrc()` are restored.
|
||||
|
||||
=== The AC97 Interface
|
||||
|
||||
The _AC97_ interface is implemented by drivers with an AC97 codec. It only has three methods:
|
||||
|
||||
* `xxxac97_init()` returns the number of ac97 codecs found.
|
||||
* `ac97_read()` and `ac97_write()` read or write a specified register.
|
||||
|
||||
The _AC97_ interface is used by the AC97 code in [.filename]#pcm# to perform higher level operations. Look at [.filename]#sound/pci/maestro3.c# or many others under [.filename]#sound/pci/# for an example.
|
||||
|
|
@ -0,0 +1,146 @@
|
|||
---
|
||||
title: Chapter 5. The SYSINIT Framework
|
||||
prev: books/arch-handbook/jail
|
||||
next: books/arch-handbook/mac
|
||||
---
|
||||
|
||||
[[sysinit]]
|
||||
= The SYSINIT Framework
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 5
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
SYSINIT is the framework for a generic call sort and dispatch mechanism. FreeBSD currently uses it for the dynamic initialization of the kernel. SYSINIT allows FreeBSD's kernel subsystems to be reordered, and added, removed, and replaced at kernel link time when the kernel or one of its modules is loaded without having to edit a statically ordered initialization routing and recompile the kernel. This system also allows kernel modules, currently called _KLD's_, to be separately compiled, linked, and initialized at boot time and loaded even later while the system is already running. This is accomplished using the "kernel linker" and "linker sets".
|
||||
|
||||
[[sysinit-term]]
|
||||
== Terminology
|
||||
|
||||
Linker Set::
|
||||
A linker technique in which the linker gathers statically declared data throughout a program's source files into a single contiguously addressable unit of data.
|
||||
|
||||
[[sysinit-operation]]
|
||||
== SYSINIT Operation
|
||||
|
||||
SYSINIT relies on the ability of the linker to take static data declared at multiple locations throughout a program's source and group it together as a single contiguous chunk of data. This linker technique is called a "linker set". SYSINIT uses two linker sets to maintain two data sets containing each consumer's call order, function, and a pointer to the data to pass to that function.
|
||||
|
||||
SYSINIT uses two priorities when ordering the functions for execution. The first priority is a subsystem ID giving an overall order for SYSINIT's dispatch of functions. Current predeclared ID's are in [.filename]#<sys/kernel.h># in the enum list `sysinit_sub_id`. The second priority used is an element order within the subsystem. Current predeclared subsystem element orders are in [.filename]#<sys/kernel.h># in the enum list `sysinit_elem_order`.
|
||||
|
||||
There are currently two uses for SYSINIT. Function dispatch at system startup and kernel module loads, and function dispatch at system shutdown and kernel module unload. Kernel subsystems often use system startup SYSINIT's to initialize data structures, for example the process scheduling subsystem uses a SYSINIT to initialize the run queue data structure. Device drivers should avoid using `SYSINIT()` directly. Instead drivers for real devices that are part of a bus structure should use `DRIVER_MODULE()` to provide a function that detects the device and, if it is present, initializes the device. It will do a few things specific to devices and then call `SYSINIT()` itself. For pseudo-devices, which are not part of a bus structure, use `DEV_MODULE()`.
|
||||
|
||||
[[sysinit-using]]
|
||||
== Using SYSINIT
|
||||
|
||||
=== Interface
|
||||
|
||||
==== Headers
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
<sys/kernel.h>
|
||||
....
|
||||
|
||||
==== Macros
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
SYSINIT(uniquifier, subsystem, order, func, ident)
|
||||
SYSUNINIT(uniquifier, subsystem, order, func, ident)
|
||||
....
|
||||
|
||||
=== Startup
|
||||
|
||||
The `SYSINIT()` macro creates the necessary SYSINIT data in SYSINIT's startup data set for SYSINIT to sort and dispatch a function at system startup and module load. `SYSINIT()` takes a uniquifier that SYSINIT uses to identify the particular function dispatch data, the subsystem order, the subsystem element order, the function to call, and the data to pass the function. All functions must take a constant pointer argument.
|
||||
|
||||
.Example of a `SYSINIT()`
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
#include <sys/kernel.h>
|
||||
|
||||
void foo_null(void *unused)
|
||||
{
|
||||
foo_doo();
|
||||
}
|
||||
SYSINIT(foo, SI_SUB_FOO, SI_ORDER_FOO, foo_null, NULL);
|
||||
|
||||
struct foo foo_voodoo = {
|
||||
FOO_VOODOO;
|
||||
}
|
||||
|
||||
void foo_arg(void *vdata)
|
||||
{
|
||||
struct foo *foo = (struct foo *)vdata;
|
||||
foo_data(foo);
|
||||
}
|
||||
SYSINIT(bar, SI_SUB_FOO, SI_ORDER_FOO, foo_arg, &foo_voodoo);
|
||||
....
|
||||
====
|
||||
|
||||
Note that `SI_SUB_FOO` and `SI_ORDER_FOO` need to be in the `sysinit_sub_id` and `sysinit_elem_order` enum's as mentioned above. Either use existing ones or add your own to the enum's. You can also use math for fine-tuning the order a SYSINIT will run in. This example shows a SYSINIT that needs to be run just barely before the SYSINIT's that handle tuning kernel parameters.
|
||||
|
||||
.Example of Adjusting `SYSINIT()` Order
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
static void
|
||||
mptable_register(void *dummy __unused)
|
||||
{
|
||||
|
||||
apic_register_enumerator(&mptable_enumerator);
|
||||
}
|
||||
|
||||
SYSINIT(mptable_register, SI_SUB_TUNABLES - 1, SI_ORDER_FIRST,
|
||||
mptable_register, NULL);
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
=== Shutdown
|
||||
|
||||
The `SYSUNINIT()` macro behaves similarly to the `SYSINIT()` macro except that it adds the SYSINIT data to SYSINIT's shutdown data set.
|
||||
|
||||
.Example of a `SYSUNINIT()`
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
#include <sys/kernel.h>
|
||||
|
||||
void foo_cleanup(void *unused)
|
||||
{
|
||||
foo_kill();
|
||||
}
|
||||
SYSUNINIT(foobar, SI_SUB_FOO, SI_ORDER_FOO, foo_cleanup, NULL);
|
||||
|
||||
struct foo_stack foo_stack = {
|
||||
FOO_STACK_VOODOO;
|
||||
}
|
||||
|
||||
void foo_flush(void *vdata)
|
||||
{
|
||||
}
|
||||
SYSUNINIT(barfoo, SI_SUB_FOO, SI_ORDER_FOO, foo_flush, &foo_stack);
|
||||
....
|
||||
|
||||
====
|
||||
167
documentation/content/en/books/arch-handbook/usb/chapter.adoc
Normal file
167
documentation/content/en/books/arch-handbook/usb/chapter.adoc
Normal file
|
|
@ -0,0 +1,167 @@
|
|||
---
|
||||
title: Chapter 13. USB Devices
|
||||
prev: books/arch-handbook/scsi
|
||||
next: books/arch-handbook/newbus
|
||||
---
|
||||
|
||||
[[usb]]
|
||||
= USB Devices
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 13
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[usb-intro]]
|
||||
== Introduction
|
||||
|
||||
The Universal Serial Bus (USB) is a new way of attaching devices to personal computers. The bus architecture features two-way communication and has been developed as a response to devices becoming smarter and requiring more interaction with the host. USB support is included in all current PC chipsets and is therefore available in all recently built PCs. Apple's introduction of the USB-only iMac has been a major incentive for hardware manufacturers to produce USB versions of their devices. The future PC specifications specify that all legacy connectors on PCs should be replaced by one or more USB connectors, providing generic plug and play capabilities. Support for USB hardware was available at a very early stage in NetBSD and was developed by Lennart Augustsson for the NetBSD project. The code has been ported to FreeBSD and we are currently maintaining a shared code base. For the implementation of the USB subsystem a number of features of USB are important.
|
||||
|
||||
_Lennart Augustsson has done most of the implementation of the USB support for the NetBSD project. Many thanks for this incredible amount of work. Many thanks also to Ardy and Dirk for their comments and proofreading of this paper._
|
||||
|
||||
* Devices connect to ports on the computer directly or on devices called hubs, forming a treelike device structure.
|
||||
* The devices can be connected and disconnected at run time.
|
||||
* Devices can suspend themselves and trigger resumes of the host system
|
||||
* As the devices can be powered from the bus, the host software has to keep track of power budgets for each hub.
|
||||
* Different quality of service requirements by the different device types together with the maximum of 126 devices that can be connected to the same bus, require proper scheduling of transfers on the shared bus to take full advantage of the 12Mbps bandwidth available. (over 400Mbps with USB 2.0)
|
||||
* Devices are intelligent and contain easily accessible information about themselves
|
||||
|
||||
The development of drivers for the USB subsystem and devices connected to it is supported by the specifications that have been developed and will be developed. These specifications are publicly available from the USB home pages. Apple has been very strong in pushing for standards based drivers, by making drivers for the generic classes available in their operating system MacOS and discouraging the use of separate drivers for each new device. This chapter tries to collate essential information for a basic understanding of the USB 2.0 implementation stack in FreeBSD/NetBSD. It is recommended however to read it together with the relevant 2.0 specifications and other developer resources:
|
||||
|
||||
* USB 2.0 Specification (http://www.usb.org/developers/docs/usb20_docs/[http://www.usb.org/developers/docs/usb20_docs/])
|
||||
* Universal Host Controller Interface (UHCI) Specification (link:ftp://ftp.netbsd.org/pub/NetBSD/misc/blymn/uhci11d.pdf[ftp://ftp.netbsd.org/pub/NetBSD/misc/blymn/uhci11d.pdf)]
|
||||
* Open Host Controller Interface (OHCI) Specification(link:ftp://ftp.compaq.com/pub/supportinformation/papers/hcir1_0a.pdf[ftp://ftp.compaq.com/pub/supportinformation/papers/hcir1_0a.pdf])
|
||||
* Developer section of USB home page (http://www.usb.org/developers/[http://www.usb.org/developers/])
|
||||
|
||||
=== Structure of the USB Stack
|
||||
|
||||
The USB support in FreeBSD can be split into three layers. The lowest layer contains the host controller driver, providing a generic interface to the hardware and its scheduling facilities. It supports initialisation of the hardware, scheduling of transfers and handling of completed and/or failed transfers. Each host controller driver implements a virtual hub providing hardware independent access to the registers controlling the root ports on the back of the machine.
|
||||
|
||||
The middle layer handles the device connection and disconnection, basic initialisation of the device, driver selection, the communication channels (pipes) and does resource management. This services layer also controls the default pipes and the device requests transferred over them.
|
||||
|
||||
The top layer contains the individual drivers supporting specific (classes of) devices. These drivers implement the protocol that is used over the pipes other than the default pipe. They also implement additional functionality to make the device available to other parts of the kernel or userland. They use the USB driver interface (USBDI) exposed by the services layer.
|
||||
|
||||
[[usb-hc]]
|
||||
== Host Controllers
|
||||
|
||||
The host controller (HC) controls the transmission of packets on the bus. Frames of 1 millisecond are used. At the start of each frame the host controller generates a Start of Frame (SOF) packet.
|
||||
|
||||
The SOF packet is used to synchronise to the start of the frame and to keep track of the frame number. Within each frame packets are transferred, either from host to device (out) or from device to host (in). Transfers are always initiated by the host (polled transfers). Therefore there can only be one host per USB bus. Each transfer of a packet has a status stage in which the recipient of the data can return either ACK (acknowledge reception), NAK (retry), STALL (error condition) or nothing (garbled data stage, device not available or disconnected). Section 8.5 of the USB 2.0 Specification explains the details of packets in more detail. Four different types of transfers can occur on a USB bus: control, bulk, interrupt and isochronous. The types of transfers and their characteristics are described below.
|
||||
|
||||
Large transfers between the device on the USB bus and the device driver are split up into multiple packets by the host controller or the HC driver.
|
||||
|
||||
Device requests (control transfers) to the default endpoints are special. They consist of two or three phases: SETUP, DATA (optional) and STATUS. The set-up packet is sent to the device. If there is a data phase, the direction of the data packet(s) is given in the set-up packet. The direction in the status phase is the opposite of the direction during the data phase, or IN if there was no data phase. The host controller hardware also provides registers with the current status of the root ports and the changes that have occurred since the last reset of the status change register. Access to these registers is provided through a virtualised hub as suggested in the USB specification. The virtual hub must comply with the hub device class given in chapter 11 of that specification. It must provide a default pipe through which device requests can be sent to it. It returns the standard andhub class specific set of descriptors. It should also provide an interrupt pipe that reports changes happening at its ports. There are currently two specifications for host controllers available: Universal Host Controller Interface (UHCI) from Intel and Open Host Controller Interface (OHCI) from Compaq, Microsoft, and National Semiconductor. The UHCI specification has been designed to reduce hardware complexity by requiring the host controller driver to supply a complete schedule of the transfers for each frame. OHCI type controllers are much more independent by providing a more abstract interface doing a lot of work themselves.
|
||||
|
||||
=== UHCI
|
||||
|
||||
The UHCI host controller maintains a framelist with 1024 pointers to per frame data structures. It understands two different data types: transfer descriptors (TD) and queue heads (QH). Each TD represents a packet to be communicated to or from a device endpoint. QHs are a means to groupTDs (and QHs) together.
|
||||
|
||||
Each transfer consists of one or more packets. The UHCI driver splits large transfers into multiple packets. For every transfer, apart from isochronous transfers, a QH is allocated. For every type of transfer these QHs are collected at a QH for that type. Isochronous transfers have to be executed first because of the fixed latency requirement and are directly referred to by the pointer in the framelist. The last isochronous TD refers to the QH for interrupt transfers for that frame. All QHs for interrupt transfers point at the QH for control transfers, which in turn points at the QH for bulk transfers. The following diagram gives a graphical overview of this:
|
||||
|
||||
This results in the following schedule being run in each frame. After fetching the pointer for the current frame from the framelist the controller first executes the TDs for all the isochronous packets in that frame. The last of these TDs refers to the QH for the interrupt transfers for thatframe. The host controller will then descend from that QH to the QHs for the individual interrupt transfers. After finishing that queue, the QH for the interrupt transfers will refer the controller to the QH for all control transfers. It will execute all the subqueues scheduled there, followed by all the transfers queued at the bulk QH. To facilitate the handling of finished or failed transfers different types of interrupts are generated by the hardware at the end of each frame. In the last TD for a transfer the Interrupt-On Completion bit is set by the HC driver to flag an interrupt when the transfer has completed. An error interrupt is flagged if a TD reaches its maximum error count. If the short packet detect bit is set in a TD and less than the set packet length is transferred this interrupt is flagged to notify the controller driver of the completed transfer. It is the host controller driver's task to find out which transfer has completed or produced an error. When called the interrupt service routine will locate all the finished transfers and call their callbacks.
|
||||
|
||||
Refer to the UHCI Specification for a more elaborate description.
|
||||
|
||||
=== OHCI
|
||||
|
||||
Programming an OHCI host controller is much simpler. The controller assumes that a set of endpoints is available, and is aware of scheduling priorities and the ordering of the types of transfers in a frame. The main data structure used by the host controller is the endpoint descriptor (ED) to which a queue of transfer descriptors (TDs) is attached. The ED contains the maximum packet size allowed for an endpoint and the controller hardware does the splitting into packets. The pointers to the data buffers are updated after each transfer and when the start and end pointer are equal, the TD is retired to the done-queue. The four types of endpoints (interrupt, isochronous, control, and bulk) have their own queues. Control and bulk endpoints are queued each at their own queue. Interrupt EDs are queued in a tree, with the level in the tree defining the frequency at which they run.
|
||||
|
||||
The schedule being run by the host controller in each frame looks as follows. The controller will first run the non-periodic control and bulk queues, up to a time limit set by the HC driver. Then the interrupt transfers for that frame number are run, by using the lower five bits of the frame number as an index into level 0 of the tree of interrupts EDs. At the end of this tree the isochronous EDs are connected and these are traversed subsequently. The isochronous TDs contain the frame number of the first frame the transfer should be run in. After all the periodic transfers have been run, the control and bulk queues are traversed again. Periodically the interrupt service routine is called to process the done queue and call the callbacks for each transfer and reschedule interrupt and isochronous endpoints.
|
||||
|
||||
See the UHCI Specification for a more elaborate description. The middle layer provides access to the device in a controlled way and maintains resources in use by the different drivers and the services layer. The layer takes care of the following aspects:
|
||||
|
||||
* The device configuration information
|
||||
* The pipes to communicate with a device
|
||||
* Probing and attaching and detaching form a device.
|
||||
|
||||
[[usb-dev]]
|
||||
== USB Device Information
|
||||
|
||||
=== Device Configuration Information
|
||||
|
||||
Each device provides different levels of configuration information. Each device has one or more configurations, of which one is selected during probe/attach. A configuration provides power and bandwidth requirements. Within each configuration there can be multiple interfaces. A device interface is a collection of endpoints. For example USB speakers can have an interface for the audio data (Audio Class) and an interface for the knobs, dials and buttons (HID Class). All interfaces in a configuration are active at the same time and can be attached to by different drivers. Each interface can have alternates, providing different quality of service parameters. In for example cameras this is used to provide different frame sizes and numbers of frames per second.
|
||||
|
||||
Within each interface, 0 or more endpoints can be specified. Endpoints are the unidirectional access points for communicating with a device. They provide buffers to temporarily store incoming or outgoing data from the device. Each endpoint has a unique address within a configuration, the endpoint's number plus its direction. The default endpoint, endpoint 0, is not part of any interface and available in all configurations. It is managed by the services layer and not directly available to device drivers.
|
||||
|
||||
This hierarchical configuration information is described in the device by a standard set of descriptors (see section 9.6 of the USB specification). They can be requested through the Get Descriptor Request. The services layer caches these descriptors to avoid unnecessary transfers on the USB bus. Access to the descriptors is provided through function calls.
|
||||
|
||||
* Device descriptors: General information about the device, like Vendor, Product and Revision Id, supported device class, subclass and protocol if applicable, maximum packet size for the default endpoint, etc.
|
||||
* Configuration descriptors: The number of interfaces in this configuration, suspend and resume functionality supported and power requirements.
|
||||
* Interface descriptors: interface class, subclass and protocol if applicable, number of alternate settings for the interface and the number of endpoints.
|
||||
* Endpoint descriptors: Endpoint address, direction and type, maximum packet size supported and polling frequency if type is interrupt endpoint. There is no descriptor for the default endpoint (endpoint 0) and it is never counted in an interface descriptor.
|
||||
* String descriptors: In the other descriptors string indices are supplied for some fields.These can be used to retrieve descriptive strings, possibly in multiple languages.
|
||||
|
||||
Class specifications can add their own descriptor types that are available through the GetDescriptor Request.
|
||||
|
||||
Pipes Communication to end points on a device flows through so-called pipes. Drivers submit transfers to endpoints to a pipe and provide a callback to be called on completion or failure of the transfer (asynchronous transfers) or wait for completion (synchronous transfer). Transfers to an endpoint are serialised in the pipe. A transfer can either complete, fail or time-out (if a time-out has been set). There are two types of time-outs for transfers. Time-outs can happen due to time-out on the USBbus (milliseconds). These time-outs are seen as failures and can be due to disconnection of the device. A second form of time-out is implemented in software and is triggered when a transfer does not complete within a specified amount of time (seconds). These are caused by a device acknowledging negatively (NAK) the transferred packets. The cause for this is the device not being ready to receive data, buffer under- or overrun or protocol errors.
|
||||
|
||||
If a transfer over a pipe is larger than the maximum packet size specified in the associated endpoint descriptor, the host controller (OHCI) or the HC driver (UHCI) will split the transfer into packets of maximum packet size, with the last packet possibly smaller than the maximum packet size.
|
||||
|
||||
Sometimes it is not a problem for a device to return less data than requested. For example abulk-in-transfer to a modem might request 200 bytes of data, but the modem has only 5 bytes available at that time. The driver can set the short packet (SPD) flag. It allows the host controller to accept a packet even if the amount of data transferred is less than requested. This flag is only valid for in-transfers, as the amount of data to be sent to a device is always known beforehand. If an unrecoverable error occurs in a device during a transfer the pipe is stalled. Before any more data is accepted or sent the driver needs to resolve the cause of the stall and clear the endpoint stall condition through send the clear endpoint halt device request over the default pipe. The default endpoint should never stall.
|
||||
|
||||
There are four different types of endpoints and corresponding pipes: - Control pipe / default pipe: There is one control pipe per device, connected to the default endpoint (endpoint 0). The pipe carries the device requests and associated data. The difference between transfers over the default pipe and other pipes is that the protocol for the transfers is described in the USB specification. These requests are used to reset and configure the device. A basic set of commands that must be supported by each device is provided in chapter 9 of the USB specification. The commands supported on this pipe can be extended by a device class specification to support additional functionality.
|
||||
|
||||
* Bulk pipe: This is the USB equivalent to a raw transmission medium.
|
||||
* Interrupt pipe: The host sends a request for data to the device and if the device has nothing to send, it will NAK the data packet. Interrupt transfers are scheduled at a frequency specified when creating the pipe.
|
||||
* Isochronous pipe: These pipes are intended for isochronous data, for example video or audio streams, with fixed latency, but no guaranteed delivery. Some support for pipes of this type is available in the current implementation. Packets in control, bulk and interrupt transfers are retried if an error occurs during transmission or the device acknowledges the packet negatively (NAK) due to for example lack of buffer space to store the incoming data. Isochronous packets are however not retried in case of failed delivery or NAK of a packet as this might violate the timing constraints.
|
||||
|
||||
The availability of the necessary bandwidth is calculated during the creation of the pipe. Transfers are scheduled within frames of 1 millisecond. The bandwidth allocation within a frame is prescribed by the USB specification, section 5.6 [ 2]. Isochronous and interrupt transfers are allowed to consume up to 90% of the bandwidth within a frame. Packets for control and bulk transfers are scheduled after all isochronous and interrupt packets and will consume all the remaining bandwidth.
|
||||
|
||||
More information on scheduling of transfers and bandwidth reclamation can be found in chapter 5 of the USB specification, section 1.3 of the UHCI specification, and section 3.4.2 of the OHCI specification.
|
||||
|
||||
[[usb-devprobe]]
|
||||
== Device Probe and Attach
|
||||
|
||||
After the notification by the hub that a new device has been connected, the service layer switches on the port, providing the device with 100 mA of current. At this point the device is in its default state and listening to device address 0. The services layer will proceed to retrieve the various descriptors through the default pipe. After that it will send a Set Address request to move the device away from the default device address (address 0). Multiple device drivers might be able to support the device. For example a modem driver might be able to support an ISDN TA through the AT compatibility interface. A driver for that specific model of the ISDN adapter might however be able to provide much better support for this device. To support this flexibility, the probes return priorities indicating their level of support. Support for a specific revision of a product ranks the highest and the generic driver the lowest priority. It might also be that multiple drivers could attach to one device if there are multiple interfaces within one configuration. Each driver only needs to support a subset of the interfaces.
|
||||
|
||||
The probing for a driver for a newly attached device checks first for device specific drivers. If not found, the probe code iterates over all supported configurations until a driver attaches in a configuration. To support devices with multiple drivers on different interfaces, the probe iterates over all interfaces in a configuration that have not yet been claimed by a driver. Configurations that exceed the power budget for the hub are ignored. During attach the driver should initialise the device to its proper state, but not reset it, as this will make the device disconnect itself from the bus and restart the probing process for it. To avoid consuming unnecessary bandwidth should not claim the interrupt pipe at attach time, but should postpone allocating the pipe until the file is opened and the data is actually used. When the file is closed the pipe should be closed again, even though the device might still be attached.
|
||||
|
||||
=== Device Disconnect and Detach
|
||||
|
||||
A device driver should expect to receive errors during any transaction with the device. The design of USB supports and encourages the disconnection of devices at any point in time. Drivers should make sure that they do the right thing when the device disappears.
|
||||
|
||||
Furthermore a device that has been disconnected and reconnected will not be reattached at the same device instance. This might change in the future when more devices support serial numbers (see the device descriptor) or other means of defining an identity for a device have been developed.
|
||||
|
||||
The disconnection of a device is signaled by a hub in the interrupt packet delivered to the hub driver. The status change information indicates which port has seen a connection change. The device detach method for all device drivers for the device connected on that port are called and the structures cleaned up. If the port status indicates that in the mean time a device has been connected to that port, the procedure for probing and attaching the device will be started. A device reset will produce a disconnect-connect sequence on the hub and will be handled as described above.
|
||||
|
||||
[[usb-protocol]]
|
||||
== USB Drivers Protocol Information
|
||||
|
||||
The protocol used over pipes other than the default pipe is undefined by the USB specification. Information on this can be found from various sources. The most accurate source is the developer's section on the USB home pages. From these pages, a growing number of deviceclass specifications are available. These specifications specify what a compliant device should look like from a driver perspective, basic functionality it needs to provide and the protocol that is to be used over the communication channels. The USB specification includes the description of the Hub Class. A class specification for Human Interface Devices (HID) has been created to cater for keyboards, tablets, bar-code readers, buttons, knobs, switches, etc. A third example is the class specification for mass storage devices. For a full list of device classes see the developers section on the USB home pages.
|
||||
|
||||
For many devices the protocol information has not yet been published however. Information on the protocol being used might be available from the company making the device. Some companies will require you to sign a Non -Disclosure Agreement (NDA) before giving you the specifications. This in most cases precludes making the driver open source.
|
||||
|
||||
Another good source of information is the Linux driver sources, as a number of companies have started to provide drivers for Linux for their devices. It is always a good idea to contact the authors of those drivers for their source of information.
|
||||
|
||||
Example: Human Interface Devices The specification for the Human Interface Devices like keyboards, mice, tablets, buttons, dials,etc. is referred to in other device class specifications and is used in many devices.
|
||||
|
||||
For example audio speakers provide endpoints to the digital to analogue converters and possibly an extra pipe for a microphone. They also provide a HID endpoint in a separate interface for the buttons and dials on the front of the device. The same is true for the monitor control class. It is straightforward to build support for these interfaces through the available kernel and userland libraries together with the HID class driver or the generic driver. Another device that serves as an example for interfaces within one configuration driven by different device drivers is a cheap keyboard with built-in legacy mouse port. To avoid having the cost of including the hardware for a USB hub in the device, manufacturers combined the mouse data received from the PS/2 port on the back of the keyboard and the key presses from the keyboard into two separate interfaces in the same configuration. The mouse and keyboard drivers each attach to the appropriate interface and allocate the pipes to the two independent endpoints.
|
||||
|
||||
Example: Firmware download Many devices that have been developed are based on a general purpose processor with an additional USB core added to it. Since the development of drivers and firmware for USB devices is still very new, many devices require the downloading of the firmware after they have been connected.
|
||||
|
||||
The procedure followed is straightforward. The device identifies itself through a vendor and product Id. The first driver probes and attaches to it and downloads the firmware into it. After that the device soft resets itself and the driver is detached. After a short pause the device announces its presence on the bus. The device will have changed its vendor/product/revision Id to reflect the fact that it has been supplied with firmware and as a consequence a second driver will probe it and attach to it.
|
||||
|
||||
An example of these types of devices is the ActiveWire I/O board, based on the EZ-USB chip. For this chip a generic firmware downloader is available. The firmware downloaded into the ActiveWire board changes the revision Id. It will then perform a soft reset of the USB part of the EZ-USB chip to disconnect from the USB bus and again reconnect.
|
||||
|
||||
Example: Mass Storage Devices Support for mass storage devices is mainly built around existing protocols. The Iomega USB Zipdrive is based on the SCSI version of their drive. The SCSI commands and status messages are wrapped in blocks and transferred over the bulk pipes to and from the device, emulating a SCSI controller over the USB wire. ATAPI and UFI commands are supported in a similar fashion.
|
||||
|
||||
The Mass Storage Specification supports 2 different types of wrapping of the command block.The initial attempt was based on sending the command and status through the default pipe and using bulk transfers for the data to be moved between the host and the device. Based on experience a second approach was designed that was based on wrapping the command and status blocks and sending them over the bulk out and in endpoint. The specification specifies exactly what has to happen when and what has to be done in case an error condition is encountered. The biggest challenge when writing drivers for these devices is to fit USB based protocol into the existing support for mass storage devices. CAM provides hooks to do this in a fairly straight forward way. ATAPI is less simple as historically the IDE interface has never had many different appearances.
|
||||
|
||||
The support for the USB floppy from Y-E Data is again less straightforward as a new command set has been designed.
|
||||
108
documentation/content/en/books/arch-handbook/vm/chapter.adoc
Normal file
108
documentation/content/en/books/arch-handbook/vm/chapter.adoc
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
---
|
||||
title: Chapter 7. Virtual Memory System
|
||||
prev: books/arch-handbook/mac
|
||||
next: books/arch-handbook/smp
|
||||
---
|
||||
|
||||
[[vm]]
|
||||
= Virtual Memory System
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 7
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[vm-physmem]]
|
||||
== Management of Physical Memory `vm_page_t`
|
||||
|
||||
Physical memory is managed on a page-by-page basis through the `vm_page_t` structure. Pages of physical memory are categorized through the placement of their respective `vm_page_t` structures on one of several paging queues.
|
||||
|
||||
A page can be in a wired, active, inactive, cache, or free state. Except for the wired state, the page is typically placed in a doubly link list queue representing the state that it is in. Wired pages are not placed on any queue.
|
||||
|
||||
FreeBSD implements a more involved paging queue for cached and free pages in order to implement page coloring. Each of these states involves multiple queues arranged according to the size of the processor's L1 and L2 caches. When a new page needs to be allocated, FreeBSD attempts to obtain one that is reasonably well aligned from the point of view of the L1 and L2 caches relative to the VM object the page is being allocated for.
|
||||
|
||||
Additionally, a page may be held with a reference count or locked with a busy count. The VM system also implements an "ultimate locked" state for a page using the PG_BUSY bit in the page's flags.
|
||||
|
||||
In general terms, each of the paging queues operates in a LRU fashion. A page is typically placed in a wired or active state initially. When wired, the page is usually associated with a page table somewhere. The VM system ages the page by scanning pages in a more active paging queue (LRU) in order to move them to a less-active paging queue. Pages that get moved into the cache are still associated with a VM object but are candidates for immediate reuse. Pages in the free queue are truly free. FreeBSD attempts to minimize the number of pages in the free queue, but a certain minimum number of truly free pages must be maintained in order to accommodate page allocation at interrupt time.
|
||||
|
||||
If a process attempts to access a page that does not exist in its page table but does exist in one of the paging queues (such as the inactive or cache queues), a relatively inexpensive page reactivation fault occurs which causes the page to be reactivated. If the page does not exist in system memory at all, the process must block while the page is brought in from disk.
|
||||
|
||||
FreeBSD dynamically tunes its paging queues and attempts to maintain reasonable ratios of pages in the various queues as well as attempts to maintain a reasonable breakdown of clean versus dirty pages. The amount of rebalancing that occurs depends on the system's memory load. This rebalancing is implemented by the pageout daemon and involves laundering dirty pages (syncing them with their backing store), noticing when pages are activity referenced (resetting their position in the LRU queues or moving them between queues), migrating pages between queues when the queues are out of balance, and so forth. FreeBSD's VM system is willing to take a reasonable number of reactivation page faults to determine how active or how idle a page actually is. This leads to better decisions being made as to when to launder or swap-out a page.
|
||||
|
||||
[[vm-cache]]
|
||||
== The Unified Buffer Cache `vm_object_t`
|
||||
|
||||
FreeBSD implements the idea of a generic "VM object". VM objects can be associated with backing store of various typesunbacked, swap-backed, physical device-backed, or file-backed storage. Since the filesystem uses the same VM objects to manage in-core data relating to files, the result is a unified buffer cache.
|
||||
|
||||
VM objects can be _shadowed_. That is, they can be stacked on top of each other. For example, you might have a swap-backed VM object stacked on top of a file-backed VM object in order to implement a MAP_PRIVATE mmap()ing. This stacking is also used to implement various sharing properties, including copy-on-write, for forked address spaces.
|
||||
|
||||
It should be noted that a `vm_page_t` can only be associated with one VM object at a time. The VM object shadowing implements the perceived sharing of the same page across multiple instances.
|
||||
|
||||
[[vm-fileio]]
|
||||
== Filesystem I/O `struct buf`
|
||||
|
||||
vnode-backed VM objects, such as file-backed objects, generally need to maintain their own clean/dirty info independent from the VM system's idea of clean/dirty. For example, when the VM system decides to synchronize a physical page to its backing store, the VM system needs to mark the page clean before the page is actually written to its backing store. Additionally, filesystems need to be able to map portions of a file or file metadata into KVM in order to operate on it.
|
||||
|
||||
The entities used to manage this are known as filesystem buffers, ``struct buf``'s, or ``bp``'s. When a filesystem needs to operate on a portion of a VM object, it typically maps part of the object into a struct buf and then maps the pages in the struct buf into KVM. In the same manner, disk I/O is typically issued by mapping portions of objects into buffer structures and then issuing the I/O on the buffer structures. The underlying vm_page_t's are typically busied for the duration of the I/O. Filesystem buffers also have their own notion of being busy, which is useful to filesystem driver code which would rather operate on filesystem buffers instead of hard VM pages.
|
||||
|
||||
FreeBSD reserves a limited amount of KVM to hold mappings from struct bufs, but it should be made clear that this KVM is used solely to hold mappings and does not limit the ability to cache data. Physical data caching is strictly a function of ``vm_page_t``'s, not filesystem buffers. However, since filesystem buffers are used to placehold I/O, they do inherently limit the amount of concurrent I/O possible. However, as there are usually a few thousand filesystem buffers available, this is not usually a problem.
|
||||
|
||||
[[vm-pagetables]]
|
||||
== Mapping Page Tables `vm_map_t, vm_entry_t`
|
||||
|
||||
FreeBSD separates the physical page table topology from the VM system. All hard per-process page tables can be reconstructed on the fly and are usually considered throwaway. Special page tables such as those managing KVM are typically permanently preallocated. These page tables are not throwaway.
|
||||
|
||||
FreeBSD associates portions of vm_objects with address ranges in virtual memory through `vm_map_t` and `vm_entry_t` structures. Page tables are directly synthesized from the `vm_map_t`/`vm_entry_t`/ `vm_object_t` hierarchy. Recall that I mentioned that physical pages are only directly associated with a `vm_object`; that is not quite true. ``vm_page_t``'s are also linked into page tables that they are actively associated with. One `vm_page_t` can be linked into several _pmaps_, as page tables are called. However, the hierarchical association holds, so all references to the same page in the same object reference the same `vm_page_t` and thus give us buffer cache unification across the board.
|
||||
|
||||
[[vm-kvm]]
|
||||
== KVM Memory Mapping
|
||||
|
||||
FreeBSD uses KVM to hold various kernel structures. The single largest entity held in KVM is the filesystem buffer cache. That is, mappings relating to `struct buf` entities.
|
||||
|
||||
Unlike Linux, FreeBSD does _not_ map all of physical memory into KVM. This means that FreeBSD can handle memory configurations up to 4G on 32 bit platforms. In fact, if the mmu were capable of it, FreeBSD could theoretically handle memory configurations up to 8TB on a 32 bit platform. However, since most 32 bit platforms are only capable of mapping 4GB of ram, this is a moot point.
|
||||
|
||||
KVM is managed through several mechanisms. The main mechanism used to manage KVM is the _zone allocator_. The zone allocator takes a chunk of KVM and splits it up into constant-sized blocks of memory in order to allocate a specific type of structure. You can use `vmstat -m` to get an overview of current KVM utilization broken down by zone.
|
||||
|
||||
[[vm-tuning]]
|
||||
== Tuning the FreeBSD VM System
|
||||
|
||||
A concerted effort has been made to make the FreeBSD kernel dynamically tune itself. Typically you do not need to mess with anything beyond the `maxusers` and `NMBCLUSTERS` kernel config options. That is, kernel compilation options specified in (typically) [.filename]#/usr/src/sys/i386/conf/CONFIG_FILE#. A description of all available kernel configuration options can be found in [.filename]#/usr/src/sys/i386/conf/LINT#.
|
||||
|
||||
In a large system configuration you may wish to increase `maxusers`. Values typically range from 10 to 128. Note that raising `maxusers` too high can cause the system to overflow available KVM resulting in unpredictable operation. It is better to leave `maxusers` at some reasonable number and add other options, such as `NMBCLUSTERS`, to increase specific resources.
|
||||
|
||||
If your system is going to use the network heavily, you may want to increase `NMBCLUSTERS`. Typical values range from 1024 to 4096.
|
||||
|
||||
The `NBUF` parameter is also traditionally used to scale the system. This parameter determines the amount of KVA the system can use to map filesystem buffers for I/O. Note that this parameter has nothing whatsoever to do with the unified buffer cache! This parameter is dynamically tuned in 3.0-CURRENT and later kernels and should generally not be adjusted manually. We recommend that you _not_ try to specify an `NBUF` parameter. Let the system pick it. Too small a value can result in extremely inefficient filesystem operation while too large a value can starve the page queues by causing too many pages to become wired down.
|
||||
|
||||
By default, FreeBSD kernels are not optimized. You can set debugging and optimization flags with the `makeoptions` directive in the kernel configuration. Note that you should not use `-g` unless you can accommodate the large (typically 7 MB+) kernels that result.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
makeoptions DEBUG="-g"
|
||||
makeoptions COPTFLAGS="-O -pipe"
|
||||
....
|
||||
|
||||
Sysctl provides a way to tune kernel parameters at run-time. You typically do not need to mess with any of the sysctl variables, especially the VM related ones.
|
||||
|
||||
Run time VM and system tuning is relatively straightforward. First, use Soft Updates on your UFS/FFS filesystems whenever possible. [.filename]#/usr/src/sys/ufs/ffs/README.softupdates# contains instructions (and restrictions) on how to configure it.
|
||||
|
||||
Second, configure sufficient swap. You should have a swap partition configured on each physical disk, up to four, even on your "work" disks. You should have at least 2x the swap space as you have main memory, and possibly even more if you do not have a lot of memory. You should also size your swap partition based on the maximum memory configuration you ever intend to put on the machine so you do not have to repartition your disks later on. If you want to be able to accommodate a crash dump, your first swap partition must be at least as large as main memory and [.filename]#/var/crash# must have sufficient free space to hold the dump.
|
||||
|
||||
NFS-based swap is perfectly acceptable on 4.X or later systems, but you must be aware that the NFS server will take the brunt of the paging load.
|
||||
8
documentation/content/en/books/books.adoc
Normal file
8
documentation/content/en/books/books.adoc
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
handbook
|
||||
arch-handbook
|
||||
design-44bsd
|
||||
dev-model
|
||||
developers-handbook
|
||||
faq
|
||||
fdp-primer
|
||||
porters-handbook
|
||||
507
documentation/content/en/books/design-44bsd/_index.adoc
Normal file
507
documentation/content/en/books/design-44bsd/_index.adoc
Normal file
|
|
@ -0,0 +1,507 @@
|
|||
---
|
||||
title: The Design and Implementation of the 4.4BSD Operating System
|
||||
authors:
|
||||
- author: Marshall Kirk McKusick
|
||||
- author: Keith Bostic
|
||||
- author: Michael J. Karels
|
||||
- author: John S. Quarterman
|
||||
copyright: 1996 Addison-Wesley Longman, Inc
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["design-44bsd"]
|
||||
---
|
||||
|
||||
= The Design and Implementation of the 4.4BSD Operating System
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 2
|
||||
:icons: font
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:partnums:
|
||||
:chapter-signifier: Chapter
|
||||
:part-signifier: Part
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:sectnumoffset: 2
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
:imagesdir: ../../images/books/design-44bsd/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:imagesdir: ../../../static/images/books/design-44bsd/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:imagesdir: ../../../static/images/books/design-44bsd/
|
||||
endif::[]
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
[[overview]]
|
||||
== Design Overview of 4.4BSD
|
||||
|
||||
[[overview-facilities]]
|
||||
=== 4.4BSD Facilities and the Kernel
|
||||
|
||||
The 4.4BSD kernel provides four basic facilities: processes, a filesystem, communications, and system startup. This section outlines where each of these four basic services is described in this book.
|
||||
|
||||
. Processes constitute a thread of control in an address space. Mechanisms for creating, terminating, and otherwise controlling processes are described in Chapter 4. The system multiplexes separate virtual-address spaces for each process; this memory management is discussed in Chapter 5.
|
||||
. The user interface to the filesystem and devices is similar; common aspects are discussed in Chapter 6. The filesystem is a set of named files, organized in a tree-structured hierarchy of directories, and of operations to manipulate them, as presented in Chapter 7. Files reside on physical media such as disks. 4.4BSD supports several organizations of data on the disk, as set forth in Chapter 8. Access to files on remote machines is the subject of Chapter 9. Terminals are used to access the system; their operation is the subject of Chapter 10.
|
||||
. Communication mechanisms provided by traditional UNIX systems include simplex reliable byte streams between related processes (see pipes, Section 11.1), and notification of exceptional events (see signals, Section 4.7). 4.4BSD also has a general interprocess-communication facility. This facility, described in Chapter 11, uses access mechanisms distinct from those of the filesystem, but, once a connection is set up, a process can access it as though it were a pipe. There is a general networking framework, discussed in Chapter 12, that is normally used as a layer underlying the IPC facility. Chapter 13 describes a particular networking implementation in detail.
|
||||
. Any real operating system has operational issues, such as how to start it running. Startup and operational issues are described in Chapter 14.
|
||||
|
||||
Sections 2.3 through 2.14 present introductory material related to Chapters 3 through 14. We shall define terms, mention basic system calls, and explore historical developments. Finally, we shall give the reasons for many major design decisions.
|
||||
|
||||
==== The Kernel
|
||||
|
||||
The _kernel_ is the part of the system that runs in protected mode and mediates access by all user programs to the underlying hardware (e.g., CPU, disks, terminals, network links) and software constructs (e.g., filesystem, network protocols). The kernel provides the basic system facilities; it creates and manages processes, and provides functions to access the filesystem and communication facilities. These functions, called _system calls_ appear to user processes as library subroutines. These system calls are the only interface that processes have to these facilities. Details of the system-call mechanism are given in Chapter 3, as are descriptions of several kernel mechanisms that do not execute as the direct result of a process doing a system call.
|
||||
|
||||
A _kernel_ in traditional operating-system terminology, is a small nucleus of software that provides only the minimal facilities necessary for implementing additional operating-system services. In contemporary research operating systems -- such as Chorus <<biblio-rozier, [Rozier et al, 1988]>>, Mach <<biblio-accetta, [Accetta et al, 1986]>>, Tunis <<biblio-ewens, [Ewens et al, 1985]>>, and the V Kernel <<biblio-cheriton, [Cheriton, 1988]>> -- this division of functionality is more than just a logical one. Services such as filesystems and networking protocols are implemented as client application processes of the nucleus or kernel.
|
||||
|
||||
The 4.4BSD kernel is not partitioned into multiple processes. This basic design decision was made in the earliest versions of UNIX. The first two implementations by Ken Thompson had no memory mapping, and thus made no hardware-enforced distinction between user and kernel space <<biblio-ritchie, [Ritchie, 1988]>>. A message-passing system could have been implemented as readily as the actually implemented model of kernel and user processes. The monolithic kernel was chosen for simplicity and performance. And the early kernels were small; the inclusion of facilities such as networking into the kernel has increased its size. The current trend in operating-systems research is to reduce the kernel size by placing such services in user space.
|
||||
|
||||
Users ordinarily interact with the system through a command-language interpreter, called a _shell_, and perhaps through additional user application programs. Such programs and the shell are implemented with processes. Details of such programs are beyond the scope of this book, which instead concentrates almost exclusively on the kernel.
|
||||
|
||||
Sections 2.3 and 2.4 describe the services provided by the 4.4BSD kernel, and give an overview of the latter's design. Later chapters describe the detailed design and implementation of these services as they appear in 4.4BSD.
|
||||
|
||||
[[overview-kernel-organization]]
|
||||
=== Kernel Organization
|
||||
|
||||
In this section, we view the organization of the 4.4BSD kernel in two ways:
|
||||
|
||||
[arabic]
|
||||
. As a static body of software, categorized by the functionality offered by the modules that make up the kernel
|
||||
. By its dynamic operation, categorized according to the services provided to users
|
||||
|
||||
The largest part of the kernel implements the system services that applications access through system calls. In 4.4BSD, this software has been organized according to the following:
|
||||
|
||||
* Basic kernel facilities: timer and system-clock handling, descriptor management, and process management
|
||||
* Memory-management support: paging and swapping
|
||||
* Generic system interfaces: the I/O, control, and multiplexing operations performed on descriptors
|
||||
* The filesystem: files, directories, pathname translation, file locking, and I/O buffer management
|
||||
* Terminal-handling support: the terminal-interface driver and terminal line disciplines
|
||||
* Interprocess-communication facilities: sockets
|
||||
* Support for network communication: communication protocols and generic network facilities, such as routing
|
||||
|
||||
.Machine-independent software in the 4.4BSD kernel
|
||||
[[table-mach-indep]]
|
||||
[cols=",,",options="header",]
|
||||
|===
|
||||
|Category |Lines of code |Percentage of kernel
|
||||
|headers |9,393 |4.6
|
||||
|initialization |1,107 |0.6
|
||||
|kernel facilities |8,793 |4.4
|
||||
|generic interfaces |4,782 |2.4
|
||||
|interprocess communication |4,540 |2.2
|
||||
|terminal handling |3,911 |1.9
|
||||
|virtual memory |11,813 |5.8
|
||||
|vnode management |7,954 |3.9
|
||||
|filesystem naming |6,550 |3.2
|
||||
|fast filestore |4,365 |2.2
|
||||
|log-structure filestore |4,337 |2.1
|
||||
|memory-based filestore |645 |0.3
|
||||
|cd9660 filesystem |4,177 |2.1
|
||||
|miscellaneous filesystems (10) |12,695 |6.3
|
||||
|network filesystem |17,199 |8.5
|
||||
|network communication |8,630 |4.3
|
||||
|internet protocols |11,984 |5.9
|
||||
|ISO protocols |23,924 |11.8
|
||||
|X.25 protocols |10,626 |5.3
|
||||
|XNS protocols |5,192 |2.6
|
||||
|===
|
||||
|
||||
Most of the software in these categories is machine independent and is portable across different hardware architectures.
|
||||
|
||||
The machine-dependent aspects of the kernel are isolated from the mainstream code. In particular, none of the machine-independent code contains conditional code for specific architecture. When an architecture-dependent action is needed, the machine-independent code calls an architecture-dependent function that is located in the machine-dependent code. The software that is machine dependent includes
|
||||
|
||||
* Low-level system-startup actions
|
||||
* Trap and fault handling
|
||||
* Low-level manipulation of the run-time context of a process
|
||||
* Configuration and initialization of hardware devices
|
||||
* Run-time support for I/O devices
|
||||
|
||||
.Machine-dependent software for the HP300 in the 4.4BSD kernel
|
||||
[[table-mach-dep]]
|
||||
[cols=",,",options="header",]
|
||||
|===
|
||||
|Category |Lines of code |Percentage of kernel
|
||||
|machine dependent headers |1,562 |0.8
|
||||
|device driver headers |3,495 |1.7
|
||||
|device driver source |17,506 |8.7
|
||||
|virtual memory |3,087 |1.5
|
||||
|other machine dependent |6,287 |3.1
|
||||
|routines in assembly language |3,014 |1.5
|
||||
|HP/UX compatibility |4,683 |2.3
|
||||
|===
|
||||
|
||||
<<table-mach-indep>> summarizes the machine-independent software that constitutes the 4.4BSD kernel for the HP300. The numbers in column 2 are for lines of C source code, header files, and assembly language. Virtually all the software in the kernel is written in the C programming language; less than 2 percent is written in assembly language. As the statistics in <<table-mach-dep>> show, the machine-dependent software, excluding HP/UX and device support, accounts for a minuscule 6.9 percent of the kernel.
|
||||
|
||||
Only a small part of the kernel is devoted to initializing the system. This code is used when the system is _bootstrapped_ into operation and is responsible for setting up the kernel hardware and software environment (see Chapter 14). Some operating systems (especially those with limited physical memory) discard or _overlay_ the software that performs these functions after that software has been executed. The 4.4BSD kernel does not reclaim the memory used by the startup code because that memory space is barely 0.5 percent of the kernel resources used on a typical machine. Also, the startup code does not appear in one place in the kernel -- it is scattered throughout, and it usually appears in places logically associated with what is being initialized.
|
||||
|
||||
[[overview-kernel-service]]
|
||||
=== Kernel Services
|
||||
|
||||
The boundary between the kernel- and user-level code is enforced by hardware-protection facilities provided by the underlying hardware. The kernel operates in a separate address space that is inaccessible to user processes. Privileged operations -- such as starting I/O and halting the central processing unit (CPU) -- are available to only the kernel. Applications request services from the kernel with _system calls_. System calls are used to cause the kernel to execute complicated operations, such as writing data to secondary storage, and simple operations, such as returning the current time of day. All system calls appear _synchronous_ to applications: The application does not run while the kernel does the actions associated with a system call. The kernel may finish some operations associated with a system call after it has returned. For example, a _write_ system call will copy the data to be written from the user process to a kernel buffer while the process waits, but will usually return from the system call before the kernel buffer is written to the disk.
|
||||
|
||||
A system call usually is implemented as a hardware trap that changes the CPU's execution mode and the current address-space mapping. Parameters supplied by users in system calls are validated by the kernel before being used. Such checking ensures the integrity of the system. All parameters passed into the kernel are copied into the kernel's address space, to ensure that validated parameters are not changed as a side effect of the system call. System-call results are returned by the kernel, either in hardware registers or by their values being copied to user-specified memory addresses. Like parameters passed into the kernel, addresses used for the return of results must be validated to ensure that they are part of an application's address space. If the kernel encounters an error while processing a system call, it returns an error code to the user. For the C programming language, this error code is stored in the global variable _errno_, and the function that executed the system call returns the value -1.
|
||||
|
||||
User applications and the kernel operate independently of each other. 4.4BSD does not store I/O control blocks or other operating-system-related data structures in the application's address space. Each user-level application is provided an independent address space in which it executes. The kernel makes most state changes, such as suspending a process while another is running, invisible to the processes involved.
|
||||
|
||||
[[overview-process-management]]
|
||||
=== Process Management
|
||||
|
||||
4.4BSD supports a multitasking environment. Each task or thread of execution is termed a _process_. The _context_ of a 4.4BSD process consists of user-level state, including the contents of its address space and the run-time environment, and kernel-level state, which includes scheduling parameters, resource controls, and identification information. The context includes everything used by the kernel in providing services for the process. Users can create processes, control the processes' execution, and receive notification when the processes' execution status changes. Every process is assigned a unique value, termed a _process identifier_ (PID). This value is used by the kernel to identify a process when reporting status changes to a user, and by a user when referencing a process in a system call.
|
||||
|
||||
The kernel creates a process by duplicating the context of another process. The new process is termed a _child process_ of the original _parent process_ The context duplicated in process creation includes both the user-level execution state of the process and the process's system state managed by the kernel. Important components of the kernel state are described in Chapter 4.
|
||||
|
||||
[[fig-process-lifecycle]]
|
||||
.Process lifecycle
|
||||
image:fig1.png[Process lifecycle]
|
||||
|
||||
The process lifecycle is depicted in <<fig-process-lifecycle>>. A process may create a new process that is a copy of the original by using the _fork_ system call. The _fork_ call returns twice: once in the parent process, where the return value is the process identifier of the child, and once in the child process, where the return value is 0. The parent-child relationship induces a hierarchical structure on the set of processes in the system. The new process shares all its parent's resources, such as file descriptors, signal-handling status, and memory layout.
|
||||
|
||||
Although there are occasions when the new process is intended to be a copy of the parent, the loading and execution of a different program is a more useful and typical action. A process can overlay itself with the memory image of another program, passing to the newly created image a set of parameters, using the system call _execve_. One parameter is the name of a file whose contents are in a format recognized by the system -- either a binary-executable file or a file that causes the execution of a specified interpreter program to process its contents.
|
||||
|
||||
A process may terminate by executing an _exit_ system call, sending 8 bits of exit status to its parent. If a process wants to communicate more than a single byte of information with its parent, it must either set up an interprocess-communication channel using pipes or sockets, or use an intermediate file. Interprocess communication is discussed extensively in Chapter 11.
|
||||
|
||||
A process can suspend execution until any of its child processes terminate using the _wait_ system call, which returns the PID and exit status of the terminated child process. A parent process can arrange to be notified by a signal when a child process exits or terminates abnormally. Using the _wait4_ system call, the parent can retrieve information about the event that caused termination of the child process and about resources consumed by the process during its lifetime. If a process is orphaned because its parent exits before it is finished, then the kernel arranges for the child's exit status to be passed back to a special system process _init_: see Sections 3.1 and 14.6).
|
||||
|
||||
The details of how the kernel creates and destroys processes are given in Chapter 5.
|
||||
|
||||
Processes are scheduled for execution according to a _process-priority_ parameter. This priority is managed by a kernel-based scheduling algorithm. Users can influence the scheduling of a process by specifying a parameter (_nice_) that weights the overall scheduling priority, but are still obligated to share the underlying CPU resources according to the kernel's scheduling policy.
|
||||
|
||||
==== Signals
|
||||
|
||||
The system defines a set of _signals_ that may be delivered to a process. Signals in 4.4BSD are modeled after hardware interrupts. A process may specify a user-level subroutine to be a _handler_ to which a signal should be delivered. When a signal is generated, it is blocked from further occurrence while it is being _caught_ by the handler. Catching a signal involves saving the current process context and building a new one in which to run the handler. The signal is then delivered to the handler, which can either abort the process or return to the executing process (perhaps after setting a global variable). If the handler returns, the signal is unblocked and can be generated (and caught) again.
|
||||
|
||||
Alternatively, a process may specify that a signal is to be _ignored_, or that a default action, as determined by the kernel, is to be taken. The default action of certain signals is to terminate the process. This termination may be accompanied by creation of a _core file_ that contains the current memory image of the process for use in postmortem debugging.
|
||||
|
||||
Some signals cannot be caught or ignored. These signals include _SIGKILL_, which kills runaway processes, and the job-control signal _SIGSTOP_.
|
||||
|
||||
A process may choose to have signals delivered on a special stack so that sophisticated software stack manipulations are possible. For example, a language supporting coroutines needs to provide a stack for each coroutine. The language run-time system can allocate these stacks by dividing up the single stack provided by 4.4BSD. If the kernel does not support a separate signal stack, the space allocated for each coroutine must be expanded by the amount of space required to catch a signal.
|
||||
|
||||
All signals have the same _priority_. If multiple signals are pending simultaneously, the order in which signals are delivered to a process is implementation specific. Signal handlers execute with the signal that caused their invocation to be blocked, but other signals may yet occur. Mechanisms are provided so that processes can protect critical sections of code against the occurrence of specified signals.
|
||||
|
||||
The detailed design and implementation of signals is described in Section 4.7.
|
||||
|
||||
==== Process Groups and Sessions
|
||||
|
||||
Processes are organized into _process groups_. Process groups are used to control access to terminals and to provide a means of distributing signals to collections of related processes. A process inherits its process group from its parent process. Mechanisms are provided by the kernel to allow a process to alter its process group or the process group of its descendents. Creating a new process group is easy; the value of a new process group is ordinarily the process identifier of the creating process.
|
||||
|
||||
The group of processes in a process group is sometimes referred to as a _job_ and is manipulated by high-level system software, such as the shell. A common kind of job created by a shell is a _pipeline_ of several processes connected by pipes, such that the output of the first process is the input of the second, the output of the second is the input of the third, and so forth. The shell creates such a job by forking a process for each stage of the pipeline, then putting all those processes into a separate process group.
|
||||
|
||||
A user process can send a signal to each process in a process group, as well as to a single process. A process in a specific process group may receive software interrupts affecting the group, causing the group to suspend or resume execution, or to be interrupted or terminated.
|
||||
|
||||
A terminal has a process-group identifier assigned to it. This identifier is normally set to the identifier of a process group associated with the terminal. A job-control shell may create a number of process groups associated with the same terminal; the terminal is the _controlling terminal_ for each process in these groups. A process may read from a descriptor for its controlling terminal only if the terminal's process-group identifier matches that of the process. If the identifiers do not match, the process will be blocked if it attempts to read from the terminal. By changing the process-group identifier of the terminal, a shell can arbitrate a terminal among several different jobs. This arbitration is called _job control_ and is described, with process groups, in Section 4.8.
|
||||
|
||||
Just as a set of related processes can be collected into a process group, a set of process groups can be collected into a _session_. The main uses for sessions are to create an isolated environment for a daemon process and its children, and to collect together a user's login shell and the jobs that that shell spawns.
|
||||
|
||||
[[overview-memory-management]]
|
||||
=== Memory Management
|
||||
|
||||
Each process has its own private address space. The address space is initially divided into three logical segments: _text_, _data_, and _stack_. The text segment is read-only and contains the machine instructions of a program. The data and stack segments are both readable and writable. The data segment contains the initialized and uninitialized data portions of a program, whereas the stack segment holds the application's run-time stack. On most machines, the stack segment is extended automatically by the kernel as the process executes. A process can expand or contract its data segment by making a system call, whereas a process can change the size of its text segment only when the segment's contents are overlaid with data from the filesystem, or when debugging takes place. The initial contents of the segments of a child process are duplicates of the segments of a parent process.
|
||||
|
||||
The entire contents of a process address space do not need to be resident for a process to execute. If a process references a part of its address space that is not resident in main memory, the system _pages_ the necessary information into memory. When system resources are scarce, the system uses a two-level approach to maintain available resources. If a modest amount of memory is available, the system will take memory resources away from processes if these resources have not been used recently. Should there be a severe resource shortage, the system will resort to _swapping_ the entire context of a process to secondary storage. The _demand paging_ and _swapping_ done by the system are effectively transparent to processes. A process may, however, advise the system about expected future memory utilization as a performance aid.
|
||||
|
||||
==== BSD Memory-Management Design Decisions
|
||||
|
||||
The support of large sparse address spaces, mapped files, and shared memory was a requirement for 4.2BSD. An interface was specified, called _mmap_, that allowed unrelated processes to request a shared mapping of a file into their address spaces. If multiple processes mapped the same file into their address spaces, changes to the file's portion of an address space by one process would be reflected in the area mapped by the other processes, as well as in the file itself. Ultimately, 4.2BSD was shipped without the _mmap_ interface, because of pressure to make other features, such as networking, available.
|
||||
|
||||
Further development of the _mmap_ interface continued during the work on 4.3BSD. Over 40 companies and research groups participated in the discussions leading to the revised architecture that was described in the Berkeley Software Architecture Manual <<biblio-mckusick-1, [McKusick et al, 1994]>>. Several of the companies have implemented the revised interface <<biblio-gingell, [Gingell et al, 1987]>>.
|
||||
|
||||
Once again, time pressure prevented 4.3BSD from providing an implementation of the interface. Although the latter could have been built into the existing 4.3BSD virtual-memory system, the developers decided not to put it in because that implementation was nearly 10 years old. Furthermore, the original virtual-memory design was based on the assumption that computer memories were small and expensive, whereas disks were locally connected, fast, large, and inexpensive. Thus, the virtual-memory system was designed to be frugal with its use of memory at the expense of generating extra disk traffic. In addition, the 4.3BSD implementation was riddled with VAX memory-management hardware dependencies that impeded its portability to other computer architectures. Finally, the virtual-memory system was not designed to support the tightly coupled multiprocessors that are becoming increasingly common and important today.
|
||||
|
||||
Attempts to improve the old implementation incrementally seemed doomed to failure. A completely new design, on the other hand, could take advantage of large memories, conserve disk transfers, and have the potential to run on multiprocessors. Consequently, the virtual-memory system was completely replaced in 4.4BSD. The 4.4BSD virtual-memory system is based on the Mach 2.0 VM system <<biblio-tevanian, [Tevanian, 1987]>>. with updates from Mach 2.5 and Mach 3.0. It features efficient support for sharing, a clean separation of machine-independent and machine-dependent features, as well as (currently unused) multiprocessor support. Processes can map files anywhere in their address space. They can share parts of their address space by doing a shared mapping of the same file. Changes made by one process are visible in the address space of the other process, and also are written back to the file itself. Processes can also request private mappings of a file, which prevents any changes that they make from being visible to other processes mapping the file or being written back to the file itself.
|
||||
|
||||
Another issue with the virtual-memory system is the way that information is passed into the kernel when a system call is made. 4.4BSD always copies data from the process address space into a buffer in the kernel. For read or write operations that are transferring large quantities of data, doing the copy can be time consuming. An alternative to doing the copying is to remap the process memory into the kernel. The 4.4BSD kernel always copies the data for several reasons:
|
||||
|
||||
* Often, the user data are not page aligned and are not a multiple of the hardware page length.
|
||||
* If the page is taken away from the process, it will no longer be able to reference that page. Some programs depend on the data remaining in the buffer even after those data have been written.
|
||||
* If the process is allowed to keep a copy of the page (as it is in current 4.4BSD semantics), the page must be made _copy-on-write_. A copy-on-write page is one that is protected against being written by being made read-only. If the process attempts to modify the page, the kernel gets a write fault. The kernel then makes a copy of the page that the process can modify. Unfortunately, the typical process will immediately try to write new data to its output buffer, forcing the data to be copied anyway.
|
||||
* When pages are remapped to new virtual-memory addresses, most memory-management hardware requires that the hardware address-translation cache be purged selectively. The cache purges are often slow. The net effect is that remapping is slower than copying for blocks of data less than 4 to 8 Kbyte.
|
||||
|
||||
The biggest incentives for memory mapping are the needs for accessing big files and for passing large quantities of data between processes. The _mmap_ interface provides a way for both of these tasks to be done without copying.
|
||||
|
||||
==== Memory Management Inside the Kernel
|
||||
|
||||
The kernel often does allocations of memory that are needed for only the duration of a single system call. In a user process, such short-term memory would be allocated on the run-time stack. Because the kernel has a limited run-time stack, it is not feasible to allocate even moderate-sized blocks of memory on it. Consequently, such memory must be allocated through a more dynamic mechanism. For example, when the system must translate a pathname, it must allocate a 1-Kbyte buffer to hold the name. Other blocks of memory must be more persistent than a single system call, and thus could not be allocated on the stack even if there was space. An example is protocol-control blocks that remain throughout the duration of a network connection.
|
||||
|
||||
Demands for dynamic memory allocation in the kernel have increased as more services have been added. A generalized memory allocator reduces the complexity of writing code inside the kernel. Thus, the 4.4BSD kernel has a single memory allocator that can be used by any part of the system. It has an interface similar to the C library routines _malloc_ and _free_ that provide memory allocation to application programs <<biblio-mckusick-2, [McKusick & Karels, 1988]>>. Like the C library interface, the allocation routine takes a parameter specifying the size of memory that is needed. The range of sizes for memory requests is not constrained; however, physical memory is allocated and is not paged. The free routine takes a pointer to the storage being freed, but does not require the size of the piece of memory being freed.
|
||||
|
||||
[[overview-io-system]]
|
||||
=== I/O System
|
||||
|
||||
The basic model of the UNIX I/O system is a sequence of bytes that can be accessed either randomly or sequentially. There are no _access methods_ and no _control blocks_ in a typical UNIX user process.
|
||||
|
||||
Different programs expect various levels of structure, but the kernel does not impose structure on I/O. For instance, the convention for text files is lines of ASCII characters separated by a single newline character (the ASCII line-feed character), but the kernel knows nothing about this convention. For the purposes of most programs, the model is further simplified to being a stream of data bytes, or an _I/O stream_. It is this single common data form that makes the characteristic UNIX tool-based approach work <<biblio-kernighan, [Kernighan & Pike, 1984]>>. An I/O stream from one program can be fed as input to almost any other program. (This kind of traditional UNIX I/O stream should not be confused with the Eighth Edition stream I/O system or with the System V, Release 3 STREAMS, both of which can be accessed as traditional I/O streams.)
|
||||
|
||||
==== Descriptors and I/O
|
||||
|
||||
UNIX processes use _descriptors_ to reference I/O streams. Descriptors are small unsigned integers obtained from the _open_ and _socket_ system calls. The _open_ system call takes as arguments the name of a file and a permission mode to specify whether the file should be open for reading or for writing, or for both. This system call also can be used to create a new, empty file. A _read_ or _write_ system call can be applied to a descriptor to transfer data. The _close_ system call can be used to deallocate any descriptor.
|
||||
|
||||
Descriptors represent underlying objects supported by the kernel, and are created by system calls specific to the type of object. In 4.4BSD, three kinds of objects can be represented by descriptors: files, pipes, and sockets.
|
||||
|
||||
* A _file_ is a linear array of bytes with at least one name. A file exists until all its names are deleted explicitly and no process holds a descriptor for it. A process acquires a descriptor for a file by opening that file's name with the _open_ system call. I/O devices are accessed as files.
|
||||
* A _pipe_ is a linear array of bytes, as is a file, but it is used solely as an I/O stream, and it is unidirectional. It also has no name, and thus cannot be opened with _open_. Instead, it is created by the _pipe_ system call, which returns two descriptors, one of which accepts input that is sent to the other descriptor reliably, without duplication, and in order. The system also supports a named pipe or FIFO. A FIFO has properties identical to a pipe, except that it appears in the filesystem; thus, it can be opened using the _open_ system call. Two processes that wish to communicate each open the FIFO: One opens it for reading, the other for writing.
|
||||
* A _socket_ is a transient object that is used for interprocess communication; it exists only as long as some process holds a descriptor referring to it. A socket is created by the _socket_ system call, which returns a descriptor for it. There are different kinds of sockets that support various communication semantics, such as reliable delivery of data, preservation of message ordering, and preservation of message boundaries.
|
||||
|
||||
In systems before 4.2BSD, pipes were implemented using the filesystem; when sockets were introduced in 4.2BSD, pipes were reimplemented as sockets.
|
||||
|
||||
The kernel keeps for each process a _descriptor table_, which is a table that the kernel uses to translate the external representation of a descriptor into an internal representation. (The descriptor is merely an index into this table.) The descriptor table of a process is inherited from that process's parent, and thus access to the objects to which the descriptors refer also is inherited. The main ways that a process can obtain a descriptor are by opening or creation of an object, and by inheritance from the parent process. In addition, socket IPC allows passing of descriptors in messages between unrelated processes on the same machine.
|
||||
|
||||
Every valid descriptor has an associated _file offset_ in bytes from the beginning of the object. Read and write operations start at this offset, which is updated after each data transfer. For objects that permit random access, the file offset also may be set with the _lseek_ system call. Ordinary files permit random access, and some devices do, as well. Pipes and sockets do not.
|
||||
|
||||
When a process terminates, the kernel reclaims all the descriptors that were in use by that process. If the process was holding the final reference to an object, the object's manager is notified so that it can do any necessary cleanup actions, such as final deletion of a file or deallocation of a socket.
|
||||
|
||||
==== Descriptor Management
|
||||
|
||||
Most processes expect three descriptors to be open already when they start running. These descriptors are 0, 1, 2, more commonly known as _standard input_, _standard output_, and _standard error_, respectively. Usually, all three are associated with the user's terminal by the login process (see Section 14.6) and are inherited through _fork_ and _exec_ by processes run by the user. Thus, a program can read what the user types by reading standard input, and the program can send output to the user's screen by writing to standard output. The standard error descriptor also is open for writing and is used for error output, whereas standard output is used for ordinary output.
|
||||
|
||||
These (and other) descriptors can be mapped to objects other than the terminal; such mapping is called _I/O redirection_, and all the standard shells permit users to do it. The shell can direct the output of a program to a file by closing descriptor 1 (standard output) and opening the desired output file to produce a new descriptor 1. It can similarly redirect standard input to come from a file by closing descriptor 0 and opening the file.
|
||||
|
||||
Pipes allow the output of one program to be input to another program without rewriting or even relinking of either program. Instead of descriptor 1 (standard output) of the source program being set up to write to the terminal, it is set up to be the input descriptor of a pipe. Similarly, descriptor 0 (standard input) of the sink program is set up to reference the output of the pipe, instead of the terminal keyboard. The resulting set of two processes and the connecting pipe is known as a _pipeline_. Pipelines can be arbitrarily long series of processes connected by pipes.
|
||||
|
||||
The _open_, _pipe_, and _socket_ system calls produce new descriptors with the lowest unused number usable for a descriptor. For pipelines to work, some mechanism must be provided to map such descriptors into 0 and 1. The _dup_ system call creates a copy of a descriptor that points to the same file-table entry. The new descriptor is also the lowest unused one, but if the desired descriptor is closed first, _dup_ can be used to do the desired mapping. Care is required, however: If descriptor 1 is desired, and descriptor 0 happens also to have been closed, descriptor 0 will be the result. To avoid this problem, the system provides the _dup2_ system call; it is like _dup_, but it takes an additional argument specifying the number of the desired descriptor (if the desired descriptor was already open, _dup2_ closes it before reusing it).
|
||||
|
||||
==== Devices
|
||||
|
||||
Hardware devices have filenames, and may be accessed by the user via the same system calls used for regular files. The kernel can distinguish a _device special file_ or _special file_, and can determine to what device it refers, but most processes do not need to make this determination. Terminals, printers, and tape drives are all accessed as though they were streams of bytes, like 4.4BSD disk files. Thus, device dependencies and peculiarities are kept in the kernel as much as possible, and even in the kernel most of them are segregated in the device drivers.
|
||||
|
||||
Hardware devices can be categorized as either _structured_ or _unstructured_; they are known as _block_ or _character_ devices, respectively. Processes typically access devices through _special files_ in the filesystem. I/O operations to these files are handled by kernel-resident software modules termed _device drivers_. Most network-communication hardware devices are accessible through only the interprocess-communication facilities, and do not have special files in the filesystem name space, because the _raw-socket_ interface provides a more natural interface than does a special file.
|
||||
|
||||
Structured or block devices are typified by disks and magnetic tapes, and include most random-access devices. The kernel supports read-modify-write-type buffering actions on block-oriented structured devices to allow the latter to be read and written in a totally random byte-addressed fashion, like regular files. Filesystems are created on block devices.
|
||||
|
||||
Unstructured devices are those devices that do not support a block structure. Familiar unstructured devices are communication lines, raster plotters, and unbuffered magnetic tapes and disks. Unstructured devices typically support large block I/O transfers.
|
||||
|
||||
Unstructured files are called _character devices_ because the first of these to be implemented were terminal device drivers. The kernel interface to the driver for these devices proved convenient for other devices that were not block structured.
|
||||
|
||||
Device special files are created by the _mknod_ system call. There is an additional system call, _ioctl_, for manipulating the underlying device parameters of special files. The operations that can be done differ for each device. This system call allows the special characteristics of devices to be accessed, rather than overloading the semantics of other system calls. For example, there is an _ioctl_ on a tape drive to write an end-of-tape mark, instead of there being a special or modified version of _write_.
|
||||
|
||||
==== Socket IPC
|
||||
|
||||
The 4.2BSD kernel introduced an IPC mechanism more flexible than pipes, based on _sockets_. A socket is an endpoint of communication referred to by a descriptor, just like a file or a pipe. Two processes can each create a socket, and then connect those two endpoints to produce a reliable byte stream. Once connected, the descriptors for the sockets can be read or written by processes, just as the latter would do with a pipe. The transparency of sockets allows the kernel to redirect the output of one process to the input of another process residing on another machine. A major difference between pipes and sockets is that pipes require a common parent process to set up the communications channel. A connection between sockets can be set up by two unrelated processes, possibly residing on different machines.
|
||||
|
||||
System V provides local interprocess communication through FIFOs (also known as _named pipes_). FIFOs appear as an object in the filesystem that unrelated processes can open and send data through in the same way as they would communicate through a pipe. Thus, FIFOs do not require a common parent to set them up; they can be connected after a pair of processes are up and running. Unlike sockets, FIFOs can be used on only a local machine; they cannot be used to communicate between processes on different machines. FIFOs are implemented in 4.4BSD only because they are required by the POSIX.1 standard. Their functionality is a subset of the socket interface.
|
||||
|
||||
The socket mechanism requires extensions to the traditional UNIX I/O system calls to provide the associated naming and connection semantics. Rather than overloading the existing interface, the developers used the existing interfaces to the extent that the latter worked without being changed, and designed new interfaces to handle the added semantics. The _read_ and _write_ system calls were used for byte-stream type connections, but six new system calls were added to allow sending and receiving addressed messages such as network datagrams. The system calls for writing messages include _send_, _sendto_, and _sendmsg_. The system calls for reading messages include _recv_, _recvfrom_, and _recvmsg_. In retrospect, the first two in each class are special cases of the others; _recvfrom_ and _sendto_ probably should have been added as library interfaces to _recvmsg_ and _sendmsg_, respectively.
|
||||
|
||||
==== Scatter/Gather I/O
|
||||
|
||||
In addition to the traditional _read_ and _write_ system calls, 4.2BSD introduced the ability to do scatter/gather I/O. Scatter input uses the _readv_ system call to allow a single read to be placed in several different buffers. Conversely, the _writev_ system call allows several different buffers to be written in a single atomic write. Instead of passing a single buffer and length parameter, as is done with _read_ and _write_, the process passes in a pointer to an array of buffers and lengths, along with a count describing the size of the array.
|
||||
|
||||
This facility allows buffers in different parts of a process address space to be written atomically, without the need to copy them to a single contiguous buffer. Atomic writes are necessary in the case where the underlying abstraction is record based, such as tape drives that output a tape block on each write request. It is also convenient to be able to read a single request into several different buffers (such as a record header into one place and the data into another). Although an application can simulate the ability to scatter data by reading the data into a large buffer and then copying the pieces to their intended destinations, the cost of memory-to-memory copying in such cases often would more than double the running time of the affected application.
|
||||
|
||||
Just as _send_ and _recv_ could have been implemented as library interfaces to _sendto_ and _recvfrom_, it also would have been possible to simulate _read_ with _readv_ and _write_ with _writev_. However, _read_ and _write_ are used so much more frequently that the added cost of simulating them would not have been worthwhile.
|
||||
|
||||
==== Multiple Filesystem Support
|
||||
|
||||
With the expansion of network computing, it became desirable to support both local and remote filesystems. To simplify the support of multiple filesystems, the developers added a new virtual node or _vnode_ interface to the kernel. The set of operations exported from the vnode interface appear much like the filesystem operations previously supported by the local filesystem. However, they may be supported by a wide range of filesystem types:
|
||||
|
||||
* Local disk-based filesystems
|
||||
* Files imported using a variety of remote filesystem protocols
|
||||
* Read-only CD-ROM filesystems
|
||||
* Filesystems providing special-purpose interfaces -- for example, the `/proc` filesystem
|
||||
|
||||
A few variants of 4.4BSD, such as FreeBSD, allow filesystems to be loaded dynamically when the filesystems are first referenced by the _mount_ system call. The vnode interface is described in Section 6.5; its ancillary support routines are described in Section 6.6; several of the special-purpose filesystems are described in Section 6.7.
|
||||
|
||||
[[overview-filesystem]]
|
||||
=== Filesystems
|
||||
|
||||
A regular file is a linear array of bytes, and can be read and written starting at any byte in the file. The kernel distinguishes no record boundaries in regular files, although many programs recognize line-feed characters as distinguishing the ends of lines, and other programs may impose other structure. No system-related information about a file is kept in the file itself, but the filesystem stores a small amount of ownership, protection, and usage information with each file.
|
||||
|
||||
A _filename_ component is a string of up to 255 characters. These filenames are stored in a type of file called a _directory_. The information in a directory about a file is called a _directory entry_ and includes, in addition to the filename, a pointer to the file itself. Directory entries may refer to other directories, as well as to plain files. A hierarchy of directories and files is thus formed, and is called a _filesystem_;
|
||||
|
||||
.A small filesystem
|
||||
[[fig-small-fs]]
|
||||
image:fig2.png[A small filesystem]
|
||||
|
||||
a small one is shown in <<fig-small-fs>>. Directories may contain subdirectories, and there is no inherent limitation to the depth with which directory nesting may occur. To protect the consistency of the filesystem, the kernel does not permit processes to write directly into directories. A filesystem may include not only plain files and directories, but also references to other objects, such as devices and sockets.
|
||||
|
||||
The filesystem forms a tree, the beginning of which is the _root directory_, sometimes referred to by the name _slash_, spelled with a single solidus character (/). The root directory contains files; in our example in Fig 2.2, it contains `vmunix`, a copy of the kernel-executable object file. It also contains directories; in this example, it contains the `usr` directory. Within the `usr` directory is the `bin` directory, which mostly contains executable object code of programs, such as the files `ls` and `vi`.
|
||||
|
||||
A process identifies a file by specifying that file's _pathname_, which is a string composed of zero or more filenames separated by slash (/) characters. The kernel associates two directories with each process for use in interpreting pathnames. A process's _root directory_ is the topmost point in the filesystem that the process can access; it is ordinarily set to the root directory of the entire filesystem. A pathname beginning with a slash is called an _absolute pathname_, and is interpreted by the kernel starting with the process's root directory.
|
||||
|
||||
A pathname that does not begin with a slash is called a _relative pathname_, and is interpreted relative to the _current working directory_ of the process. (This directory also is known by the shorter names _current directory_ or _working directory_.) The current directory itself may be referred to directly by the name _dot_, spelled with a single period (`.`). The filename _dot-dot_ (`..`) refers to a directory's parent directory. The root directory is its own parent.
|
||||
|
||||
A process may set its root directory with the _chroot_ system call, and its current directory with the _chdir_ system call. Any process may do _chdir_ at any time, but _chroot_ is permitted only a process with superuser privileges. _Chroot_ is normally used to set up restricted access to the system.
|
||||
|
||||
Using the filesystem shown in Fig. 2.2, if a process has the root of the filesystem as its root directory, and has `/usr` as its current directory, it can refer to the file `vi` either from the root with the absolute pathname `/usr/bin/vi`, or from its current directory with the relative pathname `bin/vi`.
|
||||
|
||||
System utilities and databases are kept in certain well-known directories. Part of the well-defined hierarchy includes a directory that contains the _home directory_ for each user -- for example, `/usr/staff/mckusick` and `/usr/staff/karels` in Fig. 2.2. When users log in, the current working directory of their shell is set to the home directory. Within their home directories, users can create directories as easily as they can regular files. Thus, a user can build arbitrarily complex subhierarchies.
|
||||
|
||||
The user usually knows of only one filesystem, but the system may know that this one virtual filesystem is really composed of several physical filesystems, each on a different device. A physical filesystem may not span multiple hardware devices. Since most physical disk devices are divided into several logical devices, there may be more than one filesystem per physical device, but there will be no more than one per logical device. One filesystem -- the filesystem that anchors all absolute pathnames -- is called the _root filesystem_, and is always available. Others may be mounted; that is, they may be integrated into the directory hierarchy of the root filesystem. References to a directory that has a filesystem mounted on it are converted transparently by the kernel into references to the root directory of the mounted filesystem.
|
||||
|
||||
The _link_ system call takes the name of an existing file and another name to create for that file. After a successful _link_, the file can be accessed by either filename. A filename can be removed with the _unlink_ system call. When the final name for a file is removed (and the final process that has the file open closes it), the file is deleted.
|
||||
|
||||
Files are organized hierarchically in _directories_. A directory is a type of file, but, in contrast to regular files, a directory has a structure imposed on it by the system. A process can read a directory as it would an ordinary file, but only the kernel is permitted to modify a directory. Directories are created by the _mkdir_ system call and are removed by the _rmdir_ system call. Before 4.2BSD, the _mkdir_ and _rmdir_ system calls were implemented by a series of _link_ and _unlink_ system calls being done. There were three reasons for adding systems calls explicitly to create and delete directories:
|
||||
|
||||
[arabic]
|
||||
. The operation could be made atomic. If the system crashed, the directory would not be left half-constructed, as could happen when a series of link operations were used.
|
||||
. When a networked filesystem is being run, the creation and deletion of files and directories need to be specified atomically so that they can be serialized.
|
||||
. When supporting non-UNIX filesystems, such as an MS-DOS filesystem, on another partition of the disk, the other filesystem may not support link operations. Although other filesystems might support the concept of directories, they probably would not create and delete the directories with links, as the UNIX filesystem does. Consequently, they could create and delete directories only if explicit directory create and delete requests were presented.
|
||||
|
||||
The _chown_ system call sets the owner and group of a file, and _chmod_ changes protection attributes. _Stat_ applied to a filename can be used to read back such properties of a file. The _fchown_, _fchmod_, and _fstat_ system calls are applied to a descriptor, instead of to a filename, to do the same set of operations. The _rename_ system call can be used to give a file a new name in the filesystem, replacing one of the file's old names. Like the directory-creation and directory-deletion operations, the _rename_ system call was added to 4.2BSD to provide atomicity to name changes in the local filesystem. Later, it proved useful explicitly to export renaming operations to foreign filesystems and over the network.
|
||||
|
||||
The _truncate_ system call was added to 4.2BSD to allow files to be shortened to an arbitrary offset. The call was added primarily in support of the Fortran run-time library, which has the semantics such that the end of a random-access file is set to be wherever the program most recently accessed that file. Without the _truncate_ system call, the only way to shorten a file was to copy the part that was desired to a new file, to delete the old file, then to rename the copy to the original name. As well as this algorithm being slow, the library could potentially fail on a full filesystem.
|
||||
|
||||
Once the filesystem had the ability to shorten files, the kernel took advantage of that ability to shorten large empty directories. The advantage of shortening empty directories is that it reduces the time spent in the kernel searching them when names are being created or deleted.
|
||||
|
||||
Newly created files are assigned the user identifier of the process that created them and the group identifier of the directory in which they were created. A three-level access-control mechanism is provided for the protection of files. These three levels specify the accessibility of a file to
|
||||
|
||||
[arabic]
|
||||
. The user who owns the file
|
||||
. The group that owns the file
|
||||
. Everyone else
|
||||
|
||||
Each level of access has separate indicators for read permission, write permission, and execute permission.
|
||||
|
||||
Files are created with zero length, and may grow when they are written. While a file is open, the system maintains a pointer into the file indicating the current location in the file associated with the descriptor. This pointer can be moved about in the file in a random-access fashion. Processes sharing a file descriptor through a _fork_ or _dup_ system call share the current location pointer. Descriptors created by separate _open_ system calls have separate current location pointers. Files may have _holes_ in them. Holes are void areas in the linear extent of the file where data have never been written. A process can create these holes by positioning the pointer past the current end-of-file and writing. When read, holes are treated by the system as zero-valued bytes.
|
||||
|
||||
Earlier UNIX systems had a limit of 14 characters per filename component. This limitation was often a problem. For example, in addition to the natural desire of users to give files long descriptive names, a common way of forming filenames is as `basename.extension`, where the extension (indicating the kind of file, such as `.c` for C source or `.o` for intermediate binary object) is one to three characters, leaving 10 to 12 characters for the basename. Source-code-control systems and editors usually take up another two characters, either as a prefix or a suffix, for their purposes, leaving eight to 10 characters. It is easy to use 10 or 12 characters in a single English word as a basename (e.g., ``multiplexer'').
|
||||
|
||||
It is possible to keep within these limits, but it is inconvenient or even dangerous, because other UNIX systems accept strings longer than the limit when creating files, but then _truncate_ to the limit. A C language source file named `multiplexer.c` (already 13 characters) might have a source-code-control file with `s.` prepended, producing a filename `s.multiplexer` that is indistinguishable from the source-code-control file for `multiplexer.ms`, a file containing `troff` source for documentation for the C program. The contents of the two original files could easily get confused with no warning from the source-code-control system. Careful coding can detect this problem, but the long filenames first introduced in 4.2BSD practically eliminate it.
|
||||
|
||||
[[overview-filestore]]
|
||||
=== Filestores
|
||||
|
||||
The operations defined for local filesystems are divided into two parts. Common to all local filesystems are hierarchical naming, locking, quotas, attribute management, and protection. These features are independent of how the data will be stored. 4.4BSD has a single implementation to provide these semantics.
|
||||
|
||||
The other part of the local filesystem is the organization and management of the data on the storage media. Laying out the contents of files on the storage media is the responsibility of the filestore. 4.4BSD supports three different filestore layouts:
|
||||
|
||||
* The traditional Berkeley Fast Filesystem
|
||||
* The log-structured filesystem, based on the Sprite operating-system design <<biblio-rosenblum, [Rosenblum & Ousterhout, 1992]>>
|
||||
* A memory-based filesystem
|
||||
|
||||
Although the organizations of these filestores are completely different, these differences are indistinguishable to the processes using the filestores.
|
||||
|
||||
The Fast Filesystem organizes data into cylinder groups. Files that are likely to be accessed together, based on their locations in the filesystem hierarchy, are stored in the same cylinder group. Files that are not expected to accessed together are moved into different cylinder groups. Thus, files written at the same time may be placed far apart on the disk.
|
||||
|
||||
The log-structured filesystem organizes data as a log. All data being written at any point in time are gathered together, and are written at the same disk location. Data are never overwritten; instead, a new copy of the file is written that replaces the old one. The old files are reclaimed by a garbage-collection process that runs when the filesystem becomes full and additional free space is needed.
|
||||
|
||||
The memory-based filesystem is designed to store data in virtual memory. It is used for filesystems that need to support fast but temporary data, such as `/tmp`. The goal of the memory-based filesystem is to keep the storage packed as compactly as possible to minimize the usage of virtual-memory resources.
|
||||
|
||||
[[overview-nfs]]
|
||||
=== Network Filesystem
|
||||
|
||||
Initially, networking was used to transfer data from one machine to another. Later, it evolved to allowing users to log in remotely to another machine. The next logical step was to bring the data to the user, instead of having the user go to the data -- and network filesystems were born. Users working locally do not experience the network delays on each keystroke, so they have a more responsive environment.
|
||||
|
||||
Bringing the filesystem to a local machine was among the first of the major client-server applications. The _server_ is the remote machine that exports one or more of its filesystems. The _client_ is the local machine that imports those filesystems. From the local client's point of view, a remotely mounted filesystem appears in the file-tree name space just like any other locally mounted filesystem. Local clients can change into directories on the remote filesystem, and can read, write, and execute binaries within that remote filesystem identically to the way that they can do these operations on a local filesystem.
|
||||
|
||||
When the local client does an operation on a remote filesystem, the request is packaged and is sent to the server. The server does the requested operation and returns either the requested information or an error indicating why the request was denied. To get reasonable performance, the client must cache frequently accessed data. The complexity of remote filesystems lies in maintaining cache consistency between the server and its many clients.
|
||||
|
||||
Although many remote-filesystem protocols have been developed over the years, the most pervasive one in use among UNIX systems is the Network Filesystem (NFS), whose protocol and most widely used implementation were done by Sun Microsystems. The 4.4BSD kernel supports the NFS protocol, although the implementation was done independently from the protocol specification <<biblio-macklem, [Macklem, 1994]>>. The NFS protocol is described in Chapter 9.
|
||||
|
||||
[[overview-terminal]]
|
||||
=== Terminals
|
||||
|
||||
Terminals support the standard system I/O operations, as well as a collection of terminal-specific operations to control input-character editing and output delays. At the lowest level are the terminal device drivers that control the hardware terminal ports. Terminal input is handled according to the underlying communication characteristics, such as baud rate, and according to a set of software-controllable parameters, such as parity checking.
|
||||
|
||||
Layered above the terminal device drivers are line disciplines that provide various degrees of character processing. The default line discipline is selected when a port is being used for an interactive login. The line discipline is run in _canonical mode_; input is processed to provide standard line-oriented editing functions, and input is presented to a process on a line-by-line basis.
|
||||
|
||||
Screen editors and programs that communicate with other computers generally run in _noncanonical mode_ (also commonly referred to as _raw mode_ or _character-at-a-time mode_). In this mode, input is passed through to the reading process immediately and without interpretation. All special-character input processing is disabled, no erase or other line editing processing is done, and all characters are passed to the program that is reading from the terminal.
|
||||
|
||||
It is possible to configure the terminal in thousands of combinations between these two extremes. For example, a screen editor that wanted to receive user interrupts asynchronously might enable the special characters that generate signals and enable output flow control, but otherwise run in noncanonical mode; all other characters would be passed through to the process uninterpreted.
|
||||
|
||||
On output, the terminal handler provides simple formatting services, including
|
||||
|
||||
* Converting the line-feed character to the two-character carriage-return-line-feed sequence
|
||||
* Inserting delays after certain standard control characters
|
||||
* Expanding tabs
|
||||
* Displaying echoed nongraphic ASCII characters as a two-character sequence of the form ``^C'' (i.e., the ASCII caret character followed by the ASCII character that is the character's value offset from the ASCII ``@'' character).
|
||||
|
||||
Each of these formatting services can be disabled individually by a process through control requests.
|
||||
|
||||
[[overview-ipc]]
|
||||
=== Interprocess Communication
|
||||
|
||||
Interprocess communication in 4.4BSD is organized in _communication domains_. Domains currently supported include the _local domain_, for communication between processes executing on the same machine; the _internet domain_, for communication between processes using the TCP/IP protocol suite (perhaps within the Internet); the ISO/OSI protocol family for communication between sites required to run them; and the _XNS domain_, for communication between processes using the XEROX Network Systems (XNS) protocols.
|
||||
|
||||
Within a domain, communication takes place between communication endpoints known as _sockets_. As mentioned in Section 2.6, the _socket_ system call creates a socket and returns a descriptor; other IPC system calls are described in Chapter 11. Each socket has a type that defines its communications semantics; these semantics include properties such as reliability, ordering, and prevention of duplication of messages.
|
||||
|
||||
Each socket has associated with it a _communication protocol_. This protocol provides the semantics required by the socket according to the latter's type. Applications may request a specific protocol when creating a socket, or may allow the system to select a protocol that is appropriate for the type of socket being created.
|
||||
|
||||
Sockets may have addresses bound to them. The form and meaning of socket addresses are dependent on the communication domain in which the socket is created. Binding a name to a socket in the local domain causes a file to be created in the filesystem.
|
||||
|
||||
Normal data transmitted and received through sockets are untyped. Data-representation issues are the responsibility of libraries built on top of the interprocess-communication facilities. In addition to transporting normal data, communication domains may support the transmission and reception of specially typed data, termed _access rights_. The local domain, for example, uses this facility to pass descriptors between processes.
|
||||
|
||||
Networking implementations on UNIX before 4.2BSD usually worked by overloading the character-device interfaces. One goal of the socket interface was for naive programs to be able to work without change on stream-style connections. Such programs can work only if the _read_ and _write_ systems calls are unchanged. Consequently, the original interfaces were left intact, and were made to work on stream-type sockets. A new interface was added for more complicated sockets, such as those used to send datagrams, with which a destination address must be presented with each _send_ call.
|
||||
|
||||
Another benefit is that the new interface is highly portable. Shortly after a test release was available from Berkeley, the socket interface had been ported to System III by a UNIX vendor (although AT&T did not support the socket interface until the release of System V Release 4, deciding instead to use the Eighth Edition stream mechanism). The socket interface was also ported to run in many Ethernet boards by vendors, such as Excelan and Interlan, that were selling into the PC market, where the machines were too small to run networking in the main processor. More recently, the socket interface was used as the basis for Microsoft's Winsock networking interface for Windows.
|
||||
|
||||
[[overview-network-communication]]
|
||||
=== Network Communication
|
||||
|
||||
Some of the communication domains supported by the _socket_ IPC mechanism provide access to network protocols. These protocols are implemented as a separate software layer logically below the socket software in the kernel. The kernel provides many ancillary services, such as buffer management, message routing, standardized interfaces to the protocols, and interfaces to the network interface drivers for the use of the various network protocols.
|
||||
|
||||
At the time that 4.2BSD was being implemented, there were many networking protocols in use or under development, each with its own strengths and weaknesses. There was no clearly superior protocol or protocol suite. By supporting multiple protocols, 4.2BSD could provide interoperability and resource sharing among the diverse set of machines that was available in the Berkeley environment. Multiple-protocol support also provides for future changes. Today's protocols designed for 10- to 100-Mbit-per-second Ethernets are likely to be inadequate for tomorrow's 1- to 10-Gbit-per-second fiber-optic networks. Consequently, the network-communication layer is designed to support multiple protocols. New protocols are added to the kernel without the support for older protocols being affected. Older applications can continue to operate using the old protocol over the same physical network as is used by newer applications running with a newer network protocol.
|
||||
|
||||
[[overview-network-implementation]]
|
||||
=== Network Implementation
|
||||
|
||||
The first protocol suite implemented in 4.2BSD was DARPA's Transmission Control Protocol/Internet Protocol (TCP/IP). The CSRG chose TCP/IP as the first network to incorporate into the socket IPC framework, because a 4.1BSD-based implementation was publicly available from a DARPA-sponsored project at Bolt, Beranek, and Newman (BBN). That was an influential choice: The 4.2BSD implementation is the main reason for the extremely widespread use of this protocol suite. Later performance and capability improvements to the TCP/IP implementation have also been widely adopted. The TCP/IP implementation is described in detail in Chapter 13.
|
||||
|
||||
The release of 4.3BSD added the Xerox Network Systems (XNS) protocol suite, partly building on work done at the University of Maryland and at Cornell University. This suite was needed to connect isolated machines that could not communicate using TCP/IP.
|
||||
|
||||
The release of 4.4BSD added the ISO protocol suite because of the latter's increasing visibility both within and outside the United States. Because of the somewhat different semantics defined for the ISO protocols, some minor changes were required in the socket interface to accommodate these semantics. The changes were made such that they were invisible to clients of other existing protocols. The ISO protocols also required extensive addition to the two-level routing tables provided by the kernel in 4.3BSD. The greatly expanded routing capabilities of 4.4BSD include arbitrary levels of routing with variable-length addresses and network masks.
|
||||
|
||||
[[overview-operation]]
|
||||
=== System Operation
|
||||
|
||||
Bootstrapping mechanisms are used to start the system running. First, the 4.4BSD kernel must be loaded into the main memory of the processor. Once loaded, it must go through an initialization phase to set the hardware into a known state. Next, the kernel must do autoconfiguration, a process that finds and configures the peripherals that are attached to the processor. The system begins running in single-user mode while a start-up script does disk checks and starts the accounting and quota checking. Finally, the start-up script starts the general system services and brings up the system to full multiuser operation.
|
||||
|
||||
During multiuser operation, processes wait for login requests on the terminal lines and network ports that have been configured for user access. When a login request is detected, a login process is spawned and user validation is done. When the login validation is successful, a login shell is created from which the user can run additional processes.
|
||||
|
||||
:sectnums!:
|
||||
|
||||
[bibliography]
|
||||
[[references]]
|
||||
== References
|
||||
|
||||
[[biblio-accetta]] Accetta et al, 1986 Mach: A New Kernel Foundation for UNIX Development" M.Accetta R.Baron W.Bolosky D.Golub R.Rashid A.Tevanian M.Young 93-113 USENIX Association Conference Proceedings USENIX Association June 1986
|
||||
|
||||
[[biblio-cheriton]] Cheriton, 1988 The V Distributed System D. R.Cheriton 314-333 Comm ACM, 31, 3 March 1988
|
||||
|
||||
[[biblio-ewens]] Ewens et al, 1985 Tunis: A Distributed Multiprocessor Operating System P.Ewens D. R.Blythe M.Funkenhauser R. C.Holt 247-254 USENIX Assocation Conference Proceedings USENIX Association June 1985
|
||||
|
||||
[[biblio-gingell]] Gingell et al, 1987 Virtual Memory Architecture in SunOS R.Gingell J.Moran W.Shannon 81-94 USENIX Association Conference Proceedings USENIX Association June 1987
|
||||
|
||||
[[biblio-kernighan]] Kernighan & Pike, 1984 The UNIX Programming Environment B. W.Kernighan R.Pike Prentice-Hall Englewood Cliffs NJ 1984
|
||||
|
||||
[[biblio-macklem]] Macklem, 1994 The 4.4BSD NFS Implementation R.Macklem 6:1-14 4.4BSD System Manager's Manual O'Reilly & Associates, Inc. Sebastopol CA 1994
|
||||
|
||||
[[biblio-mckusick-2]] McKusick & Karels, 1988 Design of a General Purpose Memory Allocator for the 4.3BSD UNIX Kernel M. K.McKusick M. J.Karels 295-304 USENIX Assocation Conference Proceedings USENIX Assocation June 1998
|
||||
|
||||
[[biblio-mckusick-1]] McKusick et al, 1994 Berkeley Software Architecture Manual, 4.4BSD Edition M. K.McKusick M. J.Karels S. J.Leffler W. N.Joy R. S.Faber 5:1-42 4.4BSD Programmer's Supplementary Documents O'Reilly & Associates, Inc. Sebastopol CA 1994
|
||||
|
||||
[[biblio-ritchie]] Ritchie, 1988 Early Kernel Design private communication D. M.Ritchie March 1988
|
||||
|
||||
[[biblio-rosenblum]] Rosenblum & Ousterhout, 1992 The Design and Implementation of a Log-Structured File System M.Rosenblum K.Ousterhout 26-52 ACM Transactions on Computer Systems, 10, 1 Association for Computing Machinery February 1992
|
||||
|
||||
[[biblio-rozier]] Rozier et al, 1988 Chorus Distributed Operating Systems M.Rozier V.Abrossimov F.Armand I.Boule M.Gien M.Guillemont F.Herrmann C.Kaiser S.Langlois P.Leonard W.Neuhauser 305-370 USENIX Computing Systems, 1, 4 Fall 1988
|
||||
|
||||
[[biblio-tevanian]] Tevanian, 1987 Architecture-Independent Virtual Memory Management for Parallel and Distributed Environments: The Mach Approach Technical Report CMU-CS-88-106, A.Tevanian Department of Computer Science, Carnegie-Mellon University Pittsburgh PA December 1987
|
||||
|
|
@ -0,0 +1 @@
|
|||
_index.adoc
|
||||
1000
documentation/content/en/books/dev-model/_index.adoc
Normal file
1000
documentation/content/en/books/dev-model/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1 @@
|
|||
_index.adoc
|
||||
109
documentation/content/en/books/developers-handbook/_index.adoc
Normal file
109
documentation/content/en/books/developers-handbook/_index.adoc
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
---
|
||||
title: FreeBSD Developers' Handbook
|
||||
authors:
|
||||
- author: The FreeBSD Documentation Project
|
||||
copyright: 1995-2020 The FreeBSD Documentation Project
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["freebsd", "apple", "ibm", "ieee", "intel", "linux", "microsoft", "opengroup", "sun", "general"]
|
||||
---
|
||||
|
||||
= FreeBSD Developers' Handbook
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 2
|
||||
:icons: font
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:partnums:
|
||||
:chapter-signifier: Chapter
|
||||
:part-signifier: Part
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:book: true
|
||||
:pdf: false
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
:imagesdir: ../../../../images/books/developers-handbook/
|
||||
:chapters-path: content/en/books/developers-handbook/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:imagesdir: ../../../static/images/books/developers-handbook/
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:imagesdir: ../../../static/images/books/developers-handbook/
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
[.abstract-title]
|
||||
[abstract]
|
||||
Abstract
|
||||
|
||||
Welcome to the Developers' Handbook. This manual is a _work in progress_ and is the work of many individuals. Many sections do not yet exist and some of those that do exist need to be updated. If you are interested in helping with this project, send email to the {freebsd-doc}.
|
||||
|
||||
The latest version of this document is always available from the link:https://www.FreeBSD.org[FreeBSD World Wide Web server]. It may also be downloaded in a variety of formats and compression options from the link:https://download.freebsd.org/ftp/doc/[FreeBSD FTP server] or one of the numerous link:{handbook}#mirrors-ftp/[mirror sites].
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
// Section one
|
||||
[[basics]]
|
||||
= Basics
|
||||
|
||||
include::{chapters-path}introduction/chapter.adoc[leveloffset=+1, lines=10..24;35..-1]
|
||||
include::{chapters-path}tools/chapter.adoc[leveloffset=+1, lines=10..26;37..-1]
|
||||
include::{chapters-path}secure/chapter.adoc[leveloffset=+1, lines=9..23;34..-1]
|
||||
include::{chapters-path}l10n/chapter.adoc[leveloffset=+1, lines=8..22;33..-1]
|
||||
include::{chapters-path}policies/chapter.adoc[leveloffset=+1, lines=10..24;35..-1]
|
||||
include::{chapters-path}testing/chapter.adoc[leveloffset=+1, lines=8..22;33..-1]
|
||||
|
||||
// Section two
|
||||
[[ipc]]
|
||||
= Interprocess Communication
|
||||
|
||||
include::{chapters-path}sockets/chapter.adoc[leveloffset=+1, lines=9..23;35..-1]
|
||||
include::{chapters-path}ipv6/chapter.adoc[leveloffset=+1, lines=9..23;34..-1]
|
||||
|
||||
// Section three
|
||||
[[kernel]]
|
||||
= Kernel
|
||||
|
||||
include::{chapters-path}kernelbuild/chapter.adoc[leveloffset=+1, lines=8..22;33..-1]
|
||||
include::{chapters-path}kerneldebug/chapter.adoc[leveloffset=+1, lines=11..25;36..-1]
|
||||
|
||||
// Section four
|
||||
[[architectures]]
|
||||
= Architectures
|
||||
|
||||
include::{chapters-path}x86/chapter.adoc[leveloffset=+1, lines=8..22;33..-1]
|
||||
|
||||
// Appendices
|
||||
[[appendices]]
|
||||
= Appendices
|
||||
|
||||
include::{chapters-path}bibliography/chapter.adoc[leveloffset=+1, lines=6..20;29..-1]
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
title: Appendices
|
||||
prev: books/developers-handbook/partv
|
||||
---
|
||||
|
||||
[appendix]
|
||||
[[bibliography]]
|
||||
= Appendices
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums!:
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: A
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
[[COD,1]] [1] Dave A Patterson and John L Hennessy. Copyright(R) 1998 Morgan Kaufmann Publishers, Inc. 1-55860-428-6. Morgan Kaufmann Publishers, Inc. Computer Organization and Design. The Hardware / Software Interface. 1-2.
|
||||
|
||||
[[APUE, 2]] [2] W. Richard Stevens. Copyright(R) 1993 Addison Wesley Longman, Inc. 0-201-56317-7. Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment. 1-2.
|
||||
|
||||
[[DIFOS, 3]] [3] Marshall Kirk McKusick and George Neville-Neil. Copyright(R) 2004 Addison-Wesley. 0-201-70245-2. Addison-Wesley. The Design and Implementation of the FreeBSD Operating System. 1-2.
|
||||
|
||||
[[Phrack, 4]] [4] Aleph One. Phrack 49; "Smashing the Stack for Fun and Profit".
|
||||
|
||||
[[StackGuard, 5]] [5] Chrispin Cowan, Calton Pu, and Dave Maier. StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks.
|
||||
|
||||
[[OpenBSD, 6]] [6] Todd Miller and Theo de Raadt. strlcpy and strlcat -- consistent, safe string copy and concatenation.
|
||||
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
introduction/chapter.adoc
|
||||
tools/chapter.adoc
|
||||
secure/chapter.adoc
|
||||
l10n/chapter.adoc
|
||||
policies/chapter.adoc
|
||||
testing/chapter.adoc
|
||||
sockets/chapter.adoc
|
||||
ipv6/chapter.adoc
|
||||
kernelbuild/chapter.adoc
|
||||
kerneldebug/chapter.adoc
|
||||
x86/chapter.adoc
|
||||
bibliography/chapter.adoc
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
---
|
||||
title: Chapter 1. Introduction
|
||||
authors:
|
||||
- author: Murray Stokely
|
||||
- author: Jeroen Ruigrok van der Werven
|
||||
prev: books/developers-handbook/parti
|
||||
next: books/developers-handbook/tools
|
||||
---
|
||||
|
||||
[[introduction]]
|
||||
= Introduction
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 1
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[introduction-devel]]
|
||||
== Developing on FreeBSD
|
||||
|
||||
So here we are. System all installed and you are ready to start programming. But where to start? What does FreeBSD provide? What can it do for me, as a programmer?
|
||||
|
||||
These are some questions which this chapter tries to answer. Of course, programming has different levels of proficiency like any other trade. For some it is a hobby, for others it is their profession. The information in this chapter might be aimed toward the beginning programmer; indeed, it could serve useful for the programmer unfamiliar with the FreeBSD platform.
|
||||
|
||||
[[introduction-bsdvision]]
|
||||
== The BSD Vision
|
||||
|
||||
To produce the best UNIX(R) like operating system package possible, with due respect to the original software tools ideology as well as usability, performance and stability.
|
||||
|
||||
[[introduction-archguide]]
|
||||
== Architectural Guidelines
|
||||
|
||||
Our ideology can be described by the following guidelines
|
||||
|
||||
* Do not add new functionality unless an implementor cannot complete a real application without it.
|
||||
* It is as important to decide what a system is not as to decide what it is. Do not serve all the world's needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.
|
||||
* The only thing worse than generalizing from one example is generalizing from no examples at all.
|
||||
* If a problem is not completely understood, it is probably best to provide no solution at all.
|
||||
* If you can get 90 percent of the desired effect for 10 percent of the work, use the simpler solution.
|
||||
* Isolate complexity as much as possible.
|
||||
* Provide mechanism, rather than policy. In particular, place user interface policy in the client's hands.
|
||||
|
||||
From Scheifler & Gettys: "X Window System"
|
||||
|
||||
[[introduction-layout]]
|
||||
== The Layout of [.filename]#/usr/src#
|
||||
|
||||
The complete source code to FreeBSD is available from our public repository. The source code is normally installed in [.filename]#/usr/src# which contains the following subdirectories:
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Directory
|
||||
| Description
|
||||
|
||||
|[.filename]#bin/#
|
||||
|Source for files in [.filename]#/bin#
|
||||
|
||||
|[.filename]#cddl/#
|
||||
|Utilities covered by the Common Development and Distribution License
|
||||
|
||||
|[.filename]#contrib/#
|
||||
|Source for files from contributed software
|
||||
|
||||
|[.filename]#crypto/#
|
||||
|Cryptographical sources
|
||||
|
||||
|[.filename]#etc/#
|
||||
|Source for files in [.filename]#/etc#
|
||||
|
||||
|[.filename]#gnu/#
|
||||
|Utilities covered by the GNU Public License
|
||||
|
||||
|[.filename]#include/#
|
||||
|Source for files in [.filename]#/usr/include#
|
||||
|
||||
|[.filename]#kerberos5/#
|
||||
|Source for Kerberos version 5
|
||||
|
||||
|[.filename]#lib/#
|
||||
|Source for files in [.filename]#/usr/lib#
|
||||
|
||||
|[.filename]#libexec/#
|
||||
|Source for files in [.filename]#/usr/libexec#
|
||||
|
||||
|[.filename]#release/#
|
||||
|Files required to produce a FreeBSD release
|
||||
|
||||
|[.filename]#rescue/#
|
||||
|Build system for the [.filename]#/rescue# utilities
|
||||
|
||||
|[.filename]#sbin/#
|
||||
|Source for files in [.filename]#/sbin#
|
||||
|
||||
|[.filename]#secure/#
|
||||
|Contributed cryptographic sources
|
||||
|
||||
|[.filename]#share/#
|
||||
|Source for files in [.filename]#/usr/share#
|
||||
|
||||
|[.filename]#sys/#
|
||||
|Kernel source files
|
||||
|
||||
|[.filename]#tests/#
|
||||
|The FreeBSD test suite
|
||||
|
||||
|[.filename]#tools/#
|
||||
|Tools used for maintenance and testing of FreeBSD
|
||||
|
||||
|[.filename]#usr.bin/#
|
||||
|Source for files in [.filename]#/usr/bin#
|
||||
|
||||
|[.filename]#usr.sbin/#
|
||||
|Source for files in [.filename]#/usr/sbin#
|
||||
|===
|
||||
|
|
@ -0,0 +1,665 @@
|
|||
---
|
||||
title: Chapter 8. IPv6 Internals
|
||||
authors:
|
||||
- author: Yoshinobu Inoue
|
||||
prev: books/developers-handbook/sockets
|
||||
next: books/developers-handbook/partiii
|
||||
---
|
||||
|
||||
[[ipv6]]
|
||||
= IPv6 Internals
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 8
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[ipv6-implementation]]
|
||||
== IPv6/IPsec Implementation
|
||||
|
||||
This section should explain IPv6 and IPsec related implementation internals. These functionalities are derived from http://www.kame.net/[KAME project]
|
||||
|
||||
[[ipv6details]]
|
||||
=== IPv6
|
||||
|
||||
==== Conformance
|
||||
|
||||
The IPv6 related functions conforms, or tries to conform to the latest set of IPv6 specifications. For future reference we list some of the relevant documents below (_NOTE_: this is not a complete list - this is too hard to maintain...).
|
||||
|
||||
For details please refer to specific chapter in the document, RFCs, manual pages, or comments in the source code.
|
||||
|
||||
Conformance tests have been performed on the KAME STABLE kit at TAHI project. Results can be viewed at http://www.tahi.org/report/KAME/[http://www.tahi.org/report/KAME/]. We also attended University of New Hampshire IOL tests (http://www.iol.unh.edu/[http://www.iol.unh.edu/]) in the past, with our past snapshots.
|
||||
|
||||
* RFC1639: FTP Operation Over Big Address Records (FOOBAR)
|
||||
|
||||
** RFC2428 is preferred over RFC1639. FTP clients will first try RFC2428, then RFC1639 if failed.
|
||||
|
||||
* RFC1886: DNS Extensions to support IPv6
|
||||
* RFC1933: Transition Mechanisms for IPv6 Hosts and Routers
|
||||
|
||||
** IPv4 compatible address is not supported.
|
||||
** automatic tunneling (described in 4.3 of this RFC) is not supported.
|
||||
** man:gif[4] interface implements IPv[46]-over-IPv[46] tunnel in a generic way, and it covers "configured tunnel" described in the spec. See <<gif,23.5.1.5>> in this document for details.
|
||||
|
||||
* RFC1981: Path MTU Discovery for IPv6
|
||||
* RFC2080: RIPng for IPv6
|
||||
|
||||
** usr.sbin/route6d support this.
|
||||
|
||||
* RFC2292: Advanced Sockets API for IPv6
|
||||
|
||||
** For supported library functions/kernel APIs, see [.filename]#sys/netinet6/ADVAPI#.
|
||||
|
||||
* RFC2362: Protocol Independent Multicast-Sparse Mode (PIM-SM)
|
||||
|
||||
** RFC2362 defines packet formats for PIM-SM. [.filename]#draft-ietf-pim-ipv6-01.txt# is written based on this.
|
||||
|
||||
* RFC2373: IPv6 Addressing Architecture
|
||||
|
||||
** supports node required addresses, and conforms to the scope requirement.
|
||||
|
||||
* RFC2374: An IPv6 Aggregatable Global Unicast Address Format
|
||||
|
||||
** supports 64-bit length of Interface ID.
|
||||
|
||||
* RFC2375: IPv6 Multicast Address Assignments
|
||||
|
||||
** Userland applications use the well-known addresses assigned in the RFC.
|
||||
|
||||
* RFC2428: FTP Extensions for IPv6 and NATs
|
||||
|
||||
** RFC2428 is preferred over RFC1639. FTP clients will first try RFC2428, then RFC1639 if failed.
|
||||
|
||||
* RFC2460: IPv6 specification
|
||||
* RFC2461: Neighbor discovery for IPv6
|
||||
|
||||
** See <<neighbor-discovery,23.5.1.2>> in this document for details.
|
||||
|
||||
* RFC2462: IPv6 Stateless Address Autoconfiguration
|
||||
|
||||
** See <<ipv6-pnp,23.5.1.4>> in this document for details.
|
||||
|
||||
* RFC2463: ICMPv6 for IPv6 specification
|
||||
|
||||
** See <<icmpv6,23.5.1.9>> in this document for details.
|
||||
|
||||
* RFC2464: Transmission of IPv6 Packets over Ethernet Networks
|
||||
* RFC2465: MIB for IPv6: Textual Conventions and General Group
|
||||
|
||||
** Necessary statistics are gathered by the kernel. Actual IPv6 MIB support is provided as a patchkit for ucd-snmp.
|
||||
|
||||
* RFC2466: MIB for IPv6: ICMPv6 group
|
||||
|
||||
** Necessary statistics are gathered by the kernel. Actual IPv6 MIB support is provided as patchkit for ucd-snmp.
|
||||
|
||||
* RFC2467: Transmission of IPv6 Packets over FDDI Networks
|
||||
* RFC2497: Transmission of IPv6 packet over ARCnet Networks
|
||||
* RFC2553: Basic Socket Interface Extensions for IPv6
|
||||
|
||||
** IPv4 mapped address (3.7) and special behavior of IPv6 wildcard bind socket (3.8) are supported. See <<ipv6-wildcard-socket,23.5.1.12>> in this document for details.
|
||||
|
||||
* RFC2675: IPv6 Jumbograms
|
||||
|
||||
** See <<ipv6-jumbo,23.5.1.7>> in this document for details.
|
||||
|
||||
* RFC2710: Multicast Listener Discovery for IPv6
|
||||
* RFC2711: IPv6 router alert option
|
||||
* [.filename]#draft-ietf-ipngwg-router-renum-08#: Router renumbering for IPv6
|
||||
* [.filename]#draft-ietf-ipngwg-icmp-namelookups-02#: IPv6 Name Lookups Through ICMP
|
||||
* [.filename]#draft-ietf-ipngwg-icmp-name-lookups-03#: IPv6 Name Lookups Through ICMP
|
||||
* [.filename]#draft-ietf-pim-ipv6-01.txt#: PIM for IPv6
|
||||
|
||||
** man:pim6dd[8] implements dense mode. man:pim6sd[8] implements sparse mode.
|
||||
|
||||
* [.filename]#draft-itojun-ipv6-tcp-to-anycast-00#: Disconnecting TCP connection toward IPv6 anycast address
|
||||
* [.filename]#draft-yamamoto-wideipv6-comm-model-00#
|
||||
|
||||
** See <<ipv6-sas,23.5.1.6>> in this document for details.
|
||||
|
||||
* [.filename]#draft-ietf-ipngwg-scopedaddr-format-00.txt#: An Extension of Format for IPv6 Scoped Addresses
|
||||
|
||||
[[neighbor-discovery]]
|
||||
==== Neighbor Discovery
|
||||
|
||||
Neighbor Discovery is fairly stable. Currently Address Resolution, Duplicated Address Detection, and Neighbor Unreachability Detection are supported. In the near future we will be adding Proxy Neighbor Advertisement support in the kernel and Unsolicited Neighbor Advertisement transmission command as admin tool.
|
||||
|
||||
If DAD fails, the address will be marked "duplicated" and message will be generated to syslog (and usually to console). The "duplicated" mark can be checked with man:ifconfig[8]. It is administrators' responsibility to check for and recover from DAD failures. The behavior should be improved in the near future.
|
||||
|
||||
Some of the network driver loops multicast packets back to itself, even if instructed not to do so (especially in promiscuous mode). In such cases DAD may fail, because DAD engine sees inbound NS packet (actually from the node itself) and considers it as a sign of duplicate. You may want to look at #if condition marked "heuristics" in sys/netinet6/nd6_nbr.c:nd6_dad_timer() as workaround (note that the code fragment in "heuristics" section is not spec conformant).
|
||||
|
||||
Neighbor Discovery specification (RFC2461) does not talk about neighbor cache handling in the following cases:
|
||||
|
||||
. when there was no neighbor cache entry, node received unsolicited RS/NS/NA/redirect packet without link-layer address
|
||||
. neighbor cache handling on medium without link-layer address (we need a neighbor cache entry for IsRouter bit)
|
||||
|
||||
For first case, we implemented workaround based on discussions on IETF ipngwg mailing list. For more details, see the comments in the source code and email thread started from (IPng 7155), dated Feb 6 1999.
|
||||
|
||||
IPv6 on-link determination rule (RFC2461) is quite different from assumptions in BSD network code. At this moment, no on-link determination rule is supported where default router list is empty (RFC2461, section 5.2, last sentence in 2nd paragraph - note that the spec misuse the word "host" and "node" in several places in the section).
|
||||
|
||||
To avoid possible DoS attacks and infinite loops, only 10 options on ND packet is accepted now. Therefore, if you have 20 prefix options attached to RA, only the first 10 prefixes will be recognized. If this troubles you, please ask it on FREEBSD-CURRENT mailing list and/or modify nd6_maxndopt in [.filename]#sys/netinet6/nd6.c#. If there are high demands we may provide sysctl knob for the variable.
|
||||
|
||||
[[ipv6-scope-index]]
|
||||
==== Scope Index
|
||||
|
||||
IPv6 uses scoped addresses. Therefore, it is very important to specify scope index (interface index for link-local address, or site index for site-local address) with an IPv6 address. Without scope index, scoped IPv6 address is ambiguous to the kernel, and kernel will not be able to determine the outbound interface for a packet.
|
||||
|
||||
Ordinary userland applications should use advanced API (RFC2292) to specify scope index, or interface index. For similar purpose, sin6_scope_id member in sockaddr_in6 structure is defined in RFC2553. However, the semantics for sin6_scope_id is rather vague. If you care about portability of your application, we suggest you to use advanced API rather than sin6_scope_id.
|
||||
|
||||
In the kernel, an interface index for link-local scoped address is embedded into 2nd 16bit-word (3rd and 4th byte) in IPv6 address. For example, you may see something like:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
fe80:1::200:f8ff:fe01:6317
|
||||
....
|
||||
|
||||
in the routing table and interface address structure (struct in6_ifaddr). The address above is a link-local unicast address which belongs to a network interface whose interface identifier is 1. The embedded index enables us to identify IPv6 link local addresses over multiple interfaces effectively and with only a little code change.
|
||||
|
||||
Routing daemons and configuration programs, like man:route6d[8] and man:ifconfig[8], will need to manipulate the "embedded" scope index. These programs use routing sockets and ioctls (like SIOCGIFADDR_IN6) and the kernel API will return IPv6 addresses with 2nd 16bit-word filled in. The APIs are for manipulating kernel internal structure. Programs that use these APIs have to be prepared about differences in kernels anyway.
|
||||
|
||||
When you specify scoped address to the command line, NEVER write the embedded form (such as ff02:1::1 or fe80:2::fedc). This is not supposed to work. Always use standard form, like ff02::1 or fe80::fedc, with command line option for specifying interface (like `ping6 -I ne0 ff02::1`). In general, if a command does not have command line option to specify outgoing interface, that command is not ready to accept scoped address. This may seem to be opposite from IPv6's premise to support "dentist office" situation. We believe that specifications need some improvements for this.
|
||||
|
||||
Some of the userland tools support extended numeric IPv6 syntax, as documented in [.filename]#draft-ietf-ipngwg-scopedaddr-format-00.txt#. You can specify outgoing link, by using name of the outgoing interface like "fe80::1%ne0". This way you will be able to specify link-local scoped address without much trouble.
|
||||
|
||||
To use this extension in your program, you will need to use man:getaddrinfo[3], and man:getnameinfo[3] with NI_WITHSCOPEID. The implementation currently assumes 1-to-1 relationship between a link and an interface, which is stronger than what specs say.
|
||||
|
||||
[[ipv6-pnp]]
|
||||
==== Plug and Play
|
||||
|
||||
Most of the IPv6 stateless address autoconfiguration is implemented in the kernel. Neighbor Discovery functions are implemented in the kernel as a whole. Router Advertisement (RA) input for hosts is implemented in the kernel. Router Solicitation (RS) output for endhosts, RS input for routers, and RA output for routers are implemented in the userland.
|
||||
|
||||
===== Assignment of link-local, and special addresses
|
||||
|
||||
IPv6 link-local address is generated from IEEE802 address (Ethernet MAC address). Each of interface is assigned an IPv6 link-local address automatically, when the interface becomes up (IFF_UP). Also, direct route for the link-local address is added to routing table.
|
||||
|
||||
Here is an output of netstat command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Internet6:
|
||||
Destination Gateway Flags Netif Expire
|
||||
fe80:1::%ed0/64 link#1 UC ed0
|
||||
fe80:2::%ep0/64 link#2 UC ep0
|
||||
....
|
||||
|
||||
Interfaces that has no IEEE802 address (pseudo interfaces like tunnel interfaces, or ppp interfaces) will borrow IEEE802 address from other interfaces, such as Ethernet interfaces, whenever possible. If there is no IEEE802 hardware attached, a last resort pseudo-random value, MD5(hostname), will be used as source of link-local address. If it is not suitable for your usage, you will need to configure the link-local address manually.
|
||||
|
||||
If an interface is not capable of handling IPv6 (such as lack of multicast support), link-local address will not be assigned to that interface. See section 2 for details.
|
||||
|
||||
Each interface joins the solicited multicast address and the link-local all-nodes multicast addresses (e.g., fe80::1:ff01:6317 and ff02::1, respectively, on the link the interface is attached). In addition to a link-local address, the loopback address (::1) will be assigned to the loopback interface. Also, ::1/128 and ff01::/32 are automatically added to routing table, and loopback interface joins node-local multicast group ff01::1.
|
||||
|
||||
===== Stateless address autoconfiguration on Hosts
|
||||
|
||||
In IPv6 specification, nodes are separated into two categories: _routers_ and _hosts_. Routers forward packets addressed to others, hosts does not forward the packets. net.inet6.ip6.forwarding defines whether this node is router or host (router if it is 1, host if it is 0).
|
||||
|
||||
When a host hears Router Advertisement from the router, a host may autoconfigure itself by stateless address autoconfiguration. This behavior can be controlled by net.inet6.ip6.accept_rtadv (host autoconfigures itself if it is set to 1). By autoconfiguration, network address prefix for the receiving interface (usually global address prefix) is added. Default route is also configured. Routers periodically generate Router Advertisement packets. To request an adjacent router to generate RA packet, a host can transmit Router Solicitation. To generate a RS packet at any time, use the _rtsol_ command. man:rtsold[8] daemon is also available. man:rtsold[8] generates Router Solicitation whenever necessary, and it works great for nomadic usage (notebooks/laptops). If one wishes to ignore Router Advertisements, use sysctl to set net.inet6.ip6.accept_rtadv to 0.
|
||||
|
||||
To generate Router Advertisement from a router, use the man:rtadvd[8] daemon.
|
||||
|
||||
Note that, IPv6 specification assumes the following items, and nonconforming cases are left unspecified:
|
||||
|
||||
* Only hosts will listen to router advertisements
|
||||
* Hosts have single network interface (except loopback)
|
||||
|
||||
Therefore, this is unwise to enable net.inet6.ip6.accept_rtadv on routers, or multi-interface host. A misconfigured node can behave strange (nonconforming configuration allowed for those who would like to do some experiments).
|
||||
|
||||
To summarize the sysctl knob:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
accept_rtadv forwarding role of the node
|
||||
--- --- ---
|
||||
0 0 host (to be manually configured)
|
||||
0 1 router
|
||||
1 0 autoconfigured host
|
||||
(spec assumes that host has single
|
||||
interface only, autoconfigured host
|
||||
with multiple interface is
|
||||
out-of-scope)
|
||||
1 1 invalid, or experimental
|
||||
(out-of-scope of spec)
|
||||
....
|
||||
|
||||
RFC2462 has validation rule against incoming RA prefix information option, in 5.5.3 (e). This is to protect hosts from malicious (or misconfigured) routers that advertise very short prefix lifetime. There was an update from Jim Bound to ipngwg mailing list (look for "(ipng 6712)" in the archive) and it is implemented Jim's update.
|
||||
|
||||
See <<neighbor-discovery,23.5.1.2>> in the document for relationship between DAD and autoconfiguration.
|
||||
|
||||
[[gif]]
|
||||
==== Generic Tunnel Interface
|
||||
|
||||
GIF (Generic InterFace) is a pseudo interface for configured tunnel. Details are described in man:gif[4]. Currently
|
||||
|
||||
* v6 in v6
|
||||
* v6 in v4
|
||||
* v4 in v6
|
||||
* v4 in v4
|
||||
|
||||
are available. Use man:gifconfig[8] to assign physical (outer) source and destination address to gif interfaces. Configuration that uses same address family for inner and outer IP header (v4 in v4, or v6 in v6) is dangerous. It is very easy to configure interfaces and routing tables to perform infinite level of tunneling. _Please be warned_.
|
||||
|
||||
gif can be configured to be ECN-friendly. See <<ipsec-ecn,23.5.4.5>> for ECN-friendliness of tunnels, and man:gif[4] for how to configure.
|
||||
|
||||
If you would like to configure an IPv4-in-IPv6 tunnel with gif interface, read man:gif[4] carefully. You will need to remove IPv6 link-local address automatically assigned to the gif interface.
|
||||
|
||||
[[ipv6-sas]]
|
||||
==== Source Address Selection
|
||||
|
||||
Current source selection rule is scope oriented (there are some exceptions - see below). For a given destination, a source IPv6 address is selected by the following rule:
|
||||
|
||||
. If the source address is explicitly specified by the user (e.g., via the advanced API), the specified address is used.
|
||||
. If there is an address assigned to the outgoing interface (which is usually determined by looking up the routing table) that has the same scope as the destination address, the address is used.
|
||||
+
|
||||
This is the most typical case.
|
||||
. If there is no address that satisfies the above condition, choose a global address assigned to one of the interfaces on the sending node.
|
||||
. If there is no address that satisfies the above condition, and destination address is site local scope, choose a site local address assigned to one of the interfaces on the sending node.
|
||||
. If there is no address that satisfies the above condition, choose the address associated with the routing table entry for the destination. This is the last resort, which may cause scope violation.
|
||||
|
||||
For instance, ::1 is selected for ff01::1, fe80:1::200:f8ff:fe01:6317 for fe80:1::2a0:24ff:feab:839b (note that embedded interface index - described in <<ipv6-scope-index,23.5.1.3>> - helps us choose the right source address. Those embedded indices will not be on the wire). If the outgoing interface has multiple address for the scope, a source is selected longest match basis (rule 3). Suppose 2001:0DB8:808:1:200:f8ff:fe01:6317 and 2001:0DB8:9:124:200:f8ff:fe01:6317 are given to the outgoing interface. 2001:0DB8:808:1:200:f8ff:fe01:6317 is chosen as the source for the destination 2001:0DB8:800::1.
|
||||
|
||||
Note that the above rule is not documented in the IPv6 spec. It is considered "up to implementation" item. There are some cases where we do not use the above rule. One example is connected TCP session, and we use the address kept in tcb as the source. Another example is source address for Neighbor Advertisement. Under the spec (RFC2461 7.2.2) NA's source should be the target address of the corresponding NS's target. In this case we follow the spec rather than the above longest-match rule.
|
||||
|
||||
For new connections (when rule 1 does not apply), deprecated addresses (addresses with preferred lifetime = 0) will not be chosen as source address if other choices are available. If no other choices are available, deprecated address will be used as a last resort. If there are multiple choice of deprecated addresses, the above scope rule will be used to choose from those deprecated addresses. If you would like to prohibit the use of deprecated address for some reason, configure net.inet6.ip6.use_deprecated to 0. The issue related to deprecated address is described in RFC2462 5.5.4 (NOTE: there is some debate underway in IETF ipngwg on how to use "deprecated" address).
|
||||
|
||||
[[ipv6-jumbo]]
|
||||
==== Jumbo Payload
|
||||
|
||||
The Jumbo Payload hop-by-hop option is implemented and can be used to send IPv6 packets with payloads longer than 65,535 octets. But currently no physical interface whose MTU is more than 65,535 is supported, so such payloads can be seen only on the loopback interface (i.e., lo0).
|
||||
|
||||
If you want to try jumbo payloads, you first have to reconfigure the kernel so that the MTU of the loopback interface is more than 65,535 bytes; add the following to the kernel configuration file:
|
||||
|
||||
`options "LARGE_LOMTU" #To test jumbo payload`
|
||||
|
||||
and recompile the new kernel.
|
||||
|
||||
Then you can test jumbo payloads by the man:ping6[8] command with -b and -s options. The -b option must be specified to enlarge the size of the socket buffer and the -s option specifies the length of the packet, which should be more than 65,535. For example, type as follows:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% ping6 -b 70000 -s 68000 ::1
|
||||
....
|
||||
|
||||
The IPv6 specification requires that the Jumbo Payload option must not be used in a packet that carries a fragment header. If this condition is broken, an ICMPv6 Parameter Problem message must be sent to the sender. specification is followed, but you cannot usually see an ICMPv6 error caused by this requirement.
|
||||
|
||||
When an IPv6 packet is received, the frame length is checked and compared to the length specified in the payload length field of the IPv6 header or in the value of the Jumbo Payload option, if any. If the former is shorter than the latter, the packet is discarded and statistics are incremented. You can see the statistics as output of man:netstat[8] command with `-s -p ip6' option:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% netstat -s -p ip6
|
||||
ip6:
|
||||
(snip)
|
||||
1 with data size < data length
|
||||
....
|
||||
|
||||
So, kernel does not send an ICMPv6 error unless the erroneous packet is an actual Jumbo Payload, that is, its packet size is more than 65,535 bytes. As described above, currently no physical interface with such a huge MTU is supported, so it rarely returns an ICMPv6 error.
|
||||
|
||||
TCP/UDP over jumbogram is not supported at this moment. This is because we have no medium (other than loopback) to test this. Contact us if you need this.
|
||||
|
||||
IPsec does not work on jumbograms. This is due to some specification twists in supporting AH with jumbograms (AH header size influences payload length, and this makes it real hard to authenticate inbound packet with jumbo payload option as well as AH).
|
||||
|
||||
There are fundamental issues in *BSD support for jumbograms. We would like to address those, but we need more time to finalize these. To name a few:
|
||||
|
||||
* mbuf pkthdr.len field is typed as "int" in 4.4BSD, so it will not hold jumbogram with len > 2G on 32bit architecture CPUs. If we would like to support jumbogram properly, the field must be expanded to hold 4G + IPv6 header + link-layer header. Therefore, it must be expanded to at least int64_t (u_int32_t is NOT enough).
|
||||
* We mistakingly use "int" to hold packet length in many places. We need to convert them into larger integral type. It needs a great care, as we may experience overflow during packet length computation.
|
||||
* We mistakingly check for ip6_plen field of IPv6 header for packet payload length in various places. We should be checking mbuf pkthdr.len instead. ip6_input() will perform sanity check on jumbo payload option on input, and we can safely use mbuf pkthdr.len afterwards.
|
||||
* TCP code needs a careful update in bunch of places, of course.
|
||||
|
||||
==== Loop Prevention in Header Processing
|
||||
|
||||
IPv6 specification allows arbitrary number of extension headers to be placed onto packets. If we implement IPv6 packet processing code in the way BSD IPv4 code is implemented, kernel stack may overflow due to long function call chain. sys/netinet6 code is carefully designed to avoid kernel stack overflow, so sys/netinet6 code defines its own protocol switch structure, as "struct ip6protosw" (see [.filename]#netinet6/ip6protosw.h#). There is no such update to IPv4 part (sys/netinet) for compatibility, but small change is added to its pr_input() prototype. So "struct ipprotosw" is also defined. As a result, if you receive IPsec-over-IPv4 packet with massive number of IPsec headers, kernel stack may blow up. IPsec-over-IPv6 is okay. (Of-course, for those all IPsec headers to be processed, each such IPsec header must pass each IPsec check. So an anonymous attacker will not be able to do such an attack.)
|
||||
|
||||
[[icmpv6]]
|
||||
==== ICMPv6
|
||||
|
||||
After RFC2463 was published, IETF ipngwg has decided to disallow ICMPv6 error packet against ICMPv6 redirect, to prevent ICMPv6 storm on a network medium. This is already implemented into the kernel.
|
||||
|
||||
==== Applications
|
||||
|
||||
For userland programming, we support IPv6 socket API as specified in RFC2553, RFC2292 and upcoming Internet drafts.
|
||||
|
||||
TCP/UDP over IPv6 is available and quite stable. You can enjoy man:telnet[1], man:ftp[1], man:rlogin[1], man:rsh[1], man:ssh[1], etc. These applications are protocol independent. That is, they automatically chooses IPv4 or IPv6 according to DNS.
|
||||
|
||||
==== Kernel Internals
|
||||
|
||||
While ip_forward() calls ip_output(), ip6_forward() directly calls if_output() since routers must not divide IPv6 packets into fragments.
|
||||
|
||||
ICMPv6 should contain the original packet as long as possible up to 1280. UDP6/IP6 port unreach, for instance, should contain all extension headers and the *unchanged* UDP6 and IP6 headers. So, all IP6 functions except TCP never convert network byte order into host byte order, to save the original packet.
|
||||
|
||||
tcp_input(), udp6_input() and icmp6_input() can not assume that IP6 header is preceding the transport headers due to extension headers. So, in6_cksum() was implemented to handle packets whose IP6 header and transport header is not continuous. TCP/IP6 nor UDP6/IP6 header structures do not exist for checksum calculation.
|
||||
|
||||
To process IP6 header, extension headers and transport headers easily, network drivers are now required to store packets in one internal mbuf or one or more external mbufs. A typical old driver prepares two internal mbufs for 96 - 204 bytes data, however, now such packet data is stored in one external mbuf.
|
||||
|
||||
`netstat -s -p ip6` tells you whether or not your driver conforms such requirement. In the following example, "cce0" violates the requirement. (For more information, refer to Section 2.)
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Mbuf statistics:
|
||||
317 one mbuf
|
||||
two or more mbuf::
|
||||
lo0 = 8
|
||||
cce0 = 10
|
||||
3282 one ext mbuf
|
||||
0 two or more ext mbuf
|
||||
....
|
||||
|
||||
Each input function calls IP6_EXTHDR_CHECK in the beginning to check if the region between IP6 and its header is continuous. IP6_EXTHDR_CHECK calls m_pullup() only if the mbuf has M_LOOP flag, that is, the packet comes from the loopback interface. m_pullup() is never called for packets coming from physical network interfaces.
|
||||
|
||||
Both IP and IP6 reassemble functions never call m_pullup().
|
||||
|
||||
[[ipv6-wildcard-socket]]
|
||||
==== IPv4 Mapped Address and IPv6 Wildcard Socket
|
||||
|
||||
RFC2553 describes IPv4 mapped address (3.7) and special behavior of IPv6 wildcard bind socket (3.8). The spec allows you to:
|
||||
|
||||
* Accept IPv4 connections by AF_INET6 wildcard bind socket.
|
||||
* Transmit IPv4 packet over AF_INET6 socket by using special form of the address like ::ffff:10.1.1.1.
|
||||
|
||||
but the spec itself is very complicated and does not specify how the socket layer should behave. Here we call the former one "listening side" and the latter one "initiating side", for reference purposes.
|
||||
|
||||
You can perform wildcard bind on both of the address families, on the same port.
|
||||
|
||||
The following table show the behavior of FreeBSD 4.x.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
listening side initiating side
|
||||
(AF_INET6 wildcard (connection to ::ffff:10.1.1.1)
|
||||
socket gets IPv4 conn.)
|
||||
--- ---
|
||||
FreeBSD 4.x configurable supported
|
||||
default: enabled
|
||||
....
|
||||
|
||||
The following sections will give you more details, and how you can configure the behavior.
|
||||
|
||||
Comments on listening side:
|
||||
|
||||
It looks that RFC2553 talks too little on wildcard bind issue, especially on the port space issue, failure mode and relationship between AF_INET/INET6 wildcard bind. There can be several separate interpretation for this RFC which conform to it but behaves differently. So, to implement portable application you should assume nothing about the behavior in the kernel. Using man:getaddrinfo[3] is the safest way. Port number space and wildcard bind issues were discussed in detail on ipv6imp mailing list, in mid March 1999 and it looks that there is no concrete consensus (means, up to implementers). You may want to check the mailing list archives.
|
||||
|
||||
If a server application would like to accept IPv4 and IPv6 connections, there will be two alternatives.
|
||||
|
||||
One is using AF_INET and AF_INET6 socket (you will need two sockets). Use man:getaddrinfo[3] with AI_PASSIVE into ai_flags, and man:socket[2] and man:bind[2] to all the addresses returned. By opening multiple sockets, you can accept connections onto the socket with proper address family. IPv4 connections will be accepted by AF_INET socket, and IPv6 connections will be accepted by AF_INET6 socket.
|
||||
|
||||
Another way is using one AF_INET6 wildcard bind socket. Use man:getaddrinfo[3] with AI_PASSIVE into ai_flags and with AF_INET6 into ai_family, and set the 1st argument hostname to NULL. And man:socket[2] and man:bind[2] to the address returned. (should be IPv6 unspecified addr). You can accept either of IPv4 and IPv6 packet via this one socket.
|
||||
|
||||
To support only IPv6 traffic on AF_INET6 wildcard binded socket portably, always check the peer address when a connection is made toward AF_INET6 listening socket. If the address is IPv4 mapped address, you may want to reject the connection. You can check the condition by using IN6_IS_ADDR_V4MAPPED() macro.
|
||||
|
||||
To resolve this issue more easily, there is system dependent man:setsockopt[2] option, IPV6_BINDV6ONLY, used like below.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int on;
|
||||
|
||||
setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY,
|
||||
(char *)&on, sizeof (on)) < 0));
|
||||
....
|
||||
|
||||
When this call succeed, then this socket only receive IPv6 packets.
|
||||
|
||||
Comments on initiating side:
|
||||
|
||||
Advise to application implementers: to implement a portable IPv6 application (which works on multiple IPv6 kernels), we believe that the following is the key to the success:
|
||||
|
||||
* NEVER hardcode AF_INET nor AF_INET6.
|
||||
* Use man:getaddrinfo[3] and man:getnameinfo[3] throughout the system. Never use gethostby*(), getaddrby*(), inet_*() or getipnodeby*(). (To update existing applications to be IPv6 aware easily, sometime getipnodeby*() will be useful. But if possible, try to rewrite the code to use man:getaddrinfo[3] and man:getnameinfo[3].)
|
||||
* If you would like to connect to destination, use man:getaddrinfo[3] and try all the destination returned, like man:telnet[1] does.
|
||||
* Some of the IPv6 stack is shipped with buggy man:getaddrinfo[3]. Ship a minimal working version with your application and use that as last resort.
|
||||
|
||||
If you would like to use AF_INET6 socket for both IPv4 and IPv6 outgoing connection, you will need to use man:getipnodebyname[3]. When you would like to update your existing application to be IPv6 aware with minimal effort, this approach might be chosen. But please note that it is a temporal solution, because man:getipnodebyname[3] itself is not recommended as it does not handle scoped IPv6 addresses at all. For IPv6 name resolution, man:getaddrinfo[3] is the preferred API. So you should rewrite your application to use man:getaddrinfo[3], when you get the time to do it.
|
||||
|
||||
When writing applications that make outgoing connections, story goes much simpler if you treat AF_INET and AF_INET6 as totally separate address family. {set,get}sockopt issue goes simpler, DNS issue will be made simpler. We do not recommend you to rely upon IPv4 mapped address.
|
||||
|
||||
===== unified tcp and inpcb code
|
||||
|
||||
FreeBSD 4.x uses shared tcp code between IPv4 and IPv6 (from sys/netinet/tcp*) and separate udp4/6 code. It uses unified inpcb structure.
|
||||
|
||||
The platform can be configured to support IPv4 mapped address. Kernel configuration is summarized as follows:
|
||||
|
||||
* By default, AF_INET6 socket will grab IPv4 connections in certain condition, and can initiate connection to IPv4 destination embedded in IPv4 mapped IPv6 address.
|
||||
* You can disable it on entire system with sysctl like below.
|
||||
+
|
||||
`sysctl net.inet6.ip6.mapped_addr=0`
|
||||
|
||||
====== Listening Side
|
||||
|
||||
Each socket can be configured to support special AF_INET6 wildcard bind (enabled by default). You can disable it on each socket basis with man:setsockopt[2] like below.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int on;
|
||||
|
||||
setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY,
|
||||
(char *)&on, sizeof (on)) < 0));
|
||||
....
|
||||
|
||||
Wildcard AF_INET6 socket grabs IPv4 connection if and only if the following conditions are satisfied:
|
||||
|
||||
* there is no AF_INET socket that matches the IPv4 connection
|
||||
* the AF_INET6 socket is configured to accept IPv4 traffic, i.e., getsockopt(IPV6_BINDV6ONLY) returns 0.
|
||||
|
||||
There is no problem with open/close ordering.
|
||||
|
||||
====== Initiating Side
|
||||
|
||||
FreeBSD 4.x supports outgoing connection to IPv4 mapped address (::ffff:10.1.1.1), if the node is configured to support IPv4 mapped address.
|
||||
|
||||
==== sockaddr_storage
|
||||
|
||||
When RFC2553 was about to be finalized, there was discussion on how struct sockaddr_storage members are named. One proposal is to prepend "__" to the members (like "__ss_len") as they should not be touched. The other proposal was not to prepend it (like "ss_len") as we need to touch those members directly. There was no clear consensus on it.
|
||||
|
||||
As a result, RFC2553 defines struct sockaddr_storage as follows:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct sockaddr_storage {
|
||||
u_char __ss_len; /* address length */
|
||||
u_char __ss_family; /* address family */
|
||||
/* and bunch of padding */
|
||||
};
|
||||
....
|
||||
|
||||
On the contrary, XNET draft defines as follows:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct sockaddr_storage {
|
||||
u_char ss_len; /* address length */
|
||||
u_char ss_family; /* address family */
|
||||
/* and bunch of padding */
|
||||
};
|
||||
....
|
||||
|
||||
In December 1999, it was agreed that RFC2553bis should pick the latter (XNET) definition.
|
||||
|
||||
Current implementation conforms to XNET definition, based on RFC2553bis discussion.
|
||||
|
||||
If you look at multiple IPv6 implementations, you will be able to see both definitions. As an userland programmer, the most portable way of dealing with it is to:
|
||||
|
||||
. ensure ss_family and/or ss_len are available on the platform, by using GNU autoconf,
|
||||
. have -Dss_family=__ss_family to unify all occurrences (including header file) into __ss_family, or
|
||||
. never touch __ss_family. cast to sockaddr * and use sa_family like:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
struct sockaddr_storage ss;
|
||||
family = ((struct sockaddr *)&ss)->sa_family
|
||||
....
|
||||
|
||||
=== Network Drivers
|
||||
|
||||
Now following two items are required to be supported by standard drivers:
|
||||
|
||||
. mbuf clustering requirement. In this stable release, we changed MINCLSIZE into MHLEN+1 for all the operating systems in order to make all the drivers behave as we expect.
|
||||
. multicast. If man:ifmcstat[8] yields no multicast group for a interface, that interface has to be patched.
|
||||
|
||||
If any of the drivers do not support the requirements, then the drivers cannot be used for IPv6 and/or IPsec communication. If you find any problem with your card using IPv6/IPsec, then, please report it to the {freebsd-bugs}.
|
||||
|
||||
(NOTE: In the past we required all PCMCIA drivers to have a call to in6_ifattach(). We have no such requirement any more)
|
||||
|
||||
=== Translator
|
||||
|
||||
We categorize IPv4/IPv6 translator into 4 types:
|
||||
|
||||
* _Translator A_ --- It is used in the early stage of transition to make it possible to establish a connection from an IPv6 host in an IPv6 island to an IPv4 host in the IPv4 ocean.
|
||||
* _Translator B_ --- It is used in the early stage of transition to make it possible to establish a connection from an IPv4 host in the IPv4 ocean to an IPv6 host in an IPv6 island.
|
||||
* _Translator C_ --- It is used in the late stage of transition to make it possible to establish a connection from an IPv4 host in an IPv4 island to an IPv6 host in the IPv6 ocean.
|
||||
* _Translator D_ --- It is used in the late stage of transition to make it possible to establish a connection from an IPv6 host in the IPv6 ocean to an IPv4 host in an IPv4 island.
|
||||
|
||||
[[ipsec-implementation]]
|
||||
=== IPsec
|
||||
|
||||
IPsec is mainly organized by three components.
|
||||
|
||||
. Policy Management
|
||||
. Key Management
|
||||
. AH and ESP handling
|
||||
|
||||
==== Policy Management
|
||||
|
||||
The kernel implements experimental policy management code. There are two way to manage security policy. One is to configure per-socket policy using man:setsockopt[2]. In this cases, policy configuration is described in man:ipsec_set_policy[3]. The other is to configure kernel packet filter-based policy using PF_KEY interface, via man:setkey[8].
|
||||
|
||||
The policy entry is not re-ordered with its indexes, so the order of entry when you add is very significant.
|
||||
|
||||
==== Key Management
|
||||
|
||||
The key management code implemented in this kit (sys/netkey) is a home-brew PFKEY v2 implementation. This conforms to RFC2367.
|
||||
|
||||
The home-brew IKE daemon, "racoon" is included in the kit (kame/kame/racoon). Basically you will need to run racoon as daemon, then set up a policy to require keys (like `ping -P 'out ipsec esp/transport//use'`). The kernel will contact racoon daemon as necessary to exchange keys.
|
||||
|
||||
==== AH and ESP Handling
|
||||
|
||||
IPsec module is implemented as "hooks" to the standard IPv4/IPv6 processing. When sending a packet, ip{,6}_output() checks if ESP/AH processing is required by checking if a matching SPD (Security Policy Database) is found. If ESP/AH is needed, {esp,ah}{4,6}_output() will be called and mbuf will be updated accordingly. When a packet is received, {esp,ah}4_input() will be called based on protocol number, i.e., (*inetsw[proto])(). {esp,ah}4_input() will decrypt/check authenticity of the packet, and strips off daisy-chained header and padding for ESP/AH. It is safe to strip off the ESP/AH header on packet reception, since we will never use the received packet in "as is" form.
|
||||
|
||||
By using ESP/AH, TCP4/6 effective data segment size will be affected by extra daisy-chained headers inserted by ESP/AH. Our code takes care of the case.
|
||||
|
||||
Basic crypto functions can be found in directory "sys/crypto". ESP/AH transform are listed in {esp,ah}_core.c with wrapper functions. If you wish to add some algorithm, add wrapper function in {esp,ah}_core.c, and add your crypto algorithm code into sys/crypto.
|
||||
|
||||
Tunnel mode is partially supported in this release, with the following restrictions:
|
||||
|
||||
* IPsec tunnel is not combined with GIF generic tunneling interface. It needs a great care because we may create an infinite loop between ip_output() and tunnelifp->if_output(). Opinion varies if it is better to unify them, or not.
|
||||
* MTU and Don't Fragment bit (IPv4) considerations need more checking, but basically works fine.
|
||||
* Authentication model for AH tunnel must be revisited. We will need to improve the policy management engine, eventually.
|
||||
|
||||
==== Conformance to RFCs and IDs
|
||||
|
||||
The IPsec code in the kernel conforms (or, tries to conform) to the following standards:
|
||||
|
||||
"old IPsec" specification documented in [.filename]#rfc182[5-9].txt#
|
||||
|
||||
"new IPsec" specification documented in [.filename]#rfc240[1-6].txt#, [.filename]#rfc241[01].txt#, [.filename]#rfc2451.txt# and [.filename]#draft-mcdonald-simple-ipsec-api-01.txt# (draft expired, but you can take from link:ftp://ftp.kame.net/pub/internet-drafts/[ ftp://ftp.kame.net/pub/internet-drafts/]). (NOTE: IKE specifications, [.filename]#rfc241[7-9].txt# are implemented in userland, as "racoon" IKE daemon)
|
||||
|
||||
Currently supported algorithms are:
|
||||
|
||||
* old IPsec AH
|
||||
|
||||
** null crypto checksum (no document, just for debugging)
|
||||
** keyed MD5 with 128bit crypto checksum ([.filename]#rfc1828.txt#)
|
||||
** keyed SHA1 with 128bit crypto checksum (no document)
|
||||
** HMAC MD5 with 128bit crypto checksum ([.filename]#rfc2085.txt#)
|
||||
** HMAC SHA1 with 128bit crypto checksum (no document)
|
||||
|
||||
* old IPsec ESP
|
||||
|
||||
** null encryption (no document, similar to [.filename]#rfc2410.txt#)
|
||||
** DES-CBC mode ([.filename]#rfc1829.txt#)
|
||||
|
||||
* new IPsec AH
|
||||
|
||||
** null crypto checksum (no document, just for debugging)
|
||||
** keyed MD5 with 96bit crypto checksum (no document)
|
||||
** keyed SHA1 with 96bit crypto checksum (no document)
|
||||
** HMAC MD5 with 96bit crypto checksum ([.filename]#rfc2403.txt#)
|
||||
** HMAC SHA1 with 96bit crypto checksum ([.filename]#rfc2404.txt#)
|
||||
|
||||
* new IPsec ESP
|
||||
|
||||
** null encryption ([.filename]#rfc2410.txt#)
|
||||
** DES-CBC with derived IV ([.filename]#draft-ietf-ipsec-ciph-des-derived-01.txt#, draft expired)
|
||||
** DES-CBC with explicit IV ([.filename]#rfc2405.txt#)
|
||||
** 3DES-CBC with explicit IV ([.filename]#rfc2451.txt#)
|
||||
** BLOWFISH CBC ([.filename]#rfc2451.txt#)
|
||||
** CAST128 CBC ([.filename]#rfc2451.txt#)
|
||||
** RC5 CBC ([.filename]#rfc2451.txt#)
|
||||
** each of the above can be combined with:
|
||||
|
||||
*** ESP authentication with HMAC-MD5(96bit)
|
||||
*** ESP authentication with HMAC-SHA1(96bit)
|
||||
|
||||
The following algorithms are NOT supported:
|
||||
|
||||
* old IPsec AH
|
||||
|
||||
** HMAC MD5 with 128bit crypto checksum + 64bit replay prevention ([.filename]#rfc2085.txt#)
|
||||
** keyed SHA1 with 160bit crypto checksum + 32bit padding ([.filename]#rfc1852.txt#)
|
||||
|
||||
IPsec (in kernel) and IKE (in userland as "racoon") has been tested at several interoperability test events, and it is known to interoperate with many other implementations well. Also, current IPsec implementation as quite wide coverage for IPsec crypto algorithms documented in RFC (we cover algorithms without intellectual property issues only).
|
||||
|
||||
[[ipsec-ecn]]
|
||||
==== ECN Consideration on IPsec Tunnels
|
||||
|
||||
ECN-friendly IPsec tunnel is supported as described in [.filename]#draft-ipsec-ecn-00.txt#.
|
||||
|
||||
Normal IPsec tunnel is described in RFC2401. On encapsulation, IPv4 TOS field (or, IPv6 traffic class field) will be copied from inner IP header to outer IP header. On decapsulation outer IP header will be simply dropped. The decapsulation rule is not compatible with ECN, since ECN bit on the outer IP TOS/traffic class field will be lost.
|
||||
|
||||
To make IPsec tunnel ECN-friendly, we should modify encapsulation and decapsulation procedure. This is described in http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt[ http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt], chapter 3.
|
||||
|
||||
IPsec tunnel implementation can give you three behaviors, by setting net.inet.ipsec.ecn (or net.inet6.ipsec6.ecn) to some value:
|
||||
|
||||
* RFC2401: no consideration for ECN (sysctl value -1)
|
||||
* ECN forbidden (sysctl value 0)
|
||||
* ECN allowed (sysctl value 1)
|
||||
|
||||
Note that the behavior is configurable in per-node manner, not per-SA manner (draft-ipsec-ecn-00 wants per-SA configuration, but it looks too much for me).
|
||||
|
||||
The behavior is summarized as follows (see source code for more detail):
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
encapsulate decapsulate
|
||||
--- ---
|
||||
RFC2401 copy all TOS bits drop TOS bits on outer
|
||||
from inner to outer. (use inner TOS bits as is)
|
||||
|
||||
ECN forbidden copy TOS bits except for ECN drop TOS bits on outer
|
||||
(masked with 0xfc) from inner (use inner TOS bits as is)
|
||||
to outer. set ECN bits to 0.
|
||||
|
||||
ECN allowed copy TOS bits except for ECN use inner TOS bits with some
|
||||
CE (masked with 0xfe) from change. if outer ECN CE bit
|
||||
inner to outer. is 1, enable ECN CE bit on
|
||||
set ECN CE bit to 0. the inner.
|
||||
....
|
||||
|
||||
General strategy for configuration is as follows:
|
||||
|
||||
* if both IPsec tunnel endpoint are capable of ECN-friendly behavior, you should better configure both end to "ECN allowed" (sysctl value 1).
|
||||
* if the other end is very strict about TOS bit, use "RFC2401" (sysctl value -1).
|
||||
* in other cases, use "ECN forbidden" (sysctl value 0).
|
||||
|
||||
The default behavior is "ECN forbidden" (sysctl value 0).
|
||||
|
||||
For more information, please refer to:
|
||||
|
||||
http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt[ http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt], RFC2481 (Explicit Congestion Notification), src/sys/netinet6/{ah,esp}_input.c
|
||||
|
||||
(Thanks goes to Kenjiro Cho mailto:kjc@csl.sony.co.jp[kjc@csl.sony.co.jp] for detailed analysis)
|
||||
|
||||
==== Interoperability
|
||||
|
||||
Here are (some of) platforms that KAME code have tested IPsec/IKE interoperability in the past. Note that both ends may have modified their implementation, so use the following list just for reference purposes.
|
||||
|
||||
Altiga, Ashley-laurent (vpcom.com), Data Fellows (F-Secure), Ericsson ACC, FreeS/WAN, HITACHI, IBM AIX(R), IIJ, Intel, Microsoft(R) Windows NT(R), NIST (linux IPsec + plutoplus), Netscreen, OpenBSD, RedCreek, Routerware, SSH, Secure Computing, Soliton, Toshiba, VPNet, Yamaha RT100i
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
---
|
||||
title: Chapter 9. Building and Installing a FreeBSD Kernel
|
||||
authors:
|
||||
prev: books/developers-handbook/partiii
|
||||
next: books/developers-handbook/kerneldebug
|
||||
---
|
||||
|
||||
[[kernelbuild]]
|
||||
= Building and Installing a FreeBSD Kernel
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 9
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
Being a kernel developer requires understanding of the kernel build process. To debug the FreeBSD kernel it is required to be able to build one. There are two known ways to do so:
|
||||
|
||||
The supported procedure to build and install a kernel is documented in the link:{handbook}#kernelconfig-building[Building and Installing a Custom Kernel] chapter of the FreeBSD Handbook.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
It is supposed that the reader of this chapter is familiar with the information described in the link:{handbook}#kernelconfig-building[Building and Installing a Custom Kernel] chapter of the FreeBSD Handbook. If this is not the case, please read through the above mentioned chapter to understand how the build process works.
|
||||
====
|
||||
|
||||
[[kernelbuild-traditional]]
|
||||
== Building the Faster but Brittle Way
|
||||
|
||||
Building the kernel this way may be useful when working on the kernel code and it may actually be faster than the documented procedure when only a single option or two were tweaked in the kernel configuration file. On the other hand, it might lead to unexpected kernel build breakage.
|
||||
|
||||
[.procedure]
|
||||
. Run man:config[8] to generate the kernel source code:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# /usr/sbin/config MYKERNEL
|
||||
....
|
||||
|
||||
. Change into the build directory. man:config[8] will print the name of this directory after being run as above.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# cd ../compile/MYKERNEL
|
||||
....
|
||||
|
||||
. Compile the kernel:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# make depend
|
||||
# make
|
||||
....
|
||||
|
||||
. Install the new kernel:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# make install
|
||||
....
|
||||
|
|
@ -0,0 +1,737 @@
|
|||
---
|
||||
title: Chapter 10. Kernel Debugging
|
||||
authors:
|
||||
- author: Paul Richards
|
||||
- author: Jörg Wunsch
|
||||
- author: Robert Watson
|
||||
prev: books/developers-handbook/kernelbuild
|
||||
next: books/developers-handbook/partiv
|
||||
---
|
||||
|
||||
[[kerneldebug]]
|
||||
= Kernel Debugging
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 10
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[kerneldebug-obtain]]
|
||||
== Obtaining a Kernel Crash Dump
|
||||
|
||||
When running a development kernel (e.g., FreeBSD-CURRENT), such as a kernel under extreme conditions (e.g., very high load averages, tens of thousands of connections, exceedingly high number of concurrent users, hundreds of man:jail[8]s, etc.), or using a new feature or device driver on FreeBSD-STABLE (e.g., PAE), sometimes a kernel will panic. In the event that it does, this chapter will demonstrate how to extract useful information out of a crash.
|
||||
|
||||
A system reboot is inevitable once a kernel panics. Once a system is rebooted, the contents of a system's physical memory (RAM) is lost, as well as any bits that are on the swap device before the panic. To preserve the bits in physical memory, the kernel makes use of the swap device as a temporary place to store the bits that are in RAM across a reboot after a crash. In doing this, when FreeBSD boots after a crash, a kernel image can now be extracted and debugging can take place.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
A swap device that has been configured as a dump device still acts as a swap device. Dumps to non-swap devices (such as tapes or CDRWs, for example) are not supported at this time. A "swap device" is synonymous with a "swap partition."
|
||||
====
|
||||
|
||||
Several types of kernel crash dumps are available:
|
||||
|
||||
Full memory dumps::
|
||||
Hold the complete contents of physical memory.
|
||||
|
||||
Minidumps::
|
||||
Hold only memory pages in use by the kernel (FreeBSD 6.2 and higher).
|
||||
|
||||
Textdumps::
|
||||
Hold captured, scripted, or interactive debugger output (FreeBSD 7.1 and higher).
|
||||
|
||||
Minidumps are the default dump type as of FreeBSD 7.0, and in most cases will capture all necessary information present in a full memory dump, as most problems can be isolated only using kernel state.
|
||||
|
||||
[[config-dumpdev]]
|
||||
=== Configuring the Dump Device
|
||||
|
||||
Before the kernel will dump the contents of its physical memory to a dump device, a dump device must be configured. A dump device is specified by using the man:dumpon[8] command to tell the kernel where to save kernel crash dumps. The man:dumpon[8] program must be called after the swap partition has been configured with man:swapon[8]. This is normally handled by setting the `dumpdev` variable in man:rc.conf[5] to the path of the swap device (the recommended way to extract a kernel dump) or `AUTO` to use the first configured swap device. The default for `dumpdev` is `AUTO` in HEAD, and changed to `NO` on RELENG_* branches (except for RELENG_7, which was left set to `AUTO`). On FreeBSD 9.0-RELEASE and later versions, bsdinstall will ask whether crash dumps should be enabled on the target system during the install process.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
Check [.filename]#/etc/fstab# or man:swapinfo[8] for a list of swap devices.
|
||||
====
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Make sure the `dumpdir` specified in man:rc.conf[5] exists before a kernel crash!
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mkdir /var/crash
|
||||
# chmod 700 /var/crash
|
||||
....
|
||||
|
||||
Also, remember that the contents of [.filename]#/var/crash# is sensitive and very likely contains confidential information such as passwords.
|
||||
====
|
||||
|
||||
[[extract-dump]]
|
||||
=== Extracting a Kernel Dump
|
||||
|
||||
Once a dump has been written to a dump device, the dump must be extracted before the swap device is mounted. To extract a dump from a dump device, use the man:savecore[8] program. If `dumpdev` has been set in man:rc.conf[5], man:savecore[8] will be called automatically on the first multi-user boot after the crash and before the swap device is mounted. The location of the extracted core is placed in the man:rc.conf[5] value `dumpdir`, by default [.filename]#/var/crash# and will be named [.filename]#vmcore.0#.
|
||||
|
||||
In the event that there is already a file called [.filename]#vmcore.0# in [.filename]#/var/crash# (or whatever `dumpdir` is set to), the kernel will increment the trailing number for every crash to avoid overwriting an existing [.filename]#vmcore# (e.g., [.filename]#vmcore.1#). man:savecore[8] will always create a symbolic link to named [.filename]#vmcore.last# in [.filename]#/var/crash# after a dump is saved. This symbolic link can be used to locate the name of the most recent dump.
|
||||
|
||||
The man:crashinfo[8] utility generates a text file containing a summary of information from a full memory dump or minidump. If `dumpdev` has been set in man:rc.conf[5], man:crashinfo[8] will be invoked automatically after man:savecore[8]. The output is saved to a file in `dumpdir` named [.filename]#core.txt.N#.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
If you are testing a new kernel but need to boot a different one in order to get your system up and running again, boot it only into single user mode using the `-s` flag at the boot prompt, and then perform the following steps:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# fsck -p
|
||||
# mount -a -t ufs # make sure /var/crash is writable
|
||||
# savecore /var/crash /dev/ad0s1b
|
||||
# exit # exit to multi-user
|
||||
....
|
||||
|
||||
This instructs man:savecore[8] to extract a kernel dump from [.filename]#/dev/ad0s1b# and place the contents in [.filename]#/var/crash#. Do not forget to make sure the destination directory [.filename]#/var/crash# has enough space for the dump. Also, do not forget to specify the correct path to your swap device as it is likely different than [.filename]#/dev/ad0s1b#!
|
||||
====
|
||||
|
||||
=== Testing Kernel Dump Configuration
|
||||
|
||||
The kernel includes a man:sysctl[8] node that requests a kernel panic. This can be used to verify that your system is properly configured to save kernel crash dumps. You may wish to remount existing file systems as read-only in single user mode before triggering the crash to avoid data loss.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# shutdown now
|
||||
...
|
||||
Enter full pathname of shell or RETURN for /bin/sh:
|
||||
# mount -a -u -r
|
||||
# sysctl debug.kdb.panic=1
|
||||
debug.kdb.panic:panic: kdb_sysctl_panic
|
||||
...
|
||||
....
|
||||
|
||||
After rebooting, your system should save a dump in [.filename]#/var/crash# along with a matching summary from man:crashinfo[8].
|
||||
|
||||
[[kerneldebug-gdb]]
|
||||
== Debugging a Kernel Crash Dump with `kgdb`
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
This section covers man:kgdb[1]. The latest version is included in the package:devel/gdb[]. An older version is also present in FreeBSD 11 and earlier.
|
||||
====
|
||||
|
||||
To enter into the debugger and begin getting information from the dump, start kgdb:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kgdb -n N
|
||||
....
|
||||
|
||||
Where _N_ is the suffix of the [.filename]#vmcore.N# to examine. To open the most recent dump use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kgdb -n last
|
||||
....
|
||||
|
||||
Normally, man:kgdb[1] should be able to locate the kernel running at the time the dump was generated. If it is not able to locate the correct kernel, pass the pathname of the kernel and dump as two arguments to kgdb:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kgdb /boot/kernel/kernel /var/crash/vmcore.0
|
||||
....
|
||||
|
||||
You can debug the crash dump using the kernel sources just like you can for any other program.
|
||||
|
||||
This dump is from a 5.2-BETA kernel and the crash comes from deep within the kernel. The output below has been modified to include line numbers on the left. This first trace inspects the instruction pointer and obtains a back trace. The address that is used on line 41 for the `list` command is the instruction pointer and can be found on line 17. Most developers will request having at least this information sent to them if you are unable to debug the problem yourself. If, however, you do solve the problem, make sure that your patch winds its way into the source tree via a problem report, mailing lists, or by being able to commit it!
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
1:# cd /usr/obj/usr/src/sys/KERNCONF
|
||||
2:# kgdb kernel.debug /var/crash/vmcore.0
|
||||
3:GNU gdb 5.2.1 (FreeBSD)
|
||||
4:Copyright 2002 Free Software Foundation, Inc.
|
||||
5:GDB is free software, covered by the GNU General Public License, and you are
|
||||
6:welcome to change it and/or distribute copies of it under certain conditions.
|
||||
7:Type "show copying" to see the conditions.
|
||||
8:There is absolutely no warranty for GDB. Type "show warranty" for details.
|
||||
9:This GDB was configured as "i386-undermydesk-freebsd"...
|
||||
10:panic: page fault
|
||||
11:panic messages:
|
||||
12:---
|
||||
13:Fatal trap 12: page fault while in kernel mode
|
||||
14:cpuid = 0; apic id = 00
|
||||
15:fault virtual address = 0x300
|
||||
16:fault code: = supervisor read, page not present
|
||||
17:instruction pointer = 0x8:0xc0713860
|
||||
18:stack pointer = 0x10:0xdc1d0b70
|
||||
19:frame pointer = 0x10:0xdc1d0b7c
|
||||
20:code segment = base 0x0, limit 0xfffff, type 0x1b
|
||||
21: = DPL 0, pres 1, def32 1, gran 1
|
||||
22:processor eflags = resume, IOPL = 0
|
||||
23:current process = 14394 (uname)
|
||||
24:trap number = 12
|
||||
25:panic: page fault
|
||||
26 cpuid = 0;
|
||||
27:Stack backtrace:
|
||||
28
|
||||
29:syncing disks, buffers remaining... 2199 2199 panic: mi_switch: switch in a critical section
|
||||
30:cpuid = 0;
|
||||
31:Uptime: 2h43m19s
|
||||
32:Dumping 255 MB
|
||||
33: 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
|
||||
34:---
|
||||
35:Reading symbols from /boot/kernel/snd_maestro3.ko...done.
|
||||
36:Loaded symbols for /boot/kernel/snd_maestro3.ko
|
||||
37:Reading symbols from /boot/kernel/snd_pcm.ko...done.
|
||||
38:Loaded symbols for /boot/kernel/snd_pcm.ko
|
||||
39:#0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240
|
||||
40:240 dumping++;
|
||||
41:(kgdb) list *0xc0713860
|
||||
42:0xc0713860 is in lapic_ipi_wait (/usr/src/sys/i386/i386/local_apic.c:663).
|
||||
43:658 incr = 0;
|
||||
44:659 delay = 1;
|
||||
45:660 } else
|
||||
46:661 incr = 1;
|
||||
47:662 for (x = 0; x < delay; x += incr) {
|
||||
48:663 if ((lapic->icr_lo & APIC_DELSTAT_MASK) == APIC_DELSTAT_IDLE)
|
||||
49:664 return (1);
|
||||
50:665 ia32_pause();
|
||||
51:666 }
|
||||
52:667 return (0);
|
||||
53:(kgdb) backtrace
|
||||
54:#0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240
|
||||
55:#1 0xc055fd9b in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:372
|
||||
56:#2 0xc056019d in panic () at /usr/src/sys/kern/kern_shutdown.c:550
|
||||
57:#3 0xc0567ef5 in mi_switch () at /usr/src/sys/kern/kern_synch.c:470
|
||||
58:#4 0xc055fa87 in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:312
|
||||
59:#5 0xc056019d in panic () at /usr/src/sys/kern/kern_shutdown.c:550
|
||||
60:#6 0xc0720c66 in trap_fatal (frame=0xdc1d0b30, eva=0)
|
||||
61: at /usr/src/sys/i386/i386/trap.c:821
|
||||
62:#7 0xc07202b3 in trap (frame=
|
||||
63: {tf_fs = -1065484264, tf_es = -1065484272, tf_ds = -1065484272, tf_edi = 1, tf_esi = 0, tf_ebp = -602076292, tf_isp = -602076324, tf_ebx = 0, tf_edx = 0, tf_ecx = 1000000, tf_eax = 243, tf_trapno = 12, tf_err = 0, tf_eip = -1066321824, tf_cs = 8, tf_eflags = 65671, tf_esp = 243, tf_ss = 0})
|
||||
64: at /usr/src/sys/i386/i386/trap.c:250
|
||||
65:#8 0xc070c9f8 in calltrap () at {standard input}:94
|
||||
66:#9 0xc07139f3 in lapic_ipi_vectored (vector=0, dest=0)
|
||||
67: at /usr/src/sys/i386/i386/local_apic.c:733
|
||||
68:#10 0xc0718b23 in ipi_selected (cpus=1, ipi=1)
|
||||
69: at /usr/src/sys/i386/i386/mp_machdep.c:1115
|
||||
70:#11 0xc057473e in kseq_notify (ke=0xcc05e360, cpu=0)
|
||||
71: at /usr/src/sys/kern/sched_ule.c:520
|
||||
72:#12 0xc0575cad in sched_add (td=0xcbcf5c80)
|
||||
73: at /usr/src/sys/kern/sched_ule.c:1366
|
||||
74:#13 0xc05666c6 in setrunqueue (td=0xcc05e360)
|
||||
75: at /usr/src/sys/kern/kern_switch.c:422
|
||||
76:#14 0xc05752f4 in sched_wakeup (td=0xcbcf5c80)
|
||||
77: at /usr/src/sys/kern/sched_ule.c:999
|
||||
78:#15 0xc056816c in setrunnable (td=0xcbcf5c80)
|
||||
79: at /usr/src/sys/kern/kern_synch.c:570
|
||||
80:#16 0xc0567d53 in wakeup (ident=0xcbcf5c80)
|
||||
81: at /usr/src/sys/kern/kern_synch.c:411
|
||||
82:#17 0xc05490a8 in exit1 (td=0xcbcf5b40, rv=0)
|
||||
83: at /usr/src/sys/kern/kern_exit.c:509
|
||||
84:#18 0xc0548011 in sys_exit () at /usr/src/sys/kern/kern_exit.c:102
|
||||
85:#19 0xc0720fd0 in syscall (frame=
|
||||
86: {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = -1, tf_ebp = -1077940712, tf_isp = -602075788, tf_ebx = 672411944, tf_edx = 10, tf_ecx = 672411600, tf_eax = 1, tf_trapno = 12, tf_err = 2, tf_eip = 671899563, tf_cs = 31, tf_eflags = 642, tf_esp = -1077940740, tf_ss = 47})
|
||||
87: at /usr/src/sys/i386/i386/trap.c:1010
|
||||
88:#20 0xc070ca4d in Xint0x80_syscall () at {standard input}:136
|
||||
89:---Can't read userspace from dump, or kernel process---
|
||||
90:(kgdb) quit
|
||||
....
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
If your system is crashing regularly and you are running out of disk space, deleting old [.filename]#vmcore# files in [.filename]#/var/crash# could save a considerable amount of disk space!
|
||||
====
|
||||
|
||||
[[kerneldebug-online-ddb]]
|
||||
== On-Line Kernel Debugging Using DDB
|
||||
|
||||
While `kgdb` as an off-line debugger provides a very high level of user interface, there are some things it cannot do. The most important ones being breakpointing and single-stepping kernel code.
|
||||
|
||||
If you need to do low-level debugging on your kernel, there is an on-line debugger available called DDB. It allows setting of breakpoints, single-stepping kernel functions, examining and changing kernel variables, etc. However, it cannot access kernel source files, and only has access to the global and static symbols, not to the full debug information like `kgdb` does.
|
||||
|
||||
To configure your kernel to include DDB, add the options
|
||||
[.programlisting]
|
||||
....
|
||||
options KDB
|
||||
....
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
options DDB
|
||||
....
|
||||
|
||||
to your config file, and rebuild. (See link:{handbook}/[The FreeBSD Handbook] for details on configuring the FreeBSD kernel).
|
||||
|
||||
Once your DDB kernel is running, there are several ways to enter DDB. The first, and earliest way is to use the boot flag `-d`. The kernel will start up in debug mode and enter DDB prior to any device probing. Hence you can even debug the device probe/attach functions. To use this, exit the loader's boot menu and enter `boot -d` at the loader prompt.
|
||||
|
||||
The second scenario is to drop to the debugger once the system has booted. There are two simple ways to accomplish this. If you would like to break to the debugger from the command prompt, simply type the command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# sysctl debug.kdb.enter=1
|
||||
....
|
||||
|
||||
Alternatively, if you are at the system console, you may use a hot-key on the keyboard. The default break-to-debugger sequence is kbd:[Ctrl+Alt+ESC]. For syscons, this sequence can be remapped and some of the distributed maps out there do this, so check to make sure you know the right sequence to use. There is an option available for serial consoles that allows the use of a serial line BREAK on the console line to enter DDB (`options BREAK_TO_DEBUGGER` in the kernel config file). It is not the default since there are a lot of serial adapters around that gratuitously generate a BREAK condition, for example when pulling the cable.
|
||||
|
||||
The third way is that any panic condition will branch to DDB if the kernel is configured to use it. For this reason, it is not wise to configure a kernel with DDB for a machine running unattended.
|
||||
|
||||
To obtain the unattended functionality, add:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
options KDB_UNATTENDED
|
||||
....
|
||||
|
||||
to the kernel configuration file and rebuild/reinstall.
|
||||
|
||||
The DDB commands roughly resemble some `gdb` commands. The first thing you probably need to do is to set a breakpoint:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
break function-name address
|
||||
....
|
||||
|
||||
Numbers are taken hexadecimal by default, but to make them distinct from symbol names; hexadecimal numbers starting with the letters `a-f` need to be preceded with `0x` (this is optional for other numbers). Simple expressions are allowed, for example: `function-name + 0x103`.
|
||||
|
||||
To exit the debugger and continue execution, type:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
continue
|
||||
....
|
||||
|
||||
To get a stack trace of the current thread, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
trace
|
||||
....
|
||||
|
||||
To get a stack trace of an arbitrary thread, specify a process ID or thread ID as a second argument to `trace`.
|
||||
|
||||
If you want to remove a breakpoint, use
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
del
|
||||
del address-expression
|
||||
....
|
||||
|
||||
The first form will be accepted immediately after a breakpoint hit, and deletes the current breakpoint. The second form can remove any breakpoint, but you need to specify the exact address; this can be obtained from:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
show b
|
||||
....
|
||||
|
||||
or:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
show break
|
||||
....
|
||||
|
||||
To single-step the kernel, try:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
s
|
||||
....
|
||||
|
||||
This will step into functions, but you can make DDB trace them until the matching return statement is reached by:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
n
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
This is different from ``gdb``'s `next` statement; it is like ``gdb``'s `finish`. Pressing kbd:[n] more than once will cause a continue.
|
||||
====
|
||||
|
||||
To examine data from memory, use (for example):
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
x/wx 0xf0133fe0,40
|
||||
x/hd db_symtab_space
|
||||
x/bc termbuf,10
|
||||
x/s stringbuf
|
||||
....
|
||||
|
||||
for word/halfword/byte access, and hexadecimal/decimal/character/ string display. The number after the comma is the object count. To display the next 0x10 items, simply use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
x ,10
|
||||
....
|
||||
|
||||
Similarly, use
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
x/ia foofunc,10
|
||||
....
|
||||
|
||||
to disassemble the first 0x10 instructions of `foofunc`, and display them along with their offset from the beginning of `foofunc`.
|
||||
|
||||
To modify memory, use the write command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
w/b termbuf 0xa 0xb 0
|
||||
w/w 0xf0010030 0 0
|
||||
....
|
||||
|
||||
The command modifier (`b`/`h`/`w`) specifies the size of the data to be written, the first following expression is the address to write to and the remainder is interpreted as data to write to successive memory locations.
|
||||
|
||||
If you need to know the current registers, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
show reg
|
||||
....
|
||||
|
||||
Alternatively, you can display a single register value by e.g.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
p $eax
|
||||
....
|
||||
|
||||
and modify it by:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
set $eax new-value
|
||||
....
|
||||
|
||||
Should you need to call some kernel functions from DDB, simply say:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
call func(arg1, arg2, ...)
|
||||
....
|
||||
|
||||
The return value will be printed.
|
||||
|
||||
For a man:ps[1] style summary of all running processes, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ps
|
||||
....
|
||||
|
||||
Now you have examined why your kernel failed, and you wish to reboot. Remember that, depending on the severity of previous malfunctioning, not all parts of the kernel might still be working as expected. Perform one of the following actions to shut down and reboot your system:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
panic
|
||||
....
|
||||
|
||||
This will cause your kernel to dump core and reboot, so you can later analyze the core on a higher level with man:kgdb[1].
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
call boot(0)
|
||||
....
|
||||
|
||||
Might be a good way to cleanly shut down the running system, `sync()` all disks, and finally, in some cases, reboot. As long as the disk and filesystem interfaces of the kernel are not damaged, this could be a good way for an almost clean shutdown.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
reset
|
||||
....
|
||||
|
||||
This is the final way out of disaster and almost the same as hitting the Big Red Button.
|
||||
|
||||
If you need a short command summary, simply type:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
help
|
||||
....
|
||||
|
||||
It is highly recommended to have a printed copy of the man:ddb[4] manual page ready for a debugging session. Remember that it is hard to read the on-line manual while single-stepping the kernel.
|
||||
|
||||
[[kerneldebug-online-gdb]]
|
||||
== On-Line Kernel Debugging Using Remote GDB
|
||||
|
||||
This feature has been supported since FreeBSD 2.2, and it is actually a very neat one.
|
||||
|
||||
GDB has already supported _remote debugging_ for a long time. This is done using a very simple protocol along a serial line. Unlike the other methods described above, you will need two machines for doing this. One is the host providing the debugging environment, including all the sources, and a copy of the kernel binary with all the symbols in it, and the other one is the target machine that simply runs a similar copy of the very same kernel (but stripped of the debugging information).
|
||||
|
||||
You should configure the kernel in question with `config -g` if building the "traditional" way. If building the "new" way, make sure that `makeoptions DEBUG=-g` is in the configuration. In both cases, include `DDB` in the configuration, and compile it as usual. This gives a large binary, due to the debugging information. Copy this kernel to the target machine, strip the debugging symbols off with `strip -x`, and boot it using the `-d` boot option. Connect the serial line of the target machine that has "flags 080" set on its uart device to any serial line of the debugging host. See man:uart[4] for information on how to set the flags on an uart device. Now, on the debugging machine, go to the compile directory of the target kernel, and start `gdb`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% kgdb kernel
|
||||
GDB is free software and you are welcome to distribute copies of it
|
||||
under certain conditions; type "show copying" to see the conditions.
|
||||
There is absolutely no warranty for GDB; type "show warranty" for details.
|
||||
GDB 4.16 (i386-unknown-freebsd),
|
||||
Copyright 1996 Free Software Foundation, Inc...
|
||||
(kgdb)
|
||||
....
|
||||
|
||||
Initialize the remote debugging session (assuming the first serial port is being used) by:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
(kgdb) target remote /dev/cuau0
|
||||
....
|
||||
|
||||
Now, on the target host (the one that entered DDB right before even starting the device probe), type:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Debugger("Boot flags requested debugger")
|
||||
Stopped at Debugger+0x35: movb $0, edata+0x51bc
|
||||
db> gdb
|
||||
....
|
||||
|
||||
DDB will respond with:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Next trap will enter GDB remote protocol mode
|
||||
....
|
||||
|
||||
Every time you type `gdb`, the mode will be toggled between remote GDB and local DDB. In order to force a next trap immediately, simply type `s` (step). Your hosting GDB will now gain control over the target kernel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Remote debugging using /dev/cuau0
|
||||
Debugger (msg=0xf01b0383 "Boot flags requested debugger")
|
||||
at ../../i386/i386/db_interface.c:257
|
||||
(kgdb)
|
||||
....
|
||||
|
||||
You can use this session almost as any other GDB session, including full access to the source, running it in gud-mode inside an Emacs window (which gives you an automatic source code display in another Emacs window), etc.
|
||||
|
||||
[[kerneldebug-console]]
|
||||
== Debugging a Console Driver
|
||||
|
||||
Since you need a console driver to run DDB on, things are more complicated if the console driver itself is failing. You might remember the use of a serial console (either with modified boot blocks, or by specifying `-h` at the `Boot:` prompt), and hook up a standard terminal onto your first serial port. DDB works on any configured console driver, including a serial console.
|
||||
|
||||
[[kerneldebug-deadlocks]]
|
||||
== Debugging Deadlocks
|
||||
|
||||
You may experience so called deadlocks, a situation where a system stops doing useful work. To provide a helpful bug report in this situation, use man:ddb[4] as described in the previous section. Include the output of `ps` and `trace` for suspected processes in the report.
|
||||
|
||||
If possible, consider doing further investigation. The recipe below is especially useful if you suspect that a deadlock occurs in the VFS layer. Add these options to the kernel configuration file.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
makeoptions DEBUG=-g
|
||||
options INVARIANTS
|
||||
options INVARIANT_SUPPORT
|
||||
options WITNESS
|
||||
options WITNESS_SKIPSPIN
|
||||
options DEBUG_LOCKS
|
||||
options DEBUG_VFS_LOCKS
|
||||
options DIAGNOSTIC
|
||||
....
|
||||
|
||||
When a deadlock occurs, in addition to the output of the `ps` command, provide information from the `show pcpu`, `show allpcpu`, `show locks`, `show alllocks`, `show lockedvnods` and `alltrace`.
|
||||
|
||||
To obtain meaningful backtraces for threaded processes, use `thread thread-id` to switch to the thread stack, and do a backtrace with `where`.
|
||||
|
||||
[[kerneldebug-dcons]]
|
||||
== Kernel debugging with Dcons
|
||||
|
||||
man:dcons[4] is a very simple console driver that is not directly connected with any physical devices. It just reads and writes characters from and to a buffer in a kernel or loader. Due to its simple nature, it is very useful for kernel debugging, especially with a FireWire(R) device. Currently, FreeBSD provides two ways to interact with the buffer from outside of the kernel using man:dconschat[8].
|
||||
|
||||
=== Dcons over FireWire(R)
|
||||
|
||||
Most FireWire(R) (IEEE1394) host controllers are based on the OHCI specification that supports physical access to the host memory. This means that once the host controller is initialized, we can access the host memory without the help of software (kernel). We can exploit this facility for interaction with man:dcons[4]. man:dcons[4] provides similar functionality as a serial console. It emulates two serial ports, one for the console and DDB, the other for GDB. Since remote memory access is fully handled by the hardware, the man:dcons[4] buffer is accessible even when the system crashes.
|
||||
|
||||
FireWire(R) devices are not limited to those integrated into motherboards. PCI cards exist for desktops, and a cardbus interface can be purchased for laptops.
|
||||
|
||||
==== Enabling FireWire(R) and Dcons support on the target machine
|
||||
|
||||
To enable FireWire(R) and Dcons support in the kernel of the _target machine_:
|
||||
|
||||
* Make sure your kernel supports `dcons`, `dcons_crom` and `firewire`. `Dcons` should be statically linked with the kernel. For `dcons_crom` and `firewire`, modules should be OK.
|
||||
* Make sure physical DMA is enabled. You may need to add `hw.firewire.phydma_enable=1` to [.filename]#/boot/loader.conf#.
|
||||
* Add options for debugging.
|
||||
* Add `dcons_gdb=1` in [.filename]#/boot/loader.conf# if you use GDB over FireWire(R).
|
||||
* Enable `dcons` in [.filename]#/etc/ttys#.
|
||||
* Optionally, to force `dcons` to be the high-level console, add `hw.firewire.dcons_crom.force_console=1` to [.filename]#loader.conf#.
|
||||
|
||||
To enable FireWire(R) and Dcons support in man:loader[8] on i386 or amd64:
|
||||
|
||||
Add `LOADER_FIREWIRE_SUPPORT=YES` in [.filename]#/etc/make.conf# and rebuild man:loader[8]:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /sys/boot/i386 && make clean && make && make install
|
||||
....
|
||||
|
||||
To enable man:dcons[4] as an active low-level console, add `boot_multicons="YES"` to [.filename]#/boot/loader.conf#.
|
||||
|
||||
Here are a few configuration examples. A sample kernel configuration file would contain:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
device dcons
|
||||
device dcons_crom
|
||||
options KDB
|
||||
options DDB
|
||||
options GDB
|
||||
options ALT_BREAK_TO_DEBUGGER
|
||||
....
|
||||
|
||||
And a sample [.filename]#/boot/loader.conf# would contain:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
dcons_crom_load="YES"
|
||||
dcons_gdb=1
|
||||
boot_multicons="YES"
|
||||
hw.firewire.phydma_enable=1
|
||||
hw.firewire.dcons_crom.force_console=1
|
||||
....
|
||||
|
||||
==== Enabling FireWire(R) and Dcons support on the host machine
|
||||
|
||||
To enable FireWire(R) support in the kernel on the _host machine_:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload firewire
|
||||
....
|
||||
|
||||
Find out the EUI64 (the unique 64 bit identifier) of the FireWire(R) host controller, and use man:fwcontrol[8] or `dmesg` to find the EUI64 of the target machine.
|
||||
|
||||
Run man:dconschat[8], with:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# dconschat -e \# -br -G 12345 -t 00-11-22-33-44-55-66-77
|
||||
....
|
||||
|
||||
The following key combinations can be used once man:dconschat[8] is running:
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1"]
|
||||
|===
|
||||
|
||||
|kbd:[~+.]
|
||||
|Disconnect
|
||||
|
||||
|kbd:[~]
|
||||
|ALT BREAK
|
||||
|
||||
|kbd:[~]
|
||||
|RESET target
|
||||
|
||||
|kbd:[~]
|
||||
|Suspend dconschat
|
||||
|===
|
||||
|
||||
Attach remote GDB by starting man:kgdb[1] with a remote debugging session:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
kgdb -r :12345 kernel
|
||||
....
|
||||
|
||||
==== Some general tips
|
||||
|
||||
Here are some general tips:
|
||||
|
||||
To take full advantage of the speed of FireWire(R), disable other slow console drivers:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# conscontrol delete ttyd0 # serial console
|
||||
# conscontrol delete consolectl # video/keyboard
|
||||
....
|
||||
|
||||
There exists a GDB mode for man:emacs[1]; this is what you will need to add to your [.filename]#.emacs#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
(setq gud-gdba-command-name "kgdb -a -a -a -r :12345")
|
||||
(setq gdb-many-windows t)
|
||||
(xterm-mouse-mode 1)
|
||||
M-x gdba
|
||||
....
|
||||
|
||||
And for DDD ([.filename]#devel/ddd#):
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# remote serial protocol
|
||||
LANG=C ddd --debugger kgdb -r :12345 kernel
|
||||
# live core debug
|
||||
LANG=C ddd --debugger kgdb kernel /dev/fwmem0.2
|
||||
....
|
||||
|
||||
=== Dcons with KVM
|
||||
|
||||
We can directly read the man:dcons[4] buffer via [.filename]#/dev/mem# for live systems, and in the core dump for crashed systems. These give you similar output to `dmesg -a`, but the man:dcons[4] buffer includes more information.
|
||||
|
||||
==== Using Dcons with KVM
|
||||
|
||||
To use man:dcons[4] with KVM:
|
||||
|
||||
Dump a man:dcons[4] buffer of a live system:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# dconschat -1
|
||||
....
|
||||
|
||||
Dump a man:dcons[4] buffer of a crash dump:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# dconschat -1 -M vmcore.XX
|
||||
....
|
||||
|
||||
Live core debugging can be done via:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# fwcontrol -m target_eui64
|
||||
# kgdb kernel /dev/fwmem0.2
|
||||
....
|
||||
|
||||
[[kerneldebug-options]]
|
||||
== Glossary of Kernel Options for Debugging
|
||||
|
||||
This section provides a brief glossary of compile-time kernel options used for debugging:
|
||||
|
||||
* `options KDB`: compiles in the kernel debugger framework. Required for `options DDB` and `options GDB`. Little or no performance overhead. By default, the debugger will be entered on panic instead of an automatic reboot.
|
||||
* `options KDB_UNATTENDED`: change the default value of the `debug.debugger_on_panic` sysctl to 0, which controls whether the debugger is entered on panic. When `options KDB` is not compiled into the kernel, the behavior is to automatically reboot on panic; when it is compiled into the kernel, the default behavior is to drop into the debugger unless `options KDB_UNATTENDED` is compiled in. If you want to leave the kernel debugger compiled into the kernel but want the system to come back up unless you're on-hand to use the debugger for diagnostics, use this option.
|
||||
* `options KDB_TRACE`: change the default value of the `debug.trace_on_panic` sysctl to 1, which controls whether the debugger automatically prints a stack trace on panic. Especially if running with `options KDB_UNATTENDED`, this can be helpful to gather basic debugging information on the serial or firewire console while still rebooting to recover.
|
||||
* `options DDB`: compile in support for the console debugger, DDB. This interactive debugger runs on whatever the active low-level console of the system is, which includes the video console, serial console, or firewire console. It provides basic integrated debugging facilities, such as stack tracing, process and thread listing, dumping of lock state, VM state, file system state, and kernel memory management. DDB does not require software running on a second machine or being able to generate a core dump or full debugging kernel symbols, and provides detailed diagnostics of the kernel at run-time. Many bugs can be fully diagnosed using only DDB output. This option depends on `options KDB`.
|
||||
* `options GDB`: compile in support for the remote debugger, GDB, which can operate over serial cable or firewire. When the debugger is entered, GDB may be attached to inspect structure contents, generate stack traces, etc. Some kernel state is more awkward to access than in DDB, which is able to generate useful summaries of kernel state automatically, such as automatically walking lock debugging or kernel memory management structures, and a second machine running the debugger is required. On the other hand, GDB combines information from the kernel source and full debugging symbols, and is aware of full data structure definitions, local variables, and is scriptable. This option is not required to run GDB on a kernel core dump. This option depends on `options KDB`.
|
||||
* `options BREAK_TO_DEBUGGER`, `options ALT_BREAK_TO_DEBUGGER`: allow a break signal or alternative signal on the console to enter the debugger. If the system hangs without a panic, this is a useful way to reach the debugger. Due to the current kernel locking, a break signal generated on a serial console is significantly more reliable at getting into the debugger, and is generally recommended. This option has little or no performance impact.
|
||||
* `options INVARIANTS`: compile into the kernel a large number of run-time assertion checks and tests, which constantly test the integrity of kernel data structures and the invariants of kernel algorithms. These tests can be expensive, so are not compiled in by default, but help provide useful "fail stop" behavior, in which certain classes of undesired behavior enter the debugger before kernel data corruption occurs, making them easier to debug. Tests include memory scrubbing and use-after-free testing, which is one of the more significant sources of overhead. This option depends on `options INVARIANT_SUPPORT`.
|
||||
* `options INVARIANT_SUPPORT`: many of the tests present in `options INVARIANTS` require modified data structures or additional kernel symbols to be defined.
|
||||
* `options WITNESS`: this option enables run-time lock order tracking and verification, and is an invaluable tool for deadlock diagnosis. WITNESS maintains a graph of acquired lock orders by lock type, and checks the graph at each acquire for cycles (implicit or explicit). If a cycle is detected, a warning and stack trace are generated to the console, indicating that a potential deadlock might have occurred. WITNESS is required in order to use the `show locks`, `show witness` and `show alllocks` DDB commands. This debug option has significant performance overhead, which may be somewhat mitigated through the use of `options WITNESS_SKIPSPIN`. Detailed documentation may be found in man:witness[4].
|
||||
* `options WITNESS_SKIPSPIN`: disable run-time checking of spinlock lock order with WITNESS. As spin locks are acquired most frequently in the scheduler, and scheduler events occur often, this option can significantly speed up systems running with WITNESS. This option depends on `options WITNESS`.
|
||||
* `options WITNESS_KDB`: change the default value of the `debug.witness.kdb` sysctl to 1, which causes WITNESS to enter the debugger when a lock order violation is detected, rather than simply printing a warning. This option depends on `options WITNESS`.
|
||||
* `options SOCKBUF_DEBUG`: perform extensive run-time consistency checking on socket buffers, which can be useful for debugging both socket bugs and race conditions in protocols and device drivers that interact with sockets. This option significantly impacts network performance, and may change the timing in device driver races.
|
||||
* `options DEBUG_VFS_LOCKS`: track lock acquisition points for lockmgr/vnode locks, expanding the amount of information displayed by `show lockedvnods` in DDB. This option has a measurable performance impact.
|
||||
* `options DEBUG_MEMGUARD`: a replacement for the man:malloc[9] kernel memory allocator that uses the VM system to detect reads or writes from allocated memory after free. Details may be found in man:memguard[9]. This option has a significant performance impact, but can be very helpful in debugging kernel memory corruption bugs.
|
||||
* `options DIAGNOSTIC`: enable additional, more expensive diagnostic tests along the lines of `options INVARIANTS`.
|
||||
|
|
@ -0,0 +1,209 @@
|
|||
---
|
||||
title: Chapter 4. Localization and Internationalization - L10N and I18N
|
||||
authors:
|
||||
prev: books/developers-handbook/secure
|
||||
next: books/developers-handbook/policies
|
||||
---
|
||||
|
||||
[[l10n]]
|
||||
= Localization and Internationalization - L10N and I18N
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 4
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[l10n-programming]]
|
||||
== Programming I18N Compliant Applications
|
||||
|
||||
To make your application more useful for speakers of other languages, we hope that you will program I18N compliant. The GNU gcc compiler and GUI libraries like QT and GTK support I18N through special handling of strings. Making a program I18N compliant is very easy. It allows contributors to port your application to other languages quickly. Refer to the library specific I18N documentation for more details.
|
||||
|
||||
In contrast with common perception, I18N compliant code is easy to write. Usually, it only involves wrapping your strings with library specific functions. In addition, please be sure to allow for wide or multibyte character support.
|
||||
|
||||
=== A Call to Unify the I18N Effort
|
||||
|
||||
It has come to our attention that the individual I18N/L10N efforts for each country has been repeating each others' efforts. Many of us have been reinventing the wheel repeatedly and inefficiently. We hope that the various major groups in I18N could congregate into a group effort similar to the Core Team's responsibility.
|
||||
|
||||
Currently, we hope that, when you write or port I18N programs, you would send it out to each country's related FreeBSD mailing list for testing. In the future, we hope to create applications that work in all the languages out-of-the-box without dirty hacks.
|
||||
|
||||
The {freebsd-i18n} has been established. If you are an I18N/L10N developer, please send your comments, ideas, questions, and anything you deem related to it.
|
||||
|
||||
=== Perl and Python
|
||||
|
||||
Perl and Python have I18N and wide character handling libraries. Please use them for I18N compliance.
|
||||
|
||||
[[posix-nls]]
|
||||
== Localized Messages with POSIX.1 Native Language Support (NLS)
|
||||
|
||||
Beyond the basic I18N functions, like supporting various input encodings or supporting national conventions, such as the different decimal separators, at a higher level of I18N, it is possible to localize the messages written to the output by the various programs. A common way of doing this is using the POSIX.1 NLS functions, which are provided as a part of the FreeBSD base system.
|
||||
|
||||
[[nls-catalogs]]
|
||||
=== Organizing Localized Messages into Catalog Files
|
||||
|
||||
POSIX.1 NLS is based on catalog files, which contain the localized messages in the desired encoding. The messages are organized into sets and each message is identified by an integer number in the containing set. The catalog files are conventionally named after the locale they contain localized messages for, followed by the `.msg` extension. For instance, the Hungarian messages for ISO8859-2 encoding should be stored in a file called [.filename]#hu_HU.ISO8859-2#.
|
||||
|
||||
These catalog files are common text files that contain the numbered messages. It is possible to write comments by starting the line with a `$` sign. Set boundaries are also separated by special comments, where the keyword `set` must directly follow the `$` sign. The `set` keyword is then followed by the set number. For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
$set 1
|
||||
....
|
||||
|
||||
The actual message entries start with the message number and followed by the localized message. The well-known modifiers from man:printf[3] are accepted:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
15 "File not found: %s\n"
|
||||
....
|
||||
|
||||
The language catalog files have to be compiled into a binary form before they can be opened from the program. This conversion is done with the man:gencat[1] utility. Its first argument is the filename of the compiled catalog and its further arguments are the input catalogs. The localized messages can also be organized into more catalog files and then all of them can be processed with man:gencat[1].
|
||||
|
||||
[[nls-using]]
|
||||
=== Using the Catalog Files from the Source Code
|
||||
|
||||
Using the catalog files is simple. To use the related functions, [.filename]#nl_types.h# must be included. Before using a catalog, it has to be opened with man:catopen[3]. The function takes two arguments. The first parameter is the name of the installed and compiled catalog. Usually, the name of the program is used, such as grep. This name will be used when looking for the compiled catalog file. The man:catopen[3] call looks for this file in [.filename]#/usr/shared/nls/locale/catname# and in [.filename]#/usr/local/shared/nls/locale/catname#, where `locale` is the locale set and `catname` is the catalog name being discussed. The second parameter is a constant, which can have two values:
|
||||
|
||||
* `NL_CAT_LOCALE`, which means that the used catalog file will be based on `LC_MESSAGES`.
|
||||
* `0`, which means that `LANG` has to be used to open the proper catalog.
|
||||
|
||||
The man:catopen[3] call returns a catalog identifier of type `nl_catd`. Please refer to the manual page for a list of possible returned error codes.
|
||||
|
||||
After opening a catalog man:catgets[3] can be used to retrieve a message. The first parameter is the catalog identifier returned by man:catopen[3], the second one is the number of the set, the third one is the number of the messages, and the fourth one is a fallback message, which will be returned if the requested message cannot be retrieved from the catalog file.
|
||||
|
||||
After using the catalog file, it must be closed by calling man:catclose[3], which has one argument, the catalog id.
|
||||
|
||||
[[nls-example]]
|
||||
=== A Practical Example
|
||||
|
||||
The following example will demonstrate an easy solution on how to use NLS catalogs in a flexible way.
|
||||
|
||||
The below lines need to be put into a common header file of the program, which is included into all source files where localized messages are necessary:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#ifdef WITHOUT_NLS
|
||||
#define getstr(n) nlsstr[n]
|
||||
#else
|
||||
#include nl_types.h
|
||||
|
||||
extern nl_catd catalog;
|
||||
#define getstr(n) catgets(catalog, 1, n, nlsstr[n])
|
||||
#endif
|
||||
|
||||
extern char *nlsstr[];
|
||||
....
|
||||
|
||||
Next, put these lines into the global declaration part of the main source file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#ifndef WITHOUT_NLS
|
||||
#include nl_types.h
|
||||
nl_catd catalog;
|
||||
#endif
|
||||
|
||||
/*
|
||||
* Default messages to use when NLS is disabled or no catalog
|
||||
* is found.
|
||||
*/
|
||||
char *nlsstr[] = {
|
||||
"",
|
||||
/* 1*/ "some random message",
|
||||
/* 2*/ "some other message"
|
||||
};
|
||||
....
|
||||
|
||||
Next come the real code snippets, which open, read, and close the catalog:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#ifndef WITHOUT_NLS
|
||||
catalog = catopen("myapp", NL_CAT_LOCALE);
|
||||
#endif
|
||||
|
||||
...
|
||||
|
||||
printf(getstr(1));
|
||||
|
||||
...
|
||||
|
||||
#ifndef WITHOUT_NLS
|
||||
catclose(catalog);
|
||||
#endif
|
||||
....
|
||||
|
||||
==== Reducing Strings to Localize
|
||||
|
||||
There is a good way of reducing the strings that need to be localized by using libc error messages. This is also useful to just avoid duplication and provide consistent error messages for the common errors that can be encountered by a great many of programs.
|
||||
|
||||
First, here is an example that does not use libc error messages:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#include err.h
|
||||
...
|
||||
if (!S_ISDIR(st.st_mode))
|
||||
errx(1, "argument is not a directory");
|
||||
....
|
||||
|
||||
This can be transformed to print an error message by reading `errno` and printing an error message accordingly:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#include err.h
|
||||
#include errno.h
|
||||
...
|
||||
if (!S_ISDIR(st.st_mode)) {
|
||||
errno = ENOTDIR;
|
||||
err(1, NULL);
|
||||
}
|
||||
....
|
||||
|
||||
In this example, the custom string is eliminated, thus translators will have less work when localizing the program and users will see the usual "Not a directory" error message when they encounter this error. This message will probably seem more familiar to them. Please note that it was necessary to include [.filename]#errno.h# in order to directly access `errno`.
|
||||
|
||||
It is worth to note that there are cases when `errno` is set automatically by a preceding call, so it is not necessary to set it explicitly:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#include err.h
|
||||
...
|
||||
if ((p = malloc(size)) == NULL)
|
||||
err(1, NULL);
|
||||
....
|
||||
|
||||
[[nls-mk]]
|
||||
=== Making use of [.filename]#bsd.nls.mk#
|
||||
|
||||
Using the catalog files requires few repeatable steps, such as compiling the catalogs and installing them to the proper location. In order to simplify this process even more, [.filename]#bsd.nls.mk# introduces some macros. It is not necessary to include [.filename]#bsd.nls.mk# explicitly, it is pulled in from the common Makefiles, such as [.filename]#bsd.prog.mk# or [.filename]#bsd.lib.mk#.
|
||||
|
||||
Usually it is enough to define `NLSNAME`, which should have the catalog name mentioned as the first argument of man:catopen[3] and list the catalog files in `NLS` without their `.msg` extension. Here is an example, which makes it possible to to disable NLS when used with the code examples before. The `WITHOUT_NLS` man:make[1] variable has to be defined in order to build the program without NLS support.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
.if !defined(WITHOUT_NLS)
|
||||
NLS= es_ES.ISO8859-1
|
||||
NLS+= hu_HU.ISO8859-2
|
||||
NLS+= pt_BR.ISO8859-1
|
||||
.else
|
||||
CFLAGS+= -DWITHOUT_NLS
|
||||
.endif
|
||||
....
|
||||
|
||||
Conventionally, the catalog files are placed under the [.filename]#nls# subdirectory and this is the default behavior of [.filename]#bsd.nls.mk#. It is possible, though to override the location of the catalogs with the `NLSSRCDIR` man:make[1] variable. The default name of the precompiled catalog files also follow the naming convention mentioned before. It can be overridden by setting the `NLSNAME` variable. There are other options to fine tune the processing of the catalog files but usually it is not needed, thus they are not described here. For further information on [.filename]#bsd.nls.mk#, please refer to the file itself, it is short and easy to understand.
|
||||
|
|
@ -0,0 +1,271 @@
|
|||
---
|
||||
title: Chapter 5. Source Tree Guidelines and Policies
|
||||
authors:
|
||||
- author: Poul-Henning Kamp
|
||||
- author: Giorgos Keramidas
|
||||
prev: books/developers-handbook/l10n
|
||||
next: books/developers-handbook/testing
|
||||
---
|
||||
|
||||
[[policies]]
|
||||
= Source Tree Guidelines and Policies
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 5
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
This chapter documents various guidelines and policies in force for the FreeBSD source tree.
|
||||
|
||||
[[policies-style]]
|
||||
== Style Guidelines
|
||||
|
||||
Consistent coding style is extremely important, particularly with large projects like FreeBSD. Code should follow the FreeBSD coding styles described in man:style[9] and man:style.Makefile[5].
|
||||
|
||||
[[policies-maintainer]]
|
||||
== `MAINTAINER` on Makefiles
|
||||
|
||||
If a particular portion of the FreeBSD [.filename]#src/# distribution is being maintained by a person or group of persons, this is communicated through an entry in [.filename]#src/MAINTAINERS#. Maintainers of ports within the Ports Collection express their maintainership to the world by adding a `MAINTAINER` line to the [.filename]#Makefile# of the port in question:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
MAINTAINER= email-addresses
|
||||
....
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
For other parts of the repository, or for sections not listed as having a maintainer, or when you are unsure who the active maintainer is, try looking at the recent commit history of the relevant parts of the source tree. It is quite often the case that a maintainer is not explicitly named, but the people who are actively working in a part of the source tree for, say, the last couple of years are interested in reviewing changes. Even if this is not specifically mentioned in the documentation or the source itself, asking for a review as a form of courtesy is a very reasonable thing to do.
|
||||
====
|
||||
|
||||
The role of the maintainer is as follows:
|
||||
|
||||
* The maintainer owns and is responsible for that code. This means that he or she is responsible for fixing bugs and answering problem reports pertaining to that piece of the code, and in the case of contributed software, for tracking new versions, as appropriate.
|
||||
* Changes to directories which have a maintainer defined shall be sent to the maintainer for review before being committed. Only if the maintainer does not respond for an unacceptable period of time, to several emails, will it be acceptable to commit changes without review by the maintainer. However, it is suggested that you try to have the changes reviewed by someone else if at all possible.
|
||||
* It is of course not acceptable to add a person or group as maintainer unless they agree to assume this duty. On the other hand it does not have to be a committer and it can easily be a group of people.
|
||||
|
||||
[[policies-contributed]]
|
||||
== Contributed Software
|
||||
|
||||
Some parts of the FreeBSD distribution consist of software that is actively being maintained outside the FreeBSD project. For historical reasons, we call this _contributed_ software. Some examples are sendmail, gcc and patch.
|
||||
|
||||
Over the last couple of years, various methods have been used in dealing with this type of software and all have some number of advantages and drawbacks. No clear winner has emerged.
|
||||
|
||||
Since this is the case, after some debate one of these methods has been selected as the "official" method and will be required for future imports of software of this kind. Furthermore, it is strongly suggested that existing contributed software converge on this model over time, as it has significant advantages over the old method, including the ability to easily obtain diffs relative to the "official" versions of the source by everyone (even without direct repository access). This will make it significantly easier to return changes to the primary developers of the contributed software.
|
||||
|
||||
Ultimately, however, it comes down to the people actually doing the work. If using this model is particularly unsuited to the package being dealt with, exceptions to these rules may be granted only with the approval of the core team and with the general consensus of the other developers. The ability to maintain the package in the future will be a key issue in the decisions.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Because it makes it harder to import future versions minor, trivial and/or cosmetic changes are _strongly discouraged_ on files that are still tracking the vendor branch.
|
||||
====
|
||||
|
||||
[[vendor-import-svn]]
|
||||
=== Vendor Imports with SVN
|
||||
|
||||
This section describes the vendor import procedure with Subversion in details.
|
||||
|
||||
[.procedure]
|
||||
. *Preparing the Tree*
|
||||
+
|
||||
If this is your first import after the switch to SVN, you will have to flatten and clean up the vendor tree, and bootstrap merge history in the main tree. If not, you can safely omit this step.
|
||||
+
|
||||
During the conversion from CVS to SVN, vendor branches were imported with the same layout as the main tree. For example, the foo vendor sources ended up in [.filename]#vendor/foo/dist/contrib/foo#, but it is pointless and rather inconvenient. What we really want is to have the vendor source directly in [.filename]#vendor/foo/dist#, like this:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd vendor/foo/dist/contrib/foo
|
||||
% svn move $(svn list) ../..
|
||||
% cd ../..
|
||||
% svn remove contrib
|
||||
% svn propdel -R svn:mergeinfo
|
||||
% svn commit
|
||||
....
|
||||
+
|
||||
Note that, the `propdel` bit is necessary because starting with 1.5, Subversion will automatically add `svn:mergeinfo` to any directory you copy or move. In this case, you will not need this information, since you are not going to merge anything from the tree you deleted.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
You may want to flatten the tags as well. The procedure is exactly the same. If you do this, put off the commit until the end.
|
||||
====
|
||||
+
|
||||
Check the [.filename]#dist# tree and perform any cleanup that is deemed to be necessary. You may want to disable keyword expansion, as it makes no sense on unmodified vendor code. In some cases, it can be even be harmful.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% svn propdel svn:keywords -R .
|
||||
% svn commit
|
||||
....
|
||||
+
|
||||
Bootstrapping of `svn:mergeinfo` on the target directory (in the main tree) to the revision that corresponds to the last change was made to the vendor tree prior to importing new sources is also needed:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd head/contrib/foo
|
||||
% svn merge --record-only ^/vendor/foo/dist@12345678 .
|
||||
% svn commit
|
||||
....
|
||||
+
|
||||
With some shells, the `^` in the above command may need to be escaped with a backslash.
|
||||
. *Importing New Sources*
|
||||
+
|
||||
Prepare a full, clean tree of the vendor sources. With SVN, we can keep a full distribution in the vendor tree without bloating the main tree. Import everything but merge only what is needed.
|
||||
+
|
||||
Note that you will need to add any files that were added since the last vendor import, and remove any that were removed. To facilitate this, you should prepare sorted lists of the contents of the vendor tree and of the sources you are about to import:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd vendor/foo/dist
|
||||
% svn list -R | grep -v '/$' | sort > ../old
|
||||
% cd ../foo-9.9
|
||||
% find . -type f | cut -c 3- | sort > ../new
|
||||
....
|
||||
+
|
||||
With these two files, the following command will list removed files (files only in [.filename]#old#):
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% comm -23 ../old ../new
|
||||
....
|
||||
+
|
||||
While the command below will list added files (files only in [.filename]#new#):
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% comm -13 ../old ../new
|
||||
....
|
||||
+
|
||||
Let us put this together:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd vendor/foo/foo-9.9
|
||||
% tar cf - . | tar xf - -C ../dist
|
||||
% cd ../dist
|
||||
% comm -23 ../old ../new | xargs svn remove
|
||||
% comm -13 ../old ../new | xargs svn add
|
||||
....
|
||||
+
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
If there are new directories in the new distribution, the last command will fail. You will have to add the directories, and run it again. Conversely, if any directories were removed, you will have to remove them manually.
|
||||
====
|
||||
+
|
||||
Check properties on any new files:
|
||||
|
||||
** All text files should have `svn:eol-style` set to `native`.
|
||||
** All binary files should have `svn:mime-type` set to `application/octet-stream`, unless there is a more appropriate media type.
|
||||
** Executable files should have `svn:executable` set to `*`.
|
||||
** There should be no other properties on any file in the tree.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
You are ready to commit, but you should first check the output of `svn stat` and `svn diff` to make sure everything is in order.
|
||||
====
|
||||
+
|
||||
Once you have committed the new vendor release, you should tag it for future reference. The best and quickest way is to do it directly in the repository:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% svn copy ^/vendor/foo/dist svn_base/vendor/foo/9.9
|
||||
....
|
||||
+
|
||||
To get the new tag, you can update your working copy of [.filename]#vendor/foo#.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
If you choose to do the copy in the checkout instead, do not forget to remove the generated `svn:mergeinfo` as described above.
|
||||
====
|
||||
|
||||
. *Merging to __-HEAD__*
|
||||
+
|
||||
After you have prepared your import, it is time to merge. Option `--accept=postpone` tells SVN not to handle merge conflicts yet, because they will be taken care of manually:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd head/contrib/foo
|
||||
% svn update
|
||||
% svn merge --accept=postpone ^/vendor/foo/dist
|
||||
....
|
||||
+
|
||||
Resolve any conflicts, and make sure that any files that were added or removed in the vendor tree have been properly added or removed in the main tree. It is always a good idea to check differences against the vendor branch:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% svn diff --no-diff-deleted --old=^/vendor/foo/dist --new=.
|
||||
....
|
||||
+
|
||||
`--no-diff-deleted` tells SVN not to check files that are in the vendor tree but not in the main tree.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
With SVN, there is no concept of on or off the vendor branch. If a file that previously had local modifications no longer does, just remove any left-over cruft, such as FreeBSD version tags, so it no longer shows up in diffs against the vendor tree.
|
||||
====
|
||||
+
|
||||
If any changes are required for the world to build with the new sources, make them now - and test until you are satisfied that everything build and runs correctly.
|
||||
. *Commit*
|
||||
+
|
||||
Now, you are ready to commit. Make sure you get everything in one go. Ideally, you would have done all steps in a clean tree, in which case you can just commit from the top of that tree. That is the best way to avoid surprises. If you do it properly, the tree will move atomically from a consistent state with the old code to a consistent state with the new code.
|
||||
|
||||
[[policies-encumbered]]
|
||||
== Encumbered Files
|
||||
|
||||
It might occasionally be necessary to include an encumbered file in the FreeBSD source tree. For example, if a device requires a small piece of binary code to be loaded to it before the device will operate, and we do not have the source to that code, then the binary file is said to be encumbered. The following policies apply to including encumbered files in the FreeBSD source tree.
|
||||
|
||||
. Any file which is interpreted or executed by the system CPU(s) and not in source format is encumbered.
|
||||
. Any file with a license more restrictive than BSD or GNU is encumbered.
|
||||
. A file which contains downloadable binary data for use by the hardware is not encumbered, unless (1) or (2) apply to it. It must be stored in an architecture neutral ASCII format (file2c or uuencoding is recommended).
|
||||
. Any encumbered file requires specific approval from the link:https://www.FreeBSD.org/administration/#t-core[Core Team] before it is added to the repository.
|
||||
. Encumbered files go in [.filename]#src/contrib# or [.filename]#src/sys/contrib#.
|
||||
. The entire module should be kept together. There is no point in splitting it, unless there is code-sharing with non-encumbered code.
|
||||
. Object files are named [.filename]#arch/filename.o.uu>#.
|
||||
. Kernel files:
|
||||
.. Should always be referenced in [.filename]#conf/files.*# (for build simplicity).
|
||||
.. Should always be in [.filename]#LINT#, but the link:https://www.FreeBSD.org/administration/#t-core[Core Team] decides per case if it should be commented out or not. The link:https://www.FreeBSD.org/administration/#t-core[Core Team] can, of course, change their minds later on.
|
||||
.. The _Release Engineer_ decides whether or not it goes into the release.
|
||||
|
||||
. User-land files:
|
||||
.. The link:https://www.FreeBSD.org/administration/#t-core[Core team] decides if the code should be part of `make world`.
|
||||
.. The link:https://www.FreeBSD.org/administration/#t-re[Release Engineering] decides if it goes into the release.
|
||||
|
||||
[[policies-shlib]]
|
||||
== Shared Libraries
|
||||
|
||||
If you are adding shared library support to a port or other piece of software that does not have one, the version numbers should follow these rules. Generally, the resulting numbers will have nothing to do with the release version of the software.
|
||||
|
||||
The three principles of shared library building are:
|
||||
|
||||
* Start from `1.0`
|
||||
* If there is a change that is backwards compatible, bump minor number (note that ELF systems ignore the minor number)
|
||||
* If there is an incompatible change, bump major number
|
||||
|
||||
For instance, added functions and bugfixes result in the minor version number being bumped, while deleted functions, changed function call syntax, etc. will force the major version number to change.
|
||||
|
||||
Stick to version numbers of the form major.minor (`_x_._y_`). Our a.out dynamic linker does not handle version numbers of the form `_x_._y_._z_` well. Any version number after the `_y_` (i.e., the third digit) is totally ignored when comparing shared lib version numbers to decide which library to link with. Given two shared libraries that differ only in the "micro" revision, `ld.so` will link with the higher one. That is, if you link with [.filename]#libfoo.so.3.3.3#, the linker only records `3.3` in the headers, and will link with anything starting with `_libfoo.so.3_._(anything >= 3)_._(highest available)_`.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
`ld.so` will always use the highest "minor" revision. For instance, it will use [.filename]#libc.so.2.2# in preference to [.filename]#libc.so.2.0#, even if the program was initially linked with [.filename]#libc.so.2.0#.
|
||||
====
|
||||
|
||||
In addition, our ELF dynamic linker does not handle minor version numbers at all. However, one should still specify a major and minor version number as our [.filename]#Makefile#'s "do the right thing" based on the type of system.
|
||||
|
||||
For non-port libraries, it is also our policy to change the shared library version number only once between releases. In addition, it is our policy to change the major shared library version number only once between major OS releases (i.e., from 6.0 to 7.0). When you make a change to a system library that requires the version number to be bumped, check the [.filename]#Makefile#'s commit logs. It is the responsibility of the committer to ensure that the first such change since the release will result in the shared library version number in the [.filename]#Makefile# to be updated, and any subsequent changes will not.
|
||||
|
|
@ -0,0 +1,210 @@
|
|||
---
|
||||
title: Chapter 3. Secure Programming
|
||||
authors:
|
||||
- author: Murray Stokely
|
||||
prev: books/developers-handbook/tools
|
||||
next: books/developers-handbook/l10n
|
||||
---
|
||||
|
||||
[[secure]]
|
||||
= Secure Programming
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 3
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[secure-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
This chapter describes some of the security issues that have plagued UNIX(R) programmers for decades and some of the new tools available to help programmers avoid writing exploitable code.
|
||||
|
||||
[[secure-philosophy]]
|
||||
== Secure Design Methodology
|
||||
|
||||
Writing secure applications takes a very scrutinous and pessimistic outlook on life. Applications should be run with the principle of "least privilege" so that no process is ever running with more than the bare minimum access that it needs to accomplish its function. Previously tested code should be reused whenever possible to avoid common mistakes that others may have already fixed.
|
||||
|
||||
One of the pitfalls of the UNIX(R) environment is how easy it is to make assumptions about the sanity of the environment. Applications should never trust user input (in all its forms), system resources, inter-process communication, or the timing of events. UNIX(R) processes do not execute synchronously so logical operations are rarely atomic.
|
||||
|
||||
[[secure-bufferov]]
|
||||
== Buffer Overflows
|
||||
|
||||
Buffer Overflows have been around since the very beginnings of the von Neumann <<cod,1>> architecture. They first gained widespread notoriety in 1988 with the Morris Internet worm. Unfortunately, the same basic attack remains effective today. By far the most common type of buffer overflow attack is based on corrupting the stack.
|
||||
|
||||
Most modern computer systems use a stack to pass arguments to procedures and to store local variables. A stack is a last in first out (LIFO) buffer in the high memory area of a process image. When a program invokes a function a new "stack frame" is created. This stack frame consists of the arguments passed to the function as well as a dynamic amount of local variable space. The "stack pointer" is a register that holds the current location of the top of the stack. Since this value is constantly changing as new values are pushed onto the top of the stack, many implementations also provide a "frame pointer" that is located near the beginning of a stack frame so that local variables can more easily be addressed relative to this value. <<cod,1>> The return address for function calls is also stored on the stack, and this is the cause of stack-overflow exploits since overflowing a local variable in a function can overwrite the return address of that function, potentially allowing a malicious user to execute any code he or she wants.
|
||||
|
||||
Although stack-based attacks are by far the most common, it would also be possible to overrun the stack with a heap-based (malloc/free) attack.
|
||||
|
||||
The C programming language does not perform automatic bounds checking on arrays or pointers as many other languages do. In addition, the standard C library is filled with a handful of very dangerous functions.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", frame="none"]
|
||||
|===
|
||||
|
||||
|`strcpy`(char *dest, const char *src)
|
||||
|
|
||||
|
||||
May overflow the dest buffer
|
||||
|
||||
|`strcat`(char *dest, const char *src)
|
||||
|
|
||||
|
||||
May overflow the dest buffer
|
||||
|
||||
|`getwd`(char *buf)
|
||||
|
|
||||
|
||||
May overflow the buf buffer
|
||||
|
||||
|`gets`(char *s)
|
||||
|
|
||||
|
||||
May overflow the s buffer
|
||||
|
||||
|`[vf]scanf`(const char *format, ...)
|
||||
|
|
||||
|
||||
May overflow its arguments.
|
||||
|
||||
|`realpath`(char *path, char resolved_path[])
|
||||
|
|
||||
|
||||
May overflow the path buffer
|
||||
|
||||
|`[v]sprintf`(char *str, const char *format, ...)
|
||||
|
|
||||
|
||||
May overflow the str buffer.
|
||||
|===
|
||||
|
||||
=== Example Buffer Overflow
|
||||
|
||||
The following example code contains a buffer overflow designed to overwrite the return address and skip the instruction immediately following the function call. (Inspired by <<Phrack,4>>)
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#include <stdio.h>
|
||||
|
||||
void manipulate(char *buffer) {
|
||||
char newbuffer[80];
|
||||
strcpy(newbuffer,buffer);
|
||||
}
|
||||
|
||||
int main() {
|
||||
char ch,buffer[4096];
|
||||
int i=0;
|
||||
|
||||
while ((buffer[i++] = getchar()) != '\n') {};
|
||||
|
||||
i=1;
|
||||
manipulate(buffer);
|
||||
i=2;
|
||||
printf("The value of i is : %d\n",i);
|
||||
return 0;
|
||||
}
|
||||
....
|
||||
|
||||
Let us examine what the memory image of this process would look like if we were to input 160 spaces into our little program before hitting return.
|
||||
|
||||
[XXX figure here!]
|
||||
|
||||
Obviously more malicious input can be devised to execute actual compiled instructions (such as exec(/bin/sh)).
|
||||
|
||||
=== Avoiding Buffer Overflows
|
||||
|
||||
The most straightforward solution to the problem of stack-overflows is to always use length restricted memory and string copy functions. `strncpy` and `strncat` are part of the standard C library. These functions accept a length value as a parameter which should be no larger than the size of the destination buffer. These functions will then copy up to 'length' bytes from the source to the destination. However there are a number of problems with these functions. Neither function guarantees NUL termination if the size of the input buffer is as large as the destination. The length parameter is also used inconsistently between strncpy and strncat so it is easy for programmers to get confused as to their proper usage. There is also a significant performance loss compared to `strcpy` when copying a short string into a large buffer since `strncpy` NUL fills up the size specified.
|
||||
|
||||
Another memory copy implementation exists to get around these problems. The `strlcpy` and `strlcat` functions guarantee that they will always null terminate the destination string when given a non-zero length argument.
|
||||
|
||||
==== Compiler based run-time bounds checking
|
||||
|
||||
Unfortunately there is still a very large assortment of code in public use which blindly copies memory around without using any of the bounded copy routines we just discussed. Fortunately, there is a way to help prevent such attacks - run-time bounds checking, which is implemented by several C/C++ compilers.
|
||||
|
||||
ProPolice is one such compiler feature, and is integrated into man:gcc[1] versions 4.1 and later. It replaces and extends the earlier StackGuard man:gcc[1] extension.
|
||||
|
||||
ProPolice helps to protect against stack-based buffer overflows and other attacks by laying pseudo-random numbers in key areas of the stack before calling any function. When a function returns, these "canaries" are checked and if they are found to have been changed the executable is immediately aborted. Thus any attempt to modify the return address or other variable stored on the stack in an attempt to get malicious code to run is unlikely to succeed, as the attacker would have to also manage to leave the pseudo-random canaries untouched.
|
||||
|
||||
Recompiling your application with ProPolice is an effective means of stopping most buffer-overflow attacks, but it can still be compromised.
|
||||
|
||||
==== Library based run-time bounds checking
|
||||
|
||||
Compiler-based mechanisms are completely useless for binary-only software for which you cannot recompile. For these situations there are a number of libraries which re-implement the unsafe functions of the C-library (`strcpy`, `fscanf`, `getwd`, etc..) and ensure that these functions can never write past the stack pointer.
|
||||
|
||||
* libsafe
|
||||
* libverify
|
||||
* libparanoia
|
||||
|
||||
Unfortunately these library-based defenses have a number of shortcomings. These libraries only protect against a very small set of security related issues and they neglect to fix the actual problem. These defenses may fail if the application was compiled with -fomit-frame-pointer. Also, the LD_PRELOAD and LD_LIBRARY_PATH environment variables can be overwritten/unset by the user.
|
||||
|
||||
[[secure-setuid]]
|
||||
== SetUID issues
|
||||
|
||||
There are at least 6 different IDs associated with any given process, and you must therefore be very careful with the access that your process has at any given time. In particular, all seteuid applications should give up their privileges as soon as it is no longer required.
|
||||
|
||||
The real user ID can only be changed by a superuser process. The login program sets this when a user initially logs in and it is seldom changed.
|
||||
|
||||
The effective user ID is set by the `exec()` functions if a program has its seteuid bit set. An application can call `seteuid()` at any time to set the effective user ID to either the real user ID or the saved set-user-ID. When the effective user ID is set by `exec()` functions, the previous value is saved in the saved set-user-ID.
|
||||
|
||||
[[secure-chroot]]
|
||||
== Limiting your program's environment
|
||||
|
||||
The traditional method of restricting a process is with the `chroot()` system call. This system call changes the root directory from which all other paths are referenced for a process and any child processes. For this call to succeed the process must have execute (search) permission on the directory being referenced. The new environment does not actually take effect until you `chdir()` into your new environment. It should also be noted that a process can easily break out of a chroot environment if it has root privilege. This could be accomplished by creating device nodes to read kernel memory, attaching a debugger to a process outside of the man:chroot[8] environment, or in many other creative ways.
|
||||
|
||||
The behavior of the `chroot()` system call can be controlled somewhat with the kern.chroot_allow_open_directories `sysctl` variable. When this value is set to 0, `chroot()` will fail with EPERM if there are any directories open. If set to the default value of 1, then `chroot()` will fail with EPERM if there are any directories open and the process is already subject to a `chroot()` call. For any other value, the check for open directories will be bypassed completely.
|
||||
|
||||
=== FreeBSD's jail functionality
|
||||
|
||||
The concept of a Jail extends upon the `chroot()` by limiting the powers of the superuser to create a true `virtual server'. Once a prison is set up all network communication must take place through the specified IP address, and the power of "root privilege" in this jail is severely constrained.
|
||||
|
||||
While in a prison, any tests of superuser power within the kernel using the `suser()` call will fail. However, some calls to `suser()` have been changed to a new interface `suser_xxx()`. This function is responsible for recognizing or denying access to superuser power for imprisoned processes.
|
||||
|
||||
A superuser process within a jailed environment has the power to:
|
||||
|
||||
* Manipulate credential with `setuid`, `seteuid`, `setgid`, `setegid`, `setgroups`, `setreuid`, `setregid`, `setlogin`
|
||||
* Set resource limits with `setrlimit`
|
||||
* Modify some sysctl nodes (kern.hostname)
|
||||
* `chroot()`
|
||||
* Set flags on a vnode: `chflags`, `fchflags`
|
||||
* Set attributes of a vnode such as file permission, owner, group, size, access time, and modification time.
|
||||
* Bind to privileged ports in the Internet domain (ports < 1024)
|
||||
|
||||
`Jail` is a very useful tool for running applications in a secure environment but it does have some shortcomings. Currently, the IPC mechanisms have not been converted to the `suser_xxx` so applications such as MySQL cannot be run within a jail. Superuser access may have a very limited meaning within a jail, but there is no way to specify exactly what "very limited" means.
|
||||
|
||||
=== POSIX(R).1e Process Capabilities
|
||||
|
||||
POSIX(R) has released a working draft that adds event auditing, access control lists, fine grained privileges, information labeling, and mandatory access control.
|
||||
|
||||
This is a work in progress and is the focus of the http://www.trustedbsd.org/[TrustedBSD] project. Some of the initial work has been committed to FreeBSD-CURRENT (cap_set_proc(3)).
|
||||
|
||||
[[secure-trust]]
|
||||
== Trust
|
||||
|
||||
An application should never assume that anything about the users environment is sane. This includes (but is certainly not limited to): user input, signals, environment variables, resources, IPC, mmaps, the filesystem working directory, file descriptors, the # of open files, etc.
|
||||
|
||||
You should never assume that you can catch all forms of invalid input that a user might supply. Instead, your application should use positive filtering to only allow a specific subset of inputs that you deem safe. Improper data validation has been the cause of many exploits, especially with CGI scripts on the world wide web. For filenames you need to be extra careful about paths ("../", "/"), symbolic links, and shell escape characters.
|
||||
|
||||
Perl has a really cool feature called "Taint" mode which can be used to prevent scripts from using data derived outside the program in an unsafe way. This mode will check command line arguments, environment variables, locale information, the results of certain syscalls (`readdir()`, `readlink()`, `getpwxxx()`), and all file input.
|
||||
|
||||
[[secure-race-conditions]]
|
||||
== Race Conditions
|
||||
|
||||
A race condition is anomalous behavior caused by the unexpected dependence on the relative timing of events. In other words, a programmer incorrectly assumed that a particular event would always happen before another.
|
||||
|
||||
Some of the common causes of race conditions are signals, access checks, and file opens. Signals are asynchronous events by nature so special care must be taken in dealing with them. Checking access with `access(2)` then `open(2)` is clearly non-atomic. Users can move files in between the two calls. Instead, privileged applications should `seteuid()` and then call `open()` directly. Along the same lines, an application should always set a proper umask before `open()` to obviate the need for spurious `chmod()` calls.
|
||||
|
|
@ -0,0 +1,882 @@
|
|||
---
|
||||
title: Chapter 7. Sockets
|
||||
authors:
|
||||
- author: G. Adam Stanislav
|
||||
prev: books/developers-handbook/partii
|
||||
next: books/developers-handbook/ipv6
|
||||
---
|
||||
|
||||
[[sockets]]
|
||||
= Sockets
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 7
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
:imagesdir: ../../../../images/books/developers-handbook/
|
||||
|
||||
toc::[]
|
||||
|
||||
[[sockets-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
BSD sockets take interprocess communications to a new level. It is no longer necessary for the communicating processes to run on the same machine. They still _can_, but they do not have to.
|
||||
|
||||
Not only do these processes not have to run on the same machine, they do not have to run under the same operating system. Thanks to BSD sockets, your FreeBSD software can smoothly cooperate with a program running on a Macintosh(R), another one running on a Sun(TM) workstation, yet another one running under Windows(R) 2000, all connected with an Ethernet-based local area network.
|
||||
|
||||
But your software can equally well cooperate with processes running in another building, or on another continent, inside a submarine, or a space shuttle.
|
||||
|
||||
It can also cooperate with processes that are not part of a computer (at least not in the strict sense of the word), but of such devices as printers, digital cameras, medical equipment. Just about anything capable of digital communications.
|
||||
|
||||
[[sockets-diversity]]
|
||||
== Networking and Diversity
|
||||
|
||||
We have already hinted on the _diversity_ of networking. Many different systems have to talk to each other. And they have to speak the same language. They also have to _understand_ the same language the same way.
|
||||
|
||||
People often think that _body language_ is universal. But it is not. Back in my early teens, my father took me to Bulgaria. We were sitting at a table in a park in Sofia, when a vendor approached us trying to sell us some roasted almonds.
|
||||
|
||||
I had not learned much Bulgarian by then, so, instead of saying no, I shook my head from side to side, the "universal" body language for _no_. The vendor quickly started serving us some almonds.
|
||||
|
||||
I then remembered I had been told that in Bulgaria shaking your head sideways meant _yes_. Quickly, I started nodding my head up and down. The vendor noticed, took his almonds, and walked away. To an uninformed observer, I did not change the body language: I continued using the language of shaking and nodding my head. What changed was the _meaning_ of the body language. At first, the vendor and I interpreted the same language as having completely different meaning. I had to adjust my own interpretation of that language so the vendor would understand.
|
||||
|
||||
It is the same with computers: The same symbols may have different, even outright opposite meaning. Therefore, for two computers to understand each other, they must not only agree on the same _language_, but on the same _interpretation_ of the language.
|
||||
|
||||
[[sockets-protocols]]
|
||||
== Protocols
|
||||
|
||||
While various programming languages tend to have complex syntax and use a number of multi-letter reserved words (which makes them easy for the human programmer to understand), the languages of data communications tend to be very terse. Instead of multi-byte words, they often use individual _bits_. There is a very convincing reason for it: While data travels _inside_ your computer at speeds approaching the speed of light, it often travels considerably slower between two computers.
|
||||
|
||||
As the languages used in data communications are so terse, we usually refer to them as _protocols_ rather than languages.
|
||||
|
||||
As data travels from one computer to another, it always uses more than one protocol. These protocols are _layered_. The data can be compared to the inside of an onion: You have to peel off several layers of "skin" to get to the data. This is best illustrated with a picture:
|
||||
|
||||
image::layers.png[Protocol Layers]
|
||||
|
||||
In this example, we are trying to get an image from a web page we are connected to via an Ethernet.
|
||||
|
||||
The image consists of raw data, which is simply a sequence of RGB values that our software can process, i.e., convert into an image and display on our monitor.
|
||||
|
||||
Alas, our software has no way of knowing how the raw data is organized: Is it a sequence of RGB values, or a sequence of grayscale intensities, or perhaps of CMYK encoded colors? Is the data represented by 8-bit quanta, or are they 16 bits in size, or perhaps 4 bits? How many rows and columns does the image consist of? Should certain pixels be transparent?
|
||||
|
||||
I think you get the picture...
|
||||
|
||||
To inform our software how to handle the raw data, it is encoded as a PNG file. It could be a GIF, or a JPEG, but it is a PNG.
|
||||
|
||||
And PNG is a protocol.
|
||||
|
||||
At this point, I can hear some of you yelling, _"No, it is not! It is a file format!"_
|
||||
|
||||
Well, of course it is a file format. But from the perspective of data communications, a file format is a protocol: The file structure is a _language_, a terse one at that, communicating to our _process_ how the data is organized. Ergo, it is a _protocol_.
|
||||
|
||||
Alas, if all we received was the PNG file, our software would be facing a serious problem: How is it supposed to know the data is representing an image, as opposed to some text, or perhaps a sound, or what not? Secondly, how is it supposed to know the image is in the PNG format as opposed to GIF, or JPEG, or some other image format?
|
||||
|
||||
To obtain that information, we are using another protocol: HTTP. This protocol can tell us exactly that the data represents an image, and that it uses the PNG protocol. It can also tell us some other things, but let us stay focused on protocol layers here.
|
||||
|
||||
So, now we have some data wrapped in the PNG protocol, wrapped in the HTTP protocol. How did we get it from the server?
|
||||
|
||||
By using TCP/IP over Ethernet, that is how. Indeed, that is three more protocols. Instead of continuing inside out, I am now going to talk about Ethernet, simply because it is easier to explain the rest that way.
|
||||
|
||||
Ethernet is an interesting system of connecting computers in a _local area network_ (LAN). Each computer has a _network interface card_ (NIC), which has a unique 48-bit ID called its _address_. No two Ethernet NICs in the world have the same address.
|
||||
|
||||
These NICs are all connected with each other. Whenever one computer wants to communicate with another in the same Ethernet LAN, it sends a message over the network. Every NIC sees the message. But as part of the Ethernet _protocol_, the data contains the address of the destination NIC (among other things). So, only one of all the network interface cards will pay attention to it, the rest will ignore it.
|
||||
|
||||
But not all computers are connected to the same network. Just because we have received the data over our Ethernet does not mean it originated in our own local area network. It could have come to us from some other network (which may not even be Ethernet based) connected with our own network via the Internet.
|
||||
|
||||
All data is transferred over the Internet using IP, which stands for _Internet Protocol_. Its basic role is to let us know where in the world the data has arrived from, and where it is supposed to go to. It does not _guarantee_ we will receive the data, only that we will know where it came from _if_ we do receive it.
|
||||
|
||||
Even if we do receive the data, IP does not guarantee we will receive various chunks of data in the same order the other computer has sent it to us. So, we can receive the center of our image before we receive the upper left corner and after the lower right, for example.
|
||||
|
||||
It is TCP (_Transmission Control Protocol_) that asks the sender to resend any lost data and that places it all into the proper order.
|
||||
|
||||
All in all, it took _five_ different protocols for one computer to communicate to another what an image looks like. We received the data wrapped into the PNG protocol, which was wrapped into the HTTP protocol, which was wrapped into the TCP protocol, which was wrapped into the IP protocol, which was wrapped into the Ethernet protocol.
|
||||
|
||||
Oh, and by the way, there probably were several other protocols involved somewhere on the way. For example, if our LAN was connected to the Internet through a dial-up call, it used the PPP protocol over the modem which used one (or several) of the various modem protocols, et cetera, et cetera, et cetera...
|
||||
|
||||
As a developer you should be asking by now, _"How am I supposed to handle it all?"_
|
||||
|
||||
Luckily for you, you are _not_ supposed to handle it all. You _are_ supposed to handle some of it, but not all of it. Specifically, you need not worry about the physical connection (in our case Ethernet and possibly PPP, etc). Nor do you need to handle the Internet Protocol, or the Transmission Control Protocol.
|
||||
|
||||
In other words, you do not have to do anything to receive the data from the other computer. Well, you do have to _ask_ for it, but that is almost as simple as opening a file.
|
||||
|
||||
Once you have received the data, it is up to you to figure out what to do with it. In our case, you would need to understand the HTTP protocol and the PNG file structure.
|
||||
|
||||
To use an analogy, all the internetworking protocols become a gray area: Not so much because we do not understand how it works, but because we are no longer concerned about it. The sockets interface takes care of this gray area for us:
|
||||
|
||||
image::slayers.png[Sockets Covered Protocol Layers]
|
||||
|
||||
We only need to understand any protocols that tell us how to _interpret the data_, not how to _receive_ it from another process, nor how to _send_ it to another process.
|
||||
|
||||
[[sockets-model]]
|
||||
== The Sockets Model
|
||||
|
||||
BSD sockets are built on the basic UNIX(R) model: _Everything is a file._ In our example, then, sockets would let us receive an _HTTP file_, so to speak. It would then be up to us to extract the _PNG file_ from it.
|
||||
|
||||
Due to the complexity of internetworking, we cannot just use the `open` system call, or the `open()` C function. Instead, we need to take several steps to "opening" a socket.
|
||||
|
||||
Once we do, however, we can start treating the _socket_ the same way we treat any _file descriptor_: We can `read` from it, `write` to it, `pipe` it, and, eventually, `close` it.
|
||||
|
||||
[[sockets-essential-functions]]
|
||||
== Essential Socket Functions
|
||||
|
||||
While FreeBSD offers different functions to work with sockets, we only _need_ four to "open" a socket. And in some cases we only need two.
|
||||
|
||||
[[sockets-client-server]]
|
||||
=== The Client-Server Difference
|
||||
|
||||
Typically, one of the ends of a socket-based data communication is a _server_, the other is a _client_.
|
||||
|
||||
[[sockets-common-elements]]
|
||||
==== The Common Elements
|
||||
|
||||
[[sockets-socket]]
|
||||
===== `socket`
|
||||
|
||||
The one function used by both, clients and servers, is man:socket[2]. It is declared this way:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int socket(int domain, int type, int protocol);
|
||||
....
|
||||
|
||||
The return value is of the same type as that of `open`, an integer. FreeBSD allocates its value from the same pool as that of file handles. That is what allows sockets to be treated the same way as files.
|
||||
|
||||
The `domain` argument tells the system what _protocol family_ you want it to use. Many of them exist, some are vendor specific, others are very common. They are declared in [.filename]#sys/socket.h#.
|
||||
|
||||
Use `PF_INET` for UDP, TCP and other Internet protocols (IPv4).
|
||||
|
||||
Five values are defined for the `type` argument, again, in [.filename]#sys/socket.h#. All of them start with "`SOCK_`". The most common one is `SOCK_STREAM`, which tells the system you are asking for a _reliable stream delivery service_ (which is TCP when used with `PF_INET`).
|
||||
|
||||
If you asked for `SOCK_DGRAM`, you would be requesting a _connectionless datagram delivery service_ (in our case, UDP).
|
||||
|
||||
If you wanted to be in charge of the low-level protocols (such as IP), or even network interfaces (e.g., the Ethernet), you would need to specify `SOCK_RAW`.
|
||||
|
||||
Finally, the `protocol` argument depends on the previous two arguments, and is not always meaningful. In that case, use `0` for its value.
|
||||
|
||||
[NOTE]
|
||||
.The Unconnected Socket
|
||||
====
|
||||
Nowhere, in the `socket` function have we specified to what other system we should be connected. Our newly created socket remains _unconnected_.
|
||||
|
||||
This is on purpose: To use a telephone analogy, we have just attached a modem to the phone line. We have neither told the modem to make a call, nor to answer if the phone rings.
|
||||
====
|
||||
|
||||
[[sockets-sockaddr]]
|
||||
===== `sockaddr`
|
||||
|
||||
Various functions of the sockets family expect the address of (or pointer to, to use C terminology) a small area of the memory. The various C declarations in the [.filename]#sys/socket.h# refer to it as `struct sockaddr`. This structure is declared in the same file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* Structure used by kernel to store most
|
||||
* addresses.
|
||||
*/
|
||||
struct sockaddr {
|
||||
unsigned char sa_len; /* total length */
|
||||
sa_family_t sa_family; /* address family */
|
||||
char sa_data[14]; /* actually longer; address value */
|
||||
};
|
||||
#define SOCK_MAXADDRLEN 255 /* longest possible addresses */
|
||||
....
|
||||
|
||||
Please note the _vagueness_ with which the `sa_data` field is declared, just as an array of `14` bytes, with the comment hinting there can be more than `14` of them.
|
||||
|
||||
This vagueness is quite deliberate. Sockets is a very powerful interface. While most people perhaps think of it as nothing more than the Internet interface-and most applications probably use it for that nowadays-sockets can be used for just about _any_ kind of interprocess communications, of which the Internet (or, more precisely, IP) is only one.
|
||||
|
||||
The [.filename]#sys/socket.h# refers to the various types of protocols sockets will handle as _address families_, and lists them right before the definition of `sockaddr`:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* Address families.
|
||||
*/
|
||||
#define AF_UNSPEC 0 /* unspecified */
|
||||
#define AF_LOCAL 1 /* local to host (pipes, portals) */
|
||||
#define AF_UNIX AF_LOCAL /* backward compatibility */
|
||||
#define AF_INET 2 /* internetwork: UDP, TCP, etc. */
|
||||
#define AF_IMPLINK 3 /* arpanet imp addresses */
|
||||
#define AF_PUP 4 /* pup protocols: e.g. BSP */
|
||||
#define AF_CHAOS 5 /* mit CHAOS protocols */
|
||||
#define AF_NS 6 /* XEROX NS protocols */
|
||||
#define AF_ISO 7 /* ISO protocols */
|
||||
#define AF_OSI AF_ISO
|
||||
#define AF_ECMA 8 /* European computer manufacturers */
|
||||
#define AF_DATAKIT 9 /* datakit protocols */
|
||||
#define AF_CCITT 10 /* CCITT protocols, X.25 etc */
|
||||
#define AF_SNA 11 /* IBM SNA */
|
||||
#define AF_DECnet 12 /* DECnet */
|
||||
#define AF_DLI 13 /* DEC Direct data link interface */
|
||||
#define AF_LAT 14 /* LAT */
|
||||
#define AF_HYLINK 15 /* NSC Hyperchannel */
|
||||
#define AF_APPLETALK 16 /* Apple Talk */
|
||||
#define AF_ROUTE 17 /* Internal Routing Protocol */
|
||||
#define AF_LINK 18 /* Link layer interface */
|
||||
#define pseudo_AF_XTP 19 /* eXpress Transfer Protocol (no AF) */
|
||||
#define AF_COIP 20 /* connection-oriented IP, aka ST II */
|
||||
#define AF_CNT 21 /* Computer Network Technology */
|
||||
#define pseudo_AF_RTIP 22 /* Help Identify RTIP packets */
|
||||
#define AF_IPX 23 /* Novell Internet Protocol */
|
||||
#define AF_SIP 24 /* Simple Internet Protocol */
|
||||
#define pseudo_AF_PIP 25 /* Help Identify PIP packets */
|
||||
#define AF_ISDN 26 /* Integrated Services Digital Network*/
|
||||
#define AF_E164 AF_ISDN /* CCITT E.164 recommendation */
|
||||
#define pseudo_AF_KEY 27 /* Internal key-management function */
|
||||
#define AF_INET6 28 /* IPv6 */
|
||||
#define AF_NATM 29 /* native ATM access */
|
||||
#define AF_ATM 30 /* ATM */
|
||||
#define pseudo_AF_HDRCMPLT 31 /* Used by BPF to not rewrite headers
|
||||
* in interface output routine
|
||||
*/
|
||||
#define AF_NETGRAPH 32 /* Netgraph sockets */
|
||||
#define AF_SLOW 33 /* 802.3ad slow protocol */
|
||||
#define AF_SCLUSTER 34 /* Sitara cluster protocol */
|
||||
#define AF_ARP 35
|
||||
#define AF_BLUETOOTH 36 /* Bluetooth sockets */
|
||||
#define AF_MAX 37
|
||||
....
|
||||
|
||||
The one used for IP is AF_INET. It is a symbol for the constant `2`.
|
||||
|
||||
It is the _address family_ listed in the `sa_family` field of `sockaddr` that decides how exactly the vaguely named bytes of `sa_data` will be used.
|
||||
|
||||
Specifically, whenever the _address family_ is AF_INET, we can use `struct sockaddr_in` found in [.filename]#netinet/in.h#, wherever `sockaddr` is expected:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* Socket address, internet style.
|
||||
*/
|
||||
struct sockaddr_in {
|
||||
uint8_t sin_len;
|
||||
sa_family_t sin_family;
|
||||
in_port_t sin_port;
|
||||
struct in_addr sin_addr;
|
||||
char sin_zero[8];
|
||||
};
|
||||
....
|
||||
|
||||
We can visualize its organization this way:
|
||||
|
||||
image::sain.png[sockaddr_in]
|
||||
|
||||
The three important fields are `sin_family`, which is byte 1 of the structure, `sin_port`, a 16-bit value found in bytes 2 and 3, and `sin_addr`, a 32-bit integer representation of the IP address, stored in bytes 4-7.
|
||||
|
||||
Now, let us try to fill it out. Let us assume we are trying to write a client for the _daytime_ protocol, which simply states that its server will write a text string representing the current date and time to port 13. We want to use TCP/IP, so we need to specify `AF_INET` in the address family field. `AF_INET` is defined as `2`. Let us use the IP address of `192.43.244.18`, which is the time server of US federal government (`time.nist.gov`).
|
||||
|
||||
image::sainfill.png[Specific example of sockaddr_in]
|
||||
|
||||
By the way the `sin_addr` field is declared as being of the `struct in_addr` type, which is defined in [.filename]#netinet/in.h#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* Internet address (a structure for historical reasons)
|
||||
*/
|
||||
struct in_addr {
|
||||
in_addr_t s_addr;
|
||||
};
|
||||
....
|
||||
|
||||
In addition, `in_addr_t` is a 32-bit integer.
|
||||
|
||||
The `192.43.244.18` is just a convenient notation of expressing a 32-bit integer by listing all of its 8-bit bytes, starting with the _most significant_ one.
|
||||
|
||||
So far, we have viewed `sockaddr` as an abstraction. Our computer does not store `short` integers as a single 16-bit entity, but as a sequence of 2 bytes. Similarly, it stores 32-bit integers as a sequence of 4 bytes.
|
||||
|
||||
Suppose we coded something like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sa.sin_family = AF_INET;
|
||||
sa.sin_port = 13;
|
||||
sa.sin_addr.s_addr = (((((192 << 8) | 43) << 8) | 244) << 8) | 18;
|
||||
....
|
||||
|
||||
What would the result look like?
|
||||
|
||||
Well, that depends, of course. On a Pentium(R), or other x86, based computer, it would look like this:
|
||||
|
||||
image::sainlsb.png[sockaddr_in on an Intel system]
|
||||
|
||||
On a different system, it might look like this:
|
||||
|
||||
image::sainmsb.png[sockaddr_in on an MSB system]
|
||||
|
||||
And on a PDP it might look different yet. But the above two are the most common ways in use today.
|
||||
|
||||
Ordinarily, wanting to write portable code, programmers pretend that these differences do not exist. And they get away with it (except when they code in assembly language). Alas, you cannot get away with it that easily when coding for sockets.
|
||||
|
||||
Why?
|
||||
|
||||
Because when communicating with another computer, you usually do not know whether it stores data _most significant byte_ (MSB) or _least significant byte_ (LSB) first.
|
||||
|
||||
You might be wondering, _"So, will sockets not handle it for me?"_
|
||||
|
||||
It will not.
|
||||
|
||||
While that answer may surprise you at first, remember that the general sockets interface only understands the `sa_len` and `sa_family` fields of the `sockaddr` structure. You do not have to worry about the byte order there (of course, on FreeBSD `sa_family` is only 1 byte anyway, but many other UNIX(R) systems do not have `sa_len` and use 2 bytes for `sa_family`, and expect the data in whatever order is native to the computer).
|
||||
|
||||
But the rest of the data is just `sa_data[14]` as far as sockets goes. Depending on the _address family_, sockets just forwards that data to its destination.
|
||||
|
||||
Indeed, when we enter a port number, it is because we want the other computer to know what service we are asking for. And, when we are the server, we read the port number so we know what service the other computer is expecting from us. Either way, sockets only has to forward the port number as data. It does not interpret it in any way.
|
||||
|
||||
Similarly, we enter the IP address to tell everyone on the way where to send our data to. Sockets, again, only forwards it as data.
|
||||
|
||||
That is why, we (the _programmers_, not the _sockets_) have to distinguish between the byte order used by our computer and a conventional byte order to send the data in to the other computer.
|
||||
|
||||
We will call the byte order our computer uses the _host byte order_, or just the _host order_.
|
||||
|
||||
There is a convention of sending the multi-byte data over IP _MSB first_. This, we will refer to as the _network byte order_, or simply the _network order_.
|
||||
|
||||
Now, if we compiled the above code for an Intel based computer, our _host byte order_ would produce:
|
||||
|
||||
image::sainlsb.png[Host byte order on an Intel system]
|
||||
|
||||
But the _network byte order_ requires that we store the data MSB first:
|
||||
|
||||
image::sainmsb.png[Network byte order]
|
||||
|
||||
Unfortunately, our _host order_ is the exact opposite of the _network order_.
|
||||
|
||||
We have several ways of dealing with it. One would be to _reverse_ the values in our code:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sa.sin_family = AF_INET;
|
||||
sa.sin_port = 13 << 8;
|
||||
sa.sin_addr.s_addr = (((((18 << 8) | 244) << 8) | 43) << 8) | 192;
|
||||
....
|
||||
|
||||
This will _trick_ our compiler into storing the data in the _network byte order_. In some cases, this is exactly the way to do it (e.g., when programming in assembly language). In most cases, however, it can cause a problem.
|
||||
|
||||
Suppose, you wrote a sockets-based program in C. You know it is going to run on a Pentium(R), so you enter all your constants in reverse and force them to the _network byte order_. It works well.
|
||||
|
||||
Then, some day, your trusted old Pentium(R) becomes a rusty old Pentium(R). You replace it with a system whose _host order_ is the same as the _network order_. You need to recompile all your software. All of your software continues to perform well, except the one program you wrote.
|
||||
|
||||
You have since forgotten that you had forced all of your constants to the opposite of the _host order_. You spend some quality time tearing out your hair, calling the names of all gods you ever heard of (and some you made up), hitting your monitor with a nerf bat, and performing all the other traditional ceremonies of trying to figure out why something that has worked so well is suddenly not working at all.
|
||||
|
||||
Eventually, you figure it out, say a couple of swear words, and start rewriting your code.
|
||||
|
||||
Luckily, you are not the first one to face the problem. Someone else has created the man:htons[3] and man:htonl[3] C functions to convert a `short` and `long` respectively from the _host byte order_ to the _network byte order_, and the man:ntohs[3] and man:ntohl[3] C functions to go the other way.
|
||||
|
||||
On _MSB-first_ systems these functions do nothing. On _LSB-first_ systems they convert values to the proper order.
|
||||
|
||||
So, regardless of what system your software is compiled on, your data will end up in the correct order if you use these functions.
|
||||
|
||||
[[sockets-client-functions]]
|
||||
==== Client Functions
|
||||
|
||||
Typically, the client initiates the connection to the server. The client knows which server it is about to call: It knows its IP address, and it knows the _port_ the server resides at. It is akin to you picking up the phone and dialing the number (the _address_), then, after someone answers, asking for the person in charge of wingdings (the _port_).
|
||||
|
||||
[[sockets-connect]]
|
||||
===== `connect`
|
||||
|
||||
Once a client has created a socket, it needs to connect it to a specific port on a remote system. It uses man:connect[2]:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int connect(int s, const struct sockaddr *name, socklen_t namelen);
|
||||
....
|
||||
|
||||
The `s` argument is the socket, i.e., the value returned by the `socket` function. The `name` is a pointer to `sockaddr`, the structure we have talked about extensively. Finally, `namelen` informs the system how many bytes are in our `sockaddr` structure.
|
||||
|
||||
If `connect` is successful, it returns `0`. Otherwise it returns `-1` and stores the error code in `errno`.
|
||||
|
||||
There are many reasons why `connect` may fail. For example, with an attempt to an Internet connection, the IP address may not exist, or it may be down, or just too busy, or it may not have a server listening at the specified port. Or it may outright _refuse_ any request for specific code.
|
||||
|
||||
[[sockets-first-client]]
|
||||
===== Our First Client
|
||||
|
||||
We now know enough to write a very simple client, one that will get current time from `192.43.244.18` and print it to [.filename]#stdout#.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* daytime.c
|
||||
*
|
||||
* Programmed by G. Adam Stanislav
|
||||
*/
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/socket.h>
|
||||
#include <netinet/in.h>
|
||||
|
||||
int main() {
|
||||
register int s;
|
||||
register int bytes;
|
||||
struct sockaddr_in sa;
|
||||
char buffer[BUFSIZ+1];
|
||||
|
||||
if ((s = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
|
||||
perror("socket");
|
||||
return 1;
|
||||
}
|
||||
|
||||
bzero(&sa, sizeof sa);
|
||||
|
||||
sa.sin_family = AF_INET;
|
||||
sa.sin_port = htons(13);
|
||||
sa.sin_addr.s_addr = htonl((((((192 << 8) | 43) << 8) | 244) << 8) | 18);
|
||||
if (connect(s, (struct sockaddr *)&sa, sizeof sa) < 0) {
|
||||
perror("connect");
|
||||
close(s);
|
||||
return 2;
|
||||
}
|
||||
|
||||
while ((bytes = read(s, buffer, BUFSIZ)) > 0)
|
||||
write(1, buffer, bytes);
|
||||
|
||||
close(s);
|
||||
return 0;
|
||||
}
|
||||
....
|
||||
|
||||
Go ahead, enter it in your editor, save it as [.filename]#daytime.c#, then compile and run it:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% cc -O3 -o daytime daytime.c
|
||||
% ./daytime
|
||||
|
||||
52079 01-06-19 02:29:25 50 0 1 543.9 UTC(NIST) *
|
||||
%
|
||||
....
|
||||
|
||||
In this case, the date was June 19, 2001, the time was 02:29:25 UTC. Naturally, your results will vary.
|
||||
|
||||
[[sockets-server-functions]]
|
||||
==== Server Functions
|
||||
|
||||
The typical server does not initiate the connection. Instead, it waits for a client to call it and request services. It does not know when the client will call, nor how many clients will call. It may be just sitting there, waiting patiently, one moment, The next moment, it can find itself swamped with requests from a number of clients, all calling in at the same time.
|
||||
|
||||
The sockets interface offers three basic functions to handle this.
|
||||
|
||||
[[sockets-bind]]
|
||||
===== `bind`
|
||||
|
||||
Ports are like extensions to a phone line: After you dial a number, you dial the extension to get to a specific person or department.
|
||||
|
||||
There are 65535 IP ports, but a server usually processes requests that come in on only one of them. It is like telling the phone room operator that we are now at work and available to answer the phone at a specific extension. We use man:bind[2] to tell sockets which port we want to serve.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int bind(int s, const struct sockaddr *addr, socklen_t addrlen);
|
||||
....
|
||||
|
||||
Beside specifying the port in `addr`, the server may include its IP address. However, it can just use the symbolic constant INADDR_ANY to indicate it will serve all requests to the specified port regardless of what its IP address is. This symbol, along with several similar ones, is declared in [.filename]#netinet/in.h#
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#define INADDR_ANY (u_int32_t)0x00000000
|
||||
....
|
||||
|
||||
Suppose we were writing a server for the _daytime_ protocol over TCP/IP. Recall that it uses port 13. Our `sockaddr_in` structure would look like this:
|
||||
|
||||
image::sainserv.png[Example Server sockaddr_in]
|
||||
|
||||
[[sockets-listen]]
|
||||
===== `listen`
|
||||
|
||||
To continue our office phone analogy, after you have told the phone central operator what extension you will be at, you now walk into your office, and make sure your own phone is plugged in and the ringer is turned on. Plus, you make sure your call waiting is activated, so you can hear the phone ring even while you are talking to someone.
|
||||
|
||||
The server ensures all of that with the man:listen[2] function.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int listen(int s, int backlog);
|
||||
....
|
||||
|
||||
In here, the `backlog` variable tells sockets how many incoming requests to accept while you are busy processing the last request. In other words, it determines the maximum size of the queue of pending connections.
|
||||
|
||||
[[sockets-accept]]
|
||||
===== `accept`
|
||||
|
||||
After you hear the phone ringing, you accept the call by answering the call. You have now established a connection with your client. This connection remains active until either you or your client hang up.
|
||||
|
||||
The server accepts the connection by using the man:accept[2] function.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
int accept(int s, struct sockaddr *addr, socklen_t *addrlen);
|
||||
....
|
||||
|
||||
Note that this time `addrlen` is a pointer. This is necessary because in this case it is the socket that fills out `addr`, the `sockaddr_in` structure.
|
||||
|
||||
The return value is an integer. Indeed, the `accept` returns a _new socket_. You will use this new socket to communicate with the client.
|
||||
|
||||
What happens to the old socket? It continues to listen for more requests (remember the `backlog` variable we passed to `listen`?) until we `close` it.
|
||||
|
||||
Now, the new socket is meant only for communications. It is fully connected. We cannot pass it to `listen` again, trying to accept additional connections.
|
||||
|
||||
[[sockets-first-server]]
|
||||
===== Our First Server
|
||||
|
||||
Our first server will be somewhat more complex than our first client was: Not only do we have more sockets functions to use, but we need to write it as a daemon.
|
||||
|
||||
This is best achieved by creating a _child process_ after binding the port. The main process then exits and returns control to the shell (or whatever program invoked it).
|
||||
|
||||
The child calls `listen`, then starts an endless loop, which accepts a connection, serves it, and eventually closes its socket.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* daytimed - a port 13 server
|
||||
*
|
||||
* Programmed by G. Adam Stanislav
|
||||
* June 19, 2001
|
||||
*/
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/socket.h>
|
||||
#include <netinet/in.h>
|
||||
|
||||
#define BACKLOG 4
|
||||
|
||||
int main() {
|
||||
register int s, c;
|
||||
int b;
|
||||
struct sockaddr_in sa;
|
||||
time_t t;
|
||||
struct tm *tm;
|
||||
FILE *client;
|
||||
|
||||
if ((s = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
|
||||
perror("socket");
|
||||
return 1;
|
||||
}
|
||||
|
||||
bzero(&sa, sizeof sa);
|
||||
|
||||
sa.sin_family = AF_INET;
|
||||
sa.sin_port = htons(13);
|
||||
|
||||
if (INADDR_ANY)
|
||||
sa.sin_addr.s_addr = htonl(INADDR_ANY);
|
||||
|
||||
if (bind(s, (struct sockaddr *)&sa, sizeof sa) < 0) {
|
||||
perror("bind");
|
||||
return 2;
|
||||
}
|
||||
|
||||
switch (fork()) {
|
||||
case -1:
|
||||
perror("fork");
|
||||
return 3;
|
||||
break;
|
||||
default:
|
||||
close(s);
|
||||
return 0;
|
||||
break;
|
||||
case 0:
|
||||
break;
|
||||
}
|
||||
|
||||
listen(s, BACKLOG);
|
||||
|
||||
for (;;) {
|
||||
b = sizeof sa;
|
||||
|
||||
if ((c = accept(s, (struct sockaddr *)&sa, &b)) < 0) {
|
||||
perror("daytimed accept");
|
||||
return 4;
|
||||
}
|
||||
|
||||
if ((client = fdopen(c, "w")) == NULL) {
|
||||
perror("daytimed fdopen");
|
||||
return 5;
|
||||
}
|
||||
|
||||
if ((t = time(NULL)) < 0) {
|
||||
perror("daytimed time");
|
||||
|
||||
return 6;
|
||||
}
|
||||
|
||||
tm = gmtime(&t);
|
||||
fprintf(client, "%.4i-%.2i-%.2iT%.2i:%.2i:%.2iZ\n",
|
||||
tm->tm_year + 1900,
|
||||
tm->tm_mon + 1,
|
||||
tm->tm_mday,
|
||||
tm->tm_hour,
|
||||
tm->tm_min,
|
||||
tm->tm_sec);
|
||||
|
||||
fclose(client);
|
||||
}
|
||||
}
|
||||
....
|
||||
|
||||
We start by creating a socket. Then we fill out the `sockaddr_in` structure in `sa`. Note the conditional use of INADDR_ANY:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
if (INADDR_ANY)
|
||||
sa.sin_addr.s_addr = htonl(INADDR_ANY);
|
||||
....
|
||||
|
||||
Its value is `0`. Since we have just used `bzero` on the entire structure, it would be redundant to set it to `0` again. But if we port our code to some other system where INADDR_ANY is perhaps not a zero, we need to assign it to `sa.sin_addr.s_addr`. Most modern C compilers are clever enough to notice that INADDR_ANY is a constant. As long as it is a zero, they will optimize the entire conditional statement out of the code.
|
||||
|
||||
After we have called `bind` successfully, we are ready to become a _daemon_: We use `fork` to create a child process. In both, the parent and the child, the `s` variable is our socket. The parent process will not need it, so it calls `close`, then it returns `0` to inform its own parent it had terminated successfully.
|
||||
|
||||
Meanwhile, the child process continues working in the background. It calls `listen` and sets its backlog to `4`. It does not need a large value here because _daytime_ is not a protocol many clients request all the time, and because it can process each request instantly anyway.
|
||||
|
||||
Finally, the daemon starts an endless loop, which performs the following steps:
|
||||
|
||||
[.procedure]
|
||||
. Call `accept`. It waits here until a client contacts it. At that point, it receives a new socket, `c`, which it can use to communicate with this particular client.
|
||||
. It uses the C function `fdopen` to turn the socket from a low-level _file descriptor_ to a C-style `FILE` pointer. This will allow the use of `fprintf` later on.
|
||||
. It checks the time, and prints it in the _ISO 8601_ format to the `client` "file". It then uses `fclose` to close the file. That will automatically close the socket as well.
|
||||
|
||||
We can _generalize_ this, and use it as a model for many other servers:
|
||||
|
||||
image::serv.png[Sequential Server]
|
||||
|
||||
This flowchart is good for _sequential servers_, i.e., servers that can serve one client at a time, just as we were able to with our _daytime_ server. This is only possible whenever there is no real "conversation" going on between the client and the server: As soon as the server detects a connection to the client, it sends out some data and closes the connection. The entire operation may take nanoseconds, and it is finished.
|
||||
|
||||
The advantage of this flowchart is that, except for the brief moment after the parent ``fork``s and before it exits, there is always only one _process_ active: Our server does not take up much memory and other system resources.
|
||||
|
||||
Note that we have added _initialize daemon_ in our flowchart. We did not need to initialize our own daemon, but this is a good place in the flow of the program to set up any `signal` handlers, open any files we may need, etc.
|
||||
|
||||
Just about everything in the flow chart can be used literally on many different servers. The _serve_ entry is the exception. We think of it as a _"black box"_, i.e., something you design specifically for your own server, and just "plug it into the rest."
|
||||
|
||||
Not all protocols are that simple. Many receive a request from the client, reply to it, then receive another request from the same client. As a result, they do not know in advance how long they will be serving the client. Such servers usually start a new process for each client. While the new process is serving its client, the daemon can continue listening for more connections.
|
||||
|
||||
Now, go ahead, save the above source code as [.filename]#daytimed.c# (it is customary to end the names of daemons with the letter `d`). After you have compiled it, try running it:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% ./daytimed
|
||||
bind: Permission denied
|
||||
%
|
||||
....
|
||||
|
||||
What happened here? As you will recall, the _daytime_ protocol uses port 13. But all ports below 1024 are reserved to the superuser (otherwise, anyone could start a daemon pretending to serve a commonly used port, while causing a security breach).
|
||||
|
||||
Try again, this time as the superuser:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ./daytimed
|
||||
#
|
||||
....
|
||||
|
||||
What... Nothing? Let us try again:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ./daytimed
|
||||
|
||||
bind: Address already in use
|
||||
#
|
||||
....
|
||||
|
||||
Every port can only be bound by one program at a time. Our first attempt was indeed successful: It started the child daemon and returned quietly. It is still running and will continue to run until you either kill it, or any of its system calls fail, or you reboot the system.
|
||||
|
||||
Fine, we know it is running in the background. But is it working? How do we know it is a proper _daytime_ server? Simple:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% telnet localhost 13
|
||||
|
||||
Trying ::1...
|
||||
telnet: connect to address ::1: Connection refused
|
||||
Trying 127.0.0.1...
|
||||
Connected to localhost.
|
||||
Escape character is '^]'.
|
||||
2001-06-19T21:04:42Z
|
||||
Connection closed by foreign host.
|
||||
%
|
||||
....
|
||||
|
||||
telnet tried the new IPv6, and failed. It retried with IPv4 and succeeded. The daemon works.
|
||||
|
||||
If you have access to another UNIX(R) system via telnet, you can use it to test accessing the server remotely. My computer does not have a static IP address, so this is what I did:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% who
|
||||
|
||||
whizkid ttyp0 Jun 19 16:59 (216.127.220.143)
|
||||
xxx ttyp1 Jun 19 16:06 (xx.xx.xx.xx)
|
||||
% telnet 216.127.220.143 13
|
||||
|
||||
Trying 216.127.220.143...
|
||||
Connected to r47.bfm.org.
|
||||
Escape character is '^]'.
|
||||
2001-06-19T21:31:11Z
|
||||
Connection closed by foreign host.
|
||||
%
|
||||
....
|
||||
|
||||
Again, it worked. Will it work using the domain name?
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% telnet r47.bfm.org 13
|
||||
|
||||
Trying 216.127.220.143...
|
||||
Connected to r47.bfm.org.
|
||||
Escape character is '^]'.
|
||||
2001-06-19T21:31:40Z
|
||||
Connection closed by foreign host.
|
||||
%
|
||||
....
|
||||
|
||||
By the way, telnet prints the _Connection closed by foreign host_ message after our daemon has closed the socket. This shows us that, indeed, using `fclose(client);` in our code works as advertised.
|
||||
|
||||
[[sockets-helper-functions]]
|
||||
== Helper Functions
|
||||
|
||||
FreeBSD C library contains many helper functions for sockets programming. For example, in our sample client we hard coded the `time.nist.gov` IP address. But we do not always know the IP address. Even if we do, our software is more flexible if it allows the user to enter the IP address, or even the domain name.
|
||||
|
||||
[[sockets-gethostbyname]]
|
||||
=== `gethostbyname`
|
||||
|
||||
While there is no way to pass the domain name directly to any of the sockets functions, the FreeBSD C library comes with the man:gethostbyname[3] and man:gethostbyname2[3] functions, declared in [.filename]#netdb.h#.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct hostent * gethostbyname(const char *name);
|
||||
struct hostent * gethostbyname2(const char *name, int af);
|
||||
....
|
||||
|
||||
Both return a pointer to the `hostent` structure, with much information about the domain. For our purposes, the `h_addr_list[0]` field of the structure points at `h_length` bytes of the correct address, already stored in the _network byte order_.
|
||||
|
||||
This allows us to create a much more flexible-and much more useful-version of our daytime program:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/*
|
||||
* daytime.c
|
||||
*
|
||||
* Programmed by G. Adam Stanislav
|
||||
* 19 June 2001
|
||||
*/
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/socket.h>
|
||||
#include <netinet/in.h>
|
||||
#include <netdb.h>
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
register int s;
|
||||
register int bytes;
|
||||
struct sockaddr_in sa;
|
||||
struct hostent *he;
|
||||
char buf[BUFSIZ+1];
|
||||
char *host;
|
||||
|
||||
if ((s = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
|
||||
perror("socket");
|
||||
return 1;
|
||||
}
|
||||
|
||||
bzero(&sa, sizeof sa);
|
||||
|
||||
sa.sin_family = AF_INET;
|
||||
sa.sin_port = htons(13);
|
||||
|
||||
host = (argc > 1) ? (char *)argv[1] : "time.nist.gov";
|
||||
|
||||
if ((he = gethostbyname(host)) == NULL) {
|
||||
herror(host);
|
||||
return 2;
|
||||
}
|
||||
|
||||
bcopy(he->h_addr_list[0],&sa.sin_addr, he->h_length);
|
||||
|
||||
if (connect(s, (struct sockaddr *)&sa, sizeof sa) < 0) {
|
||||
perror("connect");
|
||||
return 3;
|
||||
}
|
||||
|
||||
while ((bytes = read(s, buf, BUFSIZ)) > 0)
|
||||
write(1, buf, bytes);
|
||||
|
||||
close(s);
|
||||
return 0;
|
||||
}
|
||||
....
|
||||
|
||||
We now can type a domain name (or an IP address, it works both ways) on the command line, and the program will try to connect to its _daytime_ server. Otherwise, it will still default to `time.nist.gov`. However, even in this case we will use `gethostbyname` rather than hard coding `192.43.244.18`. That way, even if its IP address changes in the future, we will still find it.
|
||||
|
||||
Since it takes virtually no time to get the time from your local server, you could run daytime twice in a row: First to get the time from `time.nist.gov`, the second time from your own system. You can then compare the results and see how exact your system clock is:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% daytime ; daytime localhost
|
||||
|
||||
52080 01-06-20 04:02:33 50 0 0 390.2 UTC(NIST) *
|
||||
2001-06-20T04:02:35Z
|
||||
%
|
||||
....
|
||||
|
||||
As you can see, my system was two seconds ahead of the NIST time.
|
||||
|
||||
[[sockets-getservbyname]]
|
||||
=== `getservbyname`
|
||||
|
||||
Sometimes you may not be sure what port a certain service uses. The man:getservbyname[3] function, also declared in [.filename]#netdb.h# comes in very handy in those cases:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct servent * getservbyname(const char *name, const char *proto);
|
||||
....
|
||||
|
||||
The `servent` structure contains the `s_port`, which contains the proper port, already in _network byte order_.
|
||||
|
||||
Had we not known the correct port for the _daytime_ service, we could have found it this way:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
struct servent *se;
|
||||
...
|
||||
if ((se = getservbyname("daytime", "tcp")) == NULL {
|
||||
fprintf(stderr, "Cannot determine which port to use.\n");
|
||||
return 7;
|
||||
}
|
||||
sa.sin_port = se->s_port;
|
||||
....
|
||||
|
||||
You usually do know the port. But if you are developing a new protocol, you may be testing it on an unofficial port. Some day, you will register the protocol and its port (if nowhere else, at least in your [.filename]#/etc/services#, which is where `getservbyname` looks). Instead of returning an error in the above code, you just use the temporary port number. Once you have listed the protocol in [.filename]#/etc/services#, your software will find its port without you having to rewrite the code.
|
||||
|
||||
[[sockets-concurrent-servers]]
|
||||
== Concurrent Servers
|
||||
|
||||
Unlike a sequential server, a _concurrent server_ has to be able to serve more than one client at a time. For example, a _chat server_ may be serving a specific client for hours-it cannot wait till it stops serving a client before it serves the next one.
|
||||
|
||||
This requires a significant change in our flowchart:
|
||||
|
||||
image::serv2.png[Concurrent Server]
|
||||
|
||||
We moved the _serve_ from the _daemon process_ to its own _server process_. However, because each child process inherits all open files (and a socket is treated just like a file), the new process inherits not only the _"accepted handle,"_ i.e., the socket returned by the `accept` call, but also the _top socket_, i.e., the one opened by the top process right at the beginning.
|
||||
|
||||
However, the _server process_ does not need this socket and should `close` it immediately. Similarly, the _daemon process_ no longer needs the _accepted socket_, and not only should, but _must_ `close` it-otherwise, it will run out of available _file descriptors_ sooner or later.
|
||||
|
||||
After the _server process_ is done serving, it should close the _accepted socket_. Instead of returning to `accept`, it now exits.
|
||||
|
||||
Under UNIX(R), a process does not really _exit_. Instead, it _returns_ to its parent. Typically, a parent process ``wait``s for its child process, and obtains a return value. However, our _daemon process_ cannot simply stop and wait. That would defeat the whole purpose of creating additional processes. But if it never does `wait`, its children will become _zombies_-no longer functional but still roaming around.
|
||||
|
||||
For that reason, the _daemon process_ needs to set _signal handlers_ in its _initialize daemon_ phase. At least a SIGCHLD signal has to be processed, so the daemon can remove the zombie return values from the system and release the system resources they are taking up.
|
||||
|
||||
That is why our flowchart now contains a _process signals_ box, which is not connected to any other box. By the way, many servers also process SIGHUP, and typically interpret as the signal from the superuser that they should reread their configuration files. This allows us to change settings without having to kill and restart these servers.
|
||||
|
|
@ -0,0 +1,167 @@
|
|||
---
|
||||
title: Chapter 6. Regression and Performance Testing
|
||||
authors:
|
||||
prev: books/developers-handbook/policies
|
||||
next: books/developers-handbook/partii
|
||||
---
|
||||
|
||||
[[testing]]
|
||||
= Regression and Performance Testing
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 6
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
Regression tests are used to exercise a particular bit of the system to check that it works as expected, and to make sure that old bugs are not reintroduced.
|
||||
|
||||
The FreeBSD regression testing tools can be found in the FreeBSD source tree in the directory [.filename]#src/tools/regression#.
|
||||
|
||||
[[testing-micro-benchmark]]
|
||||
== Micro Benchmark Checklist
|
||||
|
||||
This section contains hints for doing proper micro-benchmarking on FreeBSD or of FreeBSD itself.
|
||||
|
||||
It is not possible to use all of the suggestions below every single time, but the more used, the better the benchmark's ability to test small differences will be.
|
||||
|
||||
* Disable APM and any other kind of clock fiddling (ACPI ?).
|
||||
* Run in single user mode. E.g., man:cron[8], and other daemons only add noise. The man:sshd[8] daemon can also cause problems. If ssh access is required during testing either disable the SSHv1 key regeneration, or kill the parent `sshd` daemon during the tests.
|
||||
* Do not run man:ntpd[8].
|
||||
* If man:syslog[3] events are generated, run man:syslogd[8] with an empty [.filename]#/etc/syslogd.conf#, otherwise, do not run it.
|
||||
* Minimize disk-I/O, avoid it entirely if possible.
|
||||
* Do not mount file systems that are not needed.
|
||||
* Mount [.filename]#/#, [.filename]#/usr#, and any other file system as read-only if possible. This removes atime updates to disk (etc.) from the I/O picture.
|
||||
* Reinitialize the read/write test file system with man:newfs[8] and populate it from a man:tar[1] or man:dump[8] file before every run. Unmount and mount it before starting the test. This results in a consistent file system layout. For a worldstone test this would apply to [.filename]#/usr/obj# (just reinitialize with `newfs` and mount). To get 100% reproducibility, populate the file system from a man:dd[1] file (i.e.: `dd if=myimage of=/dev/ad0s1h bs=1m`)
|
||||
* Use malloc backed or preloaded man:md[4] partitions.
|
||||
* Reboot between individual iterations of the test, this gives a more consistent state.
|
||||
* Remove all non-essential device drivers from the kernel. For instance if USB is not needed for the test, do not put USB in the kernel. Drivers which attach often have timeouts ticking away.
|
||||
* Unconfigure hardware that are not in use. Detach disks with man:atacontrol[8] and man:camcontrol[8] if the disks are not used for the test.
|
||||
* Do not configure the network unless it is being tested, or wait until after the test has been performed to ship the results off to another computer.
|
||||
+
|
||||
If the system must be connected to a public network, watch out for spikes of broadcast traffic. Even though it is hardly noticeable, it will take up CPU cycles. Multicast has similar caveats.
|
||||
* Put each file system on its own disk. This minimizes jitter from head-seek optimizations.
|
||||
* Minimize output to serial or VGA consoles. Running output into files gives less jitter. (Serial consoles easily become a bottleneck.) Do not touch keyboard while the test is running, even kbd:[space] or kbd:[back-space] shows up in the numbers.
|
||||
* Make sure the test is long enough, but not too long. If the test is too short, timestamping is a problem. If it is too long temperature changes and drift will affect the frequency of the quartz crystals in the computer. Rule of thumb: more than a minute, less than an hour.
|
||||
* Try to keep the temperature as stable as possible around the machine. This affects both quartz crystals and disk drive algorithms. To get real stable clock, consider stabilized clock injection. E.g., get a OCXO + PLL, inject output into clock circuits instead of motherboard xtal. Contact {phk} for more information about this.
|
||||
* Run the test at least 3 times but it is better to run more than 20 times both for "before" and "after" code. Try to interleave if possible (i.e.: do not run 20 times before then 20 times after), this makes it possible to spot environmental effects. Do not interleave 1:1, but 3:3, this makes it possible to spot interaction effects.
|
||||
+
|
||||
A good pattern is: `bababa{bbbaaa}*`. This gives hint after the first 1+1 runs (so it is possible to stop the test if it goes entirely the wrong way), a standard deviation after the first 3+3 (gives a good indication if it is going to be worth a long run) and trending and interaction numbers later on.
|
||||
* Use man:ministat[1] to see if the numbers are significant. Consider buying "Cartoon guide to statistics" ISBN: 0062731025, highly recommended, if you have forgotten or never learned about standard deviation and Student's T.
|
||||
* Do not use background man:fsck[8] unless the test is a benchmark of background `fsck`. Also, disable `background_fsck` in [.filename]#/etc/rc.conf# unless the benchmark is not started at least 60+"``fsck`` runtime" seconds after the boot, as man:rc[8] wakes up and checks if `fsck` needs to run on any file systems when background `fsck` is enabled. Likewise, make sure there are no snapshots lying around unless the benchmark is a test with snapshots.
|
||||
* If the benchmark show unexpected bad performance, check for things like high interrupt volume from an unexpected source. Some versions of ACPI have been reported to "misbehave" and generate excess interrupts. To help diagnose odd test results, take a few snapshots of `vmstat -i` and look for anything unusual.
|
||||
* Make sure to be careful about optimization parameters for kernel and userspace, likewise debugging. It is easy to let something slip through and realize later the test was not comparing the same thing.
|
||||
* Do not ever benchmark with the `WITNESS` and `INVARIANTS` kernel options enabled unless the test is interested to benchmarking those features. `WITNESS` can cause 400%+ drops in performance. Likewise, userspace man:malloc[3] parameters default differently in -CURRENT from the way they ship in production releases.
|
||||
|
||||
[[testing-tinderbox]]
|
||||
== The FreeBSD Source Tinderbox
|
||||
|
||||
The source Tinderbox consists of:
|
||||
|
||||
* A build script, [.filename]#tinderbox#, that automates checking out a specific version of the FreeBSD source tree and building it.
|
||||
* A supervisor script, [.filename]#tbmaster#, that monitors individual Tinderbox instances, logs their output, and emails failure notices.
|
||||
* A CGI script named [.filename]#index.cgi# that reads a set of tbmaster logs and presents an easy-to-read HTML summary of them.
|
||||
* A set of build servers that continually test the tip of the most important FreeBSD code branches.
|
||||
* A webserver that keeps a complete set of Tinderbox logs and displays an up-to-date summary.
|
||||
|
||||
The scripts are maintained and were developed by {des}, and are now written in Perl, a move on from their original incarnation as shell scripts. All scripts and configuration files are kept in https://www.freebsd.org/cgi/cvsweb.cgi/projects/tinderbox/[/projects/tinderbox/].
|
||||
|
||||
For more information about the tinderbox and tbmaster scripts at this stage, see their respective man pages: tinderbox(1) and tbmaster(1).
|
||||
|
||||
== The index.cgi Script
|
||||
|
||||
The [.filename]#index.cgi# script generates the HTML summary of tinderbox and tbmaster logs. Although originally intended to be used as a CGI script, as indicated by its name, this script can also be run from the command line or from a man:cron[8] job, in which case it will look for logs in the directory where the script is located. It will automatically detect context, generating HTTP headers when it is run as a CGI script. It conforms to XHTML standards and is styled using CSS.
|
||||
|
||||
The script starts in the `main()` block by attempting to verify that it is running on the official Tinderbox website. If it is not, a page indicating it is not an official website is produced, and a URL to the official site is provided.
|
||||
|
||||
Next, it scans the log directory to get an inventory of configurations, branches and architectures for which log files exist, to avoid hard-coding a list into the script and potentially ending up with blank rows or columns. This information is derived from the names of the log files matching the following pattern:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
tinderbox-$config-$branch-$arch-$machine.{brief,full}
|
||||
....
|
||||
|
||||
The configurations used on the official Tinderbox build servers are named for the branches they build. For example, the `releng_8` configuration is used to build `RELENG_8` as well as all still-supported release branches.
|
||||
|
||||
Once all of this startup procedure has been successfully completed, `do_config()` is called for each configuration.
|
||||
|
||||
The `do_config()` function generates HTML for a single Tinderbox configuration.
|
||||
|
||||
It works by first generating a header row, then iterating over each branch build with the specified configuration, producing a single row of results for each in the following manner:
|
||||
|
||||
* For each item:
|
||||
|
||||
** For each machine within that architecture:
|
||||
|
||||
*** If a brief log file exists, then:
|
||||
|
||||
**** Call `success()` to determine the outcome of the build.
|
||||
**** Output the modification size.
|
||||
**** Output the size of the brief log file with a link to the log file itself.
|
||||
**** If a full log file also exists, then:
|
||||
|
||||
***** Output the size of the full log file with a link to the log file itself.
|
||||
|
||||
*** Otherwise:
|
||||
|
||||
**** No output.
|
||||
|
||||
The `success()` function mentioned above scans a brief log file for the string "tinderbox run completed" in order to determine whether the build was successful.
|
||||
|
||||
Configurations and branches are sorted according to their branch rank. This is computed as follows:
|
||||
|
||||
* `HEAD` and `CURRENT` have rank 9999.
|
||||
* `RELENG_x` has rank __``xx``__99.
|
||||
* `RELENG_x_y` has rank _xxyy_.
|
||||
|
||||
This means that `HEAD` always ranks highest, and `RELENG` branches are ranked in numerical order, with each `STABLE` branch ranking higher than the release branches forked off of it. For instance, for FreeBSD 8, the order from highest to lowest would be:
|
||||
|
||||
* `RELENG_8` (branch rank 899).
|
||||
* `RELENG_8_3` (branch rank 803).
|
||||
* `RELENG_8_2` (branch rank 802).
|
||||
* `RELENG_8_1` (branch rank 801).
|
||||
* `RELENG_8_0` (branch rank 800).
|
||||
|
||||
The colors that Tinderbox uses for each cell in the table are defined by CSS. Successful builds are displayed with green text; unsuccessful builds are displayed with red text. The color fades as time passes since the corresponding build, with every half an hour bringing the color closer to grey.
|
||||
|
||||
== Official Build Servers
|
||||
|
||||
The official Tinderbox build servers are hosted by http://www.sentex.ca[Sentex Data Communications], who also host the FreeBSD Netperf Cluster.
|
||||
|
||||
Three build servers currently exist:
|
||||
|
||||
_freebsd-current.sentex.ca_ builds:
|
||||
|
||||
* `HEAD` for amd64, arm, i386, i386/pc98, ia64, mips, powerpc, powerpc64, and sparc64.
|
||||
* `RELENG_9` and supported 9._X_ branches for amd64, arm, i386, i386/pc98, ia64, mips, powerpc, powerpc64, and sparc64.
|
||||
|
||||
_freebsd-stable.sentex.ca_ builds:
|
||||
|
||||
* `RELENG_8` and supported 8._X_ branches for amd64, i386, i386/pc98, ia64, mips, powerpc and sparc64.
|
||||
|
||||
_freebsd-legacy.sentex.ca_ builds:
|
||||
|
||||
* `RELENG_7` and supported 7._X_ branches for amd64, i386, i386/pc98, ia64, powerpc, and sparc64.
|
||||
|
||||
== Official Summary Site
|
||||
|
||||
Summaries and logs from the official build servers are available online at http://tinderbox.FreeBSD.org[http://tinderbox.FreeBSD.org], hosted by {des} and set up as follows:
|
||||
|
||||
* A man:cron[8] job checks the build servers at regular intervals and downloads any new log files using man:rsync[1].
|
||||
* Apache is set up to use [.filename]#index.cgi# as `DirectoryIndex`.
|
||||
File diff suppressed because it is too large
Load diff
3851
documentation/content/en/books/developers-handbook/x86/chapter.adoc
Normal file
3851
documentation/content/en/books/developers-handbook/x86/chapter.adoc
Normal file
File diff suppressed because it is too large
Load diff
2642
documentation/content/en/books/faq/_index.adoc
Normal file
2642
documentation/content/en/books/faq/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
1
documentation/content/en/books/faq/chapters-order.adoc
Normal file
1
documentation/content/en/books/faq/chapters-order.adoc
Normal file
|
|
@ -0,0 +1 @@
|
|||
_index.adoc
|
||||
115
documentation/content/en/books/fdp-primer/_index.adoc
Normal file
115
documentation/content/en/books/fdp-primer/_index.adoc
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
---
|
||||
title: FreeBSD Documentation Project Primer for New Contributors
|
||||
authors:
|
||||
- author: The FreeBSD Documentation Project
|
||||
copyright: 1998-2021 DocEng
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["general"]
|
||||
---
|
||||
|
||||
= FreeBSD Documentation Project Primer for New Contributors
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 2
|
||||
:icons: font
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:partnums:
|
||||
:chapter-signifier: Chapter
|
||||
:part-signifier: Part
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:book: true
|
||||
:pdf: false
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
:chapters-path: content/en/books/fdp-primer/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
[.abstract-title]
|
||||
[abstract]
|
||||
Abstract
|
||||
|
||||
Thank you for becoming a part of the FreeBSD Documentation Project. Your contribution is extremely valuable, and we appreciate it.
|
||||
|
||||
This primer covers details needed to start contributing to the FreeBSD Documentation Project, or FDP, including tools, software, and the philosophy behind the Documentation Project.
|
||||
|
||||
This is a work in progress. Corrections and additions are always welcome.
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
include::{chapters-path}toc-figures.adoc[]
|
||||
|
||||
include::{chapters-path}toc-tables.adoc[]
|
||||
|
||||
include::{chapters-path}toc-examples.adoc[]
|
||||
|
||||
:sectnums!:
|
||||
|
||||
include::{chapters-path}preface/chapter.adoc[leveloffset=+1, lines=7..-1]
|
||||
|
||||
:sectnums:
|
||||
|
||||
include::{chapters-path}overview/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}tools/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}working-copy/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}structure/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}doc-build/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}asciidoctor-primer/chapter.adoc[leveloffset=+1, lines=7..21; 27..-1]
|
||||
|
||||
include::{chapters-path}rosetta/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}translations/chapter.adoc[leveloffset=+1, lines=7..21; 28..-1]
|
||||
|
||||
include::{chapters-path}po-translations/chapter.adoc[leveloffset=+1, lines=7..21; 27..-1]
|
||||
|
||||
include::{chapters-path}manual-pages/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}writing-style/chapter.adoc[leveloffset=+1, lines=7..21; 27..-1]
|
||||
|
||||
include::{chapters-path}editor-config/chapter.adoc[leveloffset=+1, lines=7..21; 25..-1]
|
||||
|
||||
include::{chapters-path}see-also/chapter.adoc[leveloffset=+1, lines=7..21; 27..-1]
|
||||
|
||||
:sectnums!:
|
||||
|
||||
include::{chapters-path}examples/chapter.adoc[leveloffset=+1, lines=6..21; 25..-1]
|
||||
|
||||
:sectnums:
|
||||
|
|
@ -0,0 +1,204 @@
|
|||
---
|
||||
title: Chapter 6. AsciiDoctor Primer
|
||||
prev: books/fdp-primer/doc-build
|
||||
next: books/fdp-primer/rosetta
|
||||
---
|
||||
|
||||
[[asciidoctor-primer]]
|
||||
= AsciiDoctor Primer
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 6
|
||||
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
Most FDP documentation is written with AsciiDoc. This chapter explains what that means, how to read and understand the documentation source, and the techniques used. To get a complete reference of the AsciiDoctor capabilities please consult the link:https://docs.asciidoctor.org/home/[Asciidoctor documentation]. Some of the examples used in this chapter have been taken from the link:https://docs.asciidoctor.org/asciidoc/latest/syntax-quick-reference[AsciiDoc Syntax Quick Reference].
|
||||
|
||||
[[asciidoctor-primer-overview]]
|
||||
== Overview
|
||||
|
||||
In the original days of computers, electronic text was simple. There were a few character sets like ASCII or EBCDIC, but that was about it. Text was text, and what you saw really was what you got. No frills, no formatting, no intelligence.
|
||||
|
||||
In the original days of computers, electronic text was simple. There were a few character sets like ASCII or EBCDIC, but that was about it. Text was text, and what you saw really was what you got. No frills, no formatting, no intelligence.
|
||||
|
||||
Inevitably, this was not enough. When text is in a machine-usable format, machines are expected to be able to use and manipulate it intelligently. Authors want to indicate that certain phrases should be emphasized, or added to a glossary, or made into hyperlinks. Filenames could be shown in a “typewriter” style font for viewing on screen, but as “italics” when printed, or any of a myriad of other options for presentation.
|
||||
|
||||
It was once hoped that Artificial Intelligence (AI) would make this easy. The computer would read the document and automatically identify key phrases, filenames, text that the reader should type in, examples, and more. Unfortunately, real life has not happened quite like that, and computers still require assistance before they can meaningfully process text.
|
||||
|
||||
More precisely, they need help identifying what is what. Consider this text:
|
||||
|
||||
To remove [.filename]#/tmp/foo#, use man:rm[1].
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
% rm /tmp/foo
|
||||
----
|
||||
|
||||
It is easy to see which parts are filenames, which are commands to be typed in, which parts are references to manual pages, and so on. But the computer processing the document cannot. For this we need markup.
|
||||
|
||||
|
||||
The previous example is actually represented in this document like this:
|
||||
|
||||
....
|
||||
To remove [.filename]#/tmp/foo#, use man:rm[1].
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
% rm /tmp/foo
|
||||
----
|
||||
....
|
||||
|
||||
[[asciidoctor-headings]]
|
||||
== Headings
|
||||
|
||||
AsciiDoctor support six headings levels. If the document type is `article` only one level 0 (`=`) can be used. If the document type is `book` can be multiple level 0 (`=`).
|
||||
|
||||
This is an example of headings in an `article`.
|
||||
|
||||
....
|
||||
= Document Title (Level 0)
|
||||
|
||||
== Level 1 Section Title
|
||||
|
||||
=== Level 2 Section Title
|
||||
|
||||
==== Level 3 Section Title
|
||||
|
||||
===== Level 4 Section Title
|
||||
|
||||
====== Level 5 Section Title
|
||||
|
||||
== Another Level 1 Section Title
|
||||
....
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
Section levels cannot be skipped when nesting sections.
|
||||
|
||||
The following syntax is not correct.
|
||||
|
||||
....
|
||||
= Document Title
|
||||
|
||||
== Level 2
|
||||
|
||||
==== Level 4
|
||||
....
|
||||
====
|
||||
|
||||
[[asciidoctor-paragraphs]]
|
||||
== Paragraphs
|
||||
|
||||
Paragraphs don't require special markup in AsciiDoc. A paragraph is defined by one or more consecutive lines of text. To create a new paragraph leave one blank line.
|
||||
|
||||
For example, this is a heading with two paragraphs
|
||||
|
||||
....
|
||||
= This is the heading
|
||||
|
||||
This is the firts paragraph.
|
||||
|
||||
And this is the second paragraph.
|
||||
....
|
||||
|
||||
[[asciidoctor-lists]]
|
||||
== Lists
|
||||
|
||||
AsciiDoctor support two type of lists ordered and unordered. To get more information about lists check link:https://docs.asciidoctor.org/asciidoc/latest/syntax-quick-reference/#lists[AsciiDoc Syntax Quick Reference]
|
||||
|
||||
[[asciidoctor-ordered-lists]]
|
||||
=== Ordered lists
|
||||
|
||||
To create an ordered list use the `*` character.
|
||||
|
||||
For example this is an ordered list.
|
||||
|
||||
....
|
||||
* First item
|
||||
* Second item
|
||||
** Subsecond item
|
||||
* Thrid item
|
||||
....
|
||||
|
||||
And this would be rendered as.
|
||||
|
||||
* First item
|
||||
* Second item
|
||||
** Subsecond item
|
||||
* Thrid item
|
||||
|
||||
[[asciidoctor-unordered-lists]]
|
||||
=== Unordered lists
|
||||
|
||||
To create an unordered list use the `.` character.
|
||||
|
||||
For example this is an unordered lists.
|
||||
|
||||
....
|
||||
. First item
|
||||
. Second item
|
||||
.. Subsecond item
|
||||
. Thrid item
|
||||
....
|
||||
|
||||
And this would be rendered as.
|
||||
|
||||
. First item
|
||||
. Second item
|
||||
.. Subsecond item
|
||||
. Thrid item
|
||||
|
||||
[[asciidoctor-links]]
|
||||
== Links
|
||||
|
||||
[[asciidoctor-links-external]]
|
||||
=== External links
|
||||
|
||||
To point to another website the `link` macro should be used.
|
||||
|
||||
....
|
||||
link:https://www.FreeBSD.org[FreeBSD]
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
As the AsciiDoctor documentation points the `link` macro is not required when the target starts with a URL scheme like `https`. But is a good practice to ensure that AsciiDoctor renders correctly the link. Specially in non-latin languages like Japanese.
|
||||
====
|
||||
|
||||
[[asciidoctor-links-internal]]
|
||||
=== Internal link
|
||||
|
||||
To point to another book or article the AsciiDoctor variables should be used. For example, if we are in the `cups` article and we want to point to `ipsec-must` these steps should be used.
|
||||
|
||||
. Include the [.filename]#urls.adoc# file from [.filename]#~/doc/shared# folder.
|
||||
+
|
||||
....
|
||||
\include::shared/{lang}/urls.adoc[]
|
||||
....
|
||||
+
|
||||
. Then create a link using the AsciiDoctor variable to the `ipsec-must` article.
|
||||
+
|
||||
....
|
||||
link:{ipsec-must}[IPSec-Must article]
|
||||
....
|
||||
+
|
||||
And this would be rendered as.
|
||||
+
|
||||
link:{ipsec-must}[IPSec-Must article]
|
||||
|
||||
[[asciidoctor-conclusion]]
|
||||
== Conclusion
|
||||
|
||||
That is the conclusion of this AsciiDoctor primer. For reasons of space and complexity, several things have not been covered in depth (or at all).
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
preface/chapter.adoc
|
||||
overview/chapter.adoc
|
||||
tools/chapter.adoc
|
||||
working-copy/chapter.adoc
|
||||
structure/chapter.adoc
|
||||
doc-build/chapter.adoc
|
||||
asciidoctor-primer/chapter.adoc
|
||||
rosetta/chapter.adoc
|
||||
translations/chapter.adoc
|
||||
po-translations/chapter.adoc
|
||||
manual-pages/chapter.adoc
|
||||
writing-style/chapter.adoc
|
||||
editor-config/chapter.adoc
|
||||
see-also/chapter.adoc
|
||||
examples/chapter.adoc
|
||||
195
documentation/content/en/books/fdp-primer/doc-build/chapter.adoc
Normal file
195
documentation/content/en/books/fdp-primer/doc-build/chapter.adoc
Normal file
|
|
@ -0,0 +1,195 @@
|
|||
---
|
||||
title: Chapter 5. The FreeBSD Documentation Build Process
|
||||
prev: books/fdp-primer/structure
|
||||
next: books/fdp-primer/asciidoctor-primer
|
||||
---
|
||||
|
||||
[[doc-build]]
|
||||
= The FreeBSD Documentation Build Process
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 5
|
||||
|
||||
toc::[]
|
||||
|
||||
This chapter covers organization of the documentation build process and how man:make[1] is used to control it.
|
||||
|
||||
[[doc-build-rendering]]
|
||||
== Rendering AsciiDoc into Output
|
||||
|
||||
Different types of output can be produced from a single AsciiDoc source file.
|
||||
|
||||
[cols="20%,20%,60%", frame="none", options="header"]
|
||||
|===
|
||||
| Formats
|
||||
| File Type
|
||||
| Description
|
||||
|
||||
|`html`
|
||||
|HTML
|
||||
|An `article` or `book` chapter.
|
||||
|
||||
|`pdf`
|
||||
|PDF
|
||||
|Portable Document Format.
|
||||
|
||||
|`epub`
|
||||
|EPUB
|
||||
|Electronic Publication. ePub file format.
|
||||
|===
|
||||
|
||||
To render the documentation and the website use one of the following examples.
|
||||
|
||||
[[documentation-build-example]]
|
||||
.Build the documentation
|
||||
[example]
|
||||
====
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/documentation
|
||||
% make
|
||||
....
|
||||
====
|
||||
|
||||
[[website-build-example]]
|
||||
.Build the website
|
||||
[example]
|
||||
====
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/website
|
||||
% make
|
||||
....
|
||||
====
|
||||
|
||||
[[doc-build-toolset]]
|
||||
== The FreeBSD Documentation Build Toolset
|
||||
|
||||
These are the tools used to build and install the FDP documentation.
|
||||
|
||||
* The primary build tool is man:make[1], specifically Berkeley Make.
|
||||
* Python is used to generate the Table of Contents and other related Tables.
|
||||
* Hugo
|
||||
* AsciiDoctor
|
||||
* Git
|
||||
|
||||
[[doc-build-makefile]]
|
||||
== Understanding the Makefile in the Documentation Tree
|
||||
|
||||
There is only one [.filename]#Makefile# in the [.filename]#documentation# and other in the [.filename]#website#. Both are pretty similar.
|
||||
|
||||
[[documentation-makefile]]
|
||||
=== Documentation Makefile
|
||||
|
||||
This [.filename]#Makefile# take the form of:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# Generate the FreeBSD documentation
|
||||
#
|
||||
# Copyright (c) 2020-2021 The FreeBSD Documentation Project
|
||||
# Copyright (c) 2020-2021, Sergio Carlavilla <carlavilla@FreeBSD.org>
|
||||
#
|
||||
# Targets intended for use on the command line
|
||||
#
|
||||
# all (default) - generate the books TOC and compile all the documentation
|
||||
# compile - generate the books TOC and build all the documentation
|
||||
|
||||
MAINTAINER=carlavilla@FreeBSD.org <.>
|
||||
|
||||
PYTHON_CMD = /usr/local/bin/python3.7 <.>
|
||||
HUGO_CMD = /usr/local/bin/hugo <.>
|
||||
LANGUAGES = en,es,pt_BR,de,ja,zh_CN,zh_TW,ru,el,hu,it,mn,nl,pl,fr <.>
|
||||
|
||||
all: starting-message generate-books-toc run <.>
|
||||
generate: starting-message generate-books-toc build <.>
|
||||
|
||||
starting-message: <.>
|
||||
@echo ---------------------------------------------------------------
|
||||
@echo Building the documentation
|
||||
@echo ---------------------------------------------------------------
|
||||
|
||||
generate-books-toc: <.>
|
||||
${PYTHON_CMD} ./tools/books-toc-parts-creator.py -l ${LANGUAGES}
|
||||
${PYTHON_CMD} ./tools/books-toc-creator.py -l ${LANGUAGES}
|
||||
${PYTHON_CMD} ./tools/books-toc-figures-creator.py -l ${LANGUAGES}
|
||||
${PYTHON_CMD} ./tools/books-toc-tables-creator.py -l ${LANGUAGES}
|
||||
${PYTHON_CMD} ./tools/books-toc-examples-creator.py -l ${LANGUAGES}
|
||||
|
||||
run: <.>
|
||||
${HUGO_CMD} server -D
|
||||
|
||||
build: <.>
|
||||
${HUGO_CMD} --minify
|
||||
....
|
||||
|
||||
<.> The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
|
||||
<.> `PYTHON_CMD` flag specifies the location of the Python binary.
|
||||
<.> `HUGO_CMD` flag specifies the location of the Hugo binary.
|
||||
<.> `LANGUAGES` flag specifies in which languages the table of contents has to be generated.
|
||||
<.> `all` target generate the books TOCs and run the documentation in the Hugo local webserver.
|
||||
<.> `generate` target generate the books TOCs and build the documentation in the [.filename]#~/doc/documentation/public# folder. The content of this folder should be placed in a HTTP server like nginx.
|
||||
<.> `starting-message` shows a message in the CLI to show the user that the process is running.
|
||||
<.> `generate-books-toc` calls the scripts to generate the books TOCs.
|
||||
<.> `run` runs hugo webserver in a random free port.
|
||||
<.> `build` builds the documentation and puts the result in the [.filename]#~/doc/documentation/public#.
|
||||
|
||||
[[website-makefile]]
|
||||
=== Website Makefile
|
||||
|
||||
This [.filename]#Makefile# take the form of:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# Generate the FreeBSD website
|
||||
#
|
||||
# Copyright (c) 2020-2021 The FreeBSD Documentation Project
|
||||
# Copyright (c) 2020-2021, Sergio Carlavilla <carlavilla@FreeBSD.org>
|
||||
#
|
||||
# Targets intended for use on the command line
|
||||
#
|
||||
# all (default) - generate the releases.toml and compile all the website
|
||||
# generate - generate the releases.toml and build all the website
|
||||
|
||||
MAINTAINER=carlavilla@FreeBSD.org <.>
|
||||
|
||||
PYTHON_CMD = /usr/local/bin/python3.7 <.>
|
||||
HUGO_CMD = /usr/local/bin/hugo <.>
|
||||
|
||||
all: starting-message generate-releases run <.>
|
||||
generate: starting-message generate-releases build <.>
|
||||
|
||||
starting-message: <.>
|
||||
@echo ---------------------------------------------------------------
|
||||
@echo Building the website
|
||||
@echo ---------------------------------------------------------------
|
||||
|
||||
generate-releases: <.>
|
||||
${PYTHON_CMD} ./tools/releases-toml.py -p ./shared/releases.adoc
|
||||
|
||||
run: <.>
|
||||
${HUGO_CMD} server -D
|
||||
|
||||
build: <.>
|
||||
${HUGO_CMD} --minify
|
||||
....
|
||||
|
||||
<.> The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
|
||||
<.> `PYTHON_CMD` flag specifies the location of the Python binary.
|
||||
<.> `HUGO_CMD` flag specifies the location of the Hugo binary.
|
||||
<.> `all` target generate the books TOCs and run the website in the Hugo local webserver.
|
||||
<.> `generate` target generate the books TOCs and build the website in the [.filename]#~/doc/website/public# folder. The content of this folder should be placed in a HTTP server like nginx.
|
||||
<.> `starting-message` shows a message in the CLI to show the user that the process is running.
|
||||
<.> `generate-releases` calls the script used to convert from AsciiDoc variables to TOML variables. With this conversion, the releases variables can be used in AsciiDoc and in the Hugo custom templates.
|
||||
<.> `run` runs hugo webserver in a random free port.
|
||||
<.> `build` builds the website and puts the result in the [.filename]#~/doc/website/public#.
|
||||
|
|
@ -0,0 +1,217 @@
|
|||
---
|
||||
title: Chapter 12. Editor Configuration
|
||||
prev: books/fdp-primer/writing-style
|
||||
next: books/fdp-primer/see-also
|
||||
---
|
||||
|
||||
[[editor-config]]
|
||||
= Editor Configuration
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 12
|
||||
|
||||
toc::[]
|
||||
|
||||
Adjusting text editor configuration can make working on document files quicker and easier, and help documents conform to FDP guidelines.
|
||||
|
||||
[[editor-config-vim]]
|
||||
== Vim
|
||||
|
||||
Install from package:editors/vim[], package:editors/vim-console[], or package:editors/vim-tiny[] then follow the configuration instructions in <<editor-config-vim-config>>.
|
||||
|
||||
[[editor-config-vim-use]]
|
||||
=== Use
|
||||
|
||||
Press kbd:[P] to reformat paragraphs or text that has been selected in Visual mode. Press kbd:[T] to replace groups of eight spaces with a tab.
|
||||
|
||||
[[editor-config-vim-config]]
|
||||
=== Configuration
|
||||
|
||||
Edit [.filename]#~/.vimrc#, adding these lines to the end of the file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
if has("autocmd")
|
||||
au BufNewFile,BufRead *.sgml,*.ent,*.xsl,*.xml call Set_SGML()
|
||||
au BufNewFile,BufRead *.[1-9] call ShowSpecial()
|
||||
endif " has(autocmd)
|
||||
|
||||
function Set_Highlights()
|
||||
"match ExtraWhitespace /^\s* \s*\|\s\+$/
|
||||
highlight default link OverLength ErrorMsg
|
||||
match OverLength /\%71v.\+/
|
||||
return 0
|
||||
endfunction " Set_Highlights()
|
||||
|
||||
function ShowSpecial()
|
||||
setlocal list listchars=tab:>>,trail:*,eol:$
|
||||
hi def link nontext ErrorMsg
|
||||
return 0
|
||||
endfunction " ShowSpecial()
|
||||
|
||||
function Set_SGML()
|
||||
setlocal number
|
||||
syn match sgmlSpecial "&[^;]*;"
|
||||
setlocal syntax=sgml
|
||||
setlocal filetype=xml
|
||||
setlocal shiftwidth=2
|
||||
setlocal textwidth=70
|
||||
setlocal tabstop=8
|
||||
setlocal softtabstop=2
|
||||
setlocal formatprg="fmt -p"
|
||||
setlocal autoindent
|
||||
setlocal smartindent
|
||||
" Rewrap paragraphs
|
||||
noremap P gqj
|
||||
" Replace spaces with tabs
|
||||
noremap T :s/ /\t/<CR>
|
||||
call ShowSpecial()
|
||||
call Set_Highlights()
|
||||
return 0
|
||||
endfunction " Set_SGML()
|
||||
....
|
||||
|
||||
[[editor-config-emacs]]
|
||||
== Emacs
|
||||
|
||||
Install from package:editors/emacs[] or package:editors/emacs-devel[].
|
||||
|
||||
[[editor-config-emacs-validation]]
|
||||
=== Validation
|
||||
|
||||
Emacs's nxml-mode uses compact relax NG schemas for validating XML. A compact relax NG schema for FreeBSD's extension to DocBook 5.0 is included in the documentation repository. To configure nxml-mode to validate using this schema, create [.filename]#~/.emacs.d/schema/schemas.xml# and add these lines to the file:
|
||||
|
||||
....
|
||||
locatingRules xmlns="http://thaiopensource.com/ns/locating-rules/1.0"
|
||||
documentElement localName="section" typeId="DocBook"
|
||||
documentElement localName="chapter" typeId="DocBook"
|
||||
documentElement localName="article" typeId="DocBook"
|
||||
documentElement localName="book" typeId="DocBook"
|
||||
typeId id="DocBook" uri="/usr/local/shared/xml/docbook/5.0/rng/docbook.rnc"
|
||||
locatingRules
|
||||
....
|
||||
|
||||
[[editor-config-emacs-igor]]
|
||||
=== Automated Proofreading with Flycheck and Igor
|
||||
|
||||
The Flycheck package is available from Milkypostman's Emacs Lisp Package Archive (MELPA). If MELPA is not already in Emacs's packages-archives, it can be added by evaluating
|
||||
|
||||
....
|
||||
(add-to-list 'package-archives '("melpa" . "http://stable.melpa.org/packages/") t)
|
||||
....
|
||||
|
||||
Add the line to Emacs's initialization file (one of [.filename]#~/.emacs#, [.filename]#~/.emacs.el#, or [.filename]#~.emacs.d/init.el#) to make this change permanent.
|
||||
|
||||
To install Flycheck, evaluate
|
||||
|
||||
....
|
||||
(package-install 'flycheck)
|
||||
....
|
||||
|
||||
Create a Flycheck checker for package:textproc/igor[] by evaluating
|
||||
|
||||
....
|
||||
(flycheck-define-checker igor
|
||||
"FreeBSD Documentation Project sanity checker.
|
||||
|
||||
See URLs https://www.freebsd.org/docproj/ and
|
||||
http://www.freshports.org/textproc/igor/."
|
||||
:command ("igor" "-X" source-inplace)
|
||||
:error-parser flycheck-parse-checkstyle
|
||||
:modes (nxml-mode)
|
||||
:standard-input t)
|
||||
|
||||
(add-to-list 'flycheck-checkers 'igor 'append)
|
||||
....
|
||||
|
||||
Again, add these lines to Emacs's initialization file to make the changes permanent.
|
||||
|
||||
[[editor-config-emacs-specifc]]
|
||||
=== FreeBSD Documentation Specific Settings
|
||||
|
||||
To apply settings specific to the FreeBSD documentation project, create [.filename]#.dir-locals.el# in the root directory of the documentation repository and add these lines to the file:
|
||||
|
||||
....
|
||||
;;; Directory Local Variables
|
||||
;;; For more information see (info "(emacs) Directory Variables")
|
||||
|
||||
((nxml-mode
|
||||
(eval . (turn-on-auto-fill))
|
||||
(fill-column . 70)
|
||||
(eval . (require 'flycheck))
|
||||
(eval . (flycheck-mode 1))
|
||||
(flycheck-checker . igor)
|
||||
(eval . (add-to-list 'rng-schema-locating-files "~/.emacs.d/schema/schemas.xml"))))
|
||||
....
|
||||
|
||||
[[editor-config-nano]]
|
||||
== nano
|
||||
|
||||
Install from package:editors/nano[] or package:editors/nano-devel[].
|
||||
|
||||
[[editor-config-nano-config]]
|
||||
=== Configuration
|
||||
|
||||
Copy the sample XML syntax highlight file to the user's home directory:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% cp /usr/local/shared/nano/xml.nanorc ~/.nanorc
|
||||
....
|
||||
|
||||
Use an editor to replace the lines in the [.filename]#~/.nanorc# `syntax "xml"` block with these rules:
|
||||
|
||||
....
|
||||
syntax "xml" "\.([jrs]html?|xml|xslt?)$"
|
||||
# trailing whitespace
|
||||
color ,blue "[[:space:]]+$"
|
||||
# multiples of eight spaces at the start a line
|
||||
# (after zero or more tabs) should be a tab
|
||||
color ,blue "^([TAB]*[ ]{8})+"
|
||||
# tabs after spaces
|
||||
color ,yellow "( )+TAB"
|
||||
# highlight indents that have an odd number of spaces
|
||||
color ,red "^(([ ]{2})+|(TAB+))*[ ]{1}[^ ]{1}"
|
||||
# lines longer than 70 characters
|
||||
color ,yellow "^(.{71})|(TAB.{63})|(TAB{2}.{55})|(TAB{3}.{47}).+$"
|
||||
....
|
||||
|
||||
Process the file to create embedded tabs:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% perl -i'' -pe 's/TAB/\t/g' ~/.nanorc
|
||||
....
|
||||
|
||||
[[editor-config-nano-use]]
|
||||
=== Use
|
||||
|
||||
Specify additional helpful options when running the editor:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% nano -AKipwz -r 70 -T8 _index.adoc
|
||||
....
|
||||
|
||||
Users of man:csh[1] can define an alias in [.filename]#~/.cshrc# to automate these options:
|
||||
|
||||
....
|
||||
alias nano "nano -AKipwz -r 70 -T8"
|
||||
....
|
||||
|
||||
After the alias is defined, the options will be added automatically:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% nano _index.adoc
|
||||
....
|
||||
137
documentation/content/en/books/fdp-primer/examples/chapter.adoc
Normal file
137
documentation/content/en/books/fdp-primer/examples/chapter.adoc
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
---
|
||||
title: Appendix A. Examples
|
||||
prev: books/fdp-primer/see-also/
|
||||
---
|
||||
|
||||
[appendix]
|
||||
[[examples]]
|
||||
= Examples
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: A
|
||||
|
||||
toc::[]
|
||||
|
||||
These examples are not exhaustive-they do not contain all the elements that might be desirable to use, particularly in a document's front matter. For more examples of AsciiDoctor, examine the AsciiDoc source for this and other documents available in the Git `doc` repository, or available online starting at link:https://cgit.freebsd.org/doc/[https://cgit.freebsd.org/doc/].
|
||||
|
||||
[[examples-asciidoctor-book]]
|
||||
== AsciiDoctor `book`
|
||||
|
||||
.AsciiDoctor `book`
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
---
|
||||
title: An Example Book
|
||||
authors:
|
||||
- author: The FreeBSD Documentation Project
|
||||
copyright: 1995-2021 The FreeBSD Documentation Project
|
||||
releaseinfo: ""
|
||||
trademarks: ["general"]
|
||||
---
|
||||
|
||||
= An Example Book
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 2
|
||||
:icons: font
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:partnums:
|
||||
:chapter-signifier: Chapter
|
||||
:part-signifier: Part
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:book: true
|
||||
:pdf: false
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:chapters-path: content/en/books/bookname/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
[abstract]
|
||||
Abstract
|
||||
|
||||
Abstract section
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
:sectnums!:
|
||||
|
||||
\include::{chapters-path}preface/_index.adoc[leveloffset=+1]
|
||||
|
||||
:sectnums:
|
||||
|
||||
\include::{chapters-path}parti.adoc[lines=7..18]
|
||||
|
||||
\include::{chapters-path}chapter-name/_index.adoc[leveloffset=+1]
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
[[examples-asciidoctor-article]]
|
||||
== AsciiDoctor `article`
|
||||
|
||||
.AsciiDoctor `article`
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
---
|
||||
title: An Example Article
|
||||
authors:
|
||||
- author: Your name and surname
|
||||
email: foo@example.com
|
||||
trademarks: ["general"]
|
||||
---
|
||||
|
||||
= An Example Article
|
||||
:doctype: article
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
== My First Section
|
||||
|
||||
This is the first section in my article.
|
||||
|
||||
== My First Sub-Section
|
||||
|
||||
This is the first sub-section in my article.
|
||||
|
||||
....
|
||||
|
||||
====
|
||||
|
|
@ -0,0 +1,488 @@
|
|||
---
|
||||
title: Chapter 10. Manual Pages
|
||||
prev: books/fdp-primer/po-translations
|
||||
next: books/fdp-primer/writing-style
|
||||
---
|
||||
|
||||
[[manual-pages]]
|
||||
= Manual Pages
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 10
|
||||
|
||||
toc::[]
|
||||
|
||||
[[manual-pages-introduction]]
|
||||
== Introduction
|
||||
|
||||
_Manual pages_, commonly shortened to _man pages_, were conceived as readily-available reminders for command syntax, device driver details, or configuration file formats. They have become an extremely valuable quick-reference from the command line for users, system administrators, and programmers.
|
||||
|
||||
Although intended as reference material rather than tutorials, the EXAMPLES sections of manual pages often provide detailed use case.
|
||||
|
||||
Manual pages are generally shown interactively by the man:man[1] command. When the user types `man ls`, a search is performed for a manual page matching `ls`. The first matching result is displayed.
|
||||
|
||||
[[manual-pages-sections]]
|
||||
== Sections
|
||||
|
||||
Manual pages are grouped into _sections_. Each section contains manual pages for a specific category of documentation:
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Section Number
|
||||
| Category
|
||||
|
||||
|
||||
|1
|
||||
|General Commands
|
||||
|
||||
|2
|
||||
|System Calls
|
||||
|
||||
|3
|
||||
|Library Functions
|
||||
|
||||
|4
|
||||
|Kernel Interfaces
|
||||
|
||||
|5
|
||||
|File Formats
|
||||
|
||||
|6
|
||||
|Games
|
||||
|
||||
|7
|
||||
|Miscellaneous
|
||||
|
||||
|8
|
||||
|System Manager
|
||||
|
||||
|9
|
||||
|Kernel Developer
|
||||
|===
|
||||
|
||||
[[manual-pages-markup]]
|
||||
== Markup
|
||||
|
||||
Various markup forms and rendering programs have been used for manual pages. FreeBSD has used man:groff[7] and the newer man:mandoc[1]. Most existing FreeBSD manual pages, and all new ones, use the man:mdoc[7] form of markup. This is a simple line-based markup that is reasonably expressive. It is mostly semantic: parts of text are marked up for what they are, rather than for how they should appear when rendered. There is some appearance-based markup which is usually best avoided.
|
||||
|
||||
Manual page source is usually interpreted and displayed to the screen interactively. The source files can be ordinary text files or compressed with man:gzip[1] to save space.
|
||||
|
||||
Manual pages can also be rendered to other formats, including PostScript for printing or PDF generation. See man:man[1].
|
||||
|
||||
[[manual-pages-markup-sections]]
|
||||
=== Manual Page Sections
|
||||
|
||||
Manual pages are composed of several standard sections. Each section has a title in upper case, and the sections for a particular type of manual page appear in a specific order. For a category 1 General Command manual page, the sections are:
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Section Name
|
||||
| Description
|
||||
|
||||
|
||||
|NAME
|
||||
|Name of the command
|
||||
|
||||
|SYNOPSIS
|
||||
|Format of options and arguments
|
||||
|
||||
|DESCRIPTION
|
||||
|Description of purpose and usage
|
||||
|
||||
|ENVIRONMENT
|
||||
|Environment settings that affect operation
|
||||
|
||||
|EXIT STATUS
|
||||
|Error codes returned on exit
|
||||
|
||||
|EXAMPLES
|
||||
|Examples of usage
|
||||
|
||||
|COMPATIBILITY
|
||||
|Compatibility with other implementations
|
||||
|
||||
|SEE ALSO
|
||||
|Cross-reference to related manual pages
|
||||
|
||||
|STANDARDS
|
||||
|Compatibility with standards like POSIX
|
||||
|
||||
|HISTORY
|
||||
|History of implementation
|
||||
|
||||
|BUGS
|
||||
|Known bugs
|
||||
|
||||
|AUTHORS
|
||||
|People who created the command or wrote the manual page.
|
||||
|===
|
||||
|
||||
Some sections are optional, and the combination of sections for a specific type of manual page vary. Examples of the most common types are shown later in this chapter.
|
||||
|
||||
[[manual-pages-markup-macros]]
|
||||
=== Macros
|
||||
|
||||
man:mdoc[7] markup is based on _macros_. Lines that begin with a dot contain macro commands, each two or three letters long. For example, consider this portion of the man:ls[1] manual page:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
.Dd December 1, 2015
|
||||
.Dt LS 1
|
||||
.Sh NAME
|
||||
.Nm ls
|
||||
.Nd list directory contents
|
||||
.Sh SYNOPSIS
|
||||
.Nm
|
||||
.Op Fl -libxo
|
||||
.Op Fl ABCFGHILPRSTUWZabcdfghiklmnopqrstuwxy1,
|
||||
.Op Fl D Ar format
|
||||
.Op Ar
|
||||
.Sh DESCRIPTION
|
||||
For each operand that names a
|
||||
.Ar file
|
||||
of a type other than
|
||||
directory,
|
||||
.Nm
|
||||
displays its name as well as any requested,
|
||||
associated information.
|
||||
For each operand that names a
|
||||
.Ar file
|
||||
of type directory,
|
||||
.Nm
|
||||
displays the names of files contained
|
||||
within that directory, as well as any requested, associated
|
||||
information.
|
||||
....
|
||||
|
||||
A _Document date_ and _Document title_ are defined.
|
||||
A _Section header_ for the NAME section is defined. Then the _Name_ of the command and a one-line _Name description_ are defined.
|
||||
The SYNOPSIS section begins. This section describes the command-line options and arguments accepted.
|
||||
_Name_ (`.Nm`) has already been defined, and repeating it here just displays the defined value in the text.
|
||||
An _Optional_ _Flag_ called `-libxo` is shown. The `Fl` macro adds a dash to the beginning of flags, so this appears in the manual page as `--libxo`.
|
||||
A long list of optional single-character flags are shown.
|
||||
An optional `-D` flag is defined. If the `-D` flag is given, it must be followed by an _Argument_. The argument is a _format_, a string that tells man:ls[1] what to display and how to display it. Details on the format string are given later in the manual page.
|
||||
A final optional argument is defined. Since no name is specified for the argument, the default of `file ...` is used.
|
||||
The _Section header_ for the DESCRIPTION section is defined.
|
||||
|
||||
When rendered with the command `man ls`, the result displayed on the screen looks like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
LS(1) FreeBSD General Commands Manual LS(1)
|
||||
|
||||
NAME
|
||||
ls - list directory contents
|
||||
|
||||
SYNOPSIS
|
||||
ls [--libxo] [-ABCFGHILPRSTUWZabcdfghiklmnopqrstuwxy1,] [-D format]
|
||||
[file ...]
|
||||
|
||||
DESCRIPTION
|
||||
For each operand that names a file of a type other than directory, ls
|
||||
displays its name as well as any requested, associated information. For
|
||||
each operand that names a file of type directory, ls displays the names
|
||||
of files contained within that directory, as well as any requested,
|
||||
associated information.
|
||||
....
|
||||
|
||||
Optional values are shown inside square brackets.
|
||||
|
||||
[[manual-pages-markup-guidelines]]
|
||||
=== Markup Guidelines
|
||||
|
||||
The man:mdoc[7] markup language is not very strict. For clarity and consistency, the FreeBSD Documentation project adds some additional style guidelines:
|
||||
|
||||
Only the first letter of macros is upper case::
|
||||
Always use upper case for the first letter of a macro and lower case for the remaining letters.
|
||||
|
||||
Begin new sentences on new lines::
|
||||
Start a new sentence on a new line, do not begin it on the same line as an existing sentence.
|
||||
|
||||
Update `.Dd` when making non-trivial changes to a manual page::
|
||||
The _Document date_ informs the reader about the last time the manual page was updated. It is important to update whenever non-trivial changes are made to the manual pages. Trivial changes like spelling or punctuation fixes that do not affect usage can be made without updating `.Dd`.
|
||||
|
||||
Give examples::
|
||||
Show the reader examples when possible. Even trivial examples are valuable, because what is trivial to the writer is not necessarily trivial to the reader. Three examples are a good goal. A trivial example shows the minimal requirements, a serious example shows actual use, and an in-depth example demonstrates unusual or non-obvious functionality.
|
||||
|
||||
Include the BSD license::
|
||||
Include the BSD license on new manual pages. The preferred license is available from the link:{committers-guide}[Committer's Guide].
|
||||
|
||||
[[manual-pages-markup-tricks]]
|
||||
=== Markup Tricks
|
||||
|
||||
Add a space before punctuation on a line with macros. Example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
.Sh SEE ALSO
|
||||
.Xr geom 4 ,
|
||||
.Xr boot0cfg 8 ,
|
||||
.Xr geom 8 ,
|
||||
.Xr gptboot 8
|
||||
....
|
||||
|
||||
Note how the commas at the end of the `.Xr` lines have been placed after a space. The `.Xr` macro expects two parameters to follow it, the name of an external manual page, and a section number. The space separates the punctuation from the section number. Without the space, the external links would incorrectly point to section `4,` or `8,`.
|
||||
|
||||
[[manual-pages-markup-important-macros]]
|
||||
=== Important Macros
|
||||
|
||||
Some very common macros will be shown here. For more usage examples, see man:mdoc[7], man:groff_mdoc[7], or search for actual use in [.filename]#/usr/shared/man/man*# directories. For example, to search for examples of the `.Bd`_Begin display_ macro:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% find /usr/shared/man/man* | xargs zgrep '.Bd'
|
||||
....
|
||||
|
||||
[[manual-pages-markup-important-macros-organizational]]
|
||||
==== Organizational Macros
|
||||
|
||||
Some macros are used to define logical blocks of a manual page.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Organizational Macro
|
||||
| Use
|
||||
|
||||
|
||||
|`.Sh`
|
||||
|Section header. Followed by the name of the section, traditionally all upper case. Think of these as chapter titles.
|
||||
|
||||
|`.Ss`
|
||||
|Subsection header. Followed by the name of the subsection. Used to divide a `.Sh` section into subsections.
|
||||
|
||||
|`.Bl`
|
||||
|Begin list. Start a list of items.
|
||||
|
||||
|`.El`
|
||||
|End a list.
|
||||
|
||||
|`.Bd`
|
||||
|Begin display. Begin a special area of text, like an indented area.
|
||||
|
||||
|`.Ed`
|
||||
|End display.
|
||||
|===
|
||||
|
||||
[[manual-pages-markup-important-macros-inline]]
|
||||
==== Inline Macros
|
||||
|
||||
Many macros are used to mark up inline text.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Inline Macro
|
||||
| Use
|
||||
|
||||
|
||||
|`.Nm`
|
||||
|Name. Called with a name as a parameter on the first use, then used later without the parameter to display the name that has already been defined.
|
||||
|
||||
|`.Pa`
|
||||
|Path to a file. Used to mark up filenames and directory paths.
|
||||
|===
|
||||
|
||||
[[manual-pages-sample-structures]]
|
||||
== Sample Manual Page Structures
|
||||
|
||||
This section shows minimal desired man page contents for several common categories of manual pages.
|
||||
|
||||
[[manual-pages-sample-structures-section-1-8]]
|
||||
=== Section 1 or 8 Command
|
||||
|
||||
The preferred basic structure for a section 1 or 8 command:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
.Dd August 25, 2017
|
||||
.Dt EXAMPLECMD 8
|
||||
.Os
|
||||
.Sh NAME
|
||||
.Nm examplecmd
|
||||
.Nd "command to demonstrate section 1 and 8 man pages"
|
||||
.Sh SYNOPSIS
|
||||
.Nm
|
||||
.Op Fl v
|
||||
.Sh DESCRIPTION
|
||||
The
|
||||
.Nm
|
||||
utility does nothing except demonstrate a trivial but complete
|
||||
manual page for a section 1 or 8 command.
|
||||
.Sh SEE ALSO
|
||||
.Xr exampleconf 5
|
||||
.Sh AUTHORS
|
||||
.An Firstname Lastname Aq Mt flastname@example.com
|
||||
....
|
||||
|
||||
[[manual-pages-sample-structures-section-4]]
|
||||
=== Section 4 Device Driver
|
||||
|
||||
The preferred basic structure for a section 4 device driver:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
.Dd August 25, 2017
|
||||
.Dt EXAMPLEDRIVER 4
|
||||
.Os
|
||||
.Sh NAME
|
||||
.Nm exampledriver
|
||||
.Nd "driver to demonstrate section 4 man pages"
|
||||
.Sh SYNOPSIS
|
||||
To compile this driver into the kernel, add this line to the
|
||||
kernel configuration file:
|
||||
.Bd -ragged -offset indent
|
||||
.Cd "device exampledriver"
|
||||
.Ed
|
||||
.Pp
|
||||
To load the driver as a module at boot, add this line to
|
||||
.Xr loader.conf 5 :
|
||||
.Bd -literal -offset indent
|
||||
exampledriver_load="YES"
|
||||
.Ed
|
||||
.Sh DESCRIPTION
|
||||
The
|
||||
.Nm
|
||||
driver provides an opportunity to show a skeleton or template
|
||||
file for section 4 manual pages.
|
||||
.Sh HARDWARE
|
||||
The
|
||||
.Nm
|
||||
driver supports these cards from the aptly-named Nonexistent
|
||||
Technologies:
|
||||
.Pp
|
||||
.Bl -bullet -compact
|
||||
.It
|
||||
NT X149.2 (single and dual port)
|
||||
.It
|
||||
NT X149.8 (single port)
|
||||
.El
|
||||
.Sh DIAGNOSTICS
|
||||
.Bl -diag
|
||||
.It "flashing green light"
|
||||
Something bad happened.
|
||||
.It "flashing red light"
|
||||
Something really bad happened.
|
||||
.It "solid black light"
|
||||
Power cord is unplugged.
|
||||
.El
|
||||
.Sh SEE ALSO
|
||||
.Xr example 8
|
||||
.Sh HISTORY
|
||||
The
|
||||
.Nm
|
||||
device driver first appeared in
|
||||
.Fx 49.2 .
|
||||
.Sh AUTHORS
|
||||
.An Firstname Lastname Aq Mt flastname@example.com
|
||||
....
|
||||
|
||||
[[manual-pages-sample-structures-section-5]]
|
||||
=== Section 5 Configuration File
|
||||
|
||||
The preferred basic structure for a section 5 configuration file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
.Dd August 25, 2017
|
||||
.Dt EXAMPLECONF 5
|
||||
.Os
|
||||
.Sh NAME
|
||||
.Nm example.conf
|
||||
.Nd "config file to demonstrate section 5 man pages"
|
||||
.Sh DESCRIPTION
|
||||
.Nm
|
||||
is an example configuration file.
|
||||
.Sh SEE ALSO
|
||||
.Xr example 8
|
||||
.Sh AUTHORS
|
||||
.An Firstname Lastname Aq Mt flastname@example.com
|
||||
....
|
||||
|
||||
[[manual-pages-testing]]
|
||||
== Testing
|
||||
|
||||
Testing a new manual page can be challenging. Fortunately there are some tools that can assist in the task. Some of them, like man:man[1], do not look in the current directory. It is a good idea to prefix the filename with `./` if the new manual page is in the current directory. An absolute path can also be used.
|
||||
|
||||
Use man:mandoc[1]'s linter to check for parsing errors:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% mandoc -T lint ./mynewmanpage.8
|
||||
....
|
||||
|
||||
Use package:textproc/igor[] to proofread the manual page:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% igor ./mynewmanpage.8
|
||||
....
|
||||
|
||||
Use man:man[1] to check the final result of your changes:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% man ./mynewmanpage.8
|
||||
....
|
||||
|
||||
You can use man:col[1] to filter the output of man:man[1] and get rid of the backspace characters before loading the result in your favorite editor for spell checking:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% man ./mynewmanpage.8 | col -b | vim -R -
|
||||
....
|
||||
|
||||
Spell-checking with fully-featured dictionaries is encouraged, and can be accomplished by using package:textproc/hunspell[] or package:textproc/aspell[] combined with package:textproc/en-hunspell[] or package:textproc/en-aspell[], respectively. For instance:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% aspell check --lang=en --mode=nroff ./mynewmanpage.8
|
||||
....
|
||||
|
||||
[[manual-pages-examples-as-templates]]
|
||||
== Example Manual Pages to Use as Templates
|
||||
|
||||
Some manual pages are suitable as in-depth examples.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Manual Page
|
||||
| Path to Source Location
|
||||
|
||||
|
||||
|man:cp[1]
|
||||
|[.filename]#/usr/src/bin/cp/cp.1#
|
||||
|
||||
|man:vt[4]
|
||||
|[.filename]#/usr/src/shared/man/man4/vt.4#
|
||||
|
||||
|man:crontab[5]
|
||||
|[.filename]#/usr/src/usr.sbin/cron/crontab/crontab.5#
|
||||
|
||||
|man:gpart[8]
|
||||
|[.filename]#/usr/src/sbin/geom/class/part/gpart.8#
|
||||
|===
|
||||
|
||||
[[manual-pages-resources]]
|
||||
== Resources
|
||||
|
||||
Resources for manual page writers:
|
||||
|
||||
* man:man[1]
|
||||
* man:mandoc[1]
|
||||
* man:groff_mdoc[7]
|
||||
* http://manpages.bsd.lv/mdoc.html[Practical UNIX Manuals: mdoc]
|
||||
* http://manpages.bsd.lv/history.html[History of UNIX Manpages]
|
||||
123
documentation/content/en/books/fdp-primer/overview/chapter.adoc
Normal file
123
documentation/content/en/books/fdp-primer/overview/chapter.adoc
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
---
|
||||
title: Chapter 1. Overview
|
||||
prev: books/fdp-primer/preface
|
||||
next: books/fdp-primer/tools
|
||||
---
|
||||
|
||||
[[overview]]
|
||||
= Overview
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 1
|
||||
|
||||
toc::[]
|
||||
|
||||
Welcome to the FreeBSD Documentation Project (FDP). Quality documentation is crucial to the success of FreeBSD, and we value your contributions very highly.
|
||||
|
||||
This document describes how the FDP is organized, how to write and submit documentation, and how to effectively use the available tools.
|
||||
|
||||
Everyone is welcome to contribute to the FDP. Willingness to contribute is the only membership requirement.
|
||||
|
||||
This primer shows how to:
|
||||
|
||||
* Identify which parts of FreeBSD are maintained by the FDP.
|
||||
* Install the required documentation tools and files.
|
||||
* Make changes to the documentation.
|
||||
* Submit changes back for review and inclusion in the FreeBSD documentation.
|
||||
|
||||
[[overview-quick-start]]
|
||||
== Quick Start
|
||||
|
||||
Some preparatory steps must be taken before editing the FreeBSD documentation. First, subscribe to the {freebsd-doc}. Some team members also interact on the `#bsddocs` IRC channel on http://www.efnet.org/[EFnet]. These people can help with questions or problems involving the documentation.
|
||||
|
||||
[.procedure]
|
||||
====
|
||||
. Install these packages. These packages are all of the software needed to edit and build FreeBSD documentation. The Git package is needed to obtain a working copy of the documentation and generate patches with.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install gohugo python3 git-lite rubygem-asciidoctor rubygem-rouge
|
||||
....
|
||||
+
|
||||
. Optional: to generate PDF documentation install `asciidoctor-pdf`
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install rubygem-asciidoctor-pdf
|
||||
....
|
||||
+
|
||||
. Install a local working copy of the documentation from the FreeBSD repository in [.filename]#~/doc# (see <<working-copy>>).
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% git clone https://git.FreeBSD.org/doc.git doc
|
||||
....
|
||||
+
|
||||
. Copy the content of [.filename]#~/doc/shared# to [.filename]#~/doc/documentation# and [.filename]#~/doc/website#.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
make move-shared
|
||||
....
|
||||
+
|
||||
. Configure the text editor:
|
||||
|
||||
** Tab stops set to 2.
|
||||
** Replace each group of 8 leading spaces with a single tab.
|
||||
+
|
||||
Specific editor configurations are listed in <<editor-config>>.
|
||||
+
|
||||
. Edit the documentation files that require changes. If a file needs major changes, consult the mailing list for input.
|
||||
+
|
||||
Review the output and edit the file to fix any problems shown, then rerun the command to find any remaining problems. Repeat until all of the errors are resolved.
|
||||
+
|
||||
. _Always_ build-test changes before submitting them. Running `make` in the top-level directory of the documentation will generate that documentation in HTML format.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
make generate
|
||||
....
|
||||
+
|
||||
. When changes are complete and tested, generate a "diff file":
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc
|
||||
% git diff > bsdinstall.diff.txt
|
||||
....
|
||||
+
|
||||
Give the diff file a descriptive name. In the example above, changes have been made to the [.filename]#bsdinstall# portion of the Handbook.
|
||||
. Submit the diff file using the web-based https://bugs.FreeBSD.org/bugzilla/enter_bug.cgi?product=Documentation[Problem Report] system. If using the web form, enter a Summary of _[patch] short description of problem_. Select the Component `Documentation`. In the Description field, enter a short description of the changes and any important details about them. Use the btn:[Add an attachment] button to attach the diff file. Finally, use the btn:[Submit Bug] button to submit your diff to the problem report system.
|
||||
====
|
||||
|
||||
[[overview-doc]]
|
||||
== The FreeBSD Documentation Set
|
||||
|
||||
The FDP is responsible for four categories of FreeBSD documentation.
|
||||
|
||||
* _Handbook_: The Handbook is the comprehensive online resource and reference for FreeBSD users.
|
||||
* _FAQ_: The FAQ uses a short question and answer format to address questions that are frequently asked on the various mailing lists and forums devoted to FreeBSD. This format does not permit long and comprehensive answers.
|
||||
* _Manual pages_: The English language system manual pages are usually not written by the FDP, as they are part of the base system. However, the FDP can reword parts of existing manual pages to make them clearer or to correct inaccuracies.
|
||||
* _Web site_: This is the main FreeBSD presence on the web, visible at https://www.freebsd.org/[https://www.FreeBSD.org/] and many mirrors around the world. The web site is typically a new user's first exposure to FreeBSD.
|
||||
|
||||
Translation teams are responsible for translating the Handbook and web site into different languages. Manual pages are not translated at present.
|
||||
|
||||
Documentation source for the FreeBSD web site, Handbook, and FAQ is available in the documentation repository at `https://cgit.freebsd.org/doc/`.
|
||||
|
||||
Source for manual pages is available in a separate source repository located at `https://cgit.freebsd.org/src/`.
|
||||
|
||||
Documentation commit messages are visible with `git log`. Commit messages are also archived at link:{git-doc-all}.
|
||||
|
||||
Web frontends to both of these repositories are available at https://cgit.freebsd.org/doc/[] and https://cgit.freebsd.org/src/[].
|
||||
|
||||
Many people have written tutorials or how-to articles about FreeBSD. Some are stored as part of the FDP files. In other cases, the author has decided to keep the documentation separate. The FDP endeavors to provide links to as much of this external documentation as possible.
|
||||
|
|
@ -0,0 +1,386 @@
|
|||
---
|
||||
title: Chapter 9. PO Translations
|
||||
prev: books/fdp-primer/translations
|
||||
next: books/fdp-primer/manual-pages
|
||||
---
|
||||
|
||||
[[po-translations]]
|
||||
= PO Translations
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 9
|
||||
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[po-translations-introduction]]
|
||||
== Introduction
|
||||
|
||||
The http://www.gnu.org/software/gettext/[GNU gettext] system offers translators an easy way to create and maintain translations of documents. Translatable strings are extracted from the original document into a PO (Portable Object) file. Translated versions of the strings are entered with a separate editor. The strings can be used directly or built into a complete translated version of the original document.
|
||||
|
||||
[[po-translations-quick-start]]
|
||||
== Quick Start
|
||||
|
||||
The procedure shown in <<overview-quick-start>> is assumed to have already been performed. The `TRANSLATOR` option is required and already enabled by default in the package:textproc/docproj[] port.
|
||||
|
||||
This example shows the creation of a Spanish translation of the short link:{leap-seconds}[Leap Seconds] article.
|
||||
|
||||
[[po-translations-quick-start-install-po-editor]]
|
||||
|
||||
[.procedure]
|
||||
====
|
||||
.Procedure: Install a PO Editor
|
||||
. A PO editor is needed to edit translation files. This example uses package:editors/poedit[].
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/editors/poedit
|
||||
# make install clean
|
||||
....
|
||||
====
|
||||
|
||||
[[po-translations-quick-start-initial-setup]]
|
||||
[.procedure]
|
||||
====
|
||||
.Procedure: Initial Setup
|
||||
|
||||
When a new translation is first created, the directory structure must be created or copied from the English original:
|
||||
|
||||
. Create a directory for the new translation. The English article source is in [.filename]#~/doc/en/articles/leap-seconds/#. The Spanish translation will go in [.filename]#~/doc/es/articles/leap-seconds/#. The path is the same except for the name of the language directory.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% mkdir ~/doc/es/articles/leap-seconds/
|
||||
....
|
||||
|
||||
. Copy the [.filename]#_index.adoc# from the original document into the translation directory:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cp ~/doc/en/articles/leap-seconds/_index.adoc \
|
||||
~/doc/es/articles/leap-seconds/
|
||||
....
|
||||
====
|
||||
|
||||
[[po-translations-quick-start-translation]]
|
||||
[.procedure]
|
||||
====
|
||||
.Procedure: Translation
|
||||
|
||||
Translating a document consists of two steps: extracting translatable strings from the original document, and entering translations for those strings. These steps are repeated until the translator feels that enough of the document has been translated to produce a usable translated document.
|
||||
|
||||
. Extract the translatable strings from the original English version into a PO file:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/es/articles/leap-seconds/
|
||||
% po4a-gettextize --format asciidoc --master _index.adoc --master-charset "UTF-8" >> es.pot
|
||||
....
|
||||
+
|
||||
. Use a PO editor to enter translations in the PO file. There are several different editors available. [.filename]#poedit# from package:editors/poedit[] is shown here.
|
||||
+
|
||||
The PO file name is the language region code. For Spanish, the file name is [.filename]#es.po#.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% poedit es.po
|
||||
....
|
||||
====
|
||||
|
||||
[[po-translations-quick-generating-a-translated-document]]
|
||||
[.procedure]
|
||||
====
|
||||
.Procedure: Generating a Translated Document
|
||||
. Generate the translated document:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/es/articles/leap-seconds/
|
||||
% po4a-translate -f asciidoc -m document.po -l document.po --keep 0 -p es.po -M UTF-8
|
||||
....
|
||||
+
|
||||
The name of the generated document matches the name of the English original, usually [.filename]#_index.adoc#.
|
||||
+
|
||||
. Check the generated file by rendering it to HTML and viewing it with a web browser:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% make
|
||||
....
|
||||
====
|
||||
|
||||
[[po-translations-creating]]
|
||||
== Creating New Translations
|
||||
|
||||
The first step to creating a new translated document is locating or creating a directory to hold it. FreeBSD puts translated documents in a subdirectory named for their language and region in the format [.filename]#lang#. _lang_ is a two-character lowercase code.
|
||||
|
||||
[[po-translations-language-names]]
|
||||
.Language Names
|
||||
[cols="1,1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Language
|
||||
| Region
|
||||
| Translated Directory Name
|
||||
| PO File Name
|
||||
|
||||
|English
|
||||
|United States
|
||||
|[.filename]#en#
|
||||
|[.filename]#en.po#
|
||||
|
||||
|Bengali
|
||||
|Bangladesh
|
||||
|[.filename]#bn#
|
||||
|[.filename]#bn.po#
|
||||
|
||||
|Danish
|
||||
|Denmark
|
||||
|[.filename]#da#
|
||||
|[.filename]#da.po#
|
||||
|
||||
|German
|
||||
|Germany
|
||||
|[.filename]#de#
|
||||
|[.filename]#de.po#
|
||||
|
||||
|Greek
|
||||
|Greece
|
||||
|[.filename]#el#
|
||||
|[.filename]#el.po#
|
||||
|
||||
|Spanish
|
||||
|Spain
|
||||
|[.filename]#es#
|
||||
|[.filename]#es.po#
|
||||
|
||||
|French
|
||||
|France
|
||||
|[.filename]#fr#
|
||||
|[.filename]#fr.po#
|
||||
|
||||
|Hungarian
|
||||
|Hungary
|
||||
|[.filename]#hu#
|
||||
|[.filename]#hu.po#
|
||||
|
||||
|Italian
|
||||
|Italy
|
||||
|[.filename]#it#
|
||||
|[.filename]#it.po#
|
||||
|
||||
|Japanese
|
||||
|Japan
|
||||
|[.filename]#ja#
|
||||
|[.filename]#ja.po#
|
||||
|
||||
|Korean
|
||||
|Korea
|
||||
|[.filename]#ko#
|
||||
|[.filename]#ko.po#
|
||||
|
||||
|Mongolian
|
||||
|Mongolia
|
||||
|[.filename]#mn#
|
||||
|[.filename]#mn.po#
|
||||
|
||||
|Dutch
|
||||
|Netherlands
|
||||
|[.filename]#nl#
|
||||
|[.filename]#nl.po#
|
||||
|
||||
|Polish
|
||||
|Poland
|
||||
|[.filename]#pl#
|
||||
|[.filename]#pl.po#
|
||||
|
||||
|Portuguese
|
||||
|Brazil
|
||||
|[.filename]#pt_BR#
|
||||
|[.filename]#pt_BR.po#
|
||||
|
||||
|Russian
|
||||
|Russia
|
||||
|[.filename]#ru#
|
||||
|[.filename]#ru.po#
|
||||
|
||||
|Turkish
|
||||
|Turkey
|
||||
|[.filename]#tr#
|
||||
|[.filename]#tr.po#
|
||||
|
||||
|Chinese
|
||||
|China
|
||||
|[.filename]#zh_CN#
|
||||
|[.filename]#zh_CN.po#
|
||||
|
||||
|Chinese
|
||||
|Taiwan
|
||||
|[.filename]#zh_TW#
|
||||
|[.filename]#zh_TW.po#
|
||||
|===
|
||||
|
||||
The translations are in subdirectories of the main documentation directory, here assumed to be [.filename]#~/doc/# as shown in <<overview-quick-start>>. For example, German translations are located in [.filename]#~/doc/de/#, and French translations are in [.filename]#~/doc/fr/#.
|
||||
|
||||
Each language directory contains separate subdirectories named for the type of documents, usually [.filename]#articles/# and [.filename]#books/#.
|
||||
|
||||
Combining these directory names gives the complete path to an article or book. For example, the French translation of the NanoBSD article is in [.filename]#~/doc/fr/articles/nanobsd/#, and the Mongolian translation of the Handbook is in [.filename]#~/doc/mn/books/handbook/#.
|
||||
|
||||
A new language directory must be created when translating a document to a new language. If the language directory already exists, only a subdirectory in the [.filename]#articles/# or [.filename]#books/# directory is needed.
|
||||
|
||||
[[po-translations-creating-example]]
|
||||
.Creating a Spanish Translation of the Porter's Handbook
|
||||
[example]
|
||||
====
|
||||
Create a new Spanish translation of the link:{porters-handbook}[Porter's Handbook]. The original is a book in [.filename]#~/doc/en/books/porters-handbook/#.
|
||||
|
||||
[.procedure]
|
||||
======
|
||||
|
||||
. The Spanish language books directory [.filename]#~/doc/es/books/# already exists, so only a new subdirectory for the Porter's Handbook is needed:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/es/books/
|
||||
% mkdir porters-handbook
|
||||
....
|
||||
|
||||
. Copy the content from the original book:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/es/books/porters-handbook
|
||||
% cp ~/doc/en/books/porters-handbook/ .
|
||||
....
|
||||
+
|
||||
Now the document structure is ready for the translator to begin translating with `po4a` command.
|
||||
======
|
||||
====
|
||||
|
||||
[[po-translations-translating]]
|
||||
== Translating
|
||||
|
||||
The gettext system greatly reduces the number of things that must be tracked by a translator. Strings to be translated are extracted from the original document into a PO file. Then a PO editor is used to enter the translated versions of each string.
|
||||
|
||||
The FreeBSD PO translation system does not overwrite PO files, so the extraction step can be run at any time to update the PO file.
|
||||
|
||||
A PO editor is used to edit the file. package:editors/poedit[] is shown in these examples because it is simple and has minimal requirements. Other PO editors offer features to make the job of translating easier. The Ports Collection offers several of these editors, including package:devel/gtranslator[].
|
||||
|
||||
It is important to preserve the PO file. It contains all of the work that translators have done.
|
||||
|
||||
[[po-translations-translating-example]]
|
||||
.Translating the Porter's Handbook to Spanish
|
||||
[example]
|
||||
====
|
||||
Enter Spanish translations of the contents of the Porter's Handbook.
|
||||
|
||||
[.procedure]
|
||||
======
|
||||
. Change to the Spanish Porter's Handbook directory and update the PO file. The generated PO file is called [.filename]#es_ES.po# as shown in <<po-translations-language-names>>.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc/es/books/porters-handbook
|
||||
% po4a-gettextize --format asciidoc --master _index.adoc --master-charset "UTF-8" >> es.pot
|
||||
....
|
||||
|
||||
. Enter translations using a PO editor:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% poedit es.pot
|
||||
....
|
||||
======
|
||||
|
||||
====
|
||||
|
||||
[[po-translations-tips]]
|
||||
== Tips for Translators
|
||||
|
||||
[[po-translations-tips-xmltags]]
|
||||
=== Preserving AsciiDoc macros
|
||||
|
||||
Preserve AsciiDoc macros that are shown in the English original.
|
||||
|
||||
.Preserving AsciiDoc macros
|
||||
[example]
|
||||
====
|
||||
English original:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
msgid ""
|
||||
"This example shows the creation of a Spanish translation of the short "
|
||||
"link:{leap-seconds}[Leap Seconds] article."
|
||||
....
|
||||
|
||||
Spanish translation:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
msgid ""
|
||||
"Este ejemplo muestra la creación de un artículo con poco contenido como el artículo "
|
||||
"link:{leap-seconds}[Leap Seconds]."
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
[[po-translations-tips-spaces]]
|
||||
=== Preserving Spaces
|
||||
|
||||
Preserve existing spaces at the beginning and end of strings to be translated. The translated version must have these spaces also.
|
||||
|
||||
[[po-translations-tips-verbatim]]
|
||||
=== Verbatim Tags
|
||||
|
||||
The contents of some tags should be copied verbatim, not translated:
|
||||
|
||||
* `man:man[1]`
|
||||
* `package:package[]`
|
||||
* `link`
|
||||
* `image`
|
||||
* `include`
|
||||
* `Admonitions`
|
||||
* `id's`
|
||||
* `Heading tags`
|
||||
* `source`
|
||||
|
||||
[[po-translations-building]]
|
||||
== Building a Translated Document
|
||||
|
||||
A translated version of the original document can be created at any time. Any untranslated portions of the original will be included in English in the resulting document. Most PO editors have an indicator that shows how much of the translation has been completed. This makes it easy for the translator to see when enough strings have been translated to make building the final document worthwhile.
|
||||
|
||||
[[po-translations-submitting]]
|
||||
== Submitting the New Translation
|
||||
|
||||
Prepare the new translation files for submission. This includes adding the files to the version control system, setting additional properties on them, then creating a diff for submission.
|
||||
|
||||
The diff files created by these examples can be attached to a https://bugs.freebsd.org/bugzilla/enter_bug.cgi?product=Documentation[documentation bug report] or https://reviews.freebsd.org/[code review].
|
||||
|
||||
[[po-translations-submitting-spanish]]
|
||||
.Spanish Translation of the NanoBSD Article
|
||||
[example]
|
||||
====
|
||||
[.procedure]
|
||||
======
|
||||
|
||||
. Create a diff of the new files from the [.filename]#~/doc/# base directory so the full path is shown with the filenames. This helps committers identify the target language directory.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc
|
||||
% git diff es/articles/nanobsd/ > /tmp/es_nanobsd.diff
|
||||
....
|
||||
======
|
||||
|
||||
====
|
||||
128
documentation/content/en/books/fdp-primer/preface/chapter.adoc
Normal file
128
documentation/content/en/books/fdp-primer/preface/chapter.adoc
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
---
|
||||
title: Preface
|
||||
prev: books/fdp-primer/
|
||||
next: books/fdp-primer/overview
|
||||
---
|
||||
|
||||
[preface]
|
||||
[[preface]]
|
||||
= Preface
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
|
||||
[[preface-prompts]]
|
||||
== Shell Prompts
|
||||
|
||||
This table shows the default system prompt and superuser prompt. The examples use these prompts to indicate which type of user is running the example.
|
||||
|
||||
[.informaltable]
|
||||
[cols="50%,50%", frame="none", options="header"]
|
||||
|===
|
||||
| User
|
||||
| Prompt
|
||||
|
||||
|
||||
|Normal user
|
||||
|%
|
||||
|
||||
|`root`
|
||||
|#
|
||||
|===
|
||||
|
||||
[[preface-conventions]]
|
||||
== Typographic Conventions
|
||||
|
||||
This table describes the typographic conventions used in this book.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Meaning
|
||||
| Examples
|
||||
|
||||
|
||||
|The names of commands.
|
||||
|Use `ls -l` to list all files.
|
||||
|
||||
|The names of files.
|
||||
|Edit [.filename]#.login#.
|
||||
|
||||
|On-screen computer output.
|
||||
a|
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
You have mail.
|
||||
....
|
||||
|
||||
|
||||
|What the user types, contrasted with on-screen computer output.
|
||||
a|
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% date +"The time is %H:%M"
|
||||
The time is 09:18
|
||||
....
|
||||
|
||||
|Manual page references.
|
||||
|Use man:su[1] to change user identity.
|
||||
|
||||
|User and group names.
|
||||
|Only `root` can do this.
|
||||
|
||||
|Emphasis.
|
||||
|The user _must_ do this.
|
||||
|
||||
|Text that the user is expected to replace with the actual text.
|
||||
|To search for a keyword in the manual pages, type `man -k _keyword_`
|
||||
|
||||
|Environment variables.
|
||||
|`$HOME` is set to the user's home directory.
|
||||
|===
|
||||
|
||||
[[preface-notes]]
|
||||
== Notes, Tips, Important Information, Warnings, and Examples
|
||||
|
||||
Notes, warnings, and examples appear within the text.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Notes are represented like this, and contain information to take note of, as it may affect what the user does.
|
||||
====
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
Tips are represented like this, and contain information helpful to the user, like showing an easier way to do something.
|
||||
====
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Important information is represented like this. Typically, these show extra steps the user may need to take.
|
||||
====
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
Warnings are represented like this, and contain information warning about possible damage if the instructions are not followed. This damage may be physical, to the hardware or the user, or it may be non-physical, such as the inadvertent deletion of important files.
|
||||
====
|
||||
|
||||
.A Sample Example
|
||||
[example]
|
||||
====
|
||||
Examples are represented like this, and typically contain examples showing a walkthrough, or the results of a particular action.
|
||||
====
|
||||
|
||||
[[preface-acknowledgements]]
|
||||
== Acknowledgments
|
||||
|
||||
My thanks to Sue Blake, Patrick Durusau, Jon Hamilton, Peter Flynn, and Christopher Maden, who took the time to read early drafts of this document and offer many valuable comments and criticisms.
|
||||
283
documentation/content/en/books/fdp-primer/rosetta/chapter.adoc
Normal file
283
documentation/content/en/books/fdp-primer/rosetta/chapter.adoc
Normal file
|
|
@ -0,0 +1,283 @@
|
|||
---
|
||||
title: Chapter 7. Rosetta Stone
|
||||
prev: books/fdp-primer/asciidoctor-primer
|
||||
next: books/fdp-primer/translations
|
||||
---
|
||||
|
||||
[[rosetta]]
|
||||
= Rosetta Stone
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 7
|
||||
|
||||
toc::[]
|
||||
|
||||
[[docbook-vs-asciidoc]]
|
||||
== Comparision between Docbook and AsciiDoc
|
||||
|
||||
This rosetta stone tries to show the differences between Docbook and AsciiDoc.
|
||||
|
||||
.Comparision between Docbook and AsciiDoc
|
||||
[cols="1,1,1"]
|
||||
|===
|
||||
|Language Feature |Docbook | AsciiDoc
|
||||
|
||||
|*Bold*
|
||||
|<keycap>bold</keycap>
|
||||
|\*bold*
|
||||
|
||||
|*Italic*
|
||||
|<emphasis>Italic</emphasis>
|
||||
|\_Italic_
|
||||
|
||||
|*Monospace*
|
||||
|<literal>Monospace</literal>
|
||||
|\`Monospace`
|
||||
|
||||
|*Paragraph*
|
||||
|<para>This is a paragraph</para>
|
||||
|This is a paragraph
|
||||
|
||||
|*Keycap*
|
||||
|<keycap>F11</keycap>
|
||||
|\kbd:[F11]
|
||||
|
||||
|*Links*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<link xlink:href="https://www.freebsd.org/where/">Download FreeBSD</link>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
link:https://www.freebsd.org/where/[Download FreeBSD]
|
||||
----
|
||||
|
||||
|*Sections*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<sect1 xml:id="id">
|
||||
<title>Section 1</title>
|
||||
</sect1>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
[[id]]
|
||||
= Section 1
|
||||
----
|
||||
|
||||
|*Unordered list*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<itemizedlist>
|
||||
<listitem>
|
||||
<para>When to build a custom kernel.</para>
|
||||
</listitem>
|
||||
|
||||
<listitem>
|
||||
<para>How to take a hardware inventory.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
* When to build a custom kernel.
|
||||
* How to take a hardware inventory.
|
||||
----
|
||||
|
||||
|*Ordered list*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<orderedlist>
|
||||
<listitem>
|
||||
<para>One</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Two</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Three</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Four</para>
|
||||
</listitem>
|
||||
</orderedlist>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
. One
|
||||
. Two
|
||||
. Three
|
||||
. Four
|
||||
----
|
||||
|
||||
|*Variable list*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term>amd64</term>
|
||||
<listitem>
|
||||
<para>This is the most common desktop...</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
amd64::
|
||||
This is the most common desktop...
|
||||
----
|
||||
|
||||
|*Source code*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<screen>
|
||||
&prompt.root; <userinput>mkdir -p /var/</userinput>
|
||||
</screen>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
....
|
||||
[source,bash]
|
||||
----
|
||||
# mkdir -p /var/spool/lpd/lp
|
||||
----
|
||||
....
|
||||
|
||||
|*Literal block*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<programlisting>
|
||||
include GENERIC
|
||||
ident MYKERNEL
|
||||
|
||||
options IPFIREWALL
|
||||
options DUMMYNET
|
||||
options IPFIREWALL_DEFAULT_TO_ACCEPT
|
||||
options IPDIVERT
|
||||
</programlisting>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
....
|
||||
include GENERIC
|
||||
ident MYKERNEL
|
||||
|
||||
options IPFIREWALL
|
||||
options DUMMYNET
|
||||
options IPFIREWALL_DEFAULT_TO_ACCEPT
|
||||
options IPDIVERT
|
||||
....
|
||||
----
|
||||
|
||||
|*Images*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<figure xml:id="bsdinstall-newboot-loader-menu">
|
||||
<title>FreeBSD Boot Loader Menu</title>
|
||||
|
||||
<mediaobject>
|
||||
<imageobject>
|
||||
<imagedata fileref="bsdinstall/bsdinstall-newboot-loader-menu"/>
|
||||
</imageobject>
|
||||
</mediaobject>
|
||||
</figure>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
[[bsdinstall-newboot-loader-menu]]
|
||||
.FreeBSD Boot Loader Menu
|
||||
image::bsdinstall/bsdinstall-newboot-loader-menu
|
||||
----
|
||||
|
||||
|*Includes*
|
||||
|n/a
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
\include::chapter.adoc[]
|
||||
----
|
||||
|
||||
|*Tables*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<table xml:id="partition-schemes" frame="none" rowsep="1" pgwide="1">
|
||||
<title>Partitioning Schemes</title>
|
||||
|
||||
<tgroup cols="2" align="left">
|
||||
<thead>
|
||||
<row>
|
||||
<entry align="left">Abbreviation</entry>
|
||||
<entry align="left">Description</entry>
|
||||
</row>
|
||||
</thead>
|
||||
|
||||
<tbody>
|
||||
<row>
|
||||
<entry>APM</entry>
|
||||
<entry>Apple Partition Map, used by PowerPC(R).</entry>
|
||||
</row>
|
||||
</tbody>
|
||||
</tgroup>
|
||||
</table>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
[[partition-schemes]]
|
||||
.Partitioning Schemes
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
\|===
|
||||
\| Abbreviation
|
||||
\| Description
|
||||
|
||||
\|APM
|
||||
\|Apple Partition Map, used by PowerPC(R).
|
||||
|
||||
\|===
|
||||
----
|
||||
|
||||
|*Admonitions*
|
||||
a|
|
||||
[source,xml]
|
||||
----
|
||||
<tip>
|
||||
<para>This is a tip</para>
|
||||
</tip>
|
||||
----
|
||||
a|
|
||||
[source]
|
||||
----
|
||||
[TIP]
|
||||
====
|
||||
This is a tip
|
||||
====
|
||||
----
|
||||
|
||||
|===
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
title: Chapter 13. See Also
|
||||
prev: books/fdp-primer/editor-config/
|
||||
next: books/fdp-primer/examples
|
||||
---
|
||||
|
||||
[[see-also]]
|
||||
= See Also
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 13
|
||||
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
This document is deliberately not an exhaustive discussion of AsciiDoc and the FreeBSD Documentation Project. For more information about these, you are encouraged to see the following web sites.
|
||||
|
||||
[[see-also-fdp]]
|
||||
== The FreeBSD Documentation Project
|
||||
|
||||
* link:https://www.FreeBSD.org/docproj/[The FreeBSD Documentation Project web pages]
|
||||
* link:{handbook}[The FreeBSD Handbook]
|
||||
|
||||
[[see-also-asciidoc]]
|
||||
== AsciiDoctor
|
||||
|
||||
* link:https://asciidoctor.org/[AsciiDoctor]
|
||||
|
||||
[[see-also-html]]
|
||||
== HTML
|
||||
|
||||
* link:http://www.w3.org/[The World Wide Web Consortium]
|
||||
* link:https://dev.w3.org/html5/spec-LC/[The HTML 5 specification]
|
||||
* link:https://www.w3.org/Style/CSS/specs.en.html[CSS specification]
|
||||
217
documentation/content/en/books/fdp-primer/structure/chapter.adoc
Normal file
217
documentation/content/en/books/fdp-primer/structure/chapter.adoc
Normal file
|
|
@ -0,0 +1,217 @@
|
|||
---
|
||||
title: Chapter 4. Documentation Directory Structure
|
||||
prev: books/fdp-primer/working-copy
|
||||
next: books/fdp-primer/doc-build
|
||||
---
|
||||
|
||||
[[structure]]
|
||||
= Documentation Directory Structure
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 4
|
||||
|
||||
toc::[]
|
||||
|
||||
Files and directories in the [.filename]#doc/# tree follow a structure meant to:
|
||||
|
||||
. Make it easy to automate converting the document to other formats.
|
||||
. Promote consistency between the different documentation organizations, to make it easier to switch between working on different documents.
|
||||
. Make it easy to decide where in the tree new documentation should be placed.
|
||||
|
||||
In addition, the documentation tree must accommodate documents in many different languages. It is important that the documentation tree structure does not enforce any particular defaults or cultural preferences.
|
||||
|
||||
[[structure-top]]
|
||||
== The Top Level, doc/
|
||||
|
||||
There are three sections under [.filename]#doc/#, documentation and website share the same structure.
|
||||
|
||||
[cols="20%,80%", frame="none", options="header"]
|
||||
|===
|
||||
| Directory
|
||||
| Usage
|
||||
|
||||
|[.filename]#documentation#
|
||||
|Contains all the articles and books in AsciiDoc format. Contains subdirectories to further categorize the information by languages
|
||||
|
||||
|[.filename]#shared#
|
||||
|Contains files that are not specific to the various translations of the documentation. Contains subdirectories to further categorize the information by languages and three files to store the authors, releases and mirrors information. This directory is shared between `documentation` and the `website`.
|
||||
|
||||
|[.filename]#website#
|
||||
|Contains the link:https://www.FreeBSD.org[FreeBSD website] in AsciiDoc format. Contains subdirectories to further categorize the information by languages.
|
||||
|===
|
||||
|
||||
[[structure-locale]]
|
||||
== The Directories
|
||||
|
||||
These directories contain the documentation and the website. The documentation is split into up to some directories this level. Following the link:https://gohugo.io/getting-started/directory-structure/[Hugo directory structure].
|
||||
|
||||
[cols="20%,80%", frame="none", options="header"]
|
||||
|===
|
||||
| Directory
|
||||
| Usage
|
||||
|
||||
|
||||
|[.filename]#archetypes#
|
||||
|Contain templates to create new articles, books and webpages. For more information take a look link:https://gohugo.io/content-management/archetypes/[here].
|
||||
|
||||
|[.filename]#config#
|
||||
|Contain the Hugo configuration files. One main file and one file per language. For more information take a look link:https://gohugo.io/getting-started/configuration/[here].
|
||||
|
||||
|[.filename]#content#
|
||||
|Contain the books, articles and webpages. One directory exists for each available translation of the documentation, for example `en` and `zh_TW`.
|
||||
|
||||
| [.filename]#data#
|
||||
| Contain custom data for build the website in link:https://en.wikipedia.org/wiki/TOML[TOML] format. This directory is used to store the events, news, press, etc. For more information take a look link:https://gohugo.io/templates/data-templates/[here].
|
||||
|
||||
| [.filename]#static#
|
||||
| Contain static assets. Images, security advisories, the pgpkeys, etc. For more information take a look link:https://gohugo.io/content-management/static-files/[here].
|
||||
|
||||
| [.filename]#themes#
|
||||
| Contain the templates in the form of `.html` files that specify how the website looks. For more information take a look link:https://gohugo.io/templates/[here].
|
||||
|
||||
| [.filename]#tools#
|
||||
| Contain tools used to enhance the documentation build. For example to generate the Table of Contents of the books, etc.
|
||||
|
||||
| [.filename]#beastie.png#
|
||||
| This image don't need an introduction ;)
|
||||
|
||||
| [.filename]#LICENSE#
|
||||
| License of the documentation, shared and website. BSD 2-Clause License.
|
||||
|
||||
| [.filename]#Makefile#
|
||||
| The [.filename]#Makefile# defines the build process of the documentation and the website.
|
||||
|===
|
||||
|
||||
[[structure-document]]
|
||||
== Document-Specific Information
|
||||
|
||||
This section contains specific notes about particular documents managed by the FDP.
|
||||
|
||||
[[structure-document-books]]
|
||||
== The Books: books/
|
||||
|
||||
The books are written in AsciiDoc.
|
||||
|
||||
The books are organized as an AsciiDoc `book`. The books are divided into ``part``s, each of which contains several ``chapter``s. ``chapter``s are further subdivided into sections (`=`) and subsections (`==`, `===`) and so on.
|
||||
|
||||
[[structure-document-books-physical]]
|
||||
=== Physical Organization
|
||||
|
||||
There are a number of files and directories within the books directory, all with the same structure.
|
||||
|
||||
[[structure-document-books-physical-index]]
|
||||
==== _index.adoc
|
||||
|
||||
The [.filename]#_index.adoc# defines some AsciiDoc variables that affect how the AsciiDoc source is converted to other formats and list the Table of Contents, Table of Examples, Table of Figures, Table of Tables and the abstract section.
|
||||
|
||||
[[structure-document-books-physical-book]]
|
||||
==== book.adoc
|
||||
|
||||
The [.filename]#_index.adoc# defines some AsciiDoc variables that affect how the AsciiDoc source is converted to other formats and list the Table of Contents, Table of Examples, Table of Figures, Table of Tables, the abstract section and all the chapters. This file is used to generate the PDF with `asciidoctor-pdf` and to generate the book in one `html` page.
|
||||
|
||||
[[structure-document-books-physical-part]]
|
||||
==== part*.adoc
|
||||
The [.filename]#part*.adoc# files stores a brief introduction of one part of the book.
|
||||
|
||||
[[structure-document-books-physical-toc]]
|
||||
==== toc*.adoc
|
||||
The [.filename]#toc*.adoc# files stores the Table of Contents, Table of Figures, Table of Examples, Table of Tables and different Table of Contents for each part. All of these files are generated by the Python `tools`. *Please don't edit them.*
|
||||
|
||||
[[structure-document-books-physical-chapters-order]]
|
||||
====chapters-order.adoc
|
||||
The [.filename]#chapters-order.adoc# file stores the order of the book chapters.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Please be careful with this file. Is used by the Python `tools` to generate the Table of Contents of the books. In case of editing this file, first contact the mailto:doceng@freebsd.org[Documentation Engineering] team.
|
||||
====
|
||||
|
||||
[[structure-document-handbook-physical-chapters]]
|
||||
==== directory/_index.adoc
|
||||
|
||||
Each chapter in the Handbook is stored in a file called [.filename]#_index.adoc# in a separate directory from the other chapters.
|
||||
|
||||
For example, this is an example of the header of one chapter:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
---
|
||||
title: Chapter 8. Configuring the FreeBSD Kernel
|
||||
part: Part II. Common Tasks
|
||||
prev: books/handbook/multimedia
|
||||
next: books/handbook/printing
|
||||
---
|
||||
|
||||
[[kernelconfig]]
|
||||
= Configuring the FreeBSD Kernel
|
||||
...
|
||||
....
|
||||
|
||||
When the HTML5 version of the Handbook is produced, this will yield [.filename]#kernelconfig/index.html#.
|
||||
|
||||
A brief look will show that there are many directories with individual [.filename]#_index.adoc# files, including [.filename]#basics/_index.adoc#, [.filename]#introduction/_index.adoc#, and [.filename]#printing/_index.xml#.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
Do not name chapters or directories after their ordering within the Handbook. This ordering can change as the content within the Handbook is reorganized. Reorganization should be possible without renaming files, unless entire chapters are being promoted or demoted within the hierarchy.
|
||||
====
|
||||
|
||||
DIFFERENT TOCS
|
||||
|
||||
[[structure-document-articles]]
|
||||
== The Articles: articles/
|
||||
|
||||
The articles are written in AsciiDoc.
|
||||
|
||||
The articles are organized as an AsciiDoc `article`. The articles are divided into sections (`=`) and subsections (`==`, `===`) and so on.
|
||||
|
||||
[[structure-document-articles-physical]]
|
||||
=== Physical Organization
|
||||
|
||||
There are a one [.filename]#_index.adoc# file per article.
|
||||
|
||||
[[structure-document-articles-physical-index]]
|
||||
==== _index.adoc
|
||||
|
||||
The [.filename]#_index.adoc# file contains all the AsciiDoc variables and the content.
|
||||
|
||||
For example, this is an example of one article, the structure is pretty similar to one book chapter:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
---
|
||||
title: Why you should use a BSD style license for your Open Source Project
|
||||
authors:
|
||||
- author: Bruce Montague
|
||||
email: brucem@alumni.cse.ucsc.edu
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["freebsd", "intel", "general"]
|
||||
---
|
||||
|
||||
\= Why you should use a BSD style license for your Open Source Project
|
||||
:doctype: article
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
[[intro]]
|
||||
\== Introduction
|
||||
....
|
||||
44
documentation/content/en/books/fdp-primer/tools/chapter.adoc
Normal file
44
documentation/content/en/books/fdp-primer/tools/chapter.adoc
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
title: Chapter 2. Tools
|
||||
prev: books/fdp-primer/overview
|
||||
next: books/fdp-primer/working-copy
|
||||
---
|
||||
|
||||
[[tools]]
|
||||
= Tools
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 2
|
||||
|
||||
toc::[]
|
||||
|
||||
Several software tools are used to manage the FreeBSD documentation and render it to different output formats. Some of these tools are required and must be installed before working through the examples in the following chapters. Some are optional, adding capabilities or making the job of creating documentation less demanding.
|
||||
|
||||
[[tools-required]]
|
||||
== Required Tools
|
||||
|
||||
Install `hugo` and `asciidoctor` as showed in the <<overview,the overview chapter>> from the Ports Collection. These applications are required to do useful work with the FreeBSD documentation. Some further notes on particular components are given below.
|
||||
|
||||
[[tools-optional]]
|
||||
== Optional Tools
|
||||
|
||||
These applications are not required, but can make working on the documentation easier or add capabilities.
|
||||
|
||||
[[tools-optional-software]]
|
||||
=== Software
|
||||
|
||||
Vim (package:editors/vim[])::
|
||||
A popular editor for working with AsciiDoctor.
|
||||
|
||||
Emacs (package:editors/emacs[])::
|
||||
Both of these editors include a special mode for editing documents. This mode includes commands to reduce the amount of typing needed, and help reduce the possibility of errors.
|
||||
|
|
@ -0,0 +1,206 @@
|
|||
---
|
||||
title: Chapter 8. Translations
|
||||
prev: books/fdp-primer/rosetta
|
||||
next: books/fdp-primer/po-translations
|
||||
---
|
||||
|
||||
[[translations]]
|
||||
= Translations
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 8
|
||||
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
This is the FAQ for people translating the FreeBSD documentation (FAQ, Handbook, tutorials, manual pages, and others) to different languages.
|
||||
|
||||
It is _very_ heavily based on the translation FAQ from the FreeBSD German Documentation Project, originally written by Frank Gründer
|
||||
mailto:elwood@mc5sys.in-berlin.de[elwood@mc5sys.in-berlin.de] and translated back to English by Bernd Warken
|
||||
mailto:bwarken@mayn.de[bwarken@mayn.de].
|
||||
|
||||
== What do i18n and l10n mean?
|
||||
|
||||
i18n means internationalization and l10n means localization. They are just a convenient shorthand.
|
||||
|
||||
i18n can be read as "i" followed by 18 letters, followed by "n". Similarly, l10n is "l" followed by 10 letters, followed by "n".
|
||||
|
||||
== Is there a mailing list for translators?
|
||||
|
||||
Yes. Different translation groups have their own mailing lists. The https://www.freebsd.org/docproj/translations[list of translation projects] has more information about the mailing lists and web sites run by each translation project. In addition there is
|
||||
mailto:freebsd-translators@freebsd.org[freebsd-translators@freebsd.org] for general translation discussion.
|
||||
|
||||
== Are more translators needed?
|
||||
|
||||
Yes. The more people work on translation the faster it gets done, and the faster changes to the English documentation are mirrored in the translated documents.
|
||||
|
||||
You do not have to be a professional translator to be able to help.
|
||||
|
||||
== What languages do I need to know?
|
||||
|
||||
Ideally, you will have a good knowledge of written English, and obviously you will need to be fluent in the language you are translating to.
|
||||
|
||||
English is not strictly necessary. For example, you could do a Hungarian translation of the FAQ from the Spanish translation.
|
||||
|
||||
== What software do I need to know?
|
||||
|
||||
It is strongly recommended that you maintain a local copy of the FreeBSD Git repository (at least the documentation part). This can be done by running:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% git clone https://git.FreeBSD.org/doc.git doc
|
||||
....
|
||||
|
||||
https://cgit.FreeBSD.org/[cgit.FreeBSD.org] is a public `git` server.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
This will require the package:git-lite[] package to be installed.
|
||||
====
|
||||
|
||||
You should be comfortable using git. This will allow you to see what has changed between different versions of the files that make up the documentation.
|
||||
|
||||
For example, to view the differences between revisions `r33733` and `r33734` of [.filename]#content/en/books/fdp-primer/book.adoc#, run:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% git diff -r33733:33734 content/en/books/fdp-primer/book.adoc
|
||||
....
|
||||
|
||||
Please see the complete explanation of using Git in FreeBSD in the link:{handbook}[FreeBSD Handbook].
|
||||
|
||||
== How do I find out who else might be translating to the same language?
|
||||
|
||||
The https://www.FreeBSD.org/docproj/translations/[Documentation Project translations page] lists the translation efforts that are currently known about. If others are already working on translating documentation to your language, please do not duplicate their efforts. Instead, contact them to see how you can help.
|
||||
|
||||
If no one is listed on that page as translating for your language, then send a message to the {freebsd-doc} in case someone else is thinking of doing a translation, but has not announced it yet.
|
||||
|
||||
== No one else is translating to my language. What do I do?
|
||||
|
||||
Congratulations, you have just started the "FreeBSD _your-language-here_ Documentation Translation Project". Welcome aboard.
|
||||
|
||||
First, decide whether or not you have got the time to spare. Since you are the only person working on your language at the moment it is going to be your responsibility to publicize your work and coordinate any volunteers that might want to help you.
|
||||
|
||||
Write an email to the Documentation Project mailing list, announcing that you are going to translate the documentation, so the Documentation Project translations page can be maintained.
|
||||
|
||||
If there is already someone in your country providing FreeBSD mirroring services you should contact them and ask if you can have some webspace for your project, and possibly an email address or mailing list services.
|
||||
|
||||
Then pick a document and start translating. It is best to start with something fairly small-either the FAQ, or one of the tutorials.
|
||||
|
||||
== I have translated some documentation, where do I send it?
|
||||
|
||||
That depends. If you are already working with a translation team (such as the Japanese team, or the German team) then they will have their own procedures for handling submitted documentation, and these will be outlined on their web pages.
|
||||
|
||||
If you are the only person working on a particular language (or you are responsible for a translation project and want to submit your changes back to the FreeBSD project) then you should send your translation to the FreeBSD project (see the next question).
|
||||
|
||||
== I am the only person working on translating to this language, how do I submit my translation?
|
||||
|
||||
First, make sure your translation is organized properly. This means that it should drop into the existing documentation tree and build straight away.
|
||||
|
||||
Currently, the FreeBSD documentation is stored in a top level directory called [.filename]#head/#. Directories below this are named according to the language code they are written in, as defined in ISO639 ([.filename]#/usr/shared/misc/iso639# on a version of FreeBSD newer than 20th January 1999).
|
||||
|
||||
If your language can be encoded in different ways (for example, Chinese) then there should be directories below this, one for each encoding format you have provided.
|
||||
|
||||
Finally, you should have directories for each document.
|
||||
|
||||
For example, a hypothetical Swedish translation might look like:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
head/
|
||||
content/
|
||||
sv/
|
||||
books/
|
||||
faq/
|
||||
book.adoc
|
||||
....
|
||||
|
||||
`sv` is the name of the translation, in [.filename]#lang# form. Note the two Makefiles, which will be used to build the documentation.
|
||||
|
||||
Use man:tar[1] and man:gzip[1] to compress up your documentation, and send it to the project.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% cd doc
|
||||
% tar cf swedish-docs.tar sv
|
||||
% gzip -9 swedish-docs.tar
|
||||
....
|
||||
|
||||
Put [.filename]#swedish-docs.tar.gz# somewhere. If you do not have access to your own webspace (perhaps your ISP does not let you have any) then you can email {doceng}, and arrange to email the files when it is convenient.
|
||||
|
||||
Either way, you should use Bugzilla to submit a report indicating that you have submitted the documentation. It would be very helpful if you could get other people to look over your translation and double check it first, since it is unlikely that the person committing it will be fluent in the language.
|
||||
|
||||
Someone (probably the Documentation Project Manager, currently {doceng}) will then take your translation and confirm that it builds. In particular, the following things will be looked at:
|
||||
|
||||
. Do all your files use RCS strings (such as "ID")?
|
||||
. Does `make` in the [.filename]#root# directory work correctly?
|
||||
|
||||
If there are any problems then whoever is looking at the submission will get back to you to work them out.
|
||||
|
||||
If there are no problems your translation will be committed as soon as possible.
|
||||
|
||||
== Can I include language or country specific text in my translation?
|
||||
|
||||
We would prefer that you did not.
|
||||
|
||||
For example, suppose that you are translating the Handbook to Korean, and want to include a section about retailers in Korea in your Handbook.
|
||||
|
||||
There is no real reason why that information should not be in the English (or German, or Spanish, or Japanese, or ...) versions as well. It is feasible that an English speaker in Korea might try to pick up a copy of FreeBSD whilst over there. It also helps increase FreeBSD's perceived presence around the globe, which is not a bad thing.
|
||||
|
||||
If you have country specific information, please submit it as a change to the English Handbook (using Bugzilla) and then translate the change back to your language in the translated Handbook.
|
||||
|
||||
Thanks.
|
||||
|
||||
=== Addressing the reader
|
||||
|
||||
In the English documents, the reader is addressed as "you", there is no formal/informal distinction as there is in some languages.
|
||||
|
||||
If you are translating to a language which does distinguish, use whichever form is typically used in other technical documentation in your language. If in doubt, use a mildly polite form.
|
||||
|
||||
=== Do I need to include any additional information in my translations?
|
||||
|
||||
Yes.
|
||||
|
||||
The header of the English version of each document will look something like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
---
|
||||
title: Why you should use a BSD style license for your Open Source Project
|
||||
releaseinfo: "$FreeBSD: head/en_US.ISO8859-1/articles/bsdl-gpl/article.xml 53942 2020-03-01 12:23:40Z carlavilla $"
|
||||
trademarks: ["freebsd", "intel", "general"]
|
||||
---
|
||||
|
||||
= Why you should use a BSD style license for your Open Source Project
|
||||
....
|
||||
|
||||
The exact boilerplate may change, but it will always include a $FreeBSD$ line and the phrase `The FreeBSD Documentation Project`. Note that the $FreeBSD$ part is expanded automatically by Git, so it should be empty (just `$FreeBSD$`) for new files.
|
||||
|
||||
Your translated documents should include their own FreeBSD line, and change the `FreeBSD Documentation Project` line to `The FreeBSD _language_ Documentation Project`.
|
||||
|
||||
In addition, you should add a third line which indicates which revision of the English text this is based on.
|
||||
|
||||
So, the Spanish version of this file might start:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
---
|
||||
title: Soporte para segundos intercalares en FreeBSD
|
||||
releaseinfo: "$FreeBSD: head/es_ES.ISO8859-1/articles/leap-seconds/article.xml 53090 2019-06-01 17:52:59Z carlavilla $"
|
||||
---
|
||||
|
||||
= Soporte para segundos intercalares en FreeBSD
|
||||
....
|
||||
|
||||
|
|
@ -0,0 +1,113 @@
|
|||
---
|
||||
title: Chapter 3. The Working Copy
|
||||
prev: books/fdp-primer/tools
|
||||
next: books/fdp-primer/structure
|
||||
---
|
||||
|
||||
[[working-copy]]
|
||||
= The Working Copy
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 3
|
||||
|
||||
toc::[]
|
||||
|
||||
The _working copy_ is a copy of the FreeBSD repository documentation tree downloaded onto the local computer. Changes are made to the local working copy, tested, and then submitted as patches to be committed to the main repository.
|
||||
|
||||
A full copy of the documentation tree can occupy 700 megabytes of disk space. Allow for a full gigabyte of space to have room for temporary files and test versions of various output formats.
|
||||
|
||||
link:https://git-scm.com/[Git] is used to manage the FreeBSD documentation files. It is obtained by installing the Git package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install git-lite
|
||||
....
|
||||
|
||||
[[working-copy-doc-and-src]]
|
||||
== Documentation and Manual Pages
|
||||
|
||||
FreeBSD documentation is not just books and articles. Manual pages for all the commands and configuration files are also part of the documentation, and part of the FDP's territory. Two repositories are involved: `doc` for the books and articles, and `src` for the operating system and manual pages. To edit manual pages, the `src` repository must be checked out separately.
|
||||
|
||||
Repositories may contain multiple versions of documentation and source code. New modifications are almost always made only to the latest version, called `main`.
|
||||
|
||||
[[working-copy-choosing-directory]]
|
||||
== Choosing a Directory
|
||||
|
||||
FreeBSD documentation is traditionally stored in [.filename]#/usr/doc/#, and system source code with manual pages in [.filename]#/usr/src/#. These directory trees are relocatable, and users may want to put the working copies in other locations to avoid interfering with existing information in the main directories. The examples that follow use [.filename]#~/doc# and [.filename]#~/src#, both subdirectories of the user's home directory.
|
||||
|
||||
[[working-copy-checking-out]]
|
||||
== Checking Out a Copy
|
||||
|
||||
A download of a working copy from the repository is called a _clone_, and done with `git clone`. This example clones a copy of the latest version (`main`) of the main documentation tree:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% git clone https://git.FreeBSD.org/doc.git ~/doc
|
||||
....
|
||||
|
||||
A checkout of the source code to work on manual pages is very similar:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% git clone https://git.FreeBSD.org/src.git ~/src
|
||||
....
|
||||
|
||||
[[working-copy-updating]]
|
||||
== Updating a Working Copy
|
||||
|
||||
The documents and files in the FreeBSD repository change daily. People modify files and commit changes frequently. Even a short time after an initial checkout, there will already be differences between the local working copy and the main FreeBSD repository. To update the local version with the changes that have been made to the main repository, use `git pull` on the directory containing the local working copy:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% git pull ~/doc
|
||||
....
|
||||
|
||||
Get in the protective habit of using `git pull` before editing document files. Someone else may have edited that file very recently, and the local working copy will not include the latest changes until it has been updated. Editing the newest version of a file is much easier than trying to combine an older, edited local file with the newer version from the repository.
|
||||
|
||||
[[working-copy-revert]]
|
||||
== Reverting Changes
|
||||
|
||||
Sometimes it turns out that changes were not necessary after all, or the writer just wants to start over. Files can be "reset" to their unchanged form with `git restore`. For example, to erase the edits made to [.filename]#_index.adoc# and reset it to unmodified form:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% git restore _index.adoc
|
||||
....
|
||||
|
||||
[[working-copy-making-diff]]
|
||||
== Making a Diff
|
||||
|
||||
After edits to a file or group of files are completed, the differences between the local working copy and the version on the FreeBSD repository must be collected into a single file for submission. These _diff_ files are produced by redirecting the output of `git diff` into a file:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc
|
||||
% git diff doc-fix-spelling.diff
|
||||
....
|
||||
|
||||
Give the file a meaningful name that identifies the contents. The example above is for spelling fixes to the whole documentation tree.
|
||||
|
||||
If the diff file is to be submitted with the web "link:https://bugs.FreeBSD.org/bugzilla/enter_bug.cgi[Submit a FreeBSD problem report]" interface, add a [.filename]#.txt# extension to give the earnest and simple-minded web form a clue that the contents are plain text.
|
||||
|
||||
Be careful: `git diff` includes all changes made in the current directory and any subdirectories. If there are files in the working copy with edits that are not ready to be submitted yet, provide a list of only the files that are to be included:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% cd ~/doc
|
||||
% git diff disks/_index.adoc printers/_index.adoc disks-printers.diff
|
||||
....
|
||||
|
||||
[[working-copy-git-references]]
|
||||
== Git References
|
||||
|
||||
These examples show very basic usage of Git. More detail is available in the https://git-scm.com/book/en/v2[Git Book] and the https://git-scm.com/doc[Git documentation].
|
||||
|
|
@ -0,0 +1,173 @@
|
|||
---
|
||||
title: Chapter 11. Writing Style
|
||||
prev: books/fdp-primer/manual-pages
|
||||
next: books/fdp-primer/editor-config
|
||||
---
|
||||
|
||||
[[writing-style]]
|
||||
= Writing Style
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 11
|
||||
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[writing-style-tips]]
|
||||
== Tips
|
||||
|
||||
Technical documentation can be improved by consistent use of several principles. Most of these can be classified into three goals: _be clear_, _be complete_, and _be concise_. These goals can conflict with each other. Good writing consists of a balance between them.
|
||||
|
||||
[[writing-style-be-clear]]
|
||||
=== Be Clear
|
||||
|
||||
Clarity is extremely important. The reader may be a novice, or reading the document in a second language. Strive for simple, uncomplicated text that clearly explains the concepts.
|
||||
|
||||
Avoid flowery or embellished speech, jokes, or colloquial expressions. Write as simply and clearly as possible. Simple text is easier to understand and translate.
|
||||
|
||||
Keep explanations as short, simple, and clear as possible. Avoid empty phrases like "in order to", which usually just means "to". Avoid potentially patronizing words like "basically". Avoid Latin terms like "i.e.," or "cf.", which may be unknown outside of academic or scientific groups.
|
||||
|
||||
Write in a formal style. Avoid addressing the reader as "you". For example, say "copy the file to [.filename]#/tmp#" rather than "you can copy the file to [.filename]#/tmp#".
|
||||
|
||||
Give clear, correct, _tested_ examples. A trivial example is better than no example. A good example is better yet. Do not give bad examples, identifiable by apologies or sentences like "but really it should never be done that way". Bad examples are worse than no examples. Give good examples, because _even when warned not to use the example as shown_, the reader will usually just use the example as shown.
|
||||
|
||||
Avoid _weasel words_ like "should", "might", "try", or "could". These words imply that the speaker is unsure of the facts, and create doubt in the reader.
|
||||
|
||||
Similarly, give instructions as imperative commands: not "you should do this", but merely "do this".
|
||||
|
||||
[[writing-style-be-complete]]
|
||||
=== Be Complete
|
||||
|
||||
Do not make assumptions about the reader's abilities or skill level. Tell them what they need to know. Give links to other documents to provide background information without having to recreate it. Put yourself in the reader's place, anticipate the questions they will ask, and answer them.
|
||||
|
||||
[[writing-style-be-concise]]
|
||||
=== Be Concise
|
||||
|
||||
While features should be documented completely, sometimes there is so much information that the reader cannot easily find the specific detail needed. The balance between being complete and being concise is a challenge. One approach is to have an introduction, then a "quick start" section that describes the most common situation, followed by an in-depth reference section.
|
||||
|
||||
[[writing-style-guidelines]]
|
||||
== Guidelines
|
||||
|
||||
To promote consistency between the myriad authors of the FreeBSD documentation, some guidelines have been drawn up for authors to follow.
|
||||
|
||||
Use American English Spelling::
|
||||
There are several variants of English, with different spellings for the same word. Where spellings differ, use the American English variant. "color", not "colour", "rationalize", not "rationalise", and so on.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
The use of British English may be accepted in the case of a contributed article, however the spelling must be consistent within the whole document. The other documents such as books, web site, manual pages, etc. will have to use American English.
|
||||
====
|
||||
|
||||
Do not use contractions::
|
||||
Do not use contractions. Always spell the phrase out in full. "Don't use contractions" is wrong.
|
||||
+
|
||||
Avoiding contractions makes for a more formal tone, is more precise, and is slightly easier for translators.
|
||||
|
||||
Use the serial comma::
|
||||
In a list of items within a paragraph, separate each item from the others with a comma. Separate the last item from the others with a comma and the word "and".
|
||||
+
|
||||
For example:
|
||||
+
|
||||
This is a list of one, two and three items.
|
||||
+
|
||||
Is this a list of three items, "one", "two", and "three", or a list of two items, "one" and "two and three"?
|
||||
+
|
||||
It is better to be explicit and include a serial comma:
|
||||
+
|
||||
This is a list of one, two, and three items.
|
||||
Avoid redundant phrases::
|
||||
Do not use redundant phrases. In particular, "the command", "the file", and "man command" are often redundant.
|
||||
+
|
||||
For example, commands:
|
||||
+
|
||||
Wrong: Use the `git` command to update sources.
|
||||
+
|
||||
Right: Use `git` to update sources.
|
||||
+
|
||||
Filenames:
|
||||
+
|
||||
Wrong: ... in the filename [.filename]#/etc/rc.local#...
|
||||
+
|
||||
Right: ... in [.filename]#/etc/rc.local#...
|
||||
+
|
||||
Manual page references (the second example uses `citerefentry` with the man:csh[1] entity):.
|
||||
+
|
||||
Wrong: See `man csh` for more information.
|
||||
+
|
||||
Right: See man:csh[1].
|
||||
|
||||
For more information about writing style, see http://www.bartleby.com/141/[Elements of Style], by William Strunk.
|
||||
|
||||
[[writing-style-guide]]
|
||||
== Style Guide
|
||||
|
||||
To keep the source for the documentation consistent when many different people are editing it, please follow these style conventions.
|
||||
|
||||
[[writing-style-acronyms]]
|
||||
=== Acronyms
|
||||
|
||||
Acronyms should be defined the first time they appear in a document, as in: "Network Time Protocol (NTP)". After the acronym has been defined, use the acronym alone unless it makes more sense contextually to use the whole term. Acronyms are usually defined only once per chapter or per document.
|
||||
|
||||
All acronyms should be enclosed using the ` character.
|
||||
|
||||
[[writing-style-word-list]]
|
||||
== Word List
|
||||
|
||||
This list of words shows the correct spelling and capitalization when used in FreeBSD documentation. If a word is not on this list, ask about it on the {freebsd-doc}.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Name
|
||||
| Syntax
|
||||
| Rendered
|
||||
|
||||
|
||||
| Copyright
|
||||
| +(C)+
|
||||
| (C)
|
||||
|
||||
| Registered
|
||||
| +(R)+
|
||||
| (R)
|
||||
|
||||
| Trademark
|
||||
| +(TM)+
|
||||
| (TM)
|
||||
|
||||
| Em dash
|
||||
| +--+
|
||||
| --
|
||||
|
||||
| Ellipses
|
||||
| +...+
|
||||
| ...
|
||||
|
||||
| Single right arrow
|
||||
| +->+
|
||||
| ->
|
||||
|
||||
| Double right arrow
|
||||
| +=>+
|
||||
| =>
|
||||
|
||||
| Single left arrow
|
||||
| +<-+
|
||||
| <-
|
||||
|
||||
| Double left arrow
|
||||
| +<=+
|
||||
| <=
|
||||
|
||||
|===
|
||||
39
documentation/content/en/books/handbook/_index.adoc
Normal file
39
documentation/content/en/books/handbook/_index.adoc
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
title: FreeBSD Handbook
|
||||
authors:
|
||||
- author: The FreeBSD Documentation Project
|
||||
copyright: 1995-2021 The FreeBSD Documentation Project
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["freebsd", "ibm", "ieee", "redhat", "3com", "adobe", "apple", "intel", "linux", "microsoft", "opengroup", "sun", "realnetworks", "oracle", "3ware", "arm", "adaptec", "google", "heidelberger", "intuit", "lsilogic", "themathworks", "thomson", "vmware", "wolframresearch", "xiph", "xfree86", "general"]
|
||||
next: books/handbook/preface
|
||||
---
|
||||
|
||||
= FreeBSD Handbook
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
|
||||
[.abstract-title]
|
||||
Abstract
|
||||
|
||||
Welcome to FreeBSD! This handbook covers the installation and day to day use of _FreeBSD {rel122-current}-RELEASE_, _FreeBSD {rel121-current}-RELEASE_ and _FreeBSD {rel114-current}-RELEASE_. This book is the result of ongoing work by many individuals. Some sections might be outdated. Those interested in helping to update and expand this document should send email to the {freebsd-doc}.
|
||||
|
||||
The latest version of this book is available from the https://www.FreeBSD.org/[FreeBSD web site]. Previous versions can be obtained from https://docs.FreeBSD.org/doc/[https://docs.FreeBSD.org/doc/]. The book can be downloaded in a variety of formats and compression options from the https://download.freebsd.org/ftp/doc/[FreeBSD FTP server] or one of the numerous link:./mirrors#mirrors-ftp[mirror sites]. Printed copies can be purchased at the https://www.freebsdmall.com/[FreeBSD Mall]. Searches can be performed on the handbook and other documents on the link:https://www.FreeBSD.org/search/[search page].
|
||||
|
||||
'''
|
||||
|
||||
include::content/en/books/handbook/toc.adoc[]
|
||||
|
||||
include::content/en/books/handbook/toc-figures.adoc[]
|
||||
|
||||
include::content/en/books/handbook/toc-tables.adoc[]
|
||||
|
||||
include::content/en/books/handbook/toc-examples.adoc[]
|
||||
File diff suppressed because it is too large
Load diff
401
documentation/content/en/books/handbook/audit/_index.adoc
Normal file
401
documentation/content/en/books/handbook/audit/_index.adoc
Normal file
|
|
@ -0,0 +1,401 @@
|
|||
---
|
||||
title: Chapter 17. Security Event Auditing
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/mac
|
||||
next: books/handbook/disks
|
||||
---
|
||||
|
||||
[[audit]]
|
||||
= Security Event Auditing
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 17
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/audit/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/audit/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/audit/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[audit-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
The FreeBSD operating system includes support for security event auditing. Event auditing supports reliable, fine-grained, and configurable logging of a variety of security-relevant system events, including logins, configuration changes, and file and network access. These log records can be invaluable for live system monitoring, intrusion detection, and postmortem analysis. FreeBSD implements Sun(TM)'s published Basic Security Module (BSM) Application Programming Interface (API) and file format, and is interoperable with the Solaris(TM) and Mac OS(R) X audit implementations.
|
||||
|
||||
This chapter focuses on the installation and configuration of event auditing. It explains audit policies and provides an example audit configuration.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* What event auditing is and how it works.
|
||||
* How to configure event auditing on FreeBSD for users and processes.
|
||||
* How to review the audit trail using the audit reduction and review tools.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Understand UNIX(R) and FreeBSD basics (crossref:basics[basics,FreeBSD Basics]).
|
||||
* Be familiar with the basics of kernel configuration/compilation (crossref:kernelconfig[kernelconfig,Configuring the FreeBSD Kernel]).
|
||||
* Have some familiarity with security and how it pertains to FreeBSD (crossref:security[security,Security]).
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
The audit facility has some known limitations. Not all security-relevant system events are auditable and some login mechanisms, such as Xorg-based display managers and third-party daemons, do not properly configure auditing for user login sessions.
|
||||
|
||||
The security event auditing facility is able to generate very detailed logs of system activity. On a busy system, trail file data can be very large when configured for high detail, exceeding gigabytes a week in some configurations. Administrators should take into account the disk space requirements associated with high volume audit configurations. For example, it may be desirable to dedicate a file system to [.filename]#/var/audit# so that other file systems are not affected if the audit file system becomes full.
|
||||
====
|
||||
|
||||
[[audit-inline-glossary]]
|
||||
== Key Terms
|
||||
|
||||
The following terms are related to security event auditing:
|
||||
|
||||
* _event_: an auditable event is any event that can be logged using the audit subsystem. Examples of security-relevant events include the creation of a file, the building of a network connection, or a user logging in. Events are either "attributable", meaning that they can be traced to an authenticated user, or "non-attributable". Examples of non-attributable events are any events that occur before authentication in the login process, such as bad password attempts.
|
||||
* _class_: a named set of related events which are used in selection expressions. Commonly used classes of events include "file creation" (fc), "exec" (ex), and "login_logout" (lo).
|
||||
* _record_: an audit log entry describing a security event. Records contain a record event type, information on the subject (user) performing the action, date and time information, information on any objects or arguments, and a success or failure condition.
|
||||
* _trail_: a log file consisting of a series of audit records describing security events. Trails are in roughly chronological order with respect to the time events completed. Only authorized processes are allowed to commit records to the audit trail.
|
||||
* _selection expression_: a string containing a list of prefixes and audit event class names used to match events.
|
||||
* _preselection_: the process by which the system identifies which events are of interest to the administrator. The preselection configuration uses a series of selection expressions to identify which classes of events to audit for which users, as well as global settings that apply to both authenticated and unauthenticated processes.
|
||||
* _reduction_: the process by which records from existing audit trails are selected for preservation, printing, or analysis. Likewise, the process by which undesired audit records are removed from the audit trail. Using reduction, administrators can implement policies for the preservation of audit data. For example, detailed audit trails might be kept for one month, but after that, trails might be reduced in order to preserve only login information for archival purposes.
|
||||
|
||||
[[audit-config]]
|
||||
== Audit Configuration
|
||||
|
||||
User space support for event auditing is installed as part of the base FreeBSD operating system. Kernel support is available in the [.filename]#GENERIC# kernel by default, and man:auditd[8] can be enabled by adding the following line to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
auditd_enable="YES"
|
||||
....
|
||||
|
||||
Then, start the audit daemon:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service auditd start
|
||||
....
|
||||
|
||||
Users who prefer to compile a custom kernel must include the following line in their custom kernel configuration file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
options AUDIT
|
||||
....
|
||||
|
||||
=== Event Selection Expressions
|
||||
|
||||
Selection expressions are used in a number of places in the audit configuration to determine which events should be audited. Expressions contain a list of event classes to match. Selection expressions are evaluated from left to right, and two expressions are combined by appending one onto the other.
|
||||
|
||||
<<event-selection>> summarizes the default audit event classes:
|
||||
|
||||
[[event-selection]]
|
||||
.Default Audit Event Classes
|
||||
[cols="1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Class Name
|
||||
| Description
|
||||
| Action
|
||||
|
||||
|all
|
||||
|all
|
||||
|Match all event classes.
|
||||
|
||||
|aa
|
||||
|authentication and authorization
|
||||
|
|
||||
|
||||
|ad
|
||||
|administrative
|
||||
|Administrative actions performed on the system as a whole.
|
||||
|
||||
|ap
|
||||
|application
|
||||
|Application defined action.
|
||||
|
||||
|cl
|
||||
|file close
|
||||
|Audit calls to the `close` system call.
|
||||
|
||||
|ex
|
||||
|exec
|
||||
|Audit program execution. Auditing of command line arguments and environmental variables is controlled via man:audit_control[5] using the `argv` and `envv` parameters to the `policy` setting.
|
||||
|
||||
|fa
|
||||
|file attribute access
|
||||
|Audit the access of object attributes such as man:stat[1] and man:pathconf[2].
|
||||
|
||||
|fc
|
||||
|file create
|
||||
|Audit events where a file is created as a result.
|
||||
|
||||
|fd
|
||||
|file delete
|
||||
|Audit events where file deletion occurs.
|
||||
|
||||
|fm
|
||||
|file attribute modify
|
||||
|Audit events where file attribute modification occurs, such as by man:chown[8], man:chflags[1], and man:flock[2].
|
||||
|
||||
|fr
|
||||
|file read
|
||||
|Audit events in which data is read or files are opened for reading.
|
||||
|
||||
|fw
|
||||
|file write
|
||||
|Audit events in which data is written or files are written or modified.
|
||||
|
||||
|io
|
||||
|ioctl
|
||||
|Audit use of the `ioctl` system call.
|
||||
|
||||
|ip
|
||||
|ipc
|
||||
|Audit various forms of Inter-Process Communication, including POSIX pipes and System V IPC operations.
|
||||
|
||||
|lo
|
||||
|login_logout
|
||||
|Audit man:login[1] and man:logout[1] events.
|
||||
|
||||
|na
|
||||
|non attributable
|
||||
|Audit non-attributable events.
|
||||
|
||||
|no
|
||||
|invalid class
|
||||
|Match no audit events.
|
||||
|
||||
|nt
|
||||
|network
|
||||
|Audit events related to network actions such as man:connect[2] and man:accept[2].
|
||||
|
||||
|ot
|
||||
|other
|
||||
|Audit miscellaneous events.
|
||||
|
||||
|pc
|
||||
|process
|
||||
|Audit process operations such as man:exec[3] and man:exit[3].
|
||||
|===
|
||||
|
||||
These audit event classes may be customized by modifying the [.filename]#audit_class# and [.filename]#audit_event# configuration files.
|
||||
|
||||
Each audit event class may be combined with a prefix indicating whether successful/failed operations are matched, and whether the entry is adding or removing matching for the class and type. <<event-prefixes>> summarizes the available prefixes:
|
||||
|
||||
[[event-prefixes]]
|
||||
.Prefixes for Audit Event Classes
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Prefix
|
||||
| Action
|
||||
|
||||
|+
|
||||
|Audit successful events in this class.
|
||||
|
||||
|-
|
||||
|Audit failed events in this class.
|
||||
|
||||
|^
|
||||
|Audit neither successful nor failed events in this class.
|
||||
|
||||
|^+
|
||||
|Do not audit successful events in this class.
|
||||
|
||||
|^-
|
||||
|Do not audit failed events in this class.
|
||||
|===
|
||||
|
||||
If no prefix is present, both successful and failed instances of the event will be audited.
|
||||
|
||||
The following example selection string selects both successful and failed login/logout events, but only successful execution events:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lo,+ex
|
||||
....
|
||||
|
||||
=== Configuration Files
|
||||
|
||||
The following configuration files for security event auditing are found in [.filename]#/etc/security#:
|
||||
|
||||
* [.filename]#audit_class#: contains the definitions of the audit classes.
|
||||
* [.filename]#audit_control#: controls aspects of the audit subsystem, such as default audit classes, minimum disk space to leave on the audit log volume, and maximum audit trail size.
|
||||
* [.filename]#audit_event#: textual names and descriptions of system audit events and a list of which classes each event is in.
|
||||
* [.filename]#audit_user#: user-specific audit requirements to be combined with the global defaults at login.
|
||||
* [.filename]#audit_warn#: a customizable shell script used by man:auditd[8] to generate warning messages in exceptional situations, such as when space for audit records is running low or when the audit trail file has been rotated.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
Audit configuration files should be edited and maintained carefully, as errors in configuration may result in improper logging of events.
|
||||
====
|
||||
|
||||
In most cases, administrators will only need to modify [.filename]#audit_control# and [.filename]#audit_user#. The first file controls system-wide audit properties and policies and the second file may be used to fine-tune auditing by user.
|
||||
|
||||
[[audit-auditcontrol]]
|
||||
==== The [.filename]#audit_control# File
|
||||
|
||||
A number of defaults for the audit subsystem are specified in [.filename]#audit_control#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
dir:/var/audit
|
||||
dist:off
|
||||
flags:lo,aa
|
||||
minfree:5
|
||||
naflags:lo,aa
|
||||
policy:cnt,argv
|
||||
filesz:2M
|
||||
expire-after:10M
|
||||
....
|
||||
|
||||
The `dir` entry is used to set one or more directories where audit logs will be stored. If more than one directory entry appears, they will be used in order as they fill. It is common to configure audit so that audit logs are stored on a dedicated file system, in order to prevent interference between the audit subsystem and other subsystems if the file system fills.
|
||||
|
||||
If the `dist` field is set to `on` or `yes`, hard links will be created to all trail files in [.filename]#/var/audit/dist#.
|
||||
|
||||
The `flags` field sets the system-wide default preselection mask for attributable events. In the example above, successful and failed login/logout events as well as authentication and authorization are audited for all users.
|
||||
|
||||
The `minfree` entry defines the minimum percentage of free space for the file system where the audit trail is stored.
|
||||
|
||||
The `naflags` entry specifies audit classes to be audited for non-attributed events, such as the login/logout process and authentication and authorization.
|
||||
|
||||
The `policy` entry specifies a comma-separated list of policy flags controlling various aspects of audit behavior. The `cnt` indicates that the system should continue running despite an auditing failure (this flag is highly recommended). The other flag, `argv`, causes command line arguments to the man:execve[2] system call to be audited as part of command execution.
|
||||
|
||||
The `filesz` entry specifies the maximum size for an audit trail before automatically terminating and rotating the trail file. A value of `0` disables automatic log rotation. If the requested file size is below the minimum of 512k, it will be ignored and a log message will be generated.
|
||||
|
||||
The `expire-after` field specifies when audit log files will expire and be removed.
|
||||
|
||||
[[audit-audituser]]
|
||||
==== The [.filename]#audit_user# File
|
||||
|
||||
The administrator can specify further audit requirements for specific users in [.filename]#audit_user#. Each line configures auditing for a user via two fields: the `alwaysaudit` field specifies a set of events that should always be audited for the user, and the `neveraudit` field specifies a set of events that should never be audited for the user.
|
||||
|
||||
The following example entries audit login/logout events and successful command execution for `root` and file creation and successful command execution for `www`. If used with the default [.filename]#audit_control#, the `lo` entry for `root` is redundant, and login/logout events will also be audited for `www`.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
root:lo,+ex:no
|
||||
www:fc,+ex:no
|
||||
....
|
||||
|
||||
[[audit-administration]]
|
||||
== Working with Audit Trails
|
||||
|
||||
Since audit trails are stored in the BSM binary format, several built-in tools are available to modify or convert these trails to text. To convert trail files to a simple text format, use `praudit`. To reduce the audit trail file for analysis, archiving, or printing purposes, use `auditreduce`. This utility supports a variety of selection parameters, including event type, event class, user, date or time of the event, and the file path or object acted on.
|
||||
|
||||
For example, to dump the entire contents of a specified audit log in plain text:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# praudit /var/audit/AUDITFILE
|
||||
....
|
||||
|
||||
Where _AUDITFILE_ is the audit log to dump.
|
||||
|
||||
Audit trails consist of a series of audit records made up of tokens, which `praudit` prints sequentially, one per line. Each token is of a specific type, such as `header` (an audit record header) or `path` (a file path from a name lookup). The following is an example of an `execve` event:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
header,133,10,execve(2),0,Mon Sep 25 15:58:03 2006, + 384 msec
|
||||
exec arg,finger,doug
|
||||
path,/usr/bin/finger
|
||||
attribute,555,root,wheel,90,24918,104944
|
||||
subject,robert,root,wheel,root,wheel,38439,38032,42086,128.232.9.100
|
||||
return,success,0
|
||||
trailer,133
|
||||
....
|
||||
|
||||
This audit represents a successful `execve` call, in which the command `finger doug` has been run. The `exec arg` token contains the processed command line presented by the shell to the kernel. The `path` token holds the path to the executable as looked up by the kernel. The `attribute` token describes the binary and includes the file mode. The `subject` token stores the audit user ID, effective user ID and group ID, real user ID and group ID, process ID, session ID, port ID, and login address. Notice that the audit user ID and real user ID differ as the user `robert` switched to the `root` account before running this command, but it is audited using the original authenticated user. The `return` token indicates the successful execution and the `trailer` concludes the record.
|
||||
|
||||
XML output format is also supported and can be selected by including `-x`.
|
||||
|
||||
Since audit logs may be very large, a subset of records can be selected using `auditreduce`. This example selects all audit records produced for the user `trhodes` stored in [.filename]#AUDITFILE#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# auditreduce -u trhodes /var/audit/AUDITFILE | praudit
|
||||
....
|
||||
|
||||
Members of the `audit` group have permission to read audit trails in [.filename]#/var/audit#. By default, this group is empty, so only the `root` user can read audit trails. Users may be added to the `audit` group in order to delegate audit review rights. As the ability to track audit log contents provides significant insight into the behavior of users and processes, it is recommended that the delegation of audit review rights be performed with caution.
|
||||
|
||||
=== Live Monitoring Using Audit Pipes
|
||||
|
||||
Audit pipes are cloning pseudo-devices which allow applications to tap the live audit record stream. This is primarily of interest to authors of intrusion detection and system monitoring applications. However, the audit pipe device is a convenient way for the administrator to allow live monitoring without running into problems with audit trail file ownership or log rotation interrupting the event stream. To track the live audit event stream:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# praudit /dev/auditpipe
|
||||
....
|
||||
|
||||
By default, audit pipe device nodes are accessible only to the `root` user. To make them accessible to the members of the `audit` group, add a `devfs` rule to [.filename]#/etc/devfs.rules#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
add path 'auditpipe*' mode 0440 group audit
|
||||
....
|
||||
|
||||
See man:devfs.rules[5] for more information on configuring the devfs file system.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
It is easy to produce audit event feedback cycles, in which the viewing of each audit event results in the generation of more audit events. For example, if all network I/O is audited, and `praudit` is run from an SSH session, a continuous stream of audit events will be generated at a high rate, as each event being printed will generate another event. For this reason, it is advisable to run `praudit` on an audit pipe device from sessions without fine-grained I/O auditing.
|
||||
====
|
||||
|
||||
=== Rotating and Compressing Audit Trail Files
|
||||
|
||||
Audit trails are written to by the kernel and managed by the audit daemon, man:auditd[8]. Administrators should not attempt to use man:newsyslog.conf[5] or other tools to directly rotate audit logs. Instead, `audit` should be used to shut down auditing, reconfigure the audit system, and perform log rotation. The following command causes the audit daemon to create a new audit log and signal the kernel to switch to using the new log. The old log will be terminated and renamed, at which point it may then be manipulated by the administrator:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# audit -n
|
||||
....
|
||||
|
||||
If man:auditd[8] is not currently running, this command will fail and an error message will be produced.
|
||||
|
||||
Adding the following line to [.filename]#/etc/crontab# will schedule this rotation every twelve hours:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
0 */12 * * * root /usr/sbin/audit -n
|
||||
....
|
||||
|
||||
The change will take effect once [.filename]#/etc/crontab# is saved.
|
||||
|
||||
Automatic rotation of the audit trail file based on file size is possible using `filesz` in [.filename]#audit_control# as described in <<audit-auditcontrol>>.
|
||||
|
||||
As audit trail files can become very large, it is often desirable to compress or otherwise archive trails once they have been closed by the audit daemon. The [.filename]#audit_warn# script can be used to perform customized operations for a variety of audit-related events, including the clean termination of audit trails when they are rotated. For example, the following may be added to [.filename]#/etc/security/audit_warn# to compress audit trails on close:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#
|
||||
# Compress audit trail files on close.
|
||||
#
|
||||
if [ "$1" = closefile ]; then
|
||||
gzip -9 $2
|
||||
fi
|
||||
....
|
||||
|
||||
Other archiving activities might include copying trail files to a centralized server, deleting old trail files, or reducing the audit trail to remove unneeded records. This script will be run only when audit trail files are cleanly terminated. It will not be run on trails left unterminated following an improper shutdown.
|
||||
1544
documentation/content/en/books/handbook/basics/_index.adoc
Normal file
1544
documentation/content/en/books/handbook/basics/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
151
documentation/content/en/books/handbook/bibliography/_index.adoc
Normal file
151
documentation/content/en/books/handbook/bibliography/_index.adoc
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
---
|
||||
title: Appendix B. Bibliography
|
||||
part: Part V. Appendices
|
||||
prev: books/handbook/mirrors
|
||||
next: books/handbook/eresources
|
||||
---
|
||||
|
||||
[appendix]
|
||||
[[bibliography]]
|
||||
= Bibliography
|
||||
:doctype: book
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: B
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
While manual pages provide a definitive reference for individual pieces of the FreeBSD operating system, they seldom illustrate how to put the pieces together to make the whole operating system run smoothly. For this, there is no substitute for a good book or users' manual on UNIX(R) system administration.
|
||||
|
||||
[[bibliography-freebsd]]
|
||||
== Books Specific to FreeBSD
|
||||
|
||||
International books:
|
||||
|
||||
* http://jdli.tw.FreeBSD.org/publication/book/freebsd2/index.htm[Using FreeBSD] (in Traditional Chinese), published by http://www.drmaster.com.tw/[Drmaster], 1997. ISBN 9-578-39435-7.
|
||||
* FreeBSD Unleashed (Simplified Chinese translation), published by http://www.hzbook.com/[China Machine Press]. ISBN 7-111-10201-0.
|
||||
* FreeBSD From Scratch Second Edition (in Simplified Chinese), published by China Machine Press. ISBN 7-111-10286-X.
|
||||
* FreeBSD Handbook Second Edition (Simplified Chinese translation), published by http://www.ptpress.com.cn/[Posts & Telecom Press]. ISBN 7-115-10541-3.
|
||||
* FreeBSD & Windows (in Simplified Chinese), published by http://www.tdpress.com/[China Railway Publishing House]. ISBN 7-113-03845-X
|
||||
* FreeBSD Internet Services HOWTO (in Simplified Chinese), published by China Railway Publishing House. ISBN 7-113-03423-3
|
||||
* FreeBSD (in Japanese), published by CUTT. ISBN 4-906391-22-2 C3055 P2400E.
|
||||
* http://www.shoeisha.com/book/Detail.asp?bid=650[Complete Introduction to FreeBSD] (in Japanese), published by http://www.shoeisha.co.jp/[Shoeisha Co., Ltd]. ISBN 4-88135-473-6 P3600E.
|
||||
* http://www.ascii.co.jp/pb/book1/shinkan/detail/1322785.html[Personal UNIX Starter Kit FreeBSD] (in Japanese), published by http://www.ascii.co.jp/[ASCII]. ISBN 4-7561-1733-3 P3000E.
|
||||
* FreeBSD Handbook (Japanese translation), published by http://www.ascii.co.jp/[ASCII]. ISBN 4-7561-1580-2 P3800E.
|
||||
* FreeBSD mit Methode (in German), published by http://www.cul.de[Computer und Literatur Verlag]/Vertrieb Hanser, 1998. ISBN 3-932311-31-0.
|
||||
* http://www.mitp.de/vmi/mitp/detail/pWert/1343/[FreeBSD de Luxe] (in German), published by http://www.mitp.de[Verlag Modere Industrie], 2003. ISBN 3-8266-1343-0.
|
||||
* http://www.pc.mycom.co.jp/FreeBSD/install-manual.html[FreeBSD Install and Utilization Manual] (in Japanese), published by http://www.pc.mycom.co.jp/[Mainichi Communications Inc.], 1998. ISBN 4-8399-0112-0.
|
||||
* Onno W Purbo, Dodi Maryanto, Syahrial Hubbany, Widjil Widodo _http://maxwell.itb.ac.id/[Building Internet Server with FreeBSD]_ (in Indonesia Language), published by http://www.elexmedia.co.id/[Elex Media Komputindo].
|
||||
* Absolute BSD: The Ultimate Guide to FreeBSD (Traditional Chinese translation), published by http://www.grandtech.com.tw/[GrandTech Press], 2003. ISBN 986-7944-92-5.
|
||||
* http://www.twbsd.org/cht/book/[The FreeBSD 6.0 Book] (in Traditional Chinese), published by Drmaster, 2006. ISBN 9-575-27878-X.
|
||||
|
||||
English language books:
|
||||
|
||||
* http://www.absoluteFreeBSD.com/[Absolute FreeBSD, 2nd Edition: The Complete Guide to FreeBSD], published by http://www.nostarch.com/[No Starch Press], 2007. ISBN: 978-1-59327-151-0
|
||||
* http://www.freebsdmall.com/cgi-bin/fm/bsdcomp[The Complete FreeBSD], published by http://www.oreilly.com/[O'Reilly], 2003. ISBN: 0596005164
|
||||
* http://www.freebsd-corp-net-guide.com/[The FreeBSD Corporate Networker's Guide], published by http://www.awl.com/aw/[Addison-Wesley], 2000. ISBN: 0201704811
|
||||
* http://andrsn.stanford.edu/FreeBSD/introbook/[FreeBSD: An Open-Source Operating System for Your Personal Computer], published by The Bit Tree Press, 2001. ISBN: 0971204500
|
||||
* Teach Yourself FreeBSD in 24 Hours, published by http://www.samspublishing.com/[Sams], 2002. ISBN: 0672324245
|
||||
* FreeBSD 6 Unleashed, published by http://www.samspublishing.com/[Sams], 2006. ISBN: 0672328755
|
||||
* FreeBSD: The Complete Reference, published by http://books.mcgraw-hill.com[McGrawHill], 2003. ISBN: 0072224096
|
||||
|
||||
[[bibliography-userguides]]
|
||||
== Users' Guides
|
||||
|
||||
* Ohio State University has written a http://www.cs.duke.edu/csl/docs/unix_course/[UNIX Introductory Course] which is available online in HTML and PostScript format.
|
||||
+
|
||||
An Italian https://www.FreeBSD.org/doc/it_IT.ISO8859-15/books/unix-introduction/[translation] of this document is available as part of the FreeBSD Italian Documentation Project.
|
||||
* http://www.jp.FreeBSD.org/[Jpman Project, Japan FreeBSD Users Group]. FreeBSD User's Reference Manual (Japanese translation). http://www.pc.mycom.co.jp/[Mainichi Communications Inc.], 1998. ISBN4-8399-0088-4 P3800E.
|
||||
* http://www.ed.ac.uk/[Edinburgh University] has written an http://www.ed.ac.uk/information-services/help-consultancy/is-skills/catalogue/program-op-sys-catalogue/unix1[Online Guide] for newcomers to the UNIX environment.
|
||||
|
||||
[[bibliography-adminguides]]
|
||||
== Administrators' Guides
|
||||
|
||||
* http://www.jp.FreeBSD.org/[Jpman Project, Japan FreeBSD Users Group]. FreeBSD System Administrator's Manual (Japanese translation). http://www.pc.mycom.co.jp/[Mainichi Communications Inc.], 1998. ISBN4-8399-0109-0 P3300E.
|
||||
* Dreyfus, Emmanuel. http://www.eyrolles.com/Informatique/Livre/9782212114638/[Cahiers de l'Admin: BSD] 2nd Ed. (in French), Eyrolles, 2004. ISBN 2-212-11463-X
|
||||
|
||||
[[bibliography-programmers]]
|
||||
== Programmers' Guides
|
||||
|
||||
* Computer Systems Research Group, UC Berkeley. _4.4BSD Programmer's Reference Manual_. O'Reilly & Associates, Inc., 1994. ISBN 1-56592-078-3
|
||||
* Computer Systems Research Group, UC Berkeley. _4.4BSD Programmer's Supplementary Documents_. O'Reilly & Associates, Inc., 1994. ISBN 1-56592-079-1
|
||||
* Harbison, Samuel P. and Steele, Guy L. Jr. _C: A Reference Manual_. 4th Ed. Prentice Hall, 1995. ISBN 0-13-326224-3
|
||||
* Kernighan, Brian and Dennis M. Ritchie. _The C Programming Language_. 2nd Ed. PTR Prentice Hall, 1988. ISBN 0-13-110362-8
|
||||
* Lehey, Greg. _Porting UNIX Software_. O'Reilly & Associates, Inc., 1995. ISBN 1-56592-126-7
|
||||
* Plauger, P. J. _The Standard C Library_. Prentice Hall, 1992. ISBN 0-13-131509-9
|
||||
* Spinellis, Diomidis. http://www.spinellis.gr/codereading/[Code Reading: The Open Source Perspective]. Addison-Wesley, 2003. ISBN 0-201-79940-5
|
||||
* Spinellis, Diomidis. http://www.spinellis.gr/codequality/[Code Quality: The Open Source Perspective]. Addison-Wesley, 2006. ISBN 0-321-16607-8
|
||||
* Stevens, W. Richard and Stephen A. Rago. _Advanced Programming in the UNIX Environment_. 2nd Ed. Reading, Mass. : Addison-Wesley, 2005. ISBN 0-201-43307-9
|
||||
* Stevens, W. Richard. _UNIX Network Programming_. 2nd Ed, PTR Prentice Hall, 1998. ISBN 0-13-490012-X
|
||||
|
||||
[[bibliography-osinternals]]
|
||||
== Operating System Internals
|
||||
|
||||
* Andleigh, Prabhat K. _UNIX System Architecture_. Prentice-Hall, Inc., 1990. ISBN 0-13-949843-5
|
||||
* Jolitz, William. "Porting UNIX to the 386". _Dr. Dobb's Journal_. January 1991-July 1992.
|
||||
* Leffler, Samuel J., Marshall Kirk McKusick, Michael J Karels and John Quarterman _The Design and Implementation of the 4.3BSD UNIX Operating System_. Reading, Mass. : Addison-Wesley, 1989. ISBN 0-201-06196-1
|
||||
* Leffler, Samuel J., Marshall Kirk McKusick, _The Design and Implementation of the 4.3BSD UNIX Operating System: Answer Book_. Reading, Mass. : Addison-Wesley, 1991. ISBN 0-201-54629-9
|
||||
* McKusick, Marshall Kirk, Keith Bostic, Michael J Karels, and John Quarterman. _The Design and Implementation of the 4.4BSD Operating System_. Reading, Mass. : Addison-Wesley, 1996. ISBN 0-201-54979-4
|
||||
+
|
||||
(Chapter 2 of this book is available link:{design-44bsd}[online] as part of the FreeBSD Documentation Project.)
|
||||
* Marshall Kirk McKusick, George V. Neville-Neil _The Design and Implementation of the FreeBSD Operating System_. Boston, Mass. : Addison-Wesley, 2004. ISBN 0-201-70245-2
|
||||
* Marshall Kirk McKusick, George V. Neville-Neil, Robert N. M. Watson _The Design and Implementation of the FreeBSD Operating System, 2nd Ed._. Westford, Mass. : Pearson Education, Inc., 2014. ISBN 0-321-96897-2
|
||||
* Stevens, W. Richard. _TCP/IP Illustrated, Volume 1: The Protocols_. Reading, Mass. : Addison-Wesley, 1996. ISBN 0-201-63346-9
|
||||
* Schimmel, Curt. _Unix Systems for Modern Architectures_. Reading, Mass. : Addison-Wesley, 1994. ISBN 0-201-63338-8
|
||||
* Stevens, W. Richard. _TCP/IP Illustrated, Volume 3: TCP for Transactions, HTTP, NNTP and the UNIX Domain Protocols_. Reading, Mass. : Addison-Wesley, 1996. ISBN 0-201-63495-3
|
||||
* Vahalia, Uresh. _UNIX Internals -- The New Frontiers_. Prentice Hall, 1996. ISBN 0-13-101908-2
|
||||
* Wright, Gary R. and W. Richard Stevens. _TCP/IP Illustrated, Volume 2: The Implementation_. Reading, Mass. : Addison-Wesley, 1995. ISBN 0-201-63354-X
|
||||
|
||||
[[bibliography-security]]
|
||||
== Security Reference
|
||||
|
||||
* Cheswick, William R. and Steven M. Bellovin. _Firewalls and Internet Security: Repelling the Wily Hacker_. Reading, Mass. : Addison-Wesley, 1995. ISBN 0-201-63357-4
|
||||
* Garfinkel, Simson. _PGP Pretty Good Privacy_ O'Reilly & Associates, Inc., 1995. ISBN 1-56592-098-8
|
||||
|
||||
[[bibliography-hardware]]
|
||||
== Hardware Reference
|
||||
|
||||
* Anderson, Don and Tom Shanley. _Pentium Processor System Architecture_. 2nd Ed. Reading, Mass. : Addison-Wesley, 1995. ISBN 0-201-40992-5
|
||||
* Ferraro, Richard F. _Programmer's Guide to the EGA, VGA, and Super VGA Cards_. 3rd ed. Reading, Mass. : Addison-Wesley, 1995. ISBN 0-201-62490-7
|
||||
* Intel Corporation publishes documentation on their CPUs, chipsets and standards on their http://developer.intel.com/[developer web site], usually as PDF files.
|
||||
* Shanley, Tom. _80486 System Architecture_. 3rd Ed. Reading, Mass. : Addison-Wesley, 1995. ISBN 0-201-40994-1
|
||||
* Shanley, Tom. _ISA System Architecture_. 3rd Ed. Reading, Mass. : Addison-Wesley, 1995. ISBN 0-201-40996-8
|
||||
* Shanley, Tom. _PCI System Architecture_. 4th Ed. Reading, Mass. : Addison-Wesley, 1999. ISBN 0-201-30974-2
|
||||
* Van Gilluwe, Frank. _The Undocumented PC_, 2nd Ed. Reading, Mass: Addison-Wesley Pub. Co., 1996. ISBN 0-201-47950-8
|
||||
* Messmer, Hans-Peter. _The Indispensable PC Hardware Book_, 4th Ed. Reading, Mass : Addison-Wesley Pub. Co., 2002. ISBN 0-201-59616-4
|
||||
|
||||
[[bibliography-history]]
|
||||
== UNIX(R) History
|
||||
|
||||
* Lion, John _Lion's Commentary on UNIX, 6th Ed. With Source Code_. ITP Media Group, 1996. ISBN 1573980137
|
||||
* Raymond, Eric S. _The New Hacker's Dictionary, 3rd edition_. MIT Press, 1996. ISBN 0-262-68092-0. Also known as the http://www.catb.org/~esr/jargon/html/index.html[Jargon File]
|
||||
* Salus, Peter H. _A quarter century of UNIX_. Addison-Wesley Publishing Company, Inc., 1994. ISBN 0-201-54777-5
|
||||
* Simon Garfinkel, Daniel Weise, Steven Strassmann. _The UNIX-HATERS Handbook_. IDG Books Worldwide, Inc., 1994. ISBN 1-56884-203-1. Out of print, but available http://www.simson.net/ref/ugh.pdf[online].
|
||||
* Don Libes, Sandy Ressler _Life with UNIX_ - special edition. Prentice-Hall, Inc., 1989. ISBN 0-13-536657-7
|
||||
* _The BSD family tree_. https://svnweb.freebsd.org/base/head/shared/misc/bsd-family-tree?view=co[https://svnweb.freebsd.org/base/head/shared/misc/bsd-family-tree?view=co] or link:file://localhost/usr/shared/misc/bsd-family-tree[/usr/shared/misc/bsd-family-tree] on a FreeBSD machine.
|
||||
* _Networked Computer Science Technical Reports Library_.
|
||||
* _Old BSD releases from the Computer Systems Research group (CSRG)_. http://www.mckusick.com/csrg/[http://www.mckusick.com/csrg/]: The 4CD set covers all BSD versions from 1BSD to 4.4BSD and 4.4BSD-Lite2 (but not 2.11BSD, unfortunately). The last disk also holds the final sources plus the SCCS files.
|
||||
* Kernighan, Brian _Unix: A History and a Memoir_. Kindle Direct Publishing, 2020. ISBN 978-169597855-3
|
||||
|
||||
[[bibliography-journals]]
|
||||
== Periodicals, Journals, and Magazines
|
||||
|
||||
* http://www.admin-magazin.de/[Admin Magazin] (in German), published by Medialinx AG. ISSN: 2190-1066
|
||||
* http://www.bsdmag.org/[BSD Magazine], published by Software Press Sp. z o.o. SK. ISSN: 1898-9144
|
||||
* http://www.bsdnow.tv/[BSD Now - Video Podcast], published by Jupiter Broadcasting LLC
|
||||
* http://bsdtalk.blogspot.com/[BSD Talk Podcast], by Will Backman
|
||||
* http://freebsdjournal.com/[FreeBSD Journal], published by S&W Publishing, sponsored by The FreeBSD Foundation. ISBN: 978-0-615-88479-0
|
||||
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
167
documentation/content/en/books/handbook/book.adoc
Normal file
167
documentation/content/en/books/handbook/book.adoc
Normal file
|
|
@ -0,0 +1,167 @@
|
|||
---
|
||||
title: FreeBSD Handbook
|
||||
authors:
|
||||
- author: The FreeBSD Documentation Project
|
||||
copyright: 1995-2021 The FreeBSD Documentation Project
|
||||
releaseinfo: "$FreeBSD$"
|
||||
trademarks: ["freebsd", "ibm", "ieee", "redhat", "3com", "adobe", "apple", "intel", "linux", "microsoft", "opengroup", "sun", "realnetworks", "oracle", "3ware", "arm", "adaptec", "google", "heidelberger", "intuit", "lsilogic", "themathworks", "thomson", "vmware", "wolframresearch", "xiph", "xfree86", "general"]
|
||||
---
|
||||
|
||||
= FreeBSD Handbook
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 2
|
||||
:icons: font
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:partnums:
|
||||
:chapter-signifier: Chapter
|
||||
:part-signifier: Part
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:book: true
|
||||
:pdf: false
|
||||
:pgpkeys-path: ../../../../../
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
:chapters-path: content/en/books/handbook/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
include::../../../../shared/mirrors.adoc[]
|
||||
include::../../../../shared/authors.adoc[]
|
||||
include::../../../../shared/releases.adoc[]
|
||||
include::../../../../shared/en/mailing-lists.adoc[]
|
||||
include::../../../../shared/en/teams.adoc[]
|
||||
include::../../../../shared/en/urls.adoc[]
|
||||
:chapters-path:
|
||||
endif::[]
|
||||
|
||||
[.abstract-title]
|
||||
[abstract]
|
||||
Abstract
|
||||
|
||||
Welcome to FreeBSD! This handbook covers the installation and day to day use of _FreeBSD {rel122-current}-RELEASE_, _FreeBSD {rel121-current}-RELEASE_ and _FreeBSD {rel114-current}-RELEASE_. This book is the result of ongoing work by many individuals. Some sections might be outdated. Those interested in helping to update and expand this document should send email to the {freebsd-doc}.
|
||||
|
||||
The latest version of this book is available from the https://www.FreeBSD.org/[FreeBSD web site]. Previous versions can be obtained from https://docs.FreeBSD.org/doc/[https://docs.FreeBSD.org/doc/]. The book can be downloaded in a variety of formats and compression options from the https://download.freebsd.org/ftp/doc/[FreeBSD FTP server] or one of the numerous crossref:mirrors[mirrors-ftp,mirror sites]. Printed copies can be purchased at the https://www.freebsdmall.com/[FreeBSD Mall]. Searches can be performed on the handbook and other documents on the link:https://www.FreeBSD.org/search/[search page].
|
||||
|
||||
'''
|
||||
|
||||
toc::[]
|
||||
|
||||
:sectnums!:
|
||||
|
||||
include::{chapters-path}preface/_index.adoc[leveloffset=+1, lines=7..-1]
|
||||
|
||||
:sectnums:
|
||||
|
||||
// Section one
|
||||
include::{chapters-path}parti.adoc[lines=7..18]
|
||||
|
||||
include::{chapters-path}introduction/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}bsdinstall/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}basics/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}ports/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}x11/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
// Section two
|
||||
include::{chapters-path}partii.adoc[lines=7..18]
|
||||
|
||||
include::{chapters-path}desktop/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}multimedia/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}kernelconfig/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}printing/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}linuxemu/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}wine/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
// Section three
|
||||
include::{chapters-path}partiii.adoc[lines=7..12]
|
||||
|
||||
include::{chapters-path}config/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}boot/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}security/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}jails/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}mac/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}audit/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}disks/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}geom/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}zfs/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}filesystems/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}virtualization/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}l10n/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}cutting-edge/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}dtrace/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}usb-device-mode/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
// Section four
|
||||
include::{chapters-path}partiv.adoc[lines=7..19]
|
||||
|
||||
include::{chapters-path}serialcomms/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}ppp-and-slip/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}mail/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}network-servers/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}firewalls/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
include::{chapters-path}advanced-networking/_index.adoc[leveloffset=+1, lines=8..34;44..-1]
|
||||
|
||||
// Section five
|
||||
include::{chapters-path}partv.adoc[lines=7..8]
|
||||
|
||||
:sectnums!:
|
||||
|
||||
include::{chapters-path}mirrors/_index.adoc[leveloffset=+1, lines=8..21;30..-1]
|
||||
|
||||
include::{chapters-path}bibliography/_index.adoc[leveloffset=+1, lines=8..21;29..-1]
|
||||
|
||||
include::{chapters-path}eresources/_index.adoc[leveloffset=+1, lines=8..21;30..-1]
|
||||
|
||||
include::{chapters-path}pgpkeys/_index.adoc[leveloffset=+1, lines=8..21;31..-1]
|
||||
|
||||
:sectnums:
|
||||
415
documentation/content/en/books/handbook/boot/_index.adoc
Normal file
415
documentation/content/en/books/handbook/boot/_index.adoc
Normal file
|
|
@ -0,0 +1,415 @@
|
|||
---
|
||||
title: Chapter 13. The FreeBSD Booting Process
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/config
|
||||
next: books/handbook/security
|
||||
---
|
||||
|
||||
[[boot]]
|
||||
= The FreeBSD Booting Process
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 13
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/boot/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/boot/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/boot/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[boot-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
The process of starting a computer and loading the operating system is referred to as "the bootstrap process", or "booting". FreeBSD's boot process provides a great deal of flexibility in customizing what happens when the system starts, including the ability to select from different operating systems installed on the same computer, different versions of the same operating system, or a different installed kernel.
|
||||
|
||||
This chapter details the configuration options that can be set. It demonstrates how to customize the FreeBSD boot process, including everything that happens until the FreeBSD kernel has started, probed for devices, and started man:init[8]. This occurs when the text color of the boot messages changes from bright white to grey.
|
||||
|
||||
After reading this chapter, you will recognize:
|
||||
|
||||
* The components of the FreeBSD bootstrap system and how they interact.
|
||||
* The options that can be passed to the components in the FreeBSD bootstrap in order to control the boot process.
|
||||
* How to configure a customized boot splash screen.
|
||||
* The basics of setting device hints.
|
||||
* How to boot into single- and multi-user mode and how to properly shut down a FreeBSD system.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
This chapter only describes the boot process for FreeBSD running on x86 and amd64 systems.
|
||||
====
|
||||
|
||||
[[boot-introduction]]
|
||||
== FreeBSD Boot Process
|
||||
|
||||
Turning on a computer and starting the operating system poses an interesting dilemma. By definition, the computer does not know how to do anything until the operating system is started. This includes running programs from the disk. If the computer can not run a program from the disk without the operating system, and the operating system programs are on the disk, how is the operating system started?
|
||||
|
||||
This problem parallels one in the book The Adventures of Baron Munchausen. A character had fallen part way down a manhole, and pulled himself out by grabbing his bootstraps and lifting. In the early days of computing, the term _bootstrap_ was applied to the mechanism used to load the operating system. It has since become shortened to "booting".
|
||||
|
||||
On x86 hardware, the Basic Input/Output System (BIOS) is responsible for loading the operating system. The BIOS looks on the hard disk for the Master Boot Record (MBR), which must be located in a specific place on the disk. The BIOS has enough knowledge to load and run the MBR, and assumes that the MBR can then carry out the rest of the tasks involved in loading the operating system, possibly with the help of the BIOS.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
FreeBSD provides for booting from both the older MBR standard, and the newer GUID Partition Table (GPT). GPT partitioning is often found on computers with the Unified Extensible Firmware Interface (UEFI). However, FreeBSD can boot from GPT partitions even on machines with only a legacy BIOS with man:gptboot[8]. Work is under way to provide direct UEFI booting.
|
||||
====
|
||||
|
||||
The code within the MBR is typically referred to as a _boot manager_, especially when it interacts with the user. The boot manager usually has more code in the first track of the disk or within the file system. Examples of boot managers include the standard FreeBSD boot manager boot0, also called Boot Easy, and Grub, which is used by many Linux(R) distributions.
|
||||
|
||||
If only one operating system is installed, the MBR searches for the first bootable (active) slice on the disk, and then runs the code on that slice to load the remainder of the operating system. When multiple operating systems are present, a different boot manager can be installed to display a list of operating systems so the user can select one to boot.
|
||||
|
||||
The remainder of the FreeBSD bootstrap system is divided into three stages. The first stage knows just enough to get the computer into a specific state and run the second stage. The second stage can do a little bit more, before running the third stage. The third stage finishes the task of loading the operating system. The work is split into three stages because the MBR puts limits on the size of the programs that can be run at stages one and two. Chaining the tasks together allows FreeBSD to provide a more flexible loader.
|
||||
|
||||
The kernel is then started and begins to probe for devices and initialize them for use. Once the kernel boot process is finished, the kernel passes control to the user process man:init[8], which makes sure the disks are in a usable state, starts the user-level resource configuration which mounts file systems, sets up network cards to communicate on the network, and starts the processes which have been configured to run at startup.
|
||||
|
||||
This section describes these stages in more detail and demonstrates how to interact with the FreeBSD boot process.
|
||||
|
||||
[[boot-boot0]]
|
||||
=== The Boot Manager
|
||||
|
||||
The boot manager code in the MBR is sometimes referred to as _stage zero_ of the boot process. By default, FreeBSD uses the boot0 boot manager.
|
||||
|
||||
The MBR installed by the FreeBSD installer is based on [.filename]#/boot/boot0#. The size and capability of boot0 is restricted to 446 bytes due to the slice table and `0x55AA` identifier at the end of the MBR. If boot0 and multiple operating systems are installed, a message similar to this example will be displayed at boot time:
|
||||
|
||||
[[boot-boot0-example]]
|
||||
.[.filename]#boot0# Screenshot
|
||||
[example]
|
||||
====
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
F1 Win
|
||||
F2 FreeBSD
|
||||
|
||||
Default: F2
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
Other operating systems will overwrite an existing MBR if they are installed after FreeBSD. If this happens, or to replace the existing MBR with the FreeBSD MBR, use the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# fdisk -B -b /boot/boot0 device
|
||||
....
|
||||
|
||||
where _device_ is the boot disk, such as [.filename]#ad0# for the first IDE disk, [.filename]#ad2# for the first IDE disk on a second IDE controller, or [.filename]#da0# for the first SCSI disk. To create a custom configuration of the MBR, refer to man:boot0cfg[8].
|
||||
|
||||
[[boot-boot1]]
|
||||
=== Stage One and Stage Two
|
||||
|
||||
Conceptually, the first and second stages are part of the same program on the same area of the disk. Due to space constraints, they have been split into two, but are always installed together. They are copied from the combined [.filename]#/boot/boot# by the FreeBSD installer or `bsdlabel`.
|
||||
|
||||
These two stages are located outside file systems, in the first track of the boot slice, starting with the first sector. This is where boot0, or any other boot manager, expects to find a program to run which will continue the boot process.
|
||||
|
||||
The first stage, [.filename]#boot1#, is very simple, since it can only be 512 bytes in size. It knows just enough about the FreeBSD _bsdlabel_, which stores information about the slice, to find and execute [.filename]#boot2#.
|
||||
|
||||
Stage two, [.filename]#boot2#, is slightly more sophisticated, and understands the FreeBSD file system enough to find files. It can provide a simple interface to choose the kernel or loader to run. It runs loader, which is much more sophisticated and provides a boot configuration file. If the boot process is interrupted at stage two, the following interactive screen is displayed:
|
||||
|
||||
[[boot-boot2-example]]
|
||||
.[.filename]#boot2# Screenshot
|
||||
[example]
|
||||
====
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
>> FreeBSD/i386 BOOT
|
||||
Default: 0:ad(0,a)/boot/loader
|
||||
boot:
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
To replace the installed [.filename]#boot1# and [.filename]#boot2#, use `bsdlabel`, where _diskslice_ is the disk and slice to boot from, such as [.filename]#ad0s1# for the first slice on the first IDE disk:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# bsdlabel -B diskslice
|
||||
....
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
If just the disk name is used, such as [.filename]#ad0#, `bsdlabel` will create the disk in "dangerously dedicated mode", without slices. This is probably not the desired action, so double check the _diskslice_ before pressing kbd:[Return].
|
||||
====
|
||||
|
||||
[[boot-loader]]
|
||||
=== Stage Three
|
||||
|
||||
The loader is the final stage of the three-stage bootstrap process. It is located on the file system, usually as [.filename]#/boot/loader#.
|
||||
|
||||
The loader is intended as an interactive method for configuration, using a built-in command set, backed up by a more powerful interpreter which has a more complex command set.
|
||||
|
||||
During initialization, loader will probe for a console and for disks, and figure out which disk it is booting from. It will set variables accordingly, and an interpreter is started where user commands can be passed from a script or interactively.
|
||||
|
||||
The loader will then read [.filename]#/boot/loader.rc#, which by default reads in [.filename]#/boot/defaults/loader.conf# which sets reasonable defaults for variables and reads [.filename]#/boot/loader.conf# for local changes to those variables. [.filename]#loader.rc# then acts on these variables, loading whichever modules and kernel are selected.
|
||||
|
||||
Finally, by default, loader issues a 10 second wait for key presses, and boots the kernel if it is not interrupted. If interrupted, the user is presented with a prompt which understands the command set, where the user may adjust variables, unload all modules, load modules, and then finally boot or reboot. <<boot-loader-commands>> lists the most commonly used loader commands. For a complete discussion of all available commands, refer to man:loader[8].
|
||||
|
||||
[[boot-loader-commands]]
|
||||
.Loader Built-In Commands
|
||||
[cols="20%,80%", frame="none", options="header"]
|
||||
|===
|
||||
| Variable
|
||||
| Description
|
||||
|
||||
|autoboot _seconds_
|
||||
|Proceeds to boot the kernel if not interrupted within the time span given, in seconds. It displays a countdown, and the default time span is 10 seconds.
|
||||
|
||||
|boot [`-options`] [`kernelname`]
|
||||
|Immediately proceeds to boot the kernel, with any specified options or kernel name. Providing a kernel name on the command-line is only applicable after an `unload` has been issued. Otherwise, the previously-loaded kernel will be used. If _kernelname_ is not qualified, it will be searched under _/boot/kernel_ and _/boot/modules_.
|
||||
|
||||
|boot-conf
|
||||
|Goes through the same automatic configuration of modules based on specified variables, most commonly `kernel`. This only makes sense if `unload` is used first, before changing some variables.
|
||||
|
||||
|help [`_topic_`]
|
||||
|Shows help messages read from [.filename]#/boot/loader.help#. If the topic given is `index`, the list of available topics is displayed.
|
||||
|
||||
|include `_filename_` ...
|
||||
|Reads the specified file and interprets it line by line. An error immediately stops the `include`.
|
||||
|
||||
|load [-t ``_type_``] `_filename_`
|
||||
|Loads the kernel, kernel module, or file of the type given, with the specified filename. Any arguments after _filename_ are passed to the file. If _filename_ is not qualified, it will be searched under _/boot/kernel_ and _/boot/modules_.
|
||||
|
||||
|ls [-l] [``_path_``]
|
||||
|Displays a listing of files in the given path, or the root directory, if the path is not specified. If `-l` is specified, file sizes will also be shown.
|
||||
|
||||
|lsdev [`-v`]
|
||||
|Lists all of the devices from which it may be possible to load modules. If `-v` is specified, more details are printed.
|
||||
|
||||
|lsmod [`-v`]
|
||||
|Displays loaded modules. If `-v` is specified, more details are shown.
|
||||
|
||||
|more `_filename_`
|
||||
|Displays the files specified, with a pause at each `LINES` displayed.
|
||||
|
||||
|reboot
|
||||
|Immediately reboots the system.
|
||||
|
||||
|set `_variable_`, set `_variable=value_`
|
||||
|Sets the specified environment variables.
|
||||
|
||||
|unload
|
||||
|Removes all loaded modules.
|
||||
|===
|
||||
|
||||
Here are some practical examples of loader usage. To boot the usual kernel in single-user mode :
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
boot -s
|
||||
....
|
||||
|
||||
To unload the usual kernel and modules and then load the previous or another, specified kernel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
unload
|
||||
load /path/to/kernelfile
|
||||
....
|
||||
|
||||
Use the qualified [.filename]#/boot/GENERIC/kernel# to refer to the default kernel that comes with an installation, or [.filename]#/boot/kernel.old/kernel#, to refer to the previously installed kernel before a system upgrade or before configuring a custom kernel.
|
||||
|
||||
Use the following to load the usual modules with another kernel. Note that in this case it is not necessary the qualified name:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
unload
|
||||
set kernel="mykernel"
|
||||
boot-conf
|
||||
....
|
||||
|
||||
To load an automated kernel configuration script:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
load -t userconfig_script /boot/kernel.conf
|
||||
....
|
||||
|
||||
[[boot-init]]
|
||||
=== Last Stage
|
||||
|
||||
Once the kernel is loaded by either loader or by boot2, which bypasses loader, it examines any boot flags and adjusts its behavior as necessary. <<boot-kernel>> lists the commonly used boot flags. Refer to man:boot[8] for more information on the other boot flags.
|
||||
|
||||
[[boot-kernel]]
|
||||
.Kernel Interaction During Boot
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Option
|
||||
| Description
|
||||
|
||||
|`-a`
|
||||
|During kernel initialization, ask for the device to mount as the root file system.
|
||||
|
||||
|`-C`
|
||||
|Boot the root file system from a CDROM.
|
||||
|
||||
|`-s`
|
||||
|Boot into single-user mode.
|
||||
|
||||
|`-v`
|
||||
|Be more verbose during kernel startup.
|
||||
|===
|
||||
|
||||
Once the kernel has finished booting, it passes control to the user process man:init[8], which is located at [.filename]#/sbin/init#, or the program path specified in the `init_path` variable in `loader`. This is the last stage of the boot process.
|
||||
|
||||
The boot sequence makes sure that the file systems available on the system are consistent. If a UFS file system is not, and `fsck` cannot fix the inconsistencies, init drops the system into single-user mode so that the system administrator can resolve the problem directly. Otherwise, the system boots into multi-user mode.
|
||||
|
||||
[[boot-singleuser]]
|
||||
==== Single-User Mode
|
||||
|
||||
A user can specify this mode by booting with `-s` or by setting the `boot_single` variable in loader. It can also be reached by running `shutdown now` from multi-user mode. Single-user mode begins with this message:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
Enter full pathname of shell or RETURN for /bin/sh:
|
||||
....
|
||||
|
||||
If the user presses kbd:[Enter], the system will enter the default Bourne shell. To specify a different shell, input the full path to the shell.
|
||||
|
||||
Single-user mode is usually used to repair a system that will not boot due to an inconsistent file system or an error in a boot configuration file. It can also be used to reset the `root` password when it is unknown. These actions are possible as the single-user mode prompt gives full, local access to the system and its configuration files. There is no networking in this mode.
|
||||
|
||||
While single-user mode is useful for repairing a system, it poses a security risk unless the system is in a physically secure location. By default, any user who can gain physical access to a system will have full control of that system after booting into single-user mode.
|
||||
|
||||
If the system `console` is changed to `insecure` in [.filename]#/etc/ttys#, the system will first prompt for the `root` password before initiating single-user mode. This adds a measure of security while removing the ability to reset the `root` password when it is unknown.
|
||||
|
||||
[[boot-insecure-console]]
|
||||
.Configuring an Insecure Console in [.filename]#/etc/ttys#
|
||||
[example]
|
||||
====
|
||||
[.programlisting]
|
||||
....
|
||||
# name getty type status comments
|
||||
#
|
||||
# If console is marked "insecure", then init will ask for the root password
|
||||
# when going to single-user mode.
|
||||
console none unknown off insecure
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
An `insecure` console means that physical security to the console is considered to be insecure, so only someone who knows the `root` password may use single-user mode.
|
||||
|
||||
[[boot-multiuser]]
|
||||
==== Multi-User Mode
|
||||
|
||||
If init finds the file systems to be in order, or once the user has finished their commands in single-user mode and has typed `exit` to leave single-user mode, the system enters multi-user mode, in which it starts the resource configuration of the system.
|
||||
|
||||
The resource configuration system reads in configuration defaults from [.filename]#/etc/defaults/rc.conf# and system-specific details from [.filename]#/etc/rc.conf#. It then proceeds to mount the system file systems listed in [.filename]#/etc/fstab#. It starts up networking services, miscellaneous system daemons, then the startup scripts of locally installed packages.
|
||||
|
||||
To learn more about the resource configuration system, refer to man:rc[8] and examine the scripts located in [.filename]#/etc/rc.d#.
|
||||
|
||||
[[boot-splash]]
|
||||
== Configuring Boot Time Splash Screens
|
||||
|
||||
Typically when a FreeBSD system boots, it displays its progress as a series of messages at the console. A boot splash screen creates an alternate boot screen that hides all of the boot probe and service startup messages. A few boot loader messages, including the boot options menu and a timed wait countdown prompt, are displayed at boot time, even when the splash screen is enabled. The display of the splash screen can be turned off by hitting any key on the keyboard during the boot process.
|
||||
|
||||
There are two basic environments available in FreeBSD. The first is the default legacy virtual console command line environment. After the system finishes booting, a console login prompt is presented. The second environment is a configured graphical environment. Refer to crossref:x11[x11,The X Window System] for more information on how to install and configure a graphical display manager and a graphical login manager.
|
||||
|
||||
Once the system has booted, the splash screen defaults to being a screen saver. After a time period of non-use, the splash screen will display and will cycle through steps of changing intensity of the image, from bright to very dark and over again. The configuration of the splash screen saver can be overridden by adding a `saver=` line to [.filename]#/etc/rc.conf#. Several built-in screen savers are available and described in man:splash[4]. The `saver=` option only applies to virtual consoles and has no effect on graphical display managers.
|
||||
|
||||
By installing the package:sysutils/bsd-splash-changer[] package or port, a random splash image from a collection will display at boot. The splash screen function supports 256-colors in the bitmap ([.filename]#.bmp#), ZSoft PCX ([.filename]#.pcx#), or TheDraw ([.filename]#.bin#) formats. The [.filename]#.bmp#, [.filename]#.pcx#, or [.filename]#.bin# image has to be placed on the root partition, for example in [.filename]#/boot#. The splash image files must have a resolution of 320 by 200 pixels or less in order to work on standard VGA adapters. For the default boot display resolution of 256-colors and 320 by 200 pixels or less, add the following lines to [.filename]#/boot/loader.conf#. Replace _splash.bmp_ with the name of the bitmap file to use:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
splash_bmp_load="YES"
|
||||
bitmap_load="YES"
|
||||
bitmap_name="/boot/splash.bmp"
|
||||
....
|
||||
|
||||
To use a PCX file instead of a bitmap file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
splash_pcx_load="YES"
|
||||
bitmap_load="YES"
|
||||
bitmap_name="/boot/splash.pcx"
|
||||
....
|
||||
|
||||
To instead use ASCII art in the https://en.wikipedia.org/wiki/TheDraw[https://en.wikipedia.org/wiki/TheDraw] format:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
splash_txt="YES"
|
||||
bitmap_load="YES"
|
||||
bitmap_name="/boot/splash.bin"
|
||||
....
|
||||
|
||||
Other interesting [.filename]#loader.conf# options include:
|
||||
|
||||
`beastie_disable="YES"`::
|
||||
This will stop the boot options menu from being displayed, but the timed wait count down prompt will still be present. Even with the display of the boot options menu disabled, entering an option selection at the timed wait count down prompt will enact the corresponding boot option.
|
||||
|
||||
`loader_logo="beastie"`::
|
||||
This will replace the default words "FreeBSD", which are displayed to the right of the boot options menu, with the colored beastie logo.
|
||||
|
||||
For more information, refer to man:splash[4], man:loader.conf[5], and man:vga[4].
|
||||
|
||||
[[device-hints]]
|
||||
== Device Hints
|
||||
|
||||
During initial system startup, the boot man:loader[8] reads man:device.hints[5]. This file stores kernel boot information known as variables, sometimes referred to as "device hints". These "device hints" are used by device drivers for device configuration.
|
||||
|
||||
Device hints may also be specified at the Stage 3 boot loader prompt, as demonstrated in <<boot-loader>>. Variables can be added using `set`, removed with `unset`, and viewed `show`. Variables set in [.filename]#/boot/device.hints# can also be overridden. Device hints entered at the boot loader are not permanent and will not be applied on the next reboot.
|
||||
|
||||
Once the system is booted, man:kenv[1] can be used to dump all of the variables.
|
||||
|
||||
The syntax for [.filename]#/boot/device.hints# is one variable per line, using the hash "#" as comment markers. Lines are constructed as follows:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
hint.driver.unit.keyword="value"
|
||||
....
|
||||
|
||||
The syntax for the Stage 3 boot loader is:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
set hint.driver.unit.keyword=value
|
||||
....
|
||||
|
||||
where `driver` is the device driver name, `unit` is the device driver unit number, and `keyword` is the hint keyword. The keyword may consist of the following options:
|
||||
|
||||
* `at`: specifies the bus which the device is attached to.
|
||||
* `port`: specifies the start address of the I/O to be used.
|
||||
* `irq`: specifies the interrupt request number to be used.
|
||||
* `drq`: specifies the DMA channel number.
|
||||
* `maddr`: specifies the physical memory address occupied by the device.
|
||||
* `flags`: sets various flag bits for the device.
|
||||
* `disabled`: if set to `1` the device is disabled.
|
||||
|
||||
Since device drivers may accept or require more hints not listed here, viewing a driver's manual page is recommended. For more information, refer to man:device.hints[5], man:kenv[1], man:loader.conf[5], and man:loader[8].
|
||||
|
||||
[[boot-shutdown]]
|
||||
== Shutdown Sequence
|
||||
|
||||
Upon controlled shutdown using man:shutdown[8], man:init[8] will attempt to run the script [.filename]#/etc/rc.shutdown#, and then proceed to send all processes the `TERM` signal, and subsequently the `KILL` signal to any that do not terminate in a timely manner.
|
||||
|
||||
To power down a FreeBSD machine on architectures and systems that support power management, use `shutdown -p now` to turn the power off immediately. To reboot a FreeBSD system, use `shutdown -r now`. One must be `root` or a member of `operator` in order to run man:shutdown[8]. One can also use man:halt[8] and man:reboot[8]. Refer to their manual pages and to man:shutdown[8] for more information.
|
||||
|
||||
Modify group membership by referring to crossref:basics[users-synopsis,“Users and Basic Account Management”].
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Power management requires man:acpi[4] to be loaded as a module or statically compiled into a custom kernel.
|
||||
====
|
||||
1067
documentation/content/en/books/handbook/bsdinstall/_index.adoc
Normal file
1067
documentation/content/en/books/handbook/bsdinstall/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
42
documentation/content/en/books/handbook/chapters-order.adoc
Normal file
42
documentation/content/en/books/handbook/chapters-order.adoc
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
preface/_index.adoc
|
||||
parti.adoc
|
||||
introduction/_index.adoc
|
||||
bsdinstall/_index.adoc
|
||||
basics/_index.adoc
|
||||
ports/_index.adoc
|
||||
x11/_index.adoc
|
||||
partii.adoc
|
||||
desktop/_index.adoc
|
||||
multimedia/_index.adoc
|
||||
kernelconfig/_index.adoc
|
||||
printing/_index.adoc
|
||||
linuxemu/_index.adoc
|
||||
wine/_index.adoc
|
||||
partiii.adoc
|
||||
config/_index.adoc
|
||||
boot/_index.adoc
|
||||
security/_index.adoc
|
||||
jails/_index.adoc
|
||||
mac/_index.adoc
|
||||
audit/_index.adoc
|
||||
disks/_index.adoc
|
||||
geom/_index.adoc
|
||||
zfs/_index.adoc
|
||||
filesystems/_index.adoc
|
||||
virtualization/_index.adoc
|
||||
l10n/_index.adoc
|
||||
cutting-edge/_index.adoc
|
||||
dtrace/_index.adoc
|
||||
usb-device-mode/_index.adoc
|
||||
partiv.adoc
|
||||
serialcomms/_index.adoc
|
||||
ppp-and-slip/_index.adoc
|
||||
mail/_index.adoc
|
||||
network-servers/_index.adoc
|
||||
firewalls/_index.adoc
|
||||
advanced-networking/_index.adoc
|
||||
partv.adoc
|
||||
mirrors/_index.adoc
|
||||
bibliography/_index.adoc
|
||||
eresources/_index.adoc
|
||||
pgpkeys/_index.adoc
|
||||
1520
documentation/content/en/books/handbook/config/_index.adoc
Normal file
1520
documentation/content/en/books/handbook/config/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
918
documentation/content/en/books/handbook/cutting-edge/_index.adoc
Normal file
918
documentation/content/en/books/handbook/cutting-edge/_index.adoc
Normal file
|
|
@ -0,0 +1,918 @@
|
|||
---
|
||||
title: Chapter 24. Updating and Upgrading FreeBSD
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/l10n
|
||||
next: books/handbook/dtrace
|
||||
---
|
||||
|
||||
[[updating-upgrading]]
|
||||
= Updating and Upgrading FreeBSD
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 24
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/cutting-edge/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/cutting-edge/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/cutting-edge/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[updating-upgrading-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
FreeBSD is under constant development between releases. Some people prefer to use the officially released versions, while others prefer to keep in sync with the latest developments. However, even official releases are often updated with security and other critical fixes. Regardless of the version used, FreeBSD provides all the necessary tools to keep the system updated, and allows for easy upgrades between versions. This chapter describes how to track the development system and the basic tools for keeping a FreeBSD system up-to-date.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* How to keep a FreeBSD system up-to-date with freebsd-update or Subversion.
|
||||
* How to compare the state of an installed system against a known pristine copy.
|
||||
* How to keep the installed documentation up-to-date with Subversion or documentation ports.
|
||||
* The difference between the two development branches: FreeBSD-STABLE and FreeBSD-CURRENT.
|
||||
* How to rebuild and reinstall the entire base system.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Properly set up the network connection (crossref:advanced-networking[advanced-networking,Advanced Networking]).
|
||||
* Know how to install additional third-party software (crossref:ports[ports,Installing Applications: Packages and Ports]).
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Throughout this chapter, `svnlite` is used to obtain and update FreeBSD sources. Optionally, the package:devel/subversion[] port or package may be used.
|
||||
====
|
||||
|
||||
[[updating-upgrading-freebsdupdate]]
|
||||
== FreeBSD Update
|
||||
|
||||
Applying security patches in a timely manner and upgrading to a newer release of an operating system are important aspects of ongoing system administration. FreeBSD includes a utility called `freebsd-update` which can be used to perform both these tasks.
|
||||
|
||||
This utility supports binary security and errata updates to FreeBSD, without the need to manually compile and install the patch or a new kernel. Binary updates are available for all architectures and releases currently supported by the security team. The list of supported releases and their estimated end-of-life dates are listed at https://www.FreeBSD.org/security/[https://www.FreeBSD.org/security/].
|
||||
|
||||
This utility also supports operating system upgrades to minor point releases as well as upgrades to another release branch. Before upgrading to a new release, review its release announcement as it contains important information pertinent to the release. Release announcements are available from https://www.FreeBSD.org/releases/[https://www.FreeBSD.org/releases/].
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
If a `crontab` utilizing the features of man:freebsd-update[8] exists, it must be disabled before upgrading the operating system.
|
||||
====
|
||||
|
||||
This section describes the configuration file used by `freebsd-update`, demonstrates how to apply a security patch and how to upgrade to a minor or major operating system release, and discusses some of the considerations when upgrading the operating system.
|
||||
|
||||
[[freebsdupdate-config-file]]
|
||||
=== The Configuration File
|
||||
|
||||
The default configuration file for `freebsd-update` works as-is. Some users may wish to tweak the default configuration in [.filename]#/etc/freebsd-update.conf#, allowing better control of the process. The comments in this file explain the available options, but the following may require a bit more explanation:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# Components of the base system which should be kept updated.
|
||||
Components world kernel
|
||||
....
|
||||
|
||||
This parameter controls which parts of FreeBSD will be kept up-to-date. The default is to update the entire base system and the kernel. Individual components can instead be specified, such as `src/base` or `src/sys`. However, the best option is to leave this at the default as changing it to include specific items requires every needed item to be listed. Over time, this could have disastrous consequences as source code and binaries may become out of sync.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# Paths which start with anything matching an entry in an IgnorePaths
|
||||
# statement will be ignored.
|
||||
IgnorePaths /boot/kernel/linker.hints
|
||||
....
|
||||
|
||||
To leave specified directories, such as [.filename]#/bin# or [.filename]#/sbin#, untouched during the update process, add their paths to this statement. This option may be used to prevent `freebsd-update` from overwriting local modifications.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# Paths which start with anything matching an entry in an UpdateIfUnmodified
|
||||
# statement will only be updated if the contents of the file have not been
|
||||
# modified by the user (unless changes are merged; see below).
|
||||
UpdateIfUnmodified /etc/ /var/ /root/ /.cshrc /.profile
|
||||
....
|
||||
|
||||
This option will only update unmodified configuration files in the specified directories. Any changes made by the user will prevent the automatic updating of these files. There is another option, `KeepModifiedMetadata`, which will instruct `freebsd-update` to save the changes during the merge.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# When upgrading to a new FreeBSD release, files which match MergeChanges
|
||||
# will have any local changes merged into the version from the new release.
|
||||
MergeChanges /etc/ /var/named/etc/ /boot/device.hints
|
||||
....
|
||||
|
||||
List of directories with configuration files that `freebsd-update` should attempt to merge. The file merge process is a series of man:diff[1] patches similar to man:mergemaster[8], but with fewer options. Merges are either accepted, open an editor, or cause `freebsd-update` to abort. When in doubt, backup [.filename]#/etc# and just accept the merges. See man:mergemaster[8] for more information about `mergemaster`.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# Directory in which to store downloaded updates and temporary
|
||||
# files used by FreeBSD Update.
|
||||
# WorkDir /var/db/freebsd-update
|
||||
....
|
||||
|
||||
This directory is where all patches and temporary files are placed. In cases where the user is doing a version upgrade, this location should have at least a gigabyte of disk space available.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# When upgrading between releases, should the list of Components be
|
||||
# read strictly (StrictComponents yes) or merely as a list of components
|
||||
# which *might* be installed of which FreeBSD Update should figure out
|
||||
# which actually are installed and upgrade those (StrictComponents no)?
|
||||
# StrictComponents no
|
||||
....
|
||||
|
||||
When this option is set to `yes`, `freebsd-update` will assume that the `Components` list is complete and will not attempt to make changes outside of the list. Effectively, `freebsd-update` will attempt to update every file which belongs to the `Components` list.
|
||||
|
||||
[[freebsdupdate-security-patches]]
|
||||
=== Applying Security Patches
|
||||
|
||||
The process of applying FreeBSD security patches has been simplified, allowing an administrator to keep a system fully patched using `freebsd-update`. More information about FreeBSD security advisories can be found in crossref:security[security-advisories,"FreeBSD Security Advisories"].
|
||||
|
||||
FreeBSD security patches may be downloaded and installed using the following commands. The first command will determine if any outstanding patches are available, and if so, will list the files that will be modifed if the patches are applied. The second command will apply the patches.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update fetch
|
||||
# freebsd-update install
|
||||
....
|
||||
|
||||
If the update applies any kernel patches, the system will need a reboot in order to boot into the patched kernel. If the patch was applied to any running binaries, the affected applications should be restarted so that the patched version of the binary is used.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Usually, the user needs to be prepared to reboot the system. To know if a reboot is required by a kernel update, execute the commands `freebsd-version -k` and `uname -r` and if it differs a reboot is required.
|
||||
====
|
||||
|
||||
The system can be configured to automatically check for updates once every day by adding this entry to [.filename]#/etc/crontab#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
@daily root freebsd-update cron
|
||||
....
|
||||
|
||||
If patches exist, they will automatically be downloaded but will not be applied. The `root` user will be sent an email so that the patches may be reviewed and manually installed with `freebsd-update install`.
|
||||
|
||||
If anything goes wrong, `freebsd-update` has the ability to roll back the last set of changes with the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update rollback
|
||||
Uninstalling updates... done.
|
||||
....
|
||||
|
||||
Again, the system should be restarted if the kernel or any kernel modules were modified and any affected binaries should be restarted.
|
||||
|
||||
Only the [.filename]#GENERIC# kernel can be automatically updated by `freebsd-update`. If a custom kernel is installed, it will have to be rebuilt and reinstalled after `freebsd-update` finishes installing the updates. The default kernel name is _GENERIC_. The man:uname[1] command may be used to verify its installation.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Always keep a copy of the [.filename]#GENERIC# kernel in [.filename]#/boot/GENERIC#. It will be helpful in diagnosing a variety of problems and in performing version upgrades. Refer to <<freebsd-update-custom-kernel-9x>> for instructions on how to get a copy of the [.filename]#GENERIC# kernel.
|
||||
====
|
||||
|
||||
Unless the default configuration in [.filename]#/etc/freebsd-update.conf# has been changed, `freebsd-update` will install the updated kernel sources along with the rest of the updates. Rebuilding and reinstalling a new custom kernel can then be performed in the usual way.
|
||||
|
||||
The updates distributed by `freebsd-update` do not always involve the kernel. It is not necessary to rebuild a custom kernel if the kernel sources have not been modified by `freebsd-update install`. However, `freebsd-update` will always update [.filename]#/usr/src/sys/conf/newvers.sh#. The current patch level, as indicated by the `-p` number reported by `uname -r`, is obtained from this file. Rebuilding a custom kernel, even if nothing else changed, allows `uname` to accurately report the current patch level of the system. This is particularly helpful when maintaining multiple systems, as it allows for a quick assessment of the updates installed in each one.
|
||||
|
||||
[[freebsdupdate-upgrade]]
|
||||
=== Performing Major and Minor Version Upgrades
|
||||
|
||||
Upgrades from one minor version of FreeBSD to another, like from FreeBSD 9.0 to FreeBSD 9.1, are called _minor version_ upgrades. _Major version_ upgrades occur when FreeBSD is upgraded from one major version to another, like from FreeBSD 9.X to FreeBSD 10.X. Both types of upgrades can be performed by providing `freebsd-update` with a release version target.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
If the system is running a custom kernel, make sure that a copy of the [.filename]#GENERIC# kernel exists in [.filename]#/boot/GENERIC# before starting the upgrade. Refer to <<freebsd-update-custom-kernel-9x>> for instructions on how to get a copy of the [.filename]#GENERIC# kernel.
|
||||
====
|
||||
|
||||
The following command, when run on a FreeBSD 9.0 system, will upgrade it to FreeBSD 9.1:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update -r 9.1-RELEASE upgrade
|
||||
....
|
||||
|
||||
After the command has been received, `freebsd-update` will evaluate the configuration file and current system in an attempt to gather the information necessary to perform the upgrade. A screen listing will display which components have and have not been detected. For example:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Looking up update.FreeBSD.org mirrors... 1 mirrors found.
|
||||
Fetching metadata signature for 9.0-RELEASE from update1.FreeBSD.org... done.
|
||||
Fetching metadata index... done.
|
||||
Inspecting system... done.
|
||||
|
||||
The following components of FreeBSD seem to be installed:
|
||||
kernel/smp src/base src/bin src/contrib src/crypto src/etc src/games
|
||||
src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue
|
||||
src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin
|
||||
world/base world/info world/lib32 world/manpages
|
||||
|
||||
The following components of FreeBSD do not seem to be installed:
|
||||
kernel/generic world/catpages world/dict world/doc world/games
|
||||
world/proflibs
|
||||
|
||||
Does this look reasonable (y/n)? y
|
||||
....
|
||||
|
||||
At this point, `freebsd-update` will attempt to download all files required for the upgrade. In some cases, the user may be prompted with questions regarding what to install or how to proceed.
|
||||
|
||||
When using a custom kernel, the above step will produce a warning similar to the following:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
WARNING: This system is running a "MYKERNEL" kernel, which is not a
|
||||
kernel configuration distributed as part of FreeBSD 9.0-RELEASE.
|
||||
This kernel will not be updated: you MUST update the kernel manually
|
||||
before running "/usr/sbin/freebsd-update install"
|
||||
....
|
||||
|
||||
This warning may be safely ignored at this point. The updated [.filename]#GENERIC# kernel will be used as an intermediate step in the upgrade process.
|
||||
|
||||
Once all the patches have been downloaded to the local system, they will be applied. This process may take a while, depending on the speed and workload of the machine. Configuration files will then be merged. The merging process requires some user intervention as a file may be merged or an editor may appear on screen for a manual merge. The results of every successful merge will be shown to the user as the process continues. A failed or ignored merge will cause the process to abort. Users may wish to make a backup of [.filename]#/etc# and manually merge important files, such as [.filename]#master.passwd# or [.filename]#group# at a later time.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The system is not being altered yet as all patching and merging is happening in another directory. Once all patches have been applied successfully, all configuration files have been merged and it seems the process will go smoothly, the changes can be committed to disk by the user using the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update install
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
The kernel and kernel modules will be patched first. If the system is running with a custom kernel, use man:nextboot[8] to set the kernel for the next boot to the updated [.filename]#/boot/GENERIC#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# nextboot -k GENERIC
|
||||
....
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
Before rebooting with the [.filename]#GENERIC# kernel, make sure it contains all the drivers required for the system to boot properly and connect to the network, if the machine being updated is accessed remotely. In particular, if the running custom kernel contains built-in functionality usually provided by kernel modules, make sure to temporarily load these modules into the [.filename]#GENERIC# kernel using the [.filename]#/boot/loader.conf# facility. It is recommended to disable non-essential services as well as any disk and network mounts until the upgrade process is complete.
|
||||
====
|
||||
|
||||
The machine should now be restarted with the updated kernel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# shutdown -r now
|
||||
....
|
||||
|
||||
Once the system has come back online, restart `freebsd-update` using the following command. Since the state of the process has been saved, `freebsd-update` will not start from the beginning, but will instead move on to the next phase and remove all old shared libraries and object files.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update install
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Depending upon whether any library version numbers were bumped, there may only be two install phases instead of three.
|
||||
====
|
||||
|
||||
The upgrade is now complete. If this was a major version upgrade, reinstall all ports and packages as described in <<freebsdupdate-portsrebuild>>.
|
||||
|
||||
[[freebsd-update-custom-kernel-9x]]
|
||||
==== Custom Kernels with FreeBSD 9.X and Later
|
||||
|
||||
Before using `freebsd-update`, ensure that a copy of the [.filename]#GENERIC# kernel exists in [.filename]#/boot/GENERIC#. If a custom kernel has only been built once, the kernel in [.filename]#/boot/kernel.old# is the `GENERIC` kernel. Simply rename this directory to [.filename]#/boot/GENERIC#.
|
||||
|
||||
If a custom kernel has been built more than once or if it is unknown how many times the custom kernel has been built, obtain a copy of the `GENERIC` kernel that matches the current version of the operating system. If physical access to the system is available, a copy of the `GENERIC` kernel can be installed from the installation media:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mount /cdrom
|
||||
# cd /cdrom/usr/freebsd-dist
|
||||
# tar -C/ -xvf kernel.txz boot/kernel/kernel
|
||||
....
|
||||
|
||||
Alternately, the `GENERIC` kernel may be rebuilt and installed from source:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
# make kernel __MAKE_CONF=/dev/null SRCCONF=/dev/null
|
||||
....
|
||||
|
||||
For this kernel to be identified as the `GENERIC` kernel by `freebsd-update`, the [.filename]#GENERIC# configuration file must not have been modified in any way. It is also suggested that the kernel is built without any other special options.
|
||||
|
||||
Rebooting into the [.filename]#GENERIC# kernel is not required as `freebsd-update` only needs [.filename]#/boot/GENERIC# to exist.
|
||||
|
||||
[[freebsdupdate-portsrebuild]]
|
||||
==== Upgrading Packages After a Major Version Upgrade
|
||||
|
||||
Generally, installed applications will continue to work without problems after minor version upgrades. Major versions use different Application Binary Interfaces (ABIs), which will break most third-party applications. After a major version upgrade, all installed packages and ports need to be upgraded. Packages can be upgraded using `pkg upgrade`. To upgrade installed ports, use a utility such as package:ports-mgmt/portmaster[].
|
||||
|
||||
A forced upgrade of all installed packages will replace the packages with fresh versions from the repository even if the version number has not increased. This is required because of the ABI version change when upgrading between major versions of FreeBSD. The forced upgrade can be accomplished by performing:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg-static upgrade -f
|
||||
....
|
||||
|
||||
A rebuild of all installed applications can be accomplished with this command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# portmaster -af
|
||||
....
|
||||
|
||||
This command will display the configuration screens for each application that has configurable options and wait for the user to interact with those screens. To prevent this behavior, and use only the default options, include `-G` in the above command.
|
||||
|
||||
Once the software upgrades are complete, finish the upgrade process with a final call to `freebsd-update` in order to tie up all the loose ends in the upgrade process:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update install
|
||||
....
|
||||
|
||||
If the [.filename]#GENERIC# kernel was temporarily used, this is the time to build and install a new custom kernel using the instructions in crossref:kernelconfig[kernelconfig,Configuring the FreeBSD Kernel].
|
||||
|
||||
Reboot the machine into the new FreeBSD version. The upgrade process is now complete.
|
||||
|
||||
[[freebsdupdate-system-comparison]]
|
||||
=== System State Comparison
|
||||
|
||||
The state of the installed FreeBSD version against a known good copy can be tested using `freebsd-update IDS`. This command evaluates the current version of system utilities, libraries, and configuration files and can be used as a built-in Intrusion Detection System (IDS).
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
This command is not a replacement for a real IDS such as package:security/snort[]. As `freebsd-update` stores data on disk, the possibility of tampering is evident. While this possibility may be reduced using `kern.securelevel` and by storing the `freebsd-update` data on a read-only file system when not in use, a better solution would be to compare the system against a secure disk, such as a DVD or securely stored external USB disk device. An alternative method for providing IDS functionality using a built-in utility is described in crossref:security[security-ids,"Binary Verification"]
|
||||
====
|
||||
|
||||
To begin the comparison, specify the output file to save the results to:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# freebsd-update IDS >> outfile.ids
|
||||
....
|
||||
|
||||
The system will now be inspected and a lengthy listing of files, along with the SHA256 hash values for both the known value in the release and the current installation, will be sent to the specified output file.
|
||||
|
||||
The entries in the listing are extremely long, but the output format may be easily parsed. For instance, to obtain a list of all files which differ from those in the release, issue the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cat outfile.ids | awk '{ print $1 }' | more
|
||||
/etc/master.passwd
|
||||
/etc/motd
|
||||
/etc/passwd
|
||||
/etc/pf.conf
|
||||
....
|
||||
|
||||
This sample output has been truncated as many more files exist. Some files have natural modifications. For example, [.filename]#/etc/passwd# will be modified if users have been added to the system. Kernel modules may differ as `freebsd-update` may have updated them. To exclude specific files or directories, add them to the `IDSIgnorePaths` option in [.filename]#/etc/freebsd-update.conf#.
|
||||
|
||||
[[updating-upgrading-documentation]]
|
||||
== Updating the Documentation Set
|
||||
|
||||
Documentation is an integral part of the FreeBSD operating system. While an up-to-date version of the FreeBSD documentation is always available on the FreeBSD web site (link:https://www.FreeBSD.org/doc/[https://www.freebsd.org/doc/]), it can be handy to have an up-to-date, local copy of the FreeBSD website, handbooks, FAQ, and articles.
|
||||
|
||||
This section describes how to use either source or the FreeBSD Ports Collection to keep a local copy of the FreeBSD documentation up-to-date.
|
||||
|
||||
For information on editing and submitting corrections to the documentation, refer to the FreeBSD Documentation Project Primer for New Contributors (link:{fdp-primer}[FreeBSD Documentation Project Primer for New Contributors]).
|
||||
|
||||
[[updating-installed-documentation]]
|
||||
=== Updating Documentation from Source
|
||||
|
||||
Rebuilding the FreeBSD documentation from source requires a collection of tools which are not part of the FreeBSD base system. The required tools can be installed from the package:textproc/docproj[] package or port developed by the FreeBSD Documentation Project.
|
||||
|
||||
Once installed, use svnlite to fetch a clean copy of the documentation source:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svnlite checkout https://svn.FreeBSD.org/doc/head /usr/doc
|
||||
....
|
||||
|
||||
The initial download of the documentation sources may take a while. Let it run until it completes.
|
||||
|
||||
Future updates of the documentation sources may be fetched by running:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svnlite update /usr/doc
|
||||
....
|
||||
|
||||
Once an up-to-date snapshot of the documentation sources has been fetched to [.filename]#/usr/doc#, everything is ready for an update of the installed documentation.
|
||||
|
||||
A full update of all available languages may be performed by typing:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/doc
|
||||
# make install clean
|
||||
....
|
||||
|
||||
If an update of only a specific language is desired, `make` can be invoked in a language-specific subdirectory of [.filename]#/usr/doc#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/doc/en_US.ISO8859-1
|
||||
# make install clean
|
||||
....
|
||||
|
||||
An alternative way of updating the documentation is to run this command from [.filename]#/usr/doc# or the desired language-specific subdirectory:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make update
|
||||
....
|
||||
|
||||
The output formats that will be installed may be specified by setting `FORMATS`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/doc
|
||||
# make FORMATS='html html-split' install clean
|
||||
....
|
||||
|
||||
Several options are available to ease the process of updating only parts of the documentation, or the build of specific translations. These options can be set either as system-wide options in [.filename]#/etc/make.conf#, or as command-line options passed to `make`.
|
||||
|
||||
The options include:
|
||||
|
||||
`DOC_LANG`::
|
||||
The list of languages and encodings to build and install, such as `en_US.ISO8859-1` for English documentation.
|
||||
|
||||
`FORMATS`::
|
||||
A single format or a list of output formats to be built. Currently, `html`, `html-split`, `txt`, `ps`, and `pdf` are supported.
|
||||
|
||||
`DOCDIR`::
|
||||
Where to install the documentation. It defaults to [.filename]#/usr/shared/doc#.
|
||||
|
||||
For more `make` variables supported as system-wide options in FreeBSD, refer to man:make.conf[5].
|
||||
|
||||
[[doc-ports-install-package]]
|
||||
=== Updating Documentation from Ports
|
||||
|
||||
The previous section presented a method for updating the FreeBSD documentation from sources. This section describes an alternative method which uses the Ports Collection and makes it possible to:
|
||||
|
||||
* Install pre-built packages of the documentation, without having to locally build anything or install the documentation toolchain.
|
||||
* Build the documentation sources through the ports framework, making the checkout and build steps a bit easier.
|
||||
|
||||
This method of updating the FreeBSD documentation is supported by a set of documentation ports and packages which are updated by the {doceng} on a monthly basis. These are listed in the FreeBSD Ports Collection, under the docs category (http://www.freshports.org/docs/[http://www.freshports.org/docs/]).
|
||||
|
||||
Organization of the documentation ports is as follows:
|
||||
|
||||
* The package:misc/freebsd-doc-en[] package or port installs all of the English documentation.
|
||||
* The package:misc/freebsd-doc-all[] meta-package or port installs all documentation in all available languages.
|
||||
* There is a package and port for each translation, such as package:misc/freebsd-doc-hu[] for the Hungarian documentation.
|
||||
|
||||
When binary packages are used, the FreeBSD documentation will be installed in all available formats for the given language. For example, the following command will install the latest package of the Hungarian documentation:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install hu-freebsd-doc
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Packages use a format that differs from the corresponding port's name: `_lang_-freebsd-doc`, where _lang_ is the short format of the language code, such as `hu` for Hungarian, or `zh_cn` for Simplified Chinese.
|
||||
====
|
||||
|
||||
To specify the format of the documentation, build the port instead of installing the package. For example, to build and install the English documentation:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/misc/freebsd-doc-en
|
||||
# make install clean
|
||||
....
|
||||
|
||||
The port provides a configuration menu where the format to build and install can be specified. By default, split HTML, similar to the format used on http://www.FreeBSD.org[http://www.FreeBSD.org], and PDF are selected.
|
||||
|
||||
Alternately, several `make` options can be specified when building a documentation port, including:
|
||||
|
||||
`WITH_HTML`::
|
||||
Builds the HTML format with a single HTML file per document. The formatted documentation is saved to a file called [.filename]#article.html#, or [.filename]#book.html#.
|
||||
|
||||
`WITH_PDF`::
|
||||
The formatted documentation is saved to a file called [.filename]#article.pdf# or [.filename]#book.pdf#.
|
||||
|
||||
`DOCBASE`::
|
||||
Specifies where to install the documentation. It defaults to [.filename]#/usr/local/shared/doc/freebsd#.
|
||||
|
||||
This example uses variables to install the Hungarian documentation as a PDF in the specified directory:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/misc/freebsd-doc-hu
|
||||
# make -DWITH_PDF DOCBASE=share/doc/freebsd/hu install clean
|
||||
....
|
||||
|
||||
Documentation packages or ports can be updated using the instructions in crossref:ports[ports,Installing Applications: Packages and Ports]. For example, the following command updates the installed Hungarian documentation using package:ports-mgmt/portmaster[] by using packages only:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# portmaster -PP hu-freebsd-doc
|
||||
....
|
||||
|
||||
[[current-stable]]
|
||||
== Tracking a Development Branch
|
||||
|
||||
FreeBSD has two development branches: FreeBSD-CURRENT and FreeBSD-STABLE.
|
||||
|
||||
This section provides an explanation of each branch and its intended audience, as well as how to keep a system up-to-date with each respective branch.
|
||||
|
||||
[[current]]
|
||||
=== Using FreeBSD-CURRENT
|
||||
|
||||
FreeBSD-CURRENT is the "bleeding edge" of FreeBSD development and FreeBSD-CURRENT users are expected to have a high degree of technical skill. Less technical users who wish to track a development branch should track FreeBSD-STABLE instead.
|
||||
|
||||
FreeBSD-CURRENT is the very latest source code for FreeBSD and includes works in progress, experimental changes, and transitional mechanisms that might or might not be present in the next official release. While many FreeBSD developers compile the FreeBSD-CURRENT source code daily, there are short periods of time when the source may not be buildable. These problems are resolved as quickly as possible, but whether or not FreeBSD-CURRENT brings disaster or new functionality can be a matter of when the source code was synced.
|
||||
|
||||
FreeBSD-CURRENT is made available for three primary interest groups:
|
||||
|
||||
. Members of the FreeBSD community who are actively working on some part of the source tree.
|
||||
. Members of the FreeBSD community who are active testers. They are willing to spend time solving problems, making topical suggestions on changes and the general direction of FreeBSD, and submitting patches.
|
||||
. Users who wish to keep an eye on things, use the current source for reference purposes, or make the occasional comment or code contribution.
|
||||
|
||||
FreeBSD-CURRENT should _not_ be considered a fast-track to getting new features before the next release as pre-release features are not yet fully tested and most likely contain bugs. It is not a quick way of getting bug fixes as any given commit is just as likely to introduce new bugs as to fix existing ones. FreeBSD-CURRENT is not in any way "officially supported".
|
||||
|
||||
To track FreeBSD-CURRENT:
|
||||
|
||||
. Join the {freebsd-current} and the {svn-src-head} lists. This is _essential_ in order to see the comments that people are making about the current state of the system and to receive important bulletins about the current state of FreeBSD-CURRENT.
|
||||
+
|
||||
The {svn-src-head} list records the commit log entry for each change as it is made, along with any pertinent information on possible side effects.
|
||||
+
|
||||
To join these lists, go to {mailman-lists}, click on the list to subscribe to, and follow the instructions. In order to track changes to the whole source tree, not just the changes to FreeBSD-CURRENT, subscribe to the {svn-src-all}.
|
||||
. Synchronize with the FreeBSD-CURRENT sources. Typically, <<svn,svnlite>> is used to check out the -CURRENT code from the `head` branch of one of the Subversion mirror sites listed in crossref:mirrors[svn-mirrors,“Subversion Mirror Sites”].
|
||||
. Due to the size of the repository, some users choose to only synchronize the sections of source that interest them or which they are contributing patches to. However, users that plan to compile the operating system from source must download _all_ of FreeBSD-CURRENT, not just selected portions.
|
||||
+
|
||||
Before compiling FreeBSD-CURRENT , read [.filename]#/usr/src/Makefile# very carefully and follow the instructions in <<makeworld>>. Read the {freebsd-current} and [.filename]#/usr/src/UPDATING# to stay up-to-date on other bootstrapping procedures that sometimes become necessary on the road to the next release.
|
||||
. Be active! FreeBSD-CURRENT users are encouraged to submit their suggestions for enhancements or bug fixes. Suggestions with accompanying code are always welcome.
|
||||
|
||||
[[stable]]
|
||||
=== Using FreeBSD-STABLE
|
||||
|
||||
FreeBSD-STABLE is the development branch from which major releases are made. Changes go into this branch at a slower pace and with the general assumption that they have first been tested in FreeBSD-CURRENT. This is _still_ a development branch and, at any given time, the sources for FreeBSD-STABLE may or may not be suitable for general use. It is simply another engineering development track, not a resource for end-users. Users who do not have the resources to perform testing should instead run the most recent release of FreeBSD.
|
||||
|
||||
Those interested in tracking or contributing to the FreeBSD development process, especially as it relates to the next release of FreeBSD, should consider following FreeBSD-STABLE.
|
||||
|
||||
While the FreeBSD-STABLE branch should compile and run at all times, this cannot be guaranteed. Since more people run FreeBSD-STABLE than FreeBSD-CURRENT, it is inevitable that bugs and corner cases will sometimes be found in FreeBSD-STABLE that were not apparent in FreeBSD-CURRENT. For this reason, one should not blindly track FreeBSD-STABLE. It is particularly important _not_ to update any production servers to FreeBSD-STABLE without thoroughly testing the code in a development or testing environment.
|
||||
|
||||
To track FreeBSD-STABLE:
|
||||
|
||||
. Join the {freebsd-stable} in order to stay informed of build dependencies that may appear in FreeBSD-STABLE or any other issues requiring special attention. Developers will also make announcements in this mailing list when they are contemplating some controversial fix or update, giving the users a chance to respond if they have any issues to raise concerning the proposed change.
|
||||
+
|
||||
Join the relevant svn list for the branch being tracked. For example, users tracking the 9-STABLE branch should join the {svn-src-stable-9}. This list records the commit log entry for each change as it is made, along with any pertinent information on possible side effects.
|
||||
+
|
||||
To join these lists, go to {mailman-lists}, click on the list to subscribe to, and follow the instructions. In order to track changes for the whole source tree, subscribe to {svn-src-all}.
|
||||
. To install a new FreeBSD-STABLE system, install the most recent FreeBSD-STABLE release from the crossref:mirrors[mirrors,FreeBSD mirror sites] or use a monthly snapshot built from FreeBSD-STABLE. Refer to link:https://www.FreeBSD.org/snapshots/[www.freebsd.org/snapshots] for more information about snapshots.
|
||||
+
|
||||
To compile or upgrade to an existing FreeBSD system to FreeBSD-STABLE, use crossref:mirrors[svn,svn] to check out the source for the desired branch. Branch names, such as `stable/9`, are listed at link:https://www.FreeBSD.org/releng/[www.freebsd.org/releng].
|
||||
. Before compiling or upgrading to FreeBSD-STABLE , read [.filename]#/usr/src/Makefile# carefully and follow the instructions in <<makeworld>>. Read the {freebsd-stable} and [.filename]#/usr/src/UPDATING# to keep up-to-date on other bootstrapping procedures that sometimes become necessary on the road to the next release.
|
||||
|
||||
[[makeworld]]
|
||||
== Updating FreeBSD from Source
|
||||
|
||||
Updating FreeBSD by compiling from source offers several advantages over binary updates. Code can be built with options to take advantage of specific hardware. Parts of the base system can be built with non-default settings, or left out entirely where they are not needed or desired. The build process takes longer to update a system than just installing binary updates, but allows complete customization to produce a tailored version of FreeBSD.
|
||||
|
||||
[[updating-src-quick-start]]
|
||||
=== Quick Start
|
||||
|
||||
This is a quick reference for the typical steps used to update FreeBSD by building from source. Later sections describe the process in more detail.
|
||||
|
||||
[.procedure]
|
||||
====
|
||||
* Update and Build
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# svnlite update /usr/src <.>
|
||||
check /usr/src/UPDATING <.>
|
||||
# cd /usr/src <.>
|
||||
# make -j4 buildworld <.>
|
||||
# make -j4 kernel <.>
|
||||
# shutdown -r now <.>
|
||||
# cd /usr/src <.>
|
||||
# make installworld <.>
|
||||
# mergemaster -Ui <.>
|
||||
# shutdown -r now 1<.>
|
||||
....
|
||||
|
||||
<.> Get the latest version of the source. See <<updating-src-obtaining-src>> for more information on obtaining and updating source.
|
||||
|
||||
<.> Check [.filename]#/usr/src/UPDATING# for any manual steps required before or after building from source.
|
||||
|
||||
<.> Go to the source directory.
|
||||
|
||||
<.> Compile the world, everything except the kernel.
|
||||
|
||||
<.> Compile and install the kernel. This is equivalent to `make buildkernel installkernel`.
|
||||
|
||||
<.> Reboot the system to the new kernel.
|
||||
|
||||
<.> Go to the source directory.
|
||||
|
||||
<.> Install the world.
|
||||
|
||||
<.> Update and merge configuration files in [.filename]#/etc/#.
|
||||
|
||||
<.> Restart the system to use the newly-built world and kernel.
|
||||
====
|
||||
|
||||
[[updating-src-preparing]]
|
||||
=== Preparing for a Source Update
|
||||
|
||||
Read [.filename]#/usr/src/UPDATING#. Any manual steps that must be performed before or after an update are described in this file.
|
||||
|
||||
[[updating-src-obtaining-src]]
|
||||
=== Updating the Source
|
||||
|
||||
FreeBSD source code is located in [.filename]#/usr/src/#. The preferred method of updating this source is through the Subversion version control system. Verify that the source code is under version control:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svnlite info /usr/src
|
||||
Path: /usr/src
|
||||
Working Copy Root Path: /usr/src
|
||||
...
|
||||
....
|
||||
|
||||
This indicates that [.filename]#/usr/src/# is under version control and can be updated with man:svnlite[1]:
|
||||
|
||||
[[synching]]
|
||||
[source,bash]
|
||||
....
|
||||
# svnlite update /usr/src
|
||||
....
|
||||
|
||||
The update process can take some time if the directory has not been updated recently. After it finishes, the source code is up to date and the build process described in the next section can begin.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
*Obtaining the Source:* +
|
||||
|
||||
If the output says `'/usr/src' is not a working copy`, the files there are missing or were installed with a different method. A new checkout of the source is required.
|
||||
|
||||
[[updating-src-obtaining-src-repopath]]
|
||||
.FreeBSD Versions and Repository Paths
|
||||
[cols="10%,10%,80%", options="header"]
|
||||
|===
|
||||
| uname -r Output
|
||||
| Repository Path
|
||||
| Description
|
||||
|
||||
|`_X.Y_-RELEASE`
|
||||
|`base/releng/_X.Y_`
|
||||
|The Release version plus only critical security and bug fix patches. This branch is recommended for most users.
|
||||
|
||||
|`_X.Y_-STABLE`
|
||||
|`base/stable/_X_`
|
||||
|
|
||||
|
||||
The Release version plus all additional development on that branch. _STABLE_ refers to the Applications Binary Interface (ABI) not changing, so software compiled for earlier versions still runs. For example, software compiled to run on FreeBSD 10.1 will still run on FreeBSD 10-STABLE compiled later.
|
||||
|
||||
STABLE branches occasionally have bugs or incompatibilities which might affect users, although these are typically fixed quickly.
|
||||
|
||||
|`_X_-CURRENT`
|
||||
|`base/head/`
|
||||
|The latest unreleased development version of FreeBSD. The CURRENT branch can have major bugs or incompatibilities and is recommended only for advanced users.
|
||||
|===
|
||||
|
||||
Determine which version of FreeBSD is being used with man:uname[1]:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# uname -r
|
||||
10.3-RELEASE
|
||||
....
|
||||
|
||||
Based on <<updating-src-obtaining-src-repopath>>, the source used to update `10.3-RELEASE` has a repository path of `base/releng/10.3`. That path is used when checking out the source:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mv /usr/src /usr/src.bak <.>
|
||||
# svnlite checkout https://svn.freebsd.org/base/releng/10.3 /usr/src <.>
|
||||
....
|
||||
|
||||
<.> Move the old directory out of the way. If there are no local modifications in this directory, it can be deleted.
|
||||
|
||||
<.> The path from <<updating-src-obtaining-src-repopath>> is added to the repository URL. The third parameter is the destination directory for the source code on the local system.
|
||||
====
|
||||
|
||||
[[updating-src-building]]
|
||||
=== Building from Source
|
||||
|
||||
The _world_, or all of the operating system except the kernel, is compiled. This is done first to provide up-to-date tools to build the kernel. Then the kernel itself is built:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
# make buildworld
|
||||
# make buildkernel
|
||||
....
|
||||
|
||||
The compiled code is written to [.filename]#/usr/obj#.
|
||||
|
||||
These are the basic steps. Additional options to control the build are described below.
|
||||
|
||||
[[updating-src-building-clean-build]]
|
||||
==== Performing a Clean Build
|
||||
|
||||
Some versions of the FreeBSD build system leave previously-compiled code in the temporary object directory, [.filename]#/usr/obj#. This can speed up later builds by avoiding recompiling code that has not changed. To force a clean rebuild of everything, use `cleanworld` before starting a build:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make cleanworld
|
||||
....
|
||||
|
||||
[[updating-src-building-jobs]]
|
||||
==== Setting the Number of Jobs
|
||||
|
||||
Increasing the number of build jobs on multi-core processors can improve build speed. Determine the number of cores with `sysctl hw.ncpu`. Processors vary, as do the build systems used with different versions of FreeBSD, so testing is the only sure method to tell how a different number of jobs affects the build speed. For a starting point, consider values between half and double the number of cores. The number of jobs is specified with `-j`.
|
||||
|
||||
[[updating-src-building-jobs-example]]
|
||||
.Increasing the Number of Build Jobs
|
||||
[example]
|
||||
====
|
||||
Building the world and kernel with four jobs:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make -j4 buildworld buildkernel
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
[[updating-src-building-only-kernel]]
|
||||
==== Building Only the Kernel
|
||||
|
||||
A `buildworld` must be completed if the source code has changed. After that, a `buildkernel` to build a kernel can be run at any time. To build just the kernel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
# make buildkernel
|
||||
....
|
||||
|
||||
[[updating-src-building-custom-kernel]]
|
||||
==== Building a Custom Kernel
|
||||
|
||||
The standard FreeBSD kernel is based on a _kernel config file_ called [.filename]#GENERIC#. The [.filename]#GENERIC# kernel includes the most commonly-needed device drivers and options. Sometimes it is useful or necessary to build a custom kernel, adding or removing device drivers or options to fit a specific need.
|
||||
|
||||
For example, someone developing a small embedded computer with severely limited RAM could remove unneeded device drivers or options to make the kernel slightly smaller.
|
||||
|
||||
Kernel config files are located in [.filename]#/usr/src/sys/arch/conf/#, where _arch_ is the output from `uname -m`. On most computers, that is `amd64`, giving a config file directory of [.filename]#/usr/src/sys/amd64/conf/#.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
[.filename]#/usr/src# can be deleted or recreated, so it is preferable to keep custom kernel config files in a separate directory, like [.filename]#/root#. Link the kernel config file into the [.filename]#conf# directory. If that directory is deleted or overwritten, the kernel config can be re-linked into the new one.
|
||||
====
|
||||
|
||||
A custom config file can be created by copying the [.filename]#GENERIC# config file. In this example, the new custom kernel is for a storage server, so is named [.filename]#STORAGESERVER#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cp /usr/src/sys/amd64/conf/GENERIC /root/STORAGESERVER
|
||||
# cd /usr/src/sys/amd64/conf
|
||||
# ln -s /root/STORAGESERVER .
|
||||
....
|
||||
|
||||
[.filename]#/root/STORAGESERVER# is then edited, adding or removing devices or options as shown in man:config[5].
|
||||
|
||||
The custom kernel is built by setting `KERNCONF` to the kernel config file on the command line:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make buildkernel KERNCONF=STORAGESERVER
|
||||
....
|
||||
|
||||
[[updating-src-installing]]
|
||||
=== Installing the Compiled Code
|
||||
|
||||
After the `buildworld` and `buildkernel` steps have been completed, the new kernel and world are installed:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
# make installkernel
|
||||
# shutdown -r now
|
||||
# cd /usr/src
|
||||
# make installworld
|
||||
# shutdown -r now
|
||||
....
|
||||
|
||||
If a custom kernel was built, `KERNCONF` must also be set to use the new custom kernel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
# make installkernel KERNCONF=STORAGESERVER
|
||||
# shutdown -r now
|
||||
# cd /usr/src
|
||||
# make installworld
|
||||
# shutdown -r now
|
||||
....
|
||||
|
||||
[[updating-src-completing]]
|
||||
=== Completing the Update
|
||||
|
||||
A few final tasks complete the update. Any modified configuration files are merged with the new versions, outdated libraries are located and removed, then the system is restarted.
|
||||
|
||||
[[updating-src-completing-merge-mergemaster]]
|
||||
==== Merging Configuration Files with man:mergemaster[8]
|
||||
|
||||
man:mergemaster[8] provides an easy way to merge changes that have been made to system configuration files with new versions of those files.
|
||||
|
||||
With `-Ui`, man:mergemaster[8] automatically updates files that have not been user-modified and installs new files that are not already present:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mergemaster -Ui
|
||||
....
|
||||
|
||||
If a file must be manually merged, an interactive display allows the user to choose which portions of the files are kept. See man:mergemaster[8] for more information.
|
||||
|
||||
[[updating-src-completing-check-old]]
|
||||
==== Checking for Outdated Files and Libraries
|
||||
|
||||
Some obsolete files or directories can remain after an update. These files can be located:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make check-old
|
||||
....
|
||||
|
||||
and deleted:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make delete-old
|
||||
....
|
||||
|
||||
Some obsolete libraries can also remain. These can be detected with:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make check-old-libs
|
||||
....
|
||||
|
||||
and deleted with
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make delete-old-libs
|
||||
....
|
||||
|
||||
Programs which were still using those old libraries will stop working when the library has been deleted. These programs must be rebuilt or replaced after deleting the old libraries.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
When all the old files or directories are known to be safe to delete, pressing kbd:[y] and kbd:[Enter] to delete each file can be avoided by setting `BATCH_DELETE_OLD_FILES` in the command. For example:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make BATCH_DELETE_OLD_FILES=yes delete-old-libs
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
[[updating-src-completing-restart]]
|
||||
==== Restarting After the Update
|
||||
|
||||
The last step after updating is to restart the computer so all the changes take effect:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# shutdown -r now
|
||||
....
|
||||
|
||||
[[small-lan]]
|
||||
== Tracking for Multiple Machines
|
||||
|
||||
When multiple machines need to track the same source tree, it is a waste of disk space, network bandwidth, and CPU cycles to have each system download the sources and rebuild everything. The solution is to have one machine do most of the work, while the rest of the machines mount that work via NFS. This section outlines a method of doing so. For more information about using NFS, refer to crossref:network-servers[network-nfs,"Network File System (NFS)"].
|
||||
|
||||
First, identify a set of machines which will run the same set of binaries, known as a _build set_. Each machine can have a custom kernel, but will run the same userland binaries. From that set, choose a machine to be the _build machine_ that the world and kernel are built on. Ideally, this is a fast machine that has sufficient spare CPU to run `make buildworld` and `make buildkernel`.
|
||||
|
||||
Select a machine to be the _test machine_, which will test software updates before they are put into production. This _must_ be a machine that can afford to be down for an extended period of time. It can be the build machine, but need not be.
|
||||
|
||||
All the machines in this build set need to mount [.filename]#/usr/obj# and [.filename]#/usr/src# from the build machine via NFS. For multiple build sets, [.filename]#/usr/src# should be on one build machine, and NFS mounted on the rest.
|
||||
|
||||
Ensure that [.filename]#/etc/make.conf# and [.filename]#/etc/src.conf# on all the machines in the build set agree with the build machine. That means that the build machine must build all the parts of the base system that any machine in the build set is going to install. Also, each build machine should have its kernel name set with `KERNCONF` in [.filename]#/etc/make.conf#, and the build machine should list them all in its `KERNCONF`, listing its own kernel first. The build machine must have the kernel configuration files for each machine in its [.filename]#/usr/src/sys/arch/conf#.
|
||||
|
||||
On the build machine, build the kernel and world as described in <<makeworld>>, but do not install anything on the build machine. Instead, install the built kernel on the test machine. On the test machine, mount [.filename]#/usr/src# and [.filename]#/usr/obj# via NFS. Then, run `shutdown now` to go to single-user mode in order to install the new kernel and world and run `mergemaster` as usual. When done, reboot to return to normal multi-user operations.
|
||||
|
||||
After verifying that everything on the test machine is working properly, use the same procedure to install the new software on each of the other machines in the build set.
|
||||
|
||||
The same methodology can be used for the ports tree. The first step is to share [.filename]#/usr/ports# via NFS to all the machines in the build set. To configure [.filename]#/etc/make.conf# to share distfiles, set `DISTDIR` to a common shared directory that is writable by whichever user `root` is mapped to by the NFS mount. Each machine should set `WRKDIRPREFIX` to a local build directory, if ports are to be built locally. Alternately, if the build system is to build and distribute packages to the machines in the build set, set `PACKAGES` on the build system to a directory similar to `DISTDIR`.
|
||||
570
documentation/content/en/books/handbook/desktop/_index.adoc
Normal file
570
documentation/content/en/books/handbook/desktop/_index.adoc
Normal file
|
|
@ -0,0 +1,570 @@
|
|||
---
|
||||
title: Chapter 6. Desktop Applications
|
||||
part: Part II. Common Tasks
|
||||
prev: books/handbook/partii
|
||||
next: books/handbook/multimedia
|
||||
---
|
||||
|
||||
[[desktop]]
|
||||
= Desktop Applications
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 6
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/desktop/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/desktop/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/desktop/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[desktop-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
While FreeBSD is popular as a server for its performance and stability, it is also suited for day-to-day use as a desktop. With over {numports} applications available as FreeBSD packages or ports, it is easy to build a customized desktop that runs a wide variety of desktop applications. This chapter demonstrates how to install numerous desktop applications, including web browsers, productivity software, document viewers, and financial software.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Users who prefer to install a pre-built desktop version of FreeBSD rather than configuring one from scratch should refer to https://ghostbsd.org[GhostBSD], https://www.midnightbsd.org[MidnightBSD] or https://www.nomad.org[NomadBSD].
|
||||
====
|
||||
|
||||
Readers of this chapter should know how to:
|
||||
|
||||
* Install additional software using packages or ports as described in crossref:ports[ports,Installing Applications: Packages and Ports].
|
||||
* Install X and a window manager as described in crossref:x11[x11,The X Window System].
|
||||
|
||||
For information on how to configure a multimedia environment, refer to crossref:multimedia[multimedia,Multimedia].
|
||||
|
||||
[[desktop-browsers]]
|
||||
== Browsers
|
||||
|
||||
FreeBSD does not come with a pre-installed web browser. Instead, the https://www.FreeBSD.org/ports/[www] category of the Ports Collection contains many browsers which can be installed as a package or compiled from the Ports Collection.
|
||||
|
||||
The KDE and GNOME desktop environments include their own HTML browser. Refer to crossref:x11[x11-wm,“Desktop Environments”] for more information on how to set up these complete desktops.
|
||||
|
||||
Some lightweight browsers include package:www/dillo2[], package:www/links[], and package:www/w3m[].
|
||||
|
||||
This section demonstrates how to install the following popular web browsers and indicates if the application is resource-heavy, takes time to compile from ports, or has any major dependencies.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Application Name
|
||||
| Resources Needed
|
||||
| Installation from Ports
|
||||
| Notes
|
||||
|
||||
|Firefox
|
||||
|medium
|
||||
|heavy
|
||||
|FreeBSD, Linux(R), and localized versions are available
|
||||
|
||||
|Konqueror
|
||||
|medium
|
||||
|heavy
|
||||
|Requires KDE libraries
|
||||
|
||||
|Chromium
|
||||
|medium
|
||||
|heavy
|
||||
|Requires Gtk+
|
||||
|===
|
||||
|
||||
=== Firefox
|
||||
|
||||
Firefox is an open source browser that features a standards-compliant HTML display engine, tabbed browsing, popup blocking, extensions, improved security, and more. Firefox is based on the Mozilla codebase.
|
||||
|
||||
To install the package of the latest release version of Firefox, type:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install firefox
|
||||
....
|
||||
|
||||
To instead install Firefox Extended Support Release (ESR) version, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install firefox-esr
|
||||
....
|
||||
|
||||
The Ports Collection can instead be used to compile the desired version of Firefox from source code. This example builds package:www/firefox[], where `firefox` can be replaced with the ESR or localized version to install.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/www/firefox
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== Konqueror
|
||||
|
||||
Konqueror is more than a web browser as it is also a file manager and a multimedia viewer. Supports WebKit as well as its own KHTML. WebKit is a rendering engine used by many modern browsers including Chromium.
|
||||
|
||||
Konqueror can be installed as a package by typing:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install konqueror
|
||||
....
|
||||
|
||||
To install from the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/x11-fm/konqueror/
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== Chromium
|
||||
|
||||
Chromium is an open source browser project that aims to build a safer, faster, and more stable web browsing experience. Chromium features tabbed browsing, popup blocking, extensions, and much more. Chromium is the open source project upon which the Google Chrome web browser is based.
|
||||
|
||||
Chromium can be installed as a package by typing:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install chromium
|
||||
....
|
||||
|
||||
Alternatively, Chromium can be compiled from source using the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/www/chromium
|
||||
# make install clean
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The executable for Chromium is [.filename]#/usr/local/bin/chrome#, not [.filename]#/usr/local/bin/chromium#.
|
||||
====
|
||||
|
||||
[[desktop-productivity]]
|
||||
== Productivity
|
||||
|
||||
When it comes to productivity, users often look for an office suite or an easy-to-use word processor. While some <<x11-wm,desktop environments>> like KDE provide an office suite, there is no default productivity package. Several office suites and graphical word processors are available for FreeBSD, regardless of the installed window manager.
|
||||
|
||||
This section demonstrates how to install the following popular productivity software and indicates if the application is resource-heavy, takes time to compile from ports, or has any major dependencies.
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Application Name
|
||||
| Resources Needed
|
||||
| Installation from Ports
|
||||
| Major Dependencies
|
||||
|
||||
|Calligra
|
||||
|light
|
||||
|heavy
|
||||
|KDE
|
||||
|
||||
|AbiWord
|
||||
|light
|
||||
|light
|
||||
|Gtk+ or GNOME
|
||||
|
||||
|The Gimp
|
||||
|light
|
||||
|heavy
|
||||
|Gtk+
|
||||
|
||||
|Apache OpenOffice
|
||||
|heavy
|
||||
|huge
|
||||
|JDK(TM) and Mozilla
|
||||
|
||||
|LibreOffice
|
||||
|somewhat heavy
|
||||
|huge
|
||||
|Gtk+, or KDE/ GNOME, or JDK(TM)
|
||||
|===
|
||||
|
||||
=== Calligra
|
||||
|
||||
The KDE desktop environment includes an office suite which can be installed separately from KDE. Calligra includes standard components that can be found in other office suites. Words is the word processor, Sheets is the spreadsheet program, Stage manages slide presentations, and Karbon is used to draw graphical documents.
|
||||
|
||||
In FreeBSD, package:editors/calligra[] can be installed as a package or a port. To install the package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install calligra
|
||||
....
|
||||
|
||||
If the package is not available, use the Ports Collection instead:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/editors/calligra
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== AbiWord
|
||||
|
||||
AbiWord is a free word processing program similar in look and feel to Microsoft(R) Word. It is fast, contains many features, and is user-friendly.
|
||||
|
||||
AbiWord can import or export many file formats, including some proprietary ones like Microsoft(R) [.filename]#.rtf#.
|
||||
|
||||
To install the AbiWord package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install abiword
|
||||
....
|
||||
|
||||
If the package is not available, it can be compiled from the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/editors/abiword
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== The GIMP
|
||||
|
||||
For image authoring or picture retouching, The GIMP provides a sophisticated image manipulation program. It can be used as a simple paint program or as a quality photo retouching suite. It supports a large number of plugins and features a scripting interface. The GIMP can read and write a wide range of file formats and supports interfaces with scanners and tablets.
|
||||
|
||||
To install the package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install gimp
|
||||
....
|
||||
|
||||
Alternately, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/graphics/gimp
|
||||
# make install clean
|
||||
....
|
||||
|
||||
The graphics category (https://www.FreeBSD.org/ports/graphics/[freebsd.org/ports/graphics/]) of the Ports Collection contains several GIMP-related plugins, help files, and user manuals.
|
||||
|
||||
=== Apache OpenOffice
|
||||
|
||||
Apache OpenOffice is an open source office suite which is developed under the wing of the Apache Software Foundation's Incubator. It includes all of the applications found in a complete office productivity suite: a word processor, spreadsheet, presentation manager, and drawing program. Its user interface is similar to other office suites, and it can import and export in various popular file formats. It is available in a number of different languages and internationalization has been extended to interfaces, spell checkers, and dictionaries.
|
||||
|
||||
The word processor of Apache OpenOffice uses a native XML file format for increased portability and flexibility. The spreadsheet program features a macro language which can be interfaced with external databases. Apache OpenOffice is stable and runs natively on Windows(R), Solaris(TM), Linux(R), FreeBSD, and Mac OS(R) X. More information about Apache OpenOffice can be found at http://openoffice.org/[openoffice.org]. For FreeBSD specific information refer to http://porting.openoffice.org/freebsd/[porting.openoffice.org/freebsd/].
|
||||
|
||||
To install the Apache OpenOffice package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install apache-openoffice
|
||||
....
|
||||
|
||||
Once the package is installed, type the following command to launch Apache OpenOffice:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% openoffice-X.Y.Z
|
||||
....
|
||||
|
||||
where _X.Y.Z_ is the version number of the installed version of Apache OpenOffice. The first time Apache OpenOffice launches, some questions will be asked and a [.filename]#.openoffice.org# folder will be created in the user's home directory.
|
||||
|
||||
If the desired Apache OpenOffice package is not available, compiling the port is still an option. However, this requires a lot of disk space and a fairly long time to compile:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/editors/openoffice-4
|
||||
# make install clean
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
To build a localized version, replace the previous command with:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# make LOCALIZED_LANG=your_language install clean
|
||||
....
|
||||
|
||||
Replace _your_language_ with the correct language ISO-code. A list of supported language codes is available in [.filename]#files/Makefile.localized#, located in the port's directory.
|
||||
====
|
||||
|
||||
=== LibreOffice
|
||||
|
||||
LibreOffice is a free software office suite developed by http://www.documentfoundation.org/[documentfoundation.org]. It is compatible with other major office suites and available on a variety of platforms. It is a rebranded fork of Apache OpenOffice and includes applications found in a complete office productivity suite: a word processor, spreadsheet, presentation manager, drawing program, database management program, and a tool for creating and editing mathematical formulæ. It is available in a number of different languages and internationalization has been extended to interfaces, spell checkers, and dictionaries.
|
||||
|
||||
The word processor of LibreOffice uses a native XML file format for increased portability and flexibility. The spreadsheet program features a macro language which can be interfaced with external databases. LibreOffice is stable and runs natively on Windows(R), Linux(R), FreeBSD, and Mac OS(R) X. More information about LibreOffice can be found at http://www.libreoffice.org/[libreoffice.org].
|
||||
|
||||
To install the English version of the LibreOffice package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install libreoffice
|
||||
....
|
||||
|
||||
The editors category (https://www.FreeBSD.org/ports/editors/[freebsd.org/ports/editors/]) of the Ports Collection contains several localizations for LibreOffice. When installing a localized package, replace `libreoffice` with the name of the localized package.
|
||||
|
||||
Once the package is installed, type the following command to run LibreOffice:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% libreoffice
|
||||
....
|
||||
|
||||
During the first launch, some questions will be asked and a [.filename]#.libreoffice# folder will be created in the user's home directory.
|
||||
|
||||
If the desired LibreOffice package is not available, compiling the port is still an option. However, this requires a lot of disk space and a fairly long time to compile. This example compiles the English version:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/editors/libreoffice
|
||||
# make install clean
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
To build a localized version, `cd` into the port directory of the desired language. Supported languages can be found in the editors category (https://www.FreeBSD.org/ports/editors/[freebsd.org/ports/editors/]) of the Ports Collection.
|
||||
====
|
||||
|
||||
[[desktop-viewers]]
|
||||
== Document Viewers
|
||||
|
||||
Some new document formats have gained popularity since the advent of UNIX(R) and the viewers they require may not be available in the base system. This section demonstrates how to install the following document viewers:
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Application Name
|
||||
| Resources Needed
|
||||
| Installation from Ports
|
||||
| Major Dependencies
|
||||
|
||||
|Xpdf
|
||||
|light
|
||||
|light
|
||||
|FreeType
|
||||
|
||||
|gv
|
||||
|light
|
||||
|light
|
||||
|Xaw3d
|
||||
|
||||
|Geeqie
|
||||
|light
|
||||
|light
|
||||
|Gtk+ or GNOME
|
||||
|
||||
|ePDFView
|
||||
|light
|
||||
|light
|
||||
|Gtk+
|
||||
|
||||
|Okular
|
||||
|light
|
||||
|heavy
|
||||
|KDE
|
||||
|===
|
||||
|
||||
=== Xpdf
|
||||
|
||||
For users that prefer a small FreeBSD PDF viewer, Xpdf provides a light-weight and efficient viewer which requires few resources. It uses the standard X fonts and does not require any additional toolkits.
|
||||
|
||||
To install the Xpdf package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install xpdf
|
||||
....
|
||||
|
||||
If the package is not available, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/graphics/xpdf
|
||||
# make install clean
|
||||
....
|
||||
|
||||
Once the installation is complete, launch `xpdf` and use the right mouse button to activate the menu.
|
||||
|
||||
=== gv
|
||||
|
||||
gv is a PostScript(R) and PDF viewer. It is based on ghostview, but has a nicer look as it is based on the Xaw3d widget toolkit. gv has many configurable features, such as orientation, paper size, scale, and anti-aliasing. Almost any operation can be performed with either the keyboard or the mouse.
|
||||
|
||||
To install gv as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install gv
|
||||
....
|
||||
|
||||
If a package is unavailable, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/print/gv
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== Geeqie
|
||||
|
||||
Geeqie is a fork from the unmaintained GQView project, in an effort to move development forward and integrate the existing patches. Geeqie is an image manager which supports viewing a file with a single click, launching an external editor, and thumbnail previews. It also features a slideshow mode and some basic file operations, making it easy to manage image collections and to find duplicate files. Geeqie supports full screen viewing and internationalization.
|
||||
|
||||
To install the Geeqie package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install geeqie
|
||||
....
|
||||
|
||||
If the package is not available, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/graphics/geeqie
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== ePDFView
|
||||
|
||||
ePDFView is a lightweight `PDF` document viewer that only uses the Gtk+ and Poppler libraries. It is currently under development, but already opens most `PDF` files (even encrypted), save copies of documents, and has support for printing using CUPS.
|
||||
|
||||
To install ePDFView as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install epdfview
|
||||
....
|
||||
|
||||
If a package is unavailable, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/graphics/epdfview
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== Okular
|
||||
|
||||
Okular is a universal document viewer based on KPDF for KDE. It can open many document formats, including `PDF`, PostScript(R), DjVu, `CHM`, `XPS`, and ePub.
|
||||
|
||||
To install Okular as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install okular
|
||||
....
|
||||
|
||||
If a package is unavailable, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/graphics/okular
|
||||
# make install clean
|
||||
....
|
||||
|
||||
[[desktop-finance]]
|
||||
== Finance
|
||||
|
||||
For managing personal finances on a FreeBSD desktop, some powerful and easy-to-use applications can be installed. Some are compatible with widespread file formats, such as the formats used by Quicken and Excel.
|
||||
|
||||
This section covers these programs:
|
||||
|
||||
[.informaltable]
|
||||
[cols="1,1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Application Name
|
||||
| Resources Needed
|
||||
| Installation from Ports
|
||||
| Major Dependencies
|
||||
|
||||
|GnuCash
|
||||
|light
|
||||
|heavy
|
||||
|GNOME
|
||||
|
||||
|Gnumeric
|
||||
|light
|
||||
|heavy
|
||||
|GNOME
|
||||
|
||||
|KMyMoney
|
||||
|light
|
||||
|heavy
|
||||
|KDE
|
||||
|===
|
||||
|
||||
=== GnuCash
|
||||
|
||||
GnuCash is part of the GNOME effort to provide user-friendly, yet powerful, applications to end-users. GnuCash can be used to keep track of income and expenses, bank accounts, and stocks. It features an intuitive interface while remaining professional.
|
||||
|
||||
GnuCash provides a smart register, a hierarchical system of accounts, and many keyboard accelerators and auto-completion methods. It can split a single transaction into several more detailed pieces. GnuCash can import and merge Quicken QIF files. It also handles most international date and currency formats.
|
||||
|
||||
To install the GnuCash package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install gnucash
|
||||
....
|
||||
|
||||
If the package is not available, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/finance/gnucash
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== Gnumeric
|
||||
|
||||
Gnumeric is a spreadsheet program developed by the GNOME community. It features convenient automatic guessing of user input according to the cell format with an autofill system for many sequences. It can import files in a number of popular formats, including Excel, Lotus 1-2-3, and Quattro Pro. It has a large number of built-in functions and allows all of the usual cell formats such as number, currency, date, time, and much more.
|
||||
|
||||
To install Gnumeric as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install gnumeric
|
||||
....
|
||||
|
||||
If the package is not available, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/math/gnumeric
|
||||
# make install clean
|
||||
....
|
||||
|
||||
=== KMyMoney
|
||||
|
||||
KMyMoney is a personal finance application created by the KDE community. KMyMoney aims to provide the important features found in commercial personal finance manager applications. It also highlights ease-of-use and proper double-entry accounting among its features. KMyMoney imports from standard Quicken QIF files, tracks investments, handles multiple currencies, and provides a wealth of reports.
|
||||
|
||||
To install KMyMoney as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install kmymoney-kde4
|
||||
....
|
||||
|
||||
If the package is not available, use the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/finance/kmymoney-kde4
|
||||
# make install clean
|
||||
....
|
||||
2151
documentation/content/en/books/handbook/disks/_index.adoc
Normal file
2151
documentation/content/en/books/handbook/disks/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
229
documentation/content/en/books/handbook/dtrace/_index.adoc
Normal file
229
documentation/content/en/books/handbook/dtrace/_index.adoc
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
---
|
||||
title: Chapter 25. DTrace
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/cutting-edge
|
||||
next: books/handbook/usb-device-mode
|
||||
---
|
||||
|
||||
[[dtrace]]
|
||||
= DTrace
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 25
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/dtrace/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/dtrace/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/dtrace/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[dtrace-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
DTrace, also known as Dynamic Tracing, was developed by Sun(TM) as a tool for locating performance bottlenecks in production and pre-production systems. In addition to diagnosing performance problems, DTrace can be used to help investigate and debug unexpected behavior in both the FreeBSD kernel and in userland programs.
|
||||
|
||||
DTrace is a remarkable profiling tool, with an impressive array of features for diagnosing system issues. It may also be used to run pre-written scripts to take advantage of its capabilities. Users can author their own utilities using the DTrace D Language, allowing them to customize their profiling based on specific needs.
|
||||
|
||||
The FreeBSD implementation provides full support for kernel DTrace and experimental support for userland DTrace. Userland DTrace allows users to perform function boundary tracing for userland programs using the `pid` provider, and to insert static probes into userland programs for later tracing. Some ports, such as package:databases/postgresql12-server[] and package:lang/php74[] have a DTrace option to enable static probes.
|
||||
|
||||
The official guide to DTrace is maintained by the Illumos project at http://dtrace.org/guide[DTrace Guide].
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* What DTrace is and what features it provides.
|
||||
* Differences between the Solaris(TM) DTrace implementation and the one provided by FreeBSD.
|
||||
* How to enable and use DTrace on FreeBSD.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Understand UNIX(R) and FreeBSD basics (crossref:basics[basics,FreeBSD Basics]).
|
||||
* Have some familiarity with security and how it pertains to FreeBSD (crossref:security[security,Security]).
|
||||
|
||||
[[dtrace-implementation]]
|
||||
== Implementation Differences
|
||||
|
||||
While the DTrace in FreeBSD is similar to that found in Solaris(TM), differences do exist. The primary difference is that in FreeBSD, DTrace is implemented as a set of kernel modules and DTrace can not be used until the modules are loaded. To load all of the necessary modules:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload dtraceall
|
||||
....
|
||||
|
||||
Beginning with FreeBSD 10.0-RELEASE, the modules are automatically loaded when `dtrace` is run.
|
||||
|
||||
FreeBSD uses the `DDB_CTF` kernel option to enable support for loading `CTF` data from kernel modules and the kernel itself. `CTF` is the Solaris(TM) Compact C Type Format which encapsulates a reduced form of debugging information similar to `DWARF` and the venerable stabs. `CTF` data is added to binaries by the `ctfconvert` and `ctfmerge` build tools. The `ctfconvert` utility parses `DWARF``ELF` debug sections created by the compiler and `ctfmerge` merges `CTF``ELF` sections from objects into either executables or shared libraries.
|
||||
|
||||
Some different providers exist for FreeBSD than for Solaris(TM). Most notable is the `dtmalloc` provider, which allows tracing `malloc()` by type in the FreeBSD kernel. Some of the providers found in Solaris(TM), such as `cpc` and `mib`, are not present in FreeBSD. These may appear in future versions of FreeBSD. Moreover, some of the providers available in both operating systems are not compatible, in the sense that their probes have different argument types. Thus, `D` scripts written on Solaris(TM) may or may not work unmodified on FreeBSD, and vice versa.
|
||||
|
||||
Due to security differences, only `root` may use DTrace on FreeBSD. Solaris(TM) has a few low level security checks which do not yet exist in FreeBSD. As such, the [.filename]#/dev/dtrace/dtrace# is strictly limited to `root`.
|
||||
|
||||
DTrace falls under the Common Development and Distribution License (`CDDL`) license. To view this license on FreeBSD, see [.filename]#/usr/src/cddl/contrib/opensolaris/OPENSOLARIS.LICENSE# or view it online at http://opensource.org/licenses/CDDL-1.0[http://opensource.org/licenses/CDDL-1.0]. While a FreeBSD kernel with DTrace support is `BSD` licensed, the `CDDL` is used when the modules are distributed in binary form or the binaries are loaded.
|
||||
|
||||
[[dtrace-enable]]
|
||||
== Enabling DTrace Support
|
||||
|
||||
In FreeBSD 9.2 and 10.0, DTrace support is built into the [.filename]#GENERIC# kernel. Users of earlier versions of FreeBSD or who prefer to statically compile in DTrace support should add the following lines to a custom kernel configuration file and recompile the kernel using the instructions in crossref:kernelconfig[kernelconfig,Configuring the FreeBSD Kernel]:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
options KDTRACE_HOOKS
|
||||
options DDB_CTF
|
||||
makeoptions DEBUG=-g
|
||||
makeoptions WITH_CTF=1
|
||||
....
|
||||
|
||||
Users of the AMD64 architecture should also add this line:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
options KDTRACE_FRAME
|
||||
....
|
||||
|
||||
This option provides support for `FBT`. While DTrace will work without this option, there will be limited support for function boundary tracing.
|
||||
|
||||
Once the FreeBSD system has rebooted into the new kernel, or the DTrace kernel modules have been loaded using `kldload dtraceall`, the system will need support for the Korn shell as the DTrace Toolkit has several utilities written in `ksh`. Make sure that the package:shells/ksh93[] package or port is installed. It is also possible to run these tools under package:shells/pdksh[] or package:shells/mksh[].
|
||||
|
||||
Finally, install the current DTrace Toolkit, a collection of ready-made scripts for collecting system information. There are scripts to check open files, memory, `CPU` usage, and a lot more. FreeBSD 10 installs a few of these scripts into [.filename]#/usr/shared/dtrace#. On other FreeBSD versions, or to install the full DTrace Toolkit, use the package:sysutils/DTraceToolkit[] package or port.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The scripts found in [.filename]#/usr/shared/dtrace# have been specifically ported to FreeBSD. Not all of the scripts found in the DTrace Toolkit will work as-is on FreeBSD and some scripts may require some effort in order for them to work on FreeBSD.
|
||||
====
|
||||
|
||||
The DTrace Toolkit includes many scripts in the special language of DTrace. This language is called the D language and it is very similar to C++. An in depth discussion of the language is beyond the scope of this document. It is covered extensively in the http://www.dtrace.org/guide[Illumos Dynamic Tracing Guide].
|
||||
|
||||
[[dtrace-using]]
|
||||
== Using DTrace
|
||||
|
||||
DTrace scripts consist of a list of one or more _probes_, or instrumentation points, where each probe is associated with an action. Whenever the condition for a probe is met, the associated action is executed. For example, an action may occur when a file is opened, a process is started, or a line of code is executed. The action might be to log some information or to modify context variables. The reading and writing of context variables allows probes to share information and to cooperatively analyze the correlation of different events.
|
||||
|
||||
To view all probes, the administrator can execute the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# dtrace -l | more
|
||||
....
|
||||
|
||||
Each probe has an `ID`, a `PROVIDER` (dtrace or fbt), a `MODULE`, and a `FUNCTION NAME`. Refer to man:dtrace[1] for more information about this command.
|
||||
|
||||
The examples in this section provide an overview of how to use two of the fully supported scripts from the DTrace Toolkit: the [.filename]#hotkernel# and [.filename]#procsystime# scripts.
|
||||
|
||||
The [.filename]#hotkernel# script is designed to identify which function is using the most kernel time. It will produce output similar to the following:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/local/shared/dtrace-toolkit
|
||||
# ./hotkernel
|
||||
Sampling... Hit Ctrl-C to end.
|
||||
....
|
||||
|
||||
As instructed, use the kbd:[Ctrl+C] key combination to stop the process. Upon termination, the script will display a list of kernel functions and timing information, sorting the output in increasing order of time:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
kernel`_thread_lock_flags 2 0.0%
|
||||
0xc1097063 2 0.0%
|
||||
kernel`sched_userret 2 0.0%
|
||||
kernel`kern_select 2 0.0%
|
||||
kernel`generic_copyin 3 0.0%
|
||||
kernel`_mtx_assert 3 0.0%
|
||||
kernel`vm_fault 3 0.0%
|
||||
kernel`sopoll_generic 3 0.0%
|
||||
kernel`fixup_filename 4 0.0%
|
||||
kernel`_isitmyx 4 0.0%
|
||||
kernel`find_instance 4 0.0%
|
||||
kernel`_mtx_unlock_flags 5 0.0%
|
||||
kernel`syscall 5 0.0%
|
||||
kernel`DELAY 5 0.0%
|
||||
0xc108a253 6 0.0%
|
||||
kernel`witness_lock 7 0.0%
|
||||
kernel`read_aux_data_no_wait 7 0.0%
|
||||
kernel`Xint0x80_syscall 7 0.0%
|
||||
kernel`witness_checkorder 7 0.0%
|
||||
kernel`sse2_pagezero 8 0.0%
|
||||
kernel`strncmp 9 0.0%
|
||||
kernel`spinlock_exit 10 0.0%
|
||||
kernel`_mtx_lock_flags 11 0.0%
|
||||
kernel`witness_unlock 15 0.0%
|
||||
kernel`sched_idletd 137 0.3%
|
||||
0xc10981a5 42139 99.3%
|
||||
....
|
||||
|
||||
This script will also work with kernel modules. To use this feature, run the script with `-m`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ./hotkernel -m
|
||||
Sampling... Hit Ctrl-C to end.
|
||||
^C
|
||||
MODULE COUNT PCNT
|
||||
0xc107882e 1 0.0%
|
||||
0xc10e6aa4 1 0.0%
|
||||
0xc1076983 1 0.0%
|
||||
0xc109708a 1 0.0%
|
||||
0xc1075a5d 1 0.0%
|
||||
0xc1077325 1 0.0%
|
||||
0xc108a245 1 0.0%
|
||||
0xc107730d 1 0.0%
|
||||
0xc1097063 2 0.0%
|
||||
0xc108a253 73 0.0%
|
||||
kernel 874 0.4%
|
||||
0xc10981a5 213781 99.6%
|
||||
....
|
||||
|
||||
The [.filename]#procsystime# script captures and prints the system call time usage for a given process `ID` (`PID`) or process name. In the following example, a new instance of [.filename]#/bin/csh# was spawned. Then, [.filename]#procsystime# was executed and remained waiting while a few commands were typed on the other incarnation of `csh`. These are the results of this test:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ./procsystime -n csh
|
||||
Tracing... Hit Ctrl-C to end...
|
||||
^C
|
||||
|
||||
Elapsed Times for processes csh,
|
||||
|
||||
SYSCALL TIME (ns)
|
||||
getpid 6131
|
||||
sigreturn 8121
|
||||
close 19127
|
||||
fcntl 19959
|
||||
dup 26955
|
||||
setpgid 28070
|
||||
stat 31899
|
||||
setitimer 40938
|
||||
wait4 62717
|
||||
sigaction 67372
|
||||
sigprocmask 119091
|
||||
gettimeofday 183710
|
||||
write 263242
|
||||
execve 492547
|
||||
ioctl 770073
|
||||
vfork 3258923
|
||||
sigsuspend 6985124
|
||||
read 3988049784
|
||||
....
|
||||
|
||||
As shown, the `read()` system call used the most time in nanoseconds while the `getpid()` system call used the least amount of time.
|
||||
1194
documentation/content/en/books/handbook/eresources/_index.adoc
Normal file
1194
documentation/content/en/books/handbook/eresources/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,94 @@
|
|||
---
|
||||
title: Chapter 21. Other File Systems
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/zfs
|
||||
next: books/handbook/virtualization
|
||||
---
|
||||
|
||||
[[filesystems]]
|
||||
= Other File Systems
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 21
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/filesystems/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/filesystems/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/filesystems/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[filesystems-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
File systems are an integral part of any operating system. They allow users to upload and store files, provide access to data, and make hard drives useful. Different operating systems differ in their native file system. Traditionally, the native FreeBSD file system has been the Unix File System UFS which has been modernized as UFS2. Since FreeBSD 7.0, the Z File System (ZFS) is also available as a native file system. See crossref:zfs[zfs,The Z File System (ZFS)] for more information.
|
||||
|
||||
In addition to its native file systems, FreeBSD supports a multitude of other file systems so that data from other operating systems can be accessed locally, such as data stored on locally attached USB storage devices, flash drives, and hard disks. This includes support for the Linux(R) Extended File System (EXT).
|
||||
|
||||
There are different levels of FreeBSD support for the various file systems. Some require a kernel module to be loaded and others may require a toolset to be installed. Some non-native file system support is full read-write while others are read-only.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* The difference between native and supported file systems.
|
||||
* Which file systems are supported by FreeBSD.
|
||||
* How to enable, configure, access, and make use of non-native file systems.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Understand UNIX(R) and crossref:basics[basics,FreeBSD basics].
|
||||
* Be familiar with the basics of crossref:kernelconfig[kernelconfig,kernel configuration and compilation].
|
||||
* Feel comfortable crossref:ports[ports,installing software] in FreeBSD.
|
||||
* Have some familiarity with crossref:disks[disks,disks], storage, and device names in FreeBSD.
|
||||
|
||||
[[filesystems-linux]]
|
||||
== Linux(R) File Systems
|
||||
|
||||
FreeBSD provides built-in support for several Linux(R) file systems. This section demonstrates how to load support for and how to mount the supported Linux(R) file systems.
|
||||
|
||||
=== ext2
|
||||
|
||||
Kernel support for ext2 file systems has been available since FreeBSD 2.2. In FreeBSD 8.x and earlier, the code is licensed under the GPL. Since FreeBSD 9.0, the code has been rewritten and is now BSD licensed.
|
||||
|
||||
The man:ext2fs[5] driver allows the FreeBSD kernel to both read and write to ext2 file systems.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
This driver can also be used to access ext3 and ext4 file systems. The man:ext2fs[5] filesystem has full read and write support for ext4 as of FreeBSD 12.0-RELEASE. Additionally, extended attributes and ACLs are also supported, while journalling and encryption are not. Starting with FreeBSD 12.1-RELEASE, a DTrace provider will be available as well. Prior versions of FreeBSD can access ext4 in read and write mode using package:sysutils/fusefs-ext2[].
|
||||
====
|
||||
|
||||
To access an ext file system, first load the kernel loadable module:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload ext2fs
|
||||
....
|
||||
|
||||
Then, mount the ext volume by specifying its FreeBSD partition name and an existing mount point. This example mounts [.filename]#/dev/ad1s1# on [.filename]#/mnt#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mount -t ext2fs /dev/ad1s1 /mnt
|
||||
....
|
||||
2225
documentation/content/en/books/handbook/firewalls/_index.adoc
Normal file
2225
documentation/content/en/books/handbook/firewalls/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
1129
documentation/content/en/books/handbook/geom/_index.adoc
Normal file
1129
documentation/content/en/books/handbook/geom/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
223
documentation/content/en/books/handbook/introduction/_index.adoc
Normal file
223
documentation/content/en/books/handbook/introduction/_index.adoc
Normal file
|
|
@ -0,0 +1,223 @@
|
|||
---
|
||||
title: Chapter 1. Introduction
|
||||
part: Part I. Getting Started
|
||||
prev: books/handbook/parti
|
||||
next: books/handbook/bsdinstall
|
||||
---
|
||||
|
||||
[[introduction]]
|
||||
= Introduction
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 1
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/introduction/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/introduction/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/introduction/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[introduction-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
Thank you for your interest in FreeBSD! The following chapter covers various aspects of the FreeBSD Project, such as its history, goals, development model, and so on.
|
||||
|
||||
After reading this chapter you will know:
|
||||
|
||||
* How FreeBSD relates to other computer operating systems.
|
||||
* The history of the FreeBSD Project.
|
||||
* The goals of the FreeBSD Project.
|
||||
* The basics of the FreeBSD open-source development model.
|
||||
* And of course: where the name "FreeBSD" comes from.
|
||||
|
||||
[[nutshell]]
|
||||
== Welcome to FreeBSD!
|
||||
|
||||
FreeBSD is an Open Source, standards-compliant Unix-like operating system for x86 (both 32 and 64 bit), ARM(R), AArch64, RISC-V(R), MIPS(R), POWER(R), PowerPC(R), and Sun UltraSPARC(R) computers. It provides all the features that are nowadays taken for granted, such as preemptive multitasking, memory protection, virtual memory, multi-user facilities, SMP support, all the Open Source development tools for different languages and frameworks, and desktop features centered around X Window System, KDE, or GNOME. Its particular strengths are:
|
||||
|
||||
* _Liberal Open Source license_, which grants you rights to freely modify and extend its source code and incorporate it in both Open Source projects and closed products without imposing restrictions typical to copyleft licenses, as well as avoiding potential license incompatibility problems.
|
||||
* _Strong TCP/IP networking_ - FreeBSD implements industry standard protocols with ever increasing performance and scalability. This makes it a good match in both server, and routing/firewalling roles - and indeed many companies and vendors use it precisely for that purpose.
|
||||
* _Fully integrated OpenZFS support_, including root-on-ZFS, ZFS Boot Environments, fault management, administrative delegation, support for jails, FreeBSD specific documentation, and system installer support.
|
||||
* _Extensive security features_, from the Mandatory Access Control framework to Capsicum capability and sandbox mechanisms.
|
||||
* _Over 30 thousand prebuilt packages_ for all supported architectures, and the Ports Collection which makes it easy to build your own, customized ones.
|
||||
* _Documentation_ - in addition to Handbook and books from different authors that cover topics ranging from system administration to kernel internals, there are also the man:man[1] pages, not only for userspace daemons, utilities, and configuration files, but also for kernel driver APIs (section 9) and individual drivers (section 4).
|
||||
* _Simple and consistent repository structure and build system_ - FreeBSD uses a single repository for all of its components, both kernel and userspace. This, along with an unified and easy to customize build system and a well thought out development process makes it easy to integrate FreeBSD with build infrastructure for your own product.
|
||||
* _Staying true to Unix philosophy_, preferring composability instead of monolithic "all in one" daemons with hardcoded behavior.
|
||||
* _Binary compatibility_ with Linux, which makes it possible to run many Linux binaries without the need for virtualisation.
|
||||
|
||||
FreeBSD is based on the 4.4BSD-Lite release from Computer Systems Research Group (CSRG) at the University of California at Berkeley, and carries on the distinguished tradition of BSD systems development. In addition to the fine work provided by CSRG, the FreeBSD Project has put in many thousands of man-hours into extending the functionality and fine-tuning the system for maximum performance and reliability in real-life load situations. FreeBSD offers performance and reliability on par with other Open Source and commercial offerings, combined with cutting-edge features not available anywhere else.
|
||||
|
||||
[[os-overview]]
|
||||
=== What Can FreeBSD Do?
|
||||
|
||||
The applications to which FreeBSD can be put are truly limited only by your own imagination. From software development to factory automation, inventory control to azimuth correction of remote satellite antennae; if it can be done with a commercial UNIX(R) product then it is more than likely that you can do it with FreeBSD too! FreeBSD also benefits significantly from literally thousands of high quality applications developed by research centers and universities around the world, often available at little to no cost.
|
||||
|
||||
Because the source code for FreeBSD itself is freely available, the system can also be customized to an almost unheard of degree for special applications or projects, and in ways not generally possible with operating systems from most major commercial vendors. Here is just a sampling of some of the applications in which people are currently using FreeBSD:
|
||||
|
||||
* _Internet Services:_ The robust TCP/IP networking built into FreeBSD makes it an ideal platform for a variety of Internet services such as:
|
||||
|
||||
** Web servers
|
||||
** IPv4 and IPv6 routing
|
||||
** Firewalls and NAT ("IP masquerading") gateways
|
||||
** FTP servers
|
||||
** Email servers
|
||||
** And more...
|
||||
|
||||
* _Education:_ Are you a student of computer science or a related engineering field? There is no better way of learning about operating systems, computer architecture and networking than the hands on, under the hood experience that FreeBSD can provide. A number of freely available CAD, mathematical and graphic design packages also make it highly useful to those whose primary interest in a computer is to get _other_ work done!
|
||||
* _Research:_ With source code for the entire system available, FreeBSD is an excellent platform for research in operating systems as well as other branches of computer science. FreeBSD's freely available nature also makes it possible for remote groups to collaborate on ideas or shared development without having to worry about special licensing agreements or limitations on what may be discussed in open forums.
|
||||
* _Networking:_ Need a new router? A name server (DNS)? A firewall to keep people out of your internal network? FreeBSD can easily turn that unused PC sitting in the corner into an advanced router with sophisticated packet-filtering capabilities.
|
||||
* _Embedded:_ FreeBSD makes an excellent platform to build embedded systems upon. With support for the ARM(R), MIPS(R) and PowerPC(R) platforms, coupled with a robust network stack, cutting edge features and the permissive link:{faq}#bsd-license-restrictions[BSD license] FreeBSD makes an excellent foundation for building embedded routers, firewalls, and other devices.
|
||||
* _Desktop:_ FreeBSD makes a fine choice for an inexpensive desktop solution using the freely available X11 server. FreeBSD offers a choice from many open-source desktop environments, including the standard GNOME and KDE graphical user interfaces. FreeBSD can even boot "diskless" from a central server, making individual workstations even cheaper and easier to administer.
|
||||
* _Software Development:_ The basic FreeBSD system comes with a full suite of development tools including a full C/C++ compiler and debugger suite. Support for many other languages are also available through the ports and packages collection.
|
||||
|
||||
FreeBSD is available to download free of charge, or can be obtained on either CD-ROM or DVD. Please see crossref:mirrors[mirrors, Obtaining FreeBSD] for more information about obtaining FreeBSD.
|
||||
|
||||
[[introduction-nutshell-users]]
|
||||
=== Who Uses FreeBSD?
|
||||
|
||||
FreeBSD has been known for its web serving capabilities - sites that run on FreeBSD include https://news.ycombinator.com/[Hacker News], http://www.netcraft.com/[Netcraft], http://www.163.com/[NetEase], https://signup.netflix.com/openconnect[Netflix], http://www.sina.com/[Sina], http://www.sony.co.jp/[Sony Japan], http://www.rambler.ru/[Rambler], http://www.yahoo.com/[Yahoo!], and http://www.yandex.ru/[Yandex].
|
||||
|
||||
FreeBSD's advanced features, proven security, predictable release cycle, and permissive license have led to its use as a platform for building many commercial and open source appliances, devices, and products. Many of the world's largest IT companies use FreeBSD:
|
||||
|
||||
* http://www.apache.org/[Apache] - The Apache Software Foundation runs most of its public facing infrastructure, including possibly one of the largest SVN repositories in the world with over 1.4 million commits, on FreeBSD.
|
||||
* http://www.apple.com/[Apple] - OS X borrows heavily from FreeBSD for the network stack, virtual file system, and many userland components. Apple iOS also contains elements borrowed from FreeBSD.
|
||||
* http://www.cisco.com/[Cisco] - IronPort network security and anti-spam appliances run a modified FreeBSD kernel.
|
||||
* http://www.citrix.com/[Citrix] - The NetScaler line of security appliances provide layer 4-7 load balancing, content caching, application firewall, secure VPN, and mobile cloud network access, along with the power of a FreeBSD shell.
|
||||
* https://www.emc.com/isilon[Dell EMC Isilon] - Isilon's enterprise storage appliances are based on FreeBSD. The extremely liberal FreeBSD license allowed Isilon to integrate their intellectual property throughout the kernel and focus on building their product instead of an operating system.
|
||||
* http://www.quest.com/KACE[Quest KACE] - The KACE system management appliances run FreeBSD because of its reliability, scalability, and the community that supports its continued development.
|
||||
* http://www.ixsystems.com/[iXsystems] - The TrueNAS line of unified storage appliances is based on FreeBSD. In addition to their commercial products, iXsystems also manages development of the open source projects TrueOS and FreeNAS.
|
||||
* http://www.juniper.net/[Juniper] - The JunOS operating system that powers all Juniper networking gear (including routers, switches, security, and networking appliances) is based on FreeBSD. Juniper is one of many vendors that showcases the symbiotic relationship between the project and vendors of commercial products. Improvements generated at Juniper are upstreamed into FreeBSD to reduce the complexity of integrating new features from FreeBSD back into JunOS in the future.
|
||||
* http://www.mcafee.com/[McAfee] - SecurOS, the basis of McAfee enterprise firewall products including Sidewinder is based on FreeBSD.
|
||||
* http://www.netapp.com/[NetApp] - The Data ONTAP GX line of storage appliances are based on FreeBSD. In addition, NetApp has contributed back many features, including the new BSD licensed hypervisor, bhyve.
|
||||
* http://www.netflix.com/[Netflix] - The OpenConnect appliance that Netflix uses to stream movies to its customers is based on FreeBSD. Netflix has made extensive contributions to the codebase and works to maintain a zero delta from mainline FreeBSD. Netflix OpenConnect appliances are responsible for delivering more than 32% of all Internet traffic in North America.
|
||||
* http://www.sandvine.com/[Sandvine] - Sandvine uses FreeBSD as the basis of their high performance real-time network processing platforms that make up their intelligent network policy control products.
|
||||
* http://www.sony.com/[Sony] - The PlayStation 4 gaming console runs a modified version of FreeBSD.
|
||||
* http://www.sophos.com/[Sophos] - The Sophos Email Appliance product is based on a hardened FreeBSD and scans inbound mail for spam and viruses, while also monitoring outbound mail for malware as well as the accidental loss of sensitive information.
|
||||
* http://www.spectralogic.com/[Spectra Logic] - The nTier line of archive grade storage appliances run FreeBSD and OpenZFS.
|
||||
* https://www.stormshield.eu[Stormshield] - Stormshield Network Security appliances are based on a hardened version of FreeBSD. The BSD license allows them to integrate their own intellectual property with the system while returning a great deal of interesting development to the community.
|
||||
* http://www.weather.com/[The Weather Channel] - The IntelliStar appliance that is installed at each local cable provider's headend and is responsible for injecting local weather forecasts into the cable TV network's programming runs FreeBSD.
|
||||
* http://www.verisign.com/[Verisign] - Verisign is responsible for operating the .com and .net root domain registries as well as the accompanying DNS infrastructure. They rely on a number of different network operating systems including FreeBSD to ensure there is no common point of failure in their infrastructure.
|
||||
* http://www.voxer.com/[Voxer] - Voxer powers their mobile voice messaging platform with ZFS on FreeBSD. Voxer switched from a Solaris derivative to FreeBSD because of its superior documentation, larger and more active community, and more developer friendly environment. In addition to critical features like ZFS and DTrace, FreeBSD also offers TRIM support for ZFS.
|
||||
* https://fudosecurity.com/en/[Fudo Security] - The FUDO security appliance allows enterprises to monitor, control, record, and audit contractors and administrators who work on their systems. Based on all of the best security features of FreeBSD including ZFS, GELI, Capsicum, HAST, and auditdistd.
|
||||
|
||||
FreeBSD has also spawned a number of related open source projects:
|
||||
|
||||
* http://bsdrp.net/[BSD Router] - A FreeBSD based replacement for large enterprise routers designed to run on standard PC hardware.
|
||||
* http://www.freenas.org/[FreeNAS] - A customized FreeBSD designed to be used as a network file server appliance. Provides a python based web interface to simplify the management of both the UFS and ZFS file systems. Includes support for NFS, SMB/CIFS, AFP, FTP, and iSCSI. Includes an extensible plugin system based on FreeBSD jails.
|
||||
* https://ghostbsd.org/[GhostBSD] - is derived from FreeBSD, uses the GTK environment to provide a beautiful looks and comfortable experience on the modern BSD platform offering a natural and native UNIX(R) work environment.
|
||||
* http://mfsbsd.vx.sk/[mfsBSD] - A toolkit for building a FreeBSD system image that runs entirely from memory.
|
||||
* http://www.nas4free.org/[NAS4Free] - A file server distribution based on FreeBSD with a PHP powered web interface.
|
||||
* http://www.opnsense.org/[OPNSense] - OPNsense is an open source, easy-to-use and easy-to-build FreeBSD based firewall and routing platform. OPNsense includes most of the features available in expensive commercial firewalls, and more in many cases. It brings the rich feature set of commercial offerings with the benefits of open and verifiable sources.
|
||||
* https://www.trueos.org[TrueOS] - TrueOS is based on the legendary security and stability of FreeBSD. TrueOS follows FreeBSD-CURRENT, with the latest drivers, security updates, and packages available.
|
||||
* https://www.midnightbsd.org[MidnightBSD] - is a FreeBSD derived operating system developed with desktop users in mind. It includes all the software you'd expect for your daily tasks: mail, web browsing, word processing, gaming, and much more.
|
||||
* https://www.nomadbsd.org[NomadBSD] - is a persistent live system for USB flash drives, based on FreeBSD. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD's hardware compatibility.
|
||||
* http://www.pfsense.org/[pfSense] - A firewall distribution based on FreeBSD with a huge array of features and extensive IPv6 support.
|
||||
* http://zrouter.org/[ZRouter] - An open source alternative firmware for embedded devices based on FreeBSD. Designed to replace the proprietary firmware on off-the-shelf routers.
|
||||
|
||||
A list of https://www.freebsdfoundation.org/about/testimonials/[testimonials from companies basing their products and services on FreeBSD] can be found at the FreeBSD Foundation website. Wikipedia also maintains a https://en.wikipedia.org/wiki/List_of_products_based_on_FreeBSD[list of products based on FreeBSD].
|
||||
|
||||
[[history]]
|
||||
== About the FreeBSD Project
|
||||
|
||||
The following section provides some background information on the project, including a brief history, project goals, and the development model of the project.
|
||||
|
||||
[[intro-history]]
|
||||
=== A Brief History of FreeBSD
|
||||
|
||||
The FreeBSD Project had its genesis in the early part of 1993, partially as the brainchild of the Unofficial 386BSDPatchkit's last 3 coordinators: Nate Williams, Rod Grimes and Jordan Hubbard.
|
||||
|
||||
The original goal was to produce an intermediate snapshot of 386BSD in order to fix a number of problems that the patchkit mechanism was just not capable of solving. The early working title for the project was 386BSD 0.5 or 386BSD Interim in reference of that fact.
|
||||
|
||||
386BSD was Bill Jolitz's operating system, which had been up to that point suffering rather severely from almost a year's worth of neglect. As the patchkit swelled ever more uncomfortably with each passing day, they decided to assist Bill by providing this interim "cleanup" snapshot. Those plans came to a rude halt when Bill Jolitz suddenly decided to withdraw his sanction from the project without any clear indication of what would be done instead.
|
||||
|
||||
The trio thought that the goal remained worthwhile, even without Bill's support, and so they adopted the name "FreeBSD" coined by David Greenman. The initial objectives were set after consulting with the system's current users and, once it became clear that the project was on the road to perhaps even becoming a reality, Jordan contacted Walnut Creek CDROM with an eye toward improving FreeBSD's distribution channels for those many unfortunates without easy access to the Internet. Walnut Creek CDROM not only supported the idea of distributing FreeBSD on CD but also went so far as to provide the project with a machine to work on and a fast Internet connection. Without Walnut Creek CDROM's almost unprecedented degree of faith in what was, at the time, a completely unknown project, it is quite unlikely that FreeBSD would have gotten as far, as fast, as it has today.
|
||||
|
||||
The first CD-ROM (and general net-wide) distribution was FreeBSD 1.0, released in December of 1993. This was based on the 4.3BSD-Lite ("Net/2") tape from U.C. Berkeley, with many components also provided by 386BSD and the Free Software Foundation. It was a fairly reasonable success for a first offering, and they followed it with the highly successful FreeBSD 1.1 release in May of 1994.
|
||||
|
||||
Around this time, some rather unexpected storm clouds formed on the horizon as Novell and U.C. Berkeley settled their long-running lawsuit over the legal status of the Berkeley Net/2 tape. A condition of that settlement was U.C. Berkeley's concession that large parts of Net/2 were "encumbered" code and the property of Novell, who had in turn acquired it from AT&T some time previously. What Berkeley got in return was Novell's "blessing" that the 4.4BSD-Lite release, when it was finally released, would be declared unencumbered and all existing Net/2 users would be strongly encouraged to switch. This included FreeBSD, and the project was given until the end of July 1994 to stop shipping its own Net/2 based product. Under the terms of that agreement, the project was allowed one last release before the deadline, that release being FreeBSD 1.1.5.1.
|
||||
|
||||
FreeBSD then set about the arduous task of literally re-inventing itself from a completely new and rather incomplete set of 4.4BSD-Lite bits. The "Lite" releases were light in part because Berkeley's CSRG had removed large chunks of code required for actually constructing a bootable running system (due to various legal requirements) and the fact that the Intel port of 4.4 was highly incomplete. It took the project until November of 1994 to make this transition, and in December it released FreeBSD 2.0 to the world. Despite being still more than a little rough around the edges, the release was a significant success and was followed by the more robust and easier to install FreeBSD 2.0.5 release in June of 1995.
|
||||
|
||||
Since that time, FreeBSD has made a series of releases each time improving the stability, speed, and feature set of the previous version.
|
||||
|
||||
For now, long-term development projects continue to take place in the 10.X-CURRENT (trunk) branch, and snapshot releases of 10.X are continually made available from link:ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/[the snapshot server] as work progresses.
|
||||
|
||||
[[goals]]
|
||||
=== FreeBSD Project Goals
|
||||
|
||||
The goals of the FreeBSD Project are to provide software that may be used for any purpose and without strings attached. Many of us have a significant investment in the code (and project) and would certainly not mind a little financial compensation now and then, but we are definitely not prepared to insist on it. We believe that our first and foremost "mission" is to provide code to any and all comers, and for whatever purpose, so that the code gets the widest possible use and provides the widest possible benefit. This is, I believe, one of the most fundamental goals of Free Software and one that we enthusiastically support.
|
||||
|
||||
That code in our source tree which falls under the GNU General Public License (GPL) or Library General Public License (LGPL) comes with slightly more strings attached, though at least on the side of enforced access rather than the usual opposite. Due to the additional complexities that can evolve in the commercial use of GPL software we do, however, prefer software submitted under the more relaxed BSD license when it is a reasonable option to do so.
|
||||
|
||||
[[development]]
|
||||
=== The FreeBSD Development Model
|
||||
|
||||
The development of FreeBSD is a very open and flexible process, being literally built from the contributions of thousands of people around the world, as can be seen from our link:{contributors}[list of contributors]. FreeBSD's development infrastructure allow these thousands of contributors to collaborate over the Internet. We are constantly on the lookout for new developers and ideas, and those interested in becoming more closely involved with the project need simply contact us at the {freebsd-hackers}. The {freebsd-announce} is also available to those wishing to make other FreeBSD users aware of major areas of work.
|
||||
|
||||
Useful things to know about the FreeBSD Project and its development process, whether working independently or in close cooperation:
|
||||
|
||||
The SVN repositories[[development-cvs-repository]]::
|
||||
For several years, the central source tree for FreeBSD was maintained by http://www.nongnu.org/cvs/[CVS] (Concurrent Versions System), a freely available source code control tool. In June 2008, the Project switched to using http://subversion.tigris.org[SVN] (Subversion). The switch was deemed necessary, as the technical limitations imposed by CVS were becoming obvious due to the rapid expansion of the source tree and the amount of history already stored. The Documentation Project and Ports Collection repositories also moved from CVS to SVN in May 2012 and July 2012, respectively. Please refer to the crossref:cutting-edge[synching, Obtaining the Source] section for more information on obtaining the FreeBSD `src/` repository and crossref:ports[ports-using, Using the Ports Collection] for details on obtaining the FreeBSD Ports Collection.
|
||||
|
||||
The committers list[[development-committers]]::
|
||||
The _committers_ are the people who have _write_ access to the Subversion tree, and are authorized to make modifications to the FreeBSD source (the term "committer" comes from `commit`, the source control command which is used to bring new changes into the repository). Anyone can submit a bug to the https://bugs.FreeBSD.org/submit/[Bug Database]. Before submitting a bug report, the FreeBSD mailing lists, IRC channels, or forums can be used to help verify that an issue is actually a bug.
|
||||
|
||||
The FreeBSD core team[[development-core]]::
|
||||
The _FreeBSD core team_ would be equivalent to the board of directors if the FreeBSD Project were a company. The primary task of the core team is to make sure the project, as a whole, is in good shape and is heading in the right directions. Inviting dedicated and responsible developers to join our group of committers is one of the functions of the core team, as is the recruitment of new core team members as others move on. The current core team was elected from a pool of committer candidates in June 2020. Elections are held every 2 years.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
Like most developers, most members of the core team are also volunteers when it comes to FreeBSD development and do not benefit from the project financially, so "commitment" should also not be misconstrued as meaning "guaranteed support." The "board of directors" analogy above is not very accurate, and it may be more suitable to say that these are the people who gave up their lives in favor of FreeBSD against their better judgement!
|
||||
====
|
||||
|
||||
Outside contributors::
|
||||
Last, but definitely not least, the largest group of developers are the users themselves who provide feedback and bug fixes to us on an almost constant basis. The primary way of keeping in touch with FreeBSD's more non-centralized development is to subscribe to the {freebsd-hackers} where such things are discussed. See crossref:eresources[eresources, Resources on the Internet] for more information about the various FreeBSD mailing lists.
|
||||
+
|
||||
link:{contributors}[The FreeBSD Contributors List] is a long and growing one, so why not join it by contributing something back to FreeBSD today?
|
||||
+
|
||||
Providing code is not the only way of contributing to the project; for a more complete list of things that need doing, please refer to the link:https://www.FreeBSD.org/[FreeBSD Project web site].
|
||||
|
||||
In summary, our development model is organized as a loose set of concentric circles. The centralized model is designed for the convenience of the _users_ of FreeBSD, who are provided with an easy way of tracking one central code base, not to keep potential contributors out! Our desire is to present a stable operating system with a large set of coherent crossref:ports[ports,application programs] that the users can easily install and use - this model works very well in accomplishing that.
|
||||
|
||||
All we ask of those who would join us as FreeBSD developers is some of the same dedication its current people have to its continued success!
|
||||
|
||||
[[third-party-programs]]
|
||||
=== Third Party Programs
|
||||
|
||||
In addition to the base distributions, FreeBSD offers a ported software collection with thousands of commonly sought-after programs. At the time of this writing, there were over {numports} ports! The list of ports ranges from http servers, to games, languages, editors, and almost everything in between. The entire Ports Collection requires approximately {ports-size}. To compile a port, you simply change to the directory of the program you wish to install, type `make install`, and let the system do the rest. The full original distribution for each port you build is retrieved dynamically so you need only enough disk space to build the ports you want. Almost every port is also provided as a pre-compiled "package", which can be installed with a simple command (`pkg install`) by those who do not wish to compile their own ports from source. More information on packages and ports can be found in crossref:ports[ports,Installing Applications: Packages and Ports].
|
||||
|
||||
=== Additional Documentation
|
||||
|
||||
All supported FreeBSD versions provide an option in the installer to install additional documentation under [.filename]#/usr/local/shared/doc/freebsd# during the initial system setup. Documentation may also be installed at any later time using packages as described in crossref:cutting-edge[doc-ports-install-package,“Updating Documentation from Ports”]. You may view the locally installed manuals with any HTML capable browser using the following URLs:
|
||||
|
||||
The FreeBSD Handbook::
|
||||
[.filename]#link:file://localhost/usr/local/shared/doc/freebsd/handbook/index.html[/usr/local/shared/doc/freebsd/handbook/index.html]#
|
||||
|
||||
The FreeBSD FAQ::
|
||||
[.filename]#link:file://localhost/usr/local/shared/doc/freebsd/faq/index.html[/usr/local/shared/doc/freebsd/faq/index.html]#
|
||||
|
||||
You can also view the master (and most frequently updated) copies at https://www.FreeBSD.org/[https://www.FreeBSD.org/].
|
||||
1101
documentation/content/en/books/handbook/jails/_index.adoc
Normal file
1101
documentation/content/en/books/handbook/jails/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
294
documentation/content/en/books/handbook/kernelconfig/_index.adoc
Normal file
294
documentation/content/en/books/handbook/kernelconfig/_index.adoc
Normal file
|
|
@ -0,0 +1,294 @@
|
|||
---
|
||||
title: Chapter 8. Configuring the FreeBSD Kernel
|
||||
part: Part II. Common Tasks
|
||||
prev: books/handbook/multimedia
|
||||
next: books/handbook/printing
|
||||
---
|
||||
|
||||
[[kernelconfig]]
|
||||
= Configuring the FreeBSD Kernel
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 8
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/kernelconfig/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/kernelconfig/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/kernelconfig/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[kernelconfig-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
The kernel is the core of the FreeBSD operating system. It is responsible for managing memory, enforcing security controls, networking, disk access, and much more. While much of FreeBSD is dynamically configurable, it is still occasionally necessary to configure and compile a custom kernel.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* When to build a custom kernel.
|
||||
* How to take a hardware inventory.
|
||||
* How to customize a kernel configuration file.
|
||||
* How to use the kernel configuration file to create and build a new kernel.
|
||||
* How to install the new kernel.
|
||||
* How to troubleshoot if things go wrong.
|
||||
|
||||
All of the commands listed in the examples in this chapter should be executed as `root`.
|
||||
|
||||
[[kernelconfig-custom-kernel]]
|
||||
== Why Build a Custom Kernel?
|
||||
|
||||
Traditionally, FreeBSD used a monolithic kernel. The kernel was one large program, supported a fixed list of devices, and in order to change the kernel's behavior, one had to compile and then reboot into a new kernel.
|
||||
|
||||
Today, most of the functionality in the FreeBSD kernel is contained in modules which can be dynamically loaded and unloaded from the kernel as necessary. This allows the running kernel to adapt immediately to new hardware and for new functionality to be brought into the kernel. This is known as a modular kernel.
|
||||
|
||||
Occasionally, it is still necessary to perform static kernel configuration. Sometimes the needed functionality is so tied to the kernel that it can not be made dynamically loadable. Some security environments prevent the loading and unloading of kernel modules and require that only needed functionality is statically compiled into the kernel.
|
||||
|
||||
Building a custom kernel is often a rite of passage for advanced BSD users. This process, while time consuming, can provide benefits to the FreeBSD system. Unlike the [.filename]#GENERIC# kernel, which must support a wide range of hardware, a custom kernel can be stripped down to only provide support for that computer's hardware. This has a number of benefits, such as:
|
||||
|
||||
* Faster boot time. Since the kernel will only probe the hardware on the system, the time it takes the system to boot can decrease.
|
||||
* Lower memory usage. A custom kernel often uses less memory than the [.filename]#GENERIC# kernel by omitting unused features and device drivers. This is important because the kernel code remains resident in physical memory at all times, preventing that memory from being used by applications. For this reason, a custom kernel is useful on a system with a small amount of RAM.
|
||||
* Additional hardware support. A custom kernel can add support for devices which are not present in the [.filename]#GENERIC# kernel.
|
||||
|
||||
Before building a custom kernel, consider the reason for doing so. If there is a need for specific hardware support, it may already exist as a module.
|
||||
|
||||
Kernel modules exist in [.filename]#/boot/kernel# and may be dynamically loaded into the running kernel using man:kldload[8]. Most kernel drivers have a loadable module and manual page. For example, the man:ath[4] wireless Ethernet driver has the following information in its manual page:
|
||||
|
||||
[source,bash,subs="macros"]
|
||||
....
|
||||
Alternatively, to load the driver as a module at boot time, place the
|
||||
following line in man:loader.conf[5]:
|
||||
|
||||
if_ath_load="YES"
|
||||
....
|
||||
|
||||
Adding `if_ath_load="YES"` to [.filename]#/boot/loader.conf# will load this module dynamically at boot time.
|
||||
|
||||
In some cases, there is no associated module in [.filename]#/boot/kernel#. This is mostly true for certain subsystems.
|
||||
|
||||
[[kernelconfig-devices]]
|
||||
== Finding the System Hardware
|
||||
|
||||
Before editing the kernel configuration file, it is recommended to perform an inventory of the machine's hardware. On a dual-boot system, the inventory can be created from the other operating system. For example, Microsoft(R)'s Device Manager contains information about installed devices.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Some versions of Microsoft(R) Windows(R) have a System icon which can be used to access Device Manager.
|
||||
====
|
||||
|
||||
If FreeBSD is the only installed operating system, use man:dmesg[8] to determine the hardware that was found and listed during the boot probe. Most device drivers on FreeBSD have a manual page which lists the hardware supported by that driver. For example, the following lines indicate that the man:psm[4] driver found a mouse:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
psm0: <PS/2 Mouse> irq 12 on atkbdc0
|
||||
psm0: [GIANT-LOCKED]
|
||||
psm0: [ITHREAD]
|
||||
psm0: model Generic PS/2 mouse, device ID 0
|
||||
....
|
||||
|
||||
Since this hardware exists, this driver should not be removed from a custom kernel configuration file.
|
||||
|
||||
If the output of `dmesg` does not display the results of the boot probe output, instead read the contents of [.filename]#/var/run/dmesg.boot#.
|
||||
|
||||
Another tool for finding hardware is man:pciconf[8], which provides more verbose output. For example:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% pciconf -lv
|
||||
ath0@pci0:3:0:0: class=0x020000 card=0x058a1014 chip=0x1014168c rev=0x01 hdr=0x00
|
||||
vendor = 'Atheros Communications Inc.'
|
||||
device = 'AR5212 Atheros AR5212 802.11abg wireless'
|
||||
class = network
|
||||
subclass = ethernet
|
||||
....
|
||||
|
||||
This output shows that the [.filename]#ath# driver located a wireless Ethernet device.
|
||||
|
||||
The `-k` flag of man:man[1] can be used to provide useful information. For example, it can be used to display a list of manual pages which contain a particular device brand or name:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# man -k Atheros
|
||||
ath(4) - Atheros IEEE 802.11 wireless network driver
|
||||
ath_hal(4) - Atheros Hardware Access Layer (HAL)
|
||||
....
|
||||
|
||||
Once the hardware inventory list is created, refer to it to ensure that drivers for installed hardware are not removed as the custom kernel configuration is edited.
|
||||
|
||||
[[kernelconfig-config]]
|
||||
== The Configuration File
|
||||
|
||||
In order to create a custom kernel configuration file and build a custom kernel, the full FreeBSD source tree must first be installed.
|
||||
|
||||
If [.filename]#/usr/src/# does not exist or it is empty, source has not been installed. Source can be installed using Subversion and the instructions in crossref:mirrors[svn,“Using Subversion”].
|
||||
|
||||
Once source is installed, review the contents of [.filename]#/usr/src/sys#. This directory contains a number of subdirectories, including those which represent the following supported architectures: [.filename]#amd64#, [.filename]#i386#, [.filename]#powerpc#, and [.filename]#sparc64#. Everything inside a particular architecture's directory deals with that architecture only and the rest of the code is machine independent code common to all platforms. Each supported architecture has a [.filename]#conf# subdirectory which contains the [.filename]#GENERIC# kernel configuration file for that architecture.
|
||||
|
||||
Do not make edits to [.filename]#GENERIC#. Instead, copy the file to a different name and make edits to the copy. The convention is to use a name with all capital letters. When maintaining multiple FreeBSD machines with different hardware, it is a good idea to name it after the machine's hostname. This example creates a copy, named [.filename]#MYKERNEL#, of the [.filename]#GENERIC# configuration file for the `amd64` architecture:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src/sys/amd64/conf
|
||||
# cp GENERIC MYKERNEL
|
||||
....
|
||||
|
||||
[.filename]#MYKERNEL# can now be customized with any `ASCII` text editor. The default editor is vi, though an easier editor for beginners, called ee, is also installed with FreeBSD.
|
||||
|
||||
The format of the kernel configuration file is simple. Each line contains a keyword that represents a device or subsystem, an argument, and a brief description. Any text after a `#` is considered a comment and ignored. To remove kernel support for a device or subsystem, put a `#` at the beginning of the line representing that device or subsystem. Do not add or remove a `#` for any line that you do not understand.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
It is easy to remove support for a device or option and end up with a broken kernel. For example, if the man:ata[4] driver is removed from the kernel configuration file, a system using `ATA` disk drivers may not boot. When in doubt, just leave support in the kernel.
|
||||
====
|
||||
|
||||
In addition to the brief descriptions provided in this file, additional descriptions are contained in [.filename]#NOTES#, which can be found in the same directory as [.filename]#GENERIC# for that architecture. For architecture independent options, refer to [.filename]#/usr/src/sys/conf/NOTES#.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
When finished customizing the kernel configuration file, save a backup copy to a location outside of [.filename]#/usr/src#.
|
||||
|
||||
Alternately, keep the kernel configuration file elsewhere and create a symbolic link to the file:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src/sys/amd64/conf
|
||||
# mkdir /root/kernels
|
||||
# cp GENERIC /root/kernels/MYKERNEL
|
||||
# ln -s /root/kernels/MYKERNEL
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
An `include` directive is available for use in configuration files. This allows another configuration file to be included in the current one, making it easy to maintain small changes relative to an existing file. If only a small number of additional options or drivers are required, this allows a delta to be maintained with respect to [.filename]#GENERIC#, as seen in this example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
include GENERIC
|
||||
ident MYKERNEL
|
||||
|
||||
options IPFIREWALL
|
||||
options DUMMYNET
|
||||
options IPFIREWALL_DEFAULT_TO_ACCEPT
|
||||
options IPDIVERT
|
||||
....
|
||||
|
||||
Using this method, the local configuration file expresses local differences from a [.filename]#GENERIC# kernel. As upgrades are performed, new features added to [.filename]#GENERIC# will also be added to the local kernel unless they are specifically prevented using `nooptions` or `nodevice`. A comprehensive list of configuration directives and their descriptions may be found in man:config[5].
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
To build a file which contains all available options, run the following command as `root`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src/sys/arch/conf && make LINT
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
[[kernelconfig-building]]
|
||||
== Building and Installing a Custom Kernel
|
||||
|
||||
Once the edits to the custom configuration file have been saved, the source code for the kernel can be compiled using the following steps:
|
||||
|
||||
[.procedure]
|
||||
*Procedure: Building a Kernel*
|
||||
. Change to this directory:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
....
|
||||
|
||||
. Compile the new kernel by specifying the name of the custom kernel configuration file:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# make buildkernel KERNCONF=MYKERNEL
|
||||
....
|
||||
|
||||
. Install the new kernel associated with the specified kernel configuration file. This command will copy the new kernel to [.filename]#/boot/kernel/kernel# and save the old kernel to [.filename]#/boot/kernel.old/kernel#:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# make installkernel KERNCONF=MYKERNEL
|
||||
....
|
||||
|
||||
. Shutdown the system and reboot into the new kernel. If something goes wrong, refer to <<kernelconfig-noboot, The kernel does not boot>>.
|
||||
|
||||
By default, when a custom kernel is compiled, all kernel modules are rebuilt. To update a kernel faster or to build only custom modules, edit [.filename]#/etc/make.conf# before starting to build the kernel.
|
||||
|
||||
For example, this variable specifies the list of modules to build instead of using the default of building all modules:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
MODULES_OVERRIDE = linux acpi
|
||||
....
|
||||
|
||||
Alternately, this variable lists which modules to exclude from the build process:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
WITHOUT_MODULES = linux acpi sound
|
||||
....
|
||||
|
||||
Additional variables are available. Refer to man:make.conf[5] for details.
|
||||
|
||||
[[kernelconfig-trouble]]
|
||||
== If Something Goes Wrong
|
||||
|
||||
There are four categories of trouble that can occur when building a custom kernel:
|
||||
|
||||
`config` fails::
|
||||
If `config` fails, it will print the line number that is incorrect. As an example, for the following message, make sure that line 17 is typed correctly by comparing it to [.filename]#GENERIC# or [.filename]#NOTES#:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
config: line 17: syntax error
|
||||
....
|
||||
|
||||
`make` fails::
|
||||
If `make` fails, it is usually due to an error in the kernel configuration file which is not severe enough for `config` to catch. Review the configuration, and if the problem is not apparent, send an email to the {freebsd-questions} which contains the kernel configuration file.
|
||||
[[kernelconfig-noboot]]
|
||||
The kernel does not boot::
|
||||
If the new kernel does not boot or fails to recognize devices, do not panic! Fortunately, FreeBSD has an excellent mechanism for recovering from incompatible kernels. Simply choose the kernel to boot from at the FreeBSD boot loader. This can be accessed when the system boot menu appears by selecting the "Escape to a loader prompt" option. At the prompt, type `boot _kernel.old_`, or the name of any other kernel that is known to boot properly.
|
||||
+
|
||||
After booting with a good kernel, check over the configuration file and try to build it again. One helpful resource is [.filename]#/var/log/messages# which records the kernel messages from every successful boot. Also, man:dmesg[8] will print the kernel messages from the current boot.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
When troubleshooting a kernel, make sure to keep a copy of [.filename]#GENERIC#, or some other kernel that is known to work, as a different name that will not get erased on the next build. This is important because every time a new kernel is installed, [.filename]#kernel.old# is overwritten with the last installed kernel, which may or may not be bootable. As soon as possible, move the working kernel by renaming the directory containing the good kernel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mv /boot/kernel /boot/kernel.bad
|
||||
# mv /boot/kernel.good /boot/kernel
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
The kernel works, but man:ps[1] does not::
|
||||
If the kernel version differs from the one that the system utilities have been built with, for example, a kernel built from -CURRENT sources is installed on a -RELEASE system, many system status commands like man:ps[1] and man:vmstat[8] will not work. To fix this, crossref:cutting-edge[makeworld,recompile and install a world] built with the same version of the source tree as the kernel. It is never a good idea to use a different version of the kernel than the rest of the operating system.
|
||||
583
documentation/content/en/books/handbook/l10n/_index.adoc
Normal file
583
documentation/content/en/books/handbook/l10n/_index.adoc
Normal file
|
|
@ -0,0 +1,583 @@
|
|||
---
|
||||
title: Chapter 23. Localization - i18n/L10n Usage and Setup
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/virtualization
|
||||
next: books/handbook/cutting-edge
|
||||
---
|
||||
|
||||
[[l10n]]
|
||||
= Localization - i18n/L10n Usage and Setup
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 23
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/l10n/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/l10n/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/l10n/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[l10n-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
FreeBSD is a distributed project with users and contributors located all over the world. As such, FreeBSD supports localization into many languages, allowing users to view, input, or process data in non-English languages. One can choose from most of the major languages, including, but not limited to: Chinese, German, Japanese, Korean, French, Russian, and Vietnamese.
|
||||
|
||||
The term internationalization has been shortened to i18n, which represents the number of letters between the first and the last letters of `internationalization`. L10n uses the same naming scheme, but from `localization`. The i18n/L10n methods, protocols, and applications allow users to use languages of their choice.
|
||||
|
||||
This chapter discusses the internationalization and localization features of FreeBSD. After reading this chapter, you will know:
|
||||
|
||||
* How locale names are constructed.
|
||||
* How to set the locale for a login shell.
|
||||
* How to configure the console for non-English languages.
|
||||
* How to configure Xorg for different languages.
|
||||
* How to find i18n-compliant applications.
|
||||
* Where to find more information for configuring specific languages.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Know how to crossref:ports[ports,install additional third-party applications].
|
||||
|
||||
[[using-localization]]
|
||||
== Using Localization
|
||||
|
||||
Localization settings are based on three components: the language code, country code, and encoding. Locale names are constructed from these parts as follows:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
LanguageCode_CountryCode.Encoding
|
||||
....
|
||||
|
||||
The _LanguageCode_ and _CountryCode_ are used to determine the country and the specific language variation. <<locale-lang-country>> provides some examples of __LanguageCode_CountryCode__:
|
||||
|
||||
[[locale-lang-country]]
|
||||
.Common Language and Country Codes
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| LanguageCode_Country Code
|
||||
| Description
|
||||
|
||||
|en_US
|
||||
|English, United States
|
||||
|
||||
|ru_RU
|
||||
|Russian, Russia
|
||||
|
||||
|zh_TW
|
||||
|Traditional Chinese, Taiwan
|
||||
|===
|
||||
|
||||
A complete listing of available locales can be found by typing:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% locale -a | more
|
||||
....
|
||||
|
||||
To determine the current locale setting:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% locale
|
||||
....
|
||||
|
||||
Language specific character sets, such as ISO8859-1, ISO8859-15, KOI8-R, and CP437, are described in man:multibyte[3]. The active list of character sets can be found at the http://www.iana.org/assignments/character-sets[IANA Registry].
|
||||
|
||||
Some languages, such as Chinese or Japanese, cannot be represented using ASCII characters and require an extended language encoding using either wide or multibyte characters. Examples of wide or multibyte encodings include EUC and Big5. Older applications may mistake these encodings for control characters while newer applications usually recognize these characters. Depending on the implementation, users may be required to compile an application with wide or multibyte character support, or to configure it correctly.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
FreeBSD uses Xorg-compatible locale encodings.
|
||||
====
|
||||
|
||||
The rest of this section describes the various methods for configuring the locale on a FreeBSD system. The next section will discuss the considerations for finding and compiling applications with i18n support.
|
||||
|
||||
[[setting-locale]]
|
||||
=== Setting Locale for Login Shell
|
||||
|
||||
Locale settings are configured either in a user's [.filename]#~/.login_conf# or in the startup file of the user's shell: [.filename]#~/.profile#, [.filename]#~/.bashrc#, or [.filename]#~/.cshrc#.
|
||||
|
||||
Two environment variables should be set:
|
||||
|
||||
* `LANG`, which sets the locale
|
||||
* `MM_CHARSET`, which sets the MIME character set used by applications
|
||||
|
||||
In addition to the user's shell configuration, these variables should also be set for specific application configuration and Xorg configuration.
|
||||
|
||||
Two methods are available for making the needed variable assignments: the <<login-class,login class>> method, which is the recommended method, and the <<startup-file,startup file>> method. The next two sections demonstrate how to use both methods.
|
||||
|
||||
[[login-class]]
|
||||
==== Login Classes Method
|
||||
|
||||
This first method is the recommended method as it assigns the required environment variables for locale name and MIME character sets for every possible shell. This setup can either be performed by each user or it can be configured for all users by the superuser.
|
||||
|
||||
This minimal example sets both variables for Latin-1 encoding in the [.filename]#.login_conf# of an individual user's home directory:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
me:\
|
||||
:charset=ISO-8859-1:\
|
||||
:lang=de_DE.ISO8859-1:
|
||||
....
|
||||
|
||||
Here is an example of a user's [.filename]#~/.login_conf# that sets the variables for Traditional Chinese in BIG-5 encoding. More variables are needed because some applications do not correctly respect locale variables for Chinese, Japanese, and Korean:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#Users who do not wish to use monetary units or time formats
|
||||
#of Taiwan can manually change each variable
|
||||
me:\
|
||||
:lang=zh_TW.Big5:\
|
||||
:setenv=LC_ALL=zh_TW.Big5,LC_COLLATE=zh_TW.Big5,LC_CTYPE=zh_TW.Big5,LC_MESSAGES=zh_TW.Big5,LC_MONETARY=zh_TW.Big5,LC_NUMERIC=zh_TW.Big5,LC_TIME=zh_TW.Big5:\
|
||||
:charset=big5:\
|
||||
:xmodifiers="@im=gcin": #Set gcin as the XIM Input Server
|
||||
....
|
||||
|
||||
Alternately, the superuser can configure all users of the system for localization. The following variables in [.filename]#/etc/login.conf# are used to set the locale and MIME character set:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
language_name|Account Type Description:\
|
||||
:charset=MIME_charset:\
|
||||
:lang=locale_name:\
|
||||
:tc=default:
|
||||
....
|
||||
|
||||
So, the previous Latin-1 example would look like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
german|German Users Accounts:\
|
||||
:charset=ISO-8859-1:\
|
||||
:lang=de_DE.ISO8859-1:\
|
||||
:tc=default:
|
||||
....
|
||||
|
||||
See man:login.conf[5] for more details about these variables. Note that it already contains pre-defined _russian_ class.
|
||||
|
||||
Whenever [.filename]#/etc/login.conf# is edited, remember to execute the following command to update the capability database:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cap_mkdb /etc/login.conf
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
For an end user, the `cap_mkdb` command will nee to be run on their [.filename]#~/.login_conf# for need to be run on their [.filename]#~/.login_conf# for any changes to take effect.
|
||||
====
|
||||
|
||||
===== Utilities Which Change Login Classes
|
||||
|
||||
In addition to manually editing [.filename]#/etc/login.conf#, several utilities are available for setting the locale for newly created users.
|
||||
|
||||
When using `vipw` to add new users, specify the _language_ to set the locale:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
user:password:1111:11:language:0:0:User Name:/home/user:/bin/sh
|
||||
....
|
||||
|
||||
When using `adduser` to add new users, the default language can be pre-configured for all new users or specified for an individual user.
|
||||
|
||||
If all new users use the same language, set `defaultclass=_language_` in [.filename]#/etc/adduser.conf#.
|
||||
|
||||
To override this setting when creating a user, either input the required locale at this prompt:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Enter login class: default []:
|
||||
....
|
||||
|
||||
or specify the locale to set when invoking `adduser`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# adduser -class language
|
||||
....
|
||||
|
||||
If `pw` is used to add new users, specify the locale as follows:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pw useradd user_name -L language
|
||||
....
|
||||
|
||||
To change the login class of an existing user, `chpass` can be used. Invoke it as superuser and provide the username to edit as the argument.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chpass user_name
|
||||
....
|
||||
|
||||
|
||||
[[startup-file]]
|
||||
==== Shell Startup File Method
|
||||
|
||||
This second method is not recommended as each shell that is used requires manual configuration, where each shell has a different configuration file and differing syntax. As an example, to set the German language for the `sh` shell, these lines could be added to [.filename]#~/.profile# to set the shell for that user only. These lines could also be added to [.filename]#/etc/profile# or [.filename]#/usr/share/skel/dot.profile# to set that shell for all users:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
LANG=de_DE.ISO8859-1; export LANG
|
||||
MM_CHARSET=ISO-8859-1; export MM_CHARSET
|
||||
....
|
||||
|
||||
However, the name of the configuration file and the syntax used differs for the `csh` shell. These are the equivalent settings for [.filename]#~/.csh.login#, [.filename]#/etc/csh.login#, or [.filename]#/usr/share/skel/dot.login#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
setenv LANG de_DE.ISO8859-1
|
||||
setenv MM_CHARSET ISO-8859-1
|
||||
....
|
||||
|
||||
To complicate matters, the syntax needed to configure Xorg in [.filename]#~/.xinitrc# also depends upon the shell. The first example is for the `sh` shell and the second is for the `csh` shell:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
LANG=de_DE.ISO8859-1; export LANG
|
||||
....
|
||||
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
setenv LANG de_DE.ISO8859-1
|
||||
....
|
||||
|
||||
|
||||
[[setting-console]]
|
||||
=== Console Setup
|
||||
|
||||
Several localized fonts are available for the console. To see a listing of available fonts, type `ls /usr/share/syscons/fonts`. To configure the console font, specify the _font_name_, without the [.filename]#.fnt# suffix, in [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
font8x16=font_name
|
||||
font8x14=font_name
|
||||
font8x8=font_name
|
||||
....
|
||||
|
||||
The keymap and screenmap can be set by adding the following to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
scrnmap=screenmap_name
|
||||
keymap=keymap_name
|
||||
keychange="fkey_number sequence"
|
||||
....
|
||||
|
||||
To see the list of available screenmaps, type `ls /usr/share/syscons/scrnmaps`. Do not include the [.filename]#.scm# suffix when specifying _screenmap_name_. A screenmap with a corresponding mapped font is usually needed as a workaround for expanding bit 8 to bit 9 on a VGA adapter's font character matrix so that letters are moved out of the pseudographics area if the screen font uses a bit 8 column.
|
||||
|
||||
To see the list of available keymaps, type `ls /usr/share/syscons/keymaps`. When specifying the _keymap_name_, do not include the [.filename]#.kbd# suffix. To test keymaps without rebooting, use man:kbdmap[1].
|
||||
|
||||
The `keychange` entry is usually needed to program function keys to match the selected terminal type because function key sequences cannot be defined in the keymap.
|
||||
|
||||
Next, set the correct console terminal type in [.filename]#/etc/ttys# for all virtual terminal entries. <<locale-charset>> summarizes the available terminal types.:
|
||||
[[locale-charset]]
|
||||
.Defined Terminal Types for Character Sets
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Character Set
|
||||
| Terminal Type
|
||||
|
||||
|ISO8859-1 or ISO8859-15
|
||||
|`cons25l1`
|
||||
|
||||
|ISO8859-2
|
||||
|`cons25l2`
|
||||
|
||||
|ISO8859-7
|
||||
|`cons25l7`
|
||||
|
||||
|KOI8-R
|
||||
|`cons25r`
|
||||
|
||||
|KOI8-U
|
||||
|`cons25u`
|
||||
|
||||
|CP437 (VGA default)
|
||||
|`cons25`
|
||||
|
||||
|US-ASCII
|
||||
|`cons25w`
|
||||
|===
|
||||
|
||||
For languages with wide or multibyte characters, install a console for that language from the FreeBSD Ports Collection. The available ports are summarized in <<locale-console>>. Once installed, refer to the port's [.filename]#pkg-message# or man pages for configuration and usage instructions.
|
||||
|
||||
[[locale-console]]
|
||||
.Available Console from Ports Collection
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Language
|
||||
| Port Location
|
||||
|
||||
|Traditional Chinese (BIG-5)
|
||||
|package:chinese/big5con[]
|
||||
|
||||
|Chinese/Japanese/Korean
|
||||
|package:chinese/cce[]
|
||||
|
||||
|Chinese/Japanese/Korean
|
||||
|package:chinese/zhcon[]
|
||||
|
||||
|Japanese
|
||||
|package:chinese/kon2[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/kon2-14dot[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/kon2-16dot[]
|
||||
|===
|
||||
|
||||
If moused is enabled in [.filename]#/etc/rc.conf#, additional configuration may be required. By default, the mouse cursor of the man:syscons[4] driver occupies the `0xd0`-`0xd3` range in the character set. If the language uses this range, move the cursor's range by adding the following line to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
mousechar_start=3
|
||||
....
|
||||
|
||||
=== Xorg Setup
|
||||
|
||||
crossref:x11[x11,The X Window System] describes how to install and configure Xorg. When configuring Xorg for localization, additional fonts and input methods are available from the FreeBSD Ports Collection. Application specific i18n settings such as fonts and menus can be tuned in [.filename]#~/.Xresources# and should allow users to view their selected language in graphical application menus.
|
||||
|
||||
The X Input Method (XIM) protocol is an Xorg standard for inputting non-English characters. <<locale-xim>> summarizes the input method applications which are available in the FreeBSD Ports Collection. Additional Fcitx and Uim applications are also available.
|
||||
|
||||
[[locale-xim]]
|
||||
.Available Input Methods
|
||||
[cols="1,1", frame="none", options="header"]
|
||||
|===
|
||||
| Language
|
||||
| Input Method
|
||||
|
||||
|Chinese
|
||||
|package:chinese/gcin[]
|
||||
|
||||
|Chinese
|
||||
|package:chinese/ibus-chewing[]
|
||||
|
||||
|Chinese
|
||||
|package:chinese/ibus-pinyin[]
|
||||
|
||||
|Chinese
|
||||
|package:chinese/oxim[]
|
||||
|
||||
|Chinese
|
||||
|package:chinese/scim-fcitx[]
|
||||
|
||||
|Chinese
|
||||
|package:chinese/scim-pinyin[]
|
||||
|
||||
|Chinese
|
||||
|package:chinese/scim-tables[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/ibus-anthy[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/ibus-mozc[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/ibus-skk[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/im-ja[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/kinput2[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-anthy[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-canna[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-honoka[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-honoka-plugin-romkan[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-honoka-plugin-wnn[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-prime[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-skk[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-tables[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-tomoe[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/scim-uim[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/skkinput[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/skkinput3[]
|
||||
|
||||
|Japanese
|
||||
|package:japanese/uim-anthy[]
|
||||
|
||||
|Korean
|
||||
|package:korean/ibus-hangul[]
|
||||
|
||||
|Korean
|
||||
|package:korean/imhangul[]
|
||||
|
||||
|Korean
|
||||
|package:korean/nabi[]
|
||||
|
||||
|Korean
|
||||
|package:korean/scim-hangul[]
|
||||
|
||||
|Korean
|
||||
|package:korean/scim-tables[]
|
||||
|
||||
|Vietnamese
|
||||
|package:vietnamese/xvnkb[]
|
||||
|
||||
|Vietnamese
|
||||
|package:vietnamese/x-unikey[]
|
||||
|===
|
||||
|
||||
[[l10n-compiling]]
|
||||
== Finding i18n Applications
|
||||
|
||||
i18n applications are programmed using i18n kits under libraries. These allow developers to write a simple file and translate displayed menus and texts to each language.
|
||||
|
||||
The link:https://www.FreeBSD.org/ports/[FreeBSD Ports Collection] contains many applications with built-in support for wide or multibyte characters for several languages. Such applications include `i18n` in their names for easy identification. However, they do not always support the language needed.
|
||||
|
||||
Some applications can be compiled with the specific charset. This is usually done in the port's [.filename]#Makefile# or by passing a value to configure. Refer to the i18n documentation in the respective FreeBSD port's source for more information on how to determine the needed configure value or the port's [.filename]#Makefile# to determine which compile options to use when building the port.
|
||||
|
||||
[[lang-setup]]
|
||||
== Locale Configuration for Specific Languages
|
||||
|
||||
This section provides configuration examples for localizing a FreeBSD system for the Russian language. It then provides some additional resources for localizing other languages.
|
||||
|
||||
[[ru-localize]]
|
||||
=== Russian Language (KOI8-R Encoding)
|
||||
|
||||
This section shows the specific settings needed to localize a FreeBSD system for the Russian language. Refer to <<using-localization,Using Localization>> for a more complete description of each type of setting.
|
||||
|
||||
To set this locale for the login shell, add the following lines to each user's [.filename]#~/.login_conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
me:My Account:\
|
||||
:charset=KOI8-R:\
|
||||
:lang=ru_RU.KOI8-R:
|
||||
....
|
||||
|
||||
To configure the console, add the following lines to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
keymap="ru.utf-8"
|
||||
scrnmap="utf-82cp866"
|
||||
font8x16="cp866b-8x16"
|
||||
font8x14="cp866-8x14"
|
||||
font8x8="cp866-8x8"
|
||||
mousechar_start=3
|
||||
....
|
||||
|
||||
For each `ttyv` entry in [.filename]#/etc/ttys#, use `cons25r` as the terminal type.
|
||||
|
||||
To configure printing, a special output filter is needed to convert from KOI8-R to CP866 since most printers with Russian characters come with hardware code page CP866. FreeBSD includes a default filter for this purpose, [.filename]#/usr/libexec/lpr/ru/koi2alt#. To use this filter, add this entry to [.filename]#/etc/printcap#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lp|Russian local line printer:\
|
||||
:sh:of=/usr/libexec/lpr/ru/koi2alt:\
|
||||
:lp=/dev/lpt0:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs:
|
||||
....
|
||||
|
||||
Refer to man:printcap[5] for a more detailed explanation.
|
||||
|
||||
To configure support for Russian filenames in mounted MS-DOS(R) file systems, include `-L` and the locale name when adding an entry to [.filename]#/etc/fstab#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/dev/ad0s2 /dos/c msdos rw,-Lru_RU.KOI8-R 0 0
|
||||
....
|
||||
|
||||
Refer to man:mount_msdosfs[8] for more details.
|
||||
|
||||
To configure Russian fonts for Xorg, install the package:x11-fonts/xorg-fonts-cyrillic[] package. Then, check the `"Files"` section in [.filename]#/etc/X11/xorg.conf#. The following line must be added _before_ any other `FontPath` entries:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
FontPath "/usr/local/lib/X11/fonts/cyrillic"
|
||||
....
|
||||
|
||||
Additional Cyrillic fonts are available in the Ports Collection.
|
||||
|
||||
To activate a Russian keyboard, add the following to the `"Keyboard"` section of [.filename]#/etc/xorg.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
Option "XkbLayout" "us,ru"
|
||||
Option "XkbOptions" "grp:toggle"
|
||||
....
|
||||
|
||||
Make sure that `XkbDisable` is commented out in that file.
|
||||
|
||||
For `grp:toggle` use kbd:[Right Alt], for `grp:ctrl_shift_toggle` use kbd[Ctrl+Shift]. For `grp:caps_toggle` use kbd:[CapsLock]. The old kbd:[CapsLock] function is still available in LAT mode only using kbd[Shift+CapsLock]. `grp:caps_toggle` does not work in Xorg for some unknown reason.
|
||||
|
||||
If the keyboard has "Windows(R)" keys, and some non-alphabetical keys are mapped incorrectly, add the following line to [.filename]#/etc/xorg.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
Option "XkbVariant" ",winkeys"
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The Russian XKB keyboard may not work with non-localized applications. Minimally localized applications should call a `XtSetLanguageProc (NULL, NULL, NULL);` function early in the program.
|
||||
====
|
||||
|
||||
See http://koi8.pp.ru/xwin.html[http://koi8.pp.ru/xwin.html] for more instructions on localizing Xorg applications. For more general information about KOI8-R encoding, refer to http://koi8.pp.ru/[http://koi8.pp.ru/].
|
||||
|
||||
=== Additional Language-Specific Resources
|
||||
|
||||
This section lists some additional resources for configuring other locales.
|
||||
|
||||
Traditional Chinese for Taiwan::
|
||||
The FreeBSD-Taiwan Project has a Chinese HOWTO for FreeBSD at http://netlab.cse.yzu.edu.tw/\~statue/freebsd/zh-tut/[http://netlab.cse.yzu.edu.tw/~statue/freebsd/zh-tut/].
|
||||
|
||||
Greek Language Localization::
|
||||
A complete article on Greek support in FreeBSD is available https://www.FreeBSD.org/doc/gr/articles/greek-language-support/[here], in Greek only, as part of the official FreeBSD Greek documentation.
|
||||
|
||||
Japanese and Korean Language Localization::
|
||||
For Japanese, refer to http://www.jp.FreeBSD.org/[http://www.jp.FreeBSD.org/], and for Korean, refer to http://www.kr.FreeBSD.org/[http://www.kr.FreeBSD.org/].
|
||||
|
||||
Non-English FreeBSD Documentation::
|
||||
Some FreeBSD contributors have translated parts of the FreeBSD documentation to other languages. They are available through links on the link:https://www.FreeBSD.org/[FreeBSD web site] or in [.filename]#/usr/shared/doc#.
|
||||
254
documentation/content/en/books/handbook/linuxemu/_index.adoc
Normal file
254
documentation/content/en/books/handbook/linuxemu/_index.adoc
Normal file
|
|
@ -0,0 +1,254 @@
|
|||
---
|
||||
title: Chapter 10. Linux® Binary Compatibility
|
||||
part: Part II. Common Tasks
|
||||
prev: books/handbook/printing
|
||||
next: books/handbook/wine
|
||||
---
|
||||
|
||||
[[linuxemu]]
|
||||
= Linux(R) Binary Compatibility
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 10
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/linuxemu/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/linuxemu/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/linuxemu/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[linuxemu-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
FreeBSD provides binary compatibility with Linux(R), allowing users to install and run most Linux(R) binaries on a FreeBSD system without having to first modify the binary. It has even been reported that, in some situations, Linux(R) binaries perform better on FreeBSD than they do on Linux(R).
|
||||
|
||||
However, some Linux(R)-specific operating system features are not supported under FreeBSD. For example, Linux(R) binaries will not work on FreeBSD if they overly use i386(TM) specific calls, such as enabling virtual 8086 mode.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Support for 64-bit binary compatibility with Linux(R) was added in FreeBSD 10.3.
|
||||
====
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* How to enable Linux(R) binary compatibility on a FreeBSD system.
|
||||
* How to install additional Linux(R) shared libraries.
|
||||
* How to install Linux(R) applications on a FreeBSD system.
|
||||
* The implementation details of Linux(R) compatibility in FreeBSD.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Know how to install crossref:ports[ports,additional third-party software].
|
||||
|
||||
[[linuxemu-lbc-install]]
|
||||
== Configuring Linux(R) Binary Compatibility
|
||||
|
||||
By default, Linux(R) libraries are not installed and Linux(R) binary compatibility is not enabled. Linux(R) libraries can either be installed manually or from the FreeBSD Ports Collection.
|
||||
|
||||
Before attempting to build the port, load the Linux(R) kernel module, otherwise the build will fail:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload linux
|
||||
....
|
||||
|
||||
For 64-bit compatibility:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload linux64
|
||||
....
|
||||
|
||||
To verify that the module is loaded:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% kldstat
|
||||
Id Refs Address Size Name
|
||||
1 2 0xc0100000 16bdb8 kernel
|
||||
7 1 0xc24db000 d000 linux.ko
|
||||
....
|
||||
|
||||
The package:emulators/linux_base-c7[] package or port is the easiest way to install a base set of Linux(R) libraries and binaries on a FreeBSD system. To install the port:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install emulators/linux_base-c7
|
||||
....
|
||||
|
||||
For Linux(R) compatibility to be enabled at boot time, add this line to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
linux_enable="YES"
|
||||
....
|
||||
|
||||
On 64-bit machines, [.filename]#/etc/rc.d/abi# will automatically load the module for 64-bit emulation.
|
||||
|
||||
Since the Linux(R) binary compatibility layer has gained support for running both 32- and 64-bit Linux(R) binaries (on 64-bit x86 hosts), it is no longer possible to link the emulation functionality statically into a custom kernel.
|
||||
|
||||
[[linuxemu-libs-manually]]
|
||||
=== Installing Additional Libraries Manually
|
||||
|
||||
If a Linux(R) application complains about missing shared libraries after configuring Linux(R) binary compatibility, determine which shared libraries the Linux(R) binary needs and install them manually.
|
||||
|
||||
From a Linux(R) system, `ldd` can be used to determine which shared libraries the application needs. For example, to check which shared libraries `linuxdoom` needs, run this command from a Linux(R) system that has Doom installed:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% ldd linuxdoom
|
||||
libXt.so.3 (DLL Jump 3.1) => /usr/X11/lib/libXt.so.3.1.0
|
||||
libX11.so.3 (DLL Jump 3.1) => /usr/X11/lib/libX11.so.3.1.0
|
||||
libc.so.4 (DLL Jump 4.5pl26) => /lib/libc.so.4.6.29
|
||||
....
|
||||
|
||||
Then, copy all the files in the last column of the output from the Linux(R) system into [.filename]#/compat/linux# on the FreeBSD system. Once copied, create symbolic links to the names in the first column. This example will result in the following files on the FreeBSD system:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
/compat/linux/usr/X11/lib/libXt.so.3.1.0
|
||||
/compat/linux/usr/X11/lib/libXt.so.3 -> libXt.so.3.1.0
|
||||
/compat/linux/usr/X11/lib/libX11.so.3.1.0
|
||||
/compat/linux/usr/X11/lib/libX11.so.3 -> libX11.so.3.1.0
|
||||
/compat/linux/lib/libc.so.4.6.29
|
||||
/compat/linux/lib/libc.so.4 -> libc.so.4.6.29
|
||||
....
|
||||
|
||||
If a Linux(R) shared library already exists with a matching major revision number to the first column of the `ldd` output, it does not need to be copied to the file named in the last column, as the existing library should work. It is advisable to copy the shared library if it is a newer version, though. The old one can be removed, as long as the symbolic link points to the new one.
|
||||
|
||||
For example, these libraries already exist on the FreeBSD system:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
/compat/linux/lib/libc.so.4.6.27
|
||||
/compat/linux/lib/libc.so.4 -> libc.so.4.6.27
|
||||
....
|
||||
|
||||
and `ldd` indicates that a binary requires a later version:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
libc.so.4 (DLL Jump 4.5pl26) -> libc.so.4.6.29
|
||||
....
|
||||
|
||||
Since the existing library is only one or two versions out of date in the last digit, the program should still work with the slightly older version. However, it is safe to replace the existing [.filename]#libc.so# with the newer version:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
/compat/linux/lib/libc.so.4.6.29
|
||||
/compat/linux/lib/libc.so.4 -> libc.so.4.6.29
|
||||
....
|
||||
|
||||
Generally, one will need to look for the shared libraries that Linux(R) binaries depend on only the first few times that a Linux(R) program is installed on FreeBSD. After a while, there will be a sufficient set of Linux(R) shared libraries on the system to be able to run newly installed Linux(R) binaries without any extra work.
|
||||
|
||||
=== Installing Linux(R) ELF Binaries
|
||||
|
||||
ELF binaries sometimes require an extra step. When an unbranded ELF binary is executed, it will generate an error message:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% ./my-linux-elf-binary
|
||||
ELF binary type not known
|
||||
Abort
|
||||
....
|
||||
|
||||
To help the FreeBSD kernel distinguish between a FreeBSD ELF binary and a Linux(R) binary, use man:brandelf[1]:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% brandelf -t Linux my-linux-elf-binary
|
||||
....
|
||||
|
||||
Since the GNU toolchain places the appropriate branding information into ELF binaries automatically, this step is usually not necessary.
|
||||
|
||||
=== Installing a Linux(R) RPM Based Application
|
||||
|
||||
To install a Linux(R) RPM-based application, first install the package:archivers/rpm4[] package or port. Once installed, `root` can use this command to install a [.filename]#.rpm#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /compat/linux
|
||||
# rpm2cpio < /path/to/linux.archive.rpm | cpio -id
|
||||
....
|
||||
|
||||
If necessary, `brandelf` the installed ELF binaries. Note that this will prevent a clean uninstall.
|
||||
|
||||
=== Configuring the Hostname Resolver
|
||||
|
||||
If DNS does not work or this error appears:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
resolv+: "bind" is an invalid keyword resolv+:
|
||||
"hosts" is an invalid keyword
|
||||
....
|
||||
|
||||
configure [.filename]#/compat/linux/etc/host.conf# as follows:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
order hosts, bind
|
||||
multi on
|
||||
....
|
||||
|
||||
This specifies that [.filename]#/etc/hosts# is searched first and DNS is searched second. When [.filename]#/compat/linux/etc/host.conf# does not exist, Linux(R) applications use [.filename]#/etc/host.conf# and complain about the incompatible FreeBSD syntax. Remove `bind` if a name server is not configured using [.filename]#/etc/resolv.conf#.
|
||||
|
||||
[[linuxemu-advanced]]
|
||||
== Advanced Topics
|
||||
|
||||
This section describes how Linux(R) binary compatibility works and is based on an email written to {freebsd-chat} by Terry Lambert mailto:tlambert@primenet.com[tlambert@primenet.com] (Message ID: `<199906020108.SAA07001@usr09.primenet.com>`).
|
||||
|
||||
FreeBSD has an abstraction called an "execution class loader". This is a wedge into the man:execve[2] system call.
|
||||
|
||||
Historically, the UNIX(R) loader examined the magic number (generally the first 4 or 8 bytes of the file) to see if it was a binary known to the system, and if so, invoked the binary loader.
|
||||
|
||||
If it was not the binary type for the system, the man:execve[2] call returned a failure, and the shell attempted to start executing it as shell commands. The assumption was a default of "whatever the current shell is".
|
||||
|
||||
Later, a hack was made for man:sh[1] to examine the first two characters, and if they were `:\n`, it invoked the man:csh[1] shell instead.
|
||||
|
||||
FreeBSD has a list of loaders, instead of a single loader, with a fallback to the `#!` loader for running shell interpreters or shell scripts.
|
||||
|
||||
For the Linux(R) ABI support, FreeBSD sees the magic number as an ELF binary. The ELF loader looks for a specialized _brand_, which is a comment section in the ELF image, and which is not present on SVR4/Solaris(TM) ELF binaries.
|
||||
|
||||
For Linux(R) binaries to function, they must be _branded_ as type `Linux` using man:brandelf[1]:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# brandelf -t Linux file
|
||||
....
|
||||
|
||||
When the ELF loader sees the `Linux` brand, the loader replaces a pointer in the `proc` structure. All system calls are indexed through this pointer. In addition, the process is flagged for special handling of the trap vector for the signal trampoline code, and several other (minor) fix-ups that are handled by the Linux(R) kernel module.
|
||||
|
||||
The Linux(R) system call vector contains, among other things, a list of `sysent[]` entries whose addresses reside in the kernel module.
|
||||
|
||||
When a system call is called by the Linux(R) binary, the trap code dereferences the system call function pointer off the `proc` structure, and gets the Linux(R), not the FreeBSD, system call entry points.
|
||||
|
||||
Linux(R) mode dynamically _reroots_ lookups. This is, in effect, equivalent to `union` to file system mounts. First, an attempt is made to lookup the file in [.filename]#/compat/linux/original-path#. If that fails, the lookup is done in [.filename]#/original-path#. This makes sure that binaries that require other binaries can run. For example, the Linux(R) toolchain can all run under Linux(R) ABI support. It also means that the Linux(R) binaries can load and execute FreeBSD binaries, if there are no corresponding Linux(R) binaries present, and that a man:uname[1] command can be placed in the [.filename]#/compat/linux# directory tree to ensure that the Linux(R) binaries cannot tell they are not running on Linux(R).
|
||||
|
||||
In effect, there is a Linux(R) kernel in the FreeBSD kernel. The various underlying functions that implement all of the services provided by the kernel are identical to both the FreeBSD system call table entries, and the Linux(R) system call table entries: file system operations, virtual memory operations, signal delivery, and System V IPC. The only difference is that FreeBSD binaries get the FreeBSD _glue_ functions, and Linux(R) binaries get the Linux(R) _glue_ functions. The FreeBSD _glue_ functions are statically linked into the kernel, and the Linux(R) _glue_ functions can be statically linked, or they can be accessed via a kernel module.
|
||||
|
||||
Technically, this is not really emulation, it is an ABI implementation. It is sometimes called "Linux(R) emulation" because the implementation was done at a time when there was no other word to describe what was going on. Saying that FreeBSD ran Linux(R) binaries was not true, since the code was not compiled in.
|
||||
804
documentation/content/en/books/handbook/mac/_index.adoc
Normal file
804
documentation/content/en/books/handbook/mac/_index.adoc
Normal file
|
|
@ -0,0 +1,804 @@
|
|||
---
|
||||
title: Chapter 16. Mandatory Access Control
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/jails
|
||||
next: books/handbook/audit
|
||||
---
|
||||
|
||||
[[mac]]
|
||||
= Mandatory Access Control
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 16
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/mac/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/mac/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/mac/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[mac-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
FreeBSD supports security extensions based on the POSIX(R).1e draft. These security mechanisms include file system Access Control Lists (crossref:security[fs-acl,“Access Control Lists”]) and Mandatory Access Control (MAC). MAC allows access control modules to be loaded in order to implement security policies. Some modules provide protections for a narrow subset of the system, hardening a particular service. Others provide comprehensive labeled security across all subjects and objects. The mandatory part of the definition indicates that enforcement of controls is performed by administrators and the operating system. This is in contrast to the default security mechanism of Discretionary Access Control (DAC) where enforcement is left to the discretion of users.
|
||||
|
||||
This chapter focuses on the MAC framework and the set of pluggable security policy modules FreeBSD provides for enabling various security mechanisms.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* The terminology associated with the MAC framework.
|
||||
* The capabilities of MAC security policy modules as well as the difference between a labeled and non-labeled policy.
|
||||
* The considerations to take into account before configuring a system to use the MAC framework.
|
||||
* Which MAC security policy modules are included in FreeBSD and how to configure them.
|
||||
* How to implement a more secure environment using the MAC framework.
|
||||
* How to test the MAC configuration to ensure the framework has been properly implemented.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Understand UNIX(R) and FreeBSD basics (crossref:basics[basics,FreeBSD Basics]).
|
||||
* Have some familiarity with security and how it pertains to FreeBSD (crossref:security[security,Security]).
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
Improper MAC configuration may cause loss of system access, aggravation of users, or inability to access the features provided by Xorg. More importantly, MAC should not be relied upon to completely secure a system. The MAC framework only augments an existing security policy. Without sound security practices and regular security checks, the system will never be completely secure.
|
||||
|
||||
The examples contained within this chapter are for demonstration purposes and the example settings should _not_ be implemented on a production system. Implementing any security policy takes a good deal of understanding, proper design, and thorough testing.
|
||||
====
|
||||
|
||||
While this chapter covers a broad range of security issues relating to the MAC framework, the development of new MAC security policy modules will not be covered. A number of security policy modules included with the MAC framework have specific characteristics which are provided for both testing and new module development. Refer to man:mac_test[4], man:mac_stub[4] and man:mac_none[4] for more information on these security policy modules and the various mechanisms they provide.
|
||||
|
||||
[[mac-inline-glossary]]
|
||||
== Key Terms
|
||||
|
||||
The following key terms are used when referring to the MAC framework:
|
||||
|
||||
* _compartment_: a set of programs and data to be partitioned or separated, where users are given explicit access to specific component of a system. A compartment represents a grouping, such as a work group, department, project, or topic. Compartments make it possible to implement a need-to-know-basis security policy.
|
||||
* _integrity_: the level of trust which can be placed on data. As the integrity of the data is elevated, so does the ability to trust that data.
|
||||
* _level_: the increased or decreased setting of a security attribute. As the level increases, its security is considered to elevate as well.
|
||||
* _label_: a security attribute which can be applied to files, directories, or other items in the system. It could be considered a confidentiality stamp. When a label is placed on a file, it describes the security properties of that file and will only permit access by files, users, and resources with a similar security setting. The meaning and interpretation of label values depends on the policy configuration. Some policies treat a label as representing the integrity or secrecy of an object while other policies might use labels to hold rules for access.
|
||||
* _multilabel_: this property is a file system option which can be set in single-user mode using man:tunefs[8], during boot using man:fstab[5], or during the creation of a new file system. This option permits an administrator to apply different MAC labels on different objects. This option only applies to security policy modules which support labeling.
|
||||
* _single label_: a policy where the entire file system uses one label to enforce access control over the flow of data. Whenever `multilabel` is not set, all files will conform to the same label setting.
|
||||
* _object_: an entity through which information flows under the direction of a _subject_. This includes directories, files, fields, screens, keyboards, memory, magnetic storage, printers or any other data storage or moving device. An object is a data container or a system resource. Access to an object effectively means access to its data.
|
||||
* _subject_: any active entity that causes information to flow between _objects_ such as a user, user process, or system process. On FreeBSD, this is almost always a thread acting in a process on behalf of a user.
|
||||
* _policy_: a collection of rules which defines how objectives are to be achieved. A policy usually documents how certain items are to be handled. This chapter considers a policy to be a collection of rules which controls the flow of data and information and defines who has access to that data and information.
|
||||
* _high-watermark_: this type of policy permits the raising of security levels for the purpose of accessing higher level information. In most cases, the original level is restored after the process is complete. Currently, the FreeBSD MAC framework does not include this type of policy.
|
||||
* _low-watermark_: this type of policy permits lowering security levels for the purpose of accessing information which is less secure. In most cases, the original security level of the user is restored after the process is complete. The only security policy module in FreeBSD to use this is man:mac_lomac[4].
|
||||
* _sensitivity_: usually used when discussing Multilevel Security (MLS). A sensitivity level describes how important or secret the data should be. As the sensitivity level increases, so does the importance of the secrecy, or confidentiality, of the data.
|
||||
|
||||
[[mac-understandlabel]]
|
||||
== Understanding MAC Labels
|
||||
|
||||
A MAC label is a security attribute which may be applied to subjects and objects throughout the system. When setting a label, the administrator must understand its implications in order to prevent unexpected or undesired behavior of the system. The attributes available on an object depend on the loaded policy module, as policy modules interpret their attributes in different ways.
|
||||
|
||||
The security label on an object is used as a part of a security access control decision by a policy. With some policies, the label contains all of the information necessary to make a decision. In other policies, the labels may be processed as part of a larger rule set.
|
||||
|
||||
There are two types of label policies: single label and multi label. By default, the system will use single label. The administrator should be aware of the pros and cons of each in order to implement policies which meet the requirements of the system's security model.
|
||||
|
||||
A single label security policy only permits one label to be used for every subject or object. Since a single label policy enforces one set of access permissions across the entire system, it provides lower administration overhead, but decreases the flexibility of policies which support labeling. However, in many environments, a single label policy may be all that is required.
|
||||
|
||||
A single label policy is somewhat similar to DAC as `root` configures the policies so that users are placed in the appropriate categories and access levels. A notable difference is that many policy modules can also restrict `root`. Basic control over objects will then be released to the group, but `root` may revoke or modify the settings at any time.
|
||||
|
||||
When appropriate, a multi label policy can be set on a UFS file system by passing `multilabel` to man:tunefs[8]. A multi label policy permits each subject or object to have its own independent MAC label. The decision to use a multi label or single label policy is only required for policies which implement the labeling feature, such as `biba`, `lomac`, and `mls`. Some policies, such as `seeotheruids`, `portacl` and `partition`, do not use labels at all.
|
||||
|
||||
Using a multi label policy on a partition and establishing a multi label security model can increase administrative overhead as everything in that file system has a label. This includes directories, files, and even device nodes.
|
||||
|
||||
The following command will set `multilabel` on the specified UFS file system. This may only be done in single-user mode and is not a requirement for the swap file system:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# tunefs -l enable /
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Some users have experienced problems with setting the `multilabel` flag on the root partition. If this is the case, please review <<mac-troubleshoot>>.
|
||||
====
|
||||
|
||||
Since the multi label policy is set on a per-file system basis, a multi label policy may not be needed if the file system layout is well designed. Consider an example security MAC model for a FreeBSD web server. This machine uses the single label, `biba/high`, for everything in the default file systems. If the web server needs to run at `biba/low` to prevent write up capabilities, it could be installed to a separate UFS [.filename]#/usr/local# file system set at `biba/low`.
|
||||
|
||||
=== Label Configuration
|
||||
|
||||
Virtually all aspects of label policy module configuration will be performed using the base system utilities. These commands provide a simple interface for object or subject configuration or the manipulation and verification of the configuration.
|
||||
|
||||
All configuration may be done using `setfmac`, which is used to set MAC labels on system objects, and `setpmac`, which is used to set the labels on system subjects. For example, to set the `biba` MAC label to `high` on [.filename]#test#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setfmac biba/high test
|
||||
....
|
||||
|
||||
If the configuration is successful, the prompt will be returned without error. A common error is `Permission denied` which usually occurs when the label is being set or modified on a restricted object. Other conditions may produce different failures. For instance, the file may not be owned by the user attempting to relabel the object, the object may not exist, or the object may be read-only. A mandatory policy will not allow the process to relabel the file, maybe because of a property of the file, a property of the process, or a property of the proposed new label value. For example, if a user running at low integrity tries to change the label of a high integrity file, or a user running at low integrity tries to change the label of a low integrity file to a high integrity label, these operations will fail.
|
||||
|
||||
The system administrator may use `setpmac` to override the policy module's settings by assigning a different label to the invoked process:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setfmac biba/high test
|
||||
Permission denied
|
||||
# setpmac biba/low setfmac biba/high test
|
||||
# getfmac test
|
||||
test: biba/high
|
||||
....
|
||||
|
||||
For currently running processes, such as sendmail, `getpmac` is usually used instead. This command takes a process ID (PID) in place of a command name. If users attempt to manipulate a file not in their access, subject to the rules of the loaded policy modules, the `Operation not permitted` error will be displayed.
|
||||
|
||||
=== Predefined Labels
|
||||
|
||||
A few FreeBSD policy modules which support the labeling feature offer three predefined labels: `low`, `equal`, and `high`, where:
|
||||
|
||||
* `low` is considered the lowest label setting an object or subject may have. Setting this on objects or subjects blocks their access to objects or subjects marked high.
|
||||
* `equal` sets the subject or object to be disabled or unaffected and should only be placed on objects considered to be exempt from the policy.
|
||||
* `high` grants an object or subject the highest setting available in the Biba and MLS policy modules.
|
||||
|
||||
Such policy modules include man:mac_biba[4], man:mac_mls[4] and man:mac_lomac[4]. Each of the predefined labels establishes a different information flow directive. Refer to the manual page of the module to determine the traits of the generic label configurations.
|
||||
|
||||
=== Numeric Labels
|
||||
|
||||
The Biba and MLS policy modules support a numeric label which may be set to indicate the precise level of hierarchical control. This numeric level is used to partition or sort information into different groups of classification, only permitting access to that group or a higher group level. For example:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
biba/10:2+3+6(5:2+3-20:2+3+4+5+6)
|
||||
....
|
||||
|
||||
may be interpreted as "Biba Policy Label/Grade 10:Compartments 2, 3 and 6: (grade 5 ...")
|
||||
|
||||
In this example, the first grade would be considered the effective grade with effective compartments, the second grade is the low grade, and the last one is the high grade. In most configurations, such fine-grained settings are not needed as they are considered to be advanced configurations.
|
||||
|
||||
System objects only have a current grade and compartment. System subjects reflect the range of available rights in the system, and network interfaces, where they are used for access control.
|
||||
|
||||
The grade and compartments in a subject and object pair are used to construct a relationship known as _dominance_, in which a subject dominates an object, the object dominates the subject, neither dominates the other, or both dominate each other. The "both dominate" case occurs when the two labels are equal. Due to the information flow nature of Biba, a user has rights to a set of compartments that might correspond to projects, but objects also have a set of compartments. Users may have to subset their rights using `su` or `setpmac` in order to access objects in a compartment from which they are not restricted.
|
||||
|
||||
=== User Labels
|
||||
|
||||
Users are required to have labels so that their files and processes properly interact with the security policy defined on the system. This is configured in [.filename]#/etc/login.conf# using login classes. Every policy module that uses labels will implement the user class setting.
|
||||
|
||||
To set the user class default label which will be enforced by MAC, add a `label` entry. An example `label` entry containing every policy module is displayed below. Note that in a real configuration, the administrator would never enable every policy module. It is recommended that the rest of this chapter be reviewed before any configuration is implemented.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
default:\
|
||||
:copyright=/etc/COPYRIGHT:\
|
||||
:welcome=/etc/motd:\
|
||||
:setenv=MAIL=/var/mail/$,BLOCKSIZE=K:\
|
||||
:path=~/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:\
|
||||
:manpath=/usr/shared/man /usr/local/man:\
|
||||
:nologin=/usr/sbin/nologin:\
|
||||
:cputime=1h30m:\
|
||||
:datasize=8M:\
|
||||
:vmemoryuse=100M:\
|
||||
:stacksize=2M:\
|
||||
:memorylocked=4M:\
|
||||
:memoryuse=8M:\
|
||||
:filesize=8M:\
|
||||
:coredumpsize=8M:\
|
||||
:openfiles=24:\
|
||||
:maxproc=32:\
|
||||
:priority=0:\
|
||||
:requirehome:\
|
||||
:passwordtime=91d:\
|
||||
:umask=022:\
|
||||
:ignoretime@:\
|
||||
:label=partition/13,mls/5,biba/10(5-15),lomac/10[2]:
|
||||
....
|
||||
|
||||
While users can not modify the default value, they may change their label after they login, subject to the constraints of the policy. The example above tells the Biba policy that a process's minimum integrity is `5`, its maximum is `15`, and the default effective label is `10`. The process will run at `10` until it chooses to change label, perhaps due to the user using `setpmac`, which will be constrained by Biba to the configured range.
|
||||
|
||||
After any change to [.filename]#login.conf#, the login class capability database must be rebuilt using `cap_mkdb`.
|
||||
|
||||
Many sites have a large number of users requiring several different user classes. In depth planning is required as this can become difficult to manage.
|
||||
|
||||
=== Network Interface Labels
|
||||
|
||||
Labels may be set on network interfaces to help control the flow of data across the network. Policies using network interface labels function in the same way that policies function with respect to objects. Users at high settings in Biba, for example, will not be permitted to access network interfaces with a label of `low`.
|
||||
|
||||
When setting the MAC label on network interfaces, `maclabel` may be passed to `ifconfig`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ifconfig bge0 maclabel biba/equal
|
||||
....
|
||||
|
||||
This example will set the MAC label of `biba/equal` on the `bge0` interface. When using a setting similar to `biba/high(low-high)`, the entire label should be quoted to prevent an error from being returned.
|
||||
|
||||
Each policy module which supports labeling has a tunable which may be used to disable the MAC label on network interfaces. Setting the label to `equal` will have a similar effect. Review the output of `sysctl`, the policy manual pages, and the information in the rest of this chapter for more information on those tunables.
|
||||
|
||||
[[mac-planning]]
|
||||
== Planning the Security Configuration
|
||||
|
||||
Before implementing any MAC policies, a planning phase is recommended. During the planning stages, an administrator should consider the implementation requirements and goals, such as:
|
||||
|
||||
* How to classify information and resources available on the target systems.
|
||||
* Which information or resources to restrict access to along with the type of restrictions that should be applied.
|
||||
* Which MAC modules will be required to achieve this goal.
|
||||
|
||||
A trial run of the trusted system and its configuration should occur _before_ a MAC implementation is used on production systems. Since different environments have different needs and requirements, establishing a complete security profile will decrease the need of changes once the system goes live.
|
||||
|
||||
Consider how the MAC framework augments the security of the system as a whole. The various security policy modules provided by the MAC framework could be used to protect the network and file systems or to block users from accessing certain ports and sockets. Perhaps the best use of the policy modules is to load several security policy modules at a time in order to provide a MLS environment. This approach differs from a hardening policy, which typically hardens elements of a system which are used only for specific purposes. The downside to MLS is increased administrative overhead.
|
||||
|
||||
The overhead is minimal when compared to the lasting effect of a framework which provides the ability to pick and choose which policies are required for a specific configuration and which keeps performance overhead down. The reduction of support for unneeded policies can increase the overall performance of the system as well as offer flexibility of choice. A good implementation would consider the overall security requirements and effectively implement the various security policy modules offered by the framework.
|
||||
|
||||
A system utilizing MAC guarantees that a user will not be permitted to change security attributes at will. All user utilities, programs, and scripts must work within the constraints of the access rules provided by the selected security policy modules and control of the MAC access rules is in the hands of the system administrator.
|
||||
|
||||
It is the duty of the system administrator to carefully select the correct security policy modules. For an environment that needs to limit access control over the network, the man:mac_portacl[4], man:mac_ifoff[4], and man:mac_biba[4] policy modules make good starting points. For an environment where strict confidentiality of file system objects is required, consider the man:mac_bsdextended[4] and man:mac_mls[4] policy modules.
|
||||
|
||||
Policy decisions could be made based on network configuration. If only certain users should be permitted access to man:ssh[1], the man:mac_portacl[4] policy module is a good choice. In the case of file systems, access to objects might be considered confidential to some users, but not to others. As an example, a large development team might be broken off into smaller projects where developers in project A might not be permitted to access objects written by developers in project B. Yet both projects might need to access objects created by developers in project C. Using the different security policy modules provided by the MAC framework, users could be divided into these groups and then given access to the appropriate objects.
|
||||
|
||||
Each security policy module has a unique way of dealing with the overall security of a system. Module selection should be based on a well thought out security policy which may require revision and reimplementation. Understanding the different security policy modules offered by the MAC framework will help administrators choose the best policies for their situations.
|
||||
|
||||
The rest of this chapter covers the available modules, describes their use and configuration, and in some cases, provides insight on applicable situations.
|
||||
|
||||
[CAUTION]
|
||||
====
|
||||
|
||||
Implementing MAC is much like implementing a firewall since care must be taken to prevent being completely locked out of the system. The ability to revert back to a previous configuration should be considered and the implementation of MAC over a remote connection should be done with extreme caution.
|
||||
====
|
||||
|
||||
[[mac-policies]]
|
||||
== Available MAC Policies
|
||||
|
||||
The default FreeBSD kernel includes `options MAC`. This means that every module included with the MAC framework can be loaded with `kldload` as a run-time kernel module. After testing the module, add the module name to [.filename]#/boot/loader.conf# so that it will load during boot. Each module also provides a kernel option for those administrators who choose to compile their own custom kernel.
|
||||
|
||||
FreeBSD includes a group of policies that will cover most security requirements. Each policy is summarized below. The last three policies support integer settings in place of the three default labels.
|
||||
|
||||
[[mac-seeotheruids]]
|
||||
=== The MAC See Other UIDs Policy
|
||||
|
||||
Module name: [.filename]#mac_seeotheruids.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_SEEOTHERUIDS`
|
||||
|
||||
Boot option: `mac_seeotheruids_load="YES"`
|
||||
|
||||
The man:mac_seeotheruids[4] module extends the `security.bsd.see_other_uids` and `security.bsd.see_other_gids sysctl` tunables. This option does not require any labels to be set before configuration and can operate transparently with other modules.
|
||||
|
||||
After loading the module, the following `sysctl` tunables may be used to control its features:
|
||||
|
||||
* `security.mac.seeotheruids.enabled` enables the module and implements the default settings which deny users the ability to view processes and sockets owned by other users.
|
||||
* `security.mac.seeotheruids.specificgid_enabled` allows specified groups to be exempt from this policy. To exempt specific groups, use the `security.mac.seeotheruids.specificgid=_XXX_ sysctl` tunable, replacing _XXX_ with the numeric group ID to be exempted.
|
||||
* `security.mac.seeotheruids.primarygroup_enabled` is used to exempt specific primary groups from this policy. When using this tunable, `security.mac.seeotheruids.specificgid_enabled` may not be set.
|
||||
|
||||
[[mac-bsdextended]]
|
||||
=== The MAC BSD Extended Policy
|
||||
|
||||
Module name: [.filename]#mac_bsdextended.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_BSDEXTENDED`
|
||||
|
||||
Boot option: `mac_bsdextended_load="YES"`
|
||||
|
||||
The man:mac_bsdextended[4] module enforces a file system firewall. It provides an extension to the standard file system permissions model, permitting an administrator to create a firewall-like ruleset to protect files, utilities, and directories in the file system hierarchy. When access to a file system object is attempted, the list of rules is iterated until either a matching rule is located or the end is reached. This behavior may be changed using `security.mac.bsdextended.firstmatch_enabled`. Similar to other firewall modules in FreeBSD, a file containing the access control rules can be created and read by the system at boot time using an man:rc.conf[5] variable.
|
||||
|
||||
The rule list may be entered using man:ugidfw[8] which has a syntax similar to man:ipfw[8]. More tools can be written by using the functions in the man:libugidfw[3] library.
|
||||
|
||||
After the man:mac_bsdextended[4] module has been loaded, the following command may be used to list the current rule configuration:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ugidfw list
|
||||
0 slots, 0 rules
|
||||
....
|
||||
|
||||
By default, no rules are defined and everything is completely accessible. To create a rule which blocks all access by users but leaves `root` unaffected:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ugidfw add subject not uid root new object not uid root mode n
|
||||
....
|
||||
|
||||
While this rule is simple to implement, it is a very bad idea as it blocks all users from issuing any commands. A more realistic example blocks `user1` all access, including directory listings, to ``_user2_``'s home directory:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ugidfw set 2 subject uid user1 object uid user2 mode n
|
||||
# ugidfw set 3 subject uid user1 object gid user2 mode n
|
||||
....
|
||||
|
||||
Instead of `user1`, `not uid _user2_` could be used in order to enforce the same access restrictions for all users. However, the `root` user is unaffected by these rules.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Extreme caution should be taken when working with this module as incorrect use could block access to certain parts of the file system.
|
||||
====
|
||||
|
||||
[[mac-ifoff]]
|
||||
=== The MAC Interface Silencing Policy
|
||||
|
||||
Module name: [.filename]#mac_ifoff.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_IFOFF`
|
||||
|
||||
Boot option: `mac_ifoff_load="YES"`
|
||||
|
||||
The man:mac_ifoff[4] module is used to disable network interfaces on the fly and to keep network interfaces from being brought up during system boot. It does not use labels and does not depend on any other MAC modules.
|
||||
|
||||
Most of this module's control is performed through these `sysctl` tunables:
|
||||
|
||||
* `security.mac.ifoff.lo_enabled` enables or disables all traffic on the loopback, man:lo[4], interface.
|
||||
* `security.mac.ifoff.bpfrecv_enabled` enables or disables all traffic on the Berkeley Packet Filter interface, man:bpf[4].
|
||||
* `security.mac.ifoff.other_enabled` enables or disables traffic on all other interfaces.
|
||||
|
||||
One of the most common uses of man:mac_ifoff[4] is network monitoring in an environment where network traffic should not be permitted during the boot sequence. Another use would be to write a script which uses an application such as package:security/aide[] to automatically block network traffic if it finds new or altered files in protected directories.
|
||||
|
||||
[[mac-portacl]]
|
||||
=== The MAC Port Access Control List Policy
|
||||
|
||||
Module name: [.filename]#mac_portacl.ko#
|
||||
|
||||
Kernel configuration line: `MAC_PORTACL`
|
||||
|
||||
Boot option: `mac_portacl_load="YES"`
|
||||
|
||||
The man:mac_portacl[4] module is used to limit binding to local TCP and UDP ports, making it possible to allow non-`root` users to bind to specified privileged ports below 1024.
|
||||
|
||||
Once loaded, this module enables the MAC policy on all sockets. The following tunables are available:
|
||||
|
||||
* `security.mac.portacl.enabled` enables or disables the policy completely.
|
||||
* `security.mac.portacl.port_high` sets the highest port number that man:mac_portacl[4] protects.
|
||||
* `security.mac.portacl.suser_exempt`, when set to a non-zero value, exempts the `root` user from this policy.
|
||||
* `security.mac.portacl.rules` specifies the policy as a text string of the form `rule[,rule,...]`, with as many rules as needed, and where each rule is of the form `idtype:id:protocol:port`. The `idtype` is either `uid` or `gid`. The `protocol` parameter can be `tcp` or `udp`. The `port` parameter is the port number to allow the specified user or group to bind to. Only numeric values can be used for the user ID, group ID, and port parameters.
|
||||
|
||||
By default, ports below 1024 can only be used by privileged processes which run as `root`. For man:mac_portacl[4] to allow non-privileged processes to bind to ports below 1024, set the following tunables as follows:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# sysctl security.mac.portacl.port_high=1023
|
||||
# sysctl net.inet.ip.portrange.reservedlow=0
|
||||
# sysctl net.inet.ip.portrange.reservedhigh=0
|
||||
....
|
||||
|
||||
To prevent the `root` user from being affected by this policy, set `security.mac.portacl.suser_exempt` to a non-zero value.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# sysctl security.mac.portacl.suser_exempt=1
|
||||
....
|
||||
|
||||
To allow the `www` user with UID 80 to bind to port 80 without ever needing `root` privilege:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# sysctl security.mac.portacl.rules=uid:80:tcp:80
|
||||
....
|
||||
|
||||
This next example permits the user with the UID of 1001 to bind to TCP ports 110 (POP3) and 995 (POP3s):
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# sysctl security.mac.portacl.rules=uid:1001:tcp:110,uid:1001:tcp:995
|
||||
....
|
||||
|
||||
[[mac-partition]]
|
||||
=== The MAC Partition Policy
|
||||
|
||||
Module name: [.filename]#mac_partition.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_PARTITION`
|
||||
|
||||
Boot option: `mac_partition_load="YES"`
|
||||
|
||||
The man:mac_partition[4] policy drops processes into specific "partitions" based on their MAC label. Most configuration for this policy is done using man:setpmac[8]. One `sysctl` tunable is available for this policy:
|
||||
|
||||
* `security.mac.partition.enabled` enables the enforcement of MAC process partitions.
|
||||
|
||||
When this policy is enabled, users will only be permitted to see their processes, and any others within their partition, but will not be permitted to work with utilities outside the scope of this partition. For instance, a user in the `insecure` class will not be permitted to access `top` as well as many other commands that must spawn a process.
|
||||
|
||||
This example adds `top` to the label set on users in the `insecure` class. All processes spawned by users in the `insecure` class will stay in the `partition/13` label.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setpmac partition/13 top
|
||||
....
|
||||
|
||||
This command displays the partition label and the process list:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ps Zax
|
||||
....
|
||||
|
||||
This command displays another user's process partition label and that user's currently running processes:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ps -ZU trhodes
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Users can see processes in ``root``'s label unless the man:mac_seeotheruids[4] policy is loaded.
|
||||
====
|
||||
|
||||
[[mac-mls]]
|
||||
=== The MAC Multi-Level Security Module
|
||||
|
||||
Module name: [.filename]#mac_mls.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_MLS`
|
||||
|
||||
Boot option: `mac_mls_load="YES"`
|
||||
|
||||
The man:mac_mls[4] policy controls access between subjects and objects in the system by enforcing a strict information flow policy.
|
||||
|
||||
In MLS environments, a "clearance" level is set in the label of each subject or object, along with compartments. Since these clearance levels can reach numbers greater than several thousand, it would be a daunting task to thoroughly configure every subject or object. To ease this administrative overhead, three labels are included in this policy: `mls/low`, `mls/equal`, and `mls/high`, where:
|
||||
|
||||
* Anything labeled with `mls/low` will have a low clearance level and not be permitted to access information of a higher level. This label also prevents objects of a higher clearance level from writing or passing information to a lower level.
|
||||
* `mls/equal` should be placed on objects which should be exempt from the policy.
|
||||
* `mls/high` is the highest level of clearance possible. Objects assigned this label will hold dominance over all other objects in the system; however, they will not permit the leaking of information to objects of a lower class.
|
||||
|
||||
MLS provides:
|
||||
|
||||
* A hierarchical security level with a set of non-hierarchical categories.
|
||||
* Fixed rules of `no read up, no write down`. This means that a subject can have read access to objects on its own level or below, but not above. Similarly, a subject can have write access to objects on its own level or above, but not beneath.
|
||||
* Secrecy, or the prevention of inappropriate disclosure of data.
|
||||
* A basis for the design of systems that concurrently handle data at multiple sensitivity levels without leaking information between secret and confidential.
|
||||
|
||||
The following `sysctl` tunables are available:
|
||||
|
||||
* `security.mac.mls.enabled` is used to enable or disable the MLS policy.
|
||||
* `security.mac.mls.ptys_equal` labels all man:pty[4] devices as `mls/equal` during creation.
|
||||
* `security.mac.mls.revocation_enabled` revokes access to objects after their label changes to a label of a lower grade.
|
||||
* `security.mac.mls.max_compartments` sets the maximum number of compartment levels allowed on a system.
|
||||
|
||||
To manipulate MLS labels, use man:setfmac[8]. To assign a label to an object:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setfmac mls/5 test
|
||||
....
|
||||
|
||||
To get the MLS label for the file [.filename]#test#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# getfmac test
|
||||
....
|
||||
|
||||
Another approach is to create a master policy file in [.filename]#/etc/# which specifies the MLS policy information and to feed that file to `setfmac`.
|
||||
|
||||
When using the MLS policy module, an administrator plans to control the flow of sensitive information. The default `block read up block write down` sets everything to a low state. Everything is accessible and an administrator slowly augments the confidentiality of the information.
|
||||
|
||||
Beyond the three basic label options, an administrator may group users and groups as required to block the information flow between them. It might be easier to look at the information in clearance levels using descriptive words, such as classifications of `Confidential`, `Secret`, and `Top Secret`. Some administrators instead create different groups based on project levels. Regardless of the classification method, a well thought out plan must exist before implementing a restrictive policy.
|
||||
|
||||
Some example situations for the MLS policy module include an e-commerce web server, a file server holding critical company information, and financial institution environments.
|
||||
|
||||
[[mac-biba]]
|
||||
=== The MAC Biba Module
|
||||
|
||||
Module name: [.filename]#mac_biba.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_BIBA`
|
||||
|
||||
Boot option: `mac_biba_load="YES"`
|
||||
|
||||
The man:mac_biba[4] module loads the MAC Biba policy. This policy is similar to the MLS policy with the exception that the rules for information flow are slightly reversed. This is to prevent the downward flow of sensitive information whereas the MLS policy prevents the upward flow of sensitive information.
|
||||
|
||||
In Biba environments, an "integrity" label is set on each subject or object. These labels are made up of hierarchical grades and non-hierarchical components. As a grade ascends, so does its integrity.
|
||||
|
||||
Supported labels are `biba/low`, `biba/equal`, and `biba/high`, where:
|
||||
|
||||
* `biba/low` is considered the lowest integrity an object or subject may have. Setting this on objects or subjects blocks their write access to objects or subjects marked as `biba/high`, but will not prevent read access.
|
||||
* `biba/equal` should only be placed on objects considered to be exempt from the policy.
|
||||
* `biba/high` permits writing to objects set at a lower label, but does not permit reading that object. It is recommended that this label be placed on objects that affect the integrity of the entire system.
|
||||
|
||||
Biba provides:
|
||||
|
||||
* Hierarchical integrity levels with a set of non-hierarchical integrity categories.
|
||||
* Fixed rules are `no write up, no read down`, the opposite of MLS. A subject can have write access to objects on its own level or below, but not above. Similarly, a subject can have read access to objects on its own level or above, but not below.
|
||||
* Integrity by preventing inappropriate modification of data.
|
||||
* Integrity levels instead of MLS sensitivity levels.
|
||||
|
||||
The following tunables can be used to manipulate the Biba policy:
|
||||
|
||||
* `security.mac.biba.enabled` is used to enable or disable enforcement of the Biba policy on the target machine.
|
||||
* `security.mac.biba.ptys_equal` is used to disable the Biba policy on man:pty[4] devices.
|
||||
* `security.mac.biba.revocation_enabled` forces the revocation of access to objects if the label is changed to dominate the subject.
|
||||
|
||||
To access the Biba policy setting on system objects, use `setfmac` and `getfmac`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setfmac biba/low test
|
||||
# getfmac test
|
||||
test: biba/low
|
||||
....
|
||||
|
||||
Integrity, which is different from sensitivity, is used to guarantee that information is not manipulated by untrusted parties. This includes information passed between subjects and objects. It ensures that users will only be able to modify or access information they have been given explicit access to. The man:mac_biba[4] security policy module permits an administrator to configure which files and programs a user may see and invoke while assuring that the programs and files are trusted by the system for that user.
|
||||
|
||||
During the initial planning phase, an administrator must be prepared to partition users into grades, levels, and areas. The system will default to a high label once this policy module is enabled, and it is up to the administrator to configure the different grades and levels for users. Instead of using clearance levels, a good planning method could include topics. For instance, only allow developers modification access to the source code repository, source code compiler, and other development utilities. Other users would be grouped into other categories such as testers, designers, or end users and would only be permitted read access.
|
||||
|
||||
A lower integrity subject is unable to write to a higher integrity subject and a higher integrity subject cannot list or read a lower integrity object. Setting a label at the lowest possible grade could make it inaccessible to subjects. Some prospective environments for this security policy module would include a constrained web server, a development and test machine, and a source code repository. A less useful implementation would be a personal workstation, a machine used as a router, or a network firewall.
|
||||
|
||||
[[mac-lomac]]
|
||||
=== The MAC Low-watermark Module
|
||||
|
||||
Module name: [.filename]#mac_lomac.ko#
|
||||
|
||||
Kernel configuration line: `options MAC_LOMAC`
|
||||
|
||||
Boot option: `mac_lomac_load="YES"`
|
||||
|
||||
Unlike the MAC Biba policy, the man:mac_lomac[4] policy permits access to lower integrity objects only after decreasing the integrity level to not disrupt any integrity rules.
|
||||
|
||||
The Low-watermark integrity policy works almost identically to Biba, with the exception of using floating labels to support subject demotion via an auxiliary grade compartment. This secondary compartment takes the form `[auxgrade]`. When assigning a policy with an auxiliary grade, use the syntax `lomac/10[2]`, where `2` is the auxiliary grade.
|
||||
|
||||
This policy relies on the ubiquitous labeling of all system objects with integrity labels, permitting subjects to read from low integrity objects and then downgrading the label on the subject to prevent future writes to high integrity objects using `[auxgrade]`. The policy may provide greater compatibility and require less initial configuration than Biba.
|
||||
|
||||
Like the Biba and MLS policies, `setfmac` and `setpmac` are used to place labels on system objects:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setfmac /usr/home/trhodes lomac/high[low]
|
||||
# getfmac /usr/home/trhodes lomac/high[low]
|
||||
....
|
||||
|
||||
The auxiliary grade `low` is a feature provided only by the MACLOMAC policy.
|
||||
|
||||
[[mac-userlocked]]
|
||||
== User Lock Down
|
||||
|
||||
This example considers a relatively small storage system with fewer than fifty users. Users will have login capabilities and are permitted to store data and access resources.
|
||||
|
||||
For this scenario, the man:mac_bsdextended[4] and man:mac_seeotheruids[4] policy modules could co-exist and block access to system objects while hiding user processes.
|
||||
|
||||
Begin by adding the following line to [.filename]#/boot/loader.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
mac_seeotheruids_load="YES"
|
||||
....
|
||||
|
||||
The man:mac_bsdextended[4] security policy module may be activated by adding this line to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
ugidfw_enable="YES"
|
||||
....
|
||||
|
||||
Default rules stored in [.filename]#/etc/rc.bsdextended# will be loaded at system initialization. However, the default entries may need modification. Since this machine is expected only to service users, everything may be left commented out except the last two lines in order to force the loading of user owned system objects by default.
|
||||
|
||||
Add the required users to this machine and reboot. For testing purposes, try logging in as a different user across two consoles. Run `ps aux` to see if processes of other users are visible. Verify that running man:ls[1] on another user's home directory fails.
|
||||
|
||||
Do not try to test with the `root` user unless the specific ``sysctl``s have been modified to block super user access.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
When a new user is added, their man:mac_bsdextended[4] rule will not be in the ruleset list. To update the ruleset quickly, unload the security policy module and reload it again using man:kldunload[8] and man:kldload[8].
|
||||
====
|
||||
|
||||
[[mac-implementing]]
|
||||
== Nagios in a MAC Jail
|
||||
|
||||
This section demonstrates the steps that are needed to implement the Nagios network monitoring system in a MAC environment. This is meant as an example which still requires the administrator to test that the implemented policy meets the security requirements of the network before using in a production environment.
|
||||
|
||||
This example requires `multilabel` to be set on each file system. It also assumes that package:net-mgmt/nagios-plugins[], package:net-mgmt/nagios[], and package:www/apache22[] are all installed, configured, and working correctly before attempting the integration into the MAC framework.
|
||||
|
||||
=== Create an Insecure User Class
|
||||
|
||||
Begin the procedure by adding the following user class to [.filename]#/etc/login.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
insecure:\
|
||||
:copyright=/etc/COPYRIGHT:\
|
||||
:welcome=/etc/motd:\
|
||||
:setenv=MAIL=/var/mail/$,BLOCKSIZE=K:\
|
||||
:path=~/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
|
||||
:manpath=/usr/shared/man /usr/local/man:\
|
||||
:nologin=/usr/sbin/nologin:\
|
||||
:cputime=1h30m:\
|
||||
:datasize=8M:\
|
||||
:vmemoryuse=100M:\
|
||||
:stacksize=2M:\
|
||||
:memorylocked=4M:\
|
||||
:memoryuse=8M:\
|
||||
:filesize=8M:\
|
||||
:coredumpsize=8M:\
|
||||
:openfiles=24:\
|
||||
:maxproc=32:\
|
||||
:priority=0:\
|
||||
:requirehome:\
|
||||
:passwordtime=91d:\
|
||||
:umask=022:\
|
||||
:ignoretime@:\
|
||||
:label=biba/10(10-10):
|
||||
....
|
||||
|
||||
Then, add the following line to the default user class section:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:label=biba/high:
|
||||
....
|
||||
|
||||
Save the edits and issue the following command to rebuild the database:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cap_mkdb /etc/login.conf
|
||||
....
|
||||
|
||||
=== Configure Users
|
||||
|
||||
Set the `root` user to the default class using:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pw usermod root -L default
|
||||
....
|
||||
|
||||
All user accounts that are not `root` will now require a login class. The login class is required, otherwise users will be refused access to common commands. The following `sh` script should do the trick:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# for x in `awk -F: '($3 >= 1001) && ($3 != 65534) { print $1 }' \
|
||||
/etc/passwd`; do pw usermod $x -L default; done;
|
||||
....
|
||||
|
||||
Next, drop the `nagios` and `www` accounts into the insecure class:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pw usermod nagios -L insecure
|
||||
# pw usermod www -L insecure
|
||||
....
|
||||
|
||||
=== Create the Contexts File
|
||||
|
||||
A contexts file should now be created as [.filename]#/etc/policy.contexts#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# This is the default BIBA policy for this system.
|
||||
|
||||
# System:
|
||||
/var/run(/.*)? biba/equal
|
||||
|
||||
/dev/(/.*)? biba/equal
|
||||
|
||||
/var biba/equal
|
||||
/var/spool(/.*)? biba/equal
|
||||
|
||||
/var/log(/.*)? biba/equal
|
||||
|
||||
/tmp(/.*)? biba/equal
|
||||
/var/tmp(/.*)? biba/equal
|
||||
|
||||
/var/spool/mqueue biba/equal
|
||||
/var/spool/clientmqueue biba/equal
|
||||
|
||||
# For Nagios:
|
||||
/usr/local/etc/nagios(/.*)? biba/10
|
||||
|
||||
/var/spool/nagios(/.*)? biba/10
|
||||
|
||||
# For apache
|
||||
/usr/local/etc/apache(/.*)? biba/10
|
||||
....
|
||||
|
||||
This policy enforces security by setting restrictions on the flow of information. In this specific configuration, users, including `root`, should never be allowed to access Nagios. Configuration files and processes that are a part of Nagios will be completely self contained or jailed.
|
||||
|
||||
This file will be read after running `setfsmac` on every file system. This example sets the policy on the root file system:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setfsmac -ef /etc/policy.contexts /
|
||||
....
|
||||
|
||||
Next, add these edits to the main section of [.filename]#/etc/mac.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
default_labels file ?biba
|
||||
default_labels ifnet ?biba
|
||||
default_labels process ?biba
|
||||
default_labels socket ?biba
|
||||
....
|
||||
|
||||
=== Loader Configuration
|
||||
|
||||
To finish the configuration, add the following lines to [.filename]#/boot/loader.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
mac_biba_load="YES"
|
||||
mac_seeotheruids_load="YES"
|
||||
security.mac.biba.trust_all_interfaces=1
|
||||
....
|
||||
|
||||
And the following line to the network card configuration stored in [.filename]#/etc/rc.conf#. If the primary network configuration is done via DHCP, this may need to be configured manually after every system boot:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
maclabel biba/equal
|
||||
....
|
||||
|
||||
=== Testing the Configuration
|
||||
|
||||
First, ensure that the web server and Nagios will not be started on system initialization and reboot. Ensure that `root` cannot access any of the files in the Nagios configuration directory. If `root` can list the contents of [.filename]#/var/spool/nagios#, something is wrong. Instead, a "permission denied" error should be returned.
|
||||
|
||||
If all seems well, Nagios, Apache, and Sendmail can now be started:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /etc/mail && make stop && \
|
||||
setpmac biba/equal make start && setpmac biba/10\(10-10\) apachectl start && \
|
||||
setpmac biba/10\(10-10\) /usr/local/etc/rc.d/nagios.sh forcestart
|
||||
....
|
||||
|
||||
Double check to ensure that everything is working properly. If not, check the log files for error messages. If needed, use man:sysctl[8] to disable the man:mac_biba[4] security policy module and try starting everything again as usual.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The `root` user can still change the security enforcement and edit its configuration files. The following command will permit the degradation of the security policy to a lower grade for a newly spawned shell:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# setpmac biba/10 csh
|
||||
....
|
||||
|
||||
To block this from happening, force the user into a range using man:login.conf[5]. If man:setpmac[8] attempts to run a command outside of the compartment's range, an error will be returned and the command will not be executed. In this case, set root to `biba/high(high-high)`.
|
||||
====
|
||||
|
||||
[[mac-troubleshoot]]
|
||||
== Troubleshooting the MAC Framework
|
||||
|
||||
This section discusses common configuration errors and how to resolve them.
|
||||
|
||||
The `multilabel` flag does not stay enabled on the root ([.filename]#/#) partition:::
|
||||
The following steps may resolve this transient error:
|
||||
|
||||
[.procedure]
|
||||
====
|
||||
. Edit [.filename]#/etc/fstab# and set the root partition to `ro` for read-only.
|
||||
. Reboot into single user mode.
|
||||
. Run `tunefs -l enable` on [.filename]#/#.
|
||||
. Reboot the system.
|
||||
. Run `mount -urw`[.filename]#/# and change the `ro` back to `rw` in [.filename]#/etc/fstab# and reboot the system again.
|
||||
. Double-check the output from `mount` to ensure that `multilabel` has been properly set on the root file system.
|
||||
====
|
||||
|
||||
After establishing a secure environment with MAC, Xorg no longer starts:::
|
||||
This could be caused by the MAC `partition` policy or by a mislabeling in one of the MAC labeling policies. To debug, try the following:
|
||||
|
||||
[.procedure]
|
||||
====
|
||||
. Check the error message. If the user is in the `insecure` class, the `partition` policy may be the culprit. Try setting the user's class back to the `default` class and rebuild the database with `cap_mkdb`. If this does not alleviate the problem, go to step two.
|
||||
. Double-check that the label policies are set correctly for the user, Xorg, and the [.filename]#/dev# entries.
|
||||
. If neither of these resolve the problem, send the error message and a description of the environment to the {freebsd-questions}.
|
||||
====
|
||||
|
||||
The `_secure_path: unable to stat .login_conf` error appears:::
|
||||
This error can appear when a user attempts to switch from the `root` user to another user in the system. This message usually occurs when the user has a higher label setting than that of the user they are attempting to become. For instance, if `joe` has a default label of `biba/low` and `root` has a label of `biba/high`, `root` cannot view ``joe``'s home directory. This will happen whether or not `root` has used `su` to become `joe` as the Biba integrity model will not permit `root` to view objects set at a lower integrity level.
|
||||
|
||||
The system no longer recognizes `root`:::
|
||||
When this occurs, `whoami` returns `0` and `su` returns `who are you?`.
|
||||
+
|
||||
This can happen if a labeling policy has been disabled by man:sysctl[8] or the policy module was unloaded. If the policy is disabled, the login capabilities database needs to be reconfigured. Double check [.filename]#/etc/login.conf# to ensure that all `label` options have been removed and rebuild the database with `cap_mkdb`.
|
||||
+
|
||||
This may also happen if a policy restricts access to [.filename]#master.passwd#. This is usually caused by an administrator altering the file under a label which conflicts with the general policy being used by the system. In these cases, the user information would be read by the system and access would be blocked as the file has inherited the new label. Disable the policy using man:sysctl[8] and everything should return to normal.
|
||||
933
documentation/content/en/books/handbook/mail/_index.adoc
Normal file
933
documentation/content/en/books/handbook/mail/_index.adoc
Normal file
|
|
@ -0,0 +1,933 @@
|
|||
---
|
||||
title: Chapter 29. Electronic Mail
|
||||
part: IV. Network Communication
|
||||
prev: books/handbook/ppp-and-slip
|
||||
next: books/handbook/network-servers
|
||||
---
|
||||
|
||||
[[mail]]
|
||||
= Electronic Mail
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 29
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/mail/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/mail/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/mail/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[mail-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
"Electronic Mail", better known as email, is one of the most widely used forms of communication today. This chapter provides a basic introduction to running a mail server on FreeBSD, as well as an introduction to sending and receiving email using FreeBSD. For more complete coverage of this subject, refer to the books listed in crossref:bibliography[bibliography,Bibliography].
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* Which software components are involved in sending and receiving electronic mail.
|
||||
* Where basic Sendmail configuration files are located in FreeBSD.
|
||||
* The difference between remote and local mailboxes.
|
||||
* How to block spammers from illegally using a mail server as a relay.
|
||||
* How to install and configure an alternate Mail Transfer Agent, replacing Sendmail.
|
||||
* How to troubleshoot common mail server problems.
|
||||
* How to set up the system to send mail only.
|
||||
* How to use mail with a dialup connection.
|
||||
* How to configure SMTP authentication for added security.
|
||||
* How to install and use a Mail User Agent, such as mutt, to send and receive email.
|
||||
* How to download mail from a remote POP or IMAP server.
|
||||
* How to automatically apply filters and rules to incoming email.
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Properly set up a network connection (crossref:advanced-networking[advanced-networking,Advanced Networking]).
|
||||
* Properly set up the DNS information for a mail host (crossref:network-servers[network-servers,Network Servers]).
|
||||
* Know how to install additional third-party software (crossref:ports[ports,Installing Applications: Packages and Ports]).
|
||||
|
||||
[[mail-using]]
|
||||
== Mail Components
|
||||
|
||||
There are five major parts involved in an email exchange: the Mail User Agent (MUA), the Mail Transfer Agent (MTA), a mail host, a remote or local mailbox, and DNS. This section provides an overview of these components.
|
||||
|
||||
Mail User Agent (MUA)::
|
||||
The Mail User Agent (MUA) is an application which is used to compose, send, and receive emails. This application can be a command line program, such as the built-in `mail` utility or a third-party application from the Ports Collection, such as mutt, alpine, or elm. Dozens of graphical programs are also available in the Ports Collection, including Claws Mail, Evolution, and Thunderbird. Some organizations provide a web mail program which can be accessed through a web browser. More information about installing and using a MUA on FreeBSD can be found in <<mail-agents>>.
|
||||
|
||||
Mail Transfer Agent (MTA)::
|
||||
The Mail Transfer Agent (MTA) is responsible for receiving incoming mail and delivering outgoing mail. FreeBSD ships with Sendmail as the default MTA, but it also supports numerous other mail server daemons, including Exim, Postfix, and qmail. Sendmail configuration is described in <<sendmail>>. If another MTA is installed using the Ports Collection, refer to its post-installation message for FreeBSD-specific configuration details and the application's website for more general configuration instructions.
|
||||
|
||||
Mail Host and Mailboxes::
|
||||
The mail host is a server that is responsible for delivering and receiving mail for a host or a network. The mail host collects all mail sent to the domain and stores it either in the default [.filename]#mbox# or the alternative Maildir format, depending on the configuration. Once mail has been stored, it may either be read locally using a MUA or remotely accessed and collected using protocols such as POP or IMAP. If mail is read locally, a POP or IMAP server does not need to be installed.
|
||||
+
|
||||
To access mailboxes remotely, a POP or IMAP server is required as these protocols allow users to connect to their mailboxes from remote locations. IMAP offers several advantages over POP. These include the ability to store a copy of messages on a remote server after they are downloaded and concurrent updates. IMAP can be useful over low-speed links as it allows users to fetch the structure of messages without downloading them. It can also perform tasks such as searching on the server in order to minimize data transfer between clients and servers.
|
||||
+
|
||||
Several POP and IMAP servers are available in the Ports Collection. These include package:mail/qpopper[], package:mail/imap-uw[], package:mail/courier-imap[], and package:mail/dovecot2[].
|
||||
+
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
It should be noted that both POP and IMAP transmit information, including username and password credentials, in clear-text. To secure the transmission of information across these protocols, consider tunneling sessions over man:ssh[1] (crossref:security[security-ssh-tunneling,"SSH Tunneling"]) or using SSL (crossref:security[openssl,"OpenSSL"]).
|
||||
====
|
||||
|
||||
Domain Name System (DNS)::
|
||||
The Domain Name System (DNS) and its daemon `named` play a large role in the delivery of email. In order to deliver mail from one site to another, the MTA will look up the remote site in DNS to determine which host will receive mail for the destination. This process also occurs when mail is sent from a remote host to the MTA.
|
||||
+
|
||||
In addition to mapping hostnames to IP addresses, DNS is responsible for storing information specific to mail delivery, known as Mail eXchanger MX records. The MX record specifies which hosts will receive mail for a particular domain.
|
||||
+
|
||||
To view the MX records for a domain, specify the type of record. Refer to man:host[1], for more details about this command:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% host -t mx FreeBSD.org
|
||||
FreeBSD.org mail is handled by 10 mx1.FreeBSD.org
|
||||
....
|
||||
+
|
||||
Refer to crossref:network-servers[network-dns,"Domain Name System (DNS)"] for more information about DNS and its configuration.
|
||||
|
||||
[[sendmail]]
|
||||
== Sendmail Configuration Files
|
||||
|
||||
Sendmail is the default MTA installed with FreeBSD. It accepts mail from MUAs and delivers it to the appropriate mail host, as defined by its configuration. Sendmail can also accept network connections and deliver mail to local mailboxes or to another program.
|
||||
|
||||
The configuration files for Sendmail are located in [.filename]#/etc/mail#. This section describes these files in more detail.
|
||||
|
||||
[.filename]#/etc/mail/access#::
|
||||
This access database file defines which hosts or IP addresses have access to the local mail server and what kind of access they have. Hosts listed as `OK`, which is the default option, are allowed to send mail to this host as long as the mail's final destination is the local machine. Hosts listed as `REJECT` are rejected for all mail connections. Hosts listed as `RELAY` are allowed to send mail for any destination using this mail server. Hosts listed as `ERROR` will have their mail returned with the specified mail error. If a host is listed as `SKIP`, Sendmail will abort the current search for this entry without accepting or rejecting the mail. Hosts listed as `QUARANTINE` will have their messages held and will receive the specified text as the reason for the hold.
|
||||
+
|
||||
Examples of using these options for both IPv4 and IPv6 addresses can be found in the FreeBSD sample configuration, [.filename]#/etc/mail/access.sample#:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
# $FreeBSD$
|
||||
#
|
||||
# Mail relay access control list. Default is to reject mail unless the
|
||||
# destination is local, or listed in /etc/mail/local-host-names
|
||||
#
|
||||
## Examples (commented out for safety)
|
||||
#From:cyberspammer.com ERROR:"550 We don't accept mail from spammers"
|
||||
#From:okay.cyberspammer.com OK
|
||||
#Connect:sendmail.org RELAY
|
||||
#To:sendmail.org RELAY
|
||||
#Connect:128.32 RELAY
|
||||
#Connect:128.32.2 SKIP
|
||||
#Connect:IPv6:1:2:3:4:5:6:7 RELAY
|
||||
#Connect:suspicious.example.com QUARANTINE:Mail from suspicious host
|
||||
#Connect:[127.0.0.3] OK
|
||||
#Connect:[IPv6:1:2:3:4:5:6:7:8] OK
|
||||
....
|
||||
+
|
||||
To configure the access database, use the format shown in the sample to make entries in [.filename]#/etc/mail/access#, but do not put a comment symbol (`#`) in front of the entries. Create an entry for each host or network whose access should be configured. Mail senders that match the left side of the table are affected by the action on the right side of the table.
|
||||
+
|
||||
Whenever this file is updated, update its database and restart Sendmail:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# makemap hash /etc/mail/access < /etc/mail/access
|
||||
# service sendmail restart
|
||||
....
|
||||
|
||||
[.filename]#/etc/mail/aliases#::
|
||||
This database file contains a list of virtual mailboxes that are expanded to users, files, programs, or other aliases. Here are a few entries to illustrate the file format:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
root: localuser
|
||||
ftp-bugs: joe,eric,paul
|
||||
bit.bucket: /dev/null
|
||||
procmail: "|/usr/local/bin/procmail"
|
||||
....
|
||||
+
|
||||
The mailbox name on the left side of the colon is expanded to the target(s) on the right. The first entry expands the `root` mailbox to the `localuser` mailbox, which is then looked up in the [.filename]#/etc/mail/aliases# database. If no match is found, the message is delivered to `localuser`. The second entry shows a mail list. Mail to `ftp-bugs` is expanded to the three local mailboxes `joe`, `eric`, and `paul`. A remote mailbox could be specified as _user@example.com_. The third entry shows how to write mail to a file, in this case [.filename]#/dev/null#. The last entry demonstrates how to send mail to a program, [.filename]#/usr/local/bin/procmail#, through a UNIX(R) pipe. Refer to man:aliases[5] for more information about the format of this file.
|
||||
+
|
||||
Whenever this file is updated, run `newaliases` to update and initialize the aliases database.
|
||||
|
||||
[.filename]#/etc/mail/sendmail.cf#::
|
||||
This is the master configuration file for Sendmail. It controls the overall behavior of Sendmail, including everything from rewriting email addresses to printing rejection messages to remote mail servers. Accordingly, this configuration file is quite complex. Fortunately, this file rarely needs to be changed for standard mail servers.
|
||||
+
|
||||
The master Sendmail configuration file can be built from man:m4[1] macros that define the features and behavior of Sendmail. Refer to [.filename]#/usr/src/contrib/sendmail/cf/README# for some of the details.
|
||||
+
|
||||
Whenever changes to this file are made, Sendmail needs to be restarted for the changes to take effect.
|
||||
|
||||
[.filename]#/etc/mail/virtusertable#::
|
||||
This database file maps mail addresses for virtual domains and users to real mailboxes. These mailboxes can be local, remote, aliases defined in [.filename]#/etc/mail/aliases#, or files. This allows multiple virtual domains to be hosted on one machine.
|
||||
+
|
||||
FreeBSD provides a sample configuration file in [.filename]#/etc/mail/virtusertable.sample# to further demonstrate its format. The following example demonstrates how to create custom entries using that format:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
root@example.com root
|
||||
postmaster@example.com postmaster@noc.example.net
|
||||
@example.com joe
|
||||
....
|
||||
+
|
||||
This file is processed in a first match order. When an email address matches the address on the left, it is mapped to the local mailbox listed on the right. The format of the first entry in this example maps a specific email address to a local mailbox, whereas the format of the second entry maps a specific email address to a remote mailbox. Finally, any email address from `example.com` which has not matched any of the previous entries will match the last mapping and be sent to the local mailbox `joe`. When creating custom entries, use this format and add them to [.filename]#/etc/mail/virtusertable#. Whenever this file is edited, update its database and restart Sendmail:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# makemap hash /etc/mail/virtusertable < /etc/mail/virtusertable
|
||||
# service sendmail restart
|
||||
....
|
||||
|
||||
[.filename]#/etc/mail/relay-domains#::
|
||||
In a default FreeBSD installation, Sendmail is configured to only send mail from the host it is running on. For example, if a POP server is available, users will be able to check mail from remote locations but they will not be able to send outgoing emails from outside locations. Typically, a few moments after the attempt, an email will be sent from `MAILER-DAEMON` with a `5.7 Relaying Denied` message.
|
||||
+
|
||||
The most straightforward solution is to add the ISP's FQDN to [.filename]#/etc/mail/relay-domains#. If multiple addresses are needed, add them one per line:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
your.isp.example.com
|
||||
other.isp.example.net
|
||||
users-isp.example.org
|
||||
www.example.org
|
||||
....
|
||||
+
|
||||
After creating or editing this file, restart Sendmail with `service sendmail restart`.
|
||||
+
|
||||
Now any mail sent through the system by any host in this list, provided the user has an account on the system, will succeed. This allows users to send mail from the system remotely without opening the system up to relaying SPAM from the Internet.
|
||||
|
||||
[[mail-changingmta]]
|
||||
== Changing the Mail Transfer Agent
|
||||
|
||||
FreeBSD comes with Sendmail already installed as the MTA which is in charge of outgoing and incoming mail. However, the system administrator can change the system's MTA. A wide choice of alternative MTAs is available from the `mail` category of the FreeBSD Ports Collection.
|
||||
|
||||
Once a new MTA is installed, configure and test the new software before replacing Sendmail. Refer to the documentation of the new MTA for information on how to configure the software.
|
||||
|
||||
Once the new MTA is working, use the instructions in this section to disable Sendmail and configure FreeBSD to use the replacement MTA.
|
||||
|
||||
[[mail-disable-sendmail]]
|
||||
=== Disable Sendmail
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
If Sendmail's outgoing mail service is disabled, it is important that it is replaced with an alternative mail delivery system. Otherwise, system functions such as man:periodic[8] will be unable to deliver their results by email. Many parts of the system expect a functional MTA. If applications continue to use Sendmail's binaries to try to send email after they are disabled, mail could go into an inactive Sendmail queue and never be delivered.
|
||||
====
|
||||
|
||||
In order to completely disable Sendmail, add or edit the following lines in [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sendmail_enable="NO"
|
||||
sendmail_submit_enable="NO"
|
||||
sendmail_outbound_enable="NO"
|
||||
sendmail_msp_queue_enable="NO"
|
||||
....
|
||||
|
||||
To only disable Sendmail's incoming mail service, use only this entry in [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sendmail_enable="NO"
|
||||
....
|
||||
|
||||
More information on Sendmail's startup options is available in man:rc.sendmail[8].
|
||||
|
||||
=== Replace the Default MTA
|
||||
|
||||
When a new MTA is installed using the Ports Collection, its startup script is also installed and startup instructions are mentioned in its package message. Before starting the new MTA, stop the running Sendmail processes. This example stops all of these services, then starts the Postfix service:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service sendmail stop
|
||||
# service postfix start
|
||||
....
|
||||
|
||||
To start the replacement MTA at system boot, add its configuration line to [.filename]#/etc/rc.conf#. This entry enables the Postfix MTA:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
postfix_enable="YES"
|
||||
....
|
||||
|
||||
Some extra configuration is needed as Sendmail is so ubiquitous that some software assumes it is already installed and configured. Check [.filename]#/etc/periodic.conf# and make sure that these values are set to `NO`. If this file does not exist, create it with these entries:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
daily_clean_hoststat_enable="NO"
|
||||
daily_status_mail_rejects_enable="NO"
|
||||
daily_status_include_submit_mailq="NO"
|
||||
daily_submit_queuerun="NO"
|
||||
....
|
||||
|
||||
Some alternative MTAs provide their own compatible implementations of the Sendmail command-line interface in order to facilitate using them as drop-in replacements for Sendmail. However, some MUAs may try to execute standard Sendmail binaries instead of the new MTA's binaries. FreeBSD uses [.filename]#/etc/mail/mailer.conf# to map the expected Sendmail binaries to the location of the new binaries. More information about this mapping can be found in man:mailwrapper[8].
|
||||
|
||||
The default [.filename]#/etc/mail/mailer.conf# looks like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
# $FreeBSD$
|
||||
#
|
||||
# Execute the "real" sendmail program, named /usr/libexec/sendmail/sendmail
|
||||
#
|
||||
sendmail /usr/libexec/sendmail/sendmail
|
||||
send-mail /usr/libexec/sendmail/sendmail
|
||||
mailq /usr/libexec/sendmail/sendmail
|
||||
newaliases /usr/libexec/sendmail/sendmail
|
||||
hoststat /usr/libexec/sendmail/sendmail
|
||||
purgestat /usr/libexec/sendmail/sendmail
|
||||
....
|
||||
|
||||
When any of the commands listed on the left are run, the system actually executes the associated command shown on the right. This system makes it easy to change what binaries are executed when these default binaries are invoked.
|
||||
|
||||
Some MTAs, when installed using the Ports Collection, will prompt to update this file for the new binaries. For example, Postfix will update the file like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#
|
||||
# Execute the Postfix sendmail program, named /usr/local/sbin/sendmail
|
||||
#
|
||||
sendmail /usr/local/sbin/sendmail
|
||||
send-mail /usr/local/sbin/sendmail
|
||||
mailq /usr/local/sbin/sendmail
|
||||
newaliases /usr/local/sbin/sendmail
|
||||
....
|
||||
|
||||
If the installation of the MTA does not automatically update [.filename]#/etc/mail/mailer.conf#, edit this file in a text editor so that it points to the new binaries. This example points to the binaries installed by package:mail/ssmtp[]:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sendmail /usr/local/sbin/ssmtp
|
||||
send-mail /usr/local/sbin/ssmtp
|
||||
mailq /usr/local/sbin/ssmtp
|
||||
newaliases /usr/local/sbin/ssmtp
|
||||
hoststat /usr/bin/true
|
||||
purgestat /usr/bin/true
|
||||
....
|
||||
|
||||
Once everything is configured, it is recommended to reboot the system. Rebooting provides the opportunity to ensure that the system is correctly configured to start the new MTA automatically on boot.
|
||||
|
||||
[[mail-trouble]]
|
||||
== Troubleshooting
|
||||
|
||||
=== Why do I have to use the FQDN for hosts on my site?
|
||||
|
||||
The host may actually be in a different domain. For example, in order for a host in `foo.bar.edu` to reach a host called `mumble` in the `bar.edu` domain, refer to it by the Fully-Qualified Domain Name FQDN, `mumble.bar.edu`, instead of just `mumble`.
|
||||
|
||||
This is because the version of BIND which ships with FreeBSD no longer provides default abbreviations for non-FQDNs other than the local domain. An unqualified host such as `mumble` must either be found as `mumble.foo.bar.edu`, or it will be searched for in the root domain.
|
||||
|
||||
In older versions of BIND, the search continued across `mumble.bar.edu`, and `mumble.edu`. RFC 1535 details why this is considered bad practice or even a security hole.
|
||||
|
||||
As a good workaround, place the line:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
search foo.bar.edu bar.edu
|
||||
....
|
||||
|
||||
instead of the previous:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
domain foo.bar.edu
|
||||
....
|
||||
|
||||
into [.filename]#/etc/resolv.conf#. However, make sure that the search order does not go beyond the "boundary between local and public administration", as RFC 1535 calls it.
|
||||
|
||||
=== How can I run a mail server on a dial-up PPP host?
|
||||
|
||||
Connect to a FreeBSD mail gateway on the LAN. The PPP connection is non-dedicated.
|
||||
|
||||
One way to do this is to get a full-time Internet server to provide secondary MX services for the domain. In this example, the domain is `example.com` and the ISP has configured `example.net` to provide secondary MX services to the domain:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
example.com. MX 10 example.com.
|
||||
MX 20 example.net.
|
||||
....
|
||||
|
||||
Only one host should be specified as the final recipient. For Sendmail, add `Cw example.com` in [.filename]#/etc/mail/sendmail.cf# on `example.com`.
|
||||
|
||||
When the sending MTA attempts to deliver mail, it will try to connect to the system, `example.com`, over the PPP link. This will time out if the destination is offline. The MTA will automatically deliver it to the secondary MX site at the Internet Service Provider (ISP), `example.net`. The secondary MX site will periodically try to connect to the primary MX host, `example.com`.
|
||||
|
||||
Use something like this as a login script:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#!/bin/sh
|
||||
# Put me in /usr/local/bin/pppmyisp
|
||||
( sleep 60 ; /usr/sbin/sendmail -q ) &
|
||||
/usr/sbin/ppp -direct pppmyisp
|
||||
....
|
||||
|
||||
When creating a separate login script for users, instead use `sendmail -qRexample.com` in the script above. This will force all mail in the queue for `example.com` to be processed immediately.
|
||||
|
||||
A further refinement of the situation can be seen from this example from the {freebsd-isp}:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
> we provide the secondary MX for a customer. The customer connects to
|
||||
> our services several times a day automatically to get the mails to
|
||||
> his primary MX (We do not call his site when a mail for his domains
|
||||
> arrived). Our sendmail sends the mailqueue every 30 minutes. At the
|
||||
> moment he has to stay 30 minutes online to be sure that all mail is
|
||||
> gone to the primary MX.
|
||||
>
|
||||
> Is there a command that would initiate sendmail to send all the mails
|
||||
> now? The user has not root-privileges on our machine of course.
|
||||
|
||||
In the privacy flags section of sendmail.cf, there is a
|
||||
definition Opgoaway,restrictqrun
|
||||
|
||||
Remove restrictqrun to allow non-root users to start the queue processing.
|
||||
You might also like to rearrange the MXs. We are the 1st MX for our
|
||||
customers like this, and we have defined:
|
||||
|
||||
# If we are the best MX for a host, try directly instead of generating
|
||||
# local config error.
|
||||
OwTrue
|
||||
|
||||
That way a remote site will deliver straight to you, without trying
|
||||
the customer connection. You then send to your customer. Only works for
|
||||
hosts, so you need to get your customer to name their mail
|
||||
machine customer.com as well as
|
||||
hostname.customer.com in the DNS. Just put an A record in
|
||||
the DNS for customer.com.
|
||||
....
|
||||
|
||||
[[mail-advanced]]
|
||||
== Advanced Topics
|
||||
|
||||
This section covers more involved topics such as mail configuration and setting up mail for an entire domain.
|
||||
|
||||
[[mail-config]]
|
||||
=== Basic Configuration
|
||||
|
||||
Out of the box, one can send email to external hosts as long as [.filename]#/etc/resolv.conf# is configured or the network has access to a configured DNS server. To have email delivered to the MTA on the FreeBSD host, do one of the following:
|
||||
|
||||
* Run a DNS server for the domain.
|
||||
* Get mail delivered directly to the FQDN for the machine.
|
||||
|
||||
In order to have mail delivered directly to a host, it must have a permanent static IP address, not a dynamic IP address. If the system is behind a firewall, it must be configured to allow SMTP traffic. To receive mail directly at a host, one of these two must be configured:
|
||||
|
||||
* Make sure that the lowest-numbered MX record in DNS points to the host's static IP address.
|
||||
* Make sure there is no MX entry in the DNS for the host.
|
||||
|
||||
Either of the above will allow mail to be received directly at the host.
|
||||
|
||||
Try this:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# hostname
|
||||
example.FreeBSD.org
|
||||
# host example.FreeBSD.org
|
||||
example.FreeBSD.org has address 204.216.27.XX
|
||||
....
|
||||
|
||||
In this example, mail sent directly to mailto:yourlogin@example.FreeBSD.org[yourlogin@example.FreeBSD.org] should work without problems, assuming Sendmail is running correctly on `example.FreeBSD.org`.
|
||||
|
||||
For this example:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# host example.FreeBSD.org
|
||||
example.FreeBSD.org has address 204.216.27.XX
|
||||
example.FreeBSD.org mail is handled (pri=10) by nevdull.FreeBSD.org
|
||||
....
|
||||
|
||||
All mail sent to `example.FreeBSD.org` will be collected on `hub` under the same username instead of being sent directly to your host.
|
||||
|
||||
The above information is handled by the DNS server. The DNS record that carries mail routing information is the MX entry. If no MX record exists, mail will be delivered directly to the host by way of its IP address.
|
||||
|
||||
The MX entry for `freefall.FreeBSD.org` at one time looked like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
freefall MX 30 mail.crl.net
|
||||
freefall MX 40 agora.rdrop.com
|
||||
freefall MX 10 freefall.FreeBSD.org
|
||||
freefall MX 20 who.cdrom.com
|
||||
....
|
||||
|
||||
`freefall` had many MX entries. The lowest MX number is the host that receives mail directly, if available. If it is not accessible for some reason, the next lower-numbered host will accept messages temporarily, and pass it along when a lower-numbered host becomes available.
|
||||
|
||||
Alternate MX sites should have separate Internet connections in order to be most useful. Your ISP can provide this service.
|
||||
|
||||
[[mail-domain]]
|
||||
=== Mail for a Domain
|
||||
|
||||
When configuring a MTA for a network, any mail sent to hosts in its domain should be diverted to the MTA so that users can receive their mail on the master mail server.
|
||||
|
||||
To make life easiest, a user account with the same _username_ should exist on both the MTA and the system with the MUA. Use man:adduser[8] to create the user accounts.
|
||||
|
||||
The MTA must be the designated mail exchanger for each workstation on the network. This is done in theDNS configuration with an MX record:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
example.FreeBSD.org A 204.216.27.XX ; Workstation
|
||||
MX 10 nevdull.FreeBSD.org ; Mailhost
|
||||
....
|
||||
|
||||
This will redirect mail for the workstation to the MTA no matter where the A record points. The mail is sent to the MX host.
|
||||
|
||||
This must be configured on a DNS server. If the network does not run its own DNS server, talk to the ISP or DNS provider.
|
||||
|
||||
The following is an example of virtual email hosting. Consider a customer with the domain `customer1.org`, where all the mail for `customer1.org` should be sent to `mail.myhost.com`. The DNS entry should look like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
customer1.org MX 10 mail.myhost.com
|
||||
....
|
||||
|
||||
An `A` record is _not_ needed for `customer1.org` in order to only handle email for that domain. However, running `ping` against `customer1.org` will not work unless an `A` record exists for it.
|
||||
|
||||
Tell the MTA which domains and/or hostnames it should accept mail for. Either of the following will work for Sendmail:
|
||||
|
||||
* Add the hosts to [.filename]#/etc/mail/local-host-names# when using the `FEATURE(use_cw_file)`.
|
||||
* Add a `Cwyour.host.com` line to [.filename]#/etc/sendmail.cf#.
|
||||
|
||||
[[outgoing-only]]
|
||||
== Setting Up to Send Only
|
||||
|
||||
There are many instances where one may only want to send mail through a relay. Some examples are:
|
||||
|
||||
* The computer is a desktop machine that needs to use programs such as man:mail[1], using the ISP's mail relay.
|
||||
* The computer is a server that does not handle mail locally, but needs to pass off all mail to a relay for processing.
|
||||
|
||||
While any MTA is capable of filling this particular niche, it can be difficult to properly configure a full-featured MTA just to handle offloading mail. Programs such as Sendmail and Postfix are overkill for this use.
|
||||
|
||||
Additionally, a typical Internet access service agreement may forbid one from running a "mail server".
|
||||
|
||||
The easiest way to fulfill those needs is to install the package:mail/ssmtp[] port:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/mail/ssmtp
|
||||
# make install replace clean
|
||||
....
|
||||
|
||||
Once installed, package:mail/ssmtp[] can be configured with [.filename]#/usr/local/etc/ssmtp/ssmtp.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
root=yourrealemail@example.com
|
||||
mailhub=mail.example.com
|
||||
rewriteDomain=example.com
|
||||
hostname=_HOSTNAME_
|
||||
....
|
||||
|
||||
Use the real email address for `root`. Enter the ISP's outgoing mail relay in place of `mail.example.com`. Some ISPs call this the "outgoing mail server" or "SMTP server".
|
||||
|
||||
Make sure to disable Sendmail, including the outgoing mail service. See <<mail-disable-sendmail>> for details.
|
||||
|
||||
package:mail/ssmtp[] has some other options available. Refer to the examples in [.filename]#/usr/local/etc/ssmtp# or the manual page of ssmtp for more information.
|
||||
|
||||
Setting up ssmtp in this manner allows any software on the computer that needs to send mail to function properly, while not violating the ISP's usage policy or allowing the computer to be hijacked for spamming.
|
||||
|
||||
[[SMTP-dialup]]
|
||||
== Using Mail with a Dialup Connection
|
||||
|
||||
When using a static IP address, one should not need to adjust the default configuration. Set the hostname to the assigned Internet name and Sendmail will do the rest.
|
||||
|
||||
When using a dynamically assigned IP address and a dialup PPP connection to the Internet, one usually has a mailbox on the ISP's mail server. In this example, the ISP's domain is `example.net`, the user name is `user`, the hostname is `bsd.home`, and the ISP has allowed `relay.example.net` as a mail relay.
|
||||
|
||||
In order to retrieve mail from the ISP's mailbox, install a retrieval agent from the Ports Collection. package:mail/fetchmail[] is a good choice as it supports many different protocols. Usually, the ISP will provide POP. When using user PPP, email can be automatically fetched when an Internet connection is established with the following entry in [.filename]#/etc/ppp/ppp.linkup#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
MYADDR:
|
||||
!bg su user -c fetchmail
|
||||
....
|
||||
|
||||
When using Sendmail to deliver mail to non-local accounts, configure Sendmail to process the mail queue as soon as the Internet connection is established. To do this, add this line after the above `fetchmail` entry in [.filename]#/etc/ppp/ppp.linkup#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
!bg su user -c "sendmail -q"
|
||||
....
|
||||
|
||||
In this example, there is an account for `user` on `bsd.home`. In the home directory of `user` on `bsd.home`, create a [.filename]#.fetchmailrc# which contains this line:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
poll example.net protocol pop3 fetchall pass MySecret
|
||||
....
|
||||
|
||||
This file should not be readable by anyone except `user` as it contains the password `MySecret`.
|
||||
|
||||
In order to send mail with the correct `from:` header, configure Sendmail to use mailto:user@example.net[user@example.net] rather than mailto:user@bsd.home[user@bsd.home] and to send all mail via `relay.example.net`, allowing quicker mail transmission.
|
||||
|
||||
The following [.filename]#.mc# should suffice:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
VERSIONID(`bsd.home.mc version 1.0')
|
||||
OSTYPE(bsd4.4)dnl
|
||||
FEATURE(nouucp)dnl
|
||||
MAILER(local)dnl
|
||||
MAILER(smtp)dnl
|
||||
Cwlocalhost
|
||||
Cwbsd.home
|
||||
MASQUERADE_AS(`example.net')dnl
|
||||
FEATURE(allmasquerade)dnl
|
||||
FEATURE(masquerade_envelope)dnl
|
||||
FEATURE(nocanonify)dnl
|
||||
FEATURE(nodns)dnl
|
||||
define(`SMART_HOST', `relay.example.net')
|
||||
Dmbsd.home
|
||||
define(`confDOMAIN_NAME',`bsd.home')dnl
|
||||
define(`confDELIVERY_MODE',`deferred')dnl
|
||||
....
|
||||
|
||||
Refer to the previous section for details of how to convert this file into the [.filename]#sendmail.cf# format. Do not forget to restart Sendmail after updating [.filename]#sendmail.cf#.
|
||||
|
||||
[[SMTP-Auth]]
|
||||
== SMTP Authentication
|
||||
|
||||
Configuring SMTP authentication on the MTA provides a number of benefits. SMTP authentication adds a layer of security to Sendmail, and provides mobile users who switch hosts the ability to use the same MTA without the need to reconfigure their mail client's settings each time.
|
||||
|
||||
[.procedure]
|
||||
. Install package:security/cyrus-sasl2[] from the Ports Collection. This port supports a number of compile-time options. For the SMTP authentication method demonstrated in this example, make sure that `LOGIN` is not disabled.
|
||||
. After installing package:security/cyrus-sasl2[], edit [.filename]#/usr/local/lib/sasl2/Sendmail.conf#, or create it if it does not exist, and add the following line:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
pwcheck_method: saslauthd
|
||||
....
|
||||
|
||||
. Next, install package:security/cyrus-sasl2-saslauthd[] and add the following line to [.filename]#/etc/rc.conf#:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
saslauthd_enable="YES"
|
||||
....
|
||||
+
|
||||
Finally, start the saslauthd daemon:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# service saslauthd start
|
||||
....
|
||||
+
|
||||
This daemon serves as a broker for Sendmail to authenticate against the FreeBSD man:passwd[5] database. This saves the trouble of creating a new set of usernames and passwords for each user that needs to use SMTP authentication, and keeps the login and mail password the same.
|
||||
. Next, edit [.filename]#/etc/make.conf# and add the following lines:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
SENDMAIL_CFLAGS=-I/usr/local/include/sasl -DSASL
|
||||
SENDMAIL_LDADD=/usr/local/lib/libsasl2.so
|
||||
....
|
||||
+
|
||||
These lines provide Sendmail the proper configuration options for linking to package:cyrus-sasl2[] at compile time. Make sure that package:cyrus-sasl2[] has been installed before recompiling Sendmail.
|
||||
. Recompile Sendmail by executing the following commands:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src/lib/libsmutil
|
||||
# make cleandir && make obj && make
|
||||
# cd /usr/src/lib/libsm
|
||||
# make cleandir && make obj && make
|
||||
# cd /usr/src/usr.sbin/sendmail
|
||||
# make cleandir && make obj && make && make install
|
||||
....
|
||||
+
|
||||
This compile should not have any problems if [.filename]#/usr/src# has not changed extensively and the shared libraries it needs are available.
|
||||
. After Sendmail has been compiled and reinstalled, edit [.filename]#/etc/mail/freebsd.mc# or the local [.filename]#.mc#. Many administrators choose to use the output from man:hostname[1] as the name of [.filename]#.mc# for uniqueness. Add these lines:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
dnl set SASL options
|
||||
TRUST_AUTH_MECH(`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
|
||||
define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
|
||||
....
|
||||
+
|
||||
These options configure the different methods available to Sendmail for authenticating users. To use a method other than pwcheck, refer to the Sendmail documentation.
|
||||
. Finally, run man:make[1] while in [.filename]#/etc/mail#. That will run the new [.filename]#.mc# and create a [.filename]#.cf# named either [.filename]#freebsd.cf# or the name used for the local [.filename]#.mc#. Then, run `make install restart`, which will copy the file to [.filename]#sendmail.cf#, and properly restart Sendmail. For more information about this process, refer to [.filename]#/etc/mail/Makefile#.
|
||||
|
||||
To test the configuration, use a MUA to send a test message. For further investigation, set the `LogLevel` of Sendmail to `13` and watch [.filename]#/var/log/maillog# for any errors.
|
||||
|
||||
For more information, refer to http://www.sendmail.org/~ca/email/auth.html[SMTP authentication].
|
||||
|
||||
[[mail-agents]]
|
||||
== Mail User Agents
|
||||
|
||||
A MUA is an application that is used to send and receive email. As email "evolves" and becomes more complex, MUAs are becoming increasingly powerful and provide users increased functionality and flexibility. The `mail` category of the FreeBSD Ports Collection contains numerous MUAs. These include graphical email clients such as Evolution or Balsa and console based clients such as mutt or alpine.
|
||||
|
||||
[[mail-command]]
|
||||
=== `mail`
|
||||
|
||||
man:mail[1] is the default MUA installed with FreeBSD. It is a console based MUA that offers the basic functionality required to send and receive text-based email. It provides limited attachment support and can only access local mailboxes.
|
||||
|
||||
Although `mail` does not natively support interaction with POP or IMAP servers, these mailboxes may be downloaded to a local [.filename]#mbox# using an application such as fetchmail.
|
||||
|
||||
In order to send and receive email, run `mail`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% mail
|
||||
....
|
||||
|
||||
The contents of the user's mailbox in [.filename]#/var/mail# are automatically read by `mail`. Should the mailbox be empty, the utility exits with a message indicating that no mail could be found. If mail exists, the application interface starts, and a list of messages will be displayed. Messages are automatically numbered, as can be seen in the following example:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Mail version 8.1 6/6/93. Type ? for help.
|
||||
"/var/mail/marcs": 3 messages 3 new
|
||||
>N 1 root@localhost Mon Mar 8 14:05 14/510 "test"
|
||||
N 2 root@localhost Mon Mar 8 14:05 14/509 "user account"
|
||||
N 3 root@localhost Mon Mar 8 14:05 14/509 "sample"
|
||||
....
|
||||
|
||||
Messages can now be read by typing kbd:[t] followed by the message number. This example reads the first email:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
& t 1
|
||||
Message 1:
|
||||
From root@localhost Mon Mar 8 14:05:52 2004
|
||||
X-Original-To: marcs@localhost
|
||||
Delivered-To: marcs@localhost
|
||||
To: marcs@localhost
|
||||
Subject: test
|
||||
Date: Mon, 8 Mar 2004 14:05:52 +0200 (SAST)
|
||||
From: root@localhost (Charlie Root)
|
||||
|
||||
This is a test message, please reply if you receive it.
|
||||
....
|
||||
|
||||
As seen in this example, the message will be displayed with full headers. To display the list of messages again, press kbd:[h].
|
||||
|
||||
If the email requires a reply, press either kbd:[R] or kbd:[r] `mail` keys. kbd:[R] instructs `mail` to reply only to the sender of the email, while kbd:[r] replies to all other recipients of the message. These commands can be suffixed with the mail number of the message to reply to. After typing the response, the end of the message should be marked by a single kbd:[.] on its own line. An example can be seen below:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
& R 1
|
||||
To: root@localhost
|
||||
Subject: Re: test
|
||||
|
||||
Thank you, I did get your email.
|
||||
.
|
||||
EOT
|
||||
....
|
||||
|
||||
In order to send a new email, press kbd:[m], followed by the recipient email address. Multiple recipients may be specified by separating each address with the kbd:[,] delimiter. The subject of the message may then be entered, followed by the message contents. The end of the message should be specified by putting a single kbd:[.] on its own line.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
& mail root@localhost
|
||||
Subject: I mastered mail
|
||||
|
||||
Now I can send and receive email using mail ... :)
|
||||
.
|
||||
EOT
|
||||
....
|
||||
|
||||
While using `mail`, press kbd:[?] to display help at any time. Refer to man:mail[1] for more help on how to use `mail`.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
man:mail[1] was not designed to handle attachments and thus deals with them poorly. Newer MUAs handle attachments in a more intelligent way. Users who prefer to use `mail` may find the package:converters/mpack[] port to be of considerable use.
|
||||
====
|
||||
|
||||
[[mutt-command]]
|
||||
=== mutt
|
||||
|
||||
mutt is a powerful MUA, with many features, including:
|
||||
|
||||
* The ability to thread messages.
|
||||
* PGP support for digital signing and encryption of email.
|
||||
* MIME support.
|
||||
* Maildir support.
|
||||
* Highly customizable.
|
||||
|
||||
Refer to http://www.mutt.org[http://www.mutt.org] for more information on mutt.
|
||||
|
||||
mutt may be installed using the package:mail/mutt[] port. After the port has been installed, mutt can be started by issuing the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% mutt
|
||||
....
|
||||
|
||||
mutt will automatically read and display the contents of the user mailbox in [.filename]#/var/mail#. If no mails are found, mutt will wait for commands from the user. The example below shows mutt displaying a list of messages:
|
||||
|
||||
image::mutt1.png[]
|
||||
|
||||
To read an email, select it using the cursor keys and press kbd:[Enter]. An example of mutt displaying email can be seen below:
|
||||
|
||||
image::mutt2.png[]
|
||||
|
||||
Similar to man:mail[1], mutt can be used to reply only to the sender of the message as well as to all recipients. To reply only to the sender of the email, press kbd:[r]. To send a group reply to the original sender as well as all the message recipients, press kbd:[g].
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
By default, mutt uses the man:vi[1] editor for creating and replying to emails. Each user can customize this by creating or editing the [.filename]#.muttrc# in their home directory and setting the `editor` variable or by setting the `EDITOR` environment variable. Refer to http://www.mutt.org/[http://www.mutt.org/] for more information about configuring mutt.
|
||||
====
|
||||
|
||||
To compose a new mail message, press kbd:[m]. After a valid subject has been given, mutt will start man:vi[1] so the email can be written. Once the contents of the email are complete, save and quit from `vi`. mutt will resume, displaying a summary screen of the mail that is to be delivered. In order to send the mail, press kbd:[y]. An example of the summary screen can be seen below:
|
||||
|
||||
image::mutt3.png[]
|
||||
|
||||
mutt contains extensive help which can be accessed from most of the menus by pressing kbd:[?]. The top line also displays the keyboard shortcuts where appropriate.
|
||||
|
||||
[[alpine-command]]
|
||||
=== alpine
|
||||
|
||||
alpine is aimed at a beginner user, but also includes some advanced features.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
alpine has had several remote vulnerabilities discovered in the past, which allowed remote attackers to execute arbitrary code as users on the local system, by the action of sending a specially-prepared email. While _known_ problems have been fixed, alpine code is written in an insecure style and the FreeBSD Security Officer believes there are likely to be other undiscovered vulnerabilities. Users install alpine at their own risk.
|
||||
====
|
||||
|
||||
The current version of alpine may be installed using the package:mail/alpine[] port. Once the port has installed, alpine can be started by issuing the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% alpine
|
||||
....
|
||||
|
||||
The first time alpine runs, it displays a greeting page with a brief introduction, as well as a request from the alpine development team to send an anonymous email message allowing them to judge how many users are using their client. To send this anonymous message, press kbd:[Enter]. Alternatively, press kbd:[E] to exit the greeting without sending an anonymous message. An example of the greeting page is shown below:
|
||||
|
||||
image::pine1.png[]
|
||||
|
||||
The main menu is then presented, which can be navigated using the cursor keys. This main menu provides shortcuts for the composing new mails, browsing mail directories, and administering address book entries. Below the main menu, relevant keyboard shortcuts to perform functions specific to the task at hand are shown.
|
||||
|
||||
The default directory opened by alpine is [.filename]#inbox#. To view the message index, press kbd:[I], or select the [.guimenuitem]#MESSAGE INDEX# option shown below:
|
||||
|
||||
image::pine2.png[]
|
||||
|
||||
The message index shows messages in the current directory and can be navigated by using the cursor keys. Highlighted messages can be read by pressing kbd:[Enter].
|
||||
|
||||
image::pine3.png[]
|
||||
|
||||
In the screenshot below, a sample message is displayed by alpine. Contextual keyboard shortcuts are displayed at the bottom of the screen. An example of one of a shortcut is kbd:[r], which tells the MUA to reply to the current message being displayed.
|
||||
|
||||
image::pine4.png[]
|
||||
|
||||
Replying to an email in alpine is done using the pico editor, which is installed by default with alpine. pico makes it easy to navigate the message and is easier for novice users to use than man:vi[1] or man:mail[1]. Once the reply is complete, the message can be sent by pressing kbd:[Ctrl+X]. alpine will ask for confirmation before sending the message.
|
||||
|
||||
image::pine5.png[]
|
||||
|
||||
alpine can be customized using the [.guimenuitem]#SETUP# option from the main menu. Consult http://www.washington.edu/alpine/[http://www.washington.edu/alpine/] for more information.
|
||||
|
||||
[[mail-fetchmail]]
|
||||
== Using fetchmail
|
||||
|
||||
fetchmail is a full-featured IMAP and POP client. It allows users to automatically download mail from remote IMAP and POP servers and save it into local mailboxes where it can be accessed more easily. fetchmail can be installed using the package:mail/fetchmail[] port, and offers various features, including:
|
||||
|
||||
* Support for the POP3, APOP, KPOP, IMAP, ETRN and ODMR protocols.
|
||||
* Ability to forward mail using SMTP, which allows filtering, forwarding, and aliasing to function normally.
|
||||
* May be run in daemon mode to check periodically for new messages.
|
||||
* Can retrieve multiple mailboxes and forward them, based on configuration, to different local users.
|
||||
|
||||
This section explains some of the basic features of fetchmail. This utility requires a [.filename]#.fetchmailrc# configuration in the user's home directory in order to run correctly. This file includes server information as well as login credentials. Due to the sensitive nature of the contents of this file, it is advisable to make it readable only by the user, with the following command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% chmod 600 .fetchmailrc
|
||||
....
|
||||
|
||||
The following [.filename]#.fetchmailrc# serves as an example for downloading a single user mailbox using POP. It tells fetchmail to connect to `example.com` using a username of `joesoap` and a password of `XXX`. This example assumes that the user `joesoap` exists on the local system.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
poll example.com protocol pop3 username "joesoap" password "XXX"
|
||||
....
|
||||
|
||||
The next example connects to multiple POP and IMAP servers and redirects to different local usernames where applicable:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
poll example.com proto pop3:
|
||||
user "joesoap", with password "XXX", is "jsoap" here;
|
||||
user "andrea", with password "XXXX";
|
||||
poll example2.net proto imap:
|
||||
user "john", with password "XXXXX", is "myth" here;
|
||||
....
|
||||
|
||||
fetchmail can be run in daemon mode by running it with `-d`, followed by the interval (in seconds) that fetchmail should poll servers listed in [.filename]#.fetchmailrc#. The following example configures fetchmail to poll every 600 seconds:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% fetchmail -d 600
|
||||
....
|
||||
|
||||
More information on fetchmail can be found at http://www.fetchmail.info/[http://www.fetchmail.info/].
|
||||
|
||||
[[mail-procmail]]
|
||||
== Using procmail
|
||||
|
||||
procmail is a powerful application used to filter incoming mail. It allows users to define "rules" which can be matched to incoming mails to perform specific functions or to reroute mail to alternative mailboxes or email addresses. procmail can be installed using the package:mail/procmail[] port. Once installed, it can be directly integrated into most MTAs. Consult the MTA documentation for more information. Alternatively, procmail can be integrated by adding the following line to a [.filename]#.forward# in the home directory of the user:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
"|exec /usr/local/bin/procmail || exit 75"
|
||||
....
|
||||
|
||||
The following section displays some basic procmail rules, as well as brief descriptions of what they do. Rules must be inserted into a [.filename]#.procmailrc#, which must reside in the user's home directory.
|
||||
|
||||
The majority of these rules can be found in man:procmailex[5].
|
||||
|
||||
To forward all mail from mailto:user@example.com[user@example.com] to an external address of mailto:goodmail@example2.com[goodmail@example2.com]:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:0
|
||||
* ^From.*user@example.com
|
||||
! goodmail@example2.com
|
||||
....
|
||||
|
||||
To forward all mails shorter than 1000 bytes to an external address of mailto:goodmail@example2.com[goodmail@example2.com]:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:0
|
||||
* < 1000
|
||||
! goodmail@example2.com
|
||||
....
|
||||
|
||||
To send all mail sent to mailto:alternate@example.com[alternate@example.com] to a mailbox called [.filename]#alternate#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:0
|
||||
* ^TOalternate@example.com
|
||||
alternate
|
||||
....
|
||||
|
||||
To send all mail with a subject of "Spam" to [.filename]#/dev/null#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:0
|
||||
^Subject:.*Spam
|
||||
/dev/null
|
||||
....
|
||||
|
||||
A useful recipe that parses incoming `FreeBSD.org` mailing lists and places each list in its own mailbox:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:0
|
||||
* ^Sender:.owner-freebsd-\/[^@]+@FreeBSD.ORG
|
||||
{
|
||||
LISTNAME=${MATCH}
|
||||
:0
|
||||
* LISTNAME??^\/[^@]+
|
||||
FreeBSD-${MATCH}
|
||||
}
|
||||
....
|
||||
598
documentation/content/en/books/handbook/mirrors/_index.adoc
Normal file
598
documentation/content/en/books/handbook/mirrors/_index.adoc
Normal file
|
|
@ -0,0 +1,598 @@
|
|||
---
|
||||
title: Appendix A. Obtaining FreeBSD
|
||||
part: Part V. Appendices
|
||||
prev: books/handbook/partv
|
||||
next: books/handbook/bibliography
|
||||
---
|
||||
|
||||
[appendix]
|
||||
[[mirrors]]
|
||||
= Obtaining FreeBSD
|
||||
:doctype: book
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: A
|
||||
|
||||
include::shared/mirrors.adoc[]
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
[[mirrors-cdrom]]
|
||||
== CD and DVD Sets
|
||||
|
||||
FreeBSD CD and DVD sets are available from several online retailers:
|
||||
|
||||
* FreeBSD Mall, Inc. +
|
||||
2420 Sand Creek Rd C-1 #347 +
|
||||
Brentwood, CA +
|
||||
94513 +
|
||||
USA +
|
||||
Phone: +1 925 240-6652 +
|
||||
Fax: +1 925 674-0821 +
|
||||
Email: <info@freebsdmall.com> +
|
||||
WWW: https://www.freebsdmall.com
|
||||
|
||||
* Getlinux +
|
||||
78 Rue de la Croix Rochopt +
|
||||
Épinay-sous-Sénart +
|
||||
91860 +
|
||||
France +
|
||||
Email: <contact@getlinux.fr> +
|
||||
WWW: http://www.getlinux.fr/
|
||||
|
||||
* Dr. Hinner EDV +
|
||||
Kochelseestr. 11 +
|
||||
D-81371 München +
|
||||
Germany +
|
||||
Phone: (0177) 428 419 0 +
|
||||
Email: <infow@hinner.de> +
|
||||
WWW: http://www.hinner.de/linux/freebsd.html
|
||||
|
||||
* Linux Center +
|
||||
Galernaya Street, 55 +
|
||||
Saint-Petersburg +
|
||||
190000 +
|
||||
Russia +
|
||||
Phone: +7-812-309-06-86 +
|
||||
Email: <info@linuxcenter.ru> +
|
||||
WWW: http://linuxcenter.ru/shop/freebsd
|
||||
|
||||
[[mirrors-ftp]]
|
||||
== FTP Sites
|
||||
|
||||
The official sources for FreeBSD are available via anonymous FTP from a worldwide set of mirror sites. The site link:ftp://ftp.FreeBSD.org/pub/FreeBSD/[ftp://ftp.FreeBSD.org/pub/FreeBSD/] is available via HTTP and FTP. It is made up of many machines operated by the project cluster administrators and behind GeoDNS to direct users to the closest available mirror.
|
||||
|
||||
Additionally, FreeBSD is available via anonymous FTP from the following mirror sites. When obtaining FreeBSD via anonymous FTP, please try to use a nearby site. The mirror sites listed as "Primary Mirror Sites" typically have the entire FreeBSD archive (all the currently available versions for each of the architectures) but faster download speeds are probably available from a site that is in your country or region. The regional sites carry the most recent versions for the most popular architecture(s) but might not carry the entire FreeBSD archive. All sites provide access via anonymous FTP but some sites also provide access via other methods. The access methods available for each site are provided in parentheses after the hostname.
|
||||
|
||||
<<central, {central}>>, <<primary, {mirrors-primary}>>, <<armenia, {mirrors-armenia}>>, <<australia, {mirrors-australia}>>, <<austria, {mirrors-austria}>>, <<brazil, {mirrors-brazil}>>, <<czech-republic, {mirrors-czech}>>, <<denmark, {mirrors-denmark}>>, <<estonia, {mirrors-estonia}>>, <<finland, {mirrors-finland}>>, <<france, {mirrors-france}>>, <<germany, {mirrors-germany}>>, <<greece, {mirrors-greece}>>, <<hong-kong, {mirrors-hongkong}>>, <<ireland, {mirrors-ireland}>>, <<japan, {mirrors-japan}>>, <<korea, {mirrors-korea}>>, <<latvia, {mirrors-latvia}>>, <<lithuania, {mirrors-lithuania}>>, <<netherlands, {mirrors-netherlands}>>, <<new-zealand, {mirrors-new-zealand}>>, <<norway, {mirrors-norway}>>, <<poland, {mirrors-poland}>>, <<russia, {mirrors-russia}>>, <<saudi-arabia, {mirrors-saudi-arabia}>>, <<slovenia, {mirrors-slovenia}>>, <<south-africa, {mirrors-south-africa}>>, <<spain, {mirrors-spain}>>, <<sweden, {mirrors-sweden}>>, <<switzerland, {mirrors-switzerland}>>, <<taiwan, {mirrors-taiwan}>>, <<ukraine, {mirrors-ukraine}>>, <<uk, {mirrors-uk}>>, <<usa, {mirrors-us}>>.
|
||||
|
||||
(as of UTC)
|
||||
|
||||
[[central]]
|
||||
*{central}*
|
||||
|
||||
{central-ftp} (ftp / ftpv6 / {central-http} / {central-httpv6})
|
||||
|
||||
[[primary]]
|
||||
*{mirrors-primary}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-primary-email}>` for this domain.
|
||||
|
||||
* {mirrors-primary-ftp1} (ftp)
|
||||
* {mirrors-primary-ftp2} (ftp)
|
||||
* {mirrors-primary-ftp3} (ftp)
|
||||
* {mirrors-primary-ftp4} (ftp / ftpv6 / {mirrors-primary-ftp4-http} / {mirrors-primary-ftp4-httpv6})
|
||||
* {mirrors-primary-ftp5} (ftp)
|
||||
* {mirrors-primary-ftp6} (ftp)
|
||||
* {mirrors-primary-ftp7} (ftp)
|
||||
* {mirrors-primary-ftp10} (ftp / ftpv6 / {mirrors-primary-ftp10-http} / {mirrors-primary-ftp10-httpv6})
|
||||
* {mirrors-primary-ftp11} (ftp)
|
||||
* {mirrors-primary-ftp13} (ftp)
|
||||
* {mirrors-primary-ftp14} (ftp / {mirrors-primary-ftp14-http})
|
||||
|
||||
[[armenia]]
|
||||
*{mirrors-armenia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-armenia-email}>` for this domain.
|
||||
|
||||
* {mirrors-armenia-ftp} (ftp / {mirrors-armenia-ftp-http} / rsync)
|
||||
|
||||
[[australia]]
|
||||
*{mirrors-australia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-australia-email}>` for this domain.
|
||||
|
||||
* {mirrors-australia-ftp} (ftp)
|
||||
* {mirrors-australia-ftp2} (ftp)
|
||||
* {mirrors-australia-ftp3} (ftp)
|
||||
|
||||
[[austria]]
|
||||
*{mirrors-austria}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-austria-email}>` for this domain.
|
||||
|
||||
* {mirrors-austria-ftp} (ftp / ftpv6 / {mirrors-austria-ftp-http} / {mirrors-austria-ftp-httpv6})
|
||||
|
||||
[[brazil]]
|
||||
*{mirrors-brazil}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-brazil-email}>` for this domain.
|
||||
|
||||
* {mirrors-brazil-ftp2} (ftp / {mirrors-brazil-ftp2-http})
|
||||
* {mirrors-brazil-ftp3} (ftp / rsync)
|
||||
* {mirrors-brazil-ftp4} (ftp)
|
||||
|
||||
[[czech-republic]]
|
||||
*{mirrors-czech}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-czech-email}>` for this domain.
|
||||
|
||||
* {mirrors-czech-ftp} (ftp / {mirrors-czech-ftpv6} / {mirrors-czech-ftp-http} / {mirrors-czech-ftp-httpv6} / rsync / rsyncv6)
|
||||
* {mirrors-czech-ftp2} (ftp / {mirrors-czech-ftp2-http})
|
||||
|
||||
[[denmark]]
|
||||
*{mirrors-denmark}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-denmark-email}>` for this domain.
|
||||
|
||||
* {mirrors-denmark-ftp} (ftp / ftpv6 / {mirrors-denmark-ftp-http} / {mirrors-denmark-ftp-httpv6})
|
||||
|
||||
[[estonia]]
|
||||
*{mirrors-estonia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-estonia-email}>` for this domain.
|
||||
|
||||
* {mirrors-estonia-ftp} (ftp)
|
||||
|
||||
[[finland]]
|
||||
*{mirrors-finland}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-finland-email}>` for this domain.
|
||||
|
||||
* {mirrors-finland-ftp} (ftp)
|
||||
|
||||
[[france]]
|
||||
*{mirrors-france}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-france-email}>` for this domain.
|
||||
|
||||
* {mirrors-france-ftp} (ftp)
|
||||
* {mirrors-france-ftp1} (ftp / {mirrors-france-ftp1-http} / rsync)
|
||||
* {mirrors-france-ftp3} (ftp)
|
||||
* {mirrors-france-ftp5} (ftp)
|
||||
* {mirrors-france-ftp6} (ftp / rsync)
|
||||
* {mirrors-france-ftp7} (ftp)
|
||||
* {mirrors-france-ftp8} (ftp)
|
||||
|
||||
[[germany]]
|
||||
*{mirrors-germany}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-germany-email}>` for this domain.
|
||||
|
||||
* ftp://ftp.de.FreeBSD.org/pub/FreeBSD/ (ftp)
|
||||
* ftp://ftp1.de.FreeBSD.org/freebsd/ (ftp / http://www1.de.FreeBSD.org/freebsd/ / rsync://rsync3.de.FreeBSD.org/freebsd/)
|
||||
* ftp://ftp2.de.FreeBSD.org/pub/FreeBSD/ (ftp / http://ftp2.de.FreeBSD.org/pub/FreeBSD/ / rsync)
|
||||
* ftp://ftp4.de.FreeBSD.org/FreeBSD/ (ftp / http://ftp4.de.FreeBSD.org/pub/FreeBSD/)
|
||||
* ftp://ftp5.de.FreeBSD.org/pub/FreeBSD/ (ftp)
|
||||
* ftp://ftp7.de.FreeBSD.org/pub/FreeBSD/ (ftp / http://ftp7.de.FreeBSD.org/pub/FreeBSD/)
|
||||
* ftp://ftp8.de.FreeBSD.org/pub/FreeBSD/ (ftp)
|
||||
|
||||
[[greece]]
|
||||
*{mirrors-greece}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-greece-email}>` for this domain.
|
||||
|
||||
* {mirrors-greece-ftp} (ftp)
|
||||
* {mirrors-greece-ftp2} (ftp)
|
||||
|
||||
[[hong-kong]]
|
||||
*{mirrors-hongkong}*
|
||||
|
||||
{mirrors-hongkong-ftp} (ftp)
|
||||
|
||||
[[ireland]]
|
||||
*{mirrors-ireland}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-ireland-email}>` for this domain.
|
||||
|
||||
* {mirrors-ireland-ftp} (ftp / rsync)
|
||||
|
||||
[[japan]]
|
||||
*{mirrors-japan}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-japan-email}>` for this domain.
|
||||
|
||||
* {mirrors-japan-ftp} (ftp)
|
||||
* {mirrors-japan-ftp2} (ftp)
|
||||
* {mirrors-japan-ftp3} (ftp)
|
||||
* {mirrors-japan-ftp4} (ftp)
|
||||
* {mirrors-japan-ftp5} (ftp)
|
||||
* {mirrors-japan-ftp6} (ftp)
|
||||
* {mirrors-japan-ftp7} (ftp)
|
||||
* {mirrors-japan-ftp8} (ftp)
|
||||
* {mirrors-japan-ftp9} (ftp)
|
||||
|
||||
[[korea]]
|
||||
*{mirrors-korea}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-korea-email}>` for this domain.
|
||||
|
||||
* {mirrors-korea-ftp} (ftp / rsync)
|
||||
* {mirrors-korea-ftp2} (ftp / {mirrors-korea-ftp2-http})
|
||||
|
||||
[[latvia]]
|
||||
*{mirrors-latvia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-latvia-email}>` for this domain.
|
||||
|
||||
* {mirrors-latvia-ftp} (ftp / {mirrors-latvia-ftp-http})
|
||||
|
||||
[[lithuania]]
|
||||
*{mirrors-lithuania}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-lithuania-email}>` for this domain.
|
||||
|
||||
* {mirrors-lithuania-ftp} (ftp / {mirrors-lithuania-ftp-http})
|
||||
|
||||
[[netherlands]]
|
||||
*{mirrors-netherlands}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-netherlands-email}>` for this domain.
|
||||
|
||||
* {mirrors-netherlands-ftp} (ftp / {mirrors-netherlands-ftp-http} / rsync)
|
||||
* {mirrors-netherlands-ftp2} (ftp)
|
||||
|
||||
[[new-zealand]]
|
||||
*{mirrors-new-zealand}*
|
||||
|
||||
* {mirrors-new-zealand-ftp} (ftp / {mirrors-new-zealand-ftp-http})
|
||||
|
||||
[[norway]]
|
||||
*{mirrors-norway}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-norway-email}>` for this domain.
|
||||
|
||||
* {mirrors-norway-ftp} (ftp / rsync)
|
||||
|
||||
[[poland]]
|
||||
*{mirrors-poland}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-poland-email}>` for this domain.
|
||||
|
||||
* {mirrors-poland-ftp} (ftp)
|
||||
* ftp2.pl.FreeBSD.org
|
||||
|
||||
[[russia]]
|
||||
*{mirrors-russia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-russia-email}>` for this domain.
|
||||
|
||||
* {mirrors-russia-ftp} (ftp / {mirrors-russia-ftp-http} / rsync)
|
||||
* {mirrors-russia-ftp2} (ftp / {mirrors-russia-ftp2-http} / rsync)
|
||||
* {mirrors-russia-ftp4} (ftp)
|
||||
* {mirrors-russia-ftp5} (ftp / {mirrors-russia-ftp5-http} / rsync)
|
||||
* {mirrors-russia-ftp6} (ftp)
|
||||
|
||||
[[saudi-arabia]]
|
||||
*{mirrors-saudi-arabia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-saudi-arabia-email}>` for this domain.
|
||||
|
||||
* {mirrors-saudi-arabia-ftp} (ftp)
|
||||
|
||||
[[slovenia]]
|
||||
*{mirrors-slovenia}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-slovenia-email}>` for this domain.
|
||||
|
||||
* {mirrors-slovenia-ftp} (ftp)
|
||||
|
||||
[[south-africa]]
|
||||
*{mirrors-south-africa}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-south-africa-email}>` for this domain.
|
||||
|
||||
* {mirrors-south-africa-ftp} (ftp)
|
||||
* {mirrors-south-africa-ftp2} (ftp)
|
||||
* {mirrors-south-africa-ftp4} (ftp)
|
||||
|
||||
[[spain]]
|
||||
*{mirrors-spain}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-spain-email}>` for this domain.
|
||||
|
||||
* {mirrors-spain-ftp} (ftp / {mirrors-spain-ftp-http})
|
||||
* {mirrors-spain-ftp3} (ftp)
|
||||
|
||||
[[sweden]]
|
||||
*{mirrors-sweden}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-sweden-email}>` for this domain.
|
||||
|
||||
* {mirrors-sweden-ftp} (ftp)
|
||||
* {mirrors-sweden-ftp2} (ftp / {mirrors-sweden-ftp2-rsync})
|
||||
* {mirrors-sweden-ftp3} (ftp)
|
||||
* {mirrors-sweden-ftp4} (ftp / {mirrors-sweden-ftp4v6} / {mirrors-sweden-ftp4-http} / {mirrors-sweden-ftp4-httpv6} / {mirrors-sweden-ftp4-rsync} / {mirrors-sweden-ftp4-rsyncv6})
|
||||
* {mirrors-sweden-ftp6} (ftp / {mirrors-sweden-ftp6-http})
|
||||
|
||||
[[switzerland]]
|
||||
*{mirrors-switzerland}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-switzerland-email}>` for this domain.
|
||||
|
||||
* {mirrors-switzerland-ftp} (ftp / {mirrors-switzerland-ftp-http})
|
||||
|
||||
[[taiwan]]
|
||||
*{mirrors-taiwan}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-taiwan-email}>` for this domain.
|
||||
|
||||
* {mirrors-taiwan-ftp} (ftp / {mirrors-taiwan-ftpv6} / rsync / rsyncv6)
|
||||
* {mirrors-taiwan-ftp2} (ftp / {mirrors-taiwan-ftp2v6} / {mirrors-taiwan-ftp2-http} / {mirrors-taiwan-ftp2-httpv6} / rsync / rsyncv6)
|
||||
* {mirrors-taiwan-ftp4} (ftp)
|
||||
* {mirrors-taiwan-ftp5} (ftp)
|
||||
* {mirrors-taiwan-ftp6} (ftp / {mirrors-taiwan-ftp6v6} / rsync)
|
||||
* {mirrors-taiwan-ftp7} (ftp)
|
||||
* {mirrors-taiwan-ftp8} (ftp)
|
||||
* {mirrors-taiwan-ftp11} (ftp / {mirrors-taiwan-ftp11-http})
|
||||
* {mirrors-taiwan-ftp12} (ftp)
|
||||
* {mirrors-taiwan-ftp13} (ftp)
|
||||
* {mirrors-taiwan-ftp14} (ftp)
|
||||
* {mirrors-taiwan-ftp15} (ftp)
|
||||
|
||||
[[ukraine]]
|
||||
*{mirrors-ukraine}*
|
||||
|
||||
* {mirrors-ukraine-ftp} (ftp / {mirrors-ukraine-ftp-http})
|
||||
* {mirrors-ukraine-ftp6} (ftp / {mirrors-ukraine-ftp6-http} / {mirrors-ukraine-ftp6-rsync})
|
||||
* {mirrors-ukraine-ftp7} (ftp)
|
||||
|
||||
[[uk]]
|
||||
*{mirrors-uk}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-uk-email}>` for this domain.
|
||||
|
||||
* {mirrors-uk-ftp} (ftp)
|
||||
* {mirrors-uk-ftp2} (ftp / {mirrors-uk-ftp2-rsync})
|
||||
* {mirrors-uk-ftp3} (ftp)
|
||||
* {mirrors-uk-ftp4} (ftp)
|
||||
* {mirrors-uk-ftp5} (ftp)
|
||||
|
||||
[[usa]]
|
||||
*{mirrors-us}*
|
||||
|
||||
In case of problems, please contact the hostmaster `<{mirrors-us-email}>` for this domain.
|
||||
|
||||
* {mirrors-us-ftp} (ftp)
|
||||
* {mirrors-us-ftp2} (ftp)
|
||||
* {mirrors-us-ftp3} (ftp)
|
||||
* {mirrors-us-ftp4} (ftp / ftpv6 / {mirrors-us-ftp4-http} / {mirrors-us-ftp4-httpv6})
|
||||
* {mirrors-us-ftp5} (ftp)
|
||||
* {mirrors-us-ftp6} (ftp)
|
||||
* {mirrors-us-ftp8} (ftp)
|
||||
* {mirrors-us-ftp10} (ftp)
|
||||
* {mirrors-us-ftp11} (ftp)
|
||||
* {mirrors-us-ftp13} (ftp / {mirrors-us-ftp13-http} / rsync)
|
||||
* {mirrors-us-ftp14} (ftp / {mirrors-us-ftp14-http})
|
||||
* {mirrors-us-ftp15} (ftp)
|
||||
|
||||
[[svn]]
|
||||
== Using Subversion
|
||||
|
||||
[[svn-intro]]
|
||||
=== Introduction
|
||||
|
||||
As of July 2012, FreeBSD uses Subversion as the only version control system for storing all of FreeBSD's source code, documentation, and the Ports Collection.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Subversion is generally a developer tool. Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection.
|
||||
====
|
||||
|
||||
This section demonstrates how to install Subversion on a FreeBSD system and use it to create a local copy of a FreeBSD repository. Additional information on the use of Subversion is included.
|
||||
|
||||
[[svn-ssl-certificates]]
|
||||
=== Root SSL Certificates
|
||||
|
||||
Installing package:security/ca_root_nss[] allows Subversion to verify the identity of HTTPS repository servers. The root SSL certificates can be installed from a port:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/security/ca_root_nss
|
||||
# make install clean
|
||||
....
|
||||
|
||||
or as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install ca_root_nss
|
||||
....
|
||||
|
||||
[[svn-svnlite]]
|
||||
=== Svnlite
|
||||
|
||||
A lightweight version of Subversion is already installed on FreeBSD as `svnlite`. The port or package version of Subversion is only needed if the Python or Perl API is needed, or if a later version of Subversion is desired.
|
||||
|
||||
The only difference from normal Subversion use is that the command name is `svnlite`.
|
||||
|
||||
[[svn-install]]
|
||||
=== Installation
|
||||
|
||||
If `svnlite` is unavailable or the full version of Subversion is needed, then it must be installed.
|
||||
|
||||
Subversion can be installed from the Ports Collection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/ports/devel/subversion
|
||||
# make install clean
|
||||
....
|
||||
|
||||
Subversion can also be installed as a package:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pkg install subversion
|
||||
....
|
||||
|
||||
[[svn-usage]]
|
||||
=== Running Subversion
|
||||
|
||||
To fetch a clean copy of the sources into a local directory, use `svn`. The files in this directory are called a _local working copy_.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
Move or delete an existing destination directory before using `checkout` for the first time.
|
||||
|
||||
Checkout over an existing non-`svn` directory can cause conflicts between the existing files and those brought in from the repository.
|
||||
====
|
||||
|
||||
Subversion uses URLs to designate a repository, taking the form of _protocol://hostname/path_. The first component of the path is the FreeBSD repository to access. There are three different repositories, `base` for the FreeBSD base system source code, `ports` for the Ports Collection, and `doc` for documentation. For example, the URL `https://svn.FreeBSD.org/ports/head/` specifies the main branch of the ports repository, using the `https` protocol.
|
||||
|
||||
A checkout from a given repository is performed with a command like this:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svn checkout https://svn.FreeBSD.org/repository/branch lwcdir
|
||||
....
|
||||
|
||||
where:
|
||||
|
||||
* _repository_ is one of the Project repositories: `base`, `ports`, or `doc`.
|
||||
* _branch_ depends on the repository used. `ports` and `doc` are mostly updated in the `head` branch, while `base` maintains the latest version of -CURRENT under `head` and the respective latest versions of the -STABLE branches under `stable/9` (9._x_) and `stable/10` (10._x_).
|
||||
* _lwcdir_ is the target directory where the contents of the specified branch should be placed. This is usually [.filename]#/usr/ports# for `ports`, [.filename]#/usr/src# for `base`, and [.filename]#/usr/doc# for `doc`.
|
||||
|
||||
This example checks out the Ports Collection from the FreeBSD repository using the HTTPS protocol, placing the local working copy in [.filename]#/usr/ports#. If [.filename]#/usr/ports# is already present but was not created by `svn`, remember to rename or delete it before the checkout.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svn checkout https://svn.FreeBSD.org/ports/head /usr/ports
|
||||
....
|
||||
|
||||
Because the initial checkout must download the full branch of the remote repository, it can take a while. Please be patient.
|
||||
|
||||
After the initial checkout, the local working copy can be updated by running:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svn update lwcdir
|
||||
....
|
||||
|
||||
To update [.filename]#/usr/ports# created in the example above, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# svn update /usr/ports
|
||||
....
|
||||
|
||||
The update is much quicker than a checkout, only transferring files that have changed.
|
||||
|
||||
An alternate way of updating the local working copy after checkout is provided by the [.filename]#Makefile# in the [.filename]#/usr/ports#, [.filename]#/usr/src#, and [.filename]#/usr/doc# directories. Set `SVN_UPDATE` and use the `update` target. For example, to update [.filename]#/usr/src#:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cd /usr/src
|
||||
# make update SVN_UPDATE=yes
|
||||
....
|
||||
|
||||
[[svn-mirrors]]
|
||||
=== Subversion Mirror Sites
|
||||
|
||||
The FreeBSD Subversion repository is:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
svn.FreeBSD.org
|
||||
....
|
||||
|
||||
This is a publicly accessible mirror network that uses GeoDNS to select an appropriate back end server. To view the FreeBSD Subversion repositories through a browser, use https://svnweb.FreeBSD.org/[https://svnweb.FreeBSD.org/].
|
||||
|
||||
HTTPS is the preferred protocol, but the [.filename]#security/ca_root_nss# package will need to be installed in order to automatically validate certificates.
|
||||
|
||||
=== For More Information
|
||||
|
||||
For other information about using Subversion, please see the "Subversion Book", titled http://svnbook.red-bean.com/[Version Control with Subversion], or the http://subversion.apache.org/docs/[Subversion Documentation].
|
||||
|
||||
[[mirrors-rsync]]
|
||||
== Using rsync
|
||||
|
||||
These sites make FreeBSD available through the rsync protocol. The rsync utility transfers only the differences between two sets of files. This is useful for mirror sites of the FreeBSD FTP server. The rsync suite is available for many operating systems, on FreeBSD, see the package:net/rsync[] port or use the package.
|
||||
|
||||
Czech Republic::
|
||||
rsync://ftp.cz.FreeBSD.org/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** ftp: A partial mirror of the FreeBSD FTP server.
|
||||
** FreeBSD: A full mirror of the FreeBSD FTP server.
|
||||
|
||||
Netherlands::
|
||||
rsync://ftp.nl.FreeBSD.org/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** FreeBSD: A full mirror of the FreeBSD FTP server.
|
||||
|
||||
Russia::
|
||||
rsync://ftp.mtu.ru/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** FreeBSD: A full mirror of the FreeBSD FTP server.
|
||||
** FreeBSD-Archive: The mirror of FreeBSD Archive FTP server.
|
||||
|
||||
Sweden::
|
||||
rsync://ftp4.se.freebsd.org/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** FreeBSD: A full mirror of the FreeBSD FTP server.
|
||||
|
||||
Taiwan::
|
||||
rsync://ftp.tw.FreeBSD.org/
|
||||
+
|
||||
rsync://ftp2.tw.FreeBSD.org/
|
||||
+
|
||||
rsync://ftp6.tw.FreeBSD.org/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** FreeBSD: A full mirror of the FreeBSD FTP server.
|
||||
|
||||
United Kingdom::
|
||||
rsync://rsync.mirrorservice.org/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** ftp.freebsd.org: A full mirror of the FreeBSD FTP server.
|
||||
|
||||
United States of America::
|
||||
rsync://ftp-master.FreeBSD.org/
|
||||
+
|
||||
This server may only be used by FreeBSD primary mirror sites.
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** FreeBSD: The master archive of the FreeBSD FTP server.
|
||||
** acl: The FreeBSD master ACL list.
|
||||
+
|
||||
rsync://ftp13.FreeBSD.org/
|
||||
+
|
||||
Available collections:
|
||||
|
||||
** FreeBSD: A full mirror of the FreeBSD FTP server.
|
||||
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
1069
documentation/content/en/books/handbook/multimedia/_index.adoc
Normal file
1069
documentation/content/en/books/handbook/multimedia/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
2563
documentation/content/en/books/handbook/network-servers/_index.adoc
Normal file
2563
documentation/content/en/books/handbook/network-servers/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
20
documentation/content/en/books/handbook/parti.adoc
Normal file
20
documentation/content/en/books/handbook/parti.adoc
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
---
|
||||
title: Part I. Getting Started
|
||||
prev: books/handbook/preface
|
||||
next: books/handbook/introduction
|
||||
---
|
||||
|
||||
[[getting-started]]
|
||||
= Getting Started
|
||||
|
||||
This part of the handbook is for users and administrators who are new to FreeBSD. These chapters:
|
||||
|
||||
* Introduce FreeBSD.
|
||||
* Guide readers through the installation process.
|
||||
* Teach UNIX(R) basics and fundamentals.
|
||||
* Show how to install the wealth of third party applications available for FreeBSD.
|
||||
* Introduce X, the UNIX(R) windowing system, and detail how to configure a desktop environment that makes users more productive.
|
||||
|
||||
The number of forward references in the text have been kept to a minimum so that this section can be read from front to back with minimal page flipping.
|
||||
|
||||
include::content/en/books/handbook/toc-1.adoc[]
|
||||
20
documentation/content/en/books/handbook/partii.adoc
Normal file
20
documentation/content/en/books/handbook/partii.adoc
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
---
|
||||
title: Part II. Common Tasks
|
||||
prev: books/handbook/x11
|
||||
next: books/handbook/desktop
|
||||
---
|
||||
|
||||
[[common-tasks]]
|
||||
= Common Tasks
|
||||
|
||||
Now that the basics have been covered, this part of the book discusses some frequently used features of FreeBSD. These chapters:
|
||||
|
||||
* Introduce popular and useful desktop applications: browsers, productivity tools, document viewers, and more.
|
||||
* Introduce a number of multimedia tools available for FreeBSD.
|
||||
* Explain the process of building a customized FreeBSD kernel to enable extra functionality.
|
||||
* Describe the print system in detail, both for desktop and network-connected printer setups.
|
||||
* Show how to run Linux applications on the FreeBSD system.
|
||||
|
||||
Some of these chapters recommend prior reading, and this is noted in the synopsis at the beginning of each chapter.
|
||||
|
||||
include::content/en/books/handbook/toc-2.adoc[]
|
||||
14
documentation/content/en/books/handbook/partiii.adoc
Normal file
14
documentation/content/en/books/handbook/partiii.adoc
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
---
|
||||
title: Part III. System Administration
|
||||
prev: books/handbook/linuxemu
|
||||
next: books/handbook/config
|
||||
---
|
||||
|
||||
[[system-administration]]
|
||||
= System Administration
|
||||
|
||||
The remaining chapters cover all aspects of FreeBSD system administration. Each chapter starts by describing what will be learned as a result of reading the chapter, and also details what the reader is expected to know before tackling the material.
|
||||
|
||||
These chapters are designed to be read as the information is needed. They do not need to be read in any particular order, nor must all of them be read before beginning to use FreeBSD.
|
||||
|
||||
include::content/en/books/handbook/toc-3.adoc[]
|
||||
21
documentation/content/en/books/handbook/partiv.adoc
Normal file
21
documentation/content/en/books/handbook/partiv.adoc
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
title: Part IV. Network Communication
|
||||
prev: books/handbook/usb-device-mode
|
||||
next: books/handbook/serialcomms
|
||||
---
|
||||
|
||||
[[network-communication]]
|
||||
= Network Communication
|
||||
|
||||
FreeBSD is one of the most widely deployed operating systems for high performance network servers. The chapters in this part cover:
|
||||
|
||||
* Serial communication
|
||||
* `PPP` and `PPP` over Ethernet
|
||||
* Electronic Mail
|
||||
* Running Network Servers
|
||||
* Firewalls
|
||||
* Other Advanced Networking Topics
|
||||
|
||||
These chapters are designed to be read when the information is needed. They do not need to be read in any particular order, nor is it necessary to read all of them before using FreeBSD in a network environment.
|
||||
|
||||
include::content/en/books/handbook/toc-4.adoc[]
|
||||
10
documentation/content/en/books/handbook/partv.adoc
Normal file
10
documentation/content/en/books/handbook/partv.adoc
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
---
|
||||
title: Part V. Appendices
|
||||
prev: books/handbook/advanced-networking
|
||||
next: books/handbook/mirrors
|
||||
---
|
||||
|
||||
[[appendices]]
|
||||
= Appendices
|
||||
|
||||
include::content/en/books/handbook/toc-5.adoc[]
|
||||
52
documentation/content/en/books/handbook/pgpkeys/_index.adoc
Normal file
52
documentation/content/en/books/handbook/pgpkeys/_index.adoc
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
title: Appendix D. OpenPGP Keys
|
||||
part: Part V. Appendices
|
||||
prev: books/handbook/eresources
|
||||
next: books/handbook/freebsd-glossary
|
||||
---
|
||||
|
||||
[appendix]
|
||||
[[pgpkeys]]
|
||||
= OpenPGP Keys
|
||||
:doctype: book
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: D
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
:pgpkeys-path:
|
||||
|
||||
The OpenPGP keys of the `FreeBSD.org` officers are shown here. These keys can be used to verify a signature or send encrypted email to one of the officers. A full list of FreeBSD OpenPGP keys is available in the link:{pgpkeys}[PGP Keys] article. The complete keyring can be downloaded at https://www.FreeBSD.org/doc/pgpkeyring.txt[https://www.FreeBSD.org/doc/pgpkeyring.txt].
|
||||
|
||||
[[pgpkeys-officers]]
|
||||
== Officers
|
||||
|
||||
=== {security-officer-name} `<{security-officer-email}>`
|
||||
include::{pgpkeys-path}static/pgpkeys/security-officer.key[]
|
||||
|
||||
=== {secteam-secretary-name} `<{secteam-secretary-email}>`
|
||||
include::{pgpkeys-path}static/pgpkeys/secteam-secretary.key[]
|
||||
|
||||
=== {core-secretary-name} `<{core-secretary-email}>`
|
||||
include::{pgpkeys-path}static/pgpkeys/core-secretary.key[]
|
||||
|
||||
=== {portmgr-secretary-name} `<{portmgr-secretary-email}>`
|
||||
include::{pgpkeys-path}static/pgpkeys/portmgr-secretary.key[]
|
||||
|
||||
=== `{doceng-secretary-email}`
|
||||
include::{pgpkeys-path}static/pgpkeys/doceng-secretary.key[]
|
||||
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
1147
documentation/content/en/books/handbook/ports/_index.adoc
Normal file
1147
documentation/content/en/books/handbook/ports/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
827
documentation/content/en/books/handbook/ppp-and-slip/_index.adoc
Normal file
827
documentation/content/en/books/handbook/ppp-and-slip/_index.adoc
Normal file
|
|
@ -0,0 +1,827 @@
|
|||
---
|
||||
title: Chapter 28. PPP
|
||||
part: IV. Network Communication
|
||||
prev: books/handbook/serialcomms
|
||||
next: books/handbook/mail
|
||||
---
|
||||
|
||||
[[ppp-and-slip]]
|
||||
= PPP
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 28
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/ppp-and-slip/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/ppp-and-slip/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/ppp-and-slip/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[ppp-and-slip-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
FreeBSD supports the Point-to-Point (PPP) protocol which can be used to establish a network or Internet connection using a dial-up modem. This chapter describes how to configure modem-based communication services in FreeBSD.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* How to configure, use, and troubleshoot a PPP connection.
|
||||
* How to set up PPP over Ethernet (PPPoE).
|
||||
* How to set up PPP over ATM (PPPoA).
|
||||
|
||||
Before reading this chapter, you should:
|
||||
|
||||
* Be familiar with basic network terminology.
|
||||
* Understand the basics and purpose of a dial-up connection and PPP.
|
||||
|
||||
[[userppp]]
|
||||
== Configuring PPP
|
||||
|
||||
FreeBSD provides built-in support for managing dial-up PPP connections using man:ppp[8]. The default FreeBSD kernel provides support for [.filename]#tun# which is used to interact with a modem hardware. Configuration is performed by editing at least one configuration file, and configuration files containing examples are provided. Finally, `ppp` is used to start and manage connections.
|
||||
|
||||
In order to use a PPP connection, the following items are needed:
|
||||
|
||||
* A dial-up account with an Internet Service Provider (ISP).
|
||||
* A dial-up modem.
|
||||
* The dial-up number for the ISP.
|
||||
* The login name and password assigned by the ISP.
|
||||
* The IP address of one or more DNS servers. Normally, the ISP provides these addresses. If it did not, FreeBSD can be configured to use DNS negotiation.
|
||||
|
||||
If any of the required information is missing, contact the ISP.
|
||||
|
||||
The following information may be supplied by the ISP, but is not necessary:
|
||||
|
||||
* The IP address of the default gateway. If this information is unknown, the ISP will automatically provide the correct value during connection setup. When configuring PPP on FreeBSD, this address is referred to as `HISADDR`.
|
||||
* The subnet mask. If the ISP has not provided one, `255.255.255.255` will be used in the man:ppp[8] configuration file.
|
||||
*
|
||||
+
|
||||
If the ISP has assigned a static IP address and hostname, it should be input into the configuration file. Otherwise, this information will be automatically provided during connection setup.
|
||||
|
||||
The rest of this section demonstrates how to configure FreeBSD for common PPP connection scenarios. The required configuration file is [.filename]#/etc/ppp/ppp.conf# and additional files and examples are available in [.filename]#/usr/shared/examples/ppp/#.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Throughout this section, many of the file examples display line numbers. These line numbers have been added to make it easier to follow the discussion and are not meant to be placed in the actual file.
|
||||
|
||||
When editing a configuration file, proper indentation is important. Lines that end in a `:` start in the first column (beginning of the line) while all other lines should be indented as shown using spaces or tabs.
|
||||
====
|
||||
|
||||
[[userppp-staticIP]]
|
||||
=== Basic Configuration
|
||||
|
||||
In order to configure a PPP connection, first edit [.filename]#/etc/ppp/ppp.conf# with the dial-in information for the ISP. This file is described as follows:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
1 default:
|
||||
2 set log Phase Chat LCP IPCP CCP tun command
|
||||
3 ident user-ppp VERSION
|
||||
4 set device /dev/cuau0
|
||||
5 set speed 115200
|
||||
6 set dial "ABORT BUSY ABORT NO\\sCARRIER TIMEOUT 5 \
|
||||
7 \"\" AT OK-AT-OK ATE1Q0 OK \\dATDT\\T TIMEOUT 40 CONNECT"
|
||||
8 set timeout 180
|
||||
9 enable dns
|
||||
10
|
||||
11 provider:
|
||||
12 set phone "(123) 456 7890"
|
||||
13 set authname foo
|
||||
14 set authkey bar
|
||||
15 set timeout 300
|
||||
16 set ifaddr x.x.x.x/0 y.y.y.y/0 255.255.255.255 0.0.0.0
|
||||
17 add default HISADDR
|
||||
....
|
||||
|
||||
Line 1:::
|
||||
Identifies the `default` entry. Commands in this entry (lines 2 through 9) are executed automatically when `ppp` is run.
|
||||
|
||||
Line 2:::
|
||||
Enables verbose logging parameters for testing the connection. Once the configuration is working satisfactorily, this line should be reduced to:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
set log phase tun
|
||||
....
|
||||
|
||||
Line 3:::
|
||||
Displays the version of man:ppp[8] to the PPP software running on the other side of the connection.
|
||||
|
||||
Line 4:::
|
||||
Identifies the device to which the modem is connected, where [.filename]#COM1# is [.filename]#/dev/cuau0# and [.filename]#COM2# is [.filename]#/dev/cuau1#.
|
||||
|
||||
Line 5:::
|
||||
Sets the connection speed. If `115200` does not work on an older modem, try `38400` instead.
|
||||
|
||||
Lines 6 & 7:::
|
||||
The dial string written as an expect-send syntax. Refer to man:chat[8] for more information.
|
||||
+
|
||||
Note that this command continues onto the next line for readability. Any command in [.filename]#ppp.conf# may do this if the last character on the line is `\`.
|
||||
|
||||
Line 8:::
|
||||
Sets the idle timeout for the link in seconds.
|
||||
|
||||
Line 9:::
|
||||
Instructs the peer to confirm the DNS settings. If the local network is running its own DNS server, this line should be commented out, by adding a `#` at the beginning of the line, or removed.
|
||||
|
||||
Line 10:::
|
||||
A blank line for readability. Blank lines are ignored by man:ppp[8].
|
||||
|
||||
Line 11:::
|
||||
Identifies an entry called `provider`. This could be changed to the name of the ISP so that `load _ISP_` can be used to start the connection.
|
||||
|
||||
Line 12:::
|
||||
Use the phone number for the ISP. Multiple phone numbers may be specified using the colon (`:`) or pipe character (`|`) as a separator. To rotate through the numbers, use a colon. To always attempt to dial the first number first and only use the other numbers if the first number fails, use the pipe character. Always enclose the entire set of phone numbers between quotation marks (`"`) to prevent dialing failures.
|
||||
|
||||
Lines 13 & 14:::
|
||||
Use the user name and password for the ISP.
|
||||
|
||||
Line 15:::
|
||||
Sets the default idle timeout in seconds for the connection. In this example, the connection will be closed automatically after 300 seconds of inactivity. To prevent a timeout, set this value to zero.
|
||||
|
||||
Line 16:::
|
||||
Sets the interface addresses. The values used depend upon whether a static IP address has been obtained from the ISP or if it instead negotiates a dynamic IP address during connection.
|
||||
+
|
||||
If the ISP has allocated a static IP address and default gateway, replace _x.x.x.x_ with the static IP address and replace _y.y.y.y_ with the IP address of the default gateway. If the ISP has only provided a static IP address without a gateway address, replace _y.y.y.y_ with `10.0.0.2/0`.
|
||||
+
|
||||
If the IP address changes whenever a connection is made, change this line to the following value. This tells man:ppp[8] to use the IP Configuration Protocol (IPCP) to negotiate a dynamic IP address:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
set ifaddr 10.0.0.1/0 10.0.0.2/0 255.255.255.255 0.0.0.0
|
||||
....
|
||||
|
||||
Line 17:::
|
||||
Keep this line as-is as it adds a default route to the gateway. The `HISADDR` will automatically be replaced with the gateway address specified on line 16. It is important that this line appears after line 16.
|
||||
|
||||
Depending upon whether man:ppp[8] is started manually or automatically, a [.filename]#/etc/ppp/ppp.linkup# may also need to be created which contains the following lines. This file is required when running `ppp` in `-auto` mode. This file is used after the connection has been established. At this point, the IP address will have been assigned and it is now be possible to add the routing table entries. When creating this file, make sure that _provider_ matches the value demonstrated in line 11 of [.filename]#ppp.conf#.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
provider:
|
||||
add default HISADDR
|
||||
....
|
||||
|
||||
This file is also needed when the default gateway address is "guessed" in a static IP address configuration. In this case, remove line 17 from [.filename]#ppp.conf# and create [.filename]#/etc/ppp/ppp.linkup# with the above two lines. More examples for this file can be found in [.filename]#/usr/shared/examples/ppp/#.
|
||||
|
||||
By default, `ppp` must be run as `root`. To change this default, add the account of the user who should run `ppp` to the `network` group in [.filename]#/etc/group#.
|
||||
|
||||
Then, give the user access to one or more entries in [.filename]#/etc/ppp/ppp.conf# with `allow`. For example, to give `fred` and `mary` permission to only the `provider:` entry, add this line to the `provider:` section:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
allow users fred mary
|
||||
....
|
||||
|
||||
To give the specified users access to all entries, put that line in the `default` section instead.
|
||||
|
||||
=== Advanced Configuration
|
||||
|
||||
It is possible to configure PPP to supply DNS and NetBIOS nameserver addresses on demand.
|
||||
|
||||
To enable these extensions with PPP version 1.x, the following lines might be added to the relevant section of [.filename]#/etc/ppp/ppp.conf#.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
enable msext
|
||||
set ns 203.14.100.1 203.14.100.2
|
||||
set nbns 203.14.100.5
|
||||
....
|
||||
|
||||
And for PPP version 2 and above:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
accept dns
|
||||
set dns 203.14.100.1 203.14.100.2
|
||||
set nbns 203.14.100.5
|
||||
....
|
||||
|
||||
This will tell the clients the primary and secondary name server addresses, and a NetBIOS nameserver host.
|
||||
|
||||
In version 2 and above, if the `set dns` line is omitted, PPP will use the values found in [.filename]#/etc/resolv.conf#.
|
||||
|
||||
[[userppp-PAPnCHAP]]
|
||||
==== PAP and CHAP Authentication
|
||||
|
||||
Some ISPs set their system up so that the authentication part of the connection is done using either of the PAP or CHAP authentication mechanisms. If this is the case, the ISP will not give a `login:` prompt at connection, but will start talking PPP immediately.
|
||||
|
||||
PAP is less secure than CHAP, but security is not normally an issue here as passwords, although being sent as plain text with PAP, are being transmitted down a serial line only. There is not much room for crackers to "eavesdrop".
|
||||
|
||||
The following alterations must be made:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
13 set authname MyUserName
|
||||
14 set authkey MyPassword
|
||||
15 set login
|
||||
....
|
||||
|
||||
Line 13:::
|
||||
This line specifies the PAP/CHAP user name. Insert the correct value for _MyUserName_.
|
||||
|
||||
Line 14:::
|
||||
This line specifies the PAP/CHAP password. Insert the correct value for _MyPassword_. You may want to add an additional line, such as:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
16 accept PAP
|
||||
....
|
||||
+
|
||||
or
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
16 accept CHAP
|
||||
....
|
||||
+
|
||||
to make it obvious that this is the intention, but PAP and CHAP are both accepted by default.
|
||||
|
||||
Line 15:::
|
||||
The ISP will not normally require a login to the server when using PAP or CHAP. Therefore, disable the "set login" string.
|
||||
|
||||
[[userppp-nat]]
|
||||
==== Using PPP Network Address Translation Capability
|
||||
|
||||
PPP has ability to use internal NAT without kernel diverting capabilities. This functionality may be enabled by the following line in [.filename]#/etc/ppp/ppp.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
nat enable yes
|
||||
....
|
||||
|
||||
Alternatively, NAT may be enabled by command-line option `-nat`. There is also [.filename]#/etc/rc.conf# knob named `ppp_nat`, which is enabled by default.
|
||||
|
||||
When using this feature, it may be useful to include the following [.filename]#/etc/ppp/ppp.conf# options to enable incoming connections forwarding:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
nat port tcp 10.0.0.2:ftp ftp
|
||||
nat port tcp 10.0.0.2:http http
|
||||
....
|
||||
|
||||
or do not trust the outside at all
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
nat deny_incoming yes
|
||||
....
|
||||
|
||||
[[userppp-final]]
|
||||
=== Final System Configuration
|
||||
|
||||
While `ppp` is now configured, some edits still need to be made to [.filename]#/etc/rc.conf#.
|
||||
|
||||
Working from the top down in this file, make sure the `hostname=` line is set:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
hostname="foo.example.com"
|
||||
....
|
||||
|
||||
If the ISP has supplied a static IP address and name, use this name as the host name.
|
||||
|
||||
Look for the `network_interfaces` variable. To configure the system to dial the ISP on demand, make sure the [.filename]#tun0# device is added to the list, otherwise remove it.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
network_interfaces="lo0 tun0"
|
||||
ifconfig_tun0=
|
||||
....
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The `ifconfig_tun0` variable should be empty, and a file called [.filename]#/etc/start_if.tun0# should be created. This file should contain the line:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
ppp -auto mysystem
|
||||
....
|
||||
|
||||
This script is executed at network configuration time, starting the ppp daemon in automatic mode. If this machine acts as a gateway, consider including `-alias`. Refer to the manual page for further details.
|
||||
====
|
||||
|
||||
Make sure that the router program is set to `NO` with the following line in [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
router_enable="NO"
|
||||
....
|
||||
|
||||
It is important that the `routed` daemon is not started, as `routed` tends to delete the default routing table entries created by `ppp`.
|
||||
|
||||
It is probably a good idea to ensure that the `sendmail_flags` line does not include the `-q` option, otherwise `sendmail` will attempt to do a network lookup every now and then, possibly causing your machine to dial out. You may try:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
sendmail_flags="-bd"
|
||||
....
|
||||
|
||||
The downside is that `sendmail` is forced to re-examine the mail queue whenever the ppp link. To automate this, include `!bg` in [.filename]#ppp.linkup#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
1 provider:
|
||||
2 delete ALL
|
||||
3 add 0 0 HISADDR
|
||||
4 !bg sendmail -bd -q30m
|
||||
....
|
||||
|
||||
An alternative is to set up a "dfilter" to block SMTP traffic. Refer to the sample files for further details.
|
||||
|
||||
=== Using `ppp`
|
||||
|
||||
All that is left is to reboot the machine. After rebooting, either type:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ppp
|
||||
....
|
||||
|
||||
and then `dial provider` to start the PPP session, or, to configure `ppp` to establish sessions automatically when there is outbound traffic and [.filename]#start_if.tun0# does not exist, type:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ppp -auto provider
|
||||
....
|
||||
|
||||
It is possible to talk to the `ppp` program while it is running in the background, but only if a suitable diagnostic port has been set up. To do this, add the following line to the configuration:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
set server /var/run/ppp-tun%d DiagnosticPassword 0177
|
||||
....
|
||||
|
||||
This will tell PPP to listen to the specified UNIX(R) domain socket, asking clients for the specified password before allowing access. The `%d` in the name is replaced with the [.filename]#tun# device number that is in use.
|
||||
|
||||
Once a socket has been set up, the man:pppctl[8] program may be used in scripts that wish to manipulate the running program.
|
||||
|
||||
[[userppp-mgetty]]
|
||||
=== Configuring Dial-in Services
|
||||
|
||||
crossref:serialcomms[dialup,“Dial-in Service”] provides a good description on enabling dial-up services using man:getty[8].
|
||||
|
||||
An alternative to `getty` is package:comms/mgetty+sendfax[] port), a smarter version of `getty` designed with dial-up lines in mind.
|
||||
|
||||
The advantages of using `mgetty` is that it actively _talks_ to modems, meaning if port is turned off in [.filename]#/etc/ttys# then the modem will not answer the phone.
|
||||
|
||||
Later versions of `mgetty` (from 0.99beta onwards) also support the automatic detection of PPP streams, allowing clients scriptless access to the server.
|
||||
|
||||
Refer to http://mgetty.greenie.net/doc/mgetty_toc.html[http://mgetty.greenie.net/doc/mgetty_toc.html] for more information on `mgetty`.
|
||||
|
||||
By default the package:comms/mgetty+sendfax[] port comes with the `AUTO_PPP` option enabled allowing `mgetty` to detect the LCP phase of PPP connections and automatically spawn off a ppp shell. However, since the default login/password sequence does not occur it is necessary to authenticate users using either PAP or CHAP.
|
||||
|
||||
This section assumes the user has successfully compiled, and installed the package:comms/mgetty+sendfax[] port on his system.
|
||||
|
||||
Ensure that [.filename]#/usr/local/etc/mgetty+sendfax/login.config# has the following:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
/AutoPPP/ - - /etc/ppp/ppp-pap-dialup
|
||||
....
|
||||
|
||||
This tells `mgetty` to run [.filename]#ppp-pap-dialup# for detected PPP connections.
|
||||
|
||||
Create an executable file called [.filename]#/etc/ppp/ppp-pap-dialup# containing the following:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#!/bin/sh
|
||||
exec /usr/sbin/ppp -direct pap$IDENT
|
||||
....
|
||||
|
||||
For each dial-up line enabled in [.filename]#/etc/ttys#, create a corresponding entry in [.filename]#/etc/ppp/ppp.conf#. This will happily co-exist with the definitions we created above.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
pap:
|
||||
enable pap
|
||||
set ifaddr 203.14.100.1 203.14.100.20-203.14.100.40
|
||||
enable proxy
|
||||
....
|
||||
|
||||
Each user logging in with this method will need to have a username/password in [.filename]#/etc/ppp/ppp.secret#, or alternatively add the following option to authenticate users via PAP from [.filename]#/etc/passwd#.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
enable passwdauth
|
||||
....
|
||||
|
||||
To assign some users a static IP number, specify the number as the third argument in [.filename]#/etc/ppp/ppp.secret#. See [.filename]#/usr/shared/examples/ppp/ppp.secret.sample# for examples.
|
||||
|
||||
[[ppp-troubleshoot]]
|
||||
== Troubleshooting PPP Connections
|
||||
|
||||
This section covers a few issues which may arise when using PPP over a modem connection. Some ISPs present the `ssword` prompt while others present `password`. If the `ppp` script is not written accordingly, the login attempt will fail. The most common way to debug `ppp` connections is by connecting manually as described in this section.
|
||||
|
||||
=== Check the Device Nodes
|
||||
|
||||
When using a custom kernel, make sure to include the following line in the kernel configuration file:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
device uart
|
||||
....
|
||||
|
||||
The [.filename]#uart# device is already included in the `GENERIC` kernel, so no additional steps are necessary in this case. Just check the `dmesg` output for the modem device with:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# dmesg | grep uart
|
||||
....
|
||||
|
||||
This should display some pertinent output about the [.filename]#uart# devices. These are the COM ports we need. If the modem acts like a standard serial port, it should be listed on [.filename]#uart1#, or [.filename]#COM2#. If so, a kernel rebuild is not required. When matching up, if the modem is on [.filename]#uart1#, the modem device would be [.filename]#/dev/cuau1#.
|
||||
|
||||
=== Connecting Manually
|
||||
|
||||
Connecting to the Internet by manually controlling `ppp` is quick, easy, and a great way to debug a connection or just get information on how the ISP treats `ppp` client connections. Lets start PPP from the command line. Note that in all of our examples we will use _example_ as the hostname of the machine running PPP. To start `ppp`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ppp
|
||||
....
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ppp ON example> set device /dev/cuau1
|
||||
....
|
||||
|
||||
This second command sets the modem device to [.filename]#cuau1#.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ppp ON example> set speed 115200
|
||||
....
|
||||
|
||||
This sets the connection speed to 115,200 kbps.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ppp ON example> enable dns
|
||||
....
|
||||
|
||||
This tells `ppp` to configure the resolver and add the nameserver lines to [.filename]#/etc/resolv.conf#. If `ppp` cannot determine the hostname, it can manually be set later.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ppp ON example> term
|
||||
....
|
||||
|
||||
This switches to "terminal" mode in order to manually control the modem.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
deflink: Entering terminal mode on /dev/cuau1
|
||||
type '~h' for help
|
||||
....
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
at
|
||||
OK
|
||||
atdt123456789
|
||||
....
|
||||
|
||||
Use `at` to initialize the modem, then use `atdt` and the number for the ISP to begin the dial in process.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
CONNECT
|
||||
....
|
||||
|
||||
Confirmation of the connection, if we are going to have any connection problems, unrelated to hardware, here is where we will attempt to resolve them.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ISP Login:myusername
|
||||
....
|
||||
|
||||
At this prompt, return the prompt with the username that was provided by the ISP.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ISP Pass:mypassword
|
||||
....
|
||||
|
||||
At this prompt, reply with the password that was provided by the ISP. Just like logging into FreeBSD, the password will not echo.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Shell or PPP:ppp
|
||||
....
|
||||
|
||||
Depending on the ISP, this prompt might not appear. If it does, it is asking whether to use a shell on the provider or to start `ppp`. In this example, `ppp` was selected in order to establish an Internet connection.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Ppp ON example>
|
||||
....
|
||||
|
||||
Notice that in this example the first `p` has been capitalized. This shows that we have successfully connected to the ISP.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
Ppp ON example>
|
||||
....
|
||||
|
||||
We have successfully authenticated with our ISP and are waiting for the assigned IP address.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
PPP ON example>
|
||||
....
|
||||
|
||||
We have made an agreement on an IP address and successfully completed our connection.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
PPP ON example>add default HISADDR
|
||||
....
|
||||
|
||||
Here we add our default route, we need to do this before we can talk to the outside world as currently the only established connection is with the peer. If this fails due to existing routes, put a bang character `!` in front of the `add`. Alternatively, set this before making the actual connection and it will negotiate a new route accordingly.
|
||||
|
||||
If everything went good we should now have an active connection to the Internet, which could be thrown into the background using kbd:[CTRL+z] If `PPP` returns to `ppp` then the connection has bee lost. This is good to know because it shows the connection status. Capital P's represent a connection to the ISP and lowercase p's show that the connection has been lost.
|
||||
|
||||
=== Debugging
|
||||
|
||||
If a connection cannot be established, turn hardware flow CTS/RTS to off using `set ctsrts off`. This is mainly the case when connected to some PPP-capable terminal servers, where PPP hangs when it tries to write data to the communication link, and waits for a Clear To Send (CTS) signal which may never come. When using this option, include `set accmap` as it may be required to defeat hardware dependent on passing certain characters from end to end, most of the time XON/XOFF. Refer to man:ppp[8] for more information on this option and how it is used.
|
||||
|
||||
An older modem may need `set parity even`. Parity is set at none be default, but is used for error checking with a large increase in traffic, on older modems.
|
||||
|
||||
PPP may not return to the command mode, which is usually a negotiation error where the ISP is waiting for negotiating to begin. At this point, using `~p` will force ppp to start sending the configuration information.
|
||||
|
||||
If a login prompt never appears, PAP or CHAP authentication is most likely required. To use PAP or CHAP, add the following options to PPP before going into terminal mode:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ppp ON example> set authname myusername
|
||||
....
|
||||
|
||||
Where _myusername_ should be replaced with the username that was assigned by the ISP.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
ppp ON example> set authkey mypassword
|
||||
....
|
||||
|
||||
Where _mypassword_ should be replaced with the password that was assigned by the ISP.
|
||||
|
||||
If a connection is established, but cannot seem to find any domain name, try to man:ping[8] an IP address. If there is 100 percent (100%) packet loss, it is likely that a default route was not assigned. Double check that `add default HISADDR` was set during the connection. If a connection can be made to a remote IP address, it is possible that a resolver address has not been added to [.filename]#/etc/resolv.conf#. This file should look like:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
domain example.com
|
||||
nameserver x.x.x.x
|
||||
nameserver y.y.y.y
|
||||
....
|
||||
|
||||
Where _x.x.x.x_ and _y.y.y.y_ should be replaced with the IP address of the ISP's DNS servers.
|
||||
|
||||
To configure man:syslog[3] to provide logging for the PPP connection, make sure this line exists in [.filename]#/etc/syslog.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
!ppp
|
||||
*.* /var/log/ppp.log
|
||||
....
|
||||
|
||||
[[pppoe]]
|
||||
== Using PPP over Ethernet (PPPoE)
|
||||
|
||||
This section describes how to set up PPP over Ethernet (PPPoE).
|
||||
|
||||
Here is an example of a working [.filename]#ppp.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
default:
|
||||
set log Phase tun command # you can add more detailed logging if you wish
|
||||
set ifaddr 10.0.0.1/0 10.0.0.2/0
|
||||
|
||||
name_of_service_provider:
|
||||
set device PPPoE:xl1 # replace xl1 with your Ethernet device
|
||||
set authname YOURLOGINNAME
|
||||
set authkey YOURPASSWORD
|
||||
set dial
|
||||
set login
|
||||
add default HISADDR
|
||||
....
|
||||
|
||||
As `root`, run:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ppp -ddial name_of_service_provider
|
||||
....
|
||||
|
||||
Add the following to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
ppp_enable="YES"
|
||||
ppp_mode="ddial"
|
||||
ppp_nat="YES" # if you want to enable nat for your local network, otherwise NO
|
||||
ppp_profile="name_of_service_provider"
|
||||
....
|
||||
|
||||
=== Using a PPPoE Service Tag
|
||||
|
||||
Sometimes it will be necessary to use a service tag to establish the connection. Service tags are used to distinguish between different PPPoE servers attached to a given network.
|
||||
|
||||
Any required service tag information should be in the documentation provided by the ISP.
|
||||
|
||||
As a last resort, one could try installing the package:net/rr-pppoe[] package or port. Bear in mind however, this may de-program your modem and render it useless, so think twice before doing it. Simply install the program shipped with the modem. Then, access the menu:System[] menu from the program. The name of the profile should be listed there. It is usually _ISP_.
|
||||
|
||||
The profile name (service tag) will be used in the PPPoE configuration entry in [.filename]#ppp.conf# as the provider part for `set device`. Refer to man:ppp[8] for full details. It should look like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
set device PPPoE:xl1:ISP
|
||||
....
|
||||
|
||||
Do not forget to change _xl1_ to the proper device for the Ethernet card.
|
||||
|
||||
Do not forget to change _ISP_ to the profile.
|
||||
|
||||
For additional information, refer to http://renaud.waldura.com/doc/freebsd/pppoe/[Cheaper Broadband with FreeBSD on DSL] by Renaud Waldura.
|
||||
|
||||
[[ppp-3com]]
|
||||
=== PPPoE with a 3Com(R) HomeConnect(TM) ADSL Modem Dual Link
|
||||
|
||||
This modem does not follow the PPPoE specification defined in http://www.faqs.org/rfcs/rfc2516.html[RFC 2516].
|
||||
|
||||
In order to make FreeBSD capable of communicating with this device, a sysctl must be set. This can be done automatically at boot time by updating [.filename]#/etc/sysctl.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
net.graph.nonstandard_pppoe=1
|
||||
....
|
||||
|
||||
or can be done immediately with the command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# sysctl net.graph.nonstandard_pppoe=1
|
||||
....
|
||||
|
||||
Unfortunately, because this is a system-wide setting, it is not possible to talk to a normal PPPoE client or server and a 3Com(R) HomeConnect(TM) ADSL Modem at the same time.
|
||||
|
||||
[[pppoa]]
|
||||
== Using PPP over ATM (PPPoA)
|
||||
|
||||
The following describes how to set up PPP over ATM (PPPoA). PPPoA is a popular choice among European DSL providers.
|
||||
|
||||
=== Using mpd
|
||||
|
||||
The mpd application can be used to connect to a variety of services, in particular PPTP services. It can be installed using the package:net/mpd5[] package or port. Many ADSL modems require that a PPTP tunnel is created between the modem and computer.
|
||||
|
||||
Once installed, configure mpd to suit the provider's settings. The port places a set of sample configuration files which are well documented in [.filename]#/usr/local/etc/mpd/#. A complete guide to configure mpd is available in HTML format in [.filename]#/usr/ports/shared/doc/mpd/#. Here is a sample configuration for connecting to an ADSL service with mpd. The configuration is spread over two files, first the [.filename]#mpd.conf#:
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
This example [.filename]#mpd.conf# only works with mpd 4.x.
|
||||
====
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
default:
|
||||
load adsl
|
||||
|
||||
adsl:
|
||||
new -i ng0 adsl adsl
|
||||
set bundle authname username <.>
|
||||
set bundle password password <.>
|
||||
set bundle disable multilink
|
||||
|
||||
set link no pap acfcomp protocomp
|
||||
set link disable chap
|
||||
set link accept chap
|
||||
set link keep-alive 30 10
|
||||
|
||||
set ipcp no vjcomp
|
||||
set ipcp ranges 0.0.0.0/0 0.0.0.0/0
|
||||
|
||||
set iface route default
|
||||
set iface disable on-demand
|
||||
set iface enable proxy-arp
|
||||
set iface idle 0
|
||||
|
||||
open
|
||||
....
|
||||
|
||||
<.> The username used to authenticate with your ISP.
|
||||
<.> The password used to authenticate with your ISP.
|
||||
|
||||
Information about the link, or links, to establish is found in [.filename]#mpd.links#. An example [.filename]#mpd.links# to accompany the above example is given beneath:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
adsl:
|
||||
set link type pptp
|
||||
set pptp mode active
|
||||
set pptp enable originate outcall
|
||||
set pptp self 10.0.0.1 <.>
|
||||
set pptp peer 10.0.0.138 <.>
|
||||
....
|
||||
|
||||
<.> The IP address of FreeBSD computer running mpd.
|
||||
<.> The IP address of the ADSL modem. The Alcatel SpeedTouch(TM) Home defaults to `10.0.0.138`.
|
||||
|
||||
It is possible to initialize the connection easily by issuing the following command as `root`:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mpd -b adsl
|
||||
....
|
||||
|
||||
To view the status of the connection:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% ifconfig ng0
|
||||
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> mtu 1500
|
||||
inet 216.136.204.117 --> 204.152.186.171 netmask 0xffffffff
|
||||
....
|
||||
|
||||
Using mpd is the recommended way to connect to an ADSL service with FreeBSD.
|
||||
|
||||
=== Using pptpclient
|
||||
|
||||
It is also possible to use FreeBSD to connect to other PPPoA services using package:net/pptpclient[].
|
||||
|
||||
To use package:net/pptpclient[] to connect to a DSL service, install the port or package, then edit [.filename]#/etc/ppp/ppp.conf#. An example section of [.filename]#ppp.conf# is given below. For further information on [.filename]#ppp.conf# options consult man:ppp[8].
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
adsl:
|
||||
set log phase chat lcp ipcp ccp tun command
|
||||
set timeout 0
|
||||
enable dns
|
||||
set authname username <.>
|
||||
set authkey password <.>
|
||||
set ifaddr 0 0
|
||||
add default HISADDR
|
||||
....
|
||||
|
||||
<.> The username for the DSL provider.
|
||||
<.> The password for your account.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
|
||||
Since the account's password is added to [.filename]#ppp.conf# in plain text form, make sure nobody can read the contents of this file:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chown root:wheel /etc/ppp/ppp.conf
|
||||
# chmod 600 /etc/ppp/ppp.conf
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
This will open a tunnel for a PPP session to the DSL router. Ethernet DSL modems have a preconfigured LAN IP address to connect to. In the case of the Alcatel SpeedTouch(TM) Home, this address is `10.0.0.138`. The router's documentation should list the address the device uses. To open the tunnel and start a PPP session:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# pptp address adsl
|
||||
....
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
If an ampersand ("&") is added to the end of this command, pptp will return the prompt.
|
||||
====
|
||||
|
||||
A [.filename]#tun# virtual tunnel device will be created for interaction between the pptp and ppp processes. Once the prompt is returned, or the pptp process has confirmed a connection, examine the tunnel:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% ifconfig tun0
|
||||
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
|
||||
inet 216.136.204.21 --> 204.152.186.171 netmask 0xffffff00
|
||||
Opened by PID 918
|
||||
....
|
||||
|
||||
If the connection fails, check the configuration of the router, which is usually accessible using a web browser. Also, examine the output of `pptp` and the contents of the log file, [.filename]#/var/log/ppp.log# for clues.
|
||||
246
documentation/content/en/books/handbook/preface/_index.adoc
Normal file
246
documentation/content/en/books/handbook/preface/_index.adoc
Normal file
|
|
@ -0,0 +1,246 @@
|
|||
---
|
||||
title: Preface
|
||||
prev: books/handbook/
|
||||
next: books/handbook/parti
|
||||
---
|
||||
|
||||
[preface]
|
||||
[[book-preface]]
|
||||
= Preface
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
|
||||
[[preface-audience]]
|
||||
== Intended Audience
|
||||
|
||||
The FreeBSD newcomer will find that the first section of this book guides the user through the FreeBSD installation process and gently introduces the concepts and conventions that underpin UNIX(R). Working through this section requires little more than the desire to explore, and the ability to take on board new concepts as they are introduced.
|
||||
|
||||
Once you have traveled this far, the second, far larger, section of the Handbook is a comprehensive reference to all manner of topics of interest to FreeBSD system administrators. Some of these chapters may recommend that you do some prior reading, and this is noted in the synopsis at the beginning of each chapter.
|
||||
|
||||
For a list of additional sources of information, please see crossref:bibliography[bibliography,Bibliography].
|
||||
|
||||
[[preface-changes-from3]]
|
||||
== Changes from the Third Edition
|
||||
|
||||
The current online version of the Handbook represents the cumulative effort of many hundreds of contributors over the past 10 years. The following are some of the significant changes since the two volume third edition was published in 2004:
|
||||
|
||||
* crossref:dtrace[dtrace,DTrace] has been added with information about the powerful DTrace performance analysis tool.
|
||||
* crossref:filesystems[filesystems,Other File Systems] has been added with information about non-native file systems in FreeBSD, such as ZFS from Sun(TM).
|
||||
* crossref:audit[audit,Security Event Auditing] has been added to cover the new auditing capabilities in FreeBSD and explain its use.
|
||||
* crossref:virtualization[virtualization,Virtualization] has been added with information about installing FreeBSD on virtualization software.
|
||||
* crossref:bsdinstall[bsdinstall,Installing FreeBSD] has been added to cover installation of FreeBSD using the new installation utility, bsdinstall.
|
||||
|
||||
[[preface-changes-from2]]
|
||||
== Changes from the Second Edition (2004)
|
||||
|
||||
The third edition was the culmination of over two years of work by the dedicated members of the FreeBSD Documentation Project. The printed edition grew to such a size that it was necessary to publish as two separate volumes. The following are the major changes in this new edition:
|
||||
|
||||
* crossref:config[config-tuning,Configuration and Tuning] has been expanded with new information about the ACPI power and resource management, the `cron` system utility, and more kernel tuning options.
|
||||
* crossref:security[security,Security] has been expanded with new information about virtual private networks (VPNs), file system access control lists (ACLs), and security advisories.
|
||||
* crossref:mac[mac,Mandatory Access Control] is a new chapter with this edition. It explains what MAC is and how this mechanism can be used to secure a FreeBSD system.
|
||||
* crossref:disks[disks,Storage] has been expanded with new information about USB storage devices, file system snapshots, file system quotas, file and network backed filesystems, and encrypted disk partitions.
|
||||
* A troubleshooting section has been added to crossref:ppp-and-slip[ppp-and-slip,PPP].
|
||||
* crossref:mail[mail,Electronic Mail] has been expanded with new information about using alternative transport agents, SMTP authentication, UUCP, fetchmail, procmail, and other advanced topics.
|
||||
* crossref:network-servers[network-servers,Network Servers] is all new with this edition. This chapter includes information about setting up the Apache HTTP Server, ftpd, and setting up a server for Microsoft(R) Windows(R) clients with Samba. Some sections from crossref:advanced-networking[advanced-networking,Advanced Networking] were moved here to improve the presentation.
|
||||
* crossref:advanced-networking[advanced-networking,Advanced Networking] has been expanded with new information about using Bluetooth(R) devices with FreeBSD, setting up wireless networks, and Asynchronous Transfer Mode (ATM) networking.
|
||||
* A glossary has been added to provide a central location for the definitions of technical terms used throughout the book.
|
||||
* A number of aesthetic improvements have been made to the tables and figures throughout the book.
|
||||
|
||||
[[preface-changes]]
|
||||
== Changes from the First Edition (2001)
|
||||
|
||||
The second edition was the culmination of over two years of work by the dedicated members of the FreeBSD Documentation Project. The following were the major changes in this edition:
|
||||
|
||||
* A complete Index has been added.
|
||||
* All ASCII figures have been replaced by graphical diagrams.
|
||||
* A standard synopsis has been added to each chapter to give a quick summary of what information the chapter contains, and what the reader is expected to know.
|
||||
* The content has been logically reorganized into three parts: "Getting Started", "System Administration", and "Appendices".
|
||||
* crossref:basics[basics,FreeBSD Basics] has been expanded to contain additional information about processes, daemons, and signals.
|
||||
* crossref:ports[ports,Installing Applications: Packages and Ports] has been expanded to contain additional information about binary package management.
|
||||
* crossref:x11[x11,The X Window System] has been completely rewritten with an emphasis on using modern desktop technologies such as KDE and GNOME on XFree86(TM) 4.X.
|
||||
* crossref:boot[boot,The FreeBSD Booting Process] has been expanded.
|
||||
* crossref:disks[disks,Storage] has been written from what used to be two separate chapters on "Disks" and "Backups". We feel that the topics are easier to comprehend when presented as a single chapter. A section on RAID (both hardware and software) has also been added.
|
||||
* crossref:serialcomms[serialcomms,Serial Communications] has been completely reorganized and updated for FreeBSD 4.X/5.X.
|
||||
* crossref:ppp-and-slip[ppp-and-slip,PPP] has been substantially updated.
|
||||
* Many new sections have been added to crossref:advanced-networking[advanced-networking,Advanced Networking].
|
||||
* crossref:mail[mail,Electronic Mail] has been expanded to include more information about configuring sendmail.
|
||||
* crossref:linuxemu[linuxemu,Linux® Binary Compatibility] has been expanded to include information about installing Oracle(R) and SAP(R) R/3(R).
|
||||
* The following new topics are covered in this second edition:
|
||||
|
||||
** crossref:config[config-tuning,Configuration and Tuning].
|
||||
** crossref:multimedia[multimedia,Multimedia].
|
||||
|
||||
[[preface-overview]]
|
||||
== Organization of This Book
|
||||
|
||||
This book is split into five logically distinct sections. The first section, _Getting Started_, covers the installation and basic usage of FreeBSD. It is expected that the reader will follow these chapters in sequence, possibly skipping chapters covering familiar topics. The second section, _Common Tasks_, covers some frequently used features of FreeBSD. This section, and all subsequent sections, can be read out of order. Each chapter begins with a succinct synopsis that describes what the chapter covers and what the reader is expected to already know. This is meant to allow the casual reader to skip around to find chapters of interest. The third section, _System Administration_, covers administration topics. The fourth section, _Network Communication_, covers networking and server topics. The fifth section contains appendices of reference information.
|
||||
|
||||
_crossref:introduction[introduction,Introduction]_::
|
||||
Introduces FreeBSD to a new user. It describes the history of the FreeBSD Project, its goals and development model.
|
||||
|
||||
_crossref:bsdinstall[bsdinstall,Installing FreeBSD]_::
|
||||
Walks a user through the entire installation process of FreeBSD 9._x_ and later using bsdinstall.
|
||||
|
||||
_crossref:basics[basics,FreeBSD Basics]_::
|
||||
Covers the basic commands and functionality of the FreeBSD operating system. If you are familiar with Linux(R) or another flavor of UNIX(R) then you can probably skip this chapter.
|
||||
|
||||
_crossref:ports[ports,Installing Applications: Packages and Ports]_::
|
||||
Covers the installation of third-party software with both FreeBSD's innovative "Ports Collection" and standard binary packages.
|
||||
|
||||
_crossref:x11[x11,The X Window System]_::
|
||||
Describes the X Window System in general and using X11 on FreeBSD in particular. Also describes common desktop environments such as KDE and GNOME.
|
||||
|
||||
_crossref:desktop[desktop,Desktop Applications]_::
|
||||
Lists some common desktop applications, such as web browsers and productivity suites, and describes how to install them on FreeBSD.
|
||||
|
||||
_crossref:multimedia[multimedia,Multimedia]_::
|
||||
Shows how to set up sound and video playback support for your system. Also describes some sample audio and video applications.
|
||||
|
||||
_crossref:kernelconfig[kernelconfig,Configuring the FreeBSD Kernel]_::
|
||||
Explains why you might need to configure a new kernel and provides detailed instructions for configuring, building, and installing a custom kernel.
|
||||
|
||||
_crossref:printing[printing,Printing]_::
|
||||
Describes managing printers on FreeBSD, including information about banner pages, printer accounting, and initial setup.
|
||||
|
||||
_crossref:linuxemu[linuxemu,Linux® Binary Compatibility]_::
|
||||
Describes the Linux(R) compatibility features of FreeBSD. Also provides detailed installation instructions for many popular Linux(R) applications such as Oracle(R) and Mathematica(R).
|
||||
|
||||
_crossref:config[config-tuning,Configuration and Tuning]_::
|
||||
Describes the parameters available for system administrators to tune a FreeBSD system for optimum performance. Also describes the various configuration files used in FreeBSD and where to find them.
|
||||
|
||||
_crossref:boot[boot,The FreeBSD Booting Process]_::
|
||||
Describes the FreeBSD boot process and explains how to control this process with configuration options.
|
||||
|
||||
_crossref:security[security,Security]_::
|
||||
Describes many different tools available to help keep your FreeBSD system secure, including Kerberos, IPsec and OpenSSH.
|
||||
|
||||
_crossref:jails[jails,Jails]_::
|
||||
Describes the jails framework, and the improvements of jails over the traditional chroot support of FreeBSD.
|
||||
|
||||
_crossref:mac[mac,Mandatory Access Control]_::
|
||||
Explains what Mandatory Access Control (MAC) is and how this mechanism can be used to secure a FreeBSD system.
|
||||
|
||||
_crossref:audit[audit,Security Event Auditing]_::
|
||||
Describes what FreeBSD Event Auditing is, how it can be installed, configured, and how audit trails can be inspected or monitored.
|
||||
|
||||
_crossref:disks[disks,Storage]_::
|
||||
Describes how to manage storage media and filesystems with FreeBSD. This includes physical disks, RAID arrays, optical and tape media, memory-backed disks, and network filesystems.
|
||||
|
||||
_crossref:geom[geom,GEOM: Modular Disk Transformation Framework]_::
|
||||
Describes what the GEOM framework in FreeBSD is and how to configure various supported RAID levels.
|
||||
|
||||
_crossref:filesystems[filesystems,Other File Systems]_::
|
||||
Examines support of non-native file systems in FreeBSD, like the Z File System from Sun(TM).
|
||||
|
||||
_crossref:virtualization[virtualization,Virtualization]_::
|
||||
Describes what virtualization systems offer, and how they can be used with FreeBSD.
|
||||
|
||||
_crossref:l10n[l10n,Localization - i18n/L10n Usage and Setup]_::
|
||||
Describes how to use FreeBSD in languages other than English. Covers both system and application level localization.
|
||||
|
||||
_crossref:cutting-edge[updating-upgrading,Updating and Upgrading FreeBSD]_::
|
||||
Explains the differences between FreeBSD-STABLE, FreeBSD-CURRENT, and FreeBSD releases. Describes which users would benefit from tracking a development system and outlines that process. Covers the methods users may take to update their system to the latest security release.
|
||||
|
||||
_crossref:dtrace[dtrace,DTrace]_::
|
||||
Describes how to configure and use the DTrace tool from Sun(TM) in FreeBSD. Dynamic tracing can help locate performance issues, by performing real time system analysis.
|
||||
|
||||
_crossref:serialcomms[serialcomms,Serial Communications]_::
|
||||
Explains how to connect terminals and modems to your FreeBSD system for both dial in and dial out connections.
|
||||
|
||||
_crossref:ppp-and-slip[ppp-and-slip,PPP]_::
|
||||
Describes how to use PPP to connect to remote systems with FreeBSD.
|
||||
|
||||
_crossref:mail[mail,Electronic Mail]_::
|
||||
Explains the different components of an email server and dives into simple configuration topics for the most popular mail server software: sendmail.
|
||||
|
||||
_crossref:network-servers[network-servers,Network Servers]_::
|
||||
Provides detailed instructions and example configuration files to set up your FreeBSD machine as a network filesystem server, domain name server, network information system server, or time synchronization server.
|
||||
|
||||
_crossref:firewalls[firewalls,Firewalls]_::
|
||||
Explains the philosophy behind software-based firewalls and provides detailed information about the configuration of the different firewalls available for FreeBSD.
|
||||
|
||||
_crossref:advanced-networking[advanced-networking,Advanced Networking]_::
|
||||
Describes many networking topics, including sharing an Internet connection with other computers on your LAN, advanced routing topics, wireless networking, Bluetooth(R), ATM, IPv6, and much more.
|
||||
|
||||
_crossref:mirrors[mirrors,Obtaining FreeBSD]_::
|
||||
Lists different sources for obtaining FreeBSD media on CDROM or DVD as well as different sites on the Internet that allow you to download and install FreeBSD.
|
||||
|
||||
_crossref:bibliography[bibliography,Bibliography]_::
|
||||
This book touches on many different subjects that may leave you hungry for a more detailed explanation. The bibliography lists many excellent books that are referenced in the text.
|
||||
|
||||
_crossref:eresources[eresources,Resources on the Internet]_::
|
||||
Describes the many forums available for FreeBSD users to post questions and engage in technical conversations about FreeBSD.
|
||||
|
||||
_crossref:pgpkeys[pgpkeys,OpenPGP Keys]_::
|
||||
Lists the PGP fingerprints of several FreeBSD Developers.
|
||||
|
||||
[[preface-conv]]
|
||||
== Conventions used in this book
|
||||
|
||||
To provide a consistent and easy to read text, several conventions are followed throughout the book.
|
||||
|
||||
[[preface-conv-typographic]]
|
||||
=== Typographic Conventions
|
||||
|
||||
_Italic_::
|
||||
An _italic_ font is used for filenames, URLs, emphasized text, and the first usage of technical terms.
|
||||
|
||||
`Monospace`::
|
||||
A `monospaced` font is used for error messages, commands, environment variables, names of ports, hostnames, user names, group names, device names, variables, and code fragments.
|
||||
|
||||
Bold::
|
||||
A *bold* font is used for applications, commands, and keys.
|
||||
|
||||
[[preface-conv-commands]]
|
||||
=== User Input
|
||||
|
||||
Keys are shown in *bold* to stand out from other text. Key combinations that are meant to be typed simultaneously are shown with `+` between the keys, such as:
|
||||
|
||||
kbd:[Ctrl+Alt+Del]
|
||||
|
||||
Meaning the user should type the kbd:[Ctrl], kbd:[Alt], and kbd:[Del] keys at the same time.
|
||||
|
||||
Keys that are meant to be typed in sequence will be separated with commas, for example:
|
||||
|
||||
kbd:[Ctrl+X], kbd:[Ctrl+S]
|
||||
|
||||
Would mean that the user is expected to type the kbd:[Ctrl] and kbd:[X] keys simultaneously and then to type the kbd:[Ctrl] and kbd:[S] keys simultaneously.
|
||||
|
||||
[[preface-conv-examples]]
|
||||
=== Examples
|
||||
|
||||
Examples starting with [.filename]#C:\># indicate a MS-DOS(R) command. Unless otherwise noted, these commands may be executed from a "Command Prompt" window in a modern Microsoft(R) Windows(R) environment.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
E:\> tools\fdimage floppies\kern.flp A:
|
||||
....
|
||||
|
||||
Examples starting with # indicate a command that must be invoked as the superuser in FreeBSD. You can login as `root` to type the command, or login as your normal account and use man:su[1] to gain superuser privileges.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# dd if=kern.flp of=/dev/fd0
|
||||
....
|
||||
|
||||
Examples starting with % indicate a command that should be invoked from a normal user account. Unless otherwise noted, C-shell syntax is used for setting environment variables and other shell commands.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% top
|
||||
....
|
||||
|
||||
[[preface-acknowledgements]]
|
||||
== Acknowledgments
|
||||
|
||||
The book you are holding represents the efforts of many hundreds of people around the world. Whether they sent in fixes for typos, or submitted complete chapters, all the contributions have been useful.
|
||||
|
||||
Several companies have supported the development of this document by paying authors to work on it full-time, paying for publication, etc. In particular, BSDi (subsequently acquired by http://www.windriver.com[Wind River Systems]) paid members of the FreeBSD Documentation Project to work on improving this book full time leading up to the publication of the first printed edition in March 2000 (ISBN 1-57176-241-8). Wind River Systems then paid several additional authors to make a number of improvements to the print-output infrastructure and to add additional chapters to the text. This work culminated in the publication of the second printed edition in November 2001 (ISBN 1-57176-303-1). In 2003-2004, http://www.freebsdmall.com[FreeBSD Mall, Inc], paid several contributors to improve the Handbook in preparation for the third printed edition.
|
||||
768
documentation/content/en/books/handbook/printing/_index.adoc
Normal file
768
documentation/content/en/books/handbook/printing/_index.adoc
Normal file
|
|
@ -0,0 +1,768 @@
|
|||
---
|
||||
title: Chapter 9. Printing
|
||||
part: Part II. Common Tasks
|
||||
prev: books/handbook/kernelconfig
|
||||
next: books/handbook/linuxemu
|
||||
---
|
||||
|
||||
[[printing]]
|
||||
= Printing
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 9
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/printing/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/printing/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/printing/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
Putting information on paper is a vital function, despite many attempts to eliminate it. Printing has two basic components. The data must be delivered to the printer, and must be in a form that the printer can understand.
|
||||
|
||||
[[printing-quick-start]]
|
||||
== Quick Start
|
||||
|
||||
Basic printing can be set up quickly. The printer must be capable of printing plain `ASCII` text. For printing to other types of files, see <<printing-lpd-filters>>.
|
||||
|
||||
[.procedure]
|
||||
****
|
||||
. Create a directory to store files while they are being printed:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# mkdir -p /var/spool/lpd/lp
|
||||
# chown daemon:daemon /var/spool/lpd/lp
|
||||
# chmod 770 /var/spool/lpd/lp
|
||||
....
|
||||
+
|
||||
. As `root`, create [.filename]#/etc/printcap# with these contents:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
lp:\
|
||||
:lp=/dev/unlpt0:\ <.>
|
||||
:sh:\
|
||||
:mx#0:\
|
||||
:sd=/var/spool/lpd/lp:\
|
||||
:lf=/var/log/lpd-errs:
|
||||
....
|
||||
+
|
||||
<.> This line is for a printer connected to a `USB` port.
|
||||
+
|
||||
For a printer connected to a parallel or "printer" port, use:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
:lp=/dev/lpt0:\
|
||||
....
|
||||
+
|
||||
For a printer connected directly to a network, use:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
:lp=:rm=network-printer-name:rp=raw:\
|
||||
....
|
||||
+
|
||||
Replace _network-printer-name_ with the `DNS` host name of the network printer.
|
||||
+
|
||||
. Enable LPD by editing [.filename]#/etc/rc.conf#, adding this line:
|
||||
+
|
||||
[.programlisting]
|
||||
....
|
||||
lpd_enable="YES"
|
||||
....
|
||||
+
|
||||
Start the service:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# service lpd start
|
||||
Starting lpd.
|
||||
....
|
||||
+
|
||||
. Print a test:
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
# printf "1. This printer can print.\n2. This is the second line.\n" | lpr
|
||||
....
|
||||
+
|
||||
[TIP]
|
||||
====
|
||||
|
||||
If both lines do not start at the left border, but "stairstep" instead, see <<printing-lpd-filters-stairstep>>.
|
||||
====
|
||||
+
|
||||
Text files can now be printed with `lpr`. Give the filename on the command line, or pipe output directly into `lpr`.
|
||||
+
|
||||
[source,bash]
|
||||
....
|
||||
% lpr textfile.txt
|
||||
% ls -lh | lpr
|
||||
....
|
||||
****
|
||||
|
||||
[[printing-connections]]
|
||||
== Printer Connections
|
||||
|
||||
Printers are connected to computer systems in a variety of ways. Small desktop printers are usually connected directly to a computer's `USB` port. Older printers are connected to a parallel or "printer" port. Some printers are directly connected to a network, making it easy for multiple computers to share them. A few printers use a rare serial port connection.
|
||||
|
||||
FreeBSD can communicate with all of these types of printers.
|
||||
|
||||
[[printing-connections-usb]]
|
||||
`USB`::
|
||||
`USB` printers can be connected to any available `USB` port on the computer.
|
||||
+
|
||||
When FreeBSD detects a `USB` printer, two device entries are created: [.filename]#/dev/ulpt0# and [.filename]#/dev/unlpt0#. Data sent to either device will be relayed to the printer. After each print job, [.filename]#ulpt0# resets the `USB` port. Resetting the port can cause problems with some printers, so the [.filename]#unlpt0# device is usually used instead. [.filename]#unlpt0# does not reset the USB port at all.
|
||||
[[printing-connections-parallel]]
|
||||
Parallel (`IEEE`-1284)::
|
||||
The parallel port device is [.filename]#/dev/lpt0#. This device appears whether a printer is attached or not, it is not autodetected.
|
||||
+
|
||||
Vendors have largely moved away from these "legacy" ports, and many computers no longer have them. Adapters can be used to connect a parallel printer to a `USB` port. With such an adapter, the printer can be treated as if it were actually a `USB` printer. Devices called _print servers_ can also be used to connect parallel printers directly to a network.
|
||||
[[printing-connections-serial]]
|
||||
Serial (RS-232)::
|
||||
Serial ports are another legacy port, rarely used for printers except in certain niche applications. Cables, connectors, and required wiring vary widely.
|
||||
+
|
||||
For serial ports built into a motherboard, the serial device name is [.filename]#/dev/cuau0# or [.filename]#/dev/cuau1#. Serial `USB` adapters can also be used, and these will appear as [.filename]#/dev/cuaU0#.
|
||||
+
|
||||
Several communication parameters must be known to communicate with a serial printer. The most important are _baud rate_ or `BPS` (Bits Per Second) and _parity_. Values vary, but typical serial printers use a baud rate of 9600 and no parity.
|
||||
[[printing-connections-network]]
|
||||
Network::
|
||||
Network printers are connected directly to the local computer network.
|
||||
+
|
||||
The `DNS` hostname of the printer must be known. If the printer is assigned a dynamic address by `DHCP`, `DNS` should be dynamically updated so that the host name always has the correct `IP` address. Network printers are often given static `IP` addresses to avoid this problem.
|
||||
+
|
||||
Most network printers understand print jobs sent with the LPD protocol. A print queue name can also be specified. Some printers process data differently depending on which queue is used. For example, a `raw` queue prints the data unchanged, while the `text` queue adds carriage returns to plain text.
|
||||
+
|
||||
Many network printers can also print data sent directly to port 9100.
|
||||
|
||||
[[printing-connections-summary]]
|
||||
=== Summary
|
||||
|
||||
Wired network connections are usually the easiest to set up and give the fastest printing. For direct connection to the computer, `USB` is preferred for speed and simplicity. Parallel connections work but have limitations on cable length and speed. Serial connections are more difficult to configure. Cable wiring differs between models, and communication parameters like baud rate and parity bits must add to the complexity. Fortunately, serial printers are rare.
|
||||
|
||||
[[printing-pdls]]
|
||||
== Common Page Description Languages
|
||||
|
||||
Data sent to a printer must be in a language that the printer can understand. These languages are called Page Description Languages, or PDLs.
|
||||
[[print-pdls-ascii]]
|
||||
`ASCII`::
|
||||
Plain `ASCII` text is the simplest way to send data to a printer. Characters correspond one to one with what will be printed: an `A` in the data prints an `A` on the page. Very little formatting is available. There is no way to select a font or proportional spacing. The forced simplicity of plain `ASCII` means that text can be printed straight from the computer with little or no encoding or translation. The printed output corresponds directly with what was sent.
|
||||
+
|
||||
Some inexpensive printers cannot print plain `ASCII` text. This makes them more difficult to set up, but it is usually still possible.
|
||||
[[print-pdls-postscript]]
|
||||
PostScript(R)::
|
||||
PostScript(R) is almost the opposite of `ASCII`. Rather than simple text, a PostScript(R) program is a set of instructions that draw the final document. Different fonts and graphics can be used. However, this power comes at a price. The program that draws the page must be written. Usually this program is generated by application software, so the process is invisible to the user.
|
||||
+
|
||||
Inexpensive printers sometimes leave out PostScript(R) compatibility as a cost-saving measure.
|
||||
|
||||
[[print-pdls-pcl]]
|
||||
`PCL` (Printer Command Language)::
|
||||
`PCL` is an extension of `ASCII`, adding escape sequences for formatting, font selection, and printing graphics. Many printers provide `PCL5` support. Some support the newer `PCL6` or `PCLXL`. These later versions are supersets of `PCL5` and can provide faster printing.
|
||||
[[print-pdls-host-based]]
|
||||
Host-Based::
|
||||
Manufacturers can reduce the cost of a printer by giving it a simple processor and very little memory. These printers are not capable of printing plain text. Instead, bitmaps of text and graphics are drawn by a driver on the host computer and then sent to the printer. These are called _host-based_ printers.
|
||||
+
|
||||
Communication between the driver and a host-based printer is often through proprietary or undocumented protocols, making them functional only on the most common operating systems.
|
||||
|
||||
[[print-pdls-table]]
|
||||
=== Converting PostScript(R) to Other PDLs
|
||||
|
||||
Many applications from the Ports Collection and FreeBSD utilities produce PostScript(R) output. This table shows the utilities available to convert that into other common PDLs:
|
||||
|
||||
[[print-pdls-ps-to-other-tbl]]
|
||||
.Output PDLs
|
||||
[cols="1,1,1", frame="none", options="header"]
|
||||
|===
|
||||
<| Output PDL
|
||||
<| Generated By
|
||||
<| Notes
|
||||
|
||||
|`PCL` or `PCL5`
|
||||
|package:print/ghostscript9-base[]
|
||||
|`-sDEVICE=ljet4` for monochrome, `-sDEVICE=cljet5` for color
|
||||
|
||||
|`PCLXL` or `PCL6`
|
||||
|package:print/ghostscript9-base[]
|
||||
|`-sDEVICE=pxlmono` for monochrome, `-sDEVICE=pxlcolor` for color
|
||||
|
||||
|`ESC/P2`
|
||||
|package:print/ghostscript9-base[]
|
||||
|`-sDEVICE=uniprint`
|
||||
|
||||
|`XQX`
|
||||
|package:print/foo2zjs[]
|
||||
|
|
||||
|===
|
||||
|
||||
[[print-pdls-summary]]
|
||||
=== Summary
|
||||
|
||||
For the easiest printing, choose a printer that supports PostScript(R). Printers that support `PCL` are the next preferred. With package:print/ghostscript9-base[], these printers can be used as if they understood PostScript(R) natively. Printers that support PostScript(R) or `PCL` directly almost always support direct printing of plain `ASCII` text files also.
|
||||
|
||||
Line-based printers like typical inkjets usually do not support PostScript(R) or `PCL`. They often can print plain `ASCII` text files. package:print/ghostscript9-base[] supports the PDLs used by some of these printers. However, printing an entire graphic-based page on these printers is often very slow due to the large amount of data to be transferred and printed.
|
||||
|
||||
Host-based printers are often more difficult to set up. Some cannot be used at all because of proprietary PDLs. Avoid these printers when possible.
|
||||
|
||||
Descriptions of many PDLs can be found at http://www.undocprint.org/formats/page_description_languages[]. The particular `PDL` used by various models of printers can be found at http://www.openprinting.org/printers[].
|
||||
|
||||
[[printing-direct]]
|
||||
== Direct Printing
|
||||
|
||||
For occasional printing, files can be sent directly to a printer device without any setup. For example, a file called [.filename]#sample.txt# can be sent to a `USB` printer:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cp sample.txt /dev/unlpt0
|
||||
....
|
||||
|
||||
Direct printing to network printers depends on the abilities of the printer, but most accept print jobs on port 9100, and man:nc[1] can be used with them. To print the same file to a printer with the `DNS` hostname of _netlaser_:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# nc netlaser 9100 < sample.txt
|
||||
....
|
||||
|
||||
[[printing-lpd]]
|
||||
== LPD (Line Printer Daemon)
|
||||
|
||||
Printing a file in the background is called _spooling_. A spooler allows the user to continue with other programs on the computer without waiting for the printer to slowly complete the print job.
|
||||
|
||||
FreeBSD includes a spooler called man:lpd[8]. Print jobs are submitted with man:lpr[1].
|
||||
|
||||
[[printing-lpd-setup]]
|
||||
=== Initial Setup
|
||||
|
||||
A directory for storing print jobs is created, ownership is set, and the permissions are set to prevent other users from viewing the contents of those files:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# mkdir -p /var/spool/lpd/lp
|
||||
# chown daemon:daemon /var/spool/lpd/lp
|
||||
# chmod 770 /var/spool/lpd/lp
|
||||
....
|
||||
|
||||
Printers are defined in [.filename]#/etc/printcap#. An entry for each printer includes details like a name, the port where it is attached, and various other settings. Create [.filename]#/etc/printcap# with these contents:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lp:\ <.>
|
||||
:lp=/dev/unlpt0:\ <.>
|
||||
:sh:\ <.>
|
||||
:mx#0:\ <.>
|
||||
:sd=/var/spool/lpd/lp:\ <.>
|
||||
:lf=/var/log/lpd-errs: <.>
|
||||
....
|
||||
|
||||
<.> The name of this printer. man:lpr[1] sends print jobs to the `lp` printer unless another printer is specified with `-P`, so the default printer should be named `lp`.
|
||||
|
||||
<.> The device where the printer is connected. Replace this line with the appropriate one for the connection type shown here.
|
||||
|
||||
<.> Suppress the printing of a header page at the start of a print job.
|
||||
|
||||
<.> Do not limit the maximum size of a print job.
|
||||
|
||||
<.> The path to the spooling directory for this printer. Each printer uses its own spooling directory.
|
||||
|
||||
<.> The log file where errors on this printer will be reported.
|
||||
|
||||
After creating [.filename]#/etc/printcap#, use man:chkprintcap[8] to test it for errors:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chkprintcap
|
||||
....
|
||||
|
||||
Fix any reported problems before continuing.
|
||||
|
||||
Enable man:lpd[8] in [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lpd_enable="YES"
|
||||
....
|
||||
|
||||
Start the service:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service lpd start
|
||||
....
|
||||
|
||||
[[printing-lpd-lpr]]
|
||||
=== Printing with man:lpr[1]
|
||||
|
||||
Documents are sent to the printer with `lpr`. A file to be printed can be named on the command line or piped into `lpr`. These two commands are equivalent, sending the contents of [.filename]#doc.txt# to the default printer:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpr doc.txt
|
||||
% cat doc.txt | lpr
|
||||
....
|
||||
|
||||
Printers can be selected with `-P`. To print to a printer called _laser_:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpr -Plaser doc.txt
|
||||
....
|
||||
|
||||
[[printing-lpd-filters]]
|
||||
=== Filters
|
||||
|
||||
The examples shown so far have sent the contents of a text file directly to the printer. As long as the printer understands the content of those files, output will be printed correctly.
|
||||
|
||||
Some printers are not capable of printing plain text, and the input file might not even be plain text.
|
||||
|
||||
_Filters_ allow files to be translated or processed. The typical use is to translate one type of input, like plain text, into a form that the printer can understand, like PostScript(R) or `PCL`. Filters can also be used to provide additional features, like adding page numbers or highlighting source code to make it easier to read.
|
||||
|
||||
The filters discussed here are _input filters_ or _text filters_. These filters convert the incoming file into different forms. Use man:su[1] to become `root` before creating the files.
|
||||
|
||||
Filters are specified in [.filename]#/etc/printcap# with the `if=` identifier. To use [.filename]#/usr/local/libexec/lf2crlf# as a filter, modify [.filename]#/etc/printcap# like this:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lp:\
|
||||
:lp=/dev/unlpt0:\
|
||||
:sh:\
|
||||
:mx#0:\
|
||||
:sd=/var/spool/lpd/lp:\
|
||||
:if=/usr/local/libexec/lf2crlf:\ <.>
|
||||
:lf=/var/log/lpd-errs:
|
||||
....
|
||||
|
||||
<.> `if=` identifies the _input filter_ that will be used on incoming text.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
|
||||
The backslash _line continuation_ characters at the end of the lines in [.filename]#printcap# entries reveal that an entry for a printer is really just one long line with entries delimited by colon characters. An earlier example can be rewritten as a single less-readable line:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lp:lp=/dev/unlpt0:sh:mx#0:sd=/var/spool/lpd/lp:if=/usr/local/libexec/lf2crlf:lf=/var/log/lpd-errs:
|
||||
....
|
||||
|
||||
====
|
||||
|
||||
[[printing-lpd-filters-stairstep]]
|
||||
==== Preventing Stairstepping on Plain Text Printers
|
||||
|
||||
Typical FreeBSD text files contain only a single line feed character at the end of each line. These lines will "stairstep" on a standard printer:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
A printed file looks
|
||||
like the steps of a staircase
|
||||
scattered by the wind
|
||||
....
|
||||
|
||||
A filter can convert the newline characters into carriage returns and newlines. The carriage returns make the printer return to the left after each line. Create [.filename]#/usr/local/libexec/lf2crlf# with these contents:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#!/bin/sh
|
||||
CR=$'\r'
|
||||
/usr/bin/sed -e "s/$/${CR}/g"
|
||||
....
|
||||
|
||||
Set the permissions and make it executable:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chmod 555 /usr/local/libexec/lf2crlf
|
||||
....
|
||||
|
||||
Modify [.filename]#/etc/printcap# to use the new filter:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:if=/usr/local/libexec/lf2crlf:\
|
||||
....
|
||||
|
||||
Test the filter by printing the same plain text file. The carriage returns will cause each line to start at the left side of the page.
|
||||
|
||||
[[printing-lpd-filters-enscript]]
|
||||
==== Fancy Plain Text on PostScript(R) Printers with package:print/enscript[]
|
||||
|
||||
GNUEnscript converts plain text files into nicely-formatted PostScript(R) for printing on PostScript(R) printers. It adds page numbers, wraps long lines, and provides numerous other features to make printed text files easier to read. Depending on the local paper size, install either package:print/enscript-letter[] or package:print/enscript-a4[] from the Ports Collection.
|
||||
|
||||
Create [.filename]#/usr/local/libexec/enscript# with these contents:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#!/bin/sh
|
||||
/usr/local/bin/enscript -o -
|
||||
....
|
||||
|
||||
Set the permissions and make it executable:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chmod 555 /usr/local/libexec/enscript
|
||||
....
|
||||
|
||||
Modify [.filename]#/etc/printcap# to use the new filter:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:if=/usr/local/libexec/enscript:\
|
||||
....
|
||||
|
||||
Test the filter by printing a plain text file.
|
||||
|
||||
[[printing-lpd-filters-ps2pcl]]
|
||||
==== Printing PostScript(R) to `PCL` Printers
|
||||
|
||||
Many programs produce PostScript(R) documents. However, inexpensive printers often only understand plain text or `PCL`. This filter converts PostScript(R) files to `PCL` before sending them to the printer.
|
||||
|
||||
Install the Ghostscript PostScript(R) interpreter, package:print/ghostscript9-base[], from the Ports Collection.
|
||||
|
||||
Create [.filename]#/usr/local/libexec/ps2pcl# with these contents:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#!/bin/sh
|
||||
/usr/local/bin/gs -dSAFER -dNOPAUSE -dBATCH -q -sDEVICE=ljet4 -sOutputFile=- -
|
||||
....
|
||||
|
||||
Set the permissions and make it executable:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chmod 555 /usr/local/libexec/ps2pcl
|
||||
....
|
||||
|
||||
PostScript(R) input sent to this script will be rendered and converted to `PCL` before being sent on to the printer.
|
||||
|
||||
Modify [.filename]#/etc/printcap# to use this new input filter:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:if=/usr/local/libexec/ps2pcl:\
|
||||
....
|
||||
|
||||
Test the filter by sending a small PostScript(R) program to it:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% printf "%%\!PS \n /Helvetica findfont 18 scalefont setfont \
|
||||
72 432 moveto (PostScript printing successful.) show showpage \004" | lpr
|
||||
....
|
||||
|
||||
[[printing-lpd-filters-smart]]
|
||||
==== Smart Filters
|
||||
|
||||
A filter that detects the type of input and automatically converts it to the correct format for the printer can be very convenient. The first two characters of a PostScript(R) file are usually `%!`. A filter can detect those two characters. PostScript(R) files can be sent on to a PostScript(R) printer unchanged. Text files can be converted to PostScript(R) with Enscript as shown earlier. Create [.filename]#/usr/local/libexec/psif# with these contents:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
#!/bin/sh
|
||||
#
|
||||
# psif - Print PostScript or plain text on a PostScript printer
|
||||
#
|
||||
IFS="" read -r first_line
|
||||
first_two_chars=`expr "$first_line" : '\(..\)'`
|
||||
|
||||
case "$first_two_chars" in
|
||||
%!)
|
||||
# %! : PostScript job, print it.
|
||||
echo "$first_line" && cat && exit 0
|
||||
exit 2
|
||||
;;
|
||||
*)
|
||||
# otherwise, format with enscript
|
||||
( echo "$first_line"; cat ) | /usr/local/bin/enscript -o - && exit 0
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
....
|
||||
|
||||
Set the permissions and make it executable:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# chmod 555 /usr/local/libexec/psif
|
||||
....
|
||||
|
||||
Modify [.filename]#/etc/printcap# to use this new input filter:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
:if=/usr/local/libexec/psif:\
|
||||
....
|
||||
|
||||
Test the filter by printing PostScript(R) and plain text files.
|
||||
|
||||
[[printing-lpd-filters-othersmart]]
|
||||
==== Other Smart Filters
|
||||
|
||||
Writing a filter that detects many different types of input and formats them correctly is challenging. package:print/apsfilter[] from the Ports Collection is a smart "magic" filter that detects dozens of file types and automatically converts them to the `PDL` understood by the printer. See http://www.apsfilter.org[] for more details.
|
||||
|
||||
[[printing-lpd-queues]]
|
||||
=== Multiple Queues
|
||||
|
||||
The entries in [.filename]#/etc/printcap# are really definitions of _queues_. There can be more than one queue for a single printer. When combined with filters, multiple queues provide users more control over how their jobs are printed.
|
||||
|
||||
As an example, consider a networked PostScript(R) laser printer in an office. Most users want to print plain text, but a few advanced users want to be able to print PostScript(R) files directly. Two entries can be created for the same printer in [.filename]#/etc/printcap#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
textprinter:\
|
||||
:lp=9100@officelaser:\
|
||||
:sh:\
|
||||
:mx#0:\
|
||||
:sd=/var/spool/lpd/textprinter:\
|
||||
:if=/usr/local/libexec/enscript:\
|
||||
:lf=/var/log/lpd-errs:
|
||||
|
||||
psprinter:\
|
||||
:lp=9100@officelaser:\
|
||||
:sh:\
|
||||
:mx#0:\
|
||||
:sd=/var/spool/lpd/psprinter:\
|
||||
:lf=/var/log/lpd-errs:
|
||||
....
|
||||
|
||||
Documents sent to `textprinter` will be formatted by the [.filename]#/usr/local/libexec/enscript# filter shown in an earlier example. Advanced users can print PostScript(R) files on `psprinter`, where no filtering is done.
|
||||
|
||||
This multiple queue technique can be used to provide direct access to all kinds of printer features. A printer with a duplexer could use two queues, one for ordinary single-sided printing, and one with a filter that sends the command sequence to enable double-sided printing and then sends the incoming file.
|
||||
|
||||
[[printing-lpd-monitor]]
|
||||
=== Monitoring and Controlling Printing
|
||||
|
||||
Several utilities are available to monitor print jobs and check and control printer operation.
|
||||
|
||||
[[printing-lpd-monitor-lpq]]
|
||||
==== man:lpq[1]
|
||||
|
||||
man:lpq[1] shows the status of a user's print jobs. Print jobs from other users are not shown.
|
||||
|
||||
Show the current user's pending jobs on a single printer:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpq -Plp
|
||||
Rank Owner Job Files Total Size
|
||||
1st jsmith 0 (standard input) 12792 bytes
|
||||
....
|
||||
|
||||
Show the current user's pending jobs on all printers:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpq -a
|
||||
lp:
|
||||
Rank Owner Job Files Total Size
|
||||
1st jsmith 1 (standard input) 27320 bytes
|
||||
|
||||
laser:
|
||||
Rank Owner Job Files Total Size
|
||||
1st jsmith 287 (standard input) 22443 bytes
|
||||
....
|
||||
|
||||
[[printing-lpd-monitor-lprm]]
|
||||
==== man:lprm[1]
|
||||
|
||||
man:lprm[1] is used to remove print jobs. Normal users are only allowed to remove their own jobs. `root` can remove any or all jobs.
|
||||
|
||||
Remove all pending jobs from a printer:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# lprm -Plp -
|
||||
dfA002smithy dequeued
|
||||
cfA002smithy dequeued
|
||||
dfA003smithy dequeued
|
||||
cfA003smithy dequeued
|
||||
dfA004smithy dequeued
|
||||
cfA004smithy dequeued
|
||||
....
|
||||
|
||||
Remove a single job from a printer. man:lpq[1] is used to find the job number.
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpq
|
||||
Rank Owner Job Files Total Size
|
||||
1st jsmith 5 (standard input) 12188 bytes
|
||||
|
||||
% lprm -Plp 5
|
||||
dfA005smithy dequeued
|
||||
cfA005smithy dequeued
|
||||
....
|
||||
|
||||
[[printing-lpd-monitor-lpc]]
|
||||
==== man:lpc[8]
|
||||
|
||||
man:lpc[8] is used to check and modify printer status. `lpc` is followed by a command and an optional printer name. `all` can be used instead of a specific printer name, and the command will be applied to all printers. Normal users can view status with man:lpc[8]. Only `root` can use commands which modify printer status.
|
||||
|
||||
Show the status of all printers:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpc status all
|
||||
lp:
|
||||
queuing is enabled
|
||||
printing is enabled
|
||||
1 entry in spool area
|
||||
printer idle
|
||||
laser:
|
||||
queuing is enabled
|
||||
printing is enabled
|
||||
1 entry in spool area
|
||||
waiting for laser to come up
|
||||
....
|
||||
|
||||
Prevent a printer from accepting new jobs, then begin accepting new jobs again:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# lpc disable lp
|
||||
lp:
|
||||
queuing disabled
|
||||
# lpc enable lp
|
||||
lp:
|
||||
queuing enabled
|
||||
....
|
||||
|
||||
Stop printing, but continue to accept new jobs. Then begin printing again:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# lpc stop lp
|
||||
lp:
|
||||
printing disabled
|
||||
# lpc start lp
|
||||
lp:
|
||||
printing enabled
|
||||
daemon started
|
||||
....
|
||||
|
||||
Restart a printer after some error condition:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# lpc restart lp
|
||||
lp:
|
||||
no daemon to abort
|
||||
printing enabled
|
||||
daemon restarted
|
||||
....
|
||||
|
||||
Turn the print queue off and disable printing, with a message to explain the problem to users:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# lpc down lp Repair parts will arrive on Monday
|
||||
lp:
|
||||
printer and queuing disabled
|
||||
status message is now: Repair parts will arrive on Monday
|
||||
....
|
||||
|
||||
Re-enable a printer that is down:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# lpc up lp
|
||||
lp:
|
||||
printing enabled
|
||||
daemon started
|
||||
....
|
||||
|
||||
See man:lpc[8] for more commands and options.
|
||||
|
||||
[[printing-lpd-shared]]
|
||||
=== Shared Printers
|
||||
|
||||
Printers are often shared by multiple users in businesses and schools. Additional features are provided to make sharing printers more convenient.
|
||||
|
||||
[[printing-shared-aliases]]
|
||||
==== Aliases
|
||||
|
||||
The printer name is set in the first line of the entry in [.filename]#/etc/printcap#. Additional names, or _aliases_, can be added after that name. Aliases are separated from the name and each other by vertical bars:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
lp|repairsprinter|salesprinter:\
|
||||
....
|
||||
|
||||
Aliases can be used in place of the printer name. For example, users in the Sales department print to their printer with
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpr -Psalesprinter sales-report.txt
|
||||
....
|
||||
|
||||
Users in the Repairs department print to _their_ printer with
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
% lpr -Prepairsprinter repairs-report.txt
|
||||
....
|
||||
|
||||
All of the documents print on that single printer. When the Sales department grows enough to need their own printer, the alias can be removed from the shared printer entry and used as the name of a new printer. Users in both departments continue to use the same commands, but the Sales documents are sent to the new printer.
|
||||
|
||||
[[printing-shared-headers]]
|
||||
==== Header Pages
|
||||
|
||||
It can be difficult for users to locate their documents in the stack of pages produced by a busy shared printer. _Header pages_ were created to solve this problem. A header page with the user name and document name is printed before each print job. These pages are also sometimes called _banner_ or _separator_ pages.
|
||||
|
||||
Enabling header pages differs depending on whether the printer is connected directly to the computer with a `USB`, parallel, or serial cable, or is connected remotely over a network.
|
||||
|
||||
Header pages on directly-connected printers are enabled by removing the `:sh:\` (Suppress Header) line from the entry in [.filename]#/etc/printcap#. These header pages only use line feed characters for new lines. Some printers will need the [.filename]#/usr/shared/examples/printing/hpif# filter to prevent stairstepped text. The filter configures `PCL` printers to print both carriage returns and line feeds when a line feed is received.
|
||||
|
||||
Header pages for network printers must be configured on the printer itself. Header page entries in [.filename]#/etc/printcap# are ignored. Settings are usually available from the printer front panel or a configuration web page accessible with a web browser.
|
||||
|
||||
[[printing-lpd-references]]
|
||||
=== References
|
||||
|
||||
Example files: [.filename]#/usr/shared/examples/printing/#.
|
||||
|
||||
The _4.3BSD Line Printer Spooler Manual_, [.filename]#/usr/shared/doc/smm/07.lpd/paper.ascii.gz#.
|
||||
|
||||
Manual pages: man:printcap[5], man:lpd[8], man:lpr[1], man:lpc[8], man:lprm[1], man:lpq[1].
|
||||
|
||||
[[printing-other]]
|
||||
== Other Printing Systems
|
||||
|
||||
Several other printing systems are available in addition to the built-in man:lpd[8]. These systems offer support for other protocols or additional features.
|
||||
|
||||
[[printing-other-cups]]
|
||||
=== CUPS (Common UNIX(R) Printing System)
|
||||
|
||||
CUPS is a popular printing system available on many operating systems. Using CUPS on FreeBSD is documented in a separate article: link:{cups}[CUPS]
|
||||
|
||||
[[printing-other-hplip]]
|
||||
=== HPLIP
|
||||
|
||||
Hewlett Packard provides a printing system that supports many of their inkjet and laser printers. The port is package:print/hplip[]. The main web page is at http://hplipopensource.com/hplip-web/index.html[]. The port handles all the installation details on FreeBSD. Configuration information is shown at http://hplipopensource.com/hplip-web/install/manual/hp_setup.html[].
|
||||
|
||||
[[printing-other-lprng]]
|
||||
=== LPRng
|
||||
|
||||
LPRng was developed as an enhanced alternative to man:lpd[8]. The port is package:sysutils/LPRng[]. For details and documentation, see http://www.lprng.com/[].
|
||||
2127
documentation/content/en/books/handbook/security/_index.adoc
Normal file
2127
documentation/content/en/books/handbook/security/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
1008
documentation/content/en/books/handbook/serialcomms/_index.adoc
Normal file
1008
documentation/content/en/books/handbook/serialcomms/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,260 @@
|
|||
---
|
||||
title: Chapter 26. USB Device Mode / USB OTG
|
||||
part: Part III. System Administration
|
||||
prev: books/handbook/dtrace
|
||||
next: books/handbook/partiv
|
||||
---
|
||||
|
||||
[[usb-device-mode]]
|
||||
= USB Device Mode / USB OTG
|
||||
:doctype: book
|
||||
:toc: macro
|
||||
:toclevels: 1
|
||||
:icons: font
|
||||
:sectnums:
|
||||
:sectnumlevels: 6
|
||||
:source-highlighter: rouge
|
||||
:experimental:
|
||||
:skip-front-matter:
|
||||
:xrefstyle: basic
|
||||
:relfileprefix: ../
|
||||
:outfilesuffix:
|
||||
:sectnumoffset: 26
|
||||
|
||||
ifeval::["{backend}" == "html5"]
|
||||
:imagesdir: ../../../../images/books/handbook/usb-device-mode/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "pdf"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/usb-device-mode/
|
||||
endif::[]
|
||||
|
||||
ifeval::["{backend}" == "epub3"]
|
||||
:imagesdir: ../../../../static/images/books/handbook/usb-device-mode/
|
||||
endif::[]
|
||||
|
||||
include::shared/authors.adoc[]
|
||||
include::shared/releases.adoc[]
|
||||
include::shared/en/mailing-lists.adoc[]
|
||||
include::shared/en/teams.adoc[]
|
||||
include::shared/en/urls.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
[[usb-device-mode-synopsis]]
|
||||
== Synopsis
|
||||
|
||||
This chapter covers the use of USB Device Mode and USB On The Go (USB OTG) in FreeBSD. This includes virtual serial consoles, virtual network interfaces, and virtual USB drives.
|
||||
|
||||
When running on hardware that supports USB device mode or USB OTG, like that built into many embedded boards, the FreeBSD USB stack can run in _device mode_. Device mode makes it possible for the computer to present itself as different kinds of USB device classes, including serial ports, network adapters, and mass storage, or a combination thereof. A USB host like a laptop or desktop computer is able to access them just like physical USB devices. Device mode is sometimes called the "USB gadget mode".
|
||||
|
||||
There are two basic ways the hardware can provide the device mode functionality: with a separate "client port", which only supports the device mode, and with a USB OTG port, which can provide both device and host mode. For USB OTG ports, the USB stack switches between host-side and device-side automatically, depending on what is connected to the port. Connecting a USB device like a memory stick to the port causes FreeBSD to switch to host mode. Connecting a USB host like a computer causes FreeBSD to switch to device mode. Single purpose "client ports" always work in device mode.
|
||||
|
||||
What FreeBSD presents to the USB host depends on the `hw.usb.template` sysctl. Some templates provide a single device, such as a serial terminal; others provide multiple ones, which can all be used at the same time. An example is the template 10, which provides a mass storage device, a serial console, and a network interface. See man:usb_template[4] for the list of available values.
|
||||
|
||||
Note that in some cases, depending on the hardware and the hosts operating system, for the host to notice the configuration change, it must be either physically disconnected and reconnected, or forced to rescan the USB bus in a system-specific way. When FreeBSD is running on the host, man:usbconfig[8] `reset` can be used. This also must be done after loading [.filename]#usb_template.ko# if the USB host was already connected to the USBOTG socket.
|
||||
|
||||
After reading this chapter, you will know:
|
||||
|
||||
* How to set up USB Device Mode functionality on FreeBSD.
|
||||
* How to configure the virtual serial port on FreeBSD.
|
||||
* How to connect to the virtual serial port from various operating systems.
|
||||
* How to configure FreeBSD to provide a virtual USB network interface.
|
||||
* How to configure FreeBSD to provide a virtual USB storage device.
|
||||
|
||||
[[usb-device-mode-terminals]]
|
||||
== USB Virtual Serial Ports
|
||||
|
||||
=== Configuring USB Device Mode Serial Ports
|
||||
|
||||
Virtual serial port support is provided by templates number 3, 8, and 10. Note that template 3 works with Microsoft Windows 10 without the need for special drivers and INF files. Other host operating systems work with all three templates. Both man:usb_template[4] and man:umodem[4] kernel modules must be loaded.
|
||||
|
||||
To enable USB device mode serial ports, add those lines to [.filename]#/etc/ttys#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
ttyU0 "/usr/libexec/getty 3wire" vt100 onifconsole secure
|
||||
ttyU1 "/usr/libexec/getty 3wire" vt100 onifconsole secure
|
||||
....
|
||||
|
||||
Then add these lines to [.filename]#/etc/devd.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
notify 100 {
|
||||
match "system" "DEVFS";
|
||||
match "subsystem" "CDEV";
|
||||
match "type" "CREATE";
|
||||
match "cdev" "ttyU[0-9]+";
|
||||
action "/sbin/init q";
|
||||
};
|
||||
....
|
||||
|
||||
Reload the configuration if man:devd[8] is already running:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service devd restart
|
||||
....
|
||||
|
||||
Make sure the necessary modules are loaded and the correct template is set at boot by adding those lines to [.filename]#/boot/loader.conf#, creating it if it does not already exist:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
umodem_load="YES"
|
||||
hw.usb.template=3
|
||||
....
|
||||
|
||||
To load the module and set the template without rebooting use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload umodem
|
||||
# sysctl hw.usb.template=3
|
||||
....
|
||||
|
||||
=== Connecting to USB Device Mode Serial Ports from FreeBSD
|
||||
|
||||
To connect to a board configured to provide USB device mode serial ports, connect the USB host, such as a laptop, to the boards USB OTG or USB client port. Use `pstat -t` on the host to list the terminal lines. Near the end of the list you should see a USB serial port, eg "ttyU0". To open the connection, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cu -l /dev/ttyU0
|
||||
....
|
||||
|
||||
After pressing the kbd:[Enter] key a few times you will see a login prompt.
|
||||
|
||||
=== Connecting to USB Device Mode Serial Ports from macOS
|
||||
|
||||
To connect to a board configured to provide USB device mode serial ports, connect the USB host, such as a laptop, to the boards USB OTG or USB client port. To open the connection, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# cu -l /dev/cu.usbmodemFreeBSD1
|
||||
....
|
||||
|
||||
=== Connecting to USB Device Mode Serial Ports from Linux
|
||||
|
||||
To connect to a board configured to provide USB device mode serial ports, connect the USB host, such as a laptop, to the boards USB OTG or USB client port. To open the connection, use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# minicom -D /dev/ttyACM0
|
||||
....
|
||||
|
||||
=== Connecting to USB Device Mode Serial Ports from Microsoft Windows 10
|
||||
|
||||
To connect to a board configured to provide USB device mode serial ports, connect the USB host, such as a laptop, to the boards USB OTG or USB client port. To open a connection you will need a serial terminal program, such as PuTTY. To check the COM port name used by Windows, run Device Manager, expand "Ports (COM & LPT)". You will see a name similar to "USB Serial Device (COM4)". Run serial terminal program of your choice, for example PuTTY. In the PuTTY dialog set "Connection type" to "Serial", type the COMx obtained from Device Manager in the "Serial line" dialog box and click Open.
|
||||
|
||||
[[usb-device-mode-network]]
|
||||
== USB Device Mode Network Interfaces
|
||||
|
||||
Virtual network interfaces support is provided by templates number 1, 8, and 10. Note that none of them works with Microsoft Windows. Other host operating systems work with all three templates. Both man:usb_template[4] and man:if_cdce[4] kernel modules must be loaded.
|
||||
|
||||
Make sure the necessary modules are loaded and the correct template is set at boot by adding those lines to [.filename]#/boot/loader.conf#, creating it if it does not already exist:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
if_cdce_load="YES"
|
||||
hw.usb.template=1
|
||||
....
|
||||
|
||||
To load the module and set the template without rebooting use:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload if_cdce
|
||||
# sysctl hw.usb.template=1
|
||||
....
|
||||
|
||||
[[usb-device-mode-storage]]
|
||||
== USB Virtual Storage Device
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The man:cfumass[4] driver is a USB device mode driver first available in FreeBSD 12.0.
|
||||
====
|
||||
|
||||
Mass Storage target is provided by templates 0 and 10. Both man:usb_template[4] and man:cfumass[4] kernel modules must be loaded. man:cfumass[4] interfaces to the CTL subsystem, the same one that is used for iSCSI or Fibre Channel targets. On the host side, USB Mass Storage initiators can only access a single LUN, LUN 0.
|
||||
|
||||
=== Configuring USB Mass Storage Target Using the cfumass Startup Script
|
||||
|
||||
The simplest way to set up a read-only USB storage target is to use the [.filename]#cfumass# rc script. To configure it this way, copy the files to be presented to the USB host machine into the `/var/cfumass` directory, and add this line to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
cfumass_enable="YES"
|
||||
....
|
||||
|
||||
To configure the target without restarting, run this command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service cfumass start
|
||||
....
|
||||
|
||||
Differently from serial and network functionality, the template should not be set to 0 or 10 in [.filename]#/boot/loader.conf#. This is because the LUN must be set up before setting the template. The cfumass startup script sets the correct template number automatically when started.
|
||||
|
||||
=== Configuring USB Mass Storage Using Other Means
|
||||
|
||||
The rest of this chapter provides detailed description of setting the target without using the cfumass rc file. This is necessary if eg one wants to provide a writeable LUN.
|
||||
|
||||
USB Mass Storage does not require the man:ctld[8] daemon to be running, although it can be used if desired. This is different from iSCSI. Thus, there are two ways to configure the target: man:ctladm[8], or man:ctld[8]. Both require the [.filename]#cfumass.ko# kernel module to be loaded. The module can be loaded manually:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# kldload cfumass
|
||||
....
|
||||
|
||||
If [.filename]#cfumass.ko# has not been built into the kernel, [.filename]#/boot/loader.conf# can be set to load the module at boot:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
cfumass_load="YES"
|
||||
....
|
||||
|
||||
A LUN can be created without the man:ctld[8] daemon:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# ctladm create -b block -o file=/data/target0
|
||||
....
|
||||
|
||||
This presents the contents of the image file [.filename]#/data/target0# as a LUN to the USB host. The file must exist before executing the command. To configure the LUN at system startup, add the command to [.filename]#/etc/rc.local#.
|
||||
|
||||
man:ctld[8] can also be used to manage LUNs. Create [.filename]#/etc/ctl.conf#, add a line to [.filename]#/etc/rc.conf# to make sure man:ctld[8] is automatically started at boot, and then start the daemon.
|
||||
|
||||
This is an example of a simple [.filename]#/etc/ctl.conf# configuration file. Refer to man:ctl.conf[5] for a more complete description of the options.
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
target naa.50015178f369f092 {
|
||||
lun 0 {
|
||||
path /data/target0
|
||||
size 4G
|
||||
}
|
||||
}
|
||||
....
|
||||
|
||||
The example creates a single target with a single LUN. The `naa.50015178f369f092` is a device identifier composed of 32 random hexadecimal digits. The `path` line defines the full path to a file or zvol backing the LUN. That file must exist before starting man:ctld[8]. The second line is optional and specifies the size of the LUN.
|
||||
|
||||
To make sure the man:ctld[8] daemon is started at boot, add this line to [.filename]#/etc/rc.conf#:
|
||||
|
||||
[.programlisting]
|
||||
....
|
||||
ctld_enable="YES"
|
||||
....
|
||||
|
||||
To start man:ctld[8] now, run this command:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service ctld start
|
||||
....
|
||||
|
||||
As the man:ctld[8] daemon is started, it reads [.filename]#/etc/ctl.conf#. If this file is edited after the daemon starts, reload the changes so they take effect immediately:
|
||||
|
||||
[source,bash]
|
||||
....
|
||||
# service ctld reload
|
||||
....
|
||||
1069
documentation/content/en/books/handbook/virtualization/_index.adoc
Normal file
1069
documentation/content/en/books/handbook/virtualization/_index.adoc
Normal file
File diff suppressed because it is too large
Load diff
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue