doc/en_US.ISO8859-1/articles/problem-reports/article.sgml
2003-12-24 00:26:41 +00:00

749 lines
29 KiB
Text

<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
<!ENTITY % man PUBLIC "-//FreeBSD//ENTITIES DocBook Manual Page Entities//EN">
%man;
<!ENTITY % mailing-lists PUBLIC "-//FreeBSD//ENTITIES DocBook Mailing List Entities//EN">
%mailing-lists;
<!ENTITY % trademarks PUBLIC "-//FreeBSD//ENTITIES DocBook Trademark Entities//EN">
%trademarks;
]>
<article>
<articleinfo>
<title>Writing FreeBSD Problem Reports</title>
<pubdate>$FreeBSD$</pubdate>
<legalnotice id="trademarks" role="trademarks">
&tm-attrib.freebsd;
&tm-attrib.cvsup;
&tm-attrib.ibm;
&tm-attrib.intel;
&tm-attrib.sparc;
&tm-attrib.sun;
&tm-attrib.general;
</legalnotice>
<abstract>
<para>This article describes how to best formulate and submit a
problem report to the FreeBSD Project.</para>
</abstract>
<authorgroup>
<author>
<firstname>Dag-Erling</firstname>
<surname>Sm&oslash;rgrav</surname>
<contrib>Contributed by </contrib>
</author>
</authorgroup>
</articleinfo>
<indexterm><primary>problem reports</primary></indexterm>
<section id="pr-intro">
<title>Introduction</title>
<para>One of the most frustrating experiences one can have as a
software user is to submit a problem report only to have it
summarily closed with a terse and unhelpful explanation like
<quote>not a bug</quote> or <quote>bogus PR</quote>. Similarly,
one of the most frustrating experiences as a software developer
is to be flooded with problem reports that are not really
problem reports but requests for support, or that contain little
or no information about what the problem is and how to reproduce
it.</para>
<para>This document attempts to describe how to write good problem
reports. What, you ask, is a good problem report? Well, to go
straight to the bottom line, a good problem report is one that
can be analyzed and dealt with swiftly, to the mutual
satisfaction of both user and developer.</para>
<para>Although the primary focus of this article is on FreeBSD
problem reports, most of it should apply quite well to other
software projects.</para>
<para>Note that this article is organized thematically, not
chronologically, so you should read through the entire document
before submitting a problem report, rather than treat it as a
step-by-step tutorial.</para>
</section>
<section id="pr-when">
<title>When to submit a problem report</title>
<para>There are many types of problems, and not all of them should
engender a problem report. Of course, nobody is perfect, and
there will be times when you are convinced you have found a bug
in a program when in fact you have misunderstood the syntax for
a command or made a typographical error in a configuration file (though that in
itself may sometimes be indicative of poor documentation or poor
error handling in the application). There are still many cases
where submitting a problem report is clearly <emphasis>not</emphasis> the right
course of action, and will only serve to frustrate you and the
developers. Conversely, there are cases where it might be
appropriate to submit a problem report about something else than
a bug&mdash;an enhancement or a feature request, for
instance.</para>
<para>So how do you determine what is a bug and what is not? As a
simple rule of thumb your problem is <emphasis>not</emphasis> a
bug if it can be expressed as a question (usually of the form
<quote>How do I do X?</quote> or <quote>Where can I find
Y?</quote>). It is not always quite so black and white, but the
question rule covers a large majority of cases. If you are looking
for an answer, consider posing your question to the
&a.questions;.</para>
<para>Some cases where it may be appropriate to submit a problem
report about something that is not a bug are:</para>
<itemizedlist>
<listitem>
<para>Requests for feature enhancements. It is generally a
good idea to air these on the mailing lists before
submitting a problem report.</para>
</listitem>
<listitem>
<para>Notification of updates to externally maintained
software (mainly ports, but also externally maintained base
system components such as BIND or various GNU
utilities).</para>
</listitem>
</itemizedlist>
<para>Another thing is that if the system on which you experienced
the bug is not fairly up-to-date, you should seriously consider
upgrading and trying to reproduce the problem on an up-to-date
system before submitting a problem report. There are few things
that will annoy a developer more than receiving a problem report
about a bug she has already fixed.</para>
<para>Finally, a bug that can not be reproduced can rarely be
fixed. If the bug only occurred once and you can not reproduce
it, and it does not seem to happen to anybody else, chances are
none of the developers will be able to reproduce it or figure
out what is wrong. That does not mean it did not happen, but it
does mean that the chances of your problem report ever leading
to a bug fix are very slim, and you should consider letting the
matter drop.</para>
</section>
<section id="pr-prep">
<title>Preparations</title>
<para>A good rule to follow is to always do a background search
before submitting a problem report. Maybe your problem has
already been reported; maybe it is being discussed on the
mailing lists, or recently was; it may even already be fixed in
a newer version than what you are running. You should therefore
check all the obvious places before submitting your problem
report. For FreeBSD, this means:</para>
<itemizedlist>
<listitem>
<para>The FreeBSD
<ulink URL="../../books/faq/index.html">Frequently Asked
Questions</ulink> (FAQ) list.
The FAQ attempts to provide answers for a wide range of questions,
such as those concerning
<ulink URL="../../books/faq/hardware.html">hardware
compatibility</ulink>,
<ulink URL="../../books/faq/applications.html">user
applications</ulink>,
and <ulink URL="../../books/faq/kernelconfig.html">kernel
configuration</ulink>.</para>
</listitem>
<listitem>
<para>The
<ulink
URL="../../books/handbook/eresources.html#ERESOURCES-MAIL">mailing
lists</ulink>&mdash;if you are not subscribed, use
<ulink
URL="http://www.FreeBSD.org/search/search.html#mailinglists">the
searchable archives</ulink> on the FreeBSD web site. If your
problem has not been discussed on the lists, you might try
posting a message about it and waiting a few days to see if
someone can spot something you have overlooked.</para>
</listitem>
<listitem>
<para>Optionally, the entire web&mdash;use your favorite
search engine to locate any references to your problem. You
may even get hits from archived mailing lists or newsgroups
you did not know of or had not thought to search
through.</para>
</listitem>
<listitem>
<para>Next, the searchable
<ulink URL="http://www.FreeBSD.org/cgi/query-pr-summary.cgi?query">
FreeBSD PR database</ulink> (GNATS). Unless your problem
is recent or obscure, there is a fair chance it has already
been reported.</para>
</listitem>
<listitem>
<para>Finally, if you are upgrading from one version to
another&mdash;especially if you are upgrading to the
<literal>-current</literal> branch&mdash;you should
carefully study the contents of the
<filename>/usr/src/UPDATING</filename> file on your system
or its latest version at
<ulink URL="http://www.FreeBSD.org/cgi/cvsweb.cgi/src/UPDATING"></ulink>.
This file contains many pieces of vital information.</para>
</listitem>
</itemizedlist>
<para>Next, you need to make sure your problem report goes to the
right people.</para>
<para>The first catch here is that if the problem is a bug in
third-party software (a port or a package you have installed), you
should report the bug to the original author, not to the FreeBSD
Project. There are two exceptions to this rule: the first is if
the bug does not occur on other platforms, in which case the
problem may lie in how the software was ported to FreeBSD; the
second is if the original author has already fixed the bug and
released a patch or a new version of his software, and the
FreeBSD port has not been updated yet.</para>
<para>The second catch is that FreeBSD's bug tracking system sorts
problem reports according to the category the originator
selected. Therefore, if you select the wrong category when you
submit your problem report, there is a good chance that it will
go unnoticed for a while, until someone re-categorizes
it.</para>
</section>
<section id="pr-writing">
<title>Writing the problem report</title>
<para>Now that you have decided that your issue merits a problem
report, and that it is a FreeBSD problem, it is time to write
the actual problem report. Before we get into the mechanics
of the program used to generate and submit PRs, here are some
tips and tricks to help make sure that your PR will be most
effective.</para>
<section>
<title>Tips and tricks for writing a good problem report</title>
<itemizedlist>
<listitem>
<para><emphasis>Do not leave the <quote>Synopsis</quote>
line empty.</emphasis> The PRs go both onto a mailing list
that goes all over the world (where the <quote>Synopsis</quote> is used
for the <literal>Subject:</literal> line), but also into a
database. Anyone who comes along later and browses the
database by synopsis, and finds a PR with a blank subject
line, tends just to skip over it. Remember that PRs stay
in this database until they are closed by someone; an
anonymous one will usually just disappear in the
noise.</para>
</listitem>
<listitem>
<para><emphasis>Avoid using a weak <quote>Synopsis</quote>
line.</emphasis> You should not assume that anyone reading
your PR has any context for your submission, so the more
you provide, the better. For instance, what part of the
system does the problem apply to? Do you only see the
problem while installing, or while running? To
illustrate, instead of <literal>Synopsis: portupgrade is
broken</literal>, see how much more informative this
seems: <literal>Synopsis: port sysutils/portupgrade
coredumps on -current</literal>. (In the case of ports,
it is especially helpful to have both the category and
portname in the <quote>Synopsis</quote> line.)</para>
</listitem>
<listitem>
<para><emphasis>If you have a patch, say so.</emphasis>
A PR with a patch included is much more likely to be
looked at than one without. If you are including one,
put the string <literal>[patch]</literal> at the
beginning of the <quote>Synopsis</quote>. (Although it is
not mandatory to use that exact string, by convention,
that is the one that is used.)</para>
</listitem>
<listitem>
<para><emphasis>If you are a maintainer, say so.</emphasis>
If you are maintaining a part of the source code (for
instance, a port), you might consider adding the string
<literal>[maintainer update]</literal> at the beginning of
your synopsis line and/or set the <quote>Class</quote> of
your PR to <literal>maintainer-update</literal>. This way
any committer that handles your PR will not have to check
with the <filename>Makefile</filename> of the PR every
time the PR is viewed to make sure this is an update sent
by the maintainer of the port.</para>
</listitem>
<listitem>
<para><emphasis>Be specific.</emphasis>
The more information you supply about what problem you
are having, the better your chance of getting a response.
You should include such things as what version you are
running (there is a place to put that, see below);
which architecture you are running on;
whether you are running from a release CDROM, or from
a system maintained by &man.cvsup.1; (and, if so, how
recently you updated); and, if a kernel problem,
if you have read <literal>src/UPDATING</literal>
(someone is guaranteed to ask). You do not necessarily
have to provide your kernel configuration, which ports
you have available, and a core dump (including these
by default only tends to fill up the database), but you
should be prepared to make them available, either
privately or publicly, if so asked.</para>
</listitem>
<listitem>
<para><emphasis>Avoid vague requests for features.</emphasis>
PRs of the form <quote>someone should really implement something
that does so-and-so</quote> are less likely to get results than
very specific requests. Remember, the source is available
to everyone, so if you want a feature, the best way to
ensure it being included is to get to work! Also consider
the fact that many things like this would make a better
topic for discussion on <literal>freebsd-questions</literal>
than an entry in the PR database, as discussed above.</para>
</listitem>
<listitem>
<para><emphasis>Make sure no one else has already submitted
a similar PR.</emphasis> Although this has already been
mentioned above, it bears repeating here. It only take a
minute or two to use the web-based search engine at
<ulink URL="http://www.FreeBSD.org/cgi/query-pr-summary.cgi?query"></ulink>.
(Of course, everyone is guilty of forgetting to do this
now and then.)</para> </listitem>
<listitem>
<para><emphasis>Avoid controversial requests.</emphasis>
If your PR addresses an area that has been controversial
in the past, you should probably be prepared to not only
offer patches, but also justification for why the patches
are <quote>The Right Thing To Do</quote>. As noted above,
a careful search of the mailing lists using the archives
at <ulink URL="http://www.FreeBSD.org/search/search.html#mailinglists"></ulink>
is always good preparation.</para>
</listitem>
<listitem>
<para><emphasis>Be polite.</emphasis>
Almost anyone who would potentially work on your PR is a
volunteer. No one likes to be told that they have to do
something when they are already doing it for some
motivation other than monetary gain. This is a good thing
to keep in mind at all times on Open Source
projects.</para>
</listitem>
</itemizedlist>
</section>
<section>
<title>Before you begin</title>
<para>Before running the &man.send-pr.1; program, make sure your
<envar>VISUAL</envar> (or <envar>EDITOR</envar> if
<envar>VISUAL</envar> is not set) environment variable is set
to something sensible.</para>
<para>You should also make sure that mail delivery works fine.
&man.send-pr.1; uses mail messages for the submission and
tracking of problem reports. If you cannot post mail messages
from the machine you're running &man.send-pr.1; on, your
problem report will not reach the GNATS database. For details
on the setup of mail on FreeBSD, see the <quote>Electronic
Mail</quote> chapter of the FreeBSD Handbook at
<ulink URL="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/mail.html"></ulink>.</para>
</section>
<section>
<title>Attaching patches or files</title>
<para>The &man.send-pr.1; program has provisions for attaching
files to a problem report. You can attach as many files as
you want provided that each has a unique base name (i.e. the
name of the file proper, without the path). Just use the
<option>-a</option> command-line option to specify the names
of the files you wish to attach:</para>
<screen>&prompt.user; <userinput>send-pr -a /var/run/dmesg -a /tmp/errors</userinput></screen>
<para>Do not worry about binary files, they will be automatically
encoded so as not to upset your mail agent.</para>
<para>If you attach a patch, make sure you use the
<option>-c</option> or <option>-u</option> option to
&man.diff.1; to create a context or unified diff (unified is
preferred), and make
sure to specify the exact CVS revision numbers of the files
you modified so the developers who read your report will be
able to apply them easily. A patch against the CURRENT or HEAD
CVS branch is preferred since all new code should be applied
and tested there first. After appropriate or substantial testing
has been done, the code will be merged/migrated to the STABLE
branch.</para>
<para>If you attach a patch inline, instead of as an attachment,
note that the most common problem by far is the tendency of some
email programs to render tabs as spaces, which will completely
ruin anything intended to be part of a Makefile.</para>
<para>Also note that while including small patches in a PR is
generally all right&mdash;particularly when they fix the problem
described in the PR&mdash;large patches and especially new code
which may require substantial review before committing should
be placed on a web or ftp server, and the URL should be
included in the PR instead of the patch. Patches in email
tend to get mangled, especially when GNATS is involved, and
the larger the patch, the harder it will be for interested
parties to unmangle it. Also, posting a patch on the web
allows you to modify it without having to resubmit the entire
patch in a followup to the original PR.</para>
<para>You should also take note that unless you explicitly
specify otherwise in your PR or in the patch itself, any
patches you submit will be assumed to be licensed under the
same terms as the original file you modified.</para>
</section>
<section>
<title>Filling out the template</title>
<para>When you run &man.send-pr.1;, you are presented with a
template. The template consists of a list of fields, some of
which are pre-filled, and some of which have comments explaining
their purpose or listing acceptable values. Do not worry
about the comments; they will be removed automatically if you
do not modify them or remove them yourself.</para>
<para>At the top of the template, below the
<literal>SEND-PR:</literal> lines, are the email headers. You
do not normally need to modify these, unless you are sending
the problem report from a machine or account that can send but
not receive mail, in which case you will want to set the
<literal>From:</literal> and <literal>Reply-To:</literal> to
your real email address. You may also want to send yourself
(or someone else) a carbon copy of the problem report by
adding one or more email addresses to the
<literal>Cc:</literal> header.</para>
<para>Next comes a series of single-line fields:</para>
<itemizedlist>
<listitem>
<para><emphasis>Submitter-Id:</emphasis> Do not change this.
The default value of <literal>current-users</literal> is
correct, even if you run FreeBSD-STABLE.</para>
</listitem>
<listitem>
<para><emphasis>Originator:</emphasis> This is normally
prefilled with the <literal>gecos</literal> field of the
currently logged-in
user. Please specify your real name, optionally followed
by your email address in angle brackets.</para>
</listitem>
<listitem>
<para><emphasis>Organization:</emphasis> Whatever you feel
like. This field is not used for anything
significant.</para>
</listitem>
<listitem>
<para><emphasis>Confidential:</emphasis> This is prefilled
to <literal>no</literal>. Changing it makes no sense as
there is no such thing as a confidential FreeBSD problem
report&mdash;the PR database is distributed worldwide by
<application>CVSup</application>.</para>
</listitem>
<listitem>
<para><emphasis>Synopsis:</emphasis> Fill this out with a
short and accurate description of the problem. The
synopsis is used as the subject of the problem report
email, and is used in problem report listings and
summaries; problem reports with obscure synopses tend to
get ignored.</para>
<para>As noted above, if your problem report includes a patch,
please have the synopsis start with <literal>[patch]</literal>;
if you are a maintainer, you may consider adding
<literal>[maintainer update]</literal> and/or set the
<quote>Class</quote> of your PR to
<literal>maintainer-update</literal>.</para>
</listitem>
<listitem>
<para><emphasis>Severity:</emphasis> One of
<literal>non-critical</literal>,
<literal>serious</literal> or
<literal>critical</literal>. Do not overreact; refrain
from labeling your problem <literal>critical</literal>
unless it really is (e.g. <username>root</username> exploit, easily
reproducible panic). Developers tend to ignore this and
the next field, precisely because problem report
submitters tend to overrate their problems.</para>
</listitem>
<listitem>
<para><emphasis>Priority:</emphasis> One of
<literal>low</literal>, <literal>medium</literal> or
<literal>high</literal>. See above.</para>
</listitem>
<listitem>
<para><emphasis>Category:</emphasis> Choose one of the
following:</para>
<itemizedlist>
<listitem>
<para><literal>advocacy:</literal> problems relating to
FreeBSD's public image. Rarely used.</para>
</listitem>
<listitem>
<para><literal>alpha:</literal> problems specific to the
Alpha platform.</para>
</listitem>
<listitem>
<para><literal>amd64:</literal> problems specific to the
AMD64 platform.</para>
</listitem>
<listitem>
<para><literal>bin:</literal> problems with userland
programs in the base system.</para>
</listitem>
<listitem>
<para><literal>conf:</literal> problems with
configuration files, default values etc.</para>
</listitem>
<listitem>
<para><literal>docs:</literal> problems with manual pages
or on-line documentation.</para>
</listitem>
<listitem>
<para><literal>gnu:</literal> problems with GNU software
such as &man.gcc.1; or &man.grep.1;.</para>
</listitem>
<listitem>
<para><literal>i386:</literal> problems specific to the
&i386; platform.</para>
</listitem>
<listitem>
<para><literal>ia64:</literal> problems specific to the
ia64 platform.</para>
</listitem>
<listitem>
<para><literal>java:</literal> problems related to &java;.
</para>
</listitem>
<listitem>
<para><literal>kern:</literal> problems with
kernel.</para>
</listitem>
<listitem>
<para><literal>misc:</literal> anything that does not fit
in any of the other categories. (Note that it is
easy for things to get lost in this category).</para>
</listitem>
<listitem>
<para><literal>ports:</literal> problems relating to the
ports tree.</para>
</listitem>
<listitem>
<para><literal>powerpc:</literal> problems specific to the
&powerpc; platform.</para>
</listitem>
<listitem>
<para><literal>sparc64:</literal> problems specific to the
&sparc64; platform.</para>
</listitem>
<listitem>
<para><literal>standards:</literal> Standards conformance
issues.</para>
</listitem>
<listitem>
<para><literal>www:</literal> Changes or enhancements to
the FreeBSD website.</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para><emphasis>Class:</emphasis> Choose one of the
following:</para>
<itemizedlist>
<listitem>
<para><literal>sw-bug:</literal> software bugs.</para>
</listitem>
<listitem>
<para><literal>doc-bug:</literal> errors in
documentation.</para>
</listitem>
<listitem>
<para><literal>change-request:</literal> requests for
additional features or changes in existing
features.</para>
</listitem>
<listitem>
<para><literal>update:</literal> updates to ports or
other contributed software.</para>
</listitem>
<listitem>
<para><literal>maintainer-update:</literal> updates to
ports for which you are the maintainer.</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para><emphasis>Release:</emphasis> The version of FreeBSD
that you are running. This is filled out automatically by
&man.send-pr.1; and need only be changed if you are
sending a problem report from a different system than the
one that exhibits the problem.</para>
</listitem>
</itemizedlist>
<para>Finally, there is a series of multi-line fields:</para>
<itemizedlist>
<listitem>
<para><emphasis>Environment:</emphasis> This should
describe, as accurately as possible, the environment in
which the problem has been observed. This includes the
operating system version, the version of the specific
program or file that contains the problem, and any other
relevant items such as system configuration, other
installed software that influences the problem,
etc.&mdash;quite simply everything a developer needs to
know to reconstruct the environment in which the problem
occurs.</para>
</listitem>
<listitem>
<para><emphasis>Description:</emphasis> A complete and
accurate description of the problem you are experiencing.
Try to avoid speculating about the causes of the problem
unless you are certain that you are on the right track, as
it may mislead a developer into making incorrect
assumptions about the problem.</para>
</listitem>
<listitem>
<para><emphasis>How-To-Repeat:</emphasis> A summary of the
actions you need to take to reproduce the problem.</para>
</listitem>
<listitem>
<para><emphasis>Fix:</emphasis> Preferably a patch, or at
least a workaround (which not only helps other people with
the same problem work around it, but may also help a
developer understand the cause for the problem), but if
you do not have any firm ideas for either, it is better to
leave this field blank than to speculate.</para>
</listitem>
</itemizedlist>
</section>
<section>
<title>Sending off the problem report</title>
<para>Once you are done filling out the template, have saved it,
and exit your editor, &man.send-pr.1; will prompt you with
<prompt>s)end, e)dit or a)bort?</prompt>. You can then hit
<userinput>s</userinput> to go ahead and submit the problem report,
<userinput>e</userinput> to restart the editor and make
further modifications, or <userinput>a</userinput> to abort.
If you choose the latter, your problem report will remain on
disk (&man.send-pr.1; will tell you the filename before it
terminates), so you can edit it at your leisure, or maybe
transfer it to a system with better net connectivity, before
sending it with the <option>-f</option> to
&man.send-pr.1;:</para>
<screen>&prompt.user; <userinput>send-pr -f ~/my-problem-report</userinput></screen>
<para>This will read the specified file, validate the contents,
strip comments and send it off.</para>
</section>
</section>
<section id="pr-followup">
<title>Follow-up</title>
<para>Once your problem report has been filed, you will receive a
confirmation by email which will include the tracking number
that was assigned to your problem report and a URL you can use
to check its status. With a little luck, someone will take an
interest in your problem and try to address it, or, as the case
may be, explain why it is not a problem. You will be
automatically notified of any change of status, and you will
receive copies of any comments or patches someone may attach to
your problem report's audit trail.</para>
<para>If someone requests additional information from you, or you
remember or discover something you did not mention in the
initial report, just mail it to
<email>bug-followup@FreeBSD.org</email>, making sure that the
tracking number is included in the subject so the bug tracking
system will know what problem report to attach it to.</para>
<para>If the problem report remains open after the problem has
gone away, just send a follow-up (in the manner prescribed
above) saying that the problem report can be closed, and, if
possible, explaining how or when the problem was fixed.</para>
</section>
<section id="pr-further">
<title>Further Reading</title>
<para>This is a list of resources relevant to the proper writing
and processing of problem reports. It is by no means complete.</para>
<itemizedlist>
<listitem>
<para><ulink
url="http://www.chiark.greenend.org.uk/~sgtatham/bugs.html">
How to Report Bugs Effectively</ulink>&mdash;an excellent
essay by Simon G. Tatham on composing useful (non-FreeBSD-specific)
problem reports.</para>
</listitem>
<listitem>
<para><ulink
url="../../articles/pr-guidelines/article.html">Problem
Report Handling Guidelines</ulink>&mdash;valuable insight
into how problem reports are handled by the FreeBSD
developers.</para>
</listitem>
</itemizedlist>
</section>
</article>