<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
<!ENTITY % man PUBLIC "-//FreeBSD//ENTITIES DocBook Manual Page Entities//EN">
%man;
]>

<article>
  <articleinfo>
    <title>Writing FreeBSD Problem Reports</title>

    <pubdate>$FreeBSD$</pubdate>

    <abstract>
      <para>This article describes how to best formulate and submit a
        problem report to the FreeBSD Project.</para>
    </abstract>

    <authorgroup>
      <author>
	<firstname>Dag-Erling</firstname>
	<surname>Sm&oslash;rgrav</surname>
	<contrib>Contributed by </contrib>
      </author>
    </authorgroup>
  </articleinfo>

  <indexterm><primary>problem reports</primary></indexterm>

  <sect1 id="pr-intro">
    <title>Introduction</title>

    <para>One of the most frustrating experiences one can have as a
      software user is to submit a problem report only to have it
      summarily closed with a terse and unhelpful explanation like
      <quote>not a bug</quote> or <quote>bogus PR</quote>.  Similarly,
      one of the most frustrating experiences as a software developer
      is to be flooded with problem reports that are not really
      problem reports but requests for support, or that contain little
      or no information about what the problem is and how to reproduce
      it.</para>

    <para>This document attempts to describe how to write good problem
      reports.  What, you ask, is a good problem report?  Well, to go
      straight to the bottom line, a good problem report is one that
      can be analyzed and dealt with swiftly, to the mutual
      satisfaction of both user and developer.</para>

    <para>Although the primary focus of this article is on FreeBSD
      problem reports, most of it should apply quite well to other
      software projects.</para>

    <para>Note that this article is organized thematically, not
      chronologically, so you should read through the entire document
      before submitting a problem report, rather than treat it as a
      step-by-step tutorial.</para>
  </sect1>

  <sect1 id="pr-when">
    <title>When to submit a problem report</title>

    <para>There are many types of problems, and not all of them should
      engender a problem report.  Of course, nobody is perfect, and
      there will be times when you are convinced you have found a bug
      in a program when in fact you have misunderstood the syntax for
      a command or made a typo in a configuration file (though that in
      itself may sometimes be indicative of poor documentation or poor
      error handling in the application).  There are still many cases
      where submitting a problem report is clearly not the right
      course of action, and will only serve to frustrate you and the
      developers.  Conversely, there are cases where it might be
      appropriate to submit a problem report about something else than
      a bug&mdash;an enhancement or a feature request, for
      instance.</para>

    <para>So how do you determine what is a bug and what is not?  As a
      simple rule of thumb your problem is <emphasis>not</emphasis> a
      bug if it can be expressed as a question (usually of the form
      <quote>How do I do X?</quote> or <quote>Where can I find
      Y?</quote>).  It is not always quite so black and white, but the
      question rule covers a large majority of cases.</para>

    <para>Some cases where it may be appropriate to submit a problem
      report about something that is not a bug are:</para>

    <itemizedlist>
      <listitem>
        <para>Requests for feature enhancements.  It is generally a
          good idea to air these on the mailing lists before
          submitting a problem report.</para>
      </listitem>

      <listitem>
        <para>Notification of updates to externally maintained
          software (mainly ports, but also externally maintained base
          system components such as BIND or various GNU
          utilities). </para>
      </listitem>
    </itemizedlist>

    <para>Another thing is that if the system on which you experienced
      the bug is not fairly up-to-date, you should seriously consider
      upgrading and trying to reproduce the problem on an up-to-date
      system before submitting a problem report.  There are few things
      that will annoy a developer more than receiving a problem report
      about a bug she's already fixed.</para>

    <para>Finally, a bug that can not be reproduced can rarely be
      fixed.  If the bug only occurred once and you can not reproduce
      it, and it does not seem to happen to anybody else, chances are
      none of the developers will be able to reproduce it or figure
      out what's wrong.  That does not mean it did not happen, but it
      does mean that the chances of your problem report ever leading
      to a bug fix are very slim, and you should consider letting the
      matter drop.</para>
  </sect1>

  <sect1 id="pr-prep">
    <title>Preparations</title>

    <para>A good rule to follow is to always do a background search
      before submitting a problem report.  Maybe your problem has
      already been reported; maybe it is being discussed on the
      mailing lists, or recently was; it may even already be fixed in
      a newer version than what you are running.  You should therefore
      check all the obvious places before submitting your problem
      report.  For FreeBSD, this means:</para>

    <itemizedlist>
      <listitem>
        <para>The FAQ.</para>
      </listitem>

      <listitem>
        <para>The mailing lists&mdash;if you're not subscribed, use
          the searchable archives on the FreeBSD web site.  If your
          problem has not been discussed on the lists, you might try
          posting a message about it and waiting a few days to see if
          someone can spot something you have overlooked.</para>
      </listitem>

      <listitem>
        <para>Optionally, the entire web&mdash;use your favorite
          search engine to locate any references to your problem.  You
          may even get hits from archived mailing lists or newsgroups
          you did not know of or had not thought to search
          through.</para>
      </listitem>

      <listitem>
        <para>Finally, the FreeBSD PR database.  Unless your problem
          is recent or obscure, there is a fair chance it has already
          been reported.</para>
      </listitem>
    </itemizedlist>

    <para>Next, you need to make sure your problem report goes to the
      right people.</para>

    <para>The first catch here is that if the problem is a bug in
      third-party software (a port or a package you've installed), you
      should report the bug to the original author, not to the FreeBSD
      Project.  There are two exceptions to this rule: the first is if
      the bug does not occur on other platforms, in which case the
      problem may lie in how the software was ported to FreeBSD; the
      second is if the original author has already fixed the bug and
      released a patch or a new version of his software, and the
      FreeBSD port has not been updated yet.</para>

    <para>The second catch is that FreeBSD's bug tracking system sorts
      problem reports according to the category the originator
      selected.  Therefore, if you select the wrong category when you
      submit your problem report, there is a good chance that it will
      go unnoticed for a while, until someone re-categorizes
      it.</para>
  </sect1>

  <sect1 id="pr-writing">
    <title>Writing the problem report</title>

    <para>Now that you have decided that your issue merits a problem
      report, and that it is a FreeBSD problem, it is time to write
      the actual problem report.  Make sure your <envar>VISUAL</envar>
      (or <envar>EDITOR</envar> if <envar>VISUAL</envar> is not set)
      environment variable is set to something sensible, and run
      &man.send-pr.1;.</para>

    <sect2>
      <title>Attaching patches or files</title>

      <para>The &man.send-pr.1; program has provisions for attaching
        files to a problem report.  You can attach as many files as
        you want provided that each has a unique base name (i.e. the
        name of the file proper, without the path).  Just use the
        <option>-a</option> command-line option to specify the names
        of the files you wish to attach:</para>

<screen>&prompt.user; <userinput>send-pr -a /var/run/dmesg -a /tmp/errors</userinput>
</screen>

      <para>Don't worry about binary files; they will be automatically
        encoded so as not to upset your mail agent.</para>

      <para>If you attach a patch, make sure you use the
        <option>-c</option> or <option>-u</option> option to
        &man.diff.1; to create a context or unified diff, and make
        sure to specify the exact CVS revision numbers of the files
        you modified so the developers who read your report will be
        able to apply them easily.</para>

      <para>You should also take note that unless you explicitly
        specify otherwise in your PR, any patches you submit will be
        assumed to be licensed under the same terms as the original
        file you modified.</para>
    </sect2>

    <sect2>
      <title>Filling out the template</title>

      <para>The template consists of a list of fields, some of which
        are pre-filled, and some of which have comments explaining
        their purpose or listing acceptable values.  Do not worry
        about the comments; they will be removed automatically if you
        do not modify them or remove them yourself.</para>

      <para>At the top of the template, below the
        <literal>SEND-PR:</literal> lines, are the email headers.  You
        do not normally need to modify these, unless you are sending
        the problem report from a machine or account that can send but
        not receive mail, in which case you will want to set the
        <literal>From:</literal> and <literal>Reply-To:</literal> to
        your real email address.  You may also want to send yourself
        (or someone else) a carbon copy of the problem report by
        adding one or more email addresses to the
        <literal>Cc:</literal> header.</para>

      <para>Next comes a series of single-line fields:</para>

      <itemizedlist>
        <listitem>
	  <para><emphasis>Submitter-Id:</emphasis> Do not change this.
	    The default value of <literal>current-users</literal> is
	    correct, even if you run FreeBSD-STABLE.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Originator:</emphasis> This is normally
  	    prefilled with the gecos field of the currently logged-in
  	    user.  Please specify your real name, optionally followed
  	    by your email address in angle brackets.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Organization:</emphasis> Whatever you feel
	    like.  This field is not used for anything
	    significant.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Confidential:</emphasis> This is prefilled
  	    to <literal>no</literal>; changing it makes no sense as
  	    there is no such thing as a confidential FreeBSD problem
  	    report&mdash;the PR database is distributed worldwide by
  	    CVSup.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Synopsis:</emphasis> Fill this out with a
 	    short and accurate description of the problem.  The
 	    synopsis is used as the subject of the problem report
 	    email, and is used in problem report listings and
 	    summaries; problem reports with obscure synopses tend to
 	    get ignored.</para>

	  <para>If your problem report includes a patch, please have
 	    the synopsis start with <literal>[PATCH]</literal>.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Severity:</emphasis> One of
  	    <literal>non-critical</literal>,
  	    <literal>serious</literal> or
  	    <literal>critical</literal>.  Do not overreact; refrain
  	    from labeling your problem <literal>critical</literal>
  	    unless it really is (e.g. root exploit, easily
  	    reproducible panic).  Developers tend to ignore this and
  	    the next field, precisely because problem report
  	    submitters tend to overrate their problems.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Priority:</emphasis> One of
  	    <literal>low</literal>, <literal>medium</literal> or
  	    <literal>high</literal>.  See above.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Category:</emphasis> Choose one of the
	    following:</para>
	  <itemizedlist>
	    <listitem>
	      <para><literal>advocacy:</literal> problems relating to
	        FreeBSD's public image.  Rarely used.</para>
	    </listitem>

	    <listitem>
	      <para><literal>alpha:</literal> problems specific to the
	        Alpha platform.</para>
	    </listitem>

	    <listitem>
	      <para><literal>bin:</literal> problems with userland
	        programs in the base system.</para>
	    </listitem>

	    <listitem>
	      <para><literal>conf:</literal> problems with
	        configuration files, default values etc.</para>
	    </listitem>

	    <listitem>
	      <para><literal>docs:</literal> problems with man pages
	        or on-line documentation.</para>
	    </listitem>

	    <listitem>
	      <para><literal>gnu:</literal> problems with GNU software
	        such as &man.gcc.1; or &man.grep.1;.</para>
	    </listitem>

	    <listitem>
	      <para><literal>i386:</literal> problems specific to the
	        i386 platform.</para>
	    </listitem>

	    <listitem>
	      <para><literal>kern:</literal> problems with
	        kernel.</para>
	    </listitem>

	    <listitem>
	      <para><literal>misc:</literal> anything that does not fit
	        in any of the other categories.</para>
	    </listitem>
	
	    <listitem>
	      <para><literal>ports:</literal> problems relating to the
	        ports tree.</para>
	    </listitem>
	
	    <listitem>
	      <para><literal>sparc:</literal> problems specific to the
	        Sparc platform.</para>
	    </listitem>
	  </itemizedlist>
	</listitem>
	
	<listitem>
	  <para><emphasis>Class:</emphasis> Choose one of the
	    following:</para>
	
	  <itemizedlist>
	    <listitem>
	      <para><literal>sw-bug:</literal> software bugs.</para>
	    </listitem>
	
	    <listitem>
	      <para><literal>doc-bug:</literal> errors in
	        documentation.</para>
	    </listitem>
	
	    <listitem>
	      <para><literal>change-request:</literal> requests for
	        additional features or changes in existing
	        features.</para>
	    </listitem>
	
	    <listitem>
	      <para><literal>update:</literal> updates to ports or
	        other contributed software.</para>
	    </listitem>
	
	    <listitem>
	      <para><literal>maintainer-update:</literal> updates to
	        ports for which you are the maintainer.</para>
	    </listitem>
	  </itemizedlist>
	</listitem>

	<listitem>
	  <para><emphasis>Release:</emphasis> The version of FreeBSD
	    that you are running.  This is filled out automatically by
	    &man.send-pr.1; and need only be changed if you are
	    sending a problem report from a different system than the
	    one that exhibits the problem.</para>
	</listitem>
      </itemizedlist>

      <para>Finally, there is a series of multi-line fields:</para>

      <itemizedlist>
        <listitem>
	  <para><emphasis>Environment:</emphasis> This should
            describe, as accurately as possible, the environment in
            which the problem has been observed.  This includes the
            operating system version, the version of the specific
            program or file that contains the problem, and any other
            relevant items such as system configuration, other
            installed software that influences the problem,
            etc.&mdash;quite simply everything a developer needs to
            know to reconstruct the environment in which the problem
            occurs.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Description:</emphasis> A complete and
	    accurate description of the problem you are experiencing.
	    Try to avoid speculating about the causes of the problem
	    unless you are certain that you are on the right track, as
	    it may mislead a developer into making incorrect
	    assumptions about the problem.</para>
	</listitem>

	<listitem>
	  <para><emphasis>How-To-Repeat:</emphasis> A summary of the
	    actions you need to take to reproduce the problem.</para>
	</listitem>

	<listitem>
	  <para><emphasis>Fix:</emphasis> Preferably a patch, or at
	    least a workaround (which not only helps other people with
	    the same problem work around it, but may also help a
	    developer understand the cause for the problem), but if
	    you do not have any firm ideas for either, it's better to
	    leave this field blank than to speculate.</para>
	</listitem>
      </itemizedlist>
    </sect2>

    <sect2>
      <title>Sending off the problem report</title>

      <para>Once you are done filling out the template, have saved it,
        and exit your editor, &man.send-pr.1; will prompt you with
        <prompt>s)end, e)dit or a)bort?</prompt>.  You can then hit
        <userinput>s</userinput> to go ahead and submit the problem report,
        <userinput>e</userinput> to restart the editor and make
        further modifications, or <userinput>a</userinput> to abort.
        If you choose the latter, your problem report will remain on
        disk (&man.send-pr.1; will tell you the filename before it
        terminates), so you can edit it at your leisure, or maybe
        transfer it to a system with better net connectivity, before
        sending it with the <option>-f</option> to
        &man.send-pr.1;:</para>

<screen>&prompt.user; <userinput>send-pr -f ~/my-problem-report</userinput></screen>

      <para>This will read the specified file, validate the contents,
        strip comments and send it off.</para>
    </sect2>

  </sect1>

  <sect1 id="pr-followup">
    <title>Follow-up</title>

    <para>Once your problem report has been filed, you will receive a
      confirmation by email which will include the tracking number
      that was assigned to your problem report and a URL you can use
      to check its status.  With a little luck, someone will take an
      interest in your problem and try to address it, or, as the case
      may be, explain why it is not a problem.  You will be
      automatically notified of any change of status, and you will
      receive copies of any comments or patches someone may attach to
      your problem report's audit trail.</para>

    <para>If someone requests additional information from you, or you
      remember or discover something you did not mention in the
      initial report, just mail it to
      <email>bug-followup@FreeBSD.org</email>, making sure that the
      tracking number is included in the subject so the bug tracking
      system will know what problem report to attach it to.</para>

    <para>If the problem report remains open after the problem has
      gone away, just send a follow-up (in the manner prescribed
      above) saying that the problem report can be closed, and, if
      possible, explaining how or when the problem was fixed.</para>
  </sect1>

  <sect1 id="pr-further">
    <title>Further Reading</title>

    <para>This is a list of resources relevant to the proper writing
      and processing of problem reports.  It is by no means complete.</para>

    <itemizedlist>
      <listitem>
        <para><ulink
	    url="http://www.chiark.greenend.org.uk/~sgtatham/bugs.html">
	    How to Report Bugs Effectively</ulink>&mdash;an excellent
	    essay by Simon G. Tatham on composing useful (non-FreeBSD-specific)
	    problem reports.</para>
      </listitem>
    </itemizedlist>
  </sect1>
</article>
