<!DOCTYPE HTML PUBLIC "-//FreeBSD//DTD HTML 4.01 Transitional-Based Extension//EN" [
<!ENTITY base CDATA "..">
<!ENTITY date "$FreeBSD$">
<!ENTITY title "Quality Assurance Tasks for the Ports Management Team">
<!ENTITY % navinclude.about "INCLUDE">
]>

<html>
&header;

<p>There are a number of tasks that the Ports Management Team undertakes
to try to improve the quality of the Ports Collection.  These fall into
two main categories:
<a href="#qa-before-release">activities during a release cycle</a> and
<a href="#qa-between-releases">activities between release cycles</a>.
</p>

<h3><a name="qa-before-release">Activities During a Release Cycle</a></h3>

<ul>
  <li>
    <p>Work with the
      <a href="../releng/index.html">Release Engineering Team</a>
      to coordinate the release schedule.</p>
  </li>
    
  <li>
    <p>Work with the RE team to determine which pre-built packages
      can be included on the default install ISOs.</p>
  </li>
    
  <li>
    <p>Manage commits to the CVS tree for package builds via the
      following steps:</p>

    <ol>
      <li>
	<p>Institute a freeze and produce packages for all the
	  appropriate architectures.  Often this process has to be
	  repeated because either bugs are identified in various ports,
          or changes to the src tree create a risk that the packages
	  that have already been built would not work with those
	  changes.</p>

	<p>To make sure that package builds are consistent and correct,
	  <i>all</i> commits must be approved by portmgr during a
	  freeze.  Changes that are generally approved are:</p>

	  <ul>
	    <li><p>fixes to make a package build at all;</p></li>
	    <li><p>security fixes to critical packages;</p></li>
	    <li><p>problems that are noticed with licensing issues.</p></li>
	  </ul>

	<p>Unfortunately, due to the sheer size of the Ports Collection
	  and the speed that applications are developed, it is
	  impossible to fix every single problem for a release.</p>
      </li>

      <li>
	<p>The tree is then locked for all commits and a CVS tag is laid
	  down.</p>
      </li>

      <li>
	<p>The tree is then unlocked and a <tt>slush</tt> is
	  announced.  The intent of this state is to allow routine
	  changes to be made to the Ports Collection, but with the note
	  that these changes will not ship on the release ISOs.  What
	  we particularly want to avoid is
	  <a href="implementation.html#sweeping_changes">
	  sweeping changes</a>.</p>

	<p>The reason we want to avoid these commits is if some kind
	  of show-stopper problem is found (either security- or license-
	  related) such that we need to make a change that can go on
	  the release ISOs, we will need to slip the CVS tag on the
	  changed file(s).  By allowing unlimited commits, the risk is
	  high that any such change would involve having to recreate all
	  the packages all over again, resulting in an endless release
	  cycle.</p>
      </li>
    </ol>

    <p>Only once the RE team and portmgr are happy with the final
      state of the release ISOs is the ports tree completely available
      for commits again.</p>
  </li>

</ul>

<h3><a name="qa-between-releases">Activities Between Release Cycles</a></h3>

<ul>
  <li>
    <p>Manage the <a href="http://pointyhat.FreeBSD.org">Ports Build
      Cluster</a> machines.  These machines continually build packages
      on all possible combinations of OS release and CPU architecture
      (in our terminology, <tt>build environments</tt>.)</p>

    <p>These builds also produce error logs for packages that do not
      build correctly (see the above URL).  Periodically, the team
      marks these ports as BROKEN so that maintainers may be notified.
      (See below.)</p>

    <p>Successfully built packages (at least, the ones that are freely
      redistributable) are also copied to the master FTP server and thus
      become the default &quot;latest package&quot; for installations
      that use packages rather than ports.</p>
  </li>

  <li>
    <p>Notify the FreeBSD community of problems within the Ports
      Collection so that problems do not get overlooked.  To do this,
      there are a number of emailed reports.  Ones marked
      <tt>public</tt> are posted to freebsd-ports.</p>

    <ul>

      <li><p>a public list of all ports to be removed due to security
	problems, build failures, or general obsolescence, unless they
	are fixed first.</p></li>

      <li><p>private email to all maintainers of the affected ports
	(including ports dependent on the above).</p></li>

      <li><p>private email to all maintainers of ports that are already
	marked BROKEN and/or FORBIDDEN.</p></li>

      <li><p>private email to maintainers who are not committers, who have
	PRs filed against their ports (to flag PRs that might never have
	been Cc:ed to them).</p></li>

      <li><p>public email about port commits that break building of
	INDEX.</p></li>

      <li><p>public email about port commits that send the revision
	metadata backwards (and thus confuse tools like portupgrade).</p></li>

      <li><p>a public list of all ports that have at least one file
	that fails to fetch from any non-FreeBSD mastersite.  For the
	complete list of results for all files versus all mastersites,
	see <a href="http://people.freebsd.org/~ehaupt/distilator/">
	Emanuel Haupt's port survey</a>.</p></li>

      <li><p>private email to an affected port maintainer when a port 
	is about to be marked BROKEN, Cc:ed to the last committer to
	the port.  (This email is not automated but it should be sent
	as a courtesy.)</p></li>

      <li><p>a list of ports that do not set NO_LATEST_LINK.  (Ports
	that have a stable version, and a development version, will
	generally have the development version set to a later revision.
	If it is desirable that users should install the stable version
	from packages, rather than the development version, this flag
	should be set; otherwise, users will get the latest version by
	default.)</p></li>

    </ul>

  </li>

  <li>
    <p>Remove expired ports.  Ports that have been marked BROKEN for
      some time are marked DEPRECATED (with an EXPIRATION_DATE) and then
      are removed if no one has fixed them by that time.  The intent of
      this this process is to try to insure that if a user installs a port,
      there is the best possible change that it can be made to work.</p>

    <p>In other cases, ports are marked DEPRECATED when they have been
      replaced by a newer version and the older version is no longer
      maintained by the authors.  The EXPIRATION_DATE should generally
      be set at least two months in the future to allow everyone sufficient
      time to upgrade.</p>
  </li>
</ul>

&footer;
</body>
</html>
