<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
<!ENTITY % man PUBLIC "-//FreeBSD//ENTITIES DocBook Manual Page Entities//EN">
%man;
<!ENTITY % authors PUBLIC "-//FreeBSD//ENTITIES DocBook Author Entities//EN">
%authors;
]>

<article>
  <articleinfo>
    <title>Setting up a CVS repository - the FreeBSD way</title>

    <author>
      <firstname>Stijn</firstname>
      <surname>Hoop</surname>
      <affiliation>
        <address><email>stijn@win.tue.nl</email></address>
      </affiliation>
    </author>

    <copyright>
      <year>2001</year>
      <holder role="mailto:stijn@win.tue.nl">Stijn Hoop</holder>
    </copyright>

    <pubdate role="rcs">$Date: 2002-04-10 12:54:25 $</pubdate>

    <releaseinfo>$FreeBSD$</releaseinfo>

    <abstract>
      <para>This article describes the steps I took to setup a CVS repository
        that uses the same scripts the FreeBSD project uses in their setup.
        This has several advantages over a stock CVS setup, including more
        granular access control to the source tree and generation of readable
        email of every commit.</para>
    </abstract>
  </articleinfo>

  <sect1>
    <title>Introduction</title>

    <para>Most of the open source software projects use
      <application>CVS</application> as their source code control system.
      While <application>CVS</application> is pretty good at this, it has its
      share of flaws and weaknesses.  One of these is that sharing a source
      tree with other developers can quickly lead to a system administration
      nightmare, especially if one wishes to protect parts of the tree from
      general access.</para>

    <para>FreeBSD is one of the projects using <application>CVS</application>.
      It also has a large base of developers located around the world.
      They developed some scripts to make management of the repository easier.
      Recently, these scripts were revisited and normalized by &a.joe;
      to make it easier to reuse them in other projects. This
      article describes one method of using the new scripts.</para>

    <para>To make the most use of the information in this article, you need to
      be familiar with the basic method of operation of
      <application>CVS</application>.</para>
  </sect1>

  <sect1>
    <title>First setup</title>

    <warning>
      <para>It might be best to first perform this procedure with an empty
        test repository, to make sure you understand all consequences.
        As always, make sure you have recent, readable backups!</para>
    </warning>

    <sect2>
      <title>Initializing the repository</title>

      <para>The first thing to do when setting up a new repository is to tell
        <application>CVS</application> to initialize it:

        <screen>&prompt.user; <userinput>cvs -d <replaceable>path-to-repository</replaceable> init</userinput></screen>

        This tells <application>CVS</application> to create the
        <filename>CVSROOT</filename> administrative directory, where all the
        customization takes place.</para>
    </sect2>

    <sect2>
      <title>The repository group</title>

      <para>Now we will create the group which will own the repository.
        All committers need to be in this group, so that they can write to the
        repository. We will assume the FreeBSD default of
        <literal>ncvs</literal> for this group.

        <screen>&prompt.root; <userinput>pw groupadd <replaceable>ncvs</replaceable></userinput></screen>

        Next, you should &man.chown.8; the directory to the group
        you just added:

        <screen>&prompt.root; <userinput>chown -R :<replaceable>ncvs</replaceable> <replaceable>path-to-your-repository</replaceable></userinput></screen>

        This ensures that no one can write to the repository without proper
        group permissions.</para>
    </sect2>

    <sect2>
      <title>Getting the sources</title>

      <para>Now you need to obtain the <filename>CVSROOT</filename> directory
        from the FreeBSD repository. This is most easily done by checking it
        out from a FreeBSD anonymous CVS mirror. See <ulink
          url="../../books/handbook/anoncvs.html">the relevant chapter in
        the handbook</ulink> for more information. Let us assume that the
        sources are stored in <filename>CVSROOT-freebsd</filename> in the
        current directory.</para>
    </sect2>

    <sect2>
      <title>Copying the FreeBSD scripts</title>

      <para>Next, we will copy the FreeBSD <filename>CVSROOT</filename>
        sources into your own repository. If you are accustomed to
        <application>CVS</application>, you might be thinking that you can just
        import the scripts, in an attempt to make synchronizing with later
        versions easier. However, it turns out that
        <application>CVS</application> has a deficiency in this area:
        when importing sources into the <filename>CVSROOT</filename> directory,
        it will not update the needed administrative files. In order to make
        it recognize those, you will need to checkin each file after importing
        them, losing the value of <literal>cvs import</literal>. Therefore,
        the recommended method is to simply copy over the scripts.</para>

      <para>It does not matter if the above paragraph did not make sense to
        you&mdash;the end result is the same. Simply check out your
        <filename>CVSROOT</filename> and copy the FreeBSD files over your
        local (untouched) copies:

        <screen>&prompt.user; <userinput>cvs -d <replaceable>path-to-your-repository</replaceable> checkout CVSROOT</userinput>
&prompt.user; <userinput>cd CVSROOT</userinput>
&prompt.user; <userinput>cp ../CVSROOT-freebsd/* .</userinput>
&prompt.user; <userinput>cvs add *</userinput></screen>

        Note that you will probably get a few warnings about some directories
        not being copied; this is normal, you don't need those.</para>
    </sect2>

    <sect2>
      <title>The scripts</title>

      <para>Now you have in your working directory an exact copy of the scripts
        that the FreeBSD project itself uses for their repository. A summary
        of what each file is used for is included below.</para>

      <itemizedlist>
        <listitem>
          <para><filename>access</filename> - this file is not used in the
            default setup. It is used in <link linkend="freebsdspecific">the
            FreeBSD project specific setup</link>, where it controls access to
            the repository. You can remove this file if you
            do not wish to use this setup.</para>
        </listitem>

        <listitem>
          <para><filename>avail</filename> - this file controls access to the
            repository. In this, you can specify groups of people that are
            allowed access to the repository, as well as disallow commits on a
            per-directory basis. You should tailor it to contain the groups
            and directories that will be in your repository.</para>
        </listitem>

        <listitem>
          <para><filename>cfg.pm</filename> - this file parses your
            configuration, and provides the default configuration.  You should
            <emphasis>not</emphasis> make changes to this file. Instead, put
            your configuration changes in
            <filename>cfg_local.pm</filename>.</para>
        </listitem>

        <listitem>
          <para><filename>cfg_local.pm</filename> - this file contains all
            configurable parameters of the system. You should configure all
            sorts of settings here, such as where commit mail is send, on what
            hosts people can commit, and others. More information on this
            below.</para>
        </listitem>

        <listitem>
          <para><filename>checkoutlist</filename> - this files lists all
            files under control of <application>CVS</application> in this
            directory. You should edit this to remove some FreeBSD specific
            files.</para>
        </listitem>

        <listitem>
          <para><filename>commit_prep.pl</filename> - this script performs
            various pre-commit checks, based on whether you enabled them in your
            <filename>cfg_local.pm</filename>. You should not have to touch
            this.</para>
        </listitem>

        <listitem>
          <para><filename>commitcheck</filename> - this script is invoked
            directly from <application>CVS</application>. It first checks if
            the committer has access to the specified part of the tree, and
            then runs <filename>commit_prep.pl</filename> for the various
            pre-commit checks. If those are OK, <application>CVS</application>
            will allow the commit to proceed. You should not have to touch
            this file.</para>
        </listitem>

        <listitem>
          <para><filename>commitinfo</filename> - this file is used by
            <application>CVS</application> to determine which script to run
            before a commit. You should not have to touch this file.</para>
        </listitem>

        <listitem>
          <para><filename>config</filename> - the configuration file for
            this repository. You should change this as needed, but most
            administrators can probably leave the defaults. More information on
            the options that can be set here can be found in the
            <application>CVS</application> manual.</para>
        </listitem>

        <listitem>
          <para><filename>cvs_acls.pl</filename> - this script determines
            the committers identity, and whether he/she is allowed access to the
            tree. It does this based on the <filename>avail</filename> file.
            You should not have to touch this file.</para>
        </listitem>

        <listitem>
          <para><filename>cvsignore</filename> - this file specifies files
            that <application>CVS</application> should not checkin in the
            repository. You can edit this as you wish. More information about
            this file is available in the <application>CVS</application>
            manual.</para>
        </listitem>

        <listitem>
          <para><filename>cvswrappers</filename> - this file is used by
            <application>CVS</application> to enable or disable keyword
            expansion, or whether a file should be considered binary. You
            can edit this as you wish. More information about this file
            is available in the <application>CVS</application> manual.</para>
        </listitem>

        <listitem>
          <para><filename>edithook</filename> - this file is not used
            any more, but kept for historic reasons. You should not have to
            touch this file.</para>
        </listitem>

        <listitem>
          <para><filename>editinfo</filename> - <application>CVS</application>
            uses this file for editor overrides. FreeBSD does not use this
            functionality, as parsing the log message is done in the
            <filename>verifymsg</filename> and <filename>logcheck</filename>
            files. You should not have to touch this file.</para>
        </listitem>

        <listitem>
          <para><filename>exclude</filename> - this file lists regular
            expressions that are used to determine files which cannot contain a
            revision header. In the FreeBSD setup, all files under revision
            control need to have a revision header (like
            &dollar;FreeBSD&dollar;). All filenames that match one of the lines
            in this file are exempted from this check. You should add
            expressions to this file as you checkin files that cannot have a
            revision header. For the purpose of installing the scripts, it
            may be best to exclude <filename>CVSROOT/</filename> from header
            checks.</para>
        </listitem>

        <listitem>
          <para><filename>log_accum.pl</filename> - this is a script that
            takes the log message as provided by the
            <filename>logcheck</filename> script, and appends it to a log file
            in the repository for backup purposes. It also handles mailing out a
            message to an email address you provide (in
            <filename>cfg_local.pm</filename>). You should not have to touch
            this file.</para>
        </listitem>

        <listitem>
          <para><filename>logcheck</filename> - this file parses the commit
            log message that committers provide, and attempts to sanitize it
            somewhat.</para>

          <note><para>This script depends on a local FreeBSD hack of
            <application>CVS</application>: this version reads the log message
            back in after this script has modified it. The stock version of
            <application>CVS</application> apparently does not, which would
            make this script useless.</para></note>
        </listitem>

        <listitem>
          <para><filename>loginfo</filename> - this file is used by
            <application>CVS</application> to control where log
            information is sent. You should not have to touch this
            file.</para>
        </listitem>

        <listitem>
          <para><filename>modules</filename> - this file retains its
            traditional meaning in <application>CVS</application>. You should
            remove the FreeBSD modules from the stock version. You can edit this
            as you wish. More information about this file is available in the
            <application>CVS</application> manual.</para>
        </listitem>

        <listitem>
          <para><filename>notify</filename> - this file is used by
            <application>CVS</application> in case someone sets a watch on a
            file. It is not used in the FreeBSD repository. You can edit this as
            you wish. More information about this file is available in the
            <application>CVS</application> manual.</para>
        </listitem>

        <listitem>
          <para><filename>options</filename> - this file is specific to
            the FreeBSD version of <application>CVS</application>. It contains
            the keyword to expand in revision headers. You should alter this to
            match the keyword you specified in
            <filename>cfg_local.pm</filename> (if you use that feature, which
            is FreeBSD specific for now).</para>
        </listitem>

        <listitem>
          <para><filename>rcsinfo</filename> - this file maps directories in
            the repository to template files. By default, FreeBSD uses one
            template for the whole repository. You can add others to this file
            if you wish.</para>
        </listitem>

        <listitem>
          <para><filename>rcstemplate</filename> - this file is the actual
            template committers will see when they make a checkin. You should
            edit this to describe the various extra parameters you defined in
            <filename>cfg_local.pm</filename>.</para>
        </listitem>

        <listitem>
          <para><filename>tagcheck</filename> - this files controls access
            to tagging in the repository. The stock FreeBSD version disallows
            tags with names of RELENG*, because of the release engineering
            process. You should edit this file as desired.</para>
        </listitem>

        <listitem>
          <para><filename>taginfo</filename> - this file maps tag operations
            on repository directories to access control scripts. You should not
            have to touch this file.</para>
        </listitem>

        <listitem>
          <para><filename>unwrap</filename> - this script is needed to
            automatically <quote>unwrap</quote> binary files (see
            <filename>cvswrappers</filename>) on checkout. It is not used in the
            current FreeBSD setup. You should not have to touch this
            file.</para>
        </listitem>

        <listitem>
          <para><filename>verifymsg</filename> - this file maps repository
            directories to post processor scripts of log messages. You should
            not have to touch this file.</para>
        </listitem>

        <listitem>
          <para><filename>wrap</filename> - this script is needed to
            automatically <quote>wrap</quote> binary files (see
            <filename>cvswrappers</filename>) on checkin. It is not used in the
            current FreeBSD setup. You should not have to touch this
            file.</para>
        </listitem>
      </itemizedlist>
    </sect2>

    <sect2>
      <title>Customizing the scripts</title>

      <para>The next step is to setup the scripts so that they work in
        your environment. You should go over all files in the directory and
        make your customizations. In particular, you might want to do edit the
        following files:</para>

      <procedure>
        <step>
          <para>If you do not wish to use the <link linkend="freebsdspecific">
            FreeBSD specific features</link> of the scripts, you can safely
            remove the <filename>access</filename> file:

            <screen>&prompt.user; <userinput>cvs rm -f access</userinput></screen></para>
        </step>

        <step>
          <para>Edit <filename>avail</filename> to contain the various
            repository directories in which you want to control access. Make
            sure you retain the <literal>avail||CVSROOT</literal> line,
            otherwise you will lock yourself out in the next step.</para>

          <para>The other thing you can add in this file are committer groups.
            By default, FreeBSD uses the <filename>access</filename> file to
            list all its committers in, but you can use any file you wish. You
            can also add groups if you want (the syntax is specified at the
            top of <filename>cvs_acls.pl</filename>).</para>
        </step>

        <step>
          <para>Edit <filename>cfg_local.pm</filename> to contain the options
            you want. In particular, you should take a look at the following
            configurable items:

            <itemizedlist>
              <listitem>
                <para><literal>%TEMPLATE_HEADERS</literal> - these get
                  processed by the log scripts, and inserted below the
                  commit mail if present and non-empty in the commit
                  message. You can probably remove the <literal>PR</literal>
                  and <literal>MFC after</literal> entries. And of course
                  you can add your own.</para>
              </listitem>

              <listitem>
                <para><literal>$MAIL_BRANCH_HDR</literal> - if you want
                  to insert a header into each commit mail describing the
                  branch on which the commit was made, define this to match
                  your setup. Or leave it empty if you don't want such a
                  header.</para>
              </listitem>

              <listitem>
                <para><literal>@COMMIT_HOSTS</literal> - define this to
                  be a list of hosts on which people can commit.</para>
              </listitem>

              <listitem>
                <para><literal>$MAILADDRS</literal> - set this to the
                  admin or list address that should receive commit mail.</para>
              </listitem>

              <listitem>
                <para><literal>@LOG_FILE_MAP</literal> - change this array
                  as you wish - each regexp is matched on the directory of
                  the commit, and the commit log message gets stored in
                  the <filename>commitlogs</filename> subdirectory in
                  the filename mentioned.</para>
              </listitem>

              <listitem>
                <para><literal>$COMMITCHECK_EXTRA</literal> - if you do not
                  want to use <link linkend="freebsdspecific">the FreeBSD
                  specific access checks</link>, you should remove the
                  definition of <literal>$COMMITCHECK_EXTRA</literal> from
                  this file.</para>
              </listitem>
            </itemizedlist>

          <note><para>Changing the <literal>$IDHEADER</literal> parameter
            is only guaranteed to work on FreeBSD platforms; it depends on
            FreeBSD specific modifications to
            <application>CVS</application>.</para></note>

          You can check <filename>cfg.pm</filename> to see which other
          options can be changed, but the above is a reasonable subset.</para>
        </step>

        <step>
          <para>Edit <filename>exclude</filename> to remove the FreeBSD specific
            entries (such as all lines beginning with <literal>^ports/</literal>
            etc.). Furthermore, comment out the lines beginning with
            <literal>^CVSROOT/</literal>, and add one line with only
            <literal>^CVSROOT/</literal> on it. After the wrapper is
            installed, you can add your header to the files in the
            <filename>CVSROOT</filename> directory and restore these lines,
            but for now they will only be in the way when you try to commit
            later on.</para>
        </step>

        <step>
          <para>Edit <filename>modules</filename>, and delete all FreeBSD
            stuff. Add your own modules if you wish.</para>
        </step>

        <step>
          <note><para>This step is only necessary if you specified a
            value for <literal>$IDHEADER</literal> in
            <filename>cfg_local.pm</filename> (which only works using a
            FreeBSD modified <application>CVS</application>).</para></note>

          <para>Edit <filename>options</filename> to match the tag you
            specified in <filename>cfg_local.pm</filename>. A global
            search and replace of <literal>FreeBSD</literal> with your
            tag should suffice.</para>
        </step>

        <step>
          <para>Edit <filename>rcstemplate</filename> to contain the same
            keywords as specified in <filename>cfg_local.pm</filename>.</para>
        </step>

        <step>
          <para>Optionally remove the FreeBSD checks from
             <filename>tagcheck</filename>. You can simply add
             <literal>exit 0</literal> to the top of the file to disable all
             checks on tagging.</para>
        </step>

        <step>
          <para>The last thing to do before you are finished, is to make sure
            the commitlogs can be stored. By default these are stored in
            the repository, in the <filename>commitlogs</filename> subdirectory
            of the <filename>CVSROOT</filename> directory. This directory
            needs to be created, so do the following:

            <screen>&prompt.user; <userinput>mkdir commitlogs</userinput>
&prompt.user; <userinput>cvs add commitlogs</userinput></screen></para>
        </step>
      </procedure>

      <para>Now, after careful review, you should commit your changes. Be
        sure that you have granted yourself access to the
        <filename>CVSROOT</filename> directory in your
        <filename>avail</filename> before you do this, because otherwise you
        will lock yourself out. So make sure everything is as you intend, and
        then do the following:

        <screen>&prompt.user; <userinput>cvs commit -m '<replaceable>- Initial FreeBSD scripts commit</replaceable>'</userinput></screen></para>
    </sect2>

    <sect2>
      <title>Testing the setup</title>

      <para>You are ready for the first test: a forced commit to the
        <filename>avail</filename> file, to make sure everything works as
        expected.

        <screen>&prompt.user; <userinput>cvs commit -f -m '<replaceable>Forced commit to test the new CVSROOT scripts</replaceable>' avail</userinput></screen>

        If everything works, congratulations! You now have a working setup
        of the FreeBSD scripts for your repository. If
        <application>CVS</application> still complains about something, go
        back and recheck if all of the above steps have been performed
        correctly.</para>
    </sect2>
  </sect1>

  <sect1 id="freebsdspecific">
    <title>FreeBSD specific setup</title>

    <para>The FreeBSD project itself uses a slightly different setup, which
      also uses files from the <filename>freebsd</filename> subdirectory of
      the FreeBSD <filename>CVSROOT</filename>. The project uses this because
      of the large number of committers, which all would have to be in the
      same group. So, a simple wrapper was written which ensures that people
      have the correct credentials to commit, and then sets the group id
      to that of the repository.</para>

    <para>If your repository also needs this, the steps to set this up are
      documented below. But first an overview of the files involved.</para>

    <sect2>
      <title>Files used in the FreeBSD setup</title>

      <para>
        <itemizedlist>
          <listitem>
            <para><filename>access</filename> - this file controls access
              information. You should edit this file to include all members
              of your project.</para>
          </listitem>

          <listitem>
            <para><filename>freebsd/commitmail.pl</filename> - this file is
              not used any more, but kept for historic reasons. You should not
              have to touch this file.</para>
          </listitem>

          <listitem>
            <para><filename>freebsd/cvswrap.c</filename> - this is the source
              to the CVS wrapper that you will need to install to make all
              access checks actually work. More information on this below. You
              should edit the paths in the <literal>ACCESS</literal> and
              <literal>REALCVS</literal> macros to match your setup.</para>
          </listitem>

          <listitem>
            <para><filename>freebsd/mailsend.c</filename> - this file is
              needed by the FreeBSD setup of the mailing lists. You should
              not have to touch this file.</para>
          </listitem>
        </itemizedlist>
      </para>
    </sect2>

    <sect2>
      <title>The procedure</title>

      <procedure>
        <step>
          <para>Edit the <filename>access</filename> file to contain only
            your username.</para>
        </step>

        <step>
          <para>Edit <filename>cvswrap.c</filename> to contain the
            correct path for your setup. This is defined in a macro named
            <literal>ACCESS</literal>. You should also change the location of
            the real <command>cvs</command> binary if it is not appropriate to
            your situation.  The stock <filename>cvswrap.c</filename> expects
            to be a replacement for the systemwide cvs command, which will be
            moved to <filename>/usr/bin/ncvs</filename>.</para>

          <para>My copy of <filename>cvswrap.c</filename> has this:</para>

          <programlisting>#define ACCESS "/local/cvsroot/CVSROOT/access"
#define REALCVS "/usr/bin/ncvs"</programlisting>
        </step>

        <step>
          <para>Next up is installing the wrapper to ensure you become the
            correct group when committing. The sources for this live in
            <filename>cvswrap.c</filename> in your
            <filename>CVSROOT</filename>.</para>

          <para>Compile the sources that you edited to include the correct
            paths:

            <screen>&prompt.user; <userinput>cc -o cvs cvswrap.c</userinput></screen>

            And then install them (you have to be root for this step):

            <screen>&prompt.root; <userinput>mv /usr/bin/cvs /usr/bin/ncvs</userinput>
&prompt.root; <userinput>mv cvs /usr/bin/cvs</userinput>
&prompt.root; <userinput>chown root:<replaceable>ncvs</replaceable> /usr/bin/cvs /usr/bin/ncvs</userinput>
&prompt.root; <userinput>chmod o-rx /usr/bin/ncvs</userinput>
&prompt.root; <userinput>chmod u-w,g+s /usr/bin/cvs</userinput></screen>

            This installs the wrapper as the default <command>cvs</command>
            command, making sure that anyone who wants to use the repository
            has to have the correct access levels.</para>
        </step>

        <step>
          <para>You can now remove everyone from your repository group. All
            access control is done by your wrapper, and this wrapper will
            set the correct group for access.</para>
      </procedure>
    </sect2>

    <sect2>
      <title>Testing the setup</title>

      <para>Your wrapper should now be setup. You can of course test this by
        making a forced commit to the <filename>access</filename> file:

        <screen>&prompt.user; <userinput>cvs commit -f -m '<replaceable>Forced commit to test the new CVSROOT scripts</replaceable>' access</userinput></screen>

        Again, if this fails, check to see whether all of the above steps have
        been executed correctly.</para>
    </sect2>
  </sect1>
</article>
