ea2dde38df
Remove old a.out information. Update the policy to match our current practice of rarely, if ever, bumping the major version of libraries, but instead relying on symbol versioning to maintain a stable ABI. Move from paragraph form to bullet points. Also, tweak the 'once per major release' policy to 'as needed' because the conditions that existed for the former no longer apply. Discussed with: jhb@ Reviewed by: kib@, debdrub@ Differential Revision: https://reviews.freebsd.org/D28486
356 lines
17 KiB
Text
356 lines
17 KiB
Text
---
|
|
title: Chapter 5. Source Tree Guidelines and Policies
|
|
authors:
|
|
- author: Poul-Henning Kamp
|
|
- author: Giorgos Keramidas
|
|
prev: books/developers-handbook/l10n
|
|
next: books/developers-handbook/testing
|
|
---
|
|
|
|
[[policies]]
|
|
= Source Tree Guidelines and Policies
|
|
:doctype: book
|
|
:toc: macro
|
|
:toclevels: 1
|
|
:icons: font
|
|
:sectnums:
|
|
:sectnumlevels: 6
|
|
:source-highlighter: rouge
|
|
:experimental:
|
|
:skip-front-matter:
|
|
:xrefstyle: basic
|
|
:relfileprefix: ../
|
|
:outfilesuffix:
|
|
:sectnumoffset: 5
|
|
|
|
include::shared/mirrors.adoc[]
|
|
include::shared/authors.adoc[]
|
|
include::shared/releases.adoc[]
|
|
include::shared/en/mailing-lists.adoc[]
|
|
include::shared/en/teams.adoc[]
|
|
include::shared/en/urls.adoc[]
|
|
|
|
toc::[]
|
|
|
|
This chapter documents various guidelines and policies in force for the FreeBSD source tree.
|
|
|
|
[[policies-style]]
|
|
== Style Guidelines
|
|
|
|
Consistent coding style is extremely important, particularly with large projects like FreeBSD. Code should follow the FreeBSD coding styles described in man:style[9] and man:style.Makefile[5].
|
|
|
|
[[policies-maintainer]]
|
|
== `MAINTAINER` on Makefiles
|
|
|
|
If a particular portion of the FreeBSD [.filename]#src/# distribution is being maintained by a person or group of persons, this is communicated through an entry in [.filename]#src/MAINTAINERS#. Maintainers of ports within the Ports Collection express their maintainership to the world by adding a `MAINTAINER` line to the [.filename]#Makefile# of the port in question:
|
|
|
|
[.programlisting]
|
|
....
|
|
MAINTAINER= email-addresses
|
|
....
|
|
|
|
[TIP]
|
|
====
|
|
|
|
For other parts of the repository, or for sections not listed as having a maintainer, or when you are unsure who the active maintainer is, try looking at the recent commit history of the relevant parts of the source tree. It is quite often the case that a maintainer is not explicitly named, but the people who are actively working in a part of the source tree for, say, the last couple of years are interested in reviewing changes. Even if this is not specifically mentioned in the documentation or the source itself, asking for a review as a form of courtesy is a very reasonable thing to do.
|
|
====
|
|
|
|
The role of the maintainer is as follows:
|
|
|
|
* The maintainer owns and is responsible for that code. This means that he or she is responsible for fixing bugs and answering problem reports pertaining to that piece of the code, and in the case of contributed software, for tracking new versions, as appropriate.
|
|
* Changes to directories which have a maintainer defined shall be sent to the maintainer for review before being committed. Only if the maintainer does not respond for an unacceptable period of time, to several emails, will it be acceptable to commit changes without review by the maintainer. However, it is suggested that you try to have the changes reviewed by someone else if at all possible.
|
|
* It is of course not acceptable to add a person or group as maintainer unless they agree to assume this duty. On the other hand it does not have to be a committer and it can easily be a group of people.
|
|
|
|
[[policies-contributed]]
|
|
== Contributed Software
|
|
|
|
Some parts of the FreeBSD distribution consist of software that is actively being maintained outside the FreeBSD project. For historical reasons, we call this _contributed_ software. Some examples are sendmail, gcc and patch.
|
|
|
|
Over the last couple of years, various methods have been used in dealing with this type of software and all have some number of advantages and drawbacks. No clear winner has emerged.
|
|
|
|
Since this is the case, after some debate one of these methods has been selected as the "official" method and will be required for future imports of software of this kind. Furthermore, it is strongly suggested that existing contributed software converge on this model over time, as it has significant advantages over the old method, including the ability to easily obtain diffs relative to the "official" versions of the source by everyone (even without direct repository access). This will make it significantly easier to return changes to the primary developers of the contributed software.
|
|
|
|
Ultimately, however, it comes down to the people actually doing the work. If using this model is particularly unsuited to the package being dealt with, exceptions to these rules may be granted only with the approval of the link:https://www.FreeBSD.org/administration/#t-core[Core Team] and with the general consensus of the other developers. The ability to maintain the package in the future will be a key issue in the decisions.
|
|
|
|
[NOTE]
|
|
====
|
|
Because it makes it harder to import future versions minor, trivial and/or cosmetic changes are _strongly discouraged_ on files that are still tracking the vendor branch.
|
|
====
|
|
|
|
[[vendor-import-git]]
|
|
=== Vendor Imports with Git
|
|
|
|
This section describes the vendor import procedure with Git in detail.
|
|
|
|
==== Branch naming convention
|
|
All vendor branches and tags start with `vendor/`. These branches and
|
|
tags are visible by default.
|
|
|
|
[NOTE]
|
|
====
|
|
This chapter follows the convention that the `freebsd` origin is the source of truth. If you use a different convention, replace `freebsd` with your name as appopriate.
|
|
====
|
|
|
|
|
|
We'll explore an example for updating NetBSD's mtree that's in our
|
|
tree. The vendor branch for this is `vendor/NetBSD/mtree`.
|
|
|
|
==== Updating an old vendor import
|
|
Since the trees we have in vendor branches are usually a tiny subset of
|
|
the FreeBSD, it's best to do them with work trees since the process is
|
|
quite fast. Make sure that whatever directory you choose (the
|
|
`../mtree`) argument is empty and doesn't conflict.
|
|
[source,bash]
|
|
....
|
|
% git worktree add ../mtree vendor/NetBSD/mtree
|
|
....
|
|
==== Update the Sources in the Vendor Branch
|
|
|
|
Prepare a full, clean tree of the vendor sources. Import everything but merge only what is needed.
|
|
|
|
I have my copy of NetBSD checked out from their GitHub mirror in
|
|
`~/git/NetBSD`, so I'll update from there: Note that "upstream" might
|
|
have added or removed files, so we want to make sure deletions are
|
|
propagated as well. rsync(1) is commonly installed, so I'll use that.
|
|
[source,bash]
|
|
....
|
|
% cd ../mtree
|
|
% rsync -va --del --exclude=".git" ~/git/NetBSD/usr.sbin/mtree/ .
|
|
% git add -A
|
|
% git status
|
|
...
|
|
% git diff --staged
|
|
...
|
|
% git commit -m"Vendor import of NetBSD's mtree at 2020-12-11"
|
|
[vendor/NetBSD/mtree 8e7aa25fcf1] Vendor import of NetBSD's mtree at 2020-12-11
|
|
7 files changed, 114 insertions(+), 82 deletions(-)
|
|
% git tag -a vendor/NetBSD/mtree/20201211
|
|
....
|
|
|
|
Note: I run the `git diff` and `git status` commands to make sure nothing weird
|
|
was present. Also I used `-m` to illustrate, but you should compose a proper
|
|
message in an editor (using a commit message template).
|
|
|
|
It's also important to create an annotated tag, otherwise the push
|
|
will be rejected. Only annotated tags are allowed to be pushed. The
|
|
annoated tag gives you a chance to enter a commit message. Enter
|
|
the version you are importing, along with any salient new features
|
|
or fixes in that version.
|
|
|
|
==== Updating the FreeBSD Copy
|
|
At this point you can push the import to vendor into our repo.
|
|
[source,bash]
|
|
....
|
|
% git push --follow-tags freebsd vendor/NetBSD/mtree
|
|
....
|
|
|
|
`--follow-tags` tells `git push` to also push tags associated with the locally committed revision.
|
|
==== Updating the FreeBSD source tree
|
|
Now you need to update the mtree in FreeBSD. The sources live in
|
|
`contrib/mtree` since it is upstream software.
|
|
[source,bash]
|
|
....
|
|
% cd ../src
|
|
% git subtree merge -P contrib/mtree vendor/NetBSD/mtree
|
|
....
|
|
This would generate a subtree merge commit of `contrib/mtree` against the local `vendor/NetBSD/mtree` branch. If there were conflicts, you would need to fix them before committing.
|
|
|
|
==== Rebasing your change against latest FreeBSD source tree
|
|
Because the current policy recommends against using merges, if the upstream FreeBSD `main` moved forward
|
|
before you get a chance to push, you would have to redo the merge.
|
|
|
|
Regular `git rebase` or `git pull --rebase` doesn't know how to rebase a merge commit **as a merge commit**,
|
|
so instead of that you would have to recreate the commit.
|
|
|
|
The easiest way to do this would be to create a side branch with the **contents** of the merged tree:
|
|
|
|
[source,bash]
|
|
....
|
|
% cd ../src
|
|
% git fetch freebsd
|
|
% git checkout -b merge_result
|
|
% git merge freebsd/main
|
|
....
|
|
|
|
Typically, there would be no merge conflicts here (because developers tends to work on different components).
|
|
In the worst case scenario, you would still have to resolve merge conflicts, if there was any, but this
|
|
should be really rare.
|
|
|
|
Now, checkout `freebsd/main` again as `new_merge`, and redo the merge:
|
|
[source,bash]
|
|
....
|
|
% git checkout -b new_merge freebsd/main
|
|
% git subtree merge -P contrib/mtree vendor/NetBSD/mtree
|
|
....
|
|
Instead of resolving the conflicts, perform this instead:
|
|
[source,bash]
|
|
....
|
|
% git checkout merge_result .
|
|
....
|
|
Which will overwrite the files with conflicts with the version found in `merge_result`.
|
|
|
|
Examine the tree against `merge_result` to make sure that you haven't missed deleted files:
|
|
[source,bash]
|
|
....
|
|
% git diff merge_result
|
|
....
|
|
==== Pushing the changes
|
|
Once you are sure that you have a set of deltas you think is good, you can push it to
|
|
a fork off github or gitlab for others to review. Once nice thin about Git is that it
|
|
allows you to publish rough drafts of your work for others to review.
|
|
|
|
After review, when you are sure it is a good change, you can push it to the FreeBSD repo:
|
|
[source,bash]
|
|
....
|
|
% git push freebsd main
|
|
....
|
|
|
|
=== Creating a new vendor branch
|
|
There's a number of ways to create a new vendor branch. The recommended way is
|
|
to create a new repository and then merge that with FreeBSD. Let's say
|
|
we're importing `glorbnitz` into the FreeBSD tree, release 3.1415. For
|
|
the sake of simplicity, we'll not trim this release. It's a user
|
|
command that puts the nitz device into different magical glorb states.
|
|
|
|
==== Create the repo
|
|
[source,bash]
|
|
....
|
|
% cd /some/where
|
|
% mkdir glorbnitz
|
|
% cd glorbnitz
|
|
% git init
|
|
% git checkout -b vendor/glorbnitz
|
|
....
|
|
At this point, you have a new repo, where all new commits will go on
|
|
the `vendor/glorbnitz` branch.
|
|
|
|
(Git professionals can also do this right in their FreeBSD clone, if they know
|
|
how to create a new root commit that's not attached to anything, e.g.
|
|
`git checkout --orphan vendor/glorbnitz`)
|
|
|
|
==== Copy the sources in
|
|
Since this is a new import, you can just cp the sources in, or use tar or
|
|
even rsync as shown above. And we'll add everything, assuming no dot files.
|
|
[source,bash]
|
|
....
|
|
% cp -r ~/glorbnitz/* .
|
|
% git add *
|
|
....
|
|
|
|
At this point, you should have a pristine copy of glorbnitz ready to commit.
|
|
|
|
[source,bash]
|
|
....
|
|
% git commit -m"Import GlorbNitz frobnosticator revision 3.1415"
|
|
....
|
|
As above, I used `-m` for simplicity, but you should likely create a
|
|
commit message that explains what a Glorb is and why you'd use a Nitz
|
|
to get it. Not everybody will know.
|
|
|
|
==== Now import it into our repository
|
|
Now you need to import the branch into our repository.
|
|
[source,bash]
|
|
....
|
|
% cd /path/to/freebsd/repo/src
|
|
% git remote add glorbnitz /some/where/glorbnitz
|
|
% git fetch glorbnitz vendor/glorbnitz
|
|
....
|
|
Note the vendor/glorbnitz branch is in the repo. At this point the
|
|
`/some/where/glorbnitz` can be deleted, if you like. It was only a means
|
|
to an end.
|
|
|
|
==== Tag and push
|
|
Steps from here on out are much the same as they are in the case of
|
|
updating a vendor branch, though w/o the updating the vendor
|
|
branch step.
|
|
[source,bash]
|
|
....
|
|
% git worktree add ../glorbnitz vendor/glorbnitz
|
|
% cd ../glorbnitz
|
|
% git tag --annotate vendor/glorbnitz/3.1415
|
|
# Make sure it's good
|
|
% git push --follow-tags freebsd vendor/glorbnitz
|
|
....
|
|
By 'good' we mean:
|
|
|
|
. All the right files are present
|
|
. None of the wrong files are present
|
|
. The vendor branch points at something sensible
|
|
. The tag looks good, and is annotated.
|
|
|
|
==== Time to finally merge it into the base tree*
|
|
[source,bash]
|
|
....
|
|
% cd ../src
|
|
% git subtree add -P contrib/glorbnitz vendor/glorbnitz
|
|
# Make sure it's good
|
|
% git push freebsd
|
|
....
|
|
Here 'good' means:
|
|
|
|
. All the right files, and none of the wrong ones, were merged into contrib/glorbnitz.
|
|
. No other changes are in the tree
|
|
. The commit messages look good.
|
|
|
|
[NOTE]
|
|
====
|
|
This hasn't connected `glorbnitz` to the build yet. How so do that is specific to the software being imported.
|
|
====
|
|
[[policies-encumbered]]
|
|
== Encumbered Files
|
|
|
|
It might occasionally be necessary to include an encumbered file in the FreeBSD source tree. For example, if a device requires a small piece of binary code to be loaded to it before the device will operate, and we do not have the source to that code, then the binary file is said to be encumbered. The following policies apply to including encumbered files in the FreeBSD source tree.
|
|
|
|
. Any file which is interpreted or executed by the system CPU(s) and not in source format is encumbered.
|
|
. Any file with a license more restrictive than BSD or GNU is encumbered.
|
|
. A file which contains downloadable binary data for use by the hardware is not encumbered, unless (1) or (2) apply to it. It must be stored in an architecture neutral ASCII format (file2c or uuencoding is recommended).
|
|
. Any encumbered file requires specific approval from the link:https://www.FreeBSD.org/administration/#t-core[Core Team] before it is added to the repository.
|
|
. Encumbered files go in [.filename]#src/contrib# or [.filename]#src/sys/contrib#.
|
|
. The entire module should be kept together. There is no point in splitting it, unless there is code-sharing with non-encumbered code.
|
|
. Object files are named [.filename]#arch/filename.o.uu#.
|
|
. Firmware files are named [.filename]#filename.fw.uu#.
|
|
. Kernel files:
|
|
.. Should always be referenced in [.filename]#conf/files.*# (for build simplicity).
|
|
.. Should always be in [.filename]#LINT#, but when the consensus isn't clear the link:https://www.FreeBSD.org/administration/#t-core[Core Team] decides if it should be commented out or not.
|
|
.. The link:https://www.FreeBSD.org/administration/#t-re[Release Engineering] team decides whether or not it goes into the release.
|
|
|
|
. User-land files:
|
|
.. The link:https://www.FreeBSD.org/administration/#t-core[Core team] decides if the code should be part of `make world` by default if there is no clear consensus.
|
|
.. The link:https://www.FreeBSD.org/administration/#t-re[Release Engineering] team decides if it goes into the release.
|
|
|
|
[[policies-shlib]]
|
|
== Shared Libraries
|
|
|
|
If you are adding shared library support to a port or other piece of software that does not have one, the version numbers should follow these rules. Generally, the resulting numbers will have nothing to do with the release version of the software.
|
|
|
|
FreeBSD tries to maintain ABI stability. Most libraries in the system use symbol versioning to remain compatible with old versions of the library. When updating libraries with versioned symbols:
|
|
|
|
. Create new symbols in the symbol map for otherwise incompatible changes.
|
|
. When deleting old functions, only remove visibility of symbols from newer versions (that is, do not delete functions).
|
|
. Avoid gratuitous rearrangement of structures. This will keep compatibility symbols to a minimum.
|
|
. If changes require a version bump, re-evaluate whether they are needed at all.
|
|
|
|
For project-maintained libraries, please try to follow the following guidelines for major version numbers:
|
|
|
|
. New libraries
|
|
.. Should start with major version 1, though third-party libraries may start at a different number if warranted
|
|
.. All new libraries should either be symvers or private unless there's some compelling reason to deviate.
|
|
. For non-symvers libraries, avoid changes that would require a version bump.
|
|
.. When making a change that requires one, seriously consider adopting symbol versions at the same time
|
|
. For symvers libraries, generally, avoid version bumps
|
|
. Changes to structures can be worked around by providing the old structure and old functions to access it (if the layout of the structure is embedded in its clients).
|
|
. Changes to function signatures can be implemented by creating compatibility versions of the function (often calling the new version of the function in a specific way).
|
|
. If the benefits of a version bump outweight the costs to do it, follow up to ensure compat packages are built, etc
|
|
. Carefully consider the impact of a major version bump. When a major version change cannot be avoided, add a version map to avoid future bumps.
|
|
. For libraries where the ABI cannot be guaranteed, make them FreeBSD private where possible. Third party libraries that have poor ABI guarantees are generally made private, though there are a few exceptions for historical reasons.
|
|
. It is strong policy not to change the ABI of dynamicly linked libararies in an non-backwards compatible way
|
|
. Every reasonable effort should be made to make the changes in a binary compatible manner using tools like symbol versioning
|
|
. When major version changes cannot be avoided, the policy is to change only between major releases.
|
|
.. Minimize the number of major version bumps where possible
|
|
.. Multiple bumps are allowed between releases where necessary
|
|
.. Where possible, batch multiple changes that each would require a major bump together
|
|
.. Ensure that the compatibility ports / packages are updated as appropriate
|
|
.. Request an exp-run of ports to understand the full impact of the change to the base system
|
|
. Any changes to major libraries must still allow build from source upgrades to succeed
|
|
|
|
Changing the ABI is allowed, otherwise FreeBSD cannot evolve. However, the project has generally chosen to pay the added cost of retaining ABI backward-compatibility.
|