diff --git a/en_US.ISO8859-1/articles/hubs/article.xml b/en_US.ISO8859-1/articles/hubs/article.xml index cbbd159860..2c800fbd31 100644 --- a/en_US.ISO8859-1/articles/hubs/article.xml +++ b/en_US.ISO8859-1/articles/hubs/article.xml @@ -57,7 +57,7 @@ and degree of completeness you want to mirror, a huge amount of disk space may be consumed. Also keep in mind that official mirrors are probably required to be - complete. The CVS repository and the web pages should + complete. The web pages should always be mirrored completely. Also note that the numbers stated here are reflecting the current state (at &rel2.current;-RELEASE/&rel.current;-RELEASE). Further development and @@ -67,10 +67,9 @@ Here are some approximate figures: - Full FTP Distribution: 1.1 TB - CVS repository: 5.4 GB - CTM deltas: 3.2 GB - Web pages: 463 MB + Full FTP Distribution: 1.4 TB + CTM deltas: 10 GB + Web pages: 1GB The current disk usage of FTP Distribution can be found at @@ -113,7 +112,7 @@ For a moderately visited site that offers - Rsync, you might + rsync, you might consider a current CPU with around 800MHz - 1 GHz, and at least 512MB RAM. This is probably the minimum you want for an official @@ -127,7 +126,7 @@ You also want to consider a fast disk subsystem. - Operations on the CVS repository require a fast + Operations on the SVN repository require a fast disk subsystem (RAID is highly advised). A SCSI controller that has a cache of its own can also speed up things since most of these services incur a @@ -181,17 +180,9 @@ ftp/twoftpd: As above. ftp/vsftpd: The very secure ftpd. - - ftp/wu-ftpd: The ftpd from Washington - University. It has become infamous, because of the huge - amount of security issues that have been found in it. - If you do choose to use this software be sure to - keep it up to date. - - - FreeBSD's ftpd, proftpd, - wu-ftpd and maybe ncftpd + FreeBSD's ftpd, proftpd + and maybe ncftpd are among the most commonly used FTPds. The others do not have a large userbase among mirror sites. One thing to consider is that you may need flexibility in limiting @@ -260,109 +251,6 @@ - - CVSup (desired for CVS repository) - - CVSup is a very efficient way of distributing files. - It works similar to rsync, but was specially designed for - use with CVS repositories. If you want to offer the - FreeBSD CVS repository, you really want to consider - offering it via CVSup. It is possible to offer - the CVS repository via AnonCVS, FTP, - rsync or HTTP, but - people would benefit much more from CVSup access. - CVSup was developed by &a.jdp.email;. - It is a bit tricky to install on non-FreeBSD platforms, - since it is written in Modula-3 and therefore requires - a Modula-3 environment. &a.jdp; has built a - stripped down version of M3 that is sufficient to - run CVSup, and can be installed much easier. - See Ezm3 - for details. Related ports are: - - - - net/cvsup: The native CVSup port (client and server) - which requires lang/ezm3 now. - - - net/cvsup-mirror: The CVSup mirror kit, which requires - net/cvsup-without-gui, and configures it mirror-ready. Some - site administrators may want a different setup though. - - - - - There are a few more like - net/cvsup-without-gui you might want to have - a look at. If you prefer a static binary package, take a look - here. - This page still refers to the S1G bug that was present - in CVSup. Maybe - John will set up a generic download-site to get - static binaries for various platforms. - - - It is possible to use CVSup to offer - any kind of fileset, not just CVS repositories, - but configuration can be complex. - CVSup is known to eat some CPU on both the server and the - client, since it needs to compare lots of files. - - - - AnonCVS (optional for CVS repository) - - If you have the CVS repository, you may want to offer - anonymous CVS access. A short warning first: - There is not much demand for it, - it requires some experience, and you need to know - what you are doing. - - - Generally there are two ways - to access a CVS repository remotely: via - pserver or via ssh - (we do not consider rsh). - For anonymous access, pserver is - very well suited, but some still offer ssh - access as well. There is a custom crafted - wrapper - in the CVS repository, to be used as a login-shell for the - anonymous ssh account. It does a chroot, and therefore - requires the CVS repository to be available under the - anonymous user's home-directory. This may not be possible - for all sites. If you just offer pserver - this restriction does not apply, but you may run with - more security risks. You do not need to install any special - software, since &man.cvs.1; comes with - FreeBSD. You need to enable access via inetd, - so add an entry into your /etc/inetd.conf - like this: - -cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --allow-root=/home/ncvs pserver - - See the manpage for details of the options. Also see the CVS info - page about additional ways to make sure access is read-only. - It is advised that you create an unprivileged account, - preferably called anoncvs. - Also you need to create a file passwd - in your /home/ncvs/CVSROOT and assign a - CVS password (empty or anoncvs) to that user. - The directory /anoncvstmp is a special - purpose memory based file system. It is not required but - advised since &man.cvs.1; creates a shadow directory - structure in your /tmp which is - not used after the operation but slows things - dramatically if real disk operations are required. - Here is an excerpt from /etc/fstab, - how to set up such a MFS: - -/dev/da0s1b /anoncvstmp mfs rw,-s=786432,-b=4096,-f=512,-i=560,-c=3,-m=0,nosuid 0 0 - - This is (of course) tuned a lot, and was suggested by &a.jdp.email;. - - @@ -374,8 +262,8 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all the various parts of FreeBSD, what tools to use, and where to mirror from. - - FTP + + Mirroring the FTP site The FTP area is the largest amount of data that needs to be mirrored. It includes the distribution @@ -388,39 +276,8 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all for various FreeBSD versions, and various architectures. - - With FTP mirror - You can use a FTP mirror - program to get the files. Some of the most commonly used are: - - ftp/mirror - ftp/ftpmirror - ftp/emirror - ftp/spegla - ftp/omi - ftp/wget - - - ftp/mirror was very popular, but seemed - to have some drawbacks, as it is written in &man.perl.1;, - and had real problems with mirroring large - directories like a FreeBSD site. There are rumors that - the current version has fixed this by allowing - a different algorithm for comparing - the directory structure to be specified. - - - In general FTP is not really good for mirroring. It transfers - the whole file if it has changed, and does - not create a single data stream which would benefit from - a large TCP congestion window. - - - - With rsync - - A better way to mirror the FTP area is rsync. + The best way to mirror the FTP area is rsync. You can install the port net/rsync and then use rsync to sync with your upstream host. rsync is already mentioned @@ -440,7 +297,7 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all A command line to mirror FreeBSD might look like: - &prompt.user; rsync -vaz --delete ftp4.de.FreeBSD.org::FreeBSD/ /pub/FreeBSD/ + &prompt.user; rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/ Consult the documentation for rsync, which is also available at @@ -453,188 +310,15 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all want to set up a script framework that calls such a command via &man.cron.8;. - - - With CVSup - - A few sites, including the one-and-only ftp-master.FreeBSD.org - even offer CVSup to mirror the contents of - the FTP space. You need to install a CVSup - client, preferably from the port net/cvsup. - (Also reread .) - A sample supfile suitable for ftp-master.FreeBSD.org - looks like this: - - # - # FreeBSD archive supfile from master server - # - *default host=ftp-master.FreeBSD.org - *default base=/usr - *default prefix=/pub - #*default release=all - *default delete use-rel-suffix - *default umask=002 - - # If your network link is a T1 or faster, comment out the following line. - #*default compress - - FreeBSD-archive release=all preserve - - - It seems CVSup would be the best - way to mirror the archive in terms of efficiency, but - it is only available from few sites. - - - Please have look at the CVSup documentation - like &man.cvsup.1; and consider using the - option. This reduces I/O operations by assuming the - recorded information about each file is correct. - - - - - Mirroring the CVS repository - There are various ways to mirror the CVS repository. - CVSup is the most common method. - - - Using CVSup - - CVSup is described in some - detail in and . - - It is very easy to setup a - CVSup mirror. Installing - net/cvsup-mirror will - make sure all of the needed programs are installed and then - gather all the needed information to configure the mirror. - - - Please do not forget to consider the hint - mentioned in this note - above. - - - - - Using other methods - - Using other methods than CVSup is - generally not recommended. We describe them in short here - anyway. Since most sites offer the CVS repository as - part of the FTP fileset under the path - /pub/FreeBSD/development/FreeBSD-CVS, - the following methods could be used. - - FTP - Rsync - HTTP - - - - AnonCVS cannot be used to mirror the CVS repository - since CVS does not allow you to access the repository - itself, only checked out versions of the modules. - - Mirroring the WWW pages - The best way is to check out the www - distribution from CVS. If you have a local mirror of the - CVS repository, it is as easy as: - &prompt.user; cvs -d /home/ncvs co www - and a cronjob, that calls cvs up -d -P - on a regular basis, maybe just after your repository was updated. - Of course, the files need to remain in a directory available - for public WWW access. The installation and configuration of a - web server is not discussed here. - - - - If you do not have a local repository, you can use - CVSup to maintain an up to date copy - of the www pages. A sample supfile can be found in - /usr/share/examples/cvsup/www-supfile and - could look like this: - - # - # WWW module supfile for FreeBSD - # - *default host=cvsup3.de.FreeBSD.org - *default base=/usr - *default prefix=/usr/local - *default release=cvs tag=. - *default delete use-rel-suffix - - # If your network link is a T1 or faster, comment out the following line. - *default compress - - # This collection retrieves the www/ tree of the FreeBSD repository - www - - - Using ftp/wget or other web-mirror tools is - not recommended. - - - Mirroring the FreeBSD documentation - - Since the documentation is referenced a lot from the - web pages, it is recommended that you mirror the - FreeBSD documentation as well. However, this is not - as trivial as the www-pages alone. - - - First of all, you should get the doc sources, - again preferably via CVSup. - Here is a corresponding sample supfile: - - # - # FreeBSD documentation supfile - # - *default host=cvsup3.de.FreeBSD.org - *default base=/usr - *default prefix=/usr/share - *default release=cvs tag=. - *default delete use-rel-suffix - - # If your network link is a T1 or faster, comment out the following line. - #*default compress - - # This will retrieve the entire doc branch of the FreeBSD repository. - # This includes the handbook, FAQ, and translations thereof. - doc-all - - - Then you need to install a couple of ports. - You are lucky, there is a meta-port: - textproc/docproj to do the work - for you. You need to set up some - environment variables, like - SGML_CATALOG_FILES. - Also have a look at your /etc/make.conf - (copy /usr/share/examples/etc/make.conf if - you do not have one), and look at the - DOC_LANG variable. - Now you are probably ready to run make - in your doc directory (/usr/share/doc - by default) and build the documentation. - Again you need to make it accessible for your web server - and make sure the links point to the right location. - - - The building of the documentation, as well as lots - of side issues, is documented itself in the - &os; Documentation - Project Primer. - Please read this piece of documentation, especially if you - have problems building the documentation. - - - + The FreeBSD website should only be mirrored via + rsync. + A command line to mirror the FreeBSD web site might look like: + &prompt.user; rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/ + How often should I mirror? @@ -683,7 +367,6 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all Here are some recommended schedules: FTP fileset: daily - CVS repository: hourly WWW pages: daily @@ -708,7 +391,7 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all N is a number, telling that the host would be the Nth mirror in that country. - (Same applies to cvsupN.CC.FreeBSD.org, + (Same applies to wwwN.CC.FreeBSD.org, etc.) There are mirrors with no CC part. These are the mirror sites that are very well connected and @@ -812,9 +495,7 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all that. - There are three master sites for the FTP fileset and - one for the CVS repository (the web pages and docs are - obtained from CVS, so there is no need for master). + There is one master site for the FTP fileset. ftp-master.FreeBSD.org @@ -823,11 +504,9 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all ftp-master.FreeBSD.org provides - rsync and CVSup + rsync access, in addition to FTP. - Refer to and - how to access - via these protocols. + Refer to . Mirrors are also encouraged to allow rsync @@ -835,32 +514,6 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all Tier-1-mirrors. - - cvsup-master.FreeBSD.org - - This is the master site for the CVS repository. - - - cvsup-master.FreeBSD.org provides - CVSup access only. - See for details. - - - To get access, you need to contact the &a.cvsup-master;. - Make sure you read the - FreeBSD CVSup Access Policy - first! - - - Set up the required authentication by following - these - instructions. Make sure you specify the server as - freefall.FreeBSD.org on the cvpasswd - command line, as described in this document, - even when you are contacting - cvsup-master.FreeBSD.org - - @@ -897,14 +550,6 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --all what official tier-1 mirrors are required to. All other official mirrors can consider this a big should. - - - The following applies mainly to the FTP fileset, - since a CVS repository should always be mirrored - completely, and the web pages are a case of - its own. - - Tier-1 mirrors are required to: