From 6d1356c4ef14e1ac70b29b3ee959883193c3976a Mon Sep 17 00:00:00 2001 From: Max Laier Date: Sat, 23 Jul 2005 13:16:32 +0000 Subject: [PATCH] Add the March-June 2005 Status Report. --- en/news/status/Makefile | 3 +- en/news/status/report-2005-03-2005-06.xml | 2171 ++++++++++++++++++ en/news/status/report-mar-2005-june-2005.xml | 2171 ++++++++++++++++++ en/news/status/status.sgml | 6 +- 4 files changed, 4349 insertions(+), 2 deletions(-) create mode 100644 en/news/status/report-2005-03-2005-06.xml create mode 100644 en/news/status/report-mar-2005-june-2005.xml diff --git a/en/news/status/Makefile b/en/news/status/Makefile index 037c77337c..25c6e6fef5 100644 --- a/en/news/status/Makefile +++ b/en/news/status/Makefile @@ -1,4 +1,4 @@ -# $FreeBSD: www/en/news/status/Makefile,v 1.26 2005/01/17 08:53:20 scottl Exp $ +# $FreeBSD: www/en/news/status/Makefile,v 1.27 2005/04/21 17:55:46 scottl Exp $ .if exists(../Makefile.conf) .include "../Makefile.conf" @@ -30,6 +30,7 @@ DATA+= report-mar-2004-apr-2004.html DATA+= report-may-2004-june-2004.html DATA+= report-july-2004-dec-2004.html DATA+= report-jan-2005-mar-2005.html +DATA+= report-mar-2005-june-2005.html # Install a sample entry. DATA+= report-sample.xml diff --git a/en/news/status/report-2005-03-2005-06.xml b/en/news/status/report-2005-03-2005-06.xml new file mode 100644 index 0000000000..07acbdbc67 --- /dev/null +++ b/en/news/status/report-2005-03-2005-06.xml @@ -0,0 +1,2171 @@ + + + March-June + + 2005 + + +
+ Introduction + +

The second quarter of 2005 has again been very exciting. The + BSDCan and MeetBSD conferences were both very interesting and and the + sources of very good times. I highly recommend attending them again + next year.

+ +

The Google Summer of Code project has also generated quite a bit + of excitement. FreeBSD has been granted 18 funded mentorship spots, + the fourth most of all of participating organizations. Projects being + worked on range from UFS Journalling to porting the new BSDInstaller + to redesigning the venerable www.FreeBSD.org website. We are quite + pleased to be working with so many talented students, and eagerly + await the results of their work. More information and status can be + found at the Wiki site at + + http://wikitest.freebsd.org/moin.cgi/SummerOfCode2005 + + .

+ +

The FreeBSD 6.0 release cycle is also starting up. The purpose of + quickly jumping from 5.x to 6.0 is to reduce the amount of transition + pain that most users and developers felt when switching from 4-STABLE + to 5.x. 6.0 will feature improved performance and stability over 5.x, + experimental PowerPC support, and many new WiFi/802.11 features. The + 5.x series will continue for at least one more release this fall, and + will then be supported by the security team for at least 2 years + after that. We encourage everyone to give the 6.0-BETA snapshots a + try and help us make it ready for production. We hope to release + FreeBSD 6.0 by the end of August.

+ +

Thanks again to everyone who submitted reports, and thanks to Max + Laier for running the show and putting the reports together. Enjoy + reading!

+
+ + + soc + + Google summer of code + + + + proj + + Projects + + + + doc + + Documentation + + + + kern + + Kernel + + + + net + + Network infrastructure + + + + bin + + Userland programs + + + + arch + + Architectures + + + + ports + + Ports + + + + vendor + + Vendor / 3rd Party Software + + + + misc + + Miscellaneous + + + + BSDCan + + + + + Dan + + Langille + + + dan@langille.org + + + + + + + + +

The second annual + BSDCan + + conference was well presented, well attended, and everyone went + away with good stories to tell. If you know anything that attended, + get them to tell you what they did, who they met with, and talks + they listened to.

+ +

We had 197 people from 15 different countries. That's a strong + turnout by any definition.

+ +

We'll be adding more people to the program committee for BSDCan + 2006. This job involves prodding and poking people from your + respective projects. You get them to submit papers. There are a lot + of very interesting projects out there and not all of them submit a + paper.

+ +

If you know someone doing interesting work, please let me know + and urge them to start thinking about BSDCan 2006.

+ + + + + Integrate the BSD Installer into FreeBSD + + + + + Andrew + + Turner + + + soc-andrew@FreeBSD.org + + + + + The BSD Installer + + BSD + Installer Wiki page + + + BSD Installer Perforce tree + + + +

Progress towards integrating the BSD Installer for Google's + Summer of Code is coming along nicely. The installation CD will + boot to multi-user mode and run both the front and back ends. It + can then partition a hard drive, install the base distribution and + make the disk bootable.

+ + + + Test in non-i386 + + Investigate installing from other media + + Many more tasks + + + + + FreshPorts + + + + + Dan + + Langille + + + dan@langille.org + + + + + + + + +

The following new features have been added to FreshPorts:

+ + + + + + I've noticed that FreshPorts is incorrectly reporting + vulnerabilities under a + + very specific situation + + . The fix is sitting in BETA, waiting to be moved to + production. + + I've been working on added Last-Modified to the headers. At + present, there are none. Most of the pages on the BETA website have + been completed. I need to move this to production soon. + + Customized news feeds are in the works. You'll be able to + create a news feed for each of your watch lists This work is + contingent upon finishing the Last-Modified headers. + + + + + Fundraising - TCP & IP Routing Optimization + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + +

The TCP code in FreeBSD has evolved significantly since the fork + from 4.4BSD-Lite2 in 1994 primarily due to new features and + refinements of the TCP specifications.

+ +

The TCP code now needs a general overhaul, streamlining and + cleanup to make it easily comprehensible, maintainable and + extensible again. In addition there are many little optimizations + that can be done during such an operation, propelling FreeBSD back + at the top of the best performing TCP/IP stacks again, a position + it has held for the longest time in the 90's.

+ +

This overhaul is a very involved and delicate matter and needs + extensive formal and actual testing to ensure no regressions + compared to the current code. The effort needed for this work is + about three man-month of fully focused and dedicated time. To get + it done I need funding to take time off my day job and to dedicate + me to FreeBSD work much the way PHK did with his buffer cache and + vnode rework projects.

+ +

I've got the opportunity to work up to three man-month + exclusively full-time on FreeBSD during the second half of 2005. + That means up to 720 hours of full-steam coding (at 60 hours/week)! + I will work as much time as the fundraise provides.

+ +

I need to raise enough money for each month from donations from + the FreeBSD community to cover my fixed cost of living, office and + associated overhead. These fixed cost amount to US$6,300/month + (EUR5,200 or CHF8,000). Yes, Switzerland is not the cheapest place + to live. :)

+ +

A detailed description of the tasks involved and the code I will + write is on my FreeBSD website; Follow the link above.

+ + + + Raise enough money to get all the almost finished TCP and IP + code into the tree. + + + + + CPU Cache Prefetching + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + + + +

Modern CPU's can only perform to their maximum if their working + code is in fast L1-3 cache memory instead of the bulk main memory. + All of today's CPU's support certain L1-3 cache prefetching + instructions which cause data to be retrieved from main memory to + the cache ahead of the time that it is already in place when it is + eventually accessed by the CPU.

+ +

CPU Cache Prefetching however is not a silver bullet and has to + be used with extreme care and only in very specific places to be + beneficial. Incorrect usage can lead to massive cache pollution and + a drop in effective performance. Correct and very carefully usage + on the other can lead to drastic performance increases in common + operations.

+ +

In the linked patch CPU cache prefetching has been used to + prefetch the packet header (OSI layer 2 to 4) into the CPU caches + right after entering into the network stack. This avoids a complete + CPU stall on the first access to the packet header because packets + get DMA'd into main memory and thus never are already pre-cache in + the CPU caches. A second use in the patch is in the TCP input code + to prefetch the entire struct tcpcb which is very large and used + with a very high probability. Use in both of these places show a + very significant performance gain but not yet fully quantified.

+ +

The final patch will include documentation and a guide to + evaluate and assess the use of CPU cache prefetch instructions in + the kernel.

+ + + + Need funding, see "Fundraising - TCP & IP Routing + Optimization". + + + + + TCP Reassembly Rewrite and Optimization + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + + + + + +

Currently TCP segment reassembly is implemented as a linked list + of segments. With today's high bandwidth links and large + bandwidth*delay products this doesn't scale and perform well.

+ +

The rewrite optimizes a large number of operational aspects of + the segments reassembly process. For example it is very likely that + the just arrived segment attaches to the end of the reassembly + queue, so we check that first. Second we check if it is the missing + segment or alternatively attaches to the start of the reassembly + queue. Third consecutive segments are merged together (logically) + and are skipped over in one jump for linear searches instead of + each segment at a time.

+ +

Further optimizations prototyped merge consecutive segments on + the mbuf level instead of only logically. This is expected to give + another significant performance gain. The new reassembly queue is + tracking all holes in the queue and it may be beneficial to + integrate this with the scratch pad of SACK in the future.

+ +

Andrew Gallatin was able to get 3.7Gb/sec TCP performance on + dual-2Gbit Myrinet cards with severe packet reordering (due to a + firmware bug) with the new TCP reassembly code. See second + link.

+ + + + Need funding, see "Fundraising - TCP & IP Routing + Optimization". + + + + + TTCPv2: Transactional TCP version 2 + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + + + +

The old TTCP according to RFC1644 was insecure, intrusive, + complicated and has been removed from FreeBSD >= 5.3. Although + the idea and semantics behind it are still sound and valid.

+ +

The rewrite uses a much easier and more secure system with 24bit + long client and server cookies which are transported in the TCP + options. Client cookies protect against various kinds of blind + injection attacks and can be used as well to generally secure TCP + sessions (for BGP for example). Server cookies are only exchanged + during the SYN-SYN/ACK phase and allow a server to ensure that it + has communicated with this particular client before. The first + connection is always performing a 3WHS and assigning a server + cookie to a client. Subsequent connections can send the cookie back + to the server and short-cut the 3WHS to SYN->OPEN on the + server.

+ +

TTCPv2 is fully configurable per-socket via the setsockopt() + system call. Clients and server not capable of TTCPv2 remain fully + compatible and just continue using the normal 3WHS without any + delay or other complications.

+ +

Work on implementing TTCPv2 is done to 90% and expected to be + available by early February 2005. Writing the implementation + specification (RFC Draft) has just started.

+ + + + Need funding, see "Fundraising - TCP & IP Routing + Optimization". + + + + + Network Interface API Cleanup + + + + + Anders + + Persson + + + soc-anders@freebsd.org + + + + + + + + +

The goal of this project is to review the network interface API + and try to remove references to kernel-only data structures by + removing the use of libkvm and instead rely on other interfaces to + provide information. If there are no adequate interfaces, they + would be created.

+ +

Currently netstat is being reviewed and parts of it have been + modified to use sysctl rather than libkvm to provide the + information.

+ +

A big thank you to Brooks Davis for mentoring :-)

+ + + + + FreeBSD Security Officer and Security Team + + + + + Security + + Officer + + + security-officer@FreeBSD.org + + + + + Security + + Team + + + security-team@FreeBSD.org + + + + + + + + + + + + +

In May 2005, Remko Lodder joined the FreeBSD Security Team, + followed by Christian S.J. Peron in July 2005. In the same time + period, Gregory Shapiro and Josef El-Rayes resigned from the team + in order to devote their time to other projects. The current + Security Team membership is published on the web site.

+ +

In the time since the last FreeBSD status report, twelve + security advisories have been issued concerning problems in the + base system of FreeBSD; of these, six problems were in + "contributed" code, while five problems were in code maintained + within FreeBSD. The Vulnerabilities and Exposures Markup Language + (VuXML) document has continued to be updated by the Security Team + and the Ports Committers documenting new vulnerabilities in the + FreeBSD Ports Collection; since the last status report, 97 new + entries have been added, bringing the total up to 519.

+ +

The following FreeBSD releases are supported by the FreeBSD + Security Team: FreeBSD 4.10, FreeBSD 4.11, FreeBSD 5.3, and FreeBSD + 5.4. Their respective End of Life dates are listed on the web + site.

+ + + + + Dingo + + + + + Several + + + + + + somewhat out of + date + + + +

Currently trying to restart bits of the project. Cleaning up the + p4 branch. Recently more people have volunteered to help as well. + Brooks Davis has completed removing the ifnet from the softc.

+ + + + See the web page. + + + + + The FreeBSD Dutch Documentation Project + + + + + Remko + + Lodder + + + remko@FreeBSD.org + + + + + Siebrand + + Mazeland + + + siebrand.mazeland@xs4all.nl + + + + + Rene + + Ladan + + + r.c.ladan@student.tue.nl + + + + + The Dutch + Handbook + + The + Dutch Project Site + + The Dutch + Preview Documentation + + The Dutch + FreeBSD Flyer + + + +

The FreeBSD Dutch Documentation Project is a ongoing project in + translating the english documentation to the Dutch language. + Currently we are almost done with the FreeBSD Handbook. Finishing + the Handbook is our first priority, and we could use your help. + Please contact Siebrand or myself if you want to helpout. After the + handbook we will focus on other documents as well, so feel free to + help us there as well

+ + + + FreeBSD Handbook translation. Finish the translation from + English to Dutch + + FreeBSD Handbook review. Finish the review of the translated + documents + + FreeBSD Articles. Start translating the articles from English + to the Dutch Language + + FreeBSD www. Start translating the website from English to + the Dutch Language + + The rest of the FreeBSD Documents. Start translating them + from English to the Dutch Language. + + + + + Transparent support for superpages in the FreeBSD + Kernel + + + + + Alan L. + + Cox + + + alc@cs.rice.edu + + + + + Olivier + + Crameri + + + olivier.crameri@epfl.ch + + + + + + + +

We are currently working on an updated implementation of + Juan + Navarro's transparent support for superpages in FreeBSD +

+ +

The idea is to take advantage of the architectural support for + big memory pages (superpages) by using a reservation mechanism + allowing us to transparently promote groups of base pages into + superpages and demote superpages into several smaller superpages or + base pages.

+ +

The advantage of using superpages vs. base pages is to + significantly improve the TLB coverage of the physical memory, thus + improving the peformance by reducing the number of TLB misses.

+ +

The modification of the FreeBSD kernel that we are working on + involves the replacement of the current list based page allocation + mechanism with a system using a buddy allocator to reserve groups + of pages for a memory object. The promotion and demotion of the + pages occur directly within the pmap module.

+ +

The former implementation was supporting the alpha and IA64 + architectures. We are adding the support for amd64. We currently + have an almost complete implementation. Once completed we will make + a performance study with a particular emphasis on TLB and cache + misses.

+ + + + + Wireless Networking Support + + + + + Sam + + Leffler + + + sam@freebsd.org + + + + + + + +

A lot of bugs were fixed in preparation for the 6.0 release. 6.0 + will be the first release to include full WPA support (both + supplicant and authenticator).

+ +

A presentation on the forthcoming multi-bss support was given at + BSDCan 2005. The slides from the talk are available at + + http://www.freebsd.org/~sam/BSDCan2005.pdf + + . The plan is to commit this work to HEAD after 6.0 is released + which means the first release that will have it is 7.0.

+ + + + hostapd needs work to support the IAPP and 802.11i + preauthentication protocols (these are simple conversions of + existing Linux code). + + + + + FreeSBIE toolkit integration + + + + + Dario + + Freni + + + saturnero@freesbie.org + + + + + FreeSBIE main site + + My page + on FreeBSD wiki + + + +

My Summer of Code project is reengineering and rewrite of + FreeSBIE toolkit, in order to include it in the source tree. Let's + call it FreeSBIE 2

+ +

Before being accepted, I worked hard on the FreeSBIE 1 toolkit + to make it more flexible. It now supports amd64 and powerpc + architecture. The built filesystem can now boot from almost every + media, from dvd to compact flash or hard disk. Also on i386 is + possible to include the BSD Installer on the livefs. We've received + reports that our toolkit is successfully used for install cd of + pfSense + + and + PC-BSD + + projects.

+ +

My future goals are to make the toolkit even more flexible, + capable to build embedded images (like nanoBSD) or big Live-DVD + systems, depending on user's choice, to support all the + architectures supported by FreeBSD and to write a set of tools for + making a netboot server with a FreeSBIE image.

+ + + + + PowerPC Port + + + + + Peter + + Grehan + + + grehan@FreeBSD.org + + + + + FreeBSD/PPC + Platform page. + + + +

Florent Thoumie has updated the massively out-of-date platform + page. Work continues to creating a 6.0 release of the PowerPC + port.

+ + + + + GEOM Gate rewrite + + + + + Pawel Jakub + + Dawidek + + + pjd@FreeBSD.org + + + + + + + + + + +

GGATE is a machinism for exporting storage devices over the + network. It was reimplemented to be much faster and to handle + network failures better. The ggatec uses two threads now: sendtd, + which takes I/O request from the kernel and sends it to ggated; + recvtd, which receives finished requests and forwards them to the + kernel. The ggated uses three threads: recvtd, which receives I/O + requests from ggatec; disktd, which executes I/O requests (reads or + writes data); sendtd, which sends finished requests to ggatec. The + new ggate has been committed to 6.x.

+ +

The work was sponsored by + Wheel Sp. z o.o. +

+ + + + + gjournal + + + + + Ivan + + Voras + + + ivoras@gmail.com + + + + + gjournal + wiki + + + +

The schedule (as stated on the wiki page) is honoured, which + means that the development has started, but there's not enough code + for testing. Many details have been thought-out and the development + is ongoing.

+ + + + + FreeBSD Summer of Code + + + + + Summer of Code + + Mentors + + + soc-mentors@FreeBSD.org + + + + + + + + +

Google has generously funded 18 students to spend the summer + working on FreeBSD related projects. Each student is working with + one or more mentors to learn about how open source software + development is done with FreeBSD. This development work is + happening in the Perforce repository as //depot/projects/soc2005. + This tree will soon be exported via CVSup -- check the Wiki for + more information.

+ + + + + gvinum 'move', 'rename' + + + + + Chris + + Jones + + + soc-cjones@freebsd.org + + + + + + gvinum 'move', 'rename' wiki entry + + + +

With the releases of FreeBSD 5.3 and 5.4, FreeBSD has been + moving away from "old-style" vinum towards GEOM-enabled gvinum for + logical volume management. While gvinum is a mostly + feature-complete replacement for vinum, it does not implement the + 'move' or 'rename' verbs which are rather useful when reorganizing + one's volume layout, the alternative being a tedious process of + deleting and recreating subdisks, plexes, or volumes. Additionally, + gvinum is nearly completely undocumented, which contributes to the + perception of gvinum as an unfinished project.

+ +

I'm working on implementing 'move' (being able to move a subdisk + from one drive to another) and 'rename' (being able to rename an + subdisk, plex, volume, or drive), as well as on documentation for + gvinum.

+ +

So far, I've come up with a plan of attack with le@ and phk@, + and implemented the bulk of the userland code for gvinum 'move' and + 'rename'. Still to come are the kernel-side code and + documentation.

+ + + + 'move' and 'rename' userland implementation + + 'move' and 'rename' kernel-side implementation + + Outline new handbook section and man page + + Implement new handbook section and man page + + + + + if_bridge + + + + + Andrew + + Thompson + + + thompsa@freebsd.org + + + + + + + +

This was committed to current on 5 Jun 2005 and will first + appear in the 6.0 release, thanks to everyone who tested. Recent + improvements include:

+ +
    +
  • IPFW layer2 filtering
    • + +
  • DUMMYNET support
    • + +
  • IP header alignment checking
    • +
+ +

There is ongoing work to bring in some of the advanced features + from OpenBSD such as IPSec bridging. People are encouraged to use + if_bridge and report any problems to the mailing lists.

+ + + + + IPv6 Support for IPFW + + + + + Max + + Laier + + + mlaier@freebsd.org + + + + + Brooks + + Davis + + + brooks@freebsd.org + + + + + + + +

At the developer summit before BSDCan it was decided to remove + IP6FW from the tree as it has a couple of problems. The most + pressing one is the lack of synchronization and thus the need for + debug.mpsafenet=0. As a replacement Brooks Davis has imported + patches to teach the existing and well-locked IPFW2 code about + IPv6.

+ +

Since the initial import I have added some features required to + manage IPv4 and IPv6 in a single ruleset. I have also extended + existing opcodes to work with IPv6. There are, however, still some + opcodes that do not work with IPv6 and most of the more exotic ones + haven't been tested. As long as IPFW2+v6 does not provide enough + functionality and stability to work as a drop-in replacement for + IP6FW, we won't remove IP6FW.

+ +

In order to get the new code to that point we + really + + need more testers with real world IPv6 deployment and interest in + IPFW+v6. The lack thereof (I haven't received a single answer on my + requests to various FreeBSD mailing lists) has made it hard to + progress.

+ + + + Properly implement O_REJECT for IPv6 + + Maybe implement O_LOG + + Test new(er) IPFW2 opcodes with IPv6 + + Test + + Test + + Test + + + + + launchd(8) for FreeBSD + + + + + R. Tyler + + Ballance + + + tyler@tamu.edu + + + + + Wiki + Project Page + + + Apple's launchd(8) man page + + + +

So far progress has been slow, the autoconf build system has + been removed from all of the launchd(8) code, and launchctl(1) is + building and semi-functional on FreeBSD-CURRENT (i.e. + CoreFoundation hooks have been removed)

+ +

I'm currently working on porting "liblaunch" which is the core + backend to both launchd(8) (the actual daemon) and launchctl(1), + there are some mach/xnu specific hooks and calls that need to be + remove and either reimplemented or worked around.

+ +

We're also waiting on a response from Apple on a possible + BSD-licensed version of the code (it's currently under the APSL) + Progress is slow, but steady.

+ + + + + Removable interface improvements + + + + + Brooks + + Davis + + + brooks@FreeBSD.org + + + + + + + + + + +

This project is an attempt to clean up handling of network + interfaces in order to allow interfaces to be removed reliably. + Current problems include panics if Dummynet is delaying packets to + an interface when it is removed.

+ +

I have removed struct ifnet's and layer two common structures + from device driver structures. This will eventually allow them to + be managed properly upon device removal. This code has been + committed and will appear in 6.0. Popular drivers have generally + been fixed, but more testing is needed.

+ + + + + OpenBSD dhclient import. + + + + + Brooks + + Davis + + + brooks@FreeBSD.org + + + + + Sam + + Leffler + + + sam@FreeBSD.org + + + + + + + +

The OpenBSD rewrite of dhclient has been imported, replacing the + ISC dhclient. The OpenBSD client provides better support for + roaming on wireless networks and a simpler model of operation. + Instead of a single dhclient process per system, there is on per + network interface. This instance automatically goes away in the + even of link loss and is restarted via devd when link is + reacquired. To support this change, many aspects of the network + interface configuration process were overhauled.

+ +

The current code works well in most circumstances, but more + testing and polishing is needed.

+ + + + + Move ARP out of routing table + + + + + Qing + + Li + + + qingli@freebsd.org + + + + + + + + +

I've sent the patch to jinmei@isl.rdc.toshiba.co.jp @KAME for + review. I'm still waiting for feedback from Andre. There hasn't + been any major change since the last report. I've kept the code in + sync with CURRENT. Gleb has created a separate P4 branch and has + been helping out on the locking side. Gleb is also helping out on + the testing front.

+ + + + I'm waiting for review feedback from my mentor Andre on the + overall design and code. I'm waiting for feedback from Andre on + Gleb's suggested modification. + + + + + Nsswitch / Caching daemon + + + + + Michael + + Bushkov + + + soc-bushman@rsu.ru + + + + + + + + + + +

The + nsswitch / caching daemon + + project is being developed within the Google's Summer Of Code + program. The first goal of this project is to implement a set of + patches to extend the use of nsswitch subsystem. The second goal is + the development of the caching library and daemon to add the + caching ability to the nsswitch.

+ +

Currently services, protocols, rpc and openssh patches are + finished. Support for services, services_compat, rpc, protocols, + and ssh_host_keys databases is added with 'files', 'nis' and + 'compat' (for services) sources possible. The nsswitch-friendly + openssh port is amlost completed.

+ + + + Implement set of patches to make nsswitch support + globus grid security files + + , + MAC and audit related configuration files + + databases. + + Implement the caching library and the caching daemon and + patch nsdispatch function to support caching. + + + + + OpenBSD packet filter - pf + + + + + Max + + Laier + + + mlaier@freebsd.org + + + + + + + +

We will have pf as of OpenBSD 3.7 for RELENG_6. Import has been + completed in early May and FreeBSD release 6.0 will ship with + it.

+ +

A few serious issues with pfsync on SMP have been discovered + since CARP is around and more and more people use it on big iron. + Everything that has been discovered is fixed in HEAD and (if + applicable) MFCed back to RELENG_5. Some functional changes are + undergoing testing right now and will be MFCed in the coming + days.

+ +

With the import of if_bridge from Net/OpenBSD we finally have a + bridge implementation that allows for stateful filtering as well as + IPv6 filtering. Please see the respective report.

+ + + + Shared lock implementation? + + + + + Low-overhead performance monitoring for FreeBSD + + + + + Joseph + + Koshy + + + jkoshy@FreeBSD.org + + + + + + Project home page + + + +

Modern CPUs have on-chip performance monitoring counters (PMCs) + that may be used to count low-level hardware events like + instruction retirals, branch mispredictions, and cache misses. PMC + architectures and capabilities vary between CPU vendors and between + CPU generations from the same vendor, making the creation of + portable applications difficult. This project implements a + cross-platform PMC management API for applications, and implements + the infrastructure to "virtualize" and manage these PMCs. The + creation of performance analysis tools that use this infrastructure + is also part of the project's goals.

+ +

Work since the last status report:

+ +
    +
  • Sampling mode support for P4 and AMD64 PMCs has been + implemented.
    • + +
  • A pmclog(3) API for parsing hwpmc(4) log files has been + added.
    • + +
  • A number of bugs in libpmc(3), hwpmc(4) and pmcstat(8) have + been fixed.
    • +
+ +

Future work:

+ +
    +
  • Creating user documentation showing a few real-world uses of + the currently available tools.
    • + +
  • Testing, improving the stability of the code, and + characterizing its overheads.
    • + +
  • Implementing P5 PMC support.
    • +
+ + + + + Improve libalias + + + + + Paolo + + Pisati + + + soc-pisati@freebsd.org + + + + + Wiki + page about libalias work. + + + +

My SoC project is about improving libalias and integrating it + with ipfw2, adding nat support into the firewall. Till now i ported + libalias (as a kld) and ng_nat to 4.x and 5.x branches, and i've + already a first working patchset that adds 'nat' action into ipfw. + Next step will be to add a complete syntax to ipfw that will let us + manipulate libalias operations, much like we already do with queue + and pipes for dummynet. In the end the entire work will compile and + work out of the box for 4.x, 5.x and 6.x. More details about the + project and its status are available on wiki page.

+ + + + + TODO list for volunteers + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + + +

Since Google's "Summer of Code" resulted in a lot of interest in + open projects, I'm in the process of compiling a list of nice + projects for volunteers. Unlike Google's SoC those projects aren't + backed with money (but this doesn't means nobody is allowed to + sponsor one of those projects), so we can only guarantee the social + aspects (some "Thank you!" and "That's great!" messages). So far + the list has several entries where the difficulty ranges from + "someone just has to sit down and spend some time on it" up to "we + need a guru for this".

+ + + + Merging untaken entries from the SoC list as soon as the + official participants/tasks in the SoC are announced. + + Sending the document to some doc people for review. + + Commit the list. + + + + + Removing of old basesystem files and directories + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + Patch + + + +

FreeBSD lacks a way to remove old/outdated files and directories + in the basesystem. I have a patch which removes obsolete files in a + safe way (interactively, since only the administrator really knows + if there's a need to keed an old file or not; there's a switch for + batch-processing). This feature may or may not be available for + 6.0-RELEASE, depending on the decission from the Release + Engineering team.

+ + + + Respect the NO_* switches and remove those files too. This is + easy to do with the current implementation, but isn't needed to + commit the removal of obsolete files feature. + + + + + Porting v9 of Intels C/C++ Compiler + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + + +

Intel released version 9 of its C/C++ compiler. Work to port the + x86 version to FreeBSD is in progress as time permits. Porting the + EM64T (amd64) version is on the TODO list too, but is subject to + enough free time and access to appropriate hardware.

+ + + + + Update of the Linux userland infrastructure + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + Emulation + + Mailinglist + + + emulation@FreeBSD.org + + + + + + + +

The cleanup/streamlining and the possibility of overriding the + default Linux base as reported in the last report happened without + major problems. Work on the open tasks hasn't started yet, but is + scheduled to start "soon". If a volunteer wants to spend some hours + on one of the open tasks, he should tell it on the emulation + mailinglist.

+ + + + Refactoring the common RPM code in + x11-toolkits/linux-gtk/Makefile into bsd.rpm.mk. + + Determining which up-to-date Linux distribution to use as the + next default Linux base. Important criteria: +
    +
  • RPM based (to be able to use the existing + infrastructure)
    • + +
  • good track record regarding availability of security + fixes
    • + +
  • packages available from several mirror sites
    • + +
  • available for several hardware architectures (e.g. i386, + amd64, sparc64; Note: not all architectures have a working + linuxolator for their native bit with, but as long as there are + no userland bits available, no motivation regarding writing the + kernel bits will arise)
    • +
+ + + Moving the linuxolator userland to an up-to-date version (see + above). + + + + + Autotuning of the page queue coloring algorithm + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + Patch + + + +

The VM subsystem has code to reduce the amount of cache + collisions of VM pages. Currently this code needs to be tuned with + a kernel option. I have a patch which changes this to auto-tuning + at boot time. The auto-tuning is MI, the cache size detection is + MD. Cache size detection is currently available for x86/amd64 (on + other systems it uses default values).

+ + + + Add cache-detection code for other arches too (Marius told me + how to do this for sparc64). + + Analyze why the cache detection on Athlons doesn't work (no + problems on a P4, but it uses a different code-path). + + + + + FreeBSD website improvements + + + + + Emily + + Boyd + + + soc-emily@freebsd.org + + + + + + + +

As part of the Google Summer of Code, I'm working on + improvements to the FreeBSD website (including a proposed website + redesign). My mentor for this project is Murray Stokely.

+ + + + + UFSJ -- Journaling for UFS + + + + + Brian + + Wilson + + + polytopes@gmail.com + + + + + Scott + + Long + + + scottl@FreeBSD.org + + + + + + + +

filesystem. Journaling helps ensure the filesystem's integrity + should the system crash. Journaling eliminates the need for + fsck'ing a filesystem, as the filesystem is never in an + inconsistent state (barring hardware failure). This implementation + is inspired by Darwin's HFS+ filesystem and the SGI XFS filesystem. + This is a Summer of Code project, with Scott Long as the mentor and + Brian Wilson as the developer/mentee. Currently this project is + still in the early stages, but will be in a usable state by + September 1 (the Google Summer of Code completion date).

+ + + + Finish making the file system log metadata updates. + + Add facilities to replay the log on dirty file + systems. + + Make snapshots work with journaling. + + + + + SEBSD + + + + + Yanjun + + Wu + + + yanjun03@ios.cn + + + + + Show + status in wiki, update more frequently. + + + +
    +
  1. Setup a local P4 workspace of sebsd source and Setup lxr for + TrustedBSD source for studying source code.
    2. + +
  2. Test a simple policy configuration for vsftpd.
    3. + +
  3. Writing a HOWTO document + Getting Started with SEBSD HOWTO + + by deriving the existing + Getting Started with SELinux HOWTO + + .
    4. +
+ +

Thanks Robert Watson and Scott Long for their kind help.

+ + + + When writing the document, try to figure out the sebsd + userland utils that need to be ported. + + Test and edit more policies for BSD environment. + + + + + VFS SMP + + + + + Jeff + + Roberson + + + jeff@freebsd.org + + + + + + + +

FreeBSD's VFS layer has been fine grain locked along with the + FFS filesystem for the FreeBSD 6.0 release. The locking has been + underway for several years, with the project really picking up over + the last 6 months thanks largely to sponsorship provided by Isilon + Systems, Inc. a leading vendor of clustered storage systems. The + project has entered a stabilization phase, with a few bugs being + reported in extreme circumstances while the majority of users have + seen no problems. Tests on a 8 and 16 way machines yield reasonable + parallelization, however, it will be beneficial to do lock + contention analysis once things are fully stable.

+ +

For those interested in technical details, there have been a few + relatively significant changes with vnode life-cycle management. + Vnode reference counting and recycling is now no longer an ad-hoc + process involving a variety of flags, a use count and the hold + count. A single hold count is used to track all vnode references + and a destroyed vnode is freed in the context of the caller when + the last ref is lost. The old system would never reclaim memory + used by vnodes and also had pathlogical behavior with unreferenced + vnode caching under pressure. The new system is much simpler than + the old one, however, callers are now required to vhold a vnode + that they lock directly without going through vget to prevent it + from being recycled while they are waiting on a lock. Relying on + 'location stable storage', which is a more strict version of 'type + stable storage' is no longer a valid approach.

+ +

Some other side effects include a much simpler and faster nullfs + implementation, an improved buf daemon flushing algorithm which + eliminated high latency that caused audio skipping, and a lots of + minor cleanups and debugging aids.

+ + + + + EuroBSDCon 2005 - Basel + + + + + Information + + + info@eurobsdcon.org + + + + + Homepage + + Call for papers + + + +

The fourth European BSD conference in Basel, Switzerland is a + great opportunity to present new ideas to the community and to meet + some of the developers behind the different BSDs.

+ +

The two day conference program (Nov 26 and 27) will be + complemented by a tutorial day preceeding the conference (Nov + 25).

+ +

The program committee is looking for tutorial and paper + submissions. For details, please see: The + call for papers + + online.

+ + + + + SMP Network Stack + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + Netperf home + page + + + +

Significant work has occurred over the last few months relating + to the SMP network stack work. A few of the highlights are covered + here at a high level:

+ +
    +
  • The UMA(9) per-CPU caches have been modified to use critical + sections instead of mutexes. Recent critical section + optimizations make this a performance win for both UP and SMP + systems. This results in a several percent improvement in a + number of user space benchmarks, and larger improvement for + kernel-only network forwarding and processing benchmarks.
    • + +
  • The malloc(9) allocator has been modified to store statistics + per-CPU instead of using a cross-CPU statistics pool, with each + per-CPU pool now using critical sections to synchronize access. + This results in a measurable performance win, especially on SMP + systems
    • + +
  • The netnatm ATM code is now MPSAFE.
    • + +
  • netipx MPSAFEty has been merged to RELENG_5.
    • + +
  • The netperf cluster has now been expanded to include two + additional quad-CPU systems (one dual dual-core AMD system, one + quad-CPU PIII system).
    • + +
  • libmemsetat(3) (see separate report) now corrects SMP-related + races in the measuring of mbuf allocator statistics, as well as + substantially improving kernel memory monitoring capabilities and + tools.
    • + +
  • A range of locking bug fixes, and general network stack bug + fixes.
    • + +
  • Significant updates to the SMPng web page (still more to + do!).
    • + +
  • Identification of all non-MPSAFE network device drivers, with + ultimatum issued, on freebsd-arch. Quite a bit of new driver + locking work as a result (if_ed, if_de, ...).
    • + +
  • Lots of other stuff.
    • +
+ +

In most cases, these changes will appear in FreeBSD 6.0-RELEASE; + some have been, or will be, merged to FreeBSD 5.x.

+ +

On-going tasks include:

+ +
    +
  • Review and improvement of ifnet locking, such as address + lists and flags.
    • + +
  • Optimization of interface start hand-off.
    • + +
  • Prototyping of queue-oriented packet hand-off in the + stack.
    • + +
  • Performance measurement and analysis.
    • + +
  • Prototype rewrite and simplification of socket locking.
    • +
+ + + + + TrustedBSD SEBSD + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + TrustedBSD/SEBSD + web page + + + +

The TrustedBSD Project has released a new snapshot of "SEBSD", a + port of NSA's SELinux FLASK and Type Enforcement implementation to + FreeBSD based on a late 2005 FreeBSD 6.x snapshot. The SEBSD + distribution has now been updated in Perforce to a recent 6.x + snapshot, and a new distribution will be made available in the near + future.

+ +

Work has been performed to merge additional dependencies for + SEBSD back into the base FreeBSD tree, including most recently, + changes to devfs, and System V and POSIX IPC.

+ + + + Update to new NSA FLASK implementation, which has improved + MLS support. + + Merge remaining kernel changes to support SEBSD back to the + base FreeBSD CVS repository, including file descriptor labeling and + access control (in contrast to file labeling and access control), + and categorization of kernel privileges. + + + + + TrustedBSD Audit + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + Wayne + + Salamon + + + wsalmon@FreeBSD.org + + + + trustedbsd-discuss@TrustedBSD.org + + + + + + + + +

In the past few months, significant work has been done relating + to the TrustedBSD audit implementation, including preparatory work + to merge audit into the FreeBSD CVS repository for FreeBSD 6.x. In + particular:

+ +
    +
  • The user space components, such as libbsm, include files, and + command line utilities have been broken out into an OpenBSM + distribution in Perforce. Improvements in OpenBSM will be made + available separately for use by projects such as Darwin, and + imported into the contrib area of FreeBSD.
    • + +
  • The system call table format has been updated to include an + audit event identifier for each system call across all hardware + platforms and ABIs (merged), and all system calls have been + assigned event identifiers (not yet merged).
    • + +
  • The audit management daemon has been rewritten to run on + FreeBSD (originally derived from Darwin) using /dev/audit to + track kernel events.
    • + +
  • Many system calls now properly audit their arguments.
    • + +
  • The TrustedBSD audit3 branch has been updated to a recent + 6.x-CURRENT.
    • + +
  • Significant work has gone into synchronizing the audit event + tables between FreeBSD, Darwin, and OpenSolaris to make sure file + formats and events are portable.
    • + +
  • OpenBSM has been adapted to consume and generate + endian-independent event streams.
    • + +
  • OpenBSM documentation has been created.
    • +
+ +

The hope is still to provide audit as "experimental" in 6.0; the + primary blocking factor is our awaiting relicensing of the last + remaining audit files from Apple's APSL license to BSDL so that + they can be included in the FreeBSD kernel. This is anticipated to + complete in the near future. Once this is done, the changes can be + merged to CVS, and then MFC'd to RELENG_6. If this is not complete + by 6.0-RELEASE, the work will be merged shortly after the release, + as all ABI-sensitive data structures have been updated as + needed.

+ + + + + libmemstat(3), UMA(9) and malloc(9) statistics + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + + libmemstat(3)-derived tools + + + +

libmemstat(3) provides a user space library API to monitor + kernel memory allocators, currently uma(9) and malloc(9), with the + following benefits:

+ +
    +
  • ABI-robust interface making use of accessor functions, in + order to divorce monitoring applications from kernel/user ABI + changes.
    • + +
  • Allocator-independent interfaces, allowing monitoring of + multiple allocators using the same interface.
    • + +
  • CPU-cache awareness, allowing tracking of memory use across + multiple CPUs for allocators aware of caches. Unlike previous + interfaces, libmemstat(3) coalesces per-CPU stats in user space + rather than kernel, and exposes per-CPU stats to interested + applications.
    • + +
  • Ability to track memory types over multiple queries, and + update existing structures, allowing easy tracking of statistics + over time.
    • +
+ +

libmemstat(3) and the the appropriate allocator changes for + uma(9) and malloc(9) are currently in HEAD (7-CURRENT), and MFC has + been approved to RELENG_6 for inclusion in 6.0-RELEASE. These + changes may also be backported to 5.x.

+ +

Sample applications include memstat(8), an allocator-independent + statistics viewing tool, memtop(8), which provides a top(1)-like + interface for monitoring kernel memory use and active memory types. + None of these are "pretty".

+ +

netstat -mb has also been updated to use libmemstat(3) to track + network memory use using uma(9), rather than the less reliable mbuf + allocator statistics interface. As a result, the statistics are now + more reliable on SMP systems (this corrects the bug in which mbuf + statistics sometimes "leaked", even though memory didn't), and more + informative (cache information is now displayed, as well as mbuf + tag information).

+ + + + Teach libmemstat(3) to speak libkvm(3) in order to allow + tools linked -lmemstat to interogate kernel core dumps. + + Teach libmemstat(3) to interface with user space malloc and + track malloc allocations for user space applications. + + Update vmstat(8) -m and -z implementations to use + libmemstat(3) instead of the old monitoring interfaces. Code to do + this exists in the sample libmemstat(3) applications. + + Identify how to make streams or the library endian-aware so + that streams dumped from a kernel of alternative endian could be + processed using libmemstat(3) on another system. + + Identify any remaining caching allocators in the kernel, such + as the sfbuf allocator, and teach libmemstat(3) how to interface + with them. + + + + diff --git a/en/news/status/report-mar-2005-june-2005.xml b/en/news/status/report-mar-2005-june-2005.xml new file mode 100644 index 0000000000..07acbdbc67 --- /dev/null +++ b/en/news/status/report-mar-2005-june-2005.xml @@ -0,0 +1,2171 @@ + + + March-June + + 2005 + + +
+ Introduction + +

The second quarter of 2005 has again been very exciting. The + BSDCan and MeetBSD conferences were both very interesting and and the + sources of very good times. I highly recommend attending them again + next year.

+ +

The Google Summer of Code project has also generated quite a bit + of excitement. FreeBSD has been granted 18 funded mentorship spots, + the fourth most of all of participating organizations. Projects being + worked on range from UFS Journalling to porting the new BSDInstaller + to redesigning the venerable www.FreeBSD.org website. We are quite + pleased to be working with so many talented students, and eagerly + await the results of their work. More information and status can be + found at the Wiki site at + + http://wikitest.freebsd.org/moin.cgi/SummerOfCode2005 + + .

+ +

The FreeBSD 6.0 release cycle is also starting up. The purpose of + quickly jumping from 5.x to 6.0 is to reduce the amount of transition + pain that most users and developers felt when switching from 4-STABLE + to 5.x. 6.0 will feature improved performance and stability over 5.x, + experimental PowerPC support, and many new WiFi/802.11 features. The + 5.x series will continue for at least one more release this fall, and + will then be supported by the security team for at least 2 years + after that. We encourage everyone to give the 6.0-BETA snapshots a + try and help us make it ready for production. We hope to release + FreeBSD 6.0 by the end of August.

+ +

Thanks again to everyone who submitted reports, and thanks to Max + Laier for running the show and putting the reports together. Enjoy + reading!

+
+ + + soc + + Google summer of code + + + + proj + + Projects + + + + doc + + Documentation + + + + kern + + Kernel + + + + net + + Network infrastructure + + + + bin + + Userland programs + + + + arch + + Architectures + + + + ports + + Ports + + + + vendor + + Vendor / 3rd Party Software + + + + misc + + Miscellaneous + + + + BSDCan + + + + + Dan + + Langille + + + dan@langille.org + + + + + + + + +

The second annual + BSDCan + + conference was well presented, well attended, and everyone went + away with good stories to tell. If you know anything that attended, + get them to tell you what they did, who they met with, and talks + they listened to.

+ +

We had 197 people from 15 different countries. That's a strong + turnout by any definition.

+ +

We'll be adding more people to the program committee for BSDCan + 2006. This job involves prodding and poking people from your + respective projects. You get them to submit papers. There are a lot + of very interesting projects out there and not all of them submit a + paper.

+ +

If you know someone doing interesting work, please let me know + and urge them to start thinking about BSDCan 2006.

+ + + + + Integrate the BSD Installer into FreeBSD + + + + + Andrew + + Turner + + + soc-andrew@FreeBSD.org + + + + + The BSD Installer + + BSD + Installer Wiki page + + + BSD Installer Perforce tree + + + +

Progress towards integrating the BSD Installer for Google's + Summer of Code is coming along nicely. The installation CD will + boot to multi-user mode and run both the front and back ends. It + can then partition a hard drive, install the base distribution and + make the disk bootable.

+ + + + Test in non-i386 + + Investigate installing from other media + + Many more tasks + + + + + FreshPorts + + + + + Dan + + Langille + + + dan@langille.org + + + + + + + + +

The following new features have been added to FreshPorts:

+ + + + + + I've noticed that FreshPorts is incorrectly reporting + vulnerabilities under a + + very specific situation + + . The fix is sitting in BETA, waiting to be moved to + production. + + I've been working on added Last-Modified to the headers. At + present, there are none. Most of the pages on the BETA website have + been completed. I need to move this to production soon. + + Customized news feeds are in the works. You'll be able to + create a news feed for each of your watch lists This work is + contingent upon finishing the Last-Modified headers. + + + + + Fundraising - TCP & IP Routing Optimization + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + +

The TCP code in FreeBSD has evolved significantly since the fork + from 4.4BSD-Lite2 in 1994 primarily due to new features and + refinements of the TCP specifications.

+ +

The TCP code now needs a general overhaul, streamlining and + cleanup to make it easily comprehensible, maintainable and + extensible again. In addition there are many little optimizations + that can be done during such an operation, propelling FreeBSD back + at the top of the best performing TCP/IP stacks again, a position + it has held for the longest time in the 90's.

+ +

This overhaul is a very involved and delicate matter and needs + extensive formal and actual testing to ensure no regressions + compared to the current code. The effort needed for this work is + about three man-month of fully focused and dedicated time. To get + it done I need funding to take time off my day job and to dedicate + me to FreeBSD work much the way PHK did with his buffer cache and + vnode rework projects.

+ +

I've got the opportunity to work up to three man-month + exclusively full-time on FreeBSD during the second half of 2005. + That means up to 720 hours of full-steam coding (at 60 hours/week)! + I will work as much time as the fundraise provides.

+ +

I need to raise enough money for each month from donations from + the FreeBSD community to cover my fixed cost of living, office and + associated overhead. These fixed cost amount to US$6,300/month + (EUR5,200 or CHF8,000). Yes, Switzerland is not the cheapest place + to live. :)

+ +

A detailed description of the tasks involved and the code I will + write is on my FreeBSD website; Follow the link above.

+ + + + Raise enough money to get all the almost finished TCP and IP + code into the tree. + + + + + CPU Cache Prefetching + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + + + +

Modern CPU's can only perform to their maximum if their working + code is in fast L1-3 cache memory instead of the bulk main memory. + All of today's CPU's support certain L1-3 cache prefetching + instructions which cause data to be retrieved from main memory to + the cache ahead of the time that it is already in place when it is + eventually accessed by the CPU.

+ +

CPU Cache Prefetching however is not a silver bullet and has to + be used with extreme care and only in very specific places to be + beneficial. Incorrect usage can lead to massive cache pollution and + a drop in effective performance. Correct and very carefully usage + on the other can lead to drastic performance increases in common + operations.

+ +

In the linked patch CPU cache prefetching has been used to + prefetch the packet header (OSI layer 2 to 4) into the CPU caches + right after entering into the network stack. This avoids a complete + CPU stall on the first access to the packet header because packets + get DMA'd into main memory and thus never are already pre-cache in + the CPU caches. A second use in the patch is in the TCP input code + to prefetch the entire struct tcpcb which is very large and used + with a very high probability. Use in both of these places show a + very significant performance gain but not yet fully quantified.

+ +

The final patch will include documentation and a guide to + evaluate and assess the use of CPU cache prefetch instructions in + the kernel.

+ + + + Need funding, see "Fundraising - TCP & IP Routing + Optimization". + + + + + TCP Reassembly Rewrite and Optimization + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + + + + + +

Currently TCP segment reassembly is implemented as a linked list + of segments. With today's high bandwidth links and large + bandwidth*delay products this doesn't scale and perform well.

+ +

The rewrite optimizes a large number of operational aspects of + the segments reassembly process. For example it is very likely that + the just arrived segment attaches to the end of the reassembly + queue, so we check that first. Second we check if it is the missing + segment or alternatively attaches to the start of the reassembly + queue. Third consecutive segments are merged together (logically) + and are skipped over in one jump for linear searches instead of + each segment at a time.

+ +

Further optimizations prototyped merge consecutive segments on + the mbuf level instead of only logically. This is expected to give + another significant performance gain. The new reassembly queue is + tracking all holes in the queue and it may be beneficial to + integrate this with the scratch pad of SACK in the future.

+ +

Andrew Gallatin was able to get 3.7Gb/sec TCP performance on + dual-2Gbit Myrinet cards with severe packet reordering (due to a + firmware bug) with the new TCP reassembly code. See second + link.

+ + + + Need funding, see "Fundraising - TCP & IP Routing + Optimization". + + + + + TTCPv2: Transactional TCP version 2 + + + + + Andre + + Oppermann + + + andre@freebsd.org + + + + + + + + + + +

The old TTCP according to RFC1644 was insecure, intrusive, + complicated and has been removed from FreeBSD >= 5.3. Although + the idea and semantics behind it are still sound and valid.

+ +

The rewrite uses a much easier and more secure system with 24bit + long client and server cookies which are transported in the TCP + options. Client cookies protect against various kinds of blind + injection attacks and can be used as well to generally secure TCP + sessions (for BGP for example). Server cookies are only exchanged + during the SYN-SYN/ACK phase and allow a server to ensure that it + has communicated with this particular client before. The first + connection is always performing a 3WHS and assigning a server + cookie to a client. Subsequent connections can send the cookie back + to the server and short-cut the 3WHS to SYN->OPEN on the + server.

+ +

TTCPv2 is fully configurable per-socket via the setsockopt() + system call. Clients and server not capable of TTCPv2 remain fully + compatible and just continue using the normal 3WHS without any + delay or other complications.

+ +

Work on implementing TTCPv2 is done to 90% and expected to be + available by early February 2005. Writing the implementation + specification (RFC Draft) has just started.

+ + + + Need funding, see "Fundraising - TCP & IP Routing + Optimization". + + + + + Network Interface API Cleanup + + + + + Anders + + Persson + + + soc-anders@freebsd.org + + + + + + + + +

The goal of this project is to review the network interface API + and try to remove references to kernel-only data structures by + removing the use of libkvm and instead rely on other interfaces to + provide information. If there are no adequate interfaces, they + would be created.

+ +

Currently netstat is being reviewed and parts of it have been + modified to use sysctl rather than libkvm to provide the + information.

+ +

A big thank you to Brooks Davis for mentoring :-)

+ + + + + FreeBSD Security Officer and Security Team + + + + + Security + + Officer + + + security-officer@FreeBSD.org + + + + + Security + + Team + + + security-team@FreeBSD.org + + + + + + + + + + + + +

In May 2005, Remko Lodder joined the FreeBSD Security Team, + followed by Christian S.J. Peron in July 2005. In the same time + period, Gregory Shapiro and Josef El-Rayes resigned from the team + in order to devote their time to other projects. The current + Security Team membership is published on the web site.

+ +

In the time since the last FreeBSD status report, twelve + security advisories have been issued concerning problems in the + base system of FreeBSD; of these, six problems were in + "contributed" code, while five problems were in code maintained + within FreeBSD. The Vulnerabilities and Exposures Markup Language + (VuXML) document has continued to be updated by the Security Team + and the Ports Committers documenting new vulnerabilities in the + FreeBSD Ports Collection; since the last status report, 97 new + entries have been added, bringing the total up to 519.

+ +

The following FreeBSD releases are supported by the FreeBSD + Security Team: FreeBSD 4.10, FreeBSD 4.11, FreeBSD 5.3, and FreeBSD + 5.4. Their respective End of Life dates are listed on the web + site.

+ + + + + Dingo + + + + + Several + + + + + + somewhat out of + date + + + +

Currently trying to restart bits of the project. Cleaning up the + p4 branch. Recently more people have volunteered to help as well. + Brooks Davis has completed removing the ifnet from the softc.

+ + + + See the web page. + + + + + The FreeBSD Dutch Documentation Project + + + + + Remko + + Lodder + + + remko@FreeBSD.org + + + + + Siebrand + + Mazeland + + + siebrand.mazeland@xs4all.nl + + + + + Rene + + Ladan + + + r.c.ladan@student.tue.nl + + + + + The Dutch + Handbook + + The + Dutch Project Site + + The Dutch + Preview Documentation + + The Dutch + FreeBSD Flyer + + + +

The FreeBSD Dutch Documentation Project is a ongoing project in + translating the english documentation to the Dutch language. + Currently we are almost done with the FreeBSD Handbook. Finishing + the Handbook is our first priority, and we could use your help. + Please contact Siebrand or myself if you want to helpout. After the + handbook we will focus on other documents as well, so feel free to + help us there as well

+ + + + FreeBSD Handbook translation. Finish the translation from + English to Dutch + + FreeBSD Handbook review. Finish the review of the translated + documents + + FreeBSD Articles. Start translating the articles from English + to the Dutch Language + + FreeBSD www. Start translating the website from English to + the Dutch Language + + The rest of the FreeBSD Documents. Start translating them + from English to the Dutch Language. + + + + + Transparent support for superpages in the FreeBSD + Kernel + + + + + Alan L. + + Cox + + + alc@cs.rice.edu + + + + + Olivier + + Crameri + + + olivier.crameri@epfl.ch + + + + + + + +

We are currently working on an updated implementation of + Juan + Navarro's transparent support for superpages in FreeBSD +

+ +

The idea is to take advantage of the architectural support for + big memory pages (superpages) by using a reservation mechanism + allowing us to transparently promote groups of base pages into + superpages and demote superpages into several smaller superpages or + base pages.

+ +

The advantage of using superpages vs. base pages is to + significantly improve the TLB coverage of the physical memory, thus + improving the peformance by reducing the number of TLB misses.

+ +

The modification of the FreeBSD kernel that we are working on + involves the replacement of the current list based page allocation + mechanism with a system using a buddy allocator to reserve groups + of pages for a memory object. The promotion and demotion of the + pages occur directly within the pmap module.

+ +

The former implementation was supporting the alpha and IA64 + architectures. We are adding the support for amd64. We currently + have an almost complete implementation. Once completed we will make + a performance study with a particular emphasis on TLB and cache + misses.

+ + + + + Wireless Networking Support + + + + + Sam + + Leffler + + + sam@freebsd.org + + + + + + + +

A lot of bugs were fixed in preparation for the 6.0 release. 6.0 + will be the first release to include full WPA support (both + supplicant and authenticator).

+ +

A presentation on the forthcoming multi-bss support was given at + BSDCan 2005. The slides from the talk are available at + + http://www.freebsd.org/~sam/BSDCan2005.pdf + + . The plan is to commit this work to HEAD after 6.0 is released + which means the first release that will have it is 7.0.

+ + + + hostapd needs work to support the IAPP and 802.11i + preauthentication protocols (these are simple conversions of + existing Linux code). + + + + + FreeSBIE toolkit integration + + + + + Dario + + Freni + + + saturnero@freesbie.org + + + + + FreeSBIE main site + + My page + on FreeBSD wiki + + + +

My Summer of Code project is reengineering and rewrite of + FreeSBIE toolkit, in order to include it in the source tree. Let's + call it FreeSBIE 2

+ +

Before being accepted, I worked hard on the FreeSBIE 1 toolkit + to make it more flexible. It now supports amd64 and powerpc + architecture. The built filesystem can now boot from almost every + media, from dvd to compact flash or hard disk. Also on i386 is + possible to include the BSD Installer on the livefs. We've received + reports that our toolkit is successfully used for install cd of + pfSense + + and + PC-BSD + + projects.

+ +

My future goals are to make the toolkit even more flexible, + capable to build embedded images (like nanoBSD) or big Live-DVD + systems, depending on user's choice, to support all the + architectures supported by FreeBSD and to write a set of tools for + making a netboot server with a FreeSBIE image.

+ + + + + PowerPC Port + + + + + Peter + + Grehan + + + grehan@FreeBSD.org + + + + + FreeBSD/PPC + Platform page. + + + +

Florent Thoumie has updated the massively out-of-date platform + page. Work continues to creating a 6.0 release of the PowerPC + port.

+ + + + + GEOM Gate rewrite + + + + + Pawel Jakub + + Dawidek + + + pjd@FreeBSD.org + + + + + + + + + + +

GGATE is a machinism for exporting storage devices over the + network. It was reimplemented to be much faster and to handle + network failures better. The ggatec uses two threads now: sendtd, + which takes I/O request from the kernel and sends it to ggated; + recvtd, which receives finished requests and forwards them to the + kernel. The ggated uses three threads: recvtd, which receives I/O + requests from ggatec; disktd, which executes I/O requests (reads or + writes data); sendtd, which sends finished requests to ggatec. The + new ggate has been committed to 6.x.

+ +

The work was sponsored by + Wheel Sp. z o.o. +

+ + + + + gjournal + + + + + Ivan + + Voras + + + ivoras@gmail.com + + + + + gjournal + wiki + + + +

The schedule (as stated on the wiki page) is honoured, which + means that the development has started, but there's not enough code + for testing. Many details have been thought-out and the development + is ongoing.

+ + + + + FreeBSD Summer of Code + + + + + Summer of Code + + Mentors + + + soc-mentors@FreeBSD.org + + + + + + + + +

Google has generously funded 18 students to spend the summer + working on FreeBSD related projects. Each student is working with + one or more mentors to learn about how open source software + development is done with FreeBSD. This development work is + happening in the Perforce repository as //depot/projects/soc2005. + This tree will soon be exported via CVSup -- check the Wiki for + more information.

+ + + + + gvinum 'move', 'rename' + + + + + Chris + + Jones + + + soc-cjones@freebsd.org + + + + + + gvinum 'move', 'rename' wiki entry + + + +

With the releases of FreeBSD 5.3 and 5.4, FreeBSD has been + moving away from "old-style" vinum towards GEOM-enabled gvinum for + logical volume management. While gvinum is a mostly + feature-complete replacement for vinum, it does not implement the + 'move' or 'rename' verbs which are rather useful when reorganizing + one's volume layout, the alternative being a tedious process of + deleting and recreating subdisks, plexes, or volumes. Additionally, + gvinum is nearly completely undocumented, which contributes to the + perception of gvinum as an unfinished project.

+ +

I'm working on implementing 'move' (being able to move a subdisk + from one drive to another) and 'rename' (being able to rename an + subdisk, plex, volume, or drive), as well as on documentation for + gvinum.

+ +

So far, I've come up with a plan of attack with le@ and phk@, + and implemented the bulk of the userland code for gvinum 'move' and + 'rename'. Still to come are the kernel-side code and + documentation.

+ + + + 'move' and 'rename' userland implementation + + 'move' and 'rename' kernel-side implementation + + Outline new handbook section and man page + + Implement new handbook section and man page + + + + + if_bridge + + + + + Andrew + + Thompson + + + thompsa@freebsd.org + + + + + + + +

This was committed to current on 5 Jun 2005 and will first + appear in the 6.0 release, thanks to everyone who tested. Recent + improvements include:

+ +
    +
  • IPFW layer2 filtering
    • + +
  • DUMMYNET support
    • + +
  • IP header alignment checking
    • +
+ +

There is ongoing work to bring in some of the advanced features + from OpenBSD such as IPSec bridging. People are encouraged to use + if_bridge and report any problems to the mailing lists.

+ + + + + IPv6 Support for IPFW + + + + + Max + + Laier + + + mlaier@freebsd.org + + + + + Brooks + + Davis + + + brooks@freebsd.org + + + + + + + +

At the developer summit before BSDCan it was decided to remove + IP6FW from the tree as it has a couple of problems. The most + pressing one is the lack of synchronization and thus the need for + debug.mpsafenet=0. As a replacement Brooks Davis has imported + patches to teach the existing and well-locked IPFW2 code about + IPv6.

+ +

Since the initial import I have added some features required to + manage IPv4 and IPv6 in a single ruleset. I have also extended + existing opcodes to work with IPv6. There are, however, still some + opcodes that do not work with IPv6 and most of the more exotic ones + haven't been tested. As long as IPFW2+v6 does not provide enough + functionality and stability to work as a drop-in replacement for + IP6FW, we won't remove IP6FW.

+ +

In order to get the new code to that point we + really + + need more testers with real world IPv6 deployment and interest in + IPFW+v6. The lack thereof (I haven't received a single answer on my + requests to various FreeBSD mailing lists) has made it hard to + progress.

+ + + + Properly implement O_REJECT for IPv6 + + Maybe implement O_LOG + + Test new(er) IPFW2 opcodes with IPv6 + + Test + + Test + + Test + + + + + launchd(8) for FreeBSD + + + + + R. Tyler + + Ballance + + + tyler@tamu.edu + + + + + Wiki + Project Page + + + Apple's launchd(8) man page + + + +

So far progress has been slow, the autoconf build system has + been removed from all of the launchd(8) code, and launchctl(1) is + building and semi-functional on FreeBSD-CURRENT (i.e. + CoreFoundation hooks have been removed)

+ +

I'm currently working on porting "liblaunch" which is the core + backend to both launchd(8) (the actual daemon) and launchctl(1), + there are some mach/xnu specific hooks and calls that need to be + remove and either reimplemented or worked around.

+ +

We're also waiting on a response from Apple on a possible + BSD-licensed version of the code (it's currently under the APSL) + Progress is slow, but steady.

+ + + + + Removable interface improvements + + + + + Brooks + + Davis + + + brooks@FreeBSD.org + + + + + + + + + + +

This project is an attempt to clean up handling of network + interfaces in order to allow interfaces to be removed reliably. + Current problems include panics if Dummynet is delaying packets to + an interface when it is removed.

+ +

I have removed struct ifnet's and layer two common structures + from device driver structures. This will eventually allow them to + be managed properly upon device removal. This code has been + committed and will appear in 6.0. Popular drivers have generally + been fixed, but more testing is needed.

+ + + + + OpenBSD dhclient import. + + + + + Brooks + + Davis + + + brooks@FreeBSD.org + + + + + Sam + + Leffler + + + sam@FreeBSD.org + + + + + + + +

The OpenBSD rewrite of dhclient has been imported, replacing the + ISC dhclient. The OpenBSD client provides better support for + roaming on wireless networks and a simpler model of operation. + Instead of a single dhclient process per system, there is on per + network interface. This instance automatically goes away in the + even of link loss and is restarted via devd when link is + reacquired. To support this change, many aspects of the network + interface configuration process were overhauled.

+ +

The current code works well in most circumstances, but more + testing and polishing is needed.

+ + + + + Move ARP out of routing table + + + + + Qing + + Li + + + qingli@freebsd.org + + + + + + + + +

I've sent the patch to jinmei@isl.rdc.toshiba.co.jp @KAME for + review. I'm still waiting for feedback from Andre. There hasn't + been any major change since the last report. I've kept the code in + sync with CURRENT. Gleb has created a separate P4 branch and has + been helping out on the locking side. Gleb is also helping out on + the testing front.

+ + + + I'm waiting for review feedback from my mentor Andre on the + overall design and code. I'm waiting for feedback from Andre on + Gleb's suggested modification. + + + + + Nsswitch / Caching daemon + + + + + Michael + + Bushkov + + + soc-bushman@rsu.ru + + + + + + + + + + +

The + nsswitch / caching daemon + + project is being developed within the Google's Summer Of Code + program. The first goal of this project is to implement a set of + patches to extend the use of nsswitch subsystem. The second goal is + the development of the caching library and daemon to add the + caching ability to the nsswitch.

+ +

Currently services, protocols, rpc and openssh patches are + finished. Support for services, services_compat, rpc, protocols, + and ssh_host_keys databases is added with 'files', 'nis' and + 'compat' (for services) sources possible. The nsswitch-friendly + openssh port is amlost completed.

+ + + + Implement set of patches to make nsswitch support + globus grid security files + + , + MAC and audit related configuration files + + databases. + + Implement the caching library and the caching daemon and + patch nsdispatch function to support caching. + + + + + OpenBSD packet filter - pf + + + + + Max + + Laier + + + mlaier@freebsd.org + + + + + + + +

We will have pf as of OpenBSD 3.7 for RELENG_6. Import has been + completed in early May and FreeBSD release 6.0 will ship with + it.

+ +

A few serious issues with pfsync on SMP have been discovered + since CARP is around and more and more people use it on big iron. + Everything that has been discovered is fixed in HEAD and (if + applicable) MFCed back to RELENG_5. Some functional changes are + undergoing testing right now and will be MFCed in the coming + days.

+ +

With the import of if_bridge from Net/OpenBSD we finally have a + bridge implementation that allows for stateful filtering as well as + IPv6 filtering. Please see the respective report.

+ + + + Shared lock implementation? + + + + + Low-overhead performance monitoring for FreeBSD + + + + + Joseph + + Koshy + + + jkoshy@FreeBSD.org + + + + + + Project home page + + + +

Modern CPUs have on-chip performance monitoring counters (PMCs) + that may be used to count low-level hardware events like + instruction retirals, branch mispredictions, and cache misses. PMC + architectures and capabilities vary between CPU vendors and between + CPU generations from the same vendor, making the creation of + portable applications difficult. This project implements a + cross-platform PMC management API for applications, and implements + the infrastructure to "virtualize" and manage these PMCs. The + creation of performance analysis tools that use this infrastructure + is also part of the project's goals.

+ +

Work since the last status report:

+ +
    +
  • Sampling mode support for P4 and AMD64 PMCs has been + implemented.
    • + +
  • A pmclog(3) API for parsing hwpmc(4) log files has been + added.
    • + +
  • A number of bugs in libpmc(3), hwpmc(4) and pmcstat(8) have + been fixed.
    • +
+ +

Future work:

+ +
    +
  • Creating user documentation showing a few real-world uses of + the currently available tools.
    • + +
  • Testing, improving the stability of the code, and + characterizing its overheads.
    • + +
  • Implementing P5 PMC support.
    • +
+ + + + + Improve libalias + + + + + Paolo + + Pisati + + + soc-pisati@freebsd.org + + + + + Wiki + page about libalias work. + + + +

My SoC project is about improving libalias and integrating it + with ipfw2, adding nat support into the firewall. Till now i ported + libalias (as a kld) and ng_nat to 4.x and 5.x branches, and i've + already a first working patchset that adds 'nat' action into ipfw. + Next step will be to add a complete syntax to ipfw that will let us + manipulate libalias operations, much like we already do with queue + and pipes for dummynet. In the end the entire work will compile and + work out of the box for 4.x, 5.x and 6.x. More details about the + project and its status are available on wiki page.

+ + + + + TODO list for volunteers + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + + +

Since Google's "Summer of Code" resulted in a lot of interest in + open projects, I'm in the process of compiling a list of nice + projects for volunteers. Unlike Google's SoC those projects aren't + backed with money (but this doesn't means nobody is allowed to + sponsor one of those projects), so we can only guarantee the social + aspects (some "Thank you!" and "That's great!" messages). So far + the list has several entries where the difficulty ranges from + "someone just has to sit down and spend some time on it" up to "we + need a guru for this".

+ + + + Merging untaken entries from the SoC list as soon as the + official participants/tasks in the SoC are announced. + + Sending the document to some doc people for review. + + Commit the list. + + + + + Removing of old basesystem files and directories + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + Patch + + + +

FreeBSD lacks a way to remove old/outdated files and directories + in the basesystem. I have a patch which removes obsolete files in a + safe way (interactively, since only the administrator really knows + if there's a need to keed an old file or not; there's a switch for + batch-processing). This feature may or may not be available for + 6.0-RELEASE, depending on the decission from the Release + Engineering team.

+ + + + Respect the NO_* switches and remove those files too. This is + easy to do with the current implementation, but isn't needed to + commit the removal of obsolete files feature. + + + + + Porting v9 of Intels C/C++ Compiler + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + + +

Intel released version 9 of its C/C++ compiler. Work to port the + x86 version to FreeBSD is in progress as time permits. Porting the + EM64T (amd64) version is on the TODO list too, but is subject to + enough free time and access to appropriate hardware.

+ + + + + Update of the Linux userland infrastructure + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + Emulation + + Mailinglist + + + emulation@FreeBSD.org + + + + + + + +

The cleanup/streamlining and the possibility of overriding the + default Linux base as reported in the last report happened without + major problems. Work on the open tasks hasn't started yet, but is + scheduled to start "soon". If a volunteer wants to spend some hours + on one of the open tasks, he should tell it on the emulation + mailinglist.

+ + + + Refactoring the common RPM code in + x11-toolkits/linux-gtk/Makefile into bsd.rpm.mk. + + Determining which up-to-date Linux distribution to use as the + next default Linux base. Important criteria: +
    +
  • RPM based (to be able to use the existing + infrastructure)
    • + +
  • good track record regarding availability of security + fixes
    • + +
  • packages available from several mirror sites
    • + +
  • available for several hardware architectures (e.g. i386, + amd64, sparc64; Note: not all architectures have a working + linuxolator for their native bit with, but as long as there are + no userland bits available, no motivation regarding writing the + kernel bits will arise)
    • +
+ + + Moving the linuxolator userland to an up-to-date version (see + above). + + + + + Autotuning of the page queue coloring algorithm + + + + + Alexander + + Leidinger + + + netchild@FreeBSD.org + + + + + + Patch + + + +

The VM subsystem has code to reduce the amount of cache + collisions of VM pages. Currently this code needs to be tuned with + a kernel option. I have a patch which changes this to auto-tuning + at boot time. The auto-tuning is MI, the cache size detection is + MD. Cache size detection is currently available for x86/amd64 (on + other systems it uses default values).

+ + + + Add cache-detection code for other arches too (Marius told me + how to do this for sparc64). + + Analyze why the cache detection on Athlons doesn't work (no + problems on a P4, but it uses a different code-path). + + + + + FreeBSD website improvements + + + + + Emily + + Boyd + + + soc-emily@freebsd.org + + + + + + + +

As part of the Google Summer of Code, I'm working on + improvements to the FreeBSD website (including a proposed website + redesign). My mentor for this project is Murray Stokely.

+ + + + + UFSJ -- Journaling for UFS + + + + + Brian + + Wilson + + + polytopes@gmail.com + + + + + Scott + + Long + + + scottl@FreeBSD.org + + + + + + + +

filesystem. Journaling helps ensure the filesystem's integrity + should the system crash. Journaling eliminates the need for + fsck'ing a filesystem, as the filesystem is never in an + inconsistent state (barring hardware failure). This implementation + is inspired by Darwin's HFS+ filesystem and the SGI XFS filesystem. + This is a Summer of Code project, with Scott Long as the mentor and + Brian Wilson as the developer/mentee. Currently this project is + still in the early stages, but will be in a usable state by + September 1 (the Google Summer of Code completion date).

+ + + + Finish making the file system log metadata updates. + + Add facilities to replay the log on dirty file + systems. + + Make snapshots work with journaling. + + + + + SEBSD + + + + + Yanjun + + Wu + + + yanjun03@ios.cn + + + + + Show + status in wiki, update more frequently. + + + +
    +
  1. Setup a local P4 workspace of sebsd source and Setup lxr for + TrustedBSD source for studying source code.
    2. + +
  2. Test a simple policy configuration for vsftpd.
    3. + +
  3. Writing a HOWTO document + Getting Started with SEBSD HOWTO + + by deriving the existing + Getting Started with SELinux HOWTO + + .
    4. +
+ +

Thanks Robert Watson and Scott Long for their kind help.

+ + + + When writing the document, try to figure out the sebsd + userland utils that need to be ported. + + Test and edit more policies for BSD environment. + + + + + VFS SMP + + + + + Jeff + + Roberson + + + jeff@freebsd.org + + + + + + + +

FreeBSD's VFS layer has been fine grain locked along with the + FFS filesystem for the FreeBSD 6.0 release. The locking has been + underway for several years, with the project really picking up over + the last 6 months thanks largely to sponsorship provided by Isilon + Systems, Inc. a leading vendor of clustered storage systems. The + project has entered a stabilization phase, with a few bugs being + reported in extreme circumstances while the majority of users have + seen no problems. Tests on a 8 and 16 way machines yield reasonable + parallelization, however, it will be beneficial to do lock + contention analysis once things are fully stable.

+ +

For those interested in technical details, there have been a few + relatively significant changes with vnode life-cycle management. + Vnode reference counting and recycling is now no longer an ad-hoc + process involving a variety of flags, a use count and the hold + count. A single hold count is used to track all vnode references + and a destroyed vnode is freed in the context of the caller when + the last ref is lost. The old system would never reclaim memory + used by vnodes and also had pathlogical behavior with unreferenced + vnode caching under pressure. The new system is much simpler than + the old one, however, callers are now required to vhold a vnode + that they lock directly without going through vget to prevent it + from being recycled while they are waiting on a lock. Relying on + 'location stable storage', which is a more strict version of 'type + stable storage' is no longer a valid approach.

+ +

Some other side effects include a much simpler and faster nullfs + implementation, an improved buf daemon flushing algorithm which + eliminated high latency that caused audio skipping, and a lots of + minor cleanups and debugging aids.

+ + + + + EuroBSDCon 2005 - Basel + + + + + Information + + + info@eurobsdcon.org + + + + + Homepage + + Call for papers + + + +

The fourth European BSD conference in Basel, Switzerland is a + great opportunity to present new ideas to the community and to meet + some of the developers behind the different BSDs.

+ +

The two day conference program (Nov 26 and 27) will be + complemented by a tutorial day preceeding the conference (Nov + 25).

+ +

The program committee is looking for tutorial and paper + submissions. For details, please see: The + call for papers + + online.

+ + + + + SMP Network Stack + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + Netperf home + page + + + +

Significant work has occurred over the last few months relating + to the SMP network stack work. A few of the highlights are covered + here at a high level:

+ +
    +
  • The UMA(9) per-CPU caches have been modified to use critical + sections instead of mutexes. Recent critical section + optimizations make this a performance win for both UP and SMP + systems. This results in a several percent improvement in a + number of user space benchmarks, and larger improvement for + kernel-only network forwarding and processing benchmarks.
    • + +
  • The malloc(9) allocator has been modified to store statistics + per-CPU instead of using a cross-CPU statistics pool, with each + per-CPU pool now using critical sections to synchronize access. + This results in a measurable performance win, especially on SMP + systems
    • + +
  • The netnatm ATM code is now MPSAFE.
    • + +
  • netipx MPSAFEty has been merged to RELENG_5.
    • + +
  • The netperf cluster has now been expanded to include two + additional quad-CPU systems (one dual dual-core AMD system, one + quad-CPU PIII system).
    • + +
  • libmemsetat(3) (see separate report) now corrects SMP-related + races in the measuring of mbuf allocator statistics, as well as + substantially improving kernel memory monitoring capabilities and + tools.
    • + +
  • A range of locking bug fixes, and general network stack bug + fixes.
    • + +
  • Significant updates to the SMPng web page (still more to + do!).
    • + +
  • Identification of all non-MPSAFE network device drivers, with + ultimatum issued, on freebsd-arch. Quite a bit of new driver + locking work as a result (if_ed, if_de, ...).
    • + +
  • Lots of other stuff.
    • +
+ +

In most cases, these changes will appear in FreeBSD 6.0-RELEASE; + some have been, or will be, merged to FreeBSD 5.x.

+ +

On-going tasks include:

+ +
    +
  • Review and improvement of ifnet locking, such as address + lists and flags.
    • + +
  • Optimization of interface start hand-off.
    • + +
  • Prototyping of queue-oriented packet hand-off in the + stack.
    • + +
  • Performance measurement and analysis.
    • + +
  • Prototype rewrite and simplification of socket locking.
    • +
+ + + + + TrustedBSD SEBSD + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + TrustedBSD/SEBSD + web page + + + +

The TrustedBSD Project has released a new snapshot of "SEBSD", a + port of NSA's SELinux FLASK and Type Enforcement implementation to + FreeBSD based on a late 2005 FreeBSD 6.x snapshot. The SEBSD + distribution has now been updated in Perforce to a recent 6.x + snapshot, and a new distribution will be made available in the near + future.

+ +

Work has been performed to merge additional dependencies for + SEBSD back into the base FreeBSD tree, including most recently, + changes to devfs, and System V and POSIX IPC.

+ + + + Update to new NSA FLASK implementation, which has improved + MLS support. + + Merge remaining kernel changes to support SEBSD back to the + base FreeBSD CVS repository, including file descriptor labeling and + access control (in contrast to file labeling and access control), + and categorization of kernel privileges. + + + + + TrustedBSD Audit + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + Wayne + + Salamon + + + wsalmon@FreeBSD.org + + + + trustedbsd-discuss@TrustedBSD.org + + + + + + + + +

In the past few months, significant work has been done relating + to the TrustedBSD audit implementation, including preparatory work + to merge audit into the FreeBSD CVS repository for FreeBSD 6.x. In + particular:

+ +
    +
  • The user space components, such as libbsm, include files, and + command line utilities have been broken out into an OpenBSM + distribution in Perforce. Improvements in OpenBSM will be made + available separately for use by projects such as Darwin, and + imported into the contrib area of FreeBSD.
    • + +
  • The system call table format has been updated to include an + audit event identifier for each system call across all hardware + platforms and ABIs (merged), and all system calls have been + assigned event identifiers (not yet merged).
    • + +
  • The audit management daemon has been rewritten to run on + FreeBSD (originally derived from Darwin) using /dev/audit to + track kernel events.
    • + +
  • Many system calls now properly audit their arguments.
    • + +
  • The TrustedBSD audit3 branch has been updated to a recent + 6.x-CURRENT.
    • + +
  • Significant work has gone into synchronizing the audit event + tables between FreeBSD, Darwin, and OpenSolaris to make sure file + formats and events are portable.
    • + +
  • OpenBSM has been adapted to consume and generate + endian-independent event streams.
    • + +
  • OpenBSM documentation has been created.
    • +
+ +

The hope is still to provide audit as "experimental" in 6.0; the + primary blocking factor is our awaiting relicensing of the last + remaining audit files from Apple's APSL license to BSDL so that + they can be included in the FreeBSD kernel. This is anticipated to + complete in the near future. Once this is done, the changes can be + merged to CVS, and then MFC'd to RELENG_6. If this is not complete + by 6.0-RELEASE, the work will be merged shortly after the release, + as all ABI-sensitive data structures have been updated as + needed.

+ + + + + libmemstat(3), UMA(9) and malloc(9) statistics + + + + + Robert + + Watson + + + rwatson@FreeBSD.org + + + + + + libmemstat(3)-derived tools + + + +

libmemstat(3) provides a user space library API to monitor + kernel memory allocators, currently uma(9) and malloc(9), with the + following benefits:

+ +
    +
  • ABI-robust interface making use of accessor functions, in + order to divorce monitoring applications from kernel/user ABI + changes.
    • + +
  • Allocator-independent interfaces, allowing monitoring of + multiple allocators using the same interface.
    • + +
  • CPU-cache awareness, allowing tracking of memory use across + multiple CPUs for allocators aware of caches. Unlike previous + interfaces, libmemstat(3) coalesces per-CPU stats in user space + rather than kernel, and exposes per-CPU stats to interested + applications.
    • + +
  • Ability to track memory types over multiple queries, and + update existing structures, allowing easy tracking of statistics + over time.
    • +
+ +

libmemstat(3) and the the appropriate allocator changes for + uma(9) and malloc(9) are currently in HEAD (7-CURRENT), and MFC has + been approved to RELENG_6 for inclusion in 6.0-RELEASE. These + changes may also be backported to 5.x.

+ +

Sample applications include memstat(8), an allocator-independent + statistics viewing tool, memtop(8), which provides a top(1)-like + interface for monitoring kernel memory use and active memory types. + None of these are "pretty".

+ +

netstat -mb has also been updated to use libmemstat(3) to track + network memory use using uma(9), rather than the less reliable mbuf + allocator statistics interface. As a result, the statistics are now + more reliable on SMP systems (this corrects the bug in which mbuf + statistics sometimes "leaked", even though memory didn't), and more + informative (cache information is now displayed, as well as mbuf + tag information).

+ + + + Teach libmemstat(3) to speak libkvm(3) in order to allow + tools linked -lmemstat to interogate kernel core dumps. + + Teach libmemstat(3) to interface with user space malloc and + track malloc allocations for user space applications. + + Update vmstat(8) -m and -z implementations to use + libmemstat(3) instead of the old monitoring interfaces. Code to do + this exists in the sample libmemstat(3) applications. + + Identify how to make streams or the library endian-aware so + that streams dumped from a kernel of alternative endian could be + processed using libmemstat(3) on another system. + + Identify any remaining caching allocators in the kernel, such + as the sfbuf allocator, and teach libmemstat(3) how to interface + with them. + + + + diff --git a/en/news/status/status.sgml b/en/news/status/status.sgml index b4d94beeee..4ffd95c4f0 100644 --- a/en/news/status/status.sgml +++ b/en/news/status/status.sgml @@ -1,6 +1,6 @@ - + %includes; ]> @@ -8,6 +8,7 @@ &header; +

One of the benefits of the FreeBSD development model is a focus on centralized design and implementation, in which the operating system is @@ -41,6 +43,8 @@

2005