From 7f4c52153d395c20f53a9d490a5e908804959af4 Mon Sep 17 00:00:00 2001 From: Murray Stokely Date: Sat, 16 Mar 2002 08:47:55 +0000 Subject: [PATCH] Add notes from the BSDCon 2002 FreeBSD Developer Summit. This is deliberately not connected to the build, as Robert still may have some updates to add before we make this public. Notes prepared by: gnn and rwatson --- en/events/2002/Makefile | 12 + en/events/2002/bsdcon-devsummit.sgml | 1919 ++++++++++++++++++++++++++ en/events/Makefile.inc | 3 + en/events/events.css | 18 + 4 files changed, 1952 insertions(+) create mode 100644 en/events/2002/Makefile create mode 100644 en/events/2002/bsdcon-devsummit.sgml create mode 100644 en/events/Makefile.inc create mode 100644 en/events/events.css diff --git a/en/events/2002/Makefile b/en/events/2002/Makefile new file mode 100644 index 0000000000..c746a42428 --- /dev/null +++ b/en/events/2002/Makefile @@ -0,0 +1,12 @@ +# $FreeBSD$ + +.if exists(../Makefile.conf) +.include "../Makefile.conf" +.endif +.if exists(../Makefile.inc) +.include "../Makefile.inc" +.endif + +DOCS?= bsdcon-devsummit.sgml + +.include "${WEB_PREFIX}/share/mk/web.site.mk" diff --git a/en/events/2002/bsdcon-devsummit.sgml b/en/events/2002/bsdcon-devsummit.sgml new file mode 100644 index 0000000000..721b83daae --- /dev/null +++ b/en/events/2002/bsdcon-devsummit.sgml @@ -0,0 +1,1919 @@ + + + + + + %includes; + %developers; +]> + + +&header; + +

Notes from FreeBSD Developer Summit, February 15, 2002 Cathedral + Hill Hotel, San Francisco, CA.
Notes taken by George Neville-Neil, + edited by Robert Watson.

+ +

Meeting began at 09:30am, ended at 5:00pm.

+ +

Agenda

+ + +

Attending:

+ +

In person:

+ + + +

On The Phone:

+ + + +

Via webcast:

+ +

Joe Karthauser is recording the call and is web casting.

+ +

The meeting followed a format where each section was led by an + individual and then a discussion ensued. Not all of the discussion + was caught but I have tried to make those sections understandable.

+ +
+
+

Opening Remarks - Robert Watson

+ +

Welcome to the second FreeBSD Developer Summit at BSDCon 2002. + This event is sponsored by:

+ + + +

The sponsors covered the cost of the room, food, telephone access, + etc. In addition, Joe Karthauser is providing webcast access, and + Yahoo! is providing bandwidth for that using the FreeBSD.org + cluster.

+ +

This is actually the second FreeBSD Developer Summit -- the first + was at the USENIX Annual Technical Conference in Boston last summer.

+ +

The goals are to provide the opportunity for insight into on-going + work, and to solicit comments, design input, and help on parts of + the work.

+ +

There are rules of engagement. Presenters should attempt to + remember to check for questions on the phone, respect people asking + questions, keep moving, and if told to stop, stop, as well as + provide notes on the presentations later. Others should do the + same, especially with regards to stopping when asked to stop.

+ +

(Tentative schedule reviewed)

+ +

People will be on their own for lunch.

+ +

Let's go around the room, please give your name, and say something + about what you do or why you're here.

+
+ +
+
+ +

Inheritance Added to KOBJ - Justin Gibbs

+ +

Inheritance models have been put into newbus to reduce code + duplication. This was done because he was trying to get cardbus to + work. Code did not adhere with the spec. There were conflicts with + copying from PCI. Cardbus is just an extension to PCI.

+ +

If you look at the current cardbus under BSD it's just all of PCI + with symbols renamed. Newbus is an OO type framework was and was + half way there. KOBJ and Newbus work today with a table of methods. + To invoke a method an indirection is done to a lookup. This makes + it so that the nvoker doest not need to know where the invoked + method is.

+ +

The extension that was added is that instead of a single table it's + a list of interfaces. Every interface can inherit from a parent. + In cardbus there is a list. One method is the device interface for + probe/attach. Then there is an interface for the bus. The third is + a sys parsing interface that can be shared with pccardd.

+ +

Inside this the interfaces inherit from PCI and then he + overrides a few methods.

+ +

The amount of code to support cardbus drops from 1000s of + lines to a few hundred.

+ +

The system is implemented through the indirection already in + newbus. When a class is registered the way you declare it is a list + of interfaces. Each interface can declare a parent. The compiler + walks the list to find the correct function. This allows you to + call your superclass.

+ +

The only thing that gets a little ugly is that some of the ways you + specify your class or invoke a superclass. You can generate a macro + wrapper but you have to know which interface is yours.

+ +

Another upshot is that the way most drivers are implemented today + if they need a generic method they have to explicitly put that into + their method table. If they don't need it then they don't. With + the inheritance model you just create a null interface.

+ +

Diffs on his web site.

+ +

Whole method tables collapse to one or two entries.

+ +

Open issues:

+ + + +

His question is what release should this go into and how do we + manage the transition if we decide to do this?

+ +

How do we design the versioning? Run time? Link time? In the + multipathing case is an array of dev_t sufficient?

+ +

Discussion

+ +
+ +

Anon : I hesitate to throw this + out. The way you do version verification in Solaris is that an init + gets called which passes a version to the api to check it. Another + thing you get from init.

+ +

In the solaris case there is a single number. How do I automate + that versioning check with more interfaces?

+ +

Paul : I don't think we should + over engineer the problem. We only bump per release in libraries.

+ +

Justin : When you bump the version + is one part of it.

+ +

Julian : Do you want more than one + version at once?

+ +

Justin : Assuming we use name + munging you could have.

+ +

Gnn : Have you looked at COM?

+ +

Warner : I have a few comments. + There are several problems in one here. I like the multiple + inheritance. The versioning seems orthogonal. The multipath is + orthogonal. We should break this apart.

+ +

Justin : I have 14K lines of diffs + just to make multiple-inheritance work on my laptop.

+ +

Julian : The other drivers will + still work.

+ +

Justin : No they will not, they'll + die early in init. I could turn those into fatal errors.

+ +

PoulHK : If the change is so + significant then the first thing we want to do is versioning.

+ +

Justin : Its almost as if you want + a version of newbus.

+ +

PoulHK : A change of this scale is + not very easy to get all right. Particular because we can't change + all drivers.

+ +

Justin : The only saving grace is + that the transition can be mechanical.

+ +

PoulHK : Why not a compatability + layer while doing conversion?

+ +

Justin : Sure and then you could + use Perl to change the whole tree. I hesitate because in 90% of my + own changes led to code reduction and we should do that.

+ +

PoulHK : I think we'll have the + human eyes.

+ +

David O : How long to get your + basic framework working?

+ +

Justin : Right now my stuff is + done, the rest of the tree is a man/week. I'd like to get + versioning done now.

+ +

Mark M : Does that include the + script?

+ +

Justin : No manually.

+ +

Julian : The problem is the + requirement where you can't break things. Perhaps we should have + official breakage. For instance for a week.

+ +

Nick : If you're going to do that + then lay down a tag before the breakage.

+ +

PoulHK : Development model + discussion. Lets limit this discussion.

+ +

Robert : Lets assume the + discussion of how it gets into the tree is not relevant. Do you + want the versioning before then?

+ +

PoulHK : Two questions we should + figure out first. 1) Any objections?

+ +

Warner : One last point. It seems + like we're spending a lot of effort re-inventing the object wheel. + We should just shoot KOBJ and consider using a subset of C++ to do + this. Then we can leverage the language.

+ +

Justin : One problem is that C++ + does not allow this easily. If you do eC++ then there is + multiple-inheritance.

+ +

Warner : I mean C++ with no + exceptions but not eC++.

+ +

Justin : Then people say they want + exceptions.

+ +

Julian : I wouldn't mind having + our own pre-processors.

+ +

Robert : Let's move on.

+ +

Nectar : COM does work its in + Mozilla we can use this kind of technology. We don't need C++ to do + those things.

+ +

Justin : One last thing is the + problem in this inheritance model is how you deal with IVARS. The + problem is that those name spaces are not unique. What I'm thinking + of doing is passing the interface object along. This gives you a + further

+ +

Brian : C++ won't help us solve + versioning. Although COM would solve some of these problems it + should not be in the fast path.

+ +

Robert : Lets not go there.

+ +

Warner : The other issue is the + softc issue. Only one can own it.

+ +

Justin : You can use ivars for + that.

+ +

Warner : There are issues with + bridges.

+ +

Justin : You use a method to get + to your ivars which hang off of the softc.

+ +

PoulHK : Half of us have no idea + because we have no docs.

+ +

Robert : Lets go to the phone.

+ +

Robert : No technical questions on + the phone.

+
+
+ +
+
+ +

New Architectures

+ +

PowerPC - Benno Rice

+ +

Has now commited the page map code with something more sane. Has + also got the system to the point where it tries to mount root. + Turning on invariants blows it up. Hoping to get to single user in + a month or so on the simulator. With some luck we may get to + multi-user by 5.0. Needs some more help. Possibly on real + hardware. That's about it.

+ +

Discussion:

+ +
+

Robert : This relates to release + engineering later.

+ +

David : Is everything in CVS?

+ +

Benno : Not everything yet, but + things have to be cleaned up. Some time in the next week after the + invariants problem is fixed.

+ +

PoulHK : What hardware?

+ +

Benno : Right now on pSim which is + in ports. Once that's working the first hardware will be new world + apple hardware. Some old world apple hardware and then a Motorola + board.

+ +

Justin : What are the prospects + for using this in embedded?

+ +

Benno : Depends on whether you've + got open firmware or not.

+ +

David : Targetting embedded is + very difficult. Pick a reference platform then move on. Wind River + has 20+ boards etc.

+ +

Benno : What I'm hoping to do is + to make this as easy as possible.

+ +

David : Given a reference platform + its easier to build from there.

+ +

Benno : The other note that I will + make is that I'm only targetting PowerPC similar to 700.

+ +

Warner : I was going to comment + that NetBSD has done well with little tiny ports to different + boards.

+ +

Justin : How different are these?

+ +

Gnn : Very different.

+ +

Anon : Have you had any help from + Apple or hardware vendors?

+ +

Benno : I have a bit of help from + them. In terms of hardware support I've had none except for the + Motorola board.

+ +
+ +

IA64 - David O'Brien

+ +

Kind of hosed now due to toolchain issues. It will take + significant effort to bootstrap this. It may take a month to get to + multi-user.

+ +

When Peter arrives later, he may have more to add.

+ +

x86-64 - David O'Brien

+ +

We have a toolchain that works and is mostly in the tree. There is + a simulator issue. I need a new simulator from AMD but they're not + responsive. Hardware not available yet but we're not in a rush.

+ +

Questions

+ +
+

Justin : During the NetBSD there + was some talk about PAE coming for free. Is that true and will it + affect us?

+ +

David : Peter is already adding + this stuff.

+ +

Justin : Peter is looking at the + stuff that was presented here.

+ +

Warner : Isn't part of this + meaning that ethernet drivers have to use DMA.

+ +

Justin : It's not as bad as it + looks because all the devices don't need bounce buffers.

+
+ +

Sparc64 - Jake Burk Burkholder

+ +

Basic status is we boot multi-user on real hardware. Looking at + targetting Ultra 2, 5, 10 and Blade 100. Something for 5.0 but it + may be a very manual install procedure. Toolchain is native but we + haven't tested it much. It is a hosted tool chain. Runs on + Sparc64, generates Sparc64 binaries, but it's not the full usual + thing. gcc3.1 and binutils 2.12

+ +

Discussion

+ +
+

Anon : What is the + bootstrapping?

+ +

Jake : TFTP.

+ +

Robert : Bootloader?

+ +

Jake : We can mount Solaris + disks.

+ +

Robert : How likely that all of + world will be self hosting by 5.0?

+ +

Jake : Right now we're having + problems with Perl.

+ +

David : There is polishing to be + done but even if you're not a hacker it's fine.

+ +

Jake : Still finding endianness + problems.

+ +

Terry : If someone wanted to use + that for 32 bit how tough would it be?

+ +

Jake : You'd need to rewrite the + kernel.

+ +

Terry : Do you document that?

+ +

Jake : It's mostly pmap.

+ +

Anon : SBUS support?

+ +

Jake : Ultra 2 has sbus support.

+ +

Benno : Have you got the gem + ethernet driver working?

+ +

Jake : Yes.

+ +

Robert : Any questions on the + phone?

+
+ +

StrongArm

+ +
+

Robert : We did have a slot for + ARM related ports.

+ +

David : On StrongARM we can get to + copyright messages etc. This is a bunch of code off to the side. + What path do we want to take on this? Wait until it works? Check + it in now?

+ +

Justin : Who has been working on + this?

+ +

David : Someone in Canada.

+ +

Robert : Is the person who's doing + this work willing to go all the way to maintain it etc.

+ +

David : The basic idea is the + iPAQ. The idea is more of a reference port.

+ +

Warner : MIPS port is unchanged. + Some people have this and are just deciding whether to contribute + it. Chicken and an egg problem.

+ +

Justin : Interesting embedded + platform.

+ +

David : If it's on a DECStation + then it's not embedded.

+ +

Warner : This is on current MIPS + technology for a router company.

+
+
+ +
+
+ +

Toolchain - David O'Brien

+ +

Questions:

+ +
+

John : What are the plans 5.0?

+ +

David : Interest is in new ports. + For instance C++ will lag. Like you say we need gcc3.1 and + binutils. Will commit to get kernel and interesting parts of user + land working. Very soon there will be something that those who want + to play with it can download. It's debatable who will clean this up + for new hardware.

+ +

Robert : We'll discuss the release + engineering later. There is a notion of supported vs. non-supported + and toolchains will have to follow.

+ +

Julian : There are some thread + fixes that went in. Patches to gdb?

+ +

David : I'm trying to get them out + of someone. If I get a patch I'll put it in. I need paperwork from + the FSF to commit it.

+ +

Julian : I'm going to need help to + beat up gdb for KSE.

+ +

David : Talk to John, DFR and + yourself on this.

+ +

Nick : How much change goes in to + gcc for a FreeBSD release?

+ +

David : The issue is the dual + a.out/elf stuff, that's the problem.

+
+
+ +
+
+

Architectural Issues (General Discussions)

+ +
+

PoulHK : I have 3 issues. One is + endianess in the on disk file system. Do we want to be able to move + a disk from Sparc64 to x86. I also need to collect the various disk + label formats. What do we do about "you broke world on foobar + architecture" issue?

+ +

Warner : I'll address the last + question first. We need to build up a set of machines or build up + cross building.

+ +

David : I don't know if anyone set + up an environment.

+ +

Warner : NetBSD has done some + things to deal with this.

+ +

Robert : Supported release? The + same thing can be done in the cross platform build. If a + particular arch is going to be supported then we must have a + build cluster. + +

Benno : On the subject of PPC I'd + be willing to offer them. + +

Nik C : The NetBSD folks are + talking about having a tinderbox environment. We might talk + to them about sharing. + +

PoulHK : On the subject of + tinderbox. About 2 years ago I set up a machine to test this kind + of thing but got a lot of negative feedback. If we want to do a + tinderbox like system it will require buy in by the commiters.

+ +

Nik C : I think the Mozilla team + do it more with a web page status.

+ +

David : The gcc group does a + tinderbox and it knows who did the commit.

+ +

Action item : This could be farmed + out to sysadmins who want to contribute.

+ +

Justin : From my standpoint, if I + could cross build the Sparc64 that would help.

+ +

Robert : If we're going to commit + to having multiple platforms we need to solve this.

+ +

Nik C : Thre are also issues of + regression testing.

+ +

David : I don't know what could be + done with it.

+ +

PoulHK : This is a purely + organizational question. What does it take to do this. Do we + discuss this on developer?

+ +

Greg : I really do think we should + try to find a way to be endian clean.

+ +

PoulHK : We're not going to take + suggestions. There will be a UFS2 and it will be 64bit. As part of + that we're still not sure if we will have to fork the UFS tree. One + of the things I was considering doing was at a slight performance + hit doing big or little endianness on any disk. Is that the way to + do it?

+ +

Alfred : NetBSD has this and it's + fast.

+ +

Justin : I would think of it terms + of having two modules, one for each endian.

+ +

Benno : It would be nice to be + able to do this when bringing up new big endian systems.

+ +

PoulHK : The performance is an + issue but not as big as the code intrusion. Should we do it as two + seperate file systems or should we put this functionality directly + into UFS2?

+ +

Matt : Two comments on the FS + stuff. One just from the point of view of fixing bugs, I would + prefer a single copy. You could also do this via a conversion + program.

+ +

PoulHK : It's not just a question + about sticking in macros. Soft updates makes this much more + complex.

+ +

Jake : On big endian machines I + just use NFS.

+ +

Greg : Conversion does not make + any sense.

+ +

Justin : The other thing I want to + say is that you can just have two modules.

+ +

Robert : Before we go too much + further we should look at NetBSD.

+ +

PoulHK : Is there interest?

+ +

Gnn : Removable media is reason + enough to do it.

+ +

Warner : When I've talked to + NetBSD about this they consider it a feature they don't want to give + up.

+ +

Robert : Any questions on the + phone for architectures.

+ +

None.

+
+
+ +
+
+ +

GEOM - Poul-Henning Kamp

+ +

This changes the semantics of how we handle disks. There also may + be a slight performance hit.

+ +

The way it works is that there are methods that do transformation + on the data on the disk. Simple translations move data, complex + transformations include encryption.

+ +

A method exposes one or more logical providers. It exposes an + interface where you can read/write a given size.

+ +

Other methods connect to the providers (consumers). All providers + have a dev method which allows it to show up in devfs. There is a + locking mechanism so you don't get parallel write problems.

+ +

The system is autoconfiguring.

+ +

The locking method. When you open a device somewhere there are + three counters associated with it, read, write, and exclusive. + Exclusive bit prevents anyone else from opening it for write.

+ +

The intent is that the modules that recognize the on disk format + will be endianness aware. We hope that types will be explicit. I + need support from people to collect information on disk label + formats.

+ +

Some current drivers do RAID etc. but I'd like to get that out of + the drivers, again this requires information about the on disk + format. This would allow us to come up single user with a mirrored + root.

+ +

I'd like input on :

+ +

I/O Statistics (What to collect?)

+ +
+

Greg : Read/Writes/Data transfered + etc. A way of finding the % busy time of each device.

+ +

PoulHK : With tagged queuing that + is kind of useless.

+ +

Anon : The dev stack does keep + track of how long a device has been busy (transaction outstanding).

+ +

PoulHK : One of the things there + is an explicit cloning of the struct bio. So you have one for each + edge in the graph. One of the things I want to be able to do is put + in a transparent node. This allows the moving of file systems.

+ +

Anon : You have to have per + transaction storage for this to work.

+ +

Justin : You want to have + something to make it so that softupdates does not need to do the + sleep/nice trick.

+ +

Matt : The real problem with fsck + is that when you're skipping around the disk the 3ms seek can hurt + other things.

+ +

Justin : You were saying that if I + open ad0s1 read only then ad0 inherits that.

+ +

PoulHK : It depends on where you + are in the tree etc.

+ +

Nick C : From the work you've done + so far, do you have performance numbers?

+ +

PoulHK : No, because I'm in the + simulator. It does cost copying two struct bios.

+ +

Alfred : To modify the on disk + disk label when its mounted read only can you modify it?

+ +

PoulHK : The BSD module decides + that. You cannot go down to the raw disk and do that.

+ +

Alfred : What if I want to expand + the root file system?

+ +

PoulHK : Making a partition larger + while its open is fine, making it smaller will be problematic.

+ +

Alfred : I think this locking is a + bad idea.

+ +

PoulHK : The only thing that's + going to be magic is having the root partition in single user. I'm + not going make an escape hatch for this.

+ +

Robert : Are there any other + issues?

+ +

Justin : What is the name space?

+ +

PoulHK : The control will be + sysctl. I want to remove the ioctls from these. I haven't really + decided on the name space. I want to make sure that /dev does not + change.

+ +

Julian : Have you considered using + the earlier discussed methods and inheritance so that this all works + together?

+ +

PoulHK : I discounted them for + performance reasons. I discounted newbus because it has a one way + tree. The one thing we're having a lot of problems now is something + going away. That does not work today.

+ +

Julian : How do you handle the + case when the device rips out at the bottom when you've got a + downwards going command?

+ +

PoulHK : There's two things to it. + A struct bio travelling down will either be in the consumer or the + provider and that's where the lock is. Modules can also be shut off + safely.

+ +

Alfred : Couldn't you just provide + a generic callback?

+ +

PoulHK : I can do that because + they're all sitting in the module. I need to negotiate with the + device driver writers about that.

+ +

Nik C : Is this an implementation + based on new research work or wholly new?

+ +

PoulHK : I've read what I could + find but most current systems have a fixed hierarchy.

+ +

Robert : Will this give us a + chance to retrofit the system with 64 bits?

+ +

PoulHK : struct bio will have 64 + bit numbers.

+
+
+ +
+
+ +

Network Stack - Luigo Rizzo

+ +

Recent work has been on performance. Removal of unnecessary + copies. Using clusters etc. There is the polling code but it's + only on a few devices. Need to add support for more devices.

+ +

Do people like polling?

+ +
+

PoulHK : I worry about + interactions with the rest of the system. We should probably spend + some time looking at that.

+ +

Luigi : The next thing to do is + add mixed mode operation.

+
+ +

Jonathan has something to say about how network interrupts are + dispatched.

+ +

Some minor fixes to the stack. FIFO buffers on UDP

+ +

I have a version of PGM (reliable multicast). I plan to commit to + current and stable if people ask.

+ +

The ICSI folks have changes to multicast API which will help with + IGMPv3.

+ +

Some of this code I do for myself. How do we do kernel + patches?

+ +

Questions

+ +
+

Alfred : There's two things. With + 256byte MBUFS and clusters for DMA people are seeing really bad + utilization of memory for networking packets. Certain application + servers with small packets have problems.

+ +

Luigi : I've talked to Peter Wemm + about this stuff.

+ +

Julian : First, I have some times + worried about the flexibility of polling regarding different + networking types. For instance the number of packets on Gigabit + vs. PPP. I'm worried about the extremes.

+ +

Luigi : This will become + completely irrelevant when I implement mixed mode.

+ +

Julian : Second is to do with the + TCP stack trying to remove copies. Have you got any intention of + evaluating the recent SACK implementation?

+ +

Luigi : That broke standard TCP.

+ +

Alfred : Actually SACK is out of + style now. One other thing on performance is that the drivers do a + mget/mclget at once.

+ +

Julian : Third question is the + ability to add meta data to packets using m_aux?

+ +

Luigi : My major concern is that a + generic system is very slow because of scanning for data.

+ +

Robert : I think that that's worth + doing. Julian will you own that?

+ +

Julian : Yes.

+ +

Anon : Impact of polling scheme on + SMP?

+ +

Luigi : They don't compile + together. Right now I only have one polling loop. With a giant + lock around the stack what's the point?

+ +

PoulHK : To what extent do we want + to use the netgraph code? How do we deal with the multiple ATM + stacks.

+ +

Warner : Lets lose the ones that + don't work.

+ +

PoulHK : None of them work + now.

+ +

Greg : Just losing them could get + us into a bounce. Maybe we should try to encourage using it.

+ +

PoulHK : Is ATM interesting to + FreeBSD at all?

+ +

Gnn : ATM is necessary for DSL

+ +

Justin : What about DAFS? That + uses ATM.

+ +

Julian : Since our end user + community does not use 5.0 that's part of the problem.

+ +

PoulHK : It's a perfectly valid + point. Who's going to fix it?

+ +

Anon : The ATM list is somewhat + active.

+ +

PoulHK : There are two stacks one + that is used by the Japanese and one that does a bunch of wierd + stuff that no one uses.

+ +

Robert : Action item is to query + the atm list to see what's up with this.

+ +

Action Item : Query the ATM list + about which stack they want/use.

+ +

Julian : We want to be careful + about ripping it out if people are using it with DSL cards.

+ +

Warner : One of the weaknesses in + the current network stack is dealing with removable devices.

+ +

Peter : You have to eject a button + on M$. Laughter.

+ +

Robert : Part of the problem is + the ifnet pointers from mbufs etc. We do need a long term solution. + If you ifconfig down that doesn't fix it.

+ +

Luigi : You could try to keep the + ifnet structure alive. Dummynet can scan all the mbufs whenever you + delete a pipe. It's expensive but you could do it on eject.

+ +

Alfred : The drivers have to be + more robust.

+ +

Julian : I actually did some work + on adding reference counts to all consumers of ifnet structures. It + reference counted everything but it didn't cause a performance + issue.

+ +

Luigi : This becomes an issue only + at higher speeds.

+ +

Robert : We may want to defer this + to SMPng.

+ +

Gnn : You could have a two level + hierarchy of device drivers.

+ +

Terry : Virtual interfaces.

+ +

PoulHK : This ties into another + issue about how we look at our interfaces. No one notices when I + unplug my device.

+ +

Peter Wemm : The problem isn't + deleting the routes its adding more routing messages.

+ +

Nick S : Also includes + dhclient.

+ +

Gnn : Need new routing messages.

+ +

Paul Richards : Wants activities + brought up to userland for devd.

+ +

Anon : At BEOS we had the user + list all the potential interfaces.

+ +

Robert : I think we still need a + routing socket event.

+ +

All : Discuss on mailing list.

+ +

Jonathan L : If we're talking + about doing this with cables this works with MII but it's only in + kevent and not in routing sockets.

+ +

Matt : Having a carrier loss flag + an existing route is the right answer.

+ +

PoulHK : Re raise netgraph issue. + What is the future of netgraph in the tree right now. We have very + few users now.

+ +

Julian : What parts are not + done?

+ +

PoulHK : Configuration etc.

+ +

Alfred : Netgraph is extremely + useful. It needs to be documented and a bit more bolted down.

+ +

Julian : It is bolted down. For + 5.0 only one API changed. There's plenty of documentation.

+ +

Peter W : Originally when it first + came up it was not meant for high speed.

+ +

Julian : I was being cautious. We + could switch over all of PPP to use it etc.

+ +

Nick S : I'd like to see the mpd + netgraph configuration files become more /usr/sbin/ppp.

+ +

Julian : Brian has been toying + with having ppp take over mpd. + +

Gnn : Can we use netgraph for SMP? + +

Peter W : I'm a refugee from a + streams based system. It's scary. Be a little bit careful.

+ +

Terry : I think it would be very + hard to for example take the Rice work and make it work in the + context of netgraph where you're processing interrupts to + completion. The advantage of going to completion.

+ +

Luigi : I think I'm doing the same + thing .

+ +

Jonathan L : I have code that does + that. I've replaced all the queueing calls with a single call.

+ +

Robert : Bring this to an end.

+
+
+ +

LUNCH

+ +
+
+ +

TrustedBSD - Robert Watson

+ +

DARPA funding has accelerated work.

+ +

Goal is to introduce security features for new consumers.

+ +

Most basic component is ACL work (fine grained ACLs). We needed + extended attributes, so those are in too. Currently low performance + and low reliability. Userland still needs new utilities.

+ +

The more interesting work is in the Mandatory Access Control. Goal + is to enforce new restrictions in the kernel. Multi-level security + uses this.

+ +

Part of the work is to port SE Linux stuff to BSD.

+ +

This stuff interacts with other subsystems including the network.

+ +

Mandatory policies. Discretionary rights are you proteting your + own data. This is very hard to manage. MAC addresses this by + defining policies for users in the system. Where you have many + users on the same machine. There are a couple of traditional + systems that are in military systems and trusted systems.

+ +

MLS is a confidentiality policy. Who can read/write things based + on clearance. Doing things based on the "need to know."

+ +

BIBA integrity system is the other. Secure levels are somewhat + like this but are not comprehensive.

+ +

Type Enforcement. Flexible MAC based on administrator rules.

+ +

You can plug different models into the framework.

+ +

MAC requires pervasive enforcement. Current system can restrict + access to interfaces. Can mark packets for security. Can control + sockets.

+ +

What the framework does is provide a single framework to plug these + into.

+ +

The framework provides APIs. You put these throughout the code. + They are ifdef'd. If you don't compile with "options mac" you don't + get this.

+ +

What happens with the framework a module can declare at boot time or + you can do an lkm on it.

+ +

Right now these API calls are in a perforce branch. They are + pervasive. They don't touch every part of the system, only the + parts that NAI use.

+ +

There isn't a generic label structure. To add new labels you must + recompile the kernel. Real key is to keep the costs low.

+ +

We don't allow for garbage collection on lables. Binary block that + gets carried around.

+ +

This is not really integrated into userland.

+ +

Reduced the number of total security checks in the kernel by + unifying on this i/f.

+ +
+

PoulHK : Does this flexibility + include removing the existing checks?

+ +

Robert : No. You can only extend + checks.

+ +

Directions are flexibiity , add more hooks in to the MAC for the + kernel, integrate other work like LOMAC. Need to teach userland + something.

+ +

Alfred : Are you moving to per + operation?

+ +

Robert : Right now we only do at + open() time. We want to do read/write and send/recv.

+ +

Robert : Once we have read and + write we'll be able to revoke access.

+ +

We'd like to commit this before 5.0. It's available in PerForce. + PerForce now exports through cvsup.

+ +

We have not done any micro-benchmarks. Once we know then we can + make a decision to leave it on or not.

+ +

Alfred : What about active mode + tripwire like system?

+ +

Robert : You can do a number of + things like that with the current framework.

+ +

PoulHK : One of the other things I + would like to do is a best effort MD5 on files. This would make + tripwire and mtree checks faster.

+ +

Robert : The problem with that is + you don't get the data during these operations.

+ +

Right now the struct mbuf is extended but it would be nice to have + a better system.

+ +

Alfred : m_aux

+ +

Robert : sounds expensive because + of the list stuff

+ +

Terry : You said something about + the NSA linux code. Independant? Licensing?

+ +

Robert : Interesting issue. All + TrustedBSD work is under BSD license. The NSA stuff will not be + under a BSD license but will be a module.

+ +

Terry : By compiled do you mean a + loaded module? Binaries?

+ +

Robert : Not binary, you could + imagine a package.

+ +

Anon : My understanding was that + any project done by the gov't was PD. But that doesn't seem to be + OK.

+ +

Robert : I can't say. Part of our + contract was to release as open source.

+ +

Alfred : About compatability. How + compatable are we with others?

+ +

Robert : We've tried to follow the + specs.

+ +

Alfred : Perhaps support NFS + extended attribute stuff?

+ +

Robert : Right now everyone does + RPCs for ACLs and they're incompatbile. Not in Posix 1e. We tried + to work with others but some are not tracking (Linux).

+
+
+ +
+
+ +

Capabilities - Brian Feldman

+ +

Largely complete and stable. Unknown performance hit.

+
+ +
+
+ +

KSE - Julian Elischer

+ +

Very quick status report. It's an attempt to produce support for + high quality threading within the kernel. Threading is outside the + kernel, the support is inside. Taking advantage of work by Anderson + (Scheduler Activations). NetBSD is sticking closer to the paper. + We're doing a variant based on discussions with a lot of people. + The basic concept is the same.

+ +

The threading comes with the ability to make async syscalls. Any + syscall you do from the point of the view of the thread looks like + its stopped but this does not stop the whole process. A new thread + is produced on the fly. We've extended this to produce multiple + upcall points, 1 per CPU. This is so you can run multiple upcalls + on different CPUs. The upcalls occur to different stack contexts.

+ +

Status Report

+ +

There are a set of patches available on Julian's website. Gets us + as far as pthread. Kernel supported, single CPU, threads. All + syscalls are async, but only on one CPU at a time.

+ +

This was shown in the WIP.

+ +

Up next is to do multi-processor.

+ +

Next stage is to make it complete or even safe. What I've got + works but I've broken ptrace so we can't debug processes. What I + have checked in are a number of changes that were part of this + change. This reduces the amount of diffs vs. the patch. It's not a + terribly huge diff at the moment.

+ +

Next is to get gdb working again.

+ +

The next change would be to make the changes for multiple CPUs.

+ +

Need a more complete version of the API. Just have thread creation + and thread kill right now.

+ +

I'm expecting that I'll have the current stuff checked in within + the month. Depends on issues with gdb. I'm hoping June or July for + the full multi-processor version.

+ +

I hope to check in soon so that user land folks can work with + it.

+ +

Questions

+ +
+

Julian : Does anyone think this is + a bad idea?

+ +

Greg : We never finished our + discussion on Tuesday. 3 layers is too many.

+ +

Julian : There are 4 layers but 2 + are basically null.

+ +

John : We talked about this, it is + the right thing.

+ +

PoulHK : Sounds like a blue print + paper.

+ +

Alfred : What do you plan on + implementing in the upcall?

+ +

Julian : It's not an explicit call + to allocate and deallocate threads. You do a call which says "I'm + going to go into this mode" and when something blocks come back to + me as an Nth or an N+1.

+ +

Alfred : OK.

+ +

Peter W : It works just like fork + except instead of retuning just once in the parent/child it returns + over and over again.

+ +

Matt : What I would like you to do + is to provide us how much this simplifies the user land thread + library.

+ +

Julian : Just to get basic threads + on normal code (reads, writes, anything that could block) there is a + set of code (~5000 lines of C) that has to keep track of this. The + entire user land thread scheduler is 10 lines.

+ +

Peter W : The user level thread + scheduler only works for networking but dies on disk stuff.

+ +

Robert : You get disk I/O + parallelism.

+ +

Alfred : I'd like to still be able + to use user land threading for networking.

+ +

Julian : There is no point in not + using. All we're allocating for you is a stack.

+ +

Matt : There is a partitioning + here. In I/O reading there is a problem, but in writing there is + not. Only 1% of writes now block. If you're reading 1000s of + sockets there is an issue.

+ +

Robert : KSEs are very general. + Doing the basic stuff is the right answer.

+ +

Julian : My theory is in fact that + nwe'll keep the current code and provide a new library. I don't want + to be responsible for the entire threading system.

+ +

Matt : We can always change the + default.

+ +

Julian : Just as an aside. As + part of this work I had to rearrange the way in which threads are + done. We now have a pool of free threads. It turns out that I now + have a cache of threads. Thread reapers go in wait() (called by + exit()).

+ +

Nick S : Corner case. Simple app + that you register KSE callback thing and then it makes a call into a + blocking syscall and blocks.

+ +

Julian : That thread is blocked.

+ +

Nick S : What happens when there + is nothing to do?

+ +

Julian : It calls yield() and gets + discarded. There is a mailbox between that could be used to tell + the kernel "never call me" when the process knows that its + blocked.

+ +

Greg : What are the performance + implications?

+ +

Julian : None.

+ +

Peter W : For disk performance it + will be great.

+ +

Julian : For a non-threaded + process on a non-KSE kernel I can't see any difference.

+ +

Robert : Questions on the + phone?

+ +

Phone : At what level does user + land thread scheduler operate?

+ +

Julian : It's a library that you + link with.

+ +

Phone : What about other + languages?

+ +

Julian : You need to write the + library. It's all very short.

+ +

Warner : Does this mean it breaks + the one application that we have that's not written in C?

+ +

Peter W: cvsup will work.

+
+
+ +
+
+ +

SMPng - John Baldwin

+ +

I actually attempted to set up a BOF on this at the conference. + The biggest topic of discussion was "how much do we expect ot have + done by 5.0?" A very conservative viewpoint is:

+ +

Overhaul process cred stuff.

+ +

Finish ktrace to work in an async kthread

+ +

Networking stack because its part of the target market and is a big + net gain.

+ +

Much of the discussion centered around sockets.

+ +

Another suggestion was to trace down VOP read and write path and + push down giant into that.

+ +

The last thing would be to make the kernel fully preemptable.

+ +
+

Greg : Where is the architectural + overview?

+ +

John : I'm working on that.

+ +

Greg : What about light weight + interrupts?

+ +

John : No real need.

+ +

Greg : I think we're going to fail + unless we have a good theoretical base.

+ +

David : You've told us 3 things + you've wanted for 5.0. These are micro-goals. What is the big + picture?

+ +

John : The direction is a "fine + grained locked kernel".

+ +

PoulHK : I'm sure we can come up + with an architectural paper. Very little of 4BSD ever made it in + either. I don't believe that can we make a full map.

+ +

Greg : I think we can.

+ +

PoulHK : We're talking about + redoing sections of code.

+ +

John : What key milestones?

+ +

Greg : Light weight threads.

+ +

Peter/John : Already done.

+ +

Justin : Having been at the SMPng + meeting the general consensus was to come up with a framework. LWT + is an optimization. Maybe only to 2 CPUs at 5.0

+ +

David : What is it that you've + accomplished from a high level?

+ +

John : Almost all the work is + infrastructure. When BSD/OS did SMP they added mutexes and are + using lock manager locks.

+ +

Bosko : LWI for x86 we just switch + contexts it has a very minimal impact. The code is available

+ +

We don't generate code on the fly.

+ +

John : Current Status con't. + We've taken more time to get it right. We've added common things + like semaphores, reader writer locks. I've been making the kernel + fully pre-emptible. I've commited half of this to current now. The + ktrace is another infrastructural issue so its feasible.

+ +

David : Then general framework + will be done by 5.0. Second to test the infrastructure you've been + locking the proc structure. For 5.0 we're still not talking about + super strong, fine grained SMP.

+ +

John : That's why we want to do + the network stuff.

+ +

PoulHK : Geom and devfs can do + this now. We don't need to wait. We have various pieces of code in + the tree that can be taken out from under giant.

+ +

Matt : Just an example of what + we'll probably be able to do fairly soon are the fsops in the file. + For things like pipes, /dev/null/, /dev/zero. The i/o paths we have + to concentrate on the most are read/write for vfsops. If we can't + fine grain the others in 5.0 oh well.

+ +

Anon : What specifically are you + planning in terms of performance gains before the relase? Do we + have any more firm of a schedule?

+ +

John : 2nd question (schedule) is + for later discussion. 1st is "no more than 5% loss." I want to run + some real benchmarks.

+ +

Warner : What benchmark are you + going to use?

+ +

Luigi : What if we totally miss + the numbers?

+ +

John : We'll have to revisit the + whole release.

+ +

Peter : Turning spls into mutexs + doesn't help us.

+ +

John : spls as mutexes still don't + get us out from giant.

+ +

PoulHK : I need documentation.

+ +

Anon : I will be happy to help you + with words.

+ +

Warner : I signed up to do locking + for newbus.

+ +

Paul Richards : I'll help on + documentation. We still need a roadmap.

+ +

Robert : The one thing on the must + lock list is the network.

+ +

PoulHK : We can just put one lock + around the network.

+ +

Nick S : On a uniprocessor the + locks are just null right?

+ +

John : Yes on spin locks.

+ +

Nick S : Nevermind.

+ +

Luigi : Do we care about + performance on uniprocessors on 5.0?

+ +

John : One thign that SMPng may + help buy is that if you have two network interfaces then you can + handle more stuff.

+ +

Luigi : Do we care or not about + uniprocessor on 5.0.

+ +

John : Yes

+ +

Julian : In the uniprocessor case + KSE degrades down to forkeed processors.

+ +

Peter W : Need a graph of the + locking the subsystems.

+
+
+ +

Break

+ +
+ +

5.0 Release Engineering - Murray Stokely

+ +

There are a number changes to the team. Murray, Robert, John, + Bruce Mah. Change review committee. Documented the process.

+ +

We think its pretty important to take a more active role. Want to + do developer previews (polished snapshot). April 1 will be preview + 1. Goals are:

+ + + +

Got a bit of a feature list for 5.0 final. SMPng is broken down + into several sections. UFS2. KSE. PAM overhaul. TrustedBSD etc.

+ +

I'd like some feeback on this.

+ +

Discussion

+ +
+

Warner : Are we doing to try to + have the developer release 2 done so we can hand out CDs at + Usenix?

+ +

John : That might be pushing + it.

+ +

Anon : Can we push Usenix back a + bit? (Laughter)

+ +

David : Feature freeze is a code + slush. Will I as a commiter see a freeze?

+ +

Murray : You will not have to + worry about bug fixes but you must act rationally.

+ +

Paul R : Do we really need + multiple release branches?

+ +

Robert : Good to have around.

+ +

Alfred : Could we reach a + consensus on what sort of debugging will be in 5.0?

+ +

Robert : We want to get this to + early access people.

+ +

Murray : This is an opt in thing + anyway.

+ +

Alfred : We need a list of the + debug options etc.

+ +

Warner : The cardbus will ship + with debugging turned on but its tunable.

+ +

PoulHK : We need to know if we're + going to turn off the a/j options on malloc3().

+ +

Doug : I don't know how tied you + are to the release schedule. If you want to spend all of October + polishing. If we go backwards from October we can do Release 1 on + May 15. April 1st is too soon and puts you in wierd catagories + relative to Usenix.

+ +

Murray : The way we have it set up + now... (Graphic)

+ +
	April 1 (DP 1)
+
+	June 9-14 (Usenix)
+
+	June 25 (DP2)
+
+	October 1st is Feature Freeze
+
+	October 20th is Code Freeze
+
+	Nov 20th 5.0 Release
+
+ALTERNATE
+
+	May 15 DP 1
+
+	July DP 2
+ +

Murray : The number of people + running current is too small.

+ +

Paul Richards : Not much point in + doing DP 1 if DP2 is totally different.

+ +

Murray : But that means that other + non SMP stuff is still not in.

+ +

David : People are talking about + what they're planning for 5.0 Perhaps you might to poll the people + who have balls in the air. Is there anything that April 1 is going + to give us?

+ +

Murray : Wide spread testing.

+ +

Alfred : I think the sooner the + better.

+ +

Justin : Why do we think that just + because we put together a shrink wrap that people will use it?

+ +

Murray : Because its a discipline + on all of us to get this stuff in there. If we're moving towards a + goal we need to do that.

+ +

Justin : This is not about the + final release but about what is in the tree today. Therefore the + first preview is not an interesting release.

+ +

Murray : Its concrete steps + towards 5.0 release.

+ +

Justin : I don't know that it's + necessary.

+ +

Doug : We're going to have people + generating bug reports against things that are spurious problem + reports.

+ +

Warner : Cardbus vs. PCI interrupt + routing is an example of this.

+ +

Justin : I guess the way I look at + this is that we force productization is to force the date. Why is + it that on a daily basis that things suck so bad?

+ +

David : Why does it suck so bad?

+ +

Justin : Because people don't get + shat on for breaking things.

+ +

Robert : Why does it suck?

+ +

Justin : I put it on my server and + its slow. Instead of fixing PCI interrupts.

+ +

PoulHK : I'd like to point out + that we have substantial historical experience with all .0 releases. + It is indisputable that a snapshot CDROM makes people run it. + Getting something out there is crucial to the testing of CURRENT. + If we simply call this current snapshot that will be best. Set your + date and roll your CD.

+ +

Gnn : These are different issues.

+ +

Justin : The people that used to + complain about it but don't anymore.

+ +

Alfred : We have 4 more platforms, + it's more difficult now.

+ +

Murray : We have build machines + etc.

+ +

Justin : When I first joined + people were in it because they had to do stuff incorrectly because + they had to do things on time. If we're going to engineer a real + product then yeah, it's difficult but that's the reason why this is + a cool project. So just giving up and saying it is hard is BS.

+ +

Gnn : Process, process, process...

+ +

Julian : Breaking the build is not + as bad as breaking the kernel. What's harder is committing a + subsystem that affects another subsystem. In terms of the proecess + I'd like to see a best practices document. On how people develop + patches etc. A list of things you should do etc.

+ +

David : You can do that from + freefall.

+ +

Julian : As soon as I find out its + broken I'll fix it.

+ +

David : I have posted them many + times.

+ +

Justin : One other thing I would + suggest is that instead of becoming active only during release you + should be doing this full time. If people start putting stuff in + the tree today that's not going in to 5.0 then slap their hands. + Make a window so that people...

+ +

Peter : The Mozilla tackled the + build problems with tinderbox. This will solve a lot of + problems.

+ +
	while (1)
+	{
+	      build
+	      if (break)
+		 send mail to those who committed most recently
+	      else
+	         clear list of recent committers
+	}
+ +

Nick S : The suggestion that + tangential features should be barred until after 5.0 bugs me.

+ +

Justin : It should not be your + arbitrary decision. It should be whatever body is empowered to make + that decision.

+ +

Robert : We're using the Usenix to + November window.

+ +

Paul Richards : This is a + volunteer project. In our volunteer time we wanted to what we + couldn't do in our normal jobs. Volunteers will do things by rules.

+ +

PoulHK : This project was started + by people for fun. Every time you use the word enforce you get + fun--. It is much better to inspire than to enforce. You should + not let people get away with things. I hate to say this but some + times you need to slap fingers. Many times people will thank you + for it. We see it again that people go off and need to be brought + back.

+ +

Robert : We have 10 minutes left. + Let's wrap up.

+ +

Paul Richards : Can we take + Justin's proposal?

+ +

Peter : Going from chaos to clamp + down. This can push a code fork.

+ +

Doug : One is that in regards to + PoulHK said in addition to the potential cost of disciplining a + commiter you have to measure the cost against the others who want + that person gone. How many more people could we attract if that + stress wasn't present.

+ +

Justin : You either have a good + manager or a bad manager. Good managers trust you. Are the people + in the release engineering team going to be good managers?

+ +

Peter : The MFC process seems to + work nicely but going further may not be the best idea.

+ +

Robert : Do you think the schedule + is going too far?

+ +

Julian : Usenix may not be such a + large audience to worry about. People there are technical enough. + On how many snapshots we have. I've been wondering whether we + should have 5.0 on the disc but 5.0 and the cvs tree and cvsup so + people can easily upgrade. Bootstrap CDs.

+ +

Murray : Any issues with the + name?

+ + +

PoulHK : Why not snapshot?

+ +

Warner : I think the dates look + good but I would pick a different.

+
+
+ +
+
+ +

Concluding Remarks - Robert Watson

+ +
+ +

Robert : Should we do another of + these at Usenix?

+ +

All : Yes.

+ +

John : 2 days?

+ +

Robert : What could we do + better?

+ + +

Sundry : parking validation, + connectivity, projector, catered lunch, ...

+ +

PoulHK : I would like to propose + we make a formal hour every month to talk about on irc?

+
+
+ + &footer; + + diff --git a/en/events/Makefile.inc b/en/events/Makefile.inc new file mode 100644 index 0000000000..9e715a77ff --- /dev/null +++ b/en/events/Makefile.inc @@ -0,0 +1,3 @@ +# $FreeBSD$ + +WEB_PREFIX?= ${.CURDIR}/../../.. diff --git a/en/events/events.css b/en/events/events.css new file mode 100644 index 0000000000..2dfea2af7d --- /dev/null +++ b/en/events/events.css @@ -0,0 +1,18 @@ + + +div.topic { + margin-top: 1em; + margin-bottom: 1em; +} + +div.discussion { + margin-left: 1cm; +} + +strong.speaker { + color: #660000; +} + +strong.action { + color: red; +}