Fix a deficiency in the selinfo interface:

If a selinfo object is recorded (via selrecord()) and then it is
quickly destroyed, with the waiters missing the opportunity to awake,
at the next iteration they will find the selinfo object destroyed,
causing a PF#.

That happens because the selinfo interface has no way to drain the
waiters before to destroy the registered selinfo object. Also this
race is quite rare to get in practice, because it would require a
selrecord(), a poll request by another thread and a quick destruction
of the selrecord()'ed selinfo object.

Fix this by adding the seldrain() routine which should be called
before to destroy the selinfo objects (in order to avoid such case),
and fix the present cases where it might have already been called.
Sometimes, the context is safe enough to prevent this type of race,
like it happens in device drivers which installs selinfo objects on
poll callbacks. There, the destruction of the selinfo object happens
at driver detach time, when all the filedescriptors should be already
closed, thus there cannot be a race.
For this case, mfi(4) device driver can be set as an example, as it
implements a full correct logic for preventing this from happening.

Sponsored by:	Sandvine Incorporated
Reported by:	rstone
Tested by:	pluknet
Reviewed by:	jhb, kib
Approved by:	re (bz)
MFC after:	3 weeks

2 files changed, 20 insertions, 3 deletions

diff --git a/share/man/man9/Makefile b/share/man/man9/Makefile
index a04cb68..05bab57 100644
--- a/share/man/man9/Makefile
+++ b/share/man/man9/Makefile
@@ -1075,7 +1075,7 @@ MLINKS+=scheduler.9 curpriority_cmp.9 \
 	scheduler.9 setrunnable.9 \
 	scheduler.9 updatepri.9
 MLINKS+=securelevel_gt.9 securelevel_ge.9
-MLINKS+=selrecord.9 selwakeup.9
+MLINKS+=seldrain.9 selrecord.9 selwakeup.9
 MLINKS+=sema.9 sema_destroy.9 \
 	sema.9 sema_init.9 \
 	sema.9 sema_post.9 \
diff --git a/share/man/man9/selrecord.9 b/share/man/man9/selrecord.9
index 15c5a18..f7a0113 100644
--- a/share/man/man9/selrecord.9
+++ b/share/man/man9/selrecord.9
@@ -26,10 +26,11 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd June 13, 2007
+.Dd August 25, 2011
 .Dt SELRECORD 9
 .Os
 .Sh NAME
+.Nm seldrain ,
 .Nm selrecord ,
 .Nm selwakeup
 .Nd "record and wakeup select requests"
@@ -37,14 +38,17 @@
 .In sys/param.h
 .In sys/selinfo.h
 .Ft void
+.Fn seldrain "struct selinfo *sip"
+.Ft void
 .Fn selrecord "struct thread *td" "struct selinfo *sip"
 .Ft void
 .Fn selwakeup "struct selinfo *sip"
 .Sh DESCRIPTION
+.Fn seldrain ,
 .Fn selrecord
 and
 .Fn selwakeup
-are the two central functions used by
+are the three central functions used by
 .Xr select 2 ,
 .Xr poll 2
 and the objects that are being selected on.
@@ -86,6 +90,15 @@ and
 .Xr poll 2
 when they wake up.
 .Pp
+.Fn seldrain
+will flush the waiters queue on a specified object before its
+destruction.
+The object handling code must ensure that
+.Fa *sip
+cannot be used once
+.Fn seldrain
+has been called.
+.Pp
 The contents of
 .Fa *sip
 must be zeroed, such as by softc initialization, before any call to
@@ -98,6 +111,10 @@ acquires and releases
 .Va sellock
 and may acquire and release
 .Va sched_lock .
+.Fn seldrain
+could usually be just a wrapper for
+.Fn selwakeup ,
+but consumers should not generally rely on this feature.
 .Sh SEE ALSO
 .Xr poll 2 ,
 .Xr select 2