Add a mechanism for associating a mutex with a callout when the

callout is first initialised, using a new function callout_init_mtx().
The callout system will acquire this mutex before calling the callout
function and release it on return.

In addition, the callout system uses the mutex to avoid most of the
complications and race conditions inherent in asynchronous timer
facilities, so mutex-protected callouts have much simpler semantics.
As long as the mutex is held when invoking callout_stop() or
callout_reset(), then these functions will guarantee that the callout
will be stopped, even if softclock() had already begun to process
the callout.

Existing Giant-locked callouts will automatically pick up the new
race-free semantics. This should close a number of race conditions
in the USB code and probably other areas of the kernel too.

There should be no change in behaviour for "MP-safe" callouts; these
still need to use the techniques mentioned in timeout(9) to avoid
race conditions.

2 files changed, 56 insertions, 2 deletions

diff --git a/share/man/man9/Makefile b/share/man/man9/Makefile
index 660dda3..7d5c499 100644
--- a/share/man/man9/Makefile
+++ b/share/man/man9/Makefile
@@ -919,9 +919,14 @@ MLINKS+=taskqueue.9 TASK_INIT.9 \
 MLINKS+=time.9 boottime.9 \
 	time.9 time_second.9 \
 	time.9 time_uptime.9
-MLINKS+=timeout.9 callout_drain.9 \
+MLINKS+=timeout.9 callout.9 \
+	timeout.9 callout_active.9 \
+	timeout.9 callout_deactivate.9 \
+	timeout.9 callout_drain.9 \
 	timeout.9 callout_handle_init.9 \
 	timeout.9 callout_init.9 \
+	timeout.9 callout_init_mtx.9 \
+	timeout.9 callout_pending.9 \
 	timeout.9 callout_reset.9 \
 	timeout.9 callout_stop.9 \
 	timeout.9 untimeout.9
diff --git a/share/man/man9/timeout.9 b/share/man/man9/timeout.9
index 19eaaf6..fa4c5b9 100644
--- a/share/man/man9/timeout.9
+++ b/share/man/man9/timeout.9
@@ -36,7 +36,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd January 23, 2005
+.Dd February 6, 2005
 .Dt TIMEOUT 9
 .Os
 .Sh NAME
@@ -44,6 +44,7 @@
 .Nm untimeout ,
 .Nm callout_handle_init ,
 .Nm callout_init ,
+.Nm callout_init_mtx ,
 .Nm callout_stop ,
 .Nm callout_drain ,
 .Nm callout_reset ,
@@ -70,6 +71,8 @@ struct callout_handle handle = CALLOUT_HANDLE_INITIALIZER(&handle)
 .Fn untimeout "timeout_t *func" "void *arg" "struct callout_handle handle"
 .Ft void
 .Fn callout_init "struct callout *c" "int mpsafe"
+.Ft void
+.Fn callout_init_mtx "struct callout *c" "struct mtx *mtx" "int flags"
 .Ft int
 .Fn callout_stop "struct callout *c"
 .Ft int
@@ -178,6 +181,7 @@ Thus they are protected from re-entrancy.
 .Pp
 The functions
 .Fn callout_init ,
+.Fn callout_init_mtx ,
 .Fn callout_stop ,
 .Fn callout_drain
 and
@@ -202,6 +206,26 @@ that is,
 the Giant lock will be acquired before calling the callout function,
 and released when the callout function returns.
 .Pp
+The
+.Fn callout_init_mtx
+function may be used as an alternative to
+.Fn callout_init .
+The parameter
+.Fa mtx
+specifies a mutex that is to be acquired by the callout subsystem
+before calling the callout function, and released when the callout
+function returns.
+The following
+.Fa flags
+may be specified:
+.Bl -tag -width CALLOUT_RETURNUNLOCKED
+.It Dv CALLOUT_RETURNUNLOCKED
+The callout function will release
+.Fa mtx
+itself, so the callout subsystem should not attempt to unlock it
+after the callout function returns.
+.El
+.Pp
 The function
 .Fn callout_stop
 cancels a callout if it is currently pending.
@@ -210,6 +234,8 @@ If the callout is pending, then
 will return a non-zero value.
 If the callout is not set, has already been serviced or is currently
 being serviced, then zero will be returned.
+If the callout has an associated mutex, then that mutex must be
+held when this function is called.
 .Pp
 The function
 .Fn callout_drain
@@ -234,6 +260,8 @@ first performs the equivalent of
 to disestablish the callout, and then establishes a new callout in the
 same manner as
 .Fn timeout .
+If the callout has an associated mutex, then that mutex must be
+held when this function is called.
 .Pp
 The macros
 .Fn callout_pending ,
@@ -295,6 +323,27 @@ The callout subsystem provides a number of mechanisms to address these
 synchronization concerns:
 .Bl -enum -offset indent
 .It
+If the callout has an associated mutex that was specified using the
+.Fn callout_init_mtx
+function (or implicitly specified as the
+.Va Giant
+mutex using
+.Fn callout_init
+with
+.Fa mpsafe
+set to
+.Dv FALSE ) ,
+then this mutex is used to avoid the race conditions.
+The associated mutex must be acquired by the caller before calling
+.Fn callout_stop
+or
+.Fn callout_reset
+and it is guaranteed that the callout will be correctly stopped
+or reset as expected.
+Note that it is still necessary to use
+.Fn callout_drain
+before destroying the callout or its associated mutex.
+.It
 The return value from
 .Fn callout_stop
 indicates whether or not the callout was removed.