Add code to make default mutexes adaptive if the ADAPTIVE_MUTEXES kernel

option is used (not on by default).

- In the case of trying to lock a mutex, if the MTX_CONTESTED flag is set,
  then we can safely read the thread pointer from the mtx_lock member while
  holding sched_lock.  We then examine the thread to see if it is currently
  executing on another CPU.  If it is, then we keep looping instead of
  blocking.
- In the case of trying to unlock a mutex, it is now possible for a mutex
  to have MTX_CONTESTED set in mtx_lock but to not have any threads
  actually blocked on it, so we need to handle that case.  In that case,
  we just release the lock as if MTX_CONTESTED was not set and return.
- We do not adaptively spin on Giant as Giant is held for long times and
  it slows SMP systems down to a crawl (it was taking several minutes,
  like 5-10 or so for my test alpha and sparc64 SMP boxes to boot up when
  they adaptively spinned on Giant).
- We only compile in the code to do this for SMP kernels, it doesn't make
  sense for UP kernels.

Tested on:	i386, alpha, sparc64

2 files changed, 6 insertions, 0 deletions

diff --git a/sys/conf/NOTES b/sys/conf/NOTES
index 1d431e3..688de2f 100644
--- a/sys/conf/NOTES
+++ b/sys/conf/NOTES
@@ -120,6 +120,11 @@ options 	ROOTDEVNAME=\"ufs:da0s2e\"
 # Mandatory:
 options 	SMP			# Symmetric MultiProcessor Kernel
 
+# ADAPTIVE_MUTEXES changes the behavior of blocking mutexes to spin
+# if the thread that currently owns the mutex is executing on another
+# CPU.
+options 	ADAPTIVE_MUTEXES
+
 # SMP Debugging Options:
 #
 # MUTEX_DEBUG enables various extra assertions in the mutex code.
diff --git a/sys/conf/options b/sys/conf/options
index 14133e2..5caf9b5 100644
--- a/sys/conf/options
+++ b/sys/conf/options
@@ -57,6 +57,7 @@ ADW_ALLOW_MEMIO		opt_adw.h	# Allow PCI devices to use memory
 
 # Miscellaneous options.
 GEOM		opt_geom.h
+ADAPTIVE_MUTEXES
 COMPAT_43	opt_compat.h
 COMPAT_SUNOS	opt_compat.h
 COMPILING_LINT	opt_global.h