MFC r311172,r311194,r311226,r312389,r312390:

mtx: reduce lock accesses

Instead of spuriously re-reading the lock value, read it once.

This change also has a side effect of fixing a performance bug:
on failed _mtx_obtain_lock, it was possible that re-read would find
the lock is unowned, but in this case the primitive would make a trip
through turnstile code.

This is diff reduction to a variant which uses atomic_fcmpset.

==

Reduce lock accesses in thread lock similarly to r311172

==

mtx: plug open-coded mtx_lock access missed in r311172

==

rwlock: reduce lock accesses similarly to r311172

==

sx: reduce lock accesses similarly to r311172

1 files changed, 7 insertions, 1 deletions

diff --git a/sys/sys/mutex.h b/sys/sys/mutex.h
index 374aaab..11ed9d7 100644
--- a/sys/sys/mutex.h
+++ b/sys/sys/mutex.h
@@ -420,9 +420,15 @@ extern struct mtx_pool *mtxpool_sleep;
 	_sleep((chan), &(mtx)->lock_object, (pri), (wmesg),		\
 	    tick_sbt * (timo), 0, C_HARDCLOCK)
 
+#define	MTX_READ_VALUE(m)	((m)->mtx_lock)
+
 #define	mtx_initialized(m)	lock_initialized(&(m)->lock_object)
 
-#define mtx_owned(m)	(((m)->mtx_lock & ~MTX_FLAGMASK) == (uintptr_t)curthread)
+#define lv_mtx_owner(v)	((struct thread *)((v) & ~MTX_FLAGMASK))
+
+#define mtx_owner(m)	lv_mtx_owner(MTX_READ_VALUE(m))
+
+#define mtx_owned(m)	(mtx_owner(m) == curthread)
 
 #define mtx_recursed(m)	((m)->mtx_recurse != 0)