summaryrefslogtreecommitdiffstats
path: root/sys/kern/subr_lock.c
Commit message (Collapse)AuthorAgeFilesLines
* - Implement run-time expansion of the KTR buffer via sysctl.jeff2012-11-151-25/+4
| | | | | | | | | | - Implement a function to ensure that all preempted threads have switched back out at least once. Use this to make sure there are no stale references to the old ktr_buf or the lock profiling buffers before updating them. Reviewed by: marius (sparc64 parts), attilio (earlier patch) Sponsored by: EMC / Isilon Storage Division
* panic: add a switch and infrastructure for stopping other CPUs in SMP caseavg2011-12-111-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Historical behavior of letting other CPUs merily go on is a default for time being. The new behavior can be switched on via kern.stop_scheduler_on_panic tunable and sysctl. Stopping of the CPUs has (at least) the following benefits: - more of the system state at panic time is preserved intact - threads and interrupts do not interfere with dumping of the system state Only one thread runs uninterrupted after panic if stop_scheduler_on_panic is set. That thread might call code that is also used in normal context and that code might use locks to prevent concurrent execution of certain parts. Those locks might be held by the stopped threads and would never be released. To work around this issue, it was decided that instead of explicit checks for panic context, we would rather put those checks inside the locking primitives. This change has substantial portions written and re-written by attilio and kib at various times. Other changes are heavily based on the ideas and patches submitted by jhb and mdf. bde has provided many insights into the details and history of the current code. The new behavior may cause problems for systems that use a USB keyboard for interfacing with system console. This is because of some unusual locking patterns in the ukbd code which have to be used because on one hand ukbd is below syscons, but on the other hand it has to interface with other usb code that uses regular mutexes/Giant for its concurrency protection. Dumping to USB-connected disks may also be affected. PR: amd64/139614 (at least) In cooperation with: attilio, jhb, kib, mdf Discussed with: arch@, bde Tested by: Eugene Grosbein <eugen@grosbein.net>, gnn, Steven Hartland <killing@multiplay.co.uk>, glebius, Andrew Boyer <aboyer@averesystems.com> (various versions of the patch) MFC after: 3 months (or never)
* Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.ed2011-11-071-2/+3
| | | | | | The SYSCTL_NODE macro defines a list that stores all child-elements of that node. If there's no SYSCTL_DECL macro anywhere else, there's no reason why it shouldn't be static.
* Explicitly wire the user buffer rather than doing it implicitly inmdf2011-01-271-0/+3
| | | | | | | | sbuf_new_for_sysctl(9). This allows using an sbuf with a SYSCTL_OUT drain for extremely large amounts of data where the caller knows that appropriate references are held, and sleeping is not an issue. Inspired by: rwatson
* Remove unneeded includes of <sys/linker_set.h>. Other headers that usejhb2011-01-111-1/+0
| | | | | | it internally contain nested includes. Reviewed by: bde
* Fix typos.brucec2010-11-091-1/+1
| | | | | PR: bin/148894 Submitted by: olgeni
* Re-add r212370 now that the LOR in powerpc64 has been resolved:mdf2010-09-161-14/+6
| | | | | | | | | | | | Add a drain function for struct sysctl_req, and use it for a variety of handlers, some of which had to do awkward things to get a large enough SBUF_FIXEDLEN buffer. Note that some sysctl handlers were explicitly outputting a trailing NUL byte. This behaviour was preserved, though it should not be necessary. Reviewed by: phk (original patch)
* Revert r212370, as it causes a LOR on powerpc. powerpc does a fewmdf2010-09-131-6/+14
| | | | | | | unexpected things in copyout(9) and so wiring the user buffer is not sufficient to perform a copyout(9) while holding a random mutex. Requested by: nwhitehorn
* Add a drain function for struct sysctl_req, and use it for a variety ofmdf2010-09-091-14/+6
| | | | | | | | | | handlers, some of which had to do awkward things to get a large enough FIXEDLEN buffer. Note that some sysctl handlers were explicitly outputting a trailing NUL byte. This behaviour was preserved, though it should not be necessary. Reviewed by: phk
* Use ISO C99 integer types in sys/kern where possible.ed2010-06-211-6/+6
| | | | | | There are only about 100 occurences of the BSD-specific u_int*_t datatypes in sys/kern. The ISO C99 integer types are used here more often.
* lock_profile_release_lock: do not compare unsigned with zeroavg2010-06-171-3/+5
| | | | | | | Found by: Coverity Prevent CID: 3660 Reviewed by: jhb MFC after: 2 weeks
* Update several places that iterate over CPUs to use CPU_FOREACH().jhb2010-06-111-3/+1
|
* - Implement a new mechanism for resetting lock profiling. We nowjeff2009-03-151-9/+101
| | | | | | | | | | | | | | | | | guarantee that all cpus have acknowledged the cleared enable int by scheduling the resetting thread on each cpu in succession. Since all lock profiling happens within a critical section this guarantees that all cpus have left lock profiling before we clear the datastructures. - Assert that the per-thread queue of locks lock profiling is aware of is clear on thread exit. There were several cases where this was not true that slows lock profiling and leaks information. - Remove all objects from all lists before clearing any per-cpu information in reset. Lock profiling objects can migrate between per-cpu caches and previously these migrated objects could be zero'd before they'd been removed Discussed with: attilio Sponsored by: Nokia
* - track maximum wait timekmacy2008-07-271-4/+9
| | | | | | - resize columns based on actual observed numerical values MFC after: 3 days
* - Embed the recursion counter for any locking primitive directly in theattilio2008-05-151-4/+1
| | | | | | | | | | | | | | | | | | | | | | | lock_object, using an unified field called lo_data. - Replace lo_type usage with the w_name usage and at init time pass the lock "type" directly to witness_init() from the parent lock init function. Handle delayed initialization before than witness_initialize() is called through the witness_pendhelp structure. - Axe out LO_ENROLLPEND as it is not really needed. The case where the mutex init delayed wants to be destroyed can't happen because witness_destroy() checks for witness_cold and panic in case. - In enroll(), if we cannot allocate a new object from the freelist, notify that to userspace through a printf(). - Modify the depart function in order to return nothing as in the current CVS version it always returns true and adjust callers accordingly. - Fix the witness_addgraph() argument name prototype. - Remove unuseful code from itismychild(). This commit leads to a shrinked struct lock_object and so smaller locks, in particular on amd64 where 2 uintptr_t (16 bytes per-primitive) are gained. Reviewed by: jhb
* Really, no explicit checks against against lock_class_* object should beattilio2008-02-061-2/+2
| | | | | | | | | | | | | done in consumers code: using locks properties is much more appropriate. Fix current code doing these bogus checks. Note: Really, callout are not usable by all !(LC_SPINLOCK | LC_SLEEPABLE) primitives like rmlocks doesn't implement the generic lock layer functions, but they can be equipped for this, so the check is still valid. Tested by: matteo, kris (earlier version) Reviewed by: jhb
* Fix logic in skipcount handling (used to sample every 1/N lock operationskris2008-01-081-1/+1
| | | | to reduce profiling overhead)
* - Pause a while after disabling lock profiling and before resetting itjeff2007-12-311-5/+11
| | | | | to be sure that all participating CPUs have stopped updating it. - Restore the behavior of printing the name of the lock type in the output.
* - Re-implement lock profiling in such a way that it no longer breaksjeff2007-12-151-222/+434
| | | | | | | | | | | | | | | | | | | | | | the ABI when enabled. There is no longer an embedded lock_profile_object in each lock. Instead a list of lock_profile_objects is kept per-thread for each lock it may own. The cnt_hold statistic is now always 0 to facilitate this. - Support shared locking by tracking individual lock instances and statistics in the per-thread per-instance lock_profile_object. - Make the lock profiling hash table a per-cpu singly linked list with a per-cpu static lock_prof allocator. This removes the need for an array of spinlocks and reduces cache contention between cores. - Use a seperate hash for spinlocks and other locks so that only a critical_enter() is required and not a spinlock_enter() to modify the per-cpu tables. - Count time spent spinning in the lock statistics. - Remove the LOCK_PROFILE_SHARED option as it is always supported now. - Specifically drop and release the scheduler locks in both schedulers since we track owners now. In collaboration with: Kip Macy Sponsored by: Nokia
* Initial checkin for rmlock (read mostly lock) a multi reader single writerups2007-11-081-0/+1
| | | | | | | | lock optimized for almost exclusive reader access. (see also rmlock.9) TODO: Convert to per cpu variables linkerset as soon as it is available. Optimize UP (single processor) case.
* Currently the LO_NOPROFILE flag (which is masked on upper level code byattilio2007-09-141-1/+1
| | | | | | | | | | | | | | | | | | per-primitive macros like MTX_NOPROFILE, SX_NOPROFILE or RW_NOPROFILE) is not really honoured. In particular lock_profile_obtain_lock_failure() and lock_profile_obtain_lock_success() are naked respect this flag. The bug leads to locks marked with no-profiling to be profiled as well. In the case of the clock_lock, used by the timer i8254 this leads to unpredictable behaviour both on amd64 and ia32 (double faults panic, sudden reboots, etc.). The amd64 clock_lock is also not marked as not profilable as it should be. Fix these bugs adding proper checks in the lock profiling code and at clock_lock initialization time. i8254 bug pointed out by: kris Tested by: matteo, Giuseppe Cocomazzi <sbudella at libero dot it> Approved by: jeff (mentor) Approved by: re
* Revert some debugging KTRs that were added during development.kris2007-06-031-4/+0
|
* Move lock_profile_object_{init,destroy}() into lock_{init,destroy}().jhb2007-05-181-0/+2
|
* skip call to _lock_profile_obtain_lock_success entirely if acquisition time ↵kmacy2007-04-031-14/+11
| | | | | | is non-zero (i.e. recursing or adding sharers)
* general LOCK_PROFILING cleanupkmacy2007-02-261-55/+6
| | | | | | | | | | | | - only collect timestamps when a lock is contested - this reduces the overhead of collecting profiles from 20x to 5x - remove unused function from subr_lock.c - generalize cnt_hold and cnt_lock statistics to be kept for all locks - NOTE: rwlock profiling generates invalid statistics (and most likely always has) someone familiar with that should review
* Bug fix for obscenely large wait times on uncontested lockskmacy2006-12-041-4/+4
| | | | | | | if waittime was zero (the lock was uncontested) l->lpo_waittime in the hash table would not get initialized. Inspection prompted by questions from: Attilio Rao
* track lock class name in a way that doesn't break WITNESSkmacy2006-11-131-3/+4
|
* Unbreak witnesskmacy2006-11-121-1/+1
|
* show lock class in profiling output for default case where type is not ↵kmacy2006-11-121-19/+22
| | | | | | specified when initializing the lock Approved by: scottl (standing in for mentor rwatson)
* tinderbox fixkmacy2006-11-111-1/+1
|
* remove lingering call to rd(tick)kmacy2006-11-111-1/+1
|
* missed nits replacing mutex with lockkmacy2006-11-111-4/+4
|
* MUTEX_PROFILING has been generalized to LOCK_PROFILING. We now profilekmacy2006-11-111-0/+291
| | | | | | | | | | | wait (time waited to acquire) and hold times for *all* kernel locks. If the architecture has a system synchronized TSC, the profiling code will use that - thereby minimizing profiling overhead. Large chunks of profiling code have been moved out of line, the overhead measured on the T1 for when it is compiled in but not enabled is < 1%. Approved by: scottl (standing in for mentor rwatson) Reviewed by: des and jhb
* Add a basic reader/writer lock implementation to the kernel. Thisjhb2006-01-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | implementation is by no means perfect as far as some of the algorithms that it uses and the fact that it is missing some functionality (try locks and upgrades/downgrades are not there yet), however it does seem to work in my local testing. There is more detail in the comments in the code, but the short version follows. A reader/writer lock is very much like a regular mutex: it cannot be held across a voluntary sleep; it can be acquired in an interrupt thread; if the lock is held by a writer then the priority of any threads that block on the lock will be lent to the owner; the simple case lock operations all are done in a single atomic op. It also shares some similiarities with sx locks: it supports reader/writer semantics (multiple readers, but single writers); readers are allowed to recurse, but writers are not. We can extend this implementation further by either improving algorithms or adding new functionality, but this should at least give us a base to work with now. Reviewed by: arch (in theory) Tested on: i386 (4 cpu box with a kernel module that used 4 threads that randomly chose between read locks and write locks that ran w/o panicing for over a day solid. It usually panic'd within a few seconds when there were bugs during testing. :) The kernel module source is available on request.)
* Always include the lock_classes[] array in the kernel. Thejhb2006-01-181-2/+0
| | | | | | | "is it a spinlock" test in mtx_destroy() needs it even in non-debug kernels. Reported by: danfe
* Bah. Fix 'show lock' to actually be compiled in. I had just fixed this injhb2006-01-171-0/+3
| | | | p4 but had an older subr_lock.c on the machine I committed to CVS from.
* Add a new file (kern/subr_lock.c) for holding code related to structjhb2006-01-171-0/+113
lock_obj objects: - Add new lock_init() and lock_destroy() functions to setup and teardown lock_object objects including KTR logging and registering with WITNESS. - Move all the handling of LO_INITIALIZED out of witness and the various lock init functions into lock_init() and lock_destroy(). - Remove the constants for static indices into the lock_classes[] array and change the code outside of subr_lock.c to use LOCK_CLASS to compare against a known lock class. - Move the 'show lock' ddb function and lock_classes[] array out of kern_mutex.c over to subr_lock.c.
OpenPOWER on IntegriCloud