summaryrefslogtreecommitdiffstats
path: root/sys/kern/kern_synch.c
Commit message (Collapse)AuthorAgeFilesLines
* In a threaded world, differnt priorirites become properties ofjulian2002-02-111-22/+26
| | | | | | different entities. Make it so. Reviewed by: jhb@freebsd.org (john baldwin)
* Change the preemption code for software interrupt thread schedules andjhb2002-01-051-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mutex releases to not require flags for the cases when preemption is not allowed: The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent switching to a higher priority thread on mutex releease and swi schedule, respectively when that switch is not safe. Now that the critical section API maintains a per-thread nesting count, the kernel can easily check whether or not it should switch without relying on flags from the programmer. This fixes a few bugs in that all current callers of swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from fast interrupt handlers and the swi_sched of softclock needed this flag. Note that to ensure that swi_sched()'s in clock and fast interrupt handlers do not switch, these handlers have to be explicitly wrapped in critical_enter/exit pairs. Presently, just wrapping the handlers is sufficient, but in the future with the fully preemptive kernel, the interrupt must be EOI'd before critical_exit() is called. (critical_exit() can switch due to a deferred preemption in a fully preemptive kernel.) I've tested the changes to the interrupt code on i386 and alpha. I have not tested ia64, but the interrupt code is almost identical to the alpha code, so I expect it will work fine. PowerPC and ARM do not yet have interrupt code in the tree so they shouldn't be broken. Sparc64 is broken, but that's been ok'd by jake and tmm who will be fixing the interrupt code for sparc64 shortly. Reviewed by: peter Tested on: i386, alpha
* Modify the critical section API as follows:jhb2001-12-181-3/+0
| | | | | | | | | | | | | | | | | | | - The MD functions critical_enter/exit are renamed to start with a cpu_ prefix. - MI wrapper functions critical_enter/exit maintain a per-thread nesting count and a per-thread critical section saved state set when entering a critical section while at nesting level 0 and restored when exiting to nesting level 0. This moves the saved state out of spin mutexes so that interlocking spin mutexes works properly. - Most low-level MD code that used critical_enter/exit now use cpu_critical_enter/exit. MI code such as device drivers and spin mutexes use the MI wrappers. Note that since the MI wrappers store the state in the current thread, they do not have any return values or arguments. - mtx_intr_enable() is replaced with a constant CRITICAL_FORK which is assigned to curthread->td_savecrit during fork_exit(). Tested on: i386, alpha
* Add/correct description for some sysctl variables where it was missing.luigi2001-12-161-1/+2
| | | | | | | | The description field is unused in -stable, so the MFC there is equivalent to a comment. It can be done at any time, i am just setting a reminder in 45 days when hopefully we are past 4.5-release. MFC after: 45 days
* Assert that Giant is not held in mi_switch() unless the process statejhb2001-10-231-0/+4
| | | | is SMTX or SRUN.
* Introduce some jitter to the timing of the samples that determineiedowse2001-10-201-4/+15
| | | | | | | | | | | | the system load average. Previously, the load average measurement was susceptible to synchronisation with processes that run at regular intervals such as the system bufdaemon process. Each interval is now chosen at random within the range of 4 to 6 seconds. This large variation is chosen so that over the shorter 5-minute load average timescale there is a good dispersion of samples across the 5-second sample period (the time to perform 60 5-second samples now has a standard deviation of approx 4.5 seconds).
* Move the code that computes the system load average from vm_meter.ciedowse2001-10-201-3/+49
| | | | | | | | | | | | to kern_synch.c in preparation for adding some jitter to the inter-sample time. Note that the "vm.loadavg" sysctl still lives in vm_meter.c which isn't the right place, but it is appropriate for the current (bad) name of that sysctl. Suggested by: jhb (some time ago) Reviewed by: bde
* GC some #if 0'd code.jhb2001-09-211-8/+2
|
* Whitespace and spelling fixes.jhb2001-09-211-2/+2
|
* KSE Milestone 2julian2001-09-121-167/+220
| | | | | | | | | | | | | | Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
* Make yield() MPSAFE.dillon2001-09-011-1/+6
| | | | | | Synchronize syscalls.master with all MPSAFE changes to date. Synchronize new syscall generation follows because yield() will panic if it is out of sync with syscalls.master.
* Release the sched_lock before bombing out in mi_switch() via db_error().jhb2001-08-211-1/+3
| | | | | This makes things slightly easier if you call a function that calls mi_switch() as it keeps the locking before and after closer.
* Add a hook to mi_switch() to abort via db_error() if we attempt tojhb2001-08-211-0/+12
| | | | | | perform a context switch from DDB. Consulting from: bde
* - Fix a bug in the previous workaround for the tsleep/endtsleep race.jhb2001-08-211-2/+5
| | | | | | | | | | | | | | | | callout_stop() would fail in two cases: 1) The timeout was currently executing, and 2) The timeout had already executed. We only needed to work around the race for 1). We caught some instances of 2) via the PS_TIMEOUT flag, however, if endtsleep() fired after the process had been woken up but before it had resumed execution, PS_TIMEOUT would not be set, but callout_stop() would fail, so we would block the process until endtsleep() resumed it. Except that endtsleep() had already run and couldn't resume it. This adds a new flag PS_TIMOFAIL to indicate the case of 2) when PS_TIMEOUT isn't set. - Implement this race fix for condition variables as well. Tested by: sos
* - Close races with signals and other AST's being triggered while we are injhb2001-08-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
* Work around a race between msleep() and endtsleep() where it was possiblejhb2001-08-101-3/+23
| | | | | | | | | for endtsleep() to be executing when msleep() resumed, for endtsleep() to spin on sched_lock long enough for the other process to loop on msleep() and sleep again resulting in endtsleep() waking up the "wrong" msleep. Obtained from: BSD/OS
* Style nit: covert a couple of if (p_wchan) tests to if (p_wchan != NULL).jhb2001-08-101-3/+3
|
* - Remove asleep(), await(), and M_ASLEEP.jhb2001-08-101-181/+1
| | | | | | | | | - Callers of asleep() and await() have been converted to calling tsleep(). The only caller outside of M_ASLEEP was the ata driver, which called both asleep() and await() with spl-raised, so there was no need for the asleep() and await() pair. M_ASLEEP was unused. Reviewed by: jasone, peter
* Use 'p' instead of the potentially more expensive 'curproc' inside ofjhb2001-08-021-5/+5
| | | | mi_switch().
* Apply the cluebat to myself and undo the await() -> mawait() rename. Thejhb2001-07-311-31/+15
| | | | | | | | | | | | | | | | | | | | asleep() and await() functions split the functionality of msleep() up into two halves. Only the asleep() half (which is what puts the process on the sleep queue) actually needs the lock usually passed to msleep() held to prevent lost wakeups. await() does not need the lock held, so the lock can be released prior to calling await() and does not need to be passed in to the await() function. Typical usage of these functions would be as follows: mtx_lock(&foo_mtx); ... do stuff ... asleep(&foo_cond, PRIxx, "foowt", hz); ... mtx_unlock&foo_mtx); ... await(-1, -1); Inspired by: dillon on the couch at Usenix
* Add a safety belt to mawait() for the (cold || panicstr) case identical tojhb2001-07-311-0/+12
| | | | | | | the one in msleep() such that we return immediately rather than blocking. Submitted by: peter Prodded by: sheldonh
* Backout mwakeup, etc.jake2001-07-061-13/+4
|
* Implement mwakeup, mwakeup_one, cv_signal_drop and cv_broadcast_drop.jake2001-07-041-4/+13
| | | | | | | | | These take an additional mutex argument, which is dropped before any processes are made runnable. This can avoid contention on the mutex if the processes would immediately acquire it, and is done in such a way that wakeups will not be lost. Reviewed by: jhb
* Remove commented-out garbage that skipped updating schedcpu() stats forjhb2001-07-031-2/+0
| | | | ithreads in SWAIT.
* Just check p_oncpu when determining if a process is executing or not.jhb2001-07-031-4/+1
| | | | | | | We already did this in the SMP case, and it is now maintained in the UP case as well, and makes the code slightly more readable. Note that curproc is always executing, thus the p != curproc test does not need to be performed if the p_oncpu check is made.
* Axe spl's that are covered by the sched_lock (and have been for quitejhb2001-07-031-30/+4
| | | | some time.)
* Include the wait message and channel for msleep() in the KTR tracepoint.jhb2001-07-031-1/+2
|
* Remove bogus need_resched() of the current CPU in roundrobin().jhb2001-07-031-3/+6
| | | | | | | We don't actually need to force a context switch of the current process. The act of firing the event triggers a context switch to softclock() and then switching back out again which is equivalent to a preemption, thus no further work is needed on the local CPU.
* Make the schedlock saved critical section state a per-thread property.jhb2001-06-301-0/+3
|
* - Lock CURSIG() with the proc lock to close the signal race with psignal.jhb2001-06-221-99/+67
| | | | | | | | | | | | | | | | - Grab Giant around ktrace points. - Clean up KTR_PROC tracepoints to not display the value of sched_lock.mtx_lock as it isn't really needed anymore and just obfuscates the messages. - Add a few if conditions to replace gotos. - Ensure that every msleep KTR event ends up with a matching msleep resume KTR event (this was broken when we didn't do a mi_switch()). - Only note via ktrace that we resumed from a switch once rather than twice in several places in msleep(). - Remove spl's rom asleep and await as the proc lock and sched_lock provide all the needed locking. - In mawait() add in a needed ktrace point for noting that we are about to switch out.
* Add in assertions to ensure that we always call msleep or mawait withjhb2001-05-231-0/+4
| | | | | | | either a timeout or a held mutex to detect unprotected infinite sleeps that can easily lead to deadlock. Submitted by: alfred
* Remove KASSERT test for sleeping on mv_mtx, instead let WITNESS catchalfred2001-05-221-2/+0
| | | | | | it. Requested by: jhb
* remove my private assertions from tsleep.alfred2001-05-191-7/+2
| | | | add one assertion to ensure we don't sleep while holding vm.
* Introduce a global lock for the vm subsystem (vm_mtx).alfred2001-05-191-0/+7
| | | | | | | | | | | | | | | | | | | vm_mtx does not recurse and is required for most low level vm operations. faults can not be taken without holding Giant. Memory subsystems can now call the base page allocators safely. Almost all atomic ops were removed as they are covered under the vm mutex. Alpha and ia64 now need to catch up to i386's trap handlers. FFS and NFS have been tested, other filesystems will need minor changes (grabbing the vm lock when twiddling page properties). Reviewed (partially) by: jake, jhb
* - Remove unneeded include of sys/ipl.h.jhb2001-05-151-1/+2
| | | | | - Lock the process before calling killproc() to kill it for exceeding the maximum CPU limit.
* Overhaul of the SMP code. Several portions of the SMP kernel support havejhb2001-04-271-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | been made machine independent and various other adjustments have been made to support Alpha SMP. - It splits the per-process portions of hardclock() and statclock() off into hardclock_process() and statclock_process() respectively. hardclock() and statclock() call the *_process() functions for the current process so that UP systems will run as before. For SMP systems, it is simply necessary to ensure that all other processors execute the *_process() functions when the main clock functions are triggered on one CPU by an interrupt. For the alpha 4100, clock interrupts are delievered in a staggered broadcast fashion, so we simply call hardclock/statclock on the boot CPU and call the *_process() functions on the secondaries. For x86, we call statclock and hardclock as usual and then call forward_hardclock/statclock in the MD code to send an IPI to cause the AP's to execute forwared_hardclock/statclock which then call the *_process() functions. - forward_signal() and forward_roundrobin() have been reworked to be MI and to involve less hackery. Now the cpu doing the forward sets any flags, etc. and sends a very simple IPI_AST to the other cpu(s). AST IPIs now just basically return so that they can execute ast() and don't bother with setting the astpending or needresched flags themselves. This also removes the loop in forward_signal() as sched_lock closes the race condition that the loop worked around. - need_resched(), resched_wanted() and clear_resched() have been changed to take a process to act on rather than assuming curproc so that they can be used to implement forward_roundrobin() as described above. - Various other SMP variables have been moved to a MI subr_smp.c and a new header sys/smp.h declares MI SMP variables and API's. The IPI API's from machine/ipl.h have moved to machine/smp.h which is included by sys/smp.h. - The globaldata_register() and globaldata_find() functions as well as the SLIST of globaldata structures has become MI and moved into subr_smp.c. Also, the globaldata list is only available if SMP support is compiled in. Reviewed by: jake, peter Looked over by: eivind
* Convert the allproc and proctree locks from lockmgr locks to sx locks.jhb2001-03-281-6/+7
|
* Rework the witness code to work with sx locks as well as mutexes.jhb2001-03-281-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Introduce lock classes and lock objects. Each lock class specifies a name and set of flags (or properties) shared by all locks of a given type. Currently there are three lock classes: spin mutexes, sleep mutexes, and sx locks. A lock object specifies properties of an additional lock along with a lock name and all of the extra stuff needed to make witness work with a given lock. This abstract lock stuff is defined in sys/lock.h. The lockmgr constants, types, and prototypes have been moved to sys/lockmgr.h. For temporary backwards compatability, sys/lock.h includes sys/lockmgr.h. - Replace proc->p_spinlocks with a per-CPU list, PCPU(spinlocks), of spin locks held. By making this per-cpu, we do not have to jump through magic hoops to deal with sched_lock changing ownership during context switches. - Replace proc->p_heldmtx, formerly a list of held sleep mutexes, with proc->p_sleeplocks, which is a list of held sleep locks including sleep mutexes and sx locks. - Add helper macros for logging lock events via the KTR_LOCK KTR logging level so that the log messages are consistent. - Add some new flags that can be passed to mtx_init(): - MTX_NOWITNESS - specifies that this lock should be ignored by witness. This is used for the mutex that blocks a sx lock for example. - MTX_QUIET - this is not new, but you can pass this to mtx_init() now and no events will be logged for this lock, so that one doesn't have to change all the individual mtx_lock/unlock() operations. - All lock objects maintain an initialized flag. Use this flag to export a mtx_initialized() macro that can be safely called from drivers. Also, we on longer walk the all_mtx list if MUTEX_DEBUG is defined as witness performs the corresponding checks using the initialized flag. - The lock order reversal messages have been improved to output slightly more accurate file and line numbers.
* - Proc locking.jhb2001-03-071-25/+7
| | | | - Remove unneeded spl()'s.
* Add a mtx_assert() in maybe_resched() just to be sure it's always calledjhb2001-02-221-0/+1
| | | | with sched_lock held.
* - Use the new NOCPU constant.jhb2001-02-221-2/+2
| | | | | | - Fix a warning. Noticed by: bde (2)
* - Don't call clear_resched() in userret(), instead, clear the resched flagjhb2001-02-201-0/+8
| | | | | | | | | | | | in mi_switch() just before calling cpu_switch() so that the first switch after a resched request will satisfy the request. - While I'm at it, move a few things into mi_switch() and out of cpu_switch(), specifically set the p_oncpu and p_lastcpu members of proc in mi_switch(), and handle the sched_lock state change across a context switch in mi_switch(). - Since cpu_switch() no longer handles the sched_lock state change, we have to setup an initial state for sched_lock in fork_exit() before we release it.
* Implement a unified run queue and adjust priority levels accordingly.jake2001-02-121-58/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - All processes go into the same array of queues, with different scheduling classes using different portions of the array. This allows user processes to have their priorities propogated up into interrupt thread range if need be. - I chose 64 run queues as an arbitrary number that is greater than 32. We used to have 4 separate arrays of 32 queues each, so this may not be optimal. The new run queue code was written with this in mind; changing the number of run queues only requires changing constants in runq.h and adjusting the priority levels. - The new run queue code takes the run queue as a parameter. This is intended to be used to create per-cpu run queues. Implement wrappers for compatibility with the old interface which pass in the global run queue structure. - Group the priority level, user priority, native priority (before propogation) and the scheduling class into a struct priority. - Change any hard coded priority levels that I found to use symbolic constants (TTIPRI and TTOPRI). - Remove the curpriority global variable and use that of curproc. This was used to detect when a process' priority had lowered and it should yield. We now effectively yield on every interrupt. - Activate propogate_priority(). It should now have the desired effect without needing to also propogate the scheduling class. - Temporarily comment out the call to vm_page_zero_idle() in the idle loop. It interfered with propogate_priority() because the idle process needed to do a non-blocking acquire of Giant and then other processes would try to propogate their priority onto it. The idle process should not do anything except idle. vm_page_zero_idle() will return in the form of an idle priority kernel thread which is woken up at apprioriate times by the vm system. - Update struct kinfo_proc to the new priority interface. Deliberately change its size by adjusting the spare fields. It remained the same size, but the layout has changed, so userland processes that use it would parse the data incorrectly. The size constraint should really be changed to an arbitrary version number. Also add a debug.sizeof sysctl node for struct kinfo_proc.
* Acquire sched_lock around need_resched() in roundrobin() to satisfyjake2001-02-101-8/+4
| | | | | assertions that it is held. Since roundrobin() is a timeout there's no possible way that it could be called with sched_lock held.
* Change and clean the mutex lock interface.bmilekic2001-02-091-42/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mtx_enter(lock, type) becomes: mtx_lock(lock) for sleep locks (MTX_DEF-initialized locks) mtx_lock_spin(lock) for spin locks (MTX_SPIN-initialized) similarily, for releasing a lock, we now have: mtx_unlock(lock) for MTX_DEF and mtx_unlock_spin(lock) for MTX_SPIN. We change the caller interface for the two different types of locks because the semantics are entirely different for each case, and this makes it explicitly clear and, at the same time, it rids us of the extra `type' argument. The enter->lock and exit->unlock change has been made with the idea that we're "locking data" and not "entering locked code" in mind. Further, remove all additional "flags" previously passed to the lock acquire/release routines with the exception of two: MTX_QUIET and MTX_NOSWITCH The functionality of these flags is preserved and they can be passed to the lock/unlock routines by calling the corresponding wrappers: mtx_{lock, unlock}_flags(lock, flag(s)) and mtx_{lock, unlock}_spin_flags(lock, flag(s)) for MTX_DEF and MTX_SPIN locks, respectively. Re-inline some lock acq/rel code; in the sleep lock case, we only inline the _obtain_lock()s in order to ensure that the inlined code fits into a cache line. In the spin lock case, we inline recursion and actually only perform a function call if we need to spin. This change has been made with the idea that we generally tend to avoid spin locks and that also the spin locks that we do have and are heavily used (i.e. sched_lock) do recurse, and therefore in an effort to reduce function call overhead for some architectures (such as alpha), we inline recursion for this case. Create a new malloc type for the witness code and retire from using the M_DEV type. The new type is called M_WITNESS and is only declared if WITNESS is enabled. Begin cleaning up some machdep/mutex.h code - specifically updated the "optimized" inlined code in alpha/mutex.h and wrote MTX_LOCK_SPIN and MTX_UNLOCK_SPIN asm macros for the i386/mutex.h as we presently need those. Finally, caught up to the interface changes in all sys code. Contributors: jake, jhb, jasone (in no particular order)
* Zap last remaining references to (and a use use of) of simple_locks.peter2001-01-311-4/+0
|
* - Catch up to proc flag changes.jhb2001-01-241-25/+29
| | | | | | - Add in some locking ops that might fix SIGXCPU, but don't enable them yet. - Assert that sched_lock is not recursed when mi_switch() is called.
* Do not do the commenting out the way that saves bytes and looks cleanermjacob2001-01-231-1/+4
| | | | to you. Do it the way Vox Populi wants it.
* Move (now) unused variable declaration inside the block (now commented out).mjacob2001-01-221-2/+1
|
* Temporarily disable the printf() for micruptime() going backwards, thejhb2001-01-201-0/+5
| | | | | | | SIGXCPU signal, and killing of processes that exceed their allowed run time until they can play nice with sched_lock. Right now they are just potentital panics waiting to happen. The printf() has bitten several people.
OpenPOWER on IntegriCloud