summaryrefslogtreecommitdiffstats
path: root/sys/kern/kern_intr.c
Commit message (Collapse)AuthorAgeFilesLines
* fix typo in code normally not compiled in.julian2007-10-291-1/+1
|
* Fix typo in code obviously not being compiled on any of my machines.julian2007-10-281-1/+1
| | | | found by: rdivacky@
* rename the process to 'idle' and 'intr' as per jhb.julian2007-10-271-2/+2
|
* if one changes a function's arguments, one must also change the callers.julian2007-10-261-2/+2
|
* Introduce a way to make pure kernal threads.julian2007-10-261-20/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | kthread_add() takes the same parameters as the old kthread_create() plus a pointer to a process structure, and adds a kernel thread to that process. kproc_kthread_add() takes the parameters for kthread_add, plus a process name and a pointer to a pointer to a process instead of just a pointer, and if the proc * is NULL, it creates the process to the specifications required, before adding the thread to it. All other old kthread_xxx() calls return, but act on (struct thread *) instead of (struct proc *). One reason to change the name is so that any old kernel modules that are lying around and expect kthread_create() to make a process will not just accidentally link. fix top to show kernel threads by their thread name in -SH mode add a tdnam formatting option to ps to show thread names. make all idle threads actual kthreads and put them into their own idled process. make all interrupt threads kthreads and put them in an interd process (mainly for aesthetic and accounting reasons) rename proc 0 to be 'kernel' and it's swapper thread is now 'swapper' man page fixes to follow.
* Rename the kthread_xxx (e.g. kthread_create()) callsjulian2007-10-201-6/+6
| | | | | | | | | | | to kproc_xxx as they actually make whole processes. Thos makes way for us to add REAL kthread_create() and friends that actually make theads. it turns out that most of these calls actually end up being moved back to the thread version when it's added. but we need to make this cosmetic change first. I'd LOVE to do this rename in 7.0 so that we can eventually MFC the new kthread_xxx() calls.
* Commit 14/14 of sched_lock decomposition.jeff2007-06-051-24/+24
| | | | | | | | | | | - Use thread_lock() rather than sched_lock for per-thread scheduling sychronization. - Use the per-process spinlock rather than the sched_lock for per-process scheduling synchronization. Tested by: kris, current@ Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc. Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each)
* Rework the PCPU_* (MD) interface:attilio2007-06-041-1/+1
| | | | | | | | | | | | - Rename PCPU_LAZY_INC into PCPU_INC - Add the PCPU_ADD interface which just does an add on the pcpu member given a specific value. Note that for most architectures PCPU_INC and PCPU_ADD are not safe. This is a point that needs some discussions/work in the next days. Reviewed by: alc, bde Approved by: jeff (mentor)
* In some particular cases (like in pccard and pccbb), the real devicepiso2007-05-311-3/+0
| | | | | | | | | | | handler is wrapped in a couple of functions - a filter wrapper and an ithread wrapper. In this case (and just in this case), the filter wrapper could ask the system to schedule the ithread and mask the interrupt source if the wrapped handler is composed of just an ithread handler: modify the "old" interrupt code to make it support this situation, while the "new" interrupt code is already ok. Discussed with: jhb
* Bring in the reminaing bits to make interrupt filtering work:piso2007-05-061-6/+570
| | | | | | | | | | | | | | | o push much of the i386 and amd64 MD interrupt handling code (intr_machdep.c::intr_execute_handlers()) into MI code (kern_intr.c::ithread_loop()) o move filter handling to kern_intr.c::intr_filter_loop() o factor out the code necessary to mask and ack an interrupt event (intr_machdep.c::intr_eoi_src() and intr_machdep.c::intr_disab_eoi_src()), and make them part of 'struct intr_event', passing them as arguments to kern_intr.c::intr_event_create(). o spawn a private ithread per handler (struct intr_handler::ih_thread) with filter and ithread functions. Approved by: re (implicit?)
* Bump the interrupt storm detection counter to 1000. My slow fileservernjl2007-04-191-4/+4
| | | | | | | | | | | gets a bogus irq storm detected when periodic daily kicks off at 3 am and disconnects the disk. Change the print logic to print once per second when the storm is occurring instead of only once. Otherwise, it appeared that something else was causing the errors each night at 3 am since the print only occurred the first time. Reviewed by: jhb MFC after: 1 week
* - Don't do the interrupt storm protection stuff for software interruptjhb2007-03-021-2/+3
| | | | | | | handlers. - Use pause() when throtting during an interrupt storm. Reported by: kris (1)
* Do not execute filter only handlers in ithread_execute_handlers():piso2007-02-271-0/+4
| | | | | this fixes the panics when filter only and ithread only handlers where sharing the same irq .
* o break newbus api: add a new argument of type driver_filter_t topiso2007-02-231-17/+8
| | | | | | | | | | | | | bus_setup_intr() o add an int return code to all fast handlers o retire INTR_FAST/IH_FAST For more info: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=465712+0+current/freebsd-current Reviewed by: many Approved by: re@
* - Remove setrunqueue and replace it with direct calls to sched_add().jeff2007-01-231-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | setrunqueue() was mostly empty. The few asserts and thread state setting were moved to the individual schedulers. sched_add() was chosen to displace it for naming consistency reasons. - Remove adjustrunqueue, it was 4 lines of code that was ifdef'd to be different on all three schedulers where it was only called in one place each. - Remove the long ifdef'd out remrunqueue code. - Remove the now redundant ts_state. Inspect the thread state directly. - Don't set TSF_* flags from kern_switch.c, we were only doing this to support a feature in one scheduler. - Change sched_choose() to return a thread rather than a td_sched. Also, rely on the schedulers to return the idlethread. This simplifies the logic in choosethread(). Aside from the run queue links kern_switch.c mostly does not care about the contents of td_sched. Discussed with: julian - Move the idle thread loop into the per scheduler area. ULE wants to do something different from the other schedulers. Suggested by: jhb Tested on: x86/amd64 sched_{4BSD, ULE, CORE}.
* Add a function to return the MD interrupt source cookie associated withjhb2006-12-121-0/+20
| | | | | an interrupt event. Use this in the x86 code to fixup the intrcnt names when an interrupt handler is removed.
* Add a comment and fix a whitespace nit.jhb2006-12-121-1/+2
|
* Threading cleanup.. part 2 of several.julian2006-12-061-5/+1
| | | | | | | | | | | | | | | | | | | | | | Make part of John Birrell's KSE patch permanent.. Specifically, remove: Any reference of the ksegrp structure. This feature was never fully utilised and made things overly complicated. All code in the scheduler that tried to make threaded programs fair to unthreaded programs. Libpthread processes will already do this to some extent and libthr processes already disable it. Also: Since this makes such a big change to the scheduler(s), take the opportunity to rename some structures and elements that had to be moved anyhow. This makes the code a lot more readable. The ULE scheduler compiles again but I have no idea if it works. The 4bsd scheduler still reqires a little cleaning and some functions that now do ALMOST nothing will go away, but I thought I'd do that as a separate commit. Tested by David Xu, and Dan Eischen using libthr and libpthread.
* Make KSE a kernel option, turned on by default in all GENERICjb2006-10-261-0/+4
| | | | | | | kernel configs except sun4v (which doesn't process signals properly with KSE). Reviewed by: davidxu@
* kern_intr.c:bde2006-10-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | - Count (scheduling of) software interrupts (SWIs) as SWIs, not as hardware interrupts. - Don't count (scheduling of) delayed SWIs as interrupts at all, since in the delayed case it is expected that there are many more scheduling calls than handling calls. Perhaps all interrupts should be counted only when they are handled, but it is only counts of delayed SWIs that shouldn never be combined with the other counts. subr_trap.c: - Count (handling of) Asynchronous System Traps (ASTs) as traps, not as software interrupts. Before these changes, the counter for SWIs only counted ASTs, and SWIs weren't counted separately, but a subcounter for ASTs alone is less needed than for most other exception sources. 4.4BSD-Lite uses the counters for similar things (actually matching their names) on its main arches (hp300, ..., !i386) where more of the exceptions are in hardware.
* Simplify the pager support in DDB. Allowing different db commands tojhb2006-07-121-6/+4
| | | | | | | | | | | | | | | | install custom pager functions didn't actually happen in practice (they all just used the simple pager and passed in a local quit pointer). So, just hardcode the simple pager as the only pager and make it set a global db_pager_quit flag that db commands can check when the user hits 'q' (or a suitable variant) at the pager prompt. Also, now that it's easy to do so, enable paging by default for all ddb commands. Any command that wishes to honor the quit flag can do so by checking db_pager_quit. Note that the pager can also be effectively disabled by setting $lines to 0. Other fixes: - 'show idt' on i386 and pc98 now actually checks the quit flag and terminates early. - 'show intr' now actually checks the quit flag and terminates early.
* Change msleep() and tsleep() to not alter the calling thread's priorityjhb2006-04-171-5/+3
| | | | | | | | | | | | if the specified priority is zero. This avoids a race where the calling thread could read a snapshot of it's current priority, then a different thread could change the first thread's priority, then the original thread would call sched_prio() inside msleep() undoing the change made by the second thread. I used a priority of zero as no thread that calls msleep() or tsleep() should be specifying a priority of zero anyway. The various places that passed 'curthread->td_priority' or some variant as the priority now pass 0.
* Take a better stab at making this compile.scottl2006-04-151-1/+1
|
* Take a stab at making this compile.scottl2006-04-151-1/+1
|
* Turn on ithread_destroy() and call it from intr_event_destroy() to tearjhb2006-04-131-5/+7
| | | | down an interrupt event's associated thread (if it has one).
* Add a swi_remove() function to teardown software interrupt handlers. Forjhb2005-10-261-0/+17
| | | | | now it just calls intr_event_remove_handler(), but at some point it might also be responsible for tearing down interrupt events created via swi_add.
* Reorganize the interrupt handling code a bit to make a few things cleanerjhb2005-10-251-268/+435
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and increase flexibility to allow various different approaches to be tried in the future. - Split struct ithd up into two pieces. struct intr_event holds the list of interrupt handlers associated with interrupt sources. struct intr_thread contains the data relative to an interrupt thread. Currently we still provide a 1:1 relationship of events to threads with the exception that events only have an associated thread if there is at least one threaded interrupt handler attached to the event. This means that on x86 we no longer have 4 bazillion interrupt threads with no handlers. It also means that interrupt events with only INTR_FAST handlers no longer have an associated thread either. - Renamed struct intrhand to struct intr_handler to follow the struct intr_foo naming convention. This did require renaming the powerpc MD struct intr_handler to struct ppc_intr_handler. - INTR_FAST no longer implies INTR_EXCL on all architectures except for powerpc. This means that multiple INTR_FAST handlers can attach to the same interrupt and that INTR_FAST and non-INTR_FAST handlers can attach to the same interrupt. Sharing INTR_FAST handlers may not always be desirable, but having sio(4) and uhci(4) fight over an IRQ isn't fun either. Drivers can always still use INTR_EXCL to ask for an interrupt exclusively. The way this sharing works is that when an interrupt comes in, all the INTR_FAST handlers are executed first, and if any threaded handlers exist, the interrupt thread is scheduled afterwards. This type of layout also makes it possible to investigate using interrupt filters ala OS X where the filter determines whether or not its companion threaded handler should run. - Aside from the INTR_FAST changes above, the impact on MD interrupt code is mostly just 's/ithread/intr_event/'. - A new MI ddb command 'show intrs' walks the list of interrupt events dumping their state. It also has a '/v' verbose switch which dumps info about all of the handlers attached to each event. - We currently don't destroy an interrupt thread when the last threaded handler is removed because it would suck for things like ppbus(8)'s braindead behavior. The code is present, though, it is just under #if 0 for now. - Move the code to actually execute the threaded handlers for an interrrupt event into a separate function so that ithread_loop() becomes more readable. Previously this code was all in the middle of ithread_loop() and indented halfway across the screen. - Made struct intr_thread private to kern_intr.c and replaced td_ithd with a thread private flag TDP_ITHREAD. - In statclock, check curthread against idlethread directly rather than curthread's proc against idlethread's proc. (Not really related to intr changes) Tested on: alpha, amd64, i386, sparc64 Tested on: arm, ia64 (older version of patch by cognet and marcel)
* Don't disallow sleeping for handlers on swi's since some swi handlersjhb2005-09-151-2/+4
| | | | | | (like CAM) do sleep in their handlers. Requested by: scottl
* - Add a new simple facility for marking the current thread as being in ajhb2005-09-151-0/+2
| | | | | | | | | | | | | state where sleeping on a sleep queue is not allowed. The facility doesn't support recursion but uses a simple private per-thread flag (TDP_NOSLEEPING). The sleepq_add() function will panic if the flag is set and INVARIANTS is enabled. - Use this new facility to replace the g_xup and g_xdown mutexes that were (ab)used to achieve similar behavior. - Disallow sleeping in interrupt threads when invoking interrupt handlers. MFC after: 1 week Reviewed by: phk
* Simplify the storming logic and remove a variable as a result.jhb2005-06-201-8/+3
| | | | Approved by: re (dwhite)
* Use PCPU_LAZY_INC() for cnt.v_{intr,trap,syscalls} rather than atomicjhb2005-04-121-1/+1
| | | | operations in some places and simple non-per CPU math in others.
* /* -> /*- for copyright notices, minor format tweaks as necessaryimp2005-01-061-1/+1
|
* Stop explicitly touching td_base_pri outside of the scheduler and simplyjhb2004-12-301-4/+3
| | | | | set a thread's priority via sched_prio() when that is the desired action. The schedulers will start managing td_base_pri internally shortly.
* Don't bother exiting storming mode once a second to see if it has gonejhb2004-11-171-12/+6
| | | | | | | away, instead only exit storming mode when an interrupt stops firing long enough for the ithread to exit the loop and go back to sleep. Tested by: macrus (cruder version)
* Adjust the interrupt storm handling code to better handle a storm. Whenjhb2004-11-161-5/+23
| | | | | | | | | | | a storm is detected, enter "storming" mode which throttles the interrupt source such that the handlers are run once every clock tick. Previously we allowed a full set of storm_threshold interations through the handler before going back to sleep. Also, this currently will intentionally exit storming mode once a second to see if the storm has passed. Tested by: marcus Discussed with: bde
* - Make setting of IT_ENTROPY a bit simpler in ithread_update().jhb2004-11-051-10/+10
| | | | | | | | | - Tweak the updating of the ithread name in ithread_update() so that the '+' and '*' characters for device names that were too short only get added at the end after as many device names as possible were fit into the allocated space. Prior to this, some long devices would result in '+' chars showing up between two different devices rather than at the end.
* Revert most of 1.109. Although it improved the situation on one particularjhb2004-11-031-42/+12
| | | | | | | | | | | | | motherboard, in practice the changes resulted in many false positives for heavy network loads, etc. resulting in poor performance. Also, the motherboard referenced in the 1.109 log has other problems and simply does not seem to work with the APIC enabled even with the changes in 1.109. The correct fix for that board seems to be to not use the APIC at all. One thing kept from 1.109 is that throttled interrupts are now effectively polled on every clock tick rather than just 10 times per second. MFC after: 1 month Tested by: Shunsuke SHINOMIYA shino at fornext dot org
* - Change the ddb paging "support" to use a variable (db_lines_per_page) tojhb2004-11-011-1/+1
| | | | | | | | | | | | | | | | | control the number of lines per page rather than a constant. The variable can be examined and changed in ddb as '$lines'. Setting the variable to 0 will effectively turn off paging. - Change db_putchar() to force out pending whitespace before outputting newlines and carriage returns so that one can rub out content on the current line via '\r \r' type strings. - Change the simple pager to rub out the --More-- prompt explicitly when the routine exits. - Add some aliases to the simple pager to make it more compatible with more(1): 'e' and 'j' do a single line. 'd' does half a page, and 'f' does a full page. MFC after: 1 month Inspired by: kris
* Refactor a bunch of scheduler code to give basically the same behaviourjulian2004-09-051-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | but with slightly cleaned up interfaces. The KSE structure has become the same as the "per thread scheduler private data" structure. In order to not make the diffs too great one is #defined as the other at this time. The KSE (or td_sched) structure is now allocated per thread and has no allocation code of its own. Concurrency for a KSEGRP is now kept track of via a simple pair of counters rather than using KSE structures as tokens. Since the KSE structure is different in each scheduler, kern_switch.c is now included at the end of each scheduler. Nothing outside the scheduler knows the contents of the KSE (aka td_sched) structure. The fields in the ksegrp structure that are to do with the scheduler's queueing mechanisms are now moved to the kg_sched structure. (per ksegrp scheduler private data structure). In other words how the scheduler queues and keeps track of threads is no-one's business except the scheduler's. This should allow people to write experimental schedulers with completely different internal structuring. A scheduler call sched_set_concurrency(kg, N) has been added that notifies teh scheduler that no more than N threads from that ksegrp should be allowed to be on concurrently scheduled. This is also used to enforce 'fainess' at this time so that a ksegrp with 10000 threads can not swamp a the run queue and force out a process with 1 thread, since the current code will not set the concurrency above NCPU, and both schedulers will not allow more than that many onto the system run queue at a time. Each scheduler should eventualy develop their own methods to do this now that they are effectively separated. Rejig libthr's kernel interface to follow the same code paths as linkse for scope system threads. This has slightly hurt libthr's performance but I will work to recover as much of it as I can. Thread exit code has been cleaned up greatly. exit and exec code now transitions a process back to 'standard non-threaded mode' before taking the next step. Reviewed by: scottl, peter MFC after: 1 week
* Give setrunqueue() and sched_add() more of a clue as tojulian2004-09-011-2/+2
| | | | | | where they are coming from and what is expected from them. MFC after: 2 days
* Annotate call to DELAY() in interrupt storm mitigation as beingrwatson2004-08-171-0/+3
| | | | | | something to revisit. Approved by: re (scottl)
* In ithread_schedule(), when we plan to go harvest some entropy asrwatson2004-08-061-2/+4
| | | | | | a result of scheduling an ithread, cut a KTR_INTR trace record so that it's clear in tracing interrupt activity where and when the entropy harvesting code is invoked.
* Implement preemption of kernel threads natively in the scheduler ratherjhb2004-07-021-16/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | than as one-off hacks in various other parts of the kernel: - Add a function maybe_preempt() that is called from sched_add() to determine if a thread about to be added to a run queue should be preempted to directly. If it is not safe to preempt or if the new thread does not have a high enough priority, then the function returns false and sched_add() adds the thread to the run queue. If the thread should be preempted to but the current thread is in a nested critical section, then the flag TDF_OWEPREEMPT is set and the thread is added to the run queue. Otherwise, mi_switch() is called immediately and the thread is never added to the run queue since it is switch to directly. When exiting an outermost critical section, if TDF_OWEPREEMPT is set, then clear it and call mi_switch() to perform the deferred preemption. - Remove explicit preemption from ithread_schedule() as calling setrunqueue() now does all the correct work. This also removes the do_switch argument from ithread_schedule(). - Do not use the manual preemption code in mtx_unlock if the architecture supports native preemption. - Don't call mi_switch() in a loop during shutdown to give ithreads a chance to run if the architecture supports native preemption since the ithreads will just preempt DELAY(). - Don't call mi_switch() from the page zeroing idle thread for architectures that support native preemption as it is unnecessary. - Native preemption is enabled on the same archs that supported ithread preemption, namely alpha, i386, and amd64. This change should largely be a NOP for the default case as committed except that we will do fewer context switches in a few cases and will avoid the run queues completely when preempting. Approved by: scottl (with his re@ hat)
* - Change mi_switch() and sched_switch() to accept an optional thread tojhb2004-07-021-2/+2
| | | | | | | | | | | | | switch to. If a non-NULL thread pointer is passed in, then the CPU will switch to that thread directly rather than calling choosethread() to pick a thread to choose to. - Make sched_switch() aware of idle threads and know to do TD_SET_CAN_RUN() instead of sticking them on the run queue rather than requiring all callers of mi_switch() to know to do this if they can be called from an idlethread. - Move constants for arguments to mi_switch() and thread_single() out of the middle of the function prototypes and up above into their own section.
* Detect interrupt storms better. The storm detection didn't work at allbde2004-06-051-23/+50
| | | | | | | | | | | | | | | | | with an ASUS A7N8X-E motherboard in APIC mode, since storming interrupts don't repeat immediately. Use DELAY(1) to wait a bit for them to repeat. This affects all systems. Only delay for the first (10 * intr_storm_threshold) interrupts (per interrupt handler) so that this is only a pessimization while warming up. Throttle after calling the sub-handlers instead of before so that the long delay given by throttling can be used instead of the DELAY(1) to detect storms after warming up. Reduced the throttling period from 1/10 second to 1/hz seconds so that throttling doesn't destroy performance so much. Interrupts that are detected as storming are effectively handled by polling at a frequency of hz Hz. On A7N8X-E's there is another hardware or configuration bug that makes the throttled frequency closer to 2*hz Hz.
* Fixed some style bugs in previous commit (mainly an insertion sort errorbde2004-04-171-9/+10
| | | | | | for declarations, and poorly worded messages). Fixed some nearby style bugs (unsorted declarations).
* - Enable (unmask) interrupt sources earlier in the ithread loop.jhb2004-04-161-8/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Specifically, we used to enable the source after locking sched_lock and just before we had already decided to do a context switch. This meant that an ithread could never process more than one interrupt per context switch. Enabling earlier in the loop before sched_lock is acquired allows an ithread to handle multiple interrupts per context switch if interrupts fire very rapidly. For the case of heavy interrupt load this can reduce the number of context switches (and thus overhead) as well as reduce interrupt latency. - Now that we can handle multiple interrupts per context switch, add simple interrupt storm protection to threaded interrupts. If X number of consecutive interrupts are triggered before the itherad voluntarily yields to another thread, then the interrupt thread will sleep with the associated interrupt source disabled (masked) for 1/10th of a second. The default value of X is 500, but it can be tweaked via the tunable/ sysctl hw.intr_storm_threshold. If an interrupt storm is detected, then a message is output to the kernel console on the first occurrence per interrupt thread. Interrupt storm protection can be disabled completely by setting this value to 0. There is no scientific reasoning for the 1/10th of a second or 500 interrupts values, so they may require tweaking at some point in the future. Tested by: rwatson (an earlier version w/o the storm protection) Tested by: mux (reportedly made a machine with two PCI interrupts storming usable rather than hard locked) Reviewed by: imp
* kthread_exit() no longer requires Giant, so don't force callers to acquirejhb2004-03-051-1/+0
| | | | | | Giant just to call kthread_exit(). Requested by: many
* - Add a flags parameter to mi_switch. The value of flags may be SW_VOL orjeff2004-01-251-4/+2
| | | | | | | | | | SW_INVOL. Assert that one of these is set in mi_switch() and propery adjust the rusage statistics. This is to simplify the large number of users of this interface which were previously all required to adjust the proper counter prior to calling mi_switch(). This also facilitates more switch and locking optimizations. - Change all callers of mi_switch() to pass the appropriate paramter and remove direct references to the process statistics.
* If a device attach routine fails during boot and calls bus_teardown_intr(),truckman2004-01-131-1/+5
| | | | | | | | | | | | | | ithread_remove_handler() may fail to remove the interrupt handler if it decides to let the ithread do the removal. The problem is that during boot "cold" is set, which causes msleep() to return immediately. This will cause ithread_remove_handler() to fail to wait for the ithread to do the removal from the handler TAILQ before freeing the handler back to the heap. Bad things will happen when some other user of the TAILQ, such as ithread_add_handler() or the actual ithread attempts to use the freed handler. Fix the problem by forcing ithread_remove_handler() to do the actual removal itself if the "cold" flag is set. Reviewed by: jhb
OpenPOWER on IntegriCloud