summaryrefslogtreecommitdiffstats
path: root/sys/kern/kern_intr.c
Commit message (Collapse)AuthorAgeFilesLines
* Do not add handler to event handlers list until ithread is created.kan2012-08-061-25/+27
| | | | | | | | | | | | In rare event when fast and ithread interrupts share the same vector and the fast handler was registered first, we can end up trying to schedule the ithread that is not created yet. The kernel built with INVARIANTS then triggers an assertion. Change the order to create the ithread first and only then add the handler that needs it to the interrupt event handlers list. Reviewed by: jhb
* Export intrcnt correctly when running under 32-bit compatibility.jmallett2012-03-091-0/+18
| | | | Reviewed by: gonzo, nwhitehorn
* Add a new sched_clear_name() method to the scheduler interface to clearjhb2012-03-081-0/+3
| | | | | | | | the cached name used for KTR_SCHED traces when a thread's name changes. This way KTR_SCHED traces (and thus schedgraph) will notice when a thread's name changes, most commonly via execve(). MFC after: 2 weeks
* Be pedantic and change // comment to C-style one.pluknet2012-01-161-1/+1
| | | | Noticed by: Bruce Evans
* Use strchr() and strrchr().ed2012-01-021-4/+4
| | | | | | | | It seems strchr() and strrchr() are used more often than index() and rindex(). Therefore, simply migrate all kernel code to use it. For the XFS code, remove an empty line to make the code identical to the code in the Linux kernel.
* - Remove the eintrcnt/eintrnames usage and introduce the concept ofattilio2011-07-181-5/+6
| | | | | | | | | | | | | | | | sintrcnt/sintrnames which are symbols containing the size of the 2 tables. - For amd64/i386 remove the storage of intr* stuff from assembly files. This area can be widely improved by applying the same to other architectures and likely finding an unified approach among them and move the whole code to be MI. More work in this area is expected to happen fairly soon. No MFC is previewed for this patch. Tested by: pluknet Reviewed by: jhb Approved by: re (kib)
* - Catch up to falloc() changes.jeff2011-04-261-4/+11
| | | | | - PHOLD() before using a task structure on the stack. - Fix a LOR between the sleepq lock and thread lock in _intr_drain().
* - Merge changes to the base system to support OFED. These includejeff2011-03-211-2/+56
| | | | | a wider arg2 for sysctl, updates to vlan code, IFT_INFINIBAND, and other miscellaneous small features.
* - Retire some unused ithread priorities: PI_TTYHIGH, PI_TAPE, andjhb2011-01-111-14/+5
| | | | | | PI_DISKLOW. While here, rename PI_TTYLOW to PI_TTY. - Add a macro PI_SWI() that takes a SWI_* constant as an argument and returns the suitable thread priority.
* Store interrupt trap frame into struct thread. It allows interrupt handlermav2010-06-101-0/+8
| | | | | | | to obtain both trap frame and opaque argument submitted on registrction. After kernel and all drivers get used to it, legacy hack can be removed. Reviewed by: jhb@
* KASSERT that return value of interrupt filter complies with contractavg2010-01-271-1/+11
| | | | | | | | For example a return value of zero could lead to a stuck level-triggered interrupt line. Reviewed by: jhb (for INTR_FILTER case) MFC after: 3 weeks
* Split P_NOLOAD into a per-thread flag (TDF_NOLOAD).attilio2009-11-031-5/+5
| | | | | | | | | | This improvements aims for avoiding further cache-misses in scheduler specific functions which need to keep track of average thread running time and further locking in places setting for this flag. Reported by: jeff (originally), kris (currently) Reviewed by: jhb Tested by: Giuseppe Cocomazzi <sbudella at email dot it>
* Use language more closely resembling English in a panic message.jhb2009-10-151-1/+1
| | | | | Pointy hat to: jhb Submitted by: pluknet
* Add a facility for associating optional descriptions with active interruptjhb2009-10-151-2/+57
| | | | | | | | | | | | | | | | | | | | | | | | | handlers. This is primarily intended as a way to allow devices that use multiple interrupts (e.g. MSI) to meaningfully distinguish the various interrupt handlers. - Add a new BUS_DESCRIBE_INTR() method to the bus interface to associate a description with an active interrupt handler setup by BUS_SETUP_INTR. It has a default method (bus_generic_describe_intr()) which simply passes the request up to the parent device. - Add a bus_describe_intr() wrapper around BUS_DESCRIBE_INTR() that supports printf(9) style formatting using var args. - Reserve MAXCOMLEN bytes in the intr_handler structure to hold the name of an interrupt handler and copy the name passed to intr_event_add_handler() into that buffer instead of just saving the pointer to the name. - Add a new intr_event_describe_handler() which appends a description string to an interrupt handler's name. - Implement support for interrupt descriptions on amd64 and i386 by having the nexus(4) driver supply a custom bus_describe_intr method that invokes a new intr_describe() MD routine which in turn looks up the associated interrupt event and invokes intr_event_describe_handler(). Requested by: many Reviewed by: scottl MFC after: 2 weeks
* Improve the handling of cpuset with interrupts.jhb2009-07-011-3/+23
| | | | | | | | | | | | | | | | | | | | | - For x86, change the interrupt source method to assign an interrupt source to a specific CPU to return an error value instead of void, thus allowing it to fail. - If moving an interrupt to a CPU fails due to a lack of IDT vectors in the destination CPU, fail the request with ENOSPC rather than panicing. - For MSI interrupts on x86 (but not MSI-X), only allow cpuset to be used on the first interrupt in a group. Moving the first interrupt in a group moves the entire group. - Use the icu_lock to protect intr_next_cpu() on x86 instead of the intr_table_lock to fix a LOR introduced in the last set of MSI changes. - Add a new privilege PRIV_SCHED_CPUSET_INTR for using cpuset with interrupts. Previously, binding an interrupt to a CPU only performed a privilege check if the interrupt had an interrupt thread. Interrupts without a thread could be bound by non-root users as a result. - If an interrupt event's assign_cpu method fails, then restore the original cpuset mask for the associated interrupt thread. Approved by: re (kib)
* Return errors from intr_event_bind() to the caller of intr_set_affinity().jhb2009-06-251-2/+1
| | | | | | | | Specifically, if a non-root user attempts to bind an interrupt the request will now report failure with EPERM rather than silently failing with a successful return code. MFC after: 1 week
* Binding interrupts to a CPU consists of two parts: setting up CPUrwatson2009-05-181-1/+13
| | | | | | | | | | | affinity for the interrupt thread, and requesting that underlying hardware direct interrupts to the CPU. For software interrupt threads, implement a no-op interrupt event binder that returns success, so that the interrupt management code will just set the ithread's affinity and succeed. Reviewed by: jhb MFC after: 1 week
* style(9)obrien2008-09-231-1/+1
|
* Expose a new public routine intr_event_execute_handlers() which executesjhb2008-09-151-6/+16
| | | | | | all the non-filter handlers attached to an interrupt event. This can be used by device drivers which multiplex their interrupt onto the interrupt handlers for child devices.
* Submit a band-aid for interrupt set up race.kmacy2008-08-221-0/+6
| | | | MFC after: 1 month
* revert change from local treekmacy2008-07-181-6/+0
|
* import vendor fixes to cxgbkmacy2008-07-181-0/+6
|
* Remove an unneeded error variable to make clear that if reachingbz2008-06-291-3/+1
| | | | the end of the function we never return an error.
* - Make SCHED_STATS more generic by adding a wrapper to create thejeff2008-04-171-2/+2
| | | | | | | | | | | | | | | | | | variables and sysctl nodes. - In reset walk the children of kern_sched_stats and reset the counters via the oid_arg1 pointer. This allows us to add arbitrary counters to the tree and still reset them properly. - Define a set of switch types to be passed with flags to mi_switch(). These types are named SWT_*. These types correspond to SCHED_STATS counters and are automatically handled in this way. - Make the new SWT_ types more specific than the older switch stats. There are now stats for idle switches, remote idle wakeups, remote preemption ithreads idling, etc. - Add switch statistics for ULE's pickcpu algorithm. These stats include how much migration there is, how often affinity was successful, how often threads were migrated to the local cpu on wakeup, etc. Sponsored by: Nokia
* - Add the interrupt vector number to intr_event_create so MI code canjeff2008-04-111-59/+97
| | | | | | | | | | | | lookup hard interrupt events by number. Ignore the irq# for soft intrs. - Add support to cpuset for binding hardware interrupts. This has the side effect of binding any ithread associated with the hard interrupt. As per restrictions imposed by MD code we can only bind interrupts to a single cpu presently. Interrupts can be 'unbound' by binding them to all cpus. Reviewed by: jhb Sponsored by: Nokia
* Move INTR_FILTER from opt_global.h to its own header.jhb2008-04-051-0/+1
|
* Add a MI intr_event_handle() routine for the non-INTR_FILTER case. Thisjhb2008-04-051-21/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | allows all the INTR_FILTER #ifdef's to be removed from the MD interrupt code. - Rename the intr_event 'eoi', 'disable', and 'enable' hooks to 'post_filter', 'pre_ithread', and 'post_ithread' to be less x86-centric. Also, add a comment describe what the MI code expects them to do. - On amd64, i386, and powerpc this is effectively a NOP. - On arm, don't bother masking the interrupt unless the ithread is scheduled in the non-INTR_FILTER case to match what INTR_FILTER did. Also, don't bother unmasking the interrupt in the post_filter case if we never masked it. The INTR_FILTER case had been doing this by having arm_unmask_irq for the post_filter (formerly 'eoi') hook. - On ia64, stray interrupts are now masked for the non-INTR_FILTER case. They were already masked in the INTR_FILTER case. - On sparc64, use the a NULL pre_ithread hook and use intr_enable_eoi() for both the 'post_filter' and 'post_ithread' hooks to match what the non-INTR_FILTER code did. - On sun4v, retire the ithread wrapper hack by using an appropriate 'post_ithread' hook instead (it's what 'post_ithread'/'enable' was designed to do even in 5.x). Glanced at by: piso Reviewed by: marius Requested by: marius [1], [5] Tested on: amd64, i386, arm, sparc64
* - Fix a mis-merge that crept in during the softclock changes.jeff2008-04-041-2/+0
| | | | Spotted by: jhb
* Implement per-cpu callout threads, wheels, and locks.jeff2008-04-021-11/+14
| | | | | | | | | | | | | | | | | | | | | - Move callout thread creation from kern_intr.c to kern_timeout.c - Call callout_tick() on every processor via hardclock_cpu() rather than inspecting callout internal details in kern_clock.c. - Remove callout implementation details from callout.h - Package up all of the global variables into a per-cpu callout structure. - Start one thread per-cpu. Threads are not strictly bound. They prefer to execute on the native cpu but may migrate temporarily if interrupts are starving callout processing. - Run all callouts by default in the thread for cpu0 to maintain current ordering and concurrency guarantees. Many consumers may not properly handle concurrent execution. - The new callout_reset_on() api allows specifying a particular cpu to execute the callout on. This may migrate a callout to a new cpu. callout_reset() schedules on the last assigned cpu while callout_reset_curcpu() schedules on the current cpu. Reviewed by: phk Sponsored by: Nokia
* Simplify the interrupt code a bit:jhb2008-03-171-46/+6
| | | | | | | | | | | | - Always include the ie_disable and ie_eoi methods in 'struct intr_event' and collapse down to one intr_event_create() routine. The disable and eoi hooks simply aren't used currently in the !INTR_FILTER case. - Expand 'disab' to 'disable' in a few places. - Use function casts for arm and i386:intr_eoi_src() instead of wrapper routines since to trim one extra indirection. Compiled on: {arm,amd64,i386,ia64,ppc,sparc64} x {FILTER, !FILTER} Tested on: {amd64,i386} x {FILTER, !FILTER}
* In keeping with style(9)'s recommendations on macros, use a ';'rwatson2008-03-161-1/+2
| | | | | | | | | after each SYSINIT() macro invocation. This makes a number of lightweight C parsers much happier with the FreeBSD kernel source, including cflow's prcc and lxr. MFC after: 1 month Discussed with: imp, rink
* Add preliminary support for binding interrupts to CPUs:jhb2008-03-141-4/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - Add a new intr_event method ie_assign_cpu() that is invoked when the MI code wishes to bind an interrupt source to an individual CPU. The MD code may reject the binding with an error. If an assign_cpu function is not provided, then the kernel assumes the platform does not support binding interrupts to CPUs and fails all requests to do so. - Bind ithreads to CPUs on their next execution loop once an interrupt event is bound to a CPU. Only shared ithreads are bound. We currently leave private ithreads for drivers using filters + ithreads in the INTR_FILTER case unbound. - A new intr_event_bind() routine is used to bind an interrupt event to a CPU. - Implement binding on amd64 and i386 by way of the existing pic_assign_cpu PIC method. - For x86, provide a 'intr_bind(IRQ, cpu)' wrapper routine that looks up an interrupt source and binds its interrupt event to the specified CPU. MI code can currently (ab)use this by doing: intr_bind(rman_get_start(irq_res), cpu); however, I plan to add a truly MI interface (probably a bus_bind_intr(9)) where the implementation in the x86 nexus(4) driver would end up calling intr_bind() internally. Requested by: kmacy, gallatin, jeff Tested on: {amd64, i386} x {regular, INTR_FILTER}
* Remove kernel support for M:N threading.jeff2008-03-121-1/+0
| | | | | | | | While the KSE project was quite successful in bringing threading to FreeBSD, the M:N approach taken by the kse library was never developed to its full potential. Backwards compatibility will be provided via libmap.conf for dynamically linked binaries and static binaries will be broken.
* fix typo in code normally not compiled in.julian2007-10-291-1/+1
|
* Fix typo in code obviously not being compiled on any of my machines.julian2007-10-281-1/+1
| | | | found by: rdivacky@
* rename the process to 'idle' and 'intr' as per jhb.julian2007-10-271-2/+2
|
* if one changes a function's arguments, one must also change the callers.julian2007-10-261-2/+2
|
* Introduce a way to make pure kernal threads.julian2007-10-261-20/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | kthread_add() takes the same parameters as the old kthread_create() plus a pointer to a process structure, and adds a kernel thread to that process. kproc_kthread_add() takes the parameters for kthread_add, plus a process name and a pointer to a pointer to a process instead of just a pointer, and if the proc * is NULL, it creates the process to the specifications required, before adding the thread to it. All other old kthread_xxx() calls return, but act on (struct thread *) instead of (struct proc *). One reason to change the name is so that any old kernel modules that are lying around and expect kthread_create() to make a process will not just accidentally link. fix top to show kernel threads by their thread name in -SH mode add a tdnam formatting option to ps to show thread names. make all idle threads actual kthreads and put them into their own idled process. make all interrupt threads kthreads and put them in an interd process (mainly for aesthetic and accounting reasons) rename proc 0 to be 'kernel' and it's swapper thread is now 'swapper' man page fixes to follow.
* Rename the kthread_xxx (e.g. kthread_create()) callsjulian2007-10-201-6/+6
| | | | | | | | | | | to kproc_xxx as they actually make whole processes. Thos makes way for us to add REAL kthread_create() and friends that actually make theads. it turns out that most of these calls actually end up being moved back to the thread version when it's added. but we need to make this cosmetic change first. I'd LOVE to do this rename in 7.0 so that we can eventually MFC the new kthread_xxx() calls.
* Commit 14/14 of sched_lock decomposition.jeff2007-06-051-24/+24
| | | | | | | | | | | - Use thread_lock() rather than sched_lock for per-thread scheduling sychronization. - Use the per-process spinlock rather than the sched_lock for per-process scheduling synchronization. Tested by: kris, current@ Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc. Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each)
* Rework the PCPU_* (MD) interface:attilio2007-06-041-1/+1
| | | | | | | | | | | | - Rename PCPU_LAZY_INC into PCPU_INC - Add the PCPU_ADD interface which just does an add on the pcpu member given a specific value. Note that for most architectures PCPU_INC and PCPU_ADD are not safe. This is a point that needs some discussions/work in the next days. Reviewed by: alc, bde Approved by: jeff (mentor)
* In some particular cases (like in pccard and pccbb), the real devicepiso2007-05-311-3/+0
| | | | | | | | | | | handler is wrapped in a couple of functions - a filter wrapper and an ithread wrapper. In this case (and just in this case), the filter wrapper could ask the system to schedule the ithread and mask the interrupt source if the wrapped handler is composed of just an ithread handler: modify the "old" interrupt code to make it support this situation, while the "new" interrupt code is already ok. Discussed with: jhb
* Bring in the reminaing bits to make interrupt filtering work:piso2007-05-061-6/+570
| | | | | | | | | | | | | | | o push much of the i386 and amd64 MD interrupt handling code (intr_machdep.c::intr_execute_handlers()) into MI code (kern_intr.c::ithread_loop()) o move filter handling to kern_intr.c::intr_filter_loop() o factor out the code necessary to mask and ack an interrupt event (intr_machdep.c::intr_eoi_src() and intr_machdep.c::intr_disab_eoi_src()), and make them part of 'struct intr_event', passing them as arguments to kern_intr.c::intr_event_create(). o spawn a private ithread per handler (struct intr_handler::ih_thread) with filter and ithread functions. Approved by: re (implicit?)
* Bump the interrupt storm detection counter to 1000. My slow fileservernjl2007-04-191-4/+4
| | | | | | | | | | | gets a bogus irq storm detected when periodic daily kicks off at 3 am and disconnects the disk. Change the print logic to print once per second when the storm is occurring instead of only once. Otherwise, it appeared that something else was causing the errors each night at 3 am since the print only occurred the first time. Reviewed by: jhb MFC after: 1 week
* - Don't do the interrupt storm protection stuff for software interruptjhb2007-03-021-2/+3
| | | | | | | handlers. - Use pause() when throtting during an interrupt storm. Reported by: kris (1)
* Do not execute filter only handlers in ithread_execute_handlers():piso2007-02-271-0/+4
| | | | | this fixes the panics when filter only and ithread only handlers where sharing the same irq .
* o break newbus api: add a new argument of type driver_filter_t topiso2007-02-231-17/+8
| | | | | | | | | | | | | bus_setup_intr() o add an int return code to all fast handlers o retire INTR_FAST/IH_FAST For more info: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=465712+0+current/freebsd-current Reviewed by: many Approved by: re@
* - Remove setrunqueue and replace it with direct calls to sched_add().jeff2007-01-231-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | setrunqueue() was mostly empty. The few asserts and thread state setting were moved to the individual schedulers. sched_add() was chosen to displace it for naming consistency reasons. - Remove adjustrunqueue, it was 4 lines of code that was ifdef'd to be different on all three schedulers where it was only called in one place each. - Remove the long ifdef'd out remrunqueue code. - Remove the now redundant ts_state. Inspect the thread state directly. - Don't set TSF_* flags from kern_switch.c, we were only doing this to support a feature in one scheduler. - Change sched_choose() to return a thread rather than a td_sched. Also, rely on the schedulers to return the idlethread. This simplifies the logic in choosethread(). Aside from the run queue links kern_switch.c mostly does not care about the contents of td_sched. Discussed with: julian - Move the idle thread loop into the per scheduler area. ULE wants to do something different from the other schedulers. Suggested by: jhb Tested on: x86/amd64 sched_{4BSD, ULE, CORE}.
* Add a function to return the MD interrupt source cookie associated withjhb2006-12-121-0/+20
| | | | | an interrupt event. Use this in the x86 code to fixup the intrcnt names when an interrupt handler is removed.
* Add a comment and fix a whitespace nit.jhb2006-12-121-1/+2
|
OpenPOWER on IntegriCloud