summaryrefslogtreecommitdiffstats
path: root/sys/powerpc/aim/trap.c
Commit message (Collapse)AuthorAgeFilesLines
* Rework SLB trap handling so that double-faults into an SLB trap handler arenwhitehorn2012-01-151-25/+59
| | | | | | | | | | | | | | possible, and double faults within an SLB trap handler are not. The result is that it possible to take an SLB fault at any time, on any address, for any reason, at any point in the kernel. This lets us do two important things. First, it removes the (soft) 16 GB RAM ceiling on PPC64 as well as any architectural limitations on KVA space. Second, it lets the kernel tolerate poorly designed hypervisors that have a tendency to fail to restore the SLB properly after a hypervisor context switch. MFC after: 6 weeks
* Implement hwpmc counting PMC support for PowerPC G4+ (MPC745x/MPC744x).jhibbits2011-12-241-0/+15
| | | | | | | Sampling is in progress. Approved by: nwhitehorn (mentor) MFC after: 9.0-RELEASE
* Inline the syscallenter() and syscallret(). This reduces the time measuredkib2011-09-111-0/+2
| | | | | | | | by the syscall entry speed microbenchmarks by ~10% on amd64. Submitted by: jhb Approved by: re (bz) MFC after: 2 weeks
* Use the ABI-mandated thread pointer register (r2 for ppc32, r13 for ppc64)nwhitehorn2011-06-231-3/+3
| | | | | | | | | | | | | | | instead of a PCPU field for curthread. This averts a race on SMP systems with a high interrupt rate where the thread looking up the value of curthread could be preempted and migrated between obtaining the PCPU pointer and reading the value of pc_curthread, resulting in curthread being observed to be the current thread on the thread's original CPU. This played merry havoc with the system, in particular with mutexes. Many thanks to jhb for helping me work this one out. Note that Book-E is in principle susceptible to the same problem, but has not been modified yet due to lack of Book-E hardware. MFC after: 2 weeks
* Add macro to test the sv_flags of any process. Change some places to testdchagin2011-01-261-3/+3
| | | | | | | the flags instead of explicit comparing with address of known sysentvec structures. MFC after: 1 month
* Fix handling of NX pages on capable CPUs. Thanks to kib for prodding menwhitehorn2011-01-131-1/+3
| | | | in the right direction.
* Fix two mistakes on 32-bit systems. The slbmte code in syscall() is 64-bitnwhitehorn2010-11-031-0/+2
| | | | | | only, and should be protected with an ifdef, and the no-execute bit in 32-bit set_user_sr() should be set before the comparison, not after, or it will never match.
* Clean up the user segment handling code a little more. Now thatnwhitehorn2010-11-031-6/+7
| | | | | | | | set_user_sr() itself caches the user segment VSID, there is no need for cpu_switch() to do it again. This change also unifies the 32 and 64-bit code paths for kernel faults on user pages and remaps the user SLB slot on 64-bit systems when taking a syscall to avoid some unnecessary segment exception traps.
* Restructure the way the copyin/copyout segment is stored to prevent anwhitehorn2010-10-301-1/+9
| | | | | | | | | | | | | | | | | concurrency bug. Since all SLB/SR entries were invalidated during an exception, a decrementer exception could cause the user segment to be invalidated during a copyin()/copyout() without a thread switch that would cause it to be restored from the PCB, potentially causing the operation to continue on invalid memory. This is now handled by explicit restoration of segment 12 from the PCB on 32-bit systems and a check in the Data Segment Exception handler on 64-bit. While here, cause copyin()/copyout() to check whether the requested user segment is already installed, saving some pipeline flushes, and fix the synchronization primitives around the mtsr and slbmte instructions to prevent accessing stale segments. MFC after: 2 weeks
* Handle vector assist traps without a kernel panic, by setting denormalizednwhitehorn2010-10-051-3/+13
| | | | | | | | values to zero. A correct solution would involve emulating vector operations on denormalized values, but this has little effect on accuracy and is much less complicated for now. MFC after: 2 weeks
* Split the SLB mirror cache into two kinds of object, one for kernel mapsnwhitehorn2010-09-161-10/+8
| | | | | | | which are similar to the previous ones, and one for user maps, which are arrays of pointers into the SLB tree. This changes makes user SLB updates atomic, closing a window for memory corruption. While here, rearrange the allocation functions to make context switches faster.
* Replace the SLB backing store splay tree used on 64-bit PowerPC AIMnwhitehorn2010-09-161-25/+17
| | | | | | | | hardware with a lockless sparse tree design. This marginally improves the performance of PMAP and allows copyin()/copyout() to run without acquiring locks when used on wired mappings. Submitted by: mdf
* Remove unused KTRACE includes.jhb2010-08-191-5/+0
|
* MFppc64:nwhitehorn2010-07-131-19/+130
| | | | | | | Kernel sources for 64-bit PowerPC, along with build-system changes to keep 32-bit kernels compiling (build system changes for 64-bit kernels are coming later). Existing 32-bit PowerPC kernel configurations must be updated after this change to specify their architecture.
* Reorganize syscall entry and leave handling.kib2010-05-231-88/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extend struct sysvec with three new elements: sv_fetch_syscall_args - the method to fetch syscall arguments from usermode into struct syscall_args. The structure is machine-depended (this might be reconsidered after all architectures are converted). sv_set_syscall_retval - the method to set a return value for usermode from the syscall. It is a generalization of cpu_set_syscall_retval(9) to allow ABIs to override the way to set a return value. sv_syscallnames - the table of syscall names. Use sv_set_syscall_retval in kern_sigsuspend() instead of hardcoding the call to cpu_set_syscall_retval(). The new functions syscallenter(9) and syscallret(9) are provided that use sv_*syscall* pointers and contain the common repeated code from the syscall() implementations for the architecture-specific syscall trap handlers. Syscallenter() fetches arguments, calls syscall implementation from ABI sysent table, and set up return frame. The end of syscall bookkeeping is done by syscallret(). Take advantage of single place for MI syscall handling code and implement ptrace_lwpinfo pl_flags PL_FLAG_SCE, PL_FLAG_SCX and PL_FLAG_EXEC. The SCE and SCX flags notify the debugger that the thread is stopped at syscall entry or return point respectively. The EXEC flag augments SCX and notifies debugger that the process address space was changed by one of exec(2)-family syscalls. The i386, amd64, sparc64, sun4v, powerpc and ia64 syscall()s are changed to use syscallenter()/syscallret(). MIPS and arm are not converted and use the mostly unchanged syscall() implementation. Reviewed by: jhb, marcel, marius, nwhitehorn, stas Tested by: marcel (ia64), marius (sparc64), nwhitehorn (powerpc), stas (mips) MFC after: 1 month
* Allow user programs to execute mfpvr instructions. Linux allows this, andnwhitehorn2010-02-221-1/+22
| | | | | | | some math-related software like GMP expects to be able to use it to pick a target appropriately. MFC after: 1 week
* Simplify the invocation of vm_fault(). Specifically, eliminate the flagalc2009-11-271-3/+1
| | | | | | | VM_FAULT_DIRTY. The information provided by this flag can be trivially inferred by vm_fault(). Discussed with: kib
* Garbage collect some code that was never compiled in to handle Altivecnwhitehorn2009-11-221-6/+0
| | | | during traps. It predates actual Altivec support and was never used.
* Extract the code that records syscall results in the frame into MDkib2009-11-101-37/+2
| | | | | | | | | | | function cpu_set_syscall_retval(). Suggested by: marcel Reviewed by: marcel, davidxu PowerPC, ARM, ia64 changes: marcel Sparc64 tested and reviewed by: marius, also sunv reviewed MIPS tested by: gonzo MFC after: 1 month
* Garbage collect set_user_sr(), which is declared static inline andnwhitehorn2009-10-311-9/+0
| | | | never called.
* Fix a typo in the SRR1 comparison for program exceptions. While here,nwhitehorn2009-04-191-3/+2
| | | | | | | | | | | replace magic numbers with constants to keep this from happening again. Without this fix, some programs would occasionally get SIGTRAP instead of SIGILL on an illegal instruction. This affected Altivec detection in pixman, and possibly other software. Reported by: Andreas Tobler MFC after: 1 week
* Add Altivec support for supported CPUs. This is derived from the FPU supportnwhitehorn2009-02-201-12/+5
| | | | | | | | code, and also reducing the size of trapcode to fit inside a 32 byte handler slot. Reviewed by: grehan MFC after: 2 weeks
* Add link register to fatal trap printout to better diagnose NULLgrehan2008-06-041-0/+1
| | | | function pointer derefs.
* Remove kernel support for M:N threading.jeff2008-03-121-5/+0
| | | | | | | | While the KSE project was quite successful in bringing threading to FreeBSD, the M:N approach taken by the kse library was never developed to its full potential. Backwards compatibility will be provided via libmap.conf for dynamically linked binaries and static binaries will be broken.
* Unify and generalize PowerPC headers, adjust AIM code accordingly.raj2008-03-021-7/+7
| | | | | | | | | | | | | | | | | | | | Rework of this area is a pre-requirement for importing e500 support (and other PowerPC core variations in the future). Mainly the following headers are refactored so that we can cover for low-level differences between various machines within PowerPC architecture: <machine/pcpu.h> <machine/pcb.h> <machine/kdb.h> <machine/hid.h> <machine/frame.h> Areas which use the above are adjusted and cleaned up. Credits for this rework go to marcel@ Approved by: cognet (mentor) MFp4: e500
* A bunch more files that should probably print out a thread namejulian2007-11-141-3/+3
| | | | instead of a process name.
* generally we are interested in what thread did something asjulian2007-11-141-1/+1
| | | | | | opposed to what process. Since threads by default have teh name of the process unless over-written with more useful information, just print the thread name instead.
* Remove redundant prototype.marcel2007-08-071-2/+0
| | | | Approved by: re (blanket)
* Fix backward compatibility of the "old" (i.e. FreeBSD6) lseekmarcel2007-07-311-2/+2
| | | | | | | | | | | syscall. It was broken when a new lseek syscall was introduced. The problem is that we need to swap the 32-bit td_retval values for the __syscall indirect syscall when the actual syscall has a 32-bit return value. Hence, we need to exclude lseek(2). And this means the "old" lseek(2) as well -- which we didn't. Based on a patch from: grehan@ Approved by: re (rwatson)
* Rework the PCPU_* (MD) interface:attilio2007-06-041-2/+2
| | | | | | | | | | | | - Rename PCPU_LAZY_INC into PCPU_INC - Add the PCPU_ADD interface which just does an add on the pcpu member given a specific value. Note that for most architectures PCPU_INC and PCPU_ADD are not safe. This is a point that needs some discussions/work in the next days. Reviewed by: alc, bde Approved by: jeff (mentor)
* Over NFS, an open() call could result in multiple over-the-wiremohans2007-03-091-0/+2
| | | | | | | | | | | | GETATTRs being generated - one from lookup()/namei() and the other from nfs_open() (for cto consistency). This change eliminates the GETATTR in nfs_open() if an otw GETATTR was done from the namei() path. Instead of extending the vop interface, we timestamp each attr load, and use this to detect whether a GETATTR was done from namei() for this syscall. Introduces a thread-local variable that counts the syscalls made by the thread and uses <pid, tid, thread syscalls> as the attrload timestamp. Thanks to jhb@ and peter@ for a discussion on thread state that could be used as the timestamp with minimal overhead.
* Make KSE a kernel option, turned on by default in all GENERICjb2006-10-261-0/+2
| | | | | | | kernel configs except sun4v (which doesn't process signals properly with KSE). Reviewed by: davidxu@
* Add audit hooks for ppc, ia64 system call paths.rwatson2006-09-161-0/+4
| | | | | | Reviewed by: marcel (ia64) Obtained from: TrustedBSD Project MFC after: 3 days
* Retire SYF_ARGMASK and remove both SYF_MPSAFE and SYF_ARGMASK. sy_narg isjhb2006-07-281-1/+1
| | | | now back to just being an argument count.
* Now that all system calls are MPSAFE, retire the SYF_MPSAFE flag used tojhb2006-07-281-9/+0
| | | | | | | | | mark system calls as being MPSAFE: - Stop conditionally acquiring Giant around system call invocations. - Remove all of the 'M' prefixes from the master system call files. - Remove support for the 'M' prefix from the script that generates the syscall-related files from the master system call files. - Don't explicitly set SYF_MPSAFE when registering nfssvc.
* Unify the checking for lock misbehavior in the various syscall()jhb2006-07-271-5/+13
| | | | | | | | | | | | | | implementations and adjust some of the checks while I'm here: - Add a new check to make sure we don't return from a syscall in a critical section. - Add a new explicit check before userret() to make sure we don't return with any locks held. The advantage here is that we can include the syscall number and name in syscall() whereas that info is not available in userret(). - Drop the mtx_assert()'s of sched_lock and Giant. They are replaced by the more general checks just added. MFC after: 2 weeks
* Add missing ptrace(2) system-call stops to various syscall()jhb2006-07-271-0/+5
| | | | | | implementations. MFC after: 1 week
* Simplify system time accounting for profiling.phk2006-02-081-4/+3
| | | | | | | | | | Rename struct thread's td_sticks to td_pticks, we will need the other name for more appropriately named use shortly. Reduce it from uint64_t to u_int. Clear td_pticks whenever we enter the kernel instead of recording its value as reference for userret(). Use the absolute value of td->pticks in userret() and eliminate third argument.
* 1. Change prototype of trapsignal and sendsig to use ksiginfo_t *, mostdavidxu2005-10-141-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | changes in MD code are trivial, before this change, trapsignal and sendsig use discrete parameters, now they uses member fields of ksiginfo_t structure. For sendsig, this change allows us to pass POSIX realtime signal value to user code. 2. Remove cpu_thread_siginfo, it is no longer needed because we now always generate ksiginfo_t data and feed it to libpthread. 3. Add p_sigqueue to proc structure to hold shared signals which were blocked by all threads in the proc. 4. Add td_sigqueue to thread structure to hold all signals delivered to thread. 5. i386 and amd64 now return POSIX standard si_code, other arches will be fixed. 6. In this sigqueue implementation, pending signal set is kept as before, an extra siginfo list holds additional siginfo_t data for signals. kernel code uses psignal() still behavior as before, it won't be failed even under memory pressure, only exception is when deleting a signal, we should call sigqueue_delete to remove signal from sigqueue but not SIGDELSET. Current there is no kernel code will deliver a signal with additional data, so kernel should be as stable as before, a ksiginfo can carry more information, for example, allow signal to be delivered but throw away siginfo data if memory is not enough. SIGKILL and SIGSTOP have fast path in sigqueue_add, because they can not be caught or masked. The sigqueue() syscall allows user code to queue a signal to target process, if resource is unavailable, EAGAIN will be returned as specification said. Just before thread exits, signal queue memory will be freed by sigqueue_flush. Current, all signals are allowed to be queued, not only realtime signals. Earlier patch reviewed by: jhb, deischen Tested on: i386, amd64
* Temporary band-aid to fix hang when a process exec's Altivec instructions.grehan2005-07-301-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | trap_subr.S: declare a stub for the a-unavailable trap that does an absolute jump to the vector-assist trap. This is due to the fact that the vec-unavail trap doesn't start at a 256-byte boundary, so the trick of masking the bottom 8 bits of the link register to identify the interrupt doesn't work, so let the vec-assist case handle Altivec-disabled for the time being. Note that this will be fixed in the future with a much smaller vector code-stub (< 16 bytes) that will allow use of strange vector offsets that are also present in 4xx processors, and also allow smaller differences in vector codepaths on the G5. trap.c: Treat altivec-unavailable/assist process traps as SIGILL. Not quite correct, since altivec-assist should really be a panic, but it is fine for the moment due to the above measure. machdep.c Install the stub code for the altivec-unavailable trap, and the standard trap code at the altivec-assist. Reported by: Andreas Tobler <toa at pop agri ch> MFC after: 3 days
* Don't enter the debugger if KDB_UNATTENDED is set or ifps2005-04-201-2/+0
| | | | | | debug.debugger_on_panic=0. MFC after: 2 weeks
* Use PCPU_LAZY_INC() for cnt.v_{intr,trap,syscalls} rather than atomicjhb2005-04-121-2/+2
| | | | operations in some places and simple non-per CPU math in others.
* Include <sys/signalvar.h> for trapsignal prototype.grehan2005-03-151-0/+1
|
* /* -> /*- for license, minor formatting changesimp2005-01-071-1/+1
|
* Don't include sys/user.h merely for its side-effect of recursivelydas2004-11-271-1/+0
| | | | including other headers.
* Remove an unneeded argument..julian2004-08-311-1/+1
| | | | | | | | | The removed argument could trivially be derived from the remaining one. That in turn should be the same as curthread, but it is possible that curthread could be expensive to derive on some syste,s so leave it as an argument. Having both proc and thread as an argumen tjust gives an opportunity for them to get out sync. MFC after: 3 days
* Remove sched_free_thread() which was only usedjulian2004-08-311-3/+0
| | | | | | | | in diagnostics. It has outlived its usefulness and has started causing panics for people who turn on DIAGNOSTIC, in what is otherwise good code. MFC after: 2 days
* Kernel traps were not being passed to trap_fatal in somegrehan2004-08-021-1/+2
| | | | | | circumstances. Spotted by: gallatin
* Bring into KDB new order.grehan2004-07-121-9/+6
|
* - correctly set the return value for the copyin/out fault buffer to 1grehan2004-07-091-2/+10
| | | | | | | | | | | so setfault would return correctly when a page fault was invalid (e.g. a syscall with a bad parameter). This caused an endless DSI loop, seen when running sendmail which does a setlogin() call with a NULL pointer. - introduce KTR_SYSC tracing. expose the syscallnames[] array to make the tracing more readable.
OpenPOWER on IntegriCloud