summaryrefslogtreecommitdiffstats
path: root/sys/amd64/amd64/exception.S
Commit message (Collapse)AuthorAgeFilesLines
* Merge remote-tracking branch 'origin/releng/11.1' into RELENG_2_4Luiz Souza2018-05-081-1/+115
|\
| * Fix mishandling of x86 debug exceptions. [SA-18:06.debugreg]gordon2018-05-081-1/+115
| | | | | | | | | | | | | | | | | | Bump newvers.sh and UPDATING for today's patches. Approved by: so Security: CVE-2018-8897 Security: FreeBSD-SA-18:06.debugreg Sponsored by: The FreeBSD Foundation
| * Add mitigations for two classes of speculative execution vulnerabilitiesgordon2018-03-141-174/+388
| | | | | | | | | | | | | | | | | | on amd64. [FreeBSD-SA-18:03.speculative_execution] Approved by: so Security: FreeBSD-SA-18:03.speculative_execution Security: CVE-2017-5715 Security: CVE-2017-5754
* | Revert "Proposed fix for CVE-2018-8897"Renato Botelho2018-05-081-115/+1
| | | | | | | | This reverts commit 70d1caf0ad967030b2ce835dc0f116ed1733c82c.
* | Proposed fix for CVE-2018-8897Renato Botelho2018-05-081-1/+115
| |
* | Revert "Revert "MFC ↵Luiz Souza2018-02-231-173/+376
| | | | | | | | | | | | r328083,328096,328116,328119,328120,328128,328135,328153,328157,"" This reverts commit d3d59b01294138e59995b31d2bcbbbdf45e26a3c.
* | Revert "Revert "MFC r322940:""Luiz Souza2018-02-231-3/+3
| | | | | | | | This reverts commit 72b499fe038338698da9878361ba68f79cd05af6.
* | Revert "Revert "MFC r322762, r322799, r322832, r322833:""Luiz Souza2018-02-231-13/+54
| | | | | | | | This reverts commit 5919c0a9658dde48bd090704915aa3a85a6c0d26.
* | Revert "MFC r322762, r322799, r322832, r322833:"Luiz Souza2018-02-211-54/+13
| | | | | | | | This reverts commit 2589da26b930eaf9441b6bf27c0f410062adf507.
* | Revert "MFC r322940:"Luiz Souza2018-02-211-3/+3
| | | | | | | | This reverts commit a9197dec5d4dc4631abb11db58f5cc72ce0625fd.
* | Revert "MFC r328083,328096,328116,328119,328120,328128,328135,328153,328157,"Luiz Souza2018-02-211-376/+173
| | | | | | | | This reverts commit 430a2bea3907149b30cc75fc722b6cf1f81da82a.
* | MFC r328083,328096,328116,328119,328120,328128,328135,328153,328157,kib2018-02-191-173/+376
| | | | | | | | | | | | | | | | | | | | | | | | | | | | 328166,328177,328199,328202,328205,328468,328470,328624,328625,328627, 328628,329214,329297,329365: Meltdown mitigation by PTI, PCID optimization of PTI, and kernel use of IBRS for some mitigations of Spectre. Tested by: emaste, Arshan Khanifar <arshankhanifar@gmail.com> Discussed with: jkim Sponsored by: The FreeBSD Foundation (cherry picked from commit 6dd025b40ee6870bea6ba670f30dcf684edc3f6c)
* | MFC r322940:rlibby2018-02-191-3/+3
| | | | | | | | | | | | amd64: drop q suffix from rd[fg]sbase for gas compatibility (cherry picked from commit c78f11f66bbfbc66d4b5ed31a9dc66831eacdf19)
* | MFC r322762, r322799, r322832, r322833:kib2018-02-191-13/+54
|/ | | | | | Make WRFSBASE and WRGSBASE instructions functional. (cherry picked from commit b1a7a7418e73251aad628dc4f9418e550a9fd3d7)
* sys/amd64: Small spelling fixes.pfg2016-05-031-1/+1
| | | | No functional change.
* Expose doreti as a global symbol on amd64 and i386.jhb2016-04-131-0/+1
| | | | | | | | | | doreti provides the common code path for returning from interrupt andlers on x86. Exposing doreti as a global symbol allows kernel modules to include low-level interrupt handlers instead of requiring all low-level handlers to be statically compiled into the kernel. Submitted by: Howard Su <howard0su@gmail.com> Reviewed by: kib
* [amd64] dtrace_invop handler is to be called only for kernel exceptionsavg2016-04-121-0/+2
| | | | | | | | | | | | DTrace-related exceptions in userland code are handled elsewhere. One practical problem was a crash in dtrace_invop_start() when saved %rsp pointed to a virtual address that was not backed. i386 code already ignored userland exceptions. Reviewed by: markj, kib MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D5906
* Remove an unneeded instruction.markj2015-08-281-1/+0
| | | | MFC after: 1 week
* The iret instruction may generate #np and #ss fault, besides #gp.kib2014-12-161-2/+7
| | | | | | | | | When returning to usermode, the handler for that exceptions is also executed with wrong gs base. Handle all three possible faults in the same way, checking for iret fault, and performing full iret. Sponsored by: The FreeBSD Foundation MFC after: 3 days
* Use a C wrapper for trap() instead of checking and calling the DTrace trapmarkj2014-07-191-18/+1
| | | | | | | | hook in assembly. Suggested by: kib Reviewed by: kib (original version) X-MFC-With: r268600
* Invoke the DTrace trap handler before calling trap() on amd64. This matchesmarkj2014-07-141-0/+17
| | | | | | | | | | | the upstream implementation and helps ensure that a trap induced by tracing fbt::trap:entry is handled without recursively generating another trap. This makes it possible to run most (but not all) of the DTrace tests under common/safety/ without triggering a kernel panic. Submitted by: Anton Rang <anton.rang@isilon.com> (original version) Phabric: D95
* When usermode loaded non-default segment selector into the %gs,kib2014-05-291-3/+20
| | | | | | | | | correctly prepare KGSBASE msr to restore the user descriptor base on the last swapgs during return to usermode. Reported and tested by: peterj Sponsored by: The FreeBSD Foundation MFC after: 1 week
* - For kernel compiled only with KDTRACE_HOOKS and not any lock debuggingattilio2013-11-251-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | option, unbreak the lock tracing release semantic by embedding calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined version of the releasing functions for mutex, rwlock and sxlock. Failing to do so skips the lockstat_probe_func invokation for unlocking. - As part of the LOCKSTAT support is inlined in mutex operation, for kernel compiled without lock debugging options, potentially every consumer must be compiled including opt_kdtrace.h. Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES is linked there and it is only used as a compile-time stub [0]. [0] immediately shows some new bug as DTRACE-derived support for debug in sfxge is broken and it was never really tested. As it was not including correctly opt_kdtrace.h before it was never enabled so it was kept broken for a while. Fix this by using a protection stub, leaving sfxge driver authors the responsibility for fixing it appropriately [1]. Sponsored by: EMC / Isilon storage division Discussed with: rstone [0] Reported by: rstone [1] Discussed with: philip
* Some clarifications and updates for the comments, mostly retrievedkib2013-06-191-12/+16
| | | | | | from Bruce Evans. Trim the trailing spaces. MFC after: 1 week
* When handling an exception from the attempt from loading the faultingkib2013-05-271-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | context on return from the trap handler, re-enable the interrupts on i386 and amd64. The trap return path have to disable interrupts since the sequence of loading the machine state is not atomic. The trap() function which transfers the control to the special handler would enable the interrupt, but an iret loads the previous eflags with PSL_I clear. Then, the special handler calls trap() on its own, which now sees the original eflags with PSL_I set and does not enable interrupts. The end result is that signal delivery and process exiting code could be executed with interrupts disabled, which is generally wrong and triggers several assertions. For amd64, the interrupts are enabled conditionally based on PSL_I in the eflags of the outer frame, as it is already done for doreti_iret_fault. For i386, the interrupts are enabled unconditionally, the ast loop could have opened a window with interrupts enabled just before the iret anyway. Reported and tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 1 week
* Microoptimize the return path for the fast syscalls on amd64. Arrangekib2011-09-151-10/+10
| | | | | | | | | | the code to have the fall-through path to follow the likely target. Do not use intermediate register to reload user %rsp. Proposed by: alc Reviewed by: alc, jhb Approved by: re (bz) MFC after: 2 weeks
* The jump target shall be after the padding, not into it.kib2011-09-111-1/+2
| | | | | | Reported by: alc Approved by: re (bz) MFC after: 2 weeks
* Perform amd64-specific microoptimizations for native syscall entrykib2011-09-111-8/+6
| | | | | | | | | | | | sequence. The effect is ~1% on the microbenchmark. In particular, do not restore registers which are preserved by the C calling sequence. Align the jump target. Avoid unneeded memory accesses by calculating some data in syscall entry trampoline. Reviewed by: jhb Approved by: re (bz) MFC after: 2 weeks
* Disable local interrupts before testing the PCB_FULL_IRET flag.kib2011-04-081-1/+2
| | | | | | | | | Thread might be preempted after testing, which causes the flag to be cleared. If ast was not delivered, we will do sysret with potentially wrong fs/gs bases. Reviewed by: jhb, jkim MFC after: 1 week (together with r220430, r220452)
* Fix a bug in the previous change to restore the fast path for syscalljhb2011-04-081-6/+5
| | | | | | | | | | | | | return. The ast() function may cause a context switch in which case PCB_FULL_IRET would be set in the pcb. However, the code was not rechecking the flag after ast() returned and would not properly restore the FSBASE and GSBASE MSRs. To fix, recheck the PCB_FULL_IRET flag after ast() returns. While here, trim an instruction (and memory access) from the doreti path and fix a typo in a comment. MFC after: 1 week
* pcb_flags is an int, so use testl rather than testq.jhb2011-04-071-1/+1
| | | | | | Pointy hat to: jhb Submitted by: jkim MFC after: 1 week
* If a system call does not request a full interrupt return, use a fastjhb2011-04-071-0/+33
| | | | | | | | | | path via the sysretq instruction to return from the system call. This was removed in 190620 and not quite fully restored in 195486. This resolves most of the performance regression in system call microbenchmarks between 7 and 8 on amd64. Reviewed by: kib MFC after: 1 week
* Increase size of pcb_flags to four bytes.jkim2010-12-221-6/+6
| | | | Requested by: bde, jhb
* Improve PCB flags handling and make it more robust. Add two new functionsjkim2010-12-221-7/+7
| | | | | | | | | | | | | | | | for manipulating pcb_flags. These inline functions are very similar to atomic_set_char(9) and atomic_clear_char(9) but without unnecessary LOCK prefix for SMP. Add comments about the rationale[1]. Use these functions wherever possible. Although there are some places where it is not strictly necessary (e.g., a PCB is copied to create a new PCB), it is done across the board for sake of consistency. Turn pcb_full_iret into a PCB flag as it is safe now. Move rarely used fields before pcb_flags and reduce size of pcb_flags to one byte. Fix some style(9) nits in pcb.h while I am in the neighborhood. Reviewed by: kib Submitted by: kib[1] MFC after: 2 months
* Retire write-only PCB_FULLCTX pcb flag on amd64.kib2010-12-071-1/+0
| | | | | | Reminded by: Petr Salinger <Petr.Salinger seznam cz> Tested by: pho MFC after: 1 week
* Register an interrupt vector for DTrace return probes. There is somerpaulo2010-08-281-0/+4
| | | | | | | code missing in lapic to make sure that we don't overwrite this entry, but this will be done on a sequent commit. Sponsored by: The FreeBSD Foundation
* Clear DF bit in eflags/rflags on the kernel entry. The i386 and amd64kib2010-06-231-0/+4
| | | | | | | | | ABI specifies the DF should be zero, and newer compilers do not clear DF before using DF-sensitive instructions. The DF clearing for signal handlers was done some time ago. MFC after: 1 week
* Route all returns from the interrupts and faults through the doreti_iretkib2010-05-121-1/+1
| | | | | | | | | | | | | | | | | labeled iretq instruction. Suppose that multithreaded process executes two threads, currently scheduled on different processors. Let assume that thread A executes using %cs or %ss pointing into the descriptor from LDT. If IPI comes which handler does not return by jump to doreti, and meantime thread B invalidates descriptor pointed to by %cs or %ss, then iretq from IPI handler could fault. Routing the return by doreti_iret allows kernel to catch the situation and recover from it by sending signal to the usermode. Tested by: pho MFC after: 1 week
* Style and comment adjustements.kib2010-05-031-34/+45
| | | | | Suggested and reviewed by: bde MFC after: 3 days
* ld_gs_base is executing with stack containing only the frame,kib2010-04-141-1/+0
| | | | | | | temporary pushed %rflags has been popped already. Pointy hat to: kib MFC after: 3 days
* Handle a case when non-canonical address is loaded into the fsbase orkib2010-04-101-2/+28
| | | | | | gsbase MSR. MFC after: 3 days
* Provide groundwork for 32-bit binary compatibility on non-x86 platforms,nwhitehorn2010-03-111-1/+1
| | | | | | | | | for upcoming 64-bit PowerPC and MIPS support. This renames the COMPAT_IA32 option to COMPAT_FREEBSD32, removes some IA32-specific code from MI parts of the kernel and enhances the freebsd32 compatibility code to support big-endian platforms. Reviewed by: kib, jhb
* Introduce the new kernel sub-tree x86 which should contain all the codeattilio2010-02-251-1/+1
| | | | | | | | | | | | | | shared and generalized between our current amd64, i386 and pc98. This is just an initial step that should lead to a more complete effort. For the moment, a very simple porting of cpufreq modules, BIOS calls and the whole MD specific ISA bus part is added to the sub-tree but ideally a lot of code might be added and more shared support should grow. Sponsored by: Sandvine Incorporated Reviewed by: emaste, kib, jhb, imp Discussed on: arch MFC: 3 weeks
* When amd64 CPU cannot load segment descriptor during trap return tokib2009-07-101-12/+12
| | | | | | | | | | | | | | usermode, it generates GPF, that is mirrored to user mode as SIGSEGV. The offending register in mcontext should contain the value loading of which generated the GPF, and it is so on i386. On amd64, we currently report segment descriptor in tf_err, while segment register contains the corrected value loaded by trap handler. Fix the issue by behaving like i386, reloading segment register in trap frame after signal frame is pushed onto user stack. Noted and tested by: pho Approved by: re (kensmith)
* Restore the segment registers and segment base MSRs for amd64 syscallkib2009-07-091-9/+19
| | | | | | | | | | | | | | | | | return path only when neither thread was context switched while executing syscall code nor syscall explicitely modified LDT or MSRs. Save segment registers in trap handlers before interrupts are enabled, to not allow context switches to happen before registers are saved. Use separated byte in pcb for indication of fast/full return, since pcb_flags are not synchronized with context switches. The change puts back syscall microbenchmark numbers that were slowed down after commit of the support for LDT on amd64. Reviewed by: jeff Tested (and tested, and tested ...) by: pho Approved by: re (kensmith)
* Save and restore segment registers on amd64 when entering and leavingkib2009-04-011-50/+147
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the kernel on amd64. Fill and read segment registers for mcontext and signals. Handle traps caused by restoration of the invalidated selectors. Implement user-mode creation and manipulation of the process-specific LDT descriptors for amd64, see sysarch(2). Implement support for TSS i/o port access permission bitmap for amd64. Context-switch LDT and TSS. Do not save and restore segment registers on the context switch, that is handled by kernel enter/leave trampolines now. Remove segment restore code from the signal trampolines for freebsd/amd64, freebsd/ia32 and linux/i386 for the same reason. Implement amd64-specific compat shims for sysarch. Linuxolator (temporary ?) switched to use gsbase for thread_area pointer. TODO: Currently, gdb is not adapted to show segment registers from struct reg. Also, no machine-depended ptrace command is added to set segment registers for debugged process. In collaboration with: pho Discussed with: peter Reviewed by: jhb Linuxolator tested by: dchagin
* Improve robustness of NMI handling, for NMIs recognized in kerneljkoshy2009-02-031-25/+52
| | | | | | | | | | | | | | | | | | | mode. - Make the NMI handler run on its own stack (TSS_IST2). - Store the GSBASE value for each CPU just before the start of each NMI stack, permitting efficient retrieval using %rsp-relative addressing. - For NMIs taken from kernel mode, program MSR_GSBASE explicitly since one or both of MSR_GSBASE and MSR_KGSBASE can be potentially invalid. The current contents of MSR_GSBASE are saved and restored at exit. - For NMIs handled from user mode, continue to use 'swapgs' to load the per-CPU GSBASE. Reviewed by: jeff Debugging help: jeff Tested by: gnn, Artem Belevich <artemb at gmail dot com>
* Disable interrupts, if they were enabled, before doing swapgs.kib2009-01-141-0/+1
| | | | | | | | Otherwise, interrupt may happen while we run with kernel CS and usermode gsbase. Reviewed by: jeff MFC after: 1 week
* Bug fix: %ebx needs to be preserved in the user callchain capturejkoshy2008-12-141-4/+4
| | | | path.
* - Bug fix: prevent a thread from migrating between CPUs between thejkoshy2008-12-131-6/+10
| | | | | | | | | | | | | time it is marked for user space callchain capture in the NMI handler and the time the callchain capture callback runs. - Improve code and control flow clarity by invoking hwpmc(4)'s user space callchain capture callback directly from low-level code. Reviewed by: jhb (kern/subr_trap.c) Testing (various patch revisions): gnn, Fabien Thomas <fabien dot thomas at netasq dot com>, Artem Belevich <artemb at gmail dot com>
OpenPOWER on IntegriCloud