summaryrefslogtreecommitdiffstats
path: root/sys/i386/include
Commit message (Collapse)AuthorAgeFilesLines
* Revert "Revert "MFC r319871:""Luiz Souza2018-02-231-7/+7
| | | | This reverts commit 63302e53ed4b3fe59711d939ba87433a9a12199d.
* Revert "Revert "MFC 322323 by jkim""Luiz Souza2018-02-231-1/+0
| | | | This reverts commit b634270605d0823a7110d2385de63e1e3a74f786.
* Revert "MFC 322323 by jkim"Luiz Souza2018-02-211-0/+1
| | | | This reverts commit cf2b1f5568d7328668ab821286d6b946275f8d6f.
* Revert "MFC r319871:"Luiz Souza2018-02-211-7/+7
| | | | This reverts commit 045793a68906243d204d36b336867187c1a33f00.
* MFC r319871:kib2018-02-191-7/+7
| | | | | | | Make struct syscall_args visible to userspace compilation environment from machine/proc.h, consistently on all architectures. (cherry picked from commit 06d5fa0600b92e97e90e41785ef10f641bdec89f)
* MFC 322323 by jkimsephe2018-02-191-1/+0
| | | | | | | | | | | | Split identify_cpu() into two functions for amd64 as we do for i386. This reduces diff between amd64 and i386. Also, it fixes a regression introduced in r322076, i.e., identify_hypervisor() failed to identify some hypervisors. This function assumes cpu_feature2 is already initialized. Reported by: dexuan Tested by: dexuan (cherry picked from commit 173ac9160e83a9396d44938ef9789b6e62e48e70)
* MFC r314310alc2017-06-281-0/+1
| | | | | | | | | | Refine the fix from r312954. Specifically, add a new PDE-only flag, PG_PROMOTED, that indicates whether lingering 4KB page mappings might need to be flushed on a PDE change that restricts or destroys a 2MB page mapping. This flag allows the pmap to avoid range invalidations that are both unnecessary and costly. Approved by: re (kib)
* MFC efivar(8) (by imp):kib2017-05-201-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | List of revisions merged: r307070 r307071 r307072 r307074 r307189 r307224 r307339 r307390 r307391 r309776 r314231 r314232 r314615 r314616 r314617 r314618 r314619 r314620 r314621 r314623 r314890 r314925 r314926 r314927 r314928 r315770 r315771 Discussed with: gjb (re), imp Sponsored by: The FreeBSD Foundation
* MFC r303261,r315059:mmel2017-04-161-0/+3
| | | | | | | | r303261: Add more UEFI/e820 memory types from latest specifications. r315059: Split overbloated machep.c to multiple files and do basic cleanup of these fragments.
* MFC r311169,r311898,r312925,r312973,r312975,r313007,r313040,r313080,mjg2017-03-161-0/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | r313254,r313341 amd64: add atomic_fcmpset == sparc64: add atomic_fcmpset == Implement atomic_fcmpset_* for arm and arm64. == Add atomic_fcmpset_*() inlines for powerpc Summary: atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value from the target memory location into the 'old' pointer in the case of failure. == i386: add atomic_fcmpset == Don't retry a lost reservation in atomic_fcmpset() The desired behavior of atomic_fcmpset_() is to always exit on error. Instead of retrying on lost reservation, leave the retry to the caller, and return == Add atomic_fcmpset_*() inlines for MIPS atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value from the target memory location into the 'old' pointer. == i386: fixup fcmpset An incorrect output specifier was used which worked with clang by accident, but breaks with the in-tree gcc version. While here plug a whitespace nit. == Implement atomic_fcmpset_*() for RISC-V. == Use 64bit store instruction in atomic_fcmpset_64.
* MFC r313154:kib2017-02-241-29/+0
| | | | | For i386, remove config options CPU_DISABLE_CMPXCHG, CPU_DISABLE_SSE and device npx.
* MFC r312952:jah2017-02-171-0/+10
| | | | | | | | Implement get_pcpu() for i386 and use it to replace pcpu_find(curcpu) in the i386 pmap. The curcpu macro loads the per-cpu data pointer as its first step, so the remaining steps of pcpu_find(curcpu) are circular.
* MFC r312555:kib2017-02-031-0/+7
| | | | Use SFENCE for ordering CLFLUSHOPT.
* MFC r310481:jah2017-01-071-1/+9
| | | | | | | | | | | | | | | | Move the objects used to create temporary mappings for i386 pmap zero and copy operations to the MD PCPU region. Change sysmap initialization to only allocate KVA pages for CPUs that are actually present. As a minor optimization, this also prevents false sharing between adjacent sysmap objects since the pcpu struct is already cacheline-aligned. While here, move pc_qmap_addr initialization for the BSP into pmap_bootstrap(), which allows use of pmap_quick* functions during early boot. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D8833
* MFC 305836: Remove 'cpu' and 'cpu_class' on amd64.jhb2016-11-082-0/+6
| | | | | | | | The 'cpu' and 'cpu_class' variables were always set to the same value on amd64 and are legacy holdovers from i386. Remove them entirely on amd64. Requested by: kib (MFC)
* Merge bde improvements for ddb on x86, mostly for single-stepping andkib2016-11-071-5/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | vm86 mode. MFC r304085 (by bde): Fix the variables $esp, $ds, $es, $fs, $gs and $ss in vm86 mode. Fix PC_REGS() so that printing of instructions works in some useful cases. MFC r304962 (by bde): Expand error messages: print symbol names, parentheses and shift tokens, and negative shift counts. Fix error messages. MFC r305612 (by bde): Fix single-stepping of instructions emulated by vm86. MFC r305661 (by bde): Give the full syntax of the 'count' arg for all commmands that support it. Give the full syntax of the 'addr' arg for these commands and some others. Rename it from 'address' for the generic command. Fix description of how 'count' is supposed to work for the 'break' command. Don't (mis)describe the syntax of the comma for the 'step' command. Expand the description for the generic command. Give the full syntax for the 'examine' command. It was also missing the possible values for the modifier. MFC r305663 (by bde): Fix stopping when the specified breakpoint count is reached. MFC r305665 (by bde): Pass the trap type and code down from db_trap() to db_stop_at_pc() so that the latter can easily determine what the trap type actually is after callers are fixed to encode the type unambigously. MFC r305807 (by bde): Use the MI macro TRAPF_USERMODE() instead of open-coded checks for SEL_UPL and sometimes PSL_VM. Fix logic errors in treating vm86 bioscall mode as kernel mode. The main place checked all the necessary flags, but put the necessary parentheses for the PSL_VM and PCB_VM86CALL checks in the wrong place. MFC r305811 (by bz): Try to fix LINT builds after r305807. MFC r305840 (by bde): Abort single stepping in ddb if the trap is not for single-stepping. MFC r305862 (by bde): Ifdef the new dr6 variable for KDB. MFC r305864 (by bde): Statically initialize the run mode to the one that will become current on first entry. Don't reset to the run mode to STEP_NONE when stopping, and remove STEP_NONE. MFC r305865 (by bde): Fix decoding of tf_rsp on amd64, and move TF_HAS_STACKREGS() to the i386-only section, and fix a comment about the amd64 kernel trapframe not having stackregs. MFC r305897 (by bde): Silently ignore unexpected single-step traps. MFC r306311 (by bde): Determine the operand/address size of %cs in a new function db_segsize(). Use db_segsize() to set the default operand/address size for disassembling. Fix db_print_loc_and_inst() to ask for the normal format and not the alternate in normal operation. Use db_segsize() to avoid trying to print a garbage stack trace if %cs is 16 bits.
* MFC r306680:kib2016-10-241-1/+2
| | | | Reduce the cost of TLB invalidation on x86 by using per-CPU completion flags.
* MFC 304637: Fix build for !SMP kernels after the Xen MSIX workaround.jhb2016-09-091-1/+2
| | | | | | | Move msix_disable_migration under #ifdef SMP since it doesn't make sense for !SMP kernels. PR: 212014
* MFC r302635:royger2016-07-151-0/+2
| | | | | | xen: automatically disable MSI-X interrupt migration Approved by: re (kib)
* Replace a number of conflations of mp_ncpus and mp_maxid with eithernwhitehorn2016-07-061-3/+3
| | | | | | | | | | | | | | | | | | | mp_maxid or CPU_FOREACH() as appropriate. This fixes a number of places in the kernel that assumed CPU IDs are dense in [0, mp_ncpus) and would try, for example, to run tasks on CPUs that did not exist or to allocate too few buffers on systems with sparse CPU IDs in which there are holes in the range and mp_maxid > mp_ncpus. Such circumstances generally occur on systems with SMT, but on which SMT is disabled. This patch restores system operation at least on POWER8 systems configured in this way. There are a number of other places in the kernel with potential problems in these situations, but where sparse CPU IDs are not currently known to occur, mostly in the ARM machine-dependent code. These will be fixed in a follow-up commit after the stable/11 branch. PR: kern/210106 Reviewed by: jhb Approved by: re (glebius)
* atomic: Add testandclear on i386/amd64sephe2016-05-161-0/+26
| | | | | | Reviewed by: kib Sponsored by: Microsoft OSTC Differential Revision: https://reviews.freebsd.org/D6381
* Add a new bus method to fetch device-specific CPU sets.jhb2016-05-091-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bus_get_cpus() returns a specified set of CPUs for a device. It accepts an enum for the second parameter that indicates the type of cpuset to request. Currently two valus are supported: - LOCAL_CPUS (on x86 this returns all the CPUs in the package closest to the device when DEVICE_NUMA is enabled) - INTR_CPUS (like LOCAL_CPUS but only returns 1 SMT thread for each core) For systems that do not support NUMA (or if it is not enabled in the kernel config), LOCAL_CPUS fails with EINVAL. INTR_CPUS is mapped to 'all_cpus' by default. The idea is that INTR_CPUS should always return a valid set. Device drivers which want to use per-CPU interrupts should start using INTR_CPUS instead of simply assigning interrupts to all available CPUs. In the future we may wish to add tunables to control the policy of INTR_CPUS (e.g. should it be local-only or global, should it ignore SMT threads or not). The x86 nexus driver exposes the internal set of interrupt CPUs from the the x86 interrupt code via INTR_CPUS. The ACPI bus driver and PCI bridge drivers use _PXM to return a suitable LOCAL_CPUS set when _PXM exists and DEVICE_NUMA is enabled. They also and the global INTR_CPUS set from the nexus driver with the per-domain set from _PXM to generate a local INTR_CPUS set for child devices. Compared to the r298933, this version uses 'struct _cpuset' in <sys/bus.h> instead of 'cpuset_t' to avoid requiring <sys/param.h> (<sys/_cpuset.h> still requires <sys/param.h> for MAXCPU even though <sys/_bitset.h> does not after recent changes).
* xen/i386: enable the platform hypercall for i386royger2016-05-031-2/+2
| | | | | | | Not sure why the platform hypercall was disabled on i386, just enable it in order to fix compilation of the PV timer on i386. Sponsored by: Citrix Systems R&D
* Type of the interrupt handlers on x86 cannot be expressed in C.kib2016-03-292-2/+1
| | | | | | | | Simplify and unify placeholder type definitions. Reviewed by: jhb Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D5771
* Move amd64 metadata.h to x86 and share with i386emaste2016-01-071-34/+3
| | | | MFC after: 1 week
* Move shared variables from {amd64,i386}/initcpu.c to x86/identcpu.c.jhb2015-12-231-21/+2
| | | | | | | | While here, move the common bits of <machine/cputypes.h> to <x86/cputypes.h> as well. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D4670
* Merge common parts of i386 and amd64 md_var.h and smp.h intokib2015-12-072-145/+6
| | | | | | | | new headers x86/include x86_var.h and x86_smp.h. Reviewed by: emaste, jhb Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D4358
* For amd64 non-PCID machines, and for i386 machines with support forkib2015-12-032-8/+3
| | | | | | | | | | | | | | | | | | | | | | | | | the PG_G global pte flag, pmap_invalidate_all() fails to flush global TLB entries [*]. This is because TLB shootdown handler for such configs reloads CR3, and on i386 pmap_invalidate_all() does the same for the initiating CPU. Note that current code does not issue total invalidation requests for the kernel_pmap. Rename amd64 function invltlb_globpcid() to invltlb_glob(), it is not specific for PCID for quite some time, and implement the same functionality for i386. Use the function instead of invltlb() in shootdown handlers and in i386 pmap_invalidate_all(), but only for the kernel pmap (which maps pages with the PG_G attribute set), which takes care of PG_G TLB entries on flush. To detect the affected pmap in i386 TLB shootdown handler, pmap should be passed to the smp_masked_invltlb() function, which makes amd64 and i386 TLB shootdown code almost identical. Merge the code under x86/. Noted by: jhb [*] Reviewed by: cem, jhb, pho Tested by: pho Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D4346
* Export various helper variables describing the layout and size ofjhb2015-11-121-6/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | certain kernel structures for use by debuggers. This mostly aids in examining cores from a kernel without debug symbols as a debugger can infer these values if debug symbols are available. One set of variables describes the layout of 'struct linker_file' to walk the list of loaded kernel modules. A second set of variables describes the layout of 'struct proc' and 'struct thread' to walk the list of processes in the kernel and the threads in each process. The 'pcb_size' variable is used to index into the stoppcbs[] array. The 'vm_maxuser_address' is used to distinguish kernel virtual addresses from user addresses. This doesn't have to be perfect, and 'vm_maxuser_address' is a cheap and simple way to differentiate kernel pointers from simple values like TIDs and PIDs. While here, annotate the fields in struct pcb used by kgdb on amd64 and i386 to note that their ABI should be preserved. Annotations for other platforms will be added in the future. Reviewed by: kib MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3773
* The prefix for CLFLUSHOPT is 0x66. It was right on amd64.kib2015-10-301-1/+1
| | | | Sponsored by: The FreeBSD Foundation
* Use movw instead of movl (or plain mov) when moving segment registersjhb2015-10-291-3/+3
| | | | | | | into memory. This is a nop on clang's assembler, but some assemblers complain if the size suffix is incorrect. Submitted by: bde
* Add CLFLUSHOPT instruction wrappers.kib2015-10-231-0/+7
| | | | | Sponsored by: The FreeBSD Foundation MFC after: 1 week
* x86/xen: Consolidate xen-os.h in a single placeroyger2015-10-211-186/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | amd64 and i386 platform code contain very similar xen/xen-os.h The only differences are: - Functions/variables/types which were unused in i386/xen/xen-os.h: * xen_xchg * __xchg_dummy * __xg * __xchg * atomic_t * atomic_inc * rdtscll The functions/variables/types unused in xen-os.h can be dropped and there is no more differences betwen amd64 and i386. The new header is placed in x86/include/xen and each platform will have dummy headers include x86/xen/*.h. This is to be able to include machine/xen/*.h in the PV drivers. Submitted by: Julien Grall <julien.grall@citrix.com> Reviewed by: royger Differential Revision: https://reviews.freebsd.org/D3880 Sponsored by: Citrix Systems R&D
* xen/console: Introduce a new console driver for Xen guestroyger2015-10-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current Xen console driver is crashing very quickly when using it on an ARM guest. This is because the console lock is recursive and it may lead to recursion on the tty lock and/or corrupt the ring pointer. Furthermore, the console lock is not always taken where it should be and has to be released too early because of the way the console has been designed. Over the years, code has been modified to support various new features but the driver has not been reworked. This new driver has been rewritten with the idea of only having a small set of specific function to write either via the shared ring or the hypercall interface. Note that HVM support has been left aside for now because it requires additional features which are not yet supported. A follow-up patch will be sent with HVM guest support. List of items that may be good to have but not mandatory: - Avoid to flush for each character written when using the tty - Support multiple consoles Submitted by: Julien Grall <julien.grall@citrix.com> Reviewed by: royger Differential Revision: https://reviews.freebsd.org/D3698 Sponsored by: Citrix Systems R&D
* Update Xen headers from 4.2 to 4.6royger2015-10-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull the latest headers for Xen which allow us to add support for ARM and use new features in FreeBSD. This is a verbatim copy of the xen/include/public so every headers which don't exits anymore in the Xen repositories have been dropped. Note the interface version hasn't been bumped, it will be done in a follow-up. Although, it requires fix in the code to get it compiled: - sys/xen/xen_intr.h: evtchn_port_t is already defined in the headers so drop it. - {amd64,i386}/include/intr_machdep.h: NR_EVENT_CHANNELS now depends on xen/interface/event_channel.h, so include it. - {amd64,i386}/{amd64,i386}/support.S: It's not neccessary to include machine/intr_machdep.h. This is also fixing build compilation with the new headers. - dev/xen/blkfront/blkfront.c: The typedef for blkif_request_segmenthas been dropped. So directly use struct blkif_request_segment Finally, modify xen/interface/xen-compat.h to throw a preprocessing error if __XEN_INTERFACE_VERSION__ is not set. This is allow us to catch any file where xen/xen-os.h is not correctly included. Submitted by: Julien Grall <julien.grall@citrix.com> Reviewed by: royger Differential Revision: https://reviews.freebsd.org/D3805 Sponsored by: Citrix Systems R&D
* Merge stack(9) implementations for i386 and amd64 under x86/.markj2015-09-111-39/+3
| | | | | | Reviewed by: jhb, kib Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3255
* Remove unused i386 header privatespace.h. For the native kernel, itskib2015-08-071-49/+0
| | | | | | | | | use was removed in r173592 (Nov 2007), yet Xen PV bits continued referencing the privatespace structure, and were removed in r282274 (Apr 2015). Discussed with: jhb Sponsored by: The FreeBSD Foundation
* Remove some more vestiges of the Xen PV domu support. Specifically,jhb2015-08-062-117/+0
| | | | | | | | | use vtophys() directly instead of vtomach() and retire the no-longer-used headers <machine/xenfunc.h> and <machine/xenvar.h>. Reported by: bde (stale bits in <machine/xenfunc.h>) Reviewed by: royger (earlier version) Differential Revision: https://reviews.freebsd.org/D3266
* Rationalize BSD license on sys/*/include/in_cksum.hemaste2015-08-051-1/+1
| | | | | | | Remove the advertising clause from the Regents of the University of California's license, per the letter dated July 22, 1999. Update clause numbering.
* Add two new pmap functions:jah2015-08-041-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
* Give large kernel stack to the initial thread . Otherwise, ZFSkib2015-08-041-0/+5
| | | | | | | | overflows the stack during root mount in some configurations. Tested by: Fabian Keil <freebsd-listen@fabiankeil.de> (previous version) Sponsored by: The FreeBSD Foundation MFC after: 2 weeks
* Clear the IA32_MISC_ENABLE MSR bit, which limits the max CPUIDkib2015-08-031-0/+1
| | | | | | | | | | | | | | reported, on APs. We already did this on BSP. Otherwise, the userspace software which depends on the features reported by the high CPUID levels is misbehaving. In particular, AVX detection is non-functional, depending on which CPU thread happens to execute when doing CPUID. Another victim is the libthr signal handlers interposer, which needs to save full FPU extended state. Reported and tested by: Andre Meiser <ortadur@web.de> Sponsored by: The FreeBSD Foundation MFC after: 2 weeks
* Improve comments.kib2015-07-301-3/+3
| | | | | Submitted by: bde MFC after: 2 weeks
* Use private cache line for the locked nop in *mb() on i386.kib2015-07-301-32/+39
| | | | | | | | Suggested by: alc Reviewed by: alc, bde Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 2 weeks
* MFamd64 r285934: Remove store/load (= full) barrier from the i386kib2015-07-291-17/+7
| | | | | | | | | | atomic_load_acq_*(). Noted by: alc (long time ago) Reviewed by: alc, bde Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 2 weeks
* Remove duplicate and useless declarations.kib2015-07-221-1/+0
| | | | Submitted by: bde
* Duplicate the copyright from the i386/i386/machdep.c intokib2015-07-101-6/+36
| | | | | | | | | | | i386/include/frame.h after a code was moved from machdep.c to frame.h in r284925. Use include guards style similar to other guards. Noted by: bde Sponsored by: The FreeBSD Foundation MFC after: 1 week
* Add the atomic_thread_fence() family of functions with intent tokib2015-07-081-0/+32
| | | | | | | | | | | | | | | | | | | | | | | provide a semantic defined by the C11 fences with corresponding memory_order. atomic_thread_fence_acq() gives r | r, w, where r and w are read and write accesses, and | denotes the fence itself. atomic_thread_fence_rel() is r, w | w. atomic_thread_fence_acq_rel() is the combination of the acquire and release in single operation. Note that reads after the acq+rel fence could be made visible before writes preceeding the fence. atomic_thread_fence_seq_cst() orders all accesses before/after the fence, and the fence itself is globally ordered against other sequentially consistent atomic operations. Reviewed by: alc Discussed with: bde Sponsored by: The FreeBSD Foundation MFC after: 3 weeks
* Use single instance of the identical INKERNEL() and PMC_IN_KERNEL()kib2015-07-023-4/+4
| | | | | | | | | | | | | macros on amd64 and i386. Move the definition to machine/param.h. kgdb defines INKERNEL() too, the conflict is resolved by renaming kgdb version to PINKERNEL(). On i386, correct the lowest kernel address. After the shared page was introduced, USRSTACK no longer points to the last user address + 1 [*] Submitted by: Oliver Pinter [*] Sponsored by: The FreeBSD Foundation MFC after: 1 week
* Provide npx_get_fsave(9) and npx_set_fsave(9) functions to obtain andkib2015-06-291-0/+5
| | | | | | | | | | | | | | | | | | | restore the FPU state from the format of machine FSAVE area. The intended use is for ABI emulators to provide FSAVE-formatted FPU state to usermode requiring it, while kernel could use FXSAVE due to XMM/XSAVE. The core functionality to convert from/to FXSAVE format is shared with the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions to npx.c and rename them to npx_fill_fpregs_xmm() and npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since our mcontext contains padding to be zeroed or ignored. fill_fpregs() and set_fpregs() could be converted to use the new interface, but there are small differences to handle. Sponsored by: The FreeBSD Foundation MFC after: 1 week
OpenPOWER on IntegriCloud