| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
This reverts commit 63302e53ed4b3fe59711d939ba87433a9a12199d.
|
|
|
|
| |
This reverts commit b634270605d0823a7110d2385de63e1e3a74f786.
|
|
|
|
| |
This reverts commit cf2b1f5568d7328668ab821286d6b946275f8d6f.
|
|
|
|
| |
This reverts commit 045793a68906243d204d36b336867187c1a33f00.
|
|
|
|
|
|
|
| |
Make struct syscall_args visible to userspace compilation environment
from machine/proc.h, consistently on all architectures.
(cherry picked from commit 06d5fa0600b92e97e90e41785ef10f641bdec89f)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Split identify_cpu() into two functions for amd64 as we do for i386. This
reduces diff between amd64 and i386. Also, it fixes a regression introduced
in r322076, i.e., identify_hypervisor() failed to identify some hypervisors.
This function assumes cpu_feature2 is already initialized.
Reported by: dexuan
Tested by: dexuan
(cherry picked from commit 173ac9160e83a9396d44938ef9789b6e62e48e70)
|
|
|
|
|
|
|
|
|
|
| |
Refine the fix from r312954. Specifically, add a new PDE-only flag,
PG_PROMOTED, that indicates whether lingering 4KB page mappings might
need to be flushed on a PDE change that restricts or destroys a 2MB
page mapping. This flag allows the pmap to avoid range invalidations
that are both unnecessary and costly.
Approved by: re (kib)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
List of revisions merged:
r307070
r307071
r307072
r307074
r307189
r307224
r307339
r307390
r307391
r309776
r314231
r314232
r314615
r314616
r314617
r314618
r314619
r314620
r314621
r314623
r314890
r314925
r314926
r314927
r314928
r315770
r315771
Discussed with: gjb (re), imp
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
r303261:
Add more UEFI/e820 memory types from latest specifications.
r315059:
Split overbloated machep.c to multiple files and do basic cleanup of these
fragments.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r313254,r313341
amd64: add atomic_fcmpset
==
sparc64: add atomic_fcmpset
==
Implement atomic_fcmpset_* for arm and arm64.
==
Add atomic_fcmpset_*() inlines for powerpc
Summary:
atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value
from the target memory location into the 'old' pointer in the case of failure.
==
i386: add atomic_fcmpset
==
Don't retry a lost reservation in atomic_fcmpset()
The desired behavior of atomic_fcmpset_() is to always exit on error. Instead
of retrying on lost reservation, leave the retry to the caller, and return
==
Add atomic_fcmpset_*() inlines for MIPS
atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the
read value from the target memory location into the 'old' pointer.
==
i386: fixup fcmpset
An incorrect output specifier was used which worked with clang by accident,
but breaks with the in-tree gcc version.
While here plug a whitespace nit.
==
Implement atomic_fcmpset_*() for RISC-V.
==
Use 64bit store instruction in atomic_fcmpset_64.
|
|
|
|
|
| |
For i386, remove config options CPU_DISABLE_CMPXCHG, CPU_DISABLE_SSE
and device npx.
|
|
|
|
|
|
|
|
| |
Implement get_pcpu() for i386 and use it to replace pcpu_find(curcpu)
in the i386 pmap.
The curcpu macro loads the per-cpu data pointer as its first step,
so the remaining steps of pcpu_find(curcpu) are circular.
|
|
|
|
| |
Use SFENCE for ordering CLFLUSHOPT.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move the objects used to create temporary mappings for i386 pmap zero
and copy operations to the MD PCPU region. Change sysmap
initialization to only allocate KVA pages for CPUs that are actually
present. As a minor optimization, this also prevents false sharing
between adjacent sysmap objects since the pcpu struct is already
cacheline-aligned.
While here, move pc_qmap_addr initialization for the BSP into
pmap_bootstrap(), which allows use of pmap_quick* functions during
early boot.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D8833
|
|
|
|
|
|
|
|
| |
The 'cpu' and 'cpu_class' variables were always set to the same value
on amd64 and are legacy holdovers from i386. Remove them entirely on
amd64.
Requested by: kib (MFC)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
vm86 mode.
MFC r304085 (by bde):
Fix the variables $esp, $ds, $es, $fs, $gs and $ss in vm86 mode. Fix
PC_REGS() so that printing of instructions works in some useful cases.
MFC r304962 (by bde):
Expand error messages: print symbol names, parentheses and shift tokens,
and negative shift counts. Fix error messages.
MFC r305612 (by bde):
Fix single-stepping of instructions emulated by vm86.
MFC r305661 (by bde):
Give the full syntax of the 'count' arg for all commmands that support
it. Give the full syntax of the 'addr' arg for these commands and some
others. Rename it from 'address' for the generic command. Fix
description of how 'count' is supposed to work for the 'break'
command.
Don't (mis)describe the syntax of the comma for the 'step' command.
Expand the description for the generic command.
Give the full syntax for the 'examine' command. It was also missing
the possible values for the modifier.
MFC r305663 (by bde):
Fix stopping when the specified breakpoint count is reached.
MFC r305665 (by bde):
Pass the trap type and code down from db_trap() to db_stop_at_pc() so
that the latter can easily determine what the trap type actually is
after callers are fixed to encode the type unambigously.
MFC r305807 (by bde):
Use the MI macro TRAPF_USERMODE() instead of open-coded checks for
SEL_UPL and sometimes PSL_VM. Fix logic errors in treating vm86
bioscall mode as kernel mode. The main place checked all the
necessary flags, but put the necessary parentheses for the PSL_VM and
PCB_VM86CALL checks in the wrong place.
MFC r305811 (by bz):
Try to fix LINT builds after r305807.
MFC r305840 (by bde):
Abort single stepping in ddb if the trap is not for single-stepping.
MFC r305862 (by bde):
Ifdef the new dr6 variable for KDB.
MFC r305864 (by bde):
Statically initialize the run mode to the one that will become current
on first entry. Don't reset to the run mode to STEP_NONE when
stopping, and remove STEP_NONE.
MFC r305865 (by bde):
Fix decoding of tf_rsp on amd64, and move TF_HAS_STACKREGS() to the
i386-only section, and fix a comment about the amd64 kernel trapframe
not having stackregs.
MFC r305897 (by bde):
Silently ignore unexpected single-step traps.
MFC r306311 (by bde):
Determine the operand/address size of %cs in a new function
db_segsize(). Use db_segsize() to set the default operand/address
size for disassembling.
Fix db_print_loc_and_inst() to ask for the normal format and not the
alternate in normal operation. Use db_segsize() to avoid trying to
print a garbage stack trace if %cs is 16 bits.
|
|
|
|
| |
Reduce the cost of TLB invalidation on x86 by using per-CPU completion flags.
|
|
|
|
|
|
|
| |
Move msix_disable_migration under #ifdef SMP since it doesn't make sense
for !SMP kernels.
PR: 212014
|
|
|
|
|
|
| |
xen: automatically disable MSI-X interrupt migration
Approved by: re (kib)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mp_maxid or CPU_FOREACH() as appropriate. This fixes a number of places in
the kernel that assumed CPU IDs are dense in [0, mp_ncpus) and would try,
for example, to run tasks on CPUs that did not exist or to allocate too
few buffers on systems with sparse CPU IDs in which there are holes in the
range and mp_maxid > mp_ncpus. Such circumstances generally occur on
systems with SMT, but on which SMT is disabled. This patch restores system
operation at least on POWER8 systems configured in this way.
There are a number of other places in the kernel with potential problems
in these situations, but where sparse CPU IDs are not currently known
to occur, mostly in the ARM machine-dependent code. These will be fixed
in a follow-up commit after the stable/11 branch.
PR: kern/210106
Reviewed by: jhb
Approved by: re (glebius)
|
|
|
|
|
|
| |
Reviewed by: kib
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6381
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bus_get_cpus() returns a specified set of CPUs for a device. It accepts
an enum for the second parameter that indicates the type of cpuset to
request. Currently two valus are supported:
- LOCAL_CPUS (on x86 this returns all the CPUs in the package closest to
the device when DEVICE_NUMA is enabled)
- INTR_CPUS (like LOCAL_CPUS but only returns 1 SMT thread for each core)
For systems that do not support NUMA (or if it is not enabled in the kernel
config), LOCAL_CPUS fails with EINVAL. INTR_CPUS is mapped to 'all_cpus'
by default. The idea is that INTR_CPUS should always return a valid set.
Device drivers which want to use per-CPU interrupts should start using
INTR_CPUS instead of simply assigning interrupts to all available CPUs.
In the future we may wish to add tunables to control the policy of
INTR_CPUS (e.g. should it be local-only or global, should it ignore
SMT threads or not).
The x86 nexus driver exposes the internal set of interrupt CPUs from the
the x86 interrupt code via INTR_CPUS.
The ACPI bus driver and PCI bridge drivers use _PXM to return a suitable
LOCAL_CPUS set when _PXM exists and DEVICE_NUMA is enabled. They also and
the global INTR_CPUS set from the nexus driver with the per-domain set from
_PXM to generate a local INTR_CPUS set for child devices.
Compared to the r298933, this version uses 'struct _cpuset' in
<sys/bus.h> instead of 'cpuset_t' to avoid requiring <sys/param.h>
(<sys/_cpuset.h> still requires <sys/param.h> for MAXCPU even though
<sys/_bitset.h> does not after recent changes).
|
|
|
|
|
|
|
| |
Not sure why the platform hypercall was disabled on i386, just enable it in
order to fix compilation of the PV timer on i386.
Sponsored by: Citrix Systems R&D
|
|
|
|
|
|
|
|
| |
Simplify and unify placeholder type definitions.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D5771
|
|
|
|
| |
MFC after: 1 week
|
|
|
|
|
|
|
|
| |
While here, move the common bits of <machine/cputypes.h> to
<x86/cputypes.h> as well.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D4670
|
|
|
|
|
|
|
|
| |
new headers x86/include x86_var.h and x86_smp.h.
Reviewed by: emaste, jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4358
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the PG_G global pte flag, pmap_invalidate_all() fails to flush global
TLB entries [*]. This is because TLB shootdown handler for such
configs reloads CR3, and on i386 pmap_invalidate_all() does the same
for the initiating CPU. Note that current code does not issue total
invalidation requests for the kernel_pmap.
Rename amd64 function invltlb_globpcid() to invltlb_glob(), it is not
specific for PCID for quite some time, and implement the same
functionality for i386. Use the function instead of invltlb() in
shootdown handlers and in i386 pmap_invalidate_all(), but only for the
kernel pmap (which maps pages with the PG_G attribute set), which
takes care of PG_G TLB entries on flush.
To detect the affected pmap in i386 TLB shootdown handler, pmap should
be passed to the smp_masked_invltlb() function, which makes amd64 and
i386 TLB shootdown code almost identical. Merge the code under x86/.
Noted by: jhb [*]
Reviewed by: cem, jhb, pho
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4346
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
certain kernel structures for use by debuggers. This mostly aids
in examining cores from a kernel without debug symbols as a debugger
can infer these values if debug symbols are available.
One set of variables describes the layout of 'struct linker_file' to
walk the list of loaded kernel modules.
A second set of variables describes the layout of 'struct proc' and
'struct thread' to walk the list of processes in the kernel and the
threads in each process.
The 'pcb_size' variable is used to index into the stoppcbs[] array.
The 'vm_maxuser_address' is used to distinguish kernel virtual addresses
from user addresses. This doesn't have to be perfect, and
'vm_maxuser_address' is a cheap and simple way to differentiate kernel
pointers from simple values like TIDs and PIDs.
While here, annotate the fields in struct pcb used by kgdb on amd64
and i386 to note that their ABI should be preserved. Annotations for
other platforms will be added in the future.
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D3773
|
|
|
|
| |
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
| |
into memory. This is a nop on clang's assembler, but some assemblers
complain if the size suffix is incorrect.
Submitted by: bde
|
|
|
|
|
| |
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
amd64 and i386 platform code contain very similar xen/xen-os.h
The only differences are:
- Functions/variables/types which were unused in i386/xen/xen-os.h:
* xen_xchg
* __xchg_dummy
* __xg
* __xchg
* atomic_t
* atomic_inc
* rdtscll
The functions/variables/types unused in xen-os.h can be dropped and there
is no more differences betwen amd64 and i386.
The new header is placed in x86/include/xen and each platform will have
dummy headers include x86/xen/*.h. This is to be able to include
machine/xen/*.h in the PV drivers.
Submitted by: Julien Grall <julien.grall@citrix.com>
Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D3880
Sponsored by: Citrix Systems R&D
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The current Xen console driver is crashing very quickly when using it on
an ARM guest. This is because the console lock is recursive and it may
lead to recursion on the tty lock and/or corrupt the ring pointer.
Furthermore, the console lock is not always taken where it should be and has
to be released too early because of the way the console has been designed.
Over the years, code has been modified to support various new features but
the driver has not been reworked.
This new driver has been rewritten with the idea of only having a small set
of specific function to write either via the shared ring or the hypercall
interface.
Note that HVM support has been left aside for now because it requires
additional features which are not yet supported. A follow-up patch will be
sent with HVM guest support.
List of items that may be good to have but not mandatory:
- Avoid to flush for each character written when using the tty
- Support multiple consoles
Submitted by: Julien Grall <julien.grall@citrix.com>
Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D3698
Sponsored by: Citrix Systems R&D
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Pull the latest headers for Xen which allow us to add support for ARM and
use new features in FreeBSD.
This is a verbatim copy of the xen/include/public so every headers which
don't exits anymore in the Xen repositories have been dropped.
Note the interface version hasn't been bumped, it will be done in a
follow-up. Although, it requires fix in the code to get it compiled:
- sys/xen/xen_intr.h: evtchn_port_t is already defined in the headers so
drop it.
- {amd64,i386}/include/intr_machdep.h: NR_EVENT_CHANNELS now depends on
xen/interface/event_channel.h, so include it.
- {amd64,i386}/{amd64,i386}/support.S: It's not neccessary to include
machine/intr_machdep.h. This is also fixing build compilation with the
new headers.
- dev/xen/blkfront/blkfront.c: The typedef for blkif_request_segmenthas
been dropped. So directly use struct blkif_request_segment
Finally, modify xen/interface/xen-compat.h to throw a preprocessing error if
__XEN_INTERFACE_VERSION__ is not set. This is allow us to catch any file
where xen/xen-os.h is not correctly included.
Submitted by: Julien Grall <julien.grall@citrix.com>
Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D3805
Sponsored by: Citrix Systems R&D
|
|
|
|
|
|
| |
Reviewed by: jhb, kib
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3255
|
|
|
|
|
|
|
|
|
| |
use was removed in r173592 (Nov 2007), yet Xen PV bits continued
referencing the privatespace structure, and were removed in r282274
(Apr 2015).
Discussed with: jhb
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
|
| |
use vtophys() directly instead of vtomach() and retire the no-longer-used
headers <machine/xenfunc.h> and <machine/xenvar.h>.
Reported by: bde (stale bits in <machine/xenfunc.h>)
Reviewed by: royger (earlier version)
Differential Revision: https://reviews.freebsd.org/D3266
|
|
|
|
|
|
|
| |
Remove the advertising clause from the Regents of the University of
California's license, per the letter dated July 22, 1999.
Update clause numbering.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)
These will create and destroy a temporary, CPU-local KVA mapping of a specified page.
Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread
Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page
The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.
NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.
Reviewed by: kib
Approved by: kib (mentor)
Differential Revision: http://reviews.freebsd.org/D3013
|
|
|
|
|
|
|
|
| |
overflows the stack during root mount in some configurations.
Tested by: Fabian Keil <freebsd-listen@fabiankeil.de> (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reported, on APs. We already did this on BSP.
Otherwise, the userspace software which depends on the features
reported by the high CPUID levels is misbehaving. In particular, AVX
detection is non-functional, depending on which CPU thread happens to
execute when doing CPUID. Another victim is the libthr signal
handlers interposer, which needs to save full FPU extended state.
Reported and tested by: Andre Meiser <ortadur@web.de>
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
|
|
|
|
|
| |
Submitted by: bde
MFC after: 2 weeks
|
|
|
|
|
|
|
|
| |
Suggested by: alc
Reviewed by: alc, bde
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
|
|
|
|
|
|
|
|
|
|
| |
atomic_load_acq_*().
Noted by: alc (long time ago)
Reviewed by: alc, bde
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
|
|
|
|
| |
Submitted by: bde
|
|
|
|
|
|
|
|
|
|
|
| |
i386/include/frame.h after a code was moved from machdep.c to frame.h
in r284925.
Use include guards style similar to other guards.
Noted by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
provide a semantic defined by the C11 fences with corresponding
memory_order.
atomic_thread_fence_acq() gives r | r, w, where r and w are read and
write accesses, and | denotes the fence itself.
atomic_thread_fence_rel() is r, w | w.
atomic_thread_fence_acq_rel() is the combination of the acquire and
release in single operation. Note that reads after the acq+rel fence
could be made visible before writes preceeding the fence.
atomic_thread_fence_seq_cst() orders all accesses before/after the
fence, and the fence itself is globally ordered against other
sequentially consistent atomic operations.
Reviewed by: alc
Discussed with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
macros on amd64 and i386. Move the definition to machine/param.h.
kgdb defines INKERNEL() too, the conflict is resolved by renaming kgdb
version to PINKERNEL().
On i386, correct the lowest kernel address. After the shared page was
introduced, USRSTACK no longer points to the last user address + 1 [*]
Submitted by: Oliver Pinter [*]
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
restore the FPU state from the format of machine FSAVE area. The
intended use is for ABI emulators to provide FSAVE-formatted FPU state
to usermode requiring it, while kernel could use FXSAVE due to
XMM/XSAVE.
The core functionality to convert from/to FXSAVE format is shared with
the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions
to npx.c and rename them to npx_fill_fpregs_xmm() and
npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since
our mcontext contains padding to be zeroed or ignored.
fill_fpregs() and set_fpregs() could be converted to use the new
interface, but there are small differences to handle.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|