| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
| |
r307334:
Create macros for the MAIR memory attributes. While here add an uncached
memory type, however the VM code still needs to be taught about this.
r318252:
Add the VM_MEMATTR_WRITE_THROUGH memory type to arm64 and use it to support
VM_MEMATTR_WRITE_COMBINING in the kernel. This fixes a bug where Xorg would
use write back cached memory for its graphics buffers. This would produce
artifacts on the screen as cachelines were written to memory.
|
|
|
|
|
|
|
|
| |
Add reclaim_pv_chunk on arm64. This is based on the amd64 code so should
operate similarly, other than not needing the delayed invalidation.
It has been tested with artificial injection of vm_page_alloc failures
while running 'sort /dev/zero'.
|
|
|
|
|
|
| |
Call the PSCI reset from cpu_reset on arm64. When rebooting from DDB the
kernel calls this directly so the event handler is not called, meaning
the computer fails to reboot.
|
|
|
|
|
|
| |
Restrict the arm64 supervisor all instructions to only allow a zero
immediate value for system calls. We may wish to use other values in the
future for other purposes.
|
|
|
|
|
|
| |
Push loading curthread into assembly in the synchronous exception handlers.
This will help investigating the performance impact of moving parts of the
switch statement in do_el0_sync into assembly.
|
|
|
|
|
| |
Push reading of ESR_EL1 to assembly. Among other uses this will allow us
to expose this to signal handlers, e.g. for the clang sanitizers.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r316734:
Start to use the User and Privileged execute-never bits in the arm64
pagetables. This sets both bits when entering an address we know shouldn't
be executed.
I expect we could mark all userspace pages as Privileged execute-never to
ensure the kernel doesn't branch to one of these addresses.
While here add the ARMv8.1 upper attributes.
r316761:
Set the arm64 Execute-never bits in more places.
We need to set the Execute-never bits when mapping device memory as the
hardware may perform speculative instruction fetches.
Set the Privileged Execute-ever bit on userspace memory to stop the kernel
if it is tricked into executing it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r316732:
Use the unprivileged variant of the load and store instructions most
places possible in the kernel. This forces these functions to fail if
userspace is unable to access a given memory location, even if it is in
the user memory range.
This will simplify adding Privileged Access Never support later.
r316756:
In ARMv8.1 ARM has added a process state bit to disable access to userspace
from the kernel. Make use of this to restrict accessing userspace to just
the functions that explicitly handle crossing the user kernel boundary.
|
|
|
|
|
| |
Add SCTLR bits added in ARMv8.1 and ARMv8.2 and start to use them in the
early boot code.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Avoid use of contiguous memory allocations in busdma when possible.
This patch improves the boundary checks in busdma to allow more cases
using the regular page based kernel memory allocator. Especially in
the case of having a non-zero boundary in the parent DMA tag. For
example AMD64 based platforms set the PCI DMA tag boundary to
PCI_DMA_BOUNDARY, 4GB, which before this patch caused contiguous
memory allocations to be preferred when allocating more than PAGE_SIZE
bytes. Even if the required alignment was less than PAGE_SIZE bytes.
This patch also fixes the nsegments check for using kmem_alloc_attr()
when the maximum segment size is less than PAGE_SIZE bytes.
Updated some comments describing the code in question.
Differential Revision: https://reviews.freebsd.org/D10645
Reviewed by: kib, jhb, gallatin, scottl
Sponsored by: Mellanox Technologies
|
|
|
|
|
| |
r308489, r308706:
Add PQ_LAUNDRY and remove PG_CACHED pages.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
List of revisions merged:
r307070
r307071
r307072
r307074
r307189
r307224
r307339
r307390
r307391
r309776
r314231
r314232
r314615
r314616
r314617
r314618
r314619
r314620
r314621
r314623
r314890
r314925
r314926
r314927
r314928
r315770
r315771
Discussed with: gjb (re), imp
Sponsored by: The FreeBSD Foundation
|
|
|
|
| |
Do not lose dirty bits for removing PROT_WRITE on arm64.
|
|
|
|
|
|
|
|
| |
Load the new sp_el0 with interrupts disabled in fork_trampoline. If an
interrupt arrives in fork_trampoline after sp_el0 was written we may then
switch to a new thread, enter userland so change this stack pointer, then
return to this code with the wrong value. This fixes this case by moving
the load of sp_el0 until after interrupts have been disabled.
|
|
|
|
|
|
|
|
| |
r303261:
Add more UEFI/e820 memory types from latest specifications.
r315059:
Split overbloated machep.c to multiple files and do basic cleanup of these
fragments.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
MFC r315701 (by ed):
Set the interpreter path to /nonexistent.
MFC r315749:
Adjust r314851 to not require every brand to specify interpreter path.
MFC r315753:
Add a flag BI_BRAND_ONLY_STATIC to specify that the brand only
matches static binaries.
MFC r315754:
Update r315753 with the proper flag name.
MFC r316211:
A followup to r315749, two more places where brand->interp_path was
accessed unconditionally.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
MFC r312353, r312354 and r312355:
Sync in the latest CloudABI generated source files.
Languages like C++17 and Go provide direct support for slice types:
pointer/length pairs. The CloudABI generator now has more complete for
this, meaning that for the C binding, pointer/length pairs now use an
automatic naming scheme of ${name} and ${name}_len.
Apart from this change and some reformatting, the ABI definitions are
identical. Binary compatibility is preserved entirely.
MFC r315700:
Make file descriptor passing work for CloudABI's sendmsg().
Reduce the potential amount of code duplication between cloudabi32 and
cloudabi64 by creating a cloudabi_sock_recv() utility function. The
cloudabi32 and cloudabi64 modules will then only contain code to convert
the iovecs to the native pointer size.
In cloudabi_sock_recv(), we can now construct an SCM_RIGHTS cmsghdr in
an mbuf and pass that on to kern_sendit().
MFC r315736:
Make file descriptor passing for CloudABI's recvmsg() work.
Similar to the change for sendmsg(), create a pointer size independent
implementation of recvmsg() and let cloudabi32 and cloudabi64 call into
it. In case userspace requests one or more file descriptors, call
kern_recvit() in such a way that we get the control message headers in
an mbuf. Iterate over all of the headers and copy the file descriptors
to userspace.
|
|
|
|
|
|
| |
Stop providing the compat_3_brand.
As of r315860, the ELF image activator works fine for CloudABI without it.
|
|
|
|
|
|
|
| |
- Merge ACPICA 20170303.
- Remove '-vd' option to make iasl(8) reproducible.
Relnotes: yes
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r313254,r313341
amd64: add atomic_fcmpset
==
sparc64: add atomic_fcmpset
==
Implement atomic_fcmpset_* for arm and arm64.
==
Add atomic_fcmpset_*() inlines for powerpc
Summary:
atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value
from the target memory location into the 'old' pointer in the case of failure.
==
i386: add atomic_fcmpset
==
Don't retry a lost reservation in atomic_fcmpset()
The desired behavior of atomic_fcmpset_() is to always exit on error. Instead
of retrying on lost reservation, leave the retry to the caller, and return
==
Add atomic_fcmpset_*() inlines for MIPS
atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the
read value from the target memory location into the 'old' pointer.
==
i386: fixup fcmpset
An incorrect output specifier was used which worked with clang by accident,
but breaks with the in-tree gcc version.
While here plug a whitespace nit.
==
Implement atomic_fcmpset_*() for RISC-V.
==
Use 64bit store instruction in atomic_fcmpset_64.
|
|
|
|
|
|
|
| |
Update arm and arm64 counters MD bits.
MFC r313394 (by manu):
subr_sfbus.c need sys/proc.h for struct thread definition.
|
|
|
|
| |
Define the vm_ooffset_t and vm_pindex_t types as machine-independend.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
307332:
Drop support for using mmap() with /dev/kmem.
Using the device pager with /dev/kmem is not stable since KVA mappings
are transient, but the device pager caches the PA associated with a
given offset forever. Interestingly, mips' implementation of
memmap() already refused requests for /dev/kmem.
Note that kvm_read/kvm_write do not use mmap, but use read and write on
/dev/kmem, so this should not affect libkvm users.
312086:
Trim a few comments on platforms that did not implement mmap of /dev/kmem.
After r307332, no platforms implement mmap for /dev/kmem, so the lack of
it for these platforms is no longer unique.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r306899:
Fix release MSI method for ARM GIC
r307059:
INTRNG - fix MSI/MSIX release path
Use isrc in attached MSI data structure instead of using map's
isrc directly. map's isrc is set to NULL on IRQ deactivation
which happens prior to pci_release_msi so MSI_RELEASE_MSI
receives array of NULLs
Reviewed by: mmel
Differential Revision: https://reviews.freebsd.org/D8206
r307151:
INTRNG: Propagate IRQ activation error to API consumer
Keep resource state consistent with INTRNG state - if intr_activate_irq
fails - deactivate resource and propagate error to calling function
Reviewed by: mmel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r304459:
INTRNG: Rework handling with resources. Partially revert r301453.
- Read interrupt properties at bus enumeration time and store
it into global mapping table.
- At bus_activate_resource() time, given mapping entry is resolved and
connected to real interrupt source. A copy of mapping entry is attached
to given resource.
- At bus_setup_intr() time, mapping entry stored in resource is used
for delivery of requested interrupt configuration.
- For MSI/MSIX interrupts, mapping entry is created within
pci_alloc_msi()/pci_alloc_msix() call.
- For legacy PCI interrupts, mapping entry must be created within
pcib_route_interrupt() by pcib driver itself.
r305527:
Fix MIPS INTRNG (both FDT and non-FDT) behaviour broken by r304459
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r304555, r304556, r304557, r304558, r304559, r304561, r304563, r304564,
r304565, r304615, r304742, r304743, r304744, r304745, r304748, r304886,
r304991, r305928, r305938, r305987, r306185:
Bring CloudABI support back in sync with HEAD.
- Add support for running 32-bit executables on amd64, armv6 and i386.
- As these new architectures require the use of the vDSO, merge back
vDSO support for 64-bit executables running on amd64 and arm64 as
well. This has the advantage that support for vDSO-less execution
can be phased out when 11.0 becomes unsupported, as opposed to 11.x.
This change has been tested by running the cloudlibc unit tests on all
supported architectures, which seems to work fine.
|
|
|
|
|
|
| |
Add a way for the architecture to specify the calling ABI for methods
in the EFI Runtime Services Table. On amd64, the calling conventions
are MS.
|
|
|
|
| |
Consolidate four efi_next_descriptor() definitions.
|
|
|
|
|
|
|
|
|
| |
Map coherent memory in a non-coherent dma tag as uncached. This is similar
to what the 32-bit arm code does, with the exception that it always assumes
the tag is non-coherent.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
Add a pc_clock pcpu field and use it to implement cpu_est_clockrate. This
will allow drivers that manage the clock frequency to communicate this with
the reset of the kernel.
Sponsored by: ABT Systems Ltd
|
|
|
|
|
|
|
|
| |
Print both the kernel read and write translation in DDB when asking for
a virtual to physical translation. These may be different, e.g. when a
page is mapped as read-only.
Sponsored by: ABT Systems Ltd
|
|
|
|
|
|
|
|
| |
Trap msr/mrs instructions. These are privileged arm64 instructions and
shouldn't normally be used.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
Also handle instruction traps. We might hit these when the page we are
executing is being promoted to a superpage.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
|
|
| |
When synchronising the instruction and data caches we only need to clean
the data cache to the point of unification. This is the point where the
two caches are unified to a single unified cache so cleaning past here
is just extra unneeded work.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only call cpu_icache_sync_range when inserting an executable page. If the
page is non-executable the contents of the i-cache are unimportant so this
call is just adding unneeded overhead when inserting pages.
While doing research using gem5 with an O3 pipeline and 1k/32k/1M iTLB/L1
iCache/L2 Bjoern Zeeb (bz@) observed a fairly high rate of calls into
arm64_icache_sync_range() from pmap_enter() along with a high number of
instruction fetches and iTLB/iCache hits.
Limiting the calls to arm64_icache_sync_range() to only executable pages,
we observe the iTLB and iCache Hit going down by about 43%. These numbers
are quite misleading when looked at alone as at the same time instructions
retired were reduced by 19.2% and instruction fetches were reduced by 38.8%.
Overall this reduced the runtime of the test program by 22.4%.
On Juno hardware, in steady-state, running the same test, using the cycle
count to determine runtime, we do see a reduction of up to 28.9% in runtime.
While these numbers certainly depend on the program executed, we expect an
overall performance improvement.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
304688, 304689, 304746, 304749, 304750, 304806, 305071, 305191:
Merge arm64 superpage support, however leave it disabled by default.
MFC after: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
Uncomment the vm.kvm_size and vm.kvm_free sysctls. These work as expected so
there is no reason to leave them commented out.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
Implement pmap_align_superpage on arm64 based on the amd64 implementation.
This will be needed when superpage support is added.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
|
| |
Remove the pvh_global_lock lock from the arm64 pmap. It is unneeded on arm64
as invalidation will have completed before the pmap_invalidate_* functions
have complete.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
Don't panic when we don't handle a userland exception, not all we may see
are currently handled.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
| |
Implement userspace gettimeofday(2) with HPET timecounter.
|
|
|
|
|
| |
Fix the arm64 kernel build when DDB is disabled, debug_monitor.c depends on
DDB, and is unused when it's disabled.
|
|
|
|
|
|
| |
Add a warning about a known erratum we have observed on ThunderX pass 1.1.
As this is evaluation hardware with only a few users, and there is a lack
of information add a warning when booting on this hardware.
|
|
|
|
|
|
|
|
| |
- Enable the generic OHCI driver on arm64
- Add a generic EHCI USB driver based on the Allwinner A10 driver
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
| |
Extract the common parts of pmap_kenter_device to a new function
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
| |
- Relax the barriers around a TLB invalidation
- Fix the comment above pmap_invalidate_page
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
| |
Add more AArch64 special register macros
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
| |
Fix the arm64 non-SMP build.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
|
| |
- Rework how we number CPUs on arm64 to try and keep clusters together.
- Fix a typo in a string in a KASSERT to sanity check the CPU IDs.
- Remove an unused variable.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
| |
Remove trailing whitespace from the arm64 pmap
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
|