summaryrefslogtreecommitdiffstats
path: root/sys/arm64
Commit message (Collapse)AuthorAgeFilesLines
* MFC r307334, r318252: Support write-through caches on arm64andrew2017-05-303-5/+17
| | | | | | | | | | | | r307334: Create macros for the MAIR memory attributes. While here add an uncached memory type, however the VM code still needs to be taught about this. r318252: Add the VM_MEMATTR_WRITE_THROUGH memory type to arm64 and use it to support VM_MEMATTR_WRITE_COMBINING in the kernel. This fixes a bug where Xorg would use write back cached memory for its graphics buffers. This would produce artifacts on the screen as cachelines were written to memory.
* MFC r317824:andrew2017-05-301-1/+122
| | | | | | | | Add reclaim_pv_chunk on arm64. This is based on the amd64 code so should operate similarly, other than not needing the delayed invalidation. It has been tested with artificial injection of vm_page_alloc failures while running 'sort /dev/zero'.
* MFC r317361:andrew2017-05-301-1/+11
| | | | | | Call the PSCI reset from cpu_reset on arm64. When rebooting from DDB the kernel calls this directly so the event handler is not called, meaning the computer fails to reboot.
* MFC r317197:andrew2017-05-301-2/+7
| | | | | | Restrict the arm64 supervisor all instructions to only allow a zero immediate value for system calls. We may wish to use other values in the future for other purposes.
* MFC r317192:andrew2017-05-303-24/+20
| | | | | | Push loading curthread into assembly in the synchronous exception handlers. This will help investigating the performance impact of moving parts of the switch statement in do_el0_sync into assembly.
* MFC r313396:andrew2017-05-304-5/+9
| | | | | Push reading of ESR_EL1 to assembly. Among other uses this will allow us to expose this to signal handlers, e.g. for the clang sanitizers.
* MFC r316734, r316761: Support Execute-Never bit in the arm64 pmap.andrew2017-05-302-13/+41
| | | | | | | | | | | | | | | | | | | | | r316734: Start to use the User and Privileged execute-never bits in the arm64 pagetables. This sets both bits when entering an address we know shouldn't be executed. I expect we could mark all userspace pages as Privileged execute-never to ensure the kernel doesn't branch to one of these addresses. While here add the ARMv8.1 upper attributes. r316761: Set the arm64 Execute-never bits in more places. We need to set the Execute-never bits when mapping device memory as the hardware may perform speculative instruction fetches. Set the Privileged Execute-ever bit on userspace memory to stop the kernel if it is tricked into executing it.
* MFC r316732, r316756: Enable Privileged Access Never on arm64.andrew2017-05-307-13/+82
| | | | | | | | | | | | | | | r316732: Use the unprivileged variant of the load and store instructions most places possible in the kernel. This forces these functions to fail if userspace is unable to access a given memory location, even if it is in the user memory range. This will simplify adding Privileged Access Never support later. r316756: In ARMv8.1 ARM has added a process state bit to disable access to userspace from the kernel. Make use of this to restrict accessing userspace to just the functions that explicitly handle crossing the user kernel boundary.
* MFC r316755:andrew2017-05-302-5/+10
| | | | | Add SCTLR bits added in ARMv8.1 and ARMv8.2 and start to use them in the early boot code.
* MFC r318353:hselasky2017-05-271-8/+21
| | | | | | | | | | | | | | | | | | | | | Avoid use of contiguous memory allocations in busdma when possible. This patch improves the boundary checks in busdma to allow more cases using the regular page based kernel memory allocator. Especially in the case of having a non-zero boundary in the parent DMA tag. For example AMD64 based platforms set the PCI DMA tag boundary to PCI_DMA_BOUNDARY, 4GB, which before this patch caused contiguous memory allocations to be preferred when allocating more than PAGE_SIZE bytes. Even if the required alignment was less than PAGE_SIZE bytes. This patch also fixes the nsegments check for using kmem_alloc_attr() when the maximum segment size is less than PAGE_SIZE bytes. Updated some comments describing the code in question. Differential Revision: https://reviews.freebsd.org/D10645 Reviewed by: kib, jhb, gallatin, scottl Sponsored by: Mellanox Technologies
* MFC r308474, r308691, r309203, r309365, r309703, r309898, r310720,markj2017-05-231-23/+8
| | | | | r308489, r308706: Add PQ_LAUNDRY and remove PG_CACHED pages.
* MFC efivar(8) (by imp):kib2017-05-201-26/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | List of revisions merged: r307070 r307071 r307072 r307074 r307189 r307224 r307339 r307390 r307391 r309776 r314231 r314232 r314615 r314616 r314617 r314618 r314619 r314620 r314621 r314623 r314890 r314925 r314926 r314927 r314928 r315770 r315771 Discussed with: gjb (re), imp Sponsored by: The FreeBSD Foundation
* MFC r316679:kib2017-04-241-0/+5
| | | | Do not lose dirty bits for removing PROT_WRITE on arm64.
* MFC 313772:andrew2017-04-191-7/+6
| | | | | | | | Load the new sp_el0 with interrupts disabled in fork_trampoline. If an interrupt arrives in fork_trampoline after sp_el0 was written we may then switch to a new thread, enter userland so change this stack pointer, then return to this code with the wrong value. This fixes this case by moving the load of sp_el0 until after interrupts have been disabled.
* MFC r303261,r315059:mmel2017-04-161-2/+9
| | | | | | | | r303261: Add more UEFI/e820 memory types from latest specifications. r315059: Split overbloated machep.c to multiple files and do basic cleanup of these fragments.
* Improvements for the brand detection and prioritization.kib2017-04-061-1/+1
| | | | | | | | | | | | | | | | | | | MFC r315701 (by ed): Set the interpreter path to /nonexistent. MFC r315749: Adjust r314851 to not require every brand to specify interpreter path. MFC r315753: Add a flag BI_BRAND_ONLY_STATIC to specify that the brand only matches static binaries. MFC r315754: Update r315753 with the proper flag name. MFC r316211: A followup to r315749, two more places where brand->interp_path was accessed unconditionally.
* Bring kernel space CloudABI code in sync with HEAD.ed2017-04-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MFC r312353, r312354 and r312355: Sync in the latest CloudABI generated source files. Languages like C++17 and Go provide direct support for slice types: pointer/length pairs. The CloudABI generator now has more complete for this, meaning that for the C binding, pointer/length pairs now use an automatic naming scheme of ${name} and ${name}_len. Apart from this change and some reformatting, the ABI definitions are identical. Binary compatibility is preserved entirely. MFC r315700: Make file descriptor passing work for CloudABI's sendmsg(). Reduce the potential amount of code duplication between cloudabi32 and cloudabi64 by creating a cloudabi_sock_recv() utility function. The cloudabi32 and cloudabi64 modules will then only contain code to convert the iovecs to the native pointer size. In cloudabi_sock_recv(), we can now construct an SCM_RIGHTS cmsghdr in an mbuf and pass that on to kern_sendit(). MFC r315736: Make file descriptor passing for CloudABI's recvmsg() work. Similar to the change for sendmsg(), create a pointer size independent implementation of recvmsg() and let cloudabi32 and cloudabi64 call into it. In case userspace requests one or more file descriptors, call kern_recvit() in such a way that we get the control message headers in an mbuf. Iterate over all of the headers and copy the file descriptors to userspace.
* MFC r315861:ed2017-04-061-1/+0
| | | | | | Stop providing the compat_3_brand. As of r315860, the ELF image activator works fine for CloudABI without it.
* MFC: r306686, r308953, r311462, r311529, r312438, r314611jkim2017-03-301-0/+1
| | | | | | | - Merge ACPICA 20170303. - Remove '-vd' option to make iasl(8) reproducible. Relnotes: yes
* MFC r311169,r311898,r312925,r312973,r312975,r313007,r313040,r313080,mjg2017-03-161-0/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | r313254,r313341 amd64: add atomic_fcmpset == sparc64: add atomic_fcmpset == Implement atomic_fcmpset_* for arm and arm64. == Add atomic_fcmpset_*() inlines for powerpc Summary: atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value from the target memory location into the 'old' pointer in the case of failure. == i386: add atomic_fcmpset == Don't retry a lost reservation in atomic_fcmpset() The desired behavior of atomic_fcmpset_() is to always exit on error. Instead of retrying on lost reservation, leave the retry to the caller, and return == Add atomic_fcmpset_*() inlines for MIPS atomic_fcmpset_*() is analogous to atomic_cmpset(), but saves off the read value from the target memory location into the 'old' pointer. == i386: fixup fcmpset An incorrect output specifier was used which worked with clang by accident, but breaks with the in-tree gcc version. While here plug a whitespace nit. == Implement atomic_fcmpset_*() for RISC-V. == Use 64bit store instruction in atomic_fcmpset_64.
* MFC r313345:kib2017-02-201-14/+6
| | | | | | | Update arm and arm64 counters MD bits. MFC r313394 (by manu): subr_sfbus.c need sys/proc.h for struct thread definition.
* MFC r313194:kib2017-02-111-2/+0
| | | | Define the vm_ooffset_t and vm_pindex_t types as machine-independend.
* MFC 307332,312086: Drop support for using mmap() with /dev/kmem.jhb2017-01-181-5/+4
| | | | | | | | | | | | | | | | | | | 307332: Drop support for using mmap() with /dev/kmem. Using the device pager with /dev/kmem is not stable since KVA mappings are transient, but the device pager caches the PA associated with a given offset forever. Interestingly, mips' implementation of memmap() already refused requests for /dev/kmem. Note that kvm_read/kvm_write do not use mmap, but use read and write on /dev/kmem, so this should not affect libkvm users. 312086: Trim a few comments on platforms that did not implement mmap of /dev/kmem. After r307332, no platforms implement mmap for /dev/kmem, so the lack of it for these platforms is no longer unique.
* MFC r306899, r307059, r307151gonzo2016-11-061-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | r306899: Fix release MSI method for ARM GIC r307059: INTRNG - fix MSI/MSIX release path Use isrc in attached MSI data structure instead of using map's isrc directly. map's isrc is set to NULL on IRQ deactivation which happens prior to pci_release_msi so MSI_RELEASE_MSI receives array of NULLs Reviewed by: mmel Differential Revision: https://reviews.freebsd.org/D8206 r307151: INTRNG: Propagate IRQ activation error to API consumer Keep resource state consistent with INTRNG state - if intr_activate_irq fails - deactivate resource and propagate error to calling function Reviewed by: mmel
* MFC r304459,r305527:mmel2016-11-051-8/+25
| | | | | | | | | | | | | | | | | | r304459: INTRNG: Rework handling with resources. Partially revert r301453. - Read interrupt properties at bus enumeration time and store it into global mapping table. - At bus_activate_resource() time, given mapping entry is resolved and connected to real interrupt source. A copy of mapping entry is attached to given resource. - At bus_setup_intr() time, mapping entry stored in resource is used for delivery of requested interrupt configuration. - For MSI/MSIX interrupts, mapping entry is created within pci_alloc_msi()/pci_alloc_msix() call. - For legacy PCI interrupts, mapping entry must be created within pcib_route_interrupt() by pcib driver itself. r305527: Fix MIPS INTRNG (both FDT and non-FDT) behaviour broken by r304459
* MFC r303818, r303833, r303941, r304478, r304481, r304483, r304484, r304554,ed2016-10-121-3/+2
| | | | | | | | | | | | | | | | | | r304555, r304556, r304557, r304558, r304559, r304561, r304563, r304564, r304565, r304615, r304742, r304743, r304744, r304745, r304748, r304886, r304991, r305928, r305938, r305987, r306185: Bring CloudABI support back in sync with HEAD. - Add support for running 32-bit executables on amd64, armv6 and i386. - As these new architectures require the use of the vDSO, merge back vDSO support for 64-bit executables running on amd64 and arm64 as well. This has the advantage that support for vDSO-less execution can be phased out when 11.0 becomes unsupported, as opposed to 11.x. This change has been tested by running the cloudlibc unit tests on all supported architectures, which seems to work fine.
* MFC r306091:kib2016-09-281-0/+37
| | | | | | Add a way for the architecture to specify the calling ABI for methods in the EFI Runtime Services Table. On amd64, the calling conventions are MS.
* MFC r305942:kib2016-09-251-3/+0
| | | | Consolidate four efi_next_descriptor() definitions.
* MFC 304799:andrew2016-09-211-0/+7
| | | | | | | | | Map coherent memory in a non-coherent dma tag as uncached. This is similar to what the 32-bit arm code does, with the exception that it always assumes the tag is non-coherent. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 305285:andrew2016-09-212-2/+12
| | | | | | | | Add a pc_clock pcpu field and use it to implement cpu_est_clockrate. This will allow drivers that manage the clock frequency to communicate this with the reset of the kernel. Sponsored by: ABT Systems Ltd
* MFC r304892:andrew2016-09-211-1/+3
| | | | | | | | Print both the kernel read and write translation in DDB when asking for a virtual to physical translation. These may be different, e.g. when a page is mapped as read-only. Sponsored by: ABT Systems Ltd
* MFC 305607:andrew2016-09-161-0/+4
| | | | | | | | Trap msr/mrs instructions. These are privileged arm64 instructions and shouldn't normally be used. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 305128:andrew2016-09-161-0/+1
| | | | | | | | Also handle instruction traps. We might hit these when the page we are executing is being promoted to a superpage. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 305546:andrew2016-09-161-1/+1
| | | | | | | | | | When synchronising the instruction and data caches we only need to clean the data cache to the point of unification. This is the point where the two caches are unified to a single unified cache so cleaning past here is just extra unneeded work. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 305545:andrew2016-09-161-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Only call cpu_icache_sync_range when inserting an executable page. If the page is non-executable the contents of the i-cache are unimportant so this call is just adding unneeded overhead when inserting pages. While doing research using gem5 with an O3 pipeline and 1k/32k/1M iTLB/L1 iCache/L2 Bjoern Zeeb (bz@) observed a fairly high rate of calls into arm64_icache_sync_range() from pmap_enter() along with a high number of instruction fetches and iTLB/iCache hits. Limiting the calls to arm64_icache_sync_range() to only executable pages, we observe the iTLB and iCache Hit going down by about 43%. These numbers are quite misleading when looked at alone as at the same time instructions retired were reduced by 19.2% and instruction fetches were reduced by 38.8%. Overall this reduced the runtime of the test program by 22.4%. On Juno hardware, in steady-state, running the same test, using the cycle count to determine runtime, we do see a reduction of up to 28.9% in runtime. While these numbers certainly depend on the program executed, we expect an overall performance improvement. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 304004, 304596, 304598, 304599, 304600, 304604, 304620, 304685, 304687,andrew2016-09-163-57/+1229
| | | | | | | | 304688, 304689, 304746, 304749, 304750, 304806, 305071, 305191: Merge arm64 superpage support, however leave it disabled by default. MFC after: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303904:andrew2016-09-161-2/+0
| | | | | | | | Uncomment the vm.kvm_size and vm.kvm_free sysctls. These work as expected so there is no reason to leave them commented out. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303903:andrew2016-09-161-0/+14
| | | | | | | | Implement pmap_align_superpage on arm64 based on the amd64 implementation. This will be needed when superpage support is added. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303744:andrew2016-09-161-43/+17
| | | | | | | | | Remove the pvh_global_lock lock from the arm64 pmap. It is unneeded on arm64 as invalidation will have completed before the pmap_invalidate_* functions have complete. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 305605:andrew2016-09-161-3/+3
| | | | | | | | Don't panic when we don't handle a userland exception, not all we may see are currently handled. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC r304285:kib2016-09-163-16/+2
| | | | Implement userspace gettimeofday(2) with HPET timecounter.
* MFC 305771, 305772:andrew2016-09-133-1/+5
| | | | | Fix the arm64 kernel build when DDB is disabled, debug_monitor.c depends on DDB, and is unused when it's disabled.
* MFC 305767:andrew2016-09-131-0/+22
| | | | | | Add a warning about a known erratum we have observed on ThunderX pass 1.1. As this is evaluation hardware with only a few users, and there is a lack of information add a warning when booting on this hardware.
* MFC 303299, 303475:andrew2016-09-071-0/+2
| | | | | | | | - Enable the generic OHCI driver on arm64 - Add a generic EHCI USB driver based on the Allwinner A10 driver Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303592:andrew2016-09-071-9/+15
| | | | | | | Extract the common parts of pmap_kenter_device to a new function Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303585, 303587:andrew2016-09-071-8/+7
| | | | | | | | - Relax the barriers around a TLB invalidation - Fix the comment above pmap_invalidate_page Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303594, 303610:andrew2016-09-071-0/+47
| | | | | | | Add more AArch64 special register macros Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 304625:andrew2016-09-071-2/+2
| | | | | | | Fix the arm64 non-SMP build. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303307, 303308, 303309 clean up the arm64 MP code:andrew2016-09-071-12/+9
| | | | | | | | | - Rework how we number CPUs on arm64 to try and keep clusters together. - Fix a typo in a string in a KASSERT to sanity check the CPU IDs. - Remove an unused variable. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
* MFC 303661:andrew2016-09-071-13/+13
| | | | | | | Remove trailing whitespace from the arm64 pmap Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
OpenPOWER on IntegriCloud