summaryrefslogtreecommitdiffstats
path: root/sys/powerpc/aim/mmu_oea64.c
Commit message (Collapse)AuthorAgeFilesLines
* Only manipulate the PGA_EXECUTABLE flag on managed pages. This is a proxynwhitehorn2012-04-111-14/+10
| | | | | | | | for whether the page is physical. On dense phys mem systems (32-bit), VM_PHYS_TO_PAGE will not return NULL for device memory pages if device memory is above physical memory even if there is no allocated vm_page. Attempting to use the returned page could then cause either memory corruption or a page fault.
* Substantially reduce the scope of the locks held in pmap_enter(), whichnwhitehorn2012-04-061-34/+8
| | | | improves concurrency slightly.
* Reduce the frequency that the PowerPC/AIM pmaps invalidate instructionnwhitehorn2012-04-061-22/+21
| | | | | | | | caches, by invalidating kernel icaches only when needed and not flushing user caches for shared pages. Suggested by: kib MFC after: 2 weeks
* More PMAP performance improvements: skip 256 MB segments entirely if theynwhitehorn2012-03-281-4/+20
| | | | | | are are not mapped during ranged operations and reduce the scope of the tlbie lock only to the actual tlbie instruction instead of the entire sequence. There are a few more optimization possibilities here as well.
* Make sure to call vm_page_dirty() before the pmap lock is released tonwhitehorn2012-03-271-2/+2
| | | | | | prevent a race where another process could conclude the page was clean. Submitted by: alc
* More PMAP concurrency improvements: replace the table lock and (almost) allnwhitehorn2012-03-271-86/+100
| | | | | | | | uses of the page queues mutex with a new rwlock that protects the page table and the PV lists. This reduces system time during a parallel buildworld by 35%. Reviewed by: alc
* Only call vm_page_dirty() on pages that are writable in order not tonwhitehorn2012-03-241-4/+12
| | | | confuse the VM.
* Following suggestions from alc, skip wired mappings in pmap_remove_pages()nwhitehorn2012-03-241-51/+29
| | | | | and remove moea64_attr_*() in favor of direct calls to vm_page_dirty() and friends.
* Remove acquisition of VM page queues lock from pmap_protect(). Any actualnwhitehorn2012-03-181-2/+0
| | | | | | | | | manipulation of the pvo_vlink and pvo_olink entries is already protected by the table lock, so most remaining instances of the acquisition of the page queues lock can likely be replaced with the table lock, or removed if the table lock is already held. Reviewed by: alc
* Implement pmap_remove_pages(). This will be added later to the 32-bit MMUnwhitehorn2012-03-151-0/+18
| | | | | | module. Suggested by: alc
* Improve algorithm for deciding whether to loop through all process pagesnwhitehorn2012-03-151-40/+58
| | | | | | | | or look them up individually in pmap_remove() and apply the same logic in the other ranged operation (pmap_protect). This speeds up make installworld by a factor of 2 on powerpc64. MFC after: 1 week
* Use LIST_FOREACH_SAFE() instead of LIST_FOREACH() in pmap_remove(), sincenwhitehorn2012-03-141-2/+2
| | | | | | the point of this loop is to remove elements. This worked by accident before. MFC after: 2 days
* Fix build for the case of powerpc64 kernel without COMPAT_FREEBSD32.kib2012-01-301-0/+3
| | | | MFC after: 2 months
* Finally, try to enable the nxstacks on amd64 and powerpc64 for both 64bitkib2012-01-301-0/+4
| | | | | | | and 32bit ABIs. Also try to enable nxstacks for PAE/i386 when supported, and some variants of powerpc32. MFC after: 2 months (if ever)
* Eliminate vestiges of page coloring.alc2011-12-151-2/+1
|
* Keep track of PVO entries in each pmap, which allows much fasternwhitehorn2011-12-111-4/+21
| | | | | | | pmap_remove() for large sparse requests. This can prevent pmap_remove() operations on 64-bit process destruction or swapout that would take several hundred times the lifetime of the universe to complete. This behavior is largely indistinguishable from a hang.
* Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomickib2011-09-061-13/+13
| | | | | | | | | | | | | | | | | flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
* - Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flagkib2011-08-091-19/+14
| | | | | | | | | | | | | | to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
* With retirement of cpumask_t and usage of cpuset_t for representing aattilio2011-07-041-6/+2
| | | | | | | | | | | | | | | mask of CPUs, pc_other_cpus and pc_cpumask become highly inefficient. Remove them and replace their usage with custom pc_cpuid magic (as, atm, pc_cpumask can be easilly represented by (1 << pc_cpuid) and pc_other_cpus by (all_cpus & ~(1 << pc_cpuid))). This change is not targeted for MFC because of struct pcpu members removal and dependency by cpumask_t retirement. MD review by: marcel, marius, alc Tested by: pluknet MD testing by: marcel, marius, gonzo, andreast
* Fix merge typo.andreast2011-06-231-1/+1
|
* MFCattilio2011-06-041-2/+2
|\
| * Fix a typo derived from a mismerge from mmu_oea that would causenwhitehorn2011-06-041-2/+2
| | | | | | | | | | | | | | | | | | pmap_sync_icache() to sync random (possibly uncached or nonexisting!) memory, causing kernel page faults or machine checks, most easily triggered by using GDB. While here, add an additional safeguard to only sync cacheable memory. MFC after: 2 days
* | MFCattilio2011-06-031-22/+4
|\ \ | |/
| * Remove some dead code: unnecessary isyncs and memory sorting, which arenwhitehorn2011-06-021-22/+4
| | | | | | | | handled in mtmsr() and mem_regions(), respectively.
| * Remove a useless check that served only to make 64-bit PPC systemsnwhitehorn2011-05-161-5/+0
| | | | | | | | | | | | | | unbootable after r221855. Submitted by: andreast MFC after: 1 week
* | MFCattilio2011-05-161-5/+0
| |
* | Add the powerpc support.attilio2011-05-091-3/+10
|/ | | | | | | Note that there is a dirty hack for calling openpic_write(), but nwhitehorn approved it. Discussed with: nwhitehorn
* Adjust debugging string to match the actual function.andreast2011-04-141-1/+1
| | | | Approved by: nwhitehorn (mentor)
* The macro MOEA_PVO_CHECK is empty and not used. It is a left over from theandreast2011-04-141-9/+0
| | | | | | NetBSD import. Remove the definition and all its occurrences. Approved by: nwhitehorn (mentor)
* Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize.pluknet2011-01-211-2/+2
| | | | | | | Submitted by: perryh pluto.rain.com (previous version) Reviewed by: jhb Approved by: kib (mentor) Tested by: universe
* Fix handling of NX pages on capable CPUs. Thanks to kib for prodding menwhitehorn2011-01-131-10/+6
| | | | in the right direction.
* Garbage-collect unused variable.nwhitehorn2010-12-191-4/+2
|
* Add some isync()s related to the 64-bit MMU scratch page to avoid racenwhitehorn2010-12-111-0/+1
| | | | conditions on its invalidation.
* Add an abstraction layer to the 64-bit AIM MMU's page table manipulationnwhitehorn2010-12-041-626/+154
| | | | | | | | | logic to support modifying the page table through a hypervisor. This uses KOBJ inheritance to provide subclasses of the base 64-bit AIM MMU class with additional methods for page table manipulation. Many thanks to Peter Grehan for suggesting this design and implementing the MMU KOBJ inheritance mechanism.
* Remove use of a separate ofw_pmap on 32-bit CPUs. Many Open Firmwarenwhitehorn2010-11-121-9/+0
| | | | | | | | | | mappings need to end up in the kernel anyway since the kernel begins executing in OF context. Separating them adds needless complexity, especially since the powerpc64 and mmu_oea64 code gave up on it a long time ago. As a side effect, the PPC ofw_machdep code is no longer AIM-specific, so move it to powerpc/ofw.
* Remove or conditionalize some hypervisor-unfriendly instruction sequences.nwhitehorn2010-11-121-4/+0
|
* Add some platform KOBJ extensions and continue integrating PowerPCnwhitehorn2010-11-121-0/+3
| | | | | | | | | | hypervisor infrastructure support: - Fix coexistence of multiple platform modules in the same kernel - Allow platform modules to provide an SMP topology - PowerPC hypervisors limit the amount of memory accessible in real mode. Allow the platform modules to specify the maximum real-mode address, and modify the bits of the kernel that need to allocate real-mode-accessible buffers to respect this limits.
* Fix an error in r215067. An existing /chosen/mmu but missing translationsnwhitehorn2010-11-121-5/+5
| | | | | property just means we shouldn't add any translations, not that we should panic.
* Make AIM early-boot code function correctly without Open Firmware.nwhitehorn2010-11-091-6/+2
|
* Implement pmap_is_prefaultable().alc2010-11-011-0/+15
| | | | Reviewed by: nwhitehorn
* Follow exactly the steps in architecture manual for correctly invalidatingnwhitehorn2010-10-041-2/+2
| | | | TLB entries instead of trying to cut corners.
* Fix pmap_page_set_memattr() behavior in the presence of fictitious pagesnwhitehorn2010-10-011-21/+9
| | | | | | by just caching the mode for later use by pmap_enter(), following amd64. While here, correct some mismerges from mmu_oea64 -> mmu_oea and clean up some dead code found while fixing the fictitious page behavior.
* Add support for memory attributes (pmap_mapdev_attr() and friends) onnwhitehorn2010-09-301-15/+73
| | | | | PowerPC/AIM. This is currently stubbed out on Book-E, since I have no idea how to implement it there.
* Split the SLB mirror cache into two kinds of object, one for kernel mapsnwhitehorn2010-09-161-1/+2
| | | | | | | which are similar to the previous ones, and one for user maps, which are arrays of pointers into the SLB tree. This changes makes user SLB updates atomic, closing a window for memory corruption. While here, rearrange the allocation functions to make context switches faster.
* Replace the SLB backing store splay tree used on 64-bit PowerPC AIMnwhitehorn2010-09-161-9/+16
| | | | | | | | hardware with a lockless sparse tree design. This marginally improves the performance of PMAP and allows copyin()/copyout() to run without acquiring locks when used on wired mappings. Submitted by: mdf
* Introduce inheritance into the PowerPC MMU kobj interface.grehan2010-09-151-6/+1
| | | | | | | | | | | | | | | | | include/mmuvar.h - Change the MMU_DEF macro to also create the class definition as well as define the DATA_SET. Add a macro, MMU_DEF_INHERIT, which has an extra parameter specifying the MMU class to inherit methods from. Update the comments at the start of the header file to describe the new macros. booke/pmap.c aim/mmu_oea.c aim/mmu_oea64.c - Collapse mmu_def_t declaration into updated MMU_DEF macro The MMU_DEF_INHERIT macro will be used in the PS3 MMU implementation to allow it to inherit the stock powerpc64 MMU methods. Reviewed by: nwhitehorn
* Reorder statistics tracking and table lock acquisitions already in placenwhitehorn2010-09-091-4/+6
| | | | to avoid race conditions updating the PVO statistics.
* Fix a printf specifier on 64-bit systems.nwhitehorn2010-09-081-1/+1
|
* Fix a typo in the original import of this code from NetBSD that caused thenwhitehorn2010-09-081-1/+3
| | | | | | | | | wrong element of the VSID bitmap array to be examined after a collision, leading to reallocation of in-use VSIDs under some circumstances, with attendant memory corruption. Also add an assert to check for this kind of problem in the future. MFC after: 4 days
* Fix an error made in r209975 related to context ID allocation for 64-bitnwhitehorn2010-09-071-4/+7
| | | | | | | | PowerPC CPUs running a 32-bit kernel. This bug could cause in-use VSIDs to be allocated again to another process, causing memory space overlaps and corruption. Reported by: linimon
OpenPOWER on IntegriCloud