summaryrefslogtreecommitdiffstats
path: root/sys/vm
Commit message (Collapse)AuthorAgeFilesLines
* Fix a leak of the wired pages when unwiring of the PROT_NONE-mappedkib2014-09-016-109/+159
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | wired region. Rework the handling of unwire to do the it in batch, both at pmap and object level. All commits below are by alc. MFC r268327: Introduce pmap_unwire(). MFC r268591: Implement pmap_unwire() for powerpc. MFC r268776: Implement pmap_unwire() for arm. MFC r268806: pmap_unwire(9) man page. MFC r269134: When unwiring a region of an address space, do not assume that the underlying physical pages are mapped by the pmap. This fixes a leak of the wired pages on the unwiring of the region mapped with no access allowed. MFC r269339: In the implementation of the new function pmap_unwire(), the call to MOEA64_PVO_TO_PTE() must be performed before any changes are made to the PVO. Otherwise, MOEA64_PVO_TO_PTE() will panic. MFC r269365: Correct a long-standing problem in moea{,64}_pvo_enter() that was revealed by the combination of r268591 and r269134: When we attempt to add the wired attribute to an existing mapping, moea{,64}_pvo_enter() do nothing. (They only set the wired attribute on newly created mappings.) MFC r269433: Handle wiring failures in vm_map_wire() with the new functions pmap_unwire() and vm_object_unwire(). Retire vm_fault_{un,}wire(), since they are no longer used. MFC r269438: Rewrite a loop in vm_map_wire() so that gcc doesn't think that the variable "rv" is uninitialized. MFC r269485: Retire pmap_change_wiring(). Reviewed by: alc
* MFC r270011:kib2014-08-251-4/+51
| | | | | | | | | Implement 'fast path' for the vm page fault handler. MFC r270387 (by alc): Relax one of the conditions for mapping a page on the fast path. Approved by: re (gjb)
* MFC r261647 (by alc):kib2014-08-251-1/+4
| | | | Don't call vm_fault_prefault() on zero-fill faults.
* MFC r261412 (by alc):kib2014-08-251-24/+32
| | | | Make prefaulting more aggressive on hard faults.
* MFC r269978 (by alc):kib2014-08-251-2/+3
| | | | Avoid pointless (but harmless) actions on unmanaged pages.
* MFC r269746:kib2014-08-242-11/+27
| | | | | Adapt vm_page_aflag_set(PGA_WRITEABLE) to the locking of pmap_enter(PMAP_ENTER_NOSLEEP).
* Merge the changes to pmap_enter(9) for sleep-less operation (requestedkib2014-08-243-9/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | by flag). The ia64 pmap.c changes are direct commit, since ia64 is removed on head. MFC r269368 (by alc): Retire PVO_EXECUTABLE. MFC r269728: Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). MFC r269759 (by alc): Update the text of a KASSERT() to reflect the changes in r269728. MFC r269822 (by alc): Change {_,}pmap_allocpte() so that they look for the flag PMAP_ENTER_NOSLEEP instead of M_NOWAIT/M_WAITOK when deciding whether to sleep on page table page allocation. MFC r270151 (by alc): Replace KASSERT that no PV list locks are held with a conditional unlock. Reviewed by: alc Approved by: re (gjb) Sponsored by: The FreeBSD Foundation
* MFC r269907:kib2014-08-201-0/+1
| | | | Fix leaks of unqueued unwired pages.
* MFC r269643:kib2014-08-131-1/+1
| | | | | Weaken the requirement for the vm object lock by only asserting locked object in vm_pager_page_unswapped(), instead of locked exclusively.
* MFC r269642:kib2014-08-131-0/+4
| | | | Add wrappers to assert that vm object is unlocked and for try upgrade.
* MFC r268615:kib2014-07-282-5/+8
| | | | | | | | | | | Add OBJ_TMPFS_NODE flag. MFC r268616: Set the OBJ_TMPFS_NODE flag for vm_object of VREG tmpfs node. MFC r269053: Correct assertion. tmpfs vm object is always at the bottom of the shadow chain.
* MFC r267213 (by alc):kib2014-07-244-14/+62
| | | | | | Add a page size field to struct vm_page. Approved by: alc
* MFC r267664:kib2014-06-271-0/+9
| | | | | | Assert that the new entry is inserted into the right location in the map entries list, and that it does not overlap with the previous and next entries.
* MFC r267630:kib2014-06-263-1/+7
| | | | Add MAP_EXCL flag for mmap(2).
* MFC r267766:kib2014-06-261-1/+1
| | | | Use correct names for the flags.
* MFC r267254:kib2014-06-232-40/+68
| | | | | | | Make mmap(MAP_STACK) search for the available address space. MFC r267497 (by alc): Use local variable instead of sgrowsiz.
* MFC r267391:mav2014-06-221-1/+2
| | | | | Introduce new "256 Bucket" zone to split requests and reduce congestion on "128 Bucket" zone lock.
* MFC r267387:mav2014-06-221-0/+2
| | | | | | | | | Allocating new bucket for bucket zone, never take it from the zone itself, since it will almost certanly fail. Take next bigger zone instead. This situation should not happen with original bucket zones configuration: "32 Bucket" zone uses "64 Bucket" and vice versa. But if "64 Bucket" zone lock is congested, zone may grow its bucket size and start biting itself.
* MFC r266780:kib2014-06-041-4/+1
| | | | Remove the assert which can be triggered by the userspace.
* MFC r266491:kib2014-05-241-12/+10
| | | | Remove redundand loop.
* MFC r259107alc2014-05-233-10/+7
| | | | | | Eliminate a redundant parameter to vm_radix_replace(). Improve the wording of the comment describing vm_radix_replace().
* MFC r265886, r265948alc2014-05-231-7/+14
| | | | | | | | | | | | | | | | | | With the new-and-improved vm_fault_copy_entry() (r265843), we can always avoid soft page faults when adding write access to user wired entries in vm_map_protect(). Previously, we only avoided the soft page fault when the underlying pages were copy-on-write. In other words, we avoided the pages faults that might sleep on page allocation, but not the trivial page faults to update the physical map. On a fork allow read-only wired pages to be copy-on-write shared between the parent and child processes. Previously, we copied these pages even though they are read only. However, the reason for copying them is historical and no longer exists. In recent times, vm_map_protect() has developed the ability to copy pages when write access is added to wired copy-on-write pages. So, in this case, copy-on-write sharing of wired pages is not to be feared. It is not going to lead to copy-on-write faults on wired memory.
* MFC r266464:kib2014-05-231-1/+3
| | | | | In execve(2), postpone the free of old vmspace until the threads are resumed and exited.
* MFC r265534:pho2014-05-211-1/+1
| | | | | | | | msync(2) must return ENOMEM and not EINVAL when the address is outside the allowed range or when one or more pages are not mapped. This according to The Open Group Base Specifications Issue 7. Sponsored by: EMC / Isilon storage division
* MFC r265850alc2014-05-171-1/+2
| | | | | | About 9% of the pmap_protect() calls being performed by vm_map_copy_entry() are unnecessary. Eliminate the unnecessary calls.
* MFC r265843:kib2014-05-171-40/+68
| | | | | | | | For the upgrade case in vm_fault_copy_entry(), when the entry does not need COW and is writeable, do not create a new backing object for the entry. MFC r265887: Fix locking.
* MFC r265825:kib2014-05-171-1/+1
| | | | | When printing the map with the ddb 'show procvm' command, do not dump page queues for the backing objects.
* MFC r265824:kib2014-05-171-1/+2
| | | | Print the entry address in addition to the object.
* MFC r265418alc2014-05-131-3/+5
| | | | | | | | | | | | | | | | | | | | Prior to r254304, a separate function, vm_pageout_page_stats(), was used to periodically update the reference status of the active pages. This function was called, instead of vm_pageout_scan(), when memory was not scarce. The objective was to provide up to date reference status for active pages in case memory did become scarce and active pages needed to be deactivated. The active page queue scan performed by vm_pageout_page_stats() was virtually identical to that performed by vm_pageout_scan(), and so r254304 eliminated vm_pageout_page_stats(). Instead, vm_pageout_scan() is called with the parameter "pass" set to zero. The intention was that when pass is zero, vm_pageout_scan() would only scan the active queue. However, the variable page_shortage can still be greater than zero when memory is not scarce and vm_pageout_scan() is called with pass equal to zero. Consequently, the inactive queue may be scanned and dirty pages laundered even though that was not intended by r254304. This revision fixes that.
* MFC r260567alc2014-05-131-15/+17
| | | | | | | | | | | | | | | | | | | | | | | | Correctly update the count of stuck pages, "addl_page_shortage", in vm_pageout_scan(). There were missing increments in two less common cases. Don't conflate the count of stuck pages and the pageout deficit provided by vm_page_alloc{,_contig}(). Handle held pages consistently in the inactive queue scan. In the more common case, we did not move the page to the tail of the queue. Whereas, in the less common case, we did. There's no particular reason to move the page in the less common case, so remove it. Perform the calculation of the page shortage for the active queue scan a little earlier, before the active queue lock is acquired. The correctness of this calculation doesn't depend on the active queue lock being held. Eliminate a redundant variable, "pcount". Use the more descriptive variable, "maxscan", in its place. Apply a few nearby style fixes, e.g., eliminate stray whitespace and excess parentheses.
* MFH (r264966): add sysctl OIDs for actual swap zone size and capacitydes2014-05-121-3/+11
|
* MFC r265100:kib2014-05-061-1/+3
| | | | Fix the comparision for the end of range in vm_phys_fictitious_reg_range().
* MFC r265002:kib2014-05-041-8/+15
| | | | | Fix vm_fault_copy_entry() operation on upgrade; allow it to find the pages in the shadowed objects.
* MFC r263475:kib2014-03-281-0/+4
| | | | | | | | | | | | | | | | Fix two issues with /dev/mem access on amd64, both causing kernel page faults. First, for accesses to direct map region should check for the limit by which direct map is instantiated. Second, for accesses to the kernel map, use a new thread private flag TDP_DEVMEMIO, which instructs vm_fault() to return error when fault happens on the MAP_ENTRY_NOFAULT entry, instead of panicing. MFC r263498: Add change forgotten in r263475. Make dmaplimit accessible outside amd64/pmap.c.
* MFC r263471:kib2014-03-241-0/+1
| | | | Initialize vm_map_entry member wiring_thread on the map entry creation.
* MFC r263095:kib2014-03-191-0/+1
| | | | Initialize paddr to handle the case of zero size.
* MFC r263092:kib2014-03-191-1/+2
| | | | | Do not vdrop() the tmpfs vnode until it is unlocked. The hold reference might be the last, and then vdrop() would free the vnode.
* Merge r261722, r261723, r261724, r261725 from head:glebius2014-03-042-5/+23
| | | | several minor improvements for UMA_ZPCPU_ZONE zones.
* Merge 261593 from head:glebius2014-03-041-0/+29
| | | | | Provide macros that allow easily export uma(9) zone limits and current usage via sysctl(9).
* MFC r261867:attilio2014-02-211-2/+4
| | | | Use the right index to free swapspace after vm_page_rename().
* MFC r261896:dim2014-02-171-4/+1
| | | | | | | After r251709, avoid a clang 3.4 warning about an unused static const variable (uma_max_ipers), when asserts are disabled. Reviewed by: glebius
* MFC r259908:marcel2014-02-161-0/+10
| | | | For ia64, use pmap_remove_pages() and not pmap_remove().
* MFC r258716:mav2014-01-041-5/+34
| | | | | - Add bucket size column to `show uma` DDB command. - Add `show umacache` command to show alike stats for cache-only UMA zones.
* MFC r258693:mav2014-01-041-2/+16
| | | | | | | | | | Make UMA to not blindly force offpage slab header allocation for large (> PAGE_SIZE) zones. If zone is not multiple to PAGE_SIZE, there may be enough space for the header at the last page, so we may avoid extra header memory allocation and hash table update/lookup. ZFS creates bunch of odd-sized UMA zones (5120, 6144, 7168, 10240, 14336). This change gives good use to at least some of otherwise lost memory there.
* MFC r258691:mav2014-01-041-5/+3
| | | | | | | | Don't count bucket allocation failures for UMA zones as their own failures. There are good reasons for this to happen, such as recursion prevention, etc. and they are not fatal since buckets are just an optimization mechanism. Real bucket allocation failures are any way counted by the bucket zones themselves, and we don't need double accounting there.
* MFC r258340, r258497:mav2014-01-041-0/+89
| | | | | | | | Implement mechanism to safely but slowly purge UMA per-CPU caches. This is a last resort for very low memory condition in case other measures to free memory were ineffective. Sequentially cycle through all CPUs and extract per-CPU cache buckets into zone cache from where they can be freed.
* MFC r258338:mav2014-01-041-2/+13
| | | | | | | | Grow UMA zone bucket size also on lock congestion during item free. Lock congestion is the same, whether it happens on alloc or free, so handle it equally. Now that we have back pressure, there is no problem to grow buckets a bit faster. Any way growth is much slower then in 9.x.
* MFC r258337:mav2014-01-041-0/+2
| | | | | | | | | | | | Add two new UMA bucket zones to store 3 and 9 items per bucket. These new buckets make bucket size self-tuning more soft and precise. Without them there are buckets for 1, 5, 13, 29, ... items. While at bigger sizes difference about 2x is fine, at smallest ones it is 5x and 2.6x respectively. New buckets make that line look like 1, 3, 5, 9, 13, 29, reducing jumps between steps, making algorithm work softer, allocating and freeing memory in better fitting chunks. Otherwise there is quite a big gap between allocating 128K and 5x128K of RAM at once.
* MFC r258336:mav2014-01-042-1/+11
| | | | | | | | | | | | | Implement soft pressure on UMA cache bucket sizes. Every time system detects low memory condition decrease bucket sizes for each zone by one item. As result, higher memory pressure will push to smaller bucket sizes and so smaller per-CPU caches and so more efficient memory use. Before this change there was no force to oppose buckets growth as result of practically inevitable zone lock conflicts, and after some run time per-CPU caches could consume enough RAM to kill the system.
* Merge r258690 by mav from head:glebius2014-01-041-3/+4
| | | | | | | | Fix bug introduced at r252226, when udata argument passed to bucket_alloc() was used without making sure first that it was really passed for us. On some of my systems this bug made user argument passed by ZFS code to uma_zalloc_arg() unexpectedly block UMA per-CPU caches for those zones.
OpenPOWER on IntegriCloud