summaryrefslogtreecommitdiffstats
path: root/sys/vm/vm_contig.c
Commit message (Collapse)AuthorAgeFilesLines
* The synchronization provided by vm object locking has eliminated thealc2004-11-031-2/+0
| | | | | | | | | | | | | | | | | need for most calls to vm_page_busy(). Specifically, most calls to vm_page_busy() occur immediately prior to a call to vm_page_remove(). In such cases, the containing vm object is locked across both calls. Consequently, the setting of the vm page's PG_BUSY flag is not even visible to other threads that are following the synchronization protocol. This change (1) eliminates the calls to vm_page_busy() that immediately precede a call to vm_page_remove() or functions, such as vm_page_free() and vm_page_rename(), that call it and (2) relaxes the requirement in vm_page_remove() that the vm page's PG_BUSY flag is set. Now, the vm page's PG_BUSY flag is set only when the vm object lock is released while the vm page is still in transition. Typically, this is when it is undergoing I/O.
* Acquire the vm object lock before rather than after callingalc2004-10-241-4/+5
| | | | | | vm_page_sleep_if_busy(). (The motivation being to transition synchronization of the vm_page's PG_BUSY flag from the global page queues lock to the per-object lock.)
* Turn on the new contigmalloc(9) by default. There should not actuallygreen2004-08-051-1/+1
| | | | | be a reason to use the old contigmalloc(9), but if desired, it the vm.old_contigmalloc setting can be tuned/sysctld back to 0 for now.
* Remove extraneous locks on the VM free page queue mutex; it is notgreen2004-07-191-2/+0
| | | | | | | meant to be recursed upon, and could cauuse a deadlock inside the new contigmalloc (vm.old_contigmalloc=0) code. Submitted by: alc
* Reimplement contigmalloc(9) with an algorithm which stands a greatly-green2004-07-191-36/+270
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | improved chance of working despite pressure from running programs. Instead of trying to throw a bunch of pages out to swap and hope for the best, only a range that can potentially fulfill contigmalloc(9)'s request will have its contents paged out (potentially, not forcibly) at a time. The new contigmalloc operation still operates in three passes, but it could potentially be tuned to more or less. The first pass only looks at pages in the cache and free pages, so they would be thrown out without having to block. If this is not enough, the subsequent passes page out any unwired memory. To combat memory pressure refragmenting the section of memory being laundered, each page is removed from the systems' free memory queue once it has been freed so that blocking later doesn't cause the memory laundered so far to get reallocated. The page-out operations are now blocking, as it would make little sense to try to push out a page, then get its status immediately afterward to remove it from the available free pages queue, if it's unlikely to have been freed. Another change is that if KVA allocation fails, the allocated memory segment will be freed and not leaked. There is a sysctl/tunable, defaulting to on, which causes the old contigmalloc() algorithm to be used. Nonetheless, I have been using vm.old_contigmalloc=0 for over a month. It is safe to switch at run-time to see the difference it makes. A new interface has been used which does not require mapping the allocated pages into KVA: vm_page.h functions vm_page_alloc_contig() and vm_page_release_contig(). These are what vm.old_contigmalloc=0 uses internally, so the sysctl/tunable does not affect their operation. When using the contigmalloc(9) and contigfree(9) interfaces, memory is now tracked with malloc(9) stats. Several functions have been exported from kern_malloc.c to allow other subsystems to use these statistics, as well. This invalidates the BUGS section of the contigmalloc(9) manpage.
* Make contigmalloc() more reliable:green2004-06-151-6/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | 1. Remove a race whereby contigmalloc() would deadlock against the running processes in the system if they kept reinstantiating the memory on the active and inactive page queues that it was trying to flush out. The process doing the contigmalloc() would sit in "swwrt" forever and the swap pager would be going at full force, but never get anywhere. Instead of doing it until the queues are empty, launder for as many iterations as there are pages in the queue. 2. Do all laundering to swap synchronously; previously, the vnode laundering was synchronous and the swap laundering not. 3. Increase the number of launder-or-allocate passes to three, from two, while failing without bothering to do all the laundering on the third pass if allocation was not possible. This effectively gives exactly two chances to launder enough contiguous memory, helpful with high memory churn where a lot of memory from one pass to the next (and during a single laundering loop) becomes dirtied again. I can now reliably hot-plug hardware requiring a 256KB contigmalloc() without having the kldload/cbb ithread sit around failing to make progress, while running a busy X session. Previously, it took killing X to get contigmalloc() to get further (that is, quiescing the system), and even then contigmalloc() returned failure.
* Remove advertising clause from University of California Regent's license,imp2004-04-061-4/+0
| | | | | | per letter dated July 22, 1999. Approved by: core
* Remove GIANT_REQUIRED from contigfree().alc2004-03-131-1/+1
|
* In the last revision, I introduced a physical contiguity check that is bothalc2004-03-051-3/+1
| | | | | | | | | | unnecessary and wrong. While it is necessary to verify that the page is still free after dropping and reacquiring the free page queue lock, the physical contiguity of the page can not change, making this check unnecessary. This check was wrong in that it could cause an out-of-bounds array access. Tested by: rwatson
* Modify contigmalloc1() so that the free page queues lock is not held whenalc2004-03-021-1/+13
| | | | | | vm_page_free() is called. The problem with holding this lock is that it is a spin lock and vm_page_free() may attempt the acquisition of a different default-type lock.
* Correct a long-standing race condition in vm_contig_launder() that couldalc2004-02-161-0/+2
| | | | | | | | | result in a panic "vm_page_cache: caching a dirty page, ...": Access to the page must be restricted or removed before calling vm_page_cache(). This race condition is identical in nature to that which was addressed by vm_pageout.c's revision 1.251 and vm_page.c's revision 1.275. MFC after: 7 days
* Remove vm_page_alloc_contig(). It's now unused.alc2004-01-141-15/+0
|
* - Unmanage pages allocated by contigmalloc1(). (There is no point inalc2004-01-101-6/+2
| | | | | having PV entries for these pages.) - Remove splvm() and splx() calls.
* - Enable recursive acquisition of the mutex synchronizing access to thealc2004-01-081-2/+6
| | | | | | | | | | | | | | | | | free pages queue. This is presently needed by contigmalloc1(). - Move a sanity check against attempted double allocation of two pages to the same vm object offset from vm_page_alloc() to vm_page_insert(). This provides better protection because double allocation could occur through a direct call to vm_page_insert(), such as that by vm_page_rename(). - Modify contigmalloc1() to hold the mutex synchronizing access to the free pages queue while it scans vm_page_array in search of free pages. - Correct a potential leak of pages by contigmalloc1() that I introduced in revision 1.20: We must convert all cache queue pages to free pages before we begin removing free pages from the free queue. Otherwise, if we have to restart the scan because we are unable to acquire the vm object lock that is necessary to convert a cache queue page to a free page, we leak those free pages already removed from the free queue.
* Don't bother clearing PG_ZERO in contigmalloc1(), kmem_alloc(), oralc2004-01-061-1/+0
| | | | kmem_malloc(). It serves no purpose.
* - Increase the object lock's scope in vm_contig_launder() so that accessalc2003-10-181-4/+11
| | | | | | | | | to the object's type field and the call to vm_pageout_flush() are synchronized. - The above change allows for the eliminaton of the last parameter to vm_pageout_flush(). - Synchronize access to the page's valid field in vm_pageout_flush() using the containing object's lock.
* Add the mlockall() and munlockall() system calls.bms2003-08-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | - All those diffs to syscalls.master for each architecture *are* necessary. This needed clarification; the stub code generation for mlockall() was disabled, which would prevent applications from linking to this API (suggested by mux) - Giant has been quoshed. It is no longer held by the code, as the required locking has been pushed down within vm_map.c. - Callers must specify VM_MAP_WIRE_HOLESOK or VM_MAP_WIRE_NOHOLES to express their intention explicitly. - Inspected at the vmstat, top and vm pager sysctl stats level. Paging-in activity is occurring correctly, using a test harness. - The RES size for a process may appear to be greater than its SIZE. This is believed to be due to mappings of the same shared library page being wired twice. Further exploration is needed. - Believed to back out of allocations and locks correctly (tested with WITNESS, MUTEX_PROFILING, INVARIANTS and DIAGNOSTIC). PR: kern/43426, standards/54223 Reviewed by: jake, alc Approved by: jake (mentor) MFC after: 2 weeks
* Use pmap_zero_page() to zero pages instead of bzero() becausemux2003-07-271-1/+1
| | | | they haven't been vm_map_wire()'d yet.
* Acquire Giant rather than asserting it is held in contigmalloc(). This isalc2003-07-261-1/+2
| | | | a prerequisite to removing further uses of Giant from UMA.
* Add support for the M_ZERO flag to contigmalloc().mux2003-07-251-1/+5
| | | | Reviewed by: jeff
* Lock a vm object when freeing a page from it.alc2003-07-051-0/+7
|
* Fix a few style(9) nits.mux2003-07-021-13/+9
|
* Use __FBSDID().obrien2003-06-111-1/+3
|
* - Acquire the vm_object's lock when performing vm_object_page_clean().alc2003-04-241-1/+3
| | | | | | - Add a parameter to vm_pageout_flush() that tells vm_pageout_flush() whether its caller has locked the vm_object. (This is a temporary measure to bootstrap vm_object locking.)
* Update locking on the kernel_object to use the new macros.alc2003-04-141-2/+2
|
* - Add vm_paddr_t, a physical address type. This is required for systemsjake2003-03-251-7/+8
| | | | | | | | | | | | | | | where physical addresses larger than virtual addresses, such as i386s with PAE. - Use this to represent physical addresses in the MI vm system and in the i386 pmap code. This also changes the paddr parameter to d_mmap_t. - Fix printf formats to handle physical addresses >4G in the i386 memory detection code, and due to kvtop returning vm_paddr_t instead of u_long. Note that this is a name change only; vm_paddr_t is still the same as vm_offset_t on all currently supported platforms. Sponsored by: DARPA, Network Associates Laboratories Discussed with: re, phk (cdevsw change)
* - Hold the kernel_object's lock around vm_page_insert(..., kernel_object,alc2002-12-231-0/+2
| | | | ...).
* o Extend the scope of the page queues lock in contigmalloc1().alc2002-08-041-8/+8
| | | | | o Replace vm_page_sleep_busy() with vm_page_sleep_if_busy() in vm_contig_launder().
* o Require that the page queues lock is held on entry to vm_pageout_clean()alc2002-07-271-0/+2
| | | | | | and vm_pageout_flush(). o Acquire the page queues lock before calling vm_pageout_clean() or vm_pageout_flush().
* o Lock page queue accesses by vm_page_cache() in vm_contig_launder().alc2002-07-201-2/+4
| | | | o Micro-optimize the control flow in vm_contig_launder().
* o Create vm_contig_launder() to replace code that appears twicealc2002-07-151-56/+38
| | | | in contigmalloc1().
* o Lock some (unfortunately, not yet all) accesses to the page queues.alc2002-07-121-2/+2
|
* o Lock accesses to the free page queues in contigmalloc1().alc2002-07-051-0/+2
|
* o Use vm_map_wire() and vm_map_unwire() in place of vm_map_pageable() andalc2002-06-141-1/+1
| | | | | | | | | vm_map_user_pageable(). o Remove vm_map_pageable() and vm_map_user_pageable(). o Remove vm_map_clear_recursive() and vm_map_set_recursive(). (They were only used by vm_map_pageable() and vm_map_user_pageable().) Reviewed by: tegge
* o Make contigmalloc1() static.alc2002-05-221-1/+1
|
* Call vm_pageq_remove_nowakeup() rather than duplicating it.alc2002-03-031-8/+2
|
* contigmalloc1() could cause the vm_page_zero_count to become incorrect.dillon2001-10-171-0/+2
| | | | | | Properly track the count. Submitted by: mark tinguely <tinguely@web.cs.ndsu.nodak.edu>
* Makes contigalloc[1]() create the vm_map / underlying wired pages in thedillon2001-10-131-4/+12
| | | | | | | | | | | kernel map and object in a manner that contigfree() is actually able to free. Previously contigfree() freed up the KVA space but could not unwire & free the underlying VM pages due to mismatched pageability between the map entry and the VM pages. Submitted by: Thomas Moestl <tmoestl@gmx.net> Testing by: mark tinguely <tinguely@web.cs.ndsu.nodak.edu> MFC after: 3 days
* KSE Milestone 2julian2001-09-121-4/+4
| | | | | | | | | | | | | | Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
* Reorg vm_page.c into vm_page.c, vm_pageq.c, and vm_contig.c (for contigmalloc).dillon2001-07-041-0/+309
Also removed some spl's and added some VM mutexes, but they are not actually used yet, so this commit does not really make any operational changes to the system. vm_page.c relates to vm_page_t manipulation, including high level deactivation, activation, etc... vm_pageq.c relates to finding free pages and aquiring exclusive access to a page queue (exclusivity part not yet implemented). And the world still builds... :-)
OpenPOWER on IntegriCloud