summaryrefslogtreecommitdiffstats
path: root/sys/vm/vm_map.c
Commit message (Collapse)AuthorAgeFilesLines
* Eliminate pmap_init2(). It's no longer used.alc2005-11-201-1/+0
|
* Pass a value of type vm_prot_t to pmap_enter_quick() so that it determinealc2005-09-031-2/+2
| | | | whether the mapping should permit execute access.
* Eliminate an incorrect (and unnecessary) cast.alc2005-07-201-1/+1
|
* Remove GIANT_REQUIRED from vmspace_exec().alc2005-05-021-1/+0
| | | | Prodded by: jeff
* Add checks to vm_map_findspace() to test for address wrap. The conditionsalc2005-01-181-4/+8
| | | | | | | where this could occur are very rare, but possible. Submitted by: Mark W. Krentel MFC after: 2 weeks
* /* -> /*- for license, minor formatting changesimp2005-01-071-1/+1
|
* Modify pmap_enter_quick() so that it expects the page queues to be lockedalc2004-12-231-3/+8
| | | | | | | | | | | on entry and it assumes the responsibility for releasing the page queues lock if it must sleep. Remove a bogus comment from pmap_enter_quick(). Using the first change, modify vm_map_pmap_enter() so that the page queues lock is acquired and released once, rather than each time that a page is mapped.
* In the common case, pmap_enter_quick() completes without sleeping.alc2004-12-151-9/+4
| | | | | | | | | | | | | | | | | | In such cases, the busying of the page and the unlocking of the containing object by vm_map_pmap_enter() and vm_fault_prefault() is unnecessary overhead. To eliminate this overhead, this change modifies pmap_enter_quick() so that it expects the object to be locked on entry and it assumes the responsibility for busying the page and unlocking the object if it must sleep. Note: alpha, amd64, i386 and ia64 are the only implementations optimized by this change; arm, powerpc, and sparc64 still conservatively busy the page and unlock the object within every pmap_enter_quick() call. Additionally, this change is the first case where we synchronize access to the page's PG_BUSY flag and busy field using the containing object's lock rather than the global page queues lock. (Modifications to the page's PG_BUSY flag and busy field have asserted both locks for several weeks, enabling an incremental transition.)
* Push Giant deep into vm_forkproc(), acquiring it only if the process hasalc2004-09-031-3/+0
| | | | | mapped System V shared memory segments (see shmfork_myhook()) or requires the allocation of an ldt (see vm_fault_wire()).
* - Introduce and use a new tunable "debug.mpsafevm". At present, settingalc2004-08-161-4/+4
| | | | | | | | | | | | | | "debug.mpsafevm" results in (almost) Giant-free execution of zero-fill page faults. (Giant is held only briefly, just long enough to determine if there is a vnode backing the faulting address.) Also, condition the acquisition and release of Giant around calls to pmap_remove() on "debug.mpsafevm". The effect on performance is significant. On my dual Opteron, I see a 3.6% reduction in "buildworld" time. - Use atomic operations to update several counters in vm_fault().
* Rather than bringing back all of the changes to make VM map deletiongreen2004-08-161-1/+5
| | | | | | | | wait for system wires to disappear, do so (much more trivially) by instead only checking for system wires of user maps and not kernel maps. Alternative by: tor Reviewed by: alc
* Remove spl calls.alc2004-08-141-19/+2
|
* Replace the linear search in vm_map_findspace() with an O(log n)alc2004-08-131-97/+211
| | | | | | | | | | | | | | | | | | | | | algorithm built into the map entry splay tree. This replaces the first_free hint in struct vm_map with two fields in vm_map_entry: adj_free, the amount of free space following a map entry, and max_free, the maximum amount of free space in the entry's subtree. These fields make it possible to find a first-fit free region of a given size in one pass down the tree, so O(log n) amortized using splay trees. This significantly reduces the overhead in vm_map_findspace() for applications that mmap() many hundreds or thousands of regions, and has a negligible slowdown (0.1%) on buildworld. See, for example, the discussion of a micro-benchmark titled "Some mmap observations compared to Linux 2.6/OpenBSD" on -hackers in late October 2003. OpenBSD adopted this approach in March 2002, and NetBSD added it in November 2003, both with Red-Black trees. Submitted by: Mark W. Krentel
* The vm map lock is needed in vm_fault() after the page has been found,tegge2004-08-121-6/+102
| | | | | | | | | to avoid later changes before pmap_enter() and vm_fault_prefault() has completed. Simplify deadlock avoidance by not blocking on vm map relookup. In collaboration with: alc
* Re-delete the comment from r1.352.green2004-08-121-3/+0
|
* Back out all behavioral chnages.green2004-08-101-32/+17
|
* Revamp VM map wiring.green2004-08-091-14/+32
| | | | | | | | | | | | | | | | | * Allow no-fault wiring/unwiring to succeed for consistency; however, the wired count remains at zero, so it's a special case. * Fix issues inside vm_map_wire() and vm_map_unwire() where the exact state of user wiring (one or zero) and system wiring (zero or more) could be confused; for example, system unwiring could succeed in removing a user wire, instead of being an error. * Require all mappings to be unwired before they are deleted. When VM space is still wired upon deletion, it will be waited upon for the following unwire. This makes vslock(9) work rather than allowing kernel-locked memory to be deleted out from underneath of its consumer as it would before.
* Remove a stale comment from vm_map_lookup() that pertains to share maps.alc2004-08-091-3/+0
| | | | | (The last vestiges of the share map code were removed in revisions 1.153 and 1.159.)
* - Push down the acquisition and release of Giant into pmap_enter_quick()alc2004-08-041-2/+0
| | | | | on those architectures without pmap locking. - Eliminate the acquisition and release of Giant in vm_map_pmap_enter().
* * Add a "how" argument to uma_zone constructors and initialization functionsgreen2004-08-021-7/+9
| | | | | | | | | | | | | | | | | so that they know whether the allocation is supposed to be able to sleep or not. * Allow uma_zone constructors and initialation functions to return either success or error. Almost all of the ones in the tree currently return success unconditionally, but mbuf is a notable exception: the packet zone constructor wants to be able to fail if it cannot suballocate an mbuf cluster, and the mbuf allocators want to be able to fail in general in a MAC kernel if the MAC mbuf initializer fails. This fixes the panics people are seeing when they run out of memory for mbuf clusters. * Allow debug.nosleepwithlocks on WITNESS to be disabled, without changing the default. Both bmilekic and jeff have reviewed the changes made to make failable zone allocations work.
* - Push down the acquisition and release of Giant into pmap_protect() onalc2004-07-301-2/+0
| | | | | | | | those architectures without pmap locking. - Eliminate the acquisition and release of Giant from vm_map_protect(). (Translation: mprotect(2) runs to completion without touching Giant on alpha, amd64, i386 and ia64.)
* Get rid of another lockmgr(9) consumer by using sx locks for the usermux2004-07-301-25/+17
| | | | | | | | | maps. We always acquire the sx lock exclusively here, but we can't use a mutex because we want to be able to sleep while holding the lock. This is completely equivalent to what we were doing with the lockmgr(9) locks before. Approved by: alc
* - Use atomic ops for updating the vmspace's refcnt and exitingcnt.alc2004-07-271-6/+11
| | | | | | | | - Push down Giant into shmexit(). (Giant is acquired only if the vmspace contains shm segments.) - Eliminate the acquisition of Giant from proc_rwmem(). - Reduce the scope of Giant in exit1(), uncovering the destruction of the address space.
* Make the code and comments for vm_object_coalesce() consistent.alc2004-07-251-2/+2
|
* Simplify vmspace initialization. The bcopy() of fields from the oldalc2004-07-241-11/+8
| | | | | | | vmspace to the new vmspace in vmspace_exec() is mostly wasted effort. With one exception, vm_swrss, the copied fields are immediately overwritten. Instead, initialize these fields to zero in vmspace_alloc(), eliminating a bcopy() from vmspace_exec() and a bzero() from vmspace_fork().
* Semi-gratuitous change. Move two refcount operations to their own linespeter2004-07-211-2/+4
| | | | | rather than be buried inside an if (expression). And now that the if expression is the same in both exit paths, use the same ordering.
* Move the initialization and teardown of pmaps to the vmspace zone'speter2004-07-211-3/+2
| | | | | | | | | | | | | | | | | init and fini handlers. Our vm system removes all userland mappings at exit prior to calling pmap_release. It just so happens that we might as well reuse the pmap for the next process since the userland slate has already been wiped clean. However. There is a functional benefit to this as well. For platforms that share userland and kernel context in the same pmap, it means that the kernel portion of a pmap remains valid after the vmspace has been freed (process exit) and while it is in uma's cache. This is significant for i386 SMP systems with kernel context borrowing because it avoids a LOT of IPIs from the pmap_lazyfix() cleanup in the usual case. Tested on: amd64, i386, sparc64, alpha Glanced at by: alc
* Push down the acquisition and release of the page queues lock intoalc2004-07-151-8/+0
| | | | | | pmap_protect() and pmap_remove(). In general, they require the lock in order to modify a page's pv list or flags. In some cases, however, pmap_protect() can avoid acquiring the lock.
* Use MIN() macro rather than ulmin() inline, and fix stray tabgallatin2004-06-281-3/+3
| | | | | | that snuck in with my last commit. Submitted by: green
* Fix alpha - the use of min() on longs was loosing the high bits andgallatin2004-06-281-3/+3
| | | | returning wrong answers, leading to strange values vm2->vm_{s,t,d}size.
* Correct the tracking of various bits of the process's vmspace and vm_mapgreen2004-06-241-3/+44
| | | | | | | | | | when not propogated on fork (due to minherit(2)). Consistency checks otherwise fail when the vm_map is freed and it appears to have not been emptied completely, causing an INVARIANTS panic in vm_map_zdtor(). PR: kern/68017 Submitted by: Mark W. Krentel <krentel@dreamscape.com> Reviewed by: alc
* Back out previous commit; it went to the wrong file.des2004-05-251-8/+1
|
* MFS: rev 1.187.2.27 through 1.187.2.29, fix MS_INVALIDATE semantics butdes2004-05-251-1/+8
| | | | provide a sysctl knob for reverting to old ones.
* Correct two error cases in vm_map_unwire():alc2004-05-251-4/+5
| | | | | | | | | | | | | | | | | 1. Contrary to the Single Unix Specification our implementation of munlock(2) when performed on an unwired virtual address range has returned an error. Correct this. Note, however, that the behavior of "system" unwiring is unchanged, only "user" unwiring is changed. If "system" unwiring is performed on an unwired virtual address range, an error is still returned. 2. Performing an errant "system" unwiring on a virtual address range that was "user" (i.e., mlock(2)) but not "system" wired would incorrectly undo the "user" wiring instead of returning an error. Correct this. Discussed with: green@ Reviewed by: tegge@
* To date, unwiring a fictitious page has produced a panic. The reasonalc2004-05-221-5/+13
| | | | | | | | | | | | | | | being that PHYS_TO_VM_PAGE() returns the wrong vm_page for fictitious pages but unwiring uses PHYS_TO_VM_PAGE(). The resulting panic reported an unexpected wired count. Rather than attempting to fix PHYS_TO_VM_PAGE(), this fix takes advantage of the properties of fictitious pages. Specifically, fictitious pages will never be completely unwired. Therefore, we can keep a fictitious page's wired count forever set to one and thereby avoid the use of PHYS_TO_VM_PAGE() when we know that we're working with a fictitious page, just not which one. In collaboration with: green@, tegge@ PR: kern/29915
* Properly remove MAP_FUTUREWIRE when a vm_map_entry gets torn down.green2004-05-071-0/+1
| | | | | | | | | | | | | | Previously, mlockall(2) usage would leak MAP_FUTUREWIRE of the process's vmspace::vm_map and subsequent processes would wire all of their memory. Coupled with a wired-page leak in vm_fault_unwire(), this would run the system out of free pages and cause programs to randomly SIGBUS when faulting in new pages. (Note that this is not the fix for the latter part; pages are still leaked when a wired area is unmapped in some cases.) Reviewed by: alc PR kern/62930
* In cases where a file was resident in memory mmap(..., PROT_NONE, ...)alc2004-04-241-4/+5
| | | | | | | | | | | | | | | would actually map the file with read access enabled. According to http://www.opengroup.org/onlinepubs/007904975/functions/mmap.html this is an error. Similarly, an madvise(..., MADV_WILLNEED) would enable read access on a virtual address range that was PROT_NONE. The solution implemented herein is (1) to pass a vm_prot_t to vm_map_pmap_enter() describing the allowed access and (2) to make vm_map_pmap_enter() responsible for understanding the limitations of pmap_enter_quick(). Submitted by: "Mark W. Krentel" <krentel@dreamscape.com> PR: kern/64573
* Remove advertising clause from University of California Regent's license,imp2004-04-061-4/+0
| | | | | | per letter dated July 22, 1999. Approved by: core
* Do not copy vm_exitingcnt to the new vmspace in vmspace_exec(). Copyingtjr2004-03-231-1/+2
| | | | | it led to impossibly high values in the new vmspace, causing it to never drop to 0 and be freed.
* Retire pmap_pinit2(). Alpha was the last platform that used it. However,alc2004-03-071-2/+0
| | | | | | | | | | | | | | ever since alpha/alpha/pmap.c revision 1.81 introduced the list allpmaps, there has been no reason for having this function on Alpha. Briefly, when pmap_growkernel() relied upon the list of all processes to find and update the various pmaps to reflect a growth in the kernel's valid address space, pmap_init2() served to avoid a race between pmap initialization and pmap_growkernel(). Specifically, pmap_pinit2() was responsible for initializing the kernel portions of the pmap and pmap_pinit2() was called after the process structure contained a pointer to the new pmap for use by pmap_growkernel(). Thus, an update to the kernel's address space might be applied to the new pmap unnecessarily, but an update would never be lost.
* Further reduce the use of Giant in vm_map_delete(): Perform pmap_remove()alc2004-02-121-2/+2
| | | | | | on system maps, besides the kmem_map, without Giant. In collaboration with: tegge
* - Locking for the per-process resource limits structure has eliminatedalc2004-02-051-3/+1
| | | | | | the need for Giant in vm_map_growstack(). - Use the proc * that is passed to vm_map_growstack() rather than curthread->td_proc.
* Locking for the per-process resource limits structure.jhb2004-02-041-9/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - struct plimit includes a mutex to protect a reference count. The plimit structure is treated similarly to struct ucred in that is is always copy on write, so having a reference to a structure is sufficient to read from it without needing a further lock. - The proc lock protects the p_limit pointer and must be held while reading limits from a process to keep the limit structure from changing out from under you while reading from it. - Various global limits that are ints are not protected by a lock since int writes are atomic on all the archs we support and thus a lock wouldn't buy us anything. - All accesses to individual resource limits from a process are abstracted behind a simple lim_rlimit(), lim_max(), and lim_cur() API that return either an rlimit, or the current or max individual limit of the specified resource from a process. - dosetrlimit() was renamed to kern_setrlimit() to match existing style of other similar syscall helper functions. - The alpha OSF/1 compat layer no longer calls getrlimit() and setrlimit() (it didn't used the stackgap when it should have) but uses lim_rlimit() and kern_setrlimit() instead. - The svr4 compat no longer uses the stackgap for resource limits calls, but uses lim_rlimit() and kern_setrlimit() instead. - The ibcs2 compat no longer uses the stackgap for resource limits. It also no longer uses the stackgap for accessing sysctl's for the ibcs2_sysconf() syscall but uses kernel_sysctl() instead. As a result, ibcs2_sysconf() no longer needs Giant. - The p_rlimit macro no longer exists. Submitted by: mtm (mostly, I only did a few cleanups and catchups) Tested on: i386 Compiled on: alpha, amd64
* Drop the reference count on the old vmspace after fully switching thejhb2004-02-021-2/+2
| | | | | | current thread to the new vmspace. Suggested by: dillon
* - Modify vm_object_split() to expect a locked vm object on entry andalc2003-12-301-2/+0
| | | | | return on a locked vm object on exit. Remove GIANT_REQUIRED. - Eliminate some unnecessary local variables from vm_object_split().
* Minor correction to revision 1.258: Use the proc pointer that is passed toalc2003-12-261-2/+1
| | | | vm_map_growstack() in the RLIMIT_VMEM check rather than curthread.
* - Avoid a lock-order reversal between Giant and a system map mutex thatalc2003-11-191-2/+4
| | | | | | | | | occurs when kmem_malloc() fails to allocate a sufficient number of vm pages. Specifically, we avoid the lock-order reversal by not grabbing Giant around pmap_remove() if the map is the kmem_map. Approved by: re (jhb) Reported by: Eugene <eugene3@web.de>
* Changes to msync(2)alc2003-11-141-2/+2
| | | | | | | | | | - Return EBUSY if the region was wired by mlock(2) and MS_INVALIDATE is specified to msync(2). This is required by the Open Group Base Specifications Issue 6. - vm_map_sync() doesn't return KERN_FAILURE. Thus, msync(2) can't possibly return EIO. - The second major loop in vm_map_sync() handles sub maps. Thus, failing on sub maps in the first major loop isn't necessary.
* - The Open Group Base Specifications Issue 6 specifies that an munmap(2)alc2003-11-101-14/+6
| | | | | | | | must return EINVAL if size is zero. Submitted by: tegge - In order to avoid a race condition in multithreaded applications, the check and removal operations by munmap(2) must be in the same critical section. To accomodate this, vm_map_check_protection() is modified to require its caller to obtain at least a read lock on the map.
* - Remove Giant from msync(2). Giant is still acquired by the lower layersalc2003-11-091-0/+10
| | | | | | | | | | if we drop into the pmap or vnode layers. - Migrate the handling of zero-length msync(2)s into vm_map_sync() so that multithread applications can't change the map between implementing the zero-length hack in msync(2) and reacquiring the map lock in vm_map_sync(). Reviewed by: tegge
OpenPOWER on IntegriCloud