summaryrefslogtreecommitdiffstats
path: root/sys/vm/vm_kern.c
Commit message (Collapse)AuthorAgeFilesLines
* Move what remains of vm/vm_contig.c into vm/vm_pageout.c, where similaralc2012-07-181-2/+2
| | | | | | code resides. Rename vm_contig_grow_cache() to vm_pageout_grow_cache(). Reviewed by: kib
* Move kmem_alloc_{attr,contig}() to vm/vm_kern.c, where similarly namedalc2012-07-141-0/+142
| | | | functions reside. Correct the comment describing kmem_alloc_contig().
* Simplify kmem_alloc() by eliminating code that existed on account ofalc2012-02-291-30/+0
| | | | | | | | | external pagers in Mach. FreeBSD doesn't implement external pagers. Moreover, it don't pageout the kernel object. So, the reasons for having code don't hold. Reviewed by: kib MFC after: 6 weeks
* exclude kmem_alloc'ed ARC data buffers from kernel minidumps on amd64kmacy2012-01-271-0/+2
| | | | | | | | excluding other allocations including UMA now entails the addition of a single flag to kmem_alloc or uma zone create Reviewed by: alc, avg MFC after: 2 weeks
* Eliminate vestiges of page coloring in VM_ALLOC_NOOBJ calls toalc2011-10-271-1/+1
| | | | | | vm_page_alloc(). While I'm here, for the sake of consistency, always specify the allocation class, such as VM_ALLOC_NORMAL, as the first of the flags.
* - Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flagkib2011-08-091-2/+2
| | | | | | | | | | | | | | to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
* Move the ZERO_REGION_SIZE to a machine-dependent file, as on manymdf2011-05-131-9/+6
| | | | | | | | | | | | | | | | | | architectures (i386, for example) the virtual memory space may be constrained enough that 2MB is a large chunk. Use 64K for arches other than amd64 and ia64, with special handling for sparc64 due to differing hardware. Also commit the comment changes to kmem_init_zero_region() that I missed due to not saving the file. (Darn the unfamiliar development environment). Arch maintainers, please feel free to adjust ZERO_REGION_SIZE as you see fit. Requested by: alc MFC after: 1 week MFC with: r221853
* Usa a globally visible region of zeros for both /dev/zero and the mdmdf2011-05-131-0/+34
| | | | | | | | device. There are likely other kernel uses of "blob of zeros" than can be converted. Reviewed by: alc MFC after: 1 week
* Since r218070 reenabled the call to vm_map_simplify_entry() fromkib2011-02-151-9/+23
| | | | | | | | | | | | | | | | | vm_map_insert(), the kmem_back() assumption about newly inserted entry might be broken due to interference of two factors. In the low memory condition, when vm_page_alloc() returns NULL, supplied map is unlocked. If another thread performs kmem_malloc() meantime, and its map entry is placed right next to our thread map entry in the map, both entries wire count is still 0 and entries are coalesced due to vm_map_simplify_entry(). Mark new entry with MAP_ENTRY_IN_TRANSITION to prevent coalesce. Fix some style issues, tighten the assertions to account for MAP_ENTRY_IN_TRANSITION state. Reported and tested by: pho Reviewed by: alc
* Replace an XXX comment with the appropriate code.mdf2010-09-201-5/+1
| | | | Submitted by: alc
* Rework memguard(9) to reserve significantly more KVA to detectmdf2010-08-111-8/+27
| | | | | | | | | | | | | | | | | use-after-free over a longer time. Also release the backing pages of a guarded allocation at free(9) time to reduce the overhead of using memguard(9). Allow setting and varying the malloc type at run-time. Add knobs to allow: - randomly guarding memory - adding un-backed KVA guard pages to detect underflow and overflow - a lower limit on the size of allocations that are guarded Reviewed by: alc Reviewed by: brueffer, Ulrich Spörlein <uqs spoerlein net> (man page) Silence from: -arch Approved by: zml (mentor) MFC after: 1 month
* The pages allocated by kmem_alloc_attr() and kmem_malloc() are unmanaged.alc2010-05-031-4/+0
| | | | | Consequently, neither the page lock nor the page queues lock is needed to unwire and free them.
* On Alan's advice, rather than do a wholesale conversion on a singlekmacy2010-04-301-0/+2
| | | | | | | | | | | | architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps. Supported by: Bitgravity Inc. Discussed with: alc, jeffr, and kib
* o) Add a VM find-space option, VMFS_TLB_ALIGNED_SPACE, which searches thejmallett2010-04-181-0/+29
| | | | | | | | | | | | | | | | | | | | | address space for an address as aligned by the new pmap_align_tlb() function, which is for constraints imposed by the TLB. [1] o) Add a kmem_alloc_nofault_space() function, which acts like kmem_alloc_nofault() but allows the caller to specify which find-space option to use. [1] o) Use kmem_alloc_nofault_space() with VMFS_TLB_ALIGNED_SPACE to allocate the kernel stack address on MIPS. [1] o) Make pmap_align_tlb() on MIPS align addresses so that they do not start on an odd boundary within the TLB, so that they are suitable for insertion as wired entries and do not have to share a TLB entry with another mapping, assuming they are appropriately-sized. o) Eliminate md_realstack now that the kstack will be appropriately-aligned on MIPS. o) Increase the number of guard pages to 2 so that we retain the proper alignment of the kstack address. Reviewed by: [1] alc X-MFC-after: Making sure alc has not come up with a better interface.
* Implement global and per-uid accounting of the anonymous memory. Addkib2009-06-231-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | rlimit RLIMIT_SWAP that limits the amount of swap that may be reserved for the uid. The accounting information (charge) is associated with either map entry, or vm object backing the entry, assuming the object is the first one in the shadow chain and entry does not require COW. Charge is moved from entry to object on allocation of the object, e.g. during the mmap, assuming the object is allocated, or on the first page fault on the entry. It moves back to the entry on forks due to COW setup. The per-entry granularity of accounting makes the charge process fair for processes that change uid during lifetime, and decrements charge for proper uid when region is unmapped. The interface of vm_pager_allocate(9) is extended by adding struct ucred *, that is used to charge appropriate uid when allocation if performed by kernel, e.g. md(4). Several syscalls, among them is fork(2), may now return ENOMEM when global or per-uid limits are enforced. In collaboration with: pho Reviewed by: alc Approved by: re (kensmith)
* Revert the addition of the freelist argument for the vm_map_delete()kib2009-02-241-9/+3
| | | | | | | | | function, done in r188334. Instead, collect the entries that shall be freed, in the deferred_freelist member of the map. Automatically purge the deferred freelist when map is unlocked. Tested by: pho Reviewed by: alc
* Put debug.vm_lowmem sysctl under DIAGNOSTIC.rwatson2009-02-231-0/+2
| | | | | Submitted by: sam MFC after: 3 days
* Add a debugging sysctl, debug.vm_lowmem, that when assigned a value ofrwatson2009-02-231-0/+22
| | | | | | | | 1 will trigger a pass through the VM's low-memory handlers, such as protocol and UMA drain routines. This makes it easier to exercise these otherwise rarely-invoked code paths. MFC after: 3 days
* Do not call vm_object_deallocate() from vm_map_delete(), because wekib2009-02-081-3/+9
| | | | | | | | | | hold the map lock there, and might need the vnode lock for OBJT_VNODE objects. Postpone object deallocation until caller of vm_map_delete() drops the map lock. Link the map entries to be freed into the freelist, that is released by the new helper function vm_map_entry_free_freelist(). Reviewed by: tegge, alc Tested by: pho
* Eliminate stale comments from kmem_malloc().alc2008-07-181-12/+0
|
* Make preparations for increasing the size of the kernel virtual address spacealc2008-06-221-2/+6
| | | | | | | | on the amd64 architecture. The amd64 architecture requires kernel code and global variables to reside in the highest 2GB of the 64-bit virtual address space. Thus, the memory allocated during bootstrap, before the call to kmem_init(), starts at KERNBASE, which is not necessarily the same as VM_MIN_KERNEL_ADDRESS on amd64.
* Introduce a new parameter "superpage_align" to kmem_suballoc() that isalc2008-05-101-11/+7
| | | | | | | | | | | used to request superpage alignment for the submap. Request superpage alignment for the kmem_map. Pass VMFS_ANY_SPACE instead of TRUE to vm_map_find(). (They are currently equivalent but VMFS_ANY_SPACE is the new preferred spelling.) Remove a stale comment from kmem_malloc().
* Eliminate pointless casts from kmem_suballoc().alc2008-04-281-2/+2
|
* Eliminate an unnecessary printf() from kmem_suballoc(). The subsequentalc2008-03-301-4/+2
| | | | panic() can be extended to convey the same information.
* When one tries to allocate memory with the M_WAITOK flag and we are short inpjd2008-01-101-6/+13
| | | | | | | | | | | | address space in kmem map call vm_lowmem event in a loop and wait a bit for subsystems to reclaim some memory which in turn will reclaim address space as well. Note, this is a work-around. Reviewed by: alc Approved by: alc MFC after: 3 days
* Add an access type parameter to pmap_enter(). It will be used to implementalc2008-01-031-1/+2
| | | | | | | superpage promotion. Correct a style error in kmem_malloc(): pmap_enter()'s last parameter is a Boolean.
* Change unused 'user_wait' argument to 'timo' argument, which will bepjd2007-11-071-1/+1
| | | | | | | used to specify timeout for msleep(9). Discussed with: alc Reviewed by: alc
* When KVA is exhausted, try the vm_lowmem event for the last time beforepjd2007-04-051-4/+14
| | | | panicing. This helps a lot in ZFS stability.
* Change the way that unmanaged pages are created. Specifically,alc2007-02-251-6/+4
| | | | | | | | | | | | | | immediately flag any page that is allocated to a OBJT_PHYS object as unmanaged in vm_page_alloc() rather than waiting for a later call to vm_page_unmanage(). This allows for the elimination of some uses of the page queues lock. Change the type of the kernel and kmem objects from OBJT_DEFAULT to OBJT_PHYS. This allows us to take advantage of the above change to simplify the allocation of unmanaged pages in kmem_alloc() and kmem_malloc(). Remove vm_page_unmanage(). It is no longer used.
* Declare the map entry created by kmem_init() for the range fromalc2007-01-071-1/+2
| | | | | VM_MIN_KERNEL_ADDRESS to the end of the kernel's bootstrap data as MAP_NOFAULT.
* There is no point in setting PG_REFERENCED on kmem_object pages becausealc2006-11-131-6/+1
| | | | | | they are "unmanaged", i.e., non-pageable, pages. Remove a stale comment.
* Make pmap_enter() responsible for setting PG_WRITEABLE insteadalc2006-11-121-1/+1
| | | | | of its caller. (As a beneficial side-effect, a high-contention acquisition of the page queues lock in vm_fault() is eliminated.)
* The page queues lock is no longer required by vm_page_wakeup().alc2006-10-231-1/+1
|
* /* -> /*- for license, minor formatting changesimp2005-01-071-1/+1
|
* Use VM_ALLOC_NOBUSY instead of calling vm_page_wakeup().alc2004-10-241-2/+1
|
* Back out all behavioral chnages.green2004-08-101-4/+0
|
* Revamp VM map wiring.green2004-08-091-0/+4
| | | | | | | | | | | | | | | | | * Allow no-fault wiring/unwiring to succeed for consistency; however, the wired count remains at zero, so it's a special case. * Fix issues inside vm_map_wire() and vm_map_unwire() where the exact state of user wiring (one or zero) and system wiring (zero or more) could be confused; for example, system unwiring could succeed in removing a user wire, instead of being an error. * Require all mappings to be unwired before they are deleted. When VM space is still wired upon deletion, it will be waited upon for the following unwire. This makes vslock(9) work rather than allowing kernel-locked memory to be deleted out from underneath of its consumer as it would before.
* For years, kmem_alloc_pageable() has been misused. Now that the last ofalc2004-07-251-24/+0
| | | | | these misuses has been corrected, remove it before new ones appear, such as arm/arm/pmap.c revision 1.8.
* Bring in mbuma to replace mballoc.bmilekic2004-05-311-10/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mbuma is an Mbuf & Cluster allocator built on top of a number of extensions to the UMA framework, all included herein. Extensions to UMA worth noting: - Better layering between slab <-> zone caches; introduce Keg structure which splits off slab cache away from the zone structure and allows multiple zones to be stacked on top of a single Keg (single type of slab cache); perhaps we should look into defining a subset API on top of the Keg for special use by malloc(9), for example. - UMA_ZONE_REFCNT zones can now be added, and reference counters automagically allocated for them within the end of the associated slab structures. uma_find_refcnt() does a kextract to fetch the slab struct reference from the underlying page, and lookup the corresponding refcnt. mbuma things worth noting: - integrates mbuf & cluster allocations with extended UMA and provides caches for commonly-allocated items; defines several zones (two primary, one secondary) and two kegs. - change up certain code paths that always used to do: m_get() + m_clget() to instead just use m_getcl() and try to take advantage of the newly defined secondary Packet zone. - netstat(1) and systat(1) quickly hacked up to do basic stat reporting but additional stats work needs to be done once some other details within UMA have been taken care of and it becomes clearer to how stats will work within the modified framework. From the user perspective, one implication is that the NMBCLUSTERS compile-time option is no longer used. The maximum number of clusters is still capped off according to maxusers, but it can be made unlimited by setting the kern.ipc.nmbclusters boot-time tunable to zero. Work should be done to write an appropriate sysctl handler allowing dynamic tuning of kern.ipc.nmbclusters at runtime. Additional things worth noting/known issues (READ): - One report of 'ips' (ServeRAID) driver acting really slow in conjunction with mbuma. Need more data. Latest report is that ips is equally sucking with and without mbuma. - Giant leak in NFS code sometimes occurs, can't reproduce but currently analyzing; brueffer is able to reproduce but THIS IS NOT an mbuma-specific problem and currently occurs even WITHOUT mbuma. - Issues in network locking: there is at least one code path in the rip code where one or more locks are acquired and we end up in m_prepend() with M_WAITOK, which causes WITNESS to whine from within UMA. Current temporary solution: force all UMA allocations to be M_NOWAIT from within UMA for now to avoid deadlocks unless WITNESS is defined and we can determine with certainty that we're not holding any locks when we're M_WAITOK. - I've seen at least one weird socketbuffer empty-but- mbuf-still-attached panic. I don't believe this to be related to mbuma but please keep your eyes open, turn on debugging, and capture crash dumps. This change removes more code than it adds. A paper is available detailing the change and considering various performance issues, it was presented at BSDCan2004: http://www.unixdaemons.com/~bmilekic/netbuf_bmilekic.pdf Please read the paper for Future Work and implementation details, as well as credits. Testing and Debugging: rwatson, brueffer, Ketrien I. Saihr-Kesenchedra, ... Reviewed by: Lots of people (for different parts)
* Push down the responsibility for zeroing a physical page from thealc2004-04-241-2/+0
| | | | | | | | | | | | | caller to vm_page_grab(). Although this gives VM_ALLOC_ZERO a different meaning for vm_page_grab() than for vm_page_alloc(), I feel such change is necessary to accomplish other goals. Specifically, I want to make the PG_ZERO flag immutable between the time it is allocated by vm_page_alloc() and freed by vm_page_free() or vm_page_free_zero() to avoid locking overheads. Once we gave up on the ability to automatically recognize a zeroed page upon entry to vm_page_free(), the ability to mutate the PG_ZERO flag became useless. Instead, I would like to say that "Once a page becomes valid, its PG_ZERO flag must be ignored."
* Remove advertising clause from University of California Regent's license,imp2004-04-061-4/+0
| | | | | | per letter dated July 22, 1999. Approved by: core
* Back out previous commit due to objections.des2004-02-161-2/+0
|
* Don't panic if we fail to satisfy an M_WAITOK request; return 0 instead.des2004-02-161-0/+2
| | | | The calling code will either handle that gracefully or cause a page fault.
* Unmanage pages allocated by kmem_alloc(). (There is no point in having PValc2004-01-101-0/+1
| | | | entries for these pages.)
* Don't bother clearing PG_ZERO in contigmalloc1(), kmem_alloc(), oralc2004-01-061-2/+0
| | | | kmem_malloc(). It serves no purpose.
* - Increase the scope of the kmem_object's lock in kmem_malloc(). Add aalc2004-01-011-2/+7
| | | | comment explaining why a further increase is not possible.
* Remove GIANT_REQUIRED from kmem_suballoc().alc2003-12-281-2/+0
|
* NFC: Update stale comments.mini2003-11-101-3/+3
| | | | Reviewed by: alc
* Synchronize access to a vm page's valid field using the containingalc2003-10-041-4/+4
| | | | vm object's lock.
* Call vm_page_unmanage() on pages belonging to the kmem_object. Thisalc2003-09-141-0/+1
| | | | | eliminates the unnecessary overhead of managing "PV" entries for these pages.
OpenPOWER on IntegriCloud