summaryrefslogtreecommitdiffstats
path: root/sys/vm
Commit message (Collapse)AuthorAgeFilesLines
...
* Introduce vm_reserv_reclaim_contig(). This function is used byalc2008-04-064-25/+99
| | | | | | | | | | contigmalloc(9) as a last resort to steal pages from an inactive, partially-used superpage reservation. Rename vm_reserv_reclaim() to vm_reserv_reclaim_inactive() and refactor it so that a separate subroutine is responsible for breaking the selected reservation. This subroutine is also used by vm_reserv_reclaim_contig().
* Eliminate an unnecessary test from vm_phys_unfree_page().alc2008-04-051-1/+1
|
* Update a comment to vm_map_pmap_enter().alc2008-04-041-2/+2
|
* Reintroduce UMA_SLAB_KMAP; however, change its spelling toalc2008-04-042-0/+5
| | | | | | | | | | | | | | | | | | | UMA_SLAB_KERNEL for consistency with its sibling UMA_SLAB_KMEM. (UMA_SLAB_KMAP met its original demise in revision 1.30 of vm/uma_core.c.) UMA_SLAB_KERNEL is now required by the jumbo frame allocators. Without it, UMA cannot correctly return pages from the jumbo frame zones to the VM system because it resets the pages' object field to NULL instead of the kernel object. In more detail, the jumbo frame zones are created with the option UMA_ZONE_REFCNT. This causes UMA to overwrite the pages' object field with the address of the slab. However, when UMA wants to release these pages, it doesn't know how to restore the object field, so it sets it to NULL. This change teaches UMA how to reset the object field to the kernel object. Crashes reported by: kris Fix tested by: kris Fix discussed with: jeff MFC after: 6 weeks
* Eliminate an unnecessary printf() from kmem_suballoc(). The subsequentalc2008-03-301-4/+2
| | | | panic() can be extended to convey the same information.
* - Use vm_object_reference_locked() directly fromjeff2008-03-291-15/+2
| | | | | | | | | vm_object_reference(). This is intended to get rid of vget() consumers who don't wish to acquire a lock. This is functionally the same as calling vref(). vm_object_reference_locked() already uses vref. Discussed with: alc
* Do not dereference cdev->si_cdevsw, use the dev_refthread() to properlykib2008-03-201-4/+16
| | | | | | | | obtain the reference. In particular, this fixes the panic reported in the PR. Remove the comments stating that this needs to be done. PR: kern/119422 MFC after: 1 week
* Rename vm_pageq_requeue() to vm_page_requeue() on account of its recentalc2008-03-193-15/+15
| | | | migration to vm/vm_page.c.
* - Relax requirements for p_numthreads, p_threads, p_swtick, and p_nice fromjeff2008-03-193-22/+8
| | | | | | | requiring the per-process spinlock to only requiring the process lock. - Reflect these changes in the proc.h documentation and consumers throughout the kernel. This is a substantial reduction in locking cost for these fields and was made possible by recent changes to threading support.
* Almost seven years ago, vm/vm_page.c was split into three parts:alc2008-03-183-124/+75
| | | | | | | | | | | | vm/vm_contig.c, vm/vm_page.c, and vm/vm_pageq.c. Today, vm/vm_pageq.c has withered to the point that it contains only four short functions, two of which are only used by vm/vm_page.c. Since I can't foresee any reason for vm/vm_pageq.c to grow, it is time to fold the remaining contents of vm/vm_pageq.c back into vm/vm_page.c. Add some comments. Rename one of the functions, vm_pageq_enqueue(), that is now static within vm/vm_page.c to vm_page_enqueue(). Eliminate PQ_MAXCOUNT as it no longer serves any purpose.
* Simplify the inner loop of vm_fault()'s delete-behind heuristic.alc2008-03-161-2/+2
| | | | | | Instead of checking each page for PG_UNMANAGED, perform a one-time check whether the object is OBJT_PHYS. (PG_UNMANAGED pages only belong to OBJT_PHYS objects.)
* In keeping with style(9)'s recommendations on macros, use a ';'rwatson2008-03-165-7/+9
| | | | | | | | | after each SYSINIT() macro invocation. This makes a number of lightweight C parsers much happier with the FreeBSD kernel source, including cflow's prcc and lxr. MFC after: 1 month Discussed with: imp, rink
* Remove kernel support for M:N threading.jeff2008-03-122-4/+2
| | | | | | | | While the KSE project was quite successful in bringing threading to FreeBSD, the M:N approach taken by the kse library was never developed to its full potential. Backwards compatibility will be provided via libmap.conf for dynamically linked binaries and static binaries will be broken.
* - Pass the priority argument from *sleep() into sleepq and down intojeff2008-03-122-18/+7
| | | | | | | | | | | | | | | | | sched_sleep(). This removes extra thread_lock() acquisition and allows the scheduler to decide what to do with the static boost. - Change the priority arguments to cv_* to match sleepq/msleep/etc. where 0 means no priority change. Catch -1 in cv_broadcastpri() and convert it to 0 for now. - Set a flag when sleeping in a way that is compatible with swapping since direct priority comparisons are meaningless now. - Add a sysctl to ule, kern.sched.static_boost, that defaults to on which controls the boost behavior. Turning it off gives better performance in some workloads but needs more investigation. - While we're modifying sleepq, change signal and broadcast to both return with the lock held as the lock was held on enter. Reviewed by: jhb, peter
* Eliminate an unnecessary test from vm_fault's delete-behind heuristic.alc2008-03-091-1/+1
| | | | | | | Specifically, since the delete-behind heuristic is never applied to a device-backed object, there is no point in checking whether each of the object's pages is fictitious. (Only device-backed objects have fictitious pages.)
* Make the vm_pmap field of struct vmspace the last field in themarcel2008-03-011-1/+6
| | | | | | | structure. This allows per-CPU variations of struct pmap on a single architecture without affecting the machine-independent fields. As such, the PMAP variations don't affect the ABI. They become part of it.
* Correct a long-standing error in vm_object_page_remove(). Specifically,alc2008-02-261-6/+25
| | | | | | | | | | | | pmap_remove_all() must not be called on fictitious pages. To date, fictitious pages have been allocated from zeroed memory, effectively hiding this problem because the fictitious pages appear to have an empty pv list. Submitted by: Kostik Belousov Rewrite the comments describing vm_object_page_remove() to better describe what it does. Add an assertion. Reviewed by: Kostik Belousov MFC after: 1 week
* Correct a long-standing error in vm_object_deallocate(). Specifically,alc2008-02-241-1/+4
| | | | | | | | | | | | | | | | | | | | only anonymous default (OBJT_DEFAULT) and swap (OBJT_SWAP) objects should ever have OBJ_ONEMAPPING set. However, vm_object_deallocate() was setting it on device (OBJT_DEVICE) objects. As a result, vm_object_page_remove() could be called on a device object and if that occurred pmap_remove_all() would be called on the device object's pages. However, a device object's pages are fictitious, and fictitious pages do not have an initialized pv list (struct md_page). To date, fictitious pages have been allocated from zeroed memory, effectively hiding this problem. Now, however, the conversion of rotting diagnostics to invariants in the amd64 and i386 pmaps has revealed the problem. Specifically, assertion failures have occurred during the initialization phase of the X server on some hardware. MFC after: 1 week Discussed with: Kostik Belousov Reported by: Michiel Boland
* VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used inattilio2008-01-134-6/+6
| | | | | | | | | | | conjuction with 'thread' argument passing which is always curthread. Remove the unuseful extra-argument and pass explicitly curthread to lower layer functions, when necessary. KPI results broken by this change, which should affect several ports, so version bumping and manpage update will be further committed. Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>
* When one tries to allocate memory with the M_WAITOK flag and we are short inpjd2008-01-101-6/+13
| | | | | | | | | | | | address space in kmem map call vm_lowmem event in a loop and wait a bit for subsystems to reclaim some memory which in turn will reclaim address space as well. Note, this is a work-around. Reviewed by: alc Approved by: alc MFC after: 3 days
* vn_lock() is currently only used with the 'curthread' passed as argument.attilio2008-01-104-6/+6
| | | | | | | | | | | | | | | | Remove this argument and pass curthread directly to underlying VOP_LOCK1() VFS method. This modify makes the code cleaner and in particular remove an annoying dependence helping next lockmgr() cleanup. KPI results, obviously, changed. Manpage and FreeBSD_version will be updated through further commits. As a side note, would be valuable to say that next commits will address a similar cleanup about VFS methods, in particular vop_lock1 and vop_unlock. Tested by: Diego Sardina <siarodx at gmail dot com>, Andrea Di Pasquale <whyx dot it at gmail dot com>
* Add a new file descriptor type for IPC shared memory objects and use it tojhb2008-01-081-3/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | implement shm_open(2) and shm_unlink(2) in the kernel: - Each shared memory file descriptor is associated with a swap-backed vm object which provides the backing store. Each descriptor starts off with a size of zero, but the size can be altered via ftruncate(2). The shared memory file descriptors also support fstat(2). read(2), write(2), ioctl(2), select(2), poll(2), and kevent(2) are not supported on shared memory file descriptors. - shm_open(2) and shm_unlink(2) are now implemented as system calls that manage shared memory file descriptors. The virtual namespace that maps pathnames to shared memory file descriptors is implemented as a hash table where the hash key is generated via the 32-bit Fowler/Noll/Vo hash of the pathname. - As an extension, the constant 'SHM_ANON' may be specified in place of the path argument to shm_open(2). In this case, an unnamed shared memory file descriptor will be created similar to the IPC_PRIVATE key for shmget(2). Note that the shared memory object can still be shared among processes by sharing the file descriptor via fork(2) or sendmsg(2), but it is unnamed. This effectively serves to implement the getmemfd() idea bandied about the lists several times over the years. - The backing store for shared memory file descriptors are garbage collected when they are not referenced by any open file descriptors or the shm_open(2) virtual namespace. Submitted by: dillon, peter (previous versions) Submitted by: rwatson (I based this on his version) Reviewed by: alc (suggested converting getmemfd() to shm_open())
* When MAC is enabled in the kernel, fix a panic triggered by a lockingcsjp2008-01-081-8/+8
| | | | | | | | | | | | | | assertion hit in swapoff_one() when we un-mount a swap partition. We should be using curthread where we used thread0 before. This change also replaces the thread argument with a credential argument, as the MAC framework only requires the cred. It should be noted that this allows the machine to be rebooted without panicing with "cannot differ from curthread or NULL" when MAC is enabled. Submitted by: rwatson Reviewed by: attilio MFC after: 2 weeks
* In the vm_map_stack(), check for the specified stack region wraparound.kib2008-01-041-1/+3
| | | | | | Reported and tested by: Peter Holm Reviewed by: alc MFC after: 3 days
* Add an access type parameter to pmap_enter(). It will be used to implementalc2008-01-033-6/+8
| | | | | | | superpage promotion. Correct a style error in kmem_malloc(): pmap_enter()'s last parameter is a Boolean.
* Defer setting either PG_CACHED or PG_FREE until after the free pagealc2008-01-021-2/+2
| | | | | | | | queues lock is acquired. Otherwise, the state of a reservation's pages' flags and its population count can be inconsistent. That could result in a page being freed twice. Reported by: kris
* Correct a style error that was introduced in revision 1.77.alc2008-01-011-1/+1
|
* Add the superpage reservation system. This is "part 2 of 2" of thealc2007-12-295-4/+834
| | | | | | | | | | | | machine-independent support for superpages. (The earlier part was the rewrite of the physical memory allocator.) The remainder of the code required for superpages support is machine-dependent and will be added to the various pmap implementations at a later date. Initially, I am only supporting one large page size per architecture. Moreover, I am only enabling the reservation system on amd64. (In an emergency, it can be disabled by setting VM_NRESERVLEVELS to 0 in amd64/include/vmparam.h or your kernel configuration file.)
* Add a list of reservations to the vm object structure.alc2007-12-271-0/+2
| | | | | | | | Recycle the vm object's "pg_color" field to represent the color of the first virtual page address at which the object is mapped instead of the color of the object's first physical page. Since an object may not be mapped, introduce a flag "OBJ_COLORED" that indicates whether "pg_color" is valid.
* Add the superpage reservation type.alc2007-12-271-0/+3
|
* Update the comment describing vm_phys_unfree_page().alc2007-12-211-1/+3
|
* Modify vm_phys_unfree_page() so that it no longer requires the givenalc2007-12-203-10/+19
| | | | | | | | | | page to be in the free lists. Instead, it now returns TRUE if it removed the page from the free lists and FALSE if the page was not in the free lists. This change is required to support superpage reservations. Specifically, once reservations are introduced, a cached page can either be in the free lists or a reservation.
* Correct one half of a loop continuation condition in vm_phys_unfree_page().alc2007-12-191-3/+1
| | | | | At present, this error is inconsequential; the other half of the loop continuation condition is sufficient to achieve correct execution.
* Eliminate redundant code from vm_page_startup().alc2007-12-191-17/+0
|
* Simplify vm_page_free_toq().alc2007-12-111-5/+3
|
* Correct a comment.alc2007-12-021-1/+1
|
* Modify stack(9) stack_print() and stack_sbuf_print() routines to use newrwatson2007-12-011-4/+4
| | | | | | | | | | | | | | | | linker interfaces for looking up function names and offsets from instruction pointers. Create two variants of each call: one that is "DDB-safe" and avoids locking in the linker, and one that is safe for use in live kernels, by virtue of observing locking, and in particular safe when kernel modules are being loaded and unloaded simultaneous to their use. This will allow them to be used outside of debugging contexts. Modify two of three current stack(9) consumers to use the DDB-safe interfaces, as they run in low-level debugging contexts, such as inside lockmgr(9) and the kernel memory allocator. Update man page.
* Make contigmalloc(9)'s page laundering more robust. Specifically, usealc2007-11-253-4/+8
| | | | | | | | | vm_pageout_fallback_object_lock() in vm_contig_launder_page() to better handle a lock-ordering problem. Consequently, trylock's failure on the page's containing object no longer implies that the page cannot be laundered. MFC after: 6 weeks
* Tidy up: Add comments. Eliminate the pointlessalc2007-11-251-19/+23
| | | | | | | malloc_type_allocated(..., 0) calls that occur when contigmalloc() has failed. Eliminate the acquisition and release of the page queues lock from vm_page_release_contig(). Rename contigmalloc2() to contigmapping(), reflecting what it does.
* Add a read/write sysctl for reconfiguring the maximum number of physicalalc2007-11-231-0/+2
| | | | | | | | pages that can be wired. Submitted by: Eugene Grosbein PR: 114654 MFC after: 6 weeks
* Remove an unnecessary call to pmap_remove_all() and the associated "XXX"alc2007-11-221-15/+1
| | | | | | | | comments from vnode_pager_setsize(). This call was introduced in revision 1.140 to address a problem that no longer exists. Specifically, pmap_zero_page_area() has replaced a (possibly) problematic implementation of page zeroing that was based on vm_pager_map(), bzero(), and vm_pager_unmap().
* When reactivating a cached page, reset the page's pool to the defaultalc2007-11-211-0/+1
| | | | | pool. (Not doing this before was a performance pessimization but not a cause for panic.)
* Prevent the leakage of wired pages in the following circumstances:alc2007-11-172-1/+14
| | | | | | | | | | | | | | | | | | | | | | First, a file is mmap(2)ed and then mlock(2)ed. Later, it is truncated. Under "normal" circumstances, i.e., when the file is not mlock(2)ed, the pages beyond the EOF are unmapped and freed. However, when the file is mlock(2)ed, the pages beyond the EOF are unmapped but not freed because they have a non-zero wire count. This can be a mistake. Specifically, it is a mistake if the sole reason why the pages are wired is because of wired, managed mappings. Previously, unmapping the pages destroys these wired, managed mappings, but does not reduce the pages' wire count. Consequently, when the file is unmapped, the pages are not unwired because the wired mapping has been destroyed. Moreover, when the vm object is finally destroyed, the pages are leaked because they are still wired. The fix is to reduce the pages' wired count by the number of wired, managed mappings destroyed. To do this, I introduce a new pmap function pmap_page_wired_mappings() that returns the number of managed mappings to the given physical page that are wired, and I use this function in vm_object_page_remove(). Reviewed by: tegge MFC after: 6 weeks
* Change unused 'user_wait' argument to 'timo' argument, which will bepjd2007-11-073-7/+7
| | | | | | | used to specify timeout for msleep(9). Discussed with: alc Reviewed by: alc
* Fix for the panic("vm_thread_new: kstack allocation failed") andkib2007-11-054-23/+51
| | | | | | | | | | | | | | | | | | | | silent NULL pointer dereference in the i386 and sparc64 pmap_pinit() when the kmem_alloc_nofault() failed to allocate address space. Both functions now return error instead of panicing or dereferencing NULL. As consequence, vmspace_exec() and vmspace_unshare() returns the errno int. struct vmspace arg was added to vm_forkproc() to avoid dealing with failed allocation when most of the fork1() job is already done. The kernel stack for the thread is now set up in the thread_alloc(), that itself may return NULL. Also, allocation of the first process thread is performed in the fork1() to properly deal with stack allocation failure. proc_linkup() is separated into proc_linkup() called from fork1(), and proc_linkup0(), that is used to set up the kernel process (was known as swapper). In collaboration with: Peter Holm Reviewed by: jhb
* The intent of the freeing the (zeroed) page in vm_page_cache() forkib2007-11-051-2/+5
| | | | | | | | | | | | | | | | | default object rather than cache it was to have vm_pager_has_page(object, pindex, ...) == FALSE to imply that there is no cached page in object at pindex. This allows to avoid explicit checks for cached pages in vm_object_backing_scan(). For now, we need the same bandaid for the swap object, otherwise both the vm_page_lookup() and the pager can report that there is no page at offset, while page is stored in the cache. Also, this fixes another instance of the KASSERT("object type is incompatible") failure in the vm_page_cache_transfer(). Reported and tested by: Peter Holm Reviewed by: alc MFC after: 3 days
* o Fix panic message: it's swap_pager_putpages() not swap_pager_getpages().maxim2007-11-021-1/+1
| | | | Submitted by: Mark Tinguely
* Correct a copy and paste'o in phys_pager.c, we are talking about phys hereremko2007-10-301-1/+1
| | | | | | | and not about devices. PR: 93755 Approved by: imp (mentor, implicit when re-assigning the ticket to me).
* Change vm_page_cache_transfer() such that it does not transfer pagesalc2007-10-271-10/+20
| | | | | | | | | that would have an offset beyond the end of the target object. Such pages should remain in the source object. MFC after: 3 days Diagnosed and reviewed by: Kostik Belousov Reported and tested by: Peter Holm
* Merge first in a series of TrustedBSD MAC Framework KPI changesrwatson2007-10-242-3/+3
| | | | | | | | | | | | | | | | | | | | | | | from Mac OS X Leopard--rationalize naming for entry points to the following general forms: mac_<object>_<method/action> mac_<object>_check_<method/action> The previous naming scheme was inconsistent and mostly reversed from the new scheme. Also, make object types more consistent and remove spaces from object types that contain multiple parts ("posix_sem" -> "posixsem") to make mechanical parsing easier. Introduce a new "netinet" object type for certain IPv4/IPv6-related methods. Also simplify, slightly, some entry point names. All MAC policy modules will need to be recompiled, and modules not updates as part of this commit will need to be modified to conform to the new KPI. Sponsored by: SPARTA (original patches against Mac OS X) Obtained from: TrustedBSD Project, Apple Computer
OpenPOWER on IntegriCloud