summaryrefslogtreecommitdiffstats
path: root/sys/vm/vm_fault.c
Commit message (Collapse)AuthorAgeFilesLines
* Correct a long-standing race condition in vm_fault() that could result in aalc2004-02-151-3/+1
| | | | | | | | | | panic "vm_page_cache: caching a dirty page, ...": Access to the page must be restricted or removed before calling vm_page_cache(). This race condition is identical in nature to that which was addressed by vm_pageout.c's revision 1.251 and vm_page.c's revision 1.275. Reviewed by: tegge MFC after: 7 days
* - Locking for the per-process resource limits structure has eliminatedalc2004-02-051-2/+0
| | | | | | the need for Giant in vm_map_growstack(). - Use the proc * that is passed to vm_map_growstack() rather than curthread->td_proc.
* - Reduce Giant's scope in vm_fault().alc2003-12-261-14/+10
| | | | | - Use vm_object_reference_locked() instead of vm_object_reference() in vm_fault().
* NFC: Update stale comments.mini2003-11-101-1/+1
| | | | Reviewed by: alc
* - vm_fault_copy_entry() should not assume that the source object containsalc2003-10-151-5/+19
| | | | | | | every page. If the source entry was read-only, one or more wired pages could be in backing objects. - vm_fault_copy_entry() should not set the PG_WRITEABLE flag on the page unless the destination entry is, in fact, writeable.
* Lock the destination object in vm_fault_copy_entry().alc2003-10-081-2/+7
|
* Retire vm_page_copy(). Its reason for being ended when peter@ modifiedalc2003-10-081-2/+4
| | | | | | pmap_copy_page() et al. to accept a vm_page_t rather than a physical address. Also, this change will facilitate locking access to the vm page's valid field.
* Synchronize access to a vm page's valid field using the containingalc2003-10-041-4/+8
| | | | vm object's lock.
* Migrate pmap_prefault() into the machine-independent virtual memory layer.alc2003-10-031-1/+91
| | | | | | | A small helper function pmap_is_prefaultable() is added. This function encapsulate the few lines of pmap_prefault() that actually vary from machine to machine. Note: pmap_is_prefaultable() and pmap_mincore() have much in common. Going forward, it's worth considering their merger.
* Add vm object locking to vnode_pager_lock(). (This triggers the movementalc2003-09-181-1/+1
| | | | of a VM_OBJECT_LOCK() in vm_fault().)
* To implement the sequential access optimization, vm_fault() may need toalc2003-08-231-8/+10
| | | | | | | reacquire the "first" object's lock while a backing object's lock is held. Since this is a lock-order reversal, vm_fault() uses trylock to acquire the first object's lock, skipping the sequential access optimization in the unlikely event that the trylock fails.
* Maintain a lock on the vm object of interest throughout vm_fault(),alc2003-06-221-10/+9
| | | | | | releasing the lock only if we are about to sleep (e.g., vm_pager_get_pages() or vm_pager_has_pages()). If we sleep, we have marked the vm object with the paging-in-progress flag.
* As vm_fault() descends the chain of backing objects, set paging-in-alc2003-06-221-8/+8
| | | | progress on the next object before clearing it on the current object.
* Make some style and white-space changes to the copy-on-write path throughalc2003-06-221-10/+5
| | | | vm_fault(); remove a pointless assignment statement from that path.
* Lock one of the vm objects involved in an optimized copy-on-write fault.alc2003-06-211-2/+5
|
* The so-called "optimized copy-on-write fault" case should not requirealc2003-06-201-9/+2
| | | | | | | the vm map lock. What's really needed is vm object locking, which is (for the moment) provided Giant. Reviewed by: tegge
* Fix a vm object reference leak in the page-based copy-on-write mechanismalc2003-06-191-1/+1
| | | | | | used by the zero-copy sockets implementation. Reviewed by: gallatin
* Use __FBSDID().obrien2003-06-111-2/+4
|
* Prefer the proc lock to sched_lock when testing PS_INMEM now that it isjhb2003-04-221-3/+4
| | | | safe to do so.
* - Lock the vm_object when performing vm_object_pip_wakeup().alc2003-04-201-0/+10
|
* - Lock the vm_object when performing vm_object_pip_add().alc2003-04-201-0/+4
|
* - Add vm_paddr_t, a physical address type. This is required for systemsjake2003-03-251-2/+3
| | | | | | | | | | | | | | | where physical addresses larger than virtual addresses, such as i386s with PAE. - Use this to represent physical addresses in the MI vm system and in the i386 pmap code. This also changes the paddr parameter to d_mmap_t. - Fix printf formats to handle physical addresses >4G in the i386 memory detection code, and due to kvtop returning vm_paddr_t instead of u_long. Note that this is a name change only; vm_paddr_t is still the same as vm_offset_t on all currently supported platforms. Sponsored by: DARPA, Network Associates Laboratories Discussed with: re, phk (cdevsw change)
* Zero copy send and receive fixes:ken2003-03-081-2/+9
| | | | | | | | | | | | | | | | | | - On receive, vm_map_lookup() needs to trigger the creation of a shadow object. To make that happen, call vm_map_lookup() with PROT_WRITE instead of PROT_READ in vm_pgmoveco(). - On send, a shadow object will be created by the vm_map_lookup() in vm_fault(), but vm_page_cowfault() will delete the original page from the backing object rather than simply letting the legacy COW mechanism take over. In other words, the new page should be added to the shadow object rather than replacing the old page in the backing object. (i.e. vm_page_cowfault() should not be called in this case.) We accomplish this by making sure fs.object == fs.first_object before calling vm_page_cowfault() in vm_fault(). Submitted by: gallatin, alc Tested by: ken
* Remove ENABLE_VFS_IOOPT. It is a long unfinished work-in-progress.alc2003-03-061-7/+0
| | | | Discussed on: arch@
* Merge all the various copies of vm_fault_quick() into a singledillon2003-01-161-0/+18
| | | | portable copy.
* vm_fault_copy_entry() needn't clear PG_ZERO because it didn't passalc2003-01-121-3/+0
| | | | VM_ALLOC_ZERO to vm_page_alloc().
* Reduce the number of times that we acquire and release the page queuesalc2002-12-291-2/+0
| | | | | lock by making vm_page_rename()'s caller, rather than vm_page_rename(), responsible for acquiring it.
* - Hold the page queues lock around calls to vm_page_flag_clear().alc2002-12-241-0/+2
|
* - Hold the page queues lock when performing vm_page_busy() oralc2002-12-191-1/+7
| | | | | | vm_page_flag_set(). - Replace vm_page_sleep_busy() with proper page queues locking and vm_page_sleep_if_busy().
* Now that pmap_remove_all() is exported by our pmap implementationsalc2002-11-161-2/+2
| | | | use it directly.
* When prot is VM_PROT_NONE, call pmap_page_protect() directly rather thanalc2002-11-101-2/+2
| | | | | | | | | indirectly through vm_page_protect(). The one remaining page flag that is updated by vm_page_protect() is already being updated by our various pmap implementations. Note: A later commit will similarly change the VM_PROT_READ case and eliminate vm_page_protect().
* Complete the page queues locking needed for the page-based copy-alc2002-10-191-2/+2
| | | | | | | | | on-write (COW) mechanism. (This mechanism is used by the zero-copy TCP/IP implementation.) - Extend the scope of the page queues lock in vm_fault() to cover vm_page_cowfault(). - Modify vm_page_cowfault() to release the page queues lock if it sleeps.
* o Retire pmap_pageable(). It's an advisory routine that nonealc2002-08-251-12/+0
| | | | of our platforms implements.
* o Retire vm_page_zero_fill() and vm_page_zero_fill_area(). Ever sincealc2002-08-251-1/+1
| | | | | | pmap_zero_page() and pmap_zero_page_area() were modified to accept a struct vm_page * instead of a physical address, vm_page_zero_fill() and vm_page_zero_fill_area() have served no purpose.
* o Move a call to vm_page_wakeup() inside the scope of the page queues lock.alc2002-08-101-1/+1
|
* o Remove the setting and clearing of the PG_MAPPED flag. (This flag isalc2002-08-101-2/+2
| | | | obsolete.)
* o Lock page queue accesses by vm_page_activate().alc2002-07-271-0/+6
|
* o Merge vm_fault_wire() and vm_fault_user_wire() by adding a new parameter,alc2002-07-241-49/+8
| | | | user_wire.
* o Lock page queue accesses by vm_page_free() and vm_page_deactivate().alc2002-07-211-0/+12
|
* o Lock page queue accesses by vm_page_cache() in vm_fault() andalc2002-07-201-1/+2
| | | | | vm_pageout_scan(). (The others are already locked.) o Assert that the page queues lock is held in vm_page_cache().
* o Lock some page queue accesses, in particular, those by vm_page_unwire().alc2002-07-131-1/+4
|
* At long last, commit the zero copy sockets code.ken2002-06-261-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MAKEDEV: Add MAKEDEV glue for the ti(4) device nodes. ti.4: Update the ti(4) man page to include information on the TI_JUMBO_HDRSPLIT and TI_PRIVATE_JUMBOS kernel options, and also include information about the new character device interface and the associated ioctls. man9/Makefile: Add jumbo.9 and zero_copy.9 man pages and associated links. jumbo.9: New man page describing the jumbo buffer allocator interface and operation. zero_copy.9: New man page describing the general characteristics of the zero copy send and receive code, and what an application author should do to take advantage of the zero copy functionality. NOTES: Add entries for ZERO_COPY_SOCKETS, TI_PRIVATE_JUMBOS, TI_JUMBO_HDRSPLIT, MSIZE, and MCLSHIFT. conf/files: Add uipc_jumbo.c and uipc_cow.c. conf/options: Add the 5 options mentioned above. kern_subr.c: Receive side zero copy implementation. This takes "disposable" pages attached to an mbuf, gives them to a user process, and then recycles the user's page. This is only active when ZERO_COPY_SOCKETS is turned on and the kern.ipc.zero_copy.receive sysctl variable is set to 1. uipc_cow.c: Send side zero copy functions. Takes a page written by the user and maps it copy on write and assigns it kernel virtual address space. Removes copy on write mapping once the buffer has been freed by the network stack. uipc_jumbo.c: Jumbo disposable page allocator code. This allocates (optionally) disposable pages for network drivers that want to give the user the option of doing zero copy receive. uipc_socket.c: Add kern.ipc.zero_copy.{send,receive} sysctls that are enabled if ZERO_COPY_SOCKETS is turned on. Add zero copy send support to sosend() -- pages get mapped into the kernel instead of getting copied if they meet size and alignment restrictions. uipc_syscalls.c:Un-staticize some of the sf* functions so that they can be used elsewhere. (uipc_cow.c) if_media.c: In the SIOCGIFMEDIA ioctl in ifmedia_ioctl(), avoid calling malloc() with M_WAITOK. Return an error if the M_NOWAIT malloc fails. The ti(4) driver and the wi(4) driver, at least, call this with a mutex held. This causes witness warnings for 'ifconfig -a' with a wi(4) or ti(4) board in the system. (I've only verified for ti(4)). ip_output.c: Fragment large datagrams so that each segment contains a multiple of PAGE_SIZE amount of data plus headers. This allows the receiver to potentially do page flipping on receives. if_ti.c: Add zero copy receive support to the ti(4) driver. If TI_PRIVATE_JUMBOS is not defined, it now uses the jumbo(9) buffer allocator for jumbo receive buffers. Add a new character device interface for the ti(4) driver for the new debugging interface. This allows (a patched version of) gdb to talk to the Tigon board and debug the firmware. There are also a few additional debugging ioctls available through this interface. Add header splitting support to the ti(4) driver. Tweak some of the default interrupt coalescing parameters to more useful defaults. Add hooks for supporting transmit flow control, but leave it turned off with a comment describing why it is turned off. if_tireg.h: Change the firmware rev to 12.4.11, since we're really at 12.4.11 plus fixes from 12.4.13. Add defines needed for debugging. Remove the ti_stats structure, it is now defined in sys/tiio.h. ti_fw.h: 12.4.11 firmware. ti_fw2.h: 12.4.11 firmware, plus selected fixes from 12.4.13, and my header splitting patches. Revision 12.4.13 doesn't handle 10/100 negotiation properly. (This firmware is the same as what was in the tree previously, with the addition of header splitting support.) sys/jumbo.h: Jumbo buffer allocator interface. sys/mbuf.h: Add a new external mbuf type, EXT_DISPOSABLE, to indicate that the payload buffer can be thrown away / flipped to a userland process. socketvar.h: Add prototype for socow_setup. tiio.h: ioctl interface to the character portion of the ti(4) driver, plus associated structure/type definitions. uio.h: Change prototype for uiomoveco() so that we'll know whether the source page is disposable. ufs_readwrite.c:Update for new prototype of uiomoveco(). vm_fault.c: In vm_fault(), check to see whether we need to do a page based copy on write fault. vm_object.c: Add a new function, vm_object_allocate_wait(). This does the same thing that vm_object allocate does, except that it gives the caller the opportunity to specify whether it should wait on the uma_zalloc() of the object structre. This allows vm objects to be allocated while holding a mutex. (Without generating WITNESS warnings.) vm_object_allocate() is implemented as a call to vm_object_allocate_wait() with the malloc flag set to M_WAITOK. vm_object.h: Add prototype for vm_object_allocate_wait(). vm_page.c: Add page-based copy on write setup, clear and fault routines. vm_page.h: Add page based COW function prototypes and variable in the vm_page structure. Many thanks to Drew Gallatin, who wrote the zero copy send and receive code, and to all the other folks who have tested and reviewed this code over the years.
* o Remove GIANT_REQUIRED from vm_fault_user_wire().alc2002-06-161-5/+1
| | | | | | o Move pmap_pageable() outside of Giant in vm_fault_unwire(). (pmap_pageable() is a no-op on all supported architectures.) o Remove the acquisition and release of Giant from mlock().
* o Acquire and release Giant around pmap operations in vm_fault_unwire()alc2002-05-261-1/+2
| | | | | | | and vm_map_delete(). Assert GIANT_REQUIRED in vm_map_delete() only if operating on the kernel_object or the kmem_object. o Remove GIANT_REQUIRED from vm_map_remove(). o Remove the acquisition and release of Giant from munmap().
* o Condition the compilation and use of vm_freeze_copyopts()alc2002-05-061-1/+2
| | | | on ENABLE_VFS_IOOPT.
* o Revert vm_fault1() to its original name vm_fault(), eliminating the wrapperalc2002-04-301-16/+11
| | | | that took its place for the purposes of acquiring and releasing Giant.
* Document three synchronization issues in vm_fault().alc2002-04-291-0/+8
|
* o Introduce and use vm_map_trylock() to replace several direct usesalc2002-04-281-3/+1
| | | | | | of lockmgr(). o Add missing synchronization to vmspace_swap_count(): Obtain a read lock on the vm_map before traversing it.
* o Move the acquisition of Giant from vm_fault() to the pointalc2002-04-191-12/+8
| | | | | after initialization in vm_fault1(). o Fix some style problems in vm_fault1().
* Add a comment documenting a race condition in vm_fault(): Specifically, aalc2002-04-181-0/+3
| | | | modification is made to the vm_map while only a read lock is held.
OpenPOWER on IntegriCloud