summaryrefslogtreecommitdiffstats
path: root/sys/kern/subr_uio.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge the fueword(9) and casueword(9). In particular,kib2014-11-181-0/+130
| | | | | | | | | | | | | | | | | | | | | | | MFC r273783: Add fueword(9) and casueword(9) functions. MFC note: ia64 is handled like arm, with NO_FUEWORD define. MFC r273784: Replace some calls to fuword() by fueword() with proper error checking. MFC r273785: Convert kern_umtx.c to use fueword() and casueword(). MFC note: the sys__umtx_lock and sys__umtx_unlock syscalls are not converted, they are removed from HEAD, and not used. The do_sem2*() family is not yet merged to stable/10, corresponding chunk will be merged after do_sem2* are committed. MFC r273788 (by jkim): Actually install casuword(9) to fix build. MFC r273911: Add type qualifier volatile to the base (userspace) address argument of fuword(9) and suword(9).
* MFC r263710, r273377, r273378, r273423 and r273455:hselasky2014-10-271-1/+1
| | | | | | | - De-vnet hash sizes and hash masks. - Fix multiple issues related to arguments passed to SYSCTL macros. Sponsored by: Mellanox Technologies
* Remove zero-copy sockets code. It only worked for anonymous memory,kib2013-09-161-171/+0
| | | | | | | | | | | | | and the equivalent functionality is now provided by sendfile(2) over posix shared memory filedescriptor. Remove the cow member of struct vm_page, and rearrange the remaining members. While there, make hold_count unsigned. Requested and reviewed by: alc Tested by: pho Sponsored by: The FreeBSD Foundation Approved by: re (delphij)
* On all the architectures, avoid to preallocate the physical memoryattilio2013-08-091-1/+7
| | | | | | | | | | | | | | | | | | | | | for nodes used in vm_radix. On architectures supporting direct mapping, also avoid to pre-allocate the KVA for such nodes. In order to do so make the operations derived from vm_radix_insert() to fail and handle all the deriving failure of those. vm_radix-wise introduce a new function called vm_radix_replace(), which can replace a leaf node, already present, with a new one, and take into account the possibility, during vm_radix_insert() allocation, that the operations on the radix trie can recurse. This means that if operations in vm_radix_insert() recursed vm_radix_insert() will start from scratch again. Sponsored by: EMC / Isilon storage division Reviewed by: alc (older version) Reviewed by: jeff Tested by: pho, scottl
* The soft and hard busy mechanism rely on the vm object lock to work.attilio2013-08-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
* Fix r253823. Some WIP patches snuck in.scottl2013-07-301-7/+1
| | | | Submitted by: zont
* Create a knob, kern.ipc.sfreadahead, that allows one to tune the amount ofscottl2013-07-301-1/+7
| | | | | | | readahead that sendfile() will do. Default remains the same. Obtained from: Netflix MFC after: 3 days
* Rename VM_OBJECT_LOCK(), VM_OBJECT_UNLOCK() and VM_OBJECT_TRYLOCK() toattilio2013-02-201-2/+2
| | | | | | their "write" versions. Sponsored by: EMC / Isilon storage division
* Switch vm_object lock to be a rwlock.attilio2013-02-201-1/+1
| | | | | | | | * VM_OBJECT_LOCK and VM_OBJECT_UNLOCK are mapped to write operations * VM_OBJECT_SLEEP() is introduced as a general purpose primitve to get a sleep operation using a VM_OBJECT_LOCK() as protection * The approach must bear with vm_pager.h namespace pollution so many files require including directly rwlock.h
* Reform the busdma API so that new types may be added without modifyingkib2013-02-121-0/+46
| | | | | | | | | | | | | | | | | | | | | every architecture's busdma_machdep.c. It is done by unifying the bus_dmamap_load_buffer() routines so that they may be called from MI code. The MD busdma is then given a chance to do any final processing in the complete() callback. The cam changes unify the bus_dmamap_load* handling in cam drivers. The arm and mips implementations are updated to track virtual addresses for sync(). Previously this was done in a type specific way. Now it is done in a generic way by recording the list of virtuals in the map. Submitted by: jeff (sponsored by EMC/Isilon) Reviewed by: kan (previous version), scottl, mjacob (isp(4), no objections for target mode changes) Discussed with: ian (arm changes) Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris), amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)
* Eliminate superfluous code.davidxu2012-12-061-1/+0
|
* Replace the ill-named ZERO_COPY_SOCKET kernel option with twoandre2012-10-231-5/+5
| | | | | | | | | | | | | | | | | | | | | more appropriate named kernel options for the very distinct send and receive path. "options SOCKET_SEND_COW" enables VM page copy-on-write based sending of data on an outbound socket. NB: The COW based send mechanism is not safe and may result in kernel crashes. "options SOCKET_RECV_PFLIP" enables VM kernel/userspace page flipping for special disposable pages attached as external storage to mbufs. Only the naming of the kernel options is changed and their corresponding #ifdef sections are adjusted. No functionality is added or removed. Discussed with: alc (mechanism and limitations of send side COW)
* After the PHYS_TO_VM_PAGE() function was de-inlined, the main reasonkib2012-08-051-1/+1
| | | | | | | | | | | | | to pull vm_param.h was removed. Other big dependency of vm_page.h on vm_param.h are PA_LOCK* definitions, which are only needed for in-kernel code, because modules use KBI-safe functions to lock the pages. Stop including vm_param.h into vm_page.h. Include vm_param.h explicitely for the kernel code which needs it. Suggested and reviewed by: alc MFC after: 2 weeks
* Handle spurious page faults that may occur in no-fault sections of thealc2012-03-221-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | kernel. When access restrictions are added to a page table entry, we flush the corresponding virtual address mapping from the TLB. In contrast, when access restrictions are removed from a page table entry, we do not flush the virtual address mapping from the TLB. This is exactly as recommended in AMD's documentation. In effect, when access restrictions are removed from a page table entry, AMD's MMUs will transparently refresh a stale TLB entry. In short, this saves us from having to perform potentially costly TLB flushes. In contrast, Intel's MMUs are allowed to generate a spurious page fault based upon the stale TLB entry. Usually, such spurious page faults are handled by vm_fault() without incident. However, when we are executing no-fault sections of the kernel, we are not allowed to execute vm_fault(). This change introduces special-case handling for spurious page faults that occur in no-fault sections of the kernel. In collaboration with: kib Tested by: gibbs (an earlier version) I would also like to acknowledge Hiroki Sato's assistance in diagnosing this problem. MFC after: 1 week
* Fix found places where uio_resid is truncated to int.kib2012-02-211-4/+4
| | | | | | | | | Add the sysctl debug.iosize_max_clamp, enabled by default. Setting the sysctl to zero allows to perform the SSIZE_MAX-sized i/o requests from the usermode. Discussed with: bde, das (previous versions) MFC after: 1 month
* Constify args to copyiniov and copyinuio.alfred2011-11-141-2/+2
|
* Add a facility to disable processing page faults. When activated,kib2011-07-091-10/+53
| | | | | | | | uiomove generates EFAULT if any accessed address is not mapped, as opposed to handling the fault. Sponsored by: The FreeBSD Foundation Reviewed by: alc (previous version)
* Fix the check for vm_map_remove() error.kib2011-03-281-1/+1
| | | | | Pointed out by: alc MFC after: 2 weeks
* Trim white spaces, adjust style.kib2011-03-281-10/+12
| | | | MFC after: 2 weeks
* Handle zero length in copyout_unmap().kib2011-03-281-0/+3
| | | | | Submitted by: John Wehle <john feith com> MFC after: 2 weeks
* Promote ksyms_map() and ksyms_unmap() to general facilitykib2011-03-281-0/+49
| | | | | | | copyout_map() and copyout_unmap() interfaces. Submitted by: John Wehle <john feith com>, nox MFC after: 2 weeks
* Remove the uio_yield prototype and symbol. This function has beenmdf2011-02-081-7/+0
| | | | | | | | | misnamed since it was introduced and should not be globally exposed with this name. The equivalent functionality is now available using kern_yield(curthread->td_user_pri). The function remains undocumented. Bump __FreeBSD_version.
* Based on discussions on the svn-src mailing list, rework r218195:mdf2011-02-081-24/+2
| | | | | | | | | | | | | | | | | | | | | | - entirely eliminate some calls to uio_yeild() as being unnecessary, such as in a sysctl handler. - move should_yield() and maybe_yield() to kern_synch.c and move the prototypes from sys/uio.h to sys/proc.h - add a slightly more generic kern_yield() that can replace the functionality of uio_yield(). - replace source uses of uio_yield() with the functional equivalent, or in some cases do not change the thread priority when switching. - fix a logic inversion bug in vlrureclaim(), pointed out by bde@. - instead of using the per-cpu last switched ticks, use a per thread variable for should_yield(). With PREEMPTION, the only reasonable use of this is to determine if a lock has been held a long time and relinquish it. Without PREEMPTION, this is essentially the same as the per-cpu variable.
* Put the general logic for being a CPU hog into a new functionmdf2011-02-021-6/+17
| | | | | | | | | | should_yield(). Use this in various places. Encapsulate the common case of check-and-yield into a new function maybe_yield(). Change several checks for a magic number of iterations to use should_yield() instead. MFC after: 1 week
* Push down the page queues into vm_page_cache(), vm_page_try_to_cache(), andalc2010-05-081-3/+0
| | | | | | | | | | | vm_page_try_to_free(). Consequently, push down the page queues lock into pmap_enter_quick(), pmap_page_wired_mapped(), pmap_remove_all(), and pmap_remove_write(). Push down the page queues lock into Xen's pmap_page_is_mapped(). (I overlooked the Xen pmap in r207702.) Switch to a per-processor counter for the total number of pages cached.
* Acquire the page lock around all remaining calls to vm_page_free() onalc2010-05-051-0/+2
| | | | | | | | | | | | | managed pages that didn't already have that lock held. (Freeing an unmanaged page, such as the various pmaps use, doesn't require the page lock.) This allows a change in vm_page_remove()'s locking requirements. It now expects the page lock to be held instead of the page queues lock. Consequently, the page queues lock is no longer required at all by callers to vm_page_rename(). Discussed with: kib
* Decompose the most lousy named file in sys/kern; kern_subr.c.ed2010-02-211-0/+478
Although this file has historically been used as a dumping ground for random functions, nowadays it only contains functions related to copying bits {from,to} userspace and hash table utility functions. Behold, subr_uio.c and subr_hash.c.
OpenPOWER on IntegriCloud