summaryrefslogtreecommitdiffstats
path: root/sys/kern/sys_pipe.c
Commit message (Collapse)AuthorAgeFilesLines
* Rename st_*timespec fields to st_*tim for POSIX 2008 compliance.ed2010-03-281-3/+3
| | | | | | | | | | | | | | | A nice thing about POSIX 2008 is that it finally standardizes a way to obtain file access/modification/change times in sub-second precision, namely using struct timespec, which we already have for a very long time. Unfortunately POSIX uses different names. This commit adds compatibility macros, so existing code should still build properly. Also change all source code in the kernel to work without any of the compatibility macros. This makes it all a less ambiguous. I am also renaming st_birthtime to st_birthtim, even though it was a local extension anyway. It seems Cygwin also has a st_birthtim.
* Use C99 initialization for struct filterops.rwatson2009-09-121-4/+10
| | | | | | Obtained from: Mac OS X Sponsored by: Apple Inc. MFC after: 3 weeks
* Fix poll(2) and select(2) for named pipes to return "ready for read"kib2009-07-071-6/+9
| | | | | | | | | | | | | | | | | when all writers, observed by reader, exited. Use writer generation counter for fifo, and store the snapshot of the fifo generation in the f_seqcount field of struct file, that is otherwise unused for fifos. Set FreeBSD-undocumented POLLINIGNEOF flag only when file f_seqcount is equal to fifo' fi_wgen, and revert r89376. Fix POLLINIGNEOF for sockets and pipes, and return POLLHUP for them. Note that the patch does not fix not returning POLLHUP for fifos. PR: kern/94772 Submitted by: bde (original version) Reviewed by: rwatson, jilles Approved by: re (kensmith) MFC after: 6 weeks (might be)
* Adapt vfs kqfilter to the shared vnode lock used by zfs write vop. Usekib2009-06-101-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | vnode interlock to protect the knote fields [1]. The locking assumes that shared vnode lock is held, thus we get exclusive access to knote either by exclusive vnode lock protection, or by shared vnode lock + vnode interlock. Do not use kl_locked() method to assert either lock ownership or the fact that curthread does not own the lock. For shared locks, ownership is not recorded, e.g. VOP_ISLOCKED can return LK_SHARED for the shared lock not owned by curthread, causing false positives in kqueue subsystem assertions about knlist lock. Remove kl_locked method from knlist lock vector, and add two separate assertion methods kl_assert_locked and kl_assert_unlocked, that are supposed to use proper asserts. Change knlist_init accordingly. Add convenience function knlist_init_mtx to reduce number of arguments for typical knlist initialization. Submitted by: jhb [1] Noted by: jhb [2] Reviewed by: jhb Tested by: rnoland
* Prevent integer overflow in direct pipe write code from circumventingcperciva2009-06-101-0/+2
| | | | | | | | | | | | | | virtual-to-physical page lookups. [09:09] Add missing permissions check for SIOCSIFINFO_IN6 ioctl. [09:10] Fix buffer overflow in "autokey" negotiation in ntpd(8). [09:11] Approved by: so (cperciva) Approved by: re (not really, but SVN wants this...) Security: FreeBSD-SA-09:09.pipe Security: FreeBSD-SA-09:10.ipv6 Security: FreeBSD-SA-09:11.ntpd
* Move "options MAC" from opt_mac.h to opt_global.h, as it's now in GENERICrwatson2009-06-051-2/+0
| | | | | | | | and used in a large number of files, but also because an increasing number of incorrect uses of MAC calls were sneaking in due to copy-and-paste of MAC-aware code without the associated opt_mac.h include. Discussed with: pjd
* - Make maxpipekva a signed long rather than an unsigned long as overflowjhb2009-03-101-5/+5
| | | | | | | is more likely to be noticed with signed types. - Make amountpipekva a long as well to match maxpipekva. Discussed with: bde
* Adjust some variables (mostly related to the buffer cache) that holdjhb2009-03-091-1/+1
| | | | | | | | | | | | | | | | | | | address space sizes to be longs instead of ints. Specifically, the follow values are now longs: runningbufspace, bufspace, maxbufspace, bufmallocspace, maxbufmallocspace, lobufspace, hibufspace, lorunningspace, hirunningspace, maxswzone, maxbcache, and maxpipekva. Previously, a relatively small number (~ 44000) of buffers set in kern.nbuf would result in integer overflows resulting either in hangs or bogus values of hidirtybuffers and lodirtybuffers. Now one has to overflow a long to see such problems. There was a check for a nbuf setting that would cause overflows in the auto-tuning of nbuf. I've changed it to always check and cap nbuf but warn if a user-supplied tunable would cause overflow. Note that this changes the ABI of several sysctls that are used by things like top(1), etc., so any MFC would probably require a some gross shims to allow for that. MFC after: 1 month
* Several cleanups related to pipe(2).ed2008-11-111-9/+22
| | | | | | | | | | | | | | | | | | - Use `fildes[2]' instead of `*fildes' to make more clear that pipe(2) fills an array with two descriptors. - Remove EFAULT from the manual page. Because of the current calling convention, pipe(2) raises a segmentation fault when an invalid address is passed. - Introduce kern_pipe() to make it easier for binary emulations to implement pipe(2). - Make Linux binary emulation use kern_pipe(), which means we don't have to recover td_retval after calling the FreeBSD system call. Approved by: rdivacky Discussed on: arch
* Another problem caused by the knlist_cleardel() potentially droppingkib2008-05-231-11/+22
| | | | | | | | | | | | | | | | | | | PIPE_MTX(). Since the pipe_present is cleared before (potentially) sleeping, the second thread may enter the pipeclose() for the reciprocal pipe end. The test at the end of the pipeclose() for the pipe_present == 0 would succeed, allowing the second thread to free the pipe memory. First threads then accesses the freed memory after being woken up. Properly track the closing state of the pipe in the pipe_present. Introduce the intermediate state that marks the pipe as mostly dismantled but might be sleeping waiting for the knote list to be cleared. Free the pipe pair memory only when both ends pass that point. Debugging help and tested by: pho Discussed with: jmg MFC after: 2 weeks
* Destruction of the pipe calls knlist_cleardel() to remove the knoteskib2008-05-231-6/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | monitoring the pipe. The code sets pipe_present = 0 and enters knlist_cleardel(), where the PIPE_MTX might be dropped when knl->kl_list cannot be cleared due to influx knotes. If the following often encountered code fragment if (!(kn->kn_status & KN_DETACHED)) kn->kn_fop->f_detach(kn); knote_drop(kn, td); [1] is executed while the knlist lock is dropped, then the knote memory is freed by the knote_drop() without knote being removed from the knlist, since the filt_pipedetach() contains the following: if (kn->kn_filter == EVFILT_WRITE) { if (!cpipe->pipe_peer->pipe_present) { PIPE_UNLOCK(cpipe); return; Now, the memory may be reused in the zone, causing the access to the freed memory. I got the panics caused by the marker knote appearing on the knlist, that, I believe, manifestation of the issue. In the Peter Holm test scenarious, we got unkillable processes too. The pipe_peer that has the knote for write shall be present. Ignore the pipe_present value for EVFILT_WRITE in filt_pipedetach(). Debugging help and tested by: pho Discussed with: jmg MFC after: 2 weeks
* Make ftruncate a 'struct file' operation rather than a vnode operation.jhb2008-01-071-0/+14
| | | | | | | | | | | | | | This makes it possible to support ftruncate() on non-vnode file types in the future. - 'struct fileops' grows a 'fo_truncate' method to handle an ftruncate() on a given file descriptor. - ftruncate() moves to kern/sys_generic.c and now just fetches a file object and invokes fo_truncate(). - The vnode-specific portions of ftruncate() move to vn_truncate() in vfs_vnops.c which implements fo_truncate() for vnode file types. - Non-vnode file types return EINVAL in their fo_truncate() method. Submitted by: rwatson
* Remove explicit locking of struct file.jeff2007-12-301-12/+2
| | | | | | | | | | | | | - Introduce a finit() which is used to initailize the fields of struct file in such a way that the ops vector is only valid after the data, type, and flags are valid. - Protect f_flag and f_count with atomic operations. - Remove the global list of all files and associated accounting. - Rewrite the unp garbage collection such that it no longer requires the global list of all files and instead uses a list of all unp sockets. - Mark sockets in the accept queue so we don't incorrectly gc them. Tested by: kris, pho
* Refactor select to reduce contention and hide internal implementationjeff2007-12-161-3/+6
| | | | | | | | | | | | | | | | | | | | | details from consumers. - Track individual selecters on a per-descriptor basis such that there are no longer collisions and after sleeping for events only those descriptors which triggered events must be rescaned. - Protect the selinfo (per descriptor) structure with a mtx pool mutex. mtx pool mutexes were chosen to preserve api compatibility with existing code which does nothing but bzero() to setup selinfo structures. - Use a per-thread wait channel rather than a global wait channel. - Hide select implementation details in a seltd structure which is opaque to the rest of the kernel. - Provide a 'selsocket' interface for those kernel consumers who wish to select on a socket when they have no fd so they no longer have to be aware of select implementation details. Tested by: kris Reviewed on: arch
* The kernel uses two ways to write data on a pipe:dumbbell2007-11-191-0/+4
| | | | | | | | | | | | | | | | | | | | o buffered write, for chunks smaller than PIPE_MINDIRECT bytes o direct write, for everything else A call to writev(2) may receive struct iov of various size and the kernel may have to switch from one solution to the other. Before doing this, it must wake reader processes and any select/poll/kqueue up. This commit fixes a bug where select/poll/kqueue are not triggered when switching from buffered write to direct write. It adds calls to pipeselwakeup(). I give more details on freebsd-arch@: http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html This should fix issues with Erlang (lang/erlang) and kqueue. Reported by: Rickard Green (Erlang)
* Merge first in a series of TrustedBSD MAC Framework KPI changesrwatson2007-10-241-9/+9
| | | | | | | | | | | | | | | | | | | | | | | from Mac OS X Leopard--rationalize naming for entry points to the following general forms: mac_<object>_<method/action> mac_<object>_check_<method/action> The previous naming scheme was inconsistent and mostly reversed from the new scheme. Also, make object types more consistent and remove spaces from object types that contain multiple parts ("posix_sem" -> "posixsem") to make mechanical parsing easier. Introduce a new "netinet" object type for certain IPv4/IPv6-related methods. Also simplify, slightly, some entry point names. All MAC policy modules will need to be recompiled, and modules not updates as part of this commit will need to be modified to conform to the new KPI. Sponsored by: SPARTA (original patches against Mac OS X) Obtained from: TrustedBSD Project, Apple Computer
* Remove amountpipes counter for pipes -- this replicates the function ofrwatson2007-05-271-19/+2
| | | | | | existing UMA statistics for pipes, and allows us to get rid of both the per-pipe dtor and two atomic operations per pipe required to maintain the counter.
* Further system call comment cleanup:rwatson2007-03-051-3/+2
| | | | | | | | | | - Remove also "MP SAFE" after prior "MPSAFE" pass. (suggested by bde) - Remove extra blank lines in some cases. - Add extra blank lines in some cases. - Remove no-op comments consisting solely of the function name, the word "syscall", or the system call name. - Add punctuation. - Re-wrap some comments.
* Use pipe_direct_write() optimization only if the data is in process' memory.pjd2006-12-191-2/+3
| | | | | | This fixes sending data through pipe from the kernel. Fix suggested by: rwatson
* Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.hrwatson2006-10-221-1/+2
| | | | | | | | | | | | | begun with a repo-copy of mac.h to mac_framework.h. sys/mac.h now contains the userspace and user<->kernel API and definitions, with all in-kernel interfaces moved to mac_framework.h, which is now included across most of the kernel instead. This change is the first step in a larger cleanup and sweep of MAC Framework interfaces in the kernel, and will not be MFC'd. Obtained from: TrustedBSD Project Sponsored by: SPARTA
* Move some functions and definitions from uipc_socket2.c to uipc_socket.c:rwatson2006-06-101-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | - Move sonewconn(), which creates new sockets for incoming connections on listen sockets, so that all socket allocate code is together in uipc_socket.c. - Move 'maxsockets' and associated sysctls to uipc_socket.c with the socket allocation code. - Move kern.ipc sysctl node to uipc_socket.c, add a SYSCTL_DECL() for it to sysctl.h and remove lots of scattered implementations in various IPC modules. - Sort sodealloc() after soalloc() in uipc_socket.c for dependency order reasons. Statisticize soalloc() and sodealloc() as they are now required only in uipc_socket.c, and are internal to the socket implementation. After this change, socket allocation and deallocation is entirely centralized in one file, and uipc_socket2.c consists entirely of socket buffer manipulation and default protocol switch functions. MFC after: 1 month
* - In pipe() return the error returned by pipe_create(), rather thenglebius2006-01-301-2/+3
| | | | | | | | hardcoded ENFILES, which is incorrect. pipe_create() can fail due to ENOMEM. - Update manual page, describing ENOMEM return code. Reviewed by: arch
* In pipe_write(): when uiomove() fails, do not spin on it forever.delphij2005-12-161-0/+2
| | | | | | Submitted by: Kostik Belousov <kostikbel at gmail.com> on -current@ Message-ID: <20051216151016.GE84442@deviant.zoral.local> MFC After: 3 weeks
* Fix the recent panics/LORs/hangs created by my kqueue commit by:ssouhlal2005-07-011-2/+4
| | | | | | | | | | | | | | | | | - Introducing the possibility of using locks different than mutexes for the knlist locking. In order to do this, we add three arguments to knlist_init() to specify the functions to use to lock, unlock and check if the lock is owned. If these arguments are NULL, we assume mtx_lock, mtx_unlock and mtx_owned, respectively. - Using the vnode lock for the knlist locking, when doing kqueue operations on a vnode. This way, we don't have to lock the vnode while holding a mutex, in filt_vfsread. Reviewed by: jmg Approved by: re (scottl), scottl (mentor override) Pointyhat to: ssouhlal Will be happy: everyone
* Rearrange the kninit calls for both directions of a pipe so thatsilby2005-01-171-1/+3
| | | | | | | | | they both happen before pipe backing allocation occurs. Previously, a pipe memory shortage would cause a panic due to a KNOTE call on an uninitialized si_note. Reported by: Peter Holm MFC after: 1 week
* /* -> /*- for copyright notices, minor format tweaks as necessaryimp2005-01-061-1/+1
|
* Correct a bug introduced in sys_pipe.c:1.179: in pipe_ioctl(),rwatson2004-11-231-2/+5
| | | | | | | | | release the pipe mutex before calling fsetown(), as fsetown() may block. The sigio code protects the pipe sigio data using its own mutex, and the pipe reference count held by the caller will prevent the pipe from being prematurely garbage-collected. Discovered by: imp
* Add missing break.phk2004-11-161-0/+1
|
* Straighten the ioctl function out to have only one exit point.phk2004-11-151-17/+13
|
* Introduce fdclose() which will clean an entry in a filedesc.phk2004-11-071-9/+1
| | | | | | Replace homerolled versions with call to fdclose(). Make fdunused() static to kern_descrip.c
* Major enhancements to pipe memory usage:silby2004-08-161-55/+136
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - pipespace is now able to resize non-empty pipes; this allows for many more resizing opportunities - Backing is no longer pre-allocated for the reverse direction of pipes. This direction is rarely (if ever) used, so this cuts the amount of map space allocated to a pipe in half. - Pipe growth is now much more dynamic; a pipe will now grow when the total amount of data it contains and the size of the write are larger than the size of pipe. Previously, only individual writes greater than the size of the pipe would cause growth. - In low memory situations, pipes will now shrink during both read and write operations, where possible. Once the memory shortage ends, the growth code will cause these pipes to grow back to an appropriate size. - If the full PIPE_SIZE allocation fails when a new pipe is created, the allocation will be retried with SMALL_PIPE_SIZE. This helps to deal with the situation of a fragmented map after a low memory period has ended. - Minor documentation + code changes to support the above. In total, these changes increase the total number of pipes that can be allocated simultaneously, drastically reducing the chances that pipe allocation will fail. Performance appears unchanged due to dynamic resizing.
* Add locking to the kqueue subsystem. This also makes the kqueue subsystemjmg2004-08-151-6/+11
| | | | | | | | | | | | | a more complete subsystem, and removes the knowlege of how things are implemented from the drivers. Include locking around filter ops, so a module like aio will know when not to be unloaded if there are outstanding knotes using it's filter ops. Currently, it uses the MTX_DUPOK even though it is not always safe to aquire duplicate locks. Witness currently doesn't support the ability to discover if a dup lock is ok (in some cases). Reviewed by: green, rwatson (both earlier versions)
* Standardize pipe locking, ensuring that everything is locked viasilby2004-08-031-152/+115
| | | | | | | | | pipelock(), not via a mixture of mutexes and pipelock(). Additionally, add a few KASSERTS, and change some statements that should have been KASSERTS into KASSERTS. As a result of these cleanups, some segments of code have become significantly shorter and/or easier to read.
* * Add a "how" argument to uma_zone constructors and initialization functionsgreen2004-08-021-6/+8
| | | | | | | | | | | | | | | | | so that they know whether the allocation is supposed to be able to sleep or not. * Allow uma_zone constructors and initialation functions to return either success or error. Almost all of the ones in the tree currently return success unconditionally, but mbuf is a notable exception: the packet zone constructor wants to be able to fail if it cannot suballocate an mbuf cluster, and the mbuf allocators want to be able to fail in general in a MAC kernel if the MAC mbuf initializer fails. This fixes the panics people are seeing when they run out of memory for mbuf clusters. * Allow debug.nosleepwithlocks on WITNESS to be disabled, without changing the default. Both bmilekic and jeff have reviewed the changes made to make failable zone allocations work.
* Don't perform pipe endpoint locking during pipe_create(), as the piperwatson2004-07-231-11/+20
| | | | | | | | | | can't yet be referenced by other threads. In microbenchmarks, this appears to reduce the cost of pipe();close();close() on UP by 10%, and SMP by 7%. The vast majority of the cost of allocating a pipe remains VM magic. Suggested by: silby
* Fix a minor error in pipe_stat - st_size was always reported as 0silby2004-07-201-1/+4
| | | | | when direct writes kicked in. Whether this affected any applications is unknown.
* Revise the direct or optimized case to use uiomove_fromphys() by the readeralc2004-03-271-60/+18
| | | | | | | | | | | | | | | | | instead of ephemeral mappings using pmap_qenter() by the writer. The writer is still, however, responsible for wiring the pages, just not mapping them. Consequently, the allocation of KVA for the direct case is unnecessary. Remove it and the sysctls limiting it, i.e., kern.ipc.maxpipekvawired and kern.ipc.amountpipekvawired. The number of temporarily wired pages is still, however, limited by kern.ipc.maxpipekva. Note: On platforms lacking a direct virtual-to-physical mapping, uiomove_fromphys() uses sf_bufs to cache ephemeral mappings. Thus, the number of available sf_bufs can influence the performance of pipes on platforms such i386. Surprisingly, I saw the greatest gain from this change on such a machine: lmbench's pipe bandwidth result increased from ~1050MB/s to ~1850MB/s on my 2.4GHz, 400MHz FSB P4 Xeon.
* Assert pipe mutex in pipeselwakeup(), as we manipulate pipe_staterwatson2004-02-261-0/+1
| | | | | in a non-atomic manner. It appears to always be called with the mutex (good).
* Update comment regarding MAC labels: we no longer pass endpointsrwatson2004-02-251-7/+3
| | | | | | | into the MAC Framework, just the pipe pair. GC 'hadpeer' used in pipedestroy(), which is no longer needed as we check pipe_present flags on the pair.
* Correct some major SMP-harmful problems in the pipe implementation. Firstgreen2004-02-221-41/+66
| | | | | | | | | | | | of all, PIPE_EOF is not checked pervasively after everything that can drop the pipe mutex and msleep(), so fix. Additionally, though it might not harm anything, pipelock() and pipeunlock() are not used consistently. Third, the kqueue support functions do not use the pipe mutex correctly. Last, but absolutely not least, is a race: if pipe_busy is not set on the closing side of the pipe, the other side that is trying to write to that will crash BECAUSE PIPE_EOF IS NOT SET! Unconditionally set PIPE_EOF, and get rid of all the lockups/crashes I have seen trying to build ports.
* Don't dec/inc the amountpipes counter every time we resize a pipe --rwatson2004-02-031-2/+3
| | | | | | | | | | instead, just dec/inc in the ctor/dtor. For now, increment/decrement in two's, since we're now performing the operation once per pair, not once per pipe. Not really any measurable performance change in my micro-benchmarks, but doing less work is good, especially when it comes to atomic operations. Suggested by: alc
* Catch instances of (pipe == NULL) that were obsoleted with recentrwatson2004-02-031-6/+6
| | | | | | | | changes to jointly allocated pipe pairs. Replace these checks with pipe_present checks. This avoids a NULL pointer dereference when a pipe is half-closed. Submitted by: Peter Edwards <peter.edwards@openet-telecom.com>
* Coalesce pipe allocations and frees. Previously, the pipe coderwatson2004-02-011-91/+150
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | would allocate two 'struct pipe's from the pipe zone, and malloc a mutex. - Create a new "struct pipepair" object holding the two 'struct pipe' instances, struct mutex, and struct label reference. Pipe structures now have a back-pointer to the pipe pair, and a 'pipe_present' flag to indicate whether the half has been closed. - Perform mutex init/destroy in zone init/destroy, avoiding reallocating the mutex for each pipe. Perform most pipe structure setup in zone constructor. - VM memory mappings for pageable buffers are still done outside of the UMA zone. - Change MAC API to speak 'struct pipepair' instead of 'struct pipe', update many policies. MAC labels are also handled outside of the UMA zone for now. Label-only policy modules don't have to be recompiled, but if a module is recompiled, its pipe entry points will need to be updated. If a module actually reached into the pipe structures (unlikely), that would also need to be modified. These changes substantially simplify failure handling in the pipe code as there are many fewer possible failure modes. On half-close, pipes no longer free the 'struct pipe' for the closed half until a full-close takes place. However, VM mapped buffers are still released on half-close. Some code refactoring is now possible to clean up some of the back references, etc; this patch attempts not to change the structure of most of the pipe implementation, only allocation/free code paths, so as to avoid introducing bugs (hopefully). This cuts about 8%-9% off the cost of sequential pipe allocation and free in system call tests on UP and SMP in my micro-benchmarks. May or may not make a difference in macro-benchmarks, but doing less work is good. Reviewed by: juli, tjr Testing help: dwhite, fenestro, scottl, et al
* Fix an error in a KASSERT string: it's pipe_free_kmem(), notrwatson2004-01-311-1/+1
| | | | pipespace(), that contains this KASSERT.
* New file descriptor allocation code, derived from similar code introduceddes2004-01-151-0/+1
| | | | | | | | | | | in OpenBSD by Niels Provos. The patch introduces a bitmap of allocated file descriptors which is used to locate available descriptors when a new one is needed. It also moves the task of growing the file descriptor table out of fdalloc(), reducing complexity in both fdalloc() and do_dup(). Debts of gratitude are owed to tjr@ (who provided the original patch on which this work is based), grog@ (for the gdb(4) man page) and rwatson@ (for assistance with pxeboot(8)).
* Back out 1.160, which was committed by mistake.des2004-01-111-1/+0
|
* Mechanical whitespace cleanup.des2004-01-111-27/+27
|
* Mechanical whitespace cleanup + minor style nits.des2004-01-111-1/+3
|
* Fix the maxpipekva warning message so that it points to the correctsilby2003-12-281-1/+1
| | | | | | sysctl, and shorten the message. Noticed by: bde
* - Implement selwakeuppri() which allows raising the priority of atanimura2003-11-091-1/+1
| | | | | | | | | | | | | thread being waken up. The thread waken up can run at a priority as high as after tsleep(). - Replace selwakeup()s with selwakeuppri()s and pass appropriate priorities. - Add cv_broadcastpri() which raises the priority of the broadcast threads. Used by selwakeuppri() if collision occurs. Not objected in: -arch, -current
OpenPOWER on IntegriCloud