summaryrefslogtreecommitdiffstats
path: root/sys/kern/vfs_bio.c
Commit message (Collapse)AuthorAgeFilesLines
* Normalize a significant number of kernel malloc type names:rwatson2005-10-311-1/+1
| | | | | | | | | | | | | | | | | | | - Prefer '_' to ' ', as it results in more easily parsed results in memory monitoring tools such as vmstat. - Remove punctuation that is incompatible with using memory type names as file names, such as '/' characters. - Disambiguate some collisions by adding subsystem prefixes to some memory types. - Generally prefer lower case to upper case. - If the same type is defined in multiple architecture directories, attempt to use the same name in additional cases. Not all instances were caught in this change, so more work is required to finish this conversion. Similar changes are required for UMA zone names.
* Release clean buffer with wrong size and no dependencies also for non-VMIOtegge2005-10-091-2/+1
| | | | case.
* Un-staticize waitrunningbufspace() and call it before returning fromtruckman2005-09-301-1/+1
| | | | | | | ffs_copyonwrite() if any async writes were launched. Restore the threads previous TDP_NORUNNINGBUF state before returning from ffs_copyonwrite().
* Un-staticize runningbufwakeup() and staticize updateproc.truckman2005-09-301-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new private thread flag to indicate that the thread should not sleep if runningbufspace is too large. Set this flag on the bufdaemon and syncer threads so that they skip the waitrunningbufspace() call in bufwrite() rather than than checking the proc pointer vs. the known proc pointers for these two threads. A way of preventing these threads from being starved for I/O but still placing limits on their outstanding I/O would be desirable. Set this flag in ffs_copyonwrite() to prevent bufwrite() calls from blocking on the runningbufspace check while holding snaplk. This prevents snaplk from being held for an arbitrarily long period of time if runningbufspace is high and greatly reduces the contention for snaplk. The disadvantage is that ffs_copyonwrite() can start a large amount of I/O if there are a large number of snapshots, which could cause a deadlock in other parts of the code. Call runningbufwakeup() in ffs_copyonwrite() to decrement runningbufspace before attempting to grab snaplk so that I/O requests waiting on snaplk are not counted in runningbufspace as being in-progress. Increment runningbufspace again before actually launching the original I/O request. Prior to the above two changes, the system could deadlock if enough I/O requests were blocked by snaplk to prevent runningbufspace from falling below lorunningspace and one of the bawrite() calls in ffs_copyonwrite() blocked in waitrunningbufspace() while holding snaplk. See <http://www.holm.cc/stress/log/cons143.html>
* Close a race in biodone(), whereby the bio_done field of the passedpeadar2005-09-291-3/+5
| | | | | | | | | | bio may have been freed and reassigned by the wakeup before being tested after releasing the bdonelock. There's a non-zero chance this is the cause of a few of the crashes knocking around with biodone() sitting in the stack backtrace. Reviewed By: phk@
* - Use lockmgr_printinfo rather than rolling our own. This introduces ajeff2005-08-031-3/+1
| | | | | slight problem by using printf instead of db_printf however 'show lockedvnods' does the same so I believe it is ok for now.
* Eliminate inconsistency in the setting of the B_DONE flag. Specifically,alc2005-07-201-1/+0
| | | | | | | | | | | | | | | | make the b_iodone callback responsible for setting it if it is needed. Previously, it was set unconditionally by bufdone() without holding whichever lock is shared by the b_iodone callback and the corresponding top-half function. Consequently, in a race, the top-half function could conclude that operation was done before the b_iodone callback finished. See, for example, aio_physwakeup() and aio_fphysio(). Note: I don't believe that the other, more widely-used b_iodone callbacks are affected. Discussed with: jeff Reviewed by: phk MFC after: 2 weeks
* - Add and enhance asserts related to the wrong bufobj panic.jeff2005-06-141-1/+1
| | | | | Sponsored by: Isilon Systems, Inc. Approved by: re (blanket vfs)
* - Split one KASSERT in bremfree() into two to aid in debugging.jeff2005-06-131-1/+3
| | | | Sponsored by: Isilon Systems, Inc.
* Fix a serious deadlock with the NFS client. Given a large enoughgreen2005-06-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | atomic write request, it can fill the buffer cache with the entirety of that write in order to handle retries. However, it never drops the vnode lock, or else it wouldn't be atomic, so it ends up waiting indefinitely for more buf memory that cannot be gotten as it has it all, and it waits in an uncancellable state. To fix this, hibufspace is exported and scaled to a reasonable fraction. This is used as the limit of how much of an atomic write request by the NFS client will be handled asynchronously. If the request is larger than this, it will be turned into a synchronous request which won't deadlock the system. It's possible this value is far off from what is required by some, so it shall be tunable as soon as mount_nfs(8) learns of the new field. The slowdown between an asynchronous and a synchronous write on NFS appears to be on the order of 2x-4x. General nod by: gad MFC after: 2 weeks More testing: wes PR: kern/79208
* - My sub-par public school education has been exposed. s/sentinal/sentinel/jeff2005-06-091-4/+4
| | | | Noticed by: Emil Mikulic
* - Under heavy IO load the buf daemon can run for many hundereds ofjeff2005-06-081-7/+29
| | | | | | | | | | | | | | | | | milliseconds due to what is essentially n^2 algorithmic complexity. This change makes the algorithm N*2 instead. This heavy processing manifested itself as skipping in audio and video playback due to the long scheduling latencies and contention on giant by pcm. - flushbufqueues() is now responsible for flushing multiple buffers rather than one at a time. This allows us to save our progress in the list by using a sentinal. We must do the numdirtywakeup() and waitrunningbufspace() here now rather than in buf_daemon(). - Also add a uio_yield() after we have processed the list once for bufs without deps and again for bufs with deps. This is to release Giant and allow any other giant locked code to proceed. Tested by: Many users on current@ Revealed by: schedgraph traces sent by Emil Mikulic & Anthony Ginepro
* - Add bufobj_wrefl() to add a write ref to a bufobj that is already locked.jeff2005-05-301-0/+9
|
* - Remove long dead splbio() calls and comments relating to the oldjeff2005-04-301-71/+4
| | | | synchronization mechanism.
* - Don't acquire Giant before calling b_biodone, individual consumers arejeff2005-04-301-6/+0
| | | | | | now required to do so themselves. Sponsored by: Isilon Systems, Inc.
* - Add two KASSERTs to prevent us from recycling a buf that is still on ajeff2005-04-221-0/+6
| | | | | | bufobj list. Sponsored by: Isilon Systems, Inc.
* - Add information about the buf lock to db_show_buffer.jeff2005-03-251-0/+18
| | | | | | - Add a 'show lockedbufs' command that is similar to show lockedvnods. Sponsored by: Isilon Systems, Inc.
* - Lock access to the buffer_map with the vm_map lock. In 4.x this wasjeff2005-03-081-0/+5
| | | | | | | done with splbio, in 5.x this was done with Giant. Discussed with: alc Reported by: julian, pho
* Make various vnode related functions staticphk2005-02-101-3/+3
|
* - Add more information to the getnewbuf() recycling KTR.jeff2005-02-101-2/+4
| | | | Sponsored by: Isilon Systems, Inc.
* - Remove an invalid KASSERT added in recent background write reshuffling.jeff2005-02-081-3/+0
| | | | Sponsored by: Isilon Systems, Inc.
* Background writes are entirely an FFS/Softupdates thing.phk2005-02-081-126/+5
| | | | | | | | | | | | | | | | Give FFS vnodes a specific bufwrite method which contains all the background write stuff and then calls into the default bufwrite() for the rest of the job. Remove all the background write related stuff from the normal bufwrite. This drags the softdep_move_dependencies() back into FFS. Long term, it is worth looking at simply copying the data into allocated memory and issuing the bio directly and not create the "shadow buf" in the first place (just like copy-on-write is done in snapshots for instance). I don't think we really gain anything but complexity from doing this with a buf.
* - Don't release BKGRDINPROG until after we've bufdone'd the copy.jeff2005-02-051-14/+15
| | | | Sponsored by: Isilon Systems, Inc.
* - Don't drop the wref on the bufobj until after bufdone() has completed.jeff2005-01-281-7/+10
| | | | | | | Without this, threads waiting in bufobj_wwait() may wakeup prior to bufdone() completing. Sponsored by: Isilon Systems, Inc.
* Don't use VOP_GETVOBJECT, use vp->v_object directly.phk2005-01-251-10/+8
|
* Kill the VV_OBJBUF and test the v_object for NULL instead.phk2005-01-241-4/+3
|
* - Add CTR calls to trace the lifecycle of a buffer.jeff2005-01-241-79/+84
| | | | | | | | | | | | | | | | | - Remove some KASSERTs which are invalid if the appropriate lock is not held. - Slightly restructure bremfree() so that it is more sane. - Change the flush code in bdwrite() to avoid acquiring a mutex whenever possible. - Change the flush code in bdwrite() to avoid holding the bufobj mutex while calling buf_countdeps(). This introduces a lock-order relationship with the softdep lock that can not otherwise be resolved. - Don't set B_DONE until bufdone() is complete, otherwise another processor may believe the buf is done before it is. - Only acquire Giant if the caller has set b_iodone. Don't grab giant around normal bufdone() calls. Sponsored By: Isilon Systems, Inc.
* Add BO_SYNC() and add a default which uses the secret vnode pointerphk2005-01-111-0/+8
| | | | and VOP_FSYNC() for now.
* Remove the unused credential argument from VOP_FSYNC() and VFS_SYNC().phk2005-01-111-1/+1
| | | | | | | | | | | | | | | | | | I'm not sure why a credential was added to these in the first place, it is not used anywhere and it doesn't make much sense: The credentials for syncing a file (ability to write to the file) should be checked at the system call level. Credentials for syncing one or more filesystems ("none") should be checked at the system call level as well. If the filesystem implementation needs a particular credential to carry out the syncing it would logically have to the cached mount credential, or a credential cached along with any delayed write data. Discussed with: rwatson
* - Eliminate the acquisition and release of the bqlock in bremfree() byjeff2004-11-181-38/+66
| | | | | | | | | | | | | | | | | | | | setting the B_REMFREE flag in the buf. This is done to prevent lock order reversals with code that must call bremfree() with a local lock held. This also reduces overhead by removing two lock operations per buf for fsync() and similar. - Check for the B_REMFREE flag in brelse() and bqrelse() after the bqlock has been acquired so that we may remove ourself from the free-list. - Provide a bremfreef() function to immediately remove a buf from a free-list for use only by NFS. This is done because the nfsclient code overloads the b_freelist queue for its own async. io queue. - Simplify the numfreebuffers accounting by removing a switch statement that executed the same code in every possible case. - getnewbuf() can encounter locked bufs on free-lists once Giant is removed. Remove a panic associated with this condition and delay asserts that inspect the buf until after it is locked. Reviewed by: phk Sponsored by: Isilon Systems, Inc.
* Retire b_magic now, we have the bufobj containing the same hint.phk2004-11-041-1/+0
|
* Change buf->b_object to buf->b_bufobj->bo_objectphk2004-11-041-14/+16
| | | | some whitespace fixes.
* whitespacephk2004-11-041-4/+2
|
* Remove buf->b_dev field.phk2004-11-041-7/+2
|
* The synchronization provided by vm object locking has eliminated thealc2004-11-031-2/+0
| | | | | | | | | | | | | | | | | need for most calls to vm_page_busy(). Specifically, most calls to vm_page_busy() occur immediately prior to a call to vm_page_remove(). In such cases, the containing vm object is locked across both calls. Consequently, the setting of the vm page's PG_BUSY flag is not even visible to other threads that are following the synchronization protocol. This change (1) eliminates the calls to vm_page_busy() that immediately precede a call to vm_page_remove() or functions, such as vm_page_free() and vm_page_rename(), that call it and (2) relaxes the requirement in vm_page_remove() that the vm page's PG_BUSY flag is set. Now, the vm page's PG_BUSY flag is set only when the vm object lock is released while the vm page is still in transition. Typically, this is when it is undergoing I/O.
* Remove the last call in the system to VOP_SPECSTRATEGY(): We can nophk2004-10-291-7/+2
| | | | | longer come through the VNODE layer to the disks since all the filesystems now go via geom_vfs to GEOM.
* Give dev_strategy() an explict cdev argument in preparation for removingphk2004-10-291-13/+49
| | | | | | | | | | | buf->b-dev. Put a bio between the buf passed to dev_strategy() and the device driver strategy routine in order to not clobber fields in the buf. Assert copyright on vfs_bio.c and update copyright message to canonical text. There is no legal difference between John Dysons two-clause abbreviated BSD license and the canonical text.
* Lock bp->b_bufobj->b_object instead of bp->b_objectphk2004-10-281-10/+10
|
* The island council met and voted buf_prewrite() home.phk2004-10-261-5/+3
| | | | | | | | | | | | | | Give ffs it's own bufobj->bo_ops vector and create a private strategy routine, (currently misnamed for forwards compatibility), which is just a copy of the generic bufstrategy routine except we call softdep_disk_prewrite() directly instead of through the buf_prewrite() indirection. Teach UFS about the need for softdep_disk_prewrite() and call the function directly in FFS. Remove buf_prewrite() from the default bufstrategy() and from the global bio_ops method vector.
* Put the I/O block size in bufobj->bo_bsize.phk2004-10-261-8/+1
| | | | | | | We keep si_bsize_phys around for now as that is the simplest way to pull the number out of disk device drivers in devfs_open(). The correct solution would be to do an ioctl(DIOCGSECTORSIZE), but the point is probably mooth when filesystems sit on GEOM, so don't bother for now.
* Hold the lock on the containing vm object when callingalc2004-10-261-0/+2
| | | | vm_page_sleep_if_busy().
* Remove vnode->v_bsize. This was a dead-end.phk2004-10-251-7/+0
|
* Use VM_ALLOC_NOBUSY to eliminate vm_page_wakeup() calls and the acquisitionalc2004-10-251-10/+3
| | | | | | | and release of the global page queues lock required to make the call. Remove GIANT_REQUIRED from vm_hold_free_pages(). All of its VM operations are properly synchronized.
* Collapse vnode->v_object and buf->b_object into bufobj->bo_object.phk2004-10-251-3/+6
|
* Move the buffer method vector (buf->b_op) to the bufobj.phk2004-10-241-37/+18
| | | | | | | | | | | | | | | | | Extend it with a strategy method. Add bufstrategy() which do the usual VOP_SPECSTRATEGY/VOP_STRATEGY song and dance. Rename ibwrite to bufwrite(). Move the two NFS buf_ops to more sensible places, add bufstrategy to them. Add inlines for bwrite() and bstrategy() which calls through buf->b_bufobj->b_ops->b_{write,strategy}(). Replace almost all VOP_STRATEGY()/VOP_SPECSTRATEGY() calls with bstrategy().
* Add b_bufobj to struct buf which eventually will eliminate the need for b_vp.phk2004-10-221-56/+66
| | | | | | | | | | | | | | | | | | Initialize b_bufobj for all buffers. Make incore() and gbincore() take a bufobj instead of a vnode. Make inmem() local to vfs_bio.c Change a lot of VI_[UN]LOCK(bp->b_vp) to BO_[UN]LOCK(bp->b_bufobj) also VI_MTX() to BO_MTX(), Make buf_vlist_add() take a bufobj instead of a vnode. Eliminate other uses of bp->b_vp where bp->b_bufobj will do. Various minor polishing: remove "register", turn panic into KASSERT, use new function declarations, TAILQ_FOREACH_SAFE() etc.
* Move the VI_BWAIT flag into no bo_flag element of bufobj and call it BO_WWAITphk2004-10-211-8/+66
| | | | | | | | | | Add bufobj_wref(), bufobj_wdrop() and bufobj_wwait() to handle the write count on a bufobj. Bufobj_wdrop() replaces vwakeup(). Use these functions all relevant places except in ffs_softdep.c where the use if interlocked_sleep() makes this impossible. Rename b_vnbufs to b_bobufs now that we touch all the relevant files anyway.
* use dev_re[fl]thread() rather than home rolled versions.phk2004-09-241-8/+2
|
* Eliminate DEV_STRATEGY() macro: call dev_strategy() directly.phk2004-09-231-3/+12
| | | | Make dev_strategy() handle errors and departing devices properly.
* Do not refcount the cdevsw, but rather maintain a cdev->si_threadcountphk2004-09-231-5/+11
| | | | | | | | | | of the number of threads which are inside whatever is behind the cdevsw for this particular cdev. Make the device mutex visible through dev_lock() and dev_unlock(). We may want finer granularity later. Replace spechash_mtx use with dev_lock()/dev_unlock().
OpenPOWER on IntegriCloud