summaryrefslogtreecommitdiffstats
path: root/sys/kern/kern_physio.c
Commit message (Collapse)AuthorAgeFilesLines
* MFC r290140:hselasky2015-11-121-1/+7
| | | | | | | | | | | | | | Add missing NULL check in physio(). When destroying a character device the si_devsw field is set to NULL before all references are gone, to indicate the character device is going away. This can cause a NULL-dereference fault inside physio(). The callers of physio() should own a thread reference on the cdev and if si_devsw is seen as non-NULL, it is usable during the execution of the function. Else an ENXIO error code is returned. Reviewed by: kib
* MFC r281825: Rewrite physio() to not allocate pbufs for unmapped I/O.mav2015-05-061-61/+93
| | | | | | | | | | | | | pbufs is a limited resource, and their allocator is not SMP-scalable. So instead of always allocating pbuf to immediately convert it to bio, allocate bio just here. If buffer needs kernel mapping, then pbuf is still allocated, but used only as a source of KVA and storage for a list of held pages. On 40-core system doing many 512-byte reads from user level to array of raw SSDs this change removes huge lock congestion inside pbuf allocator. It improves peak performance from ~300K to ~1.2M IOPS. On my previous 24-core system this problem also existed, but was less serious.
* Fix some issues in change 254760 pointed out by Bruce Evans:ken2013-08-291-11/+8
| | | | | | | | | | - Remove excessive parenthesis - Use KNF continuation indentation - Cut down on excessive continuation lines - More consistent style in messages - Use uprintf() instead of printf() Submitted by: bde
* Fix a printf format warning on 32-bit mips and powerpc.ken2013-08-241-2/+2
| | | | | Reported by: bde, gjb Pointy hat to: ken
* Add support to physio(9) for devices that don't want I/O split andken2013-08-241-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | configure sa(4) to request no I/O splitting by default. For tape devices, the user needs to be able to clearly understand what blocksize is actually being used when writing to a tape device. The previous behavior of physio(9) was that it would split up any I/O that was too large for the device, or too large to fit into MAXPHYS. This means that if, for instance, the user wrote a 1MB block to a tape device, and MAXPHYS was 128KB, the 1MB write would be split into 8 128K chunks. This would be done without informing the user. This has suboptimal effects, especially when trying to communicate status to the user. In the event of an error writing to a tape (e.g. physical end of tape) in the middle of a 1MB block that has been split into 8 pieces, the user could have the first two 128K pieces written successfully, the third returned with an error, and the last 5 returned with 0 bytes written. If the user is using a standard write(2) system call, all he will see is the ENOSPC error. He won't have a clue how much actually got written. (With a writev(2) system call, he should be able to determine how much got written in addition to the error.) The solution is to prevent physio(9) from splitting the I/O. The new cdev flag, SI_NOSPLIT, tells physio that the driver does not want I/O to be split beforehand. Although the sa(4) driver now enables SI_NOSPLIT by default, that can be disabled by two loader tunables for now. It will not be configurable starting in FreeBSD 11.0. kern.cam.sa.allow_io_split allows the user to configure I/O splitting for all sa(4) driver instances. kern.cam.sa.%d.allow_io_split allows the user to configure I/O splitting for a specific sa(4) instance. There are also now three sa(4) driver sysctl variables that let the users see some sa(4) driver values. kern.cam.sa.%d.allow_io_split shows whether I/O splitting is turned on. kern.cam.sa.%d.maxio shows the maximum I/O size allowed by kernel configuration parameters (e.g. MAXPHYS, DFLTPHYS) and the capabilities of the controller. kern.cam.sa.%d.cpi_maxio shows the maximum I/O size supported by the controller. Note that a better long term solution would be to implement support for chaining buffers, so that that MAXPHYS is no longer a limiting factor for I/O size to tape and disk devices. At that point, the controller and the tape drive would become the limiting factors. sys/conf.h: Add a new cdev flag, SI_NOSPLIT, that allows a driver to tell physio not to split up I/O. sys/param.h: Bump __FreeBSD_version to 1000049 for the addition of the SI_NOSPLIT cdev flag. kern_physio.c: If the SI_NOSPLIT flag is set on the cdev, return any I/O that is larger than si_iosize_max or MAXPHYS, has more than one segment, or would have to be split because of misalignment with EFBIG. (File too large). In the event of an error, print a console message to give the user a clue about what happened. scsi_sa.c: Set the SI_NOSPLIT cdev flag on the devices created for the sa(4) driver by default. Add tunables to control whether we allow I/O splitting in physio(9). Explain in the comments that allowing I/O splitting will be deprecated for the sa(4) driver in FreeBSD 11.0. Add sysctl variables to display the maximum I/O size we can do (which could be further limited by read block limits) and the maximum I/O size that the controller can do. Limit our maximum I/O size (recorded in the cdev's si_iosize_max) by MAXPHYS. This isn't strictly necessary, because physio(9) will limit it to MAXPHYS, but it will provide some clarity for the application. Record the controller's maximum I/O size reported in the Path Inquiry CCB. sa.4: Document the block size behavior, and explain that the option of allowing physio(9) to split the I/O will disappear in FreeBSD 11.0. Sponsored by: Spectra Logic
* Change the way that unmapped I/O capability is advertised.ken2013-08-151-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous method was to set the D_UNMAPPED_IO flag in the cdevsw for the driver. The problem with this is that in many cases (e.g. sa(4)) there may be some instances of the driver that can handle unmapped I/O and some that can't. The isp(4) driver can handle unmapped I/O, but the esp(4) driver currently cannot. The cdevsw is shared among all driver instances. So instead of setting a flag on the cdevsw, set a flag on the cdev. This allows drivers to indicate support for unmapped I/O on a per-instance basis. sys/conf.h: Remove the D_UNMAPPED_IO cdevsw flag and replace it with an SI_UNMAPPED cdev flag. kern_physio.c: Look at the cdev SI_UNMAPPED flag to determine whether or not a particular driver can handle unmapped I/O. geom_dev.c: Set the SI_UNMAPPED flag for all GEOM cdevs. Since GEOM will create a temporary mapping when needed, setting SI_UNMAPPED unconditionally will work. Remove the D_UNMAPPED_IO flag. nvme_ns.c: Set the SI_UNMAPPED flag on cdevs created here if NVME_UNMAPPED_BIO_SUPPORT is enabled. vfs_aio.c: In aio_qphysio(), check the SI_UNMAPPED flag on a cdev instead of the D_UNMAPPED_IO flag on the cdevsw. sys/param.h: Bump __FreeBSD_version to 1000045 for the switch from setting the D_UNMAPPED_IO flag in the cdevsw to setting SI_UNMAPPED in the cdev. Reviewed by: kib, jimharris MFC after: 1 week Sponsored by: Spectra Logic
* Add dev_strategy_csw() function, which is similar to dev_strategy()kib2013-03-271-8/+5
| | | | | | | | | | | | | | | | | but assumes that a thread reference was already obtained on the passed device. Use the function from physio(), to avoid two extra dev_mtx lock and unlock. Note that physio() is always used as the cdevsw method, or is called from a cdevsw method, and the caller already owns the reference. dev_strategy() is left to keep KPI intact, but now it is implemented as a wrapper around dev_strategy_csw(). Do some style cleanup in physio(). Requested and reviewed by: kan (previous version) Sponsored by: The FreeBSD Foundation MFC after: 2 weeks
* Do not pass unmapped buffers to drivers that cannot handle themkan2013-03-261-2/+12
| | | | | | | | | | | | | | In physio, check if device can handle unmapped IO and pass an appropriately mapped buffer to the driver strategy routine. The only driver in the tree that can handle unmapped buffers is one exposed by GEOM, so mark it as such with the new flag in the driver cdevsw structure. This fixes insta-panics on hosts, running dconschat, as /dev/fwmem is an example of the driver that makes use of physio routine, but bypasses the g_down thread, where the buffer gets mapped normally. Discussed with: kib (earlier version)
* Do not remap usermode pages into KVA for physio.kib2013-03-191-1/+1
| | | | | Sponsored by: The FreeBSD Foundation Tested by: pho
* Account i/o done on cdevs.kib2010-11-251-2/+5
| | | | | Reported and tested by: Adam Vande More <amvandemore gmail com> MFC after: 1 week
* /* -> /*- for copyright notices, minor format tweaks as necessaryimp2005-01-061-1/+1
|
* Remove buf->b_dev field.phk2004-11-041-1/+0
|
* Give dev_strategy() an explict cdev argument in preparation for removingphk2004-10-291-1/+1
| | | | | | | | | | | buf->b-dev. Put a bio between the buf passed to dev_strategy() and the device driver strategy routine in order to not clobber fields in the buf. Assert copyright on vfs_bio.c and update copyright message to canonical text. There is no legal difference between John Dysons two-clause abbreviated BSD license and the canonical text.
* Eliminate DEV_STRATEGY() macro: call dev_strategy() directly.phk2004-09-231-1/+1
| | | | Make dev_strategy() handle errors and departing devices properly.
* Eliminate the acquisition and release of Giant within physio(). Removealc2004-08-101-6/+0
| | | | | | | the spl calls. Reviewed by: phk@ Discussed with: scottl@
* Do the dreaded s/dev_t/struct cdev */phk2004-06-161-1/+1
| | | | Bump __FreeBSD_version accordingly.
* Send B_PHYS out to pasture, it no longer serves any function.phk2003-11-151-1/+1
|
* Initialize b_iooffset before calling strategyphk2003-10-181-0/+1
|
* Grab Giant in physio() since non-giant drivers are starting to appear.phk2003-08-021-0/+3
|
* Use __FBSDID().obrien2003-06-111-2/+3
|
* Don't reinitialize fields that are already initialized by getpbuf().alc2003-04-051-1/+1
|
* Sufficient access checks are performed by vmapbuf() that callingalc2003-04-051-15/+1
| | | | | | useracc() is pointless. Remove the call to useracc() from physio(). Reviewed by: tegge
* - Add a lock for protecting against msleep(bp, ...) wakeup(bp) races.jeff2003-03-131-9/+5
| | | | | | | | | | - Create a new function bdone() which sets B_DONE and calls wakup(bp). This is suitable for use as b_iodone for buf consumers who are not going through the buf cache. - Create a new function bwait() which waits for the buf to be done at a set priority and with a specific wmesg. - Replace several cases where the above functionality was implemented without locking with the new functions.
* Close the remaining user address mapping races for physicaldillon2003-01-201-1/+11
| | | | | | | I/O, CAM, and AIO. Still TODO: streamline useracc() checks. Reviewed by: alc, tegge MFC after: 7 days
* Remove unused second argument from DEV_STRATEGY().phk2003-01-031-1/+1
|
* Change iov_base's type from `char *' to the standard `void *'. Allmike2002-10-111-1/+2
| | | | | uses of iov_base which assume its type is `char *' (in order to do pointer arithmetic) have been updated to cast iov_base to `char *'.
* More caddr_t removal, make fo_ioctl take a void * instead of a caddr_t.alfred2002-06-291-2/+2
|
* Use btodb() macro.phk2002-05-181-7/+1
| | | | Sponsored by: DARPA & NAI Labs.
* Separate the struct bio related stuff out of <sys/buf.h> intophk2000-05-051-0/+1
| | | | | | | | | | | | | | | <sys/bio.h>. <sys/bio.h> is now a prerequisite for <sys/buf.h> but it shall not be made a nested include according to bdes teachings on the subject of nested includes. Diskdrivers and similar stuff below specfs::strategy() should no longer need to include <sys/buf.> unless they need caching of data. Still a few bogus uses of struct buf to track down. Repocopy by: peter
* Move B_ERROR flag to b_ioflags and call it BIO_ERROR.phk2000-04-021-2/+2
| | | | | | | | | | | | | (Much of this done by script) Move B_ORDERED flag to b_ioflags and call it BIO_ORDERED. Move b_pblkno and b_iodone_chain to struct bio while we transition, they will be obsoleted once bio structs chain/stack. Add bio_queue field for struct bio aware disksort. Address a lot of stylistic issues brought up by bde.
* Rename the existing BUF_STRATEGY() to DEV_STRATEGY()phk2000-03-201-1/+1
| | | | | | | | substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo) substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo) This patch is machine generated except for the ccd.c and buf.h parts.
* Remove B_READ, B_WRITE and B_FREEBUF and replace them with a newphk2000-03-201-4/+4
| | | | | | | | | | | | | | | | | | | | | field in struct buf: b_iocmd. The b_iocmd is enforced to have exactly one bit set. B_WRITE was bogusly defined as zero giving rise to obvious coding mistakes. Also eliminate the redundant struct buf flag B_CALL, it can just as efficiently be done by comparing b_iodone to NULL. Should you get a panic or drop into the debugger, complaining about "b_iocmd", don't continue. It is likely to write on your disk where it should have been reading. This change is a step in the direction towards a stackable BIO capability. A lot of this patch were machine generated (Thanks to style(9) compliance!) Vinum users: Greg has not had time to test this yet, be careful.
* Change useracc() and kernacc() to use VM_PROT_{READ|WRITE|EXECUTE} for thephk1999-10-301-1/+2
| | | | | | | | | "rw" argument, rather than hijacking B_{READ|WRITE}. Fix two bugs (physio & cam) resulting by the confusion caused by this. Submitted by: Tor.Egge@fast.no Reviewed by: alc, ken (partly)
* Give physio a makeover.phk1999-10-091-121/+54
| | | | | | | | | | | | | | | | | | | - Let physio take read/write compatible args and have it use uio->uio_rw to determine the direction. - physread/physwrite are now #defines for physio - Remove the inversly named minphys(), dev->si_iosize_max takes over. - Physio() always uses pbufs. - Fix the check for non page-aligned transfers, now only unaligned transfers larger than (MAXPHYS - PAGE_SIZE) get fragmented (only interesting for tapes using max blocksize). - General wash-and-clean of code. Constructive input from: bde
* Add a newline to "WARNING: %s maxphys = 0 ??" so it doesn't trip upgreen1999-10-091-1/+1
| | | | syslogd. Note of course it's simply much more polite and correct, too :)
* In some drivers we use two devices to be able to boot.sos1999-10-021-0/+1
| | | | | | | | | So if si_iosize_max is allready set, dont mess with it.. Also just log the problem with maxphys not being set once. designed by: phk tested by: sos
* Fix a problem relating to si_iosize_max which broke scsi devices.phk1999-10-021-0/+4
|
* Kill the cdevsw->d_maxio field.phk1999-09-221-7/+2
| | | | | | | | | d_maxio is replaced by the dev->si_iosize_max field which the driver should be set in all calls to cdevsw->d_open if it has a better idea than the system wide default. The field is a generic dev_t field (ie: not disk specific) so that tapes and other devices can use physio as well.
* Remove replace phygetvpbuf() with direct call to getpbuf();phk1999-09-121-19/+1
|
* $Id$ -> $FreeBSD$peter1999-08-281-1/+1
|
* Use more compiler friendly test for overflow.phk1999-08-211-3/+3
| | | | Submitted by: bde
* Detect if the the offset used to read from a raw device loose bitsphk1999-08-211-2/+8
| | | | when converted to block number.
* Spring cleaning around strategy and disklabels/slices:phk1999-08-141-6/+5
| | | | | | | | | | | | | | Introduce BUF_STRATEGY(struct buf *, int flag) macro, and use it throughout. please see comment in sys/conf.h about the flag argument. Remove strategy argument from all the diskslice/label/bad144 implementations, it should be found from the dev_t. Remove bogus and unused strategy1 routines. Remove open/close arguments from dssize(). Pick them up from dev_t. Remove unused and unfinished setgeom support from diskslice/label/bad144 code.
* Convert buffer locking from using the B_BUSY and B_WANTED flags to usingmckusick1999-06-261-19/+9
| | | | | | | lockmgr locks. This commit should be functionally equivalent to the old semantics. That is, all buffer locking is done with LK_EXCLUSIVE requests. Changes to take advantage of LK_SHARED and LK_RECURSIVE will be done in future commits.
* I got tired of seeing all the cdevsw[major(foo)] all over the place.phk1999-05-081-5/+5
| | | | | | | | Made a new (inline) function devsw(dev_t dev) and substituted it. Changed to the BDEV variant to this format as well: bdevsw(dev_t dev) DEVFS will eventually benefit from this change too.
* Introduce two functions: physread() and physwrite() and use these directlyphk1999-05-071-1/+13
| | | | | | in *devsw[] rather than the 46 local copies of the same functions. (grog will do the same for vinum when he has time)
* remove b_proc from struct buf, it's (now) unused.phk1999-05-061-2/+1
| | | | Reviewed by: dillon, bde
* Use the reference counted PHOLD()/PRELE() rather than P_PHYSIO.peter1999-04-061-9/+9
|
* This is a rather large commit that encompasses the new swapper,dillon1999-01-211-4/+4
| | | | | | | | | | changes to the VM system to support the new swapper, VM bug fixes, several VM optimizations, and some additional revamping of the VM code. The specific bug fixes will be documented with additional forced commits. This commit is somewhat rough in regards to code cleanup issues. Reviewed by: "John S. Dyson" <root@dyson.iquest.net>, "David Greenman" <dg@root.com>
* Nitpicking and dusting performed on a train. Removes trivial warningsphk1998-10-251-2/+1
| | | | about unused variables, labels and other lint.
OpenPOWER on IntegriCloud