Add support to physio(9) for devices that don't want I/O split and

configure sa(4) to request no I/O splitting by default.

For tape devices, the user needs to be able to clearly understand
what blocksize is actually being used when writing to a tape
device.  The previous behavior of physio(9) was that it would split
up any I/O that was too large for the device, or too large to fit
into MAXPHYS.  This means that if, for instance, the user wrote a
1MB block to a tape device, and MAXPHYS was 128KB, the 1MB write
would be split into 8 128K chunks.  This would be done without
informing the user.

This has suboptimal effects, especially when trying to communicate
status to the user.  In the event of an error writing to a tape
(e.g. physical end of tape) in the middle of a 1MB block that has
been split into 8 pieces, the user could have the first two 128K
pieces written successfully, the third returned with an error, and
the last 5 returned with 0 bytes written.  If the user is using
a standard write(2) system call, all he will see is the ENOSPC
error.  He won't have a clue how much actually got written.  (With
a writev(2) system call, he should be able to determine how much
got written in addition to the error.)

The solution is to prevent physio(9) from splitting the I/O.  The
new cdev flag, SI_NOSPLIT, tells physio that the driver does not
want I/O to be split beforehand.

Although the sa(4) driver now enables SI_NOSPLIT by default,
that can be disabled by two loader tunables for now.  It will not
be configurable starting in FreeBSD 11.0.  kern.cam.sa.allow_io_split
allows the user to configure I/O splitting for all sa(4) driver
instances.  kern.cam.sa.%d.allow_io_split allows the user to
configure I/O splitting for a specific sa(4) instance.

There are also now three sa(4) driver sysctl variables that let the
users see some sa(4) driver values.  kern.cam.sa.%d.allow_io_split
shows whether I/O splitting is turned on.  kern.cam.sa.%d.maxio shows
the maximum I/O size allowed by kernel configuration parameters
(e.g. MAXPHYS, DFLTPHYS) and the capabilities of the controller.
kern.cam.sa.%d.cpi_maxio shows the maximum I/O size supported by
the controller.

Note that a better long term solution would be to implement support
for chaining buffers, so that that MAXPHYS is no longer a limiting
factor for I/O size to tape and disk devices.  At that point, the
controller and the tape drive would become the limiting factors.

sys/conf.h:	Add a new cdev flag, SI_NOSPLIT, that allows a
		driver to tell physio not to split up I/O.

sys/param.h:	Bump __FreeBSD_version to 1000049 for the addition
		of the SI_NOSPLIT cdev flag.

kern_physio.c:	If the SI_NOSPLIT flag is set on the cdev, return
		any I/O that is larger than si_iosize_max or
		MAXPHYS, has more than one segment, or would have
		to be split because of misalignment with EFBIG.
		(File too large).

		In the event of an error, print a console message to
		give the user a clue about what happened.

scsi_sa.c:	Set the SI_NOSPLIT cdev flag on the devices created
		for the sa(4) driver by default.

		Add tunables to control whether we allow I/O splitting
		in physio(9).

		Explain in the comments that allowing I/O splitting
		will be deprecated for the sa(4) driver in FreeBSD
		11.0.

		Add sysctl variables to display the maximum I/O
		size we can do (which could be further limited by
		read block limits) and the maximum I/O size that
		the controller can do.

		Limit our maximum I/O size (recorded in the cdev's
		si_iosize_max) by MAXPHYS.  This isn't strictly
		necessary, because physio(9) will limit it to
		MAXPHYS, but it will provide some clarity for the
		application.

		Record the controller's maximum I/O size reported
		in the Path Inquiry CCB.

sa.4:		Document the block size behavior, and explain that
		the option of allowing physio(9) to split the I/O
		will disappear in FreeBSD 11.0.

Sponsored by:	Spectra Logic

1 files changed, 41 insertions, 0 deletions

diff --git a/sys/kern/kern_physio.c b/sys/kern/kern_physio.c
index ab9c344..88cd0cf 100644
--- a/sys/kern/kern_physio.c
+++ b/sys/kern/kern_physio.c
@@ -54,6 +54,36 @@ physio(struct cdev *dev, struct uio *uio, int ioflag)
 		dev->si_iosize_max = DFLTPHYS;
 	}
 
+	/*
+	 * If the driver does not want I/O to be split, that means that we
+	 * need to reject any requests that will not fit into one buffer.
+	 */
+	if ((dev->si_flags & SI_NOSPLIT) &&
+	    ((uio->uio_resid > dev->si_iosize_max) ||
+	     (uio->uio_resid > MAXPHYS) ||
+	     (uio->uio_iovcnt > 1))) {
+		/*
+		 * Tell the user why his I/O was rejected.
+		 */
+		if (uio->uio_resid > dev->si_iosize_max)
+			printf("%s: request size %zd > si_iosize_max=%d, "
+			    "cannot split request\n", devtoname(dev),
+			    uio->uio_resid, dev->si_iosize_max);
+
+		if (uio->uio_resid > MAXPHYS)
+			printf("%s: request size %zd > MAXPHYS=%d, "
+			    "cannot split request\n", devtoname(dev),
+			    uio->uio_resid, MAXPHYS);
+
+		if (uio->uio_iovcnt > 1)
+			printf("%s: request vectors=%d > 1, "
+			    "cannot split request\n", devtoname(dev),
+			    uio->uio_iovcnt);
+
+		error = EFBIG;
+		goto doerror;
+	}
+
 	for (i = 0; i < uio->uio_iovcnt; i++) {
 		while (uio->uio_iov[i].iov_len) {
 			bp->b_flags = 0;
@@ -83,6 +113,17 @@ physio(struct cdev *dev, struct uio *uio, int ioflag)
 			 */
 			iolen = ((vm_offset_t) bp->b_data) & PAGE_MASK;
 			if ((bp->b_bcount + iolen) > bp->b_kvasize) {
+				/*
+				 * This device does not want I/O to be split.
+				 */
+				if (dev->si_flags & SI_NOSPLIT) {
+					printf("%s: request ptr %#jx is not "
+					    "on a page boundary, cannot split "
+					    "request\n", devtoname(dev),
+					    (uintmax_t)bp->b_data);
+					error = EFBIG;
+					goto doerror;
+				}
 				bp->b_bcount = bp->b_kvasize;
 				if (iolen != 0)
 					bp->b_bcount -= PAGE_SIZE;