Move dynamic sysctl(8) variable creation for the cd(4) and da(4) drivers

out of cdregister() and daregister(), which are run from interrupt context.

The sysctl code does blocking mallocs (M_WAITOK), which causes problems
if malloc(9) actually needs to sleep.

The eventual fix for this issue will involve moving the CAM probe process
inside a kernel thread.  For now, though, I have fixed the issue by moving
dynamic sysctl variable creation for these two drivers to a task queue
running in a kernel thread.

The existing task queues (taskqueue_swi and taskqueue_swi_giant) run in
software interrupt handlers, which wouldn't fix the problem at hand.  So I
have created a new task queue, taskqueue_thread, that runs inside a kernel
thread.  (It also runs outside of Giant -- clients must explicitly acquire
and release Giant in their taskqueue functions.)

scsi_cd.c:	Remove sysctl variable creation code from cdregister(), and
		move it to a new function, cdsysctlinit().  Queue
		cdsysctlinit() to the taskqueue_thread taskqueue once we
		have fully registered the cd(4) driver instance.

scsi_da.c:	Remove sysctl variable creation code from daregister(), and
		move it to move it to a new function, dasysctlinit().
		Queue dasysctlinit() to the taskqueue_thread taskqueue once
		we have fully registered the da(4) instance.

taskqueue.h:	Declare the new taskqueue_thread taskqueue, update some
		comments.

subr_taskqueue.c:
		Create the new kernel thread taskqueue.  This taskqueue
		runs outside of Giant, so any functions queued to it would
		need to explicitly acquire/release Giant if they need it.

cd.4:		Update the cd(4) man page to talk about the minimum command
		size sysctl/loader tunable.  Also note that the changer
		variables are available as loader tunables as well.

da.4:		Update the da(4) man page to cover the retry_count,
		default_timeout and minimum_cmd_size sysctl variables/loader
		tunables.  Remove references to /dev/r???, they aren't used
		any longer.

cd.9:		Update the cd(9) man page to describe the CD_Q_10_BYTE_ONLY
		quirk.

taskqueue.9:	Update the taskqueue(9) man page to describe the new thread
		task queue, and the taskqueue_swi_giant queue.

MFC after:	3 days

4 files changed, 132 insertions, 56 deletions

diff --git a/share/man/man4/cd.4 b/share/man/man4/cd.4
index 579fcea..7c491f5 100644
--- a/share/man/man4/cd.4
+++ b/share/man/man4/cd.4
@@ -25,7 +25,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd October 10, 1998
+.Dd September 2, 2003
 .Dt CD 4
 .Os
 .Sh NAME
@@ -370,6 +370,37 @@ Some work is planned to support
 some of the more common `broken'
 .Tn CD-ROM
 drives; however, this is not yet under way.
+.Pp
+The
+.Nm
+driver attempts to automatically determine whether the drive it is talking
+to supports 6 byte or 10 byte MODE SENSE/MODE SELECT operations.  Many
+.Tn SCSI
+drives only support 6 byte commands, and
+.Tn ATAPI
+drives only support 10 byte commands.
+The
+.Nm
+driver first attempts to determine whether the protocol in use typically
+supports 6 byte commands by issuing a CAM Path Inquiry CCB.
+It will then default to 6 byte or 10 byte commands as appropriate.
+After that, the
+.Nm
+driver defaults to using 6 byte commands (assuming the protocol the drive
+speaks claims to support 6 byte commands), until one fails with a
+.Tn SCSI 
+ILLEGAL REQUEST error.  Then it tries the 10 byte version of the command to
+see if that works instead.  Users can change the default via per-drive
+sysctl variables and loader tunables.  The variable names are the same in
+both instances:
+.Pp
+.Va kern.cam.cd.%d.minimum_cmd_size
+.Pp
+Where 
+.Dq %d
+is the unit number of the drive in question.  Valid minimum command sizes
+are 6 and 10.  Any value above 6 will be rounded to 10, and any value below
+6 will be rounded to 6.
 .Sh CHANGER OPERATION
 This driver has built-in support for LUN-based CD changers.
 A LUN-based CD
@@ -399,7 +430,7 @@ If there is no outstanding I/O for
 another LUN, the driver will allow indefinite access to a given LUN.
 .Pp
 The minimum and maximum time quanta are configurable via kernel options and
-also via sysctl variables.
+also via sysctl and kernel tunable variables.
 The kernel options are:
 .Pp
 .Bl -item -compact
@@ -409,7 +440,7 @@ The kernel options are:
 .Cd "options ""CHANGER_MAX_BUSY_SECONDS=11"""
 .El
 .Pp
-The sysctl variables are:
+The sysctl/kernel tunable variables are:
 .Pp
 .Bl -item -compact
 .It
diff --git a/share/man/man4/da.4 b/share/man/man4/da.4
index 835f4fd..c35b59f 100644
--- a/share/man/man4/da.4
+++ b/share/man/man4/da.4
@@ -25,7 +25,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd October 15, 1998
+.Dd September 2, 2003
 .Dt DA 4
 .Os
 .Sh NAME
@@ -196,6 +196,59 @@ The driver
 .Em will
 write the new disklabel to the disk.
 .El
+.Sh SYSCTL VARIABLES
+The following variables are available as both
+.Xr sysctl 8
+variables and
+.Xr loader 8
+tunables:
+.Bl -tag -width 12
+.It kern.cam.da.retry_count
+.Pp
+This variable determines how many times the
+.Nm
+driver will retry a READ or WRITE command.
+This does not affect the number of retries used during probe time or for
+the
+.Nm
+driver dump routine.
+This value currently defaults to 4.
+.It kern.cam.da.default_timeout
+.Pp
+This variable determines how long the
+.Nm
+driver will wait before timing out an outstanding command.
+The units for this value are seconds, and the default is currently 60
+seconds.
+.It kern.cam.da.%d.minimum_cmd_size
+.Pp
+This variable determines what the minimum READ/WRITE CDB size is for a
+given
+.Nm
+unit.
+(The %d above denotes the unit number of the
+.Nm
+driver instance, e.g. 1, 2, 4, 8, etc.)
+Valid minimum command size values are 6, 10, 12 and 16 bytes.
+The default is 6 bytes.
+.Pp
+The
+.Nm
+driver issues a CAM Path Inquiry CCB at probe time to determine whether the
+protocol the device in question speaks (e.g. ATAPI) typically doesn't allow
+6 byte commands.
+If it doesn't, the
+.Nm
+driver will default to using at least 10 byte CDBs.
+If a 6 byte READ or WRITE fails with an ILLEGAL REQUEST error, the
+.Nm
+driver will then increase the default CDB size for the device to 10 bytes and
+retry the command.
+CDB size is always
+chosen as the smallest READ/WRITE CDB that will satisfy the specified minimum
+command size, and the LBA and length of the READ or WRITE in question.
+(e.g., a write to an LBA larger than 2^32 will require a 16 byte CDB.)
+.El
 .Sh NOTES
 If a device becomes invalidated (media is removed, device becomes unresponsive)
 the disklabel and information held within the kernel about the device will
@@ -206,25 +259,9 @@ the last file descriptor referencing the old device is closed.
 During this period, all new open attempts will be rejected.
 .Sh FILES
 .Bl -tag -width /dev/rsdXXXXX -compact
-.It Pa /dev/rda Ns Ar u
-raw mode
-.Tn SCSI
-disk unit
-.Ar u ,
-accessed as an unpartitioned device
 .Sm off
 .It Pa /dev/da Ar u Pa s Ar n
 .Sm on
-block mode
-.Tn SCSI
-disk unit
-.Ar u ,
-slice
-.Ar n ,
-accessed as an unpartitioned device
-.Sm off
-.It Pa /dev/rda Ar u Pa s Ar n
-.Sm on
 raw mode
 .Tn SCSI
 disk unit
@@ -233,15 +270,6 @@ slice
 .Ar n ,
 accessed as an unpartitioned device
 .It Pa /dev/da Ns Ar u Ns Ar p
-block mode
-.Tn SCSI
-disk unit
-.Ar u ,
-first
-.Fx
-slice, partition
-.Ar p
-.It Pa /dev/rda Ns Ar u Ns Ar p
 raw mode
 .Tn SCSI
 disk unit
@@ -259,22 +287,6 @@ slice, partition
 .Ar p
 .Xc
 .Sm on
-block mode
-.Tn SCSI
-disk unit
-.Ar u ,
-.Ar n Ns th
-slice, partition
-.Ar p
-.Sm off
-.It Xo
-.Pa /dev/rda
-.Ar u
-.Pa s
-.Ar n
-.Ar p
-.Xc
-.Sm on
 raw mode
 .Tn SCSI
 disk unit
diff --git a/share/man/man9/cd.9 b/share/man/man9/cd.9
index 036fc80..49c2ede 100644
--- a/share/man/man9/cd.9
+++ b/share/man/man9/cd.9
@@ -27,7 +27,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd October 10, 1998
+.Dd September 2, 2003
 .Dt CD 9
 .Os
 .Sh NAME
@@ -74,6 +74,22 @@ In general, the driver will figure this out automatically when it sees a
 LUN greater than 0.  Setting this flag only has the effect of telling the
 driver to run the initial read capacity command for LUN 0 of the changer
 through the changer scheduling code.
+.It Dv CD_Q_10_BYTE_ONLY
+This flag tells the driver that the given device only accepts 10 byte MODE
+SENSE/MODE SELECT commands.  In general these types of quirks should not be
+added to the
+.Xr cd 4
+driver.  The reason is that the driver does several things to attempt to
+determine whether the drive in question needs 10 byte commands.  First, it
+issues a CAM Path Inquiry command to determine whether the protocol that
+the drive speaks typically only allows 10 byte commands.  (ATAPI and USB
+are two prominent examples of protocols where you generally only want to
+send 10 byte commands.)  Then, if it gets an ILLEGAL REQUEST error back
+from a 6 byte MODE SENSE or MODE SELECT command, it attempts to send the 10
+byte version of the command instead.  The only reason you would need a
+quirk is if your drive uses a protocol (e.g. 
+.Tn SCSI )
+that typically doesn't have a problem with 6 byte commands.
 .El
 .Sh FILES
 .Bl -tag -width /sys/cam/scsi/scsi_cd.c -compact
diff --git a/share/man/man9/taskqueue.9 b/share/man/man9/taskqueue.9
index 013f737..7d35458 100644
--- a/share/man/man9/taskqueue.9
+++ b/share/man/man9/taskqueue.9
@@ -174,26 +174,43 @@ argument to the macro is executed as a C statement,
 allowing any further initialisation to be performed
 (such as registering an interrupt handler etc.)
 .Pp
-The system provides a global taskqueue,
+The system provides three global taskqueues,
 .Va taskqueue_swi ,
-which is run via a software interrupt mechanism.
-To use this queue,
+.Va taskqueue_swi_giant ,
+and 
+.Va taskqueue_thread .
+The swi taskqueues are run via a software interrupt mechanism.
+The taskqueue_swi queue runs without the protection of the Giant kernel lock,
+and the taskqueue_swi_giant queue runs with the protection of the Giant
+kernel lock.
+The thread taskqueue runs in a kernel thread context, and tasks run from
+this thread do not run under the Giant kernel lock.
+If the caller wants to run under Giant, he should explicitly acquire and
+release Giant in his taskqueue handler routine.
+
+To use these queues,
 call
 .Fn taskqueue_enqueue
-with the value of the global variable
-.Va taskqueue_swi .
-The queue will be run at
-.\" XXX	This should be a cross-reference (Xr), but there is no MANLINKS
-.\"	entry for splsofttq.9 yet.
-.Fn splsofttq .
+with the value of the global taskqueue variable for the queue you wish to
+use (
+.Va taskqueue_swi ,
+.Va taskqueue_swi_giant ,
+or
+.Va taskqueue_thread
+).
 .Pp
-This queue can be used,
+This the software interrupt queues can be used,
 for instance, for implementing interrupt handlers which must perform a
 significant amount of processing in the handler.
 The hardware interrupt handler would perform minimal processing of the
 interrupt and then enqueue a task to finish the work.
 This reduces to a minimum
 the amount of time spent with interrupts disabled.
+.Pp
+The thread queue can be used, for instance, by interrupt level routines
+that need to call kernel functions that do things that can only be done
+from a thread context.
+(e.g., call malloc with the M_WAITOK flag.)
 .Sh HISTORY
 This interface first appeared in
 .Fx 5.0 .