MFC 303164: Add more documentation regarding unsafe AIO requests.

The asynchronous I/O changes made previously result in different
behavior out of the box. Previously all AIO requests failed with
ENOSYS / SIGSYS unless aio.ko was explicitly loaded. Now, some AIO
requests complete and others ("unsafe" requests) fail with EOPNOTSUPP.

Reword the introductory paragraph in aio(4) to add a general
description of AIO before describing the vfs.aio.enable_unsafe sysctl.

Remove the ENOSYS error description from aio_fsync(2), aio_read(2),
and aio_write(2) and replace it with a description of EOPNOTSUPP.

Remove the ENOSYS error description from aio_mlock(2).

Log a message to the system log the first time a process requests an
"unsafe" AIO request that fails with EOPNOTSUPP. This is modeled on
the log message used for processes using the legacy pty devices.

Approved by:	re (gjb)

6 files changed, 41 insertions, 28 deletions

diff --git a/lib/libc/sys/aio_fsync.2 b/lib/libc/sys/aio_fsync.2
index 7eb1a3b..52b47efb 100644
--- a/lib/libc/sys/aio_fsync.2
+++ b/lib/libc/sys/aio_fsync.2
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd July 15, 2016
+.Dd July 21, 2016
 .Dt AIO_FSYNC 2
 .Os
 .Sh NAME
@@ -107,10 +107,10 @@ The request was not queued because of system resource limitations.
 The asynchronous notification method in
 .Fa iocb->aio_sigevent.sigev_notify
 is invalid or not supported.
-.It Bq Er ENOSYS
-The
-.Fn aio_fsync
-system call is not supported.
+.It Bq Er EOPNOTSUPP
+Asynchronous file synchronization operations on the file descriptor
+.Fa iocb->aio_fildes
+are unsafe and unsafe asynchronous I/O operations are disabled.
 .It Bq Er EINVAL
 A value of the
 .Fa op
diff --git a/lib/libc/sys/aio_mlock.2 b/lib/libc/sys/aio_mlock.2
index c798a37..cc588b5 100644
--- a/lib/libc/sys/aio_mlock.2
+++ b/lib/libc/sys/aio_mlock.2
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd July 15, 2016
+.Dd July 21, 2016
 .Dt AIO_MLOCK 2
 .Os
 .Sh NAME
@@ -102,10 +102,6 @@ The request was not queued because of system resource limitations.
 The asynchronous notification method in
 .Fa iocb->aio_sigevent.sigev_notify
 is invalid or not supported.
-.It Bq Er ENOSYS
-The
-.Fn aio_mlock
-system call is not supported.
 .El
 .Pp
 If the request is successfully enqueued, but subsequently cancelled
diff --git a/lib/libc/sys/aio_read.2 b/lib/libc/sys/aio_read.2
index cc0133b..2f16809 100644
--- a/lib/libc/sys/aio_read.2
+++ b/lib/libc/sys/aio_read.2
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd July 15, 2016
+.Dd July 21, 2016
 .Dt AIO_READ 2
 .Os
 .Sh NAME
@@ -125,10 +125,10 @@ The request was not queued because of system resource limitations.
 The asynchronous notification method in
 .Fa iocb->aio_sigevent.sigev_notify
 is invalid or not supported.
-.It Bq Er ENOSYS
-The
-.Fn aio_read
-system call is not supported.
+.It Bq Er EOPNOTSUPP
+Asynchronous read operations on the file descriptor
+.Fa iocb->aio_fildes
+are unsafe and unsafe asynchronous I/O operations are disabled.
 .El
 .Pp
 The following conditions may be synchronously detected when the
diff --git a/lib/libc/sys/aio_write.2 b/lib/libc/sys/aio_write.2
index 6cbc2ab..414f98f 100644
--- a/lib/libc/sys/aio_write.2
+++ b/lib/libc/sys/aio_write.2
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd July 15, 2016
+.Dd July 21, 2016
 .Dt AIO_WRITE 2
 .Os
 .Sh NAME
@@ -129,10 +129,10 @@ The request was not queued because of system resource limitations.
 The asynchronous notification method in
 .Fa iocb->aio_sigevent.sigev_notify
 is invalid or not supported.
-.It Bq Er ENOSYS
-The
-.Fn aio_write
-system call is not supported.
+.It Bq Er EOPNOTSUPP
+Asynchronous write operations on the file descriptor
+.Fa iocb->aio_fildes
+are unsafe and unsafe asynchronous I/O operations are disabled.
 .El
 .Pp
 The following conditions may be synchronously detected when the
diff --git a/share/man/man4/aio.4 b/share/man/man4/aio.4
index 460b3c8..efcfe24 100644
--- a/share/man/man4/aio.4
+++ b/share/man/man4/aio.4
@@ -27,7 +27,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd July 15, 2016
+.Dd July 21, 2016
 .Dt AIO 4
 .Os
 .Sh NAME
@@ -37,17 +37,25 @@
 The
 .Nm
 facility provides system calls for asynchronous I/O.
-However, asynchronous I/O operations are only enabled for certain file
-types by default.
-Asynchronous I/O operations for other file types may block an AIO daemon
-indefinitely resulting in process and/or system hangs.
-Asynchronous I/O operations can be enabled for all file types by setting
+Asynchronous I/O operations are not completed synchronously by the
+calling thread.
+Instead, the calling thread invokes one system call to request an
+asynchronous I/O operation.
+The status of a completed request is retrieved later via a separate
+system call.
+.Pp
+Asynchronous I/O operations on some file descriptor types may block an
+AIO daemon indefinitely resulting in process and/or system hangs.
+Operations on these file descriptor types are considered
+.Dq unsafe
+and disabled by default.
+They can be enabled by setting
 the
 .Va vfs.aio.enable_unsafe
 sysctl node to a non-zero value.
 .Pp
 Asynchronous I/O operations on sockets and raw disk devices do not block
-indefinitely and are enabled by default.
+indefinitely and are always enabled.
 .Pp
 The
 .Nm
diff --git a/sys/kern/vfs_aio.c b/sys/kern/vfs_aio.c
index 1c75ff9..ead5e7c 100644
--- a/sys/kern/vfs_aio.c
+++ b/sys/kern/vfs_aio.c
@@ -53,6 +53,7 @@ __FBSDID("$FreeBSD$");
 #include <sys/syscall.h>
 #include <sys/sysent.h>
 #include <sys/sysctl.h>
+#include <sys/syslog.h>
 #include <sys/sx.h>
 #include <sys/taskqueue.h>
 #include <sys/vnode.h>
@@ -110,6 +111,11 @@ static int enable_aio_unsafe = 0;
 SYSCTL_INT(_vfs_aio, OID_AUTO, enable_unsafe, CTLFLAG_RW, &enable_aio_unsafe, 0,
     "Permit asynchronous IO on all file types, not just known-safe types");
 
+static unsigned int unsafe_warningcnt = 1;
+SYSCTL_UINT(_vfs_aio, OID_AUTO, unsafe_warningcnt, CTLFLAG_RW,
+    &unsafe_warningcnt, 0,
+    "Warnings that will be triggered upon failed IO requests on unsafe files");
+
 static int max_aio_procs = MAX_AIO_PROCS;
 SYSCTL_INT(_vfs_aio, OID_AUTO, max_aio_procs, CTLFLAG_RW, &max_aio_procs, 0,
     "Maximum number of kernel processes to use for handling async IO ");
@@ -1697,8 +1703,11 @@ queueit:
 				safe = true;
 		}
 	}
-	if (!(safe || enable_aio_unsafe))
+	if (!(safe || enable_aio_unsafe)) {
+		counted_warning(&unsafe_warningcnt,
+		    "is attempting to use unsafe AIO requests");
 		return (EOPNOTSUPP);
+	}
 
 	if (opcode == LIO_SYNC) {
 		AIO_LOCK(ki);