ioat(4): Add support for 'fence' bit with DMA_FENCE flag

Some classes of IOAT hardware prefetch reads.  DMA operations that
depend on the result of prior DMA operations must use the DMA_FENCE flag
to prevent stale reads.

(E.g., I've hit this personally on Broadwell-EP.  The Broadwell-DE has a
different IOAT unit that is documented to not pipeline DMA operations.)

Sponsored by:	EMC / Isilon Storage Division

1 files changed, 13 insertions, 2 deletions

diff --git a/share/man/man4/ioat.4 b/share/man/man4/ioat.4
index 10f2663..e71c2e1 100644
--- a/share/man/man4/ioat.4
+++ b/share/man/man4/ioat.4
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd January 7, 2016
+.Dd January 14, 2016
 .Dt IOAT 4
 .Os
 .Sh NAME
@@ -134,7 +134,7 @@ Null operations do nothing, but may be used to test the interrupt and callback
 mechanism.
 .Pp
 All operations can optionally trigger an interrupt at completion with the
-.Ar DMA_EN_INT
+.Ar DMA_INT_EN
 flag.
 For example, a user might submit multiple operations to the same channel and
 only enable an interrupt and callback for the last operation.
@@ -160,6 +160,17 @@ flag.
 .Ar DMA_NO_WAIT
 may return NULL.)
 .Pp
+Operations that depend on the result of prior operations should use
+.Ar DMA_FENCE .
+For example, such a scenario can happen when two related DMA operations are
+queued.
+First, a DMA copy to one location (A), followed directly by a DMA copy
+from A to B.
+In this scenario, some classes of I/OAT hardware may prefetch A for the second
+operation before it is written by the first operation.
+To avoid reading a stale value in sequences of dependent operations, use
+.Ar DMA_FENCE .
+.Pp
 All operations, as well as
 .Fn ioat_get_dmaengine ,
 can return NULL in special circumstances.