MFC r308217, r308301, r311958, r312437, r318188, r318427, r318679

r308217:
Add a fallback to the device mapper logic.  We've seen systems in the field
that are apparently misconfigured by the manufacturer and cause the mapping
logic to fail.  The fallback allows drive numbers to be assigned based on the
PHY number that they're attached to.  Add sysctls and tunables to overrid
this new behavior, but they should be considered only necessary for debugging.

Reviewed by:	 imp, smh
Obtained from:	Netflix
MFC after:	3 days
Sponsored by:	D8403

r308301:
Record the LogInfo field when reporting the IOCStatus.  Helps in
debugging errors.

Submitted by:	slm
Obtained from:	Netflix
MFC after:	3 days

r311958:
Print out the number of queues/MSIx vectors.

Sponsored by:	Netflix

r312437:
Rework the debug print API. Event printing no longer gets special handling.
All of the printing from the tables file now has wrappers so that the
handling is cleaner and it's possible to print something out (say, during
development) without having to fight the global debug flags. This re-org
will also make it easier to have the tables be compiled out at build time
if desired.

Other than fixing some minor bugs, there are no user-visible changes from
this change

Sponsored by:	Netflix, Inc.
Differential Revision:	D9238

r318188:
Improve error messages during command timeout for the mpr and mps
drivers.

Sponsored by:	Netflix

r318427:
Add tri-mode support (SAS/SATA/PCIe).

This includes NVMe device support and adds support for the following adapters:
    SAS 3408
    SAS 3416
    SAS 3508
    SAS 3516
    SAS 3616
    SAS 3708
    SAS 3716

Reviewed by:    ken, scottl, asomers, mav
Approved by:	ken, scottl, mav
MFC after:      2 weeks
Relnotes:	yes
Differential Revision: https://reviews.freebsd.org/D10095

r318679:
Fix powerpc compiler error.

Approved by:	ken

1 files changed, 83 insertions, 8 deletions

diff --git a/share/man/man4/mpr.4 b/share/man/man4/mpr.4
index 370d659..e741d6e 100644
--- a/share/man/man4/mpr.4
+++ b/share/man/man4/mpr.4
@@ -1,8 +1,8 @@
 .\"
 .\" Copyright (c) 2010 Spectra Logic Corporation
 .\" Copyright (c) 2014 LSI Corp
-.\" Copyright (c) 2016 Avago Technologies
-.\" Copyright (c) 2016 Broadcom Ltd.
+.\" Copyright (c) 2017 Avago Technologies
+.\" Copyright (c) 2017 Broadcom Ltd.
 .\" All rights reserved.
 .\"
 .\" Redistribution and use in source and binary forms, with or without
@@ -38,12 +38,12 @@
 .\" $Id$
 .\" $FreeBSD$
 .\"
-.Dd July 6, 2016
+.Dd May 17, 2017
 .Dt MPR 4
 .Os
 .Sh NAME
 .Nm mpr
-.Nd "LSI Fusion-MPT 3 IT/IR 12Gb/s Serial Attached SCSI/SATA driver"
+.Nd "LSI Fusion-MPT 3/3.5 IT/IR 12Gb/s Serial Attached SCSI/SATA/PCIe driver"
 .Sh SYNOPSIS
 To compile this driver into the kernel, place these lines in the kernel
 configuration file:
@@ -62,8 +62,8 @@ mpr_load="YES"
 The
 .Nm
 driver provides support for Broadcom Ltd./Avago Tech (LSI)
-Fusion-MPT 3 IT/IR
-.Tn SAS
+Fusion-MPT 3/3.5 IT/IR
+.Tn SAS/PCIe
 controllers.
 .Sh HARDWARE
 These controllers are supported by the
@@ -81,6 +81,24 @@ Broadcom Ltd./Avago Tech (LSI) SAS 3108 (8 Port SAS)
 Broadcom Ltd./Avago Tech (LSI) SAS 3216 (16 Port SAS)
 .It
 Broadcom Ltd./Avago Tech (LSI) SAS 3224 (24 Port SAS)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3316 (16 Port SAS)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3324 (24 Port SAS)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3408 (8 Port SAS/PCIe)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3416 (16 Port SAS/PCIe)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3508 (8 Port SAS/PCIe)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3516 (16 Port SAS/PCIe)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3616 (16 Port SAS/PCIe)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3708 (8 Port SAS/PCIe)
+.It
+Broadcom Ltd./Avago Tech (LSI) SAS 3716 (16 Port SAS/PCIe)
 .El
 .Sh CONFIGURATION
 .Pp
@@ -158,6 +176,24 @@ dev.mpr.X.io_cmds_active
 variable.
 .Ed
 .Pp
+The current number of free PRP pages is stored in the
+dev.mpr.X.prp_pages_free
+.Xr sysctl 8
+variable.
+PRP pages are used by NVMe devices for I/O transfers, much like Scatter/Gather
+lists.
+.Pp
+The lowest number of free PRP pages seen since boot is stored in the
+dev.mpr.X.prp_pages_free_lowwater
+.Xr sysctl 8
+variable.
+.Pp
+The number of times that PRP page allocations have failed since boot is
+stored in the
+dev.mpr.X.prp_page_alloc_fail
+.Xr sysctl 8
+variable.
+.Pp
 To set the maximum number of pages that will be used per I/O for all adapters,
 set this tunable in
 .Xr loader.conf 5 :
@@ -231,13 +267,13 @@ Send SSU to HDDs, but not to SSDs.
 Send SSU to both HDDs and SSDs.
 .El
 .Pp
-To control the feature for a specific adapter, set this tunable value in
+To control this feature for a specific adapter, set this tunable value in
 .Xr loader.conf 5 :
 .Bd -literal -offset indent
 dev.mpr.X.enable_ssu
 .Ed
 .Pp
-The same set of values are valid when setting this tunable for all adapters.
+The same set of values are valid as when setting this tunable for all adapters.
 .Pp
 SATA disks that take several seconds to spin up and fail the SATA Identify
 command might not be discovered by the driver.
@@ -263,6 +299,45 @@ dev.mpr.X.spinup_wait_time=NNNN
 tunable.
 NNNN is the number of seconds to wait for SATA devices to spin up when they fail
 the initial SATA Identify command.
+.Pp
+The driver can map devices discovered by the adapter so that target IDs
+corresponding to a specific device persist across resets and reboots.
+In some cases it is possible for devices to lose their mapped IDs due to
+unexpected behavior from certain hardware, such as some types of enclosures.
+To overcome this problem, a tunable is provided that will force the driver to
+map devices using the Phy number associated with the device.
+This feature is not recommended if the topology includes multiple
+enclosures/expanders.
+If multiple enclosures/expanders are present in the topology, Phy numbers are
+repeated, causing all devices at these Phy numbers except the first device to
+fail enumeration.
+To control this feature for all adapters, set the
+.Bd -literal -offset indent
+hw.mpr.use_phy_num
+.Ed
+.Pp
+tunable in
+.Xr loader.conf 5
+to one of these values:
+.Bl -tag -width 6n -offset indent
+.It -1
+Only use Phy numbers to map devices and bypass the driver's mapping logic.
+.It 0
+Never use Phy numbers to map devices.
+.It 1
+Use Phy numbers to map devices, but only if the driver's mapping logic fails
+to map the device that is being enumerated.
+This is the default value.
+.El
+.Pp
+To control this feature for a specific adapter, set this tunable value in
+.Xr loader.conf 5 :
+.Bd -literal -offset indent
+dev.mpr.X.use_phy_num
+.Ed
+.Pp
+The same set of values are valid as when setting this tunable for all adapters.
+.Pp
 .Sh DEBUGGING
 To enable debugging prints from the
 .Nm