MFC 303205,303722,305032,305752: Create VF devices on Chelsio T4/T5 NICs.

303205:
Add a driver to create VF devices on Chelsio T4/T5 NICs.

Chelsio NICs are a bit unique compared to some other NICs in that they
expose different functionality on different physical functions.  In
particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex').
However, PF4 is not able to create VF devices.  Instead, VFs are only
supported by physical functions 0 through 3.  This commit adds 't4iov'
and 't5iov' drivers that attach to PF0-3.

One extra wrinkle is that the iov devices cannot enable SR-IOV until the
firwmare has been initialized by the main PF4 driver.  To handle this
case, a new t4_if kobj interface has been added to permit cross-calls
between the PF drivers.  The PF4 driver notifies sibling drivers when it
is fully attached.  It also requests sibling drivers to detach before it
detaches.  Sibling drivers query the PF4 driver during their attach
routine to see if it is attached.  If not, the sibling drivers defer
their attach actions until the PF4 driver informs them it is attached.

VF devices are associated with a single port on the NIC.  VF devices
created from PF0 are associated with the first port on the NIC, VFs
from PF1 are associated with the second port, etc.  VF devices can
only be created from a PF device that has an associated port.  Thus,
on a 2-port card, VFs are only supported on PF0 and PF1.

303722:
Use the port device name for the iov device for Chelsio T4/T5 cards.

Chelsio T4/T5 adapters are multifunction cards.  The main driver uses
physical function 4 (PF4).  However, VF devices for SR-IOV are only
supported on physical functions 0 through 3, where PF0 creates VFs tied
to port 0, etc.  The t4iov/t5iov driver was previously added to
create VF devices for ports that are present on each adapter.  This
change uses the recently added pci_iov_attach_name() function to
name the character device in /dev/iov after the associated port on
the card (e.g. /dev/iov/cxl0 is used to create VFs that share the
cxl0 port).  With this in place, mark the t4iov/t5iov devices quiet
to prevent them from cluttering dmesg.

305032:
Use device_verbose() to undo device_quiet() when detaching from t[45]iovX.

The device quiet flag is not automatically reset on detach, so it is
inherited by other device drivers (e.g. when switching a device driver
over to ppt for PCI pass through).  Cope with this behavior by explicitly
marking the device verbose during detach so that the next driver can make
its own decision.

305752:
Remove explicit device_verbose() from the t4iov driver detach routine
now that this case is handled generically.

Sponsored by:	Chelsio Communications

1 files changed, 2 insertions, 0 deletions

diff --git a/sys/conf/files b/sys/conf/files
index 865b41d..0fb6735 100644
--- a/sys/conf/files
+++ b/sys/conf/files
@@ -1244,6 +1244,8 @@ dev/cxgb/sys/uipc_mvec.c	optional cxgb pci \
 	compile-with "${NORMAL_C} -I$S/dev/cxgb"
 dev/cxgb/cxgb_t3fw.c		optional cxgb cxgb_t3fw \
 	compile-with "${NORMAL_C} -I$S/dev/cxgb"
+dev/cxgbe/t4_iov.c		optional cxgbe pci \
+	compile-with "${NORMAL_C} -I$S/dev/cxgbe"
 dev/cxgbe/t4_mp_ring.c		optional cxgbe pci \
 	compile-with "${NORMAL_C} -I$S/dev/cxgbe"
 dev/cxgbe/t4_main.c		optional cxgbe pci \