MFC r306486-r325841:

Update the mlx4, mlx4ib(4) and mlx4en(4) drivers to match FreeBSD-head
prior to the Linux 4.9 RoCE/infiniband upgrade.

Sponsored by:	Mellanox Technologies

MFC r306486:
Move the ConnectX-3 and ConnectX-2 driver from sys/ofed into sys/dev/mlx4
like other PCI network drivers. The sys/ofed directory is now mainly
reserved for generic infiniband code, with exception of the mthca driver.

- Add new manual page, mlx4en(4), describing how to configure and load
mlx4en.

- All relevant driver C-files are now prefixed mlx4, mlx4_en and
mlx4_ib respectivly to avoid object filename collisions when compiling
the kernel. This also fixes an issue with proper dependency file
generation for the C-files in question.

- Device mlxen is now device mlx4en and depends on device mlx4, see
mlx4en(4). Only the network device name remains unchanged.

- The mlx4 and mlx4en modules are now built by default on i386 and
amd64 targets. Only building the mlx4ib module depends on
WITH_OFED=YES .

Sponsored by:	Mellanox Technologies

MFC r310425:
mlx: avoid use of __DATE__ to make build reproducible

Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D8886

MFC r310488:
mlx(4): remove date from log message

Further to r310425, go one step further and just remove the date.

Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D8888

MFC r324200:
Implement SIOCGIFRSS{KEY,HASH} for the mlx4en(4) driver.

Differential Revision:	https://reviews.freebsd.org/D12176
Sponsored by:		Mellanox Technologies

MFC r324201:
Setup mbuf hash type properly when receiving IP packets in the mlx4en(4) driver.

Submitted by:		sephe@
Differential Revision:	https://reviews.freebsd.org/D12229
Sponsored by:		Mellanox Technologies

MFC r324621:
mlx4: use enum constants instead of const vars for case exprs

Follow up from r324201 to fix compilation with gcc, which complains
about non-ICE case expressions.

Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D12675

MFC r325841:
Update the mlx4 core and mlx4en(4) modules towards Linux v4.9.

Background:
The coming ibcore update forces an update of mlx4ib(4) which in turn requires
an updated mlx4 core module. This also affects the mlx4en(4) module because
commonly used APIs are updated. This commit is a middle step updating the
mlx4 modules towards the new ibcore.

This change contains no major new features.

Changes in mlx4:
  a) Improved error handling when mlx4 PCI devices are
  detached inside VMs.
  b) Major update of codebase towards Linux 4.9.

Changes in mlx4ib(4):
  a) Minimal changes needed in order to compile using the
  updated mlx4 core APIs.

Changes in mlx4en(4):
  a) Update flow steering code in mlx4en to use new APIs for
  registering MAC addresses and IP addresses.
  b) Update all statistics counters to be 64-bit.
  c) Minimal changes needed in order to compile using the
  updated mlx4 core APIs.

Sponsored by:	Mellanox Technologies

5 files changed, 60 insertions, 32 deletions

diff --git a/sys/modules/Makefile b/sys/modules/Makefile
index ab2f34a..178b785 100644
--- a/sys/modules/Makefile
+++ b/sys/modules/Makefile
@@ -245,7 +245,7 @@ SUBDIR=	\
 	mlx \
 	${_mlx4} \
 	${_mlx4ib} \
-	${_mlxen} \
+	${_mlx4en} \
 	${_mlx5} \
 	${_mlx5en} \
 	${_mlx5ib} \
@@ -666,17 +666,17 @@ _iwifw=		iwifw
 _iwmfw=		iwmfw
 _iwnfw=		iwnfw
 .endif
-.if ${MK_OFED} != "no" || defined(ALL_MODULES)
 _mlx4=		mlx4
-_mlx4ib=	mlx4ib
-_mlx5ib=	mlx5ib
-_mlxen=		mlxen
-.endif
 _mlx5=		mlx5
 .if (${MK_INET_SUPPORT} != "no" && ${MK_INET6_SUPPORT} != "no") || \
 	defined(ALL_MODULES)
+_mlx4en=	mlx4en
 _mlx5en=	mlx5en
 .endif
+.if ${MK_OFED} != "no" || defined(ALL_MODULES)
+_mlx4ib=	mlx4ib
+_mlx5ib=	mlx5ib
+.endif
 _mly=		mly
 .if ${MK_OFED} != "no" || defined(ALL_MODULES)
 _mthca=		mthca
diff --git a/sys/modules/mlx4/Makefile b/sys/modules/mlx4/Makefile
index ad0bcd2..d71f79b 100644
--- a/sys/modules/mlx4/Makefile
+++ b/sys/modules/mlx4/Makefile
@@ -1,20 +1,33 @@
 # $FreeBSD$
-.PATH:  ${SRCTOP}/sys/ofed/drivers/net/mlx4
+.PATH:  ${SRCTOP}/sys/dev/mlx4/mlx4_core
 
 KMOD=	mlx4
 SRCS=	device_if.h bus_if.h vnode_if.h pci_if.h \
 	opt_inet.h opt_inet6.h \
-	alloc.c catas.c cmd.c cq.c eq.c \
-	fw.c icm.c intf.c main.c mcg.c mr.c \
-	pd.c port.c profile.c qp.c reset.c sense.c \
-	srq.c resource_tracker.c sys_tune.c
+	mlx4_alloc.c \
+	mlx4_catas.c \
+	mlx4_cmd.c \
+	mlx4_cq.c \
+	mlx4_eq.c \
+	mlx4_fw.c \
+	mlx4_fw_qos.c \
+	mlx4_icm.c \
+	mlx4_intf.c \
+	mlx4_main.c \
+	mlx4_mcg.c \
+	mlx4_mr.c \
+	mlx4_pd.c \
+	mlx4_port.c \
+	mlx4_profile.c \
+	mlx4_qp.c \
+	mlx4_reset.c \
+	mlx4_sense.c \
+	mlx4_srq.c \
+	mlx4_resource_tracker.c
 
-CFLAGS+= -I${SRCTOP}/sys/ofed/drivers/net/mlx4
 CFLAGS+= -I${SRCTOP}/sys/ofed/include
 CFLAGS+= -I${SRCTOP}/sys/compat/linuxkpi/common/include
 
 .include <bsd.kmod.mk>
 
 CFLAGS+= -Wno-cast-qual -Wno-pointer-arith
-
-CWARNFLAGS.mcg.c=	-Wno-unused
diff --git a/sys/modules/mlx4en/Makefile b/sys/modules/mlx4en/Makefile
new file mode 100644
index 0000000..b5b57f4
--- /dev/null
+++ b/sys/modules/mlx4en/Makefile
@@ -0,0 +1,18 @@
+# $FreeBSD$
+.PATH:  ${SRCTOP}/sys/dev/mlx4/mlx4_en
+
+KMOD=	mlx4en
+SRCS=	device_if.h bus_if.h pci_if.h vnode_if.h \
+	opt_inet.h opt_inet6.h \
+	mlx4_en_cq.c \
+	mlx4_en_main.c \
+	mlx4_en_netdev.c \
+	mlx4_en_port.c \
+	mlx4_en_resources.c \
+	mlx4_en_rx.c \
+	mlx4_en_tx.c
+
+CFLAGS+= -I${SRCTOP}/sys/ofed/include
+CFLAGS+= -I${SRCTOP}/sys/compat/linuxkpi/common/include
+
+.include <bsd.kmod.mk>
diff --git a/sys/modules/mlx4ib/Makefile b/sys/modules/mlx4ib/Makefile
index 8ca2cf0..653b0f6 100644
--- a/sys/modules/mlx4ib/Makefile
+++ b/sys/modules/mlx4ib/Makefile
@@ -1,14 +1,24 @@
 # $FreeBSD$
-.PATH:  ${SRCTOP}/sys/ofed/drivers/infiniband/hw/mlx4
+.PATH:  ${SRCTOP}/sys/dev/mlx4/mlx4_ib
 
 KMOD=	mlx4ib
 SRCS=	device_if.h bus_if.h vnode_if.h pci_if.h \
 	opt_inet.h opt_inet6.h \
-	alias_GUID.c mcg.c sysfs.c ah.c cq.c \
-	mlx4_exp.c \
-	doorbell.c mad.c main.c mr.c qp.c srq.c wc.c cm.c
+	mlx4_ib_alias_GUID.c \
+	mlx4_ib_mcg.c \
+	mlx4_ib_sysfs.c \
+	mlx4_ib_ah.c \
+	mlx4_ib_cq.c \
+	mlx4_ib_exp.c \
+	mlx4_ib_doorbell.c \
+	mlx4_ib_mad.c \
+	mlx4_ib_main.c \
+	mlx4_ib_mr.c \
+	mlx4_ib_qp.c \
+	mlx4_ib_srq.c \
+	mlx4_ib_wc.c \
+	mlx4_ib_cm.c
 
-CFLAGS+= -I${SRCTOP}/sys/ofed/drivers/infiniband/hw/mlx4
 CFLAGS+= -I${SRCTOP}/sys/ofed/include
 CFLAGS+= -I${SRCTOP}/sys/compat/linuxkpi/common/include
 CFLAGS+= -DCONFIG_INFINIBAND_USER_MEM
diff --git a/sys/modules/mlxen/Makefile b/sys/modules/mlxen/Makefile
deleted file mode 100644
index 4c43998..0000000
--- a/sys/modules/mlxen/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# $FreeBSD$
-.PATH:  ${.CURDIR}/../../ofed/drivers/net/mlx4
-
-KMOD    = mlxen
-SRCS    = device_if.h bus_if.h pci_if.h vnode_if.h
-SRCS	+= en_cq.c en_main.c en_netdev.c en_port.c en_resources.c
-SRCS	+= en_rx.c en_tx.c
-SRCS	+= opt_inet.h opt_inet6.h
-CFLAGS+= -I${.CURDIR}/../../ofed/drivers/net/mlx4
-CFLAGS+= -I${.CURDIR}/../../ofed/include
-CFLAGS+= -I${.CURDIR}/../../compat/linuxkpi/common/include
-
-.include <bsd.kmod.mk>