diff options
| author | ngie <ngie@FreeBSD.org> | 2015-12-29 19:56:26 +0000 |
|---|---|---|
| committer | ngie <ngie@FreeBSD.org> | 2015-12-29 19:56:26 +0000 |
| commit | 48b72481dba08c5185c1dd6a70b82b326be43d6b (patch) | |
| tree | bfa4f047c73b138c59732a37297737fa1ffd3e91 | |
| parent | f8c5cd12c37a92500da3f42f1d25bae9cc70e30a (diff) | |
| parent | c0ae58548ada98f553239a091b8764ea9e1b474b (diff) | |
| download | FreeBSD-src-48b72481dba08c5185c1dd6a70b82b326be43d6b.zip FreeBSD-src-48b72481dba08c5185c1dd6a70b82b326be43d6b.tar.gz | |
MFhead @ r292618
577 files changed, 21646 insertions, 6360 deletions
diff --git a/MAINTAINERS b/MAINTAINERS index 5ea2483..07d0c16 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -24,6 +24,15 @@ maintainer of a sub-system is to check recent logs for that directory or sub-system. *** +*** +Maintainers are encouraged to visit: + https://reviews.freebsd.org/herald + +and configure notifications for parts of the tree which they maintain. +Notifications can automatically be sent when someone proposes a revision or +makes a commit to the specified subtree. +*** + subsystem login notes ----------------------------- atf freebsd-testing,jmmv,ngie Pre-commit review requested. @@ -47,7 +56,9 @@ isci(4) jimharris Pre-commit review requested. iwm(4) adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org iwn(4) adrian Pre-commit review requested, send to freebsd-wireless@freebsd.org kqueue jmg Pre-commit review requested. Documentation Required. +libdpv dteske Pre-commit review requested. Keep in sync with dpv(1). libfetch des Pre-commit review requested. +libfigpar dteske Pre-commit review requested. libpam des Pre-commit review requested. linprocfs des Pre-commit review requested. lpr gad Pre-commit review requested, particularly for @@ -76,6 +87,7 @@ sh(1) jilles Pre-commit review requested. This also applies compiled in as builtins. share/mk imp, bapt, bdrewery, emaste, sjg Make is hard. share/mk/*.test.mk freebsd-testing,ngie (same list as share/mk too) Pre-commit review requested. +sys/boot/forth dteske Pre-commit review requested. sys/compat/linuxkpi hselasky If in doubt, ask. sys/dev/e1000 erj Pre-commit phabricator review requested. sys/dev/ixgbe erj Pre-commit phabricator review requested. @@ -85,74 +97,8 @@ sys/dev/usb hselasky If in doubt, ask. sys/netinet/ip_carp.c glebius Pre-commit review recommended. sys/netpfil/pf kp,glebius Pre-commit review recommended. tests freebsd-testing,ngie Pre-commit review requested. +usr.sbin/bsdconfig dteske Pre-commit phabricator review requested. +usr.sbin/dpv dteske Pre-commit review requested. Keep in sync with libdpv. usr.sbin/pkg pkg@ Please coordinate behavior or flag changes with pkg team. +usr.sbin/sysrc dteske Pre-commit phabricator review requested. Keep in sync with bsdconfig(8) sysrc.subr. vmm(4) neel,grehan Pre-commit review requested. ----- OLD ---- -libc/posix1e rwatson Pre-commit review requested. -POSIX.1e ACLs rwatson Pre-commit review requested. -UFS EAs rwatson Pre-commit review requested. -MAC Framework rwatson Pre-commit review requested. -MAC Modules rwatson Pre-commit review requested. -contrib/openbsm rwatson Pre-commit review requested. -sys/security/audit rwatson Pre-commit review requested. -ahc(4) gibbs Pre-commit review requested. -ahd(4) gibbs Pre-commit review requested. -cdboot jhb Pre-commit review requested. -pxeboot jhb Pre-commit review requested. -witness jhb Pre-commit review requested. -CAM gibbs, - ken Pre-commit review requested. send to scsi@freebsd.org -devstat(9) ken Pre-commit review requested. -camcontrol(8) ken Pre-commit review requested. -libcam ken Pre-commit review requested. -libdevstat ken Pre-commit review requested. -iostat(8) ken Pre-commit review requested. -cd(4) ken Pre-commit review requested. -pass(4) ken Pre-commit review requested. -ch(4) ken Pre-commit review requested. -em(4) jfv Pre-commit review requested. -nvi peter Try not to break it. -libz peter Try not to break it. -groff ru Recommends pre-commit review. -ipfw ipfw Pre-commit review preferred. send to ipfw@freebsd.org -drm rnoland Just keep me informed of changes, try not to break it. -unifdef(1) fanf Pre-commit review requested. -ntp roberto Pre-commit review requested. -inetd dwmalone Recommends pre-commit review. -contrib/smbfs bp Open for in-tree committs. In case of functional - changes pre-commit review requested. -file obrien Insists to keep file blocked from other's unapproved - commits -contrib/bzip2 obrien Pre-commit review required. -geom freebsd-geom@FreeBSD.org -geom_concat pjd Pre-commit review preferred. -geom_gate pjd Pre-commit review preferred. -geom_label pjd Pre-commit review preferred. -geom_mirror pjd Pre-commit review preferred. -geom_nop pjd Pre-commit review preferred. -geom_raid3 pjd Pre-commit review preferred. -geom_shsec pjd Pre-commit review preferred. -geom_stripe pjd Pre-commit review preferred. -geom_zero pjd Pre-commit review preferred. -sbin/geom pjd Pre-commit review preferred. -zfs freebsd-fs@FreeBSD.org -linux emul emulation Please discuss changes here. -bs{diff,patch} cperciva Pre-commit review requested. -portsnap cperciva Pre-commit review requested. -freebsd-update cperciva Pre-commit review requested. -sys/netgraph/bluetooth emax Pre-commit review preferred. -lib/libbluetooth emax Pre-commit review preferred. -lib/libsdp emax Pre-commit review preferred. -usr.bin/bluetooth emax Pre-commit review preferred. -usr.sbin/bluetooth emax Pre-commit review preferred. -share/zoneinfo edwin Heads-up appreciated, since our data is coming - from a third party source. -usr.sbin/zic edwin Heads-up appreciated, since this code is - maintained by a third party source. -lib/libc/stdtime edwin Heads-up appreciated, since parts of this code - is maintained by a third party source. -sbin/routed bms Pre-commit review; notify vendor at rhyolite.com -cmx daniel@roe.ch Pre-commit review preferred. -filemon obrien Pre-commit review preferred. -sysdoc trhodes Pre-commit review preferred. - diff --git a/Makefile.inc1 b/Makefile.inc1 index 8a43898..a26df46 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -48,10 +48,11 @@ .error "Both TARGET and TARGET_ARCH must be defined." .endif +LOCALBASE?= /usr/local + # Cross toolchain changes must be in effect before bsd.compiler.mk # so that gets the right CC, and pass CROSS_TOOLCHAIN to submakes. .if defined(CROSS_TOOLCHAIN) -LOCALBASE?= /usr/local .include "${LOCALBASE}/share/toolchains/${CROSS_TOOLCHAIN}.mk" CROSSENV+=CROSS_TOOLCHAIN="${CROSS_TOOLCHAIN}" .endif @@ -1314,7 +1315,7 @@ packagekernel: # Build the API documentation with doxygen # doxygen: .PHONY - @if [ ! -x `/usr/bin/which doxygen` ]; then \ + @if [ ! -x ${LOCALBASE}/bin/doxygen ]; then \ echo "You need doxygen (devel/doxygen) to generate the API documentation of the kernel." | /usr/bin/fmt; \ exit 1; \ fi @@ -1676,7 +1677,7 @@ NXBMAKE= ${NXBENV} ${MAKE} \ -DNO_PIC MK_PROFILE=no -DNO_SHARED \ -DNO_CPU_CFLAGS MK_WARNS=no MK_CTF=no \ MK_CLANG_EXTRAS=no MK_CLANG_FULL=no \ - MK_LLDB=no + MK_LLDB=no MK_DEBUG_FILES=no # native-xtools is the current target for qemu-user cross builds of ports # via poudriere and the imgact_binmisc kernel module. diff --git a/ObsoleteFiles.inc b/ObsoleteFiles.inc index a07758b..cf5d2a2 100644 --- a/ObsoleteFiles.inc +++ b/ObsoleteFiles.inc @@ -38,6 +38,88 @@ # xargs -n1 | sort | uniq -d; # done +# 20151225: new clang import which bumps version from 3.7.0 to 3.7.1. +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/allocator_interface.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/asan_interface.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/common_interface_defs.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/coverage_interface.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/dfsan_interface.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/linux_syscall_hooks.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/lsan_interface.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/msan_interface.h +OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/tsan_interface_atomic.h +OLD_DIRS+=usr/lib/clang/3.7.0/include/sanitizer +OLD_FILES+=usr/lib/clang/3.7.0/include/__stddef_max_align_t.h +OLD_FILES+=usr/lib/clang/3.7.0/include/__wmmintrin_aes.h +OLD_FILES+=usr/lib/clang/3.7.0/include/__wmmintrin_pclmul.h +OLD_FILES+=usr/lib/clang/3.7.0/include/adxintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/altivec.h +OLD_FILES+=usr/lib/clang/3.7.0/include/ammintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/arm_acle.h +OLD_FILES+=usr/lib/clang/3.7.0/include/arm_neon.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx2intrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512bwintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512cdintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512dqintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512erintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512fintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512vlbwintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512vldqintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avx512vlintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/avxintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/bmi2intrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/bmiintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/cpuid.h +OLD_FILES+=usr/lib/clang/3.7.0/include/cuda_builtin_vars.h +OLD_FILES+=usr/lib/clang/3.7.0/include/emmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/f16cintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/fma4intrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/fmaintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/fxsrintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/htmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/htmxlintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/ia32intrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/immintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/lzcntintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/mm3dnow.h +OLD_FILES+=usr/lib/clang/3.7.0/include/mm_malloc.h +OLD_FILES+=usr/lib/clang/3.7.0/include/mmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/module.modulemap +OLD_FILES+=usr/lib/clang/3.7.0/include/nmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/pmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/popcntintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/prfchwintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/rdseedintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/rtmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/s390intrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/shaintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/smmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/tbmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/tmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/vadefs.h +OLD_FILES+=usr/lib/clang/3.7.0/include/vecintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/wmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/x86intrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/xmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/xopintrin.h +OLD_FILES+=usr/lib/clang/3.7.0/include/xtestintrin.h +OLD_DIRS+=usr/lib/clang/3.7.0/include +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan-i386.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan_cxx-i386.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan_cxx-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.profile-arm.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.profile-i386.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.profile-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.safestack-i386.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.safestack-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan_standalone-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan_standalone_cxx-i386.a +OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan_standalone_cxx-x86_64.a +OLD_DIRS+=usr/lib/clang/3.7.0/lib/freebsd +OLD_DIRS+=usr/lib/clang/3.7.0/lib +OLD_DIRS+=usr/lib/clang/3.7.0 # 20151130: libelf moved from /usr/lib to /lib (libkvm dependency in r291406) OLD_LIBS+=usr/lib/libelf.so.2 # 20151115: Fox bad upgrade scheme diff --git a/bin/pax/pat_rep.c b/bin/pax/pat_rep.c index e8d17db..516519e 100644 --- a/bin/pax/pat_rep.c +++ b/bin/pax/pat_rep.c @@ -878,7 +878,7 @@ rep_name(char *name, int *nlen, int prnt) * (the user already saw that substitution go by) */ pt = rephead; - (void)strcpy(buf1, name); + (void)strlcpy(buf1, name, sizeof(buf1)); inpt = buf1; outpt = nname; endpt = outpt + PAXPATHLEN; diff --git a/bin/sh/mknodes.c b/bin/sh/mknodes.c index 81f42a2..1667d77 100644 --- a/bin/sh/mknodes.c +++ b/bin/sh/mknodes.c @@ -89,7 +89,6 @@ static struct str *nodestr[MAXTYPES]; /* type of structure used by the node */ static int nstr; /* number of structures */ static struct str str[MAXTYPES]; /* the structures */ static struct str *curstr; /* current structure */ -static FILE *infp; static char line[1024]; static int linno; static char *linep; @@ -102,7 +101,7 @@ static void outfunc(FILE *, int); static void indent(int, FILE *); static int nextfield(char *); static void skipbl(void); -static int readline(void); +static int readline(FILE *); static void error(const char *, ...) __printf0like(1, 2) __dead2; static char *savestr(const char *); @@ -110,17 +109,19 @@ static char *savestr(const char *); int main(int argc, char *argv[]) { + FILE *infp; + if (argc != 3) error("usage: mknodes file"); - infp = stdin; if ((infp = fopen(argv[1], "r")) == NULL) error("Can't open %s: %s", argv[1], strerror(errno)); - while (readline()) { + while (readline(infp)) { if (line[0] == ' ' || line[0] == '\t') parsefield(); else if (line[0] != '\0') parsenode(); } + fclose(infp); output(argv[2]); exit(0); } @@ -253,6 +254,10 @@ output(char *file) fputs("union node *getfuncnode(struct funcdef *);\n", hfile); fputs("void reffunc(struct funcdef *);\n", hfile); fputs("void unreffunc(struct funcdef *);\n", hfile); + if (ferror(hfile)) + error("Can't write to nodes.h"); + if (fclose(hfile)) + error("Can't close nodes.h"); fputs(writer, cfile); while (fgets(line, sizeof line, patfile) != NULL) { @@ -266,6 +271,11 @@ output(char *file) else fputs(line, cfile); } + fclose(patfile); + if (ferror(cfile)) + error("Can't write to nodes.c"); + if (fclose(cfile)) + error("Can't close nodes.c"); } @@ -401,7 +411,7 @@ skipbl(void) static int -readline(void) +readline(FILE *infp) { char *p; diff --git a/bin/sh/tests/expansion/Makefile b/bin/sh/tests/expansion/Makefile index e359ae7..fb672da 100644 --- a/bin/sh/tests/expansion/Makefile +++ b/bin/sh/tests/expansion/Makefile @@ -90,5 +90,6 @@ FILES+= trim5.0 FILES+= trim6.0 FILES+= trim7.0 FILES+= trim8.0 +FILES+= trim9.0 .include <bsd.test.mk> diff --git a/bin/sh/tests/expansion/trim9.0 b/bin/sh/tests/expansion/trim9.0 new file mode 100644 index 0000000..47c825a --- /dev/null +++ b/bin/sh/tests/expansion/trim9.0 @@ -0,0 +1,61 @@ +# $FreeBSD$ + +# POSIX does not specify these but they occasionally occur in the wild. +# This just serves to keep working what currently works. + +failures='' +ok='' + +testcase() { + code="$1" + expected="$2" + oIFS="$IFS" + eval "$code" + IFS='|' + result="$#|$*" + IFS="$oIFS" + if [ "x$result" = "x$expected" ]; then + ok=x$ok + else + failures=x$failures + echo "For $code, expected $expected actual $result" + fi +} + +testcase 'shift $#; set -- "${*#Q}"' '1|' +testcase 'shift $#; set -- "${*##Q}"' '1|' +testcase 'shift $#; set -- "${*%Q}"' '1|' +testcase 'shift $#; set -- "${*%%Q}"' '1|' +testcase 'set -- Q R; set -- "${*#Q}"' '1| R' +testcase 'set -- Q R; set -- "${*##Q}"' '1| R' +testcase 'set -- Q R; set -- "${*%R}"' '1|Q ' +testcase 'set -- Q R; set -- "${*%%R}"' '1|Q ' +testcase 'set -- Q R; set -- "${*#S}"' '1|Q R' +testcase 'set -- Q R; set -- "${*##S}"' '1|Q R' +testcase 'set -- Q R; set -- "${*%S}"' '1|Q R' +testcase 'set -- Q R; set -- "${*%%S}"' '1|Q R' +testcase 'set -- Q R; set -- ${*#Q}' '1|R' +testcase 'set -- Q R; set -- ${*##Q}' '1|R' +testcase 'set -- Q R; set -- ${*%R}' '1|Q' +testcase 'set -- Q R; set -- ${*%%R}' '1|Q' +testcase 'set -- Q R; set -- ${*#S}' '2|Q|R' +testcase 'set -- Q R; set -- ${*##S}' '2|Q|R' +testcase 'set -- Q R; set -- ${*%S}' '2|Q|R' +testcase 'set -- Q R; set -- ${*%%S}' '2|Q|R' +testcase 'set -- Q R; set -- ${@#Q}' '1|R' +testcase 'set -- Q R; set -- ${@##Q}' '1|R' +testcase 'set -- Q R; set -- ${@%R}' '1|Q' +testcase 'set -- Q R; set -- ${@%%R}' '1|Q' +testcase 'set -- Q R; set -- ${@#S}' '2|Q|R' +testcase 'set -- Q R; set -- ${@##S}' '2|Q|R' +testcase 'set -- Q R; set -- ${@%S}' '2|Q|R' +testcase 'set -- Q R; set -- ${@%%S}' '2|Q|R' +testcase 'set -- Q R; set -- "${@#Q}"' '2||R' +testcase 'set -- Q R; set -- "${@%R}"' '2|Q|' +testcase 'set -- Q R; set -- "${@%%R}"' '2|Q|' +testcase 'set -- Q R; set -- "${@#S}"' '2|Q|R' +testcase 'set -- Q R; set -- "${@##S}"' '2|Q|R' +testcase 'set -- Q R; set -- "${@%S}"' '2|Q|R' +testcase 'set -- Q R; set -- "${@%%S}"' '2|Q|R' + +test "x$failures" = x diff --git a/cddl/contrib/opensolaris/cmd/zdb/zdb.c b/cddl/contrib/opensolaris/cmd/zdb/zdb.c index 7f25880..346d817 100644 --- a/cddl/contrib/opensolaris/cmd/zdb/zdb.c +++ b/cddl/contrib/opensolaris/cmd/zdb/zdb.c @@ -3657,7 +3657,8 @@ main(int argc, char **argv) kernel_init(FREAD); g_zfs = libzfs_init(); - ASSERT(g_zfs != NULL); + if (g_zfs == NULL) + fatal("Fail to initialize zfs"); if (dump_all) verbose = MAX(verbose, 1); diff --git a/cddl/sbin/zfs/Makefile b/cddl/sbin/zfs/Makefile index b1a104c..8e9ceb3 100644 --- a/cddl/sbin/zfs/Makefile +++ b/cddl/sbin/zfs/Makefile @@ -22,6 +22,6 @@ CFLAGS+= -I${.CURDIR}/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs CFLAGS+= -I${.CURDIR}/../../../sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${.CURDIR}/../../../sys/cddl/contrib/opensolaris/common/zfs -LIBADD= geom jail nvpair umem util uutil zfs_core zfs +LIBADD= jail nvpair uutil zfs_core zfs .include <bsd.prog.mk> diff --git a/cddl/sbin/zpool/Makefile b/cddl/sbin/zpool/Makefile index 11adad2..f900e0d 100644 --- a/cddl/sbin/zpool/Makefile +++ b/cddl/sbin/zpool/Makefile @@ -27,6 +27,6 @@ CFLAGS+= -I${.CURDIR}/../../../sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${.CURDIR}/../../../cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${.CURDIR}/../../../cddl/contrib/opensolaris/cmd/stat/common -LIBADD= avl geom nvpair umem util uutil zfs_core zfs +LIBADD= geom nvpair uutil zfs .include <bsd.prog.mk> diff --git a/cddl/usr.sbin/zdb/Makefile b/cddl/usr.sbin/zdb/Makefile index 4fbcaea..94f0412 100644 --- a/cddl/usr.sbin/zdb/Makefile +++ b/cddl/usr.sbin/zdb/Makefile @@ -24,7 +24,7 @@ CFLAGS+= -I${.CURDIR}/../../../sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${.CURDIR}/../../../cddl/contrib/opensolaris/head CFLAGS+= -I${.CURDIR}/../../lib/libumem -LIBADD= geom m nvpair pthread umem uutil zfs_core zfs zpool +LIBADD= nvpair umem uutil zfs zpool # Since there are many asserts in this program, it makes no sense to compile # it without debugging. diff --git a/cddl/usr.sbin/zhack/Makefile b/cddl/usr.sbin/zhack/Makefile index c42a49e..2973e5c 100644 --- a/cddl/usr.sbin/zhack/Makefile +++ b/cddl/usr.sbin/zhack/Makefile @@ -23,7 +23,7 @@ CFLAGS+= -I${.CURDIR}/../../../sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${.CURDIR}/../../../cddl/contrib/opensolaris/head CFLAGS+= -I${.CURDIR}/../../lib/libumem -LIBADD= geom m nvpair pthread umem uutil zfs_core zfs zpool +LIBADD= nvpair zfs zpool CFLAGS+= -DDEBUG=1 #DEBUG_FLAGS+= -g diff --git a/contrib/binutils/bfd/elf32-arm.c b/contrib/binutils/bfd/elf32-arm.c index ca40eac..753cdcc 100644 --- a/contrib/binutils/bfd/elf32-arm.c +++ b/contrib/binutils/bfd/elf32-arm.c @@ -5800,7 +5800,7 @@ elf32_arm_final_link_relocate (reloc_howto_type * howto, if (globals->use_rel) { addend = ((insn >> 4) & 0xf000) | (insn & 0xfff); - signed_addend = (addend ^ 0x10000) - 0x10000; + signed_addend = (addend ^ 0x8000) - 0x8000; } value += signed_addend; diff --git a/contrib/bmake/ChangeLog b/contrib/bmake/ChangeLog index c620c84..c3a4241 100644 --- a/contrib/bmake/ChangeLog +++ b/contrib/bmake/ChangeLog @@ -1,3 +1,9 @@ +2015-12-20 Simon J. Gerraty <sjg@bad.crufty.net> + + * Makefile (MAKE_VERSION): 20151220 + Merge with NetBSD make, pick up + o suff.c: re-initialize suffNull when clearing suffixes. + 2015-12-01 Simon J. Gerraty <sjg@bad.crufty.net> * Makefile (MAKE_VERSION): 20151201 diff --git a/contrib/bmake/Makefile b/contrib/bmake/Makefile index 861e8f5..625c254 100644 --- a/contrib/bmake/Makefile +++ b/contrib/bmake/Makefile @@ -1,7 +1,7 @@ -# $Id: Makefile,v 1.48 2015/12/02 00:36:42 sjg Exp $ +# $Id: Makefile,v 1.49 2015/12/20 22:54:40 sjg Exp $ # Base version on src date -MAKE_VERSION= 20151201 +MAKE_VERSION= 20151220 PROG= bmake diff --git a/contrib/bmake/mk/ChangeLog b/contrib/bmake/mk/ChangeLog index 965ff4d..3c728c8 100644 --- a/contrib/bmake/mk/ChangeLog +++ b/contrib/bmake/mk/ChangeLog @@ -1,3 +1,9 @@ +2015-12-12 Simon J. Gerraty <sjg@bad.crufty.net> + + * install-mk (MK_VERSION): 20151212 + * auto.obj.mk: do not require MAKEOBJDIRPREFIX to exist. + only apply :tA to __objdir when comparing to .OBJDIR + 2015-11-14 Simon J. Gerraty <sjg@bad.crufty.net> * install-mk (MK_VERSION): 20151111 diff --git a/contrib/bmake/mk/auto.obj.mk b/contrib/bmake/mk/auto.obj.mk index fee3de2..2395f64 100644 --- a/contrib/bmake/mk/auto.obj.mk +++ b/contrib/bmake/mk/auto.obj.mk @@ -1,4 +1,4 @@ -# $Id: auto.obj.mk,v 1.11 2015/06/16 06:28:21 sjg Exp $ +# $Id: auto.obj.mk,v 1.12 2015/12/16 01:57:06 sjg Exp $ # # @(#) Copyright (c) 2004, Simon J. Gerraty # @@ -40,12 +40,12 @@ MKOBJDIRS= auto .if !defined(NOOBJ) && !defined(NO_OBJ) && ${MKOBJDIRS:Uno} == auto # Use __objdir here so it is easier to tweak without impacting # the logic. -.if !empty(MAKEOBJDIRPREFIX) && exists(${MAKEOBJDIRPREFIX}) +.if !empty(MAKEOBJDIRPREFIX) __objdir?= ${MAKEOBJDIRPREFIX}${.CURDIR} .endif __objdir?= ${MAKEOBJDIR:Uobj} -__objdir:= ${__objdir:tA} -.if ${.OBJDIR} != ${__objdir} +__objdir:= ${__objdir} +.if ${.OBJDIR:tA} != ${__objdir:tA} # We need to chdir, make the directory if needed .if !exists(${__objdir}/) && \ (${.TARGETS} == "" || ${.TARGETS:Nclean*:N*clean:Ndestroy*} != "") @@ -53,11 +53,10 @@ __objdir:= ${__objdir:tA} __objdir_made != echo ${__objdir}/; umask ${OBJDIR_UMASK:U002}; \ ${ECHO_TRACE} "[Creating objdir ${__objdir}...]" >&2; \ ${Mkdirs}; Mkdirs ${__objdir} -__objdir:= ${__objdir:tA} .endif # This causes make to use the specified directory as .OBJDIR .OBJDIR: ${__objdir} -.if ${.OBJDIR} != ${__objdir} && ${__objdir_made:Uno:M${__objdir}/*} != "" +.if ${.OBJDIR:tA} != ${__objdir:tA} && ${__objdir_made:Uno:M${__objdir}/*} != "" .error could not use ${__objdir}: .OBJDIR=${.OBJDIR} .endif .endif diff --git a/contrib/bmake/mk/install-mk b/contrib/bmake/mk/install-mk index d2500da..0c3db00 100644 --- a/contrib/bmake/mk/install-mk +++ b/contrib/bmake/mk/install-mk @@ -55,7 +55,7 @@ # Simon J. Gerraty <sjg@crufty.net> # RCSid: -# $Id: install-mk,v 1.117 2015/11/14 18:09:57 sjg Exp $ +# $Id: install-mk,v 1.118 2015/12/16 01:57:06 sjg Exp $ # # @(#) Copyright (c) 1994 Simon J. Gerraty # @@ -70,7 +70,7 @@ # sjg@crufty.net # -MK_VERSION=20151111 +MK_VERSION=20151212 OWNER= GROUP= MODE=444 diff --git a/contrib/bmake/os.sh b/contrib/bmake/os.sh index 93d4387..1dd394f 100755 --- a/contrib/bmake/os.sh +++ b/contrib/bmake/os.sh @@ -17,7 +17,7 @@ # Simon J. Gerraty <sjg@crufty.net> # RCSid: -# $Id: os.sh,v 1.49 2015/10/25 00:05:40 sjg Exp $ +# $Id: os.sh,v 1.50 2015/12/17 17:06:29 sjg Exp $ # # @(#) Copyright (c) 1994 Simon J. Gerraty # @@ -56,10 +56,10 @@ Which() { case "$1" in /*) test $t $1 && echo $1;; *) - # some shells cannot correctly handle `IFS` - # in conjunction with the for loop. - _dirs=`IFS=:; echo ${2:-$PATH}` - for d in $_dirs + # some shells cannot correctly handle `IFS` + # in conjunction with the for loop. + _dirs=`IFS=:; echo ${2:-$PATH}` + for d in $_dirs do test $t $d/$1 && { echo $d/$1; break; } done @@ -70,11 +70,11 @@ Which() { # tr is insanely non-portable wrt char classes, so we need to # spell out the alphabet. sed y/// would work too. toUpper() { - ${TR:-tr} abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ + ${TR:-tr} abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ } toLower() { - ${TR:-tr} ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz + ${TR:-tr} ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz } K= @@ -91,7 +91,7 @@ SunOS) export CHOWN # Great! Solaris keeps moving arch(1) - # should just bite the bullet and use uname -p + # should just bite the bullet and use uname -p arch=`Which arch /usr/bin:/usr/ucb` MAILER=/usr/ucb/Mail @@ -105,8 +105,8 @@ SunOS) MACHINE=$MACHINE_ARCH ;; 4*) - MACHINE_ARCH=`arch` - ;; + MACHINE_ARCH=`arch` + ;; 5*) K=-k LOCAL_FS=ufs @@ -116,8 +116,8 @@ SunOS) # overwriting an existing file!!!!! We want one that works! test -x /usr/xpg4/bin/ln && LN=${LN:-/usr/xpg4/bin/ln} # wonderful, 5.8's tr again require's []'s - # but /usr/xpg4/bin/tr causes problems if LC_COLLATE is set! - # use toUpper/toLower instead. + # but /usr/xpg4/bin/tr causes problems if LC_COLLATE is set! + # use toUpper/toLower instead. ;; esac case "$OS/$MACHINE_ARCH" in @@ -142,9 +142,9 @@ SunOS) SHARE_ARCH=$OS/$HOST ;; OpenBSD) - arch=`Which arch /usr/bin:/usr/ucb:$PATH` - MACHINE_ARCH=`$arch -s` - ;; + arch=`Which arch /usr/bin:/usr/ucb:$PATH` + MACHINE_ARCH=`$arch -s` + ;; esac NAWK=awk export NAWK @@ -218,17 +218,25 @@ export HOST_TARGET case `echo -n .` in -n*) N=; C="\c";; *) N=-n; C=;; esac -export HOSTNAME HOST +Echo() { + case "$1" in + -n) _n=$N _c=$C; shift;; + *) _n= _c=;; + esac + echo $_n "$@" $_c +} + +export HOSTNAME HOST export OS MACHINE MACHINE_ARCH OSREL OSMAJOR LOCAL_FS TMP_DIRS MAILER N C K PS_AXC export LN SHARE_ARCH TR case /$0 in */os.sh) - for v in $* + for v in $* do - eval vv=\$$v - echo "$v='$vv'" + eval vv=\$$v + echo "$v='$vv'" done - ;; + ;; esac diff --git a/contrib/bmake/suff.c b/contrib/bmake/suff.c index 15a3cc8..db69643 100644 --- a/contrib/bmake/suff.c +++ b/contrib/bmake/suff.c @@ -1,4 +1,4 @@ -/* $NetBSD: suff.c,v 1.74 2015/10/11 04:51:24 sjg Exp $ */ +/* $NetBSD: suff.c,v 1.75 2015/12/20 22:44:10 sjg Exp $ */ /* * Copyright (c) 1988, 1989, 1990, 1993 @@ -69,14 +69,14 @@ */ #ifndef MAKE_NATIVE -static char rcsid[] = "$NetBSD: suff.c,v 1.74 2015/10/11 04:51:24 sjg Exp $"; +static char rcsid[] = "$NetBSD: suff.c,v 1.75 2015/12/20 22:44:10 sjg Exp $"; #else #include <sys/cdefs.h> #ifndef lint #if 0 static char sccsid[] = "@(#)suff.c 8.4 (Berkeley) 3/21/94"; #else -__RCSID("$NetBSD: suff.c,v 1.74 2015/10/11 04:51:24 sjg Exp $"); +__RCSID("$NetBSD: suff.c,v 1.75 2015/12/20 22:44:10 sjg Exp $"); #endif #endif /* not lint */ #endif @@ -553,7 +553,20 @@ Suff_ClearSuffixes(void) #endif sufflist = Lst_Init(FALSE); sNum = 0; - suffNull = emptySuff; + if (suffNull) + SuffFree(suffNull); + emptySuff = suffNull = bmake_malloc(sizeof(Suff)); + + suffNull->name = bmake_strdup(""); + suffNull->nameLen = 0; + suffNull->searchPath = Lst_Init(FALSE); + Dir_Concat(suffNull->searchPath, dirSearchPath); + suffNull->children = Lst_Init(FALSE); + suffNull->parents = Lst_Init(FALSE); + suffNull->ref = Lst_Init(FALSE); + suffNull->sNum = sNum++; + suffNull->flags = SUFF_NULL; + suffNull->refCount = 1; } /*- @@ -2524,32 +2537,18 @@ Suff_SetNull(char *name) void Suff_Init(void) { - sufflist = Lst_Init(FALSE); #ifdef CLEANUP suffClean = Lst_Init(FALSE); #endif srclist = Lst_Init(FALSE); transforms = Lst_Init(FALSE); - sNum = 0; /* * Create null suffix for single-suffix rules (POSIX). The thing doesn't * actually go on the suffix list or everyone will think that's its * suffix. */ - emptySuff = suffNull = bmake_malloc(sizeof(Suff)); - - suffNull->name = bmake_strdup(""); - suffNull->nameLen = 0; - suffNull->searchPath = Lst_Init(FALSE); - Dir_Concat(suffNull->searchPath, dirSearchPath); - suffNull->children = Lst_Init(FALSE); - suffNull->parents = Lst_Init(FALSE); - suffNull->ref = Lst_Init(FALSE); - suffNull->sNum = sNum++; - suffNull->flags = SUFF_NULL; - suffNull->refCount = 1; - + Suff_ClearSuffixes(); } diff --git a/contrib/bsnmp/snmpd/action.c b/contrib/bsnmp/snmpd/action.c index ebba0f5..e8fd715 100644 --- a/contrib/bsnmp/snmpd/action.c +++ b/contrib/bsnmp/snmpd/action.c @@ -60,29 +60,6 @@ static const struct asn_oid #endif /* - * Get a string value from the KERN sysctl subtree. - */ -static char * -act_getkernstring(int id) -{ - int mib[2]; - size_t len; - char *string; - - mib[0] = CTL_KERN; - mib[1] = id; - if (sysctl(mib, 2, NULL, &len, NULL, 0) != 0) - return (NULL); - if ((string = malloc(len)) == NULL) - return (NULL); - if (sysctl(mib, 2, string, &len, NULL, 0) != 0) { - free(string); - return (NULL); - } - return (string); -} - -/* * Get an integer value from the KERN sysctl subtree. */ static char * diff --git a/contrib/bsnmp/snmpd/main.c b/contrib/bsnmp/snmpd/main.c index e7257b5..cbd1893 100644 --- a/contrib/bsnmp/snmpd/main.c +++ b/contrib/bsnmp/snmpd/main.c @@ -119,26 +119,30 @@ static struct lmodules modules_start = TAILQ_HEAD_INITIALIZER(modules_start); struct community_list community_list = TAILQ_HEAD_INITIALIZER(community_list); /* list of all known USM users */ -struct usm_userlist usm_userlist = SLIST_HEAD_INITIALIZER(usm_userlist); +static struct usm_userlist usm_userlist = SLIST_HEAD_INITIALIZER(usm_userlist); /* A list of all VACM users configured, including v1, v2c and v3 */ -struct vacm_userlist vacm_userlist = SLIST_HEAD_INITIALIZER(vacm_userlist); +static struct vacm_userlist vacm_userlist = + SLIST_HEAD_INITIALIZER(vacm_userlist); /* A list of all VACM groups */ -struct vacm_grouplist vacm_grouplist = SLIST_HEAD_INITIALIZER(vacm_grouplist); +static struct vacm_grouplist vacm_grouplist = + SLIST_HEAD_INITIALIZER(vacm_grouplist); static struct vacm_group vacm_default_group = { .groupname = "", }; /* The list of configured access entries */ -struct vacm_accesslist vacm_accesslist = TAILQ_HEAD_INITIALIZER(vacm_accesslist); +static struct vacm_accesslist vacm_accesslist = + TAILQ_HEAD_INITIALIZER(vacm_accesslist); /* The list of configured views */ -struct vacm_viewlist vacm_viewlist = SLIST_HEAD_INITIALIZER(vacm_viewlist); +static struct vacm_viewlist vacm_viewlist = + SLIST_HEAD_INITIALIZER(vacm_viewlist); /* The list of configured contexts */ -struct vacm_contextlist vacm_contextlist = +static struct vacm_contextlist vacm_contextlist = SLIST_HEAD_INITIALIZER(vacm_contextlist); /* list of all installed object resources */ diff --git a/contrib/bsnmp/snmpd/trap.c b/contrib/bsnmp/snmpd/trap.c index bf02708..aa9dd93 100644 --- a/contrib/bsnmp/snmpd/trap.c +++ b/contrib/bsnmp/snmpd/trap.c @@ -60,15 +60,15 @@ struct trapsink_list trapsink_list = TAILQ_HEAD_INITIALIZER(trapsink_list); /* List of target addresses */ -struct target_addresslist target_addresslist = +static struct target_addresslist target_addresslist = SLIST_HEAD_INITIALIZER(target_addresslist); /* List of target parameters */ -struct target_paramlist target_paramlist = +static struct target_paramlist target_paramlist = SLIST_HEAD_INITIALIZER(target_paramlist); /* List of notification targets */ -struct target_notifylist target_notifylist = +static struct target_notifylist target_notifylist = SLIST_HEAD_INITIALIZER(target_notifylist); static const struct asn_oid oid_begemotTrapSinkTable = diff --git a/contrib/gcc/config/rs6000/sysv4.h b/contrib/gcc/config/rs6000/sysv4.h index 0b1eb1e..9b36485 100644 --- a/contrib/gcc/config/rs6000/sysv4.h +++ b/contrib/gcc/config/rs6000/sysv4.h @@ -282,7 +282,9 @@ do { \ #define RESTORE_FP_SUFFIX "_l" /* Type used for ptrdiff_t, as a string used in a declaration. */ +#ifndef PTRDIFF_TYPE #define PTRDIFF_TYPE "int" +#endif /* Type used for wchar_t, as a string used in a declaration. */ /* Override svr4.h definition. */ diff --git a/contrib/libexecinfo/backtrace.3 b/contrib/libexecinfo/backtrace.3 index d0dfb95..061a68a 100644 --- a/contrib/libexecinfo/backtrace.3 +++ b/contrib/libexecinfo/backtrace.3 @@ -28,7 +28,7 @@ .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE .\" POSSIBILITY OF SUCH DAMAGE. .\" -.Dd November 3, 2015 +.Dd December 12, 2015 .Dt BACKTRACE 3 .Os .Sh NAME @@ -47,7 +47,7 @@ .Ft "char **" .Fn backtrace_symbols_fmt "void * const *addrlist" "size_t len" "const char *fmt" .Ft int -.Fn backtrace_symbols_fd_fmt "void * const *addrlist" "size_t len" "const char *fmt" "int fd" +.Fn backtrace_symbols_fd_fmt "void * const *addrlist" "size_t len" "int fd" "const char *fmt" .Sh DESCRIPTION The .Fn backtrace @@ -85,7 +85,7 @@ the /proc filesystem is available to determine the executable path. The difference of the symbol address and the address element printed using 0x%tx. .It Dv D -The difference of the symbol addresss and the address element printed using +The difference of the symbol address and the address element printed using +0x%tx if non-zero, or nothing if zero. .It Dv f The filename of the symbol as determined by diff --git a/contrib/llvm/include/llvm-c/Core.h b/contrib/llvm/include/llvm-c/Core.h index 1529007..9dbcbfe 100644 --- a/contrib/llvm/include/llvm-c/Core.h +++ b/contrib/llvm/include/llvm-c/Core.h @@ -2675,7 +2675,8 @@ LLVMValueRef LLVMBuildInvoke(LLVMBuilderRef, LLVMValueRef Fn, LLVMBasicBlockRef Then, LLVMBasicBlockRef Catch, const char *Name); LLVMValueRef LLVMBuildLandingPad(LLVMBuilderRef B, LLVMTypeRef Ty, - unsigned NumClauses, const char *Name); + LLVMValueRef PersFn, unsigned NumClauses, + const char *Name); LLVMValueRef LLVMBuildResume(LLVMBuilderRef B, LLVMValueRef Exn); LLVMValueRef LLVMBuildUnreachable(LLVMBuilderRef); diff --git a/contrib/llvm/include/llvm/CodeGen/CommandFlags.h b/contrib/llvm/include/llvm/CodeGen/CommandFlags.h index 4b2e0b0..bedb7d5 100644 --- a/contrib/llvm/include/llvm/CodeGen/CommandFlags.h +++ b/contrib/llvm/include/llvm/CodeGen/CommandFlags.h @@ -21,7 +21,7 @@ #include "llvm/IR/Intrinsics.h" #include "llvm/IR/Module.h" #include "llvm/MC/MCTargetOptionsCommandFlags.h" -#include "llvm//MC/SubtargetFeature.h" +#include "llvm/MC/SubtargetFeature.h" #include "llvm/Support/CodeGen.h" #include "llvm/Support/CommandLine.h" #include "llvm/Support/Host.h" diff --git a/contrib/llvm/lib/CodeGen/AsmPrinter/WinException.cpp b/contrib/llvm/lib/CodeGen/AsmPrinter/WinException.cpp index 71c7781..a2b9316 100644 --- a/contrib/llvm/lib/CodeGen/AsmPrinter/WinException.cpp +++ b/contrib/llvm/lib/CodeGen/AsmPrinter/WinException.cpp @@ -169,7 +169,7 @@ void WinException::endFunction(const MachineFunction *MF) { Asm->OutStreamer->PopSection(); } - if (shouldEmitMoves) + if (shouldEmitMoves || shouldEmitPersonality) Asm->OutStreamer->EmitWinCFIEndProc(); } diff --git a/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp index 21ab072..fbc8f1e 100644 --- a/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp +++ b/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp @@ -439,7 +439,7 @@ ExpandUnalignedLoad(LoadSDNode *LD, SelectionDAG &DAG, ISD::ANY_EXTEND, dl, VT, Result); ValResult = Result; - ChainResult = Chain; + ChainResult = newLoad.getValue(1); return; } diff --git a/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp b/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp index a7392fa..54cfaf5 100644 --- a/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp +++ b/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp @@ -1010,6 +1010,8 @@ SDValue DAGTypeLegalizer::GetVectorElementPointer(SDValue VecPtr, EVT EltVT, // Calculate the element offset and add it to the pointer. unsigned EltSize = EltVT.getSizeInBits() / 8; // FIXME: should be ABI size. + assert(EltSize * 8 == EltVT.getSizeInBits() && + "Converting bits to bytes lost precision"); Index = DAG.getNode(ISD::MUL, dl, Index.getValueType(), Index, DAG.getConstant(EltSize, dl, Index.getValueType())); diff --git a/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp b/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp index 4348ab7..51cd661 100644 --- a/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp +++ b/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp @@ -1528,9 +1528,25 @@ SDValue DAGTypeLegalizer::SplitVecOp_EXTRACT_VECTOR_ELT(SDNode *N) { if (CustomLowerNode(N, N->getValueType(0), true)) return SDValue(); - // Store the vector to the stack. - EVT EltVT = VecVT.getVectorElementType(); + // Make the vector elements byte-addressable if they aren't already. SDLoc dl(N); + EVT EltVT = VecVT.getVectorElementType(); + if (EltVT.getSizeInBits() < 8) { + SmallVector<SDValue, 4> ElementOps; + for (unsigned i = 0; i < VecVT.getVectorNumElements(); ++i) { + ElementOps.push_back(DAG.getAnyExtOrTrunc( + DAG.getNode(ISD::EXTRACT_VECTOR_ELT, dl, EltVT, Vec, + DAG.getConstant(i, dl, MVT::i8)), + dl, MVT::i8)); + } + + EltVT = MVT::i8; + VecVT = EVT::getVectorVT(*DAG.getContext(), EltVT, + VecVT.getVectorNumElements()); + Vec = DAG.getNode(ISD::BUILD_VECTOR, dl, VecVT, ElementOps); + } + + // Store the vector to the stack. SDValue StackPtr = DAG.CreateStackTemporary(VecVT); SDValue Store = DAG.getStore(DAG.getEntryNode(), dl, Vec, StackPtr, MachinePointerInfo(), false, false, 0); diff --git a/contrib/llvm/lib/IR/AsmWriter.cpp b/contrib/llvm/lib/IR/AsmWriter.cpp index adc620d..b553f11 100644 --- a/contrib/llvm/lib/IR/AsmWriter.cpp +++ b/contrib/llvm/lib/IR/AsmWriter.cpp @@ -794,6 +794,10 @@ void SlotTracker::processFunction() { ST_DEBUG("begin processFunction!\n"); fNext = 0; + // Process function metadata if it wasn't hit at the module-level. + if (!ShouldInitializeAllMetadata) + processFunctionMetadata(*TheFunction); + // Add all the function arguments with no names. for(Function::const_arg_iterator AI = TheFunction->arg_begin(), AE = TheFunction->arg_end(); AI != AE; ++AI) @@ -807,8 +811,6 @@ void SlotTracker::processFunction() { if (!BB.hasName()) CreateFunctionSlot(&BB); - processFunctionMetadata(*TheFunction); - for (auto &I : BB) { if (!I.getType()->isVoidTy() && !I.hasName()) CreateFunctionSlot(&I); @@ -836,11 +838,11 @@ void SlotTracker::processFunction() { void SlotTracker::processFunctionMetadata(const Function &F) { SmallVector<std::pair<unsigned, MDNode *>, 4> MDs; - for (auto &BB : F) { - F.getAllMetadata(MDs); - for (auto &MD : MDs) - CreateMetadataSlot(MD.second); + F.getAllMetadata(MDs); + for (auto &MD : MDs) + CreateMetadataSlot(MD.second); + for (auto &BB : F) { for (auto &I : BB) processInstructionMetadata(I); } diff --git a/contrib/llvm/lib/IR/Core.cpp b/contrib/llvm/lib/IR/Core.cpp index e0e729d..0eb88a9 100644 --- a/contrib/llvm/lib/IR/Core.cpp +++ b/contrib/llvm/lib/IR/Core.cpp @@ -2257,7 +2257,14 @@ LLVMValueRef LLVMBuildInvoke(LLVMBuilderRef B, LLVMValueRef Fn, } LLVMValueRef LLVMBuildLandingPad(LLVMBuilderRef B, LLVMTypeRef Ty, - unsigned NumClauses, const char *Name) { + LLVMValueRef PersFn, unsigned NumClauses, + const char *Name) { + // The personality used to live on the landingpad instruction, but now it + // lives on the parent function. For compatibility, take the provided + // personality and put it on the parent function. + if (PersFn) + unwrap(B)->GetInsertBlock()->getParent()->setPersonalityFn( + cast<Function>(unwrap(PersFn))); return wrap(unwrap(B)->CreateLandingPad(unwrap(Ty), NumClauses, Name)); } diff --git a/contrib/llvm/lib/LTO/LTOCodeGenerator.cpp b/contrib/llvm/lib/LTO/LTOCodeGenerator.cpp index 149ec6a..25ae4ac 100644 --- a/contrib/llvm/lib/LTO/LTOCodeGenerator.cpp +++ b/contrib/llvm/lib/LTO/LTOCodeGenerator.cpp @@ -63,14 +63,21 @@ const char* LTOCodeGenerator::getVersionString() { #endif } +static void handleLTODiagnostic(const DiagnosticInfo &DI) { + DiagnosticPrinterRawOStream DP(errs()); + DI.print(DP); + errs() << "\n"; +} + LTOCodeGenerator::LTOCodeGenerator() - : Context(getGlobalContext()), IRLinker(new Module("ld-temp.o", Context)) { + : Context(getGlobalContext()), IRLinker(new Module("ld-temp.o", Context), + handleLTODiagnostic) { initializeLTOPasses(); } LTOCodeGenerator::LTOCodeGenerator(std::unique_ptr<LLVMContext> Context) : OwnedContext(std::move(Context)), Context(*OwnedContext), - IRLinker(new Module("ld-temp.o", *OwnedContext)) { + IRLinker(new Module("ld-temp.o", *OwnedContext), handleLTODiagnostic) { initializeLTOPasses(); } diff --git a/contrib/llvm/lib/MC/MCContext.cpp b/contrib/llvm/lib/MC/MCContext.cpp index c601c56..a85796c 100644 --- a/contrib/llvm/lib/MC/MCContext.cpp +++ b/contrib/llvm/lib/MC/MCContext.cpp @@ -82,6 +82,7 @@ void MCContext::reset() { UsedNames.clear(); Symbols.clear(); + SectionSymbols.clear(); Allocator.Reset(); Instances.clear(); CompilationDir.clear(); diff --git a/contrib/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp b/contrib/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp index 709d753..0a5309b 100644 --- a/contrib/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp @@ -264,6 +264,12 @@ void AMDGPUAsmPrinter::getSIProgramInfo(SIProgramInfo &ProgInfo, for (const MachineBasicBlock &MBB : MF) { for (const MachineInstr &MI : MBB) { // TODO: CodeSize should account for multiple functions. + + // TODO: Should we count size of debug info? + if (MI.isDebugValue()) + continue; + + // FIXME: This is reporting 0 for many instructions. CodeSize += MI.getDesc().Size; unsigned numOperands = MI.getNumOperands(); diff --git a/contrib/llvm/lib/Target/AMDGPU/AMDGPUPromoteAlloca.cpp b/contrib/llvm/lib/Target/AMDGPU/AMDGPUPromoteAlloca.cpp index 4a65bfc..57b7a73 100644 --- a/contrib/llvm/lib/Target/AMDGPU/AMDGPUPromoteAlloca.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/AMDGPUPromoteAlloca.cpp @@ -134,13 +134,17 @@ static Value* GEPToVectorIndex(GetElementPtrInst *GEP) { // // TODO: Check isTriviallyVectorizable for calls and handle other // instructions. -static bool canVectorizeInst(Instruction *Inst) { +static bool canVectorizeInst(Instruction *Inst, User *User) { switch (Inst->getOpcode()) { case Instruction::Load: - case Instruction::Store: case Instruction::BitCast: case Instruction::AddrSpaceCast: return true; + case Instruction::Store: { + // Must be the stored pointer operand, not a stored value. + StoreInst *SI = cast<StoreInst>(Inst); + return SI->getPointerOperand() == User; + } default: return false; } @@ -166,7 +170,7 @@ static bool tryPromoteAllocaToVector(AllocaInst *Alloca) { for (User *AllocaUser : Alloca->users()) { GetElementPtrInst *GEP = dyn_cast<GetElementPtrInst>(AllocaUser); if (!GEP) { - if (!canVectorizeInst(cast<Instruction>(AllocaUser))) + if (!canVectorizeInst(cast<Instruction>(AllocaUser), Alloca)) return false; WorkList.push_back(AllocaUser); @@ -184,7 +188,7 @@ static bool tryPromoteAllocaToVector(AllocaInst *Alloca) { GEPVectorIdx[GEP] = Index; for (User *GEPUser : AllocaUser->users()) { - if (!canVectorizeInst(cast<Instruction>(GEPUser))) + if (!canVectorizeInst(cast<Instruction>(GEPUser), AllocaUser)) return false; WorkList.push_back(GEPUser); @@ -240,7 +244,12 @@ static bool collectUsesWithPtrTypes(Value *Val, std::vector<Value*> &WorkList) { for (User *User : Val->users()) { if(std::find(WorkList.begin(), WorkList.end(), User) != WorkList.end()) continue; - if (isa<CallInst>(User)) { + if (CallInst *CI = dyn_cast<CallInst>(User)) { + // TODO: We might be able to handle some cases where the callee is a + // constantexpr bitcast of a function. + if (!CI->getCalledFunction()) + return false; + WorkList.push_back(User); continue; } @@ -250,6 +259,12 @@ static bool collectUsesWithPtrTypes(Value *Val, std::vector<Value*> &WorkList) { if (UseInst && UseInst->getOpcode() == Instruction::PtrToInt) return false; + if (StoreInst *SI = dyn_cast_or_null<StoreInst>(UseInst)) { + // Reject if the stored value is not the pointer operand. + if (SI->getPointerOperand() != Val) + return false; + } + if (!User->getType()->isPointerTy()) continue; diff --git a/contrib/llvm/lib/Target/AMDGPU/AMDGPURegisterInfo.td b/contrib/llvm/lib/Target/AMDGPU/AMDGPURegisterInfo.td index 835a146..ba0490a 100644 --- a/contrib/llvm/lib/Target/AMDGPU/AMDGPURegisterInfo.td +++ b/contrib/llvm/lib/Target/AMDGPU/AMDGPURegisterInfo.td @@ -14,8 +14,7 @@ let Namespace = "AMDGPU" in { foreach Index = 0-15 in { - // Indices are used in a variety of ways here, so don't set a size/offset. - def sub#Index : SubRegIndex<-1, -1>; + def sub#Index : SubRegIndex<32, !shl(Index, 5)>; } def INDIRECT_BASE_ADDR : Register <"INDIRECT_BASE_ADDR">; diff --git a/contrib/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUAsmBackend.cpp b/contrib/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUAsmBackend.cpp index 468563c..4434d9b 100644 --- a/contrib/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUAsmBackend.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUAsmBackend.cpp @@ -71,12 +71,26 @@ void AMDGPUMCObjectWriter::writeObject(MCAssembler &Asm, } } +static unsigned getFixupKindNumBytes(unsigned Kind) { + switch (Kind) { + case FK_Data_1: + return 1; + case FK_Data_2: + return 2; + case FK_Data_4: + return 4; + case FK_Data_8: + return 8; + default: + llvm_unreachable("Unknown fixup kind!"); + } +} + void AMDGPUAsmBackend::applyFixup(const MCFixup &Fixup, char *Data, unsigned DataSize, uint64_t Value, bool IsPCRel) const { switch ((unsigned)Fixup.getKind()) { - default: llvm_unreachable("Unknown fixup kind"); case AMDGPU::fixup_si_sopp_br: { uint16_t *Dst = (uint16_t*)(Data + Fixup.getOffset()); *Dst = (Value - 4) / 4; @@ -96,6 +110,24 @@ void AMDGPUAsmBackend::applyFixup(const MCFixup &Fixup, char *Data, *Dst = Value + 4; break; } + default: { + // FIXME: Copied from AArch64 + unsigned NumBytes = getFixupKindNumBytes(Fixup.getKind()); + if (!Value) + return; // Doesn't change encoding. + MCFixupKindInfo Info = getFixupKindInfo(Fixup.getKind()); + + // Shift the value into position. + Value <<= Info.TargetOffset; + + unsigned Offset = Fixup.getOffset(); + assert(Offset + NumBytes <= DataSize && "Invalid fixup offset!"); + + // For each byte of the fragment that the fixup touches, mask in the + // bits from the fixup value. + for (unsigned i = 0; i != NumBytes; ++i) + Data[Offset + i] |= uint8_t((Value >> (i * 8)) & 0xff); + } } } diff --git a/contrib/llvm/lib/Target/AMDGPU/SIISelLowering.cpp b/contrib/llvm/lib/Target/AMDGPU/SIISelLowering.cpp index 099b0b1..c2db9ff 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIISelLowering.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/SIISelLowering.cpp @@ -157,6 +157,7 @@ SITargetLowering::SITargetLowering(TargetMachine &TM, setTruncStoreAction(MVT::i64, MVT::i32, Expand); setTruncStoreAction(MVT::v8i32, MVT::v8i16, Expand); + setTruncStoreAction(MVT::v16i32, MVT::v16i8, Expand); setTruncStoreAction(MVT::v16i32, MVT::v16i16, Expand); setOperationAction(ISD::LOAD, MVT::i1, Custom); @@ -2252,10 +2253,8 @@ MachineSDNode *SITargetLowering::buildScratchRSRC(SelectionDAG &DAG, SDValue Ptr) const { const SIInstrInfo *TII = static_cast<const SIInstrInfo *>(Subtarget->getInstrInfo()); - uint64_t Rsrc = TII->getDefaultRsrcDataFormat() | AMDGPU::RSRC_TID_ENABLE | - 0xffffffff; // Size - return buildRSRC(DAG, DL, Ptr, 0, Rsrc); + return buildRSRC(DAG, DL, Ptr, 0, TII->getScratchRsrcWords23()); } SDValue SITargetLowering::CreateLiveInRegister(SelectionDAG &DAG, diff --git a/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp index 1891061..cfd2c42 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp @@ -2778,3 +2778,16 @@ uint64_t SIInstrInfo::getDefaultRsrcDataFormat() const { return RsrcDataFormat; } + +uint64_t SIInstrInfo::getScratchRsrcWords23() const { + uint64_t Rsrc23 = getDefaultRsrcDataFormat() | + AMDGPU::RSRC_TID_ENABLE | + 0xffffffff; // Size; + + // If TID_ENABLE is set, DATA_FORMAT specifies stride bits [14:17]. + // Clear them unless we want a huge stride. + if (ST.getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS) + Rsrc23 &= ~AMDGPU::RSRC_DATA_FORMAT; + + return Rsrc23; +} diff --git a/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.h b/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.h index 015ea12..5053786 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.h +++ b/contrib/llvm/lib/Target/AMDGPU/SIInstrInfo.h @@ -353,7 +353,7 @@ public: } uint64_t getDefaultRsrcDataFormat() const; - + uint64_t getScratchRsrcWords23() const; }; namespace AMDGPU { diff --git a/contrib/llvm/lib/Target/AMDGPU/SIInstructions.td b/contrib/llvm/lib/Target/AMDGPU/SIInstructions.td index f78ffd7..e0eeea9 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIInstructions.td +++ b/contrib/llvm/lib/Target/AMDGPU/SIInstructions.td @@ -1548,6 +1548,12 @@ defm V_WRITELANE_B32 : VOP2SI_3VI_m < // These instructions only exist on SI and CI let SubtargetPredicate = isSICI in { +let isCommutable = 1 in { +defm V_MAC_LEGACY_F32 : VOP2InstSI <vop2<0x6>, "v_mac_legacy_f32", + VOP_F32_F32_F32 +>; +} // End isCommutable = 1 + defm V_MIN_LEGACY_F32 : VOP2InstSI <vop2<0xd>, "v_min_legacy_f32", VOP_F32_F32_F32, AMDGPUfmin_legacy >; @@ -1562,12 +1568,6 @@ defm V_LSHL_B32 : VOP2InstSI <vop2<0x19>, "v_lshl_b32", VOP_I32_I32_I32>; } // End isCommutable = 1 } // End let SubtargetPredicate = SICI -let isCommutable = 1 in { -defm V_MAC_LEGACY_F32 : VOP2_VI3_Inst <vop23<0x6, 0x28e>, "v_mac_legacy_f32", - VOP_F32_F32_F32 ->; -} // End isCommutable = 1 - defm V_BFM_B32 : VOP2_VI3_Inst <vop23<0x1e, 0x293>, "v_bfm_b32", VOP_I32_I32_I32 >; diff --git a/contrib/llvm/lib/Target/AMDGPU/SIPrepareScratchRegs.cpp b/contrib/llvm/lib/Target/AMDGPU/SIPrepareScratchRegs.cpp index 0a7f684..2cd600d 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIPrepareScratchRegs.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/SIPrepareScratchRegs.cpp @@ -135,8 +135,7 @@ bool SIPrepareScratchRegs::runOnMachineFunction(MachineFunction &MF) { unsigned ScratchRsrcReg = RS.scavengeRegister(&AMDGPU::SReg_128RegClass, 0); - uint64_t Rsrc = AMDGPU::RSRC_DATA_FORMAT | AMDGPU::RSRC_TID_ENABLE | - 0xffffffff; // Size + uint64_t Rsrc23 = TII->getScratchRsrcWords23(); unsigned Rsrc0 = TRI->getSubReg(ScratchRsrcReg, AMDGPU::sub0); unsigned Rsrc1 = TRI->getSubReg(ScratchRsrcReg, AMDGPU::sub1); @@ -152,11 +151,11 @@ bool SIPrepareScratchRegs::runOnMachineFunction(MachineFunction &MF) { .addReg(ScratchRsrcReg, RegState::ImplicitDefine); BuildMI(MBB, I, DL, TII->get(AMDGPU::S_MOV_B32), Rsrc2) - .addImm(Rsrc & 0xffffffff) + .addImm(Rsrc23 & 0xffffffff) .addReg(ScratchRsrcReg, RegState::ImplicitDefine); BuildMI(MBB, I, DL, TII->get(AMDGPU::S_MOV_B32), Rsrc3) - .addImm(Rsrc >> 32) + .addImm(Rsrc23 >> 32) .addReg(ScratchRsrcReg, RegState::ImplicitDefine); // Scratch Offset diff --git a/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp index 54c4d54..e9e8412 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp +++ b/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp @@ -26,23 +26,25 @@ using namespace llvm; SIRegisterInfo::SIRegisterInfo() : AMDGPURegisterInfo() {} -BitVector SIRegisterInfo::getReservedRegs(const MachineFunction &MF) const { - BitVector Reserved(getNumRegs()); - Reserved.set(AMDGPU::EXEC); +void SIRegisterInfo::reserveRegisterTuples(BitVector &Reserved, unsigned Reg) const { + MCRegAliasIterator R(Reg, this, true); - // EXEC_LO and EXEC_HI could be allocated and used as regular register, - // but this seems likely to result in bugs, so I'm marking them as reserved. - Reserved.set(AMDGPU::EXEC_LO); - Reserved.set(AMDGPU::EXEC_HI); + for (; R.isValid(); ++R) + Reserved.set(*R); +} +BitVector SIRegisterInfo::getReservedRegs(const MachineFunction &MF) const { + BitVector Reserved(getNumRegs()); Reserved.set(AMDGPU::INDIRECT_BASE_ADDR); - Reserved.set(AMDGPU::FLAT_SCR); - Reserved.set(AMDGPU::FLAT_SCR_LO); - Reserved.set(AMDGPU::FLAT_SCR_HI); + + // EXEC_LO and EXEC_HI could be allocated and used as regular register, but + // this seems likely to result in bugs, so I'm marking them as reserved. + reserveRegisterTuples(Reserved, AMDGPU::EXEC); + reserveRegisterTuples(Reserved, AMDGPU::FLAT_SCR); // Reserve some VGPRs to use as temp registers in case we have to spill VGPRs - Reserved.set(AMDGPU::VGPR255); - Reserved.set(AMDGPU::VGPR254); + reserveRegisterTuples(Reserved, AMDGPU::VGPR254); + reserveRegisterTuples(Reserved, AMDGPU::VGPR255); // Tonga and Iceland can only allocate a fixed number of SGPRs due // to a hw bug. @@ -54,10 +56,7 @@ BitVector SIRegisterInfo::getReservedRegs(const MachineFunction &MF) const { for (unsigned i = Limit; i < NumSGPRs; ++i) { unsigned Reg = AMDGPU::SGPR_32RegClass.getRegister(i); - MCRegAliasIterator R = MCRegAliasIterator(Reg, this, true); - - for (; R.isValid(); ++R) - Reserved.set(*R); + reserveRegisterTuples(Reserved, Reg); } } diff --git a/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.h b/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.h index bfdb67c..7da6de2 100644 --- a/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.h +++ b/contrib/llvm/lib/Target/AMDGPU/SIRegisterInfo.h @@ -23,7 +23,10 @@ namespace llvm { struct SIRegisterInfo : public AMDGPURegisterInfo { +private: + void reserveRegisterTuples(BitVector &, unsigned Reg) const; +public: SIRegisterInfo(); BitVector getReservedRegs(const MachineFunction &MF) const override; diff --git a/contrib/llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp b/contrib/llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp index f8f0eb2..cf6b892 100644 --- a/contrib/llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp +++ b/contrib/llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp @@ -15,6 +15,7 @@ #include "llvm/ADT/SmallVector.h" #include "llvm/ADT/StringExtras.h" #include "llvm/ADT/StringSwitch.h" +#include "llvm/ADT/Triple.h" #include "llvm/ADT/Twine.h" #include "llvm/MC/MCAsmInfo.h" #include "llvm/MC/MCAssembler.h" @@ -9104,6 +9105,10 @@ bool ARMAsmParser::parseDirectiveArch(SMLoc L) { return false; } + Triple T; + STI.setDefaultFeatures(T.getARMCPUForArch(Arch)); + setAvailableFeatures(ComputeAvailableFeatures(STI.getFeatureBits())); + getTargetStreamer().emitArch(ID); return false; } diff --git a/contrib/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp b/contrib/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp index d9e654c..9d5f1d4 100644 --- a/contrib/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp +++ b/contrib/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp @@ -50,6 +50,7 @@ private: // Complex Pattern for address selection. bool SelectAddr(SDValue Addr, SDValue &Base, SDValue &Offset); + bool SelectFIAddr(SDValue Addr, SDValue &Base, SDValue &Offset); }; } @@ -67,7 +68,7 @@ bool BPFDAGToDAGISel::SelectAddr(SDValue Addr, SDValue &Base, SDValue &Offset) { Addr.getOpcode() == ISD::TargetGlobalAddress) return false; - // Addresses of the form FI+const or FI|const + // Addresses of the form Addr+const or Addr|const if (CurDAG->isBaseWithConstantOffset(Addr)) { ConstantSDNode *CN = dyn_cast<ConstantSDNode>(Addr.getOperand(1)); if (isInt<32>(CN->getSExtValue())) { @@ -89,6 +90,31 @@ bool BPFDAGToDAGISel::SelectAddr(SDValue Addr, SDValue &Base, SDValue &Offset) { return true; } +// ComplexPattern used on BPF FI instruction +bool BPFDAGToDAGISel::SelectFIAddr(SDValue Addr, SDValue &Base, SDValue &Offset) { + SDLoc DL(Addr); + + if (!CurDAG->isBaseWithConstantOffset(Addr)) + return false; + + // Addresses of the form Addr+const or Addr|const + ConstantSDNode *CN = dyn_cast<ConstantSDNode>(Addr.getOperand(1)); + if (isInt<32>(CN->getSExtValue())) { + + // If the first operand is a FI, get the TargetFI Node + if (FrameIndexSDNode *FIN = + dyn_cast<FrameIndexSDNode>(Addr.getOperand(0))) + Base = CurDAG->getTargetFrameIndex(FIN->getIndex(), MVT::i64); + else + return false; + + Offset = CurDAG->getTargetConstant(CN->getSExtValue(), DL, MVT::i64); + return true; + } + + return false; +} + SDNode *BPFDAGToDAGISel::Select(SDNode *Node) { unsigned Opcode = Node->getOpcode(); @@ -104,13 +130,6 @@ SDNode *BPFDAGToDAGISel::Select(SDNode *Node) { // tablegen selection should be handled here. switch (Opcode) { default: break; - - case ISD::UNDEF: { - errs() << "BUG: "; Node->dump(CurDAG); errs() << '\n'; - report_fatal_error("shouldn't see UNDEF during Select"); - break; - } - case ISD::INTRINSIC_W_CHAIN: { unsigned IntNo = cast<ConstantSDNode>(Node->getOperand(1))->getZExtValue(); switch (IntNo) { diff --git a/contrib/llvm/lib/Target/BPF/BPFISelLowering.cpp b/contrib/llvm/lib/Target/BPF/BPFISelLowering.cpp index 58498a1..7341828 100644 --- a/contrib/llvm/lib/Target/BPF/BPFISelLowering.cpp +++ b/contrib/llvm/lib/Target/BPF/BPFISelLowering.cpp @@ -102,6 +102,7 @@ BPFTargetLowering::BPFTargetLowering(const TargetMachine &TM, setOperationAction(ISD::BR_CC, MVT::i64, Custom); setOperationAction(ISD::BR_JT, MVT::Other, Expand); + setOperationAction(ISD::BRIND, MVT::Other, Expand); setOperationAction(ISD::BRCOND, MVT::Other, Expand); setOperationAction(ISD::SETCC, MVT::i64, Expand); setOperationAction(ISD::SELECT, MVT::i64, Expand); @@ -128,9 +129,6 @@ BPFTargetLowering::BPFTargetLowering(const TargetMachine &TM, setOperationAction(ISD::SUBC, MVT::i64, Expand); setOperationAction(ISD::SUBE, MVT::i64, Expand); - // no UNDEF allowed - setOperationAction(ISD::UNDEF, MVT::i64, Expand); - setOperationAction(ISD::ROTR, MVT::i64, Expand); setOperationAction(ISD::ROTL, MVT::i64, Expand); setOperationAction(ISD::SHL_PARTS, MVT::i64, Expand); diff --git a/contrib/llvm/lib/Target/BPF/BPFInstrInfo.td b/contrib/llvm/lib/Target/BPF/BPFInstrInfo.td index 26b2cfe..6b73db8 100644 --- a/contrib/llvm/lib/Target/BPF/BPFInstrInfo.td +++ b/contrib/llvm/lib/Target/BPF/BPFInstrInfo.td @@ -54,7 +54,8 @@ def i64immSExt32 : PatLeaf<(imm), [{return isInt<32>(N->getSExtValue()); }]>; // Addressing modes. -def ADDRri : ComplexPattern<i64, 2, "SelectAddr", [frameindex], []>; +def ADDRri : ComplexPattern<i64, 2, "SelectAddr", [], []>; +def FIri : ComplexPattern<i64, 2, "SelectFIAddr", [add, or], []>; // Address operands def MEMri : Operand<i64> { @@ -260,6 +261,15 @@ def MOV_rr : MOV_RR<"mov">; def MOV_ri : MOV_RI<"mov">; } +def FI_ri + : InstBPF<(outs GPR:$dst), (ins MEMri:$addr), + "lea\t$dst, $addr", + [(set i64:$dst, FIri:$addr)]> { + // This is a tentative instruction, and will be replaced + // with MOV_rr and ADD_ri in PEI phase +} + + def LD_pseudo : InstBPF<(outs GPR:$dst), (ins i64imm:$pseudo, u64imm:$imm), "ld_pseudo\t$dst, $pseudo, $imm", diff --git a/contrib/llvm/lib/Target/BPF/BPFRegisterInfo.cpp b/contrib/llvm/lib/Target/BPF/BPFRegisterInfo.cpp index 8f885c3..952615b 100644 --- a/contrib/llvm/lib/Target/BPF/BPFRegisterInfo.cpp +++ b/contrib/llvm/lib/Target/BPF/BPFRegisterInfo.cpp @@ -58,14 +58,13 @@ void BPFRegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II, unsigned FrameReg = getFrameRegister(MF); int FrameIndex = MI.getOperand(i).getIndex(); + const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo(); + MachineBasicBlock &MBB = *MI.getParent(); if (MI.getOpcode() == BPF::MOV_rr) { - const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo(); int Offset = MF.getFrameInfo()->getObjectOffset(FrameIndex); MI.getOperand(i).ChangeToRegister(FrameReg, false); - - MachineBasicBlock &MBB = *MI.getParent(); unsigned reg = MI.getOperand(i - 1).getReg(); BuildMI(MBB, ++II, DL, TII.get(BPF::ADD_ri), reg) .addReg(reg) @@ -79,8 +78,24 @@ void BPFRegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II, if (!isInt<32>(Offset)) llvm_unreachable("bug in frame offset"); - MI.getOperand(i).ChangeToRegister(FrameReg, false); - MI.getOperand(i + 1).ChangeToImmediate(Offset); + if (MI.getOpcode() == BPF::FI_ri) { + // architecture does not really support FI_ri, replace it with + // MOV_rr <target_reg>, frame_reg + // ADD_ri <target_reg>, imm + unsigned reg = MI.getOperand(i - 1).getReg(); + + BuildMI(MBB, ++II, DL, TII.get(BPF::MOV_rr), reg) + .addReg(FrameReg); + BuildMI(MBB, II, DL, TII.get(BPF::ADD_ri), reg) + .addReg(reg) + .addImm(Offset); + + // Remove FI_ri instruction + MI.eraseFromParent(); + } else { + MI.getOperand(i).ChangeToRegister(FrameReg, false); + MI.getOperand(i + 1).ChangeToImmediate(Offset); + } } unsigned BPFRegisterInfo::getFrameRegister(const MachineFunction &MF) const { diff --git a/contrib/llvm/lib/Target/Mips/MipsISelLowering.h b/contrib/llvm/lib/Target/Mips/MipsISelLowering.h index 6fe8f83..b3d861d 100644 --- a/contrib/llvm/lib/Target/Mips/MipsISelLowering.h +++ b/contrib/llvm/lib/Target/Mips/MipsISelLowering.h @@ -269,6 +269,14 @@ namespace llvm { unsigned getRegisterByName(const char* RegName, EVT VT, SelectionDAG &DAG) const override; + /// Returns true if a cast between SrcAS and DestAS is a noop. + bool isNoopAddrSpaceCast(unsigned SrcAS, unsigned DestAS) const override { + // Mips doesn't have any special address spaces so we just reserve + // the first 256 for software use (e.g. OpenCL) and treat casts + // between them as noops. + return SrcAS < 256 && DestAS < 256; + } + protected: SDValue getGlobalReg(SelectionDAG &DAG, EVT Ty) const; diff --git a/contrib/llvm/lib/Target/Mips/MipsSEISelDAGToDAG.cpp b/contrib/llvm/lib/Target/Mips/MipsSEISelDAGToDAG.cpp index cb46d73..2ebfbd1 100644 --- a/contrib/llvm/lib/Target/Mips/MipsSEISelDAGToDAG.cpp +++ b/contrib/llvm/lib/Target/Mips/MipsSEISelDAGToDAG.cpp @@ -115,6 +115,11 @@ bool MipsSEDAGToDAGISel::replaceUsesWithZeroReg(MachineRegisterInfo *MRI, if (MI->isPHI() || MI->isRegTiedToDefOperand(OpNo) || MI->isPseudo()) continue; + // Also, we have to check that the register class of the operand + // contains the zero register. + if (!MRI->getRegClass(MO.getReg())->contains(ZeroReg)) + continue; + MO.setReg(ZeroReg); } diff --git a/contrib/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp b/contrib/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp index 4444466..8e118ec 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp @@ -947,11 +947,11 @@ void PPCAsmPrinter::EmitInstruction(const MachineInstr *MI) { return; } case PPC::ADDISdtprelHA: - // Transform: %Xd = ADDISdtprelHA %X3, <ga:@sym> - // Into: %Xd = ADDIS8 %X3, sym@dtprel@ha + // Transform: %Xd = ADDISdtprelHA %Xs, <ga:@sym> + // Into: %Xd = ADDIS8 %Xs, sym@dtprel@ha case PPC::ADDISdtprelHA32: { - // Transform: %Rd = ADDISdtprelHA32 %R3, <ga:@sym> - // Into: %Rd = ADDIS %R3, sym@dtprel@ha + // Transform: %Rd = ADDISdtprelHA32 %Rs, <ga:@sym> + // Into: %Rd = ADDIS %Rs, sym@dtprel@ha const MachineOperand &MO = MI->getOperand(2); const GlobalValue *GValue = MO.getGlobal(); MCSymbol *MOSymbol = getSymbol(GValue); @@ -962,7 +962,7 @@ void PPCAsmPrinter::EmitInstruction(const MachineInstr *MI) { *OutStreamer, MCInstBuilder(Subtarget->isPPC64() ? PPC::ADDIS8 : PPC::ADDIS) .addReg(MI->getOperand(0).getReg()) - .addReg(Subtarget->isPPC64() ? PPC::X3 : PPC::R3) + .addReg(MI->getOperand(1).getReg()) .addExpr(SymDtprel)); return; } diff --git a/contrib/llvm/lib/Target/PowerPC/PPCCTRLoops.cpp b/contrib/llvm/lib/Target/PowerPC/PPCCTRLoops.cpp index baadf08..fd150be 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCCTRLoops.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCCTRLoops.cpp @@ -197,10 +197,18 @@ static bool isLargeIntegerTy(bool Is32Bit, Type *Ty) { // Determining the address of a TLS variable results in a function call in // certain TLS models. static bool memAddrUsesCTR(const PPCTargetMachine *TM, - const llvm::Value *MemAddr) { + const Value *MemAddr) { const auto *GV = dyn_cast<GlobalValue>(MemAddr); - if (!GV) + if (!GV) { + // Recurse to check for constants that refer to TLS global variables. + if (const auto *CV = dyn_cast<Constant>(MemAddr)) + for (const auto &CO : CV->operands()) + if (memAddrUsesCTR(TM, CO)) + return true; + return false; + } + if (!GV->isThreadLocal()) return false; if (!TM) @@ -239,6 +247,11 @@ bool PPCCTRLoops::mightUseCTR(const Triple &TT, BasicBlock *BB) { if (F->getIntrinsicID() != Intrinsic::not_intrinsic) { switch (F->getIntrinsicID()) { default: continue; + // If we have a call to ppc_is_decremented_ctr_nonzero, or ppc_mtctr + // we're definitely using CTR. + case Intrinsic::ppc_is_decremented_ctr_nonzero: + case Intrinsic::ppc_mtctr: + return true; // VisualStudio defines setjmp as _setjmp #if defined(_MSC_VER) && defined(setjmp) && \ @@ -426,6 +439,7 @@ bool PPCCTRLoops::convertToCTRLoop(Loop *L) { // Process nested loops first. for (Loop::iterator I = L->begin(), E = L->end(); I != E; ++I) { MadeChange |= convertToCTRLoop(*I); + DEBUG(dbgs() << "Nested loop converted\n"); } // If a nested loop has been converted, then we can't convert this loop. diff --git a/contrib/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp b/contrib/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp index b6025bf..9322268 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp @@ -2570,13 +2570,25 @@ SDNode *PPCDAGToDAGISel::Select(SDNode *N) { return nullptr; } // ISD::OR doesn't get all the bitfield insertion fun. - // (and (or x, c1), c2) where isRunOfOnes(~(c1^c2)) is a bitfield insert + // (and (or x, c1), c2) where isRunOfOnes(~(c1^c2)) might be a + // bitfield insert. if (isInt32Immediate(N->getOperand(1), Imm) && N->getOperand(0).getOpcode() == ISD::OR && isInt32Immediate(N->getOperand(0).getOperand(1), Imm2)) { + // The idea here is to check whether this is equivalent to: + // (c1 & m) | (x & ~m) + // where m is a run-of-ones mask. The logic here is that, for each bit in + // c1 and c2: + // - if both are 1, then the output will be 1. + // - if both are 0, then the output will be 0. + // - if the bit in c1 is 0, and the bit in c2 is 1, then the output will + // come from x. + // - if the bit in c1 is 1, and the bit in c2 is 0, then the output will + // be 0. + // If that last condition is never the case, then we can form m from the + // bits that are the same between c1 and c2. unsigned MB, ME; - Imm = ~(Imm^Imm2); - if (isRunOfOnes(Imm, MB, ME)) { + if (isRunOfOnes(~(Imm^Imm2), MB, ME) && !(~Imm & Imm2)) { SDValue Ops[] = { N->getOperand(0).getOperand(0), N->getOperand(0).getOperand(1), getI32Imm(0, dl), getI32Imm(MB, dl), @@ -2787,6 +2799,8 @@ SDNode *PPCDAGToDAGISel::Select(SDNode *N) { SDValue Base, Offset; if (LD->isUnindexed() && + (LD->getMemoryVT() == MVT::f64 || + LD->getMemoryVT() == MVT::i64) && SelectAddrIdxOnly(LD->getBasePtr(), Base, Offset)) { SDValue Chain = LD->getChain(); SDValue Ops[] = { Base, Offset, Chain }; diff --git a/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp b/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp index 1e28913..1b8f8fb 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp @@ -431,6 +431,8 @@ PPCTargetLowering::PPCTargetLowering(const PPCTargetMachine &TM, AddPromotedToType (ISD::LOAD , VT, MVT::v4i32); setOperationAction(ISD::SELECT, VT, Promote); AddPromotedToType (ISD::SELECT, VT, MVT::v4i32); + setOperationAction(ISD::SELECT_CC, VT, Promote); + AddPromotedToType (ISD::SELECT_CC, VT, MVT::v4i32); setOperationAction(ISD::STORE, VT, Promote); AddPromotedToType (ISD::STORE, VT, MVT::v4i32); @@ -7175,7 +7177,6 @@ SDValue PPCTargetLowering::LowerVECTOR_SHUFFLE(SDValue Op, PPC::isSplatShuffleMask(SVOp, 4) || PPC::isVPKUWUMShuffleMask(SVOp, 1, DAG) || PPC::isVPKUHUMShuffleMask(SVOp, 1, DAG) || - PPC::isVPKUDUMShuffleMask(SVOp, 1, DAG) || PPC::isVSLDOIShuffleMask(SVOp, 1, DAG) != -1 || PPC::isVMRGLShuffleMask(SVOp, 1, 1, DAG) || PPC::isVMRGLShuffleMask(SVOp, 2, 1, DAG) || @@ -7183,8 +7184,10 @@ SDValue PPCTargetLowering::LowerVECTOR_SHUFFLE(SDValue Op, PPC::isVMRGHShuffleMask(SVOp, 1, 1, DAG) || PPC::isVMRGHShuffleMask(SVOp, 2, 1, DAG) || PPC::isVMRGHShuffleMask(SVOp, 4, 1, DAG) || - PPC::isVMRGEOShuffleMask(SVOp, true, 1, DAG) || - PPC::isVMRGEOShuffleMask(SVOp, false, 1, DAG)) { + (Subtarget.hasP8Altivec() && ( + PPC::isVPKUDUMShuffleMask(SVOp, 1, DAG) || + PPC::isVMRGEOShuffleMask(SVOp, true, 1, DAG) || + PPC::isVMRGEOShuffleMask(SVOp, false, 1, DAG)))) { return Op; } } @@ -7195,7 +7198,6 @@ SDValue PPCTargetLowering::LowerVECTOR_SHUFFLE(SDValue Op, unsigned int ShuffleKind = isLittleEndian ? 2 : 0; if (PPC::isVPKUWUMShuffleMask(SVOp, ShuffleKind, DAG) || PPC::isVPKUHUMShuffleMask(SVOp, ShuffleKind, DAG) || - PPC::isVPKUDUMShuffleMask(SVOp, ShuffleKind, DAG) || PPC::isVSLDOIShuffleMask(SVOp, ShuffleKind, DAG) != -1 || PPC::isVMRGLShuffleMask(SVOp, 1, ShuffleKind, DAG) || PPC::isVMRGLShuffleMask(SVOp, 2, ShuffleKind, DAG) || @@ -7203,8 +7205,10 @@ SDValue PPCTargetLowering::LowerVECTOR_SHUFFLE(SDValue Op, PPC::isVMRGHShuffleMask(SVOp, 1, ShuffleKind, DAG) || PPC::isVMRGHShuffleMask(SVOp, 2, ShuffleKind, DAG) || PPC::isVMRGHShuffleMask(SVOp, 4, ShuffleKind, DAG) || - PPC::isVMRGEOShuffleMask(SVOp, true, ShuffleKind, DAG) || - PPC::isVMRGEOShuffleMask(SVOp, false, ShuffleKind, DAG)) + (Subtarget.hasP8Altivec() && ( + PPC::isVPKUDUMShuffleMask(SVOp, ShuffleKind, DAG) || + PPC::isVMRGEOShuffleMask(SVOp, true, ShuffleKind, DAG) || + PPC::isVMRGEOShuffleMask(SVOp, false, ShuffleKind, DAG)))) return Op; // Check to see if this is a shuffle of 4-byte values. If so, we can use our diff --git a/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.cpp b/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.cpp index bf6e402..d4e666c 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.cpp @@ -309,6 +309,11 @@ PPCInstrInfo::commuteInstruction(MachineInstr *MI, bool NewMI) const { unsigned MB = MI->getOperand(4).getImm(); unsigned ME = MI->getOperand(5).getImm(); + // We can't commute a trivial mask (there is no way to represent an all-zero + // mask). + if (MB == 0 && ME == 31) + return nullptr; + if (NewMI) { // Create a new instruction. unsigned Reg0 = ChangeReg0 ? Reg2 : MI->getOperand(0).getReg(); diff --git a/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td b/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td index b50124d..24fd9bd 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td +++ b/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td @@ -2835,24 +2835,84 @@ def : Pat<(i64 (anyext i1:$in)), (SELECT_I8 $in, (LI8 1), (LI8 0))>; // match setcc on i1 variables. +// CRANDC is: +// 1 1 : F +// 1 0 : T +// 0 1 : F +// 0 0 : F +// +// LT is: +// -1 -1 : F +// -1 0 : T +// 0 -1 : F +// 0 0 : F +// +// ULT is: +// 1 1 : F +// 1 0 : F +// 0 1 : T +// 0 0 : F def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETLT)), - (CRANDC $s2, $s1)>; + (CRANDC $s1, $s2)>; def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETULT)), (CRANDC $s2, $s1)>; +// CRORC is: +// 1 1 : T +// 1 0 : T +// 0 1 : F +// 0 0 : T +// +// LE is: +// -1 -1 : T +// -1 0 : T +// 0 -1 : F +// 0 0 : T +// +// ULE is: +// 1 1 : T +// 1 0 : F +// 0 1 : T +// 0 0 : T def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETLE)), - (CRORC $s2, $s1)>; + (CRORC $s1, $s2)>; def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETULE)), (CRORC $s2, $s1)>; + def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETEQ)), (CREQV $s1, $s2)>; + +// GE is: +// -1 -1 : T +// -1 0 : F +// 0 -1 : T +// 0 0 : T +// +// UGE is: +// 1 1 : T +// 1 0 : T +// 0 1 : F +// 0 0 : T def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETGE)), - (CRORC $s1, $s2)>; + (CRORC $s2, $s1)>; def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETUGE)), (CRORC $s1, $s2)>; + +// GT is: +// -1 -1 : F +// -1 0 : F +// 0 -1 : T +// 0 0 : F +// +// UGT is: +// 1 1 : F +// 1 0 : T +// 0 1 : F +// 0 0 : F def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETGT)), - (CRANDC $s1, $s2)>; + (CRANDC $s2, $s1)>; def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETUGT)), (CRANDC $s1, $s2)>; + def : Pat<(i1 (setcc i1:$s1, i1:$s2, SETNE)), (CRXOR $s1, $s2)>; @@ -3203,18 +3263,30 @@ def : Pat<(i1 (select i1:$cond, i1:$tval, i1:$fval)), // select (lhs == rhs), tval, fval is: // ((lhs == rhs) & tval) | (!(lhs == rhs) & fval) def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETLT)), + (CROR (CRAND (CRANDC $lhs, $rhs), $tval), + (CRAND (CRORC $rhs, $lhs), $fval))>; +def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETULT)), (CROR (CRAND (CRANDC $rhs, $lhs), $tval), (CRAND (CRORC $lhs, $rhs), $fval))>; def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETLE)), + (CROR (CRAND (CRORC $lhs, $rhs), $tval), + (CRAND (CRANDC $rhs, $lhs), $fval))>; +def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETULE)), (CROR (CRAND (CRORC $rhs, $lhs), $tval), (CRAND (CRANDC $lhs, $rhs), $fval))>; def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETEQ)), (CROR (CRAND (CREQV $lhs, $rhs), $tval), (CRAND (CRXOR $lhs, $rhs), $fval))>; def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETGE)), + (CROR (CRAND (CRORC $rhs, $lhs), $tval), + (CRAND (CRANDC $lhs, $rhs), $fval))>; +def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETUGE)), (CROR (CRAND (CRORC $lhs, $rhs), $tval), (CRAND (CRANDC $rhs, $lhs), $fval))>; def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETGT)), + (CROR (CRAND (CRANDC $rhs, $lhs), $tval), + (CRAND (CRORC $lhs, $rhs), $fval))>; +def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETUGT)), (CROR (CRAND (CRANDC $lhs, $rhs), $tval), (CRAND (CRORC $rhs, $lhs), $fval))>; def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETNE)), @@ -3223,66 +3295,106 @@ def : Pat <(i1 (selectcc i1:$lhs, i1:$rhs, i1:$tval, i1:$fval, SETNE)), // match selectcc on i1 variables with non-i1 output. def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETLT)), + (SELECT_I4 (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETULT)), (SELECT_I4 (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETLE)), + (SELECT_I4 (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETULE)), (SELECT_I4 (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETEQ)), (SELECT_I4 (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETGE)), + (SELECT_I4 (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETUGE)), (SELECT_I4 (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETGT)), + (SELECT_I4 (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETUGT)), (SELECT_I4 (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(i32 (selectcc i1:$lhs, i1:$rhs, i32:$tval, i32:$fval, SETNE)), (SELECT_I4 (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETLT)), + (SELECT_I8 (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETULT)), (SELECT_I8 (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETLE)), + (SELECT_I8 (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETULE)), (SELECT_I8 (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETEQ)), (SELECT_I8 (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETGE)), + (SELECT_I8 (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETUGE)), (SELECT_I8 (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETGT)), + (SELECT_I8 (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETUGT)), (SELECT_I8 (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(i64 (selectcc i1:$lhs, i1:$rhs, i64:$tval, i64:$fval, SETNE)), (SELECT_I8 (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLT)), + (SELECT_F4 (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETULT)), (SELECT_F4 (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLE)), + (SELECT_F4 (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETULE)), (SELECT_F4 (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETEQ)), (SELECT_F4 (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGE)), + (SELECT_F4 (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGE)), (SELECT_F4 (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGT)), + (SELECT_F4 (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGT)), (SELECT_F4 (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETNE)), (SELECT_F4 (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETLT)), + (SELECT_F8 (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETULT)), (SELECT_F8 (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETLE)), + (SELECT_F8 (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETULE)), (SELECT_F8 (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETEQ)), (SELECT_F8 (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETGE)), + (SELECT_F8 (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETUGE)), (SELECT_F8 (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETGT)), + (SELECT_F8 (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETUGT)), (SELECT_F8 (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETNE)), (SELECT_F8 (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETLT)), + (SELECT_VRRC (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETULT)), (SELECT_VRRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETLE)), + (SELECT_VRRC (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETULE)), (SELECT_VRRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETEQ)), (SELECT_VRRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETGE)), + (SELECT_VRRC (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETUGE)), (SELECT_VRRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETGT)), + (SELECT_VRRC (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETUGT)), (SELECT_VRRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i32 (selectcc i1:$lhs, i1:$rhs, v4i32:$tval, v4i32:$fval, SETNE)), (SELECT_VRRC (CRXOR $lhs, $rhs), $tval, $fval)>; diff --git a/contrib/llvm/lib/Target/PowerPC/PPCInstrQPX.td b/contrib/llvm/lib/Target/PowerPC/PPCInstrQPX.td index 5c66b42..0a044c5 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCInstrQPX.td +++ b/contrib/llvm/lib/Target/PowerPC/PPCInstrQPX.td @@ -1115,40 +1115,64 @@ def : Pat<(v4f64 (PPCqbflt v4i1:$src)), (COPY_TO_REGCLASS $src, QFRC)>; def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETLT)), + (SELECT_QFRC (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETULT)), (SELECT_QFRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETLE)), + (SELECT_QFRC (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETULE)), (SELECT_QFRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETEQ)), (SELECT_QFRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETGE)), + (SELECT_QFRC (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETUGE)), (SELECT_QFRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETGT)), + (SELECT_QFRC (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETUGT)), (SELECT_QFRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f64 (selectcc i1:$lhs, i1:$rhs, v4f64:$tval, v4f64:$fval, SETNE)), (SELECT_QFRC (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETLT)), + (SELECT_QSRC (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETULT)), (SELECT_QSRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETLE)), + (SELECT_QSRC (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETULE)), (SELECT_QSRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETEQ)), (SELECT_QSRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETGE)), + (SELECT_QSRC (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETUGE)), (SELECT_QSRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETGT)), + (SELECT_QSRC (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETUGT)), (SELECT_QSRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4f32 (selectcc i1:$lhs, i1:$rhs, v4f32:$tval, v4f32:$fval, SETNE)), (SELECT_QSRC (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETLT)), + (SELECT_QBRC (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETULT)), (SELECT_QBRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETLE)), + (SELECT_QBRC (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETULE)), (SELECT_QBRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETEQ)), (SELECT_QBRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETGE)), + (SELECT_QBRC (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETUGE)), (SELECT_QBRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETGT)), + (SELECT_QBRC (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETUGT)), (SELECT_QBRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(v4i1 (selectcc i1:$lhs, i1:$rhs, v4i1:$tval, v4i1:$fval, SETNE)), (SELECT_QBRC (CRXOR $lhs, $rhs), $tval, $fval)>; diff --git a/contrib/llvm/lib/Target/PowerPC/PPCInstrVSX.td b/contrib/llvm/lib/Target/PowerPC/PPCInstrVSX.td index 20c95fe..ce63c22 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCInstrVSX.td +++ b/contrib/llvm/lib/Target/PowerPC/PPCInstrVSX.td @@ -958,27 +958,43 @@ def : Pat<(v4i32 (PPCxxswapd v4i32:$src)), (XXPERMDI $src, $src, 2)>; // Selects. def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETLT)), + (SELECT_VSRC (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETULT)), (SELECT_VSRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETLE)), + (SELECT_VSRC (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETULE)), (SELECT_VSRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETEQ)), (SELECT_VSRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETGE)), + (SELECT_VSRC (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETUGE)), (SELECT_VSRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETGT)), + (SELECT_VSRC (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETUGT)), (SELECT_VSRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(v2f64 (selectcc i1:$lhs, i1:$rhs, v2f64:$tval, v2f64:$fval, SETNE)), (SELECT_VSRC (CRXOR $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETLT)), + (SELECT_VSFRC (CRANDC $lhs, $rhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETULT)), (SELECT_VSFRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETLE)), + (SELECT_VSFRC (CRORC $lhs, $rhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETULE)), (SELECT_VSFRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETEQ)), (SELECT_VSFRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETGE)), + (SELECT_VSFRC (CRORC $rhs, $lhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETUGE)), (SELECT_VSFRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETGT)), + (SELECT_VSFRC (CRANDC $rhs, $lhs), $tval, $fval)>; +def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETUGT)), (SELECT_VSFRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(f64 (selectcc i1:$lhs, i1:$rhs, f64:$tval, f64:$fval, SETNE)), (SELECT_VSFRC (CRXOR $lhs, $rhs), $tval, $fval)>; @@ -1060,18 +1076,27 @@ let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns. (COPY_TO_REGCLASS (LXSSPX xoaddr:$src), VSFRC)>; def : Pat<(f64 (fextend f32:$src)), (COPY_TO_REGCLASS $src, VSFRC)>; + def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLT)), + (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>; + def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETULT)), (SELECT_VSSRC (CRANDC $rhs, $lhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLE)), + (SELECT_VSSRC (CRORC $lhs, $rhs), $tval, $fval)>; + def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETULE)), (SELECT_VSSRC (CRORC $rhs, $lhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETEQ)), (SELECT_VSSRC (CREQV $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGE)), + (SELECT_VSSRC (CRORC $rhs, $lhs), $tval, $fval)>; + def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGE)), (SELECT_VSSRC (CRORC $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGT)), + (SELECT_VSSRC (CRANDC $rhs, $lhs), $tval, $fval)>; + def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGT)), (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>; def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETNE)), - (SELECT_VSSRC (CRXOR $lhs, $rhs), $tval, $fval)>; + (SELECT_VSSRC (CRXOR $lhs, $rhs), $tval, $fval)>; // VSX Elementary Scalar FP arithmetic (SP) let isCommutable = 1 in { diff --git a/contrib/llvm/lib/Target/PowerPC/PPCVSXFMAMutate.cpp b/contrib/llvm/lib/Target/PowerPC/PPCVSXFMAMutate.cpp index 58d3c3d..46b8d13 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCVSXFMAMutate.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCVSXFMAMutate.cpp @@ -103,6 +103,11 @@ protected: VNInfo *AddendValNo = LIS->getInterval(MI->getOperand(1).getReg()).Query(FMAIdx).valueIn(); + if (!AddendValNo) { + // This can be null if the register is undef. + continue; + } + MachineInstr *AddendMI = LIS->getInstructionFromIndex(AddendValNo->def); // The addend and this instruction must be in the same block. @@ -181,11 +186,14 @@ protected: if (!KilledProdOp) continue; - // For virtual registers, verify that the addend source register - // is live here (as should have been assured above). - assert((!TargetRegisterInfo::isVirtualRegister(AddendSrcReg) || - LIS->getInterval(AddendSrcReg).liveAt(FMAIdx)) && - "Addend source register is not live!"); + // If the addend copy is used only by this MI, then the addend source + // register is likely not live here. This could be fixed (based on the + // legality checks above, the live range for the addend source register + // could be extended), but it seems likely that such a trivial copy can + // be coalesced away later, and thus is not worth the effort. + if (TargetRegisterInfo::isVirtualRegister(AddendSrcReg) && + !LIS->getInterval(AddendSrcReg).liveAt(FMAIdx)) + continue; // Transform: (O2 * O3) + O1 -> (O2 * O1) + O3. diff --git a/contrib/llvm/lib/Target/PowerPC/PPCVSXSwapRemoval.cpp b/contrib/llvm/lib/Target/PowerPC/PPCVSXSwapRemoval.cpp index 3fb1dcc..d7132d5 100644 --- a/contrib/llvm/lib/Target/PowerPC/PPCVSXSwapRemoval.cpp +++ b/contrib/llvm/lib/Target/PowerPC/PPCVSXSwapRemoval.cpp @@ -240,6 +240,9 @@ bool PPCVSXSwapRemoval::gatherVectorInstructions() { for (MachineBasicBlock &MBB : *MF) { for (MachineInstr &MI : MBB) { + if (MI.isDebugValue()) + continue; + bool RelevantInstr = false; bool Partial = false; diff --git a/contrib/llvm/lib/Target/Sparc/AsmParser/SparcAsmParser.cpp b/contrib/llvm/lib/Target/Sparc/AsmParser/SparcAsmParser.cpp index 4a33f7f..1c4e486 100644 --- a/contrib/llvm/lib/Target/Sparc/AsmParser/SparcAsmParser.cpp +++ b/contrib/llvm/lib/Target/Sparc/AsmParser/SparcAsmParser.cpp @@ -77,7 +77,7 @@ class SparcAsmParser : public MCTargetAsmParser { bool parseDirectiveWord(unsigned Size, SMLoc L); bool is64Bit() const { - return STI.getTargetTriple().getArchName().startswith("sparcv9"); + return STI.getTargetTriple().getArch() == Triple::sparcv9; } void expandSET(MCInst &Inst, SMLoc IDLoc, diff --git a/contrib/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp b/contrib/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp index 88e5e47..909baae 100644 --- a/contrib/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp +++ b/contrib/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp @@ -228,7 +228,7 @@ void PassManagerBuilder::populateModulePassManager( // Start of function pass. // Break up aggregate allocas, using SSAUpdater. if (UseNewSROA) - MPM.add(createSROAPass(/*RequiresDomTree*/ false)); + MPM.add(createSROAPass()); else MPM.add(createScalarReplAggregatesPass(-1, false)); MPM.add(createEarlyCSEPass()); // Catch trivial redundancies diff --git a/contrib/llvm/lib/Transforms/Scalar/GVN.cpp b/contrib/llvm/lib/Transforms/Scalar/GVN.cpp index d1eba6e..89a0d0a 100644 --- a/contrib/llvm/lib/Transforms/Scalar/GVN.cpp +++ b/contrib/llvm/lib/Transforms/Scalar/GVN.cpp @@ -1761,7 +1761,8 @@ bool GVN::processNonLocalLoad(LoadInst *LI) { if (isa<PHINode>(V)) V->takeName(LI); if (Instruction *I = dyn_cast<Instruction>(V)) - I->setDebugLoc(LI->getDebugLoc()); + if (LI->getDebugLoc()) + I->setDebugLoc(LI->getDebugLoc()); if (V->getType()->getScalarType()->isPointerTy()) MD->invalidateCachedPointerInfo(V); markInstructionForDeletion(LI); diff --git a/contrib/llvm/lib/Transforms/Utils/Local.cpp b/contrib/llvm/lib/Transforms/Utils/Local.cpp index 50ca623..ba8af47 100644 --- a/contrib/llvm/lib/Transforms/Utils/Local.cpp +++ b/contrib/llvm/lib/Transforms/Utils/Local.cpp @@ -869,6 +869,11 @@ bool llvm::EliminateDuplicatePHINodes(BasicBlock *BB) { PN->replaceAllUsesWith(*Inserted.first); PN->eraseFromParent(); Changed = true; + + // The RAUW can change PHIs that we already visited. Start over from the + // beginning. + PHISet.clear(); + I = BB->begin(); } } diff --git a/contrib/llvm/patches/README.TXT b/contrib/llvm/patches/README.TXT index 220baf0..96112b6 100644 --- a/contrib/llvm/patches/README.TXT +++ b/contrib/llvm/patches/README.TXT @@ -1,11 +1,11 @@ This is a set of individual patches, which contain all the customizations to llvm/clang currently in the FreeBSD base system. These can be applied in -alphabetical order to a pristine llvm/clang 3.7.0 source tree, for example by +alphabetical order to a pristine llvm/clang 3.7.1 source tree, for example by doing: -svn co https://llvm.org/svn/llvm-project/llvm/trunk llvm-3.7.0 -svn co https://llvm.org/svn/llvm-project/cfe/trunk llvm-3.7.0/tools/clang -cd llvm-3.7.0 +svn co https://llvm.org/svn/llvm-project/llvm/trunk llvm-3.7.1 +svn co https://llvm.org/svn/llvm-project/cfe/trunk llvm-3.7.1/tools/clang +cd llvm-3.7.1 for p in /usr/src/contrib/llvm/patches/patch-*.diff; do patch -p0 -f -F0 -E -i $p -s || break done diff --git a/contrib/llvm/patches/patch-10-clang-cc1as-dwarf2.diff b/contrib/llvm/patches/patch-08-clang-cc1as-dwarf2.diff index eb00168..eb00168 100644 --- a/contrib/llvm/patches/patch-10-clang-cc1as-dwarf2.diff +++ b/contrib/llvm/patches/patch-08-clang-cc1as-dwarf2.diff diff --git a/contrib/llvm/patches/patch-08-llvm-r250085-fix-avx-crash.diff b/contrib/llvm/patches/patch-08-llvm-r250085-fix-avx-crash.diff deleted file mode 100644 index a83c68e..0000000 --- a/contrib/llvm/patches/patch-08-llvm-r250085-fix-avx-crash.diff +++ /dev/null @@ -1,142 +0,0 @@ -Pull in r250085 from upstream llvm trunk (by Andrea Di Biagio): - - [x86] Fix wrong lowering of vsetcc nodes (PR25080). - - Function LowerVSETCC (in X86ISelLowering.cpp) worked under the wrong - assumption that for non-AVX512 targets, the source type and destination type - of a type-legalized setcc node were always the same type. - - This assumption was unfortunately incorrect; the type legalizer is not always - able to promote the return type of a setcc to the same type as the first - operand of a setcc. - - In the case of a vsetcc node, the legalizer firstly checks if the first input - operand has a legal type. If so, then it promotes the return type of the vsetcc - to that same type. Otherwise, the return type is promoted to the 'next legal - type', which, for vectors of MVT::i1 is always a 128-bit integer vector type. - - Example (-mattr=+avx): - - %0 = trunc <8 x i32> %a to <8 x i23> - %1 = icmp eq <8 x i23> %0, zeroinitializer - - The initial selection dag for the code above is: - - v8i1 = setcc t5, t7, seteq:ch - t5: v8i23 = truncate t2 - t2: v8i32,ch = CopyFromReg t0, Register:v8i32 %vreg1 - t7: v8i32 = build_vector of all zeroes. - - The type legalizer would firstly check if 't5' has a legal type. If so, then it - would reuse that same type to promote the return type of the setcc node. - Unfortunately 't5' is of illegal type v8i23, and therefore it cannot be used to - promote the return type of the setcc node. Consequently, the setcc return type - is promoted to v8i16. Later on, 't5' is promoted to v8i32 thus leading to the - following dag node: - v8i16 = setcc t32, t25, seteq:ch - - where t32 and t25 are now values of type v8i32. - - Before this patch, function LowerVSETCC would have wrongly expanded the setcc - to a single X86ISD::PCMPEQ. Surprisingly, ISel was still able to match an - instruction. In our case, ISel would have matched a VPCMPEQWrr: - t37: v8i16 = X86ISD::VPCMPEQWrr t36, t25 - - However, t36 and t25 are both VR256, while the result type is instead of class - VR128. This inconsistency ended up causing the insertion of COPY instructions - like this: - %vreg7<def> = COPY %vreg3; VR128:%vreg7 VR256:%vreg3 - - Which is an invalid full copy (not a sub register copy). - Eventually, the backend would have hit an UNREACHABLE "Cannot emit physreg copy - instruction" in the attempt to expand the malformed pseudo COPY instructions. - - This patch fixes the problem adding the missing logic in LowerVSETCC to handle - the corner case of a setcc with 128-bit return type and 256-bit operand type. - - This problem was originally reported by Dimitry as PR25080. It has been latent - for a very long time. I have added the minimal reproducible from that bugzilla - as test setcc-lowering.ll. - - Differential Revision: http://reviews.llvm.org/D13660 - -This should fix the "Cannot emit physreg copy instruction" errors when -compiling contrib/wpa/src/common/ieee802_11_common.c, and CPUTYPE is set -to a CPU supporting AVX (e.g. sandybridge, ivybridge). - -Introduced here: http://svnweb.freebsd.org/changeset/base/289221 - -Index: lib/Target/X86/X86ISelLowering.cpp -=================================================================== ---- lib/Target/X86/X86ISelLowering.cpp -+++ lib/Target/X86/X86ISelLowering.cpp -@@ -13573,6 +13573,35 @@ static SDValue LowerVSETCC(SDValue Op, const X86Su - DAG.getConstant(SSECC, dl, MVT::i8)); - } - -+ MVT VTOp0 = Op0.getSimpleValueType(); -+ assert(VTOp0 == Op1.getSimpleValueType() && -+ "Expected operands with same type!"); -+ assert(VT.getVectorNumElements() == VTOp0.getVectorNumElements() && -+ "Invalid number of packed elements for source and destination!"); -+ -+ if (VT.is128BitVector() && VTOp0.is256BitVector()) { -+ // On non-AVX512 targets, a vector of MVT::i1 is promoted by the type -+ // legalizer to a wider vector type. In the case of 'vsetcc' nodes, the -+ // legalizer firstly checks if the first operand in input to the setcc has -+ // a legal type. If so, then it promotes the return type to that same type. -+ // Otherwise, the return type is promoted to the 'next legal type' which, -+ // for a vector of MVT::i1 is always a 128-bit integer vector type. -+ // -+ // We reach this code only if the following two conditions are met: -+ // 1. Both return type and operand type have been promoted to wider types -+ // by the type legalizer. -+ // 2. The original operand type has been promoted to a 256-bit vector. -+ // -+ // Note that condition 2. only applies for AVX targets. -+ SDValue NewOp = DAG.getSetCC(dl, VTOp0, Op0, Op1, SetCCOpcode); -+ return DAG.getZExtOrTrunc(NewOp, dl, VT); -+ } -+ -+ // The non-AVX512 code below works under the assumption that source and -+ // destination types are the same. -+ assert((Subtarget->hasAVX512() || (VT == VTOp0)) && -+ "Value types for source and destination must be the same!"); -+ - // Break 256-bit integer vector compare into smaller ones. - if (VT.is256BitVector() && !Subtarget->hasInt256()) - return Lower256IntVSETCC(Op, DAG); -Index: test/CodeGen/X86/setcc-lowering.ll -=================================================================== ---- test/CodeGen/X86/setcc-lowering.ll -+++ test/CodeGen/X86/setcc-lowering.ll -@@ -0,0 +1,29 @@ -+; RUN: llc -mtriple=x86_64-unknown-unknown -mattr=+avx < %s | FileCheck %s -+ -+; Verify that we don't crash during codegen due to a wrong lowering -+; of a setcc node with illegal operand types and return type. -+ -+define <8 x i16> @pr25080(<8 x i32> %a) { -+; CHECK-LABEL: pr25080: -+; CHECK: # BB#0: # %entry -+; CHECK-NEXT: vandps {{.*}}(%rip), %ymm0, %ymm0 -+; CHECK-NEXT: vextractf128 $1, %ymm0, %xmm1 -+; CHECK-NEXT: vpxor %xmm2, %xmm2, %xmm2 -+; CHECK-NEXT: vpcmpeqd %xmm2, %xmm1, %xmm1 -+; CHECK-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,4,5,8,9,12,13,8,9,12,13,12,13,14,15] -+; CHECK-NEXT: vpshufb %xmm3, %xmm1, %xmm1 -+; CHECK-NEXT: vpcmpeqd %xmm2, %xmm0, %xmm0 -+; CHECK-NEXT: vpshufb %xmm3, %xmm0, %xmm0 -+; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0] -+; CHECK-NEXT: vpor {{.*}}(%rip), %xmm0, %xmm0 -+; CHECK-NEXT: vpsllw $15, %xmm0, %xmm0 -+; CHECK-NEXT: vpsraw $15, %xmm0, %xmm0 -+; CHECK-NEXT: vzeroupper -+; CHECK-NEXT: retq -+entry: -+ %0 = trunc <8 x i32> %a to <8 x i23> -+ %1 = icmp eq <8 x i23> %0, zeroinitializer -+ %2 = or <8 x i1> %1, <i1 true, i1 true, i1 true, i1 true, i1 false, i1 false, i1 false, i1 false> -+ %3 = sext <8 x i1> %2 to <8 x i16> -+ ret <8 x i16> %3 -+} diff --git a/contrib/llvm/patches/patch-09-clang-r250657-openmp.diff b/contrib/llvm/patches/patch-09-clang-r250657-openmp.diff deleted file mode 100644 index e784673..0000000 --- a/contrib/llvm/patches/patch-09-clang-r250657-openmp.diff +++ /dev/null @@ -1,182 +0,0 @@ -Pull in r248379 from upstream clang trunk (by Jörg Sonnenberger): - - Refactor library decision for -fopenmp support from Darwin into a - function for sharing with other platforms. - -Pull in r248424 from upstream clang trunk (by Jörg Sonnenberger): - - Push OpenMP linker flags after linker input on Darwin. Don't add any - libraries if -nostdlib is specified. Test. - -Pull in r248426 from upstream clang trunk (by Jörg Sonnenberger): - - Support linking against OpenMP runtime on NetBSD. - -Pull in r250657 from upstream clang trunk (by Dimitry Andric): - - Support linking against OpenMP runtime on FreeBSD. - -Introduced here: http://svnweb.freebsd.org/changeset/base/289523 - -Index: tools/clang/lib/Driver/Tools.cpp -=================================================================== ---- tools/clang/lib/Driver/Tools.cpp -+++ tools/clang/lib/Driver/Tools.cpp -@@ -2460,6 +2460,28 @@ static OpenMPRuntimeKind getOpenMPRuntime(const To - return RT; - } - -+static void addOpenMPRuntime(ArgStringList &CmdArgs, const ToolChain &TC, -+ const ArgList &Args) { -+ if (!Args.hasFlag(options::OPT_fopenmp, options::OPT_fopenmp_EQ, -+ options::OPT_fno_openmp, false)) -+ return; -+ -+ switch (getOpenMPRuntime(TC, Args)) { -+ case OMPRT_OMP: -+ CmdArgs.push_back("-lomp"); -+ break; -+ case OMPRT_GOMP: -+ CmdArgs.push_back("-lgomp"); -+ break; -+ case OMPRT_IOMP5: -+ CmdArgs.push_back("-liomp5"); -+ break; -+ case OMPRT_Unknown: -+ // Already diagnosed. -+ break; -+ } -+} -+ - static void addSanitizerRuntime(const ToolChain &TC, const ArgList &Args, - ArgStringList &CmdArgs, StringRef Sanitizer, - bool IsShared) { -@@ -6527,24 +6549,6 @@ void darwin::Linker::ConstructJob(Compilation &C, - - Args.AddAllArgs(CmdArgs, options::OPT_L); - -- if (Args.hasFlag(options::OPT_fopenmp, options::OPT_fopenmp_EQ, -- options::OPT_fno_openmp, false)) { -- switch (getOpenMPRuntime(getToolChain(), Args)) { -- case OMPRT_OMP: -- CmdArgs.push_back("-lomp"); -- break; -- case OMPRT_GOMP: -- CmdArgs.push_back("-lgomp"); -- break; -- case OMPRT_IOMP5: -- CmdArgs.push_back("-liomp5"); -- break; -- case OMPRT_Unknown: -- // Already diagnosed. -- break; -- } -- } -- - AddLinkerInputs(getToolChain(), Inputs, Args, CmdArgs); - // Build the input file for -filelist (list of linker input files) in case we - // need it later -@@ -6563,6 +6567,10 @@ void darwin::Linker::ConstructJob(Compilation &C, - InputFileList.push_back(II.getFilename()); - } - -+ if (!Args.hasArg(options::OPT_nostdlib) && -+ !Args.hasArg(options::OPT_nodefaultlibs)) -+ addOpenMPRuntime(CmdArgs, getToolChain(), Args); -+ - if (isObjCRuntimeLinked(Args) && !Args.hasArg(options::OPT_nostdlib) && - !Args.hasArg(options::OPT_nodefaultlibs)) { - // We use arclite library for both ARC and subscripting support. -@@ -7358,6 +7366,7 @@ void freebsd::Linker::ConstructJob(Compilation &C, - - if (!Args.hasArg(options::OPT_nostdlib) && - !Args.hasArg(options::OPT_nodefaultlibs)) { -+ addOpenMPRuntime(CmdArgs, ToolChain, Args); - if (D.CCCIsCXX()) { - ToolChain.AddCXXStdlibLibArgs(Args, CmdArgs); - if (Args.hasArg(options::OPT_pg)) -@@ -7673,6 +7682,7 @@ void netbsd::Linker::ConstructJob(Compilation &C, - - if (!Args.hasArg(options::OPT_nostdlib) && - !Args.hasArg(options::OPT_nodefaultlibs)) { -+ addOpenMPRuntime(CmdArgs, getToolChain(), Args); - if (D.CCCIsCXX()) { - getToolChain().AddCXXStdlibLibArgs(Args, CmdArgs); - CmdArgs.push_back("-lm"); -Index: tools/clang/test/Driver/fopenmp.c -=================================================================== ---- tools/clang/test/Driver/fopenmp.c -+++ tools/clang/test/Driver/fopenmp.c -@@ -1,6 +1,15 @@ - // RUN: %clang -target x86_64-linux-gnu -fopenmp=libomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP - // RUN: %clang -target x86_64-linux-gnu -fopenmp=libgomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-NO-OPENMP - // RUN: %clang -target x86_64-linux-gnu -fopenmp=libiomp5 -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP -+// RUN: %clang -target x86_64-apple-darwin -fopenmp=libomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP -+// RUN: %clang -target x86_64-apple-darwin -fopenmp=libgomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-NO-OPENMP -+// RUN: %clang -target x86_64-apple-darwin -fopenmp=libiomp5 -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP -+// RUN: %clang -target x86_64-freebsd -fopenmp=libomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP -+// RUN: %clang -target x86_64-freebsd -fopenmp=libgomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-NO-OPENMP -+// RUN: %clang -target x86_64-freebsd -fopenmp=libiomp5 -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP -+// RUN: %clang -target x86_64-netbsd -fopenmp=libomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP -+// RUN: %clang -target x86_64-netbsd -fopenmp=libgomp -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-NO-OPENMP -+// RUN: %clang -target x86_64-netbsd -fopenmp=libiomp5 -c %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-CC1-OPENMP - // - // CHECK-CC1-OPENMP: "-cc1" - // CHECK-CC1-OPENMP: "-fopenmp" -@@ -12,6 +21,30 @@ - // RUN: %clang -target x86_64-linux-gnu -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-GOMP - // RUN: %clang -target x86_64-linux-gnu -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-IOMP5 - // -+// RUN: %clang -nostdlib -target x86_64-linux-gnu -fopenmp=libomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-OMP -+// RUN: %clang -nostdlib -target x86_64-linux-gnu -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-GOMP -+// RUN: %clang -nostdlib -target x86_64-linux-gnu -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-IOMP5 -+// -+// RUN: %clang -target x86_64-darwin -fopenmp=libomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-OMP -+// RUN: %clang -target x86_64-darwin -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-GOMP -+// RUN: %clang -target x86_64-darwin -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-IOMP5 -+// -+// RUN: %clang -nostdlib -target x86_64-darwin -fopenmp=libomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-OMP -+// RUN: %clang -nostdlib -target x86_64-darwin -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-GOMP -+// RUN: %clang -nostdlib -target x86_64-darwin -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-IOMP5 -+// -+// RUN: %clang -target x86_64-netbsd -fopenmp=libomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-OMP -+// RUN: %clang -target x86_64-netbsd -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-GOMP -+// RUN: %clang -target x86_64-netbsd -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-IOMP5 -+// -+// RUN: %clang -nostdlib -target x86_64-freebsd -fopenmp=libomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-OMP -+// RUN: %clang -nostdlib -target x86_64-freebsd -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-GOMP -+// RUN: %clang -nostdlib -target x86_64-freebsd -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-IOMP5 -+// -+// RUN: %clang -nostdlib -target x86_64-netbsd -fopenmp=libomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-OMP -+// RUN: %clang -nostdlib -target x86_64-netbsd -fopenmp=libgomp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-GOMP -+// RUN: %clang -nostdlib -target x86_64-netbsd -fopenmp=libiomp5 %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-IOMP5 -+// - // CHECK-LD-OMP: "{{.*}}ld{{(.exe)?}}" - // CHECK-LD-OMP: "-lomp" - // -@@ -21,6 +54,15 @@ - // CHECK-LD-IOMP5: "{{.*}}ld{{(.exe)?}}" - // CHECK-LD-IOMP5: "-liomp5" - // -+// CHECK-NO-OMP: "{{.*}}ld{{(.exe)?}}" -+// CHECK-NO-OMP-NOT: "-lomp" -+// -+// CHECK-NO-GOMP: "{{.*}}ld{{(.exe)?}}" -+// CHECK-NO-GOMP-NOT: "-lgomp" -+// -+// CHECK-NO-IOMP5: "{{.*}}ld{{(.exe)?}}" -+// CHECK-NO-IOMP5-NOT: "-liomp5" -+// - // We'd like to check that the default is sane, but until we have the ability - // to *always* semantically analyze OpenMP without always generating runtime - // calls (in the event of an unsupported runtime), we don't have a good way to -@@ -28,6 +70,9 @@ - // OpenMP runtime. - // - // RUN: %clang -target x86_64-linux-gnu -fopenmp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-ANY -+// RUN: %clang -target x86_64-darwin -fopenmp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-ANY -+// RUN: %clang -target x86_64-freebsd -fopenmp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-ANY -+// RUN: %clang -target x86_64-netbsd -fopenmp %s -o %t -### 2>&1 | FileCheck %s --check-prefix=CHECK-LD-ANY - // - // CHECK-LD-ANY: "{{.*}}ld{{(.exe)?}}" - // CHECK-LD-ANY: "-l{{(omp|gomp|iomp5)}}" diff --git a/contrib/llvm/tools/clang/lib/Basic/Targets.cpp b/contrib/llvm/tools/clang/lib/Basic/Targets.cpp index e4db004..9e44f7d 100644 --- a/contrib/llvm/tools/clang/lib/Basic/Targets.cpp +++ b/contrib/llvm/tools/clang/lib/Basic/Targets.cpp @@ -3968,7 +3968,13 @@ public: class MinGWX86_64TargetInfo : public WindowsX86_64TargetInfo { public: MinGWX86_64TargetInfo(const llvm::Triple &Triple) - : WindowsX86_64TargetInfo(Triple) {} + : WindowsX86_64TargetInfo(Triple) { + // Mingw64 rounds long double size and alignment up to 16 bytes, but sticks + // with x86 FP ops. Weird. + LongDoubleWidth = LongDoubleAlign = 128; + LongDoubleFormat = &llvm::APFloat::x87DoubleExtended; + } + void getTargetDefines(const LangOptions &Opts, MacroBuilder &Builder) const override { WindowsX86_64TargetInfo::getTargetDefines(Opts, Builder); diff --git a/contrib/llvm/tools/clang/lib/Basic/Version.cpp b/contrib/llvm/tools/clang/lib/Basic/Version.cpp index ab7cff2..4b10dcd 100644 --- a/contrib/llvm/tools/clang/lib/Basic/Version.cpp +++ b/contrib/llvm/tools/clang/lib/Basic/Version.cpp @@ -36,7 +36,7 @@ std::string getClangRepositoryPath() { // If the SVN_REPOSITORY is empty, try to use the SVN keyword. This helps us // pick up a tag in an SVN export, for example. - StringRef SVNRepository("$URL: https://llvm.org/svn/llvm-project/cfe/tags/RELEASE_370/final/lib/Basic/Version.cpp $"); + StringRef SVNRepository("$URL: https://llvm.org/svn/llvm-project/cfe/tags/RELEASE_371/final/lib/Basic/Version.cpp $"); if (URL.empty()) { URL = SVNRepository.slice(SVNRepository.find(':'), SVNRepository.find("/lib/Basic")); diff --git a/contrib/llvm/tools/clang/lib/CodeGen/CGCall.cpp b/contrib/llvm/tools/clang/lib/CodeGen/CGCall.cpp index 3e4d7f3..0bcf59b 100644 --- a/contrib/llvm/tools/clang/lib/CodeGen/CGCall.cpp +++ b/contrib/llvm/tools/clang/lib/CodeGen/CGCall.cpp @@ -1279,12 +1279,7 @@ CodeGenTypes::GetFunctionType(const CGFunctionInfo &FI) { } break; - case ABIArgInfo::Indirect: { - assert(!retAI.getIndirectAlign() && "Align unused on indirect return."); - resultType = llvm::Type::getVoidTy(getLLVMContext()); - break; - } - + case ABIArgInfo::Indirect: case ABIArgInfo::Ignore: resultType = llvm::Type::getVoidTy(getLLVMContext()); break; diff --git a/contrib/llvm/tools/clang/lib/CodeGen/CodeGenModule.cpp b/contrib/llvm/tools/clang/lib/CodeGen/CodeGenModule.cpp index a179ad4..c9c48c7 100644 --- a/contrib/llvm/tools/clang/lib/CodeGen/CodeGenModule.cpp +++ b/contrib/llvm/tools/clang/lib/CodeGen/CodeGenModule.cpp @@ -2493,6 +2493,11 @@ void CodeGenModule::EmitAliasDefinition(GlobalDecl GD) { StringRef MangledName = getMangledName(GD); + if (AA->getAliasee() == MangledName) { + Diags.Report(AA->getLocation(), diag::err_cyclic_alias); + return; + } + // If there is a definition in the module, then it wins over the alias. // This is dubious, but allow it to be safe. Just ignore the alias. llvm::GlobalValue *Entry = GetGlobalValue(MangledName); diff --git a/contrib/llvm/tools/clang/lib/CodeGen/TargetInfo.cpp b/contrib/llvm/tools/clang/lib/CodeGen/TargetInfo.cpp index 48a8b37..25bd733 100644 --- a/contrib/llvm/tools/clang/lib/CodeGen/TargetInfo.cpp +++ b/contrib/llvm/tools/clang/lib/CodeGen/TargetInfo.cpp @@ -1552,12 +1552,10 @@ public: /// WinX86_64ABIInfo - The Windows X86_64 ABI information. class WinX86_64ABIInfo : public ABIInfo { - - ABIArgInfo classify(QualType Ty, unsigned &FreeSSERegs, - bool IsReturnType) const; - public: - WinX86_64ABIInfo(CodeGen::CodeGenTypes &CGT) : ABIInfo(CGT) {} + WinX86_64ABIInfo(CodeGen::CodeGenTypes &CGT) + : ABIInfo(CGT), + IsMingw64(getTarget().getTriple().isWindowsGNUEnvironment()) {} void computeInfo(CGFunctionInfo &FI) const override; @@ -1574,6 +1572,12 @@ public: // FIXME: Assumes vectorcall is in use. return isX86VectorCallAggregateSmallEnough(NumMembers); } + +private: + ABIArgInfo classify(QualType Ty, unsigned &FreeSSERegs, + bool IsReturnType) const; + + bool IsMingw64; }; class X86_64TargetCodeGenInfo : public TargetCodeGenInfo { @@ -3070,11 +3074,6 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs, if (RT->getDecl()->hasFlexibleArrayMember()) return ABIArgInfo::getIndirect(0, /*ByVal=*/false); - - // FIXME: mingw-w64-gcc emits 128-bit struct as i128 - if (Width == 128 && getTarget().getTriple().isWindowsGNUEnvironment()) - return ABIArgInfo::getDirect(llvm::IntegerType::get(getVMContext(), - Width)); } // vectorcall adds the concept of a homogenous vector aggregate, similar to @@ -3116,6 +3115,14 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs, if (BT && BT->getKind() == BuiltinType::Bool) return ABIArgInfo::getExtend(); + // Mingw64 GCC uses the old 80 bit extended precision floating point unit. It + // passes them indirectly through memory. + if (IsMingw64 && BT && BT->getKind() == BuiltinType::LongDouble) { + const llvm::fltSemantics *LDF = &getTarget().getLongDoubleFormat(); + if (LDF == &llvm::APFloat::x87DoubleExtended) + return ABIArgInfo::getIndirect(Align, /*ByVal=*/false); + } + return ABIArgInfo::getDirect(); } diff --git a/contrib/llvm/tools/clang/lib/Sema/SemaExprCXX.cpp b/contrib/llvm/tools/clang/lib/Sema/SemaExprCXX.cpp index 01966d5..9ad5aa5 100644 --- a/contrib/llvm/tools/clang/lib/Sema/SemaExprCXX.cpp +++ b/contrib/llvm/tools/clang/lib/Sema/SemaExprCXX.cpp @@ -2485,8 +2485,10 @@ bool MismatchingNewDeleteDetector::hasMatchingNewInCtor( MismatchingNewDeleteDetector::MismatchResult MismatchingNewDeleteDetector::analyzeInClassInitializer() { assert(Field != nullptr && "This should be called only for members"); - if (const CXXNewExpr *NE = - getNewExprFromInitListOrExpr(Field->getInClassInitializer())) { + const Expr *InitExpr = Field->getInClassInitializer(); + if (!InitExpr) + return EndOfTU ? NoMismatch : AnalyzeLater; + if (const CXXNewExpr *NE = getNewExprFromInitListOrExpr(InitExpr)) { if (NE->isArray() != IsArrayForm) { NewExprs.push_back(NE); return MemberInitMismatches; diff --git a/contrib/llvm/tools/lldb/source/Plugins/Process/FreeBSD/RegisterContextPOSIXProcessMonitor_arm64.cpp b/contrib/llvm/tools/lldb/source/Plugins/Process/FreeBSD/RegisterContextPOSIXProcessMonitor_arm64.cpp index d79def5..44cb603 100644 --- a/contrib/llvm/tools/lldb/source/Plugins/Process/FreeBSD/RegisterContextPOSIXProcessMonitor_arm64.cpp +++ b/contrib/llvm/tools/lldb/source/Plugins/Process/FreeBSD/RegisterContextPOSIXProcessMonitor_arm64.cpp @@ -260,13 +260,9 @@ RegisterContextPOSIXProcessMonitor_arm64::HardwareSingleStep(bool enable) bool RegisterContextPOSIXProcessMonitor_arm64::UpdateAfterBreakpoint() { - // PC points one byte past the int3 responsible for the breakpoint. - lldb::addr_t pc; - - if ((pc = GetPC()) == LLDB_INVALID_ADDRESS) + if (GetPC() == LLDB_INVALID_ADDRESS) return false; - SetPC(pc - 1); return true; } diff --git a/contrib/llvm/tools/llvm-lto/llvm-lto.cpp b/contrib/llvm/tools/llvm-lto/llvm-lto.cpp index 9678c83..0821898 100644 --- a/contrib/llvm/tools/llvm-lto/llvm-lto.cpp +++ b/contrib/llvm/tools/llvm-lto/llvm-lto.cpp @@ -214,8 +214,11 @@ int main(int argc, char **argv) { if (SetMergedModule && i == BaseArg) { // Transfer ownership to the code generator. CodeGen.setModule(Module.release()); - } else if (!CodeGen.addModule(Module.get())) + } else if (!CodeGen.addModule(Module.get())) { + // Print a message here so that we know addModule() did not abort. + errs() << argv[0] << ": error adding file '" << InputFilenames[i] << "'\n"; return 1; + } unsigned NumSyms = LTOMod->getSymbolCount(); for (unsigned I = 0; I < NumSyms; ++I) { diff --git a/contrib/netbsd-tests/lib/libc/gen/t_assert.c b/contrib/netbsd-tests/lib/libc/gen/t_assert.c index 140417a..a09c130 100644 --- a/contrib/netbsd-tests/lib/libc/gen/t_assert.c +++ b/contrib/netbsd-tests/lib/libc/gen/t_assert.c @@ -40,6 +40,23 @@ __RCSID("$NetBSD: t_assert.c,v 1.2 2011/06/14 05:28:00 jruoho Exp $"); #include <string.h> #include <unistd.h> +#ifdef __FreeBSD__ +#include <sys/types.h> +#include <sys/time.h> +#include <sys/resource.h> + +static void +disable_corefile(void) +{ + struct rlimit limits; + + limits.rlim_cur = 0; + limits.rlim_max = 0; + + ATF_REQUIRE(setrlimit(RLIMIT_CORE, &limits) == 0); +} +#endif + static void handler(int); static void @@ -65,6 +82,9 @@ ATF_TC_BODY(assert_false, tc) if (pid == 0) { +#ifdef __FreeBSD__ + disable_corefile(); +#endif (void)closefrom(0); (void)memset(&sa, 0, sizeof(struct sigaction)); @@ -102,6 +122,9 @@ ATF_TC_BODY(assert_true, tc) if (pid == 0) { +#ifdef __FreeBSD__ + disable_corefile(); +#endif (void)closefrom(0); (void)memset(&sa, 0, sizeof(struct sigaction)); diff --git a/contrib/netbsd-tests/lib/libc/ssp/t_ssp.sh b/contrib/netbsd-tests/lib/libc/ssp/t_ssp.sh index 362178f..e535e3e 100755 --- a/contrib/netbsd-tests/lib/libc/ssp/t_ssp.sh +++ b/contrib/netbsd-tests/lib/libc/ssp/t_ssp.sh @@ -35,6 +35,7 @@ h_fail() { echo "Executing command [ $2$1 ]" # Begin FreeBSD + ulimit -c 0 if true; then eval $2 atf_check -s signal -o ignore -e ignore $1 else diff --git a/contrib/netbsd-tests/usr.bin/grep/d_binary.out b/contrib/netbsd-tests/usr.bin/grep/d_binary.out index ce03056..f0ef988 100644 --- a/contrib/netbsd-tests/usr.bin/grep/d_binary.out +++ b/contrib/netbsd-tests/usr.bin/grep/d_binary.out @@ -1 +1 @@ -Binary file /bin/sh matches +Binary file test.file matches diff --git a/contrib/netbsd-tests/usr.bin/grep/t_grep.sh b/contrib/netbsd-tests/usr.bin/grep/t_grep.sh index f2d70f0..7e53016 100755 --- a/contrib/netbsd-tests/usr.bin/grep/t_grep.sh +++ b/contrib/netbsd-tests/usr.bin/grep/t_grep.sh @@ -43,7 +43,20 @@ binary_head() } binary_body() { + # Begin FreeBSD + # + # Generate stable output instead of depending on uname to match the + # branded OS name of /bin/sh + if true; then + dd if=/dev/zero count=1 of=test.file + echo -n "foobar" >> test.file + atf_check -o file:"$(atf_get_srcdir)/d_binary.out" grep foobar test.file + else + # End FreeBSD atf_check -o file:"$(atf_get_srcdir)/d_binary.out" grep $(uname) /bin/sh + # Begin FreeBSD + fi + # End FreeBSD } atf_test_case recurse diff --git a/contrib/smbfs/lib/smb/nb_name.c b/contrib/smbfs/lib/smb/nb_name.c index 2346ec7..a953ce0 100644 --- a/contrib/smbfs/lib/smb/nb_name.c +++ b/contrib/smbfs/lib/smb/nb_name.c @@ -143,15 +143,13 @@ nb_encname_len(const char *str) return len; } -#define NBENCODE(c) (htole16((u_short)(((u_char)(c) >> 4) | \ - (((u_char)(c) & 0xf) << 8)) + 0x4141)) - -static void -memsetw(char *dst, int n, u_short word) +static inline void +nb_char_encode(u_char **ptr, u_char c, int n) { + while (n--) { - *(u_short*)dst = word; - dst += 2; + *(*ptr)++ = 0x41 + (c >> 4); + *(*ptr)++ = 0x41 + (c & 0x0f); } } @@ -165,19 +163,15 @@ nb_name_encode(struct nb_name *np, u_char *dst) *cp++ = NB_ENCNAMELEN; name = np->nn_name; if (name[0] == '*' && name[1] == 0) { - *(u_short*)cp = NBENCODE('*'); - memsetw(cp + 2, NB_NAMELEN - 1, NBENCODE(' ')); - cp += NB_ENCNAMELEN; + nb_char_encode(&cp, '*', 1); + nb_char_encode(&cp, ' ', NB_NAMELEN - 1); } else { - for (i = 0; *name && i < NB_NAMELEN - 1; i++, cp += 2, name++) - *(u_short*)cp = NBENCODE(toupper(*name)); - i = NB_NAMELEN - i - 1; - if (i > 0) { - memsetw(cp, i, NBENCODE(' ')); - cp += i * 2; - } - *(u_short*)cp = NBENCODE(np->nn_type); - cp += 2; + for (i = 0; i < NB_NAMELEN - 1; i++) + if (*name != 0) + nb_char_encode(&cp, toupper(*name++), 1); + else + nb_char_encode(&cp, ' ', 1); + nb_char_encode(&cp, np->nn_type, 1); } *cp = 0; if (np->nn_scope == NULL) diff --git a/etc/defaults/rc.conf b/etc/defaults/rc.conf index 31a106a..12aaf31 100644 --- a/etc/defaults/rc.conf +++ b/etc/defaults/rc.conf @@ -439,9 +439,9 @@ ubthidhci_enable="NO" # Switch an USB BT controller present on ### Network link/usability verification options netwait_enable="NO" # Enable rc.d/netwait (or NO) -#netwait_ip="" # IP addresses to be pinged by netwait. +#netwait_ip="" # Wait for ping response from any IP in this list. netwait_timeout="60" # Total number of seconds to perform pings. -#netwait_if="" # Interface name to watch link state on. +#netwait_if="" # Wait for active link on each intf in this list. netwait_if_timeout="30" # Total number of seconds to monitor link state. ### Miscellaneous network options: ### diff --git a/etc/mtree/BSD.debug.dist b/etc/mtree/BSD.debug.dist index e760e01..4a499ea 100644 --- a/etc/mtree/BSD.debug.dist +++ b/etc/mtree/BSD.debug.dist @@ -27,7 +27,7 @@ .. lib clang - 3.7.0 + 3.7.1 lib freebsd .. diff --git a/etc/mtree/BSD.tests.dist b/etc/mtree/BSD.tests.dist index 7887ee4..089a419 100644 --- a/etc/mtree/BSD.tests.dist +++ b/etc/mtree/BSD.tests.dist @@ -422,6 +422,12 @@ .. kqueue .. + mac + bsdextended + .. + portacl + .. + .. mqueue .. netinet diff --git a/etc/mtree/BSD.usr.dist b/etc/mtree/BSD.usr.dist index 3a00a90..1a5835b 100644 --- a/etc/mtree/BSD.usr.dist +++ b/etc/mtree/BSD.usr.dist @@ -19,10 +19,10 @@ aout .. clang - 3.7.0 + 3.7.1 include - sanitizer - .. + sanitizer + .. .. lib freebsd diff --git a/etc/rc.d/NETWORKING b/etc/rc.d/NETWORKING index 2294628..92ba05d 100755 --- a/etc/rc.d/NETWORKING +++ b/etc/rc.d/NETWORKING @@ -4,7 +4,7 @@ # # PROVIDE: NETWORKING NETWORK -# REQUIRE: netif netoptions routing ppp ipfw stf +# REQUIRE: netif netwait netoptions routing ppp ipfw stf # REQUIRE: defaultroute routed route6d mroute6d resolv bridge # REQUIRE: static_arp static_ndp diff --git a/etc/rc.d/jail b/etc/rc.d/jail index 70d865b..a1b4bbc 100755 --- a/etc/rc.d/jail +++ b/etc/rc.d/jail @@ -28,16 +28,16 @@ extra_commands="config console status" need_dad_wait= -# extract_var jail name param num defval -# Extract value from ${jail_$jail_$name} or ${jail_$name} and +# extract_var jv name param num defval +# Extract value from ${jail_$jv_$name} or ${jail_$name} and # set it to $param. If not defined, $defval is used. -# When $num is [0-9]*, ${jail_$jail_$name$num} are looked up and +# When $num is [0-9]*, ${jail_$jv_$name$num} are looked up and # $param is set by using +=. # When $num is YN or NY, the value is interpret as boolean. extract_var() { - local i _j _name _param _num _def _name1 _name2 - _j=$1 + local i _jv _name _param _num _def _name1 _name2 + _jv=$1 _name=$2 _param=$3 _num=$4 @@ -45,7 +45,7 @@ extract_var() case $_num in YN) - _name1=jail_${_j}_${_name} + _name1=jail_${_jv}_${_name} _name2=jail_${_name} eval $_name1=\"\${$_name1:-\${$_name2:-$_def}}\" if checkyesno $_name1; then @@ -55,7 +55,7 @@ extract_var() fi ;; NY) - _name1=jail_${_j}_${_name} + _name1=jail_${_jv}_${_name} _name2=jail_${_name} eval $_name1=\"\${$_name1:-\${$_name2:-$_def}}\" if checkyesno $_name1; then @@ -67,7 +67,7 @@ extract_var() [0-9]*) i=$_num while : ; do - _name1=jail_${_j}_${_name}${i} + _name1=jail_${_jv}_${_name}${i} _name2=jail_${_name}${i} eval _tmpargs=\"\${$_name1:-\${$_name2:-$_def}}\" if [ -n "$_tmpargs" ]; then @@ -79,7 +79,7 @@ extract_var() done ;; *) - _name1=jail_${_j}_${_name} + _name1=jail_${_jv}_${_name} _name2=jail_${_name} eval _tmpargs=\"\${$_name1:-\${$_name2:-$_def}}\" if [ -n "$_tmpargs" ]; then @@ -89,22 +89,23 @@ extract_var() esac } -# parse_options _j +# parse_options _j _jv # Parse options and create a temporary configuration file if necessary. # parse_options() { - local _j _p + local _j _jv _p _j=$1 + _jv=$2 _confwarn=0 if [ -z "$_j" ]; then warn "parse_options: you must specify a jail" return fi - eval _jconf=\"\${jail_${_j}_conf:-/etc/jail.${_j}.conf}\" - eval _rootdir=\"\$jail_${_j}_rootdir\" - eval _hostname=\"\$jail_${_j}_hostname\" + eval _jconf=\"\${jail_${_jv}_conf:-/etc/jail.${_j}.conf}\" + eval _rootdir=\"\$jail_${_jv}_rootdir\" + eval _hostname=\"\$jail_${_jv}_hostname\" if [ -z "$_rootdir" -o \ -z "$_hostname" ]; then if [ -r "$_jconf" ]; then @@ -120,7 +121,7 @@ parse_options() fi return 1 fi - eval _ip=\"\$jail_${_j}_ip\" + eval _ip=\"\$jail_${_jv}_ip\" if [ -z "$_ip" ] && ! check_kern_features vimage; then warn "no ipaddress specified and no vimage support. " \ "Jail $_j was ignored." @@ -138,10 +139,10 @@ parse_options() fi /usr/bin/install -m 0644 -o root -g wheel /dev/null $_conf || return 1 - eval : \${jail_${_j}_flags:=${jail_flags}} - eval _exec=\"\$jail_${_j}_exec\" - eval _exec_start=\"\$jail_${_j}_exec_start\" - eval _exec_stop=\"\$jail_${_j}_exec_stop\" + eval : \${jail_${_jv}_flags:=${jail_flags}} + eval _exec=\"\$jail_${_jv}_exec\" + eval _exec_start=\"\$jail_${_jv}_exec_start\" + eval _exec_stop=\"\$jail_${_jv}_exec_stop\" if [ -n "${_exec}" ]; then # simple/backward-compatible execution _exec_start="${_exec}" @@ -155,20 +156,20 @@ parse_options() fi fi fi - eval _interface=\"\${jail_${_j}_interface:-${jail_interface}}\" - eval _parameters=\"\${jail_${_j}_parameters:-${jail_parameters}}\" - eval _fstab=\"\${jail_${_j}_fstab:-${jail_fstab:-/etc/fstab.$_j}}\" + eval _interface=\"\${jail_${_jv}_interface:-${jail_interface}}\" + eval _parameters=\"\${jail_${_jv}_parameters:-${jail_parameters}}\" + eval _fstab=\"\${jail_${_jv}_fstab:-${jail_fstab:-/etc/fstab.$_j}}\" ( date +"# Generated by rc.d/jail at %Y-%m-%d %H:%M:%S" echo "$_j {" - extract_var $_j hostname host.hostname - "" - extract_var $_j rootdir path - "" + extract_var $_jv hostname host.hostname - "" + extract_var $_jv rootdir path - "" if [ -n "$_ip" ]; then - extract_var $_j interface interface - "" + extract_var $_jv interface interface - "" jail_handle_ips_option $_ip $_interface alias=0 while : ; do - eval _x=\"\$jail_${_j}_ip_multi${alias}\" + eval _x=\"\$jail_${_jv}_ip_multi${alias}\" [ -z "$_x" ] && break jail_handle_ips_option $_x $_interface @@ -184,37 +185,37 @@ parse_options() ;; esac # These are applicable only to non-vimage jails. - extract_var $_j fib exec.fib - "" - extract_var $_j socket_unixiproute_only \ + extract_var $_jv fib exec.fib - "" + extract_var $_jv socket_unixiproute_only \ allow.raw_sockets NY YES else echo " vnet;" - extract_var $_j vnet_interface vnet.interface - "" + extract_var $_jv vnet_interface vnet.interface - "" fi echo " exec.clean;" echo " exec.system_user = \"root\";" echo " exec.jail_user = \"root\";" - extract_var $_j exec_prestart exec.prestart 0 "" - extract_var $_j exec_poststart exec.poststart 0 "" - extract_var $_j exec_prestop exec.prestop 0 "" - extract_var $_j exec_poststop exec.poststop 0 "" + extract_var $_jv exec_prestart exec.prestart 0 "" + extract_var $_jv exec_poststart exec.poststart 0 "" + extract_var $_jv exec_prestop exec.prestop 0 "" + extract_var $_jv exec_poststop exec.poststop 0 "" echo " exec.start += \"$_exec_start\";" - extract_var $_j exec_afterstart exec.start 1 "" + extract_var $_jv exec_afterstart exec.start 1 "" echo " exec.stop = \"$_exec_stop\";" - extract_var $_j consolelog exec.consolelog - \ + extract_var $_jv consolelog exec.consolelog - \ /var/log/jail_${_j}_console.log if [ -r $_fstab ]; then echo " mount.fstab = \"$_fstab\";" fi - eval : \${jail_${_j}_devfs_enable:=${jail_devfs_enable:-NO}} - if checkyesno jail_${_j}_devfs_enable; then + eval : \${jail_${_jv}_devfs_enable:=${jail_devfs_enable:-NO}} + if checkyesno jail_${_jv}_devfs_enable; then echo " mount.devfs;" - eval _ruleset=\${jail_${_j}_devfs_ruleset:-${jail_devfs_ruleset}} + eval _ruleset=\${jail_${_jv}_devfs_ruleset:-${jail_devfs_ruleset}} case $_ruleset in "") ;; [0-9]*) echo " devfs_ruleset = \"$_ruleset\";" ;; @@ -227,24 +228,24 @@ parse_options() *) warn "devfs_ruleset must be an integer." ;; esac fi - eval : \${jail_${_j}_fdescfs_enable:=${jail_fdescfs_enable:-NO}} - if checkyesno jail_${_j}_fdescfs_enable; then + eval : \${jail_${_jv}_fdescfs_enable:=${jail_fdescfs_enable:-NO}} + if checkyesno jail_${_jv}_fdescfs_enable; then echo " mount.fdescfs;" fi - eval : \${jail_${_j}_procfs_enable:=${jail_procfs_enable:-NO}} - if checkyesno jail_${_j}_procfs_enable; then + eval : \${jail_${_jv}_procfs_enable:=${jail_procfs_enable:-NO}} + if checkyesno jail_${_jv}_procfs_enable; then echo " mount.procfs;" fi - eval : \${jail_${_j}_mount_enable:=${jail_mount_enable:-NO}} - if checkyesno jail_${_j}_mount_enable; then + eval : \${jail_${_jv}_mount_enable:=${jail_mount_enable:-NO}} + if checkyesno jail_${_jv}_mount_enable; then echo " allow.mount;" fi - extract_var $_j set_hostname_allow allow.set_hostname YN NO - extract_var $_j sysvipc_allow allow.sysvipc YN NO - extract_var $_j osreldate osreldate - extract_var $_j osrelease osrelease + extract_var $_jv set_hostname_allow allow.set_hostname YN NO + extract_var $_jv sysvipc_allow allow.sysvipc YN NO + extract_var $_jv osreldate osreldate + extract_var $_jv osrelease osrelease for _p in $_parameters; do echo " ${_p%\;};" done @@ -382,14 +383,15 @@ jail_handle_ips_option() jail_config() { - local _j + local _j _jv case $1 in _ALL) return ;; esac for _j in $@; do _j=$(echo $_j | tr /. _) - if parse_options $_j; then + _jv=$(echo -n $_j | tr -c '[:alnum:]' _) + if parse_options $_j $_jv; then echo "$_j: parameters are in $_conf." fi done @@ -397,7 +399,7 @@ jail_config() jail_console() { - local _j _cmd + local _j _jv _cmd # One argument that is not _ALL. case $#:$1 in @@ -405,9 +407,10 @@ jail_console() 1:*) ;; esac _j=$(echo $1 | tr /. _) + _jv=$(echo -n $1 | tr -c '[:alnum:]' _) shift case $# in - 0) eval _cmd=\${jail_${_j}_consolecmd:-$jail_consolecmd} ;; + 0) eval _cmd=\${jail_${_jv}_consolecmd:-$jail_consolecmd} ;; *) _cmd=$@ ;; esac $jail_jexec $_j $_cmd @@ -421,7 +424,7 @@ jail_status() jail_start() { - local _j _jid _jl _id _name + local _j _jv _jid _jl _id _name if [ $# = 0 ]; then return @@ -454,11 +457,12 @@ jail_start() _jl= for _j in $@; do _j=$(echo $_j | tr /. _) - parse_options $_j || continue + _jv=$(echo -n $_j | tr -c '[:alnum:]' _) + parse_options $_j $_jv || continue _jl="$_jl $_j" - eval rc_flags=\${jail_${_j}_flags:-$jail_flags} - eval command=\${jail_${_j}_program:-$jail_program} + eval rc_flags=\${jail_${_jv}_flags:-$jail_flags} + eval command=\${jail_${_jv}_program:-$jail_program} command_args="-i -f $_conf -c $_j" $command $rc_flags $command_args \ >/dev/null 2>&1 </dev/null & @@ -480,10 +484,11 @@ jail_start() # for _j in $@; do _j=$(echo $_j | tr /. _) - parse_options $_j || continue + _jv=$(echo -n $_j | tr -c '[:alnum:]' _) + parse_options $_j $_jv || continue - eval rc_flags=\${jail_${_j}_flags:-$jail_flags} - eval command=\${jail_${_j}_program:-$jail_program} + eval rc_flags=\${jail_${_jv}_flags:-$jail_flags} + eval command=\${jail_${_jv}_program:-$jail_program} command_args="-i -f $_conf -c $_j" _tmp=`mktemp -t jail` || exit 3 if $command $rc_flags $command_args \ @@ -505,7 +510,7 @@ jail_start() jail_stop() { - local _j + local _j _jv if [ $# = 0 ]; then return @@ -533,11 +538,12 @@ jail_stop() esac for _j in $@; do _j=$(echo $_j | tr /. _) - parse_options $_j || continue + _jv=$(echo -n $_j | tr -c '[:alnum:]' _) + parse_options $_j $_jv || continue if ! $jail_jls -j $_j > /dev/null 2>&1; then continue fi - eval command=\${jail_${_j}_program:-$jail_program} + eval command=\${jail_${_jv}_program:-$jail_program} echo -n " ${_hostname:-${_j}}" _tmp=`mktemp -t jail` || exit 3 $command -q -f $_conf -r $_j >> $_tmp 2>&1 diff --git a/etc/rc.d/netwait b/etc/rc.d/netwait index 7ea7a4c..6ccca04 100755 --- a/etc/rc.d/netwait +++ b/etc/rc.d/netwait @@ -3,13 +3,20 @@ # $FreeBSD$ # # PROVIDE: netwait -# REQUIRE: NETWORKING +# REQUIRE: devd routing # KEYWORD: nojail # -# The netwait script is intended to be used by systems which have -# statically-configured IP addresses in rc.conf(5). If your system -# uses DHCP, you should use synchronous_dhclient="YES" in your -# /etc/rc.conf instead of using netwait. +# The netwait script helps handle two situations: +# - Systems with USB or other late-attaching network hardware which +# is initialized by devd events. The script waits for all the +# interfaces named in the netwait_if list to appear. +# - Systems with statically-configured IP addresses in rc.conf(5). +# The IP addresses in the netwait_ip list are pinged. The script +# waits for any single IP in the list to respond to the ping. If your +# system uses DHCP, you should probably use synchronous_dhclient="YES" +# in your /etc/rc.conf instead of netwait_ip. +# Either or both of the wait lists can be used (at least one must be +# non-empty if netwait is enabled). . /etc/rc.subr @@ -21,77 +28,87 @@ stop_cmd=":" netwait_start() { - local ip rc count output link + local ip rc count output link wait_if got_if any_error - if [ -z "${netwait_ip}" ]; then - err 1 "You must define one or more IP addresses in netwait_ip" + if [ -z "${netwait_if}" ] && [ -z "${netwait_ip}" ]; then + err 1 "No interface or IP addresses listed, nothing to wait for" fi if [ ${netwait_timeout} -lt 1 ]; then err 1 "netwait_timeout must be >= 1" fi - # Handle SIGINT (Ctrl-C); force abort of while() loop - trap break SIGINT - if [ -n "${netwait_if}" ]; then - echo -n "Waiting for $netwait_if to have link" - - count=1 - while [ ${count} -le ${netwait_if_timeout} ]; do - if output=`/sbin/ifconfig ${netwait_if} 2>/dev/null`; then - link=`expr "${output}" : '.*[[:blank:]]status: \(no carrier\)'` - if [ -z "${link}" ]; then - echo '.' - break + any_error=0 + for wait_if in ${netwait_if}; do + echo -n "Waiting for ${wait_if}" + link="" + got_if=0 + count=1 + # Handle SIGINT (Ctrl-C); force abort of while() loop + trap break SIGINT + while [ ${count} -le ${netwait_if_timeout} ]; do + if output=`/sbin/ifconfig ${wait_if} 2>/dev/null`; then + if [ ${got_if} -eq 0 ]; then + echo -n ", interface present" + got_if=1 + fi + link=`expr "${output}" : '.*[[:blank:]]status: \(no carrier\)'` + if [ -z "${link}" ]; then + echo ', got link.' + break + fi fi - else - echo '' - err 1 "ifconfig ${netwait_if} failed" - fi - sleep 1 - count=$((count+1)) - done - if [ -n "${link}" ]; then + sleep 1 + count=$((count+1)) + done # Restore default SIGINT handler trap - SIGINT - - echo '' - warn "Interface still has no link. Continuing with startup, but" - warn "be aware you may not have a fully functional networking" - warn "layer at this point." - return + if [ ${got_if} -eq 0 ]; then + echo ", wait failed: interface never appeared." + any_error=1 + elif [ -n "${link}" ]; then + echo ", wait failed: interface still has no link." + any_error=1 + fi + done + if [ ${any_error} -eq 1 ]; then + warn "Continuing with startup, but be aware you may not have " + warn "a fully functional networking layer at this point." fi fi + + if [ -n "${netwait_ip}" ]; then + # Handle SIGINT (Ctrl-C); force abort of for() loop + trap break SIGINT - # Handle SIGINT (Ctrl-C); force abort of while() loop - trap break SIGINT + for ip in ${netwait_ip}; do + echo -n "Waiting for ${ip} to respond to ICMP ping" - for ip in ${netwait_ip}; do - echo -n "Waiting for ${ip} to respond to ICMP" + count=1 + while [ ${count} -le ${netwait_timeout} ]; do + /sbin/ping -t 1 -c 1 -o ${ip} >/dev/null 2>&1 + rc=$? - count=1 - while [ ${count} -le ${netwait_timeout} ]; do - /sbin/ping -t 1 -c 1 -o ${ip} >/dev/null 2>&1 - rc=$? + if [ $rc -eq 0 ]; then + # Restore default SIGINT handler + trap - SIGINT - if [ $rc -eq 0 ]; then - # Restore default SIGINT handler - trap - SIGINT - - echo '.' - return - fi - count=$((count+1)) + echo ', got response.' + return + fi + count=$((count+1)) + done + echo ', failed: No response from host.' done - echo ': No response from host.' - done - # Restore default SIGINT handler - trap - SIGINT + # Restore default SIGINT handler + trap - SIGINT + + warn "Exhausted IP list. Continuing with startup, but be aware you may" + warn "not have a fully functional networking layer at this point." + fi - warn "Exhausted IP list. Continuing with startup, but be aware you may" - warn "not have a fully functional networking layer at this point." } load_rc_config $name diff --git a/etc/services b/etc/services index 3d06be6..dde71fb 100644 --- a/etc/services +++ b/etc/services @@ -1029,7 +1029,7 @@ nntps 563/udp snntp #nntp protocol over TLS/SSL whoami 565/tcp whoami 565/udp streettalk 566/tcp -streettalk 566/udp +streettalk 566/udp banyan-rpc 567/tcp banyan-rpc 567/udp ms-shuttle 568/tcp #Microsoft shuttle @@ -1099,16 +1099,16 @@ acp 599/tcp #Aeolon Core Protocol acp 599/udp #Aeolon Core Protocol ipcserver 600/tcp #Sun IPC server ipcserver 600/udp #Sun IPC server -syslog-conn 601/tcp #Reliable Syslog Service -syslog-conn 601/udp #Reliable Syslog Service -xmlrpc-beep 602/tcp #XML-RPC over BEEP -xmlrpc-beep 602/udp #XML-RPC over BEEP -idxp 603/tcp -idxp 603/udp -tunnel 604/tcp -tunnel 604/udp -soap-beep 605/tcp #SOAP over BEEP -soap-beep 605/udp #SOAP over BEEP +syslog-conn 601/tcp #Reliable Syslog Service +syslog-conn 601/udp #Reliable Syslog Service +xmlrpc-beep 602/tcp #XML-RPC over BEEP +xmlrpc-beep 602/udp #XML-RPC over BEEP +idxp 603/tcp +idxp 603/udp +tunnel 604/tcp +tunnel 604/udp +soap-beep 605/tcp #SOAP over BEEP +soap-beep 605/udp #SOAP over BEEP urm 606/tcp #Cray Unified Resource Manager urm 606/udp #Cray Unified Resource Manager nqs 607/tcp @@ -1137,14 +1137,14 @@ dei-icda 618/tcp dei-icda 618/udp compaq-evm 619/tcp #Compaq EVM compaq-evm 619/udp #Compaq EVM -sco-websrvrmgr 620/tcp #SCO WebServer Manager -sco-websrvrmgr 620/udp #SCO WebServer Manager +sco-websrvrmgr 620/tcp #SCO WebServer Manager +sco-websrvrmgr 620/udp #SCO WebServer Manager escp-ip 621/tcp #ESCP escp-ip 621/udp #ESCP collaborator 622/tcp collaborator 622/udp -asf-rmcp 623/tcp #ASF Remote Management and Control Protocol -asf-rmcp 623/udp #ASF Remote Management and Control Protocol +asf-rmcp 623/tcp #ASF Remote Management and Control Protocol +asf-rmcp 623/udp #ASF Remote Management and Control Protocol cryptoadmin 624/tcp #Crypto Admin cryptoadmin 624/udp #Crypto Admin dec_dlm 625/tcp #DEC DLM @@ -1167,12 +1167,12 @@ servstat 633/tcp #Service Status update (Sterling Software) servstat 633/udp #Service Status update (Sterling Software) ginad 634/tcp ginad 634/udp -rlzdbase 635/tcp #RLZ DBase -rlzdbase 635/udp #RLZ DBase +rlzdbase 635/tcp #RLZ DBase +rlzdbase 635/udp #RLZ DBase ldaps 636/tcp sldap #ldap protocol over TLS/SSL ldaps 636/udp sldap -lanserver 637/tcp -lanserver 637/udp +lanserver 637/tcp +lanserver 637/udp mcns-sec 638/tcp mcns-sec 638/udp msdp 639/tcp @@ -1191,12 +1191,12 @@ pssc 645/tcp pssc 645/udp ldp 646/tcp ldp 646/udp -dhcp-failover 647/tcp #DHCP Failover -dhcp-failover 647/udp #DHCP Failover -rrp 648/tcp #Registry Registrar Protocol (RRP) -rrp 648/udp #Registry Registrar Protocol (RRP) -cadview-3d 649/tcp #Cadview-3d - streaming 3d models over the internet -cadview-3d 649/udp #Cadview-3d - streaming 3d models over the internet +dhcp-failover 647/tcp #DHCP Failover +dhcp-failover 647/udp #DHCP Failover +rrp 648/tcp #Registry Registrar Protocol (RRP) +rrp 648/udp #Registry Registrar Protocol (RRP) +cadview-3d 649/tcp #Cadview-3d - streaming 3d models over the internet +cadview-3d 649/udp #Cadview-3d - streaming 3d models over the internet obex 650/tcp obex 650/udp ieee-mms 651/tcp #IEEE MMS @@ -1215,38 +1215,38 @@ rmc 657/tcp rmc 657/udp tenfold 658/tcp tenfold 658/udp -mac-srvr-admin 660/tcp #MacOS Server Admin -mac-srvr-admin 660/udp #MacOS Server Admin -hap 661/tcp -hap 661/udp -pftp 662/tcp -pftp 662/udp -purenoise 663/tcp #PureNoise -purenoise 663/udp #PureNoise -asf-secure-rmcp 664/tcp #ASF Secure Remote Management and Control Protocol -asf-secure-rmcp 664/udp #ASF Secure Remote Management and Control Protocol -sun-dr 665/tcp #Sun DR -sun-dr 665/udp #Sun DR +mac-srvr-admin 660/tcp #MacOS Server Admin +mac-srvr-admin 660/udp #MacOS Server Admin +hap 661/tcp +hap 661/udp +pftp 662/tcp +pftp 662/udp +purenoise 663/tcp #PureNoise +purenoise 663/udp #PureNoise +asf-secure-rmcp 664/tcp #ASF Secure Remote Management and Control Protocol +asf-secure-rmcp 664/udp #ASF Secure Remote Management and Control Protocol +sun-dr 665/tcp #Sun DR +sun-dr 665/udp #Sun DR mdqs 666/tcp mdqs 666/udp #PROBLEMS!=============================================== doom 666/tcp #doom Id Software doom 666/udp #doom Id Software #PROBLEMS!=============================================== -disclose 667/tcp #campaign contribution disclosures - SDR Technologies -disclose 667/udp #campaign contribution disclosures - SDR Technologies -mecomm 668/tcp -mecomm 668/udp -meregister 669/tcp -meregister 669/udp -vacdsm-sws 670/tcp -vacdsm-sws 670/udp -vacdsm-app 671/tcp -vacdsm-app 671/udp -vpps-qua 672/tcp -vpps-qua 672/udp -cimplex 673/tcp -cimplex 673/udp +disclose 667/tcp #campaign contribution disclosures - SDR Technologies +disclose 667/udp #campaign contribution disclosures - SDR Technologies +mecomm 668/tcp +mecomm 668/udp +meregister 669/tcp +meregister 669/udp +vacdsm-sws 670/tcp +vacdsm-sws 670/udp +vacdsm-app 671/tcp +vacdsm-app 671/udp +vpps-qua 672/tcp +vpps-qua 672/udp +cimplex 673/tcp +cimplex 673/udp acap 674/tcp #Application Configuration Access Protocol acap 674/udp #Application Configuration Access Protocol dctp 675/tcp @@ -1257,70 +1257,70 @@ vpp 677/tcp #Virtual Presence Protocol vpp 677/udp #Virtual Presence Protocol ggf-ncp 678/tcp #GNU Generation Foundation NCP ggf-ncp 678/udp #GNU Generation Foundation NCP -mrm 679/tcp -mrm 679/udp +mrm 679/tcp +mrm 679/udp entrust-aaas 680/tcp entrust-aaas 680/udp entrust-aams 681/tcp entrust-aams 681/udp -xfr 682/tcp -xfr 682/udp -corba-iiop 683/tcp #CORBA IIOP -corba-iiop 683/udp #CORBA IIOP +xfr 682/tcp +xfr 682/udp +corba-iiop 683/tcp #CORBA IIOP +corba-iiop 683/udp #CORBA IIOP corba-iiop-ssl 684/tcp #CORBA IIOP SSL corba-iiop-ssl 684/udp #CORBA IIOP SSL mdc-portmapper 685/tcp #MDC Port Mapper mdc-portmapper 685/udp #MDC Port Mapper -hcp-wismar 686/tcp #Hardware Control Protocol Wismar -hcp-wismar 686/udp #Hardware Control Protocol Wismar +hcp-wismar 686/tcp #Hardware Control Protocol Wismar +hcp-wismar 686/udp #Hardware Control Protocol Wismar asipregistry 687/tcp asipregistry 687/udp -realm-rusd 688/tcp #ApplianceWare management protocol -realm-rusd 688/udp #ApplianceWare management protocol -nmap 689/tcp -nmap 689/udp -vatp 690/tcp #Velazquez Application Transfer Protocol -vatp 690/udp #Velazquez Application Transfer Protocol +realm-rusd 688/tcp #ApplianceWare management protocol +realm-rusd 688/udp #ApplianceWare management protocol +nmap 689/tcp +nmap 689/udp +vatp 690/tcp #Velazquez Application Transfer Protocol +vatp 690/udp #Velazquez Application Transfer Protocol msexch-routing 691/tcp #MS Exchange Routing msexch-routing 691/udp #MS Exchange Routing hyperwave-isp 692/tcp #Hyperwave-ISP hyperwave-isp 692/udp #Hyperwave-ISP -connendp 693/tcp -connendp 693/udp -ha-cluster 694/tcp -ha-cluster 694/udp -ieee-mms-ssl 695/tcp -ieee-mms-ssl 695/udp -rushd 696/tcp -rushd 696/udp -uuidgen 697/tcp -uuidgen 697/udp -olsr 698/tcp -olsr 698/udp -accessnetwork 699/tcp #Access Network -accessnetwork 699/udp #Access Network -epp 700/tcp #Extensible Provisioning Protocol -epp 700/udp #Extensible Provisioning Protocol -lmp 701/tcp #Link Management Protocol (LMP) -lmp 701/udp #Link Management Protocol (LMP) -iris-beep 702/tcp #IRIS over BEEP -iris-beep 702/udp #IRIS over BEEP +connendp 693/tcp +connendp 693/udp +ha-cluster 694/tcp +ha-cluster 694/udp +ieee-mms-ssl 695/tcp +ieee-mms-ssl 695/udp +rushd 696/tcp +rushd 696/udp +uuidgen 697/tcp +uuidgen 697/udp +olsr 698/tcp +olsr 698/udp +accessnetwork 699/tcp #Access Network +accessnetwork 699/udp #Access Network +epp 700/tcp #Extensible Provisioning Protocol +epp 700/udp #Extensible Provisioning Protocol +lmp 701/tcp #Link Management Protocol (LMP) +lmp 701/udp #Link Management Protocol (LMP) +iris-beep 702/tcp #IRIS over BEEP +iris-beep 702/udp #IRIS over BEEP elcsd 704/tcp #errlog copy/server daemon elcsd 704/udp #errlog copy/server daemon -agentx 705/tcp #AgentX -agentx 705/udp #AgentX -silc 706/tcp -silc 706/udp -borland-dsj 707/tcp #Borland DSJ -borland-dsj 707/udp #Borland DSJ +agentx 705/tcp #AgentX +agentx 705/udp #AgentX +silc 706/tcp +silc 706/udp +borland-dsj 707/tcp #Borland DSJ +borland-dsj 707/udp #Borland DSJ entrustmanager 709/tcp #EntrustManager entrustmanager 709/udp #EntrustManager -entrust-ash 710/tcp #Entrust Administration Service Handler -entrust-ash 710/udp #Entrust Administration Service Handler -cisco-tdp 711/tcp #Cisco TDP -cisco-tdp 711/udp #Cisco TDP -tbrpf 712/tcp -tbrpf 712/udp +entrust-ash 710/tcp #Entrust Administration Service Handler +entrust-ash 710/udp #Entrust Administration Service Handler +cisco-tdp 711/tcp #Cisco TDP +cisco-tdp 711/udp #Cisco TDP +tbrpf 712/tcp +tbrpf 712/udp iris-xpc 713/tcp #IRIS over XPC iris-xpc 713/udp #IRIS over XPC iris-xpcs 714/tcp #IRIS over XPCS @@ -1419,49 +1419,49 @@ fcp-udp 810/udp #FCP Datagram itm-mcell-s 828/tcp itm-mcell-s 828/udp pkix-3-ca-ra 829/tcp #PKIX-3 CA/RA -pkix-3-ca-ra 829/udp #PKIX-3 CA/RA -netconf-ssh 830/tcp #NETCONF over SSH -netconf-ssh 830/udp #NETCONF over SSH -netconf-beep 831/tcp #NETCONF over BEEP -netconf-beep 831/udp #NETCONF over BEEP -netconfsoaphttp 832/tcp #NETCONF for SOAP over HTTPS -netconfsoaphttp 832/udp #NETCONF for SOAP over HTTPS -netconfsoapbeep 833/tcp #NETCONF for SOAP over BEEP -netconfsoapbeep 833/udp #NETCONF for SOAP over BEEP -dhcp-failover2 847/tcp #dhcp-failover 2 -dhcp-failover2 847/udp #dhcp-failover 2 -gdoi 848/tcp -gdoi 848/udp -iscsi 860/tcp -iscsi 860/udp -owamp-control 861/tcp -owamp-control 861/udp +pkix-3-ca-ra 829/udp #PKIX-3 CA/RA +netconf-ssh 830/tcp #NETCONF over SSH +netconf-ssh 830/udp #NETCONF over SSH +netconf-beep 831/tcp #NETCONF over BEEP +netconf-beep 831/udp #NETCONF over BEEP +netconfsoaphttp 832/tcp #NETCONF for SOAP over HTTPS +netconfsoaphttp 832/udp #NETCONF for SOAP over HTTPS +netconfsoapbeep 833/tcp #NETCONF for SOAP over BEEP +netconfsoapbeep 833/udp #NETCONF for SOAP over BEEP +dhcp-failover2 847/tcp #dhcp-failover 2 +dhcp-failover2 847/udp #dhcp-failover 2 +gdoi 848/tcp +gdoi 848/udp +iscsi 860/tcp +iscsi 860/udp +owamp-control 861/tcp +owamp-control 861/udp supfilesrv 871/tcp # for SUP rsync 873/tcp rsync 873/udp -iclcnet-locate 886/tcp #ICL coNETion locate server -iclcnet-locate 886/udp #ICL coNETion locate server -iclcnet_svinfo 887/tcp #ICL coNETion server info -iclcnet_svinfo 887/udp #ICL coNETion server info +iclcnet-locate 886/tcp #ICL coNETion locate server +iclcnet-locate 886/udp #ICL coNETion locate server +iclcnet_svinfo 887/tcp #ICL coNETion server info +iclcnet_svinfo 887/udp #ICL coNETion server info accessbuilder 888/tcp accessbuilder 888/udp -omginitialrefs 900/tcp #OMG Initial Refs -omginitialrefs 900/udp #OMG Initial Refs +omginitialrefs 900/tcp #OMG Initial Refs +omginitialrefs 900/udp #OMG Initial Refs swat 901/tcp # samba web configuration tool -smpnameres 901/tcp -smpnameres 901/udp -ideafarm-chat 902/tcp -ideafarm-chat 902/udp -ideafarm-catch 903/tcp -ideafarm-catch 903/udp -kink 910/tcp #Kerberized Internet Negotiation of Keys (KINK) -kink 910/udp #Kerberized Internet Negotiation of Keys (KINK) -xact-backup 911/tcp -xact-backup 911/udp -apex-mesh 912/tcp #APEX relay-relay service -apex-mesh 912/udp #APEX relay-relay service -apex-edge 913/tcp #APEX endpoint-relay service -apex-edge 913/udp #APEX endpoint-relay service +smpnameres 901/tcp +smpnameres 901/udp +ideafarm-chat 902/tcp +ideafarm-chat 902/udp +ideafarm-catch 903/tcp +ideafarm-catch 903/udp +kink 910/tcp #Kerberized Internet Negotiation of Keys (KINK) +kink 910/udp #Kerberized Internet Negotiation of Keys (KINK) +xact-backup 911/tcp +xact-backup 911/udp +apex-mesh 912/tcp #APEX relay-relay service +apex-mesh 912/udp #APEX relay-relay service +apex-edge 913/tcp #APEX endpoint-relay service +apex-edge 913/udp #APEX endpoint-relay service rndc 953/tcp # named's rndc control socket ftps-data 989/tcp # ftp protocol, data, over TLS/SSL ftps-data 989/udp @@ -1491,10 +1491,10 @@ cadlock2 1000/tcp cadlock2 1000/udp surf 1010/tcp surf 1010/udp -exp1 1021/tcp #RFC3692-style Experiment 1 (*) [RFC4727] -exp1 1021/udp #RFC3692-style Experiment 1 (*) [RFC4727] -exp2 1022/tcp #RFC3692-style Experiment 2 (*) [RFC4727] -exp2 1022/udp #RFC3692-style Experiment 2 (*) [RFC4727] +exp1 1021/tcp #RFC3692-style Experiment 1 (*) [RFC4727] +exp1 1021/udp #RFC3692-style Experiment 1 (*) [RFC4727] +exp2 1022/tcp #RFC3692-style Experiment 2 (*) [RFC4727] +exp2 1022/udp #RFC3692-style Experiment 2 (*) [RFC4727] # # REGISTERED PORT NUMBERS # @@ -2401,6 +2401,7 @@ xdsxdm 6558/udp sane-port 6566/tcp #Scanner Access Now Easy (SANE) Control Port sane-port 6566/udp #Scanner Access Now Easy (SANE) Control Port ircd 6667/tcp #Internet Relay Chat (unofficial) +ircs-u 6697/tcp #Internet Relay Chat over TLS/SSL frc-hp 6704/sctp #ForCES HP (High Priority) channel frc-mp 6705/sctp #ForCES MP (Medium Priority) channel frc-lp 6706/sctp #ForCES LP (Low priority) channel diff --git a/gnu/usr.bin/binutils/ld/Makefile b/gnu/usr.bin/binutils/ld/Makefile index 40dd7e2..5058c0a 100644 --- a/gnu/usr.bin/binutils/ld/Makefile +++ b/gnu/usr.bin/binutils/ld/Makefile @@ -48,6 +48,7 @@ CLEANFILES+= ldemul-list.h stringify.sed FILES= ${LDSCRIPTS:S|^|ldscripts/|} FILESDIR= ${SCRIPTDIR} +LINKS= ${BINDIR}/ld ${BINDIR}/ld.bfd HOST= ${TARGET_TUPLE} LIBSEARCHPATH= \"=/lib\":\"=/usr/lib\" diff --git a/include/netdb.h b/include/netdb.h index 102567f..a73d2af 100644 --- a/include/netdb.h +++ b/include/netdb.h @@ -122,7 +122,7 @@ struct protoent { struct addrinfo { int ai_flags; /* AI_PASSIVE, AI_CANONNAME, AI_NUMERICHOST */ - int ai_family; /* PF_xxx */ + int ai_family; /* AF_xxx */ int ai_socktype; /* SOCK_xxx */ int ai_protocol; /* 0 or IPPROTO_xxx for IPv4 and IPv6 */ socklen_t ai_addrlen; /* length of ai_addr */ diff --git a/include/search.h b/include/search.h index 068c82d..4c1f534 100644 --- a/include/search.h +++ b/include/search.h @@ -35,8 +35,9 @@ typedef enum { #ifdef _SEARCH_PRIVATE typedef struct node { - char *key; + void *key; struct node *llink, *rlink; + signed char balance; } node_t; struct que_elem { @@ -46,11 +47,8 @@ struct que_elem { #endif #if __BSD_VISIBLE -struct _ENTRY; struct hsearch_data { - struct _ENTRY *table; - size_t size; - size_t filled; + struct __hsearch *__hsearch; }; #endif diff --git a/lib/Makefile b/lib/Makefile index de0acb3..51532e7 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -91,7 +91,7 @@ SUBDIR= ${SUBDIR_ORDERED} \ libsbuf \ ${_libsdp} \ ${_libsm} \ - ${_libsmb} \ + libsmb \ ${_libsmdb} \ ${_libsmutil} \ libsqlite3 \ @@ -243,7 +243,6 @@ _libypclnt= libypclnt .endif .if ${MACHINE_CPUARCH} == "i386" || ${MACHINE_CPUARCH} == "amd64" -_libsmb= libsmb _libvgl= libvgl _libproc= libproc _librtld_db= librtld_db @@ -263,15 +262,9 @@ _librtld_db= librtld_db .if ${MACHINE_CPUARCH} == "powerpc" _libproc= libproc _librtld_db= librtld_db -_libsmb= libsmb -.endif - -.if ${MACHINE_CPUARCH} == "sparc64" -_libsmb= libsmb .endif .if ${MACHINE_CPUARCH} == "aarch64" || ${MACHINE_CPUARCH} == "arm" -_libsmb= libsmb _libproc= libproc _librtld_db= librtld_db .endif diff --git a/lib/clang/include/Makefile b/lib/clang/include/Makefile index bd68591..2b20cbb 100644 --- a/lib/clang/include/Makefile +++ b/lib/clang/include/Makefile @@ -8,7 +8,7 @@ LLVM_SRCS= ${.CURDIR}/../../../contrib/llvm .PATH: ${LLVM_SRCS}/tools/clang/lib/Headers -INCSDIR=${LIBDIR}/clang/3.7.0/include +INCSDIR=${LIBDIR}/clang/3.7.1/include INCS= __stddef_max_align_t.h \ __wmmintrin_aes.h \ diff --git a/lib/clang/include/clang/Basic/Version.inc b/lib/clang/include/clang/Basic/Version.inc index 6056314..5c5b264 100644 --- a/lib/clang/include/clang/Basic/Version.inc +++ b/lib/clang/include/clang/Basic/Version.inc @@ -1,11 +1,11 @@ /* $FreeBSD$ */ -#define CLANG_VERSION 3.7.0 +#define CLANG_VERSION 3.7.1 #define CLANG_VERSION_MAJOR 3 #define CLANG_VERSION_MINOR 7 -#define CLANG_VERSION_PATCHLEVEL 0 +#define CLANG_VERSION_PATCHLEVEL 1 #define CLANG_VENDOR "FreeBSD " -#define CLANG_VENDOR_SUFFIX " 20150906" +#define CLANG_VENDOR_SUFFIX " 20151225" -#define SVN_REVISION "246257" +#define SVN_REVISION "255217" diff --git a/lib/clang/include/clang/Config/config.h b/lib/clang/include/clang/Config/config.h index fecc361..892bf1b 100644 --- a/lib/clang/include/clang/Config/config.h +++ b/lib/clang/include/clang/Config/config.h @@ -31,7 +31,7 @@ /* Define if we have libxml2 */ /* #undef CLANG_HAVE_LIBXML */ -#define PACKAGE_STRING "LLVM 3.7.0" +#define PACKAGE_STRING "LLVM 3.7.1" /* The LLVM product name and version */ #define BACKEND_PACKAGE_STRING PACKAGE_STRING diff --git a/lib/clang/include/llvm/Config/config.h b/lib/clang/include/llvm/Config/config.h index ef7295c..8342fcb 100644 --- a/lib/clang/include/llvm/Config/config.h +++ b/lib/clang/include/llvm/Config/config.h @@ -505,10 +505,10 @@ #define LLVM_VERSION_MINOR 7 /* Patch version of the LLVM API */ -#define LLVM_VERSION_PATCH 0 +#define LLVM_VERSION_PATCH 1 /* LLVM version string */ -#define LLVM_VERSION_STRING "3.7.0" +#define LLVM_VERSION_STRING "3.7.1" /* The shared library extension */ #define LTDL_SHLIB_EXT ".so" @@ -524,13 +524,13 @@ #define PACKAGE_NAME "LLVM" /* Define to the full name and version of this package. */ -#define PACKAGE_STRING "LLVM 3.7.0" +#define PACKAGE_STRING "LLVM 3.7.1" /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "llvm" /* Define to the version of this package. */ -#define PACKAGE_VERSION "3.7.0" +#define PACKAGE_VERSION "3.7.1" /* Define as the return type of signal handlers (`int' or `void'). */ #define RETSIGTYPE void diff --git a/lib/clang/include/llvm/Config/llvm-config.h b/lib/clang/include/llvm/Config/llvm-config.h index 51a5918..36bcda7 100644 --- a/lib/clang/include/llvm/Config/llvm-config.h +++ b/lib/clang/include/llvm/Config/llvm-config.h @@ -98,9 +98,9 @@ #define LLVM_VERSION_MINOR 7 /* Patch version of the LLVM API */ -#define LLVM_VERSION_PATCH 0 +#define LLVM_VERSION_PATCH 1 /* LLVM version string */ -#define LLVM_VERSION_STRING "3.7.0" +#define LLVM_VERSION_STRING "3.7.1" #endif diff --git a/lib/libc/Makefile b/lib/libc/Makefile index 2f8865c..e9f14f4 100644 --- a/lib/libc/Makefile +++ b/lib/libc/Makefile @@ -26,6 +26,7 @@ LIBC_ARCH=${MACHINE_CPUARCH} LIB=c SHLIB_MAJOR= 7 SHLIB_LDSCRIPT=libc.ldscript +SHLIB_LDSCRIPT_LINKS=libxnet.so WARNS?= 2 CFLAGS+=-I${LIBC_SRCTOP}/include -I${LIBC_SRCTOP}/../../include CFLAGS+=-I${LIBC_SRCTOP}/${LIBC_ARCH} diff --git a/lib/libc/arm/sys/__vdso_gettc.c b/lib/libc/arm/sys/__vdso_gettc.c index d75d866..1f43e72 100644 --- a/lib/libc/arm/sys/__vdso_gettc.c +++ b/lib/libc/arm/sys/__vdso_gettc.c @@ -34,8 +34,10 @@ __FBSDID("$FreeBSD$"); #include <sys/time.h> #include <sys/vdso.h> #include <machine/cpufunc.h> +#include <machine/acle-compat.h> #include "libc_private.h" +#if __ARM_ARCH >= 6 static inline uint64_t cp15_cntvct_get(void) { @@ -53,6 +55,7 @@ cp15_cntpct_get(void) __asm __volatile("mrrc\tp15, 0, %Q0, %R0, c14" : "=r" (reg)); return (reg); } +#endif #pragma weak __vdso_gettc u_int @@ -60,6 +63,7 @@ __vdso_gettc(const struct vdso_timehands *th) { uint64_t val; +#if __ARM_ARCH >= 6 /* * Userspace gettimeofday() is only enabled on ARMv7 CPUs, but * libc is compiled for ARMv6. Due to clang issues, .arch @@ -67,6 +71,9 @@ __vdso_gettc(const struct vdso_timehands *th) */ __asm __volatile(".word\t0xf57ff06f" : : : "memory"); /* isb */ val = th->th_physical == 0 ? cp15_cntvct_get() : cp15_cntpct_get(); +#else + val = 0; +#endif return (val); } diff --git a/lib/libc/gen/getpeereid.c b/lib/libc/gen/getpeereid.c index 5ecb243..cedaee6 100644 --- a/lib/libc/gen/getpeereid.c +++ b/lib/libc/gen/getpeereid.c @@ -27,6 +27,7 @@ #include <sys/cdefs.h> __FBSDID("$FreeBSD$"); +#include "namespace.h" #include <sys/param.h> #include <sys/socket.h> #include <sys/ucred.h> @@ -34,6 +35,7 @@ __FBSDID("$FreeBSD$"); #include <errno.h> #include <unistd.h> +#include "un-namespace.h" int getpeereid(int s, uid_t *euid, gid_t *egid) @@ -43,7 +45,7 @@ getpeereid(int s, uid_t *euid, gid_t *egid) int error; xuclen = sizeof(xuc); - error = getsockopt(s, 0, LOCAL_PEERCRED, &xuc, &xuclen); + error = _getsockopt(s, 0, LOCAL_PEERCRED, &xuc, &xuclen); if (error != 0) return (error); if (xuc.cr_version != XUCRED_VERSION) diff --git a/lib/libc/gen/lockf.c b/lib/libc/gen/lockf.c index 2c567ba..c64a347 100644 --- a/lib/libc/gen/lockf.c +++ b/lib/libc/gen/lockf.c @@ -36,6 +36,7 @@ __FBSDID("$FreeBSD$"); #include <fcntl.h> #include <unistd.h> #include "un-namespace.h" +#include "libc_private.h" int lockf(int filedes, int function, off_t size) @@ -62,9 +63,12 @@ lockf(int filedes, int function, off_t size) break; case F_TEST: fl.l_type = F_WRLCK; - if (_fcntl(filedes, F_GETLK, &fl) == -1) + if (((int (*)(int, int, ...)) + __libc_interposing[INTERPOS_fcntl])(filedes, F_GETLK, &fl) + == -1) return (-1); - if (fl.l_type == F_UNLCK || (fl.l_sysid == 0 && fl.l_pid == getpid())) + if (fl.l_type == F_UNLCK || (fl.l_sysid == 0 && + fl.l_pid == getpid())) return (0); errno = EAGAIN; return (-1); @@ -75,5 +79,6 @@ lockf(int filedes, int function, off_t size) /* NOTREACHED */ } - return (_fcntl(filedes, cmd, &fl)); + return (((int (*)(int, int, ...)) + __libc_interposing[INTERPOS_fcntl])(filedes, cmd, &fl)); } diff --git a/lib/libc/gen/nlist.c b/lib/libc/gen/nlist.c index e93f89b..80784dd 100644 --- a/lib/libc/gen/nlist.c +++ b/lib/libc/gen/nlist.c @@ -47,8 +47,8 @@ __FBSDID("$FreeBSD$"); #include <unistd.h> #include "un-namespace.h" -/* There is no a.out support on arm64 */ -#ifndef __aarch64__ +/* i386 is the only current FreeBSD architecture that used a.out format. */ +#ifdef __i386__ #define _NLIST_DO_AOUT #endif #define _NLIST_DO_ELF diff --git a/lib/libc/gen/sysconf.c b/lib/libc/gen/sysconf.c index be3d4af..71f2321 100644 --- a/lib/libc/gen/sysconf.c +++ b/lib/libc/gen/sysconf.c @@ -36,6 +36,7 @@ static char sccsid[] = "@(#)sysconf.c 8.2 (Berkeley) 3/20/94"; #include <sys/cdefs.h> __FBSDID("$FreeBSD$"); +#include "namespace.h" #include <sys/param.h> #include <sys/time.h> #include <sys/sysctl.h> @@ -49,6 +50,7 @@ __FBSDID("$FreeBSD$"); #include <pthread.h> /* we just need the limits */ #include <time.h> #include <unistd.h> +#include "un-namespace.h" #include "../stdlib/atexit.h" #include "tzfile.h" /* from ../../../contrib/tzcode/stdtime */ @@ -575,10 +577,10 @@ yesno: case _SC_IPV6: #if _POSIX_IPV6 == 0 sverrno = errno; - value = socket(PF_INET6, SOCK_DGRAM, 0); + value = _socket(PF_INET6, SOCK_DGRAM, 0); errno = sverrno; if (value >= 0) { - close(value); + _close(value); return (200112L); } else return (0); diff --git a/lib/libc/iconv/citrus_mmap.c b/lib/libc/iconv/citrus_mmap.c index f8e96d1..83dd70b 100644 --- a/lib/libc/iconv/citrus_mmap.c +++ b/lib/libc/iconv/citrus_mmap.c @@ -27,6 +27,7 @@ * SUCH DAMAGE. */ +#include "namespace.h" #include <sys/cdefs.h> #include <sys/mman.h> #include <sys/types.h> @@ -40,6 +41,7 @@ #include <stdlib.h> #include <string.h> #include <unistd.h> +#include "un-namespace.h" #include "citrus_namespace.h" #include "citrus_region.h" @@ -57,10 +59,10 @@ _citrus_map_file(struct _citrus_region * __restrict r, _region_init(r, NULL, 0); - if ((fd = open(path, O_RDONLY | O_CLOEXEC)) == -1) + if ((fd = _open(path, O_RDONLY | O_CLOEXEC)) == -1) return (errno); - if (fstat(fd, &st) == -1) { + if (_fstat(fd, &st) == -1) { ret = errno; goto error; } @@ -78,7 +80,7 @@ _citrus_map_file(struct _citrus_region * __restrict r, _region_init(r, head, (size_t)st.st_size); error: - (void)close(fd); + (void)_close(fd); return (ret); } diff --git a/lib/libc/net/getaddrinfo.3 b/lib/libc/net/getaddrinfo.3 index 69601e6..7380428 100644 --- a/lib/libc/net/getaddrinfo.3 +++ b/lib/libc/net/getaddrinfo.3 @@ -18,7 +18,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 19, 2015 +.Dd December 21, 2015 .Dt GETADDRINFO 3 .Os .Sh NAME @@ -79,7 +79,7 @@ as defined by .Bd -literal struct addrinfo { int ai_flags; /* input flags */ - int ai_family; /* protocol family for socket */ + int ai_family; /* address family for socket */ int ai_socktype; /* socket type */ int ai_protocol; /* protocol for socket */ socklen_t ai_addrlen; /* length of socket-address */ @@ -95,12 +95,12 @@ The caller can supply the following structure elements in .Fa hints : .Bl -tag -width "ai_socktypeXX" .It Fa ai_family -The protocol family that should be used. +The address family that should be used. When .Fa ai_family is set to -.Dv PF_UNSPEC , -it means the caller will accept any protocol family supported by the +.Dv AF_UNSPEC , +it means the caller will accept any address family supported by the operating system. .It Fa ai_socktype Denotes the type of socket that is wanted: @@ -268,7 +268,7 @@ behaves as if the caller provided a with .Fa ai_family set to -.Dv PF_UNSPEC +.Dv AF_UNSPEC and all other elements set to zero or .Dv NULL . .Pp @@ -380,7 +380,7 @@ int s; const char *cause = NULL; memset(&hints, 0, sizeof(hints)); -hints.ai_family = PF_UNSPEC; +hints.ai_family = AF_UNSPEC; hints.ai_socktype = SOCK_STREAM; error = getaddrinfo("www.kame.net", "http", &hints, &res0); if (error) { @@ -423,7 +423,7 @@ int nsock; const char *cause = NULL; memset(&hints, 0, sizeof(hints)); -hints.ai_family = PF_UNSPEC; +hints.ai_family = AF_UNSPEC; hints.ai_socktype = SOCK_STREAM; hints.ai_flags = AI_PASSIVE; error = getaddrinfo(NULL, "http", &hints, &res0); diff --git a/lib/libc/net/getaddrinfo.c b/lib/libc/net/getaddrinfo.c index ed1a388..e2eb1a0 100644 --- a/lib/libc/net/getaddrinfo.c +++ b/lib/libc/net/getaddrinfo.c @@ -2164,7 +2164,11 @@ getanswer(const querybuf *answer, int anslen, const char *qname, int qtype, return sentinel.ai_next; } - RES_SET_H_ERRNO(res, NO_RECOVERY); + /* + * We could have walked a CNAME chain, but the ultimate target + * may not have what we looked for. + */ + RES_SET_H_ERRNO(res, ntohs(hp->ancount) > 0 ? NO_DATA : NO_RECOVERY); return NULL; } @@ -2341,6 +2345,7 @@ _dns_getaddrinfo(void *rv, void *cb_data, va_list ap) if (sentinel.ai_next == NULL) switch (res->res_h_errno) { case HOST_NOT_FOUND: + case NO_DATA: return NS_NOTFOUND; case TRY_AGAIN: return NS_TRYAGAIN; diff --git a/lib/libc/net/gethostbynis.c b/lib/libc/net/gethostbynis.c index c0d5177..6bafdb1 100644 --- a/lib/libc/net/gethostbynis.c +++ b/lib/libc/net/gethostbynis.c @@ -198,61 +198,6 @@ _gethostbynisaddr_r(const void *addr, socklen_t len, int af, } #endif /* YP */ -/* XXX _gethostbynisname/_gethostbynisaddr only used by getipnodeby*() */ -struct hostent * -_gethostbynisname(const char *name, int af) -{ -#ifdef YP - struct hostent *he; - struct hostent_data *hed; - u_long oresopt; - int error; - res_state statp; - - statp = __res_state(); - if ((he = __hostent_init()) == NULL || - (hed = __hostent_data_init()) == NULL) { - RES_SET_H_ERRNO(statp, NETDB_INTERNAL); - return (NULL); - } - - oresopt = statp->options; - statp->options &= ~RES_USE_INET6; - error = _gethostbynisname_r(name, af, he, hed); - statp->options = oresopt; - return (error == 0) ? he : NULL; -#else - return (NULL); -#endif -} - -struct hostent * -_gethostbynisaddr(const void *addr, socklen_t len, int af) -{ -#ifdef YP - struct hostent *he; - struct hostent_data *hed; - u_long oresopt; - int error; - res_state statp; - - statp = __res_state(); - if ((he = __hostent_init()) == NULL || - (hed = __hostent_data_init()) == NULL) { - RES_SET_H_ERRNO(statp, NETDB_INTERNAL); - return (NULL); - } - - oresopt = statp->options; - statp->options &= ~RES_USE_INET6; - error = _gethostbynisaddr_r(addr, len, af, he, hed); - statp->options = oresopt; - return (error == 0) ? he : NULL; -#else - return (NULL); -#endif -} - int _nis_gethostbyname(void *rval, void *cb_data, va_list ap) { diff --git a/lib/libc/net/map_v4v6.c b/lib/libc/net/map_v4v6.c index 09b035b..327d6b4 100644 --- a/lib/libc/net/map_v4v6.c +++ b/lib/libc/net/map_v4v6.c @@ -78,19 +78,11 @@ typedef union { void _map_v4v6_address(const char *src, char *dst) { - u_char *p = (u_char *)dst; - char tmp[NS_INADDRSZ]; - int i; - - /* Stash a temporary copy so our caller can update in place. */ - memcpy(tmp, src, NS_INADDRSZ); + /* Our caller may update in place. */ + memmove(&dst[12], src, NS_INADDRSZ); /* Mark this ipv6 addr as a mapped ipv4. */ - for (i = 0; i < 10; i++) - *p++ = 0x00; - *p++ = 0xff; - *p++ = 0xff; - /* Retrieve the saved copy and we're done. */ - memcpy((void*)p, tmp, NS_INADDRSZ); + memset(&dst[10], 0xff, 2); + memset(&dst[0], 0, 10); } void diff --git a/lib/libc/net/name6.c b/lib/libc/net/name6.c index 89effe6..51e2da15 100644 --- a/lib/libc/net/name6.c +++ b/lib/libc/net/name6.c @@ -794,10 +794,9 @@ match_addrselectpolicy(struct sockaddr *addr, struct policyhead *head) memset(&key, 0, sizeof(key)); key.sin6_family = AF_INET6; key.sin6_len = sizeof(key); - key.sin6_addr.s6_addr[10] = 0xff; - key.sin6_addr.s6_addr[11] = 0xff; - memcpy(&key.sin6_addr.s6_addr[12], - &((struct sockaddr_in *)addr)->sin_addr, 4); + _map_v4v6_address( + (char *)&((struct sockaddr_in *)addr)->sin_addr, + (char *)&key.sin6_addr); break; default: return(NULL); diff --git a/lib/libc/net/netdb_private.h b/lib/libc/net/netdb_private.h index 0eedb3c..8ab2247 100644 --- a/lib/libc/net/netdb_private.h +++ b/lib/libc/net/netdb_private.h @@ -133,8 +133,6 @@ void _endhostdnsent(void); void _endhosthtent(struct hostent_data *); void _endnetdnsent(void); void _endnethtent(struct netent_data *); -struct hostent *_gethostbynisaddr(const void *, socklen_t, int); -struct hostent *_gethostbynisname(const char *, int); void _map_v4v6_address(const char *, char *); void _map_v4v6_hostent(struct hostent *, char **, char *); void _sethostdnsent(int); diff --git a/lib/libc/net/rcmdsh.c b/lib/libc/net/rcmdsh.c index f30ad14..13278df 100644 --- a/lib/libc/net/rcmdsh.c +++ b/lib/libc/net/rcmdsh.c @@ -36,6 +36,7 @@ #include <sys/cdefs.h> __FBSDID("$FreeBSD$"); +#include "namespace.h" #include <sys/types.h> #include <sys/socket.h> #include <sys/wait.h> @@ -48,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include <stdio.h> #include <string.h> #include <unistd.h> +#include "un-namespace.h" /* * This is a replacement rcmd() function that uses the rsh(1) @@ -99,7 +101,7 @@ rcmdsh(char **ahost, int rport, const char *locuser, const char *remuser, } /* Get a socketpair we'll use for stdin and stdout. */ - if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, sp) == -1) { + if (_socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, sp) == -1) { perror("rcmdsh: socketpair"); return (-1); } @@ -112,8 +114,8 @@ rcmdsh(char **ahost, int rport, const char *locuser, const char *remuser, /* * Child. We use sp[1] to be stdin/stdout, and close sp[0]. */ - (void)close(sp[0]); - if (dup2(sp[1], 0) == -1 || dup2(0, 1) == -1) { + (void)_close(sp[0]); + if (_dup2(sp[1], 0) == -1 || _dup2(0, 1) == -1) { perror("rcmdsh: dup2 failed"); _exit(255); } @@ -156,9 +158,9 @@ rcmdsh(char **ahost, int rport, const char *locuser, const char *remuser, _exit(255); } else { /* Parent. close sp[1], return sp[0]. */ - (void)close(sp[1]); + (void)_close(sp[1]); /* Reap child. */ - (void)wait(NULL); + (void)_waitpid(cpid, NULL, 0); return (sp[0]); } /* NOTREACHED */ diff --git a/lib/libc/stdio/findfp.c b/lib/libc/stdio/findfp.c index b8bb5af..6a68958 100644 --- a/lib/libc/stdio/findfp.c +++ b/lib/libc/stdio/findfp.c @@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$"); #include <unistd.h> #include <stdio.h> #include <stdlib.h> +#include <stdint.h> #include <string.h> #include <spinlock.h> @@ -96,11 +97,22 @@ moreglue(int n) struct glue *g; static FILE empty = { ._fl_mutex = PTHREAD_MUTEX_INITIALIZER }; FILE *p; + size_t align; - g = (struct glue *)malloc(sizeof(*g) + ALIGNBYTES + n * sizeof(FILE)); + /* + * FILE has a mbstate_t variable. This variable tries to be int64_t + * aligned through its definition. int64_t may be larger than void *, + * which is the size traditionally used for ALIGNBYTES. So, use our own + * rounding instead of the MI ALIGN macros. If for some reason + * ALIGNBYTES is larger than int64_t, respect that too. There appears to + * be no portable way to ask for FILE's alignment requirements other + * than just knowing here. + */ + align = MAX(ALIGNBYTES, sizeof(int64_t)); + g = (struct glue *)malloc(sizeof(*g) + align + n * sizeof(FILE)); if (g == NULL) return (NULL); - p = (FILE *)ALIGN(g + 1); + p = (FILE *)roundup((uintptr_t)(g + 1), align); g->next = NULL; g->niobs = n; g->iobs = p; diff --git a/lib/libc/stdlib/Makefile.inc b/lib/libc/stdlib/Makefile.inc index 7cee03a..3c6ba17 100644 --- a/lib/libc/stdlib/Makefile.inc +++ b/lib/libc/stdlib/Makefile.inc @@ -6,7 +6,8 @@ MISRCS+=_Exit.c a64l.c abort.c abs.c atexit.c atof.c atoi.c atol.c atoll.c \ bsearch.c div.c exit.c getenv.c getopt.c getopt_long.c \ - getsubopt.c hcreate.c heapsort.c heapsort_b.c imaxabs.c imaxdiv.c \ + getsubopt.c hcreate.c hcreate_r.c hdestroy_r.c heapsort.c heapsort_b.c \ + hsearch_r.c imaxabs.c imaxdiv.c \ insque.c l64a.c labs.c ldiv.c llabs.c lldiv.c lsearch.c \ merge.c mergesort_b.c ptsname.c qsort.c qsort_r.c quick_exit.c \ radixsort.c rand.c \ diff --git a/lib/libc/stdlib/hcreate.3 b/lib/libc/stdlib/hcreate.3 index 2161f92..5709157 100644 --- a/lib/libc/stdlib/hcreate.3 +++ b/lib/libc/stdlib/hcreate.3 @@ -28,7 +28,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 21, 2014 +.Dd December 26, 2015 .Dt HCREATE 3 .Os .Sh NAME @@ -76,8 +76,8 @@ The .Fa nel argument is an estimate of the maximum number of entries that the table should contain. -This number may be adjusted upward by the -algorithm in order to obtain certain mathematically favorable circumstances. +As this implementation resizes the hash table dynamically, +this argument is ignored. .Pp The .Fn hdestroy @@ -274,8 +274,6 @@ functions will fail if: .Bl -tag -width Er .It Bq Er ENOMEM Insufficient memory is available. -.It Bq Er EINVAL -A table already exists. .El .Pp The diff --git a/lib/libc/stdlib/hcreate.c b/lib/libc/stdlib/hcreate.c index b3be9b4..2d5d943 100644 --- a/lib/libc/stdlib/hcreate.c +++ b/lib/libc/stdlib/hcreate.c @@ -1,9 +1,6 @@ -/* $NetBSD: hcreate.c,v 1.7 2011/09/14 23:33:51 christos Exp $ */ - -/* - * Copyright (c) 2001 Christopher G. Demetriou - * All rights reserved. - * +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,207 +9,65 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * 3. The name of the author may not be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES - * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. - * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, - * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF - * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * - * <<Id: LICENSE,v 1.2 2000/06/14 15:57:33 cgd Exp>> - */ - -/* - * hcreate() / hsearch() / hdestroy() * - * SysV/XPG4 hash table functions. - * - * Implementation done based on NetBSD manual page and Solaris manual page, - * plus my own personal experience about how they're supposed to work. - * - * I tried to look at Knuth (as cited by the Solaris manual page), but - * nobody had a copy in the office, so... + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. */ #include <sys/cdefs.h> -#if 0 -#if defined(LIBC_SCCS) && !defined(lint) -__RCSID("$NetBSD: hcreate.c,v 1.8 2011/09/17 16:54:39 christos Exp $"); -#endif /* LIBC_SCCS and not lint */ -#endif __FBSDID("$FreeBSD$"); -#include <sys/types.h> -#include <sys/queue.h> -#include <errno.h> #include <search.h> -#include <stdlib.h> -#include <string.h> +#include <stdbool.h> +#include <stddef.h> /* - * DO NOT MAKE THIS STRUCTURE LARGER THAN 32 BYTES (4 ptrs on 64-bit - * ptr machine) without adjusting MAX_BUCKETS_LG2 below. + * Thread unsafe interface: use a single process-wide hash table and + * forward calls to *_r() functions. */ -struct internal_entry { - SLIST_ENTRY(internal_entry) link; - ENTRY ent; -}; -SLIST_HEAD(internal_head, internal_entry); - -#define MIN_BUCKETS_LG2 4 -#define MIN_BUCKETS (1 << MIN_BUCKETS_LG2) -/* - * max * sizeof internal_entry must fit into size_t. - * assumes internal_entry is <= 32 (2^5) bytes. - */ -#define MAX_BUCKETS_LG2 (sizeof (size_t) * 8 - 1 - 5) -#define MAX_BUCKETS ((size_t)1 << MAX_BUCKETS_LG2) - -/* Default hash function, from db/hash/hash_func.c */ -extern u_int32_t (*__default_hash)(const void *, size_t); - -static struct hsearch_data htable; +static struct hsearch_data global_hashtable; +static bool global_hashtable_initialized = false; int hcreate(size_t nel) { - /* Make sure this is not called when a table already exists. */ - if (htable.table != NULL) { - errno = EINVAL; - return 0; - } - return hcreate_r(nel, &htable); -} - -int -hcreate_r(size_t nel, struct hsearch_data *head) -{ - struct internal_head *table; - size_t idx; - unsigned int p2; - void *p; - - /* If nel is too small, make it min sized. */ - if (nel < MIN_BUCKETS) - nel = MIN_BUCKETS; - - /* If it is too large, cap it. */ - if (nel > MAX_BUCKETS) - nel = MAX_BUCKETS; - - /* If it is not a power of two in size, round up. */ - if ((nel & (nel - 1)) != 0) { - for (p2 = 0; nel != 0; p2++) - nel >>= 1; - nel = 1 << p2; - } - - /* Allocate the table. */ - head->size = nel; - head->filled = 0; - p = malloc(nel * sizeof table[0]); - if (p == NULL) { - errno = ENOMEM; - return 0; - } - head->table = p; - table = p; - - /* Initialize it. */ - for (idx = 0; idx < nel; idx++) - SLIST_INIT(&table[idx]); - - return 1; + return (1); } void hdestroy(void) { - hdestroy_r(&htable); -} - -void -hdestroy_r(struct hsearch_data *head) -{ - struct internal_entry *ie; - size_t idx; - void *p; - struct internal_head *table; - - if (head == NULL) - return; - - p = head->table; - head->table = NULL; - table = p; - for (idx = 0; idx < head->size; idx++) { - while (!SLIST_EMPTY(&table[idx])) { - ie = SLIST_FIRST(&table[idx]); - SLIST_REMOVE_HEAD(&table[idx], link); - free(ie); - } + /* Destroy global hash table if present. */ + if (global_hashtable_initialized) { + hdestroy_r(&global_hashtable); + global_hashtable_initialized = false; } - free(table); } ENTRY * hsearch(ENTRY item, ACTION action) { - ENTRY *ep; - (void)hsearch_r(item, action, &ep, &htable); - return ep; -} - -int -hsearch_r(ENTRY item, ACTION action, ENTRY **itemp, struct hsearch_data *head) -{ - struct internal_head *table, *chain; - struct internal_entry *ie; - uint32_t hashval; - size_t len; - void *p; - - p = head->table; - table = p; + ENTRY *retval; - len = strlen(item.key); - hashval = (*__default_hash)(item.key, len); - - chain = &table[hashval & (head->size - 1)]; - ie = SLIST_FIRST(chain); - while (ie != NULL) { - if (strcmp(ie->ent.key, item.key) == 0) - break; - ie = SLIST_NEXT(ie, link); - } - - if (ie != NULL) { - *itemp = &ie->ent; - return 1; - } else if (action == FIND) { - *itemp = NULL; - errno = ESRCH; - return 1; + /* Create global hash table if needed. */ + if (!global_hashtable_initialized) { + if (hcreate_r(0, &global_hashtable) == 0) + return (NULL); + global_hashtable_initialized = true; } - - ie = malloc(sizeof *ie); - if (ie == NULL) - return 0; - ie->ent.key = item.key; - ie->ent.data = item.data; - - SLIST_INSERT_HEAD(chain, ie, link); - *itemp = &ie->ent; - head->filled++; - return 1; + if (hsearch_r(item, action, &retval, &global_hashtable) == 0) + return (NULL); + return (retval); } diff --git a/lib/libc/stdlib/hcreate_r.c b/lib/libc/stdlib/hcreate_r.c new file mode 100644 index 0000000..83e322a --- /dev/null +++ b/lib/libc/stdlib/hcreate_r.c @@ -0,0 +1,63 @@ +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <search.h> +#include <stdlib.h> + +#include "hsearch.h" + +int +hcreate_r(size_t nel, struct hsearch_data *htab) +{ + struct __hsearch *hsearch; + + /* + * Allocate a hash table object. Ignore the provided hint and start + * off with a table of sixteen entries. In most cases this hint is + * just a wild guess. Resizing the table dynamically if the use + * increases a threshold does not affect the worst-case running time. + */ + hsearch = malloc(sizeof(*hsearch)); + if (hsearch == NULL) + return 0; + hsearch->entries = calloc(16, sizeof(ENTRY)); + if (hsearch->entries == NULL) { + free(hsearch); + return 0; + } + + /* + * Pick a random initialization for the FNV-1a hashing. This makes it + * hard to come up with a fixed set of keys to force hash collisions. + */ + arc4random_buf(&hsearch->offset_basis, sizeof(hsearch->offset_basis)); + hsearch->index_mask = 0xf; + hsearch->entries_used = 0; + htab->__hsearch = hsearch; + return 1; +} diff --git a/lib/libc/stdlib/hdestroy_r.c b/lib/libc/stdlib/hdestroy_r.c new file mode 100644 index 0000000..890bd085 --- /dev/null +++ b/lib/libc/stdlib/hdestroy_r.c @@ -0,0 +1,43 @@ +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <search.h> +#include <stdlib.h> + +#include "hsearch.h" + +void +hdestroy_r(struct hsearch_data *htab) +{ + struct __hsearch *hsearch; + + /* Free hash table object and its entries. */ + hsearch = htab->__hsearch; + free(hsearch->entries); + free(hsearch); +} diff --git a/lib/libc/stdlib/hsearch.h b/lib/libc/stdlib/hsearch.h new file mode 100644 index 0000000..649933d --- /dev/null +++ b/lib/libc/stdlib/hsearch.h @@ -0,0 +1,40 @@ +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef HSEARCH_H +#define HSEARCH_H + +#include <search.h> + +struct __hsearch { + size_t offset_basis; /* Initial value for FNV-1a hashing. */ + size_t index_mask; /* Bitmask for indexing the table. */ + size_t entries_used; /* Number of entries currently used. */ + ENTRY *entries; /* Hash table entries. */ +}; + +#endif diff --git a/lib/libc/stdlib/hsearch_r.c b/lib/libc/stdlib/hsearch_r.c new file mode 100644 index 0000000..2fb5991 --- /dev/null +++ b/lib/libc/stdlib/hsearch_r.c @@ -0,0 +1,150 @@ +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <errno.h> +#include <limits.h> +#include <search.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include "hsearch.h" + +/* + * Look up an unused entry in the hash table for a given hash. For this + * implementation we use quadratic probing. Quadratic probing has the + * advantage of preventing primary clustering. + */ +static ENTRY * +hsearch_lookup_free(struct __hsearch *hsearch, size_t hash) +{ + size_t index, i; + + for (index = hash, i = 0;; index += ++i) { + ENTRY *entry = &hsearch->entries[index & hsearch->index_mask]; + if (entry->key == NULL) + return (entry); + } +} + +/* + * Computes an FNV-1a hash of the key. Depending on the pointer size, this + * either uses the 32- or 64-bit FNV prime. + */ +static size_t +hsearch_hash(size_t offset_basis, const char *str) +{ + size_t hash; + + hash = offset_basis; + while (*str != '\0') { + hash ^= (uint8_t)*str++; + if (sizeof(size_t) * CHAR_BIT <= 32) + hash *= UINT32_C(16777619); + else + hash *= UINT64_C(1099511628211); + } + return (hash); +} + +int +hsearch_r(ENTRY item, ACTION action, ENTRY **retval, struct hsearch_data *htab) +{ + struct __hsearch *hsearch; + ENTRY *entry, *old_entries, *new_entries; + size_t hash, index, i, old_hash, old_count, new_count; + + hsearch = htab->__hsearch; + hash = hsearch_hash(hsearch->offset_basis, item.key); + + /* + * Search the hash table for an existing entry for this key. + * Stop searching if we run into an unused hash table entry. + */ + for (index = hash, i = 0;; index += ++i) { + entry = &hsearch->entries[index & hsearch->index_mask]; + if (entry->key == NULL) + break; + if (strcmp(entry->key, item.key) == 0) { + *retval = entry; + return (1); + } + } + + /* Only perform the insertion if action is set to ENTER. */ + if (action == FIND) { + errno = ESRCH; + return (0); + } + + if (hsearch->entries_used * 2 >= hsearch->index_mask) { + /* Preserve the old hash table entries. */ + old_count = hsearch->index_mask + 1; + old_entries = hsearch->entries; + + /* + * Allocate and install a new table if insertion would + * yield a hash table that is more than 50% used. By + * using 50% as a threshold, a lookup will only take up + * to two steps on average. + */ + new_count = (hsearch->index_mask + 1) * 2; + new_entries = calloc(new_count, sizeof(ENTRY)); + if (new_entries == NULL) + return (0); + hsearch->entries = new_entries; + hsearch->index_mask = new_count - 1; + + /* Copy over the entries from the old table to the new table. */ + for (i = 0; i < old_count; ++i) { + entry = &old_entries[i]; + if (entry->key != NULL) { + old_hash = hsearch_hash(hsearch->offset_basis, + entry->key); + *hsearch_lookup_free(hsearch, old_hash) = + *entry; + } + } + + /* Destroy the old hash table entries. */ + free(old_entries); + + /* + * Perform a new lookup for a free table entry, so that + * we insert the entry into the new hash table. + */ + hsearch = htab->__hsearch; + entry = hsearch_lookup_free(hsearch, hash); + } + + /* Insert the new entry into the hash table. */ + *entry = item; + ++hsearch->entries_used; + *retval = entry; + return (1); +} diff --git a/lib/libc/stdlib/tdelete.c b/lib/libc/stdlib/tdelete.c index bef187e..7799f35 100644 --- a/lib/libc/stdlib/tdelete.c +++ b/lib/libc/stdlib/tdelete.c @@ -1,72 +1,213 @@ -/* $NetBSD: tdelete.c,v 1.2 1999/09/16 11:45:37 lukem Exp $ */ - -/* - * Tree search generalized from Knuth (6.2.2) Algorithm T just like - * the AT&T man page says. +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ * - * The node_t structure is for internal use only, lint doesn't grok it. + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. * - * Written by reading the System V Interface Definition, not the code. - * - * Totally public domain. + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. */ #include <sys/cdefs.h> -#if 0 -#if defined(LIBC_SCCS) && !defined(lint) -__RCSID("$NetBSD: tdelete.c,v 1.6 2012/06/25 22:32:45 abs Exp $"); -#endif /* LIBC_SCCS and not lint */ -#endif __FBSDID("$FreeBSD$"); -#define _SEARCH_PRIVATE +#define _SEARCH_PRIVATE #include <search.h> +#include <stdbool.h> #include <stdlib.h> +#include "tsearch_path.h" /* - * find a node with given key - * - * vkey: key to be found - * vrootp: address of the root of the tree - * compar: function to carry out node comparisons - */ + * Makes a step to the left along the binary search tree. This step is + * also saved, so it can be replayed while rebalancing. +*/ +#define GO_LEFT() do { \ + if ((*leaf)->balance == 0 || \ + ((*leaf)->balance < 0 && (*leaf)->rlink->balance == 0)) { \ + /* \ + * If we reach a node that is balanced, or has a child \ + * in the opposite direction that is balanced, we know \ + * that we won't need to perform any rotations above \ + * this point. In this case rotations are always \ + * capable of keeping the subtree in balance. Make \ + * this the base node and reset the path. \ + */ \ + base = leaf; \ + path_init(&path); \ + } \ + path_taking_left(&path); \ + leaf = &(*leaf)->llink; \ +} while (0) + +/* Makes a step to the right along the binary search tree. */ +#define GO_RIGHT() do { \ + if ((*leaf)->balance == 0 || \ + ((*leaf)->balance > 0 && (*leaf)->llink->balance == 0)) { \ + base = leaf; \ + path_init(&path); \ + } \ + result = &(*leaf)->key; \ + path_taking_right(&path); \ + leaf = &(*leaf)->rlink; \ +} while (0) + void * -tdelete(const void * __restrict vkey, void ** __restrict vrootp, +tdelete(const void *restrict key, void **restrict rootp, int (*compar)(const void *, const void *)) { - node_t **rootp = (node_t **)vrootp; - node_t *p, *q, *r; + struct path path; + node_t *root, **base, **leaf, *old, **n, *x, *y, *z; + void *result; int cmp; - if (rootp == NULL || (p = *rootp) == NULL) - return NULL; + /* POSIX requires that tdelete() returns NULL if rootp is NULL. */ + if (rootp == NULL) + return (NULL); + root = *rootp; + + /* + * Find the leaf that needs to be removed. Return if we cannot + * find an existing entry. Keep track of the path that is taken + * to get to the node, as we will need it to adjust the + * balances. + */ + result = (void *)1; + path_init(&path); + base = &root; + leaf = &root; + for (;;) { + if (*leaf == NULL) + return (NULL); + cmp = compar(key, (*leaf)->key); + if (cmp < 0) { + result = &(*leaf)->key; + GO_LEFT(); + } else if (cmp > 0) { + result = &(*leaf)->key; + GO_RIGHT(); + } else { + break; + } + } - while ((cmp = (*compar)(vkey, (*rootp)->key)) != 0) { - p = *rootp; - rootp = (cmp < 0) ? - &(*rootp)->llink : /* follow llink branch */ - &(*rootp)->rlink; /* follow rlink branch */ - if (*rootp == NULL) - return NULL; /* key not found */ + /* Found a matching key in the tree. Remove the node. */ + if ((*leaf)->llink == NULL) { + /* Node has no left children. Replace by its right subtree. */ + old = *leaf; + *leaf = old->rlink; + free(old); + } else { + /* + * Node has left children. Replace this node's key by + * its predecessor's and remove that node instead. + */ + void **keyp = &(*leaf)->key; + GO_LEFT(); + while ((*leaf)->rlink != NULL) + GO_RIGHT(); + old = *leaf; + *keyp = old->key; + *leaf = old->llink; + free(old); } - r = (*rootp)->rlink; /* D1: */ - if ((q = (*rootp)->llink) == NULL) /* Left NULL? */ - q = r; - else if (r != NULL) { /* Right link is NULL? */ - if (r->llink == NULL) { /* D2: Find successor */ - r->llink = q; - q = r; - } else { /* D3: Find NULL link */ - for (q = r->llink; q->llink != NULL; q = r->llink) - r = q; - r->llink = q->rlink; - q->llink = (*rootp)->llink; - q->rlink = (*rootp)->rlink; + + /* + * Walk along the same path a second time and adjust the + * balances. Though this code looks similar to the rebalancing + * performed in tsearch(), it is not identical. We now also need + * to consider the case of outward imbalance in the right-right + * and left-left case that only exists when deleting. Hence the + * duplication of code. + */ + for (n = base; n != leaf;) { + if (path_took_left(&path)) { + x = *n; + if (x->balance < 0) { + y = x->rlink; + if (y->balance > 0) { + /* Right-left case. */ + z = y->llink; + x->rlink = z->llink; + z->llink = x; + y->llink = z->rlink; + z->rlink = y; + *n = z; + + x->balance = z->balance < 0 ? 1 : 0; + y->balance = z->balance > 0 ? -1 : 0; + z->balance = 0; + } else { + /* Right-right case. */ + x->rlink = y->llink; + y->llink = x; + *n = y; + + if (y->balance < 0) { + x->balance = 0; + y->balance = 0; + } else { + x->balance = -1; + y->balance = 1; + } + } + } else { + --x->balance; + } + n = &x->llink; + } else { + x = *n; + if (x->balance > 0) { + y = x->llink; + if (y->balance < 0) { + /* Left-right case. */ + z = y->rlink; + y->rlink = z->llink; + z->llink = y; + x->llink = z->rlink; + z->rlink = x; + *n = z; + + x->balance = z->balance > 0 ? -1 : 0; + y->balance = z->balance < 0 ? 1 : 0; + z->balance = 0; + } else { + /* Left-left case. */ + x->llink = y->rlink; + y->rlink = x; + *n = y; + + if (y->balance > 0) { + x->balance = 0; + y->balance = 0; + } else { + x->balance = 1; + y->balance = -1; + } + } + } else { + ++x->balance; + } + n = &x->rlink; } } - if (p != *rootp) - free(*rootp); /* D4: Free node */ - *rootp = q; /* link parent to new node */ - return p; + + /* Return the parent of the old entry. */ + *rootp = root; + return (result); } diff --git a/lib/libc/stdlib/tsearch.3 b/lib/libc/stdlib/tsearch.3 index 7a204d0..2205f7e 100644 --- a/lib/libc/stdlib/tsearch.3 +++ b/lib/libc/stdlib/tsearch.3 @@ -27,7 +27,7 @@ .\" OpenBSD: tsearch.3,v 1.2 1998/06/21 22:13:49 millert Exp .\" $FreeBSD$ .\" -.Dd June 15, 1997 +.Dd December 6, 2015 .Dt TSEARCH 3 .Os .Sh NAME @@ -50,8 +50,12 @@ The .Fn tsearch , and .Fn twalk -functions manage binary search trees based on algorithms T and D -from Knuth (6.2.2). +functions manage binary search trees. +This implementation uses a balanced AVL tree, +which due to its strong theoretical limit on the height of the tree has +the advantage of calling the comparison function relatively +infrequently. +.Pp The comparison function passed in by the user has the same style of return values as .Xr strcmp 3 . diff --git a/lib/libc/stdlib/tsearch.c b/lib/libc/stdlib/tsearch.c index 16bbf7c..b96a275 100644 --- a/lib/libc/stdlib/tsearch.c +++ b/lib/libc/stdlib/tsearch.c @@ -1,55 +1,200 @@ -/* $NetBSD: tsearch.c,v 1.7 2012/06/25 22:32:45 abs Exp $ */ - -/* - * Tree search generalized from Knuth (6.2.2) Algorithm T just like - * the AT&T man page says. - * - * The node_t structure is for internal use only, lint doesn't grok it. +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ * - * Written by reading the System V Interface Definition, not the code. + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. * - * Totally public domain. + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. */ #include <sys/cdefs.h> -#if 0 -#if defined(LIBC_SCCS) && !defined(lint) -__RCSID("$NetBSD: tsearch.c,v 1.7 2012/06/25 22:32:45 abs Exp $"); -#endif /* LIBC_SCCS and not lint */ -#endif __FBSDID("$FreeBSD$"); -#define _SEARCH_PRIVATE +#define _SEARCH_PRIVATE #include <search.h> #include <stdlib.h> -/* find or insert datum into search tree */ +#include "tsearch_path.h" + void * -tsearch(const void *vkey, void **vrootp, +tsearch(const void *key, void **rootp, int (*compar)(const void *, const void *)) { - node_t *q; - node_t **rootp = (node_t **)vrootp; + struct path path; + node_t *root, **base, **leaf, *result, *n, *x, *y, *z; + int cmp; + /* POSIX requires that tsearch() returns NULL if rootp is NULL. */ if (rootp == NULL) - return NULL; + return (NULL); + root = *rootp; - while (*rootp != NULL) { /* Knuth's T1: */ - int r; + /* + * Find the leaf where the new key needs to be inserted. Return + * if we've found an existing entry. Keep track of the path that + * is taken to get to the node, as we will need it to adjust the + * balances. + */ + path_init(&path); + base = &root; + leaf = &root; + while (*leaf != NULL) { + if ((*leaf)->balance != 0) { + /* + * If we reach a node that has a non-zero + * balance on the way, we know that we won't + * need to perform any rotations above this + * point. In this case rotations are always + * capable of keeping the subtree in balance. + * Make this the base node and reset the path. + */ + base = leaf; + path_init(&path); + } + cmp = compar(key, (*leaf)->key); + if (cmp < 0) { + path_taking_left(&path); + leaf = &(*leaf)->llink; + } else if (cmp > 0) { + path_taking_right(&path); + leaf = &(*leaf)->rlink; + } else { + return (&(*leaf)->key); + } + } - if ((r = (*compar)(vkey, (*rootp)->key)) == 0) /* T2: */ - return *rootp; /* we found it! */ + /* Did not find a matching key in the tree. Insert a new node. */ + result = *leaf = malloc(sizeof(**leaf)); + if (result == NULL) + return (NULL); + result->key = (void *)key; + result->llink = NULL; + result->rlink = NULL; + result->balance = 0; - rootp = (r < 0) ? - &(*rootp)->llink : /* T3: follow left branch */ - &(*rootp)->rlink; /* T4: follow right branch */ + /* + * Walk along the same path a second time and adjust the + * balances. Except for the first node, all of these nodes must + * have a balance of zero, meaning that these nodes will not get + * out of balance. + */ + for (n = *base; n != *leaf;) { + if (path_took_left(&path)) { + n->balance += 1; + n = n->llink; + } else { + n->balance -= 1; + n = n->rlink; + } } - q = malloc(sizeof(node_t)); /* T5: key not found */ - if (q != 0) { /* make new node */ - *rootp = q; /* link new node to old */ - q->key = __DECONST(void *, vkey);/* initialize new node */ - q->llink = q->rlink = NULL; + /* + * Adjusting the balances may have pushed the balance of the + * base node out of range. Perform a rotation to bring the + * balance back in range. + */ + x = *base; + if (x->balance > 1) { + y = x->llink; + if (y->balance < 0) { + /* + * Left-right case. + * + * x + * / \ z + * y D / \ + * / \ --> y x + * A z /| |\ + * / \ A B C D + * B C + */ + z = y->rlink; + y->rlink = z->llink; + z->llink = y; + x->llink = z->rlink; + z->rlink = x; + *base = z; + + x->balance = z->balance > 0 ? -1 : 0; + y->balance = z->balance < 0 ? 1 : 0; + z->balance = 0; + } else { + /* + * Left-left case. + * + * x y + * / \ / \ + * y C --> A x + * / \ / \ + * A B B C + */ + x->llink = y->rlink; + y->rlink = x; + *base = y; + + x->balance = 0; + y->balance = 0; + } + } else if (x->balance < -1) { + y = x->rlink; + if (y->balance > 0) { + /* + * Right-left case. + * + * x + * / \ z + * A y / \ + * / \ --> x y + * z D /| |\ + * / \ A B C D + * B C + */ + node_t *z = y->llink; + x->rlink = z->llink; + z->llink = x; + y->llink = z->rlink; + z->rlink = y; + *base = z; + + x->balance = z->balance < 0 ? 1 : 0; + y->balance = z->balance > 0 ? -1 : 0; + z->balance = 0; + } else { + /* + * Right-right case. + * + * x y + * / \ / \ + * A y --> x C + * / \ / \ + * B C A B + */ + x->rlink = y->llink; + y->llink = x; + *base = y; + + x->balance = 0; + y->balance = 0; + } } - return q; + + /* Return the new entry. */ + *rootp = root; + return (&result->key); } diff --git a/lib/libc/stdlib/tsearch_path.h b/lib/libc/stdlib/tsearch_path.h new file mode 100644 index 0000000..934c91f --- /dev/null +++ b/lib/libc/stdlib/tsearch_path.h @@ -0,0 +1,97 @@ +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef TSEARCH_PATH_H +#define TSEARCH_PATH_H + +#include <limits.h> +#include <stdbool.h> +#include <stdint.h> + +/* + * Bookkeeping for storing a path in a balanced binary search tree from + * the root to a leaf node. + * + * For an AVL tree we know that its maximum height of a tree is bounded + * by approximately 1.44 * log2(n) - 0.328. Given that the number of + * entries of the tree is constrained by the size of the address space, + * two uintptr_t's provide sufficient space to store the path from the + * root to any leaf. + */ +struct path { + uintptr_t steps[2]; + unsigned int nsteps; +}; + +/* Initializes the path structure with a zero-length path. */ +static inline void +path_init(struct path *p) +{ + + p->nsteps = 0; +} + +#define STEPS_BIT (sizeof(uintptr_t) * CHAR_BIT) + +/* Pushes a step to the left to the end of the path. */ +static inline void +path_taking_left(struct path *p) +{ + + p->steps[p->nsteps / STEPS_BIT] |= + (uintptr_t)1 << (p->nsteps % STEPS_BIT); + ++p->nsteps; +} + +/* Pushes a step to the right to the end of the path. */ +static inline void +path_taking_right(struct path *p) +{ + + p->steps[p->nsteps / STEPS_BIT] &= + ~((uintptr_t)1 << (p->nsteps % STEPS_BIT)); + ++p->nsteps; +} + +/* + * Pops the first step from the path and returns whether it was a step + * to the left. + */ +static inline bool +path_took_left(struct path *p) +{ + bool result; + + result = p->steps[0] & 0x1; + p->steps[0] = (p->steps[0] >> 1) | (p->steps[1] << (STEPS_BIT - 1)); + p->steps[1] >>= 1; + return (result); +} + +#undef STEPS_BIT + +#endif diff --git a/lib/libc/sys/clock_gettime.2 b/lib/libc/sys/clock_gettime.2 index 583cc8f..e77b1e8 100644 --- a/lib/libc/sys/clock_gettime.2 +++ b/lib/libc/sys/clock_gettime.2 @@ -29,7 +29,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 29, 2009 +.Dd December 27, 2015 .Dt CLOCK_GETTIME 2 .Os .Sh NAME @@ -134,12 +134,10 @@ The following error codes may be set in .It Bq Er EINVAL The .Fa clock_id +or +.Fa timespec argument was not a valid value. -.It Bq Er EFAULT -The -.Fa *tp -argument address referenced invalid memory. .It Bq Er EPERM A user other than the super-user attempted to set the time. .El diff --git a/lib/libc/sys/gettimeofday.2 b/lib/libc/sys/gettimeofday.2 index 23cc059..86b6b1a 100644 --- a/lib/libc/sys/gettimeofday.2 +++ b/lib/libc/sys/gettimeofday.2 @@ -28,7 +28,7 @@ .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 .\" $FreeBSD$ .\" -.Dd May 26, 1995 +.Dd December 27, 2015 .Dt GETTIMEOFDAY 2 .Os .Sh NAME @@ -110,8 +110,10 @@ system call even when the system is secure. The following error codes may be set in .Va errno : .Bl -tag -width Er -.It Bq Er EFAULT -An argument address referenced invalid memory. +.It Bq Er EINVAL +The supplied +.Fa timeval +value is invalid. .It Bq Er EPERM A user other than the super-user attempted to set the time. .El diff --git a/lib/libc/tests/resolv/Makefile b/lib/libc/tests/resolv/Makefile index 4e4e62be..4fb43d8 100644 --- a/lib/libc/tests/resolv/Makefile +++ b/lib/libc/tests/resolv/Makefile @@ -6,7 +6,6 @@ BINDIR= ${TESTSDIR} FILES+= mach ATF_TESTS_C+= resolv_test -#TEST_METADATA.resolv_test= timeout="1800" # Note: this test relies on being dynamically linked. You will get a # spurious PASS for a statically linked test. diff --git a/lib/libc/tests/resolv/resolv_test.c b/lib/libc/tests/resolv/resolv_test.c index 5c53569..74e89b1 100644 --- a/lib/libc/tests/resolv/resolv_test.c +++ b/lib/libc/tests/resolv/resolv_test.c @@ -289,21 +289,31 @@ do { \ ATF_REQUIRE(run_tests(_hostlist_file, method) == 0); \ } while(0) -ATF_TC_WITHOUT_HEAD(getaddrinfo_test); +ATF_TC(getaddrinfo_test); +ATF_TC_HEAD(getaddrinfo_test, tc) { + atf_tc_set_md_var(tc, "timeout", "450"); +} ATF_TC_BODY(getaddrinfo_test, tc) { RUN_TESTS(tc, METHOD_GETADDRINFO); } -ATF_TC_WITHOUT_HEAD(gethostby_test); +ATF_TC(gethostby_test); +ATF_TC_HEAD(gethostby_test, tc) { + atf_tc_set_md_var(tc, "timeout", "450"); +} ATF_TC_BODY(gethostby_test, tc) { RUN_TESTS(tc, METHOD_GETHOSTBY); } -ATF_TC_WITHOUT_HEAD(getipnodeby_test); +ATF_TC(getipnodeby_test); +ATF_TC_HEAD(getipnodeby_test, tc) { + + atf_tc_set_md_var(tc, "timeout", "450"); +} ATF_TC_BODY(getipnodeby_test, tc) { diff --git a/lib/libc/tests/stdlib/Makefile b/lib/libc/tests/stdlib/Makefile index 4bc1354..87e84c5 100644 --- a/lib/libc/tests/stdlib/Makefile +++ b/lib/libc/tests/stdlib/Makefile @@ -3,6 +3,7 @@ ATF_TESTS_C+= heapsort_test ATF_TESTS_C+= mergesort_test ATF_TESTS_C+= qsort_test +ATF_TESTS_C+= tsearch_test # TODO: t_getenv_thread, t_mi_vector_hash NETBSD_ATF_TESTS_C+= abs_test diff --git a/lib/libc/tests/stdlib/tsearch_test.c b/lib/libc/tests/stdlib/tsearch_test.c new file mode 100644 index 0000000..b08fd94 --- /dev/null +++ b/lib/libc/tests/stdlib/tsearch_test.c @@ -0,0 +1,131 @@ +/*- + * Copyright (c) 2015 Nuxi, https://nuxi.nl/ + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <atf-c.h> +#define _SEARCH_PRIVATE +#include <search.h> +#include <stdbool.h> +#include <stdlib.h> + +/* Validates the integrity of an AVL tree. */ +static inline unsigned int +tnode_assert(const node_t *n) +{ + unsigned int height_left, height_right; + int balance; + + if (n == NULL) + return 0; + height_left = tnode_assert(n->llink); + height_right = tnode_assert(n->rlink); + balance = (int)height_left - (int)height_right; + ATF_CHECK(balance >= -1); + ATF_CHECK(balance <= 1); + ATF_CHECK_EQ(balance, n->balance); + return (height_left > height_right ? height_left : height_right) + 1; +} + +static int +compar(const void *a, const void *b) +{ + + return *(int *)a - *(int *)b; +} + +ATF_TC_WITHOUT_HEAD(tsearch_test); +ATF_TC_BODY(tsearch_test, tc) +{ + /* + * Run the test below in a deterministic fashion to prevent this + * test from potentially flapping. We assume that this provides + * enough coverage. + */ +#if 0 + unsigned short random_state[3]; + arc4random_buf(random_state, sizeof(random_state)); +#else + unsigned short random_state[3] = { 26554, 13330, 3246 }; +#endif + +#define NKEYS 1000 + /* Create 1000 possible keys. */ + int keys[NKEYS]; + for (int i = 0; i < NKEYS; ++i) + keys[i] = i; + + /* Apply random operations on a binary tree and check the results. */ + void *root = NULL; + bool present[NKEYS] = {}; + for (int i = 0; i < NKEYS * 10; ++i) { + int key = nrand48(random_state) % NKEYS; + switch (nrand48(random_state) % 3) { + case 0: /* tdelete(). */ + if (present[key]) { + ATF_CHECK(tdelete(&key, &root, compar) != NULL); + present[key] = false; + } else { + ATF_CHECK_EQ(NULL, + tdelete(&key, &root, compar)); + } + break; + case 1: /* tfind(). */ + if (present[key]) { + ATF_CHECK_EQ(&keys[key], + *(int **)tfind(&key, &root, compar)); + } else { + ATF_CHECK_EQ(NULL, tfind(&key, &root, compar)); + } + break; + case 2: /* tsearch(). */ + if (present[key]) { + ATF_CHECK_EQ(&keys[key], + *(int **)tsearch(&key, &root, compar)); + } else { + ATF_CHECK_EQ(&keys[key], *(int **)tsearch( + &keys[key], &root, compar)); + present[key] = true; + } + break; + } + tnode_assert(root); + } + + /* Remove all entries from the tree. */ + for (int key = 0; key < NKEYS; ++key) + if (present[key]) + ATF_CHECK(tdelete(&key, &root, compar) != NULL); + ATF_CHECK_EQ(NULL, root); +} + +ATF_TP_ADD_TCS(tp) +{ + + ATF_TP_ADD_TC(tp, tsearch_test); + + return (atf_no_error()); +} diff --git a/lib/libclang_rt/Makefile.inc b/lib/libclang_rt/Makefile.inc index f9bd454..1e656d9 100644 --- a/lib/libclang_rt/Makefile.inc +++ b/lib/libclang_rt/Makefile.inc @@ -5,7 +5,7 @@ CRTARCH=${MACHINE_CPUARCH:C/amd64/x86_64/} CRTSRC=${.CURDIR}/../../../contrib/compiler-rt -CLANGDIR=/usr/lib/clang/3.7.0 +CLANGDIR=/usr/lib/clang/3.7.1 LIBDIR=${CLANGDIR}/lib/freebsd NO_PIC= diff --git a/lib/libcrypt/Makefile b/lib/libcrypt/Makefile index 70c4634..d5d0347 100644 --- a/lib/libcrypt/Makefile +++ b/lib/libcrypt/Makefile @@ -30,6 +30,7 @@ CFLAGS+= -I${.CURDIR} -DHAS_DES -DHAS_BLOWFISH .for sym in MD4Init MD4Final MD4Update MD4Pad \ MD5Init MD5Final MD5Update MD5Pad \ SHA256_Init SHA256_Final SHA256_Update \ + SHA384_Init SHA384_Final SHA384_Update \ SHA512_Init SHA512_Final SHA512_Update CFLAGS+= -D${sym}=__${sym} .endfor diff --git a/lib/libcuse/cuse_lib.c b/lib/libcuse/cuse_lib.c index 321522d..dd25ef8 100644 --- a/lib/libcuse/cuse_lib.c +++ b/lib/libcuse/cuse_lib.c @@ -711,8 +711,8 @@ cuse_copy_out(const void *src, void *user_dst, int len) if (pe->is_local) { memcpy(user_dst, src, len); } else { - info.local_ptr = (unsigned long)src; - info.peer_ptr = (unsigned long)user_dst; + info.local_ptr = (uintptr_t)src; + info.peer_ptr = (uintptr_t)user_dst; info.length = len; error = ioctl(f_cuse, CUSE_IOCTL_WRITE_DATA, &info); @@ -744,8 +744,8 @@ cuse_copy_in(const void *user_src, void *dst, int len) if (pe->is_local) { memcpy(dst, user_src, len); } else { - info.local_ptr = (unsigned long)dst; - info.peer_ptr = (unsigned long)user_src; + info.local_ptr = (uintptr_t)dst; + info.peer_ptr = (uintptr_t)user_src; info.length = len; error = ioctl(f_cuse, CUSE_IOCTL_READ_DATA, &info); diff --git a/lib/libmd/Makefile b/lib/libmd/Makefile index c1f6eb9..43b8a5a 100644 --- a/lib/libmd/Makefile +++ b/lib/libmd/Makefile @@ -7,8 +7,9 @@ SRCS= md4c.c md5c.c md4hl.c md5hl.c \ rmd160c.c rmd160hl.c \ sha0c.c sha0hl.c sha1c.c sha1hl.c \ sha256c.c sha256hl.c \ + sha384hl.c \ sha512c.c sha512hl.c -INCS= md4.h md5.h ripemd.h sha.h sha256.h sha512.h +INCS= md4.h md5.h ripemd.h sha.h sha256.h sha384.h sha512.h WARNS?= 0 @@ -33,6 +34,10 @@ MLINKS+=sha256.3 SHA256_Init.3 sha256.3 SHA256_Update.3 MLINKS+=sha256.3 SHA256_Final.3 sha256.3 SHA256_End.3 MLINKS+=sha256.3 SHA256_File.3 sha256.3 SHA256_FileChunk.3 MLINKS+=sha256.3 SHA256_Data.3 +MLINKS+=sha512.3 SHA384_Init.3 sha512.3 SHA384_Update.3 +MLINKS+=sha512.3 SHA384_Final.3 sha512.3 SHA384_End.3 +MLINKS+=sha512.3 SHA384_File.3 sha512.3 SHA384_FileChunk.3 +MLINKS+=sha512.3 SHA384_Data.3 sha512.3 sha384.3 MLINKS+=sha512.3 SHA512_Init.3 sha512.3 SHA512_Update.3 MLINKS+=sha512.3 SHA512_Final.3 sha512.3 SHA512_End.3 MLINKS+=sha512.3 SHA512_File.3 sha512.3 SHA512_FileChunk.3 @@ -40,7 +45,8 @@ MLINKS+=sha512.3 SHA512_Data.3 CLEANFILES+= md[245]hl.c md[245].ref md[245].3 mddriver \ rmd160.ref rmd160hl.c rmddriver \ sha0.ref sha0hl.c sha1.ref sha1hl.c shadriver \ - sha256.ref sha256hl.c sha512.ref sha512hl.c + sha256.ref sha256hl.c sha384hl.c sha384.ref \ + sha512.ref sha512hl.c # Define WEAK_REFS to provide weak aliases for libmd symbols # @@ -88,6 +94,12 @@ sha256hl.c: mdXhl.c -e 's/SHA256__/SHA256_/g' \ ${.ALLSRC}) > ${.TARGET} +sha384hl.c: mdXhl.c + (echo '#define LENGTH 48'; \ + sed -e 's/mdX/sha384/g' -e 's/MDX/SHA384_/g' \ + -e 's/SHA384__/SHA384_/g' \ + ${.ALLSRC}) > ${.TARGET} + sha512hl.c: mdXhl.c (echo '#define LENGTH 64'; \ sed -e 's/mdX/sha512/g' -e 's/MDX/SHA512_/g' \ @@ -168,6 +180,21 @@ sha256.ref: @echo 'SHA-256 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") =' \ 'f371bc4a311f2b009eef952dd83ca80e2b60026c8e935592d0f9c308453c813e' >> ${.TARGET} +sha384.ref: + echo 'SHA-384 test suite:' > ${.TARGET} + @echo 'SHA-384 ("") =' \ + '38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b' >> ${.TARGET} + @echo 'SHA-384 ("abc") =' \ + 'cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7' >> ${.TARGET} + @echo 'SHA-384 ("message digest") =' \ + '473ed35167ec1f5d8e550368a3db39be54639f828868e9454c239fc8b52e3c61dbd0d8b4de1390c256dcbb5d5fd99cd5' >> ${.TARGET} + @echo 'SHA-384 ("abcdefghijklmnopqrstuvwxyz") =' \ + 'feb67349df3db6f5924815d6c3dc133f091809213731fe5c7b5f4999e463479ff2877f5f2936fa63bb43784b12f3ebb4' >> ${.TARGET} + @echo 'SHA-384 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") =' \ + '1761336e3f7cbfe51deb137f026f89e01a448e3b1fafa64039c1464ee8732f11a5341a6f41e0c202294736ed64db1a84' >> ${.TARGET} + @echo 'SHA-384 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") =' \ + 'b12932b0627d1c060942f5447764155655bd4da0c9afa6dd9b9ef53129af1b8fb0195996d2de9ca0df9d821ffee67026' >> ${.TARGET} + sha512.ref: echo 'SHA-512 test suite:' > ${.TARGET} @echo 'SHA-512 ("") =' \ @@ -196,7 +223,8 @@ rmd160.ref: @echo 'RIPEMD160 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") =' \ '9b752e45573d4b39f4dbd3323cab82bf63326bfb' >> ${.TARGET} -test: md4.ref md5.ref sha0.ref rmd160.ref sha1.ref sha256.ref sha512.ref +test: md4.ref md5.ref sha0.ref rmd160.ref sha1.ref sha256.ref sha384.ref \ + sha512.ref @${ECHO} if any of these test fail, the code produces wrong results @${ECHO} and should NOT be used. ${CC} ${CFLAGS} ${LDFLAGS} -DMD=4 -o mddriver ${.CURDIR}/mddriver.c libmd.a @@ -219,6 +247,9 @@ test: md4.ref md5.ref sha0.ref rmd160.ref sha1.ref sha256.ref sha512.ref ${CC} ${CFLAGS} ${LDFLAGS} -DSHA=256 -o shadriver ${.CURDIR}/shadriver.c libmd.a ./shadriver | cmp sha256.ref - @${ECHO} SHA-256 passed test + ${CC} ${CFLAGS} ${LDFLAGS} -DSHA=384 -o shadriver ${.CURDIR}/shadriver.c libmd.a + ./shadriver | cmp sha384.ref - + @${ECHO} SHA-384 passed test ${CC} ${CFLAGS} ${LDFLAGS} -DSHA=512 -o shadriver ${.CURDIR}/shadriver.c libmd.a ./shadriver | cmp sha512.ref - @${ECHO} SHA-512 passed test diff --git a/lib/libmd/sha512.3 b/lib/libmd/sha512.3 index bcb5252..d6b3aaf 100644 --- a/lib/libmd/sha512.3 +++ b/lib/libmd/sha512.3 @@ -9,7 +9,7 @@ .\" From: Id: mdX.3,v 1.14 1999/02/11 20:31:49 wollman Exp .\" $FreeBSD$ .\" -.Dd March 28, 2014 +.Dd October 17, 2015 .Dt SHA512 3 .Os .Sh NAME @@ -19,8 +19,15 @@ .Nm SHA512_End , .Nm SHA512_File , .Nm SHA512_FileChunk , -.Nm SHA512_Data -.Nd calculate the FIPS 180-2 ``SHA-512'' message digest +.Nm SHA512_Data , +.Nm SHA384_Init , +.Nm SHA384_Update , +.Nm SHA384_Final , +.Nm SHA384_End , +.Nm SHA384_File , +.Nm SHA384_FileChunk , +.Nm SHA384_Data +.Nd calculate the FIPS 180-4 ``SHA-512'' family of message digests .Sh LIBRARY .Lb libmd .Sh SYNOPSIS @@ -40,6 +47,20 @@ .Fn SHA512_FileChunk "const char *filename" "char *buf" "off_t offset" "off_t length" .Ft "char *" .Fn SHA512_Data "const unsigned char *data" "unsigned int len" "char *buf" +.Ft void +.Fn SHA384_Init "SHA384_CTX *context" +.Ft void +.Fn SHA384_Update "SHA384_CTX *context" "const unsigned char *data" "size_t len" +.Ft void +.Fn SHA384_Final "unsigned char digest[48]" "SHA384_CTX *context" +.Ft "char *" +.Fn SHA384_End "SHA384_CTX *context" "char *buf" +.Ft "char *" +.Fn SHA384_File "const char *filename" "char *buf" +.Ft "char *" +.Fn SHA384_FileChunk "const char *filename" "char *buf" "off_t offset" "off_t length" +.Ft "char *" +.Fn SHA384_Data "const unsigned char *data" "unsigned int len" "char *buf" .Sh DESCRIPTION The .Li SHA512_ @@ -119,6 +140,21 @@ after use. If the .Fa buf argument is non-null it must point to at least 65 characters of buffer space. +.Pp +The +.Li SHA384_ +functions are identical to the +.Li SHA512_ +functions except they use a different initial hash value and the output is +truncated to 384 bits. +.Pp +.Fn SHA384_End +is a wrapper for +.Fn SHA384_Final +which converts the return value to a 49-character +(including the terminating '\e0') +.Tn ASCII +string which represents the 384 bits in hexadecimal. .Sh SEE ALSO .Xr md4 3 , .Xr md5 3 , diff --git a/lib/libmd/shadriver.c b/lib/libmd/shadriver.c index 29a3cf0..e7e7b5d 100644 --- a/lib/libmd/shadriver.c +++ b/lib/libmd/shadriver.c @@ -22,6 +22,7 @@ __FBSDID("$FreeBSD$"); #include "sha.h" #include "sha256.h" +#include "sha384.h" #include "sha512.h" /* The following makes SHA default to SHA-1 if it has not already been @@ -36,6 +37,9 @@ __FBSDID("$FreeBSD$"); #elif SHA == 256 #undef SHA_Data #define SHA_Data SHA256_Data +#elif SHA == 384 +#undef SHA_Data +#define SHA_Data SHA384_Data #elif SHA == 512 #undef SHA_Data #define SHA_Data SHA512_Data diff --git a/lib/libstand/bootp.c b/lib/libstand/bootp.c index 904b3ba..1af7bd5 100644 --- a/lib/libstand/bootp.c +++ b/lib/libstand/bootp.c @@ -354,6 +354,7 @@ vend_rfc1048(cp, len) u_char *ep; int size; u_char tag; + const char *val; #ifdef BOOTP_DEBUG if (debug) @@ -380,15 +381,17 @@ vend_rfc1048(cp, len) } if (tag == TAG_SWAPSERVER) { /* let it override bp_siaddr */ - bcopy(cp, &rootip.s_addr, sizeof(swapip.s_addr)); + bcopy(cp, &rootip.s_addr, sizeof(rootip.s_addr)); } if (tag == TAG_ROOTPATH) { - strncpy(rootpath, (char *)cp, sizeof(rootpath)); - rootpath[size] = '\0'; + if ((val = getenv("dhcp.root-path")) == NULL) + val = (const char *)cp; + strlcpy(rootpath, val, sizeof(rootpath)); } if (tag == TAG_HOSTNAME) { - strncpy(hostname, (char *)cp, sizeof(hostname)); - hostname[size] = '\0'; + if ((val = getenv("dhcp.host-name")) == NULL) + val = (const char *)cp; + strlcpy(hostname, val, sizeof(hostname)); } #ifdef SUPPORT_DHCP if (tag == TAG_DHCP_MSGTYPE) { @@ -730,7 +733,11 @@ setenv_(u_char *cp, u_char *ep, struct dhcp_opt *opts) sprintf(env, op->desc, opts[0].desc, tag); else sprintf(env, "%s%s", opts[0].desc, op->desc); - setenv(env, buf, 1); + /* + * Do not replace existing values in the environment, so that + * locally-obtained values can override server-provided values. + */ + setenv(env, buf, 0); } } if (tp != tags) { diff --git a/lib/libsysdecode/Makefile b/lib/libsysdecode/Makefile index 8eb7908..a6d3ba1 100644 --- a/lib/libsysdecode/Makefile +++ b/lib/libsysdecode/Makefile @@ -4,10 +4,31 @@ LIB= sysdecode -SRCS= utrace.c +SRCS= ioctl.c utrace.c INCS= sysdecode.h MAN+= sysdecode.3 \ + sysdecode_ioctlname.3 \ sysdecode_utrace.3 +CLEANFILES= ioctl.c + +.if defined(COMPAT_32BIT) +CPP+= -m32 +.endif + +.if ${MK_PF} != "no" +CFLAGS+=-DPF +.endif + +# Workaround duplicate declarations in <netinet/ip_compat.h> +CFLAGS.gcc.ioctl.c+= -Wno-redundant-decls +CFLAGS.gcc+= ${CFLAGS.gcc.${.IMPSRC}} + +ioctl.c: mkioctls + env MACHINE=${MACHINE} CPP="${CPP}" \ + /bin/sh ${.CURDIR}/mkioctls ${DESTDIR}${INCLUDEDIR} > ${.TARGET} + +beforedepend: ioctl.c + .include <bsd.lib.mk> diff --git a/usr.bin/kdump/mkioctls b/lib/libsysdecode/mkioctls index a563341..e174d30 100644 --- a/usr.bin/kdump/mkioctls +++ b/lib/libsysdecode/mkioctls @@ -1,19 +1,15 @@ #!/bin/sh # # $FreeBSD$ -# -# When editing this script, keep in mind that truss also uses it. -# set -e -if [ $# -ne 2 -o \( $1 != "print" -a $1 != "return" \) ]; then - echo "usage: sh $0 print|return include-dir" +if [ $# -ne 1 ]; then + echo "usage: sh $0 include-dir" exit 1 fi -style="$1" -includedir="$2" +includedir="$1" LC_ALL=C; export LC_ALL @@ -40,7 +36,7 @@ esac awk -v x="$ioctl_includes" 'BEGIN {print x}' | $CPP -nostdinc -I$includedir -dM -DCOMPAT_43TTY - | - awk -v ioctl_includes="$ioctl_includes" -v style="$style" ' + awk -v ioctl_includes="$ioctl_includes" ' BEGIN { print "/* XXX obnoxious prerequisites. */" print "#define COMPAT_43" @@ -68,20 +64,12 @@ BEGIN { print "#include <cam/cam.h>" print "#include <stddef.h>" print "#include <stdint.h>" + print "#include <sysdecode.h>" print "" print ioctl_includes print "" - if (style == "print") { - print "void ioctlname(unsigned long val, int decimal);" - print "" - print "void" - print "ioctlname(unsigned long val, int decimal)" - } else { - print "const char *ioctlname(unsigned long val);" - print "" - print "const char *" - print "ioctlname(unsigned long val)" - } + print "const char *" + print "sysdecode_ioctlname(unsigned long val)" print "{" print "\tconst char *str = NULL;" print "" @@ -103,16 +91,7 @@ BEGIN { } END { print "" - if (style == "print") { - print "\tif (str != NULL)" - print "\t\tprintf(\"%s\", str);" - print "\telse if (decimal)" - print "\t\tprintf(\"%lu\", val);" - print "\telse" - print "\t\tprintf(\"%#lx\", val);" - } else { - print "\treturn (str);" - } + print "\treturn (str);" print "}" } ' diff --git a/lib/libsysdecode/sysdecode.3 b/lib/libsysdecode/sysdecode.3 index fd1677d..52faf53 100644 --- a/lib/libsysdecode/sysdecode.3 +++ b/lib/libsysdecode/sysdecode.3 @@ -39,6 +39,7 @@ The library includes several functions that provide descriptive names of values associated with system calls. .Sh SEE ALSO +.Xr sysdecode_ioctlname 3 , .Xr sysdecode_utrace 3 .Sh HISTORY The diff --git a/lib/libsysdecode/sysdecode.h b/lib/libsysdecode/sysdecode.h index 10feee1..aa95838 100644 --- a/lib/libsysdecode/sysdecode.h +++ b/lib/libsysdecode/sysdecode.h @@ -29,6 +29,7 @@ #ifndef __SYSDECODE_H__ #define __SYSDECODE_H__ +const char *sysdecode_ioctlname(unsigned long _val); int sysdecode_utrace(FILE *_fp, void *_buf, size_t _len); #endif /* !__SYSDECODE_H__ */ diff --git a/lib/libsysdecode/sysdecode_ioctlname.3 b/lib/libsysdecode/sysdecode_ioctlname.3 new file mode 100644 index 0000000..6479f03 --- /dev/null +++ b/lib/libsysdecode/sysdecode_ioctlname.3 @@ -0,0 +1,57 @@ +.\" +.\" Copyright (c) 2015 John Baldwin <jhb@FreeBSD.org> +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd December 12, 2015 +.Dt sysdecode_ioctlname 3 +.Os +.Sh NAME +.Nm sysdecode_ioctlname +.Nd lookup name of device control command +.Sh LIBRARY +.Lb libsysdecode +.Sh SYNOPSIS +.Ft conts char * +.Fn sysdecode_ioctlname "unsigned long request" +.Sh DESCRIPTION +The +.Fn sysdecode_ioctlname +function returns the name of a device control request identified by +.Fa request . +A table of names is generated during the build of the +.Nm sysdecode +library from system headers that maps device control request values to +the name of the corresponding C macro. +.Sh RETURN VALUES +The +.Fn sysdecode_ioctlname +function returns the name of a device control request if +.Fa request +is a known value; +otherwise +.Dv NULL . +.Sh SEE ALSO +.Xr sysdecode 3 diff --git a/lib/libthr/thread/thr_fork.c b/lib/libthr/thread/thr_fork.c index 7dc8ca8..7256b68 100644 --- a/lib/libthr/thread/thr_fork.c +++ b/lib/libthr/thread/thr_fork.c @@ -209,7 +209,7 @@ __thr_fork(void) /* reinitalize library. */ _libpthread_init(curthread); - /* atfork is reinitializeded by _libpthread_init()! */ + /* atfork is reinitialized by _libpthread_init()! */ _thr_rwl_rdlock(&_thr_atfork_lock); if (was_threaded) { diff --git a/lib/libthr/thread/thr_init.c b/lib/libthr/thread/thr_init.c index 9a1a67a..e0400e4 100644 --- a/lib/libthr/thread/thr_init.c +++ b/lib/libthr/thread/thr_init.c @@ -332,7 +332,7 @@ _libpthread_init(struct pthread *curthread) PANIC("Can't set session ID"); if (revoke(_PATH_CONSOLE) != 0) PANIC("Can't revoke console"); - if ((fd = __sys_open(_PATH_CONSOLE, O_RDWR)) < 0) + if ((fd = __sys_openat(AT_FDCWD, _PATH_CONSOLE, O_RDWR)) < 0) PANIC("Can't open console"); if (setlogin("root") == -1) PANIC("Can't set login to root"); diff --git a/lib/libthr/thread/thr_private.h b/lib/libthr/thread/thr_private.h index ed24c38..0ba123d 100644 --- a/lib/libthr/thread/thr_private.h +++ b/lib/libthr/thread/thr_private.h @@ -803,7 +803,6 @@ void _pthread_cancel_leave(int maycancel); /* #include <fcntl.h> */ #ifdef _SYS_FCNTL_H_ int __sys_fcntl(int, int, ...); -int __sys_open(const char *, int, ...); int __sys_openat(int, const char *, int, ...); #endif diff --git a/lib/msun/tests/Makefile b/lib/msun/tests/Makefile index dfac5a2..bbf0f01 100644 --- a/lib/msun/tests/Makefile +++ b/lib/msun/tests/Makefile @@ -39,12 +39,19 @@ NETBSD_ATF_TESTS_C+= tanh_test TAP_TESTS_C+= cexp_test TAP_TESTS_C+= conj_test TAP_TESTS_C+= csqrt_test +TAP_TESTS_C+= ctrig_test +TAP_TESTS_C+= exponential_test TAP_TESTS_C+= fenv_test +TAP_TESTS_C+= fma_test TAP_TESTS_C+= fmaxmin_test TAP_TESTS_C+= ilogb_test +TAP_TESTS_C+= invtrig_test TAP_TESTS_C+= invctrig_test TAP_TESTS_C+= logarithm_test TAP_TESTS_C+= lrint_test +# XXX: the testcase crashes on all platforms, but only on head +# (bug 205451) +#TAP_TESTS_C+= lround_test TAP_TESTS_C+= nan_test TAP_TESTS_C+= nearbyint_test TAP_TESTS_C+= next_test @@ -53,7 +60,6 @@ TAP_TESTS_C+= trig_test .for t in ${TAP_TESTS_C} CFLAGS.$t+= -O0 -CFLAGS.$t+= -I${SRCTOP}/tools/regression/lib/msun .endfor CSTD= c99 diff --git a/tools/regression/lib/msun/test-ctrig.c b/lib/msun/tests/ctrig_test.c index 9f9b88d..475b6c5 100644 --- a/tools/regression/lib/msun/test-ctrig.c +++ b/lib/msun/tests/ctrig_test.c @@ -427,6 +427,7 @@ test_large(void) test_odd_tol(ctanh, z, CMPLXL(1.0, 8.95257245135025991216632140458264468e-309L), DBL_ULP()); +#if !defined(__i386__) z = CMPLXL(30, 0x1p1023L); test_odd_tol(ctanh, z, CMPLXL(1.0, -1.62994325413993477997492170229268382e-26L), @@ -436,6 +437,7 @@ test_large(void) CMPLXL(0.878606311888306869546254022621986509L, -0.225462792499754505792678258169527424L), DBL_ULP()); +#endif z = CMPLXL(710.6, 0.78539816339744830961566084581987572L); test_odd_tol(csinh, z, diff --git a/tools/regression/lib/msun/test-exponential.c b/lib/msun/tests/exponential_test.c index 010e0fd..df552ee 100644 --- a/tools/regression/lib/msun/test-exponential.c +++ b/lib/msun/tests/exponential_test.c @@ -66,13 +66,22 @@ __FBSDID("$FreeBSD$"); } while (0) /* Test all the functions that compute b^x. */ -#define testall0(x, result, exceptmask, excepts) do { \ +#define _testall0(x, result, exceptmask, excepts) do { \ test(exp, x, result, exceptmask, excepts); \ test(expf, x, result, exceptmask, excepts); \ test(exp2, x, result, exceptmask, excepts); \ test(exp2f, x, result, exceptmask, excepts); \ +} while (0) + +/* Skip over exp2l on platforms that don't support it. */ +#if LDBL_PREC == 53 +#define testall0 _testall0 +#else +#define testall0(x, result, exceptmask, excepts) do { \ + _testall0(x, result, exceptmask, excepts); \ test(exp2l, x, result, exceptmask, excepts); \ } while (0) +#endif /* Test all the functions that compute b^x - 1. */ #define testall1(x, result, exceptmask, excepts) do { \ @@ -102,12 +111,14 @@ run_generic_tests(void) testall0(-INFINITY, 0.0, ALL_STD_EXCEPT, 0); testall1(-INFINITY, -1.0, ALL_STD_EXCEPT, 0); +#if !defined(__i386__) /* exp(big) == Inf, overflow exception */ testall0(50000.0, INFINITY, ALL_STD_EXCEPT & ~FE_INEXACT, FE_OVERFLOW); testall1(50000.0, INFINITY, ALL_STD_EXCEPT & ~FE_INEXACT, FE_OVERFLOW); /* exp(small) == 0, underflow and inexact exceptions */ testall0(-50000.0, 0.0, ALL_STD_EXCEPT, FE_UNDERFLOW | FE_INEXACT); +#endif testall1(-50000.0, -1.0, ALL_STD_EXCEPT, FE_INEXACT); } diff --git a/tools/regression/lib/msun/test-fma.c b/lib/msun/tests/fma_test.c index 1fcf889..dfcea2a 100644 --- a/tools/regression/lib/msun/test-fma.c +++ b/lib/msun/tests/fma_test.c @@ -31,11 +31,13 @@ #include <sys/cdefs.h> __FBSDID("$FreeBSD$"); +#include <sys/param.h> #include <assert.h> #include <fenv.h> #include <float.h> #include <math.h> #include <stdio.h> +#include <stdlib.h> #include "test-utils.h" @@ -473,44 +475,63 @@ int main(int argc, char *argv[]) { int rmodes[] = { FE_TONEAREST, FE_UPWARD, FE_DOWNWARD, FE_TOWARDZERO }; - int i; + int i, j; + +#if defined(__i386__) + printf("1..0 # SKIP all testcases fail on i386\n"); + exit(0); +#endif + + j = 1; printf("1..19\n"); - for (i = 0; i < 4; i++) { + for (i = 0; i < nitems(rmodes); i++, j++) { + printf("rmode = %d\n", rmodes[i]); fesetround(rmodes[i]); test_zeroes(); - printf("ok %d - fma zeroes\n", i + 1); + printf("ok %d - fma zeroes\n", j); } - for (i = 0; i < 4; i++) { + for (i = 0; i < nitems(rmodes); i++, j++) { +#if defined(__amd64__) + printf("ok %d # SKIP testcase fails assertion on " + "amd64\n", j); + continue; +#endif + printf("rmode = %d\n", rmodes[i]); fesetround(rmodes[i]); test_infinities(); - printf("ok %d - fma infinities\n", i + 5); + printf("ok %d - fma infinities\n", j); } fesetround(FE_TONEAREST); test_nans(); - printf("ok 9 - fma NaNs\n"); + printf("ok %d - fma NaNs\n", j); + j++; - for (i = 0; i < 4; i++) { + for (i = 0; i < nitems(rmodes); i++, j++) { + printf("rmode = %d\n", rmodes[i]); fesetround(rmodes[i]); test_small_z(); - printf("ok %d - fma small z\n", i + 10); + printf("ok %d - fma small z\n", j); } - for (i = 0; i < 4; i++) { + for (i = 0; i < nitems(rmodes); i++, j++) { + printf("rmode = %d\n", rmodes[i]); fesetround(rmodes[i]); test_big_z(); - printf("ok %d - fma big z\n", i + 14); + printf("ok %d - fma big z\n", j); } fesetround(FE_TONEAREST); test_accuracy(); - printf("ok 18 - fma accuracy\n"); + printf("ok %d - fma accuracy\n", j); + j++; test_double_rounding(); - printf("ok 19 - fma double rounding\n"); + printf("ok %d - fma double rounding\n", j); + j++; /* * TODO: diff --git a/tools/regression/lib/msun/test-invtrig.c b/lib/msun/tests/invtrig_test.c index 9b41fd5..01b0379 100644 --- a/tools/regression/lib/msun/test-invtrig.c +++ b/lib/msun/tests/invtrig_test.c @@ -61,14 +61,23 @@ __FBSDID("$FreeBSD$"); #define test(func, x, result, excepts) \ test_tol(func, (x), (result), 0, (excepts)) -#define testall_tol(prefix, x, result, tol, excepts) do { \ +#define _testall_tol(prefix, x, result, tol, excepts) do { \ test_tol(prefix, (double)(x), (double)(result), \ (tol) * ldexp(1.0, 1 - DBL_MANT_DIG), (excepts)); \ test_tol(prefix##f, (float)(x), (float)(result), \ (tol) * ldexpf(1.0, 1 - FLT_MANT_DIG), (excepts)); \ +} while (0) + +#if LDBL_PREC == 53 +#define testall_tol _testall_tol +#else +#define testall_tol(prefix, x, result, tol, excepts) do { \ + _testall_tol(prefix, x, result, tol, excepts); \ test_tol(prefix##l, (x), (result), \ (tol) * ldexpl(1.0, 1 - LDBL_MANT_DIG), (excepts)); \ } while (0) +#endif + #define testall(prefix, x, result, excepts) \ testall_tol(prefix, (x), (result), 0, (excepts)) @@ -81,14 +90,23 @@ __FBSDID("$FreeBSD$"); #define test2(func, y, x, result, excepts) \ test2_tol(func, (y), (x), (result), 0, (excepts)) -#define testall2_tol(prefix, y, x, result, tol, excepts) do { \ +#define _testall2_tol(prefix, y, x, result, tol, excepts) do { \ test2_tol(prefix, (double)(y), (double)(x), (double)(result), \ (tol) * ldexp(1.0, 1 - DBL_MANT_DIG), (excepts)); \ test2_tol(prefix##f, (float)(y), (float)(x), (float)(result), \ (tol) * ldexpf(1.0, 1 - FLT_MANT_DIG), (excepts)); \ +} while (0) + +#if LDBL_PREC == 53 +#define testall2_tol _testall2_tol +#else +#define testall2_tol(prefix, y, x, result, tol, excepts) do { \ + _testall2_tol(prefix, y, x, result, tol, excepts); \ test2_tol(prefix##l, (y), (x), (result), \ (tol) * ldexpl(1.0, 1 - LDBL_MANT_DIG), (excepts)); \ } while (0) +#endif + #define testall2(prefix, y, x, result, excepts) \ testall2_tol(prefix, (y), (x), (result), 0, (excepts)) @@ -429,6 +447,11 @@ int main(int argc, char *argv[]) { +#if defined(__i386__) + printf("1..0 # SKIP fails all assertions on i386\n"); + return (0); +#endif + printf("1..7\n"); test_special(); diff --git a/tools/regression/lib/msun/test-lround.c b/lib/msun/tests/lround_test.c index 2a37367..2a37367 100644 --- a/tools/regression/lib/msun/test-lround.c +++ b/lib/msun/tests/lround_test.c diff --git a/tools/regression/lib/msun/test-ctrig.t b/lib/msun/tests/lround_test.t index 8bdfd03..8bdfd03 100644 --- a/tools/regression/lib/msun/test-ctrig.t +++ b/lib/msun/tests/lround_test.t diff --git a/tools/regression/lib/msun/test-utils.h b/lib/msun/tests/test-utils.h index bf0d6de..bf0d6de 100644 --- a/tools/regression/lib/msun/test-utils.h +++ b/lib/msun/tests/test-utils.h diff --git a/libexec/rtld-elf/arm/reloc.c b/libexec/rtld-elf/arm/reloc.c index 9cbdc0e..d0bf987 100644 --- a/libexec/rtld-elf/arm/reloc.c +++ b/libexec/rtld-elf/arm/reloc.c @@ -15,6 +15,7 @@ __FBSDID("$FreeBSD$"); #include "debug.h" #include "rtld.h" +#include "paths.h" void init_pltgot(Obj_Entry *obj) diff --git a/libexec/rtld-elf/riscv/reloc.c b/libexec/rtld-elf/riscv/reloc.c new file mode 100644 index 0000000..c43800a --- /dev/null +++ b/libexec/rtld-elf/riscv/reloc.c @@ -0,0 +1,400 @@ +/*- + * Copyright (c) 2015 Ruslan Bukin <br@bsdpad.com> + * All rights reserved. + * + * This software was developed by SRI International and the University of + * Cambridge Computer Laboratory under DARPA/AFRL contract FA8750-10-C-0237 + * ("CTSRD"), as part of the DARPA CRASH research programme. + * + * This software was developed by the University of Cambridge Computer + * Laboratory as part of the CTSRD Project, with support from the UK Higher + * Education Innovation Fund (HEIF). + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/types.h> + +#include <stdlib.h> + +#include "debug.h" +#include "rtld.h" +#include "rtld_printf.h" + +/* + * It is possible for the compiler to emit relocations for unaligned data. + * We handle this situation with these inlines. + */ +#define RELOC_ALIGNED_P(x) \ + (((uintptr_t)(x) & (sizeof(void *) - 1)) == 0) + +void _exit(int); + +uint64_t +set_gp(Obj_Entry *obj) +{ + uint64_t old; + SymLook req; + uint64_t gp; + int res; + + __asm __volatile("mv %0, gp" : "=r"(old)); + + symlook_init(&req, "_gp"); + req.ventry = NULL; + req.flags = SYMLOOK_EARLY; + res = symlook_obj(&req, obj); + + if (res == 0) { + gp = req.sym_out->st_value; + __asm __volatile("mv gp, %0" :: "r"(gp)); + } + + return (old); +} + +void +init_pltgot(Obj_Entry *obj) +{ + + if (obj->pltgot != NULL) { + obj->pltgot[0] = (Elf_Addr)&_rtld_bind_start; + obj->pltgot[1] = (Elf_Addr)obj; + } +} + +int +do_copy_relocations(Obj_Entry *dstobj) +{ + const Obj_Entry *srcobj, *defobj; + const Elf_Rela *relalim; + const Elf_Rela *rela; + const Elf_Sym *srcsym; + const Elf_Sym *dstsym; + const void *srcaddr; + const char *name; + void *dstaddr; + SymLook req; + size_t size; + int res; + + /* + * COPY relocs are invalid outside of the main program + */ + assert(dstobj->mainprog); + + relalim = (const Elf_Rela *)((char *)dstobj->rela + + dstobj->relasize); + for (rela = dstobj->rela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) != R_RISCV_COPY) + continue; + + dstaddr = (void *)(dstobj->relocbase + rela->r_offset); + dstsym = dstobj->symtab + ELF_R_SYM(rela->r_info); + name = dstobj->strtab + dstsym->st_name; + size = dstsym->st_size; + + symlook_init(&req, name); + req.ventry = fetch_ventry(dstobj, ELF_R_SYM(rela->r_info)); + req.flags = SYMLOOK_EARLY; + + for (srcobj = dstobj->next; srcobj != NULL; + srcobj = srcobj->next) { + res = symlook_obj(&req, srcobj); + if (res == 0) { + srcsym = req.sym_out; + defobj = req.defobj_out; + break; + } + } + if (srcobj == NULL) { + _rtld_error( +"Undefined symbol \"%s\" referenced from COPY relocation in %s", + name, dstobj->path); + return (-1); + } + + srcaddr = (const void *)(defobj->relocbase + srcsym->st_value); + memcpy(dstaddr, srcaddr, size); + } + + return (0); +} + +/* + * Process the PLT relocations. + */ +int +reloc_plt(Obj_Entry *obj) +{ + const Elf_Rela *relalim; + const Elf_Rela *rela; + + relalim = (const Elf_Rela *)((char *)obj->pltrela + obj->pltrelasize); + for (rela = obj->pltrela; rela < relalim; rela++) { + Elf_Addr *where; + + assert(ELF_R_TYPE(rela->r_info) == R_RISCV_JUMP_SLOT); + + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + *where += (Elf_Addr)obj->relocbase; + } + + return (0); +} + +/* + * LD_BIND_NOW was set - force relocation for all jump slots + */ +int +reloc_jmpslots(Obj_Entry *obj, int flags, RtldLockState *lockstate) +{ + const Obj_Entry *defobj; + const Elf_Rela *relalim; + const Elf_Rela *rela; + const Elf_Sym *def; + + relalim = (const Elf_Rela *)((char *)obj->pltrela + obj->pltrelasize); + for (rela = obj->pltrela; rela < relalim; rela++) { + Elf_Addr *where; + + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + switch(ELF_R_TYPE(rela->r_info)) { + case R_RISCV_JUMP_SLOT: + def = find_symdef(ELF_R_SYM(rela->r_info), obj, + &defobj, SYMLOOK_IN_PLT | flags, NULL, lockstate); + if (def == NULL) { + dbg("reloc_jmpslots: sym not found"); + return (-1); + } + + *where = (Elf_Addr)(defobj->relocbase + def->st_value); + break; + default: + _rtld_error("Unknown relocation type %x in jmpslot", + (unsigned int)ELF_R_TYPE(rela->r_info)); + return (-1); + } + } + + return (0); +} + +int +reloc_iresolve(Obj_Entry *obj, struct Struct_RtldLockState *lockstate) +{ + + /* XXX not implemented */ + return (0); +} + +int +reloc_gnu_ifunc(Obj_Entry *obj, int flags, + struct Struct_RtldLockState *lockstate) +{ + + /* XXX not implemented */ + return (0); +} + +Elf_Addr +reloc_jmpslot(Elf_Addr *where, Elf_Addr target, const Obj_Entry *defobj, + const Obj_Entry *obj, const Elf_Rel *rel) +{ + + assert(ELF_R_TYPE(rel->r_info) == R_RISCV_JUMP_SLOT); + + if (*where != target) + *where = target; + + return target; +} + +/* + * Process non-PLT relocations + */ +int +reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, + RtldLockState *lockstate) +{ + const Obj_Entry *defobj; + const Elf_Rela *relalim; + const Elf_Rela *rela; + const Elf_Sym *def; + SymCache *cache; + Elf_Addr *where; + unsigned long symnum; + + if ((flags & SYMLOOK_IFUNC) != 0) + /* XXX not implemented */ + return (0); + + /* + * The dynamic loader may be called from a thread, we have + * limited amounts of stack available so we cannot use alloca(). + */ + if (obj == obj_rtld) + cache = NULL; + else + cache = calloc(obj->dynsymcount, sizeof(SymCache)); + /* No need to check for NULL here */ + + relalim = (const Elf_Rela *)((caddr_t)obj->rela + obj->relasize); + for (rela = obj->rela; rela < relalim; rela++) { + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + symnum = ELF_R_SYM(rela->r_info); + + switch (ELF_R_TYPE(rela->r_info)) { + case R_RISCV_JUMP_SLOT: + /* This will be handled by the plt/jmpslot routines */ + break; + case R_RISCV_NONE: + break; + case R_RISCV_64: + def = find_symdef(symnum, obj, &defobj, flags, cache, + lockstate); + if (def == NULL) + return (-1); + + *where = (Elf_Addr)(defobj->relocbase + def->st_value + + rela->r_addend); + break; + case R_RISCV_TLS_DTPMOD64: + def = find_symdef(symnum, obj, &defobj, flags, cache, + lockstate); + if (def == NULL) + return -1; + + *where += (Elf_Addr)defobj->tlsindex; + break; + case R_RISCV_COPY: + /* + * These are deferred until all other relocations have + * been done. All we do here is make sure that the + * COPY relocation is not in a shared library. They + * are allowed only in executable files. + */ + if (!obj->mainprog) { + _rtld_error("%s: Unexpected R_RISCV_COPY " + "relocation in shared library", obj->path); + return (-1); + } + break; + case R_RISCV_TLS_DTPREL64: + def = find_symdef(symnum, obj, &defobj, flags, cache, + lockstate); + if (def == NULL) + return (-1); + /* + * We lazily allocate offsets for static TLS as we + * see the first relocation that references the + * TLS block. This allows us to support (small + * amounts of) static TLS in dynamically loaded + * modules. If we run out of space, we generate an + * error. + */ + if (!defobj->tls_done) { + if (!allocate_tls_offset((Obj_Entry*) defobj)) { + _rtld_error( + "%s: No space available for static " + "Thread Local Storage", obj->path); + return (-1); + } + } + + *where += (Elf_Addr)(def->st_value + rela->r_addend + - TLS_DTV_OFFSET); + break; + case R_RISCV_TLS_TPREL64: + def = find_symdef(symnum, obj, &defobj, flags, cache, + lockstate); + if (def == NULL) + return (-1); + + /* + * We lazily allocate offsets for static TLS as we + * see the first relocation that references the + * TLS block. This allows us to support (small + * amounts of) static TLS in dynamically loaded + * modules. If we run out of space, we generate an + * error. + */ + if (!defobj->tls_done) { + if (!allocate_tls_offset((Obj_Entry*) defobj)) { + _rtld_error( + "%s: No space available for static " + "Thread Local Storage", obj->path); + return (-1); + } + } + + *where = (def->st_value + rela->r_addend + + defobj->tlsoffset - TLS_TP_OFFSET); + break; + case R_RISCV_RELATIVE: + *where = (Elf_Addr)(obj->relocbase + rela->r_addend); + break; + default: + rtld_printf("%s: Unhandled relocation %lu\n", + obj->path, ELF_R_TYPE(rela->r_info)); + return (-1); + } + } + + return (0); +} + +void +allocate_initial_tls(Obj_Entry *objs) +{ + Elf_Addr **tp; + + /* + * Fix the size of the static TLS block by using the maximum + * offset allocated so far and adding a bit for dynamic modules to + * use. + */ + tls_static_space = tls_last_offset + tls_last_size + + RTLD_STATIC_TLS_EXTRA; + + tp = (Elf_Addr **) ((char *)allocate_tls(objs, NULL, TLS_TCB_SIZE, 16) + + TLS_TP_OFFSET + TLS_TCB_SIZE); + + __asm __volatile("mv tp, %0" :: "r"(tp)); +} + +void * +__tls_get_addr(tls_index* ti) +{ + char *_tp; + void *p; + + __asm __volatile("mv %0, tp" : "=r" (_tp)); + + p = tls_get_addr_common((Elf_Addr**)((Elf_Addr)_tp - TLS_TP_OFFSET + - TLS_TCB_SIZE), ti->ti_module, ti->ti_offset); + + return (p + TLS_DTV_OFFSET); +} diff --git a/libexec/rtld-elf/riscv/rtld_machdep.h b/libexec/rtld-elf/riscv/rtld_machdep.h new file mode 100644 index 0000000..aa37000 --- /dev/null +++ b/libexec/rtld-elf/riscv/rtld_machdep.h @@ -0,0 +1,111 @@ +/*- + * Copyright (c) 1999, 2000 John D. Polstra. + * Copyright (c) 2015 Ruslan Bukin <br@bsdpad.com> + * All rights reserved. + * + * Portions of this software were developed by SRI International and the + * University of Cambridge Computer Laboratory under DARPA/AFRL contract + * FA8750-10-C-0237 ("CTSRD"), as part of the DARPA CRASH research programme. + * + * Portions of this software were developed by the University of Cambridge + * Computer Laboratory as part of the CTSRD Project, with support from the + * UK Higher Education Innovation Fund (HEIF). + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef RTLD_MACHDEP_H +#define RTLD_MACHDEP_H 1 + +#include <sys/types.h> +#include <machine/atomic.h> + +struct Struct_Obj_Entry; + +uint64_t set_gp(struct Struct_Obj_Entry *obj); + +/* Return the address of the .dynamic section in the dynamic linker. */ +#define rtld_dynamic(obj) \ +({ \ + Elf_Addr _dynamic_addr; \ + __asm __volatile("lla %0, _DYNAMIC" : "=r"(_dynamic_addr)); \ + (const Elf_Dyn *)_dynamic_addr; \ +}) +#define RTLD_IS_DYNAMIC() (1) + +Elf_Addr reloc_jmpslot(Elf_Addr *where, Elf_Addr target, + const struct Struct_Obj_Entry *defobj, + const struct Struct_Obj_Entry *obj, + const Elf_Rel *rel); + +#define make_function_pointer(def, defobj) \ + ((defobj)->relocbase + (def)->st_value) + +#define call_initfini_pointer(obj, target) \ +({ \ + uint64_t old0; \ + old0 = set_gp(obj); \ + (((InitFunc)(target))()); \ + __asm __volatile("mv gp, %0" :: "r"(old0)); \ +}) + +#define call_init_pointer(obj, target) \ +({ \ + uint64_t old1; \ + old1 = set_gp(obj); \ + (((InitArrFunc)(target))(main_argc, main_argv, environ)); \ + __asm __volatile("mv gp, %0" :: "r"(old1)); \ +}) + +/* + * Lazy binding entry point, called via PLT. + */ +void _rtld_bind_start(void); + +/* + * TLS + */ +#define TLS_TP_OFFSET 0x0 +#define TLS_DTV_OFFSET 0x800 +#define TLS_TCB_SIZE 16 + +#define round(size, align) \ + (((size) + (align) - 1) & ~((align) - 1)) +#define calculate_first_tls_offset(size, align) \ + round(16, align) +#define calculate_tls_offset(prev_offset, prev_size, size, align) \ + round(prev_offset + prev_size, align) +#define calculate_tls_end(off, size) ((off) + (size)) + +typedef struct { + unsigned long ti_module; + unsigned long ti_offset; +} tls_index; + +extern void *__tls_get_addr(tls_index* ti); + +#define RTLD_DEFAULT_STACK_PF_EXEC PF_X +#define RTLD_DEFAULT_STACK_EXEC PROT_EXEC + +#endif diff --git a/libexec/rtld-elf/riscv/rtld_start.S b/libexec/rtld-elf/riscv/rtld_start.S new file mode 100644 index 0000000..76b6dfc --- /dev/null +++ b/libexec/rtld-elf/riscv/rtld_start.S @@ -0,0 +1,129 @@ +/*- + * Copyright (c) 2015 Ruslan Bukin <br@bsdpad.com> + * All rights reserved. + * + * This software was developed by SRI International and the University of + * Cambridge Computer Laboratory under DARPA/AFRL contract FA8750-10-C-0237 + * ("CTSRD"), as part of the DARPA CRASH research programme. + * + * This software was developed by the University of Cambridge Computer + * Laboratory as part of the CTSRD Project, with support from the UK Higher + * Education Innovation Fund (HEIF). + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <machine/asm.h> +__FBSDID("$FreeBSD$"); + +/* + * func_ptr_type + * _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entry **objp) + */ + +ENTRY(.rtld_start) + mv s0, a0 /* Put ps_strings in a callee-saved register */ + mv s1, sp /* And the stack pointer */ + + addi sp, sp, -16 /* Make room for obj_main & exit proc */ + + mv a1, sp /* exit_proc */ + addi a2, a1, 8 /* obj_main */ + jal _rtld /* Call the loader */ + mv t0, a0 /* Backup the entry point */ + + ld a2, 0(sp) /* Load cleanup */ + ld a1, 8(sp) /* Load obj_main */ + mv a0, s0 /* Restore ps_strings */ + mv sp, s1 /* Restore the stack pointer */ + jr t0 /* Jump to the entry point */ +END(.rtld_start) + +/* + * t0 = obj pointer + * t1 = reloc offset + */ +ENTRY(_rtld_bind_start) + /* Save the arguments and ra */ + addi sp, sp, -(8 * 25) + sd a0, (8 * 0)(sp) + sd a1, (8 * 1)(sp) + sd a2, (8 * 2)(sp) + sd a3, (8 * 3)(sp) + sd a4, (8 * 4)(sp) + sd a5, (8 * 5)(sp) + sd a6, (8 * 6)(sp) + sd a7, (8 * 7)(sp) + sd ra, (8 * 8)(sp) +#if 0 + /* RISCVTODO VFP */ + /* Save any floating-point arguments */ + fsq fa0, (8 * 9)(sp) + fsq fa1, (8 * 11)(sp) + fsq fa2, (8 * 13)(sp) + fsq fa3, (8 * 15)(sp) + fsq fa4, (8 * 17)(sp) + fsq fa5, (8 * 19)(sp) + fsq fa6, (8 * 21)(sp) + fsq fa7, (8 * 23)(sp) +#endif + + /* Reloc offset is 3x of the .got.plt offset */ + slli a1, t1, 1 /* Mult items by 2 */ + add a1, a1, t1 /* Plus item */ + + /* Load obj */ + mv a0, t0 + + /* Call into rtld */ + jal _rtld_bind + + /* Backup the address to branch to */ + mv t0, a0 + + /* Restore the arguments and ra */ + ld a0, (8 * 0)(sp) + ld a1, (8 * 1)(sp) + ld a2, (8 * 2)(sp) + ld a3, (8 * 3)(sp) + ld a4, (8 * 4)(sp) + ld a5, (8 * 5)(sp) + ld a6, (8 * 6)(sp) + ld a7, (8 * 7)(sp) + ld ra, (8 * 8)(sp) +#if 0 + /* RISCVTODO VFP */ + /* Restore floating-point arguments */ + flq fa0, (8 * 9)(sp) + flq fa1, (8 * 11)(sp) + flq fa2, (8 * 13)(sp) + flq fa3, (8 * 15)(sp) + flq fa4, (8 * 17)(sp) + flq fa5, (8 * 19)(sp) + flq fa6, (8 * 21)(sp) + flq fa7, (8 * 23)(sp) +#endif + addi sp, sp, (8 * 25) + + /* Call into the correct function */ + jr t0 +END(_rtld_bind_start) diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c index c2e165e..4eee6c5 100644 --- a/libexec/rtld-elf/rtld.c +++ b/libexec/rtld-elf/rtld.c @@ -204,6 +204,8 @@ extern Elf_Dyn _DYNAMIC; #define RTLD_IS_DYNAMIC() (&_DYNAMIC != NULL) #endif +#define _LD(x) LD_ x + int dlclose(void *) __exported; char *dlerror(void) __exported; void *dlopen(const char *, int) __exported; @@ -417,7 +419,7 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entry **objp) trust = !issetugid(); - ld_bind_now = getenv(LD_ "BIND_NOW"); + ld_bind_now = getenv(_LD("BIND_NOW")); /* * If the process is tainted, then we un-set the dangerous environment * variables. The process will be marked as tainted until setuid(2) @@ -425,24 +427,24 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entry **objp) * future processes to honor the potentially un-safe variables. */ if (!trust) { - if (unsetenv(LD_ "PRELOAD") || unsetenv(LD_ "LIBMAP") || - unsetenv(LD_ "LIBRARY_PATH") || unsetenv(LD_ "LIBRARY_PATH_FDS") || - unsetenv(LD_ "LIBMAP_DISABLE") || - unsetenv(LD_ "DEBUG") || unsetenv(LD_ "ELF_HINTS_PATH") || - unsetenv(LD_ "LOADFLTR") || unsetenv(LD_ "LIBRARY_PATH_RPATH")) { + if (unsetenv(_LD("PRELOAD")) || unsetenv(_LD("LIBMAP")) || + unsetenv(_LD("LIBRARY_PATH")) || unsetenv(_LD("LIBRARY_PATH_FDS")) || + unsetenv(_LD("LIBMAP_DISABLE")) || + unsetenv(_LD("DEBUG")) || unsetenv(_LD("ELF_HINTS_PATH")) || + unsetenv(_LD("LOADFLTR")) || unsetenv(_LD("LIBRARY_PATH_RPATH"))) { _rtld_error("environment corrupt; aborting"); rtld_die(); } } - ld_debug = getenv(LD_ "DEBUG"); - libmap_disable = getenv(LD_ "LIBMAP_DISABLE") != NULL; - libmap_override = getenv(LD_ "LIBMAP"); - ld_library_path = getenv(LD_ "LIBRARY_PATH"); - ld_library_dirs = getenv(LD_ "LIBRARY_PATH_FDS"); - ld_preload = getenv(LD_ "PRELOAD"); - ld_elf_hints_path = getenv(LD_ "ELF_HINTS_PATH"); - ld_loadfltr = getenv(LD_ "LOADFLTR") != NULL; - library_path_rpath = getenv(LD_ "LIBRARY_PATH_RPATH"); + ld_debug = getenv(_LD("DEBUG")); + libmap_disable = getenv(_LD("LIBMAP_DISABLE")) != NULL; + libmap_override = getenv(_LD("LIBMAP")); + ld_library_path = getenv(_LD("LIBRARY_PATH")); + ld_library_dirs = getenv(_LD("LIBRARY_PATH_FDS")); + ld_preload = getenv(_LD("PRELOAD")); + ld_elf_hints_path = getenv(_LD("ELF_HINTS_PATH")); + ld_loadfltr = getenv(_LD("LOADFLTR")) != NULL; + library_path_rpath = getenv(_LD("LIBRARY_PATH_RPATH")); if (library_path_rpath != NULL) { if (library_path_rpath[0] == 'y' || library_path_rpath[0] == 'Y' || @@ -454,8 +456,8 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entry **objp) dangerous_ld_env = libmap_disable || (libmap_override != NULL) || (ld_library_path != NULL) || (ld_preload != NULL) || (ld_elf_hints_path != NULL) || ld_loadfltr; - ld_tracing = getenv(LD_ "TRACE_LOADED_OBJECTS"); - ld_utrace = getenv(LD_ "UTRACE"); + ld_tracing = getenv(_LD("TRACE_LOADED_OBJECTS")); + ld_utrace = getenv(_LD("UTRACE")); if ((ld_elf_hints_path == NULL) || strlen(ld_elf_hints_path) == 0) ld_elf_hints_path = ld_elf_hints_default; @@ -592,7 +594,7 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entry **objp) exit(0); } - if (getenv(LD_ "DUMP_REL_PRE") != NULL) { + if (getenv(_LD("DUMP_REL_PRE")) != NULL) { dump_relocations(obj_main); exit (0); } @@ -620,7 +622,7 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entry **objp) if (do_copy_relocations(obj_main) == -1) rtld_die(); - if (getenv(LD_ "DUMP_REL_POST") != NULL) { + if (getenv(_LD("DUMP_REL_POST")) != NULL) { dump_relocations(obj_main); exit (0); } @@ -1126,7 +1128,6 @@ digest_dynamic1(Obj_Entry *obj, int early, const Elf_Dyn **dyn_rpath, #ifndef __mips__ case DT_DEBUG: - /* XXX - not implemented yet */ if (!early) dbg("Filling in DT_DEBUG entry"); ((Elf_Dyn*)dynp)->d_un.d_ptr = (Elf_Addr) &r_debug; @@ -1362,22 +1363,22 @@ digest_notes(Obj_Entry *obj, Elf_Addr note_start, Elf_Addr note_end) if (note->n_namesz != sizeof(NOTE_FREEBSD_VENDOR) || note->n_descsz != sizeof(int32_t)) continue; - if (note->n_type != ABI_NOTETYPE && - note->n_type != CRT_NOINIT_NOTETYPE) + if (note->n_type != NT_FREEBSD_ABI_TAG && + note->n_type != NT_FREEBSD_NOINIT_TAG) continue; note_name = (const char *)(note + 1); if (strncmp(NOTE_FREEBSD_VENDOR, note_name, sizeof(NOTE_FREEBSD_VENDOR)) != 0) continue; switch (note->n_type) { - case ABI_NOTETYPE: + case NT_FREEBSD_ABI_TAG: /* FreeBSD osrel note */ p = (uintptr_t)(note + 1); p += roundup2(note->n_namesz, sizeof(Elf32_Addr)); obj->osrel = *(const int32_t *)(p); dbg("note osrel %d", obj->osrel); break; - case CRT_NOINIT_NOTETYPE: + case NT_FREEBSD_NOINIT_TAG: /* FreeBSD 'crt does not call init' note */ obj->crt_no_init = true; dbg("note crt_no_init"); @@ -4178,16 +4179,16 @@ trace_loaded_objects(Obj_Entry *obj) char *fmt1, *fmt2, *fmt, *main_local, *list_containers; int c; - if ((main_local = getenv(LD_ "TRACE_LOADED_OBJECTS_PROGNAME")) == NULL) + if ((main_local = getenv(_LD("TRACE_LOADED_OBJECTS_PROGNAME"))) == NULL) main_local = ""; - if ((fmt1 = getenv(LD_ "TRACE_LOADED_OBJECTS_FMT1")) == NULL) + if ((fmt1 = getenv(_LD("TRACE_LOADED_OBJECTS_FMT1"))) == NULL) fmt1 = "\t%o => %p (%x)\n"; - if ((fmt2 = getenv(LD_ "TRACE_LOADED_OBJECTS_FMT2")) == NULL) + if ((fmt2 = getenv(_LD("TRACE_LOADED_OBJECTS_FMT2"))) == NULL) fmt2 = "\t%o (%x)\n"; - list_containers = getenv(LD_ "TRACE_LOADED_OBJECTS_ALL"); + list_containers = getenv(_LD("TRACE_LOADED_OBJECTS_ALL")); for (; obj; obj = obj->next) { Needed_Entry *needed; @@ -4396,7 +4397,7 @@ tls_get_addr_common(Elf_Addr **dtvp, int index, size_t offset) } #if defined(__aarch64__) || defined(__arm__) || defined(__mips__) || \ - defined(__powerpc__) + defined(__powerpc__) || defined(__riscv__) /* * Allocate Static TLS using the Variant I method. diff --git a/sbin/gbde/Makefile b/sbin/gbde/Makefile index c33136b..f80f8cd 100644 --- a/sbin/gbde/Makefile +++ b/sbin/gbde/Makefile @@ -4,7 +4,7 @@ PROG= gbde SRCS= gbde.c template.c SRCS+= rijndael-alg-fst.c SRCS+= rijndael-api-fst.c -SRCS+= sha2.c +SRCS+= sha512c.c SRCS+= g_bde_lock.c # rijndael-fst.c does evil casting things which can results in warnings, diff --git a/sbin/gbde/gbde.c b/sbin/gbde/gbde.c index 3dca212..710ae2c 100644 --- a/sbin/gbde/gbde.c +++ b/sbin/gbde/gbde.c @@ -84,7 +84,7 @@ #include <sys/disk.h> #include <sys/stat.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha512.h> #include <sys/param.h> #include <sys/linker.h> diff --git a/sbin/geom/class/eli/Makefile b/sbin/geom/class/eli/Makefile index f8e453d..50de651 100644 --- a/sbin/geom/class/eli/Makefile +++ b/sbin/geom/class/eli/Makefile @@ -6,7 +6,8 @@ GEOM_CLASS= eli SRCS= g_eli_crypto.c SRCS+= g_eli_key.c SRCS+= pkcs5v2.c -SRCS+= sha2.c +SRCS+= sha256c.c +SRCS+= sha512c.c LIBADD= md crypto diff --git a/sbin/ifconfig/Makefile b/sbin/ifconfig/Makefile index ed9c205..b5616a4 100644 --- a/sbin/ifconfig/Makefile +++ b/sbin/ifconfig/Makefile @@ -38,7 +38,7 @@ SRCS+= sfp.c # SFP/SFP+ information LIBADD+= m SRCS+= ifieee80211.c # SIOC[GS]IEEE80211 support -LIBADD+= bsdxml sbuf 80211 +LIBADD+= 80211 SRCS+= carp.c # SIOC[GS]VH support SRCS+= ifgroup.c # ... diff --git a/sbin/ifconfig/sfp.c b/sbin/ifconfig/sfp.c index 7c090e1..306140a 100644 --- a/sbin/ifconfig/sfp.c +++ b/sbin/ifconfig/sfp.c @@ -171,7 +171,7 @@ static struct _nv fc_speed[] = { /* 10/40G Ethernet compliance codes, byte 128 + 3 */ static struct _nv eth_1040g[] = { - { 0x80, "Reserved" }, + { 0x80, "Extended" }, { 0x40, "10GBASE-LRM" }, { 0x20, "10GBASE-LR" }, { 0x10, "10GBASE-SR" }, @@ -181,6 +181,38 @@ static struct _nv eth_1040g[] = { { 0x01, "40G Active Cable" }, { 0, NULL } }; +#define SFF_8636_EXT_COMPLIANCE 0x80 + +/* SFF-8024 Rev. 3.4 table 4.4: Extended Specification Compliance */ +static struct _nv eth_extended_comp[] = { + { 0xFF, "Reserved" }, + { 0x1A, "2 lambda DWDM 100G" }, + { 0x19, "100G ACC or 25GAUI C2M ACC" }, + { 0x18, "100G AOC or 25GAUI C2M AOC" }, + { 0x17, "100G CLR4" }, + { 0x16, "10GBASE-T with SFI electrical interface" }, + { 0x15, "G959.1 profile P1L1-2D2" }, + { 0x14, "G959.1 profile P1S1-2D2" }, + { 0x13, "G959.1 profile P1I1-2D1" }, + { 0x12, "40G PSM4 Parallel SMF" }, + { 0x11, "4 x 10GBASE-SR" }, + { 0x10, "40GBASE-ER4" }, + { 0x0F, "Reserved" }, + { 0x0D, "25GBASE-CR CA-N" }, + { 0x0C, "25GBASE-CR CA-S" }, + { 0x0B, "100GBASE-CR4 or 25GBASE-CR CA-L" }, + { 0x0A, "Reserved" }, + { 0x09, "100G CWDM4 MSA without FEC" }, + { 0x08, "100G ACC (Active Copper Cable)" }, + { 0x07, "100G PSM4 Parallel SMF" }, + { 0x06, "100G CWDM4 MSA with FEC" }, + { 0x05, "100GBASE-SR10" }, + { 0x04, "100GBASE-ER4" }, + { 0x03, "100GBASE-LR4" }, + { 0x02, "100GBASE-SR4" }, + { 0x01, "100G AOC (Active Optical Cable) or 25GAUI C2M ACC" }, + { 0x00, "Unspecified" } +}; /* SFF-8636 Rev. 2.5 table 6.3: Revision compliance */ static struct _nv rev_compl[] = { @@ -371,9 +403,16 @@ get_qsfp_transceiver_class(struct i2c_info *ii, char *buf, size_t size) const char *tech_class; uint8_t code; - /* Check 10/40G Ethernet class only */ - read_i2c(ii, SFF_8436_BASE, SFF_8436_CODE_E1040G, 1, &code); - tech_class = find_zero_bit(eth_1040g, code, 1); + read_i2c(ii, SFF_8436_BASE, SFF_8436_CODE_E1040100G, 1, &code); + + /* Check for extended specification compliance */ + if (code & SFF_8636_EXT_COMPLIANCE) { + read_i2c(ii, SFF_8436_BASE, SFF_8436_OPTIONS_START, 1, &code); + tech_class = find_value(eth_extended_comp, code); + } else + /* Check 10/40G Ethernet class only */ + tech_class = find_zero_bit(eth_1040g, code, 1); + if (tech_class == NULL) tech_class = "Unknown"; @@ -873,6 +912,7 @@ sfp_status(int s, struct ifreq *ifr, int verbose) switch (id_byte) { case SFF_8024_ID_QSFP: case SFF_8024_ID_QSFPPLUS: + case SFF_8024_ID_QSFP28: print_qsfp_status(&ii, verbose); break; default: diff --git a/sbin/md5/Makefile b/sbin/md5/Makefile index b6afe4e..5f927c4 100644 --- a/sbin/md5/Makefile +++ b/sbin/md5/Makefile @@ -6,11 +6,13 @@ PROG= md5 LINKS= ${BINDIR}/md5 ${BINDIR}/rmd160 \ ${BINDIR}/md5 ${BINDIR}/sha1 \ ${BINDIR}/md5 ${BINDIR}/sha256 \ + ${BINDIR}/md5 ${BINDIR}/sha384 \ ${BINDIR}/md5 ${BINDIR}/sha512 MLINKS= md5.1 rmd160.1 \ md5.1 sha1.1 \ md5.1 sha256.1 \ + md5.1 sha384.1 \ md5.1 sha512.1 LIBADD= md diff --git a/sbin/md5/md5.1 b/sbin/md5/md5.1 index e191cd1..94cdf84 100644 --- a/sbin/md5/md5.1 +++ b/sbin/md5/md5.1 @@ -1,9 +1,9 @@ .\" $FreeBSD$ -.Dd May 17, 2014 +.Dd October 17, 2015 .Dt MD5 1 .Os .Sh NAME -.Nm md5 , sha1 , sha256 , sha512, rmd160 +.Nm md5 , sha1 , sha256 , sha384 , sha512, rmd160 .Nd calculate a message-digest fingerprint (checksum) for a file .Sh SYNOPSIS .Nm md5 @@ -21,6 +21,11 @@ .Op Fl c Ar string .Op Fl s Ar string .Op Ar +.Nm sha384 +.Op Fl pqrtx +.Op Fl c Ar string +.Op Fl s Ar string +.Op Ar .Nm sha512 .Op Fl pqrtx .Op Fl c Ar string @@ -33,7 +38,7 @@ .Op Ar .Sh DESCRIPTION The -.Nm md5 , sha1 , sha256 , sha512 +.Nm md5 , sha1 , sha256 , sha384 , sha512 and .Nm rmd160 utilities take as input a message of arbitrary length and produce as @@ -46,7 +51,7 @@ It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. The -.Tn MD5 , SHA-1 , SHA-256 , SHA-512 +.Tn MD5 , SHA-1 , SHA-256 , SHA-384 , SHA-512 and .Tn RIPEMD-160 algorithms are intended for digital signature applications, where a @@ -123,6 +128,7 @@ option. .Xr ripemd 3 , .Xr sha 3 , .Xr sha256 3 , +.Xr sha384 3 , .Xr sha512 3 .Rs .%A R. Rivest diff --git a/sbin/md5/md5.c b/sbin/md5/md5.c index f4c56ac..1172f25 100644 --- a/sbin/md5/md5.c +++ b/sbin/md5/md5.c @@ -28,6 +28,7 @@ __FBSDID("$FreeBSD$"); #include <ripemd.h> #include <sha.h> #include <sha256.h> +#include <sha384.h> #include <sha512.h> #include <stdio.h> #include <stdlib.h> @@ -55,6 +56,7 @@ typedef char *(DIGEST_End)(void *, char *); extern const char *MD5TestOutput[MDTESTCOUNT]; extern const char *SHA1_TestOutput[MDTESTCOUNT]; extern const char *SHA256_TestOutput[MDTESTCOUNT]; +extern const char *SHA384_TestOutput[MDTESTCOUNT]; extern const char *SHA512_TestOutput[MDTESTCOUNT]; extern const char *RIPEMD160_TestOutput[MDTESTCOUNT]; @@ -80,6 +82,7 @@ typedef union { MD5_CTX md5; SHA1_CTX sha1; SHA256_CTX sha256; + SHA384_CTX sha384; SHA512_CTX sha512; RIPEMD160_CTX ripemd160; } DIGEST_CTX; @@ -101,6 +104,9 @@ static const struct Algorithm_t Algorithm[] = { { "sha256", "SHA256", &SHA256_TestOutput, (DIGEST_Init*)&SHA256_Init, (DIGEST_Update*)&SHA256_Update, (DIGEST_End*)&SHA256_End, &SHA256_Data, &SHA256_File }, + { "sha384", "SHA384", &SHA384_TestOutput, (DIGEST_Init*)&SHA384_Init, + (DIGEST_Update*)&SHA384_Update, (DIGEST_End*)&SHA384_End, + &SHA384_Data, &SHA384_File }, { "sha512", "SHA512", &SHA512_TestOutput, (DIGEST_Init*)&SHA512_Init, (DIGEST_Update*)&SHA512_Update, (DIGEST_End*)&SHA512_End, &SHA512_Data, &SHA512_File }, @@ -327,6 +333,17 @@ const char *SHA256_TestOutput[MDTESTCOUNT] = { "e6eae09f10ad4122a0e2a4075761d185a272ebd9f5aa489e998ff2f09cbfdd9f" }; +const char *SHA384_TestOutput[MDTESTCOUNT] = { + "38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b", + "54a59b9f22b0b80880d8427e548b7c23abd873486e1f035dce9cd697e85175033caa88e6d57bc35efae0b5afd3145f31", + "cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7", + "473ed35167ec1f5d8e550368a3db39be54639f828868e9454c239fc8b52e3c61dbd0d8b4de1390c256dcbb5d5fd99cd5", + "feb67349df3db6f5924815d6c3dc133f091809213731fe5c7b5f4999e463479ff2877f5f2936fa63bb43784b12f3ebb4", + "1761336e3f7cbfe51deb137f026f89e01a448e3b1fafa64039c1464ee8732f11a5341a6f41e0c202294736ed64db1a84", + "b12932b0627d1c060942f5447764155655bd4da0c9afa6dd9b9ef53129af1b8fb0195996d2de9ca0df9d821ffee67026", + "99428d401bf4abcd4ee0695248c9858b7503853acfae21a9cffa7855f46d1395ef38596fcd06d5a8c32d41a839cc5dfb" +}; + const char *SHA512_TestOutput[MDTESTCOUNT] = { "cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e", "1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75", diff --git a/sbin/mount/mount.c b/sbin/mount/mount.c index 6427fc8..debe354 100644 --- a/sbin/mount/mount.c +++ b/sbin/mount/mount.c @@ -485,10 +485,18 @@ ismounted(struct fstab *fs, struct statfs *mntbuf, int mntsize) strlcpy(realfsfile, fs->fs_file, sizeof(realfsfile)); } + /* + * Consider the filesystem to be mounted if: + * It has the same mountpoint as a mounted filesytem, and + * It has the same type as that same mounted filesystem, and + * It has the same device name as that same mounted filesystem, OR + * It is a nonremountable filesystem + */ for (i = mntsize - 1; i >= 0; --i) if (strcmp(realfsfile, mntbuf[i].f_mntonname) == 0 && + strcmp(fs->fs_vfstype, mntbuf[i].f_fstypename) == 0 && (!isremountable(fs->fs_vfstype) || - strcmp(fs->fs_spec, mntbuf[i].f_mntfromname) == 0)) + (strcmp(fs->fs_spec, mntbuf[i].f_mntfromname) == 0))) return (1); return (0); } @@ -533,7 +541,7 @@ append_arg(struct cpa *sa, char *arg) { if (sa->c + 1 == sa->sz) { sa->sz = sa->sz == 0 ? 8 : sa->sz * 2; - sa->a = realloc(sa->a, sizeof(sa->a) * sa->sz); + sa->a = realloc(sa->a, sizeof(*sa->a) * sa->sz); if (sa->a == NULL) errx(1, "realloc failed"); } diff --git a/sbin/umount/umount.c b/sbin/umount/umount.c index 521bbc8..6cd8ea7 100644 --- a/sbin/umount/umount.c +++ b/sbin/umount/umount.c @@ -434,7 +434,7 @@ getmntentry(const char *fromname, const char *onname, fsid_t *fsid, dowhat what) { static struct statfs *mntbuf; static size_t mntsize = 0; - static char *mntcheck = NULL; + static int *mntcheck = NULL; struct statfs *sfs, *foundsfs; int i, count; diff --git a/share/examples/tests/tests/atf/printf_test.c b/share/examples/tests/tests/atf/printf_test.c index 04a5665..719927c 100644 --- a/share/examples/tests/tests/atf/printf_test.c +++ b/share/examples/tests/tests/atf/printf_test.c @@ -152,4 +152,6 @@ ATF_TP_ADD_TCS(tp) ATF_TP_ADD_TC(tp, snprintf__two_formatters); ATF_TP_ADD_TC(tp, snprintf__overflow); ATF_TP_ADD_TC(tp, fprintf__simple_string); + + return (atf_no_error()); } diff --git a/share/man/man4/Makefile b/share/man/man4/Makefile index 1807876..6cf392a 100644 --- a/share/man/man4/Makefile +++ b/share/man/man4/Makefile @@ -266,6 +266,7 @@ MAN= aac.4 \ malo.4 \ mcd.4 \ md.4 \ + mdio.4 \ me.4 \ mem.4 \ meteor.4 \ diff --git a/share/man/man4/mdio.4 b/share/man/man4/mdio.4 new file mode 100644 index 0000000..1a8bdd0 --- /dev/null +++ b/share/man/man4/mdio.4 @@ -0,0 +1,53 @@ +.\" Written by Landon Fuller for the FreeBSD Project. +.\" Based on the miibus(4) manual page written by Tom Rhodes. +.\" Please see the /usr/src/COPYRIGHT file for copyright information. +.\" +.\" $FreeBSD$ +.\" +.Dd December 17, 2015 +.Dt MDIO 4 +.Os +.Sh NAME +.Nm mdio +.Nd IEEE 802.3 Management Data Input/Output interface +.Sh SYNOPSIS +To compile this driver into the kernel, +place the following line in your +kernel configuration file: +.Bd -ragged -offset indent +.Cd "device mdio" +.Ed +.Sh DESCRIPTION +The +.Nm +driver provides an interconnection between the Media Access Control (MAC) +sublayer and Physical Layer (PHY) entities' control and status registers, +as defined by the IEEE 802.3 Standard. +.Pp +The +.Nm +layer allows device drivers to share common support code for various +external PHY devices. +.Pp +.Tn MDIO +is one of two signal interfaces that comprise the +Media Independent Interface (MII) defined by the IEEE 802.3 +Standard. The +.Xr miibus 4 +driver provides support for devices that require full +.Tn MII +support. +.Sh SEE ALSO +.Xr miibus 4 +.Sh STANDARDS +More information on +.Tn MDIO +can be found in the IEEE 802.3 Standard. +.Sh HISTORY +The +.Nm +driver first appeared in +.Fx 10.0 . +.Sh AUTHORS +The driver was written by +.An Stefan Bethke Aq Mt stb@lassitu.de . diff --git a/share/man/man5/procfs.5 b/share/man/man5/procfs.5 index 684f9f3..e1791a33c 100644 --- a/share/man/man5/procfs.5 +++ b/share/man/man5/procfs.5 @@ -2,7 +2,7 @@ .\" Written by Garrett Wollman .\" This file is in the public domain. .\" -.Dd April 22, 2013 +.Dd December 26, 2015 .Dt PROCFS 5 .Os .Sh NAME @@ -17,11 +17,7 @@ The process file system, or .Nm , implements a view of the system process table inside the file system. It is normally mounted on -.Pa /proc , -and is required for the complete operation of programs such as -.Xr ps 1 -and -.Xr w 1 . +.Pa /proc . .Pp The .Nm diff --git a/share/man/man9/Makefile b/share/man/man9/Makefile index 30cb503..3cd6f7f 100644 --- a/share/man/man9/Makefile +++ b/share/man/man9/Makefile @@ -1287,6 +1287,8 @@ MLINKS+=pci.9 pci_alloc_msi.9 \ pci.9 pci_iov_detach.9 \ pci.9 pci_msi_count.9 \ pci.9 pci_msix_count.9 \ + pci.9 pci_msix_pba_bar.9 \ + pci.9 pci_msix_table_bar.9 \ pci.9 pci_pending_msix.9 \ pci.9 pci_read_config.9 \ pci.9 pci_release_msi.9 \ @@ -1911,6 +1913,7 @@ MLINKS+=zone.9 uma.9 \ zone.9 uma_zone_get_cur.9 \ zone.9 uma_zone_get_max.9 \ zone.9 uma_zone_set_max.9 \ - zone.9 uma_zone_set_warning.9 + zone.9 uma_zone_set_warning.9 \ + zone.9 uma_zone_set_maxaction.9 .include <bsd.prog.mk> diff --git a/share/man/man9/pci.9 b/share/man/man9/pci.9 index 4faa5f2..ac7b28b 100644 --- a/share/man/man9/pci.9 +++ b/share/man/man9/pci.9 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd November 5, 2015 +.Dd December 23, 2015 .Dt PCI 9 .Os .Sh NAME @@ -51,6 +51,8 @@ .Nm pci_iov_detach , .Nm pci_msi_count , .Nm pci_msix_count , +.Nm pci_msix_pba_bar , +.Nm pci_msix_table_bar , .Nm pci_pending_msix , .Nm pci_read_config , .Nm pci_release_msi , @@ -107,6 +109,10 @@ .Ft int .Fn pci_msix_count "device_t dev" .Ft int +.Fn pci_msix_pba_bar "device_t dev" +.Ft int +.Fn pci_msix_table_bar "device_t dev" +.Ft int .Fn pci_pending_msix "device_t dev" "u_int index" .Ft uint32_t .Fn pci_read_config "device_t dev" "int reg" "int width" @@ -694,6 +700,37 @@ then returns zero. .Pp The +.Fn pci_msix_pba_bar +function returns the offset in configuration space of the Base Address Register +.Pq BAR +containing the MSI-X Pending Bit Array (PBA) for device +.Fa dev . +The returned value can be used as the resource ID with +.Xr bus_alloc_resource 9 +and +.Xr bus_release_resource 9 +to allocate the BAR. +If the device does not support MSI-X, +then +.Fn pci_msix_pba_bar +returns -1. +.Pp +The +.Fn pci_msix_table_bar +function returns the offset in configuration space of the BAR +containing the MSI-X vector table for device +.Fa dev . +The returned value can be used as the resource ID with +.Xr bus_alloc_resource 9 +and +.Xr bus_release_resource 9 +to allocate the BAR. +If the device does not support MSI-X, +then +.Fn pci_msix_table_bar +returns -1. +.Pp +The .Fn pci_alloc_msix function attempts to allocate .Fa *count @@ -732,12 +769,21 @@ it returns an error. Unlike MSI, MSI-X does not require message counts that are powers of two. .Pp +The BARs containing the MSI-X vector table and PBA must be +allocated via +.Xr bus_alloc_resource 9 +before calling +.Fn pci_alloc_msix +and must not be released until after calling +.Fn pci_release_msi . +Note that the vector table and PBA may be stored in the same BAR or in +different BARs. +.Pp The .Fn pci_pending_msix function examines the .Fa dev -device's Pending Bit Array -.Pq PBA +device's PBA to determine the pending status of the MSI-X message at table index .Fa index . If the indicated message is pending, @@ -853,3 +899,6 @@ These do not refer to the geographic location of PCI devices, but to the device number assigned by the combination of the PCI IDSEL mechanism and the platform firmware. This should be taken note of when working with the kernel PCI code. +.Pp +The PCI bus driver should allocate the MSI-X vector table and PBA internally +as necessary rather than requiring the caller to do so. diff --git a/share/man/man9/zone.9 b/share/man/man9/zone.9 index 2df14b02..12226d1 100644 --- a/share/man/man9/zone.9 +++ b/share/man/man9/zone.9 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd February 7, 2014 +.Dd December 20, 2015 .Dt ZONE 9 .Os .Sh NAME @@ -39,7 +39,8 @@ .Nm uma_zone_set_max, .Nm uma_zone_get_max, .Nm uma_zone_get_cur, -.Nm uma_zone_set_warning +.Nm uma_zone_set_warning, +.Nm uma_zone_set_maxaction .Nd zone allocator .Sh SYNOPSIS .In sys/param.h @@ -71,6 +72,8 @@ .Fn uma_zone_get_cur "uma_zone_t zone" .Ft void .Fn uma_zone_set_warning "uma_zone_t zone" "const char *warning" +.Ft void +.Fn uma_zone_set_maxaction "uma_zone_t zone" "void (*maxaction)(uma_zone_t)" .In sys/sysctl.h .Fn SYSCTL_UMA_MAX parent nbr name access zone descr .Fn SYSCTL_ADD_UMA_MAX ctx parent nbr name access zone descr @@ -307,13 +310,21 @@ The .Fn uma_zone_set_warning function sets a warning that will be printed on the system console when the given zone becomes full and fails to allocate an item. -The warning will be printed not often than every five minutes. +The warning will be printed no more often than every five minutes. Warnings can be turned off globally by setting the .Va vm.zone_warnings sysctl tunable to .Va 0 . .Pp The +.Fn uma_zone_set_maxaction +function sets a function that will be called when the given zone becomes full +and fails to allocate an item. +The function will be called with the zone locked. Also, the function +that called the allocation function may have held additional locks. Therefore, +this function should do very little work (similar to a signal handler). +.Pp +The .Fn SYSCTL_UMA_MAX parent nbr name access zone descr macro declares a static .Xr sysctl diff --git a/share/misc/committers-ports.dot b/share/misc/committers-ports.dot index aac0ade..15f3072 100644 --- a/share/misc/committers-ports.dot +++ b/share/misc/committers-ports.dot @@ -349,6 +349,7 @@ erwin -> lbr erwin -> lth erwin -> simon +feld -> brnrd feld -> junovitch fjoe -> danfe @@ -584,8 +585,6 @@ thierry -> riggs tmclaugh -> itetcu tmclaugh -> xride -vsevolod -> brnrd - wen -> cs wen -> culot wen -> pawel diff --git a/share/misc/organization.dot b/share/misc/organization.dot index 23ef52e..863efc3 100644 --- a/share/misc/organization.dot +++ b/share/misc/organization.dot @@ -34,7 +34,7 @@ portmgr [label="Port Management Team\nportmgr@FreeBSD.org\nantoine, bapt, bdrewe portmgrsecretary [label="Port Management Team Secretary\nportmgr-secretary@FreeBSD.org\nculot"] re [label="Primary Release Engineering Team\nre@FreeBSD.org\nkib, blackend, jpaetzel, hrs, kensmith"] secteam [label="Security Team\nsecteam@FreeBSD.org\nsimon, qingli, delphij,\nremko, philip, stas, cperciva,\ncsjp, rwatson, miwi, bz"] -portssecteam [label="Ports Security Team\nports-secteam@FreeBSD.org\ndelphij, eadler, feld, jgh, rea, sbz, simon, swills, zi"] +portssecteam [label="Ports Security Team\nports-secteam@FreeBSD.org\ndelphij, eadler, feld, jgh, miwi, rea, sbz, simon, swills, zi"] secteamsecretary [label="Security Team Secretary\nsecteam-secretary@FreeBSD.org\nremko"] securityofficer [label="Security Officer Team\nsecurity-officer@FreeBSD.org\ncperciva, simon, nectar"] srccommitters [label="Src Committers\nsrc-committers@FreeBSD.org"] diff --git a/share/mk/bsd.README b/share/mk/bsd.README index 527de95..3e3dcd3 100644 --- a/share/mk/bsd.README +++ b/share/mk/bsd.README @@ -492,6 +492,10 @@ KYUAFILE If 'auto' (the default), generate a Kyuafile out of the subdirectories providing helper programs or data files only). +LOCALBASE The --prefix for the kyua package. + + The value of LOCALBASE defaults to /usr/local . + ATF_TESTS_C The names of the ATF C test programs to build. ATF_TESTS_CXX The names of the ATF C++ test programs to build. diff --git a/share/mk/bsd.lib.mk b/share/mk/bsd.lib.mk index 6e26cff..2e1796f 100644 --- a/share/mk/bsd.lib.mk +++ b/share/mk/bsd.lib.mk @@ -368,6 +368,9 @@ _libinstall: ${INSTALL} -S -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} ${SHLIB_LINK:R}.ld \ ${DESTDIR}${_LIBDIR}/${SHLIB_LINK} +.for _SHLIB_LINK_LINK in ${SHLIB_LDSCRIPT_LINKS} + ${INSTALL_SYMLINK} ${SHLIB_LINK} ${DESTDIR}${_LIBDIR}/${_SHLIB_LINK_LINK} +.endfor .else .if ${_SHLIBDIR} == ${_LIBDIR} ${INSTALL_SYMLINK} ${SHLIB_NAME} ${DESTDIR}${_LIBDIR}/${SHLIB_LINK} diff --git a/share/mk/bsd.test.mk b/share/mk/bsd.test.mk index 8102e8a..6e68d9e 100644 --- a/share/mk/bsd.test.mk +++ b/share/mk/bsd.test.mk @@ -10,6 +10,9 @@ __<bsd.test.mk>__: +# Third-party software (kyua, etc) prefix. +LOCALBASE?= /usr/local + # Tests install directory TESTSDIR?= ${TESTSBASE}/${RELDIR:H} diff --git a/share/mk/suite.test.mk b/share/mk/suite.test.mk index 0ffffba..606514b 100644 --- a/share/mk/suite.test.mk +++ b/share/mk/suite.test.mk @@ -45,31 +45,21 @@ KYUAFILE?= auto # unqualified TEST_METADATA variable. #TEST_METADATA.<test-program>+= key="value" -# Path to the prefix of the installed Kyua CLI, if any. -# -# If kyua is installed from ports, we automatically define a realcheck target -# below to run the tests using this tool. The tools are searched for in the -# hierarchy specified by this variable. -KYUA_PREFIX?= /usr/local - -.if ${KYUAFILE:tl} == "yes" +.if ${KYUAFILE:tl} != "no" FILES+= Kyuafile FILESDIR_Kyuafile= ${TESTSDIR} +.endif -CLEANFILES+= Kyuafile.auto Kyuafile.auto.tmp -.elif ${KYUAFILE:tl} == "auto" -FILES+= Kyuafile.auto -FILESDIR_Kyuafile.auto= ${TESTSDIR} -FILESNAME_Kyuafile.auto= Kyuafile - -CLEANFILES+= Kyuafile.auto Kyuafile.auto.tmp +.if ${KYUAFILE:tl} == "auto" +CLEANFILES+= Kyuafile Kyuafile.tmp +.endif .for _T in ${_TESTS} _TEST_METADATA.${_T}= ${TEST_METADATA} ${TEST_METADATA.${_T}} .endfor -.NOPATH: Kyuafile.auto -Kyuafile.auto: Makefile +.if ${KYUAFILE:tl} == "auto" +Kyuafile: Makefile @{ \ echo '-- Automatically generated by bsd.test.mk.'; \ echo; \ @@ -77,20 +67,18 @@ Kyuafile.auto: Makefile echo; \ echo 'test_suite("${TESTSUITE}")'; \ echo; \ - } >Kyuafile.auto.tmp + } > ${.TARGET}.tmp .for _T in ${_TESTS} @echo '${TEST_INTERFACE.${_T}}_test_program{name="${_T}"${_TEST_METADATA.${_T}:C/$/,/:tW:C/^/, /W:C/,$//W}}' \ - >>Kyuafile.auto.tmp + >>${.TARGET}.tmp .endfor .for _T in ${TESTS_SUBDIRS:N.WAIT} - @echo "include(\"${_T}/Kyuafile\")" >>Kyuafile.auto.tmp + @echo "include(\"${_T}/${.TARGET}\")" >>${.TARGET}.tmp .endfor - @mv Kyuafile.auto.tmp Kyuafile.auto + @mv ${.TARGET}.tmp ${.TARGET} .endif -_kyuafile= ${DESTDIR}${TESTSDIR}/Kyuafile - -KYUA?= ${KYUA_PREFIX}/bin/kyua +KYUA= ${LOCALBASE}/bin/kyua realcheck: .PHONY .if exists(${KYUA}) diff --git a/share/mk/sys.mk b/share/mk/sys.mk index 8fe6b68..51f2818 100644 --- a/share/mk/sys.mk +++ b/share/mk/sys.mk @@ -69,6 +69,10 @@ __ENV_ONLY_OPTIONS:= \ # # The rules below use this macro to distinguish between Posix-compliant # and default behaviour. +# +# This functionality is currently broken, since make(1) processes sys.mk +# before reading any other files, and consequently has no opportunity to +# set the %POSIX macro before we read this point. .if defined(%POSIX) .SUFFIXES: .o .c .y .l .a .sh .f diff --git a/share/mk/tap.test.mk b/share/mk/tap.test.mk index ec86088..28b52af 100644 --- a/share/mk/tap.test.mk +++ b/share/mk/tap.test.mk @@ -26,7 +26,7 @@ TAP_TESTS_PERL?= TAP_TESTS_SH?= # Perl interpreter to use for test programs written in this language. -TAP_PERL_INTERPRETER?= /usr/local/bin/perl +TAP_PERL_INTERPRETER?= ${LOCALBASE}/bin/perl .if !empty(TAP_TESTS_C) PROGS+= ${TAP_TESTS_C} diff --git a/share/timedef/ja_JP.SJIS.src b/share/timedef/ja_JP.SJIS.src index 3c96a37..f5a6bff 100644 --- a/share/timedef/ja_JP.SJIS.src +++ b/share/timedef/ja_JP.SJIS.src @@ -63,7 +63,7 @@ ŒßŒã # # date_fmt -%Y”N%mŒŽ%e“ú %H:%M:%S %Z +%Y”N%mŒŽ%e“ú %A %H:%M:%S %Z # # Long month names (without case ending) 1ŒŽ diff --git a/share/timedef/ja_JP.UTF-8.src b/share/timedef/ja_JP.UTF-8.src index ea2fcbe..afa495f 100644 --- a/share/timedef/ja_JP.UTF-8.src +++ b/share/timedef/ja_JP.UTF-8.src @@ -56,14 +56,14 @@ %Y/%m/%d # # c_fmt -%Yå¹´%m月%eæ—¥ %H:%M:%S +%Yå¹´%m月%eæ—¥ %A %H:%M:%S # # AM/PM åˆå‰ åˆå¾Œ # # date_fmt -%Yå¹´%m月%eæ—¥ %H:%M:%S %Z +%Yå¹´%m月%eæ—¥ %A %H:%M:%S %Z # # Long month names (without case ending) 1月 diff --git a/share/timedef/ja_JP.eucJP.src b/share/timedef/ja_JP.eucJP.src index f98c832..3e231cd 100644 --- a/share/timedef/ja_JP.eucJP.src +++ b/share/timedef/ja_JP.eucJP.src @@ -56,14 +56,14 @@ %Y/%m/%d # # c_fmt -%Yǯ%m·î%eÆü %H:%M:%S +%Yǯ%m·î%eÆü %A %H:%M:%S # # AM/PM ¸áÁ° ¸á¸å # # date_fmt -%Yǯ%m·î%eÆü %H:%M:%S %Z +%Yǯ%m·î%eÆü %A %H:%M:%S %Z # # Long month names (without case ending) 1·î diff --git a/sys/amd64/amd64/initcpu.c b/sys/amd64/amd64/initcpu.c index f711b12..15d8171 100644 --- a/sys/amd64/amd64/initcpu.c +++ b/sys/amd64/amd64/initcpu.c @@ -55,37 +55,6 @@ SYSCTL_INT(_hw, OID_AUTO, instruction_sse, CTLFLAG_RD, */ static int hw_clflush_disable = -1; -int cpu; /* Are we 386, 386sx, 486, etc? */ -u_int cpu_feature; /* Feature flags */ -u_int cpu_feature2; /* Feature flags */ -u_int amd_feature; /* AMD feature flags */ -u_int amd_feature2; /* AMD feature flags */ -u_int amd_pminfo; /* AMD advanced power management info */ -u_int via_feature_rng; /* VIA RNG features */ -u_int via_feature_xcrypt; /* VIA ACE features */ -u_int cpu_high; /* Highest arg to CPUID */ -u_int cpu_exthigh; /* Highest arg to extended CPUID */ -u_int cpu_id; /* Stepping ID */ -u_int cpu_procinfo; /* HyperThreading Info / Brand Index / CLFUSH */ -u_int cpu_procinfo2; /* Multicore info */ -char cpu_vendor[20]; /* CPU Origin code */ -u_int cpu_vendor_id; /* CPU vendor ID */ -u_int cpu_fxsr; /* SSE enabled */ -u_int cpu_mxcsr_mask; /* Valid bits in mxcsr */ -u_int cpu_clflush_line_size = 32; -u_int cpu_stdext_feature; -u_int cpu_stdext_feature2; -u_int cpu_max_ext_state_size; -u_int cpu_mon_mwait_flags; /* MONITOR/MWAIT flags (CPUID.05H.ECX) */ -u_int cpu_mon_min_size; /* MONITOR minimum range size, bytes */ -u_int cpu_mon_max_size; /* MONITOR minimum range size, bytes */ -u_int cpu_maxphyaddr; /* Max phys addr width in bits */ - -SYSCTL_UINT(_hw, OID_AUTO, via_feature_rng, CTLFLAG_RD, - &via_feature_rng, 0, "VIA RNG feature available in CPU"); -SYSCTL_UINT(_hw, OID_AUTO, via_feature_xcrypt, CTLFLAG_RD, - &via_feature_xcrypt, 0, "VIA xcrypt feature available in CPU"); - static void init_amd(void) { diff --git a/sys/amd64/include/cputypes.h b/sys/amd64/include/cputypes.h index eeec4e0..5a70462 100644 --- a/sys/amd64/include/cputypes.h +++ b/sys/amd64/include/cputypes.h @@ -30,6 +30,8 @@ #ifndef _MACHINE_CPUTYPES_H_ #define _MACHINE_CPUTYPES_H_ +#include <x86/cputypes.h> + /* * Classes of processor. */ @@ -43,17 +45,4 @@ #define CPU_CLAWHAMMER 1 /* AMD Clawhammer */ #define CPU_SLEDGEHAMMER 2 /* AMD Sledgehammer */ -/* - * Vendors of processor. - */ -#define CPU_VENDOR_AMD 0x1022 /* AMD */ -#define CPU_VENDOR_IDT 0x111d /* Centaur/IDT/VIA */ -#define CPU_VENDOR_INTEL 0x8086 /* Intel */ -#define CPU_VENDOR_CENTAUR CPU_VENDOR_IDT - -#ifndef LOCORE -extern int cpu; -extern int cpu_class; -#endif - #endif /* !_MACHINE_CPUTYPES_H_ */ diff --git a/sys/amd64/include/md_var.h b/sys/amd64/include/md_var.h index 983485a..ddeb257 100644 --- a/sys/amd64/include/md_var.h +++ b/sys/amd64/include/md_var.h @@ -34,7 +34,6 @@ #include <x86/x86_var.h> -extern char ctx_switch_xsave[]; extern uint64_t *vm_page_dump; /* XXX */ diff --git a/sys/arm/arm/locore-v4.S b/sys/arm/arm/locore-v4.S index cc4f636..c46c0bf 100644 --- a/sys/arm/arm/locore-v4.S +++ b/sys/arm/arm/locore-v4.S @@ -42,19 +42,8 @@ __FBSDID("$FreeBSD$"); -/* - * Sanity check the configuration. - * FLASHADDR and LOADERRAMADDR depend on PHYSADDR in some cases. - * ARMv4 and ARMv5 make assumptions on where they are loaded. - * - * TODO: Fix the ARMv4/v5 case. - */ -#ifndef PHYSADDR -#error PHYSADDR must be defined for this configuration -#endif - -/* What size should this really be ? It is only used by initarm() */ -#define INIT_ARM_STACK_SIZE (2048 * 4) +/* 2K initial stack is plenty, it is only used by initarm() */ +#define INIT_ARM_STACK_SIZE 2048 #define CPWAIT_BRANCH \ sub pc, pc, #4 @@ -109,6 +98,16 @@ ASENTRY_NP(_start) msr cpsr_c, r7 #if defined (FLASHADDR) && defined(LOADERRAMADDR) +/* + * Sanity check the configuration. + * FLASHADDR and LOADERRAMADDR depend on PHYSADDR in some cases. + * ARMv4 and ARMv5 make assumptions on where they are loaded. + * TODO: Fix the ARMv4/v5 case. + */ +#ifndef PHYSADDR +#error PHYSADDR must be defined for this configuration +#endif + /* Check if we're running from flash. */ ldr r7, =FLASHADDR /* @@ -164,37 +163,45 @@ Lunmapped: * Build page table from scratch. */ - /* Find the delta between VA and PA */ + /* + * Figure out the physical address we're loaded at by assuming this + * entry point code is in the first L1 section and so if we clear the + * offset bits of the pc that will give us the section-aligned load + * address, which remains in r5 throughout all the following code. + */ + ldr r2, =(L1_S_OFFSET) + bic r5, pc, r2 + + /* Find the delta between VA and PA, result stays in r0 throughout. */ adr r0, Lpagetable bl translate_va_to_pa - /* - * Some of the older ports (the various XScale, mostly) assume - * that the memory before the kernel is mapped, and use it for - * the various stacks, page tables, etc. For those CPUs, map the - * 64 first MB of RAM, as it used to be. - */ - /* - * Map PA == VA + /* + * First map the entire 4GB address space as VA=PA. It's mapped as + * normal (cached) memory because it's for things like accessing the + * parameters passed in from the bootloader, which might be at any + * physical address, different for every platform. */ - ldr r5, =PHYSADDR - mov r1, r5 - mov r2, r5 - /* Map 64MiB, preserved over calls to build_pagetables */ - mov r3, #64 + mov r1, #0 + mov r2, #0 + mov r3, #4096 bl build_pagetables - /* Create the kernel map to jump to */ + /* + * Next we do 64MiB starting at the physical load address, mapped to + * the VA the kernel is linked for. + */ mov r1, r5 - ldr r2, =(KERNBASE) + ldr r2, =(KERNVIRTADDR) + mov r3, #64 bl build_pagetables - ldr r5, =(KERNPHYSADDR) + /* Create a device mapping for early_printf if specified. */ #if defined(SOCDEV_PA) && defined(SOCDEV_VA) - /* Create the custom map */ ldr r1, =SOCDEV_PA ldr r2, =SOCDEV_VA - bl build_pagetables + mov r3, #1 + bl build_device_pagetables #endif mcr p15, 0, r0, c2, c0, 0 /* Set TTB */ @@ -205,9 +212,6 @@ Lunmapped: mcr p15, 0, r0, c3, c0, 0 /* * Enable MMU. - * On armv6 enable extended page tables, and set alignment checking - * to modulo-4 (CPU_CONTROL_UNAL_ENABLE) for the ldrd/strd - * instructions emitted by clang. */ mrc p15, 0, r0, c1, c0, 0 orr r0, r0, #(CPU_CONTROL_MMU_ENABLE) @@ -217,6 +221,9 @@ Lunmapped: nop CPWAIT(r0) + /* Transition the PC from physical to virtual addressing. */ + ldr pc,=mmu_done + mmu_done: nop adr r1, .Lstart @@ -227,7 +234,6 @@ mmu_done: str r3, [r1], #0x0004 /* get zero init data */ subs r2, r2, #4 bgt .L1 - ldr pc, .Lvirt_done virt_done: mov r1, #28 /* loader info size is 28 bytes also second arg */ @@ -301,21 +307,25 @@ translate_va_to_pa: * * Addresses must be 1MiB aligned */ +build_device_pagetables: + ldr r4, =(L1_TYPE_S|L1_S_AP(AP_KRW)) + b 1f build_pagetables: /* Set the required page attributed */ ldr r4, =(L1_TYPE_S|L1_S_C|L1_S_AP(AP_KRW)) +1: orr r1, r4 /* Move the virtual address to the correct bit location */ lsr r2, #(L1_S_SHIFT - 2) mov r4, r3 -1: +2: str r1, [r0, r2] add r2, r2, #4 add r1, r1, #(L1_S_SIZE) adds r4, r4, #-1 - bhi 1b + bhi 2b RET diff --git a/sys/arm/arm/ofw_machdep.c b/sys/arm/arm/ofw_machdep.c new file mode 100644 index 0000000..59b85c6 --- /dev/null +++ b/sys/arm/arm/ofw_machdep.c @@ -0,0 +1,71 @@ +/*- + * Copyright (c) 2015 Ian Lepore <ian@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/bus.h> + +#include <machine/bus.h> +#include <machine/fdt.h> + +#include <dev/ofw/openfirm.h> +#include <dev/ofw/ofw_subr.h> + +int +OF_decode_addr(phandle_t dev, int regno, bus_space_tag_t *tag, + bus_space_handle_t *handle) +{ + bus_addr_t addr; + bus_size_t size; + pcell_t pci_hi; + int flags, res; + + res = ofw_reg_to_paddr(dev, regno, &addr, &size, &pci_hi); + if (res < 0) + return (res); + + /* + * Nothing special to do for PCI busses right now. + * This may need to be handled per-platform when it does come up. + */ +#ifdef notyet + if (pci_hi == OFW_PADDR_NOT_PCI) { + *tag = fdtbus_bs_tag; + flags = 0; + } else { + *tag = fdtbus_bs_tag; + flags = (pci_hi & OFW_PCI_PHYS_HI_PREFETCHABLE) ? + BUS_SPACE_MAP_PREFETCHABLE: 0; + } +#else + *tag = fdtbus_bs_tag; + flags = 0; +#endif + return (bus_space_map(*tag, addr, size, flags, handle)); +} + diff --git a/sys/arm/arm/pmap-v6-new.c b/sys/arm/arm/pmap-v6-new.c index 8d14ed7..5d7e571 100644 --- a/sys/arm/arm/pmap-v6-new.c +++ b/sys/arm/arm/pmap-v6-new.c @@ -3194,7 +3194,7 @@ setpte1: * When page is not modified, PTE2_RO can be set without * a TLB invalidation. * - * Note: When modified bit is being set, then in harware case, + * Note: When modified bit is being set, then in hardware case, * the TLB entry is re-read (updated) from PT2, and in * software case (abort), the PTE2 is read from PT2 and * TLB flushed if changed. The following cmpset() solves @@ -6170,7 +6170,7 @@ pte1_seta: /* * Handle modify bits for page and section. Note that the modify * bit is emulated by software. So PTEx_RO is software read only - * bit and PTEx_NM flag is real harware read only bit. + * bit and PTEx_NM flag is real hardware read only bit. * * QQQ: This is hardware emulation, we do not call userret() * for aborts from user mode. diff --git a/sys/arm/at91/std.bwct b/sys/arm/at91/std.bwct index 19e5356..3ec16a3 100644 --- a/sys/arm/at91/std.bwct +++ b/sys/arm/at91/std.bwct @@ -2,7 +2,6 @@ include "../at91/std.at91" makeoptions KERNPHYSADDR=0x20000000 -options KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 options KERNVIRTADDR=0xc0000000 diff --git a/sys/arm/at91/std.eb9200 b/sys/arm/at91/std.eb9200 index cc8f70f..0109fe2 100644 --- a/sys/arm/at91/std.eb9200 +++ b/sys/arm/at91/std.eb9200 @@ -2,7 +2,6 @@ include "../at91/std.at91" makeoptions KERNPHYSADDR=0x20000000 -options KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 options KERNVIRTADDR=0xc0000000 diff --git a/sys/arm/at91/std.ethernut5 b/sys/arm/at91/std.ethernut5 index 7793a83..aab78c2 100644 --- a/sys/arm/at91/std.ethernut5 +++ b/sys/arm/at91/std.ethernut5 @@ -3,7 +3,6 @@ include "../at91/std.at91sam9" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_ethernut5 diff --git a/sys/arm/at91/std.hl200 b/sys/arm/at91/std.hl200 index 5b5ed22..761c0a0 100644 --- a/sys/arm/at91/std.hl200 +++ b/sys/arm/at91/std.hl200 @@ -2,7 +2,6 @@ include "../at91/std.at91" makeoptions KERNPHYSADDR=0x20100000 -options KERNPHYSADDR=0x20100000 makeoptions KERNVIRTADDR=0xc0100000 options KERNVIRTADDR=0xc0100000 diff --git a/sys/arm/at91/std.hl201 b/sys/arm/at91/std.hl201 index 6537b27..e0c9c4b 100644 --- a/sys/arm/at91/std.hl201 +++ b/sys/arm/at91/std.hl201 @@ -3,7 +3,6 @@ include "../at91/std.at91sam9" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_hl201 diff --git a/sys/arm/at91/std.kb920x b/sys/arm/at91/std.kb920x index 2f52db5..0af4f04 100644 --- a/sys/arm/at91/std.kb920x +++ b/sys/arm/at91/std.kb920x @@ -2,7 +2,6 @@ include "../at91/std.at91" makeoptions KERNPHYSADDR=0x20000000 -options KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 options KERNVIRTADDR=0xc0000000 diff --git a/sys/arm/at91/std.qila9g20 b/sys/arm/at91/std.qila9g20 index f9dbb5f..4f99cc0 100644 --- a/sys/arm/at91/std.qila9g20 +++ b/sys/arm/at91/std.qila9g20 @@ -3,7 +3,6 @@ include "../at91/std.at91sam9" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_qila9g20 diff --git a/sys/arm/at91/std.sam9260ek b/sys/arm/at91/std.sam9260ek index fd4a7da..d424ad5 100644 --- a/sys/arm/at91/std.sam9260ek +++ b/sys/arm/at91/std.sam9260ek @@ -3,7 +3,6 @@ include "../at91/std.at91sam9" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_sam9260ek diff --git a/sys/arm/at91/std.sam9g20ek b/sys/arm/at91/std.sam9g20ek index 160a893..f8f651c 100644 --- a/sys/arm/at91/std.sam9g20ek +++ b/sys/arm/at91/std.sam9g20ek @@ -3,7 +3,6 @@ include "../at91/std.at91sam9" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_sam9g20ek diff --git a/sys/arm/at91/std.sam9x25ek b/sys/arm/at91/std.sam9x25ek index 4536b51..193d067 100644 --- a/sys/arm/at91/std.sam9x25ek +++ b/sys/arm/at91/std.sam9x25ek @@ -3,7 +3,6 @@ include "../at91/std.at91sam9" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_sam9x25ek diff --git a/sys/arm/at91/std.sn9g45 b/sys/arm/at91/std.sn9g45 index 5376955..30f3bc4 100644 --- a/sys/arm/at91/std.sn9g45 +++ b/sys/arm/at91/std.sn9g45 @@ -2,7 +2,6 @@ include "../at91/std.at91sam9g45" makeoptions KERNPHYSADDR=0x70008000 -options KERNPHYSADDR=0x70008000 makeoptions KERNVIRTADDR=0xc0008000 options KERNVIRTADDR=0xc0008000 diff --git a/sys/arm/at91/std.tsc4370 b/sys/arm/at91/std.tsc4370 index 6eb25cb..02054e5 100644 --- a/sys/arm/at91/std.tsc4370 +++ b/sys/arm/at91/std.tsc4370 @@ -3,7 +3,6 @@ include "../at91/std.at91" makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 device at91_board_tsc4370 diff --git a/sys/arm/broadcom/bcm2835/bcm2835_fbd.c b/sys/arm/broadcom/bcm2835/bcm2835_fbd.c index b47c602..4902955 100644 --- a/sys/arm/broadcom/bcm2835/bcm2835_fbd.c +++ b/sys/arm/broadcom/bcm2835/bcm2835_fbd.c @@ -59,13 +59,130 @@ __FBSDID("$FreeBSD$"); #define FB_DEPTH 24 struct bcmsc_softc { - struct fb_info *info; + struct fb_info info; + int fbswap; + struct bcm2835_fb_config fb; + device_t dev; }; static int bcm_fb_probe(device_t); static int bcm_fb_attach(device_t); static int +bcm_fb_init(struct bcmsc_softc *sc, struct bcm2835_fb_config *fb) +{ + int err; + + err = 0; + + memset(fb, 0, sizeof(*fb)); + if (bcm2835_mbox_fb_get_w_h(fb) != 0) + return (ENXIO); + fb->bpp = FB_DEPTH; + + if ((err = bcm2835_mbox_fb_init(fb)) != 0) { + device_printf(sc->dev, "bcm2835_mbox_fb_init failed, err=%d\n", err); + return (ENXIO); + } + + return (0); +} + +static int +bcm_fb_setup_fbd(struct bcmsc_softc *sc) +{ + struct bcm2835_fb_config fb; + device_t fbd; + int err; + + err = bcm_fb_init(sc, &fb); + if (err) + return (err); + + memset(&sc->info, 0, sizeof(sc->info)); + sc->info.fb_name = device_get_nameunit(sc->dev); + + sc->info.fb_vbase = (intptr_t)pmap_mapdev(fb.base, fb.size); + sc->info.fb_pbase = fb.base; + sc->info.fb_size = fb.size; + sc->info.fb_bpp = sc->info.fb_depth = fb.bpp; + sc->info.fb_stride = fb.pitch; + sc->info.fb_width = fb.xres; + sc->info.fb_height = fb.yres; + + if (sc->fbswap) { + switch (sc->info.fb_bpp) { + case 24: + vt_generate_cons_palette(sc->info.fb_cmap, + COLOR_FORMAT_RGB, 0xff, 0, 0xff, 8, 0xff, 16); + sc->info.fb_cmsize = 16; + break; + case 32: + vt_generate_cons_palette(sc->info.fb_cmap, + COLOR_FORMAT_RGB, 0xff, 16, 0xff, 8, 0xff, 0); + sc->info.fb_cmsize = 16; + break; + } + } + + fbd = device_add_child(sc->dev, "fbd", device_get_unit(sc->dev)); + if (fbd == NULL) { + device_printf(sc->dev, "Failed to add fbd child\n"); + pmap_unmapdev(sc->info.fb_vbase, sc->info.fb_size); + return (ENXIO); + } else if (device_probe_and_attach(fbd) != 0) { + device_printf(sc->dev, "Failed to attach fbd device\n"); + device_delete_child(sc->dev, fbd); + pmap_unmapdev(sc->info.fb_vbase, sc->info.fb_size); + return (ENXIO); + } + + device_printf(sc->dev, "%dx%d(%dx%d@%d,%d) %dbpp\n", fb.xres, fb.yres, + fb.vxres, fb.vyres, fb.xoffset, fb.yoffset, fb.bpp); + device_printf(sc->dev, + "fbswap: %d, pitch %d, base 0x%08x, screen_size %d\n", + sc->fbswap, fb.pitch, fb.base, fb.size); + + return (0); +} + +static int +bcm_fb_resync_sysctl(SYSCTL_HANDLER_ARGS) +{ + struct bcmsc_softc *sc = arg1; + struct bcm2835_fb_config fb; + int val; + int err; + + val = 0; + err = sysctl_handle_int(oidp, &val, 0, req); + if (err || !req->newptr) /* error || read request */ + return (err); + + bcm_fb_init(sc, &fb); + + return (0); +} + +static void +bcm_fb_sysctl_init(struct bcmsc_softc *sc) +{ + struct sysctl_ctx_list *ctx; + struct sysctl_oid *tree_node; + struct sysctl_oid_list *tree; + + /* + * Add system sysctl tree/handlers. + */ + ctx = device_get_sysctl_ctx(sc->dev); + tree_node = device_get_sysctl_tree(sc->dev); + tree = SYSCTL_CHILDREN(tree_node); + SYSCTL_ADD_PROC(ctx, tree, OID_AUTO, "resync", + CTLFLAG_RW | CTLTYPE_UINT, sc, sizeof(*sc), + bcm_fb_resync_sysctl, "IU", "Set to resync framebuffer with VC"); +} + +static int bcm_fb_probe(device_t dev) { if (!ofw_bus_is_compatible(dev, "broadcom,bcm2835-fb")) @@ -80,36 +197,15 @@ static int bcm_fb_attach(device_t dev) { char bootargs[2048], *n, *p, *v; - device_t fbd; - int fbswap, err; + int err; phandle_t chosen; - struct bcm2835_fb_config fb; struct bcmsc_softc *sc; - struct fb_info *info; sc = device_get_softc(dev); - memset(&fb, 0, sizeof(fb)); - if (bcm2835_mbox_fb_get_w_h(&fb) != 0) - return (ENXIO); - fb.bpp = FB_DEPTH; - if ((err = bcm2835_mbox_fb_init(&fb)) != 0) { - device_printf(dev, "bcm2835_mbox_fb_init failed, err=%d\n", err); - return (ENXIO); - } - - info = malloc(sizeof(struct fb_info), M_DEVBUF, M_WAITOK | M_ZERO); - info->fb_name = device_get_nameunit(dev); - info->fb_vbase = (intptr_t)pmap_mapdev(fb.base, fb.size); - info->fb_pbase = fb.base; - info->fb_size = fb.size; - info->fb_bpp = info->fb_depth = fb.bpp; - info->fb_stride = fb.pitch; - info->fb_width = fb.xres; - info->fb_height = fb.yres; - sc->info = info; + sc->dev = dev; /* Newer firmware versions needs an inverted color palette. */ - fbswap = 0; + sc->fbswap = 0; chosen = OF_finddevice("/chosen"); if (chosen != 0 && OF_getprop(chosen, "bootargs", &bootargs, sizeof(bootargs)) > 0) { @@ -120,41 +216,15 @@ bcm_fb_attach(device_t dev) n = strsep(&v, "="); if (strcmp(n, "bcm2708_fb.fbswap") == 0 && v != NULL) if (*v == '1') - fbswap = 1; + sc->fbswap = 1; } } - if (fbswap) { - switch (info->fb_bpp) { - case 24: - vt_generate_cons_palette(info->fb_cmap, - COLOR_FORMAT_RGB, 0xff, 0, 0xff, 8, 0xff, 16); - info->fb_cmsize = 16; - break; - case 32: - vt_generate_cons_palette(info->fb_cmap, - COLOR_FORMAT_RGB, 0xff, 16, 0xff, 8, 0xff, 0); - info->fb_cmsize = 16; - break; - } - } - fbd = device_add_child(dev, "fbd", device_get_unit(dev)); - if (fbd == NULL) { - device_printf(dev, "Failed to add fbd child\n"); - free(info, M_DEVBUF); - return (ENXIO); - } else if (device_probe_and_attach(fbd) != 0) { - device_printf(dev, "Failed to attach fbd device\n"); - device_delete_child(dev, fbd); - free(info, M_DEVBUF); - return (ENXIO); - } + bcm_fb_sysctl_init(sc); - device_printf(dev, "%dx%d(%dx%d@%d,%d) %dbpp\n", fb.xres, fb.yres, - fb.vxres, fb.vyres, fb.xoffset, fb.yoffset, fb.bpp); - device_printf(dev, - "fbswap: %d, pitch %d, base 0x%08x, screen_size %d\n", - fbswap, fb.pitch, fb.base, fb.size); + err = bcm_fb_setup_fbd(sc); + if (err) + return (err); return (0); } @@ -166,7 +236,7 @@ bcm_fb_helper_getinfo(device_t dev) sc = device_get_softc(dev); - return (sc->info); + return (&sc->info); } static device_method_t bcm_fb_methods[] = { diff --git a/sys/arm/cavium/cns11xx/std.econa b/sys/arm/cavium/cns11xx/std.econa index d520aa4..bc60ed1 100644 --- a/sys/arm/cavium/cns11xx/std.econa +++ b/sys/arm/cavium/cns11xx/std.econa @@ -8,7 +8,6 @@ options PHYSADDR=0x00000000 makeoptions KERNPHYSADDR=0x01000000 makeoptions KERNVIRTADDR=0xc1000000 -options KERNPHYSADDR=0x01000000 options KERNVIRTADDR=0xc1000000 # Used in ldscript.arm options FLASHADDR=0xD0000000 options LOADERRAMADDR=0x00000000 diff --git a/sys/arm/conf/ATMEL b/sys/arm/conf/ATMEL index c3b6035..f86f8e8 100644 --- a/sys/arm/conf/ATMEL +++ b/sys/arm/conf/ATMEL @@ -13,7 +13,6 @@ include "../at91/std.atmel" # at91sam9g45 or on some boards with non u-boot boot loaders. makeoptions KERNPHYSADDR=0x20000000 makeoptions KERNVIRTADDR=0xc0000000 -options KERNPHYSADDR=0x20000000 options KERNVIRTADDR=0xc0000000 makeoptions MODULES_OVERRIDE="dtb/atmel" diff --git a/sys/arm/conf/CNS11XXNAS b/sys/arm/conf/CNS11XXNAS index 710a471..2bb5920 100644 --- a/sys/arm/conf/CNS11XXNAS +++ b/sys/arm/conf/CNS11XXNAS @@ -22,7 +22,6 @@ ident CNS11XXNAS include "std.arm" #options PHYSADDR=0x10000000 -#options KERNPHYSADDR=0x10200000 #options KERNVIRTADDR=0xc0200000 # Used in ldscript.arm #options FLASHADDR=0x50000000 #options LOADERRAMADDR=0x00000000 diff --git a/sys/arm/conf/CRB b/sys/arm/conf/CRB index af500ca..c85913b 100644 --- a/sys/arm/conf/CRB +++ b/sys/arm/conf/CRB @@ -21,7 +21,6 @@ ident CRB include "std.arm" options PHYSADDR=0x00000000 -options KERNPHYSADDR=0x00200000 options KERNVIRTADDR=0xc0200000 # Used in ldscript.arm options COUNTS_PER_SEC=400000000 diff --git a/sys/arm/conf/GUMSTIX b/sys/arm/conf/GUMSTIX index 048bd3c..0b9c423 100644 --- a/sys/arm/conf/GUMSTIX +++ b/sys/arm/conf/GUMSTIX @@ -28,7 +28,6 @@ cpu CPU_XSCALE_PXA2X0 hints "GUMSTIX.hints" options PHYSADDR=0xa0000000 -options KERNPHYSADDR=0xa0200000 options KERNVIRTADDR=0xc0200000 # Used in ldscript.arm include "../xscale/pxa/std.pxa" diff --git a/sys/arm/conf/IMX6 b/sys/arm/conf/IMX6 index 7c267ff..6de7c26 100644 --- a/sys/arm/conf/IMX6 +++ b/sys/arm/conf/IMX6 @@ -133,6 +133,12 @@ device u3g # USB modems #options SC_DFLT_FONT # compile font in #makeoptions SC_DFLT_FONT=cp437 +device vt +device kbdmux +device ukbd +device videomode +device hdmi + # Flattened Device Tree options FDT # Configure using FDT/DTB data makeoptions MODULES_EXTRA=dtb/imx6 diff --git a/sys/arm/conf/NOTES b/sys/arm/conf/NOTES index 63912e1..f4259f2 100644 --- a/sys/arm/conf/NOTES +++ b/sys/arm/conf/NOTES @@ -26,7 +26,6 @@ files "../xscale/ixp425/files.ixp425" files "../xscale/pxa/files.pxa" options PHYSADDR=0x00000000 -options KERNPHYSADDR=0x00000000 options KERNVIRTADDR=0xc0000000 makeoptions LDFLAGS="-zmuldefs" diff --git a/sys/arm/conf/NSLU b/sys/arm/conf/NSLU index 665910a..a5ea6ed 100644 --- a/sys/arm/conf/NSLU +++ b/sys/arm/conf/NSLU @@ -24,7 +24,6 @@ ident NSLU include "std.arm" # XXX What is defined in std.avila does not exactly match the following: #options PHYSADDR=0x10000000 -#options KERNPHYSADDR=0x10200000 #options KERNVIRTADDR=0xc0200000 # Used in ldscript.arm #options FLASHADDR=0x50000000 #options LOADERRAMADDR=0x00000000 diff --git a/sys/arm/conf/SAM9260EK b/sys/arm/conf/SAM9260EK index 53b293c..1edee77 100644 --- a/sys/arm/conf/SAM9260EK +++ b/sys/arm/conf/SAM9260EK @@ -103,8 +103,10 @@ options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. #options BOOTP_NFSV3 #options BOOTP_WIRED_TO=macb0 -# alternatively, boot from a MMC/SD memory card -# s1 is FAT on this platform. +# s3 because s1 is reserved for the DOS parittions sometimes needed to +# boot off SD cards on the G20 and newer chips. s2 is reserved for +# nanobsd's config partition. s3 and s4 are for the ping-pong upgrade +# path. 9260 doesn't boot off SD, but let's keep things sane. options ROOTDEVNAME=\"ufs:/dev/mmcsd0s3a\" # Alternatively, boot from a USB card. diff --git a/sys/arm/conf/SAM9G20EK b/sys/arm/conf/SAM9G20EK index bd33e6b..685b2bf 100644 --- a/sys/arm/conf/SAM9G20EK +++ b/sys/arm/conf/SAM9G20EK @@ -82,9 +82,11 @@ options DDB # Enable the kernel debugger #options BOOTP_NFSV3 #options BOOTP_WIRED_TO=ate0 -# s2 because s1 is reserved for the DOS parittions sometimes needed to -# boot off SD cards on the G20 and newer chips. -options ROOTDEVNAME=\"ufs:/dev/mmcsd0s2a\" +# s3 because s1 is reserved for the DOS parittions sometimes needed to +# boot off SD cards on the G20 and newer chips. s2 is reserved for +# nanobsd's config partition. s3 and s4 are for the ping-pong upgrade +# path. +options ROOTDEVNAME=\"ufs:/dev/mmcsd0s3a\" # kernel/memory size reduction options MUTEX_NOINLINE diff --git a/sys/arm/freescale/imx/files.imx6 b/sys/arm/freescale/imx/files.imx6 index 45ff1a5..de4ff6e 100644 --- a/sys/arm/freescale/imx/files.imx6 +++ b/sys/arm/freescale/imx/files.imx6 @@ -24,6 +24,11 @@ arm/freescale/imx/imx6_sdma.c optional sdma arm/freescale/imx/imx6_audmux.c optional sound arm/freescale/imx/imx6_ssi.c optional sound +arm/arm/hdmi_if.m optional hdmi +arm/freescale/imx/imx6_hdmi.c optional hdmi + +arm/freescale/imx/imx6_ipu.c optional vt + # # Optional devices. # diff --git a/sys/arm/freescale/imx/imx6_ccm.c b/sys/arm/freescale/imx/imx6_ccm.c index cc86082..0439a5f 100644 --- a/sys/arm/freescale/imx/imx6_ccm.c +++ b/sys/arm/freescale/imx/imx6_ccm.c @@ -348,6 +348,43 @@ imx_ccm_ahb_hz(void) return (132000000); } +void +imx_ccm_ipu_enable(int ipu) +{ + struct ccm_softc *sc; + uint32_t reg; + + sc = ccm_sc; + reg = RD4(sc, CCM_CCGR3); + if (ipu == 1) + reg |= CCGR3_IPU1_IPU | CCGR3_IPU1_DI0; + else + reg |= CCGR3_IPU2_IPU | CCGR3_IPU2_DI0; + WR4(sc, CCM_CCGR3, reg); +} + +void +imx_ccm_hdmi_enable(void) +{ + struct ccm_softc *sc; + uint32_t reg; + + sc = ccm_sc; + reg = RD4(sc, CCM_CCGR2); + reg |= CCGR2_HDMI_TX | CCGR2_HDMI_TX_ISFR; + WR4(sc, CCM_CCGR2, reg); + + /* Set HDMI clock to 280MHz */ + reg = RD4(sc, CCM_CHSCCDR); + reg &= ~(CHSCCDR_IPU1_DI0_PRE_CLK_SEL_MASK | + CHSCCDR_IPU1_DI0_PODF_MASK | CHSCCDR_IPU1_DI0_CLK_SEL_MASK); + reg |= (CHSCCDR_PODF_DIVIDE_BY_3 << CHSCCDR_IPU1_DI0_PODF_SHIFT); + reg |= (CHSCCDR_IPU_PRE_CLK_540M_PFD << CHSCCDR_IPU1_DI0_PRE_CLK_SEL_SHIFT); + WR4(sc, CCM_CHSCCDR, reg); + reg |= (CHSCCDR_CLK_SEL_LDB_DI0 << CHSCCDR_IPU1_DI0_CLK_SEL_SHIFT); + WR4(sc, CCM_CHSCCDR, reg); +} + uint32_t imx_ccm_get_cacrr(void) { diff --git a/sys/arm/freescale/imx/imx6_ccmreg.h b/sys/arm/freescale/imx/imx6_ccmreg.h index bce102e..22e5995 100644 --- a/sys/arm/freescale/imx/imx6_ccmreg.h +++ b/sys/arm/freescale/imx/imx6_ccmreg.h @@ -30,6 +30,9 @@ #define IMX6_CCMREG_H #define CCM_CACCR 0x010 +#define CCM_CBCDR 0x014 +#define CBCDR_MMDC_CH1_AXI_PODF_SHIFT 3 +#define CBCDR_MMDC_CH1_AXI_PODF_MASK (7 << 3) #define CCM_CSCMR1 0x01C #define SSI1_CLK_SEL_S 10 #define SSI2_CLK_SEL_S 12 @@ -39,6 +42,7 @@ #define SSI_CLK_SEL_454_PFD 1 #define SSI_CLK_SEL_PLL4 2 #define CCM_CSCMR2 0x020 +#define CSCMR2_LDB_DI0_IPU_DIV_SHIFT 10 #define CCM_CS1CDR 0x028 #define SSI1_CLK_PODF_SHIFT 0 #define SSI1_CLK_PRED_SHIFT 6 @@ -49,6 +53,18 @@ #define CCM_CS2CDR 0x02C #define SSI2_CLK_PODF_SHIFT 0 #define SSI2_CLK_PRED_SHIFT 6 +#define LDB_DI0_CLK_SEL_SHIFT 9 +#define LDB_DI0_CLK_SEL_MASK (3 << LDB_DI0_CLK_SEL_SHIFT) +#define CCM_CHSCCDR 0x034 +#define CHSCCDR_IPU1_DI0_PRE_CLK_SEL_MASK (0x7 << 6) +#define CHSCCDR_IPU1_DI0_PRE_CLK_SEL_SHIFT 6 +#define CHSCCDR_IPU1_DI0_PODF_MASK (0x7 << 3) +#define CHSCCDR_IPU1_DI0_PODF_SHIFT 3 +#define CHSCCDR_IPU1_DI0_CLK_SEL_MASK (0x7) +#define CHSCCDR_IPU1_DI0_CLK_SEL_SHIFT 0 +#define CHSCCDR_CLK_SEL_LDB_DI0 3 +#define CHSCCDR_PODF_DIVIDE_BY_3 2 +#define CHSCCDR_IPU_PRE_CLK_540M_PFD 5 #define CCM_CSCDR2 0x038 #define CCM_CLPCR 0x054 #define CCM_CLPCR_LPM_MASK 0x03 diff --git a/sys/arm/freescale/imx/imx6_hdmi.c b/sys/arm/freescale/imx/imx6_hdmi.c new file mode 100644 index 0000000..c6e7fe4 --- /dev/null +++ b/sys/arm/freescale/imx/imx6_hdmi.c @@ -0,0 +1,768 @@ +/*- + * Copyright (c) 2015 Oleksandr Tymoshenko <gonzo@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +/* + * HDMI core module + */ + +#include <sys/param.h> +#include <sys/systm.h> +#include <sys/kernel.h> +#include <sys/module.h> +#include <sys/bus.h> +#include <sys/rman.h> + +#include <dev/ofw/ofw_bus.h> +#include <dev/ofw/ofw_bus_subr.h> + +#include <machine/bus.h> + +#include <dev/videomode/videomode.h> +#include <dev/videomode/edidvar.h> + +#include <dev/iicbus/iicbus.h> +#include <dev/iicbus/iiconf.h> + +#include <arm/freescale/imx/imx_ccmvar.h> +#include <arm/freescale/imx/imx_iomuxvar.h> +#include <arm/freescale/imx/imx_iomuxreg.h> +#include <arm/freescale/imx/imx6_hdmireg.h> + +#include "hdmi_if.h" + +#define I2C_DDC_ADDR (0x50 << 1) +#define EDID_LENGTH 0x80 + +struct imx_hdmi_softc { + device_t sc_dev; + struct resource *sc_mem_res; + int sc_mem_rid; + struct intr_config_hook sc_mode_hook; + struct videomode sc_mode; + uint8_t *sc_edid; + uint8_t sc_edid_len; + phandle_t sc_i2c_xref; +}; + +static struct ofw_compat_data compat_data[] = { + {"fsl,imx6dl-hdmi", 1}, + {"fsl,imx6q-hdmi", 1}, + {NULL, 0} +}; + +static inline uint8_t +RD1(struct imx_hdmi_softc *sc, bus_size_t off) +{ + + return (bus_read_1(sc->sc_mem_res, off)); +} + +static inline void +WR1(struct imx_hdmi_softc *sc, bus_size_t off, uint8_t val) +{ + + bus_write_1(sc->sc_mem_res, off, val); +} + +static void +imx_hdmi_phy_wait_i2c_done(struct imx_hdmi_softc *sc, int msec) +{ + uint8_t val; + + val = RD1(sc, HDMI_IH_I2CMPHY_STAT0) & + (HDMI_IH_I2CMPHY_STAT0_DONE | HDMI_IH_I2CMPHY_STAT0_ERROR); + while (val == 0) { + pause("HDMI_PHY", hz/100); + msec -= 10; + if (msec <= 0) + return; + val = RD1(sc, HDMI_IH_I2CMPHY_STAT0) & + (HDMI_IH_I2CMPHY_STAT0_DONE | HDMI_IH_I2CMPHY_STAT0_ERROR); + } +} + +static void +imx_hdmi_phy_i2c_write(struct imx_hdmi_softc *sc, unsigned short data, + unsigned char addr) +{ + + /* clear DONE and ERROR flags */ + WR1(sc, HDMI_IH_I2CMPHY_STAT0, + HDMI_IH_I2CMPHY_STAT0_DONE | HDMI_IH_I2CMPHY_STAT0_ERROR); + WR1(sc, HDMI_PHY_I2CM_ADDRESS_ADDR, addr); + WR1(sc, HDMI_PHY_I2CM_DATAO_1_ADDR, ((data >> 8) & 0xff)); + WR1(sc, HDMI_PHY_I2CM_DATAO_0_ADDR, ((data >> 0) & 0xff)); + WR1(sc, HDMI_PHY_I2CM_OPERATION_ADDR, HDMI_PHY_I2CM_OPERATION_ADDR_WRITE); + imx_hdmi_phy_wait_i2c_done(sc, 1000); +} + +static void +imx_hdmi_disable_overflow_interrupts(struct imx_hdmi_softc *sc) +{ + WR1(sc, HDMI_IH_MUTE_FC_STAT2, HDMI_IH_MUTE_FC_STAT2_OVERFLOW_MASK); + WR1(sc, HDMI_FC_MASK2, + HDMI_FC_MASK2_LOW_PRI | HDMI_FC_MASK2_HIGH_PRI); +} + +static void +imx_hdmi_av_composer(struct imx_hdmi_softc *sc) +{ + uint8_t inv_val; + int is_dvi; + int hblank, vblank, hsync_len, hbp, vbp; + + /* Set up HDMI_FC_INVIDCONF */ + inv_val = ((sc->sc_mode.flags & VID_NVSYNC) ? + HDMI_FC_INVIDCONF_VSYNC_IN_POLARITY_ACTIVE_LOW : + HDMI_FC_INVIDCONF_VSYNC_IN_POLARITY_ACTIVE_HIGH); + + inv_val |= ((sc->sc_mode.flags & VID_NHSYNC) ? + HDMI_FC_INVIDCONF_HSYNC_IN_POLARITY_ACTIVE_LOW : + HDMI_FC_INVIDCONF_HSYNC_IN_POLARITY_ACTIVE_HIGH); + + inv_val |= HDMI_FC_INVIDCONF_DE_IN_POLARITY_ACTIVE_HIGH; + + inv_val |= ((sc->sc_mode.flags & VID_INTERLACE) ? + HDMI_FC_INVIDCONF_R_V_BLANK_IN_OSC_ACTIVE_HIGH : + HDMI_FC_INVIDCONF_R_V_BLANK_IN_OSC_ACTIVE_LOW); + + inv_val |= ((sc->sc_mode.flags & VID_INTERLACE) ? + HDMI_FC_INVIDCONF_IN_I_P_INTERLACED : + HDMI_FC_INVIDCONF_IN_I_P_PROGRESSIVE); + + /* TODO: implement HDMI part */ + is_dvi = 1; + inv_val |= (is_dvi ? + HDMI_FC_INVIDCONF_DVI_MODEZ_DVI_MODE : + HDMI_FC_INVIDCONF_DVI_MODEZ_HDMI_MODE); + + WR1(sc, HDMI_FC_INVIDCONF, inv_val); + + /* Set up horizontal active pixel region width */ + WR1(sc, HDMI_FC_INHACTV1, sc->sc_mode.hdisplay >> 8); + WR1(sc, HDMI_FC_INHACTV0, sc->sc_mode.hdisplay); + + /* Set up vertical blanking pixel region width */ + WR1(sc, HDMI_FC_INVACTV1, sc->sc_mode.vdisplay >> 8); + WR1(sc, HDMI_FC_INVACTV0, sc->sc_mode.vdisplay); + + /* Set up horizontal blanking pixel region width */ + hblank = sc->sc_mode.htotal - sc->sc_mode.hdisplay; + WR1(sc, HDMI_FC_INHBLANK1, hblank >> 8); + WR1(sc, HDMI_FC_INHBLANK0, hblank); + + /* Set up vertical blanking pixel region width */ + vblank = sc->sc_mode.vtotal - sc->sc_mode.vdisplay; + WR1(sc, HDMI_FC_INVBLANK, vblank); + + /* Set up HSYNC active edge delay width (in pixel clks) */ + hbp = sc->sc_mode.htotal - sc->sc_mode.hsync_end; + WR1(sc, HDMI_FC_HSYNCINDELAY1, hbp >> 8); + WR1(sc, HDMI_FC_HSYNCINDELAY0, hbp); + + /* Set up VSYNC active edge delay (in pixel clks) */ + vbp = sc->sc_mode.vtotal - sc->sc_mode.vsync_end; + WR1(sc, HDMI_FC_VSYNCINDELAY, vbp); + + hsync_len = (sc->sc_mode.hsync_end - sc->sc_mode.hsync_start); + /* Set up HSYNC active pulse width (in pixel clks) */ + WR1(sc, HDMI_FC_HSYNCINWIDTH1, hsync_len >> 8); + WR1(sc, HDMI_FC_HSYNCINWIDTH0, hsync_len); + + /* Set up VSYNC active edge delay (in pixel clks) */ + WR1(sc, HDMI_FC_VSYNCINWIDTH, (sc->sc_mode.vsync_end - sc->sc_mode.vsync_start)); +} + +static void +imx_hdmi_phy_enable_power(struct imx_hdmi_softc *sc, uint8_t enable) +{ + uint8_t reg; + + reg = RD1(sc, HDMI_PHY_CONF0); + reg &= ~HDMI_PHY_CONF0_PDZ_MASK; + reg |= (enable << HDMI_PHY_CONF0_PDZ_OFFSET); + WR1(sc, HDMI_PHY_CONF0, reg); +} + +static void +imx_hdmi_phy_enable_tmds(struct imx_hdmi_softc *sc, uint8_t enable) +{ + uint8_t reg; + + reg = RD1(sc, HDMI_PHY_CONF0); + reg &= ~HDMI_PHY_CONF0_ENTMDS_MASK; + reg |= (enable << HDMI_PHY_CONF0_ENTMDS_OFFSET); + WR1(sc, HDMI_PHY_CONF0, reg); +} + +static void +imx_hdmi_phy_gen2_pddq(struct imx_hdmi_softc *sc, uint8_t enable) +{ + uint8_t reg; + + reg = RD1(sc, HDMI_PHY_CONF0); + reg &= ~HDMI_PHY_CONF0_GEN2_PDDQ_MASK; + reg |= (enable << HDMI_PHY_CONF0_GEN2_PDDQ_OFFSET); + WR1(sc, HDMI_PHY_CONF0, reg); +} + +static void +imx_hdmi_phy_gen2_txpwron(struct imx_hdmi_softc *sc, uint8_t enable) +{ + uint8_t reg; + + reg = RD1(sc, HDMI_PHY_CONF0); + reg &= ~HDMI_PHY_CONF0_GEN2_TXPWRON_MASK; + reg |= (enable << HDMI_PHY_CONF0_GEN2_TXPWRON_OFFSET); + WR1(sc, HDMI_PHY_CONF0, reg); +} + +static void +imx_hdmi_phy_sel_data_en_pol(struct imx_hdmi_softc *sc, uint8_t enable) +{ + uint8_t reg; + + reg = RD1(sc, HDMI_PHY_CONF0); + reg &= ~HDMI_PHY_CONF0_SELDATAENPOL_MASK; + reg |= (enable << HDMI_PHY_CONF0_SELDATAENPOL_OFFSET); + WR1(sc, HDMI_PHY_CONF0, reg); +} + +static void +imx_hdmi_phy_sel_interface_control(struct imx_hdmi_softc *sc, uint8_t enable) +{ + uint8_t reg; + + reg = RD1(sc, HDMI_PHY_CONF0); + reg &= ~HDMI_PHY_CONF0_SELDIPIF_MASK; + reg |= (enable << HDMI_PHY_CONF0_SELDIPIF_OFFSET); + WR1(sc, HDMI_PHY_CONF0, reg); +} + +static inline void +imx_hdmi_phy_test_clear(struct imx_hdmi_softc *sc, unsigned char bit) +{ + uint8_t val; + + val = RD1(sc, HDMI_PHY_TST0); + val &= ~HDMI_PHY_TST0_TSTCLR_MASK; + val |= (bit << HDMI_PHY_TST0_TSTCLR_OFFSET) & + HDMI_PHY_TST0_TSTCLR_MASK; + WR1(sc, HDMI_PHY_TST0, val); +} + +static void +imx_hdmi_clear_overflow(struct imx_hdmi_softc *sc) +{ + int count; + uint8_t val; + + /* TMDS software reset */ + WR1(sc, HDMI_MC_SWRSTZ, (uint8_t)~HDMI_MC_SWRSTZ_TMDSSWRST_REQ); + + val = RD1(sc, HDMI_FC_INVIDCONF); + + for (count = 0 ; count < 5 ; count++) + WR1(sc, HDMI_FC_INVIDCONF, val); +} + +static int +imx_hdmi_phy_configure(struct imx_hdmi_softc *sc) +{ + uint8_t val; + uint8_t msec; + + WR1(sc, HDMI_MC_FLOWCTRL, HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_BYPASS); + + /* gen2 tx power off */ + imx_hdmi_phy_gen2_txpwron(sc, 0); + + /* gen2 pddq */ + imx_hdmi_phy_gen2_pddq(sc, 1); + + /* PHY reset */ + WR1(sc, HDMI_MC_PHYRSTZ, HDMI_MC_PHYRSTZ_DEASSERT); + WR1(sc, HDMI_MC_PHYRSTZ, HDMI_MC_PHYRSTZ_ASSERT); + + WR1(sc, HDMI_MC_HEACPHY_RST, HDMI_MC_HEACPHY_RST_ASSERT); + + imx_hdmi_phy_test_clear(sc, 1); + WR1(sc, HDMI_PHY_I2CM_SLAVE_ADDR, HDMI_PHY_I2CM_SLAVE_ADDR_PHY_GEN2); + imx_hdmi_phy_test_clear(sc, 0); + + /* + * Following initialization are for 8bit per color case + */ + + /* + * PLL/MPLL config, see section 24.7.22 in TRM + * config, see section 24.7.22 + */ + if (sc->sc_mode.dot_clock*1000 <= 45250000) { + imx_hdmi_phy_i2c_write(sc, CPCE_CTRL_45_25, HDMI_PHY_I2C_CPCE_CTRL); + imx_hdmi_phy_i2c_write(sc, GMPCTRL_45_25, HDMI_PHY_I2C_GMPCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 92500000) { + imx_hdmi_phy_i2c_write(sc, CPCE_CTRL_92_50, HDMI_PHY_I2C_CPCE_CTRL); + imx_hdmi_phy_i2c_write(sc, GMPCTRL_92_50, HDMI_PHY_I2C_GMPCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 185000000) { + imx_hdmi_phy_i2c_write(sc, CPCE_CTRL_185, HDMI_PHY_I2C_CPCE_CTRL); + imx_hdmi_phy_i2c_write(sc, GMPCTRL_185, HDMI_PHY_I2C_GMPCTRL); + } else { + imx_hdmi_phy_i2c_write(sc, CPCE_CTRL_370, HDMI_PHY_I2C_CPCE_CTRL); + imx_hdmi_phy_i2c_write(sc, GMPCTRL_370, HDMI_PHY_I2C_GMPCTRL); + } + + /* + * Values described in TRM section 34.9.2 PLL/MPLL Generic + * Configuration Settings. Table 34-23. + */ + if (sc->sc_mode.dot_clock*1000 <= 54000000) { + imx_hdmi_phy_i2c_write(sc, 0x091c, HDMI_PHY_I2C_CURRCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 58400000) { + imx_hdmi_phy_i2c_write(sc, 0x091c, HDMI_PHY_I2C_CURRCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 72000000) { + imx_hdmi_phy_i2c_write(sc, 0x06dc, HDMI_PHY_I2C_CURRCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 74250000) { + imx_hdmi_phy_i2c_write(sc, 0x06dc, HDMI_PHY_I2C_CURRCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 118800000) { + imx_hdmi_phy_i2c_write(sc, 0x091c, HDMI_PHY_I2C_CURRCTRL); + } else if (sc->sc_mode.dot_clock*1000 <= 216000000) { + imx_hdmi_phy_i2c_write(sc, 0x06dc, HDMI_PHY_I2C_CURRCTRL); + } else { + panic("Unsupported mode\n"); + } + + imx_hdmi_phy_i2c_write(sc, 0x0000, HDMI_PHY_I2C_PLLPHBYCTRL); + imx_hdmi_phy_i2c_write(sc, MSM_CTRL_FB_CLK, HDMI_PHY_I2C_MSM_CTRL); + /* RESISTANCE TERM 133 Ohm */ + imx_hdmi_phy_i2c_write(sc, TXTERM_133, HDMI_PHY_I2C_TXTERM); + + /* REMOVE CLK TERM */ + imx_hdmi_phy_i2c_write(sc, CKCALCTRL_OVERRIDE, HDMI_PHY_I2C_CKCALCTRL); + + if (sc->sc_mode.dot_clock*1000 > 148500000) { + imx_hdmi_phy_i2c_write(sc,CKSYMTXCTRL_OVERRIDE | CKSYMTXCTRL_TX_SYMON | + CKSYMTXCTRL_TX_TRBON | CKSYMTXCTRL_TX_CK_SYMON, HDMI_PHY_I2C_CKSYMTXCTRL); + imx_hdmi_phy_i2c_write(sc, VLEVCTRL_TX_LVL(9) | VLEVCTRL_CK_LVL(9), + HDMI_PHY_I2C_VLEVCTRL); + } else { + imx_hdmi_phy_i2c_write(sc,CKSYMTXCTRL_OVERRIDE | CKSYMTXCTRL_TX_SYMON | + CKSYMTXCTRL_TX_TRAON | CKSYMTXCTRL_TX_CK_SYMON, HDMI_PHY_I2C_CKSYMTXCTRL); + imx_hdmi_phy_i2c_write(sc, VLEVCTRL_TX_LVL(13) | VLEVCTRL_CK_LVL(13), + HDMI_PHY_I2C_VLEVCTRL); + } + + imx_hdmi_phy_enable_power(sc, 1); + + /* toggle TMDS enable */ + imx_hdmi_phy_enable_tmds(sc, 0); + imx_hdmi_phy_enable_tmds(sc, 1); + + /* gen2 tx power on */ + imx_hdmi_phy_gen2_txpwron(sc, 1); + imx_hdmi_phy_gen2_pddq(sc, 0); + + /*Wait for PHY PLL lock */ + msec = 4; + val = RD1(sc, HDMI_PHY_STAT0) & HDMI_PHY_TX_PHY_LOCK; + while (val == 0) { + DELAY(1000); + if (msec-- == 0) { + device_printf(sc->sc_dev, "PHY PLL not locked\n"); + return (-1); + } + val = RD1(sc, HDMI_PHY_STAT0) & HDMI_PHY_TX_PHY_LOCK; + } + + return true; +} + +static void +imx_hdmi_phy_init(struct imx_hdmi_softc *sc) +{ + int i; + + /* HDMI Phy spec says to do the phy initialization sequence twice */ + for (i = 0 ; i < 2 ; i++) { + imx_hdmi_phy_sel_data_en_pol(sc, 1); + imx_hdmi_phy_sel_interface_control(sc, 0); + imx_hdmi_phy_enable_tmds(sc, 0); + imx_hdmi_phy_enable_power(sc, 0); + + /* Enable CSC */ + imx_hdmi_phy_configure(sc); + } +} + +static void +imx_hdmi_enable_video_path(struct imx_hdmi_softc *sc) +{ + uint8_t clkdis; + + /* + * Control period timing + * Values are minimal according to HDMI spec 1.4a + */ + WR1(sc, HDMI_FC_CTRLDUR, 12); + WR1(sc, HDMI_FC_EXCTRLDUR, 32); + WR1(sc, HDMI_FC_EXCTRLSPAC, 1); + + /* + * Bits to fill data lines not used to transmit preamble + * for channels 0, 1, and 2 respectively + */ + WR1(sc, HDMI_FC_CH0PREAM, 0x0B); + WR1(sc, HDMI_FC_CH1PREAM, 0x16); + WR1(sc, HDMI_FC_CH2PREAM, 0x21); + + /* Save CEC clock */ + clkdis = RD1(sc, HDMI_MC_CLKDIS) & HDMI_MC_CLKDIS_CECCLK_DISABLE; + clkdis |= ~HDMI_MC_CLKDIS_CECCLK_DISABLE; + + /* Enable pixel clock and tmds data path */ + clkdis &= ~HDMI_MC_CLKDIS_PIXELCLK_DISABLE; + WR1(sc, HDMI_MC_CLKDIS, clkdis); + + clkdis &= ~HDMI_MC_CLKDIS_TMDSCLK_DISABLE; + WR1(sc, HDMI_MC_CLKDIS, clkdis); +} + +static void +imx_hdmi_video_packetize(struct imx_hdmi_softc *sc) +{ + unsigned int color_depth = 0; + unsigned int remap_size = HDMI_VP_REMAP_YCC422_16BIT; + unsigned int output_select = HDMI_VP_CONF_OUTPUT_SELECTOR_PP; + uint8_t val; + + output_select = HDMI_VP_CONF_OUTPUT_SELECTOR_BYPASS; + color_depth = 0; + + /* set the packetizer registers */ + val = ((color_depth << HDMI_VP_PR_CD_COLOR_DEPTH_OFFSET) & + HDMI_VP_PR_CD_COLOR_DEPTH_MASK); + WR1(sc, HDMI_VP_PR_CD, val); + + val = RD1(sc, HDMI_VP_STUFF); + val &= ~HDMI_VP_STUFF_PR_STUFFING_MASK; + val |= HDMI_VP_STUFF_PR_STUFFING_STUFFING_MODE; + WR1(sc, HDMI_VP_STUFF, val); + + val = RD1(sc, HDMI_VP_CONF); + val &= ~(HDMI_VP_CONF_PR_EN_MASK | + HDMI_VP_CONF_BYPASS_SELECT_MASK); + val |= HDMI_VP_CONF_PR_EN_DISABLE | + HDMI_VP_CONF_BYPASS_SELECT_VID_PACKETIZER; + WR1(sc, HDMI_VP_CONF, val); + + val = RD1(sc, HDMI_VP_STUFF); + val &= ~HDMI_VP_STUFF_IDEFAULT_PHASE_MASK; + val |= 1 << HDMI_VP_STUFF_IDEFAULT_PHASE_OFFSET; + WR1(sc, HDMI_VP_STUFF, val); + + WR1(sc, HDMI_VP_REMAP, remap_size); + + if (output_select == HDMI_VP_CONF_OUTPUT_SELECTOR_PP) { + val = RD1(sc, HDMI_VP_CONF); + val &= ~(HDMI_VP_CONF_BYPASS_EN_MASK | + HDMI_VP_CONF_PP_EN_ENMASK | + HDMI_VP_CONF_YCC422_EN_MASK); + val |= HDMI_VP_CONF_BYPASS_EN_DISABLE | + HDMI_VP_CONF_PP_EN_ENABLE | + HDMI_VP_CONF_YCC422_EN_DISABLE; + WR1(sc, HDMI_VP_CONF, val); + } else if (output_select == HDMI_VP_CONF_OUTPUT_SELECTOR_YCC422) { + val = RD1(sc, HDMI_VP_CONF); + val &= ~(HDMI_VP_CONF_BYPASS_EN_MASK | + HDMI_VP_CONF_PP_EN_ENMASK | + HDMI_VP_CONF_YCC422_EN_MASK); + val |= HDMI_VP_CONF_BYPASS_EN_DISABLE | + HDMI_VP_CONF_PP_EN_DISABLE | + HDMI_VP_CONF_YCC422_EN_ENABLE; + WR1(sc, HDMI_VP_CONF, val); + } else if (output_select == HDMI_VP_CONF_OUTPUT_SELECTOR_BYPASS) { + val = RD1(sc, HDMI_VP_CONF); + val &= ~(HDMI_VP_CONF_BYPASS_EN_MASK | + HDMI_VP_CONF_PP_EN_ENMASK | + HDMI_VP_CONF_YCC422_EN_MASK); + val |= HDMI_VP_CONF_BYPASS_EN_ENABLE | + HDMI_VP_CONF_PP_EN_DISABLE | + HDMI_VP_CONF_YCC422_EN_DISABLE; + WR1(sc, HDMI_VP_CONF, val); + } else { + return; + } + + val = RD1(sc, HDMI_VP_STUFF); + val &= ~(HDMI_VP_STUFF_PP_STUFFING_MASK | + HDMI_VP_STUFF_YCC422_STUFFING_MASK); + val |= HDMI_VP_STUFF_PP_STUFFING_STUFFING_MODE | + HDMI_VP_STUFF_YCC422_STUFFING_STUFFING_MODE; + WR1(sc, HDMI_VP_STUFF, val); + + val = RD1(sc, HDMI_VP_CONF); + val &= ~HDMI_VP_CONF_OUTPUT_SELECTOR_MASK; + val |= output_select; + WR1(sc, HDMI_VP_CONF, val); +} + +static void +imx_hdmi_video_sample(struct imx_hdmi_softc *sc) +{ + int color_format; + uint8_t val; + + color_format = 0x01; + val = HDMI_TX_INVID0_INTERNAL_DE_GENERATOR_DISABLE | + ((color_format << HDMI_TX_INVID0_VIDEO_MAPPING_OFFSET) & + HDMI_TX_INVID0_VIDEO_MAPPING_MASK); + WR1(sc, HDMI_TX_INVID0, val); + + /* Enable TX stuffing: When DE is inactive, fix the output data to 0 */ + val = HDMI_TX_INSTUFFING_BDBDATA_STUFFING_ENABLE | + HDMI_TX_INSTUFFING_RCRDATA_STUFFING_ENABLE | + HDMI_TX_INSTUFFING_GYDATA_STUFFING_ENABLE; + WR1(sc, HDMI_TX_INSTUFFING, val); + WR1(sc, HDMI_TX_GYDATA0, 0x0); + WR1(sc, HDMI_TX_GYDATA1, 0x0); + WR1(sc, HDMI_TX_RCRDATA0, 0x0); + WR1(sc, HDMI_TX_RCRDATA1, 0x0); + WR1(sc, HDMI_TX_BCBDATA0, 0x0); + WR1(sc, HDMI_TX_BCBDATA1, 0x0); +} + +static int +imx_hdmi_set_mode(struct imx_hdmi_softc *sc) +{ + + imx_hdmi_disable_overflow_interrupts(sc); + imx_hdmi_av_composer(sc); + imx_hdmi_phy_init(sc); + imx_hdmi_enable_video_path(sc); + /* TODO: AVI infoframes */ + imx_hdmi_video_packetize(sc); + /* TODO: imx_hdmi_video_csc(sc); */ + imx_hdmi_video_sample(sc); + imx_hdmi_clear_overflow(sc); + + return (0); +} + +static int +hdmi_edid_read(struct imx_hdmi_softc *sc, uint8_t **edid, uint32_t *edid_len) +{ + device_t i2c_dev; + int result; + uint8_t addr = 0; + struct iic_msg msg[] = { + { 0, IIC_M_WR, 1, &addr }, + { 0, IIC_M_RD, EDID_LENGTH, NULL} + }; + + *edid = NULL; + *edid_len = 0; + + if (sc->sc_i2c_xref == 0) + return (ENXIO); + + i2c_dev = OF_device_from_xref(sc->sc_i2c_xref); + if (!i2c_dev) { + device_printf(sc->sc_dev, + "no actual device for \"ddc-i2c-bus\" property (handle=%x)\n", sc->sc_i2c_xref); + return (ENXIO); + } + + device_printf(sc->sc_dev, "reading EDID from %s, addr %02x\n", + device_get_nameunit(i2c_dev), I2C_DDC_ADDR/2); + + msg[0].slave = I2C_DDC_ADDR; + msg[1].slave = I2C_DDC_ADDR; + msg[1].buf = sc->sc_edid; + + result = iicbus_request_bus(i2c_dev, sc->sc_dev, IIC_INTRWAIT); + + if (result) { + device_printf(sc->sc_dev, "failed to request i2c bus: %d\n", result); + return (result); + } + + result = iicbus_transfer(i2c_dev, msg, 2); + iicbus_release_bus(i2c_dev, sc->sc_dev); + + if (result) { + device_printf(sc->sc_dev, "i2c transfer failed: %d\n", result); + return (result); + } else { + *edid_len = sc->sc_edid_len; + *edid = sc->sc_edid; + } + + return (result); +} + +static void +imx_hdmi_detect_cable(void *arg) +{ + struct imx_hdmi_softc *sc; + + sc = arg; + EVENTHANDLER_INVOKE(hdmi_event, sc->sc_dev, HDMI_EVENT_CONNECTED); + /* Finished with the interrupt hook */ + config_intrhook_disestablish(&sc->sc_mode_hook); +} + +static int +imx_hdmi_detach(device_t dev) +{ + struct imx_hdmi_softc *sc; + + sc = device_get_softc(dev); + + if (sc->sc_mem_res != NULL) + bus_release_resource(dev, SYS_RES_MEMORY, + sc->sc_mem_rid, sc->sc_mem_res); + + return (0); +} + +static int +imx_hdmi_attach(device_t dev) +{ + struct imx_hdmi_softc *sc; + int err; + uint32_t gpr3; + phandle_t node, i2c_xref; + + sc = device_get_softc(dev); + sc->sc_dev = dev; + err = 0; + + /* Allocate memory resources. */ + sc->sc_mem_rid = 0; + sc->sc_mem_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &sc->sc_mem_rid, + RF_ACTIVE); + if (sc->sc_mem_res == NULL) { + device_printf(dev, "Cannot allocate memory resources\n"); + err = ENXIO; + goto out; + } + + sc->sc_mode_hook.ich_func = imx_hdmi_detect_cable; + sc->sc_mode_hook.ich_arg = sc; + + if (config_intrhook_establish(&sc->sc_mode_hook) != 0) { + err = ENOMEM; + goto out; + } + + node = ofw_bus_get_node(dev); + if (OF_getencprop(node, "ddc-i2c-bus", &i2c_xref, sizeof(i2c_xref)) == -1) + sc->sc_i2c_xref = 0; + else + sc->sc_i2c_xref = i2c_xref; + + sc->sc_edid = malloc(EDID_LENGTH, M_DEVBUF, M_WAITOK | M_ZERO); + sc->sc_edid_len = EDID_LENGTH; + + imx_ccm_hdmi_enable(); + + device_printf(sc->sc_dev, "HDMI controller %02x:%02x:%02x:%02x\n", + RD1(sc, HDMI_DESIGN_ID), RD1(sc, HDMI_REVISION_ID), + RD1(sc, HDMI_PRODUCT_ID0), RD1(sc, HDMI_PRODUCT_ID1)); + + + gpr3 = imx_iomux_gpr_get(IOMUXC_GPR3); + gpr3 &= ~(IOMUXC_GPR3_HDMI_MASK); + gpr3 |= IOMUXC_GPR3_HDMI_IPU1_DI0; + imx_iomux_gpr_set(IOMUXC_GPR3, gpr3); + + WR1(sc, HDMI_PHY_POL0, HDMI_PHY_HPD); + WR1(sc, HDMI_IH_PHY_STAT0, HDMI_IH_PHY_STAT0_HPD); + +out: + + if (err != 0) + imx_hdmi_detach(dev); + + return (err); +} + +static int +imx_hdmi_probe(device_t dev) +{ + + if (ofw_bus_search_compatible(dev, compat_data)->ocd_data == 0) + return (ENXIO); + + device_set_desc(dev, "Freescale i.MX6 HDMI core"); + + return (BUS_PROBE_DEFAULT); +} + +static int +imx_hdmi_get_edid(device_t dev, uint8_t **edid, uint32_t *edid_len) +{ + + return (hdmi_edid_read(device_get_softc(dev), edid, edid_len)); +} + +static int +imx_hdmi_set_videomode(device_t dev, const struct videomode *mode) +{ + struct imx_hdmi_softc *sc; + + sc = device_get_softc(dev); + memcpy(&sc->sc_mode, mode, sizeof(*mode)); + imx_hdmi_set_mode(sc); + + return (0); +} + +static device_method_t imx_hdmi_methods[] = { + /* Device interface */ + DEVMETHOD(device_probe, imx_hdmi_probe), + DEVMETHOD(device_attach, imx_hdmi_attach), + DEVMETHOD(device_detach, imx_hdmi_detach), + + /* HDMI methods */ + DEVMETHOD(hdmi_get_edid, imx_hdmi_get_edid), + DEVMETHOD(hdmi_set_videomode, imx_hdmi_set_videomode), + + DEVMETHOD_END +}; + +static driver_t imx_hdmi_driver = { + "hdmi", + imx_hdmi_methods, + sizeof(struct imx_hdmi_softc) +}; + +static devclass_t imx_hdmi_devclass; + +DRIVER_MODULE(hdmi, simplebus, imx_hdmi_driver, imx_hdmi_devclass, 0, 0); diff --git a/sys/arm/freescale/imx/imx6_hdmireg.h b/sys/arm/freescale/imx/imx6_hdmireg.h new file mode 100644 index 0000000..947d873 --- /dev/null +++ b/sys/arm/freescale/imx/imx6_hdmireg.h @@ -0,0 +1,651 @@ +/*- + * Copyright (c) 2015 Oleksandr Tymoshenko <gonzo@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef __IMX6__HDMI_REGS_H__ +#define __IMX6__HDMI_REGS_H__ +#define HDMI_DESIGN_ID 0x0000 +#define HDMI_REVISION_ID 0x0001 +#define HDMI_PRODUCT_ID0 0x0002 +#define HDMI_PRODUCT_ID1 0x0003 + +/* Interrupt Registers */ +#define HDMI_IH_FC_STAT0 0x0100 +#define HDMI_IH_FC_STAT1 0x0101 +#define HDMI_IH_FC_STAT2 0x0102 +#define HDMI_IH_AS_STAT0 0x0103 +#define HDMI_IH_PHY_STAT0 0x0104 +#define HDMI_IH_PHY_STAT0_HPD (1 << 0) +#define HDMI_IH_I2CM_STAT0 0x0105 +#define HDMI_IH_CEC_STAT0 0x0106 +#define HDMI_IH_VP_STAT0 0x0107 +#define HDMI_IH_I2CMPHY_STAT0 0x0108 +#define HDMI_IH_I2CMPHY_STAT0_DONE (1 << 1) +#define HDMI_IH_I2CMPHY_STAT0_ERROR (1 << 0) +#define HDMI_IH_AHBDMAAUD_STAT0 0x0109 + +#define HDMI_IH_MUTE_FC_STAT0 0x0180 +#define HDMI_IH_MUTE_FC_STAT1 0x0181 +#define HDMI_IH_MUTE_FC_STAT2 0x0182 +#define HDMI_IH_MUTE_FC_STAT2_OVERFLOW_MASK (0x3) +#define HDMI_IH_MUTE_AS_STAT0 0x0183 +#define HDMI_IH_MUTE_PHY_STAT0 0x0184 +#define HDMI_IH_MUTE_I2CM_STAT0 0x0185 +#define HDMI_IH_MUTE_CEC_STAT0 0x0186 +#define HDMI_IH_MUTE_VP_STAT0 0x0187 +#define HDMI_IH_MUTE_I2CMPHY_STAT0 0x0188 +#define HDMI_IH_MUTE_AHBDMAAUD_STAT0 0x0189 +#define HDMI_IH_MUTE 0x01FF +#define HDMI_IH_MUTE_MUTE_WAKEUP_INTERRUPT (1<<1) +#define HDMI_IH_MUTE_MUTE_ALL_INTERRUPT (1<<0) + +/* Video Sample Registers */ +#define HDMI_TX_INVID0 0x0200 +#define HDMI_TX_INVID0_INTERNAL_DE_GENERATOR_MASK 0x80 +#define HDMI_TX_INVID0_INTERNAL_DE_GENERATOR_ENABLE 0x80 +#define HDMI_TX_INVID0_INTERNAL_DE_GENERATOR_DISABLE 0x00 +#define HDMI_TX_INVID0_VIDEO_MAPPING_MASK 0x1F +#define HDMI_TX_INVID0_VIDEO_MAPPING_OFFSET 0 +#define HDMI_TX_INSTUFFING 0x0201 +#define HDMI_TX_INSTUFFING_BDBDATA_STUFFING_MASK 0x4 +#define HDMI_TX_INSTUFFING_BDBDATA_STUFFING_ENABLE 0x4 +#define HDMI_TX_INSTUFFING_BDBDATA_STUFFING_DISABLE 0x0 +#define HDMI_TX_INSTUFFING_RCRDATA_STUFFING_MASK 0x2 +#define HDMI_TX_INSTUFFING_RCRDATA_STUFFING_ENABLE 0x2 +#define HDMI_TX_INSTUFFING_RCRDATA_STUFFING_DISABLE 0x0 +#define HDMI_TX_INSTUFFING_GYDATA_STUFFING_MASK 0x1 +#define HDMI_TX_INSTUFFING_GYDATA_STUFFING_ENABLE 0x1 +#define HDMI_TX_INSTUFFING_GYDATA_STUFFING_DISABLE 0x0 +#define HDMI_TX_GYDATA0 0x0202 +#define HDMI_TX_GYDATA1 0x0203 +#define HDMI_TX_RCRDATA0 0x0204 +#define HDMI_TX_RCRDATA1 0x0205 +#define HDMI_TX_BCBDATA0 0x0206 +#define HDMI_TX_BCBDATA1 0x0207 + +/* Video Packetizer Registers */ +#define HDMI_VP_STATUS 0x0800 +#define HDMI_VP_PR_CD 0x0801 +#define HDMI_VP_PR_CD_COLOR_DEPTH_MASK 0xF0 +#define HDMI_VP_PR_CD_COLOR_DEPTH_OFFSET 4 +#define HDMI_VP_PR_CD_DESIRED_PR_FACTOR_MASK 0x0F +#define HDMI_VP_PR_CD_DESIRED_PR_FACTOR_OFFSET 0 + +#define HDMI_VP_STUFF 0x0802 +#define HDMI_VP_STUFF_IDEFAULT_PHASE_MASK 0x20 +#define HDMI_VP_STUFF_IDEFAULT_PHASE_OFFSET 5 +#define HDMI_VP_STUFF_IFIX_PP_TO_LAST_MASK 0x10 +#define HDMI_VP_STUFF_IFIX_PP_TO_LAST_OFFSET 4 +#define HDMI_VP_STUFF_ICX_GOTO_P0_ST_MASK 0x8 +#define HDMI_VP_STUFF_ICX_GOTO_P0_ST_OFFSET 3 +#define HDMI_VP_STUFF_YCC422_STUFFING_MASK 0x4 +#define HDMI_VP_STUFF_YCC422_STUFFING_STUFFING_MODE 0x4 +#define HDMI_VP_STUFF_YCC422_STUFFING_DIRECT_MODE 0x0 +#define HDMI_VP_STUFF_PP_STUFFING_MASK 0x2 +#define HDMI_VP_STUFF_PP_STUFFING_STUFFING_MODE 0x2 +#define HDMI_VP_STUFF_PP_STUFFING_DIRECT_MODE 0x0 +#define HDMI_VP_STUFF_PR_STUFFING_MASK 0x1 +#define HDMI_VP_STUFF_PR_STUFFING_STUFFING_MODE 0x1 +#define HDMI_VP_STUFF_PR_STUFFING_DIRECT_MODE 0x0 +#define HDMI_VP_REMAP 0x0803 +#define HDMI_VP_REMAP_MASK 0x3 +#define HDMI_VP_REMAP_YCC422_24BIT 0x2 +#define HDMI_VP_REMAP_YCC422_20BIT 0x1 +#define HDMI_VP_REMAP_YCC422_16BIT 0x0 +#define HDMI_VP_CONF 0x0804 +#define HDMI_VP_CONF_BYPASS_EN_MASK 0x40 +#define HDMI_VP_CONF_BYPASS_EN_ENABLE 0x40 +#define HDMI_VP_CONF_BYPASS_EN_DISABLE 0x00 +#define HDMI_VP_CONF_PP_EN_ENMASK 0x20 +#define HDMI_VP_CONF_PP_EN_ENABLE 0x20 +#define HDMI_VP_CONF_PP_EN_DISABLE 0x00 +#define HDMI_VP_CONF_PR_EN_MASK 0x10 +#define HDMI_VP_CONF_PR_EN_ENABLE 0x10 +#define HDMI_VP_CONF_PR_EN_DISABLE 0x00 +#define HDMI_VP_CONF_YCC422_EN_MASK 0x8 +#define HDMI_VP_CONF_YCC422_EN_ENABLE 0x8 +#define HDMI_VP_CONF_YCC422_EN_DISABLE 0x0 +#define HDMI_VP_CONF_BYPASS_SELECT_MASK 0x4 +#define HDMI_VP_CONF_BYPASS_SELECT_VID_PACKETIZER 0x4 +#define HDMI_VP_CONF_BYPASS_SELECT_PIX_REPEATER 0x0 +#define HDMI_VP_CONF_OUTPUT_SELECTOR_MASK 0x3 +#define HDMI_VP_CONF_OUTPUT_SELECTOR_BYPASS 0x3 +#define HDMI_VP_CONF_OUTPUT_SELECTOR_YCC422 0x1 +#define HDMI_VP_CONF_OUTPUT_SELECTOR_PP 0x0 +#define HDMI_VP_STAT 0x0805 +#define HDMI_VP_INT 0x0806 +#define HDMI_VP_MASK 0x0807 +#define HDMI_VP_POL 0x0808 + +/* Frame Composer Registers */ +#define HDMI_FC_INVIDCONF 0x1000 +#define HDMI_FC_INVIDCONF_VSYNC_IN_POLARITY_ACTIVE_HIGH 0x40 +#define HDMI_FC_INVIDCONF_VSYNC_IN_POLARITY_ACTIVE_LOW 0x00 +#define HDMI_FC_INVIDCONF_HSYNC_IN_POLARITY_ACTIVE_HIGH 0x20 +#define HDMI_FC_INVIDCONF_HSYNC_IN_POLARITY_ACTIVE_LOW 0x00 +#define HDMI_FC_INVIDCONF_DE_IN_POLARITY_ACTIVE_HIGH 0x10 +#define HDMI_FC_INVIDCONF_DE_IN_POLARITY_ACTIVE_LOW 0x00 +#define HDMI_FC_INVIDCONF_DVI_MODEZ_HDMI_MODE 0x8 +#define HDMI_FC_INVIDCONF_DVI_MODEZ_DVI_MODE 0x0 +#define HDMI_FC_INVIDCONF_R_V_BLANK_IN_OSC_ACTIVE_HIGH 0x2 +#define HDMI_FC_INVIDCONF_R_V_BLANK_IN_OSC_ACTIVE_LOW 0x0 +#define HDMI_FC_INVIDCONF_IN_I_P_INTERLACED 0x1 +#define HDMI_FC_INVIDCONF_IN_I_P_PROGRESSIVE 0x0 +#define HDMI_FC_INHACTV0 0x1001 +#define HDMI_FC_INHACTV1 0x1002 +#define HDMI_FC_INHBLANK0 0x1003 +#define HDMI_FC_INHBLANK1 0x1004 +#define HDMI_FC_INVACTV0 0x1005 +#define HDMI_FC_INVACTV1 0x1006 +#define HDMI_FC_INVBLANK 0x1007 +#define HDMI_FC_HSYNCINDELAY0 0x1008 +#define HDMI_FC_HSYNCINDELAY1 0x1009 +#define HDMI_FC_HSYNCINWIDTH0 0x100A +#define HDMI_FC_HSYNCINWIDTH1 0x100B +#define HDMI_FC_VSYNCINDELAY 0x100C +#define HDMI_FC_VSYNCINWIDTH 0x100D +#define HDMI_FC_INFREQ0 0x100E +#define HDMI_FC_INFREQ1 0x100F +#define HDMI_FC_INFREQ2 0x1010 +#define HDMI_FC_CTRLDUR 0x1011 +#define HDMI_FC_EXCTRLDUR 0x1012 +#define HDMI_FC_EXCTRLSPAC 0x1013 +#define HDMI_FC_CH0PREAM 0x1014 +#define HDMI_FC_CH1PREAM 0x1015 +#define HDMI_FC_CH2PREAM 0x1016 +#define HDMI_FC_AVICONF3 0x1017 +#define HDMI_FC_GCP 0x1018 +#define HDMI_FC_AVICONF0 0x1019 +#define HDMI_FC_AVICONF1 0x101A +#define HDMI_FC_AVICONF2 0x101B +#define HDMI_FC_AVIVID 0x101C +#define HDMI_FC_AVIETB0 0x101D +#define HDMI_FC_AVIETB1 0x101E +#define HDMI_FC_AVISBB0 0x101F +#define HDMI_FC_AVISBB1 0x1020 +#define HDMI_FC_AVIELB0 0x1021 +#define HDMI_FC_AVIELB1 0x1022 +#define HDMI_FC_AVISRB0 0x1023 +#define HDMI_FC_AVISRB1 0x1024 +#define HDMI_FC_AUDICONF0 0x1025 +#define HDMI_FC_AUDICONF1 0x1026 +#define HDMI_FC_AUDICONF2 0x1027 +#define HDMI_FC_AUDICONF3 0x1028 +#define HDMI_FC_VSDIEEEID0 0x1029 +#define HDMI_FC_VSDSIZE 0x102A +#define HDMI_FC_VSDIEEEID1 0x1030 +#define HDMI_FC_VSDIEEEID2 0x1031 +#define HDMI_FC_VSDPAYLOAD0 0x1032 +#define HDMI_FC_VSDPAYLOAD1 0x1033 +#define HDMI_FC_VSDPAYLOAD2 0x1034 +#define HDMI_FC_VSDPAYLOAD3 0x1035 +#define HDMI_FC_VSDPAYLOAD4 0x1036 +#define HDMI_FC_VSDPAYLOAD5 0x1037 +#define HDMI_FC_VSDPAYLOAD6 0x1038 +#define HDMI_FC_VSDPAYLOAD7 0x1039 +#define HDMI_FC_VSDPAYLOAD8 0x103A +#define HDMI_FC_VSDPAYLOAD9 0x103B +#define HDMI_FC_VSDPAYLOAD10 0x103C +#define HDMI_FC_VSDPAYLOAD11 0x103D +#define HDMI_FC_VSDPAYLOAD12 0x103E +#define HDMI_FC_VSDPAYLOAD13 0x103F +#define HDMI_FC_VSDPAYLOAD14 0x1040 +#define HDMI_FC_VSDPAYLOAD15 0x1041 +#define HDMI_FC_VSDPAYLOAD16 0x1042 +#define HDMI_FC_VSDPAYLOAD17 0x1043 +#define HDMI_FC_VSDPAYLOAD18 0x1044 +#define HDMI_FC_VSDPAYLOAD19 0x1045 +#define HDMI_FC_VSDPAYLOAD20 0x1046 +#define HDMI_FC_VSDPAYLOAD21 0x1047 +#define HDMI_FC_VSDPAYLOAD22 0x1048 +#define HDMI_FC_VSDPAYLOAD23 0x1049 +#define HDMI_FC_SPDVENDORNAME0 0x104A +#define HDMI_FC_SPDVENDORNAME1 0x104B +#define HDMI_FC_SPDVENDORNAME2 0x104C +#define HDMI_FC_SPDVENDORNAME3 0x104D +#define HDMI_FC_SPDVENDORNAME4 0x104E +#define HDMI_FC_SPDVENDORNAME5 0x104F +#define HDMI_FC_SPDVENDORNAME6 0x1050 +#define HDMI_FC_SPDVENDORNAME7 0x1051 +#define HDMI_FC_SDPPRODUCTNAME0 0x1052 +#define HDMI_FC_SDPPRODUCTNAME1 0x1053 +#define HDMI_FC_SDPPRODUCTNAME2 0x1054 +#define HDMI_FC_SDPPRODUCTNAME3 0x1055 +#define HDMI_FC_SDPPRODUCTNAME4 0x1056 +#define HDMI_FC_SDPPRODUCTNAME5 0x1057 +#define HDMI_FC_SDPPRODUCTNAME6 0x1058 +#define HDMI_FC_SDPPRODUCTNAME7 0x1059 +#define HDMI_FC_SDPPRODUCTNAME8 0x105A +#define HDMI_FC_SDPPRODUCTNAME9 0x105B +#define HDMI_FC_SDPPRODUCTNAME10 0x105C +#define HDMI_FC_SDPPRODUCTNAME11 0x105D +#define HDMI_FC_SDPPRODUCTNAME12 0x105E +#define HDMI_FC_SDPPRODUCTNAME13 0x105F +#define HDMI_FC_SDPPRODUCTNAME14 0x1060 +#define HDMI_FC_SPDPRODUCTNAME15 0x1061 +#define HDMI_FC_SPDDEVICEINF 0x1062 +#define HDMI_FC_AUDSCONF 0x1063 +#define HDMI_FC_AUDSSTAT 0x1064 +#define HDMI_FC_DATACH0FILL 0x1070 +#define HDMI_FC_DATACH1FILL 0x1071 +#define HDMI_FC_DATACH2FILL 0x1072 +#define HDMI_FC_CTRLQHIGH 0x1073 +#define HDMI_FC_CTRLQLOW 0x1074 +#define HDMI_FC_ACP0 0x1075 +#define HDMI_FC_ACP28 0x1076 +#define HDMI_FC_ACP27 0x1077 +#define HDMI_FC_ACP26 0x1078 +#define HDMI_FC_ACP25 0x1079 +#define HDMI_FC_ACP24 0x107A +#define HDMI_FC_ACP23 0x107B +#define HDMI_FC_ACP22 0x107C +#define HDMI_FC_ACP21 0x107D +#define HDMI_FC_ACP20 0x107E +#define HDMI_FC_ACP19 0x107F +#define HDMI_FC_ACP18 0x1080 +#define HDMI_FC_ACP17 0x1081 +#define HDMI_FC_ACP16 0x1082 +#define HDMI_FC_ACP15 0x1083 +#define HDMI_FC_ACP14 0x1084 +#define HDMI_FC_ACP13 0x1085 +#define HDMI_FC_ACP12 0x1086 +#define HDMI_FC_ACP11 0x1087 +#define HDMI_FC_ACP10 0x1088 +#define HDMI_FC_ACP9 0x1089 +#define HDMI_FC_ACP8 0x108A +#define HDMI_FC_ACP7 0x108B +#define HDMI_FC_ACP6 0x108C +#define HDMI_FC_ACP5 0x108D +#define HDMI_FC_ACP4 0x108E +#define HDMI_FC_ACP3 0x108F +#define HDMI_FC_ACP2 0x1090 +#define HDMI_FC_ACP1 0x1091 +#define HDMI_FC_ISCR1_0 0x1092 +#define HDMI_FC_ISCR1_16 0x1093 +#define HDMI_FC_ISCR1_15 0x1094 +#define HDMI_FC_ISCR1_14 0x1095 +#define HDMI_FC_ISCR1_13 0x1096 +#define HDMI_FC_ISCR1_12 0x1097 +#define HDMI_FC_ISCR1_11 0x1098 +#define HDMI_FC_ISCR1_10 0x1099 +#define HDMI_FC_ISCR1_9 0x109A +#define HDMI_FC_ISCR1_8 0x109B +#define HDMI_FC_ISCR1_7 0x109C +#define HDMI_FC_ISCR1_6 0x109D +#define HDMI_FC_ISCR1_5 0x109E +#define HDMI_FC_ISCR1_4 0x109F +#define HDMI_FC_ISCR1_3 0x10A0 +#define HDMI_FC_ISCR1_2 0x10A1 +#define HDMI_FC_ISCR1_1 0x10A2 +#define HDMI_FC_ISCR2_15 0x10A3 +#define HDMI_FC_ISCR2_14 0x10A4 +#define HDMI_FC_ISCR2_13 0x10A5 +#define HDMI_FC_ISCR2_12 0x10A6 +#define HDMI_FC_ISCR2_11 0x10A7 +#define HDMI_FC_ISCR2_10 0x10A8 +#define HDMI_FC_ISCR2_9 0x10A9 +#define HDMI_FC_ISCR2_8 0x10AA +#define HDMI_FC_ISCR2_7 0x10AB +#define HDMI_FC_ISCR2_6 0x10AC +#define HDMI_FC_ISCR2_5 0x10AD +#define HDMI_FC_ISCR2_4 0x10AE +#define HDMI_FC_ISCR2_3 0x10AF +#define HDMI_FC_ISCR2_2 0x10B0 +#define HDMI_FC_ISCR2_1 0x10B1 +#define HDMI_FC_ISCR2_0 0x10B2 +#define HDMI_FC_DATAUTO0 0x10B3 +#define HDMI_FC_DATAUTO1 0x10B4 +#define HDMI_FC_DATAUTO2 0x10B5 +#define HDMI_FC_DATMAN 0x10B6 +#define HDMI_FC_DATAUTO3 0x10B7 +#define HDMI_FC_RDRB0 0x10B8 +#define HDMI_FC_RDRB1 0x10B9 +#define HDMI_FC_RDRB2 0x10BA +#define HDMI_FC_RDRB3 0x10BB +#define HDMI_FC_RDRB4 0x10BC +#define HDMI_FC_RDRB5 0x10BD +#define HDMI_FC_RDRB6 0x10BE +#define HDMI_FC_RDRB7 0x10BF +#define HDMI_FC_STAT0 0x10D0 +#define HDMI_FC_INT0 0x10D1 +#define HDMI_FC_MASK0 0x10D2 +#define HDMI_FC_POL0 0x10D3 +#define HDMI_FC_STAT1 0x10D4 +#define HDMI_FC_INT1 0x10D5 +#define HDMI_FC_MASK1 0x10D6 +#define HDMI_FC_POL1 0x10D7 +#define HDMI_FC_STAT2 0x10D8 +#define HDMI_FC_INT2 0x10D9 +#define HDMI_FC_MASK2 0x10DA +#define HDMI_FC_MASK2_LOW_PRI (1 << 1) +#define HDMI_FC_MASK2_HIGH_PRI (1 << 0) +#define HDMI_FC_POL2 0x10DB +#define HDMI_FC_PRCONF 0x10E0 + +#define HDMI_FC_GMD_STAT 0x1100 +#define HDMI_FC_GMD_EN 0x1101 +#define HDMI_FC_GMD_UP 0x1102 +#define HDMI_FC_GMD_CONF 0x1103 +#define HDMI_FC_GMD_HB 0x1104 +#define HDMI_FC_GMD_PB0 0x1105 +#define HDMI_FC_GMD_PB1 0x1106 +#define HDMI_FC_GMD_PB2 0x1107 +#define HDMI_FC_GMD_PB3 0x1108 +#define HDMI_FC_GMD_PB4 0x1109 +#define HDMI_FC_GMD_PB5 0x110A +#define HDMI_FC_GMD_PB6 0x110B +#define HDMI_FC_GMD_PB7 0x110C +#define HDMI_FC_GMD_PB8 0x110D +#define HDMI_FC_GMD_PB9 0x110E +#define HDMI_FC_GMD_PB10 0x110F +#define HDMI_FC_GMD_PB11 0x1110 +#define HDMI_FC_GMD_PB12 0x1111 +#define HDMI_FC_GMD_PB13 0x1112 +#define HDMI_FC_GMD_PB14 0x1113 +#define HDMI_FC_GMD_PB15 0x1114 +#define HDMI_FC_GMD_PB16 0x1115 +#define HDMI_FC_GMD_PB17 0x1116 +#define HDMI_FC_GMD_PB18 0x1117 +#define HDMI_FC_GMD_PB19 0x1118 +#define HDMI_FC_GMD_PB20 0x1119 +#define HDMI_FC_GMD_PB21 0x111A +#define HDMI_FC_GMD_PB22 0x111B +#define HDMI_FC_GMD_PB23 0x111C +#define HDMI_FC_GMD_PB24 0x111D +#define HDMI_FC_GMD_PB25 0x111E +#define HDMI_FC_GMD_PB26 0x111F +#define HDMI_FC_GMD_PB27 0x1120 + +#define HDMI_FC_DBGFORCE 0x1200 +#define HDMI_FC_DBGAUD0CH0 0x1201 +#define HDMI_FC_DBGAUD1CH0 0x1202 +#define HDMI_FC_DBGAUD2CH0 0x1203 +#define HDMI_FC_DBGAUD0CH1 0x1204 +#define HDMI_FC_DBGAUD1CH1 0x1205 +#define HDMI_FC_DBGAUD2CH1 0x1206 +#define HDMI_FC_DBGAUD0CH2 0x1207 +#define HDMI_FC_DBGAUD1CH2 0x1208 +#define HDMI_FC_DBGAUD2CH2 0x1209 +#define HDMI_FC_DBGAUD0CH3 0x120A +#define HDMI_FC_DBGAUD1CH3 0x120B +#define HDMI_FC_DBGAUD2CH3 0x120C +#define HDMI_FC_DBGAUD0CH4 0x120D +#define HDMI_FC_DBGAUD1CH4 0x120E +#define HDMI_FC_DBGAUD2CH4 0x120F +#define HDMI_FC_DBGAUD0CH5 0x1210 +#define HDMI_FC_DBGAUD1CH5 0x1211 +#define HDMI_FC_DBGAUD2CH5 0x1212 +#define HDMI_FC_DBGAUD0CH6 0x1213 +#define HDMI_FC_DBGAUD1CH6 0x1214 +#define HDMI_FC_DBGAUD2CH6 0x1215 +#define HDMI_FC_DBGAUD0CH7 0x1216 +#define HDMI_FC_DBGAUD1CH7 0x1217 +#define HDMI_FC_DBGAUD2CH7 0x1218 +#define HDMI_FC_DBGTMDS0 0x1219 +#define HDMI_FC_DBGTMDS1 0x121A +#define HDMI_FC_DBGTMDS2 0x121B + +#define HDMI_PHY_CONF0 0x3000 +#define HDMI_PHY_CONF0_PDZ_MASK 0x80 +#define HDMI_PHY_CONF0_PDZ_OFFSET 7 +#define HDMI_PHY_CONF0_ENTMDS_MASK 0x40 +#define HDMI_PHY_CONF0_ENTMDS_OFFSET 6 +#define HDMI_PHY_CONF0_SPARECTRL 0x20 +#define HDMI_PHY_CONF0_GEN2_PDDQ_MASK 0x10 +#define HDMI_PHY_CONF0_GEN2_PDDQ_OFFSET 4 +#define HDMI_PHY_CONF0_GEN2_TXPWRON_MASK 0x8 +#define HDMI_PHY_CONF0_GEN2_TXPWRON_OFFSET 3 +#define HDMI_PHY_CONF0_GEN2_ENHPDRXSENSE_MASK 0x4 +#define HDMI_PHY_CONF0_GEN2_ENHPDRXSENSE_OFFSET 2 +#define HDMI_PHY_CONF0_SELDATAENPOL_MASK 0x2 +#define HDMI_PHY_CONF0_SELDATAENPOL_OFFSET 1 +#define HDMI_PHY_CONF0_SELDIPIF_MASK 0x1 +#define HDMI_PHY_CONF0_SELDIPIF_OFFSET 0 +#define HDMI_PHY_TST0 0x3001 +#define HDMI_PHY_TST0_TSTCLR_MASK 0x20 +#define HDMI_PHY_TST0_TSTCLR_OFFSET 5 +#define HDMI_PHY_TST0_TSTEN_MASK 0x10 +#define HDMI_PHY_TST0_TSTEN_OFFSET 4 +#define HDMI_PHY_TST0_TSTCLK_MASK 0x1 +#define HDMI_PHY_TST0_TSTCLK_OFFSET 0 +#define HDMI_PHY_TST1 0x3002 +#define HDMI_PHY_TST2 0x3003 +#define HDMI_PHY_STAT0 0x3004 +#define HDMI_PHY_TX_PHY_LOCK 0x01 +#define HDMI_PHY_INT0 0x3005 +#define HDMI_PHY_MASK0 0x3006 +#define HDMI_PHY_POL0 0x3007 +#define HDMI_PHY_HPD (1 << 1) + +/* HDMI Master PHY Registers */ +#define HDMI_PHY_I2CM_SLAVE_ADDR 0x3020 +#define HDMI_PHY_I2CM_SLAVE_ADDR_PHY_GEN2 0x69 +#define HDMI_PHY_I2CM_SLAVE_ADDR_HEAC_PHY 0x49 +#define HDMI_PHY_I2CM_ADDRESS_ADDR 0x3021 +#define HDMI_PHY_I2CM_DATAO_1_ADDR 0x3022 +#define HDMI_PHY_I2CM_DATAO_0_ADDR 0x3023 +#define HDMI_PHY_I2CM_DATAI_1_ADDR 0x3024 +#define HDMI_PHY_I2CM_DATAI_0_ADDR 0x3025 +#define HDMI_PHY_I2CM_OPERATION_ADDR 0x3026 +#define HDMI_PHY_I2CM_INT_ADDR 0x3027 +#define HDMI_PHY_I2CM_CTLINT_ADDR 0x3028 +#define HDMI_PHY_I2CM_DIV_ADDR 0x3029 +#define HDMI_PHY_I2CM_SOFTRSTZ_ADDR 0x302a +#define HDMI_PHY_I2CM_SS_SCL_HCNT_1_ADDR 0x302b +#define HDMI_PHY_I2CM_SS_SCL_HCNT_0_ADDR 0x302c +#define HDMI_PHY_I2CM_SS_SCL_LCNT_1_ADDR 0x302d +#define HDMI_PHY_I2CM_SS_SCL_LCNT_0_ADDR 0x302e +#define HDMI_PHY_I2CM_FS_SCL_HCNT_1_ADDR 0x302f +#define HDMI_PHY_I2CM_FS_SCL_HCNT_0_ADDR 0x3030 +#define HDMI_PHY_I2CM_FS_SCL_LCNT_1_ADDR 0x3031 +#define HDMI_PHY_I2CM_FS_SCL_LCNT_0_ADDR 0x3032 + +/* Audio Sampler Registers */ +#define HDMI_AUD_CONF0 0x3100 +#define HDMI_AUD_CONF1 0x3101 +#define HDMI_AUD_INT 0x3102 +#define HDMI_AUD_CONF2 0x3103 +#define HDMI_AUD_N1 0x3200 +#define HDMI_AUD_N2 0x3201 +#define HDMI_AUD_N3 0x3202 +#define HDMI_AUD_CTS1 0x3203 +#define HDMI_AUD_CTS2 0x3204 +#define HDMI_AUD_CTS3 0x3205 +#define HDMI_AUD_INPUTCLKFS 0x3206 +#define HDMI_AUD_SPDIFINT 0x3302 +#define HDMI_AUD_CONF0_HBR 0x3400 +#define HDMI_AUD_HBR_STATUS 0x3401 +#define HDMI_AUD_HBR_INT 0x3402 +#define HDMI_AUD_HBR_POL 0x3403 +#define HDMI_AUD_HBR_MASK 0x3404 + +/* + * Generic Parallel Audio Interface Registers + * Not used as GPAUD interface is not enabled in hw + */ +#define HDMI_GP_CONF0 0x3500 +#define HDMI_GP_CONF1 0x3501 +#define HDMI_GP_CONF2 0x3502 +#define HDMI_GP_STAT 0x3503 +#define HDMI_GP_INT 0x3504 +#define HDMI_GP_MASK 0x3505 +#define HDMI_GP_POL 0x3506 + +/* Main Controller Registers */ +#define HDMI_MC_SFRDIV 0x4000 +#define HDMI_MC_CLKDIS 0x4001 +#define HDMI_MC_CLKDIS_HDCPCLK_DISABLE (1 << 6) +#define HDMI_MC_CLKDIS_CECCLK_DISABLE (1 << 5) +#define HDMI_MC_CLKDIS_CSCCLK_DISABLE (1 << 4) +#define HDMI_MC_CLKDIS_AUDCLK_DISABLE (1 << 3) +#define HDMI_MC_CLKDIS_PREPCLK_DISABLE (1 << 2) +#define HDMI_MC_CLKDIS_TMDSCLK_DISABLE (1 << 1) +#define HDMI_MC_CLKDIS_PIXELCLK_DISABLE (1 << 0) + +#define HDMI_MC_SWRSTZ 0x4002 +#define HDMI_MC_SWRSTZ_TMDSSWRST_REQ 0x02 +#define HDMI_MC_OPCTRL 0x4003 +#define HDMI_MC_FLOWCTRL 0x4004 +#define HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_MASK 0x1 +#define HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_IN_PATH 0x1 +#define HDMI_MC_FLOWCTRL_FEED_THROUGH_OFF_CSC_BYPASS 0x0 +#define HDMI_MC_PHYRSTZ 0x4005 +#define HDMI_MC_PHYRSTZ_ASSERT 0x0 +#define HDMI_MC_PHYRSTZ_DEASSERT 0x1 +#define HDMI_MC_LOCKONCLOCK 0x4006 +#define HDMI_MC_HEACPHY_RST 0x4007 +#define HDMI_MC_HEACPHY_RST_ASSERT 0x1 +#define HDMI_MC_HEACPHY_RST_DEASSERT 0x0 + +/* HDCP Encryption Engine Registers */ +#define HDMI_A_HDCPCFG0 0x5000 +#define HDMI_A_HDCPCFG1 0x5001 +#define HDMI_A_HDCPOBS0 0x5002 +#define HDMI_A_HDCPOBS1 0x5003 +#define HDMI_A_HDCPOBS2 0x5004 +#define HDMI_A_HDCPOBS3 0x5005 +#define HDMI_A_APIINTCLR 0x5006 +#define HDMI_A_APIINTSTAT 0x5007 +#define HDMI_A_APIINTMSK 0x5008 +#define HDMI_A_VIDPOLCFG 0x5009 +#define HDMI_A_OESSWCFG 0x500A +#define HDMI_A_TIMER1SETUP0 0x500B +#define HDMI_A_TIMER1SETUP1 0x500C +#define HDMI_A_TIMER2SETUP0 0x500D +#define HDMI_A_TIMER2SETUP1 0x500E +#define HDMI_A_100MSCFG 0x500F +#define HDMI_A_2SCFG0 0x5010 +#define HDMI_A_2SCFG1 0x5011 +#define HDMI_A_5SCFG0 0x5012 +#define HDMI_A_5SCFG1 0x5013 +#define HDMI_A_SRMVERLSB 0x5014 +#define HDMI_A_SRMVERMSB 0x5015 +#define HDMI_A_SRMCTRL 0x5016 +#define HDMI_A_SFRSETUP 0x5017 +#define HDMI_A_I2CHSETUP 0x5018 +#define HDMI_A_INTSETUP 0x5019 +#define HDMI_A_PRESETUP 0x501A +#define HDMI_A_SRM_BASE 0x5020 + +/* CEC Engine Registers */ +#define HDMI_CEC_CTRL 0x7D00 +#define HDMI_CEC_STAT 0x7D01 +#define HDMI_CEC_MASK 0x7D02 +#define HDMI_CEC_POLARITY 0x7D03 +#define HDMI_CEC_INT 0x7D04 +#define HDMI_CEC_ADDR_L 0x7D05 +#define HDMI_CEC_ADDR_H 0x7D06 +#define HDMI_CEC_TX_CNT 0x7D07 +#define HDMI_CEC_RX_CNT 0x7D08 +#define HDMI_CEC_TX_DATA0 0x7D10 +#define HDMI_CEC_TX_DATA1 0x7D11 +#define HDMI_CEC_TX_DATA2 0x7D12 +#define HDMI_CEC_TX_DATA3 0x7D13 +#define HDMI_CEC_TX_DATA4 0x7D14 +#define HDMI_CEC_TX_DATA5 0x7D15 +#define HDMI_CEC_TX_DATA6 0x7D16 +#define HDMI_CEC_TX_DATA7 0x7D17 +#define HDMI_CEC_TX_DATA8 0x7D18 +#define HDMI_CEC_TX_DATA9 0x7D19 +#define HDMI_CEC_TX_DATA10 0x7D1a +#define HDMI_CEC_TX_DATA11 0x7D1b +#define HDMI_CEC_TX_DATA12 0x7D1c +#define HDMI_CEC_TX_DATA13 0x7D1d +#define HDMI_CEC_TX_DATA14 0x7D1e +#define HDMI_CEC_TX_DATA15 0x7D1f +#define HDMI_CEC_RX_DATA0 0x7D20 +#define HDMI_CEC_RX_DATA1 0x7D21 +#define HDMI_CEC_RX_DATA2 0x7D22 +#define HDMI_CEC_RX_DATA3 0x7D23 +#define HDMI_CEC_RX_DATA4 0x7D24 +#define HDMI_CEC_RX_DATA5 0x7D25 +#define HDMI_CEC_RX_DATA6 0x7D26 +#define HDMI_CEC_RX_DATA7 0x7D27 +#define HDMI_CEC_RX_DATA8 0x7D28 +#define HDMI_CEC_RX_DATA9 0x7D29 +#define HDMI_CEC_RX_DATA10 0x7D2a +#define HDMI_CEC_RX_DATA11 0x7D2b +#define HDMI_CEC_RX_DATA12 0x7D2c +#define HDMI_CEC_RX_DATA13 0x7D2d +#define HDMI_CEC_RX_DATA14 0x7D2e +#define HDMI_CEC_RX_DATA15 0x7D2f +#define HDMI_CEC_LOCK 0x7D30 +#define HDMI_CEC_WKUPCTRL 0x7D31 + +/* I2C Master Registers (E-DDC) */ +#define HDMI_I2CM_SLAVE 0x7E00 +#define HDMI_I2CMESS 0x7E01 +#define HDMI_I2CM_DATAO 0x7E02 +#define HDMI_I2CM_DATAI 0x7E03 +#define HDMI_I2CM_OPERATION 0x7E04 +#define HDMI_PHY_I2CM_OPERATION_ADDR_WRITE 0x10 +#define HDMI_PHY_I2CM_OPERATION_ADDR_READ 0x1 +#define HDMI_I2CM_INT 0x7E05 +#define HDMI_I2CM_CTLINT 0x7E06 +#define HDMI_I2CM_DIV 0x7E07 +#define HDMI_I2CM_SEGADDR 0x7E08 +#define HDMI_I2CM_SOFTRSTZ 0x7E09 +#define HDMI_I2CM_SEGPTR 0x7E0A +#define HDMI_I2CM_SS_SCL_HCNT_1_ADDR 0x7E0B +#define HDMI_I2CM_SS_SCL_HCNT_0_ADDR 0x7E0C +#define HDMI_I2CM_SS_SCL_LCNT_1_ADDR 0x7E0D +#define HDMI_I2CM_SS_SCL_LCNT_0_ADDR 0x7E0E +#define HDMI_I2CM_FS_SCL_HCNT_1_ADDR 0x7E0F +#define HDMI_I2CM_FS_SCL_HCNT_0_ADDR 0x7E10 +#define HDMI_I2CM_FS_SCL_LCNT_1_ADDR 0x7E11 +#define HDMI_I2CM_FS_SCL_LCNT_0_ADDR 0x7E12 + +/* HDMI PHY register with access through I2C */ +#define HDMI_PHY_I2C_CKCALCTRL 0x5 +#define CKCALCTRL_OVERRIDE (1 << 15) +#define HDMI_PHY_I2C_CPCE_CTRL 0x6 +#define CPCE_CTRL_45_25 ((3 << 7) | (3 << 5)) +#define CPCE_CTRL_92_50 ((2 << 7) | (2 << 5)) +#define CPCE_CTRL_185 ((1 << 7) | (1 << 5)) +#define CPCE_CTRL_370 ((0 << 7) | (0 << 5)) +#define HDMI_PHY_I2C_CKSYMTXCTRL 0x9 +#define CKSYMTXCTRL_OVERRIDE (1 << 15) +#define CKSYMTXCTRL_TX_SYMON (1 << 3) +#define CKSYMTXCTRL_TX_TRAON (1 << 2) +#define CKSYMTXCTRL_TX_TRBON (1 << 1) +#define CKSYMTXCTRL_TX_CK_SYMON (1 << 0) +#define HDMI_PHY_I2C_VLEVCTRL 0x0E +#define HDMI_PHY_I2C_CURRCTRL 0x10 +#define HDMI_PHY_I2C_PLLPHBYCTRL 0x13 +#define VLEVCTRL_TX_LVL(x) ((x) << 5) +#define VLEVCTRL_CK_LVL(x) (x) +#define HDMI_PHY_I2C_GMPCTRL 0x15 +#define GMPCTRL_45_25 0x00 +#define GMPCTRL_92_50 0x05 +#define GMPCTRL_185 0x0a +#define GMPCTRL_370 0x0f +#define HDMI_PHY_I2C_MSM_CTRL 0x17 +#define MSM_CTRL_FB_CLK (0x3 << 1) +#define HDMI_PHY_I2C_TXTERM 0x19 +#define TXTERM_133 0x5 + +#endif /* __IMX6__HDMI_REGS_H__ */ diff --git a/sys/arm/freescale/imx/imx6_ipu.c b/sys/arm/freescale/imx/imx6_ipu.c new file mode 100644 index 0000000..d1efbff --- /dev/null +++ b/sys/arm/freescale/imx/imx6_ipu.c @@ -0,0 +1,1203 @@ +/*- + * Copyright 2015 Oleksandr Tymoshenko <gonzo@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/systm.h> +#include <sys/kernel.h> +#include <sys/module.h> +#include <sys/clock.h> +#include <sys/time.h> +#include <sys/bus.h> +#include <sys/lock.h> +#include <sys/mutex.h> +#include <sys/resource.h> +#include <sys/rman.h> +#include <sys/sysctl.h> +#include <sys/fbio.h> +#include <sys/consio.h> + +#include <machine/bus.h> + +#include <dev/fdt/fdt_common.h> +#include <dev/ofw/openfirm.h> +#include <dev/ofw/ofw_bus.h> +#include <dev/ofw/ofw_bus_subr.h> + +#include <dev/fb/fbreg.h> +#include <dev/vt/vt.h> + +#include <dev/videomode/videomode.h> +#include <dev/videomode/edidvar.h> + +#include <arm/freescale/imx/imx6_src.h> +#include <arm/freescale/imx/imx_ccmvar.h> + +#include "fb_if.h" +#include "hdmi_if.h" + +#define EDID_DEBUG_not + +static int have_ipu = 0; + +#define LDB_CLOCK_RATE 280000000 + +#define MODE_HBP(mode) ((mode)->htotal - (mode)->hsync_end) +#define MODE_HFP(mode) ((mode)->hsync_start - (mode)->hdisplay) +#define MODE_HSW(mode) ((mode)->hsync_end - (mode)->hsync_start) +#define MODE_VBP(mode) ((mode)->vtotal - (mode)->vsync_end) +#define MODE_VFP(mode) ((mode)->vsync_start - (mode)->vdisplay) +#define MODE_VSW(mode) ((mode)->vsync_end - (mode)->vsync_start) + +#define MODE_BPP 16 +#define MODE_PIXEL_CLOCK_INVERT 1 + +#define M(nm,hr,vr,clk,hs,he,ht,vs,ve,vt,f) \ + { clk, hr, hs, he, ht, vr, vs, ve, vt, f, nm } + +static struct videomode mode1024x768 = M("1024x768x60",1024,768,65000,1048,1184,1344,771,777,806,VID_NHSYNC|VID_PHSYNC); + +#define DMA_CHANNEL 23 +#define DC_CHAN5 5 +#define DI_PORT 0 + +#define IPU_LOCK(_sc) mtx_lock(&(_sc)->sc_mtx) +#define IPU_UNLOCK(_sc) mtx_unlock(&(_sc)->sc_mtx) +#define IPU_LOCK_INIT(_sc) mtx_init(&(_sc)->sc_mtx, \ + device_get_nameunit(_sc->sc_dev), "ipu", MTX_DEF) +#define IPU_LOCK_DESTROY(_sc) mtx_destroy(&(_sc)->sc_mtx) + +#define IPU_READ4(_sc, reg) bus_read_4((_sc)->sc_mem_res, (reg)) +#define IPU_WRITE4(_sc, reg, value) \ + bus_write_4((_sc)->sc_mem_res, (reg), (value)) + +#define CPMEM_BASE 0x300000 +#define DC_TEMPL_BASE 0x380000 + +/* Microcode */ +/* Word 1 */ +#define TEMPLATE_SYNC(v) ((v) << 0) +#define TEMPLATE_GLUELOGIC(v) ((v) << 4) +#define TEMPLATE_MAPPING(v) ((v) << 15) +#define TEMPLATE_WAVEFORM(v) ((v) << 11) +#define GLUELOGIC_KEEP_ASSERTED (1 << 3) +#define GLUELOGIC_KEEP_NEGATED (1 << 2) +/* Word 2 */ +#define TEMPLATE_OPCODE(v) ((v) << 4) +#define OPCODE_WROD 0x18 +#define TEMPLATE_STOP (1 << 9) + +#define IPU_CONF 0x200000 +#define IPU_CONF_DMFC_EN (1 << 10) +#define IPU_CONF_DC_EN (1 << 9) +#define IPU_CONF_DI1_EN (1 << 7) +#define IPU_CONF_DI0_EN (1 << 6) +#define IPU_CONF_DP_EN (1 << 5) +#define IPU_DISP_GEN 0x2000C4 +#define DISP_GEN_DI1_CNTR_RELEASE (1 << 25) +#define DISP_GEN_DI0_CNTR_RELEASE (1 << 24) +#define DISP_GEN_MCU_MAX_BURST_STOP (1 << 22) +#define DISP_GEN_MCU_T_SHIFT 18 +#define IPU_MEM_RST 0x2000DC +#define IPU_MEM_RST_START (1 << 31) +#define IPU_MEM_RST_ALL 0x807FFFFF +#define IPU_CH_DB_MODE_SEL_0 0x200150 +#define IPU_CH_DB_MODE_SEL_1 0x200154 +#define IPU_CUR_BUF_0 0x20023C +#define IPU_CUR_BUF_1 0x200240 + +#define IPU_IDMAC_CH_EN_1 0x208004 +#define IPU_IDMAC_CH_EN_2 0x208008 +#define IPU_IDMAC_CH_PRI_1 0x208014 +#define IPU_IDMAC_CH_PRI_2 0x208018 + +#define IPU_DI0_GENERAL 0x240000 +#define DI_CLOCK_EXTERNAL (1 << 20) +#define DI_GENERAL_POL_CLK (1 << 17) +#define DI_GENERAL_POLARITY_3 (1 << 2) +#define DI_GENERAL_POLARITY_2 (1 << 1) +#define IPU_DI0_BS_CLKGEN0 0x240004 +#define DI_BS_CLKGEN0(_int, _frac) (((_int) << 4) | (_frac)) +#define IPU_DI0_BS_CLKGEN1 0x240008 +#define DI_BS_CLKGEN1_DOWN(_int, _frac) ((((_int) << 1) | (_frac)) << 16) +#define IPU_DI0_SW_GEN0_1 0x24000C +#define DI_RUN_VALUE_M1(v) ((v) << 19) +#define DI_RUN_RESOLUTION(v) ((v) << 16) +#define DI_OFFSET_VALUE(v) ((v) << 3) +#define IPU_DI0_SW_GEN1_1 0x240030 +#define DI_CNT_POLARITY_GEN_EN(v) ((v) << 29) +#define DI_CNT_AUTO_RELOAD (1 << 28) +#define DI_CNT_CLR_SEL(v) ((v) << 25) +#define DI_CNT_DOWN(v) ((v) << 16) +#define DI_CNT_POLARITY_TRIGGER_SEL(v) ((v) << 12) +#define DI_CNT_POLARITY_CLR_SEL(v) ((v) << 9) +#define IPU_DI0_SYNC_AS_GEN 0x240054 +#define SYNC_AS_GEN_VSYNC_SEL(v) ((v) << 13) +#define SYNC_AS_GEN_SYNC_START(v) ((v) << 0) +#define IPU_DI0_DW_GEN_0 0x240058 +#define DW_GEN_DI_ACCESS_SIZE(v) ((v) << 24) +#define DW_GEN_DI_COMPONENT_SIZE(v) ((v) << 16) +#define DW_GEN_DI_SET_MASK 3 +#define DW_GEN_DI_PIN_15_SET(v) ((v) << 8) +#define IPU_DI0_DW_SET3_0 0x240118 +#define DW_SET_DATA_CNT_DOWN(v) ((v) << 16) +#define DW_SET_DATA_CNT_UP(v) ((v) << 0) +#define IPU_DI0_STP_REP 0x240148 +#define IPU_DI0_POL 0x240164 +#define DI_POL_DRDY_POLARITY_15 (1 << 4) +#define IPU_DI0_SCR_CONF 0x240170 + +#define IPU_DI1_GENERAL 0x248000 +#define IPU_DI1_BS_CLKGEN0 0x248004 +#define IPU_DI1_BS_CLKGEN1 0x248008 +#define IPU_DI1_SW_GEN0_1 0x24800C +#define IPU_DI1_SW_GEN1_1 0x248030 +#define IPU_DI1_SYNC_AS_GEN 0x248054 +#define IPU_DI1_DW_GEN_0 0x248058 +#define IPU_DI1_POL 0x248164 +#define IPU_DI1_DW_SET3_0 0x248118 +#define IPU_DI1_STP_REP 0x248148 +#define IPU_DI1_SCR_CONF 0x248170 +#define DMFC_RD_CHAN 0x260000 +#define DMFC_WR_CHAN 0x260004 +#define DMFC_WR_CHAN_BURST_SIZE_32 (0 << 6) +#define DMFC_WR_CHAN_BURST_SIZE_16 (1 << 6) +#define DMFC_WR_CHAN_BURST_SIZE_8 (2 << 6) +#define DMFC_WR_CHAN_BURST_SIZE_4 (3 << 6) +#define DMFC_WR_CHAN_BURST_SIZE_4 (3 << 6) +#define DMFC_WR_CHAN_FIFO_SIZE_128 (2 << 3) +#define DMFC_WR_CHAN_DEF 0x260008 +#define DMFC_WR_CHAN_DEF_WM_CLR_2C(v) ((v) << 29) +#define DMFC_WR_CHAN_DEF_WM_CLR_1C(v) ((v) << 21) +#define DMFC_WR_CHAN_DEF_WM_CLR_2(v) ((v) << 13) +#define DMFC_WR_CHAN_DEF_WM_CLR_1(v) ((v) << 5) +#define DMFC_WR_CHAN_DEF_WM_SET_1(v) ((v) << 2) +#define DMFC_WR_CHAN_DEF_WM_EN_1 (1 << 1) +#define DMFC_DP_CHAN 0x26000C +#define DMFC_DP_CHAN_BURST_SIZE_8 2 +#define DMFC_DP_CHAN_FIFO_SIZE_256 1 +#define DMFC_DP_CHAN_FIFO_SIZE_128 2 +#define DMFC_DP_CHAN_BURST_SIZE_5F(v) ((v) << 14) +#define DMFC_DP_CHAN_FIFO_SIZE_5F(v) ((v) << 11) +#define DMFC_DP_CHAN_ST_ADDR_SIZE_5F(v) ((v) << 8) +#define DMFC_DP_CHAN_BURST_SIZE_5B(v) ((v) << 6) +#define DMFC_DP_CHAN_FIFO_SIZE_5B(v) ((v) << 3) +#define DMFC_DP_CHAN_ST_ADDR_SIZE_5B(v) ((v) << 0) +#define DMFC_DP_CHAN_DEF 0x260010 +#define DMFC_DP_CHAN_DEF_WM_CLR_6F(v) ((v) << 29) +#define DMFC_DP_CHAN_DEF_WM_CLR_6B(v) ((v) << 21) +#define DMFC_DP_CHAN_DEF_WM_CLR_5F(v) ((v) << 13) +#define DMFC_DP_CHAN_DEF_WM_SET_5F(v) ((v) << 10) +#define DMFC_DP_CHAN_DEF_WM_EN_5F (1 << 9) +#define DMFC_DP_CHAN_DEF_WM_CLR_5B(v) ((v) << 5) +#define DMFC_DP_CHAN_DEF_WM_SET_5B(v) ((v) << 2) +#define DMFC_DP_CHAN_DEF_WM_EN_5B (1 << 1) +#define DMFC_GENERAL_1 0x260014 +#define DMFC_GENERAL_1_WAIT4EOT_5B (1 << 20) +#define DMFC_IC_CTRL 0x26001C +#define DMFC_IC_CTRL_DISABLED 0x2 + +#define DC_WRITE_CH_CONF_1 0x0025801C +#define WRITE_CH_CONF_PROG_CHAN_TYP_MASK (7 << 5) +#define WRITE_CH_CONF_PROG_CHAN_NORMAL (4 << 5) +#define DC_WRITE_CH_ADDR_1 0x00258020 +#define DC_WRITE_CH_CONF_5 0x0025805C +#define WRITE_CH_CONF_PROG_DISP_ID(v) ((v) << 3) +#define WRITE_CH_CONF_PROG_DI_ID(v) ((v) << 2) +#define WRITE_CH_CONF_PROG_W_SIZE(v) (v) +#define DC_WRITE_CH_ADDR_5 0x00258060 +#define DC_RL0_CH_5 0x00258064 +#define DC_GEN 0x002580D4 +#define DC_GEN_SYNC_PRIORITY (1 << 7) +#define DC_GEN_ASYNC (0 << 1) +#define DC_GEN_SYNC (2 << 1) +#define DC_DISP_CONF2(di) (0x002580E8 + (di) * 4) +#define DC_MAP_CONF_0 0x00258108 +#define DC_MAP_CONF_15 0x00258144 +#define DC_MAP_CONF_VAL(map) (DC_MAP_CONF_15 + ((map) / 2) * sizeof(uint32_t)) +#define MAP_CONF_VAL_MASK 0xffff +#define DC_MAP_CONF_PTR(ptr) (DC_MAP_CONF_0 + ((ptr) / 2) * sizeof(uint32_t)) +#define MAP_CONF_PTR_MASK 0x1f + +#define DI_COUNTER_INT_HSYNC 1 +#define DI_COUNTER_HSYNC 2 +#define DI_COUNTER_VSYNC 3 +#define DI_COUNTER_AD_0 4 +#define DI_COUNTER_AD_1 5 + +#define DI_SYNC_NONE 0 +#define DI_SYNC_CLK 1 +#define DI_SYNC_COUNTER(c) ((c) + 1) + +struct ipu_cpmem_word { + uint32_t data[5]; + uint32_t padding[3]; +}; + +struct ipu_cpmem_ch_param { + struct ipu_cpmem_word word[2]; +}; + +#define CH_PARAM_RESET(param) memset(param, 0, sizeof(*param)) +#define IPU_READ_CH_PARAM(_sc, ch, param) bus_read_region_4( \ + (_sc)->sc_mem_res, CPMEM_BASE + ch * (sizeof(*param)),\ + (uint32_t*)param, sizeof(*param) / 4) +#define IPU_WRITE_CH_PARAM(_sc, ch, param) bus_write_region_4( \ + (_sc)->sc_mem_res, CPMEM_BASE + ch * (sizeof(*param)),\ + (uint32_t*)param, sizeof(*param) / 4) + +#define CH_PARAM_SET_FW(param, v) ipu_ch_param_set_value((param), \ + 0, 125, 13, (v)) +#define CH_PARAM_SET_FH(param, v) ipu_ch_param_set_value((param), \ + 0, 138, 12, (v)) +#define CH_PARAM_SET_SLY(param, v) ipu_ch_param_set_value((param), \ + 1, 102, 14, (v)) +#define CH_PARAM_SET_EBA0(param, v) ipu_ch_param_set_value((param), \ + 1, 0, 29, (v)) +#define CH_PARAM_SET_EBA1(param, v) ipu_ch_param_set_value((param), \ + 1, 29, 29, (v)) +#define CH_PARAM_SET_BPP(param, v) ipu_ch_param_set_value((param), \ + 0, 107, 3, (v)) +#define CH_PARAM_SET_PFS(param, v) ipu_ch_param_set_value((param), \ + 1, 85, 4, (v)) +#define CH_PARAM_SET_NPB(param, v) ipu_ch_param_set_value((param), \ + 1, 78, 7, (v)) +#define CH_PARAM_SET_UBO(param, v) ipu_ch_param_set_value((param), \ + 0, 46, 22, (v)) +#define CH_PARAM_SET_VBO(param, v) ipu_ch_param_set_value((param), \ + 0, 68, 22, (v)) + +#define CH_PARAM_SET_RED_WIDTH(param, v) ipu_ch_param_set_value((param), \ + 1, 116, 3, (v)) +#define CH_PARAM_SET_RED_OFFSET(param, v) ipu_ch_param_set_value((param), \ + 1, 128, 5, (v)) + +#define CH_PARAM_SET_GREEN_WIDTH(param, v) ipu_ch_param_set_value((param), \ + 1, 119, 3, (v)) +#define CH_PARAM_SET_GREEN_OFFSET(param, v) ipu_ch_param_set_value((param), \ + 1, 133, 5, (v)) + +#define CH_PARAM_SET_BLUE_WIDTH(param, v) ipu_ch_param_set_value((param), \ + 1, 122, 3, (v)) +#define CH_PARAM_SET_BLUE_OFFSET(param, v) ipu_ch_param_set_value((param), \ + 1, 138, 5, (v)) + +#define CH_PARAM_SET_ALPHA_WIDTH(param, v) ipu_ch_param_set_value((param), \ + 1, 125, 3, (v)) +#define CH_PARAM_SET_ALPHA_OFFSET(param, v) ipu_ch_param_set_value((param), \ + 1, 143, 5, (v)) + +#define CH_PARAM_GET_FW(param) ipu_ch_param_get_value((param), \ + 0, 125, 13) +#define CH_PARAM_GET_FH(param) ipu_ch_param_get_value((param), \ + 0, 138, 12) +#define CH_PARAM_GET_SLY(param) ipu_ch_param_get_value((param), \ + 1, 102, 14) +#define CH_PARAM_GET_EBA0(param) ipu_ch_param_get_value((param), \ + 1, 0, 29) +#define CH_PARAM_GET_EBA1(param) ipu_ch_param_get_value((param), \ + 1, 29, 29) +#define CH_PARAM_GET_BPP(param) ipu_ch_param_get_value((param), \ + 0, 107, 3) +#define CH_PARAM_GET_PFS(param) ipu_ch_param_get_value((param), \ + 1, 85, 4) +#define CH_PARAM_GET_NPB(param) ipu_ch_param_get_value((param), \ + 1, 78, 7) +#define CH_PARAM_GET_UBO(param) ipu_ch_param_get_value((param), \ + 0, 46, 22) +#define CH_PARAM_GET_VBO(param) ipu_ch_param_get_value((param), \ + 0, 68, 22) + +#define CH_PARAM_GET_RED_WIDTH(param) ipu_ch_param_get_value((param), \ + 1, 116, 3) +#define CH_PARAM_GET_RED_OFFSET(param) ipu_ch_param_get_value((param), \ + 1, 128, 5) + +#define CH_PARAM_GET_GREEN_WIDTH(param) ipu_ch_param_get_value((param), \ + 1, 119, 3) +#define CH_PARAM_GET_GREEN_OFFSET(param) ipu_ch_param_get_value((param), \ + 1, 133, 5) + +#define CH_PARAM_GET_BLUE_WIDTH(param) ipu_ch_param_get_value((param), \ + 1, 122, 3) +#define CH_PARAM_GET_BLUE_OFFSET(param) ipu_ch_param_get_value((param), \ + 1, 138, 5) + +#define CH_PARAM_GET_ALPHA_WIDTH(param) ipu_ch_param_get_value((param), \ + 1, 125, 3) +#define CH_PARAM_GET_ALPHA_OFFSET(param) ipu_ch_param_get_value((param), \ + 1, 143, 5) + +#define IPU_PIX_FORMAT_BPP_32 0 +#define IPU_PIX_FORMAT_BPP_24 1 +#define IPU_PIX_FORMAT_BPP_18 2 +#define IPU_PIX_FORMAT_BPP_16 3 +#define IPU_PIX_FORMAT_BPP_12 4 +#define IPU_PIX_FORMAT_BPP_8 5 +#define IPU_PIX_FORMAT_BPP_ + +#define IPU_PIX_FORMAT_RGB 7 + +enum dc_event_t { + DC_EVENT_NF = 0, + DC_EVENT_NL, + DC_EVENT_EOF, + DC_EVENT_NFIELD, + DC_EVENT_EOL, + DC_EVENT_EOFIELD, + DC_EVENT_NEW_ADDR, + DC_EVENT_NEW_CHAN, + DC_EVENT_NEW_DATA +}; + +struct ipu_softc { + device_t sc_dev; + struct resource *sc_mem_res; + int sc_mem_rid; + struct resource *sc_irq_res; + int sc_irq_rid; + void *sc_intr_hl; + struct mtx sc_mtx; + struct fb_info sc_fb_info; + struct videomode *sc_mode; + + /* Framebuffer */ + bus_dma_tag_t sc_dma_tag; + bus_dmamap_t sc_dma_map; + size_t sc_fb_size; + bus_addr_t sc_fb_phys; + uint8_t *sc_fb_base; + + /* HDMI */ + eventhandler_tag sc_hdmi_evh; +}; + +static void +ipu_dmamap_cb(void *arg, bus_dma_segment_t *segs, int nseg, int err) +{ + bus_addr_t *addr; + + if (err) + return; + + addr = (bus_addr_t*)arg; + *addr = segs[0].ds_addr; +} + +static void +ipu_ch_param_set_value(struct ipu_cpmem_ch_param *param, + int word, unsigned int offset, int len, uint32_t value) +{ + uint32_t datapos, bitpos, mask; + uint32_t data, data2; + + KASSERT((len <= 32), ("%s: field len is more than 32", __func__)); + + datapos = offset / 32; + bitpos = offset % 32; + + mask = (1 << len) - 1; + data = param->word[word].data[datapos]; + data &= ~(mask << bitpos); + data |= (value << bitpos); + param->word[word].data[datapos] = data; + + if ((bitpos + len) > 32) { + len = bitpos + len - 32; + mask = (1UL << len) - 1; + data2 = param->word[word].data[datapos + 1]; + data2 &= mask; + data2 |= (value >> (32 - bitpos)); + param->word[word].data[datapos + 1] = data2; + } +} + +#ifdef DEBUG +static uint32_t +ipu_ch_param_get_value(struct ipu_cpmem_ch_param *param, + int word, unsigned int offset, int len) +{ + uint32_t datapos, bitpos, mask; + uint32_t data, data2; + + KASSERT((len <= 32), ("%s: field len is more than 32", __func__)); + + datapos = offset / 32; + bitpos = offset % 32; + mask = (1UL << len) - 1; + data = param->word[word].data[datapos]; + data = data >> bitpos; + data &= mask; + if ((bitpos + len) > 32) { + len = bitpos + len - 32; + mask = (1UL << len) - 1; + data2 = param->word[word].data[datapos + 1]; + data2 &= mask; + data |= (data2 << (32 - bitpos)); + } + + return (data); +} + +static void +ipu_print_channel(struct ipu_cpmem_ch_param *param) +{ + int offset0[] = {0, 10, 19, 32, 44, 45, 46, 68, 90, 94, 95, 113, 114, 117, 119, 120, 121, 122, 123, 124, 125, 138, 150, 151, -1}; + int offset1[] = {0, 29, 58, 78, 85, 89, 90, 93, 95, 102, 116, 119, 122, 125, 128, 133, 138, 143, 148, 149, 150, -1}; + printf("WORD0: %08x %08x %08x %08x %08x\n", + param->word[0].data[0], param->word[0].data[1], + param->word[0].data[2], param->word[0].data[3], + param->word[0].data[4]); + printf("WORD1: %08x %08x %08x %08x %08x\n", + param->word[1].data[0], param->word[1].data[1], + param->word[1].data[2], param->word[1].data[3], + param->word[1].data[4]); + + for (int i = 0; offset0[i + 1] != -1; i++) { + int len = offset0[i + 1] - offset0[i]; + printf("W0[%d:%d] = %d\n", offset0[i], + offset0[i] + len - 1, + ipu_ch_param_get_value(param, 0, offset0[i], len) + ); + } + + for (int i = 0; offset1[i + 1] != -1; i++) { + int len = offset1[i + 1] - offset1[i]; + printf("W1[%d:%d] = %d\n", offset1[i], + offset1[i] + len - 1, + ipu_ch_param_get_value(param, 1, offset1[i], len) + ); + } + + printf("FW: %d\n", CH_PARAM_GET_FW(param)); + printf("FH: %d\n", CH_PARAM_GET_FH(param)); + printf("SLY: %d\n", CH_PARAM_GET_SLY(param)); + printf("EBA0: 0x%08x\n", CH_PARAM_GET_EBA0(param)); + printf("EBA1: 0x%08x\n", CH_PARAM_GET_EBA1(param)); + printf("BPP: %d\n", CH_PARAM_GET_BPP(param)); + printf("PFS: %d\n", CH_PARAM_GET_PFS(param)); + printf("NPB: %d\n", CH_PARAM_GET_NPB(param)); + printf("UBO: %d\n", CH_PARAM_GET_UBO(param)); + printf("VBO: %d\n", CH_PARAM_GET_VBO(param)); + printf("RED: %d bits @%d\n", CH_PARAM_GET_RED_WIDTH(param) + 1, + CH_PARAM_GET_RED_OFFSET(param)); + printf("GREEN: %d bits @%d\n", CH_PARAM_GET_GREEN_WIDTH(param) + 1, + CH_PARAM_GET_GREEN_OFFSET(param)); + printf("BLUE: %d bits @%d\n", CH_PARAM_GET_BLUE_WIDTH(param) + 1, + CH_PARAM_GET_BLUE_OFFSET(param)); + printf("ALPHA: %d bits @%d\n", CH_PARAM_GET_ALPHA_WIDTH(param) + 1, + CH_PARAM_GET_ALPHA_OFFSET(param)); +} +#endif + +static void +ipu_di_enable(struct ipu_softc *sc, int di) +{ + uint32_t flag, reg; + + flag = di ? DISP_GEN_DI1_CNTR_RELEASE : DISP_GEN_DI0_CNTR_RELEASE; + reg = IPU_READ4(sc, IPU_DISP_GEN); + reg |= flag; + IPU_WRITE4(sc, IPU_DISP_GEN, reg); +} + +static void +ipu_config_wave_gen_0(struct ipu_softc *sc, int di, + int wave_gen, int run_value, int run_res, + int offset_value, int offset_res) +{ + uint32_t addr, reg; + + addr = (di ? IPU_DI1_SW_GEN0_1 : IPU_DI0_SW_GEN0_1) + + (wave_gen - 1) * sizeof(uint32_t); + reg = DI_RUN_VALUE_M1(run_value) | + DI_RUN_RESOLUTION(run_res) | + DI_OFFSET_VALUE(offset_value) | offset_res; + IPU_WRITE4(sc, addr, reg); +} + +static void +ipu_config_wave_gen_1(struct ipu_softc *sc, int di, int wave_gen, + int repeat_count, int cnt_clr_src, + int cnt_polarity_gen_en, + int cnt_polarity_clr_src, + int cnt_polarity_trigger_src, + int cnt_up, int cnt_down) +{ + uint32_t addr, reg; + + addr = (di ? IPU_DI1_SW_GEN1_1 : IPU_DI0_SW_GEN1_1) + + (wave_gen - 1) * sizeof(uint32_t); + reg = DI_CNT_POLARITY_GEN_EN(cnt_polarity_gen_en) | + DI_CNT_CLR_SEL(cnt_clr_src) | + DI_CNT_POLARITY_TRIGGER_SEL(cnt_polarity_trigger_src) | + DI_CNT_POLARITY_CLR_SEL(cnt_polarity_clr_src); + reg |= DI_CNT_DOWN(cnt_down) | cnt_up; + if (repeat_count == 0) + reg |= DI_CNT_AUTO_RELOAD; + IPU_WRITE4(sc, addr, reg); + + addr = (di ? IPU_DI1_STP_REP : IPU_DI0_STP_REP) + + (wave_gen - 1) / 2 * sizeof(uint32_t); + reg = IPU_READ4(sc, addr); + if (wave_gen % 2) { + reg &= ~(0xffff); + reg |= repeat_count; + } + else { + reg &= ~(0xffff << 16); + reg |= (repeat_count << 16); + } + IPU_WRITE4(sc, addr, reg); +} + +static void +ipu_reset_wave_gen(struct ipu_softc *sc, int di, + int wave_gen) +{ + uint32_t addr, reg; + + addr = (di ? IPU_DI1_SW_GEN0_1 : IPU_DI0_SW_GEN0_1) + + (wave_gen - 1) * sizeof(uint32_t); + IPU_WRITE4(sc, addr, 0); + + addr = (di ? IPU_DI1_SW_GEN1_1 : IPU_DI0_SW_GEN1_1) + + (wave_gen - 1) * sizeof(uint32_t); + IPU_WRITE4(sc, addr, 0); + + addr = (di ? IPU_DI1_STP_REP : IPU_DI0_STP_REP) + + (wave_gen - 1) / 2 * sizeof(uint32_t); + reg = IPU_READ4(sc, addr); + if (wave_gen % 2) + reg &= ~(0xffff); + else + reg &= ~(0xffff << 16); + IPU_WRITE4(sc, addr, reg); +} + +static void +ipu_init_microcode_template(struct ipu_softc *sc, int di, int map) +{ + uint32_t addr; + uint32_t w1, w2; + int i, word; + int glue; + + word = di ? 2 : 5; + + for (i = 0; i < 3; i++) { + if (i == 0) + glue = GLUELOGIC_KEEP_ASSERTED; + else if (i == 1) + glue = GLUELOGIC_KEEP_NEGATED; + else if (i == 2) + glue = 0; + + w1 = TEMPLATE_SYNC(5) | + TEMPLATE_GLUELOGIC(glue) | + TEMPLATE_WAVEFORM(1) | /* wave unit 0 */ + TEMPLATE_MAPPING(map + 1); + /* operand is zero */ + + /* Write data to DI and Hold data in register */ + w2 = TEMPLATE_OPCODE(OPCODE_WROD) | + TEMPLATE_STOP; + + addr = DC_TEMPL_BASE + (word + i) * 2 * sizeof(uint32_t); + IPU_WRITE4(sc, addr, w1); + IPU_WRITE4(sc, addr + sizeof(uint32_t), w2); + } +} + +static void +ipu_config_timing(struct ipu_softc *sc, int di) +{ + int div; + uint32_t di_scr_conf; + uint32_t gen_offset, gen; + uint32_t as_gen_offset, as_gen; + uint32_t dw_gen_offset, dw_gen; + uint32_t dw_set_offset, dw_set; + uint32_t bs_clkgen_offset; + int map; + + /* TODO: check mode restrictions / fixup */ + /* TODO: enable timers, get divisors */ + div = 1; + map = 0; + + bs_clkgen_offset = di ? IPU_DI1_BS_CLKGEN0 : IPU_DI0_BS_CLKGEN0; + IPU_WRITE4(sc, bs_clkgen_offset, DI_BS_CLKGEN0(div, 0)); + /* half of the divider */ + IPU_WRITE4(sc, bs_clkgen_offset + 4, DI_BS_CLKGEN1_DOWN(div / 2, div % 2)); + + /* + * TODO: Configure LLDB clock by changing following fields + * in CCM fields: + * CS2CDR_LDB_DI0_CLK_SEL + * CSCMR2_LDB_DI0_IPU_DIV + * CBCDR_MMDC_CH1_AXI_PODF + */ + + /* Setup wave generator */ + dw_gen_offset = di ? IPU_DI1_DW_GEN_0 : IPU_DI0_DW_GEN_0; + dw_gen = DW_GEN_DI_ACCESS_SIZE(div - 1) | DW_GEN_DI_COMPONENT_SIZE(div - 1); + dw_gen &= ~DW_GEN_DI_PIN_15_SET(DW_GEN_DI_SET_MASK); + dw_gen |= DW_GEN_DI_PIN_15_SET(3); /* set 3*/ + IPU_WRITE4(sc, dw_gen_offset, dw_gen); + + dw_set_offset = di ? IPU_DI1_DW_SET3_0 : IPU_DI0_DW_SET3_0; + dw_set = DW_SET_DATA_CNT_DOWN(div * 2) | DW_SET_DATA_CNT_UP(0); + IPU_WRITE4(sc, dw_set_offset, dw_set); + + /* DI_COUNTER_INT_HSYNC */ + ipu_config_wave_gen_0(sc, di, DI_COUNTER_INT_HSYNC, + sc->sc_mode->htotal - 1, DI_SYNC_CLK, 0, DI_SYNC_NONE); + ipu_config_wave_gen_1(sc, di, DI_COUNTER_INT_HSYNC, + 0, DI_SYNC_NONE, 0, DI_SYNC_NONE, DI_SYNC_NONE, 0, 0); + + /* DI_COUNTER_HSYNC */ + ipu_config_wave_gen_0(sc, di, DI_COUNTER_HSYNC, + sc->sc_mode->htotal - 1, DI_SYNC_CLK, 0, DI_SYNC_CLK); + ipu_config_wave_gen_1(sc, di, DI_COUNTER_HSYNC, + 0, DI_SYNC_NONE, 1, DI_SYNC_NONE, DI_SYNC_CLK, + 0, MODE_HSW(sc->sc_mode) * 2); + + /* DI_COUNTER_VSYNC */ + ipu_config_wave_gen_0(sc, di, DI_COUNTER_VSYNC, + sc->sc_mode->vtotal - 1, DI_SYNC_COUNTER(DI_COUNTER_INT_HSYNC), + 0, DI_SYNC_NONE); + ipu_config_wave_gen_1(sc, di, DI_COUNTER_VSYNC, + 0, DI_SYNC_NONE, 1, DI_SYNC_NONE, + DI_SYNC_COUNTER(DI_COUNTER_INT_HSYNC), + 0, MODE_VSW(sc->sc_mode) * 2); + + di_scr_conf = di ? IPU_DI1_SCR_CONF : IPU_DI0_SCR_CONF; + IPU_WRITE4(sc, di_scr_conf, sc->sc_mode->vtotal - 1); + + /* TODO: update DI_SCR_CONF */ + + /* Active Data 0 */ + ipu_config_wave_gen_0(sc, di, DI_COUNTER_AD_0, + 0, DI_SYNC_COUNTER(DI_COUNTER_HSYNC), + MODE_VSW(sc->sc_mode) + MODE_VFP(sc->sc_mode), DI_SYNC_COUNTER(DI_COUNTER_HSYNC)); + ipu_config_wave_gen_1(sc, di, DI_COUNTER_AD_0, + sc->sc_mode->vdisplay, DI_SYNC_COUNTER(DI_COUNTER_VSYNC), + 0, DI_SYNC_NONE, DI_SYNC_NONE, 0, 0); + + ipu_config_wave_gen_0(sc, di, DI_COUNTER_AD_1, + 0, DI_SYNC_CLK, MODE_HSW(sc->sc_mode) + MODE_HFP(sc->sc_mode), DI_SYNC_CLK); + ipu_config_wave_gen_1(sc, di, DI_COUNTER_AD_1, + sc->sc_mode->hdisplay, DI_SYNC_COUNTER(DI_COUNTER_AD_0), + 0, DI_SYNC_NONE, DI_SYNC_NONE, 0, 0); + + ipu_reset_wave_gen(sc, di, 6); + ipu_reset_wave_gen(sc, di, 7); + ipu_reset_wave_gen(sc, di, 8); + ipu_reset_wave_gen(sc, di, 9); + + ipu_init_microcode_template(sc, di, map); + + gen_offset = di ? IPU_DI1_GENERAL : IPU_DI0_GENERAL; + gen = IPU_READ4(sc, gen_offset); + + if (sc->sc_mode->flags & VID_NHSYNC) + gen &= ~DI_GENERAL_POLARITY_2; + else /* active high */ + gen |= DI_GENERAL_POLARITY_2; + + if (sc->sc_mode->flags & VID_NVSYNC) + gen &= ~DI_GENERAL_POLARITY_3; + else /* active high */ + gen |= DI_GENERAL_POLARITY_3; + + if (MODE_PIXEL_CLOCK_INVERT) + gen &= ~DI_GENERAL_POL_CLK; + else + gen |= DI_GENERAL_POL_CLK; + + /* Use LDB clock to drive pixel clock */ + gen |= DI_CLOCK_EXTERNAL; + + IPU_WRITE4(sc, gen_offset, gen); + + as_gen_offset = di ? IPU_DI1_SYNC_AS_GEN : IPU_DI0_SYNC_AS_GEN; + as_gen = SYNC_AS_GEN_VSYNC_SEL(DI_COUNTER_VSYNC - 1) | + SYNC_AS_GEN_SYNC_START(2); + IPU_WRITE4(sc, as_gen_offset, as_gen); + + IPU_WRITE4(sc, (di ? IPU_DI1_POL : IPU_DI0_POL), DI_POL_DRDY_POLARITY_15); + + IPU_WRITE4(sc, DC_DISP_CONF2(di), sc->sc_mode->hdisplay); +} + +static void +ipu_dc_enable(struct ipu_softc *sc) +{ + uint32_t conf; + + /* channel 1 uses DI1 */ + IPU_WRITE4(sc, DC_WRITE_CH_CONF_1, WRITE_CH_CONF_PROG_DI_ID(1)); + + conf = IPU_READ4(sc, DC_WRITE_CH_CONF_5); + conf &= ~WRITE_CH_CONF_PROG_CHAN_TYP_MASK; + conf |= WRITE_CH_CONF_PROG_CHAN_NORMAL; + IPU_WRITE4(sc, DC_WRITE_CH_CONF_5, conf); + + /* TODO: enable clock */ +} + +static void +ipu_dc_link_event(struct ipu_softc *sc, int event, int addr, int priority) +{ + uint32_t reg; + int offset; + int shift; + + if (event % 2) + shift = 16; + else + shift = 0; + + offset = DC_RL0_CH_5 + (event / 2) * sizeof(uint32_t); + + reg = IPU_READ4(sc, offset); + reg &= ~(0xFFFF << shift); + reg |= ((addr << 8) | priority) << shift; + IPU_WRITE4(sc, offset, reg); +} + +static void +ipu_dc_setup_map(struct ipu_softc *sc, int map, + int byte, int offset, int mask) +{ + uint32_t reg, shift, ptr; + + ptr = map * 3 + byte; + + reg = IPU_READ4(sc, DC_MAP_CONF_VAL(ptr)); + if (ptr & 1) + shift = 16; + else + shift = 0; + reg &= ~(0xffff << shift); + reg |= ((offset << 8) | mask) << shift; + IPU_WRITE4(sc, DC_MAP_CONF_VAL(ptr), reg); + + reg = IPU_READ4(sc, DC_MAP_CONF_PTR(map)); + if (map & 1) + shift = 16 + 5 * byte; + else + shift = 5 * byte; + reg &= ~(MAP_CONF_PTR_MASK << shift); + reg |= (ptr) << shift; + IPU_WRITE4(sc, DC_MAP_CONF_PTR(map), reg); +} + +static void +ipu_dc_reset_map(struct ipu_softc *sc, int map) +{ + uint32_t reg, shift; + + reg = IPU_READ4(sc, DC_MAP_CONF_VAL(map)); + if (map & 1) + shift = 16; + else + shift = 0; + reg &= ~(MAP_CONF_VAL_MASK << shift); + IPU_WRITE4(sc, DC_MAP_CONF_VAL(map), reg); +} + +static void +ipu_dc_init(struct ipu_softc *sc, int di_port) +{ + int addr; + uint32_t conf; + + if (di_port) + addr = 2; + else + addr = 5; + + ipu_dc_link_event(sc, DC_EVENT_NL, addr, 3); + ipu_dc_link_event(sc, DC_EVENT_EOL, addr + 1, 2); + ipu_dc_link_event(sc, DC_EVENT_NEW_DATA, addr + 2, 1); + ipu_dc_link_event(sc, DC_EVENT_NF, 0, 0); + ipu_dc_link_event(sc, DC_EVENT_NFIELD, 0, 0); + ipu_dc_link_event(sc, DC_EVENT_EOF, 0, 0); + ipu_dc_link_event(sc, DC_EVENT_EOFIELD, 0, 0); + ipu_dc_link_event(sc, DC_EVENT_NEW_CHAN, 0, 0); + ipu_dc_link_event(sc, DC_EVENT_NEW_ADDR, 0, 0); + + conf = WRITE_CH_CONF_PROG_W_SIZE(0x02) | + WRITE_CH_CONF_PROG_DISP_ID(DI_PORT) | + WRITE_CH_CONF_PROG_DI_ID(DI_PORT); + + IPU_WRITE4(sc, DC_WRITE_CH_CONF_5, conf); + IPU_WRITE4(sc, DC_WRITE_CH_ADDR_5, 0x00000000); + IPU_WRITE4(sc, DC_GEN, DC_GEN_SYNC_PRIORITY | DC_GEN_SYNC); /* High priority, sync */ +} + +static void +ipu_init_buffer(struct ipu_softc *sc) +{ + struct ipu_cpmem_ch_param param; + uint32_t stride; + uint32_t reg, db_mode_sel, cur_buf; + + stride = sc->sc_mode->hdisplay * MODE_BPP / 8; + + /* init channel paramters */ + CH_PARAM_RESET(¶m); + /* XXX: interlaced modes are not supported yet */ + CH_PARAM_SET_FW(¶m, sc->sc_mode->hdisplay - 1); + CH_PARAM_SET_FH(¶m, sc->sc_mode->vdisplay - 1); + CH_PARAM_SET_SLY(¶m, stride - 1); + + CH_PARAM_SET_EBA0(¶m, (sc->sc_fb_phys >> 3)); + CH_PARAM_SET_EBA1(¶m, (sc->sc_fb_phys >> 3)); + + CH_PARAM_SET_BPP(¶m, IPU_PIX_FORMAT_BPP_16); + CH_PARAM_SET_PFS(¶m, IPU_PIX_FORMAT_RGB); + /* 16 pixels per burst access */ + CH_PARAM_SET_NPB(¶m, 16 - 1); + + CH_PARAM_SET_RED_OFFSET(¶m, 0); + CH_PARAM_SET_RED_WIDTH(¶m, 5 - 1); + CH_PARAM_SET_GREEN_OFFSET(¶m, 5); + CH_PARAM_SET_GREEN_WIDTH(¶m, 6 - 1); + CH_PARAM_SET_BLUE_OFFSET(¶m, 11); + CH_PARAM_SET_BLUE_WIDTH(¶m, 5 - 1); + CH_PARAM_SET_ALPHA_OFFSET(¶m, 16); + CH_PARAM_SET_ALPHA_WIDTH(¶m, 8 - 1); + + CH_PARAM_SET_UBO(¶m, 0); + CH_PARAM_SET_VBO(¶m, 0); + + IPU_WRITE_CH_PARAM(sc, DMA_CHANNEL, ¶m); +#ifdef DEBUG + ipu_print_channel(¶m); +#endif + + /* init DMFC */ + IPU_WRITE4(sc, DMFC_IC_CTRL, DMFC_IC_CTRL_DISABLED); + /* High resolution DP */ + IPU_WRITE4(sc, DMFC_WR_CHAN, DMFC_WR_CHAN_BURST_SIZE_8 | + DMFC_WR_CHAN_FIFO_SIZE_128); + IPU_WRITE4(sc, DMFC_WR_CHAN_DEF, DMFC_WR_CHAN_DEF_WM_CLR_2C(1) | + DMFC_WR_CHAN_DEF_WM_CLR_1C(1) | + DMFC_WR_CHAN_DEF_WM_CLR_2(1) | + DMFC_WR_CHAN_DEF_WM_CLR_1(7) | + DMFC_WR_CHAN_DEF_WM_SET_1(5) | + DMFC_WR_CHAN_DEF_WM_EN_1); + + IPU_WRITE4(sc, DMFC_DP_CHAN, + DMFC_DP_CHAN_BURST_SIZE_5F(DMFC_DP_CHAN_BURST_SIZE_8) | + DMFC_DP_CHAN_FIFO_SIZE_5F(DMFC_DP_CHAN_FIFO_SIZE_128) | + DMFC_DP_CHAN_ST_ADDR_SIZE_5F(6) /* segment 6 */ | + DMFC_DP_CHAN_BURST_SIZE_5B(DMFC_DP_CHAN_BURST_SIZE_8) | + DMFC_DP_CHAN_FIFO_SIZE_5B(DMFC_DP_CHAN_FIFO_SIZE_256) | + DMFC_DP_CHAN_ST_ADDR_SIZE_5B(2) /* segment 2 */); + + IPU_WRITE4(sc, DMFC_DP_CHAN_DEF, DMFC_DP_CHAN_DEF_WM_CLR_6F(1) | + DMFC_DP_CHAN_DEF_WM_CLR_6B(1) | + DMFC_DP_CHAN_DEF_WM_CLR_5F(7) | + DMFC_DP_CHAN_DEF_WM_SET_5F(5) | + DMFC_DP_CHAN_DEF_WM_EN_5F | + DMFC_DP_CHAN_DEF_WM_CLR_5B(7) | + DMFC_DP_CHAN_DEF_WM_SET_5B(5) | + DMFC_DP_CHAN_DEF_WM_EN_5B); + + reg = IPU_READ4(sc, DMFC_GENERAL_1); + reg &= ~(DMFC_GENERAL_1_WAIT4EOT_5B); + IPU_WRITE4(sc, DMFC_GENERAL_1, reg); + + /* XXX: set priority? */ + + /* Set single buffer mode */ + if (DMA_CHANNEL < 32) { + db_mode_sel = IPU_CH_DB_MODE_SEL_0; + cur_buf = IPU_CUR_BUF_0; + } else { + db_mode_sel = IPU_CH_DB_MODE_SEL_1; + cur_buf = IPU_CUR_BUF_1; + } + + reg = IPU_READ4(sc, db_mode_sel); + reg |= (1UL << (DMA_CHANNEL & 0x1f)); + IPU_WRITE4(sc, db_mode_sel, reg); + + IPU_WRITE4(sc, cur_buf, (1UL << (DMA_CHANNEL & 0x1f))); +} + +static int +ipu_init(struct ipu_softc *sc) +{ + uint32_t reg, off; + int i, err; + size_t dma_size; + + IPU_WRITE4(sc, IPU_CONF, DI_PORT ? IPU_CONF_DI1_EN : IPU_CONF_DI0_EN); + + IPU_WRITE4(sc, IPU_MEM_RST, IPU_MEM_RST_ALL); + i = 1000; + while (i-- > 0) { + if (!(IPU_READ4(sc, IPU_MEM_RST) & IPU_MEM_RST_START)) + break; + DELAY(1); + } + + if (i <= 0) { + err = ETIMEDOUT; + device_printf(sc->sc_dev, "timeout while resetting memory\n"); + goto fail; + } + + ipu_dc_reset_map(sc, 0); + ipu_dc_setup_map(sc, 0, 0, 7, 0xff); + ipu_dc_setup_map(sc, 0, 1, 15, 0xff); + ipu_dc_setup_map(sc, 0, 2, 23, 0xff); + + dma_size = round_page(sc->sc_mode->hdisplay * sc->sc_mode->vdisplay * (MODE_BPP / 8)); + + /* + * Now allocate framebuffer memory + */ + err = bus_dma_tag_create( + bus_get_dma_tag(sc->sc_dev), + 4, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + dma_size, 1, /* maxsize, nsegments */ + dma_size, 0, /* maxsegsize, flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sc_dma_tag); + if (err) + goto fail; + + err = bus_dmamem_alloc(sc->sc_dma_tag, (void **)&sc->sc_fb_base, + BUS_DMA_COHERENT | BUS_DMA_ZERO, &sc->sc_dma_map); + + if (err) { + device_printf(sc->sc_dev, "cannot allocate framebuffer\n"); + goto fail; + } + + err = bus_dmamap_load(sc->sc_dma_tag, sc->sc_dma_map, sc->sc_fb_base, + dma_size, ipu_dmamap_cb, &sc->sc_fb_phys, BUS_DMA_NOWAIT); + + if (err) { + device_printf(sc->sc_dev, "cannot load DMA map\n"); + goto fail; + } + + /* Calculate actual FB Size */ + sc->sc_fb_size = sc->sc_mode->hdisplay * sc->sc_mode->vdisplay * MODE_BPP / 8; + + ipu_dc_init(sc, DI_PORT); + reg = IPU_READ4(sc, IPU_CONF); + reg |= IPU_CONF_DMFC_EN | IPU_CONF_DC_EN | IPU_CONF_DP_EN; + IPU_WRITE4(sc, IPU_CONF, reg); + + ipu_config_timing(sc, DI_PORT); + ipu_init_buffer(sc); + ipu_di_enable(sc, DI_PORT); + + /* Enable DMA channel */ + off = (DMA_CHANNEL > 31) ? IPU_IDMAC_CH_EN_2 : IPU_IDMAC_CH_EN_1; + reg = IPU_READ4(sc, off); + reg |= (1 << (DMA_CHANNEL & 0x1f)); + IPU_WRITE4(sc, off, reg); + + ipu_dc_enable(sc); + + sc->sc_fb_info.fb_name = device_get_nameunit(sc->sc_dev); + sc->sc_fb_info.fb_vbase = (intptr_t)sc->sc_fb_base; + sc->sc_fb_info.fb_pbase = sc->sc_fb_phys; + sc->sc_fb_info.fb_size = sc->sc_fb_size; + sc->sc_fb_info.fb_bpp = sc->sc_fb_info.fb_depth = MODE_BPP; + sc->sc_fb_info.fb_stride = sc->sc_mode->hdisplay * MODE_BPP / 8; + sc->sc_fb_info.fb_width = sc->sc_mode->hdisplay; + sc->sc_fb_info.fb_height = sc->sc_mode->vdisplay; + + device_t fbd = device_add_child(sc->sc_dev, "fbd", + device_get_unit(sc->sc_dev)); + if (fbd == NULL) { + device_printf(sc->sc_dev, "Failed to add fbd child\n"); + goto fail; + } + if (device_probe_and_attach(fbd) != 0) { + device_printf(sc->sc_dev, "Failed to attach fbd device\n"); + goto fail; + } + + return (0); +fail: + + return (err); +} + +static void +ipu_hdmi_event(void *arg, device_t hdmi_dev) +{ + struct ipu_softc *sc; + uint8_t *edid; + uint32_t edid_len; +#ifdef EDID_DEBUG + struct edid_info ei; +#endif + const struct videomode *videomode; + + sc = arg; + + edid = NULL; + edid_len = 0; + if (HDMI_GET_EDID(hdmi_dev, &edid, &edid_len) != 0) { + device_printf(sc->sc_dev, "failed to get EDID info from HDMI framer\n"); + } + + videomode = NULL; + +#ifdef EDID_DEBUG + if ( edid && (edid_parse(edid, &ei) == 0)) { + edid_print(&ei); + } else + device_printf(sc->sc_dev, "failed to parse EDID\n"); +#endif + + sc->sc_mode = &mode1024x768; + ipu_init(sc); + + HDMI_SET_VIDEOMODE(hdmi_dev, sc->sc_mode); +} + +static int +ipu_probe(device_t dev) +{ + + if (have_ipu) + return (ENXIO); + + if (!ofw_bus_status_okay(dev)) + return (ENXIO); + + if (!ofw_bus_is_compatible(dev, "fsl,imx6q-ipu")) + return (ENXIO); + + device_set_desc(dev, "Freescale IPU"); + + return (BUS_PROBE_DEFAULT); +} + +static int +ipu_attach(device_t dev) +{ + struct ipu_softc *sc; + + if (have_ipu) + return (ENXIO); + + sc = device_get_softc(dev); + sc->sc_dev = dev; + + sc->sc_mem_rid = 0; + sc->sc_mem_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, + &sc->sc_mem_rid, RF_ACTIVE); + if (!sc->sc_mem_res) { + device_printf(dev, "cannot allocate memory window\n"); + return (ENXIO); + } + + sc->sc_irq_rid = 0; + sc->sc_irq_res = bus_alloc_resource_any(dev, SYS_RES_IRQ, + &sc->sc_irq_rid, RF_ACTIVE); + if (!sc->sc_irq_res) { + bus_release_resource(dev, SYS_RES_MEMORY, + sc->sc_mem_rid, sc->sc_mem_res); + device_printf(dev, "cannot allocate interrupt\n"); + return (ENXIO); + } + + /* Enable IPU1 */ + imx_ccm_ipu_enable(1); + + if (src_reset_ipu() != 0) { + device_printf(dev, "failed to reset IPU\n"); + return (ENXIO); + } + + IPU_LOCK_INIT(sc); + + sc->sc_hdmi_evh = EVENTHANDLER_REGISTER(hdmi_event, + ipu_hdmi_event, sc, 0); + + have_ipu = 1; + + return (0); +} + +static int +ipu_detach(device_t dev) +{ + /* Do not let unload driver */ + return (EBUSY); +} + +static struct fb_info * +ipu_fb_getinfo(device_t dev) +{ + struct ipu_softc *sc; + + sc = device_get_softc(dev); + + return (&sc->sc_fb_info); +} + +static device_method_t ipu_methods[] = { + DEVMETHOD(device_probe, ipu_probe), + DEVMETHOD(device_attach, ipu_attach), + DEVMETHOD(device_detach, ipu_detach), + + /* Framebuffer service methods */ + DEVMETHOD(fb_getinfo, ipu_fb_getinfo), + + DEVMETHOD_END +}; + +static driver_t ipu_driver = { + "fb", + ipu_methods, + sizeof(struct ipu_softc), +}; + +static devclass_t ipu_devclass; + +DRIVER_MODULE(ipu, simplebus, ipu_driver, ipu_devclass, 0, 0); +MODULE_VERSION(ipu, 1); +MODULE_DEPEND(ipu, simplebus, 1, 1, 1); diff --git a/sys/arm/freescale/imx/imx_ccmvar.h b/sys/arm/freescale/imx/imx_ccmvar.h index 91a3e45..8e944d3 100644 --- a/sys/arm/freescale/imx/imx_ccmvar.h +++ b/sys/arm/freescale/imx/imx_ccmvar.h @@ -52,6 +52,8 @@ uint32_t imx_ccm_ahb_hz(void); void imx_ccm_usb_enable(device_t _usbdev); void imx_ccm_usbphy_enable(device_t _phydev); void imx_ccm_ssi_configure(device_t _ssidev); +void imx_ccm_hdmi_enable(void); +void imx_ccm_ipu_enable(int ipu); /* Routines to get and set the arm clock root divisor register. */ uint32_t imx_ccm_get_cacrr(void); diff --git a/sys/arm/freescale/imx/imx_iomuxreg.h b/sys/arm/freescale/imx/imx_iomuxreg.h new file mode 100644 index 0000000..897a32f --- /dev/null +++ b/sys/arm/freescale/imx/imx_iomuxreg.h @@ -0,0 +1,42 @@ +/*- + * Copyright (c) 2015 Oleksandr Tymoshenko <gonzo@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef IMX_IOMUXREG_H +#define IMX_IOMUXREG_H + +#define IOMUXC_GPR0 0x00 +#define IOMUXC_GPR1 0x04 +#define IOMUXC_GPR2 0x08 +#define IOMUXC_GPR3 0x0C +#define IOMUXC_GPR3_HDMI_MASK (3 << 2) +#define IOMUXC_GPR3_HDMI_IPU1_DI0 (0 << 2) +#define IOMUXC_GPR3_HDMI_IPU1_DI1 (1 << 2) +#define IOMUXC_GPR3_HDMI_IPU2_DI0 (2 << 2) +#define IOMUXC_GPR3_HDMI_IPU2_DI1 (3 << 2) + +#endif diff --git a/sys/arm/include/ofw_machdep.h b/sys/arm/include/ofw_machdep.h index d6bd576..54033ea 100644 --- a/sys/arm/include/ofw_machdep.h +++ b/sys/arm/include/ofw_machdep.h @@ -32,6 +32,9 @@ #ifndef _MACHINE_OFW_MACHDEP_H_ #define _MACHINE_OFW_MACHDEP_H_ +#include <sys/types.h> +#include <sys/bus.h> +#include <sys/rman.h> #include <vm/vm.h> typedef uint32_t cell_t; diff --git a/sys/arm/lpc/std.lpc b/sys/arm/lpc/std.lpc index 4c0d0d6..170df75 100644 --- a/sys/arm/lpc/std.lpc +++ b/sys/arm/lpc/std.lpc @@ -9,6 +9,5 @@ machine arm makeoptions CONF_CFLAGS="-march=armv5te" options PHYSADDR=0x80000000 makeoptions KERNPHYSADDR=0x80100000 -options KERNPHYSADDR=0x80100000 makeoptions KERNVIRTADDR=0xc0100000 options KERNVIRTADDR=0xc0100000 diff --git a/sys/arm/mv/discovery/std.db78xxx b/sys/arm/mv/discovery/std.db78xxx index 70314fd..f1e9bbc 100644 --- a/sys/arm/mv/discovery/std.db78xxx +++ b/sys/arm/mv/discovery/std.db78xxx @@ -5,7 +5,5 @@ files "../mv/discovery/files.db78xxx" makeoptions KERNPHYSADDR=0x00900000 makeoptions KERNVIRTADDR=0xc0900000 - -options KERNPHYSADDR=0x00900000 options KERNVIRTADDR=0xc0900000 options PHYSADDR=0x00000000 diff --git a/sys/arm/mv/kirkwood/std.kirkwood b/sys/arm/mv/kirkwood/std.kirkwood index abe2ddd..7fa24ba 100644 --- a/sys/arm/mv/kirkwood/std.kirkwood +++ b/sys/arm/mv/kirkwood/std.kirkwood @@ -1,14 +1,4 @@ # $FreeBSD$ -# kernel gets loaded at 0x00900000 by the loader, but runs at virtual address -# 0xc0900000. RAM starts at 0. We put the pagetable at a reasonable place -# in memory, but may need to bounce it higher if there's a problem with this. -# We could paper over this by loading the kernel at 0xc0000000 virtual, but -# that leads to other complications, so we'll just reclaim the lower region of -# ram after we're loaded. Put the page tables for startup at 1MB. -makeoptions KERNPHYSADDR=0x00900000 -makeoptions KERNVIRTADDR=0xc0900000 - -options KERNPHYSADDR=0x00900000 -options KERNVIRTADDR=0xc0900000 -options PHYSADDR=0x00000000 +makeoptions KERNVIRTADDR=0xc0000000 +options KERNVIRTADDR=0xc0000000 diff --git a/sys/arm/mv/orion/std.db88f5xxx b/sys/arm/mv/orion/std.db88f5xxx index 7b52d7d..8a13725 100644 --- a/sys/arm/mv/orion/std.db88f5xxx +++ b/sys/arm/mv/orion/std.db88f5xxx @@ -5,7 +5,5 @@ files "../mv/orion/files.db88f5xxx" makeoptions KERNPHYSADDR=0x00900000 makeoptions KERNVIRTADDR=0xc0900000 - -options KERNPHYSADDR=0x00900000 options KERNVIRTADDR=0xc0900000 options PHYSADDR=0x00000000 diff --git a/sys/arm/mv/orion/std.ts7800 b/sys/arm/mv/orion/std.ts7800 index aa5dcef..931da3a 100644 --- a/sys/arm/mv/orion/std.ts7800 +++ b/sys/arm/mv/orion/std.ts7800 @@ -5,8 +5,6 @@ files "../mv/orion/files.ts7800" makeoptions KERNPHYSADDR=0x00900000 makeoptions KERNVIRTADDR=0xc0900000 - -options KERNPHYSADDR=0x00900000 options KERNVIRTADDR=0xc0900000 options PHYSADDR=0x00000000 options LOADERRAMADDR=0x00000000 diff --git a/sys/arm/xscale/i80321/ep80219_machdep.c b/sys/arm/xscale/i80321/ep80219_machdep.c index d93ed74..6c0d1f5 100644 --- a/sys/arm/xscale/i80321/ep80219_machdep.c +++ b/sys/arm/xscale/i80321/ep80219_machdep.c @@ -343,9 +343,9 @@ initarm(struct arm_boot_params *abp) * Prepare the list of physical memory available to the vm subsystem. */ arm_physmem_hardware_region(IQ80321_SDRAM_START, memsize); - arm_physmem_exclude_region(freemem_pt, KERNPHYSADDR - + arm_physmem_exclude_region(freemem_pt, abp->abp_physaddr - freemem_pt, EXFLAG_NOALLOC); - arm_physmem_exclude_region(freemempos, KERNPHYSADDR - 0x100000 - + arm_physmem_exclude_region(freemempos, abp->abp_physaddr - 0x100000 - freemempos, EXFLAG_NOALLOC); arm_physmem_exclude_region(abp->abp_physaddr, virtual_avail - KERNVIRTADDR, EXFLAG_NOALLOC); diff --git a/sys/arm/xscale/i80321/iq31244_machdep.c b/sys/arm/xscale/i80321/iq31244_machdep.c index 52d94af..eb19f78 100644 --- a/sys/arm/xscale/i80321/iq31244_machdep.c +++ b/sys/arm/xscale/i80321/iq31244_machdep.c @@ -345,9 +345,9 @@ initarm(struct arm_boot_params *abp) * Prepare the list of physical memory available to the vm subsystem. */ arm_physmem_hardware_region(SDRAM_START, memsize); - arm_physmem_exclude_region(freemem_pt, KERNPHYSADDR - + arm_physmem_exclude_region(freemem_pt, abp->abp_physaddr - freemem_pt, EXFLAG_NOALLOC); - arm_physmem_exclude_region(freemempos, KERNPHYSADDR - 0x100000 - + arm_physmem_exclude_region(freemempos, abp->abp_physaddr - 0x100000 - freemempos, EXFLAG_NOALLOC); arm_physmem_exclude_region(abp->abp_physaddr, virtual_avail - KERNVIRTADDR, EXFLAG_NOALLOC); diff --git a/sys/arm/xscale/i8134x/crb_machdep.c b/sys/arm/xscale/i8134x/crb_machdep.c index 138ed09..2bd77a5 100644 --- a/sys/arm/xscale/i8134x/crb_machdep.c +++ b/sys/arm/xscale/i8134x/crb_machdep.c @@ -325,9 +325,9 @@ initarm(struct arm_boot_params *abp) * Prepare the list of physical memory available to the vm subsystem. */ arm_physmem_hardware_region(SDRAM_START, memsize); - arm_physmem_exclude_region(freemem_pt, KERNPHYSADDR - + arm_physmem_exclude_region(freemem_pt, abp->abp_physaddr - freemem_pt, EXFLAG_NOALLOC); - arm_physmem_exclude_region(freemempos, KERNPHYSADDR - 0x100000 - + arm_physmem_exclude_region(freemempos, abp->abp_physaddr - 0x100000 - freemempos, EXFLAG_NOALLOC); arm_physmem_exclude_region(abp->abp_physaddr, virtual_avail - KERNVIRTADDR, EXFLAG_NOALLOC); diff --git a/sys/arm/xscale/ixp425/avila_machdep.c b/sys/arm/xscale/ixp425/avila_machdep.c index 0d5d9bb..e48fbb6 100644 --- a/sys/arm/xscale/ixp425/avila_machdep.c +++ b/sys/arm/xscale/ixp425/avila_machdep.c @@ -415,9 +415,9 @@ initarm(struct arm_boot_params *abp) * Prepare the list of physical memory available to the vm subsystem. */ arm_physmem_hardware_region(PHYSADDR, memsize); - arm_physmem_exclude_region(freemem_pt, KERNPHYSADDR - + arm_physmem_exclude_region(freemem_pt, abp->abp_physaddr - freemem_pt, EXFLAG_NOALLOC); - arm_physmem_exclude_region(freemempos, KERNPHYSADDR - 0x100000 - + arm_physmem_exclude_region(freemempos, abp->abp_physaddr - 0x100000 - freemempos, EXFLAG_NOALLOC); arm_physmem_exclude_region(abp->abp_physaddr, virtual_avail - KERNVIRTADDR, EXFLAG_NOALLOC); diff --git a/sys/arm/xscale/ixp425/std.avila b/sys/arm/xscale/ixp425/std.avila index 9c00e1b..8f67379 100644 --- a/sys/arm/xscale/ixp425/std.avila +++ b/sys/arm/xscale/ixp425/std.avila @@ -13,7 +13,6 @@ files "../xscale/ixp425/files.avila" # be remapped away from 0. # options PHYSADDR=0x00000000 -options KERNPHYSADDR=0x00200000 makeoptions KERNPHYSADDR=0x00200000 options KERNVIRTADDR=0xc0200000 # Used in ldscript.arm makeoptions KERNVIRTADDR=0xc0200000 diff --git a/sys/arm/xscale/pxa/pxa_machdep.c b/sys/arm/xscale/pxa/pxa_machdep.c index 41e49c3..5cc8d46 100644 --- a/sys/arm/xscale/pxa/pxa_machdep.c +++ b/sys/arm/xscale/pxa/pxa_machdep.c @@ -336,9 +336,9 @@ initarm(struct arm_boot_params *abp) if (memsize[j] > 0) arm_physmem_hardware_region(memstart[j], memsize[j]); } - arm_physmem_exclude_region(freemem_pt, KERNPHYSADDR - + arm_physmem_exclude_region(freemem_pt, abp->abp_physaddr - freemem_pt, EXFLAG_NOALLOC); - arm_physmem_exclude_region(freemempos, KERNPHYSADDR - 0x100000 - + arm_physmem_exclude_region(freemempos, abp->abp_physaddr - 0x100000 - freemempos, EXFLAG_NOALLOC); arm_physmem_exclude_region(abp->abp_physaddr, virtual_avail - KERNVIRTADDR, EXFLAG_NOALLOC); diff --git a/sys/arm64/include/setjmp.h b/sys/arm64/include/setjmp.h index 98dc0e8..c42d237 100644 --- a/sys/arm64/include/setjmp.h +++ b/sys/arm64/include/setjmp.h @@ -43,13 +43,12 @@ * - 11 general purpose registers * - 8 floating point registers * - The signal mask (128 bits) - * i.e. 24 64-bit words, this can be rounded up to 32 to give us some - * space to expand into without affecting the ABI. - * XXX: Is this enough space for expansion? + * i.e. 24 64-bit words, round this up to 31(+1) 128-bit words to allow for + * CPU extensions with larger registers and stronger alignment requirements. * * The registers to save are: r19 to r29, and d8 to d15. */ -#define _JBLEN 32 +#define _JBLEN 31 #define _JB_SIGMASK 21 /* This should only be needed in libc and may change */ @@ -65,10 +64,10 @@ * internally to avoid some run-time errors for mismatches. */ #if __BSD_VISIBLE || __POSIX_VISIBLE || __XSI_VISIBLE -typedef struct _sigjmp_buf { long _sjb[_JBLEN + 1]; } sigjmp_buf[1]; +typedef struct _sigjmp_buf { __int128_t _sjb[_JBLEN + 1]; } sigjmp_buf[1]; #endif -typedef struct _jmp_buf { long _jb[_JBLEN + 1]; } jmp_buf[1]; +typedef struct _jmp_buf { __int128_t _jb[_JBLEN + 1]; } jmp_buf[1]; #endif /* __ASSEMBLER__ */ #endif /* !_MACHINE_SETJMP_H_ */ diff --git a/sys/boot/efi/boot1/boot1.c b/sys/boot/efi/boot1/boot1.c index eff191b..be59993 100644 --- a/sys/boot/efi/boot1/boot1.c +++ b/sys/boot/efi/boot1/boot1.c @@ -330,18 +330,21 @@ load(const char *fname) status = systab->BootServices->LoadImage(TRUE, image, bootdevpath, buffer, bufsize, &loaderhandle); if (EFI_ERROR(status)) - printf("LoadImage failed with error %lx\n", status); + printf("LoadImage failed with error %lu\n", + status & ~EFI_ERROR_MASK); status = systab->BootServices->HandleProtocol(loaderhandle, &LoadedImageGUID, (VOID**)&loaded_image); if (EFI_ERROR(status)) - printf("HandleProtocol failed with error %lx\n", status); + printf("HandleProtocol failed with error %lu\n", + status & ~EFI_ERROR_MASK); loaded_image->DeviceHandle = bootdevhandle; status = systab->BootServices->StartImage(loaderhandle, NULL, NULL); if (EFI_ERROR(status)) - printf("StartImage failed with error %lx\n", status); + printf("StartImage failed with error %lu\n", + status & ~EFI_ERROR_MASK); } static void diff --git a/sys/boot/efi/loader/Makefile b/sys/boot/efi/loader/Makefile index c607f46..d6bd9c7 100644 --- a/sys/boot/efi/loader/Makefile +++ b/sys/boot/efi/loader/Makefile @@ -67,7 +67,7 @@ HAVE_BCACHE= yes CFLAGS+= -DEFI_STAGING_SIZE=${EFI_STAGING_SIZE} .endif -# Always add MI sources +# Always add MI sources .PATH: ${.CURDIR}/../../common .include "${.CURDIR}/../../common/Makefile.inc" CFLAGS+= -I${.CURDIR}/../../common diff --git a/sys/boot/efi/loader/arch/amd64/framebuffer.c b/sys/boot/efi/loader/arch/amd64/framebuffer.c index 9a8dfdb..eb78f7b 100644 --- a/sys/boot/efi/loader/arch/amd64/framebuffer.c +++ b/sys/boot/efi/loader/arch/amd64/framebuffer.c @@ -303,7 +303,7 @@ efifb_from_uga(struct efi_fb *efifb, EFI_UGA_DRAW_PROTOCOL *uga) * offset within the frame buffer of the visible region, nor * the stride. Our only option is to look at the system and * fill in the blanks based on that. Luckily, UGA was mostly - * only used on Apple hardware. + * only used on Apple hardware. */ offset = -1; ev = getenv("smbios.system.maker"); diff --git a/sys/boot/efi/loader/bootinfo.c b/sys/boot/efi/loader/bootinfo.c index 0b7b7ef..622f4c6 100644 --- a/sys/boot/efi/loader/bootinfo.c +++ b/sys/boot/efi/loader/bootinfo.c @@ -234,44 +234,6 @@ bi_copymodules(vm_offset_t addr) } static int -bi_add_efi_data_and_exit(struct preloaded_file *kfp, - struct efi_map_header *efihdr, size_t efisz, EFI_MEMORY_DESCRIPTOR *mm, - UINTN sz) -{ - UINTN efi_mapkey; - UINTN mmsz; - UINT32 mmver; - EFI_STATUS status; - UINTN retry; - - /* - * It is possible that the first call to ExitBootServices may change - * the map key. Fetch a new map key and retry ExitBootServices in that - * case. - */ - for (retry = 2; retry > 0; retry--) { - status = BS->GetMemoryMap(&sz, mm, &efi_mapkey, &mmsz, &mmver); - if (EFI_ERROR(status)) { - printf("%s: GetMemoryMap error %lu\n", __func__, - (unsigned long)(status & ~EFI_ERROR_MASK)); - return (EINVAL); - } - status = BS->ExitBootServices(IH, efi_mapkey); - if (EFI_ERROR(status) == 0) { - efihdr->memory_size = sz; - efihdr->descriptor_size = mmsz; - efihdr->descriptor_version = mmver; - file_addmetadata(kfp, MODINFOMD_EFI_MAP, efisz + sz, - efihdr); - return (0); - } - } - printf("ExitBootServices error %lu\n", - (unsigned long)(status & ~EFI_ERROR_MASK)); - return (EINVAL); -} - -static int bi_load_efi_data(struct preloaded_file *kfp) { EFI_MEMORY_DESCRIPTOR *mm; @@ -279,7 +241,7 @@ bi_load_efi_data(struct preloaded_file *kfp) EFI_STATUS status; size_t efisz; UINTN efi_mapkey; - UINTN mmsz, pages, sz; + UINTN mmsz, pages, retry, sz; UINT32 mmver; struct efi_map_header *efihdr; @@ -304,37 +266,63 @@ bi_load_efi_data(struct preloaded_file *kfp) efisz = (sizeof(struct efi_map_header) + 0xf) & ~0xf; /* - * Allocate enough pages to hold the bootinfo block and the memory - * map EFI will return to us. The memory map has an unknown size, - * so we have to determine that first. Note that the AllocatePages - * call can itself modify the memory map, so we have to take that - * into account as well. The changes to the memory map are caused - * by splitting a range of free memory into two (AFAICT), so that - * one is marked as being loader data. + * It is possible that the first call to ExitBootServices may change + * the map key. Fetch a new map key and retry ExitBootServices in that + * case. */ - sz = 0; - BS->GetMemoryMap(&sz, NULL, &efi_mapkey, &mmsz, &mmver); - sz += mmsz; - sz = (sz + 0xf) & ~0xf; - pages = EFI_SIZE_TO_PAGES(sz + efisz); - status = BS->AllocatePages(AllocateAnyPages, EfiLoaderData, pages, - &addr); - if (EFI_ERROR(status)) { - printf("%s: AllocatePages error %lu\n", __func__, - (unsigned long)(status & ~EFI_ERROR_MASK)); - return (ENOMEM); - } + for (retry = 2; retry > 0; retry--) { + /* + * Allocate enough pages to hold the bootinfo block and the + * memory map EFI will return to us. The memory map has an + * unknown size, so we have to determine that first. Note that + * the AllocatePages call can itself modify the memory map, so + * we have to take that into account as well. The changes to + * the memory map are caused by splitting a range of free + * memory into two (AFAICT), so that one is marked as being + * loader data. + */ + sz = 0; + BS->GetMemoryMap(&sz, NULL, &efi_mapkey, &mmsz, &mmver); + sz += mmsz; + sz = (sz + 0xf) & ~0xf; + pages = EFI_SIZE_TO_PAGES(sz + efisz); + status = BS->AllocatePages(AllocateAnyPages, EfiLoaderData, + pages, &addr); + if (EFI_ERROR(status)) { + printf("%s: AllocatePages error %lu\n", __func__, + (unsigned long)(status & ~EFI_ERROR_MASK)); + return (ENOMEM); + } - /* - * Read the memory map and stash it after bootinfo. Align the - * memory map on a 16-byte boundary (the bootinfo block is page - * aligned). - */ - efihdr = (struct efi_map_header *)addr; - mm = (void *)((uint8_t *)efihdr + efisz); - sz = (EFI_PAGE_SIZE * pages) - efisz; + /* + * Read the memory map and stash it after bootinfo. Align the + * memory map on a 16-byte boundary (the bootinfo block is page + * aligned). + */ + efihdr = (struct efi_map_header *)addr; + mm = (void *)((uint8_t *)efihdr + efisz); + sz = (EFI_PAGE_SIZE * pages) - efisz; - return (bi_add_efi_data_and_exit(kfp, efihdr, efisz, mm, sz)); + status = BS->GetMemoryMap(&sz, mm, &efi_mapkey, &mmsz, &mmver); + if (EFI_ERROR(status)) { + printf("%s: GetMemoryMap error %lu\n", __func__, + (unsigned long)(status & ~EFI_ERROR_MASK)); + return (EINVAL); + } + status = BS->ExitBootServices(IH, efi_mapkey); + if (EFI_ERROR(status) == 0) { + efihdr->memory_size = sz; + efihdr->descriptor_size = mmsz; + efihdr->descriptor_version = mmver; + file_addmetadata(kfp, MODINFOMD_EFI_MAP, efisz + sz, + efihdr); + return (0); + } + BS->FreePages(addr, pages); + } + printf("ExitBootServices error %lu\n", + (unsigned long)(status & ~EFI_ERROR_MASK)); + return (EINVAL); } /* diff --git a/sys/boot/efi/loader/devicename.c b/sys/boot/efi/loader/devicename.c index a9327dc..89f9941 100644 --- a/sys/boot/efi/loader/devicename.c +++ b/sys/boot/efi/loader/devicename.c @@ -38,7 +38,7 @@ __FBSDID("$FreeBSD$"); static int efi_parsedev(struct devdesc **, const char *, const char **); -/* +/* * Point (dev) at an allocated device specifier for the device matching the * path in (devspec). If it contains an explicit device specification, * use that. If not, use the default device. diff --git a/sys/boot/efi/loader/main.c b/sys/boot/efi/loader/main.c index 8044db5..fc8949e 100644 --- a/sys/boot/efi/loader/main.c +++ b/sys/boot/efi/loader/main.c @@ -64,10 +64,10 @@ EFI_GUID fdtdtb = FDT_TABLE_GUID; EFI_STATUS main(int argc, CHAR16 *argv[]) { - char vendor[128]; + char var[128]; EFI_LOADED_IMAGE *img; EFI_GUID *guid; - int i; + int i, j, vargood; /* * XXX Chicken-and-egg problem; we want to have console output @@ -77,6 +77,29 @@ main(int argc, CHAR16 *argv[]) */ cons_probe(); + /* + * Loop through the args, and for each one that contains an '=' that is + * not the first character, add it to the environment. This allows + * loader and kernel env vars to be passed on the command line. Convert + * args from UCS-2 to ASCII (16 to 8 bit) as they are copied. + */ + for (i = 1; i < argc; i++) { + vargood = 0; + for (j = 0; argv[i][j] != 0; j++) { + if (j == sizeof(var)) { + vargood = 0; + break; + } + if (j > 0 && argv[i][j] == '=') + vargood = 1; + var[j] = (char)argv[i][j]; + } + if (vargood) { + var[j] = 0; + putenv(var); + } + } + if (efi_copy_init()) { printf("failed to allocate staging area\n"); return (EFI_BUFFER_TOO_SMALL); diff --git a/sys/boot/i386/libi386/biosmem.c b/sys/boot/i386/libi386/biosmem.c index 1a9929c..399a208 100644 --- a/sys/boot/i386/libi386/biosmem.c +++ b/sys/boot/i386/libi386/biosmem.c @@ -35,6 +35,7 @@ __FBSDID("$FreeBSD$"); #include "bootstrap.h" #include "libi386.h" #include "btxv86.h" +#include "smbios.h" vm_offset_t memtop, memtop_copyin, high_heap_base; uint32_t bios_basemem, bios_extmem, high_heap_size; diff --git a/sys/boot/i386/zfsboot/zfsboot.c b/sys/boot/i386/zfsboot/zfsboot.c index afb77b2..ca82c63 100644 --- a/sys/boot/i386/zfsboot/zfsboot.c +++ b/sys/boot/i386/zfsboot/zfsboot.c @@ -238,7 +238,7 @@ bios_getmem(void) v86.es = VTOPSEG(&smap); v86.edi = VTOPOFF(&smap); v86int(); - if ((v86.efl & 1) || (v86.eax != SMAP_SIG)) + if (V86_CY(v86.efl) || (v86.eax != SMAP_SIG)) break; /* look for a low-memory segment that's large enough */ if ((smap.type == SMAP_TYPE_MEMORY) && (smap.base == 0) && @@ -285,7 +285,7 @@ bios_getmem(void) v86.addr = 0x15; /* int 0x15 function 0xe801*/ v86.eax = 0xe801; v86int(); - if (!(v86.efl & 1)) { + if (!V86_CY(v86.efl)) { bios_extmem = ((v86.ecx & 0xffff) + ((v86.edx & 0xffff) * 64)) * 1024; } } @@ -320,7 +320,7 @@ int13probe(int drive) v86.edx = drive; v86int(); - if (!(v86.efl & 0x1) && /* carry clear */ + if (!V86_CY(v86.efl) && /* carry clear */ ((v86.edx & 0xff) != (drive & DRV_MASK))) { /* unit # OK */ if ((v86.ecx & 0x3f) == 0) { /* absurd sector size */ return(0); /* skip device */ diff --git a/sys/boot/libstand32/Makefile b/sys/boot/libstand32/Makefile index e04d0be..075a9ea 100644 --- a/sys/boot/libstand32/Makefile +++ b/sys/boot/libstand32/Makefile @@ -41,8 +41,8 @@ SRCS+= ntoh.c .PATH: ${LIBC_SRC}/string SRCS+= bcmp.c bcopy.c bzero.c ffs.c memccpy.c memchr.c memcmp.c memcpy.c \ memmove.c memset.c qdivrem.c strcat.c strchr.c strcmp.c strcpy.c \ - strcspn.c strlen.c strncat.c strncmp.c strncpy.c strpbrk.c \ - strrchr.c strsep.c strspn.c strstr.c strtok.c swab.c + strcspn.c strlcat.c strlcpy.c strlen.c strncat.c strncmp.c strncpy.c \ + strpbrk.c strrchr.c strsep.c strspn.c strstr.c strtok.c swab.c .if ${MACHINE_CPUARCH} == "arm" .PATH: ${LIBC_SRC}/arm/gen diff --git a/sys/boot/pc98/boot2/boot2.c b/sys/boot/pc98/boot2/boot2.c index 83af085..991cc53 100644 --- a/sys/boot/pc98/boot2/boot2.c +++ b/sys/boot/pc98/boot2/boot2.c @@ -327,7 +327,7 @@ bd_getbigeom(int bunit) v86.addr = 0x1b; v86.eax = 0x8400 | unit; v86int(); - if (v86.efl & 0x1) + if (V86_CY(v86.efl)) return 0x4F020F; /* 1200KB FD C:80 H:2 S:15 */ return ((v86.ecx & 0xffff) << 16) | (v86.edx & 0xffff); } diff --git a/sys/boot/pc98/libpc98/biosdisk.c b/sys/boot/pc98/libpc98/biosdisk.c index 96dbd1f..b777d25 100644 --- a/sys/boot/pc98/libpc98/biosdisk.c +++ b/sys/boot/pc98/libpc98/biosdisk.c @@ -824,7 +824,7 @@ bd_chs_io(struct open_disk *od, daddr_t dblk, int blks, caddr_t dest, int write) v86.es = VTOPSEG(dest); v86.ebp = VTOPOFF(dest); v86int(); - return (v86.efl & 0x1); + return (V86_CY(v86.efl)); } static int @@ -959,7 +959,7 @@ bd_getgeom(struct open_disk *od) od->od_cyl = v86.ecx; od->od_hds = (v86.edx >> 8) & 0xff; od->od_sec = v86.edx & 0xff; - if (v86.efl & 0x1) + if (V86_CY(v86.efl)) return(1); } @@ -1010,7 +1010,7 @@ bd_getbigeom(int bunit) v86.addr = 0x1b; v86.eax = 0x8400 | unit; v86int(); - if (v86.efl & 0x1) + if (V86_CY(v86.efl)) return 0x4F020F; /* 1200KB FD C:80 H:2 S:15 */ return ((v86.ecx & 0xffff) << 16) | (v86.edx & 0xffff); } diff --git a/sys/bsm/audit.h b/sys/bsm/audit.h index 625e816..078b333 100644 --- a/sys/bsm/audit.h +++ b/sys/bsm/audit.h @@ -26,7 +26,6 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit.h#10 * $FreeBSD$ */ diff --git a/sys/bsm/audit_domain.h b/sys/bsm/audit_domain.h index 76cd38f..22d2966 100644 --- a/sys/bsm/audit_domain.h +++ b/sys/bsm/audit_domain.h @@ -26,7 +26,6 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_domain.h#2 * $FreeBSD$ */ diff --git a/sys/bsm/audit_errno.h b/sys/bsm/audit_errno.h index d5a375f..939ddeb 100644 --- a/sys/bsm/audit_errno.h +++ b/sys/bsm/audit_errno.h @@ -26,7 +26,6 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_errno.h#7 * $FreeBSD$ */ diff --git a/sys/bsm/audit_fcntl.h b/sys/bsm/audit_fcntl.h index 15b5f51..8b5f1a5 100644 --- a/sys/bsm/audit_fcntl.h +++ b/sys/bsm/audit_fcntl.h @@ -26,7 +26,6 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_fcntl.h#2 * $FreeBSD$ */ diff --git a/sys/bsm/audit_internal.h b/sys/bsm/audit_internal.h index 8ba611f..3ec9896 100644 --- a/sys/bsm/audit_internal.h +++ b/sys/bsm/audit_internal.h @@ -30,7 +30,6 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_internal.h#6 * $FreeBSD$ */ diff --git a/sys/bsm/audit_kevents.h b/sys/bsm/audit_kevents.h index 303d37f..3c16c73 100644 --- a/sys/bsm/audit_kevents.h +++ b/sys/bsm/audit_kevents.h @@ -26,7 +26,6 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_kevents.h#7 * $FreeBSD$ */ @@ -34,7 +33,7 @@ #define _BSM_AUDIT_KEVENTS_H_ /* - * The reserved event numbers for kernel events are 1...2047 and 43001..44900. + * The reserved event numbers for kernel events are 1...2047 and 43001..44999. */ #define AUE_IS_A_KEVENT(e) (((e) > 0 && (e) < 2048) || \ ((e) > 43000 && (e) < 45000)) diff --git a/sys/bsm/audit_record.h b/sys/bsm/audit_record.h index 5da5e4f2..b72236b 100644 --- a/sys/bsm/audit_record.h +++ b/sys/bsm/audit_record.h @@ -26,7 +26,6 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_record.h#10 * $FreeBSD$ */ diff --git a/sys/bsm/audit_socket_type.h b/sys/bsm/audit_socket_type.h index 6b6bac1..1f580fc 100644 --- a/sys/bsm/audit_socket_type.h +++ b/sys/bsm/audit_socket_type.h @@ -26,7 +26,6 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. * - * P4: //depot/projects/trustedbsd/openbsm/sys/bsm/audit_socket_type.h#1 * $FreeBSD$ */ diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c index a64d6ef..e684b21 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c @@ -28,7 +28,7 @@ #include <sys/zfs_context.h> #include <sys/zio.h> #ifdef _KERNEL -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #else #include <sha256.h> #endif diff --git a/sys/cddl/dev/fbt/arm/fbt_isa.c b/sys/cddl/dev/fbt/arm/fbt_isa.c index c3b6fca..0e948dd 100644 --- a/sys/cddl/dev/fbt/arm/fbt_isa.c +++ b/sys/cddl/dev/fbt/arm/fbt_isa.c @@ -35,6 +35,7 @@ #include <sys/param.h> #include <sys/dtrace.h> +#include <machine/stack.h> #include <machine/trap.h> #include "fbt.h" @@ -42,6 +43,7 @@ #define FBT_PUSHM 0xe92d0000 #define FBT_POPM 0xe8bd0000 #define FBT_JUMP 0xea000000 +#define FBT_SUBSP 0xe24dd000 #define FBT_ENTRY "entry" #define FBT_RETURN "return" @@ -111,12 +113,18 @@ fbt_provide_module_function(linker_file_t lf, int symindx, instr = (uint32_t *)symval->value; limit = (uint32_t *)(symval->value + symval->size); - for (; instr < limit; instr++) - if ((*instr & 0xffff0000) == FBT_PUSHM && - (*instr & 0x4000) != 0) - break; + /* + * va_arg functions has first instruction of + * sub sp, sp, #? + */ + if ((*instr & 0xfffff000) == FBT_SUBSP) + instr++; - if (instr >= limit) + /* + * check if insn is a pushm with LR + */ + if ((*instr & 0xffff0000) != FBT_PUSHM || + (*instr & (1 << LR)) == 0) return (0); fbt = malloc(sizeof (fbt_probe_t), M_FBT, M_WAITOK | M_ZERO); diff --git a/sys/compat/linux/linux_futex.c b/sys/compat/linux/linux_futex.c index 84fc996..e2aad79 100644 --- a/sys/compat/linux/linux_futex.c +++ b/sys/compat/linux/linux_futex.c @@ -1099,6 +1099,8 @@ linux_get_robust_list(struct thread *td, struct linux_get_robust_list_args *args ESRCH); return (ESRCH); } + if (SV_PROC_ABI(td2->td_proc) != SV_ABI_LINUX) + return (EPERM); em = em_find(td2); KASSERT(em != NULL, ("get_robust_list: emuldata notfound.\n")); diff --git a/sys/compat/linux/linux_timer.c b/sys/compat/linux/linux_timer.c index 7dbddbe..f2819bb 100644 --- a/sys/compat/linux/linux_timer.c +++ b/sys/compat/linux/linux_timer.c @@ -57,6 +57,8 @@ linux_convert_l_sigevent(struct l_sigevent *l_sig, struct sigevent *sig) CP(*l_sig, *sig, sigev_notify); switch (l_sig->sigev_notify) { case L_SIGEV_SIGNAL: + if (!LINUX_SIG_VALID(l_sig->sigev_signo)) + return (EINVAL); sig->sigev_notify = SIGEV_SIGNAL; sig->sigev_signo = linux_to_bsd_signal(l_sig->sigev_signo); PTRIN_CP(*l_sig, *sig, sigev_value.sival_ptr); @@ -73,6 +75,8 @@ linux_convert_l_sigevent(struct l_sigevent *l_sig, struct sigevent *sig) return (EINVAL); #endif case L_SIGEV_THREAD_ID: + if (!LINUX_SIG_VALID(l_sig->sigev_signo)) + return (EINVAL); sig->sigev_notify = SIGEV_THREAD_ID; CP2(*l_sig, *sig, _l_sigev_un._tid, sigev_notify_thread_id); sig->sigev_signo = linux_to_bsd_signal(l_sig->sigev_signo); diff --git a/sys/compat/linuxkpi/common/include/linux/compiler.h b/sys/compat/linuxkpi/common/include/linux/compiler.h index 6381358..b6ea98f 100644 --- a/sys/compat/linuxkpi/common/include/linux/compiler.h +++ b/sys/compat/linuxkpi/common/include/linux/compiler.h @@ -72,4 +72,20 @@ #define barrier() __asm__ __volatile__("": : :"memory") +#define ACCESS_ONCE(x) (*(volatile __typeof(x) *)&(x)) + +#define WRITE_ONCE(x,v) do { \ + barrier(); \ + ACCESS_ONCE(x) = (v); \ + barrier(); \ +} while (0) + +#define READ_ONCE(x) ({ \ + __typeof(x) __var; \ + barrier(); \ + __var = ACCESS_ONCE(x); \ + barrier(); \ + __var; \ +}) + #endif /* _LINUX_COMPILER_H_ */ diff --git a/sys/compat/linuxkpi/common/include/linux/kobject.h b/sys/compat/linuxkpi/common/include/linux/kobject.h index 0e94809..475095e 100644 --- a/sys/compat/linuxkpi/common/include/linux/kobject.h +++ b/sys/compat/linuxkpi/common/include/linux/kobject.h @@ -103,29 +103,7 @@ kobject_get(struct kobject *kobj) return kobj; } -static inline int -kobject_set_name_vargs(struct kobject *kobj, const char *fmt, va_list args) -{ - char *old; - char *name; - - old = kobj->name; - - if (old && !fmt) - return 0; - - name = kzalloc(MAXPATHLEN, GFP_KERNEL); - if (!name) - return -ENOMEM; - vsnprintf(name, MAXPATHLEN, fmt, args); - kobj->name = name; - kfree(old); - for (; *name != '\0'; name++) - if (*name == '/') - *name = '!'; - return (0); -} - +int kobject_set_name_vargs(struct kobject *kobj, const char *fmt, va_list); int kobject_add(struct kobject *kobj, struct kobject *parent, const char *fmt, ...); diff --git a/sys/compat/linuxkpi/common/include/linux/srcu.h b/sys/compat/linuxkpi/common/include/linux/srcu.h new file mode 100644 index 0000000..c20215b --- /dev/null +++ b/sys/compat/linuxkpi/common/include/linux/srcu.h @@ -0,0 +1,72 @@ +/*- + * Copyright (c) 2015 Mellanox Technologies, Ltd. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice unmodified, this list of conditions, and the following + * disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ +#ifndef _LINUX_SRCU_H_ +#define _LINUX_SRCU_H_ + +#include <sys/param.h> +#include <sys/lock.h> +#include <sys/sx.h> + +struct srcu_struct { + struct sx sx; +}; + +static inline int +init_srcu_struct(struct srcu_struct *srcu) +{ + sx_init(&srcu->sx, "SleepableRCU"); + return (0); +} + +static inline void +cleanup_srcu_struct(struct srcu_struct *srcu) +{ + sx_destroy(&srcu->sx); +} + +static inline int +srcu_read_lock(struct srcu_struct *srcu) +{ + sx_slock(&srcu->sx); + return (0); +} + +static inline void +srcu_read_unlock(struct srcu_struct *srcu, int key) +{ + sx_sunlock(&srcu->sx); +} + +static inline void +synchronize_srcu(struct srcu_struct *srcu) +{ + sx_xlock(&srcu->sx); + sx_xunlock(&srcu->sx); +} + +#endif /* _LINUX_SRCU_H_ */ diff --git a/sys/compat/linuxkpi/common/include/linux/workqueue.h b/sys/compat/linuxkpi/common/include/linux/workqueue.h index 58ef463..7eb2301 100644 --- a/sys/compat/linuxkpi/common/include/linux/workqueue.h +++ b/sys/compat/linuxkpi/common/include/linux/workqueue.h @@ -36,10 +36,13 @@ #include <linux/timer.h> #include <linux/slab.h> +#include <asm/atomic.h> + #include <sys/taskqueue.h> struct workqueue_struct { struct taskqueue *taskqueue; + atomic_t draining; }; struct work_struct { @@ -55,6 +58,12 @@ struct delayed_work { struct callout timer; }; +extern void linux_work_fn(void *, int); +extern void linux_flush_fn(void *, int); +extern void linux_delayed_work_fn(void *); +extern struct workqueue_struct *linux_create_workqueue_common(const char *, int); +extern void destroy_workqueue(struct workqueue_struct *); + static inline struct delayed_work * to_delayed_work(struct work_struct *work) { @@ -62,21 +71,11 @@ to_delayed_work(struct work_struct *work) return container_of(work, struct delayed_work, work); } - -static inline void -_work_fn(void *context, int pending) -{ - struct work_struct *work; - - work = context; - work->fn(work); -} - #define INIT_WORK(work, func) \ do { \ (work)->fn = (func); \ (work)->taskqueue = NULL; \ - TASK_INIT(&(work)->work_task, 0, _work_fn, (work)); \ + TASK_INIT(&(work)->work_task, 0, linux_work_fn, (work)); \ } while (0) #define INIT_DELAYED_WORK(_work, func) \ @@ -85,7 +84,7 @@ do { \ callout_init(&(_work)->timer, 1); \ } while (0) -#define INIT_DEFERRABLE_WORK INIT_DELAYED_WORK +#define INIT_DEFERRABLE_WORK(...) INIT_DELAYED_WORK(__VA_ARGS__) #define schedule_work(work) \ do { \ @@ -95,20 +94,15 @@ do { \ #define flush_scheduled_work() flush_taskqueue(taskqueue_thread) -static inline int queue_work(struct workqueue_struct *q, struct work_struct *work) -{ - (work)->taskqueue = (q)->taskqueue; - /* Return opposite val to align with Linux logic */ - return !taskqueue_enqueue((q)->taskqueue, &(work)->work_task); -} - -static inline void -_delayed_work_fn(void *arg) +static inline int +queue_work(struct workqueue_struct *wq, struct work_struct *work) { - struct delayed_work *work; - - work = arg; - taskqueue_enqueue(work->work.taskqueue, &work->work.work_task); + work->taskqueue = wq->taskqueue; + /* Check for draining */ + if (atomic_read(&wq->draining) != 0) + return (!work->work_task.ta_pending); + /* Return opposite value to align with Linux logic */ + return (!taskqueue_enqueue(wq->taskqueue, &work->work_task)); } static inline int @@ -117,76 +111,65 @@ queue_delayed_work(struct workqueue_struct *wq, struct delayed_work *work, { int pending; - pending = work->work.work_task.ta_pending; work->work.taskqueue = wq->taskqueue; - if (delay != 0) - callout_reset(&work->timer, delay, _delayed_work_fn, work); - else - _delayed_work_fn((void *)work); - + if (atomic_read(&wq->draining) != 0) { + pending = work->work.work_task.ta_pending; + } else if (delay != 0) { + pending = work->work.work_task.ta_pending; + callout_reset(&work->timer, delay, linux_delayed_work_fn, work); + } else { + callout_stop(&work->timer); + pending = taskqueue_enqueue(work->work.taskqueue, + &work->work.work_task); + } return (!pending); } -static inline bool schedule_delayed_work(struct delayed_work *dwork, - unsigned long delay) -{ - struct workqueue_struct wq; - wq.taskqueue = taskqueue_thread; - return queue_delayed_work(&wq, dwork, delay); -} - -static inline struct workqueue_struct * -_create_workqueue_common(char *name, int cpus) +static inline bool +schedule_delayed_work(struct delayed_work *dwork, + unsigned long delay) { - struct workqueue_struct *wq; - - wq = kmalloc(sizeof(*wq), M_WAITOK); - wq->taskqueue = taskqueue_create((name), M_WAITOK, - taskqueue_thread_enqueue, &wq->taskqueue); - taskqueue_start_threads(&wq->taskqueue, cpus, PWAIT, "%s", name); + struct workqueue_struct wq; - return (wq); + wq.taskqueue = taskqueue_thread; + atomic_set(&wq.draining, 0); + return (queue_delayed_work(&wq, dwork, delay)); } - #define create_singlethread_workqueue(name) \ - _create_workqueue_common(name, 1) + linux_create_workqueue_common(name, 1) #define create_workqueue(name) \ - _create_workqueue_common(name, MAXCPU) + linux_create_workqueue_common(name, MAXCPU) #define alloc_ordered_workqueue(name, flags) \ - _create_workqueue_common(name, 1) + linux_create_workqueue_common(name, 1) #define alloc_workqueue(name, flags, max_active) \ - _create_workqueue_common(name, max_active) - -static inline void -destroy_workqueue(struct workqueue_struct *wq) -{ - taskqueue_free(wq->taskqueue); - kfree(wq); -} + linux_create_workqueue_common(name, max_active) #define flush_workqueue(wq) flush_taskqueue((wq)->taskqueue) static inline void -_flush_fn(void *context, int pending) -{ -} - -static inline void flush_taskqueue(struct taskqueue *tq) { struct task flushtask; PHOLD(curproc); - TASK_INIT(&flushtask, 0, _flush_fn, NULL); + TASK_INIT(&flushtask, 0, linux_flush_fn, NULL); taskqueue_enqueue(tq, &flushtask); taskqueue_drain(tq, &flushtask); PRELE(curproc); } +static inline void +drain_workqueue(struct workqueue_struct *wq) +{ + atomic_inc(&wq->draining); + flush_taskqueue(wq->taskqueue); + atomic_dec(&wq->draining); +} + static inline int cancel_work_sync(struct work_struct *work) { @@ -223,7 +206,7 @@ cancel_delayed_work_sync(struct delayed_work *work) static inline bool mod_delayed_work(struct workqueue_struct *wq, struct delayed_work *dwork, - unsigned long delay) + unsigned long delay) { cancel_delayed_work(dwork); queue_delayed_work(wq, dwork, delay); diff --git a/sys/compat/linuxkpi/common/src/linux_compat.c b/sys/compat/linuxkpi/common/src/linux_compat.c index cc0479b..91ec693 100644 --- a/sys/compat/linuxkpi/common/src/linux_compat.c +++ b/sys/compat/linuxkpi/common/src/linux_compat.c @@ -64,6 +64,7 @@ __FBSDID("$FreeBSD$"); #include <linux/vmalloc.h> #include <linux/netdevice.h> #include <linux/timer.h> +#include <linux/workqueue.h> #include <vm/vm_pager.h> @@ -93,7 +94,50 @@ panic_cmp(struct rb_node *one, struct rb_node *two) } RB_GENERATE(linux_root, rb_node, __entry, panic_cmp); - + +int +kobject_set_name_vargs(struct kobject *kobj, const char *fmt, va_list args) +{ + va_list tmp_va; + int len; + char *old; + char *name; + char dummy; + + old = kobj->name; + + if (old && fmt == NULL) + return (0); + + /* compute length of string */ + va_copy(tmp_va, args); + len = vsnprintf(&dummy, 0, fmt, tmp_va); + va_end(tmp_va); + + /* account for zero termination */ + len++; + + /* check for error */ + if (len < 1) + return (-EINVAL); + + /* allocate memory for string */ + name = kzalloc(len, GFP_KERNEL); + if (name == NULL) + return (-ENOMEM); + vsnprintf(name, len, fmt, args); + kobj->name = name; + + /* free old string */ + kfree(old); + + /* filter new string */ + for (; *name != '\0'; name++) + if (*name == '/') + *name = '!'; + return (0); +} + int kobject_set_name(struct kobject *kobj, const char *fmt, ...) { @@ -913,6 +957,50 @@ linux_completion_done(struct completion *c) return (isdone); } +void +linux_delayed_work_fn(void *arg) +{ + struct delayed_work *work; + + work = arg; + taskqueue_enqueue(work->work.taskqueue, &work->work.work_task); +} + +void +linux_work_fn(void *context, int pending) +{ + struct work_struct *work; + + work = context; + work->fn(work); +} + +void +linux_flush_fn(void *context, int pending) +{ +} + +struct workqueue_struct * +linux_create_workqueue_common(const char *name, int cpus) +{ + struct workqueue_struct *wq; + + wq = kmalloc(sizeof(*wq), M_WAITOK); + wq->taskqueue = taskqueue_create(name, M_WAITOK, + taskqueue_thread_enqueue, &wq->taskqueue); + atomic_set(&wq->draining, 0); + taskqueue_start_threads(&wq->taskqueue, cpus, PWAIT, "%s", name); + + return (wq); +} + +void +destroy_workqueue(struct workqueue_struct *wq) +{ + taskqueue_free(wq->taskqueue); + kfree(wq); +} + static void linux_compat_init(void *arg) { diff --git a/sys/conf/Makefile.mips b/sys/conf/Makefile.mips index 30f7d1e..09eca75 100644 --- a/sys/conf/Makefile.mips +++ b/sys/conf/Makefile.mips @@ -42,7 +42,7 @@ TRAMPLOADADDR?=0x807963c0 # We default to the MIPS32 ISA, if none specified in the # kernel configuration file. ARCH_FLAGS?=-march=mips32 -EXTRA_FLAGS=-DKERNLOADADDR=${KERNLOADADDR} +EXTRA_FLAGS=-fno-pic -mno-abicalls -G0 -DKERNLOADADDR=${KERNLOADADDR} HACK_EXTRA_FLAGS=-shared diff --git a/sys/conf/files b/sys/conf/files index 55174f7..feac6c0 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -557,10 +557,9 @@ crypto/rijndael/rijndael-api-fst.c optional geom_bde | random !random_loadable crypto/rijndael/rijndael-api.c optional crypto | ipsec | wlan_ccmp crypto/sha1.c optional carp | crypto | ipsec | \ netgraph_mppc_encryption | sctp -crypto/sha2/sha2.c optional crypto | geom_bde | ipsec | random !random_loadable | \ - sctp | zfs crypto/sha2/sha256c.c optional crypto | geom_bde | ipsec | random !random_loadable | \ sctp | zfs +crypto/sha2/sha512c.c optional crypto | geom_bde | ipsec | zfs crypto/siphash/siphash.c optional inet | inet6 crypto/siphash/siphash_test.c optional inet | inet6 ddb/db_access.c optional ddb @@ -1404,8 +1403,6 @@ dev/etherswitch/ip17x/ip175c.c optional ip17x dev/etherswitch/ip17x/ip175d.c optional ip17x dev/etherswitch/ip17x/ip17x_phy.c optional ip17x dev/etherswitch/ip17x/ip17x_vlans.c optional ip17x -dev/etherswitch/mdio_if.m optional miiproxy | mdio -dev/etherswitch/mdio.c optional miiproxy | mdio dev/etherswitch/miiproxy.c optional miiproxy dev/etherswitch/rtl8366/rtl8366rb.c optional rtl8366rb dev/etherswitch/ukswitch/ukswitch.c optional ukswitch @@ -1861,6 +1858,8 @@ dev/ixgbe/if_ixv.c optional ixv inet \ compile-with "${NORMAL_C} -I$S/dev/ixgbe -DSMP" dev/ixgbe/ix_txrx.c optional ix inet | ixv inet \ compile-with "${NORMAL_C} -I$S/dev/ixgbe" +dev/ixgbe/ixgbe_osdep.c optional ix inet | ixv inet \ + compile-with "${NORMAL_C} -I$S/dev/ixgbe" dev/ixgbe/ixgbe_phy.c optional ix inet | ixv inet \ compile-with "${NORMAL_C} -I$S/dev/ixgbe" dev/ixgbe/ixgbe_api.c optional ix inet | ixv inet \ @@ -1906,6 +1905,8 @@ dev/mca/mca_bus.c optional mca dev/mcd/mcd.c optional mcd isa nowerror dev/mcd/mcd_isa.c optional mcd isa nowerror dev/md/md.c optional md +dev/mdio/mdio_if.m optional miiproxy | mdio +dev/mdio/mdio.c optional miiproxy | mdio dev/mem/memdev.c optional mem dev/mem/memutil.c optional mem dev/mfi/mfi.c optional mfi @@ -2094,6 +2095,7 @@ dev/ofw/ofw_bus_subr.c optional fdt dev/ofw/ofw_fdt.c optional fdt dev/ofw/ofw_if.m optional fdt dev/ofw/ofw_iicbus.c optional fdt iicbus +dev/ofw/ofw_subr.c optional fdt dev/ofw/ofwbus.c optional fdt dev/ofw/openfirm.c optional fdt dev/ofw/openfirmio.c optional fdt @@ -3685,6 +3687,7 @@ netinet/sctp_usrreq.c optional inet sctp | inet6 sctp netinet/sctputil.c optional inet sctp | inet6 sctp netinet/siftr.c optional inet siftr alq | inet6 siftr alq netinet/tcp_debug.c optional tcpdebug +netinet/tcp_fastopen.c optional inet tcp_rfc7413 | inet6 tcp_rfc7413 netinet/tcp_hostcache.c optional inet | inet6 netinet/tcp_input.c optional inet | inet6 netinet/tcp_lro.c optional inet | inet6 diff --git a/sys/conf/files.arm b/sys/conf/files.arm index 292fd54..b3b9ea3 100644 --- a/sys/conf/files.arm +++ b/sys/conf/files.arm @@ -55,6 +55,7 @@ arm/arm/minidump_machdep.c optional mem arm/arm/mp_machdep.c optional smp arm/arm/mpcore_timer.c optional mpcore_timer arm/arm/nexus.c standard +arm/arm/ofw_machdep.c optional fdt arm/arm/physmem.c standard kern/pic_if.m optional arm_intrng arm/arm/pl190.c optional pl190 diff --git a/sys/conf/files.mips b/sys/conf/files.mips index 22b9472..da21410 100644 --- a/sys/conf/files.mips +++ b/sys/conf/files.mips @@ -30,6 +30,7 @@ mips/mips/minidump_machdep.c standard mips/mips/mp_machdep.c optional smp mips/mips/mpboot.S optional smp mips/mips/nexus.c standard +mips/mips/ofw_machdep.c optional fdt mips/mips/pm_machdep.c standard mips/mips/pmap.c standard mips/mips/ptrace_machdep.c standard diff --git a/sys/conf/files.powerpc b/sys/conf/files.powerpc index e8827d7..0a1e7c1 100644 --- a/sys/conf/files.powerpc +++ b/sys/conf/files.powerpc @@ -58,6 +58,7 @@ dev/ofw/ofw_disk.c optional ofwd aim dev/ofw/ofw_iicbus.c optional iicbus aim dev/ofw/ofwbus.c optional aim | fdt dev/ofw/ofw_standard.c optional aim powerpc +dev/ofw/ofw_subr.c optional aim powerpc dev/powermac_nvram/powermac_nvram.c optional powermac_nvram powermac dev/quicc/quicc_bfe_fdt.c optional quicc mpc85xx dev/scc/scc_bfe_macio.c optional scc powermac diff --git a/sys/conf/kern.mk b/sys/conf/kern.mk index 0a8161d..56ddbda 100644 --- a/sys/conf/kern.mk +++ b/sys/conf/kern.mk @@ -160,7 +160,6 @@ CFLAGS.gcc+= -mcall-aixdesc # For MIPS we also tell gcc to use floating point emulation # .if ${MACHINE_CPUARCH} == "mips" -CFLAGS+= -fno-pic -mno-abicalls -G0 CFLAGS+= -msoft-float INLINE_LIMIT?= 8000 .endif diff --git a/sys/conf/kmod.mk b/sys/conf/kmod.mk index 2b7fab2..50000c5 100644 --- a/sys/conf/kmod.mk +++ b/sys/conf/kmod.mk @@ -130,7 +130,7 @@ CFLAGS+= -mlongcall -fno-omit-frame-pointer .endif .if ${MACHINE_CPUARCH} == mips -CFLAGS+= -mlong-calls +CFLAGS+= -G0 -fno-pic -mno-abicalls -mlong-calls .endif .if defined(DEBUG) || defined(DEBUG_FLAGS) diff --git a/sys/conf/options b/sys/conf/options index 65932b8..acded05 100644 --- a/sys/conf/options +++ b/sys/conf/options @@ -440,6 +440,8 @@ TCPDEBUG TCPPCAP opt_global.h SIFTR TCP_OFFLOAD opt_inet.h # Enable code to dispatch TCP offloading +TCP_RFC7413 opt_inet.h +TCP_RFC7413_MAX_KEYS opt_inet.h TCP_SIGNATURE opt_inet.h VLAN_ARRAY opt_vlan.h XBONEHACK diff --git a/sys/conf/options.arm b/sys/conf/options.arm index a6dcae3..fdddecb 100644 --- a/sys/conf/options.arm +++ b/sys/conf/options.arm @@ -32,7 +32,6 @@ IPI_IRQ_END opt_smp.h FREEBSD_BOOT_LOADER opt_global.h IXP4XX_FLASH_SIZE opt_global.h KERNBASE opt_global.h -KERNPHYSADDR opt_global.h KERNVIRTADDR opt_global.h LINUX_BOOT_ABI opt_global.h LOADERRAMADDR opt_global.h diff --git a/sys/conf/options.mips b/sys/conf/options.mips index 93d4ed3..47cff06 100644 --- a/sys/conf/options.mips +++ b/sys/conf/options.mips @@ -31,6 +31,7 @@ CPU_MIPS4KC opt_global.h CPU_MIPS24KC opt_global.h CPU_MIPS74KC opt_global.h +CPU_MIPS1004KC opt_global.h CPU_MIPS32 opt_global.h CPU_MIPS64 opt_global.h CPU_SENTRY5 opt_global.h @@ -120,6 +121,18 @@ IF_RT_PHY_SUPPORT opt_if_rt.h IF_RT_RING_DATA_COUNT opt_if_rt.h # +# Options that control the Ralink/Mediatek SoC type. +# +MT7620 opt_rt305x.h +RT5350 opt_rt305x.h +RT305XF opt_rt305x.h +RT3052F opt_rt305x.h +RT3050F opt_rt305x.h +RT305X opt_rt305x.h +RT305X_UBOOT opt_rt305x.h +RT305X_USE_UART opt_rt305x.h + +# # Options that affect the pmap. # PV_STATS opt_pmap.h diff --git a/sys/contrib/ipfilter/netinet/ip_nat.c b/sys/contrib/ipfilter/netinet/ip_nat.c index 9ec450b..5d0e972 100644 --- a/sys/contrib/ipfilter/netinet/ip_nat.c +++ b/sys/contrib/ipfilter/netinet/ip_nat.c @@ -5123,7 +5123,7 @@ ipf_nat_out(fin, nat, natadd, nflags) ipf_fix_outcksum(0, &fin->fin_ip->ip_sum, msumd, 0); } #if !defined(_KERNEL) || defined(MENTAT) || defined(__sgi) || \ - defined(linux) || defined(BRIDGE_IPF) + defined(linux) || defined(BRIDGE_IPF) || defined(__FreeBSD__) else { /* * Strictly speaking, this isn't necessary on BSD @@ -5235,7 +5235,7 @@ ipf_nat_out(fin, nat, natadd, nflags) uh->uh_ulen += fin->fin_plen; uh->uh_ulen = htons(uh->uh_ulen); #if !defined(_KERNEL) || defined(MENTAT) || defined(__sgi) || \ - defined(linux) || defined(BRIDGE_IPF) + defined(linux) || defined(BRIDGE_IPF) || defined(__FreeBSD__) ipf_fix_outcksum(0, &ip->ip_sum, sumd, 0); #endif diff --git a/sys/crypto/sha2/sha2.c b/sys/crypto/sha2/sha2.c deleted file mode 100644 index 5bb52f3..0000000 --- a/sys/crypto/sha2/sha2.c +++ /dev/null @@ -1,708 +0,0 @@ -/* $KAME: sha2.c,v 1.11 2004/06/02 09:52:45 itojun Exp $ */ - -/* - * sha2.c - * - * Version 1.0.0beta1 - * - * Written by Aaron D. Gifford <me@aarongifford.com> - * - * Copyright 2000 Aaron D. Gifford. All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. Neither the name of the copyright holder nor the names of contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) AND CONTRIBUTOR(S) ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR(S) OR CONTRIBUTOR(S) BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - */ - -#include <sys/cdefs.h> -__FBSDID("$FreeBSD$"); - -#include <sys/types.h> -#include <sys/time.h> -#ifdef _KERNEL -#include <sys/systm.h> -#else -#include <string.h> -#endif -#include <machine/endian.h> -#include <crypto/sha2/sha2.h> - -/* - * ASSERT NOTE: - * Some sanity checking code is included using assert(). On my FreeBSD - * system, this additional code can be removed by compiling with NDEBUG - * defined. Check your own systems manpage on assert() to see how to - * compile WITHOUT the sanity checking code on your system. - * - * UNROLLED TRANSFORM LOOP NOTE: - * You can define SHA2_UNROLL_TRANSFORM to use the unrolled transform - * loop version for the hash transform rounds (defined using macros - * later in this file). Either define on the command line, for example: - * - * cc -DSHA2_UNROLL_TRANSFORM -o sha2 sha2.c sha2prog.c - * - * or define below: - * - * #define SHA2_UNROLL_TRANSFORM - * - */ - -#if defined(_KERNEL) && defined(__FreeBSD__) -#define assert(x) -#else -#include <assert.h> -#endif - - -/*** SHA-256/384/512 Machine Architecture Definitions *****************/ -/* - * BYTE_ORDER NOTE: - * - * Please make sure that your system defines BYTE_ORDER. If your - * architecture is little-endian, make sure it also defines - * LITTLE_ENDIAN and that the two (BYTE_ORDER and LITTLE_ENDIAN) are - * equivilent. - * - * If your system does not define the above, then you can do so by - * hand like this: - * - * #define LITTLE_ENDIAN 1234 - * #define BIG_ENDIAN 4321 - * - * And for little-endian machines, add: - * - * #define BYTE_ORDER LITTLE_ENDIAN - * - * Or for big-endian machines: - * - * #define BYTE_ORDER BIG_ENDIAN - * - * The FreeBSD machine this was written on defines BYTE_ORDER - * appropriately by including <sys/types.h> (which in turn includes - * <machine/endian.h> where the appropriate definitions are actually - * made). - */ -#if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN && BYTE_ORDER != BIG_ENDIAN) -#error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN -#endif - -/* - * Define the followingsha2_* types to types of the correct length on - * the native archtecture. Most BSD systems and Linux define u_intXX_t - * types. Machines with very recent ANSI C headers, can use the - * uintXX_t definintions from inttypes.h by defining SHA2_USE_INTTYPES_H - * during compile or in the sha.h header file. - * - * Machines that support neither u_intXX_t nor inttypes.h's uintXX_t - * will need to define these three typedefs below (and the appropriate - * ones in sha.h too) by hand according to their system architecture. - * - * Thank you, Jun-ichiro itojun Hagino, for suggesting using u_intXX_t - * types and pointing out recent ANSI C support for uintXX_t in inttypes.h. - */ -typedef uint8_t sha2_byte; /* Exactly 1 byte */ -typedef uint32_t sha2_word32; /* Exactly 4 bytes */ -typedef uint64_t sha2_word64; /* Exactly 8 bytes */ - - -/*** SHA-256/384/512 Various Length Definitions ***********************/ -/* NOTE: Most of these are in sha2.h */ -#define SHA256_SHORT_BLOCK_LENGTH (SHA256_BLOCK_LENGTH - 8) -#define SHA384_SHORT_BLOCK_LENGTH (SHA384_BLOCK_LENGTH - 16) -#define SHA512_SHORT_BLOCK_LENGTH (SHA512_BLOCK_LENGTH - 16) - - -/*** ENDIAN REVERSAL MACROS *******************************************/ -#if BYTE_ORDER == LITTLE_ENDIAN -#define REVERSE32(w,x) { \ - sha2_word32 tmp = (w); \ - tmp = (tmp >> 16) | (tmp << 16); \ - (x) = ((tmp & 0xff00ff00UL) >> 8) | ((tmp & 0x00ff00ffUL) << 8); \ -} -#define REVERSE64(w,x) { \ - sha2_word64 tmp = (w); \ - tmp = (tmp >> 32) | (tmp << 32); \ - tmp = ((tmp & 0xff00ff00ff00ff00ULL) >> 8) | \ - ((tmp & 0x00ff00ff00ff00ffULL) << 8); \ - (x) = ((tmp & 0xffff0000ffff0000ULL) >> 16) | \ - ((tmp & 0x0000ffff0000ffffULL) << 16); \ -} -#endif /* BYTE_ORDER == LITTLE_ENDIAN */ - -/* - * Macro for incrementally adding the unsigned 64-bit integer n to the - * unsigned 128-bit integer (represented using a two-element array of - * 64-bit words): - */ -#define ADDINC128(w,n) { \ - (w)[0] += (sha2_word64)(n); \ - if ((w)[0] < (n)) { \ - (w)[1]++; \ - } \ -} - -/*** THE SIX LOGICAL FUNCTIONS ****************************************/ -/* - * Bit shifting and rotation (used by the six SHA-XYZ logical functions: - * - * NOTE: The naming of R and S appears backwards here (R is a SHIFT and - * S is a ROTATION) because the SHA-256/384/512 description document - * (see http://csrc.nist.gov/cryptval/shs/sha256-384-512.pdf) uses this - * same "backwards" definition. - */ -/* Shift-right (used in SHA-256, SHA-384, and SHA-512): */ -#define R(b,x) ((x) >> (b)) -/* 64-bit Rotate-right (used in SHA-384 and SHA-512): */ -#define S64(b,x) (((x) >> (b)) | ((x) << (64 - (b)))) - -/* Two of six logical functions used in SHA-256, SHA-384, and SHA-512: */ -#define Ch(x,y,z) (((x) & (y)) ^ ((~(x)) & (z))) -#define Maj(x,y,z) (((x) & (y)) ^ ((x) & (z)) ^ ((y) & (z))) - -/* Four of six logical functions used in SHA-384 and SHA-512: */ -#define Sigma0_512(x) (S64(28, (x)) ^ S64(34, (x)) ^ S64(39, (x))) -#define Sigma1_512(x) (S64(14, (x)) ^ S64(18, (x)) ^ S64(41, (x))) -#define sigma0_512(x) (S64( 1, (x)) ^ S64( 8, (x)) ^ R( 7, (x))) -#define sigma1_512(x) (S64(19, (x)) ^ S64(61, (x)) ^ R( 6, (x))) - -/*** INTERNAL FUNCTION PROTOTYPES *************************************/ -/* NOTE: These should not be accessed directly from outside this - * library -- they are intended for private internal visibility/use - * only. - */ -static void SHA512_Last(SHA512_CTX*); -static void SHA512_Transform(SHA512_CTX*, const sha2_word64*); - - -/*** SHA-XYZ INITIAL HASH VALUES AND CONSTANTS ************************/ -/* Hash constant words K for SHA-384 and SHA-512: */ -static const sha2_word64 K512[80] = { - 0x428a2f98d728ae22ULL, 0x7137449123ef65cdULL, - 0xb5c0fbcfec4d3b2fULL, 0xe9b5dba58189dbbcULL, - 0x3956c25bf348b538ULL, 0x59f111f1b605d019ULL, - 0x923f82a4af194f9bULL, 0xab1c5ed5da6d8118ULL, - 0xd807aa98a3030242ULL, 0x12835b0145706fbeULL, - 0x243185be4ee4b28cULL, 0x550c7dc3d5ffb4e2ULL, - 0x72be5d74f27b896fULL, 0x80deb1fe3b1696b1ULL, - 0x9bdc06a725c71235ULL, 0xc19bf174cf692694ULL, - 0xe49b69c19ef14ad2ULL, 0xefbe4786384f25e3ULL, - 0x0fc19dc68b8cd5b5ULL, 0x240ca1cc77ac9c65ULL, - 0x2de92c6f592b0275ULL, 0x4a7484aa6ea6e483ULL, - 0x5cb0a9dcbd41fbd4ULL, 0x76f988da831153b5ULL, - 0x983e5152ee66dfabULL, 0xa831c66d2db43210ULL, - 0xb00327c898fb213fULL, 0xbf597fc7beef0ee4ULL, - 0xc6e00bf33da88fc2ULL, 0xd5a79147930aa725ULL, - 0x06ca6351e003826fULL, 0x142929670a0e6e70ULL, - 0x27b70a8546d22ffcULL, 0x2e1b21385c26c926ULL, - 0x4d2c6dfc5ac42aedULL, 0x53380d139d95b3dfULL, - 0x650a73548baf63deULL, 0x766a0abb3c77b2a8ULL, - 0x81c2c92e47edaee6ULL, 0x92722c851482353bULL, - 0xa2bfe8a14cf10364ULL, 0xa81a664bbc423001ULL, - 0xc24b8b70d0f89791ULL, 0xc76c51a30654be30ULL, - 0xd192e819d6ef5218ULL, 0xd69906245565a910ULL, - 0xf40e35855771202aULL, 0x106aa07032bbd1b8ULL, - 0x19a4c116b8d2d0c8ULL, 0x1e376c085141ab53ULL, - 0x2748774cdf8eeb99ULL, 0x34b0bcb5e19b48a8ULL, - 0x391c0cb3c5c95a63ULL, 0x4ed8aa4ae3418acbULL, - 0x5b9cca4f7763e373ULL, 0x682e6ff3d6b2b8a3ULL, - 0x748f82ee5defb2fcULL, 0x78a5636f43172f60ULL, - 0x84c87814a1f0ab72ULL, 0x8cc702081a6439ecULL, - 0x90befffa23631e28ULL, 0xa4506cebde82bde9ULL, - 0xbef9a3f7b2c67915ULL, 0xc67178f2e372532bULL, - 0xca273eceea26619cULL, 0xd186b8c721c0c207ULL, - 0xeada7dd6cde0eb1eULL, 0xf57d4f7fee6ed178ULL, - 0x06f067aa72176fbaULL, 0x0a637dc5a2c898a6ULL, - 0x113f9804bef90daeULL, 0x1b710b35131c471bULL, - 0x28db77f523047d84ULL, 0x32caab7b40c72493ULL, - 0x3c9ebe0a15c9bebcULL, 0x431d67c49c100d4cULL, - 0x4cc5d4becb3e42b6ULL, 0x597f299cfc657e2aULL, - 0x5fcb6fab3ad6faecULL, 0x6c44198c4a475817ULL -}; - -/* Initial hash value H for SHA-384 */ -static const sha2_word64 sha384_initial_hash_value[8] = { - 0xcbbb9d5dc1059ed8ULL, - 0x629a292a367cd507ULL, - 0x9159015a3070dd17ULL, - 0x152fecd8f70e5939ULL, - 0x67332667ffc00b31ULL, - 0x8eb44a8768581511ULL, - 0xdb0c2e0d64f98fa7ULL, - 0x47b5481dbefa4fa4ULL -}; - -/* Initial hash value H for SHA-512 */ -static const sha2_word64 sha512_initial_hash_value[8] = { - 0x6a09e667f3bcc908ULL, - 0xbb67ae8584caa73bULL, - 0x3c6ef372fe94f82bULL, - 0xa54ff53a5f1d36f1ULL, - 0x510e527fade682d1ULL, - 0x9b05688c2b3e6c1fULL, - 0x1f83d9abfb41bd6bULL, - 0x5be0cd19137e2179ULL -}; - -/* - * Constant used by SHA256/384/512_End() functions for converting the - * digest to a readable hexadecimal character string: - */ -static const char *sha2_hex_digits = "0123456789abcdef"; - - -/*** SHA-256: *********************************************************/ -char *SHA256_End(SHA256_CTX* context, char buffer[]) { - sha2_byte digest[SHA256_DIGEST_LENGTH], *d = digest; - int i; - - /* Sanity check: */ - assert(context != (SHA256_CTX*)0); - - if (buffer != (char*)0) { - SHA256_Final(digest, context); - - for (i = 0; i < SHA256_DIGEST_LENGTH; i++) { - *buffer++ = sha2_hex_digits[(*d & 0xf0) >> 4]; - *buffer++ = sha2_hex_digits[*d & 0x0f]; - d++; - } - *buffer = (char)0; - } else { - bzero(context, sizeof(*context)); - } - bzero(digest, SHA256_DIGEST_LENGTH); - return buffer; -} - -char* SHA256_Data(const void *data, unsigned int len, char *digest) { - SHA256_CTX context; - - SHA256_Init(&context); - SHA256_Update(&context, data, len); - return SHA256_End(&context, digest); -} - - -/*** SHA-512: *********************************************************/ -void SHA512_Init(SHA512_CTX* context) { - if (context == (SHA512_CTX*)0) { - return; - } - bcopy(sha512_initial_hash_value, context->state, SHA512_DIGEST_LENGTH); - bzero(context->buffer, SHA512_BLOCK_LENGTH); - context->bitcount[0] = context->bitcount[1] = 0; -} - -#ifdef SHA2_UNROLL_TRANSFORM - -/* Unrolled SHA-512 round macros: */ -#if BYTE_ORDER == LITTLE_ENDIAN - -#define ROUND512_0_TO_15(a,b,c,d,e,f,g,h) \ - REVERSE64(*data++, W512[j]); \ - T1 = (h) + Sigma1_512(e) + Ch((e), (f), (g)) + \ - K512[j] + W512[j]; \ - (d) += T1, \ - (h) = T1 + Sigma0_512(a) + Maj((a), (b), (c)), \ - j++ - - -#else /* BYTE_ORDER == LITTLE_ENDIAN */ - -#define ROUND512_0_TO_15(a,b,c,d,e,f,g,h) \ - T1 = (h) + Sigma1_512(e) + Ch((e), (f), (g)) + \ - K512[j] + (W512[j] = *data++); \ - (d) += T1; \ - (h) = T1 + Sigma0_512(a) + Maj((a), (b), (c)); \ - j++ - -#endif /* BYTE_ORDER == LITTLE_ENDIAN */ - -#define ROUND512(a,b,c,d,e,f,g,h) \ - s0 = W512[(j+1)&0x0f]; \ - s0 = sigma0_512(s0); \ - s1 = W512[(j+14)&0x0f]; \ - s1 = sigma1_512(s1); \ - T1 = (h) + Sigma1_512(e) + Ch((e), (f), (g)) + K512[j] + \ - (W512[j&0x0f] += s1 + W512[(j+9)&0x0f] + s0); \ - (d) += T1; \ - (h) = T1 + Sigma0_512(a) + Maj((a), (b), (c)); \ - j++ - -static void SHA512_Transform(SHA512_CTX* context, const sha2_word64* data) { - sha2_word64 a, b, c, d, e, f, g, h, s0, s1; - sha2_word64 T1, *W512 = (sha2_word64*)context->buffer; - int j; - - /* Initialize registers with the prev. intermediate value */ - a = context->state[0]; - b = context->state[1]; - c = context->state[2]; - d = context->state[3]; - e = context->state[4]; - f = context->state[5]; - g = context->state[6]; - h = context->state[7]; - - j = 0; - do { - ROUND512_0_TO_15(a,b,c,d,e,f,g,h); - ROUND512_0_TO_15(h,a,b,c,d,e,f,g); - ROUND512_0_TO_15(g,h,a,b,c,d,e,f); - ROUND512_0_TO_15(f,g,h,a,b,c,d,e); - ROUND512_0_TO_15(e,f,g,h,a,b,c,d); - ROUND512_0_TO_15(d,e,f,g,h,a,b,c); - ROUND512_0_TO_15(c,d,e,f,g,h,a,b); - ROUND512_0_TO_15(b,c,d,e,f,g,h,a); - } while (j < 16); - - /* Now for the remaining rounds up to 79: */ - do { - ROUND512(a,b,c,d,e,f,g,h); - ROUND512(h,a,b,c,d,e,f,g); - ROUND512(g,h,a,b,c,d,e,f); - ROUND512(f,g,h,a,b,c,d,e); - ROUND512(e,f,g,h,a,b,c,d); - ROUND512(d,e,f,g,h,a,b,c); - ROUND512(c,d,e,f,g,h,a,b); - ROUND512(b,c,d,e,f,g,h,a); - } while (j < 80); - - /* Compute the current intermediate hash value */ - context->state[0] += a; - context->state[1] += b; - context->state[2] += c; - context->state[3] += d; - context->state[4] += e; - context->state[5] += f; - context->state[6] += g; - context->state[7] += h; - - /* Clean up */ - a = b = c = d = e = f = g = h = T1 = 0; -} - -#else /* SHA2_UNROLL_TRANSFORM */ - -static void SHA512_Transform(SHA512_CTX* context, const sha2_word64* data) { - sha2_word64 a, b, c, d, e, f, g, h, s0, s1; - sha2_word64 T1 = 0, T2 = 0, *W512 = (sha2_word64*)context->buffer; - int j; - - /* Initialize registers with the prev. intermediate value */ - a = context->state[0]; - b = context->state[1]; - c = context->state[2]; - d = context->state[3]; - e = context->state[4]; - f = context->state[5]; - g = context->state[6]; - h = context->state[7]; - - j = 0; - do { -#if BYTE_ORDER == LITTLE_ENDIAN - /* Convert TO host byte order */ - REVERSE64(*data++, W512[j]); - /* Apply the SHA-512 compression function to update a..h */ - T1 = h + Sigma1_512(e) + Ch(e, f, g) + K512[j] + W512[j]; -#else /* BYTE_ORDER == LITTLE_ENDIAN */ - /* Apply the SHA-512 compression function to update a..h with copy */ - T1 = h + Sigma1_512(e) + Ch(e, f, g) + K512[j] + (W512[j] = *data++); -#endif /* BYTE_ORDER == LITTLE_ENDIAN */ - T2 = Sigma0_512(a) + Maj(a, b, c); - h = g; - g = f; - f = e; - e = d + T1; - d = c; - c = b; - b = a; - a = T1 + T2; - - j++; - } while (j < 16); - - do { - /* Part of the message block expansion: */ - s0 = W512[(j+1)&0x0f]; - s0 = sigma0_512(s0); - s1 = W512[(j+14)&0x0f]; - s1 = sigma1_512(s1); - - /* Apply the SHA-512 compression function to update a..h */ - T1 = h + Sigma1_512(e) + Ch(e, f, g) + K512[j] + - (W512[j&0x0f] += s1 + W512[(j+9)&0x0f] + s0); - T2 = Sigma0_512(a) + Maj(a, b, c); - h = g; - g = f; - f = e; - e = d + T1; - d = c; - c = b; - b = a; - a = T1 + T2; - - j++; - } while (j < 80); - - /* Compute the current intermediate hash value */ - context->state[0] += a; - context->state[1] += b; - context->state[2] += c; - context->state[3] += d; - context->state[4] += e; - context->state[5] += f; - context->state[6] += g; - context->state[7] += h; - - /* Clean up */ - a = b = c = d = e = f = g = h = T1 = T2 = 0; -} - -#endif /* SHA2_UNROLL_TRANSFORM */ - -void SHA512_Update(SHA512_CTX* context, const sha2_byte *data, size_t len) { - unsigned int freespace, usedspace; - - if (len == 0) { - /* Calling with no data is valid - we do nothing */ - return; - } - - /* Sanity check: */ - assert(context != (SHA512_CTX*)0 && data != (sha2_byte*)0); - - usedspace = (context->bitcount[0] >> 3) % SHA512_BLOCK_LENGTH; - if (usedspace > 0) { - /* Calculate how much free space is available in the buffer */ - freespace = SHA512_BLOCK_LENGTH - usedspace; - - if (len >= freespace) { - /* Fill the buffer completely and process it */ - bcopy(data, &context->buffer[usedspace], freespace); - ADDINC128(context->bitcount, freespace << 3); - len -= freespace; - data += freespace; - SHA512_Transform(context, (sha2_word64*)context->buffer); - } else { - /* The buffer is not yet full */ - bcopy(data, &context->buffer[usedspace], len); - ADDINC128(context->bitcount, len << 3); - /* Clean up: */ - usedspace = freespace = 0; - return; - } - } - while (len >= SHA512_BLOCK_LENGTH) { - /* Process as many complete blocks as we can */ - SHA512_Transform(context, (const sha2_word64*)data); - ADDINC128(context->bitcount, SHA512_BLOCK_LENGTH << 3); - len -= SHA512_BLOCK_LENGTH; - data += SHA512_BLOCK_LENGTH; - } - if (len > 0) { - /* There's left-overs, so save 'em */ - bcopy(data, context->buffer, len); - ADDINC128(context->bitcount, len << 3); - } - /* Clean up: */ - usedspace = freespace = 0; -} - -static void SHA512_Last(SHA512_CTX* context) { - unsigned int usedspace; - - usedspace = (context->bitcount[0] >> 3) % SHA512_BLOCK_LENGTH; -#if BYTE_ORDER == LITTLE_ENDIAN - /* Convert FROM host byte order */ - REVERSE64(context->bitcount[0],context->bitcount[0]); - REVERSE64(context->bitcount[1],context->bitcount[1]); -#endif - if (usedspace > 0) { - /* Begin padding with a 1 bit: */ - context->buffer[usedspace++] = 0x80; - - if (usedspace <= SHA512_SHORT_BLOCK_LENGTH) { - /* Set-up for the last transform: */ - bzero(&context->buffer[usedspace], SHA512_SHORT_BLOCK_LENGTH - usedspace); - } else { - if (usedspace < SHA512_BLOCK_LENGTH) { - bzero(&context->buffer[usedspace], SHA512_BLOCK_LENGTH - usedspace); - } - /* Do second-to-last transform: */ - SHA512_Transform(context, (sha2_word64*)context->buffer); - - /* And set-up for the last transform: */ - bzero(context->buffer, SHA512_BLOCK_LENGTH - 2); - } - } else { - /* Prepare for final transform: */ - bzero(context->buffer, SHA512_SHORT_BLOCK_LENGTH); - - /* Begin padding with a 1 bit: */ - *context->buffer = 0x80; - } - /* Store the length of input data (in bits): */ - *(sha2_word64*)&context->buffer[SHA512_SHORT_BLOCK_LENGTH] = context->bitcount[1]; - *(sha2_word64*)&context->buffer[SHA512_SHORT_BLOCK_LENGTH+8] = context->bitcount[0]; - - /* Final transform: */ - SHA512_Transform(context, (sha2_word64*)context->buffer); -} - -void SHA512_Final(sha2_byte digest[], SHA512_CTX* context) { - sha2_word64 *d = (sha2_word64*)digest; - - /* Sanity check: */ - assert(context != (SHA512_CTX*)0); - - /* If no digest buffer is passed, we don't bother doing this: */ - if (digest != (sha2_byte*)0) { - SHA512_Last(context); - - /* Save the hash data for output: */ -#if BYTE_ORDER == LITTLE_ENDIAN - { - /* Convert TO host byte order */ - int j; - for (j = 0; j < 8; j++) { - REVERSE64(context->state[j],context->state[j]); - *d++ = context->state[j]; - } - } -#else - bcopy(context->state, d, SHA512_DIGEST_LENGTH); -#endif - } - - /* Zero out state data */ - bzero(context, sizeof(*context)); -} - -char *SHA512_End(SHA512_CTX* context, char buffer[]) { - sha2_byte digest[SHA512_DIGEST_LENGTH], *d = digest; - int i; - - /* Sanity check: */ - assert(context != (SHA512_CTX*)0); - - if (buffer != (char*)0) { - SHA512_Final(digest, context); - - for (i = 0; i < SHA512_DIGEST_LENGTH; i++) { - *buffer++ = sha2_hex_digits[(*d & 0xf0) >> 4]; - *buffer++ = sha2_hex_digits[*d & 0x0f]; - d++; - } - *buffer = (char)0; - } else { - bzero(context, sizeof(*context)); - } - bzero(digest, SHA512_DIGEST_LENGTH); - return buffer; -} - -char* SHA512_Data(const sha2_byte* data, size_t len, char digest[SHA512_DIGEST_STRING_LENGTH]) { - SHA512_CTX context; - - SHA512_Init(&context); - SHA512_Update(&context, data, len); - return SHA512_End(&context, digest); -} - - -/*** SHA-384: *********************************************************/ -void SHA384_Init(SHA384_CTX* context) { - if (context == (SHA384_CTX*)0) { - return; - } - bcopy(sha384_initial_hash_value, context->state, SHA512_DIGEST_LENGTH); - bzero(context->buffer, SHA384_BLOCK_LENGTH); - context->bitcount[0] = context->bitcount[1] = 0; -} - -void SHA384_Update(SHA384_CTX* context, const sha2_byte* data, size_t len) { - SHA512_Update((SHA512_CTX*)context, data, len); -} - -void SHA384_Final(sha2_byte digest[], SHA384_CTX* context) { - sha2_word64 *d = (sha2_word64*)digest; - - /* Sanity check: */ - assert(context != (SHA384_CTX*)0); - - /* If no digest buffer is passed, we don't bother doing this: */ - if (digest != (sha2_byte*)0) { - SHA512_Last((SHA512_CTX*)context); - - /* Save the hash data for output: */ -#if BYTE_ORDER == LITTLE_ENDIAN - { - /* Convert TO host byte order */ - int j; - for (j = 0; j < 6; j++) { - REVERSE64(context->state[j],context->state[j]); - *d++ = context->state[j]; - } - } -#else - bcopy(context->state, d, SHA384_DIGEST_LENGTH); -#endif - } - - /* Zero out state data */ - bzero(context, sizeof(*context)); -} - -char *SHA384_End(SHA384_CTX* context, char buffer[]) { - sha2_byte digest[SHA384_DIGEST_LENGTH], *d = digest; - int i; - - /* Sanity check: */ - assert(context != (SHA384_CTX*)0); - - if (buffer != (char*)0) { - SHA384_Final(digest, context); - - for (i = 0; i < SHA384_DIGEST_LENGTH; i++) { - *buffer++ = sha2_hex_digits[(*d & 0xf0) >> 4]; - *buffer++ = sha2_hex_digits[*d & 0x0f]; - d++; - } - *buffer = (char)0; - } else { - bzero(context, sizeof(*context)); - } - bzero(digest, SHA384_DIGEST_LENGTH); - return buffer; -} - -char* SHA384_Data(const sha2_byte* data, size_t len, char digest[SHA384_DIGEST_STRING_LENGTH]) { - SHA384_CTX context; - - SHA384_Init(&context); - SHA384_Update(&context, data, len); - return SHA384_End(&context, digest); -} diff --git a/sys/crypto/sha2/sha2.h b/sys/crypto/sha2/sha2.h deleted file mode 100644 index 59e91f6..0000000 --- a/sys/crypto/sha2/sha2.h +++ /dev/null @@ -1,89 +0,0 @@ -/* $FreeBSD$ */ -/* $KAME: sha2.h,v 1.5 2007/06/14 12:09:41 itojun Exp $ */ - -/* - * sha2.h - * - * Version 1.0.0beta1 - * - * Written by Aaron D. Gifford <me@aarongifford.com> - * - * Copyright 2000 Aaron D. Gifford. All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. Neither the name of the copyright holder nor the names of contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) AND CONTRIBUTOR(S) ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR(S) OR CONTRIBUTOR(S) BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - */ - -#ifndef __SHA2_H__ -#define __SHA2_H__ - -#ifdef __cplusplus -extern "C" { -#endif - - -/*** SHA-256/384/512 Various Length Definitions ***********************/ -#define SHA256_BLOCK_LENGTH 64 -#define SHA256_DIGEST_LENGTH 32 -#define SHA256_DIGEST_STRING_LENGTH (SHA256_DIGEST_LENGTH * 2 + 1) -#define SHA384_BLOCK_LENGTH 128 -#define SHA384_DIGEST_LENGTH 48 -#define SHA384_DIGEST_STRING_LENGTH (SHA384_DIGEST_LENGTH * 2 + 1) -#define SHA512_BLOCK_LENGTH 128 -#define SHA512_DIGEST_LENGTH 64 -#define SHA512_DIGEST_STRING_LENGTH (SHA512_DIGEST_LENGTH * 2 + 1) - - -/*** SHA-384/512 Context Structures *******************************/ -typedef struct _SHA512_CTX { - uint64_t state[8]; - uint64_t bitcount[2]; - uint8_t buffer[SHA512_BLOCK_LENGTH]; -} SHA512_CTX; - -typedef SHA512_CTX SHA384_CTX; - - -/*** SHA-384/512 Function Prototypes ******************************/ - -void SHA384_Init(SHA384_CTX*); -void SHA384_Update(SHA384_CTX*, const u_int8_t*, size_t); -void SHA384_Final(u_int8_t[SHA384_DIGEST_LENGTH], SHA384_CTX*); -char* SHA384_End(SHA384_CTX*, char[SHA384_DIGEST_STRING_LENGTH]); -char* SHA384_Data(const u_int8_t*, size_t, char[SHA384_DIGEST_STRING_LENGTH]); - -void SHA512_Init(SHA512_CTX*); -void SHA512_Update(SHA512_CTX*, const u_int8_t*, size_t); -void SHA512_Final(u_int8_t[SHA512_DIGEST_LENGTH], SHA512_CTX*); -char* SHA512_End(SHA512_CTX*, char[SHA512_DIGEST_STRING_LENGTH]); -char* SHA512_Data(const u_int8_t*, size_t, char[SHA512_DIGEST_STRING_LENGTH]); - -#ifdef __cplusplus -} -#endif /* __cplusplus */ - -#include "sha256.h" - -#endif /* __SHA2_H__ */ diff --git a/sys/crypto/sha2/sha256.h b/sys/crypto/sha2/sha256.h index c828379..528af1e 100644 --- a/sys/crypto/sha2/sha256.h +++ b/sys/crypto/sha2/sha256.h @@ -33,10 +33,14 @@ #include <sys/types.h> #endif +#define SHA256_BLOCK_LENGTH 64 +#define SHA256_DIGEST_LENGTH 32 +#define SHA256_DIGEST_STRING_LENGTH (SHA256_DIGEST_LENGTH * 2 + 1) + typedef struct SHA256Context { uint32_t state[8]; uint64_t count; - uint8_t buf[64]; + uint8_t buf[SHA256_BLOCK_LENGTH]; } SHA256_CTX; __BEGIN_DECLS @@ -74,10 +78,10 @@ __BEGIN_DECLS void SHA256_Init(SHA256_CTX *); void SHA256_Update(SHA256_CTX *, const void *, size_t); -void SHA256_Final(unsigned char [32], SHA256_CTX *); +void SHA256_Final(unsigned char [SHA256_DIGEST_LENGTH], SHA256_CTX *); +#ifndef _KERNEL char *SHA256_End(SHA256_CTX *, char *); char *SHA256_Data(const void *, unsigned int, char *); -#ifndef _KERNEL char *SHA256_File(const char *, char *); char *SHA256_FileChunk(const char *, char *, off_t, off_t); #endif diff --git a/sys/crypto/sha2/sha384.h b/sys/crypto/sha2/sha384.h new file mode 100644 index 0000000..ae63ba9 --- /dev/null +++ b/sys/crypto/sha2/sha384.h @@ -0,0 +1,87 @@ +/*- + * Copyright 2005 Colin Percival + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _SHA384_H_ +#define _SHA384_H_ + +#ifndef _KERNEL +#include <sys/types.h> +#endif + +#define SHA384_BLOCK_LENGTH 128 +#define SHA384_DIGEST_LENGTH 48 +#define SHA384_DIGEST_STRING_LENGTH (SHA384_DIGEST_LENGTH * 2 + 1) + +typedef struct SHA384Context { + uint64_t state[8]; + uint64_t count[2]; + uint8_t buf[SHA384_BLOCK_LENGTH]; +} SHA384_CTX; + +__BEGIN_DECLS + +/* Ensure libmd symbols do not clash with libcrypto */ +#ifndef SHA384_Init +#define SHA384_Init _libmd_SHA384_Init +#endif +#ifndef SHA384_Update +#define SHA384_Update _libmd_SHA384_Update +#endif +#ifndef SHA384_Final +#define SHA384_Final _libmd_SHA384_Final +#endif +#ifndef SHA384_End +#define SHA384_End _libmd_SHA384_End +#endif +#ifndef SHA384_File +#define SHA384_File _libmd_SHA384_File +#endif +#ifndef SHA384_FileChunk +#define SHA384_FileChunk _libmd_SHA384_FileChunk +#endif +#ifndef SHA384_Data +#define SHA384_Data _libmd_SHA384_Data +#endif + +#ifndef SHA384_version +#define SHA384_version _libmd_SHA384_version +#endif + +void SHA384_Init(SHA384_CTX *); +void SHA384_Update(SHA384_CTX *, const void *, size_t); +void SHA384_Final(unsigned char [SHA384_DIGEST_LENGTH], SHA384_CTX *); +#ifndef _KERNEL +char *SHA384_End(SHA384_CTX *, char *); +char *SHA384_Data(const void *, unsigned int, char *); +char *SHA384_File(const char *, char *); +char *SHA384_FileChunk(const char *, char *, off_t, off_t); +#endif + +__END_DECLS + +#endif /* !_SHA384_H_ */ diff --git a/lib/libmd/sha512.h b/sys/crypto/sha2/sha512.h index b034ca5..da0a018 100644 --- a/lib/libmd/sha512.h +++ b/sys/crypto/sha2/sha512.h @@ -29,18 +29,23 @@ #ifndef _SHA512_H_ #define _SHA512_H_ +#ifndef _KERNEL #include <sys/types.h> +#endif + +#define SHA512_BLOCK_LENGTH 128 +#define SHA512_DIGEST_LENGTH 64 +#define SHA512_DIGEST_STRING_LENGTH (SHA512_DIGEST_LENGTH * 2 + 1) typedef struct SHA512Context { uint64_t state[8]; uint64_t count[2]; - unsigned char buf[128]; + uint8_t buf[SHA512_BLOCK_LENGTH]; } SHA512_CTX; __BEGIN_DECLS /* Ensure libmd symbols do not clash with libcrypto */ - #ifndef SHA512_Init #define SHA512_Init _libmd_SHA512_Init #endif @@ -72,11 +77,14 @@ __BEGIN_DECLS void SHA512_Init(SHA512_CTX *); void SHA512_Update(SHA512_CTX *, const void *, size_t); -void SHA512_Final(unsigned char [64], SHA512_CTX *); +void SHA512_Final(unsigned char [SHA512_DIGEST_LENGTH], SHA512_CTX *); +#ifndef _KERNEL char *SHA512_End(SHA512_CTX *, char *); +char *SHA512_Data(const void *, unsigned int, char *); char *SHA512_File(const char *, char *); char *SHA512_FileChunk(const char *, char *, off_t, off_t); -char *SHA512_Data(const void *, unsigned int, char *); +#endif + __END_DECLS #endif /* !_SHA512_H_ */ diff --git a/lib/libmd/sha512c.c b/sys/crypto/sha2/sha512c.c index e5e52a8..42ad058 100644 --- a/lib/libmd/sha512c.c +++ b/sys/crypto/sha2/sha512c.c @@ -30,9 +30,14 @@ __FBSDID("$FreeBSD$"); #include <sys/endian.h> #include <sys/types.h> +#ifdef _KERNEL +#include <sys/systm.h> +#else #include <string.h> +#endif #include "sha512.h" +#include "sha384.h" #if BYTE_ORDER == BIG_ENDIAN @@ -104,7 +109,7 @@ be64dec_vect(uint64_t *dst, const unsigned char *src, size_t len) * the 512-bit input block to produce a new state. */ static void -SHA512_Transform(uint64_t * state, const unsigned char block[128]) +SHA512_Transform(uint64_t * state, const unsigned char block[SHA512_BLOCK_LENGTH]) { uint64_t W[80]; uint64_t S[8]; @@ -112,12 +117,12 @@ SHA512_Transform(uint64_t * state, const unsigned char block[128]) int i; /* 1. Prepare message schedule W. */ - be64dec_vect(W, block, 128); + be64dec_vect(W, block, SHA512_BLOCK_LENGTH); for (i = 16; i < 80; i++) W[i] = s1(W[i - 2]) + W[i - 7] + s0(W[i - 15]) + W[i - 16]; /* 2. Initialize working variables. */ - memcpy(S, state, 64); + memcpy(S, state, SHA512_DIGEST_LENGTH); /* 3. Mix. */ RNDr(S, W, 0, 0x428a2f98d728ae22ULL); @@ -206,7 +211,7 @@ SHA512_Transform(uint64_t * state, const unsigned char block[128]) state[i] += S[i]; } -static unsigned char PAD[128] = { +static unsigned char PAD[SHA512_BLOCK_LENGTH] = { 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, @@ -279,22 +284,22 @@ SHA512_Update(SHA512_CTX * ctx, const void *in, size_t len) ctx->count[0] += bitlen[0]; /* Handle the case where we don't need to perform any transforms */ - if (len < 128 - r) { + if (len < SHA512_BLOCK_LENGTH - r) { memcpy(&ctx->buf[r], src, len); return; } /* Finish the current block */ - memcpy(&ctx->buf[r], src, 128 - r); + memcpy(&ctx->buf[r], src, SHA512_BLOCK_LENGTH - r); SHA512_Transform(ctx->state, ctx->buf); - src += 128 - r; - len -= 128 - r; + src += SHA512_BLOCK_LENGTH - r; + len -= SHA512_BLOCK_LENGTH - r; /* Perform complete blocks */ - while (len >= 128) { + while (len >= SHA512_BLOCK_LENGTH) { SHA512_Transform(ctx->state, src); - src += 128; - len -= 128; + src += SHA512_BLOCK_LENGTH; + len -= SHA512_BLOCK_LENGTH; } /* Copy left over data into buffer */ @@ -306,14 +311,64 @@ SHA512_Update(SHA512_CTX * ctx, const void *in, size_t len) * and clears the context state. */ void -SHA512_Final(unsigned char digest[64], SHA512_CTX * ctx) +SHA512_Final(unsigned char digest[SHA512_DIGEST_LENGTH], SHA512_CTX * ctx) { /* Add padding */ SHA512_Pad(ctx); /* Write the hash */ - be64enc_vect(digest, ctx->state, 64); + be64enc_vect(digest, ctx->state, SHA512_DIGEST_LENGTH); + + /* Clear the context state */ + memset((void *)ctx, 0, sizeof(*ctx)); +} + +/*** SHA-384: *********************************************************/ +/* + * the SHA384 and SHA512 transforms are identical, so SHA384 is skipped + */ + +/* SHA-384 initialization. Begins a SHA-384 operation. */ +void +SHA384_Init(SHA384_CTX * ctx) +{ + + /* Zero bits processed so far */ + ctx->count[0] = ctx->count[1] = 0; + + /* Magic initialization constants */ + ctx->state[0] = 0xcbbb9d5dc1059ed8ULL; + ctx->state[1] = 0x629a292a367cd507ULL; + ctx->state[2] = 0x9159015a3070dd17ULL; + ctx->state[3] = 0x152fecd8f70e5939ULL; + ctx->state[4] = 0x67332667ffc00b31ULL; + ctx->state[5] = 0x8eb44a8768581511ULL; + ctx->state[6] = 0xdb0c2e0d64f98fa7ULL; + ctx->state[7] = 0x47b5481dbefa4fa4ULL; +} + +/* Add bytes into the SHA-384 hash */ +void +SHA384_Update(SHA384_CTX * ctx, const void *in, size_t len) +{ + + SHA512_Update((SHA512_CTX *)ctx, in, len); +} + +/* + * SHA-384 finalization. Pads the input data, exports the hash value, + * and clears the context state. + */ +void +SHA384_Final(unsigned char digest[SHA384_DIGEST_LENGTH], SHA384_CTX * ctx) +{ + + /* Add padding */ + SHA512_Pad((SHA512_CTX *)ctx); + + /* Write the hash */ + be64enc_vect(digest, ctx->state, SHA384_DIGEST_LENGTH); /* Clear the context state */ memset((void *)ctx, 0, sizeof(*ctx)); @@ -332,4 +387,11 @@ __weak_reference(_libmd_SHA512_Update, SHA512_Update); __weak_reference(_libmd_SHA512_Final, SHA512_Final); #undef SHA512_Transform __weak_reference(_libmd_SHA512_Transform, SHA512_Transform); + +#undef SHA384_Init +__weak_reference(_libmd_SHA384_Init, SHA384_Init); +#undef SHA384_Update +__weak_reference(_libmd_SHA384_Update, SHA384_Update); +#undef SHA384_Final +__weak_reference(_libmd_SHA384_Final, SHA384_Final); #endif diff --git a/sys/dev/bxe/bxe.c b/sys/dev/bxe/bxe.c index dd994be..ddaa88a 100644 --- a/sys/dev/bxe/bxe.c +++ b/sys/dev/bxe/bxe.c @@ -472,6 +472,10 @@ static const struct { 4, STATS_FLAGS_FUNC, "rx_pkts"}, { STATS_OFFSET32(rx_tpa_pkts), 4, STATS_FLAGS_FUNC, "rx_tpa_pkts"}, + { STATS_OFFSET32(rx_erroneous_jumbo_sge_pkts), + 4, STATS_FLAGS_FUNC, "rx_erroneous_jumbo_sge_pkts"}, + { STATS_OFFSET32(rx_bxe_service_rxsgl), + 4, STATS_FLAGS_FUNC, "rx_bxe_service_rxsgl"}, { STATS_OFFSET32(rx_jumbo_sge_pkts), 4, STATS_FLAGS_FUNC, "rx_jumbo_sge_pkts"}, { STATS_OFFSET32(rx_soft_errors), @@ -585,6 +589,10 @@ static const struct { 4, "rx_pkts"}, { Q_STATS_OFFSET32(rx_tpa_pkts), 4, "rx_tpa_pkts"}, + { Q_STATS_OFFSET32(rx_erroneous_jumbo_sge_pkts), + 4, "rx_erroneous_jumbo_sge_pkts"}, + { Q_STATS_OFFSET32(rx_bxe_service_rxsgl), + 4, "rx_bxe_service_rxsgl"}, { Q_STATS_OFFSET32(rx_jumbo_sge_pkts), 4, "rx_jumbo_sge_pkts"}, { Q_STATS_OFFSET32(rx_soft_errors), @@ -728,6 +736,8 @@ static __noinline int bxe_nic_unload(struct bxe_softc *sc, static void bxe_handle_sp_tq(void *context, int pending); static void bxe_handle_fp_tq(void *context, int pending); +static int bxe_add_cdev(struct bxe_softc *sc); +static void bxe_del_cdev(struct bxe_softc *sc); /* calculate crc32 on a buffer (NOTE: crc32_length MUST be aligned to 8) */ uint32_t @@ -3475,11 +3485,14 @@ bxe_rxeof(struct bxe_softc *sc, m_adj(m, pad); m->m_pkthdr.len = m->m_len = len; - if (len != lenonbd){ + if ((len > 60) && (len > lenonbd)) { + fp->eth_q_stats.rx_bxe_service_rxsgl++; rc = bxe_service_rxsgl(fp, len, lenonbd, m, cqe_fp); if (rc) break; fp->eth_q_stats.rx_jumbo_sge_pkts++; + } else if (lenonbd < len) { + fp->eth_q_stats.rx_erroneous_jumbo_sge_pkts++; } /* assign packet to this interface interface */ @@ -4492,7 +4505,7 @@ bxe_nic_unload(struct bxe_softc *sc, sc->rx_mode = BXE_RX_MODE_NONE; /* XXX set rx mode ??? */ - if (IS_PF(sc)) { + if (IS_PF(sc) && !sc->grcdump_done) { /* set ALWAYS_ALIVE bit in shmem */ sc->fw_drv_pulse_wr_seq |= DRV_PULSE_ALWAYS_ALIVE; @@ -4512,7 +4525,8 @@ bxe_nic_unload(struct bxe_softc *sc, ; /* bxe_vfpf_close_vf(sc); */ } else if (unload_mode != UNLOAD_RECOVERY) { /* if this is a normal/close unload need to clean up chip */ - bxe_chip_cleanup(sc, unload_mode, keep_link); + if (!sc->grcdump_done) + bxe_chip_cleanup(sc, unload_mode, keep_link); } else { /* Send the UNLOAD_REQUEST to the MCP */ bxe_send_unload_req(sc, unload_mode); @@ -16122,12 +16136,12 @@ bxe_sysctl_state(SYSCTL_HANDLER_ARGS) } if (result == 1) { - uint32_t temp; + uint32_t temp; sc = (struct bxe_softc *)arg1; BLOGI(sc, "... dumping driver state ...\n"); - temp = SHMEM2_RD(sc, temperature_in_half_celsius); - BLOGI(sc, "\t Device Temperature = %d Celsius\n", (temp/2)); + temp = SHMEM2_RD(sc, temperature_in_half_celsius); + BLOGI(sc, "\t Device Temperature = %d Celsius\n", (temp/2)); } return (error); @@ -16265,6 +16279,12 @@ bxe_add_sysctls(struct bxe_softc *sc) CTLFLAG_RW, &sc->debug, "debug logging mode"); + sc->trigger_grcdump = 0; + SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "trigger_grcdump", + CTLFLAG_RW, &sc->trigger_grcdump, 0, + "set by driver when a grcdump is needed"); + + sc->rx_budget = bxe_rx_budget; SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "rx_budget", CTLFLAG_RW, &sc->rx_budget, 0, @@ -16393,8 +16413,20 @@ bxe_attach(device_t dev) return (ENXIO); } + if (bxe_add_cdev(sc) != 0) { + if (sc->ifp != NULL) { + ether_ifdetach(sc->ifp); + } + ifmedia_removeall(&sc->ifmedia); + bxe_release_mutexes(sc); + bxe_deallocate_bars(sc); + pci_disable_busmaster(dev); + return (ENXIO); + } + /* allocate device interrupts */ if (bxe_interrupt_alloc(sc) != 0) { + bxe_del_cdev(sc); if (sc->ifp != NULL) { ether_ifdetach(sc->ifp); } @@ -16408,6 +16440,7 @@ bxe_attach(device_t dev) /* allocate ilt */ if (bxe_alloc_ilt_mem(sc) != 0) { bxe_interrupt_free(sc); + bxe_del_cdev(sc); if (sc->ifp != NULL) { ether_ifdetach(sc->ifp); } @@ -16422,6 +16455,7 @@ bxe_attach(device_t dev) if (bxe_alloc_hsi_mem(sc) != 0) { bxe_free_ilt_mem(sc); bxe_interrupt_free(sc); + bxe_del_cdev(sc); if (sc->ifp != NULL) { ether_ifdetach(sc->ifp); } @@ -16493,6 +16527,8 @@ bxe_detach(device_t dev) return(EBUSY); } + bxe_del_cdev(sc); + /* stop the periodic callout */ bxe_periodic_stop(sc); @@ -18813,3 +18849,457 @@ ecore_storm_memset_struct(struct bxe_softc *sc, } } + +/* + * character device - ioctl interface definitions + */ + + +#include "bxe_dump.h" +#include "bxe_ioctl.h" +#include <sys/conf.h> + +static int bxe_eioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag, + struct thread *td); + +static struct cdevsw bxe_cdevsw = { + .d_version = D_VERSION, + .d_ioctl = bxe_eioctl, + .d_name = "bxecnic", +}; + +#define BXE_PATH(sc) (CHIP_IS_E1x(sc) ? 0 : (sc->pcie_func & 1)) + + +#define DUMP_ALL_PRESETS 0x1FFF +#define DUMP_MAX_PRESETS 13 +#define IS_E1_REG(chips) ((chips & DUMP_CHIP_E1) == DUMP_CHIP_E1) +#define IS_E1H_REG(chips) ((chips & DUMP_CHIP_E1H) == DUMP_CHIP_E1H) +#define IS_E2_REG(chips) ((chips & DUMP_CHIP_E2) == DUMP_CHIP_E2) +#define IS_E3A0_REG(chips) ((chips & DUMP_CHIP_E3A0) == DUMP_CHIP_E3A0) +#define IS_E3B0_REG(chips) ((chips & DUMP_CHIP_E3B0) == DUMP_CHIP_E3B0) + +#define IS_REG_IN_PRESET(presets, idx) \ + ((presets & (1 << (idx-1))) == (1 << (idx-1))) + + +static int +bxe_get_preset_regs_len(struct bxe_softc *sc, uint32_t preset) +{ + if (CHIP_IS_E1(sc)) + return dump_num_registers[0][preset-1]; + else if (CHIP_IS_E1H(sc)) + return dump_num_registers[1][preset-1]; + else if (CHIP_IS_E2(sc)) + return dump_num_registers[2][preset-1]; + else if (CHIP_IS_E3A0(sc)) + return dump_num_registers[3][preset-1]; + else if (CHIP_IS_E3B0(sc)) + return dump_num_registers[4][preset-1]; + else + return 0; +} + +static int +bxe_get_max_regs_len(struct bxe_softc *sc) +{ + uint32_t preset_idx; + int regdump_len32, len32; + + regdump_len32 = bxe_get_preset_regs_len(sc, 1); + + /* Calculate the total preset regs length */ + for (preset_idx = 2; preset_idx <= DUMP_MAX_PRESETS; preset_idx++) { + + len32 = bxe_get_preset_regs_len(sc, preset_idx); + + if (regdump_len32 < len32) + regdump_len32 = len32; + } + + return regdump_len32; +} + +static int +bxe_get_total_regs_len32(struct bxe_softc *sc) +{ + uint32_t preset_idx; + int regdump_len32 = 0; + + + /* Calculate the total preset regs length */ + for (preset_idx = 1; preset_idx <= DUMP_MAX_PRESETS; preset_idx++) { + regdump_len32 += bxe_get_preset_regs_len(sc, preset_idx); + } + + return regdump_len32; +} + +static const uint32_t * +__bxe_get_page_addr_ar(struct bxe_softc *sc) +{ + if (CHIP_IS_E2(sc)) + return page_vals_e2; + else if (CHIP_IS_E3(sc)) + return page_vals_e3; + else + return NULL; +} + +static uint32_t +__bxe_get_page_reg_num(struct bxe_softc *sc) +{ + if (CHIP_IS_E2(sc)) + return PAGE_MODE_VALUES_E2; + else if (CHIP_IS_E3(sc)) + return PAGE_MODE_VALUES_E3; + else + return 0; +} + +static const uint32_t * +__bxe_get_page_write_ar(struct bxe_softc *sc) +{ + if (CHIP_IS_E2(sc)) + return page_write_regs_e2; + else if (CHIP_IS_E3(sc)) + return page_write_regs_e3; + else + return NULL; +} + +static uint32_t +__bxe_get_page_write_num(struct bxe_softc *sc) +{ + if (CHIP_IS_E2(sc)) + return PAGE_WRITE_REGS_E2; + else if (CHIP_IS_E3(sc)) + return PAGE_WRITE_REGS_E3; + else + return 0; +} + +static const struct reg_addr * +__bxe_get_page_read_ar(struct bxe_softc *sc) +{ + if (CHIP_IS_E2(sc)) + return page_read_regs_e2; + else if (CHIP_IS_E3(sc)) + return page_read_regs_e3; + else + return NULL; +} + +static uint32_t +__bxe_get_page_read_num(struct bxe_softc *sc) +{ + if (CHIP_IS_E2(sc)) + return PAGE_READ_REGS_E2; + else if (CHIP_IS_E3(sc)) + return PAGE_READ_REGS_E3; + else + return 0; +} + +static bool +bxe_is_reg_in_chip(struct bxe_softc *sc, const struct reg_addr *reg_info) +{ + if (CHIP_IS_E1(sc)) + return IS_E1_REG(reg_info->chips); + else if (CHIP_IS_E1H(sc)) + return IS_E1H_REG(reg_info->chips); + else if (CHIP_IS_E2(sc)) + return IS_E2_REG(reg_info->chips); + else if (CHIP_IS_E3A0(sc)) + return IS_E3A0_REG(reg_info->chips); + else if (CHIP_IS_E3B0(sc)) + return IS_E3B0_REG(reg_info->chips); + else + return 0; +} + +static bool +bxe_is_wreg_in_chip(struct bxe_softc *sc, const struct wreg_addr *wreg_info) +{ + if (CHIP_IS_E1(sc)) + return IS_E1_REG(wreg_info->chips); + else if (CHIP_IS_E1H(sc)) + return IS_E1H_REG(wreg_info->chips); + else if (CHIP_IS_E2(sc)) + return IS_E2_REG(wreg_info->chips); + else if (CHIP_IS_E3A0(sc)) + return IS_E3A0_REG(wreg_info->chips); + else if (CHIP_IS_E3B0(sc)) + return IS_E3B0_REG(wreg_info->chips); + else + return 0; +} + +/** + * bxe_read_pages_regs - read "paged" registers + * + * @bp device handle + * @p output buffer + * + * Reads "paged" memories: memories that may only be read by first writing to a + * specific address ("write address") and then reading from a specific address + * ("read address"). There may be more than one write address per "page" and + * more than one read address per write address. + */ +static void +bxe_read_pages_regs(struct bxe_softc *sc, uint32_t *p, uint32_t preset) +{ + uint32_t i, j, k, n; + + /* addresses of the paged registers */ + const uint32_t *page_addr = __bxe_get_page_addr_ar(sc); + /* number of paged registers */ + int num_pages = __bxe_get_page_reg_num(sc); + /* write addresses */ + const uint32_t *write_addr = __bxe_get_page_write_ar(sc); + /* number of write addresses */ + int write_num = __bxe_get_page_write_num(sc); + /* read addresses info */ + const struct reg_addr *read_addr = __bxe_get_page_read_ar(sc); + /* number of read addresses */ + int read_num = __bxe_get_page_read_num(sc); + uint32_t addr, size; + + for (i = 0; i < num_pages; i++) { + for (j = 0; j < write_num; j++) { + REG_WR(sc, write_addr[j], page_addr[i]); + + for (k = 0; k < read_num; k++) { + if (IS_REG_IN_PRESET(read_addr[k].presets, preset)) { + size = read_addr[k].size; + for (n = 0; n < size; n++) { + addr = read_addr[k].addr + n*4; + *p++ = REG_RD(sc, addr); + } + } + } + } + } + return; +} + + +static int +bxe_get_preset_regs(struct bxe_softc *sc, uint32_t *p, uint32_t preset) +{ + uint32_t i, j, addr; + const struct wreg_addr *wreg_addr_p = NULL; + + if (CHIP_IS_E1(sc)) + wreg_addr_p = &wreg_addr_e1; + else if (CHIP_IS_E1H(sc)) + wreg_addr_p = &wreg_addr_e1h; + else if (CHIP_IS_E2(sc)) + wreg_addr_p = &wreg_addr_e2; + else if (CHIP_IS_E3A0(sc)) + wreg_addr_p = &wreg_addr_e3; + else if (CHIP_IS_E3B0(sc)) + wreg_addr_p = &wreg_addr_e3b0; + else + return (-1); + + /* Read the idle_chk registers */ + for (i = 0; i < IDLE_REGS_COUNT; i++) { + if (bxe_is_reg_in_chip(sc, &idle_reg_addrs[i]) && + IS_REG_IN_PRESET(idle_reg_addrs[i].presets, preset)) { + for (j = 0; j < idle_reg_addrs[i].size; j++) + *p++ = REG_RD(sc, idle_reg_addrs[i].addr + j*4); + } + } + + /* Read the regular registers */ + for (i = 0; i < REGS_COUNT; i++) { + if (bxe_is_reg_in_chip(sc, ®_addrs[i]) && + IS_REG_IN_PRESET(reg_addrs[i].presets, preset)) { + for (j = 0; j < reg_addrs[i].size; j++) + *p++ = REG_RD(sc, reg_addrs[i].addr + j*4); + } + } + + /* Read the CAM registers */ + if (bxe_is_wreg_in_chip(sc, wreg_addr_p) && + IS_REG_IN_PRESET(wreg_addr_p->presets, preset)) { + for (i = 0; i < wreg_addr_p->size; i++) { + *p++ = REG_RD(sc, wreg_addr_p->addr + i*4); + + /* In case of wreg_addr register, read additional + registers from read_regs array + */ + for (j = 0; j < wreg_addr_p->read_regs_count; j++) { + addr = *(wreg_addr_p->read_regs); + *p++ = REG_RD(sc, addr + j*4); + } + } + } + + /* Paged registers are supported in E2 & E3 only */ + if (CHIP_IS_E2(sc) || CHIP_IS_E3(sc)) { + /* Read "paged" registers */ + bxe_read_pages_regs(sc, p, preset); + } + + return 0; +} + +static int +bxe_grc_dump(struct bxe_softc *sc, bxe_grcdump_t *dump) +{ + int rval = 0; + uint32_t preset_idx; + uint8_t *buf; + uint32_t size; + struct dump_header *d_hdr; + + ecore_disable_blocks_parity(sc); + + buf = dump->grcdump; + d_hdr = dump->grcdump; + + d_hdr->header_size = (sizeof(struct dump_header) >> 2) - 1; + d_hdr->version = BNX2X_DUMP_VERSION; + d_hdr->preset = DUMP_ALL_PRESETS; + + if (CHIP_IS_E1(sc)) { + d_hdr->dump_meta_data = DUMP_CHIP_E1; + } else if (CHIP_IS_E1H(sc)) { + d_hdr->dump_meta_data = DUMP_CHIP_E1H; + } else if (CHIP_IS_E2(sc)) { + d_hdr->dump_meta_data = DUMP_CHIP_E2 | + (BXE_PATH(sc) ? DUMP_PATH_1 : DUMP_PATH_0); + } else if (CHIP_IS_E3A0(sc)) { + d_hdr->dump_meta_data = DUMP_CHIP_E3A0 | + (BXE_PATH(sc) ? DUMP_PATH_1 : DUMP_PATH_0); + } else if (CHIP_IS_E3B0(sc)) { + d_hdr->dump_meta_data = DUMP_CHIP_E3B0 | + (BXE_PATH(sc) ? DUMP_PATH_1 : DUMP_PATH_0); + } + + dump->grcdump_dwords = sizeof(struct dump_header) >> 2; + buf += sizeof(struct dump_header); + + for (preset_idx = 1; preset_idx <= DUMP_MAX_PRESETS; preset_idx++) { + + /* Skip presets with IOR */ + if ((preset_idx == 2) || (preset_idx == 5) || (preset_idx == 8) || + (preset_idx == 11)) + continue; + + rval = bxe_get_preset_regs(sc, sc->grc_dump, preset_idx); + + if (rval) + break; + + size = bxe_get_preset_regs_len(sc, preset_idx) * (sizeof (uint32_t)); + + rval = copyout(sc->grc_dump, buf, size); + + if (rval) + break; + + dump->grcdump_dwords += (size / (sizeof (uint32_t))); + + buf += size; + } + + ecore_clear_blocks_parity(sc); + ecore_enable_blocks_parity(sc); + + sc->grcdump_done = 1; + return(rval); +} + +static int +bxe_add_cdev(struct bxe_softc *sc) +{ + int max_preset_size; + + max_preset_size = bxe_get_max_regs_len(sc) * (sizeof (uint32_t)); + + sc->grc_dump = malloc(max_preset_size, M_DEVBUF, M_NOWAIT); + + if (sc->grc_dump == NULL) + return (-1); + + sc->ioctl_dev = make_dev(&bxe_cdevsw, + sc->ifp->if_dunit, + UID_ROOT, + GID_WHEEL, + 0600, + "%s", + if_name(sc->ifp)); + + if (sc->ioctl_dev == NULL) { + + free(sc->grc_dump, M_DEVBUF); + + return (-1); + } + + sc->ioctl_dev->si_drv1 = sc; + + return (0); +} + +static void +bxe_del_cdev(struct bxe_softc *sc) +{ + if (sc->ioctl_dev != NULL) + destroy_dev(sc->ioctl_dev); + + if (sc->grc_dump == NULL) + free(sc->grc_dump, M_DEVBUF); + + return; +} + +static int +bxe_eioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag, + struct thread *td) +{ + struct bxe_softc *sc; + int rval = 0; + device_t pci_dev; + bxe_grcdump_t *dump = NULL; + int grc_dump_size; + + if ((sc = (struct bxe_softc *)dev->si_drv1) == NULL) + return ENXIO; + + pci_dev= sc->dev; + + dump = (bxe_grcdump_t *)data; + + switch(cmd) { + + case BXE_GRC_DUMP_SIZE: + dump->pci_func = sc->pcie_func; + dump->grcdump_size = (bxe_get_total_regs_len32(sc) * sizeof(uint32_t)) + + sizeof(struct dump_header); + break; + + case BXE_GRC_DUMP: + + grc_dump_size = (bxe_get_total_regs_len32(sc) * sizeof(uint32_t)) + + sizeof(struct dump_header); + + if ((sc->grc_dump == NULL) || (dump->grcdump == NULL) || + (dump->grcdump_size < grc_dump_size)) { + rval = EINVAL; + break; + } + + rval = bxe_grc_dump(sc, dump); + + break; + + default: + break; + } + + return (rval); +} diff --git a/sys/dev/bxe/bxe.h b/sys/dev/bxe/bxe.h index 980a91f..f954ec0 100644 --- a/sys/dev/bxe/bxe.h +++ b/sys/dev/bxe/bxe.h @@ -1830,6 +1830,11 @@ struct bxe_softc { uint8_t prio_to_cos[BXE_MAX_PRIORITY]; int panic; + + struct cdev *ioctl_dev; + void *grc_dump; + int trigger_grcdump; + int grcdump_done; }; /* struct bxe_softc */ /* IOCTL sub-commands for edebug and firmware upgrade */ @@ -2296,6 +2301,7 @@ void ecore_storm_memset_struct(struct bxe_softc *sc, uint32_t addr, "ERROR: " format, \ ## args); \ } \ + sc->trigger_grcdump |= 0x1; \ } while(0) #ifdef ECORE_STOP_ON_ERROR diff --git a/sys/dev/bxe/bxe_dump.h b/sys/dev/bxe/bxe_dump.h new file mode 100644 index 0000000..4985003 --- /dev/null +++ b/sys/dev/bxe/bxe_dump.h @@ -0,0 +1,2231 @@ +/* + * Copyright (c) 2007-2015 QLogic Corporation. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS + * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF + * THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#ifndef __BXE_DUMP_H__ +#define __BXE_DUMP_H__ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +/* WaitP Definitions */ +#define DRV_DUMP_XSTORM_WAITP_ADDRESS 0x2b8a80 +#define DRV_DUMP_TSTORM_WAITP_ADDRESS 0x1b8a80 +#define DRV_DUMP_USTORM_WAITP_ADDRESS 0x338a80 +#define DRV_DUMP_CSTORM_WAITP_ADDRESS 0x238a80 + +/* Possible Chips */ +#define DUMP_CHIP_E1 1 +#define DUMP_CHIP_E1H 2 +#define DUMP_CHIP_E2 4 +#define DUMP_CHIP_E3A0 8 +#define DUMP_CHIP_E3B0 16 +#define DUMP_PATH_0 512 +#define DUMP_PATH_1 1024 +#define NUM_PRESETS 13 +#define NUM_CHIPS 5 + +struct dump_header { + uint32_t header_size; /* Size in DWORDs excluding this field */ + uint32_t version; + uint32_t preset; + uint32_t dump_meta_data; /* OR of CHIP and PATH. */ +}; + +#define BNX2X_DUMP_VERSION 0x61111111 +struct reg_addr { + uint32_t addr; + uint32_t size; + uint32_t chips; + uint32_t presets; +}; + +struct wreg_addr { + uint32_t addr; + uint32_t size; + uint32_t read_regs_count; + const uint32_t *read_regs; + uint32_t chips; + uint32_t presets; +}; + +#define PAGE_MODE_VALUES_E2 2 +#define PAGE_READ_REGS_E2 1 +#define PAGE_WRITE_REGS_E2 1 +static const uint32_t page_vals_e2[] = {0, 128}; +static const uint32_t page_write_regs_e2[] = {328476}; +static const struct reg_addr page_read_regs_e2[] = { + {0x58000, 4608, DUMP_CHIP_E2, 0x30} +}; + +#define PAGE_MODE_VALUES_E3 2 +#define PAGE_READ_REGS_E3 1 +#define PAGE_WRITE_REGS_E3 1 +static const uint32_t page_vals_e3[] = {0, 128}; +static const uint32_t page_write_regs_e3[] = {328476}; +static const struct reg_addr page_read_regs_e3[] = { + {0x58000, 4608, DUMP_CHIP_E3A0 | DUMP_CHIP_E3B0, 0x30} +}; + +static const struct reg_addr reg_addrs[] = { + { 0x2000, 1, 0x1f, 0xfff}, + { 0x2004, 1, 0x1f, 0x1fff}, + { 0x2008, 25, 0x1f, 0xfff}, + { 0x206c, 1, 0x1f, 0x1fff}, + { 0x2070, 313, 0x1f, 0xfff}, + { 0x2800, 103, 0x1f, 0xfff}, + { 0x3000, 287, 0x1f, 0xfff}, + { 0x3800, 331, 0x1f, 0xfff}, + { 0x8800, 6, 0x1f, 0x924}, + { 0x8818, 1, 0x1e, 0x924}, + { 0x9000, 4, 0x1c, 0x924}, + { 0x9010, 7, 0x1c, 0xfff}, + { 0x902c, 1, 0x1c, 0x924}, + { 0x9030, 1, 0x1c, 0xfff}, + { 0x9034, 13, 0x1c, 0x924}, + { 0x9068, 16, 0x1c, 0xfff}, + { 0x90a8, 98, 0x1c, 0x924}, + { 0x9230, 2, 0x1c, 0xfff}, + { 0x9238, 3, 0x1c, 0x924}, + { 0x9244, 1, 0x1c, 0xfff}, + { 0x9248, 1, 0x1c, 0x924}, + { 0x924c, 1, 0x4, 0x924}, + { 0x9250, 16, 0x1c, 0x924}, + { 0x92a8, 2, 0x1c, 0x1fff}, + { 0x92b4, 1, 0x1c, 0x1fff}, + { 0x9400, 33, 0x1c, 0x924}, + { 0x9484, 5, 0x18, 0x924}, + { 0xa000, 27, 0x1f, 0x924}, + { 0xa06c, 1, 0x3, 0x924}, + { 0xa070, 2, 0x1f, 0x924}, + { 0xa078, 1, 0x1f, 0x1fff}, + { 0xa07c, 31, 0x1f, 0x924}, + { 0xa0f8, 1, 0x1f, 0x1fff}, + { 0xa0fc, 3, 0x1f, 0x924}, + { 0xa108, 1, 0x1f, 0x1fff}, + { 0xa10c, 3, 0x1f, 0x924}, + { 0xa118, 1, 0x1f, 0x1fff}, + { 0xa11c, 28, 0x1f, 0x924}, + { 0xa18c, 4, 0x3, 0x924}, + { 0xa19c, 3, 0x1f, 0x924}, + { 0xa1a8, 1, 0x1f, 0x1fff}, + { 0xa1ac, 3, 0x1f, 0x924}, + { 0xa1b8, 1, 0x1f, 0x1fff}, + { 0xa1bc, 54, 0x1f, 0x924}, + { 0xa294, 2, 0x3, 0x924}, + { 0xa29c, 2, 0x1f, 0x924}, + { 0xa2a4, 2, 0x7, 0x924}, + { 0xa2ac, 2, 0x1f, 0x924}, + { 0xa2b4, 1, 0x1f, 0x1fff}, + { 0xa2b8, 49, 0x1f, 0x924}, + { 0xa38c, 2, 0x1f, 0x1fff}, + { 0xa398, 1, 0x1f, 0x1fff}, + { 0xa39c, 7, 0x1e, 0x924}, + { 0xa3b8, 2, 0x18, 0x924}, + { 0xa3c0, 1, 0x1e, 0x924}, + { 0xa3c4, 1, 0x1e, 0xfff}, + { 0xa3c8, 1, 0x1e, 0x924}, + { 0xa3d0, 1, 0x1e, 0x924}, + { 0xa3d8, 1, 0x1e, 0x924}, + { 0xa3e0, 1, 0x1e, 0x924}, + { 0xa3e8, 1, 0x1e, 0x924}, + { 0xa3f0, 1, 0x1e, 0x924}, + { 0xa3f8, 1, 0x1e, 0x924}, + { 0xa400, 1, 0x1f, 0x924}, + { 0xa404, 1, 0x1f, 0xfff}, + { 0xa408, 2, 0x1f, 0x1fff}, + { 0xa410, 7, 0x1f, 0x924}, + { 0xa42c, 12, 0x1f, 0xfff}, + { 0xa45c, 1, 0x1f, 0x924}, + { 0xa460, 1, 0x1f, 0x1924}, + { 0xa464, 15, 0x1f, 0x924}, + { 0xa4a0, 1, 0x7, 0x924}, + { 0xa4a4, 2, 0x1f, 0x924}, + { 0xa4ac, 2, 0x3, 0x924}, + { 0xa4b4, 1, 0x7, 0x924}, + { 0xa4b8, 2, 0x3, 0x924}, + { 0xa4c0, 3, 0x1f, 0x924}, + { 0xa4cc, 5, 0x3, 0x924}, + { 0xa4e0, 3, 0x1f, 0x924}, + { 0xa4fc, 2, 0x1f, 0x924}, + { 0xa504, 1, 0x3, 0x924}, + { 0xa508, 3, 0x1f, 0x924}, + { 0xa518, 1, 0x1f, 0x924}, + { 0xa520, 1, 0x1f, 0x924}, + { 0xa528, 1, 0x1f, 0x924}, + { 0xa530, 1, 0x1f, 0x924}, + { 0xa538, 1, 0x1f, 0x924}, + { 0xa540, 1, 0x1f, 0x924}, + { 0xa548, 1, 0x3, 0x924}, + { 0xa550, 1, 0x3, 0x924}, + { 0xa558, 1, 0x3, 0x924}, + { 0xa560, 1, 0x3, 0x924}, + { 0xa568, 1, 0x3, 0x924}, + { 0xa570, 1, 0x1f, 0x924}, + { 0xa580, 1, 0x1f, 0x1fff}, + { 0xa590, 1, 0x1f, 0x1fff}, + { 0xa5a0, 1, 0x7, 0x924}, + { 0xa5c0, 1, 0x1f, 0x924}, + { 0xa5e0, 1, 0x1e, 0x924}, + { 0xa5e8, 1, 0x1e, 0x924}, + { 0xa5f0, 1, 0x1e, 0x924}, + { 0xa5f8, 1, 0x6, 0x924}, + { 0xa5fc, 1, 0x1e, 0x924}, + { 0xa600, 5, 0x1e, 0xfff}, + { 0xa614, 1, 0x1e, 0x924}, + { 0xa618, 1, 0x1e, 0xfff}, + { 0xa61c, 1, 0x1e, 0x924}, + { 0xa620, 6, 0x1c, 0x924}, + { 0xa638, 20, 0x4, 0x924}, + { 0xa688, 35, 0x1c, 0x924}, + { 0xa714, 1, 0x1c, 0xfff}, + { 0xa718, 2, 0x1c, 0x924}, + { 0xa720, 1, 0x1c, 0xfff}, + { 0xa724, 3, 0x1c, 0x924}, + { 0xa730, 1, 0x4, 0x924}, + { 0xa734, 2, 0x1c, 0x924}, + { 0xa73c, 4, 0x4, 0x924}, + { 0xa74c, 1, 0x1c, 0x924}, + { 0xa750, 1, 0x1c, 0xfff}, + { 0xa754, 3, 0x1c, 0x924}, + { 0xa760, 5, 0x4, 0x924}, + { 0xa774, 7, 0x1c, 0x924}, + { 0xa790, 15, 0x4, 0x924}, + { 0xa7cc, 4, 0x1c, 0x924}, + { 0xa7e0, 6, 0x18, 0x924}, + { 0xa800, 18, 0x4, 0x924}, + { 0xa848, 33, 0x1c, 0x924}, + { 0xa8cc, 2, 0x18, 0x924}, + { 0xa8d4, 4, 0x1c, 0x924}, + { 0xa8e4, 1, 0x18, 0x924}, + { 0xa8e8, 1, 0x1c, 0x924}, + { 0xa8f0, 1, 0x1c, 0x924}, + { 0xa8f8, 30, 0x18, 0x924}, + { 0xa974, 73, 0x18, 0x924}, + { 0xac30, 1, 0x18, 0x924}, + { 0xac40, 1, 0x18, 0x924}, + { 0xac50, 1, 0x18, 0x924}, + { 0xac60, 1, 0x10, 0x924}, + { 0x10000, 9, 0x1f, 0x924}, + { 0x10024, 1, 0x7, 0x924}, + { 0x10028, 5, 0x1f, 0x924}, + { 0x1003c, 6, 0x7, 0x924}, + { 0x10054, 20, 0x1f, 0x924}, + { 0x100a4, 4, 0x7, 0x924}, + { 0x100b4, 11, 0x1f, 0x924}, + { 0x100e0, 4, 0x7, 0x924}, + { 0x100f0, 8, 0x1f, 0x924}, + { 0x10110, 6, 0x7, 0x924}, + { 0x10128, 110, 0x1f, 0x924}, + { 0x102e0, 4, 0x7, 0x924}, + { 0x102f0, 18, 0x1f, 0x924}, + { 0x10338, 20, 0x7, 0x924}, + { 0x10388, 10, 0x1f, 0x924}, + { 0x103d0, 2, 0x3, 0x1fff}, + { 0x103dc, 1, 0x3, 0x1fff}, + { 0x10400, 6, 0x7, 0x924}, + { 0x10418, 1, 0x1f, 0xfff}, + { 0x1041c, 1, 0x1f, 0x924}, + { 0x10420, 1, 0x1f, 0xfff}, + { 0x10424, 1, 0x1f, 0x924}, + { 0x10428, 1, 0x1f, 0xfff}, + { 0x1042c, 1, 0x1f, 0x924}, + { 0x10430, 10, 0x7, 0x924}, + { 0x10458, 2, 0x1f, 0x924}, + { 0x10460, 1, 0x1f, 0xfff}, + { 0x10464, 4, 0x1f, 0x924}, + { 0x10474, 1, 0x1f, 0xfff}, + { 0x10478, 14, 0x1f, 0x924}, + { 0x104b0, 12, 0x7, 0x924}, + { 0x104e0, 1, 0x1f, 0xfff}, + { 0x104e8, 1, 0x1f, 0x924}, + { 0x104ec, 1, 0x1f, 0xfff}, + { 0x104f4, 1, 0x1f, 0x924}, + { 0x104f8, 1, 0x1f, 0xfff}, + { 0x10500, 2, 0x1f, 0x924}, + { 0x10508, 1, 0x1f, 0xfff}, + { 0x1050c, 9, 0x1f, 0x924}, + { 0x10530, 1, 0x1f, 0xfff}, + { 0x10534, 1, 0x1f, 0x924}, + { 0x10538, 1, 0x1f, 0xfff}, + { 0x1053c, 3, 0x1f, 0x924}, + { 0x10548, 1, 0x1f, 0xfff}, + { 0x1054c, 3, 0x1f, 0x924}, + { 0x10558, 1, 0x1f, 0xfff}, + { 0x1055c, 123, 0x1f, 0x924}, + { 0x10750, 2, 0x7, 0x924}, + { 0x10760, 2, 0x7, 0x924}, + { 0x10770, 2, 0x7, 0x924}, + { 0x10780, 2, 0x7, 0x924}, + { 0x10790, 2, 0x1f, 0x924}, + { 0x107a0, 2, 0x7, 0x924}, + { 0x107b0, 2, 0x7, 0x924}, + { 0x107c0, 2, 0x7, 0x924}, + { 0x107d0, 2, 0x7, 0x924}, + { 0x107e0, 2, 0x1f, 0x924}, + { 0x10880, 2, 0x1f, 0x924}, + { 0x10900, 2, 0x1f, 0x924}, + { 0x16000, 1, 0x6, 0x924}, + { 0x16004, 25, 0x1e, 0x924}, + { 0x16070, 8, 0x1e, 0x924}, + { 0x16090, 4, 0xe, 0x924}, + { 0x160a0, 6, 0x1e, 0x924}, + { 0x160c0, 7, 0x1e, 0x924}, + { 0x160dc, 2, 0x6, 0x924}, + { 0x160e4, 6, 0x1e, 0x924}, + { 0x160fc, 4, 0x1e, 0x1fff}, + { 0x1610c, 2, 0x6, 0x924}, + { 0x16114, 6, 0x1e, 0x924}, + { 0x16140, 48, 0x1e, 0x1fff}, + { 0x16204, 5, 0x1e, 0x924}, + { 0x18000, 1, 0x1e, 0x924}, + { 0x18008, 1, 0x1e, 0x924}, + { 0x18010, 35, 0x1c, 0x924}, + { 0x180a4, 2, 0x1c, 0x924}, + { 0x180c0, 9, 0x1c, 0x924}, + { 0x180e4, 1, 0xc, 0x924}, + { 0x180e8, 2, 0x1c, 0x924}, + { 0x180f0, 1, 0xc, 0x924}, + { 0x180f4, 79, 0x1c, 0x924}, + { 0x18230, 1, 0xc, 0x924}, + { 0x18234, 2, 0x1c, 0x924}, + { 0x1823c, 1, 0xc, 0x924}, + { 0x18240, 13, 0x1c, 0x924}, + { 0x18274, 1, 0x4, 0x924}, + { 0x18278, 12, 0x1c, 0x924}, + { 0x182a8, 1, 0x1c, 0xfff}, + { 0x182ac, 3, 0x1c, 0x924}, + { 0x182b8, 1, 0x1c, 0xfff}, + { 0x182bc, 19, 0x1c, 0x924}, + { 0x18308, 1, 0x1c, 0xfff}, + { 0x1830c, 3, 0x1c, 0x924}, + { 0x18318, 1, 0x1c, 0xfff}, + { 0x1831c, 7, 0x1c, 0x924}, + { 0x18338, 1, 0x1c, 0xfff}, + { 0x1833c, 3, 0x1c, 0x924}, + { 0x18348, 1, 0x1c, 0xfff}, + { 0x1834c, 28, 0x1c, 0x924}, + { 0x183bc, 2, 0x1c, 0x1fff}, + { 0x183c8, 3, 0x1c, 0x1fff}, + { 0x183d8, 1, 0x1c, 0x1fff}, + { 0x18440, 48, 0x1c, 0x1fff}, + { 0x18500, 15, 0x1c, 0x924}, + { 0x18570, 1, 0x18, 0xfff}, + { 0x18574, 1, 0x18, 0x924}, + { 0x18578, 1, 0x18, 0xfff}, + { 0x1857c, 4, 0x18, 0x924}, + { 0x1858c, 1, 0x18, 0xfff}, + { 0x18590, 1, 0x18, 0x924}, + { 0x18594, 1, 0x18, 0xfff}, + { 0x18598, 32, 0x18, 0x924}, + { 0x18618, 5, 0x10, 0x924}, + { 0x1862c, 4, 0x10, 0xfff}, + { 0x1863c, 16, 0x10, 0x924}, + { 0x18680, 44, 0x10, 0x924}, + { 0x18748, 12, 0x10, 0x924}, + { 0x18788, 1, 0x10, 0x924}, + { 0x1879c, 6, 0x10, 0x924}, + { 0x187c4, 51, 0x10, 0x924}, + { 0x18a00, 48, 0x10, 0x924}, + { 0x20000, 24, 0x1f, 0x924}, + { 0x20060, 8, 0x1f, 0x9e4}, + { 0x20080, 94, 0x1f, 0x924}, + { 0x201f8, 1, 0x3, 0x924}, + { 0x201fc, 1, 0x1f, 0x924}, + { 0x20200, 1, 0x3, 0x924}, + { 0x20204, 1, 0x1f, 0x924}, + { 0x20208, 1, 0x3, 0x924}, + { 0x2020c, 4, 0x1f, 0x924}, + { 0x2021c, 11, 0x1f, 0xfff}, + { 0x20248, 24, 0x1f, 0x924}, + { 0x202b8, 2, 0x1f, 0x1fff}, + { 0x202c4, 1, 0x1f, 0x1fff}, + { 0x202c8, 1, 0x1c, 0x924}, + { 0x202d8, 4, 0x1c, 0x924}, + { 0x202f0, 1, 0x10, 0x924}, + { 0x20400, 1, 0x1f, 0x924}, + { 0x20404, 1, 0x1f, 0xfff}, + { 0x2040c, 2, 0x1f, 0xfff}, + { 0x20414, 2, 0x1f, 0x924}, + { 0x2041c, 2, 0x1f, 0xfff}, + { 0x20424, 2, 0x1f, 0x924}, + { 0x2042c, 18, 0x1e, 0x924}, + { 0x20480, 1, 0x1f, 0x924}, + { 0x20500, 1, 0x1f, 0x924}, + { 0x20600, 1, 0x1f, 0x924}, + { 0x28000, 1, 0x1f, 0x9e4}, + { 0x28004, 255, 0x1f, 0x180}, + { 0x28400, 1, 0x1f, 0x1c0}, + { 0x28404, 255, 0x1f, 0x180}, + { 0x28800, 1, 0x1f, 0x1c0}, + { 0x28804, 255, 0x1f, 0x180}, + { 0x28c00, 1, 0x1f, 0x1c0}, + { 0x28c04, 255, 0x1f, 0x180}, + { 0x29000, 1, 0x1f, 0x1c0}, + { 0x29004, 255, 0x1f, 0x180}, + { 0x29400, 1, 0x1f, 0x1c0}, + { 0x29404, 255, 0x1f, 0x180}, + { 0x29800, 1, 0x1f, 0x1c0}, + { 0x29804, 255, 0x1f, 0x180}, + { 0x29c00, 1, 0x1f, 0x1c0}, + { 0x29c04, 255, 0x1f, 0x180}, + { 0x2a000, 1, 0x1f, 0x1c0}, + { 0x2a004, 255, 0x1f, 0x180}, + { 0x2a400, 1, 0x1f, 0x1c0}, + { 0x2a404, 255, 0x1f, 0x180}, + { 0x2a800, 1, 0x1f, 0x1c0}, + { 0x2a804, 255, 0x1f, 0x180}, + { 0x2ac00, 1, 0x1f, 0x1c0}, + { 0x2ac04, 255, 0x1f, 0x180}, + { 0x2b000, 1, 0x1f, 0x1c0}, + { 0x2b004, 255, 0x1f, 0x180}, + { 0x2b400, 1, 0x1f, 0x1c0}, + { 0x2b404, 255, 0x1f, 0x180}, + { 0x2b800, 1, 0x1f, 0x1c0}, + { 0x2b804, 255, 0x1f, 0x180}, + { 0x2bc00, 1, 0x1f, 0x1c0}, + { 0x2bc04, 255, 0x1f, 0x180}, + { 0x2c000, 1, 0x1f, 0x1c0}, + { 0x2c004, 255, 0x1f, 0x180}, + { 0x2c400, 1, 0x1f, 0x1c0}, + { 0x2c404, 255, 0x1f, 0x180}, + { 0x2c800, 1, 0x1f, 0x1c0}, + { 0x2c804, 255, 0x1f, 0x180}, + { 0x2cc00, 1, 0x1f, 0x1c0}, + { 0x2cc04, 255, 0x1f, 0x180}, + { 0x2d000, 1, 0x1f, 0x1c0}, + { 0x2d004, 255, 0x1f, 0x180}, + { 0x2d400, 1, 0x1f, 0x1c0}, + { 0x2d404, 255, 0x1f, 0x180}, + { 0x2d800, 1, 0x1f, 0x1c0}, + { 0x2d804, 255, 0x1f, 0x180}, + { 0x2dc00, 1, 0x1f, 0x1c0}, + { 0x2dc04, 255, 0x1f, 0x180}, + { 0x2e000, 1, 0x1f, 0x1c0}, + { 0x2e004, 255, 0x1f, 0x180}, + { 0x2e400, 1, 0x1f, 0x1c0}, + { 0x2e404, 255, 0x1f, 0x180}, + { 0x2e800, 1, 0x1f, 0x1c0}, + { 0x2e804, 255, 0x1f, 0x180}, + { 0x2ec00, 1, 0x1f, 0x1c0}, + { 0x2ec04, 255, 0x1f, 0x180}, + { 0x2f000, 1, 0x1f, 0x1c0}, + { 0x2f004, 255, 0x1f, 0x180}, + { 0x2f400, 1, 0x1f, 0x1c0}, + { 0x2f404, 255, 0x1f, 0x180}, + { 0x2f800, 1, 0x1f, 0x1c0}, + { 0x2f804, 255, 0x1f, 0x180}, + { 0x2fc00, 1, 0x1f, 0x1c0}, + { 0x2fc04, 255, 0x1f, 0x180}, + { 0x30000, 1, 0x1f, 0x9e4}, + { 0x30004, 255, 0x1f, 0x180}, + { 0x30400, 1, 0x1f, 0x1c0}, + { 0x30404, 255, 0x1f, 0x180}, + { 0x30800, 1, 0x1f, 0x1c0}, + { 0x30804, 255, 0x1f, 0x180}, + { 0x30c00, 1, 0x1f, 0x1c0}, + { 0x30c04, 255, 0x1f, 0x180}, + { 0x31000, 1, 0x1f, 0x1c0}, + { 0x31004, 255, 0x1f, 0x180}, + { 0x31400, 1, 0x1f, 0x1c0}, + { 0x31404, 255, 0x1f, 0x180}, + { 0x31800, 1, 0x1f, 0x1c0}, + { 0x31804, 255, 0x1f, 0x180}, + { 0x31c00, 1, 0x1f, 0x1c0}, + { 0x31c04, 255, 0x1f, 0x180}, + { 0x32000, 1, 0x1f, 0x1c0}, + { 0x32004, 255, 0x1f, 0x180}, + { 0x32400, 1, 0x1f, 0x1c0}, + { 0x32404, 255, 0x1f, 0x180}, + { 0x32800, 1, 0x1f, 0x1c0}, + { 0x32804, 255, 0x1f, 0x180}, + { 0x32c00, 1, 0x1f, 0x1c0}, + { 0x32c04, 255, 0x1f, 0x180}, + { 0x33000, 1, 0x1f, 0x1c0}, + { 0x33004, 255, 0x1f, 0x180}, + { 0x33400, 1, 0x1f, 0x1c0}, + { 0x33404, 255, 0x1f, 0x180}, + { 0x33800, 1, 0x1f, 0x1c0}, + { 0x33804, 255, 0x1f, 0x180}, + { 0x33c00, 1, 0x1f, 0x1c0}, + { 0x33c04, 255, 0x1f, 0x180}, + { 0x34000, 1, 0x1f, 0x1c0}, + { 0x34004, 255, 0x1f, 0x180}, + { 0x34400, 1, 0x1f, 0x1c0}, + { 0x34404, 255, 0x1f, 0x180}, + { 0x34800, 1, 0x1f, 0x1c0}, + { 0x34804, 255, 0x1f, 0x180}, + { 0x34c00, 1, 0x1f, 0x1c0}, + { 0x34c04, 255, 0x1f, 0x180}, + { 0x35000, 1, 0x1f, 0x1c0}, + { 0x35004, 255, 0x1f, 0x180}, + { 0x35400, 1, 0x1f, 0x1c0}, + { 0x35404, 255, 0x1f, 0x180}, + { 0x35800, 1, 0x1f, 0x1c0}, + { 0x35804, 255, 0x1f, 0x180}, + { 0x35c00, 1, 0x1f, 0x1c0}, + { 0x35c04, 255, 0x1f, 0x180}, + { 0x36000, 1, 0x1f, 0x1c0}, + { 0x36004, 255, 0x1f, 0x180}, + { 0x36400, 1, 0x1f, 0x1c0}, + { 0x36404, 255, 0x1f, 0x180}, + { 0x36800, 1, 0x1f, 0x1c0}, + { 0x36804, 255, 0x1f, 0x180}, + { 0x36c00, 1, 0x1f, 0x1c0}, + { 0x36c04, 255, 0x1f, 0x180}, + { 0x37000, 1, 0x1f, 0x1c0}, + { 0x37004, 255, 0x1f, 0x180}, + { 0x37400, 1, 0x1f, 0x1c0}, + { 0x37404, 255, 0x1f, 0x180}, + { 0x37800, 1, 0x1f, 0x1c0}, + { 0x37804, 255, 0x1f, 0x180}, + { 0x37c00, 1, 0x1f, 0x1c0}, + { 0x37c04, 255, 0x1f, 0x180}, + { 0x38000, 1, 0x1f, 0x1c0}, + { 0x38004, 255, 0x1f, 0x180}, + { 0x38400, 1, 0x1f, 0x1c0}, + { 0x38404, 255, 0x1f, 0x180}, + { 0x38800, 1, 0x1f, 0x1c0}, + { 0x38804, 255, 0x1f, 0x180}, + { 0x38c00, 1, 0x1f, 0x1c0}, + { 0x38c04, 255, 0x1f, 0x180}, + { 0x39000, 1, 0x1f, 0x1c0}, + { 0x39004, 255, 0x1f, 0x180}, + { 0x39400, 1, 0x1f, 0x1c0}, + { 0x39404, 255, 0x1f, 0x180}, + { 0x39800, 1, 0x1f, 0x1c0}, + { 0x39804, 255, 0x1f, 0x180}, + { 0x39c00, 1, 0x1f, 0x1c0}, + { 0x39c04, 255, 0x1f, 0x180}, + { 0x3a000, 1, 0x1f, 0x1c0}, + { 0x3a004, 255, 0x1f, 0x180}, + { 0x3a400, 1, 0x1f, 0x1c0}, + { 0x3a404, 255, 0x1f, 0x180}, + { 0x3a800, 1, 0x1f, 0x1c0}, + { 0x3a804, 255, 0x1f, 0x180}, + { 0x3ac00, 1, 0x1f, 0x1c0}, + { 0x3ac04, 255, 0x1f, 0x180}, + { 0x3b000, 1, 0x1f, 0x1c0}, + { 0x3b004, 255, 0x1f, 0x180}, + { 0x3b400, 1, 0x1f, 0x1c0}, + { 0x3b404, 255, 0x1f, 0x180}, + { 0x3b800, 1, 0x1f, 0x1c0}, + { 0x3b804, 255, 0x1f, 0x180}, + { 0x3bc00, 1, 0x1f, 0x1c0}, + { 0x3bc04, 255, 0x1f, 0x180}, + { 0x3c000, 1, 0x1f, 0x1c0}, + { 0x3c004, 255, 0x1f, 0x180}, + { 0x3c400, 1, 0x1f, 0x1c0}, + { 0x3c404, 255, 0x1f, 0x180}, + { 0x3c800, 1, 0x1f, 0x1c0}, + { 0x3c804, 255, 0x1f, 0x180}, + { 0x3cc00, 1, 0x1f, 0x1c0}, + { 0x3cc04, 255, 0x1f, 0x180}, + { 0x3d000, 1, 0x1f, 0x1c0}, + { 0x3d004, 255, 0x1f, 0x180}, + { 0x3d400, 1, 0x1f, 0x1c0}, + { 0x3d404, 255, 0x1f, 0x180}, + { 0x3d800, 1, 0x1f, 0x1c0}, + { 0x3d804, 255, 0x1f, 0x180}, + { 0x3dc00, 1, 0x1f, 0x1c0}, + { 0x3dc04, 255, 0x1f, 0x180}, + { 0x3e000, 1, 0x1f, 0x1c0}, + { 0x3e004, 255, 0x1f, 0x180}, + { 0x3e400, 1, 0x1f, 0x1c0}, + { 0x3e404, 255, 0x1f, 0x180}, + { 0x3e800, 1, 0x1f, 0x1c0}, + { 0x3e804, 255, 0x1f, 0x180}, + { 0x3ec00, 1, 0x1f, 0x1c0}, + { 0x3ec04, 255, 0x1f, 0x180}, + { 0x3f000, 1, 0x1f, 0x1c0}, + { 0x3f004, 255, 0x1f, 0x180}, + { 0x3f400, 1, 0x1f, 0x1c0}, + { 0x3f404, 255, 0x1f, 0x180}, + { 0x3f800, 1, 0x1f, 0x1c0}, + { 0x3f804, 255, 0x1f, 0x180}, + { 0x3fc00, 1, 0x1f, 0x1c0}, + { 0x3fc04, 255, 0x1f, 0x180}, + { 0x40000, 85, 0x1f, 0x924}, + { 0x40154, 13, 0x1f, 0xfff}, + { 0x40198, 2, 0x1f, 0x1fff}, + { 0x401a4, 1, 0x1f, 0x1fff}, + { 0x401a8, 8, 0x1e, 0x924}, + { 0x401c8, 1, 0x2, 0x924}, + { 0x401cc, 2, 0x1e, 0x924}, + { 0x401d4, 2, 0x1c, 0x924}, + { 0x40200, 4, 0x1f, 0x924}, + { 0x40220, 6, 0x1c, 0x924}, + { 0x40238, 8, 0xc, 0x924}, + { 0x40258, 4, 0x1c, 0x924}, + { 0x40268, 2, 0x18, 0x924}, + { 0x40270, 17, 0x10, 0x924}, + { 0x40400, 43, 0x1f, 0x924}, + { 0x404bc, 2, 0x1f, 0x1fff}, + { 0x404c8, 1, 0x1f, 0x1fff}, + { 0x404cc, 3, 0x1e, 0x924}, + { 0x404e0, 1, 0x1c, 0x924}, + { 0x40500, 2, 0x1f, 0x924}, + { 0x40510, 2, 0x1f, 0x924}, + { 0x40520, 2, 0x1f, 0x924}, + { 0x40530, 2, 0x1f, 0x924}, + { 0x40540, 2, 0x1f, 0x924}, + { 0x40550, 10, 0x1c, 0x924}, + { 0x40610, 2, 0x1c, 0x924}, + { 0x42000, 164, 0x1f, 0x924}, + { 0x422b0, 2, 0x1f, 0x1fff}, + { 0x422bc, 1, 0x1f, 0x1fff}, + { 0x422c0, 4, 0x1c, 0x924}, + { 0x422d4, 5, 0x1e, 0x924}, + { 0x422e8, 1, 0x1c, 0x924}, + { 0x42400, 49, 0x1f, 0x924}, + { 0x424c8, 32, 0x1f, 0x924}, + { 0x42548, 1, 0x1f, 0xfff}, + { 0x4254c, 1, 0x1f, 0x924}, + { 0x42550, 1, 0x1f, 0xfff}, + { 0x42554, 1, 0x1f, 0x924}, + { 0x42558, 1, 0x1f, 0xfff}, + { 0x4255c, 1, 0x1f, 0x924}, + { 0x42568, 2, 0x1f, 0x924}, + { 0x42640, 5, 0x1c, 0x924}, + { 0x42800, 1, 0x1f, 0x924}, + { 0x50000, 1, 0x1f, 0x1fff}, + { 0x50004, 19, 0x1f, 0x924}, + { 0x50050, 8, 0x1f, 0x93c}, + { 0x50070, 60, 0x1f, 0x924}, + { 0x50160, 8, 0x1f, 0xfff}, + { 0x50180, 20, 0x1f, 0x924}, + { 0x501e0, 2, 0x1f, 0x1fff}, + { 0x501ec, 1, 0x1f, 0x1fff}, + { 0x501f0, 4, 0x1e, 0x924}, + { 0x50200, 1, 0x1f, 0x924}, + { 0x50204, 1, 0x1f, 0xfff}, + { 0x5020c, 2, 0x1f, 0xfff}, + { 0x50214, 2, 0x1f, 0x924}, + { 0x5021c, 1, 0x1f, 0xfff}, + { 0x50220, 2, 0x1f, 0x924}, + { 0x50228, 6, 0x1e, 0x924}, + { 0x50240, 1, 0x1f, 0x924}, + { 0x50280, 1, 0x1f, 0x924}, + { 0x50300, 1, 0x1c, 0x924}, + { 0x5030c, 1, 0x1c, 0x924}, + { 0x50318, 1, 0x1c, 0x934}, + { 0x5031c, 1, 0x1c, 0x924}, + { 0x50320, 2, 0x1c, 0x934}, + { 0x50330, 1, 0x10, 0x924}, + { 0x52000, 1, 0x1f, 0x924}, + { 0x54000, 1, 0x1f, 0x93c}, + { 0x54004, 255, 0x1f, 0x30}, + { 0x54400, 1, 0x1f, 0x38}, + { 0x54404, 255, 0x1f, 0x30}, + { 0x54800, 1, 0x1f, 0x38}, + { 0x54804, 255, 0x1f, 0x30}, + { 0x54c00, 1, 0x1f, 0x38}, + { 0x54c04, 255, 0x1f, 0x30}, + { 0x55000, 1, 0x1f, 0x38}, + { 0x55004, 255, 0x1f, 0x30}, + { 0x55400, 1, 0x1f, 0x38}, + { 0x55404, 255, 0x1f, 0x30}, + { 0x55800, 1, 0x1f, 0x38}, + { 0x55804, 255, 0x1f, 0x30}, + { 0x55c00, 1, 0x1f, 0x38}, + { 0x55c04, 255, 0x1f, 0x30}, + { 0x56000, 1, 0x1f, 0x38}, + { 0x56004, 255, 0x1f, 0x30}, + { 0x56400, 1, 0x1f, 0x38}, + { 0x56404, 255, 0x1f, 0x30}, + { 0x56800, 1, 0x1f, 0x38}, + { 0x56804, 255, 0x1f, 0x30}, + { 0x56c00, 1, 0x1f, 0x38}, + { 0x56c04, 255, 0x1f, 0x30}, + { 0x57000, 1, 0x1f, 0x38}, + { 0x57004, 255, 0x1f, 0x30}, + { 0x58000, 1, 0x1f, 0x934}, + { 0x58004, 8191, 0x3, 0x30}, + { 0x60000, 26, 0x1f, 0x924}, + { 0x60068, 8, 0x3, 0x924}, + { 0x60088, 2, 0x1f, 0x924}, + { 0x60090, 1, 0x1f, 0xfff}, + { 0x60094, 9, 0x1f, 0x924}, + { 0x600b8, 9, 0x3, 0x924}, + { 0x600dc, 1, 0x1f, 0x924}, + { 0x600e0, 5, 0x3, 0x924}, + { 0x600f4, 1, 0x7, 0x924}, + { 0x600f8, 1, 0x3, 0x924}, + { 0x600fc, 8, 0x1f, 0x924}, + { 0x6012c, 2, 0x1f, 0x1fff}, + { 0x60138, 1, 0x1f, 0x1fff}, + { 0x6013c, 24, 0x2, 0x924}, + { 0x6019c, 2, 0x1c, 0x924}, + { 0x601ac, 18, 0x1c, 0x924}, + { 0x60200, 1, 0x1f, 0xb6d}, + { 0x60204, 2, 0x1f, 0x249}, + { 0x60210, 13, 0x1c, 0x924}, + { 0x60244, 16, 0x10, 0x924}, + { 0x61000, 1, 0x1f, 0xb6d}, + { 0x61004, 511, 0x1f, 0x249}, + { 0x61800, 512, 0x18, 0x249}, + { 0x70000, 8, 0x1f, 0xb6d}, + { 0x70020, 8184, 0x1f, 0x249}, + { 0x78000, 8192, 0x18, 0x249}, + { 0x85000, 3, 0x1f, 0x1000}, + { 0x8501c, 7, 0x1f, 0x1000}, + { 0x85048, 1, 0x1f, 0x1000}, + { 0x85200, 32, 0x1f, 0x1000}, + { 0xa0000, 16384, 0x3, 0x1000}, + { 0xb0000, 16384, 0x2, 0x1000}, + { 0xc1000, 7, 0x1f, 0x924}, + { 0xc102c, 2, 0x1f, 0x1fff}, + { 0xc1038, 1, 0x1f, 0x1fff}, + { 0xc103c, 2, 0x1c, 0x924}, + { 0xc1800, 2, 0x1f, 0x924}, + { 0xc2000, 164, 0x1f, 0x924}, + { 0xc22b0, 2, 0x1f, 0x1fff}, + { 0xc22bc, 1, 0x1f, 0x1fff}, + { 0xc22c0, 5, 0x1c, 0x924}, + { 0xc22d8, 4, 0x1c, 0x924}, + { 0xc2400, 49, 0x1f, 0x924}, + { 0xc24c8, 32, 0x1f, 0x924}, + { 0xc2548, 1, 0x1f, 0xfff}, + { 0xc254c, 1, 0x1f, 0x924}, + { 0xc2550, 1, 0x1f, 0xfff}, + { 0xc2554, 1, 0x1f, 0x924}, + { 0xc2558, 1, 0x1f, 0xfff}, + { 0xc255c, 1, 0x1f, 0x924}, + { 0xc2568, 2, 0x1f, 0x924}, + { 0xc2600, 1, 0x1f, 0x924}, + { 0xc4000, 165, 0x1f, 0x924}, + { 0xc42b4, 2, 0x1f, 0x1fff}, + { 0xc42c0, 1, 0x1f, 0x1fff}, + { 0xc42d8, 2, 0x1c, 0x924}, + { 0xc42e0, 7, 0x1e, 0x924}, + { 0xc42fc, 1, 0x1c, 0x924}, + { 0xc4400, 51, 0x1f, 0x924}, + { 0xc44d0, 32, 0x1f, 0x924}, + { 0xc4550, 1, 0x1f, 0xfff}, + { 0xc4554, 1, 0x1f, 0x924}, + { 0xc4558, 1, 0x1f, 0xfff}, + { 0xc455c, 1, 0x1f, 0x924}, + { 0xc4560, 1, 0x1f, 0xfff}, + { 0xc4564, 1, 0x1f, 0x924}, + { 0xc4570, 2, 0x1f, 0x924}, + { 0xc4578, 5, 0x1c, 0x924}, + { 0xc4600, 1, 0x1f, 0x924}, + { 0xd0000, 19, 0x1f, 0x924}, + { 0xd004c, 8, 0x1f, 0x1927}, + { 0xd006c, 64, 0x1f, 0x924}, + { 0xd016c, 8, 0x1f, 0xfff}, + { 0xd018c, 19, 0x1f, 0x924}, + { 0xd01e8, 2, 0x1f, 0x1fff}, + { 0xd01f4, 1, 0x1f, 0x1fff}, + { 0xd01fc, 1, 0x1c, 0x924}, + { 0xd0200, 1, 0x1f, 0x924}, + { 0xd0204, 1, 0x1f, 0xfff}, + { 0xd020c, 3, 0x1f, 0xfff}, + { 0xd0218, 4, 0x1f, 0x924}, + { 0xd0228, 18, 0x1e, 0x924}, + { 0xd0280, 1, 0x1f, 0x924}, + { 0xd0300, 1, 0x1f, 0x924}, + { 0xd0400, 1, 0x1f, 0x924}, + { 0xd0818, 1, 0x10, 0x924}, + { 0xd4000, 1, 0x1f, 0x1927}, + { 0xd4004, 255, 0x1f, 0x6}, + { 0xd4400, 1, 0x1f, 0x1007}, + { 0xd4404, 255, 0x1f, 0x6}, + { 0xd4800, 1, 0x1f, 0x1007}, + { 0xd4804, 255, 0x1f, 0x6}, + { 0xd4c00, 1, 0x1f, 0x1007}, + { 0xd4c04, 255, 0x1f, 0x6}, + { 0xd5000, 1, 0x1f, 0x1007}, + { 0xd5004, 255, 0x1f, 0x6}, + { 0xd5400, 1, 0x1f, 0x1007}, + { 0xd5404, 255, 0x1f, 0x6}, + { 0xd5800, 1, 0x1f, 0x1007}, + { 0xd5804, 255, 0x1f, 0x6}, + { 0xd5c00, 1, 0x1f, 0x1007}, + { 0xd5c04, 255, 0x1f, 0x6}, + { 0xd6000, 1, 0x1f, 0x1007}, + { 0xd6004, 255, 0x1f, 0x6}, + { 0xd6400, 1, 0x1f, 0x1007}, + { 0xd6404, 255, 0x1f, 0x6}, + { 0xd8000, 1, 0x1f, 0x1927}, + { 0xd8004, 255, 0x1f, 0x6}, + { 0xd8400, 1, 0x1f, 0x1007}, + { 0xd8404, 255, 0x1f, 0x6}, + { 0xd8800, 1, 0x1f, 0x1007}, + { 0xd8804, 255, 0x1f, 0x6}, + { 0xd8c00, 1, 0x1f, 0x1007}, + { 0xd8c04, 255, 0x1f, 0x6}, + { 0xd9000, 1, 0x1f, 0x1007}, + { 0xd9004, 255, 0x1f, 0x6}, + { 0xd9400, 1, 0x1f, 0x1007}, + { 0xd9404, 255, 0x1f, 0x6}, + { 0xd9800, 1, 0x1f, 0x1007}, + { 0xd9804, 255, 0x1f, 0x6}, + { 0xd9c00, 1, 0x1f, 0x1007}, + { 0xd9c04, 255, 0x1f, 0x6}, + { 0xda000, 1, 0x1f, 0x1007}, + { 0xda004, 255, 0x1f, 0x6}, + { 0xda400, 1, 0x1f, 0x1007}, + { 0xda404, 255, 0x1f, 0x6}, + { 0xda800, 1, 0x1f, 0x1007}, + { 0xda804, 255, 0x1f, 0x6}, + { 0xdac00, 1, 0x1f, 0x1007}, + { 0xdac04, 255, 0x1f, 0x6}, + { 0xdb000, 1, 0x1f, 0x1007}, + { 0xdb004, 255, 0x1f, 0x6}, + { 0xdb400, 1, 0x1f, 0x1007}, + { 0xdb404, 255, 0x1f, 0x6}, + { 0xdb800, 1, 0x1f, 0x1007}, + { 0xdb804, 255, 0x1f, 0x6}, + { 0xdbc00, 1, 0x1f, 0x1007}, + { 0xdbc04, 255, 0x1f, 0x6}, + { 0xdc000, 1, 0x1f, 0x1007}, + { 0xdc004, 255, 0x1f, 0x6}, + { 0xdc400, 1, 0x1f, 0x1007}, + { 0xdc404, 255, 0x1f, 0x6}, + { 0xdc800, 1, 0x1f, 0x1007}, + { 0xdc804, 255, 0x1f, 0x6}, + { 0xdcc00, 1, 0x1f, 0x1007}, + { 0xdcc04, 255, 0x1f, 0x6}, + { 0xdd000, 1, 0x1f, 0x1007}, + { 0xdd004, 255, 0x1f, 0x6}, + { 0xdd400, 1, 0x1f, 0x1007}, + { 0xdd404, 255, 0x1f, 0x6}, + { 0xdd800, 1, 0x1f, 0x1007}, + { 0xdd804, 255, 0x1f, 0x6}, + { 0xddc00, 1, 0x1f, 0x1007}, + { 0xddc04, 255, 0x1f, 0x6}, + { 0xde000, 1, 0x1f, 0x1007}, + { 0xde004, 255, 0x1f, 0x6}, + { 0xde400, 1, 0x1f, 0x1007}, + { 0xde404, 255, 0x1f, 0x6}, + { 0xde800, 1, 0x1f, 0x1007}, + { 0xde804, 255, 0x1f, 0x6}, + { 0xdec00, 1, 0x1f, 0x1007}, + { 0xdec04, 255, 0x1f, 0x6}, + { 0xdf000, 1, 0x1f, 0x1007}, + { 0xdf004, 255, 0x1f, 0x6}, + { 0xdf400, 1, 0x1f, 0x1007}, + { 0xdf404, 255, 0x1f, 0x6}, + { 0xdf800, 1, 0x1f, 0x1007}, + { 0xdf804, 255, 0x1f, 0x6}, + { 0xdfc00, 1, 0x1f, 0x1007}, + { 0xdfc04, 255, 0x1f, 0x6}, + { 0xe0000, 21, 0x1f, 0x924}, + { 0xe0054, 8, 0x1f, 0xf24}, + { 0xe0074, 49, 0x1f, 0x924}, + { 0xe0138, 1, 0x3, 0x924}, + { 0xe013c, 6, 0x1f, 0x924}, + { 0xe0154, 8, 0x1f, 0xfff}, + { 0xe0174, 21, 0x1f, 0x924}, + { 0xe01d8, 2, 0x1f, 0x1fff}, + { 0xe01e4, 1, 0x1f, 0x1fff}, + { 0xe01f4, 1, 0x4, 0x924}, + { 0xe01f8, 1, 0x1c, 0x924}, + { 0xe0200, 1, 0x1f, 0x924}, + { 0xe0204, 1, 0x1f, 0xfff}, + { 0xe020c, 2, 0x1f, 0xfff}, + { 0xe0214, 2, 0x1f, 0x924}, + { 0xe021c, 2, 0x1f, 0xfff}, + { 0xe0224, 2, 0x1f, 0x924}, + { 0xe022c, 18, 0x1e, 0x924}, + { 0xe0280, 1, 0x1f, 0x924}, + { 0xe0300, 1, 0x1f, 0x924}, + { 0xe0400, 1, 0x10, 0x924}, + { 0xe1000, 1, 0x1f, 0x924}, + { 0xe2000, 1, 0x1f, 0xf24}, + { 0xe2004, 255, 0x1f, 0xc00}, + { 0xe2400, 1, 0x1f, 0xe00}, + { 0xe2404, 255, 0x1f, 0xc00}, + { 0xe2800, 1, 0x1f, 0xe00}, + { 0xe2804, 255, 0x1f, 0xc00}, + { 0xe2c00, 1, 0x1f, 0xe00}, + { 0xe2c04, 255, 0x1f, 0xc00}, + { 0xe3000, 1, 0x1f, 0xe00}, + { 0xe3004, 255, 0x1f, 0xc00}, + { 0xe3400, 1, 0x1f, 0xe00}, + { 0xe3404, 255, 0x1f, 0xc00}, + { 0xe3800, 1, 0x1f, 0xe00}, + { 0xe3804, 255, 0x1f, 0xc00}, + { 0xe3c00, 1, 0x1f, 0xe00}, + { 0xe3c04, 255, 0x1f, 0xc00}, + { 0xf0000, 1, 0x1f, 0xf24}, + { 0xf0004, 255, 0x1f, 0xc00}, + { 0xf0400, 1, 0x1f, 0xe00}, + { 0xf0404, 255, 0x1f, 0xc00}, + { 0xf0800, 1, 0x1f, 0xe00}, + { 0xf0804, 255, 0x1f, 0xc00}, + { 0xf0c00, 1, 0x1f, 0xe00}, + { 0xf0c04, 255, 0x1f, 0xc00}, + { 0xf1000, 1, 0x1f, 0xe00}, + { 0xf1004, 255, 0x1f, 0xc00}, + { 0xf1400, 1, 0x1f, 0xe00}, + { 0xf1404, 255, 0x1f, 0xc00}, + { 0xf1800, 1, 0x1f, 0xe00}, + { 0xf1804, 255, 0x1f, 0xc00}, + { 0xf1c00, 1, 0x1f, 0xe00}, + { 0xf1c04, 255, 0x1f, 0xc00}, + { 0xf2000, 1, 0x1f, 0xe00}, + { 0xf2004, 255, 0x1f, 0xc00}, + { 0xf2400, 1, 0x1f, 0xe00}, + { 0xf2404, 255, 0x1f, 0xc00}, + { 0xf2800, 1, 0x1f, 0xe00}, + { 0xf2804, 255, 0x1f, 0xc00}, + { 0xf2c00, 1, 0x1f, 0xe00}, + { 0xf2c04, 255, 0x1f, 0xc00}, + { 0xf3000, 1, 0x1f, 0xe00}, + { 0xf3004, 255, 0x1f, 0xc00}, + { 0xf3400, 1, 0x1f, 0xe00}, + { 0xf3404, 255, 0x1f, 0xc00}, + { 0xf3800, 1, 0x1f, 0xe00}, + { 0xf3804, 255, 0x1f, 0xc00}, + { 0xf3c00, 1, 0x1f, 0xe00}, + { 0xf3c04, 255, 0x1f, 0xc00}, + { 0xf4000, 1, 0x1f, 0xe00}, + { 0xf4004, 255, 0x1f, 0xc00}, + { 0xf4400, 1, 0x1f, 0xe00}, + { 0xf4404, 255, 0x1f, 0xc00}, + { 0xf4800, 1, 0x1f, 0xe00}, + { 0xf4804, 255, 0x1f, 0xc00}, + { 0xf4c00, 1, 0x1f, 0xe00}, + { 0xf4c04, 255, 0x1f, 0xc00}, + { 0xf5000, 1, 0x1f, 0xe00}, + { 0xf5004, 255, 0x1f, 0xc00}, + { 0xf5400, 1, 0x1f, 0xe00}, + { 0xf5404, 255, 0x1f, 0xc00}, + { 0xf5800, 1, 0x1f, 0xe00}, + { 0xf5804, 255, 0x1f, 0xc00}, + { 0xf5c00, 1, 0x1f, 0xe00}, + { 0xf5c04, 255, 0x1f, 0xc00}, + { 0xf6000, 1, 0x1f, 0xe00}, + { 0xf6004, 255, 0x1f, 0xc00}, + { 0xf6400, 1, 0x1f, 0xe00}, + { 0xf6404, 255, 0x1f, 0xc00}, + { 0xf6800, 1, 0x1f, 0xe00}, + { 0xf6804, 255, 0x1f, 0xc00}, + { 0xf6c00, 1, 0x1f, 0xe00}, + { 0xf6c04, 255, 0x1f, 0xc00}, + { 0xf7000, 1, 0x1f, 0xe00}, + { 0xf7004, 255, 0x1f, 0xc00}, + { 0xf7400, 1, 0x1f, 0xe00}, + { 0xf7404, 255, 0x1f, 0xc00}, + { 0xf7800, 1, 0x1f, 0xe00}, + { 0xf7804, 255, 0x1f, 0xc00}, + { 0xf7c00, 1, 0x1f, 0xe00}, + { 0xf7c04, 255, 0x1f, 0xc00}, + { 0xf8000, 1, 0x1f, 0xe00}, + { 0xf8004, 255, 0x1f, 0xc00}, + { 0xf8400, 1, 0x1f, 0xe00}, + { 0xf8404, 255, 0x1f, 0xc00}, + { 0xf8800, 1, 0x1f, 0xe00}, + { 0xf8804, 255, 0x1f, 0xc00}, + { 0xf8c00, 1, 0x1f, 0xe00}, + { 0xf8c04, 255, 0x1f, 0xc00}, + { 0xf9000, 1, 0x1f, 0xe00}, + { 0xf9004, 255, 0x1f, 0xc00}, + { 0xf9400, 1, 0x1f, 0xe00}, + { 0xf9404, 255, 0x1f, 0xc00}, + { 0xf9800, 1, 0x1f, 0xe00}, + { 0xf9804, 255, 0x1f, 0xc00}, + { 0xf9c00, 1, 0x1f, 0xe00}, + { 0xf9c04, 255, 0x1f, 0xc00}, + { 0xfa000, 1, 0x1f, 0xe00}, + { 0xfa004, 255, 0x1f, 0xc00}, + { 0xfa400, 1, 0x1f, 0xe00}, + { 0xfa404, 255, 0x1f, 0xc00}, + { 0xfa800, 1, 0x1f, 0xe00}, + { 0xfa804, 255, 0x1f, 0xc00}, + { 0xfac00, 1, 0x1f, 0xe00}, + { 0xfac04, 255, 0x1f, 0xc00}, + { 0xfb000, 1, 0x1f, 0xe00}, + { 0xfb004, 255, 0x1f, 0xc00}, + { 0xfb400, 1, 0x1f, 0xe00}, + { 0xfb404, 255, 0x1f, 0xc00}, + { 0xfb800, 1, 0x1f, 0xe00}, + { 0xfb804, 255, 0x1f, 0xc00}, + { 0xfbc00, 1, 0x1f, 0xe00}, + { 0xfbc04, 255, 0x1f, 0xc00}, + { 0xfc000, 1, 0x1f, 0xe00}, + { 0xfc004, 255, 0x1f, 0xc00}, + { 0xfc400, 1, 0x1f, 0xe00}, + { 0xfc404, 255, 0x1f, 0xc00}, + { 0xfc800, 1, 0x1f, 0xe00}, + { 0xfc804, 255, 0x1f, 0xc00}, + { 0xfcc00, 1, 0x1f, 0xe00}, + { 0xfcc04, 255, 0x1f, 0xc00}, + { 0xfd000, 1, 0x1f, 0xe00}, + { 0xfd004, 255, 0x1f, 0xc00}, + { 0xfd400, 1, 0x1f, 0xe00}, + { 0xfd404, 255, 0x1f, 0xc00}, + { 0xfd800, 1, 0x1f, 0xe00}, + { 0xfd804, 255, 0x1f, 0xc00}, + { 0xfdc00, 1, 0x1f, 0xe00}, + { 0xfdc04, 255, 0x1f, 0xc00}, + { 0xfe000, 1, 0x1f, 0xe00}, + { 0xfe004, 255, 0x1f, 0xc00}, + { 0xfe400, 1, 0x1f, 0xe00}, + { 0xfe404, 255, 0x1f, 0xc00}, + { 0xfe800, 1, 0x1f, 0xe00}, + { 0xfe804, 255, 0x1f, 0xc00}, + { 0xfec00, 1, 0x1f, 0xe00}, + { 0xfec04, 255, 0x1f, 0xc00}, + { 0xff000, 1, 0x1f, 0xe00}, + { 0xff004, 255, 0x1f, 0xc00}, + { 0xff400, 1, 0x1f, 0xe00}, + { 0xff404, 255, 0x1f, 0xc00}, + { 0xff800, 1, 0x1f, 0xe00}, + { 0xff804, 255, 0x1f, 0xc00}, + { 0xffc00, 1, 0x1f, 0xe00}, + { 0xffc04, 255, 0x1f, 0xc00}, + { 0x101000, 5, 0x1f, 0x924}, + { 0x101014, 1, 0x1f, 0xfff}, + { 0x101018, 6, 0x1f, 0x924}, + { 0x101040, 2, 0x1f, 0x1fff}, + { 0x10104c, 1, 0x1f, 0x1fff}, + { 0x101050, 1, 0x1e, 0x924}, + { 0x101054, 3, 0x1c, 0x924}, + { 0x101100, 1, 0x1f, 0x924}, + { 0x101800, 8, 0x1f, 0x924}, + { 0x102000, 18, 0x1f, 0x924}, + { 0x102058, 2, 0x1f, 0x1fff}, + { 0x102064, 1, 0x1f, 0x1fff}, + { 0x102068, 6, 0x1c, 0x924}, + { 0x102080, 16, 0x1f, 0xfff}, + { 0x1020c0, 1, 0x1f, 0x924}, + { 0x1020c8, 8, 0x2, 0x924}, + { 0x1020e8, 9, 0x1c, 0x924}, + { 0x102400, 1, 0x1f, 0x924}, + { 0x103000, 1, 0x1f, 0x924}, + { 0x103004, 2, 0x1f, 0xfff}, + { 0x10300c, 23, 0x1f, 0x924}, + { 0x103088, 2, 0x1f, 0x1fff}, + { 0x103094, 1, 0x1f, 0x1fff}, + { 0x103098, 1, 0x1e, 0x924}, + { 0x10309c, 2, 0x1e, 0xfff}, + { 0x1030a4, 2, 0x1e, 0x924}, + { 0x1030ac, 2, 0x1c, 0x924}, + { 0x1030b4, 1, 0x4, 0x924}, + { 0x1030b8, 2, 0x1c, 0xfff}, + { 0x1030c0, 3, 0x1c, 0x924}, + { 0x1030cc, 1, 0x1c, 0xfff}, + { 0x1030d0, 1, 0x1c, 0x924}, + { 0x1030d8, 2, 0x1c, 0x924}, + { 0x1030e0, 1, 0x1c, 0xfff}, + { 0x1030e4, 5, 0x1c, 0x924}, + { 0x103400, 136, 0x1c, 0x1fff}, + { 0x103800, 8, 0x1f, 0x924}, + { 0x104000, 1, 0x1f, 0x924}, + { 0x104004, 1, 0x1f, 0xfff}, + { 0x104008, 4, 0x1f, 0x924}, + { 0x104018, 1, 0x1f, 0xfff}, + { 0x10401c, 1, 0x1f, 0x924}, + { 0x104020, 1, 0x1f, 0xfff}, + { 0x104024, 6, 0x1f, 0x924}, + { 0x10403c, 1, 0x1f, 0xfff}, + { 0x104040, 47, 0x1f, 0x924}, + { 0x10410c, 2, 0x1f, 0x1fff}, + { 0x104118, 1, 0x1f, 0x1fff}, + { 0x10411c, 16, 0x1c, 0x924}, + { 0x104200, 17, 0x1f, 0x924}, + { 0x104400, 1, 0x1f, 0x1fff}, + { 0x104404, 63, 0x1f, 0xfff}, + { 0x104500, 192, 0x1f, 0xdb6}, + { 0x104800, 1, 0x1f, 0x1fff}, + { 0x104804, 63, 0x1f, 0xfff}, + { 0x104900, 192, 0x1f, 0xdb6}, + { 0x105000, 4, 0x1f, 0x1fff}, + { 0x105010, 252, 0x1f, 0xfff}, + { 0x105400, 768, 0x1f, 0xdb6}, + { 0x107000, 7, 0x1c, 0x924}, + { 0x10701c, 1, 0x18, 0x924}, + { 0x108000, 33, 0x3, 0x924}, + { 0x1080ac, 5, 0x2, 0x924}, + { 0x108100, 5, 0x3, 0x924}, + { 0x108120, 5, 0x3, 0x924}, + { 0x108200, 74, 0x3, 0x924}, + { 0x108400, 74, 0x3, 0x924}, + { 0x108800, 152, 0x3, 0x924}, + { 0x110000, 111, 0x1c, 0x924}, + { 0x1101cc, 2, 0x1c, 0x1fff}, + { 0x1101d8, 1, 0x1c, 0x1fff}, + { 0x1101dc, 1, 0x18, 0x924}, + { 0x110200, 4, 0x1c, 0x924}, + { 0x120000, 92, 0x1f, 0x924}, + { 0x120170, 2, 0x3, 0x924}, + { 0x120178, 14, 0x1f, 0x924}, + { 0x1201b0, 2, 0x1f, 0xfff}, + { 0x1201b8, 93, 0x1f, 0x924}, + { 0x12032c, 1, 0x1f, 0xfff}, + { 0x120330, 15, 0x1f, 0x924}, + { 0x12036c, 3, 0x1f, 0xfff}, + { 0x120378, 36, 0x1f, 0x924}, + { 0x120408, 2, 0x1f, 0xfff}, + { 0x120410, 1, 0x1f, 0x924}, + { 0x120414, 15, 0x1f, 0xfff}, + { 0x120450, 10, 0x1f, 0x924}, + { 0x120478, 2, 0x1f, 0xfff}, + { 0x120480, 43, 0x1f, 0x924}, + { 0x12052c, 1, 0x1f, 0xfff}, + { 0x120530, 5, 0x1f, 0x924}, + { 0x120544, 4, 0x3, 0x924}, + { 0x120554, 4, 0x1f, 0x924}, + { 0x120564, 2, 0x1f, 0xfff}, + { 0x12057c, 2, 0x1f, 0x1fff}, + { 0x120588, 3, 0x1f, 0x1fff}, + { 0x120598, 1, 0x1f, 0x1fff}, + { 0x12059c, 22, 0x1e, 0x924}, + { 0x1205f4, 1, 0x6, 0x924}, + { 0x1205f8, 4, 0x1c, 0x924}, + { 0x120618, 1, 0x1c, 0x924}, + { 0x12061c, 31, 0x1e, 0x924}, + { 0x120698, 3, 0x1c, 0x924}, + { 0x1206a4, 1, 0x4, 0x924}, + { 0x1206a8, 1, 0x1c, 0x924}, + { 0x1206b0, 38, 0x1c, 0x924}, + { 0x120748, 1, 0x1c, 0xfff}, + { 0x12074c, 11, 0x1c, 0x924}, + { 0x120778, 2, 0x1c, 0xfff}, + { 0x120780, 23, 0x1c, 0x924}, + { 0x1207dc, 1, 0x4, 0x924}, + { 0x1207fc, 1, 0x1c, 0x924}, + { 0x12080c, 2, 0x1f, 0xfff}, + { 0x120814, 1, 0x1f, 0x924}, + { 0x120818, 1, 0x1f, 0xfff}, + { 0x12081c, 1, 0x1f, 0x924}, + { 0x120820, 1, 0x1f, 0xfff}, + { 0x120824, 1, 0x1f, 0x924}, + { 0x120828, 1, 0x1f, 0xfff}, + { 0x12082c, 1, 0x1f, 0x924}, + { 0x120830, 1, 0x1f, 0xfff}, + { 0x120834, 1, 0x1f, 0x924}, + { 0x120838, 1, 0x1f, 0xfff}, + { 0x12083c, 1, 0x1f, 0x924}, + { 0x120840, 1, 0x1f, 0xfff}, + { 0x120844, 1, 0x1f, 0x924}, + { 0x120848, 1, 0x1f, 0xfff}, + { 0x12084c, 1, 0x1f, 0x924}, + { 0x120850, 1, 0x1f, 0xfff}, + { 0x120854, 1, 0x1f, 0x924}, + { 0x120858, 1, 0x1f, 0xfff}, + { 0x12085c, 1, 0x1f, 0x924}, + { 0x120860, 1, 0x1f, 0xfff}, + { 0x120864, 1, 0x1f, 0x924}, + { 0x120868, 1, 0x1f, 0xfff}, + { 0x12086c, 1, 0x1f, 0x924}, + { 0x120870, 1, 0x1f, 0xfff}, + { 0x120874, 1, 0x1f, 0x924}, + { 0x120878, 1, 0x1f, 0xfff}, + { 0x12087c, 1, 0x1f, 0x924}, + { 0x120880, 1, 0x1f, 0xfff}, + { 0x120884, 1, 0x1f, 0x924}, + { 0x120888, 1, 0x1f, 0xfff}, + { 0x12088c, 1, 0x1f, 0x924}, + { 0x120890, 1, 0x1f, 0xfff}, + { 0x120894, 1, 0x1f, 0x924}, + { 0x120898, 1, 0x1f, 0xfff}, + { 0x12089c, 1, 0x1f, 0x924}, + { 0x1208a0, 1, 0x1f, 0xfff}, + { 0x1208a4, 1, 0x1f, 0x924}, + { 0x1208a8, 1, 0x1f, 0xfff}, + { 0x1208ac, 1, 0x1f, 0x924}, + { 0x1208b0, 1, 0x1f, 0xfff}, + { 0x1208b4, 1, 0x1f, 0x924}, + { 0x1208b8, 1, 0x1f, 0xfff}, + { 0x1208bc, 1, 0x1f, 0x924}, + { 0x1208c0, 1, 0x1f, 0xfff}, + { 0x1208c4, 1, 0x1f, 0x924}, + { 0x1208c8, 1, 0x1f, 0xfff}, + { 0x1208cc, 1, 0x1f, 0x924}, + { 0x1208d0, 1, 0x1f, 0xfff}, + { 0x1208d4, 1, 0x1f, 0x924}, + { 0x1208d8, 1, 0x1f, 0xfff}, + { 0x1208dc, 1, 0x1f, 0x924}, + { 0x1208e0, 1, 0x1f, 0xfff}, + { 0x1208e4, 1, 0x1f, 0x924}, + { 0x1208e8, 1, 0x1f, 0xfff}, + { 0x1208ec, 1, 0x1f, 0x924}, + { 0x1208f0, 1, 0x1f, 0xfff}, + { 0x1208f4, 1, 0x1f, 0x924}, + { 0x1208f8, 1, 0x1f, 0xfff}, + { 0x1208fc, 1, 0x1f, 0x924}, + { 0x120900, 1, 0x1f, 0xfff}, + { 0x120904, 1, 0x1f, 0x924}, + { 0x120908, 1, 0x1f, 0xfff}, + { 0x12090c, 1, 0x1f, 0x924}, + { 0x120910, 7, 0x1c, 0x924}, + { 0x120930, 9, 0x1c, 0x924}, + { 0x12095c, 37, 0x18, 0x924}, + { 0x120a00, 2, 0x7, 0x924}, + { 0x120b00, 1, 0x18, 0x924}, + { 0x122000, 2, 0x1f, 0x924}, + { 0x122008, 2046, 0x1, 0x924}, + { 0x128000, 6144, 0x1e, 0x924}, + { 0x130000, 1, 0x1c, 0x1fff}, + { 0x130004, 11, 0x1c, 0x924}, + { 0x130030, 1, 0x1c, 0xfff}, + { 0x130034, 6, 0x1c, 0x924}, + { 0x13004c, 3, 0x1c, 0xfff}, + { 0x130058, 3, 0x1c, 0x924}, + { 0x130064, 2, 0x1c, 0xfff}, + { 0x13006c, 8, 0x1c, 0x924}, + { 0x13009c, 2, 0x1c, 0x1fff}, + { 0x1300a8, 1, 0x1c, 0x1fff}, + { 0x130100, 12, 0x1c, 0x924}, + { 0x130130, 1, 0x1c, 0xfff}, + { 0x130134, 14, 0x1c, 0x924}, + { 0x13016c, 1, 0x1c, 0xfff}, + { 0x130170, 1, 0x1c, 0x924}, + { 0x130180, 1, 0x1c, 0x924}, + { 0x130200, 1, 0x1c, 0x924}, + { 0x130280, 1, 0x1c, 0x924}, + { 0x130300, 1, 0x1c, 0xfff}, + { 0x130304, 4, 0x1c, 0x924}, + { 0x130380, 1, 0x1c, 0x924}, + { 0x130400, 1, 0x1c, 0x924}, + { 0x130480, 1, 0x1c, 0xfff}, + { 0x130484, 4, 0x1c, 0x924}, + { 0x130800, 72, 0x1c, 0x924}, + { 0x131000, 136, 0x1c, 0x924}, + { 0x132000, 148, 0x1c, 0x924}, + { 0x134000, 544, 0x1c, 0x924}, + { 0x140000, 1, 0x1f, 0x924}, + { 0x140004, 9, 0xf, 0x924}, + { 0x140028, 8, 0x1f, 0x924}, + { 0x140048, 5, 0xf, 0x924}, + { 0x14005c, 2, 0xf, 0xfff}, + { 0x140064, 3, 0xf, 0x924}, + { 0x140070, 1, 0x1f, 0x924}, + { 0x140074, 10, 0xf, 0x924}, + { 0x14009c, 1, 0x1f, 0x924}, + { 0x1400a0, 5, 0xf, 0x924}, + { 0x1400b4, 7, 0x1f, 0x924}, + { 0x1400d0, 2, 0xf, 0xfff}, + { 0x1400d8, 2, 0xf, 0x924}, + { 0x1400e0, 1, 0xf, 0xfff}, + { 0x1400e4, 5, 0xf, 0x924}, + { 0x1400f8, 2, 0x1f, 0x924}, + { 0x140100, 5, 0x3, 0x924}, + { 0x140114, 5, 0xf, 0x924}, + { 0x140128, 7, 0x1f, 0x924}, + { 0x140144, 9, 0xf, 0x924}, + { 0x140168, 8, 0x1f, 0x924}, + { 0x140188, 3, 0xf, 0x924}, + { 0x140194, 13, 0x1f, 0x924}, + { 0x1401d8, 2, 0x1f, 0x1fff}, + { 0x1401e4, 1, 0x1f, 0x1fff}, + { 0x140200, 6, 0xf, 0xfff}, + { 0x1402e0, 2, 0xc, 0x924}, + { 0x1402e8, 2, 0x1c, 0x924}, + { 0x1402f0, 9, 0xc, 0x924}, + { 0x140314, 9, 0x10, 0x924}, + { 0x140338, 7, 0x10, 0xfff}, + { 0x140354, 7, 0x10, 0x924}, + { 0x140370, 7, 0x10, 0xfff}, + { 0x14038c, 14, 0x10, 0x924}, + { 0x1404b0, 14, 0x10, 0x924}, + { 0x15c000, 2, 0x1e, 0x924}, + { 0x15c008, 5, 0x2, 0x924}, + { 0x15c020, 8, 0x1c, 0x924}, + { 0x15c040, 1, 0xc, 0x924}, + { 0x15c044, 2, 0x1c, 0x924}, + { 0x15c04c, 8, 0xc, 0x924}, + { 0x15c06c, 8, 0x1c, 0x924}, + { 0x15c090, 13, 0x1c, 0x924}, + { 0x15c0c8, 24, 0x1c, 0x924}, + { 0x15c128, 2, 0xc, 0x924}, + { 0x15c130, 1, 0x1c, 0x924}, + { 0x15c138, 6, 0x1c, 0x924}, + { 0x15c150, 2, 0x18, 0x924}, + { 0x15c158, 2, 0x8, 0x924}, + { 0x15c160, 23, 0x10, 0x924}, + { 0x15c1bc, 6, 0x10, 0xfff}, + { 0x15c1d4, 23, 0x10, 0x924}, + { 0x15c230, 7, 0x10, 0xfff}, + { 0x15c24c, 90, 0x10, 0x924}, + { 0x160004, 6, 0x18, 0x924}, + { 0x16003c, 1, 0x10, 0x924}, + { 0x160040, 6, 0x18, 0x924}, + { 0x16005c, 6, 0x18, 0x924}, + { 0x160074, 1, 0x10, 0x924}, + { 0x160078, 2, 0x18, 0x924}, + { 0x160300, 8, 0x18, 0x924}, + { 0x160330, 6, 0x18, 0x924}, + { 0x160404, 6, 0x18, 0x924}, + { 0x16043c, 1, 0x10, 0x924}, + { 0x160440, 6, 0x18, 0x924}, + { 0x16045c, 6, 0x18, 0x924}, + { 0x160474, 1, 0x10, 0x924}, + { 0x160478, 2, 0x18, 0x924}, + { 0x160700, 8, 0x18, 0x924}, + { 0x160730, 6, 0x18, 0x924}, + { 0x161000, 7, 0x1f, 0x924}, + { 0x16102c, 2, 0x1f, 0x1fff}, + { 0x161038, 1, 0x1f, 0x1fff}, + { 0x16103c, 2, 0x1c, 0x924}, + { 0x161800, 2, 0x1f, 0x924}, + { 0x162000, 54, 0x18, 0x924}, + { 0x162200, 60, 0x18, 0x924}, + { 0x162400, 54, 0x18, 0x924}, + { 0x162600, 60, 0x18, 0x924}, + { 0x162800, 54, 0x18, 0x924}, + { 0x162a00, 60, 0x18, 0x924}, + { 0x162c00, 54, 0x18, 0x924}, + { 0x162e00, 60, 0x18, 0x924}, + { 0x163000, 1, 0x18, 0x924}, + { 0x163008, 1, 0x18, 0x924}, + { 0x163010, 1, 0x18, 0x924}, + { 0x163018, 1, 0x18, 0x924}, + { 0x163020, 5, 0x18, 0x924}, + { 0x163038, 3, 0x18, 0x924}, + { 0x163048, 3, 0x18, 0x924}, + { 0x163058, 1, 0x18, 0x924}, + { 0x163060, 1, 0x18, 0x924}, + { 0x163068, 1, 0x18, 0x924}, + { 0x163070, 3, 0x18, 0x924}, + { 0x163080, 1, 0x18, 0x924}, + { 0x163088, 3, 0x18, 0x924}, + { 0x163098, 1, 0x18, 0x924}, + { 0x1630a0, 1, 0x18, 0x924}, + { 0x1630a8, 1, 0x18, 0x924}, + { 0x1630b0, 2, 0x10, 0x924}, + { 0x1630c0, 1, 0x18, 0x924}, + { 0x1630c8, 1, 0x18, 0x924}, + { 0x1630d0, 1, 0x18, 0x924}, + { 0x1630d8, 1, 0x18, 0x924}, + { 0x1630e0, 2, 0x18, 0x924}, + { 0x163110, 1, 0x18, 0x924}, + { 0x163120, 2, 0x18, 0x924}, + { 0x163420, 4, 0x18, 0x924}, + { 0x163438, 2, 0x18, 0x924}, + { 0x163488, 2, 0x18, 0x924}, + { 0x163520, 2, 0x18, 0x924}, + { 0x163800, 1, 0x18, 0x924}, + { 0x163808, 1, 0x18, 0x924}, + { 0x163810, 1, 0x18, 0x924}, + { 0x163818, 1, 0x18, 0x924}, + { 0x163820, 5, 0x18, 0x924}, + { 0x163838, 3, 0x18, 0x924}, + { 0x163848, 3, 0x18, 0x924}, + { 0x163858, 1, 0x18, 0x924}, + { 0x163860, 1, 0x18, 0x924}, + { 0x163868, 1, 0x18, 0x924}, + { 0x163870, 3, 0x18, 0x924}, + { 0x163880, 1, 0x18, 0x924}, + { 0x163888, 3, 0x18, 0x924}, + { 0x163898, 1, 0x18, 0x924}, + { 0x1638a0, 1, 0x18, 0x924}, + { 0x1638a8, 1, 0x18, 0x924}, + { 0x1638b0, 2, 0x10, 0x924}, + { 0x1638c0, 1, 0x18, 0x924}, + { 0x1638c8, 1, 0x18, 0x924}, + { 0x1638d0, 1, 0x18, 0x924}, + { 0x1638d8, 1, 0x18, 0x924}, + { 0x1638e0, 2, 0x18, 0x924}, + { 0x163910, 1, 0x18, 0x924}, + { 0x163920, 2, 0x18, 0x924}, + { 0x163c20, 4, 0x18, 0x924}, + { 0x163c38, 2, 0x18, 0x924}, + { 0x163c88, 2, 0x18, 0x924}, + { 0x163d20, 2, 0x18, 0x924}, + { 0x164000, 5, 0x1f, 0x924}, + { 0x164014, 2, 0x1f, 0xfff}, + { 0x16401c, 53, 0x1f, 0x924}, + { 0x164100, 2, 0x1f, 0x1fff}, + { 0x16410c, 1, 0x1f, 0x1fff}, + { 0x164110, 2, 0x1e, 0x924}, + { 0x164118, 15, 0x1c, 0x924}, + { 0x164200, 1, 0x1f, 0x924}, + { 0x164208, 1, 0x1f, 0x924}, + { 0x164210, 1, 0x1f, 0x924}, + { 0x164218, 1, 0x1f, 0x924}, + { 0x164220, 1, 0x1f, 0x924}, + { 0x164228, 1, 0x1f, 0x924}, + { 0x164230, 1, 0x1f, 0x924}, + { 0x164238, 1, 0x1f, 0x924}, + { 0x164240, 1, 0x1f, 0x924}, + { 0x164248, 1, 0x1f, 0x924}, + { 0x164250, 1, 0x1f, 0x924}, + { 0x164258, 1, 0x1f, 0x924}, + { 0x164260, 1, 0x1f, 0x924}, + { 0x164270, 2, 0x1f, 0x924}, + { 0x164280, 2, 0x1f, 0x924}, + { 0x164800, 2, 0x1f, 0x924}, + { 0x165000, 2, 0x1f, 0x924}, + { 0x166000, 164, 0x1f, 0x924}, + { 0x1662b0, 2, 0x1f, 0x1fff}, + { 0x1662bc, 1, 0x1f, 0x1fff}, + { 0x1662cc, 7, 0x1c, 0x924}, + { 0x166400, 49, 0x1f, 0x924}, + { 0x1664c8, 32, 0x1f, 0x924}, + { 0x166548, 1, 0x1f, 0xfff}, + { 0x16654c, 1, 0x1f, 0x924}, + { 0x166550, 1, 0x1f, 0xfff}, + { 0x166554, 1, 0x1f, 0x924}, + { 0x166558, 1, 0x1f, 0xfff}, + { 0x16655c, 1, 0x1f, 0x924}, + { 0x166568, 2, 0x1f, 0x924}, + { 0x166570, 5, 0x1c, 0x924}, + { 0x166800, 1, 0x1f, 0x924}, + { 0x168000, 1, 0x1f, 0xfff}, + { 0x168004, 1, 0x1f, 0x924}, + { 0x168008, 1, 0x1f, 0xfff}, + { 0x16800c, 1, 0x1f, 0x924}, + { 0x168010, 1, 0x1f, 0xfff}, + { 0x168014, 1, 0x1f, 0x924}, + { 0x168018, 1, 0x1f, 0xfff}, + { 0x16801c, 3, 0x1f, 0x924}, + { 0x168028, 2, 0x1f, 0xfff}, + { 0x168030, 10, 0x1f, 0x924}, + { 0x168058, 9, 0x1f, 0xfff}, + { 0x16807c, 106, 0x1f, 0x924}, + { 0x168224, 2, 0x3, 0x924}, + { 0x16822c, 3, 0x1f, 0x924}, + { 0x168238, 1, 0x1f, 0xfff}, + { 0x16823c, 25, 0x1f, 0x924}, + { 0x1682a0, 12, 0x3, 0x924}, + { 0x1682d0, 7, 0x1f, 0xfff}, + { 0x1682ec, 5, 0x1f, 0x924}, + { 0x168300, 2, 0x3, 0xfff}, + { 0x168308, 65, 0x1f, 0xfff}, + { 0x16840c, 1, 0x1f, 0x924}, + { 0x168410, 2, 0x1f, 0xfff}, + { 0x168418, 2, 0x3, 0x924}, + { 0x168420, 6, 0x1f, 0x924}, + { 0x168448, 2, 0x1f, 0x1fff}, + { 0x168454, 1, 0x1f, 0x1fff}, + { 0x168800, 19, 0x1f, 0x924}, + { 0x168900, 1, 0x1f, 0x924}, + { 0x168a00, 128, 0x1f, 0xfff}, + { 0x16a000, 1536, 0x1f, 0x924}, + { 0x16c000, 1536, 0x1f, 0x924}, + { 0x16e000, 16, 0x2, 0x924}, + { 0x16e040, 8, 0x1c, 0x924}, + { 0x16e100, 1, 0x2, 0x924}, + { 0x16e200, 2, 0x2, 0xfff}, + { 0x16e400, 1, 0x2, 0x924}, + { 0x16e404, 2, 0x2, 0xfff}, + { 0x16e40c, 94, 0x2, 0x924}, + { 0x16e584, 64, 0x2, 0xfff}, + { 0x16e684, 2, 0x1e, 0xfff}, + { 0x16e68c, 4, 0x2, 0xfff}, + { 0x16e69c, 8, 0x2, 0x924}, + { 0x16e6bc, 4, 0x1e, 0x924}, + { 0x16e6cc, 4, 0x2, 0x924}, + { 0x16e6e0, 2, 0x1c, 0x924}, + { 0x16e6e8, 5, 0xc, 0x924}, + { 0x16e6fc, 4, 0x1c, 0xfff}, + { 0x16e70c, 1, 0x1c, 0x924}, + { 0x16e768, 17, 0x1c, 0x924}, + { 0x16e7ac, 12, 0x10, 0xfff}, + { 0x170000, 24, 0x1f, 0x924}, + { 0x170060, 4, 0x3, 0x924}, + { 0x170070, 13, 0x1f, 0x924}, + { 0x1700a4, 1, 0x1f, 0xfff}, + { 0x1700a8, 1, 0x1f, 0x924}, + { 0x1700ac, 2, 0x1f, 0xfff}, + { 0x1700b4, 3, 0x1f, 0x924}, + { 0x1700c0, 1, 0x1f, 0xfff}, + { 0x1700c4, 44, 0x1f, 0x924}, + { 0x170184, 2, 0x1f, 0x1fff}, + { 0x170190, 1, 0x1f, 0x1fff}, + { 0x170194, 11, 0x1c, 0x924}, + { 0x1701c4, 1, 0x1c, 0x924}, + { 0x1701cc, 7, 0x1c, 0x924}, + { 0x1701e8, 1, 0x18, 0x924}, + { 0x1701ec, 1, 0x1c, 0x924}, + { 0x1701f4, 1, 0x1c, 0x924}, + { 0x170200, 4, 0x1f, 0x924}, + { 0x170214, 1, 0x1f, 0x924}, + { 0x170218, 77, 0x1c, 0x924}, + { 0x170400, 64, 0x1c, 0x924}, + { 0x178000, 1, 0x1f, 0x924}, + { 0x180000, 61, 0x1f, 0x924}, + { 0x180114, 2, 0x1f, 0x1fff}, + { 0x180120, 3, 0x1f, 0x1fff}, + { 0x180130, 1, 0x1f, 0x1fff}, + { 0x18013c, 2, 0x1e, 0x924}, + { 0x180200, 27, 0x1f, 0x924}, + { 0x18026c, 1, 0x1f, 0xfff}, + { 0x180270, 12, 0x1f, 0x924}, + { 0x1802a0, 1, 0x1f, 0xfff}, + { 0x1802a4, 17, 0x1f, 0x924}, + { 0x180340, 4, 0x1f, 0x924}, + { 0x180380, 1, 0x1c, 0x924}, + { 0x180388, 1, 0x1c, 0x924}, + { 0x180390, 1, 0x1c, 0x924}, + { 0x180398, 1, 0x1c, 0x924}, + { 0x1803a0, 5, 0x1c, 0x924}, + { 0x1803b4, 2, 0x18, 0x924}, + { 0x181000, 4, 0x1f, 0x93c}, + { 0x181010, 1020, 0x1f, 0x38}, + { 0x182000, 4, 0x18, 0x924}, + { 0x1a0000, 1, 0x1f, 0x92c}, + { 0x1a0004, 5631, 0x1f, 0x8}, + { 0x1a5800, 2560, 0x1e, 0x8}, + { 0x1a8000, 1, 0x1f, 0x92c}, + { 0x1a8004, 8191, 0x1e, 0x8}, + { 0x1b0000, 1, 0x1f, 0x92c}, + { 0x1b0004, 15, 0x2, 0x8}, + { 0x1b0040, 1, 0x1e, 0x92c}, + { 0x1b0044, 239, 0x2, 0x8}, + { 0x1b0400, 1, 0x1f, 0x92c}, + { 0x1b0404, 255, 0x2, 0x8}, + { 0x1b0800, 1, 0x1f, 0x924}, + { 0x1b0840, 1, 0x1e, 0x924}, + { 0x1b0c00, 1, 0x1f, 0x1fff}, + { 0x1b1000, 1, 0x1f, 0x1fff}, + { 0x1b1040, 1, 0x1e, 0x1fff}, + { 0x1b1400, 1, 0x1f, 0x924}, + { 0x1b1440, 1, 0x1e, 0x924}, + { 0x1b1480, 1, 0x1e, 0x924}, + { 0x1b14c0, 1, 0x1e, 0x924}, + { 0x1b1800, 128, 0x1f, 0x10}, + { 0x1b1c00, 128, 0x1f, 0x10}, + { 0x1b2000, 1, 0x1f, 0xdb6}, + { 0x1b2400, 1, 0x1e, 0x92c}, + { 0x1b2404, 5631, 0x1c, 0x8}, + { 0x1b8000, 1, 0x1f, 0xfff}, + { 0x1b8040, 1, 0x1f, 0xfff}, + { 0x1b8080, 1, 0x1f, 0xfff}, + { 0x1b80c0, 1, 0x1f, 0xfff}, + { 0x1b8100, 1, 0x1f, 0x924}, + { 0x1b8140, 1, 0x1f, 0x924}, + { 0x1b8180, 1, 0x1f, 0x924}, + { 0x1b81c0, 1, 0x1f, 0x924}, + { 0x1b8200, 1, 0x1f, 0x924}, + { 0x1b8240, 1, 0x1f, 0x924}, + { 0x1b8280, 1, 0x1f, 0x924}, + { 0x1b82c0, 1, 0x1f, 0x924}, + { 0x1b8300, 1, 0x1f, 0x924}, + { 0x1b8340, 1, 0x1f, 0x924}, + { 0x1b8380, 1, 0x1f, 0x924}, + { 0x1b83c0, 1, 0x1f, 0x924}, + { 0x1b8400, 1, 0x1f, 0x924}, + { 0x1b8440, 1, 0x1f, 0x924}, + { 0x1b8480, 1, 0x1f, 0x924}, + { 0x1b84c0, 1, 0x1f, 0x924}, + { 0x1b8500, 1, 0x1f, 0x924}, + { 0x1b8540, 1, 0x1f, 0x924}, + { 0x1b8580, 1, 0x1f, 0x924}, + { 0x1b85c0, 19, 0x1c, 0x924}, + { 0x1b8800, 1, 0x1f, 0x924}, + { 0x1b8840, 1, 0x1f, 0x924}, + { 0x1b8880, 1, 0x1f, 0x924}, + { 0x1b88c0, 1, 0x1f, 0x924}, + { 0x1b8900, 1, 0x1f, 0x924}, + { 0x1b8940, 1, 0x1f, 0x924}, + { 0x1b8980, 1, 0x1f, 0x924}, + { 0x1b89c0, 1, 0x1f, 0x924}, + { 0x1b8a00, 1, 0x1f, 0x934}, + { 0x1b8a40, 1, 0x1f, 0x924}, + { 0x1b8a80, 1, 0x1f, 0x492}, + { 0x1b8ac0, 1, 0x1f, 0x924}, + { 0x1b8b00, 1, 0x1f, 0x924}, + { 0x1b8b40, 1, 0x1f, 0x924}, + { 0x1b8b80, 1, 0x1f, 0x924}, + { 0x1b8bc0, 1, 0x1f, 0x924}, + { 0x1b8c00, 1, 0x1f, 0x924}, + { 0x1b8c40, 1, 0x1f, 0x924}, + { 0x1b8c80, 1, 0x1f, 0x924}, + { 0x1b8cc0, 1, 0x1f, 0x924}, + { 0x1b8cc4, 1, 0x1c, 0x924}, + { 0x1b8d00, 1, 0x1f, 0x924}, + { 0x1b8d40, 1, 0x1f, 0x924}, + { 0x1b8d80, 1, 0x1f, 0x924}, + { 0x1b8dc0, 1, 0x1f, 0x924}, + { 0x1b8e00, 1, 0x1f, 0x924}, + { 0x1b8e40, 1, 0x1f, 0x924}, + { 0x1b8e80, 1, 0x1f, 0x924}, + { 0x1b8e84, 1, 0x1c, 0x924}, + { 0x1b8ec0, 1, 0x1e, 0x924}, + { 0x1b8f00, 1, 0x1e, 0x924}, + { 0x1b8f40, 1, 0x1e, 0x924}, + { 0x1b8f80, 1, 0x1e, 0x924}, + { 0x1b8fc0, 1, 0x1e, 0x924}, + { 0x1b8fd4, 5, 0x1c, 0x924}, + { 0x1b8fe8, 2, 0x18, 0x924}, + { 0x1b9000, 1, 0x1c, 0x924}, + { 0x1b9040, 3, 0x1c, 0x924}, + { 0x1b905c, 1, 0x18, 0x924}, + { 0x1b9064, 1, 0x10, 0x924}, + { 0x1b9080, 10, 0x10, 0x924}, + { 0x1c0000, 2, 0x1f, 0x924}, + { 0x200000, 65, 0x1f, 0x924}, + { 0x200124, 2, 0x1f, 0x1fff}, + { 0x200130, 3, 0x1f, 0x1fff}, + { 0x200140, 1, 0x1f, 0x1fff}, + { 0x20014c, 2, 0x1e, 0x924}, + { 0x200200, 27, 0x1f, 0x924}, + { 0x20026c, 1, 0x1f, 0xfff}, + { 0x200270, 12, 0x1f, 0x924}, + { 0x2002a0, 1, 0x1f, 0xfff}, + { 0x2002a4, 17, 0x1f, 0x924}, + { 0x200340, 4, 0x1f, 0x924}, + { 0x200380, 1, 0x1c, 0x924}, + { 0x200388, 1, 0x1c, 0x924}, + { 0x200390, 1, 0x1c, 0x924}, + { 0x200398, 1, 0x1c, 0x924}, + { 0x2003a0, 1, 0x1c, 0x924}, + { 0x2003a8, 2, 0x1c, 0x924}, + { 0x202000, 4, 0x1f, 0x1927}, + { 0x202010, 2044, 0x1f, 0x1007}, + { 0x204000, 4, 0x18, 0x924}, + { 0x220000, 1, 0x1f, 0x925}, + { 0x220004, 5631, 0x1f, 0x1}, + { 0x225800, 2560, 0x1e, 0x1}, + { 0x228000, 1, 0x1f, 0x925}, + { 0x228004, 8191, 0x1e, 0x1}, + { 0x230000, 1, 0x1f, 0x925}, + { 0x230004, 15, 0x2, 0x1}, + { 0x230040, 1, 0x1e, 0x925}, + { 0x230044, 239, 0x2, 0x1}, + { 0x230400, 1, 0x1f, 0x925}, + { 0x230404, 255, 0x2, 0x1}, + { 0x230800, 1, 0x1f, 0x924}, + { 0x230840, 1, 0x1e, 0x924}, + { 0x230c00, 1, 0x1f, 0x924}, + { 0x231000, 1, 0x1f, 0x924}, + { 0x231040, 1, 0x1e, 0x924}, + { 0x231400, 1, 0x1f, 0x924}, + { 0x231440, 1, 0x1e, 0x924}, + { 0x231480, 1, 0x1e, 0x924}, + { 0x2314c0, 1, 0x1e, 0x924}, + { 0x231800, 128, 0x1f, 0x2}, + { 0x231c00, 128, 0x1f, 0x2}, + { 0x232000, 1, 0x1f, 0xdb6}, + { 0x232400, 1, 0x1e, 0x925}, + { 0x232404, 5631, 0x1c, 0x1}, + { 0x238000, 1, 0x1f, 0xfff}, + { 0x238040, 1, 0x1f, 0xfff}, + { 0x238080, 1, 0x1f, 0xfff}, + { 0x2380c0, 1, 0x1f, 0xfff}, + { 0x238100, 1, 0x1f, 0x924}, + { 0x238140, 1, 0x1f, 0x924}, + { 0x238180, 1, 0x1f, 0x924}, + { 0x2381c0, 1, 0x1f, 0x924}, + { 0x238200, 1, 0x1f, 0x924}, + { 0x238240, 1, 0x1f, 0x924}, + { 0x238280, 1, 0x1f, 0x924}, + { 0x2382c0, 1, 0x1f, 0x924}, + { 0x238300, 1, 0x1f, 0x924}, + { 0x238340, 1, 0x1f, 0x924}, + { 0x238380, 1, 0x1f, 0x924}, + { 0x2383c0, 1, 0x1f, 0x924}, + { 0x238400, 1, 0x1f, 0x924}, + { 0x238440, 1, 0x1f, 0x924}, + { 0x238480, 1, 0x1f, 0x924}, + { 0x2384c0, 1, 0x1f, 0x924}, + { 0x238500, 1, 0x1f, 0x924}, + { 0x238540, 1, 0x1f, 0x924}, + { 0x238580, 1, 0x1f, 0x924}, + { 0x2385c0, 19, 0x1c, 0x924}, + { 0x238800, 1, 0x1f, 0x924}, + { 0x238840, 1, 0x1f, 0x924}, + { 0x238880, 1, 0x1f, 0x924}, + { 0x2388c0, 1, 0x1f, 0x924}, + { 0x238900, 1, 0x1f, 0x924}, + { 0x238940, 1, 0x1f, 0x924}, + { 0x238980, 1, 0x1f, 0x924}, + { 0x2389c0, 1, 0x1f, 0x924}, + { 0x238a00, 1, 0x1f, 0x926}, + { 0x238a40, 1, 0x1f, 0x924}, + { 0x238a80, 1, 0x1f, 0x492}, + { 0x238ac0, 1, 0x1f, 0x924}, + { 0x238b00, 1, 0x1f, 0x924}, + { 0x238b40, 1, 0x1f, 0x924}, + { 0x238b80, 1, 0x1f, 0x924}, + { 0x238bc0, 1, 0x1f, 0x924}, + { 0x238c00, 1, 0x1f, 0x924}, + { 0x238c40, 1, 0x1f, 0x924}, + { 0x238c80, 1, 0x1f, 0x924}, + { 0x238cc0, 1, 0x1f, 0x924}, + { 0x238cc4, 1, 0x1c, 0x924}, + { 0x238d00, 1, 0x1f, 0x924}, + { 0x238d40, 1, 0x1f, 0x924}, + { 0x238d80, 1, 0x1f, 0x924}, + { 0x238dc0, 1, 0x1f, 0x924}, + { 0x238e00, 1, 0x1f, 0x924}, + { 0x238e40, 1, 0x1f, 0x924}, + { 0x238e80, 1, 0x1f, 0x924}, + { 0x238e84, 1, 0x1c, 0x924}, + { 0x238ec0, 1, 0x1e, 0x924}, + { 0x238f00, 1, 0x1e, 0x924}, + { 0x238f40, 1, 0x1e, 0x924}, + { 0x238f80, 1, 0x1e, 0x924}, + { 0x238fc0, 1, 0x1e, 0x924}, + { 0x238fd4, 5, 0x1c, 0x924}, + { 0x238fe8, 2, 0x18, 0x924}, + { 0x239000, 1, 0x1c, 0x924}, + { 0x239040, 3, 0x1c, 0x924}, + { 0x23905c, 1, 0x18, 0x924}, + { 0x239064, 1, 0x10, 0x924}, + { 0x239080, 10, 0x10, 0x924}, + { 0x240000, 2, 0x1f, 0x924}, + { 0x280000, 65, 0x1f, 0x924}, + { 0x280124, 2, 0x1f, 0x1fff}, + { 0x280130, 3, 0x1f, 0x1fff}, + { 0x280140, 1, 0x1f, 0x1fff}, + { 0x28014c, 2, 0x1e, 0x924}, + { 0x280200, 27, 0x1f, 0x924}, + { 0x28026c, 1, 0x1f, 0xfff}, + { 0x280270, 12, 0x1f, 0x924}, + { 0x2802a0, 1, 0x1f, 0xfff}, + { 0x2802a4, 17, 0x1f, 0x924}, + { 0x280340, 4, 0x1f, 0x924}, + { 0x280380, 1, 0x1c, 0x924}, + { 0x280388, 1, 0x1c, 0x924}, + { 0x280390, 1, 0x1c, 0x924}, + { 0x280398, 1, 0x1c, 0x924}, + { 0x2803a0, 1, 0x1c, 0x924}, + { 0x2803a8, 2, 0x1c, 0x924}, + { 0x282000, 4, 0x1f, 0x9e4}, + { 0x282010, 2044, 0x1f, 0x1c0}, + { 0x284000, 4, 0x18, 0x924}, + { 0x2a0000, 1, 0x1f, 0x964}, + { 0x2a0004, 5631, 0x1f, 0x40}, + { 0x2a5800, 2560, 0x1e, 0x40}, + { 0x2a8000, 1, 0x1f, 0x964}, + { 0x2a8004, 8191, 0x1e, 0x40}, + { 0x2b0000, 1, 0x1f, 0x964}, + { 0x2b0004, 15, 0x2, 0x40}, + { 0x2b0040, 1, 0x1e, 0x964}, + { 0x2b0044, 239, 0x2, 0x40}, + { 0x2b0400, 1, 0x1f, 0x964}, + { 0x2b0404, 255, 0x2, 0x40}, + { 0x2b0800, 1, 0x1f, 0x924}, + { 0x2b0840, 1, 0x1e, 0x924}, + { 0x2b0c00, 1, 0x1f, 0x924}, + { 0x2b1000, 1, 0x1f, 0x924}, + { 0x2b1040, 1, 0x1e, 0x924}, + { 0x2b1400, 1, 0x1f, 0x924}, + { 0x2b1440, 1, 0x1e, 0x924}, + { 0x2b1480, 1, 0x1e, 0x924}, + { 0x2b14c0, 1, 0x1e, 0x924}, + { 0x2b1800, 128, 0x1f, 0x80}, + { 0x2b1c00, 128, 0x1f, 0x80}, + { 0x2b2000, 1, 0x1f, 0xdb6}, + { 0x2b2400, 1, 0x1e, 0x964}, + { 0x2b2404, 5631, 0x1c, 0x40}, + { 0x2b8000, 1, 0x1f, 0xfff}, + { 0x2b8040, 1, 0x1f, 0xfff}, + { 0x2b8080, 1, 0x1f, 0xfff}, + { 0x2b80c0, 1, 0x1f, 0x924}, + { 0x2b8100, 1, 0x1f, 0x924}, + { 0x2b8140, 1, 0x1f, 0x924}, + { 0x2b8180, 1, 0x1f, 0x924}, + { 0x2b81c0, 1, 0x1f, 0x924}, + { 0x2b8200, 1, 0x1f, 0x924}, + { 0x2b8240, 1, 0x1f, 0x924}, + { 0x2b8280, 1, 0x1f, 0x924}, + { 0x2b82c0, 1, 0x1f, 0x924}, + { 0x2b8300, 1, 0x1f, 0x924}, + { 0x2b8340, 1, 0x1f, 0x924}, + { 0x2b8380, 1, 0x1f, 0x924}, + { 0x2b83c0, 1, 0x1f, 0x924}, + { 0x2b8400, 1, 0x1f, 0x924}, + { 0x2b8440, 1, 0x1f, 0x924}, + { 0x2b8480, 1, 0x1f, 0x924}, + { 0x2b84c0, 1, 0x1f, 0x924}, + { 0x2b8500, 1, 0x1f, 0x924}, + { 0x2b8540, 1, 0x1f, 0x924}, + { 0x2b8580, 1, 0x1f, 0x924}, + { 0x2b85c0, 19, 0x1c, 0x924}, + { 0x2b8800, 1, 0x1f, 0x924}, + { 0x2b8840, 1, 0x1f, 0x924}, + { 0x2b8880, 1, 0x1f, 0x924}, + { 0x2b88c0, 1, 0x1f, 0x924}, + { 0x2b8900, 1, 0x1f, 0x924}, + { 0x2b8940, 1, 0x1f, 0x924}, + { 0x2b8980, 1, 0x1f, 0x924}, + { 0x2b89c0, 1, 0x1f, 0x924}, + { 0x2b8a00, 1, 0x1f, 0x9a4}, + { 0x2b8a40, 1, 0x1f, 0x924}, + { 0x2b8a80, 1, 0x1f, 0x492}, + { 0x2b8ac0, 1, 0x1f, 0x924}, + { 0x2b8b00, 1, 0x1f, 0x924}, + { 0x2b8b40, 1, 0x1f, 0x924}, + { 0x2b8b80, 1, 0x1f, 0x924}, + { 0x2b8bc0, 1, 0x1f, 0x924}, + { 0x2b8c00, 1, 0x1f, 0x924}, + { 0x2b8c40, 1, 0x1f, 0x924}, + { 0x2b8c80, 1, 0x1f, 0x924}, + { 0x2b8cc0, 1, 0x1f, 0x924}, + { 0x2b8cc4, 1, 0x1c, 0x924}, + { 0x2b8d00, 1, 0x1f, 0x924}, + { 0x2b8d40, 1, 0x1f, 0x924}, + { 0x2b8d80, 1, 0x1f, 0x924}, + { 0x2b8dc0, 1, 0x1f, 0x924}, + { 0x2b8e00, 1, 0x1f, 0x924}, + { 0x2b8e40, 1, 0x1f, 0x924}, + { 0x2b8e80, 1, 0x1f, 0x924}, + { 0x2b8e84, 1, 0x1c, 0x924}, + { 0x2b8ec0, 1, 0x1e, 0x924}, + { 0x2b8f00, 1, 0x1e, 0x924}, + { 0x2b8f40, 1, 0x1e, 0x924}, + { 0x2b8f80, 1, 0x1e, 0x924}, + { 0x2b8fc0, 1, 0x1e, 0x924}, + { 0x2b8fd4, 5, 0x1c, 0x924}, + { 0x2b8fe8, 2, 0x18, 0x924}, + { 0x2b9000, 1, 0x1c, 0x924}, + { 0x2b9040, 3, 0x1c, 0x924}, + { 0x2b905c, 1, 0x18, 0x924}, + { 0x2b9064, 1, 0x10, 0x924}, + { 0x2b9080, 10, 0x10, 0x924}, + { 0x2c0000, 2, 0x1f, 0x1fff}, + { 0x300000, 65, 0x1f, 0x924}, + { 0x300124, 2, 0x1f, 0x1fff}, + { 0x300130, 3, 0x1f, 0x1fff}, + { 0x300140, 1, 0x1f, 0x1fff}, + { 0x30014c, 2, 0x1e, 0x924}, + { 0x300200, 27, 0x1f, 0x924}, + { 0x30026c, 1, 0x1f, 0xfff}, + { 0x300270, 12, 0x1f, 0x924}, + { 0x3002a0, 1, 0x1f, 0xfff}, + { 0x3002a4, 17, 0x1f, 0x924}, + { 0x300340, 4, 0x1f, 0x924}, + { 0x300380, 1, 0x1c, 0x924}, + { 0x300388, 1, 0x1c, 0x924}, + { 0x300390, 1, 0x1c, 0x924}, + { 0x300398, 1, 0x1c, 0x924}, + { 0x3003a0, 1, 0x1c, 0x924}, + { 0x3003a8, 2, 0x1c, 0x924}, + { 0x302000, 4, 0x1f, 0xf24}, + { 0x302010, 2044, 0x1f, 0xe00}, + { 0x304000, 4, 0x18, 0x924}, + { 0x320000, 1, 0x1f, 0xb24}, + { 0x320004, 5631, 0x1f, 0x200}, + { 0x325800, 2560, 0x1e, 0x200}, + { 0x328000, 1, 0x1f, 0xb24}, + { 0x328004, 8191, 0x1e, 0x200}, + { 0x330000, 1, 0x1f, 0xb24}, + { 0x330004, 15, 0x2, 0x200}, + { 0x330040, 1, 0x1e, 0xb24}, + { 0x330044, 239, 0x2, 0x200}, + { 0x330400, 1, 0x1f, 0xb24}, + { 0x330404, 255, 0x2, 0x200}, + { 0x330800, 1, 0x1f, 0x924}, + { 0x330840, 1, 0x1e, 0x924}, + { 0x330c00, 1, 0x1f, 0x924}, + { 0x331000, 1, 0x1f, 0x924}, + { 0x331040, 1, 0x1e, 0x924}, + { 0x331400, 1, 0x1f, 0x924}, + { 0x331440, 1, 0x1e, 0x924}, + { 0x331480, 1, 0x1e, 0x924}, + { 0x3314c0, 1, 0x1e, 0x924}, + { 0x331800, 128, 0x1f, 0x400}, + { 0x331c00, 128, 0x1f, 0x400}, + { 0x332000, 1, 0x1f, 0xdb6}, + { 0x332400, 1, 0x1e, 0xb24}, + { 0x332404, 5631, 0x1c, 0x200}, + { 0x338000, 1, 0x1f, 0xfff}, + { 0x338040, 1, 0x1f, 0xfff}, + { 0x338080, 1, 0x1f, 0xfff}, + { 0x3380c0, 1, 0x1f, 0xfff}, + { 0x338100, 1, 0x1f, 0x924}, + { 0x338140, 1, 0x1f, 0x924}, + { 0x338180, 1, 0x1f, 0x924}, + { 0x3381c0, 1, 0x1f, 0x924}, + { 0x338200, 1, 0x1f, 0x924}, + { 0x338240, 1, 0x1f, 0x924}, + { 0x338280, 1, 0x1f, 0x924}, + { 0x3382c0, 1, 0x1f, 0x924}, + { 0x338300, 1, 0x1f, 0x924}, + { 0x338340, 1, 0x1f, 0x924}, + { 0x338380, 1, 0x1f, 0x924}, + { 0x3383c0, 1, 0x1f, 0x924}, + { 0x338400, 1, 0x1f, 0x924}, + { 0x338440, 1, 0x1f, 0x924}, + { 0x338480, 1, 0x1f, 0x924}, + { 0x3384c0, 1, 0x1f, 0x924}, + { 0x338500, 1, 0x1f, 0x924}, + { 0x338540, 1, 0x1f, 0x924}, + { 0x338580, 1, 0x1f, 0x924}, + { 0x3385c0, 19, 0x1c, 0x924}, + { 0x338800, 1, 0x1f, 0x924}, + { 0x338840, 1, 0x1f, 0x924}, + { 0x338880, 1, 0x1f, 0x924}, + { 0x3388c0, 1, 0x1f, 0x924}, + { 0x338900, 1, 0x1f, 0x924}, + { 0x338940, 1, 0x1f, 0x924}, + { 0x338980, 1, 0x1f, 0x924}, + { 0x3389c0, 1, 0x1f, 0x924}, + { 0x338a00, 1, 0x1f, 0xd24}, + { 0x338a40, 1, 0x1f, 0x924}, + { 0x338a80, 1, 0x1f, 0x492}, + { 0x338ac0, 1, 0x1f, 0x924}, + { 0x338b00, 1, 0x1f, 0x924}, + { 0x338b40, 1, 0x1f, 0x924}, + { 0x338b80, 1, 0x1f, 0x924}, + { 0x338bc0, 1, 0x1f, 0x924}, + { 0x338c00, 1, 0x1f, 0x924}, + { 0x338c40, 1, 0x1f, 0x924}, + { 0x338c80, 1, 0x1f, 0x924}, + { 0x338cc0, 1, 0x1f, 0x924}, + { 0x338cc4, 1, 0x1c, 0x924}, + { 0x338d00, 1, 0x1f, 0x924}, + { 0x338d40, 1, 0x1f, 0x924}, + { 0x338d80, 1, 0x1f, 0x924}, + { 0x338dc0, 1, 0x1f, 0x924}, + { 0x338e00, 1, 0x1f, 0x924}, + { 0x338e40, 1, 0x1f, 0x924}, + { 0x338e80, 1, 0x1f, 0x924}, + { 0x338e84, 1, 0x1c, 0x924}, + { 0x338ec0, 1, 0x1e, 0x924}, + { 0x338f00, 1, 0x1e, 0x924}, + { 0x338f40, 1, 0x1e, 0x924}, + { 0x338f80, 1, 0x1e, 0x924}, + { 0x338fc0, 1, 0x1e, 0x924}, + { 0x338fd4, 5, 0x1c, 0x924}, + { 0x338fe8, 2, 0x18, 0x924}, + { 0x339000, 1, 0x1c, 0x924}, + { 0x339040, 3, 0x1c, 0x924}, + { 0x33905c, 1, 0x18, 0x924}, + { 0x339064, 1, 0x10, 0x924}, + { 0x339080, 10, 0x10, 0x924}, + { 0x340000, 2, 0x1f, 0x924}, + { 0x3a0000, 40960, 0x1c, 0x1000} +}; + +#define REGS_COUNT ARRAY_SIZE(reg_addrs) + +static const struct reg_addr idle_reg_addrs[] = { + { 0x2104, 1, 0x1f, 0xfff}, + { 0x2110, 2, 0x1f, 0xfff}, + { 0x211c, 8, 0x1f, 0xfff}, + { 0x2814, 1, 0x1f, 0xfff}, + { 0x281c, 2, 0x1f, 0xfff}, + { 0x2854, 1, 0x1f, 0xfff}, + { 0x285c, 1, 0x1f, 0xfff}, + { 0x3040, 1, 0x1f, 0xfff}, + { 0x9010, 7, 0x1c, 0xfff}, + { 0x9030, 1, 0x1c, 0xfff}, + { 0x9068, 16, 0x1c, 0xfff}, + { 0x9230, 2, 0x1c, 0xfff}, + { 0x9244, 1, 0x1c, 0xfff}, + { 0x9298, 1, 0x1c, 0xfff}, + { 0x92a8, 1, 0x1c, 0x1fff}, + { 0xa38c, 1, 0x1f, 0x1fff}, + { 0xa3c4, 1, 0x1e, 0xfff}, + { 0xa404, 1, 0x1f, 0xfff}, + { 0xa408, 2, 0x1f, 0x1fff}, + { 0xa42c, 12, 0x1f, 0xfff}, + { 0xa580, 1, 0x1f, 0x1fff}, + { 0xa590, 1, 0x1f, 0x1fff}, + { 0xa600, 5, 0x1e, 0xfff}, + { 0xa618, 1, 0x1e, 0xfff}, + { 0xa714, 1, 0x1c, 0xfff}, + { 0xa720, 1, 0x1c, 0xfff}, + { 0xa750, 1, 0x1c, 0xfff}, + { 0xc09c, 1, 0x3, 0xfff}, + { 0x103b0, 1, 0x1f, 0xfff}, + { 0x103c0, 1, 0x1f, 0xfff}, + { 0x103d0, 1, 0x3, 0x1fff}, + { 0x10418, 1, 0x1f, 0xfff}, + { 0x10420, 1, 0x1f, 0xfff}, + { 0x10428, 1, 0x1f, 0xfff}, + { 0x10460, 1, 0x1f, 0xfff}, + { 0x10474, 1, 0x1f, 0xfff}, + { 0x104e0, 1, 0x1f, 0xfff}, + { 0x104ec, 1, 0x1f, 0xfff}, + { 0x104f8, 1, 0x1f, 0xfff}, + { 0x10508, 1, 0x1f, 0xfff}, + { 0x10530, 1, 0x1f, 0xfff}, + { 0x10538, 1, 0x1f, 0xfff}, + { 0x10548, 1, 0x1f, 0xfff}, + { 0x10558, 1, 0x1f, 0xfff}, + { 0x182a8, 1, 0x1c, 0xfff}, + { 0x182b8, 1, 0x1c, 0xfff}, + { 0x18308, 1, 0x1c, 0xfff}, + { 0x18318, 1, 0x1c, 0xfff}, + { 0x18338, 1, 0x1c, 0xfff}, + { 0x18348, 1, 0x1c, 0xfff}, + { 0x183bc, 1, 0x1c, 0x1fff}, + { 0x183cc, 1, 0x1c, 0x1fff}, + { 0x18570, 1, 0x18, 0xfff}, + { 0x18578, 1, 0x18, 0xfff}, + { 0x1858c, 1, 0x18, 0xfff}, + { 0x18594, 1, 0x18, 0xfff}, + { 0x1862c, 4, 0x10, 0xfff}, + { 0x2021c, 11, 0x1f, 0xfff}, + { 0x202a8, 1, 0x1f, 0xfff}, + { 0x202b8, 1, 0x1f, 0x1fff}, + { 0x20404, 1, 0x1f, 0xfff}, + { 0x2040c, 2, 0x1f, 0xfff}, + { 0x2041c, 2, 0x1f, 0xfff}, + { 0x40154, 14, 0x1f, 0xfff}, + { 0x40198, 1, 0x1f, 0x1fff}, + { 0x404ac, 1, 0x1f, 0xfff}, + { 0x404bc, 1, 0x1f, 0x1fff}, + { 0x42290, 1, 0x1f, 0xfff}, + { 0x422a0, 1, 0x1f, 0xfff}, + { 0x422b0, 1, 0x1f, 0x1fff}, + { 0x42548, 1, 0x1f, 0xfff}, + { 0x42550, 1, 0x1f, 0xfff}, + { 0x42558, 1, 0x1f, 0xfff}, + { 0x50160, 8, 0x1f, 0xfff}, + { 0x501d0, 1, 0x1f, 0xfff}, + { 0x501e0, 1, 0x1f, 0x1fff}, + { 0x50204, 1, 0x1f, 0xfff}, + { 0x5020c, 2, 0x1f, 0xfff}, + { 0x5021c, 1, 0x1f, 0xfff}, + { 0x60090, 1, 0x1f, 0xfff}, + { 0x6011c, 1, 0x1f, 0xfff}, + { 0x6012c, 1, 0x1f, 0x1fff}, + { 0xc101c, 1, 0x1f, 0xfff}, + { 0xc102c, 1, 0x1f, 0x1fff}, + { 0xc2290, 1, 0x1f, 0xfff}, + { 0xc22a0, 1, 0x1f, 0xfff}, + { 0xc22b0, 1, 0x1f, 0x1fff}, + { 0xc2548, 1, 0x1f, 0xfff}, + { 0xc2550, 1, 0x1f, 0xfff}, + { 0xc2558, 1, 0x1f, 0xfff}, + { 0xc4294, 1, 0x1f, 0xfff}, + { 0xc42a4, 1, 0x1f, 0xfff}, + { 0xc42b4, 1, 0x1f, 0x1fff}, + { 0xc4550, 1, 0x1f, 0xfff}, + { 0xc4558, 1, 0x1f, 0xfff}, + { 0xc4560, 1, 0x1f, 0xfff}, + { 0xd016c, 8, 0x1f, 0xfff}, + { 0xd01d8, 1, 0x1f, 0xfff}, + { 0xd01e8, 1, 0x1f, 0x1fff}, + { 0xd0204, 1, 0x1f, 0xfff}, + { 0xd020c, 3, 0x1f, 0xfff}, + { 0xe0154, 8, 0x1f, 0xfff}, + { 0xe01c8, 1, 0x1f, 0xfff}, + { 0xe01d8, 1, 0x1f, 0x1fff}, + { 0xe0204, 1, 0x1f, 0xfff}, + { 0xe020c, 2, 0x1f, 0xfff}, + { 0xe021c, 2, 0x1f, 0xfff}, + { 0x101014, 1, 0x1f, 0xfff}, + { 0x101030, 1, 0x1f, 0xfff}, + { 0x101040, 1, 0x1f, 0x1fff}, + { 0x102058, 1, 0x1f, 0x1fff}, + { 0x102080, 16, 0x1f, 0xfff}, + { 0x103004, 2, 0x1f, 0xfff}, + { 0x103068, 1, 0x1f, 0xfff}, + { 0x103078, 1, 0x1f, 0xfff}, + { 0x103088, 1, 0x1f, 0x1fff}, + { 0x10309c, 2, 0x1e, 0xfff}, + { 0x1030b8, 2, 0x1c, 0xfff}, + { 0x1030cc, 1, 0x1c, 0xfff}, + { 0x1030e0, 1, 0x1c, 0xfff}, + { 0x104004, 1, 0x1f, 0xfff}, + { 0x104018, 1, 0x1f, 0xfff}, + { 0x104020, 1, 0x1f, 0xfff}, + { 0x10403c, 1, 0x1f, 0xfff}, + { 0x1040fc, 1, 0x1f, 0xfff}, + { 0x10410c, 1, 0x1f, 0x1fff}, + { 0x104400, 1, 0x1f, 0x1fff}, + { 0x104404, 63, 0x1f, 0xfff}, + { 0x104800, 1, 0x1f, 0x1fff}, + { 0x104804, 63, 0x1f, 0xfff}, + { 0x105000, 4, 0x1f, 0x1fff}, + { 0x105010, 252, 0x1f, 0xfff}, + { 0x108094, 1, 0x3, 0xfff}, + { 0x1201b0, 2, 0x1f, 0xfff}, + { 0x12032c, 1, 0x1f, 0xfff}, + { 0x12036c, 3, 0x1f, 0xfff}, + { 0x120408, 2, 0x1f, 0xfff}, + { 0x120414, 15, 0x1f, 0xfff}, + { 0x120478, 2, 0x1f, 0xfff}, + { 0x12052c, 1, 0x1f, 0xfff}, + { 0x120564, 3, 0x1f, 0xfff}, + { 0x12057c, 1, 0x1f, 0x1fff}, + { 0x12058c, 1, 0x1f, 0x1fff}, + { 0x120608, 1, 0x1e, 0xfff}, + { 0x120748, 1, 0x1c, 0xfff}, + { 0x120778, 2, 0x1c, 0xfff}, + { 0x120808, 3, 0x1f, 0xfff}, + { 0x120818, 1, 0x1f, 0xfff}, + { 0x120820, 1, 0x1f, 0xfff}, + { 0x120828, 1, 0x1f, 0xfff}, + { 0x120830, 1, 0x1f, 0xfff}, + { 0x120838, 1, 0x1f, 0xfff}, + { 0x120840, 1, 0x1f, 0xfff}, + { 0x120848, 1, 0x1f, 0xfff}, + { 0x120850, 1, 0x1f, 0xfff}, + { 0x120858, 1, 0x1f, 0xfff}, + { 0x120860, 1, 0x1f, 0xfff}, + { 0x120868, 1, 0x1f, 0xfff}, + { 0x120870, 1, 0x1f, 0xfff}, + { 0x120878, 1, 0x1f, 0xfff}, + { 0x120880, 1, 0x1f, 0xfff}, + { 0x120888, 1, 0x1f, 0xfff}, + { 0x120890, 1, 0x1f, 0xfff}, + { 0x120898, 1, 0x1f, 0xfff}, + { 0x1208a0, 1, 0x1f, 0xfff}, + { 0x1208a8, 1, 0x1f, 0xfff}, + { 0x1208b0, 1, 0x1f, 0xfff}, + { 0x1208b8, 1, 0x1f, 0xfff}, + { 0x1208c0, 1, 0x1f, 0xfff}, + { 0x1208c8, 1, 0x1f, 0xfff}, + { 0x1208d0, 1, 0x1f, 0xfff}, + { 0x1208d8, 1, 0x1f, 0xfff}, + { 0x1208e0, 1, 0x1f, 0xfff}, + { 0x1208e8, 1, 0x1f, 0xfff}, + { 0x1208f0, 1, 0x1f, 0xfff}, + { 0x1208f8, 1, 0x1f, 0xfff}, + { 0x120900, 1, 0x1f, 0xfff}, + { 0x120908, 1, 0x1f, 0xfff}, + { 0x130030, 1, 0x1c, 0xfff}, + { 0x13004c, 3, 0x1c, 0xfff}, + { 0x130064, 2, 0x1c, 0xfff}, + { 0x13009c, 1, 0x1c, 0x1fff}, + { 0x130130, 1, 0x1c, 0xfff}, + { 0x13016c, 1, 0x1c, 0xfff}, + { 0x130300, 1, 0x1c, 0xfff}, + { 0x130480, 1, 0x1c, 0xfff}, + { 0x14005c, 2, 0xf, 0xfff}, + { 0x1400d0, 2, 0xf, 0xfff}, + { 0x1400e0, 1, 0xf, 0xfff}, + { 0x1401c8, 1, 0xf, 0xfff}, + { 0x140200, 6, 0xf, 0xfff}, + { 0x140338, 7, 0x10, 0xfff}, + { 0x140370, 7, 0x10, 0xfff}, + { 0x15c1bc, 6, 0x10, 0xfff}, + { 0x15c230, 7, 0x10, 0xfff}, + { 0x16101c, 1, 0x1f, 0xfff}, + { 0x16102c, 1, 0x1f, 0x1fff}, + { 0x164014, 2, 0x1f, 0xfff}, + { 0x1640f0, 1, 0x1f, 0xfff}, + { 0x166290, 1, 0x1f, 0xfff}, + { 0x1662a0, 1, 0x1f, 0xfff}, + { 0x1662b0, 1, 0x1f, 0x1fff}, + { 0x166548, 1, 0x1f, 0xfff}, + { 0x166550, 1, 0x1f, 0xfff}, + { 0x166558, 1, 0x1f, 0xfff}, + { 0x168000, 1, 0x1f, 0xfff}, + { 0x168008, 1, 0x1f, 0xfff}, + { 0x168010, 1, 0x1f, 0xfff}, + { 0x168018, 1, 0x1f, 0xfff}, + { 0x168028, 2, 0x1f, 0xfff}, + { 0x168058, 9, 0x1f, 0xfff}, + { 0x168238, 1, 0x1f, 0xfff}, + { 0x1682d0, 7, 0x1f, 0xfff}, + { 0x168300, 2, 0x3, 0xfff}, + { 0x168308, 65, 0x1f, 0xfff}, + { 0x168410, 2, 0x1f, 0xfff}, + { 0x168438, 1, 0x1f, 0xfff}, + { 0x168448, 1, 0x1f, 0x1fff}, + { 0x168a00, 128, 0x1f, 0xfff}, + { 0x16e200, 128, 0x2, 0xfff}, + { 0x16e404, 2, 0x2, 0xfff}, + { 0x16e584, 64, 0x2, 0xfff}, + { 0x16e684, 2, 0x1e, 0xfff}, + { 0x16e68c, 4, 0x2, 0xfff}, + { 0x16e6fc, 4, 0x1c, 0xfff}, + { 0x16e7ac, 12, 0x10, 0xfff}, + { 0x1700a4, 1, 0x1f, 0xfff}, + { 0x1700ac, 2, 0x1f, 0xfff}, + { 0x1700c0, 1, 0x1f, 0xfff}, + { 0x170174, 1, 0x1f, 0xfff}, + { 0x170184, 1, 0x1f, 0x1fff}, + { 0x1800f4, 1, 0x1f, 0xfff}, + { 0x180104, 1, 0x1f, 0xfff}, + { 0x180114, 1, 0x1f, 0x1fff}, + { 0x180124, 1, 0x1f, 0x1fff}, + { 0x18026c, 1, 0x1f, 0xfff}, + { 0x1802a0, 1, 0x1f, 0xfff}, + { 0x1b8000, 1, 0x1f, 0xfff}, + { 0x1b8040, 1, 0x1f, 0xfff}, + { 0x1b8080, 1, 0x1f, 0xfff}, + { 0x1b80c0, 1, 0x1f, 0xfff}, + { 0x200104, 1, 0x1f, 0xfff}, + { 0x200114, 1, 0x1f, 0xfff}, + { 0x200124, 1, 0x1f, 0x1fff}, + { 0x200134, 1, 0x1f, 0x1fff}, + { 0x20026c, 1, 0x1f, 0xfff}, + { 0x2002a0, 1, 0x1f, 0xfff}, + { 0x238000, 1, 0x1f, 0xfff}, + { 0x238040, 1, 0x1f, 0xfff}, + { 0x238080, 1, 0x1f, 0xfff}, + { 0x2380c0, 1, 0x1f, 0xfff}, + { 0x280104, 1, 0x1f, 0xfff}, + { 0x280114, 1, 0x1f, 0xfff}, + { 0x280124, 1, 0x1f, 0x1fff}, + { 0x280134, 1, 0x1f, 0x1fff}, + { 0x28026c, 1, 0x1f, 0xfff}, + { 0x2802a0, 1, 0x1f, 0xfff}, + { 0x2b8000, 1, 0x1f, 0xfff}, + { 0x2b8040, 1, 0x1f, 0xfff}, + { 0x2b8080, 1, 0x1f, 0xfff}, + { 0x300104, 1, 0x1f, 0xfff}, + { 0x300114, 1, 0x1f, 0xfff}, + { 0x300124, 1, 0x1f, 0x1fff}, + { 0x300134, 1, 0x1f, 0x1fff}, + { 0x30026c, 1, 0x1f, 0xfff}, + { 0x3002a0, 1, 0x1f, 0xfff}, + { 0x338000, 1, 0x1f, 0xfff}, + { 0x338040, 1, 0x1f, 0xfff}, + { 0x338080, 1, 0x1f, 0xfff}, + { 0x3380c0, 1, 0x1f, 0xfff} +}; + +#define IDLE_REGS_COUNT ARRAY_SIZE(idle_reg_addrs) + +static const uint32_t read_reg_e1[] = { + 0x1b1000}; + +static const struct wreg_addr wreg_addr_e1 = { + 0x1b0c00, 192, 1, read_reg_e1, 0x1f, 0x1fff}; + +static const uint32_t read_reg_e1h[] = { + 0x1b1040, 0x1b1000}; + +static const struct wreg_addr wreg_addr_e1h = { + 0x1b0c00, 256, 2, read_reg_e1h, 0x1f, 0x1fff}; + +static const uint32_t read_reg_e2[] = { + 0x1b1040, 0x1b1000}; + +static const struct wreg_addr wreg_addr_e2 = { + 0x1b0c00, 128, 2, read_reg_e2, 0x1f, 0x1fff}; + +static const uint32_t read_reg_e3[] = { + 0x1b1040, 0x1b1000}; + +static const struct wreg_addr wreg_addr_e3 = { + 0x1b0c00, 128, 2, read_reg_e3, 0x1f, 0x1fff}; + +static const uint32_t read_reg_e3b0[] = { + 0x1b1040, 0x1b1000}; + +static const struct wreg_addr wreg_addr_e3b0 = { + 0x1b0c00, 128, 2, read_reg_e3b0, 0x1f, 0x1fff}; + +static const unsigned int dump_num_registers[NUM_CHIPS][NUM_PRESETS] = { + {19758, 17543, 26951, 18705, 17287, 26695, 19812, 31367, 40775, 19788, + 25223, 34631, 19074}, + {31750, 18273, 32253, 30697, 18017, 31997, 31804, 32097, 46077, 31780, + 25953, 39933, 35895}, + {36527, 17928, 33697, 35474, 18700, 34466, 36581, 31752, 47521, 36557, + 25608, 41377, 43903}, + {45239, 17936, 34387, 44186, 18708, 35156, 45293, 31760, 48211, 45269, + 25616, 42067, 43903}, + {45302, 17999, 34802, 44249, 18771, 35571, 45356, 31823, 48626, 45332, + 25679, 42482, 43903} +}; +#endif /* #ifndef __BXE_DUMP_H__ */ diff --git a/sys/dev/bxe/bxe_ioctl.h b/sys/dev/bxe/bxe_ioctl.h new file mode 100644 index 0000000..2504982 --- /dev/null +++ b/sys/dev/bxe/bxe_ioctl.h @@ -0,0 +1,57 @@ +/* + * Copyright (c) 2015-2016 Qlogic Corporation + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _BXE_IOCTL_H_ +#define _BXE_IOCTL_H_ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/ioccom.h> + + +struct bxe_grcdump { + uint16_t pci_func; + uint32_t grcdump_size; + void *grcdump; + uint32_t grcdump_dwords; +}; +typedef struct bxe_grcdump bxe_grcdump_t; + + +/* + * Read grcdump size + */ +#define BXE_GRC_DUMP_SIZE _IOWR('e', 1, bxe_grcdump_t) + +/* + * Read grcdump + */ +#define BXE_GRC_DUMP _IOWR('e', 2, bxe_grcdump_t) + + +#endif /* #ifndef _QLNX_IOCTL_H_ */ diff --git a/sys/dev/bxe/bxe_stats.c b/sys/dev/bxe/bxe_stats.c index b2cc676..6a8551c 100644 --- a/sys/dev/bxe/bxe_stats.c +++ b/sys/dev/bxe/bxe_stats.c @@ -1227,6 +1227,8 @@ bxe_drv_stats_update(struct bxe_softc *sc) UPDATE_ESTAT_QSTAT(rx_calls); UPDATE_ESTAT_QSTAT(rx_pkts); UPDATE_ESTAT_QSTAT(rx_tpa_pkts); + UPDATE_ESTAT_QSTAT(rx_erroneous_jumbo_sge_pkts); + UPDATE_ESTAT_QSTAT(rx_bxe_service_rxsgl); UPDATE_ESTAT_QSTAT(rx_jumbo_sge_pkts); UPDATE_ESTAT_QSTAT(rx_soft_errors); UPDATE_ESTAT_QSTAT(rx_hw_csum_errors); diff --git a/sys/dev/bxe/bxe_stats.h b/sys/dev/bxe/bxe_stats.h index badb4b7..a4cd152 100644 --- a/sys/dev/bxe/bxe_stats.h +++ b/sys/dev/bxe/bxe_stats.h @@ -218,6 +218,8 @@ struct bxe_eth_stats { uint32_t rx_calls; uint32_t rx_pkts; uint32_t rx_tpa_pkts; + uint32_t rx_erroneous_jumbo_sge_pkts; + uint32_t rx_bxe_service_rxsgl; uint32_t rx_jumbo_sge_pkts; uint32_t rx_soft_errors; uint32_t rx_hw_csum_errors; @@ -319,6 +321,8 @@ struct bxe_eth_q_stats { uint32_t rx_calls; uint32_t rx_pkts; uint32_t rx_tpa_pkts; + uint32_t rx_erroneous_jumbo_sge_pkts; + uint32_t rx_bxe_service_rxsgl; uint32_t rx_jumbo_sge_pkts; uint32_t rx_soft_errors; uint32_t rx_hw_csum_errors; @@ -413,6 +417,8 @@ struct bxe_eth_q_stats_old { uint32_t rx_calls_old; uint32_t rx_pkts_old; uint32_t rx_tpa_pkts_old; + uint32_t rx_erroneous_jumbo_sge_pkts_old; + uint32_t rx_bxe_service_rxsgl_old; uint32_t rx_jumbo_sge_pkts_old; uint32_t rx_soft_errors_old; uint32_t rx_hw_csum_errors_old; diff --git a/sys/dev/bxe/ecore_init.h b/sys/dev/bxe/ecore_init.h index 7eab811..31417cc 100644 --- a/sys/dev/bxe/ecore_init.h +++ b/sys/dev/bxe/ecore_init.h @@ -749,10 +749,17 @@ static inline void ecore_set_mcp_parity(struct bxe_softc *sc, uint8_t enable) for (i = 0; i < ARRSIZE(mcp_attn_ctl_regs); i++) { reg_val = REG_RD(sc, mcp_attn_ctl_regs[i].addr); +#if 0 if (enable) reg_val |= MISC_AEU_ENABLE_MCP_PRTY_BITS; /* Linux is using mcp_attn_ctl_regs[i].bits */ else reg_val &= ~MISC_AEU_ENABLE_MCP_PRTY_BITS; /* Linux is using mcp_attn_ctl_regs[i].bits */ +#else + if (enable) + reg_val |= mcp_attn_ctl_regs[i].bits; + else + reg_val &= ~mcp_attn_ctl_regs[i].bits; +#endif REG_WR(sc, mcp_attn_ctl_regs[i].addr, reg_val); } diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index b3553ca..d7a6f76 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -766,7 +766,7 @@ struct adapter { void *tom_softc; /* (struct tom_data *) */ struct tom_tunables tt; void *iwarp_softc; /* (struct c4iw_dev *) */ - void *iscsi_softc; + void *iscsi_ulp_softc; /* (struct cxgbei_data *) */ #endif struct l2t_data *l2t; /* L2 table */ struct tid_info tids; diff --git a/sys/dev/cxgbe/cxgbei/cxgbei.c b/sys/dev/cxgbe/cxgbei/cxgbei.c new file mode 100644 index 0000000..66ceb89 --- /dev/null +++ b/sys/dev/cxgbe/cxgbei/cxgbei.c @@ -0,0 +1,1131 @@ +/*- + * Copyright (c) 2012 Chelsio Communications, Inc. + * All rights reserved. + * + * Chelsio T5xx iSCSI driver + * + * Written by: Sreenivasa Honnur <shonnur@chelsio.com> + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include "opt_inet.h" +#include "opt_inet6.h" + +#include <sys/types.h> +#include <sys/param.h> +#include <sys/kernel.h> +#include <sys/module.h> +#include <sys/systm.h> + +#ifdef TCP_OFFLOAD +#include <sys/errno.h> +#include <sys/kthread.h> +#include <sys/smp.h> +#include <sys/socket.h> +#include <sys/socketvar.h> +#include <sys/mbuf.h> +#include <sys/lock.h> +#include <sys/mutex.h> +#include <sys/condvar.h> + +#include <netinet/in.h> +#include <netinet/in_pcb.h> +#include <netinet/toecore.h> +#include <netinet/tcp_var.h> +#include <netinet/tcp_fsm.h> + +#include <cam/scsi/scsi_all.h> +#include <cam/scsi/scsi_da.h> +#include <cam/ctl/ctl_io.h> +#include <cam/ctl/ctl.h> +#include <cam/ctl/ctl_backend.h> +#include <cam/ctl/ctl_error.h> +#include <cam/ctl/ctl_frontend.h> +#include <cam/ctl/ctl_debug.h> +#include <cam/ctl/ctl_ha.h> +#include <cam/ctl/ctl_ioctl.h> + +#include <dev/iscsi/icl.h> +#include <dev/iscsi/iscsi_proto.h> +#include <dev/iscsi/iscsi_ioctl.h> +#include <dev/iscsi/iscsi.h> +#include <cam/ctl/ctl_frontend_iscsi.h> + +#include <cam/cam.h> +#include <cam/cam_ccb.h> +#include <cam/cam_xpt.h> +#include <cam/cam_debug.h> +#include <cam/cam_sim.h> +#include <cam/cam_xpt_sim.h> +#include <cam/cam_xpt_periph.h> +#include <cam/cam_periph.h> +#include <cam/cam_compat.h> +#include <cam/scsi/scsi_message.h> + +#include "common/common.h" +#include "common/t4_msg.h" +#include "common/t4_regs.h" /* for PCIE_MEM_ACCESS */ +#include "tom/t4_tom.h" +#include "cxgbei.h" +#include "cxgbei_ulp2_ddp.h" + +static int worker_thread_count; +static struct cxgbei_worker_thread_softc *cwt_softc; +static struct proc *cxgbei_proc; + +/* XXXNP some header instead. */ +struct icl_pdu *icl_cxgbei_new_pdu(int); +void icl_cxgbei_new_pdu_set_conn(struct icl_pdu *, struct icl_conn *); +void icl_cxgbei_conn_pdu_free(struct icl_conn *, struct icl_pdu *); + +/* + * Direct Data Placement - + * Directly place the iSCSI Data-In or Data-Out PDU's payload into pre-posted + * final destination host-memory buffers based on the Initiator Task Tag (ITT) + * in Data-In or Target Task Tag (TTT) in Data-Out PDUs. + * The host memory address is programmed into h/w in the format of pagepod + * entries. + * The location of the pagepod entry is encoded into ddp tag which is used as + * the base for ITT/TTT. + */ + +/* + * functions to program the pagepod in h/w + */ +static void inline +ppod_set(struct pagepod *ppod, + struct cxgbei_ulp2_pagepod_hdr *hdr, + struct cxgbei_ulp2_gather_list *gl, + unsigned int pidx) +{ + int i; + + memcpy(ppod, hdr, sizeof(*hdr)); + + for (i = 0; i < (PPOD_PAGES + 1); i++, pidx++) { + ppod->addr[i] = pidx < gl->nelem ? + cpu_to_be64(gl->dma_sg[pidx].phys_addr) : 0ULL; + } +} + +static void inline +ppod_clear(struct pagepod *ppod) +{ + memset(ppod, 0, sizeof(*ppod)); +} + +static inline void +ulp_mem_io_set_hdr(struct adapter *sc, int tid, struct ulp_mem_io *req, + unsigned int wr_len, unsigned int dlen, + unsigned int pm_addr) +{ + struct ulptx_idata *idata = (struct ulptx_idata *)(req + 1); + + INIT_ULPTX_WR(req, wr_len, 0, 0); + req->cmd = cpu_to_be32(V_ULPTX_CMD(ULP_TX_MEM_WRITE) | + V_ULP_MEMIO_ORDER(is_t4(sc)) | + V_T5_ULP_MEMIO_IMM(is_t5(sc))); + req->dlen = htonl(V_ULP_MEMIO_DATA_LEN(dlen >> 5)); + req->len16 = htonl(DIV_ROUND_UP(wr_len - sizeof(req->wr), 16) + | V_FW_WR_FLOWID(tid)); + req->lock_addr = htonl(V_ULP_MEMIO_ADDR(pm_addr >> 5)); + + idata->cmd_more = htonl(V_ULPTX_CMD(ULP_TX_SC_IMM)); + idata->len = htonl(dlen); +} + +#define PPOD_SIZE sizeof(struct pagepod) +#define ULPMEM_IDATA_MAX_NPPODS 1 /* 256/PPOD_SIZE */ +#define PCIE_MEMWIN_MAX_NPPODS 16 /* 1024/PPOD_SIZE */ + +static int +ppod_write_idata(struct cxgbei_data *ci, + struct cxgbei_ulp2_pagepod_hdr *hdr, + unsigned int idx, unsigned int npods, + struct cxgbei_ulp2_gather_list *gl, + unsigned int gl_pidx, struct toepcb *toep) +{ + u_int dlen = PPOD_SIZE * npods; + u_int pm_addr = idx * PPOD_SIZE + ci->llimit; + u_int wr_len = roundup(sizeof(struct ulp_mem_io) + + sizeof(struct ulptx_idata) + dlen, 16); + struct ulp_mem_io *req; + struct ulptx_idata *idata; + struct pagepod *ppod; + u_int i; + struct wrqe *wr; + struct adapter *sc = toep->vi->pi->adapter; + + wr = alloc_wrqe(wr_len, toep->ctrlq); + if (wr == NULL) { + CXGBE_UNIMPLEMENTED("ppod_write_idata: alloc_wrqe failure"); + return (ENOMEM); + } + + req = wrtod(wr); + memset(req, 0, wr_len); + ulp_mem_io_set_hdr(sc, toep->tid, req, wr_len, dlen, pm_addr); + idata = (struct ulptx_idata *)(req + 1); + + ppod = (struct pagepod *)(idata + 1); + for (i = 0; i < npods; i++, ppod++, gl_pidx += PPOD_PAGES) { + if (!hdr) /* clear the pagepod */ + ppod_clear(ppod); + else /* set the pagepod */ + ppod_set(ppod, hdr, gl, gl_pidx); + } + + t4_wrq_tx(sc, wr); + return 0; +} + +int +t4_ddp_set_map(struct cxgbei_data *ci, void *iccp, + struct cxgbei_ulp2_pagepod_hdr *hdr, u_int idx, u_int npods, + struct cxgbei_ulp2_gather_list *gl, int reply) +{ + struct icl_cxgbei_conn *icc = (struct icl_cxgbei_conn *)iccp; + struct toepcb *toep = icc->toep; + int err; + unsigned int pidx = 0, w_npods = 0, cnt; + + /* + * on T4, if we use a mix of IMMD and DSGL with ULP_MEM_WRITE, + * the order would not be garanteed, so we will stick with IMMD + */ + gl->tid = toep->tid; + gl->port_id = toep->vi->pi->port_id; + gl->egress_dev = (void *)toep->vi->ifp; + + /* send via immediate data */ + for (; w_npods < npods; idx += cnt, w_npods += cnt, + pidx += PPOD_PAGES) { + cnt = npods - w_npods; + if (cnt > ULPMEM_IDATA_MAX_NPPODS) + cnt = ULPMEM_IDATA_MAX_NPPODS; + err = ppod_write_idata(ci, hdr, idx, cnt, gl, pidx, toep); + if (err) { + printf("%s: ppod_write_idata failed\n", __func__); + break; + } + } + return err; +} + +void +t4_ddp_clear_map(struct cxgbei_data *ci, struct cxgbei_ulp2_gather_list *gl, + u_int tag, u_int idx, u_int npods, struct icl_cxgbei_conn *icc) +{ + struct toepcb *toep = icc->toep; + int err = -1; + u_int pidx = 0; + u_int w_npods = 0; + u_int cnt; + + for (; w_npods < npods; idx += cnt, w_npods += cnt, + pidx += PPOD_PAGES) { + cnt = npods - w_npods; + if (cnt > ULPMEM_IDATA_MAX_NPPODS) + cnt = ULPMEM_IDATA_MAX_NPPODS; + err = ppod_write_idata(ci, NULL, idx, cnt, gl, 0, toep); + if (err) + break; + } +} + +static int +cxgbei_map_sg(struct cxgbei_sgl *sgl, struct ccb_scsiio *csio) +{ + unsigned int data_len = csio->dxfer_len; + unsigned int sgoffset = (uint64_t)csio->data_ptr & PAGE_MASK; + unsigned int nsge; + unsigned char *sgaddr = csio->data_ptr; + unsigned int len = 0; + + nsge = (csio->dxfer_len + sgoffset + PAGE_SIZE - 1) >> PAGE_SHIFT; + sgl->sg_addr = sgaddr; + sgl->sg_offset = sgoffset; + if (data_len < (PAGE_SIZE - sgoffset)) + len = data_len; + else + len = PAGE_SIZE - sgoffset; + + sgl->sg_length = len; + + data_len -= len; + sgaddr += len; + sgl = sgl+1; + + while (data_len > 0) { + sgl->sg_addr = sgaddr; + len = (data_len < PAGE_SIZE)? data_len: PAGE_SIZE; + sgl->sg_length = len; + sgaddr += len; + data_len -= len; + sgl = sgl + 1; + } + + return nsge; +} + +static int +cxgbei_map_sg_tgt(struct cxgbei_sgl *sgl, union ctl_io *io) +{ + unsigned int data_len, sgoffset, nsge; + unsigned char *sgaddr; + unsigned int len = 0, index = 0, ctl_sg_count, i; + struct ctl_sg_entry ctl_sg_entry, *ctl_sglist; + + if (io->scsiio.kern_sg_entries > 0) { + ctl_sglist = (struct ctl_sg_entry *)io->scsiio.kern_data_ptr; + ctl_sg_count = io->scsiio.kern_sg_entries; + } else { + ctl_sglist = &ctl_sg_entry; + ctl_sglist->addr = io->scsiio.kern_data_ptr; + ctl_sglist->len = io->scsiio.kern_data_len; + ctl_sg_count = 1; + } + + sgaddr = sgl->sg_addr = ctl_sglist[index].addr; + sgoffset = sgl->sg_offset = (uint64_t)sgl->sg_addr & PAGE_MASK; + data_len = ctl_sglist[index].len; + + if (data_len < (PAGE_SIZE - sgoffset)) + len = data_len; + else + len = PAGE_SIZE - sgoffset; + + sgl->sg_length = len; + + data_len -= len; + sgaddr += len; + sgl = sgl+1; + + len = 0; + for (i = 0; i< ctl_sg_count; i++) + len += ctl_sglist[i].len; + nsge = (len + sgoffset + PAGE_SIZE -1) >> PAGE_SHIFT; + while (data_len > 0) { + sgl->sg_addr = sgaddr; + len = (data_len < PAGE_SIZE)? data_len: PAGE_SIZE; + sgl->sg_length = len; + sgaddr += len; + data_len -= len; + sgl = sgl + 1; + if (data_len == 0) { + if (index == ctl_sg_count - 1) + break; + index++; + sgaddr = ctl_sglist[index].addr; + data_len = ctl_sglist[index].len; + } + } + + return nsge; +} + +static int +t4_sk_ddp_tag_reserve(struct cxgbei_data *ci, struct icl_cxgbei_conn *icc, + u_int xferlen, struct cxgbei_sgl *sgl, u_int sgcnt, u_int *ddp_tag) +{ + struct cxgbei_ulp2_gather_list *gl; + int err = -EINVAL; + struct toepcb *toep = icc->toep; + + gl = cxgbei_ulp2_ddp_make_gl_from_iscsi_sgvec(xferlen, sgl, sgcnt, ci, 0); + if (gl) { + err = cxgbei_ulp2_ddp_tag_reserve(ci, icc, toep->tid, + &ci->tag_format, ddp_tag, gl, 0, 0); + if (err) { + cxgbei_ulp2_ddp_release_gl(ci, gl); + } + } + + return err; +} + +static unsigned int +cxgbei_task_reserve_itt(struct icl_conn *ic, void **prv, + struct ccb_scsiio *scmd, unsigned int *itt) +{ + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + int xferlen = scmd->dxfer_len; + struct cxgbei_task_data *tdata = NULL; + struct cxgbei_sgl *sge = NULL; + struct toepcb *toep = icc->toep; + struct adapter *sc = td_adapter(toep->td); + struct cxgbei_data *ci = sc->iscsi_ulp_softc; + int err = -1; + + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + + tdata = (struct cxgbei_task_data *)*prv; + if (xferlen == 0 || tdata == NULL) + goto out; + if (xferlen < DDP_THRESHOLD) + goto out; + + if ((scmd->ccb_h.flags & CAM_DIR_MASK) == CAM_DIR_IN) { + tdata->nsge = cxgbei_map_sg(tdata->sgl, scmd); + if (tdata->nsge == 0) { + CTR1(KTR_CXGBE, "%s: map_sg failed", __func__); + return 0; + } + sge = tdata->sgl; + + tdata->sc_ddp_tag = *itt; + + CTR3(KTR_CXGBE, "%s: *itt:0x%x sc_ddp_tag:0x%x", + __func__, *itt, tdata->sc_ddp_tag); + if (cxgbei_ulp2_sw_tag_usable(&ci->tag_format, + tdata->sc_ddp_tag)) { + err = t4_sk_ddp_tag_reserve(ci, icc, scmd->dxfer_len, + sge, tdata->nsge, &tdata->sc_ddp_tag); + } else { + CTR3(KTR_CXGBE, + "%s: itt:0x%x sc_ddp_tag:0x%x not usable", + __func__, *itt, tdata->sc_ddp_tag); + } + } +out: + if (err < 0) + tdata->sc_ddp_tag = + cxgbei_ulp2_set_non_ddp_tag(&ci->tag_format, *itt); + + return tdata->sc_ddp_tag; +} + +static unsigned int +cxgbei_task_reserve_ttt(struct icl_conn *ic, void **prv, union ctl_io *io, + unsigned int *ttt) +{ + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + struct toepcb *toep = icc->toep; + struct adapter *sc = td_adapter(toep->td); + struct cxgbei_data *ci = sc->iscsi_ulp_softc; + struct cxgbei_task_data *tdata = NULL; + int xferlen, err = -1; + struct cxgbei_sgl *sge = NULL; + + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + + xferlen = (io->scsiio.kern_data_len - io->scsiio.ext_data_filled); + tdata = (struct cxgbei_task_data *)*prv; + if ((xferlen == 0) || (tdata == NULL)) + goto out; + if (xferlen < DDP_THRESHOLD) + goto out; + tdata->nsge = cxgbei_map_sg_tgt(tdata->sgl, io); + if (tdata->nsge == 0) { + CTR1(KTR_CXGBE, "%s: map_sg failed", __func__); + return 0; + } + sge = tdata->sgl; + + tdata->sc_ddp_tag = *ttt; + if (cxgbei_ulp2_sw_tag_usable(&ci->tag_format, tdata->sc_ddp_tag)) { + err = t4_sk_ddp_tag_reserve(ci, icc, xferlen, sge, + tdata->nsge, &tdata->sc_ddp_tag); + } else { + CTR2(KTR_CXGBE, "%s: sc_ddp_tag:0x%x not usable", + __func__, tdata->sc_ddp_tag); + } +out: + if (err < 0) + tdata->sc_ddp_tag = + cxgbei_ulp2_set_non_ddp_tag(&ci->tag_format, *ttt); + return tdata->sc_ddp_tag; +} + +static int +t4_sk_ddp_tag_release(struct icl_cxgbei_conn *icc, unsigned int ddp_tag) +{ + struct toepcb *toep = icc->toep; + struct adapter *sc = td_adapter(toep->td); + struct cxgbei_data *ci = sc->iscsi_ulp_softc; + + cxgbei_ulp2_ddp_tag_release(ci, ddp_tag, icc); + + return (0); +} + +static int +cxgbei_ddp_init(struct adapter *sc, struct cxgbei_data *ci) +{ + int nppods, bits, max_sz, rc; + static const u_int pgsz_order[] = {0, 1, 2, 3}; + + MPASS(sc->vres.iscsi.size > 0); + + ci->llimit = sc->vres.iscsi.start; + ci->ulimit = sc->vres.iscsi.start + sc->vres.iscsi.size - 1; + max_sz = G_MAXRXDATA(t4_read_reg(sc, A_TP_PARA_REG2)); + + nppods = sc->vres.iscsi.size >> IPPOD_SIZE_SHIFT; + if (nppods <= 1024) + return (ENXIO); + + bits = fls(nppods); + if (bits > IPPOD_IDX_MAX_SIZE) + bits = IPPOD_IDX_MAX_SIZE; + nppods = (1 << (bits - 1)) - 1; + + rc = bus_dma_tag_create(NULL, 1, 0, BUS_SPACE_MAXADDR, + BUS_SPACE_MAXADDR, NULL, NULL, UINT32_MAX , 8, BUS_SPACE_MAXSIZE, + BUS_DMA_ALLOCNOW, NULL, NULL, &ci->ulp_ddp_tag); + if (rc != 0) { + device_printf(sc->dev, "%s: failed to create DMA tag: %u.\n", + __func__, rc); + return (rc); + } + + ci->colors = malloc(nppods * sizeof(char), M_CXGBE, M_NOWAIT | M_ZERO); + ci->gl_map = malloc(nppods * sizeof(struct cxgbei_ulp2_gather_list *), + M_CXGBE, M_NOWAIT | M_ZERO); + if (ci->colors == NULL || ci->gl_map == NULL) { + bus_dma_tag_destroy(ci->ulp_ddp_tag); + free(ci->colors, M_CXGBE); + free(ci->gl_map, M_CXGBE); + return (ENOMEM); + } + + mtx_init(&ci->map_lock, "ddp lock", NULL, MTX_DEF | MTX_DUPOK); + ci->max_txsz = ci->max_rxsz = min(max_sz, ULP2_MAX_PKT_SIZE); + ci->nppods = nppods; + ci->idx_last = nppods; + ci->idx_bits = bits; + ci->idx_mask = (1 << bits) - 1; + ci->rsvd_tag_mask = (1 << (bits + IPPOD_IDX_SHIFT)) - 1; + + ci->tag_format.sw_bits = bits; + ci->tag_format.rsvd_bits = bits; + ci->tag_format.rsvd_shift = IPPOD_IDX_SHIFT; + ci->tag_format.rsvd_mask = ci->idx_mask; + + t4_iscsi_init(sc, ci->idx_mask << IPPOD_IDX_SHIFT, pgsz_order); + + return (rc); +} + +static int +do_rx_iscsi_hdr(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) +{ + struct adapter *sc = iq->adapter; + struct cpl_iscsi_hdr *cpl = mtod(m, struct cpl_iscsi_hdr *); + u_int tid = GET_TID(cpl); + struct toepcb *toep = lookup_tid(sc, tid); + struct icl_pdu *ip; + struct icl_cxgbei_pdu *icp; + + M_ASSERTPKTHDR(m); + + ip = icl_cxgbei_new_pdu(M_NOWAIT); + if (ip == NULL) + CXGBE_UNIMPLEMENTED("PDU allocation failure"); + icp = ip_to_icp(ip); + bcopy(mtod(m, caddr_t) + sizeof(*cpl), icp->ip.ip_bhs, sizeof(struct + iscsi_bhs)); + icp->pdu_seq = ntohl(cpl->seq); + icp->pdu_flags = SBUF_ULP_FLAG_HDR_RCVD; + + /* This is the start of a new PDU. There should be no old state. */ + MPASS(toep->ulpcb2 == NULL); + toep->ulpcb2 = icp; + +#if 0 + CTR4(KTR_CXGBE, "%s: tid %u, cpl->len hlen %u, m->m_len hlen %u", + __func__, tid, ntohs(cpl->len), m->m_len); +#endif + + m_freem(m); + return (0); +} + +static int +do_rx_iscsi_data(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) +{ + struct adapter *sc = iq->adapter; + struct cpl_iscsi_data *cpl = mtod(m, struct cpl_iscsi_data *); + u_int tid = GET_TID(cpl); + struct toepcb *toep = lookup_tid(sc, tid); + struct icl_cxgbei_pdu *icp = toep->ulpcb2; + + M_ASSERTPKTHDR(m); + + /* Must already have received the header (but not the data). */ + MPASS(icp != NULL); + MPASS(icp->pdu_flags == SBUF_ULP_FLAG_HDR_RCVD); + MPASS(icp->ip.ip_data_mbuf == NULL); + MPASS(icp->ip.ip_data_len == 0); + + m_adj(m, sizeof(*cpl)); + + icp->pdu_flags |= SBUF_ULP_FLAG_DATA_RCVD; + icp->ip.ip_data_mbuf = m; + icp->ip.ip_data_len = m->m_pkthdr.len; + +#if 0 + CTR4(KTR_CXGBE, "%s: tid %u, cpl->len dlen %u, m->m_len dlen %u", + __func__, tid, ntohs(cpl->len), m->m_len); +#endif + + return (0); +} + +static int +do_rx_iscsi_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) +{ + struct adapter *sc = iq->adapter; + const struct cpl_rx_data_ddp *cpl = (const void *)(rss + 1); + u_int tid = GET_TID(cpl); + struct toepcb *toep = lookup_tid(sc, tid); + struct inpcb *inp = toep->inp; + struct socket *so; + struct sockbuf *sb; + struct tcpcb *tp; + struct icl_cxgbei_conn *icc; + struct icl_conn *ic; + struct icl_cxgbei_pdu *icp = toep->ulpcb2; + struct icl_pdu *ip; + u_int pdu_len, val; + + MPASS(m == NULL); + + /* Must already be assembling a PDU. */ + MPASS(icp != NULL); + MPASS(icp->pdu_flags & SBUF_ULP_FLAG_HDR_RCVD); /* Data is optional. */ + ip = &icp->ip; + icp->pdu_flags |= SBUF_ULP_FLAG_STATUS_RCVD; + val = ntohl(cpl->ddpvld); + if (val & F_DDP_PADDING_ERR) + icp->pdu_flags |= SBUF_ULP_FLAG_PAD_ERROR; + if (val & F_DDP_HDRCRC_ERR) + icp->pdu_flags |= SBUF_ULP_FLAG_HCRC_ERROR; + if (val & F_DDP_DATACRC_ERR) + icp->pdu_flags |= SBUF_ULP_FLAG_DCRC_ERROR; + if (ip->ip_data_mbuf == NULL) { + /* XXXNP: what should ip->ip_data_len be, and why? */ + icp->pdu_flags |= SBUF_ULP_FLAG_DATA_DDPED; + } + pdu_len = ntohs(cpl->len); /* includes everything. */ + + INP_WLOCK(inp); + if (__predict_false(inp->inp_flags & (INP_DROPPED | INP_TIMEWAIT))) { + CTR4(KTR_CXGBE, "%s: tid %u, rx (%d bytes), inp_flags 0x%x", + __func__, tid, pdu_len, inp->inp_flags); + INP_WUNLOCK(inp); + icl_cxgbei_conn_pdu_free(NULL, ip); +#ifdef INVARIANTS + toep->ulpcb2 = NULL; +#endif + return (0); + } + + tp = intotcpcb(inp); + MPASS(icp->pdu_seq == tp->rcv_nxt); + MPASS(tp->rcv_wnd >= pdu_len); + tp->rcv_nxt += pdu_len; + tp->rcv_wnd -= pdu_len; + tp->t_rcvtime = ticks; + + /* update rx credits */ + toep->rx_credits += pdu_len; + t4_rcvd(&toep->td->tod, tp); /* XXX: sc->tom_softc.tod */ + + so = inp->inp_socket; + sb = &so->so_rcv; + SOCKBUF_LOCK(sb); + + icc = toep->ulpcb; + if (__predict_false(icc == NULL || sb->sb_state & SBS_CANTRCVMORE)) { + CTR5(KTR_CXGBE, + "%s: tid %u, excess rx (%d bytes), icc %p, sb_state 0x%x", + __func__, tid, pdu_len, icc, sb->sb_state); + SOCKBUF_UNLOCK(sb); + INP_WUNLOCK(inp); + + INP_INFO_RLOCK(&V_tcbinfo); + INP_WLOCK(inp); + tp = tcp_drop(tp, ECONNRESET); + if (tp) + INP_WUNLOCK(inp); + INP_INFO_RUNLOCK(&V_tcbinfo); + + icl_cxgbei_conn_pdu_free(NULL, ip); +#ifdef INVARIANTS + toep->ulpcb2 = NULL; +#endif + return (0); + } + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + ic = &icc->ic; + icl_cxgbei_new_pdu_set_conn(ip, ic); + + MPASS(m == NULL); /* was unused, we'll use it now. */ + m = sbcut_locked(sb, sbused(sb)); /* XXXNP: toep->sb_cc accounting? */ + if (__predict_false(m != NULL)) { + int len = m_length(m, NULL); + + /* + * PDUs were received before the tid transitioned to ULP mode. + * Convert them to icl_cxgbei_pdus and send them to ICL before + * the PDU in icp/ip. + */ + CTR3(KTR_CXGBE, "%s: tid %u, %u bytes in so_rcv", __func__, tid, + len); + + /* XXXNP: needs to be rewritten. */ + if (len == sizeof(struct iscsi_bhs) || len == 4 + sizeof(struct + iscsi_bhs)) { + struct icl_cxgbei_pdu *icp0; + struct icl_pdu *ip0; + + ip0 = icl_cxgbei_new_pdu(M_NOWAIT); + icl_cxgbei_new_pdu_set_conn(ip0, ic); + if (ip0 == NULL) + CXGBE_UNIMPLEMENTED("PDU allocation failure"); + icp0 = ip_to_icp(ip0); + icp0->pdu_seq = 0; /* XXX */ + icp0->pdu_flags = SBUF_ULP_FLAG_HDR_RCVD | + SBUF_ULP_FLAG_STATUS_RCVD; + m_copydata(m, 0, sizeof(struct iscsi_bhs), (void *)ip0->ip_bhs); + STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip0, ip_next); + } + m_freem(m); + } + +#if 0 + CTR4(KTR_CXGBE, "%s: tid %u, pdu_len %u, pdu_flags 0x%x", + __func__, tid, pdu_len, icp->pdu_flags); +#endif + + STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip, ip_next); + if ((icc->rx_flags & RXF_ACTIVE) == 0) { + struct cxgbei_worker_thread_softc *cwt = &cwt_softc[icc->cwt]; + + mtx_lock(&cwt->cwt_lock); + icc->rx_flags |= RXF_ACTIVE; + TAILQ_INSERT_TAIL(&cwt->rx_head, icc, rx_link); + if (cwt->cwt_state == CWT_SLEEPING) { + cwt->cwt_state = CWT_RUNNING; + cv_signal(&cwt->cwt_cv); + } + mtx_unlock(&cwt->cwt_lock); + } + SOCKBUF_UNLOCK(sb); + INP_WUNLOCK(inp); + +#ifdef INVARIANTS + toep->ulpcb2 = NULL; +#endif + + return (0); +} + +static void +t4_register_cpl_handler_with_tom(struct adapter *sc) +{ + + t4_register_cpl_handler(sc, CPL_ISCSI_HDR, do_rx_iscsi_hdr); + t4_register_cpl_handler(sc, CPL_ISCSI_DATA, do_rx_iscsi_data); + t4_register_cpl_handler(sc, CPL_RX_ISCSI_DDP, do_rx_iscsi_ddp); +} + +static void +t4_unregister_cpl_handler_with_tom(struct adapter *sc) +{ + + t4_register_cpl_handler(sc, CPL_ISCSI_HDR, NULL); + t4_register_cpl_handler(sc, CPL_ISCSI_DATA, NULL); + t4_register_cpl_handler(sc, CPL_RX_ISCSI_DDP, NULL); +} + +/* initiator */ +void +cxgbei_conn_task_reserve_itt(void *conn, void **prv, + void *scmd, unsigned int *itt) +{ + unsigned int tag; + tag = cxgbei_task_reserve_itt(conn, prv, scmd, itt); + if (tag) + *itt = htonl(tag); + return; +} + +/* target */ +void +cxgbei_conn_transfer_reserve_ttt(void *conn, void **prv, + void *scmd, unsigned int *ttt) +{ + unsigned int tag; + tag = cxgbei_task_reserve_ttt(conn, prv, scmd, ttt); + if (tag) + *ttt = htonl(tag); + return; +} + +void +cxgbei_cleanup_task(void *conn, void *ofld_priv) +{ + struct icl_conn *ic = (struct icl_conn *)conn; + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + struct cxgbei_task_data *tdata = ofld_priv; + struct adapter *sc = icc->sc; + struct cxgbei_data *ci = sc->iscsi_ulp_softc; + + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + MPASS(tdata != NULL); + + if (cxgbei_ulp2_is_ddp_tag(&ci->tag_format, tdata->sc_ddp_tag)) + t4_sk_ddp_tag_release(icc, tdata->sc_ddp_tag); + memset(tdata, 0, sizeof(*tdata)); +} + +static int +cxgbei_activate(struct adapter *sc) +{ + struct cxgbei_data *ci; + int rc; + + ASSERT_SYNCHRONIZED_OP(sc); + + if (uld_active(sc, ULD_ISCSI)) { + KASSERT(0, ("%s: iSCSI offload already enabled on adapter %p", + __func__, sc)); + return (0); + } + + if (sc->iscsicaps == 0 || sc->vres.iscsi.size == 0) { + device_printf(sc->dev, + "not iSCSI offload capable, or capability disabled.\n"); + return (ENOSYS); + } + + /* per-adapter softc for iSCSI */ + ci = malloc(sizeof(*ci), M_CXGBE, M_ZERO | M_NOWAIT); + if (ci == NULL) + return (ENOMEM); + + rc = cxgbei_ddp_init(sc, ci); + if (rc != 0) { + free(ci, M_CXGBE); + return (rc); + } + + t4_register_cpl_handler_with_tom(sc); + sc->iscsi_ulp_softc = ci; + + return (0); +} + +static int +cxgbei_deactivate(struct adapter *sc) +{ + + ASSERT_SYNCHRONIZED_OP(sc); + + if (sc->iscsi_ulp_softc != NULL) { + cxgbei_ddp_cleanup(sc->iscsi_ulp_softc); + t4_unregister_cpl_handler_with_tom(sc); + free(sc->iscsi_ulp_softc, M_CXGBE); + sc->iscsi_ulp_softc = NULL; + } + + return (0); +} + +static void +cxgbei_activate_all(struct adapter *sc, void *arg __unused) +{ + + if (begin_synchronized_op(sc, NULL, SLEEP_OK | INTR_OK, "t4isact") != 0) + return; + + /* Activate iSCSI if any port on this adapter has IFCAP_TOE enabled. */ + if (sc->offload_map && !uld_active(sc, ULD_ISCSI)) + (void) t4_activate_uld(sc, ULD_ISCSI); + + end_synchronized_op(sc, 0); +} + +static void +cxgbei_deactivate_all(struct adapter *sc, void *arg __unused) +{ + + if (begin_synchronized_op(sc, NULL, SLEEP_OK | INTR_OK, "t4isdea") != 0) + return; + + if (uld_active(sc, ULD_ISCSI)) + (void) t4_deactivate_uld(sc, ULD_ISCSI); + + end_synchronized_op(sc, 0); +} + +static struct uld_info cxgbei_uld_info = { + .uld_id = ULD_ISCSI, + .activate = cxgbei_activate, + .deactivate = cxgbei_deactivate, +}; + +static void +cwt_main(void *arg) +{ + struct cxgbei_worker_thread_softc *cwt = arg; + struct icl_cxgbei_conn *icc = NULL; + struct icl_conn *ic; + struct icl_pdu *ip; + struct sockbuf *sb; + STAILQ_HEAD(, icl_pdu) rx_pdus = STAILQ_HEAD_INITIALIZER(rx_pdus); + + MPASS(cwt != NULL); + + mtx_lock(&cwt->cwt_lock); + MPASS(cwt->cwt_state == 0); + cwt->cwt_state = CWT_RUNNING; + cv_signal(&cwt->cwt_cv); + + while (__predict_true(cwt->cwt_state != CWT_STOP)) { + cwt->cwt_state = CWT_RUNNING; + while ((icc = TAILQ_FIRST(&cwt->rx_head)) != NULL) { + TAILQ_REMOVE(&cwt->rx_head, icc, rx_link); + mtx_unlock(&cwt->cwt_lock); + + ic = &icc->ic; + sb = &ic->ic_socket->so_rcv; + + SOCKBUF_LOCK(sb); + MPASS(icc->rx_flags & RXF_ACTIVE); + if (__predict_true(!(sb->sb_state & SBS_CANTRCVMORE))) { + MPASS(STAILQ_EMPTY(&rx_pdus)); + STAILQ_SWAP(&icc->rcvd_pdus, &rx_pdus, icl_pdu); + SOCKBUF_UNLOCK(sb); + + /* Hand over PDUs to ICL. */ + while ((ip = STAILQ_FIRST(&rx_pdus)) != NULL) { + STAILQ_REMOVE_HEAD(&rx_pdus, ip_next); + ic->ic_receive(ip); + } + + SOCKBUF_LOCK(sb); + MPASS(STAILQ_EMPTY(&rx_pdus)); + } + MPASS(icc->rx_flags & RXF_ACTIVE); + if (STAILQ_EMPTY(&icc->rcvd_pdus) || + __predict_false(sb->sb_state & SBS_CANTRCVMORE)) { + icc->rx_flags &= ~RXF_ACTIVE; + } else { + /* + * More PDUs were received while we were busy + * handing over the previous batch to ICL. + * Re-add this connection to the end of the + * queue. + */ + mtx_lock(&cwt->cwt_lock); + TAILQ_INSERT_TAIL(&cwt->rx_head, icc, + rx_link); + mtx_unlock(&cwt->cwt_lock); + } + SOCKBUF_UNLOCK(sb); + + mtx_lock(&cwt->cwt_lock); + } + + /* Inner loop doesn't check for CWT_STOP, do that first. */ + if (__predict_false(cwt->cwt_state == CWT_STOP)) + break; + cwt->cwt_state = CWT_SLEEPING; + cv_wait(&cwt->cwt_cv, &cwt->cwt_lock); + } + + MPASS(TAILQ_FIRST(&cwt->rx_head) == NULL); + mtx_assert(&cwt->cwt_lock, MA_OWNED); + cwt->cwt_state = CWT_STOPPED; + cv_signal(&cwt->cwt_cv); + mtx_unlock(&cwt->cwt_lock); + kthread_exit(); +} + +static int +start_worker_threads(void) +{ + int i, rc; + struct cxgbei_worker_thread_softc *cwt; + + worker_thread_count = min(mp_ncpus, 32); + cwt_softc = malloc(worker_thread_count * sizeof(*cwt), M_CXGBE, + M_WAITOK | M_ZERO); + + MPASS(cxgbei_proc == NULL); + for (i = 0, cwt = &cwt_softc[0]; i < worker_thread_count; i++, cwt++) { + mtx_init(&cwt->cwt_lock, "cwt lock", NULL, MTX_DEF); + cv_init(&cwt->cwt_cv, "cwt cv"); + TAILQ_INIT(&cwt->rx_head); + rc = kproc_kthread_add(cwt_main, cwt, &cxgbei_proc, NULL, 0, 0, + "cxgbei", "%d", i); + if (rc != 0) { + printf("cxgbei: failed to start thread #%d/%d (%d)\n", + i + 1, worker_thread_count, rc); + mtx_destroy(&cwt->cwt_lock); + cv_destroy(&cwt->cwt_cv); + bzero(&cwt, sizeof(*cwt)); + if (i == 0) { + free(cwt_softc, M_CXGBE); + worker_thread_count = 0; + + return (rc); + } + + /* Not fatal, carry on with fewer threads. */ + worker_thread_count = i; + rc = 0; + break; + } + + /* Wait for thread to start before moving on to the next one. */ + mtx_lock(&cwt->cwt_lock); + while (cwt->cwt_state == 0) + cv_wait(&cwt->cwt_cv, &cwt->cwt_lock); + mtx_unlock(&cwt->cwt_lock); + } + + MPASS(cwt_softc != NULL); + MPASS(worker_thread_count > 0); + return (0); +} + +static void +stop_worker_threads(void) +{ + int i; + struct cxgbei_worker_thread_softc *cwt = &cwt_softc[0]; + + MPASS(worker_thread_count >= 0); + + for (i = 0, cwt = &cwt_softc[0]; i < worker_thread_count; i++, cwt++) { + mtx_lock(&cwt->cwt_lock); + MPASS(cwt->cwt_state == CWT_RUNNING || + cwt->cwt_state == CWT_SLEEPING); + cwt->cwt_state = CWT_STOP; + cv_signal(&cwt->cwt_cv); + do { + cv_wait(&cwt->cwt_cv, &cwt->cwt_lock); + } while (cwt->cwt_state != CWT_STOPPED); + mtx_unlock(&cwt->cwt_lock); + } + free(cwt_softc, M_CXGBE); +} + +/* Select a worker thread for a connection. */ +u_int +cxgbei_select_worker_thread(struct icl_cxgbei_conn *icc) +{ + struct adapter *sc = icc->sc; + struct toepcb *toep = icc->toep; + u_int i, n; + + n = worker_thread_count / sc->sge.nofldrxq; + if (n > 0) + i = toep->vi->pi->port_id * n + arc4random() % n; + else + i = arc4random() % worker_thread_count; + + CTR3(KTR_CXGBE, "%s: tid %u, cwt %u", __func__, toep->tid, i); + + return (i); +} + +static int +cxgbei_mod_load(void) +{ + int rc; + + rc = start_worker_threads(); + if (rc != 0) + return (rc); + + rc = t4_register_uld(&cxgbei_uld_info); + if (rc != 0) { + stop_worker_threads(); + return (rc); + } + + t4_iterate(cxgbei_activate_all, NULL); + + return (rc); +} + +static int +cxgbei_mod_unload(void) +{ + + t4_iterate(cxgbei_deactivate_all, NULL); + + if (t4_unregister_uld(&cxgbei_uld_info) == EBUSY) + return (EBUSY); + + stop_worker_threads(); + + return (0); +} +#endif + +static int +cxgbei_modevent(module_t mod, int cmd, void *arg) +{ + int rc = 0; + +#ifdef TCP_OFFLOAD + switch (cmd) { + case MOD_LOAD: + rc = cxgbei_mod_load(); + break; + + case MOD_UNLOAD: + rc = cxgbei_mod_unload(); + break; + + default: + rc = EINVAL; + } +#else + printf("cxgbei: compiled without TCP_OFFLOAD support.\n"); + rc = EOPNOTSUPP; +#endif + + return (rc); +} + +static moduledata_t cxgbei_mod = { + "cxgbei", + cxgbei_modevent, + NULL, +}; + +MODULE_VERSION(cxgbei, 1); +DECLARE_MODULE(cxgbei, cxgbei_mod, SI_SUB_EXEC, SI_ORDER_ANY); +MODULE_DEPEND(cxgbei, t4_tom, 1, 1, 1); +MODULE_DEPEND(cxgbei, cxgbe, 1, 1, 1); +MODULE_DEPEND(cxgbei, icl, 1, 1, 1); diff --git a/sys/dev/cxgbe/cxgbei/cxgbei.h b/sys/dev/cxgbe/cxgbei/cxgbei.h new file mode 100644 index 0000000..10e1296 --- /dev/null +++ b/sys/dev/cxgbe/cxgbei/cxgbei.h @@ -0,0 +1,167 @@ +/*- + * Copyright (c) 2012, 2015 Chelsio Communications, Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + * + */ + +#ifndef __CXGBEI_OFLD_H__ +#define __CXGBEI_OFLD_H__ + +#include <dev/iscsi/icl.h> + +enum { + CWT_SLEEPING = 1, + CWT_RUNNING = 2, + CWT_STOP = 3, + CWT_STOPPED = 4, +}; + +struct cxgbei_worker_thread_softc { + struct mtx cwt_lock; + struct cv cwt_cv; + volatile int cwt_state; + + TAILQ_HEAD(, icl_cxgbei_conn) rx_head; +} __aligned(CACHE_LINE_SIZE); + +#define CXGBEI_CONN_SIGNATURE 0x56788765 + +enum { + RXF_ACTIVE = 1 << 0, /* In the worker thread's queue */ +}; + +struct icl_cxgbei_conn { + struct icl_conn ic; + + /* cxgbei specific stuff goes here. */ + uint32_t icc_signature; + int ulp_submode; + struct adapter *sc; + struct toepcb *toep; + + /* Receive related. */ + u_int rx_flags; /* protected by so_rcv lock */ + u_int cwt; + STAILQ_HEAD(, icl_pdu) rcvd_pdus; /* protected by so_rcv lock */ + TAILQ_ENTRY(icl_cxgbei_conn) rx_link; /* protected by cwt lock */ +}; + +static inline struct icl_cxgbei_conn * +ic_to_icc(struct icl_conn *ic) +{ + + return (__containerof(ic, struct icl_cxgbei_conn, ic)); +} + +#define CXGBEI_PDU_SIGNATURE 0x12344321 + +struct icl_cxgbei_pdu { + struct icl_pdu ip; + + /* cxgbei specific stuff goes here. */ + uint32_t icp_signature; + uint32_t pdu_seq; /* For debug only */ + u_int pdu_flags; +}; + +static inline struct icl_cxgbei_pdu * +ip_to_icp(struct icl_pdu *ip) +{ + + return (__containerof(ip, struct icl_cxgbei_pdu, ip)); +} + +struct cxgbei_sgl { + int sg_flag; + void *sg_addr; + void *sg_dma_addr; + size_t sg_offset; + size_t sg_length; +}; + +#define cxgbei_scsi_for_each_sg(_sgl, _sgel, _n, _i) \ + for (_i = 0, _sgel = (cxgbei_sgl*) (_sgl); _i < _n; _i++, \ + _sgel++) +#define sg_dma_addr(_sgel) _sgel->sg_dma_addr +#define sg_virt(_sgel) _sgel->sg_addr +#define sg_len(_sgel) _sgel->sg_length +#define sg_off(_sgel) _sgel->sg_offset +#define sg_next(_sgel) _sgel + 1 + +#define SBUF_ULP_FLAG_HDR_RCVD 0x1 +#define SBUF_ULP_FLAG_DATA_RCVD 0x2 +#define SBUF_ULP_FLAG_STATUS_RCVD 0x4 +#define SBUF_ULP_FLAG_HCRC_ERROR 0x10 +#define SBUF_ULP_FLAG_DCRC_ERROR 0x20 +#define SBUF_ULP_FLAG_PAD_ERROR 0x40 +#define SBUF_ULP_FLAG_DATA_DDPED 0x80 + +/* private data for each scsi task */ +struct cxgbei_task_data { + struct cxgbei_sgl sgl[256]; + u_int nsge; + u_int sc_ddp_tag; +}; + +struct cxgbei_ulp2_tag_format { + u_char sw_bits; + u_char rsvd_bits; + u_char rsvd_shift; + u_char filler[1]; + uint32_t rsvd_mask; +}; + +struct cxgbei_data { + u_int max_txsz; + u_int max_rxsz; + u_int llimit; + u_int ulimit; + u_int nppods; + u_int idx_last; + u_char idx_bits; + uint32_t idx_mask; + uint32_t rsvd_tag_mask; + + struct mtx map_lock; + bus_dma_tag_t ulp_ddp_tag; + unsigned char *colors; + struct cxgbei_ulp2_gather_list **gl_map; + + struct cxgbei_ulp2_tag_format tag_format; +}; + +void cxgbei_conn_task_reserve_itt(void *, void **, void *, unsigned int *); +void cxgbei_conn_transfer_reserve_ttt(void *, void **, void *, unsigned int *); +void cxgbei_cleanup_task(void *, void *); +u_int cxgbei_select_worker_thread(struct icl_cxgbei_conn *); + +struct cxgbei_ulp2_pagepod_hdr; +int t4_ddp_set_map(struct cxgbei_data *, void *, + struct cxgbei_ulp2_pagepod_hdr *, u_int, u_int, + struct cxgbei_ulp2_gather_list *, int); +void t4_ddp_clear_map(struct cxgbei_data *, struct cxgbei_ulp2_gather_list *, + u_int, u_int, u_int, struct icl_cxgbei_conn *); +#endif diff --git a/sys/dev/cxgbe/cxgbei/cxgbei_ulp2_ddp.c b/sys/dev/cxgbe/cxgbei/cxgbei_ulp2_ddp.c new file mode 100644 index 0000000..fd7cd4a --- /dev/null +++ b/sys/dev/cxgbe/cxgbei/cxgbei_ulp2_ddp.c @@ -0,0 +1,417 @@ +/*- + * Copyright (c) 2012 Chelsio Communications, Inc. + * All rights reserved. + * + * Chelsio T5xx iSCSI driver + * cxgbei_ulp2_ddp.c: Chelsio iSCSI DDP Manager. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include "opt_inet.h" +#include "opt_inet6.h" + +#ifdef TCP_OFFLOAD +#include <sys/types.h> +#include <sys/module.h> +#include <sys/systm.h> +#include <sys/errno.h> +#include <sys/param.h> +#include <sys/kernel.h> +#include <sys/socket.h> +#include <sys/socketvar.h> +#include <sys/mbuf.h> +#include <sys/lock.h> +#include <sys/mutex.h> +#include <sys/condvar.h> + +#include <netinet/in.h> +#include <netinet/in_pcb.h> +#include <netinet/toecore.h> +#include <netinet/tcp_var.h> +#include <netinet/tcp_fsm.h> + +#include <dev/iscsi/icl.h> +#include <dev/iscsi/iscsi_proto.h> + +#include "common/common.h" +#include "common/t4_msg.h" +#include "common/t4_regs.h" /* for PCIE_MEM_ACCESS */ +#include "tom/t4_tom.h" +#include "cxgbei.h" +#include "cxgbei_ulp2_ddp.h" + +/* + * Map a single buffer address. + */ +static void +ulp2_dma_map_addr(void *arg, bus_dma_segment_t *segs, int nseg, int error) +{ + bus_addr_t *ba = arg; + if (error) + return; + + KASSERT(nseg == 1, ("%s: %d segments returned!", __func__, nseg)); + + *ba = segs->ds_addr; +} + +/* + * iSCSI Direct Data Placement + * + * T4/5 ulp2 h/w can directly place the iSCSI Data-In or Data-Out PDU's + * payload into pre-posted final destination host-memory buffers based on the + * Initiator Task Tag (ITT) in Data-In or Target Task Tag (TTT) in Data-Out + * PDUs. + * + * The host memory address is programmed into h/w in the format of pagepod + * entries. + * The location of the pagepod entry is encoded into ddp tag which is used or + * is the base for ITT/TTT. + */ + + +static inline int +ddp_find_unused_entries(struct cxgbei_data *ci, u_int start, u_int max, + u_int count, u_int *idx, struct cxgbei_ulp2_gather_list *gl) +{ + unsigned int i, j, k; + + /* not enough entries */ + if (max - start < count) + return (EBUSY); + + max -= count; + mtx_lock(&ci->map_lock); + for (i = start; i < max;) { + for (j = 0, k = i; j < count; j++, k++) { + if (ci->gl_map[k]) + break; + } + if (j == count) { + for (j = 0, k = i; j < count; j++, k++) + ci->gl_map[k] = gl; + mtx_unlock(&ci->map_lock); + *idx = i; + return (0); + } + i += j + 1; + } + mtx_unlock(&ci->map_lock); + return (EBUSY); +} + +static inline void +ddp_unmark_entries(struct cxgbei_data *ci, u_int start, u_int count) +{ + + mtx_lock(&ci->map_lock); + memset(&ci->gl_map[start], 0, + count * sizeof(struct cxgbei_ulp2_gather_list *)); + mtx_unlock(&ci->map_lock); +} + +static inline void +ddp_gl_unmap(struct cxgbei_data *ci, struct cxgbei_ulp2_gather_list *gl) +{ + int i; + + if (!gl->pages[0]) + return; + + for (i = 0; i < gl->nelem; i++) { + bus_dmamap_unload(ci->ulp_ddp_tag, gl->dma_sg[i].bus_map); + bus_dmamap_destroy(ci->ulp_ddp_tag, gl->dma_sg[i].bus_map); + } +} + +static inline int +ddp_gl_map(struct cxgbei_data *ci, struct cxgbei_ulp2_gather_list *gl) +{ + int i, rc; + bus_addr_t pa; + + MPASS(ci != NULL); + + mtx_lock(&ci->map_lock); + for (i = 0; i < gl->nelem; i++) { + rc = bus_dmamap_create(ci->ulp_ddp_tag, 0, + &gl->dma_sg[i].bus_map); + if (rc != 0) + goto unmap; + rc = bus_dmamap_load(ci->ulp_ddp_tag, gl->dma_sg[i].bus_map, + gl->pages[i], PAGE_SIZE, ulp2_dma_map_addr, + &pa, BUS_DMA_NOWAIT); + if (rc != 0) + goto unmap; + gl->dma_sg[i].phys_addr = pa; + } + mtx_unlock(&ci->map_lock); + + return (0); + +unmap: + if (i) { + u_int nelem = gl->nelem; + + gl->nelem = i; + ddp_gl_unmap(ci, gl); + gl->nelem = nelem; + } + return (ENOMEM); +} + +/** + * cxgbei_ulp2_ddp_make_gl_from_iscsi_sgvec - build ddp page buffer list + * @xferlen: total buffer length + * @sgl: page buffer scatter-gather list (struct cxgbei_sgl) + * @sgcnt: # of page buffers + * @gfp: allocation mode + * + * construct a ddp page buffer list from the scsi scattergather list. + * coalesce buffers as much as possible, and obtain dma addresses for + * each page. + * + * Return the cxgbei_ulp2_gather_list constructed from the page buffers if the + * memory can be used for ddp. Return NULL otherwise. + */ +struct cxgbei_ulp2_gather_list * +cxgbei_ulp2_ddp_make_gl_from_iscsi_sgvec(u_int xferlen, struct cxgbei_sgl *sgl, + u_int sgcnt, struct cxgbei_data *ci, int gfp) +{ + struct cxgbei_ulp2_gather_list *gl; + struct cxgbei_sgl *sg = sgl; + void *sgpage = (void *)((u64)sg->sg_addr & (~PAGE_MASK)); + unsigned int sglen = sg->sg_length; + unsigned int sgoffset = (u64)sg->sg_addr & PAGE_MASK; + unsigned int npages = (xferlen + sgoffset + PAGE_SIZE - 1) >> + PAGE_SHIFT; + int i = 1, j = 0; + + if (xferlen <= DDP_THRESHOLD) { + CTR2(KTR_CXGBE, "xfer %u < threshold %u, no ddp.", + xferlen, DDP_THRESHOLD); + return NULL; + } + + gl = malloc(sizeof(struct cxgbei_ulp2_gather_list) + + npages * (sizeof(struct dma_segments) + sizeof(void *)), + M_DEVBUF, M_NOWAIT | M_ZERO); + if (gl == NULL) + return (NULL); + + gl->pages = (void **)&gl->dma_sg[npages]; + gl->length = xferlen; + gl->offset = sgoffset; + gl->pages[0] = sgpage; + CTR6(KTR_CXGBE, + "%s: xferlen:0x%x len:0x%x off:0x%x sg_addr:%p npages:%d", + __func__, xferlen, gl->length, gl->offset, sg->sg_addr, npages); + + for (i = 1, sg = sg_next(sg); i < sgcnt; i++, sg = sg_next(sg)) { + void *page = sg->sg_addr; + + if (sgpage == page && sg->sg_offset == sgoffset + sglen) + sglen += sg->sg_length; + else { + /* make sure the sgl is fit for ddp: + * each has the same page size, and + * all of the middle pages are used completely + */ + if ((j && sgoffset) || + ((i != sgcnt - 1) && + ((sglen + sgoffset) & ~CXGBEI_PAGE_MASK))){ + goto error_out; + } + + j++; + if (j == gl->nelem || sg->sg_offset) { + goto error_out; + } + gl->pages[j] = page; + sglen = sg->sg_length; + sgoffset = sg->sg_offset; + sgpage = page; + } + } + gl->nelem = ++j; + + if (ddp_gl_map(ci, gl) < 0) + goto error_out; + + return gl; + +error_out: + free(gl, M_DEVBUF); + return NULL; +} + +/** + * cxgbei_ulp2_ddp_release_gl - release a page buffer list + * @gl: a ddp page buffer list + * @pdev: pci_dev used for pci_unmap + * free a ddp page buffer list resulted from cxgbei_ulp2_ddp_make_gl(). + */ +void +cxgbei_ulp2_ddp_release_gl(struct cxgbei_data *ci, + struct cxgbei_ulp2_gather_list *gl) +{ + + ddp_gl_unmap(ci, gl); + free(gl, M_DEVBUF); +} + +/** + * cxgbei_ulp2_ddp_tag_reserve - set up ddp for a data transfer + * @ci: adapter's ddp info + * @tid: connection id + * @tformat: tag format + * @tagp: contains s/w tag initially, will be updated with ddp/hw tag + * @gl: the page momory list + * @gfp: allocation mode + * + * ddp setup for a given page buffer list and construct the ddp tag. + * return 0 if success, < 0 otherwise. + */ +int +cxgbei_ulp2_ddp_tag_reserve(struct cxgbei_data *ci, void *icc, u_int tid, + struct cxgbei_ulp2_tag_format *tformat, u32 *tagp, + struct cxgbei_ulp2_gather_list *gl, int gfp, int reply) +{ + struct cxgbei_ulp2_pagepod_hdr hdr; + u_int npods, idx; + int rc; + u32 sw_tag = *tagp; + u32 tag; + + MPASS(ci != NULL); + + if (!gl || !gl->nelem || gl->length < DDP_THRESHOLD) + return (EINVAL); + + npods = (gl->nelem + IPPOD_PAGES_MAX - 1) >> IPPOD_PAGES_SHIFT; + + if (ci->idx_last == ci->nppods) + rc = ddp_find_unused_entries(ci, 0, ci->nppods, npods, &idx, + gl); + else { + rc = ddp_find_unused_entries(ci, ci->idx_last + 1, + ci->nppods, npods, &idx, gl); + if (rc && ci->idx_last >= npods) { + rc = ddp_find_unused_entries(ci, 0, + min(ci->idx_last + npods, ci->nppods), + npods, &idx, gl); + } + } + if (rc) { + CTR3(KTR_CXGBE, "xferlen %u, gl %u, npods %u NO DDP.", + gl->length, gl->nelem, npods); + return (rc); + } + + tag = cxgbei_ulp2_ddp_tag_base(idx, ci->colors, tformat, sw_tag); + CTR4(KTR_CXGBE, "%s: sw_tag:0x%x idx:0x%x tag:0x%x", + __func__, sw_tag, idx, tag); + + hdr.rsvd = 0; + hdr.vld_tid = htonl(F_IPPOD_VALID | V_IPPOD_TID(tid)); + hdr.pgsz_tag_clr = htonl(tag & ci->rsvd_tag_mask); + hdr.maxoffset = htonl(gl->length); + hdr.pgoffset = htonl(gl->offset); + + rc = t4_ddp_set_map(ci, icc, &hdr, idx, npods, gl, reply); + if (rc < 0) + goto unmark_entries; + + ci->idx_last = idx; + *tagp = tag; + return (0); + +unmark_entries: + ddp_unmark_entries(ci, idx, npods); + return (rc); +} + +/** + * cxgbei_ulp2_ddp_tag_release - release a ddp tag + * @ci: adapter's ddp info + * @tag: ddp tag + * ddp cleanup for a given ddp tag and release all the resources held + */ +void +cxgbei_ulp2_ddp_tag_release(struct cxgbei_data *ci, uint32_t tag, + struct icl_cxgbei_conn *icc) +{ + uint32_t idx; + + MPASS(ci != NULL); + MPASS(icc != NULL); + + idx = (tag >> IPPOD_IDX_SHIFT) & ci->idx_mask; + CTR3(KTR_CXGBE, "tag:0x%x idx:0x%x nppods:0x%x", + tag, idx, ci->nppods); + if (idx < ci->nppods) { + struct cxgbei_ulp2_gather_list *gl = ci->gl_map[idx]; + unsigned int npods; + + if (!gl || !gl->nelem) { + CTR4(KTR_CXGBE, + "release 0x%x, idx 0x%x, gl 0x%p, %u.", + tag, idx, gl, gl ? gl->nelem : 0); + return; + } + npods = (gl->nelem + IPPOD_PAGES_MAX - 1) >> IPPOD_PAGES_SHIFT; + CTR3(KTR_CXGBE, "ddp tag 0x%x, release idx 0x%x, npods %u.", + tag, idx, npods); + t4_ddp_clear_map(ci, gl, tag, idx, npods, icc); + ddp_unmark_entries(ci, idx, npods); + cxgbei_ulp2_ddp_release_gl(ci, gl); + } else + CTR3(KTR_CXGBE, "ddp tag 0x%x, idx 0x%x > max 0x%x.", + tag, idx, ci->nppods); +} + +/** + * cxgbei_ddp_cleanup - release the adapter's ddp resources + */ +void +cxgbei_ddp_cleanup(struct cxgbei_data *ci) +{ + int i = 0; + + while (i < ci->nppods) { + struct cxgbei_ulp2_gather_list *gl = ci->gl_map[i]; + if (gl) { + int npods = (gl->nelem + IPPOD_PAGES_MAX - 1) + >> IPPOD_PAGES_SHIFT; + free(gl, M_DEVBUF); + i += npods; + } else + i++; + } + free(ci->colors, M_CXGBE); + free(ci->gl_map, M_CXGBE); +} +#endif diff --git a/sys/dev/cxgbe/cxgbei/cxgbei_ulp2_ddp.h b/sys/dev/cxgbe/cxgbei/cxgbei_ulp2_ddp.h new file mode 100644 index 0000000..f069f09 --- /dev/null +++ b/sys/dev/cxgbe/cxgbei/cxgbei_ulp2_ddp.h @@ -0,0 +1,217 @@ +/*- + * Copyright (c) 2012 Chelsio Communications, Inc. + * All rights reserved. + * + * Chelsio T5xx iSCSI driver + * cxgbei_ulp2_ddp.c: Chelsio iSCSI DDP Manager. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + * + */ + +#ifndef __CXGBEI_ULP2_DDP_H__ +#define __CXGBEI_ULP2_DDP_H__ + +#define CXGBEI_PAGE_MASK (~(PAGE_SIZE-1)) +#define DDP_THRESHOLD 2048 + +/* + * cxgbei ddp tag are 32 bits, it consists of reserved bits used by h/w and + * non-reserved bits that can be used by the iscsi s/w. + * The reserved bits are identified by the rsvd_bits and rsvd_shift fields + * in struct cxgbei_ulp2_tag_format. + * + * The upper most reserved bit can be used to check if a tag is ddp tag or not: + * if the bit is 0, the tag is a valid ddp tag + */ + +/* + * cxgbei_ulp2_is_ddp_tag - check if a given tag is a hw/ddp tag + * @tformat: tag format information + * @tag: tag to be checked + * + * return true if the tag is a ddp tag, false otherwise. + */ +static inline int +cxgbei_ulp2_is_ddp_tag(struct cxgbei_ulp2_tag_format *tformat, uint32_t tag) +{ + + return (!(tag & (1 << (tformat->rsvd_bits + tformat->rsvd_shift - 1)))); +} + +/* + * cxgbei_ulp2_sw_tag_usable - check if s/w tag has enough bits left for hw bits + * @tformat: tag format information + * @sw_tag: s/w tag to be checked + * + * return true if the tag can be used for hw ddp tag, false otherwise. + */ +static inline int +cxgbei_ulp2_sw_tag_usable(struct cxgbei_ulp2_tag_format *tformat, + uint32_t sw_tag) +{ + + return (1); /* XXXNP: huh? */ + + sw_tag >>= (32 - tformat->rsvd_bits + tformat->rsvd_shift); + return !sw_tag; +} + +/* + * cxgbei_ulp2_set_non_ddp_tag - mark a given s/w tag as an invalid ddp tag + * @tformat: tag format information + * @sw_tag: s/w tag to be checked + * + * insert 1 at the upper most reserved bit to mark it as an invalid ddp tag. + */ +static inline uint32_t +cxgbei_ulp2_set_non_ddp_tag(struct cxgbei_ulp2_tag_format *tformat, + uint32_t sw_tag) +{ + uint32_t rsvd_bits = tformat->rsvd_bits + tformat->rsvd_shift; + if (sw_tag) { + u32 v1 = sw_tag & ((1 << (rsvd_bits - 1)) - 1); + u32 v2 = (sw_tag >> (rsvd_bits - 1)) << rsvd_bits; + return v2 | (1 << (rsvd_bits - 1)) | v1; + } + + return sw_tag | (1 << (rsvd_bits - 1)) ; +} + +struct dma_segments { + bus_dmamap_t bus_map; + bus_addr_t phys_addr; +}; +/* + * struct cxgbei_ulp2_gather_list - cxgbei direct data placement memory + * + * @tag: ddp tag + * @length: total data buffer length + * @offset: initial offset to the 1st page + * @nelem: # of pages + * @pages: page pointers + * @phys_addr: physical address + */ +struct cxgbei_ulp2_gather_list { + uint32_t tag; + uint32_t tid; + uint32_t port_id; + void *egress_dev; + unsigned int length; + unsigned int offset; + unsigned int nelem; + bus_size_t mapsize; + bus_dmamap_t bus_map; + bus_dma_segment_t *segments; + void **pages; + struct dma_segments dma_sg[0]; +}; + +#define IPPOD_SIZE sizeof(struct cxgbei_ulp2_pagepod) /* 64 */ +#define IPPOD_SIZE_SHIFT 6 + +#define IPPOD_COLOR_SHIFT 0 +#define IPPOD_COLOR_SIZE 6 +#define IPPOD_COLOR_MASK ((1 << IPPOD_COLOR_SIZE) - 1) + +#define IPPOD_IDX_SHIFT IPPOD_COLOR_SIZE +#define IPPOD_IDX_MAX_SIZE 24 + +#define S_IPPOD_TID 0 +#define M_IPPOD_TID 0xFFFFFF +#define V_IPPOD_TID(x) ((x) << S_IPPOD_TID) + +#define S_IPPOD_VALID 24 +#define V_IPPOD_VALID(x) ((x) << S_IPPOD_VALID) +#define F_IPPOD_VALID V_IPPOD_VALID(1U) + +#define S_IPPOD_COLOR 0 +#define M_IPPOD_COLOR 0x3F +#define V_IPPOD_COLOR(x) ((x) << S_IPPOD_COLOR) + +#define S_IPPOD_TAG 6 +#define M_IPPOD_TAG 0xFFFFFF +#define V_IPPOD_TAG(x) ((x) << S_IPPOD_TAG) + +#define S_IPPOD_PGSZ 30 +#define M_IPPOD_PGSZ 0x3 +#define V_IPPOD_PGSZ(x) ((x) << S_IPPOD_PGSZ) + +static inline uint32_t +cxgbei_ulp2_ddp_tag_base(u_int idx, u_char *colors, + struct cxgbei_ulp2_tag_format *tformat, uint32_t sw_tag) +{ + if (__predict_false(++colors[idx] == 1 << IPPOD_IDX_SHIFT)) + colors[idx] = 0; + + sw_tag <<= tformat->rsvd_bits + tformat->rsvd_shift; + + return (sw_tag | idx << IPPOD_IDX_SHIFT | colors[idx]); +} + +#define ISCSI_PDU_NONPAYLOAD_LEN 312 /* bhs(48) + ahs(256) + digest(8) */ + +/* + * align pdu size to multiple of 512 for better performance + */ +#define cxgbei_align_pdu_size(n) do { n = (n) & (~511); } while (0) + +#define ULP2_MAX_PKT_SIZE 16224 +#define ULP2_MAX_PDU_PAYLOAD (ULP2_MAX_PKT_SIZE - ISCSI_PDU_NONPAYLOAD_LEN) +#define IPPOD_PAGES_MAX 4 +#define IPPOD_PAGES_SHIFT 2 /* 4 pages per pod */ + +/* + * struct pagepod_hdr, pagepod - pagepod format + */ +struct cxgbei_ulp2_pagepod_hdr { + uint32_t vld_tid; + uint32_t pgsz_tag_clr; + uint32_t maxoffset; + uint32_t pgoffset; + uint64_t rsvd; +}; + +struct cxgbei_ulp2_pagepod { + struct cxgbei_ulp2_pagepod_hdr hdr; + uint64_t addr[IPPOD_PAGES_MAX + 1]; +}; + +int cxgbei_ulp2_ddp_tag_reserve(struct cxgbei_data *, void *, unsigned int, + struct cxgbei_ulp2_tag_format *, uint32_t *, + struct cxgbei_ulp2_gather_list *, int , int ); +void cxgbei_ulp2_ddp_tag_release(struct cxgbei_data *, uint32_t, + struct icl_cxgbei_conn *); + +struct cxgbei_ulp2_gather_list *cxgbei_ulp2_ddp_make_gl_from_iscsi_sgvec(u_int, + struct cxgbei_sgl *, u_int, struct cxgbei_data *, int); +void cxgbei_ulp2_ddp_release_gl(struct cxgbei_data *, + struct cxgbei_ulp2_gather_list *); + +int cxgbei_ulp2_ddp_find_page_index(u_long); +int cxgbei_ulp2_adapter_ddp_info(struct cxgbei_data *, + struct cxgbei_ulp2_tag_format *); + +void cxgbei_ddp_cleanup(struct cxgbei_data *); +#endif diff --git a/sys/dev/cxgbe/cxgbei/icl_cxgbei.c b/sys/dev/cxgbe/cxgbei/icl_cxgbei.c new file mode 100644 index 0000000..6f20281 --- /dev/null +++ b/sys/dev/cxgbe/cxgbei/icl_cxgbei.c @@ -0,0 +1,896 @@ +/*- + * Copyright (c) 2012 The FreeBSD Foundation + * Copyright (c) 2015 Chelsio Communications, Inc. + * All rights reserved. + * + * This software was developed by Edward Tomasz Napierala under sponsorship + * from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +/* + * cxgbei implementation of iSCSI Common Layer kobj(9) interface. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include "opt_inet.h" +#include "opt_inet6.h" + +#ifdef TCP_OFFLOAD +#include <sys/param.h> +#include <sys/capsicum.h> +#include <sys/condvar.h> +#include <sys/conf.h> +#include <sys/file.h> +#include <sys/kernel.h> +#include <sys/kthread.h> +#include <sys/lock.h> +#include <sys/mbuf.h> +#include <sys/mutex.h> +#include <sys/module.h> +#include <sys/protosw.h> +#include <sys/socket.h> +#include <sys/socketvar.h> +#include <sys/sysctl.h> +#include <sys/systm.h> +#include <sys/sx.h> +#include <sys/uio.h> +#include <machine/bus.h> +#include <vm/uma.h> +#include <netinet/in.h> +#include <netinet/in_pcb.h> +#include <netinet/tcp.h> +#include <netinet/tcp_var.h> +#include <netinet/toecore.h> + +#include <dev/iscsi/icl.h> +#include <dev/iscsi/iscsi_proto.h> +#include <icl_conn_if.h> + +#include "common/common.h" +#include "tom/t4_tom.h" +#include "cxgbei.h" + +SYSCTL_NODE(_kern_icl, OID_AUTO, cxgbei, CTLFLAG_RD, 0, "Chelsio iSCSI offload"); +static int coalesce = 1; +SYSCTL_INT(_kern_icl_cxgbei, OID_AUTO, coalesce, CTLFLAG_RWTUN, + &coalesce, 0, "Try to coalesce PDUs before sending"); +static int partial_receive_len = 128 * 1024; +SYSCTL_INT(_kern_icl_cxgbei, OID_AUTO, partial_receive_len, CTLFLAG_RWTUN, + &partial_receive_len, 0, "Minimum read size for partially received " + "data segment"); +static int sendspace = 1048576; +SYSCTL_INT(_kern_icl_cxgbei, OID_AUTO, sendspace, CTLFLAG_RWTUN, + &sendspace, 0, "Default send socket buffer size"); +static int recvspace = 1048576; +SYSCTL_INT(_kern_icl_cxgbei, OID_AUTO, recvspace, CTLFLAG_RWTUN, + &recvspace, 0, "Default receive socket buffer size"); + +static uma_zone_t icl_transfer_zone; + +static volatile u_int icl_cxgbei_ncons; + +#define ICL_CONN_LOCK(X) mtx_lock(X->ic_lock) +#define ICL_CONN_UNLOCK(X) mtx_unlock(X->ic_lock) +#define ICL_CONN_LOCK_ASSERT(X) mtx_assert(X->ic_lock, MA_OWNED) +#define ICL_CONN_LOCK_ASSERT_NOT(X) mtx_assert(X->ic_lock, MA_NOTOWNED) + +struct icl_pdu *icl_cxgbei_new_pdu(int); +void icl_cxgbei_new_pdu_set_conn(struct icl_pdu *, struct icl_conn *); + +static icl_conn_new_pdu_t icl_cxgbei_conn_new_pdu; +icl_conn_pdu_free_t icl_cxgbei_conn_pdu_free; +static icl_conn_pdu_data_segment_length_t + icl_cxgbei_conn_pdu_data_segment_length; +static icl_conn_pdu_append_data_t icl_cxgbei_conn_pdu_append_data; +static icl_conn_pdu_get_data_t icl_cxgbei_conn_pdu_get_data; +static icl_conn_pdu_queue_t icl_cxgbei_conn_pdu_queue; +static icl_conn_handoff_t icl_cxgbei_conn_handoff; +static icl_conn_free_t icl_cxgbei_conn_free; +static icl_conn_close_t icl_cxgbei_conn_close; +static icl_conn_task_setup_t icl_cxgbei_conn_task_setup; +static icl_conn_task_done_t icl_cxgbei_conn_task_done; +static icl_conn_transfer_setup_t icl_cxgbei_conn_transfer_setup; +static icl_conn_transfer_done_t icl_cxgbei_conn_transfer_done; + +static kobj_method_t icl_cxgbei_methods[] = { + KOBJMETHOD(icl_conn_new_pdu, icl_cxgbei_conn_new_pdu), + KOBJMETHOD(icl_conn_pdu_free, icl_cxgbei_conn_pdu_free), + KOBJMETHOD(icl_conn_pdu_data_segment_length, + icl_cxgbei_conn_pdu_data_segment_length), + KOBJMETHOD(icl_conn_pdu_append_data, icl_cxgbei_conn_pdu_append_data), + KOBJMETHOD(icl_conn_pdu_get_data, icl_cxgbei_conn_pdu_get_data), + KOBJMETHOD(icl_conn_pdu_queue, icl_cxgbei_conn_pdu_queue), + KOBJMETHOD(icl_conn_handoff, icl_cxgbei_conn_handoff), + KOBJMETHOD(icl_conn_free, icl_cxgbei_conn_free), + KOBJMETHOD(icl_conn_close, icl_cxgbei_conn_close), + KOBJMETHOD(icl_conn_task_setup, icl_cxgbei_conn_task_setup), + KOBJMETHOD(icl_conn_task_done, icl_cxgbei_conn_task_done), + KOBJMETHOD(icl_conn_transfer_setup, icl_cxgbei_conn_transfer_setup), + KOBJMETHOD(icl_conn_transfer_done, icl_cxgbei_conn_transfer_done), + { 0, 0 } +}; + +DEFINE_CLASS(icl_cxgbei, icl_cxgbei_methods, sizeof(struct icl_cxgbei_conn)); + +#if 0 +/* + * Subtract another 256 for AHS from MAX_DSL if AHS could be used. + */ +#define CXGBEI_MAX_PDU 16224 +#define CXGBEI_MAX_DSL (CXGBEI_MAX_PDU - sizeof(struct iscsi_bhs) - 8) +#endif +#define CXGBEI_MAX_DSL 8192 +#define CXGBEI_MAX_PDU (CXGBEI_MAX_DSL + sizeof(struct iscsi_bhs) + 8) + +void +icl_cxgbei_conn_pdu_free(struct icl_conn *ic, struct icl_pdu *ip) +{ +#ifdef INVARIANTS + struct icl_cxgbei_pdu *icp = ip_to_icp(ip); +#endif + + MPASS(icp->icp_signature == CXGBEI_PDU_SIGNATURE); + MPASS(ic == ip->ip_conn); + MPASS(ip->ip_bhs_mbuf != NULL); + + m_freem(ip->ip_ahs_mbuf); + m_freem(ip->ip_data_mbuf); + m_freem(ip->ip_bhs_mbuf); /* storage for icl_cxgbei_pdu itself */ + +#ifdef DIAGNOSTIC + if (__predict_true(ic != NULL)) + refcount_release(&ic->ic_outstanding_pdus); +#endif +} + +struct icl_pdu * +icl_cxgbei_new_pdu(int flags) +{ + struct icl_cxgbei_pdu *icp; + struct icl_pdu *ip; + struct mbuf *m; + uintptr_t a; + + m = m_gethdr(flags, MT_DATA); + if (__predict_false(m == NULL)) + return (NULL); + + a = roundup2(mtod(m, uintptr_t), _Alignof(struct icl_cxgbei_pdu)); + icp = (struct icl_cxgbei_pdu *)a; + bzero(icp, sizeof(*icp)); + + icp->icp_signature = CXGBEI_PDU_SIGNATURE; + ip = &icp->ip; + ip->ip_bhs_mbuf = m; + + a = roundup2((uintptr_t)(icp + 1), _Alignof(struct iscsi_bhs *)); + ip->ip_bhs = (struct iscsi_bhs *)a; +#ifdef INVARIANTS + /* Everything must fit entirely in the mbuf. */ + a = (uintptr_t)(ip->ip_bhs + 1); + MPASS(a <= (uintptr_t)m + MSIZE); +#endif + bzero(ip->ip_bhs, sizeof(*ip->ip_bhs)); + + m->m_data = (void *)ip->ip_bhs; + m->m_len = sizeof(struct iscsi_bhs); + m->m_pkthdr.len = m->m_len; + + return (ip); +} + +void +icl_cxgbei_new_pdu_set_conn(struct icl_pdu *ip, struct icl_conn *ic) +{ + + ip->ip_conn = ic; +#ifdef DIAGNOSTIC + refcount_acquire(&ic->ic_outstanding_pdus); +#endif +} + +/* + * Allocate icl_pdu with empty BHS to fill up by the caller. + */ +static struct icl_pdu * +icl_cxgbei_conn_new_pdu(struct icl_conn *ic, int flags) +{ + struct icl_pdu *ip; + + ip = icl_cxgbei_new_pdu(flags); + if (__predict_false(ip == NULL)) + return (NULL); + icl_cxgbei_new_pdu_set_conn(ip, ic); + + return (ip); +} + +static size_t +icl_pdu_data_segment_length(const struct icl_pdu *request) +{ + uint32_t len = 0; + + len += request->ip_bhs->bhs_data_segment_len[0]; + len <<= 8; + len += request->ip_bhs->bhs_data_segment_len[1]; + len <<= 8; + len += request->ip_bhs->bhs_data_segment_len[2]; + + return (len); +} + +size_t +icl_cxgbei_conn_pdu_data_segment_length(struct icl_conn *ic, + const struct icl_pdu *request) +{ + + return (icl_pdu_data_segment_length(request)); +} + +static uint32_t +icl_conn_build_tasktag(struct icl_conn *ic, uint32_t tag) +{ + return tag; +} + +static struct mbuf * +finalize_pdu(struct icl_cxgbei_conn *icc, struct icl_cxgbei_pdu *icp) +{ + struct icl_pdu *ip = &icp->ip; + uint8_t ulp_submode, padding; + struct mbuf *m, *last; + struct iscsi_bhs *bhs; + + /* + * Fix up the data segment mbuf first. + */ + m = ip->ip_data_mbuf; + ulp_submode = icc->ulp_submode; + if (m) { + last = m_last(m); + + /* + * Round up the data segment to a 4B boundary. Pad with 0 if + * necessary. There will definitely be room in the mbuf. + */ + padding = roundup2(ip->ip_data_len, 4) - ip->ip_data_len; + if (padding) { + bzero(mtod(last, uint8_t *) + last->m_len, padding); + last->m_len += padding; + } + } else { + MPASS(ip->ip_data_len == 0); + ulp_submode &= ~ULP_CRC_DATA; + padding = 0; + } + + /* + * Now the header mbuf that has the BHS. + */ + m = ip->ip_bhs_mbuf; + MPASS(m->m_pkthdr.len == sizeof(struct iscsi_bhs)); + MPASS(m->m_len == sizeof(struct iscsi_bhs)); + + bhs = ip->ip_bhs; + bhs->bhs_data_segment_len[2] = ip->ip_data_len; + bhs->bhs_data_segment_len[1] = ip->ip_data_len >> 8; + bhs->bhs_data_segment_len[0] = ip->ip_data_len >> 16; + + /* "Convert" PDU to mbuf chain. Do not use icp/ip after this. */ + m->m_pkthdr.len = sizeof(struct iscsi_bhs) + ip->ip_data_len + padding; + m->m_next = ip->ip_data_mbuf; + set_mbuf_ulp_submode(m, ulp_submode); +#ifdef INVARIANTS + bzero(icp, sizeof(*icp)); +#endif +#ifdef DIAGNOSTIC + refcount_release(&icc->ic.ic_outstanding_pdus); +#endif + + return (m); +} + +int +icl_cxgbei_conn_pdu_append_data(struct icl_conn *ic, struct icl_pdu *ip, + const void *addr, size_t len, int flags) +{ + struct mbuf *m; +#ifdef INVARIANTS + struct icl_cxgbei_pdu *icp = ip_to_icp(ip); +#endif + + MPASS(icp->icp_signature == CXGBEI_PDU_SIGNATURE); + MPASS(ic == ip->ip_conn); + KASSERT(len > 0, ("%s: len is %jd", __func__, (intmax_t)len)); + + m = ip->ip_data_mbuf; + if (m == NULL) { + m = m_getjcl(M_NOWAIT, MT_DATA, 0, MJUM16BYTES); + if (__predict_false(m == NULL)) + return (ENOMEM); + + ip->ip_data_mbuf = m; + } + + if (__predict_true(m_append(m, len, addr) != 0)) { + ip->ip_data_len += len; + MPASS(ip->ip_data_len <= CXGBEI_MAX_DSL); + return (0); + } else { + if (flags & M_WAITOK) { + CXGBE_UNIMPLEMENTED("fail safe append"); + } + ip->ip_data_len = m_length(m, NULL); + return (1); + } +} + +void +icl_cxgbei_conn_pdu_get_data(struct icl_conn *ic, struct icl_pdu *ip, + size_t off, void *addr, size_t len) +{ + struct icl_cxgbei_pdu *icp = ip_to_icp(ip); + + if (icp->pdu_flags & SBUF_ULP_FLAG_DATA_DDPED) + return; /* data is DDP'ed, no need to copy */ + m_copydata(ip->ip_data_mbuf, off, len, addr); +} + +void +icl_cxgbei_conn_pdu_queue(struct icl_conn *ic, struct icl_pdu *ip) +{ + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + struct icl_cxgbei_pdu *icp = ip_to_icp(ip); + struct socket *so = ic->ic_socket; + struct toepcb *toep = icc->toep; + struct inpcb *inp; + struct mbuf *m; + + MPASS(ic == ip->ip_conn); + MPASS(ip->ip_bhs_mbuf != NULL); + /* The kernel doesn't generate PDUs with AHS. */ + MPASS(ip->ip_ahs_mbuf == NULL && ip->ip_ahs_len == 0); + + ICL_CONN_LOCK_ASSERT(ic); + /* NOTE: sowriteable without so_snd lock is a mostly harmless race. */ + if (ic->ic_disconnecting || so == NULL || !sowriteable(so)) { + icl_cxgbei_conn_pdu_free(ic, ip); + return; + } + + m = finalize_pdu(icc, icp); + M_ASSERTPKTHDR(m); + MPASS((m->m_pkthdr.len & 3) == 0); + MPASS(m->m_pkthdr.len + 8 <= CXGBEI_MAX_PDU); + + /* + * Do not get inp from toep->inp as the toepcb might have detached + * already. + */ + inp = sotoinpcb(so); + INP_WLOCK(inp); + if (__predict_false(inp->inp_flags & (INP_DROPPED | INP_TIMEWAIT)) || + __predict_false((toep->flags & TPF_ATTACHED) == 0)) + m_freem(m); + else { + mbufq_enqueue(&toep->ulp_pduq, m); + t4_push_pdus(icc->sc, toep, 0); + } + INP_WUNLOCK(inp); +} + +static struct icl_conn * +icl_cxgbei_new_conn(const char *name, struct mtx *lock) +{ + struct icl_cxgbei_conn *icc; + struct icl_conn *ic; + + refcount_acquire(&icl_cxgbei_ncons); + + icc = (struct icl_cxgbei_conn *)kobj_create(&icl_cxgbei_class, M_CXGBE, + M_WAITOK | M_ZERO); + icc->icc_signature = CXGBEI_CONN_SIGNATURE; + STAILQ_INIT(&icc->rcvd_pdus); + + ic = &icc->ic; + ic->ic_lock = lock; + + /* XXXNP: review. Most of these icl_conn fields aren't really used */ + STAILQ_INIT(&ic->ic_to_send); + cv_init(&ic->ic_send_cv, "icl_cxgbei_tx"); + cv_init(&ic->ic_receive_cv, "icl_cxgbei_rx"); +#ifdef DIAGNOSTIC + refcount_init(&ic->ic_outstanding_pdus, 0); +#endif + ic->ic_max_data_segment_length = CXGBEI_MAX_DSL; + ic->ic_name = name; + ic->ic_offload = "cxgbei"; + + CTR2(KTR_CXGBE, "%s: icc %p", __func__, icc); + + return (ic); +} + +void +icl_cxgbei_conn_free(struct icl_conn *ic) +{ + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + + CTR2(KTR_CXGBE, "%s: icc %p", __func__, icc); + + cv_destroy(&ic->ic_send_cv); + cv_destroy(&ic->ic_receive_cv); + + kobj_delete((struct kobj *)icc, M_CXGBE); + refcount_release(&icl_cxgbei_ncons); +} + +static int +icl_cxgbei_setsockopt(struct icl_conn *ic, struct socket *so) +{ + size_t minspace; + struct sockopt opt; + int error, one = 1; + + /* + * For sendspace, this is required because the current code cannot + * send a PDU in pieces; thus, the minimum buffer size is equal + * to the maximum PDU size. "+4" is to account for possible padding. + * + * What we should actually do here is to use autoscaling, but set + * some minimal buffer size to "minspace". I don't know a way to do + * that, though. + */ + minspace = sizeof(struct iscsi_bhs) + ic->ic_max_data_segment_length + + ISCSI_HEADER_DIGEST_SIZE + ISCSI_DATA_DIGEST_SIZE + 4; + if (sendspace < minspace) + sendspace = minspace; + if (recvspace < minspace) + recvspace = minspace; + + error = soreserve(so, sendspace, recvspace); + if (error != 0) { + icl_cxgbei_conn_close(ic); + return (error); + } + SOCKBUF_LOCK(&so->so_snd); + so->so_snd.sb_flags |= SB_AUTOSIZE; + SOCKBUF_UNLOCK(&so->so_snd); + SOCKBUF_LOCK(&so->so_rcv); + so->so_rcv.sb_flags |= SB_AUTOSIZE; + SOCKBUF_UNLOCK(&so->so_rcv); + + /* + * Disable Nagle. + */ + bzero(&opt, sizeof(opt)); + opt.sopt_dir = SOPT_SET; + opt.sopt_level = IPPROTO_TCP; + opt.sopt_name = TCP_NODELAY; + opt.sopt_val = &one; + opt.sopt_valsize = sizeof(one); + error = sosetopt(so, &opt); + if (error != 0) { + icl_cxgbei_conn_close(ic); + return (error); + } + + return (0); +} + +/* + * Request/response structure used to find out the adapter offloading a socket. + */ +struct find_ofld_adapter_rr { + struct socket *so; + struct adapter *sc; /* result */ +}; + +static void +find_offload_adapter(struct adapter *sc, void *arg) +{ + struct find_ofld_adapter_rr *fa = arg; + struct socket *so = fa->so; + struct tom_data *td = sc->tom_softc; + struct tcpcb *tp; + struct inpcb *inp; + + /* Non-TCP were filtered out earlier. */ + MPASS(so->so_proto->pr_protocol == IPPROTO_TCP); + + if (fa->sc != NULL) + return; /* Found already. */ + + if (td == NULL) + return; /* TOE not enabled on this adapter. */ + + inp = sotoinpcb(so); + INP_WLOCK(inp); + if ((inp->inp_flags & (INP_DROPPED | INP_TIMEWAIT)) == 0) { + tp = intotcpcb(inp); + if (tp->t_flags & TF_TOE && tp->tod == &td->tod) + fa->sc = sc; /* Found. */ + } + INP_WUNLOCK(inp); +} + +/* XXXNP: move this to t4_tom. */ +static void +send_iscsi_flowc_wr(struct adapter *sc, struct toepcb *toep, int maxlen) +{ + struct wrqe *wr; + struct fw_flowc_wr *flowc; + const u_int nparams = 1; + u_int flowclen; + struct ofld_tx_sdesc *txsd = &toep->txsd[toep->txsd_pidx]; + + flowclen = sizeof(*flowc) + nparams * sizeof(struct fw_flowc_mnemval); + + wr = alloc_wrqe(roundup2(flowclen, 16), toep->ofld_txq); + if (wr == NULL) { + /* XXX */ + panic("%s: allocation failure.", __func__); + } + flowc = wrtod(wr); + memset(flowc, 0, wr->wr_len); + + flowc->op_to_nparams = htobe32(V_FW_WR_OP(FW_FLOWC_WR) | + V_FW_FLOWC_WR_NPARAMS(nparams)); + flowc->flowid_len16 = htonl(V_FW_WR_LEN16(howmany(flowclen, 16)) | + V_FW_WR_FLOWID(toep->tid)); + + flowc->mnemval[0].mnemonic = FW_FLOWC_MNEM_TXDATAPLEN_MAX; + flowc->mnemval[0].val = htobe32(maxlen); + + txsd->tx_credits = howmany(flowclen, 16); + txsd->plen = 0; + KASSERT(toep->tx_credits >= txsd->tx_credits && toep->txsd_avail > 0, + ("%s: not enough credits (%d)", __func__, toep->tx_credits)); + toep->tx_credits -= txsd->tx_credits; + if (__predict_false(++toep->txsd_pidx == toep->txsd_total)) + toep->txsd_pidx = 0; + toep->txsd_avail--; + + t4_wrq_tx(sc, wr); +} + +static void +set_ulp_mode_iscsi(struct adapter *sc, struct toepcb *toep, int hcrc, int dcrc) +{ + uint64_t val = 0; + + if (hcrc) + val |= ULP_CRC_HEADER; + if (dcrc) + val |= ULP_CRC_DATA; + val <<= 4; + val |= ULP_MODE_ISCSI; + + CTR4(KTR_CXGBE, "%s: tid %u, ULP_MODE_ISCSI, CRC hdr=%d data=%d", + __func__, toep->tid, hcrc, dcrc); + + t4_set_tcb_field(sc, toep, 1, 0, 0xfff, val); +} + +/* + * XXXNP: Who is responsible for cleaning up the socket if this returns with an + * error? Review all error paths. + * + * XXXNP: What happens to the socket's fd reference if the operation is + * successful, and how does that affect the socket's life cycle? + */ +int +icl_cxgbei_conn_handoff(struct icl_conn *ic, int fd) +{ + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + struct find_ofld_adapter_rr fa; + struct file *fp; + struct socket *so; + struct inpcb *inp; + struct tcpcb *tp; + struct toepcb *toep; + cap_rights_t rights; + int error; + + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + ICL_CONN_LOCK_ASSERT_NOT(ic); + + /* + * Steal the socket from userland. + */ + error = fget(curthread, fd, + cap_rights_init(&rights, CAP_SOCK_CLIENT), &fp); + if (error != 0) + return (error); + if (fp->f_type != DTYPE_SOCKET) { + fdrop(fp, curthread); + return (EINVAL); + } + so = fp->f_data; + if (so->so_type != SOCK_STREAM || + so->so_proto->pr_protocol != IPPROTO_TCP) { + fdrop(fp, curthread); + return (EINVAL); + } + + ICL_CONN_LOCK(ic); + if (ic->ic_socket != NULL) { + ICL_CONN_UNLOCK(ic); + fdrop(fp, curthread); + return (EBUSY); + } + ic->ic_disconnecting = false; + ic->ic_socket = so; + fp->f_ops = &badfileops; + fp->f_data = NULL; + fdrop(fp, curthread); + ICL_CONN_UNLOCK(ic); + + /* Find the adapter offloading this socket. */ + fa.sc = NULL; + fa.so = so; + t4_iterate(find_offload_adapter, &fa); + if (fa.sc == NULL) + return (EINVAL); + icc->sc = fa.sc; + + error = icl_cxgbei_setsockopt(ic, so); + if (error) + return (error); + + inp = sotoinpcb(so); + INP_WLOCK(inp); + tp = intotcpcb(inp); + if (inp->inp_flags & (INP_DROPPED | INP_TIMEWAIT)) + error = EBUSY; + else { + /* + * socket could not have been "unoffloaded" if here. + */ + MPASS(tp->t_flags & TF_TOE); + MPASS(tp->tod != NULL); + MPASS(tp->t_toe != NULL); + toep = tp->t_toe; + MPASS(toep->vi->pi->adapter == icc->sc); + icc->toep = toep; + icc->cwt = cxgbei_select_worker_thread(icc); + icc->ulp_submode = 0; + if (ic->ic_header_crc32c) + icc->ulp_submode |= ULP_CRC_HEADER; + if (ic->ic_data_crc32c) + icc->ulp_submode |= ULP_CRC_DATA; + so->so_options |= SO_NO_DDP; + toep->ulp_mode = ULP_MODE_ISCSI; + toep->ulpcb = icc; + + send_iscsi_flowc_wr(icc->sc, toep, CXGBEI_MAX_PDU); + set_ulp_mode_iscsi(icc->sc, toep, ic->ic_header_crc32c, + ic->ic_data_crc32c); + error = 0; + } + INP_WUNLOCK(inp); + + return (error); +} + +void +icl_cxgbei_conn_close(struct icl_conn *ic) +{ + struct icl_cxgbei_conn *icc = ic_to_icc(ic); + struct icl_pdu *ip; + struct socket *so; + struct sockbuf *sb; + struct inpcb *inp; + struct toepcb *toep = icc->toep; + + MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); + ICL_CONN_LOCK_ASSERT_NOT(ic); + + ICL_CONN_LOCK(ic); + so = ic->ic_socket; + if (ic->ic_disconnecting || so == NULL) { + CTR4(KTR_CXGBE, "%s: icc %p (disconnecting = %d), so %p", + __func__, icc, ic->ic_disconnecting, so); + ICL_CONN_UNLOCK(ic); + return; + } + ic->ic_disconnecting = true; + + /* These are unused in this driver right now. */ + MPASS(STAILQ_EMPTY(&ic->ic_to_send)); + MPASS(ic->ic_receive_pdu == NULL); + +#ifdef DIAGNOSTIC + KASSERT(ic->ic_outstanding_pdus == 0, + ("destroying session with %d outstanding PDUs", + ic->ic_outstanding_pdus)); +#endif + ICL_CONN_UNLOCK(ic); + + CTR3(KTR_CXGBE, "%s: tid %d, icc %p", __func__, toep ? toep->tid : -1, + icc); + inp = sotoinpcb(so); + sb = &so->so_rcv; + INP_WLOCK(inp); + if (toep != NULL) { /* NULL if connection was never offloaded. */ + toep->ulpcb = NULL; + mbufq_drain(&toep->ulp_pduq); + SOCKBUF_LOCK(sb); + if (icc->rx_flags & RXF_ACTIVE) { + volatile u_int *p = &icc->rx_flags; + + SOCKBUF_UNLOCK(sb); + INP_WUNLOCK(inp); + + while (*p & RXF_ACTIVE) + pause("conclo", 1); + + INP_WLOCK(inp); + SOCKBUF_LOCK(sb); + } + + while (!STAILQ_EMPTY(&icc->rcvd_pdus)) { + ip = STAILQ_FIRST(&icc->rcvd_pdus); + STAILQ_REMOVE_HEAD(&icc->rcvd_pdus, ip_next); + icl_cxgbei_conn_pdu_free(ic, ip); + } + SOCKBUF_UNLOCK(sb); + } + INP_WUNLOCK(inp); + + ICL_CONN_LOCK(ic); + ic->ic_socket = NULL; + ICL_CONN_UNLOCK(ic); + + /* + * XXXNP: we should send RST instead of FIN when PDUs held in various + * queues were purged instead of delivered reliably but soabort isn't + * really general purpose and wouldn't do the right thing here. + */ + soclose(so); +} + +int +icl_cxgbei_conn_task_setup(struct icl_conn *ic, struct ccb_scsiio *csio, + uint32_t *task_tagp, void **prvp) +{ + void *prv; + + *task_tagp = icl_conn_build_tasktag(ic, *task_tagp); + + prv = uma_zalloc(icl_transfer_zone, M_NOWAIT | M_ZERO); + if (prv == NULL) + return (ENOMEM); + + *prvp = prv; + + cxgbei_conn_task_reserve_itt(ic, prvp, csio, task_tagp); + + return (0); +} + +void +icl_cxgbei_conn_task_done(struct icl_conn *ic, void *prv) +{ + + cxgbei_cleanup_task(ic, prv); + uma_zfree(icl_transfer_zone, prv); +} + +int +icl_cxgbei_conn_transfer_setup(struct icl_conn *ic, union ctl_io *io, + uint32_t *transfer_tag, void **prvp) +{ + void *prv; + + *transfer_tag = icl_conn_build_tasktag(ic, *transfer_tag); + + prv = uma_zalloc(icl_transfer_zone, M_NOWAIT | M_ZERO); + if (prv == NULL) + return (ENOMEM); + + *prvp = prv; + + cxgbei_conn_transfer_reserve_ttt(ic, prvp, io, transfer_tag); + + return (0); +} + +void +icl_cxgbei_conn_transfer_done(struct icl_conn *ic, void *prv) +{ + cxgbei_cleanup_task(ic, prv); + uma_zfree(icl_transfer_zone, prv); +} + +static int +icl_cxgbei_limits(size_t *limitp) +{ + + *limitp = CXGBEI_MAX_DSL; + + return (0); +} + +static int +icl_cxgbei_load(void) +{ + int error; + + icl_transfer_zone = uma_zcreate("icl_transfer", + 16 * 1024, NULL, NULL, NULL, NULL, + UMA_ALIGN_PTR, 0); + + refcount_init(&icl_cxgbei_ncons, 0); + + error = icl_register("cxgbei", 100, icl_cxgbei_limits, + icl_cxgbei_new_conn); + KASSERT(error == 0, ("failed to register")); + + return (error); +} + +static int +icl_cxgbei_unload(void) +{ + + if (icl_cxgbei_ncons != 0) + return (EBUSY); + + icl_unregister("cxgbei"); + + uma_zdestroy(icl_transfer_zone); + + return (0); +} + +static int +icl_cxgbei_modevent(module_t mod, int what, void *arg) +{ + + switch (what) { + case MOD_LOAD: + return (icl_cxgbei_load()); + case MOD_UNLOAD: + return (icl_cxgbei_unload()); + default: + return (EINVAL); + } +} + +moduledata_t icl_cxgbei_data = { + "icl_cxgbei", + icl_cxgbei_modevent, + 0 +}; + +DECLARE_MODULE(icl_cxgbei, icl_cxgbei_data, SI_SUB_DRIVERS, SI_ORDER_MIDDLE); +MODULE_DEPEND(icl_cxgbei, icl, 1, 1, 1); +MODULE_VERSION(icl_cxgbei, 1); +#endif diff --git a/sys/dev/cxgbe/offload.h b/sys/dev/cxgbe/offload.h index 2a12283..2b5e4dc 100644 --- a/sys/dev/cxgbe/offload.h +++ b/sys/dev/cxgbe/offload.h @@ -156,7 +156,7 @@ int t4_register_uld(struct uld_info *); int t4_unregister_uld(struct uld_info *); int t4_activate_uld(struct adapter *, int); int t4_deactivate_uld(struct adapter *, int); -void t4_iscsi_init(struct ifnet *, unsigned int, const unsigned int *); +void t4_iscsi_init(struct adapter *, u_int, const u_int *); int uld_active(struct adapter *, int); #endif #endif diff --git a/sys/dev/cxgbe/t4_main.c b/sys/dev/cxgbe/t4_main.c index edac709..0d00a77 100644 --- a/sys/dev/cxgbe/t4_main.c +++ b/sys/dev/cxgbe/t4_main.c @@ -8801,11 +8801,8 @@ t4_ioctl(struct cdev *dev, unsigned long cmd, caddr_t data, int fflag, #ifdef TCP_OFFLOAD void -t4_iscsi_init(struct ifnet *ifp, unsigned int tag_mask, - const unsigned int *pgsz_order) +t4_iscsi_init(struct adapter *sc, u_int tag_mask, const u_int *pgsz_order) { - struct vi_info *vi = ifp->if_softc; - struct adapter *sc = vi->pi->adapter; t4_write_reg(sc, A_ULP_RX_ISCSI_TAGMASK, tag_mask); t4_write_reg(sc, A_ULP_RX_ISCSI_PSZ, V_HPZ0(pgsz_order[0]) | diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index c91c994..d58592e 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2012 Chelsio Communications, Inc. + * Copyright (c) 2012, 2015 Chelsio Communications, Inc. * All rights reserved. * Written by: Navdeep Parhar <np@FreeBSD.org> * @@ -71,33 +71,6 @@ VNET_DECLARE(int, tcp_autorcvbuf_inc); VNET_DECLARE(int, tcp_autorcvbuf_max); #define V_tcp_autorcvbuf_max VNET(tcp_autorcvbuf_max) -/* - * For ULP connections HW may add headers, e.g., for digests, that aren't part - * of the messages sent by the host but that are part of the TCP payload and - * therefore consume TCP sequence space. Tx connection parameters that - * operate in TCP sequence space are affected by the HW additions and need to - * compensate for them to accurately track TCP sequence numbers. This array - * contains the compensating extra lengths for ULP packets. It is indexed by - * a packet's ULP submode. - */ -const unsigned int t4_ulp_extra_len[] = {0, 4, 4, 8}; - -/* - * Return the length of any HW additions that will be made to a Tx packet. - * Such additions can happen for some types of ULP packets. - */ -static inline unsigned int -ulp_extra_len(struct mbuf *m, int *ulp_mode) -{ - struct m_tag *mtag; - - if ((mtag = m_tag_find(m, CXGBE_ISCSI_MBUF_TAG, NULL)) == NULL) - return (0); - *ulp_mode = *((int *)(mtag + 1)); - - return (t4_ulp_extra_len[*ulp_mode & 3]); -} - void send_flowc_wr(struct toepcb *toep, struct flowc_tx_params *ftxp) { @@ -384,13 +357,10 @@ t4_rcvd(struct toedev *tod, struct tcpcb *tp) KASSERT(toep->sb_cc >= sbused(sb), ("%s: sb %p has more data (%d) than last time (%d).", __func__, sb, sbused(sb), toep->sb_cc)); - if (toep->ulp_mode == ULP_MODE_ISCSI) { - toep->rx_credits += toep->sb_cc; - toep->sb_cc = 0; - } else { - toep->rx_credits += toep->sb_cc - sbused(sb); - toep->sb_cc = sbused(sb); - } + + toep->rx_credits += toep->sb_cc - sbused(sb); + toep->sb_cc = sbused(sb); + if (toep->rx_credits > 0 && (tp->rcv_wnd <= 32 * 1024 || toep->rx_credits >= 64 * 1024 || (toep->rx_credits >= 16 * 1024 && tp->rcv_wnd <= 128 * 1024) || @@ -490,25 +460,16 @@ max_dsgl_nsegs(int tx_credits) static inline void write_tx_wr(void *dst, struct toepcb *toep, unsigned int immdlen, - unsigned int plen, uint8_t credits, int shove, int ulp_mode, int txalign) + unsigned int plen, uint8_t credits, int shove, int ulp_submode, int txalign) { struct fw_ofld_tx_data_wr *txwr = dst; - unsigned int wr_ulp_mode; txwr->op_to_immdlen = htobe32(V_WR_OP(FW_OFLD_TX_DATA_WR) | V_FW_WR_IMMDLEN(immdlen)); txwr->flowid_len16 = htobe32(V_FW_WR_FLOWID(toep->tid) | V_FW_WR_LEN16(credits)); - - /* for iscsi, the mode & submode setting is per-packet */ - if (toep->ulp_mode == ULP_MODE_ISCSI) - wr_ulp_mode = V_TX_ULP_MODE(ulp_mode >> 4) | - V_TX_ULP_SUBMODE(ulp_mode & 3); - else - wr_ulp_mode = V_TX_ULP_MODE(toep->ulp_mode); - - txwr->lsodisable_to_flags = htobe32(wr_ulp_mode | V_TX_URG(0) | /*XXX*/ - V_TX_SHOVE(shove)); + txwr->lsodisable_to_flags = htobe32(V_TX_ULP_MODE(toep->ulp_mode) | + V_TX_ULP_SUBMODE(ulp_submode) | V_TX_URG(0) | V_TX_SHOVE(shove)); txwr->plen = htobe32(plen); if (txalign > 0) { @@ -616,6 +577,9 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) toep->ulp_mode == ULP_MODE_RDMA, ("%s: ulp_mode %u for toep %p", __func__, toep->ulp_mode, toep)); + if (__predict_false(toep->flags & TPF_ABORT_SHUTDOWN)) + return; + /* * This function doesn't resume by itself. Someone else must clear the * flag and call this function. @@ -802,56 +766,70 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) close_conn(sc, toep); } -/* Send ULP data over TOE using TX_DATA_WR. We send whole mbuf at once */ +static inline void +rqdrop_locked(struct mbufq *q, int plen) +{ + struct mbuf *m; + + while (plen > 0) { + m = mbufq_dequeue(q); + + /* Too many credits. */ + MPASS(m != NULL); + M_ASSERTPKTHDR(m); + + /* Partial credits. */ + MPASS(plen >= m->m_pkthdr.len); + + plen -= m->m_pkthdr.len; + m_freem(m); + } +} + void -t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) +t4_push_pdus(struct adapter *sc, struct toepcb *toep, int drop) { - struct mbuf *sndptr, *m = NULL; + struct mbuf *sndptr, *m; struct fw_ofld_tx_data_wr *txwr; struct wrqe *wr; - unsigned int plen, nsegs, credits, max_imm, max_nsegs, max_nsegs_1mbuf; + u_int plen, nsegs, credits, max_imm, max_nsegs, max_nsegs_1mbuf; + u_int adjusted_plen, ulp_submode; struct inpcb *inp = toep->inp; - struct tcpcb *tp; - struct socket *so; - struct sockbuf *sb; - int tx_credits, ulp_len = 0, ulp_mode = 0, qlen = 0; - int shove, compl; - struct ofld_tx_sdesc *txsd; + struct tcpcb *tp = intotcpcb(inp); + int tx_credits, shove; + struct ofld_tx_sdesc *txsd = &toep->txsd[toep->txsd_pidx]; + struct mbufq *pduq = &toep->ulp_pduq; + static const u_int ulp_extra_len[] = {0, 4, 4, 8}; INP_WLOCK_ASSERT(inp); - if (toep->flags & TPF_ABORT_SHUTDOWN) - return; - - tp = intotcpcb(inp); - so = inp->inp_socket; - sb = &so->so_snd; - txsd = &toep->txsd[toep->txsd_pidx]; - KASSERT(toep->flags & TPF_FLOWC_WR_SENT, ("%s: flowc_wr not sent for tid %u.", __func__, toep->tid)); + KASSERT(toep->ulp_mode == ULP_MODE_ISCSI, + ("%s: ulp_mode %u for toep %p", __func__, toep->ulp_mode, toep)); + + if (__predict_false(toep->flags & TPF_ABORT_SHUTDOWN)) + return; /* * This function doesn't resume by itself. Someone else must clear the * flag and call this function. */ - if (__predict_false(toep->flags & TPF_TX_SUSPENDED)) + if (__predict_false(toep->flags & TPF_TX_SUSPENDED)) { + KASSERT(drop == 0, + ("%s: drop (%d) != 0 but tx is suspended", __func__, drop)); return; + } - sndptr = t4_queue_iscsi_callback(so, toep, 1, &qlen); - if (!qlen) - return; + if (drop) + rqdrop_locked(&toep->ulp_pdu_reclaimq, drop); + + while ((sndptr = mbufq_first(pduq)) != NULL) { + M_ASSERTPKTHDR(sndptr); - do { tx_credits = min(toep->tx_credits, MAX_OFLD_TX_CREDITS); max_imm = max_imm_payload(tx_credits); max_nsegs = max_dsgl_nsegs(tx_credits); - if (drop) { - t4_cpl_iscsi_callback(toep->td, toep, &drop, - CPL_FW4_ACK); - drop = 0; - } - plen = 0; nsegs = 0; max_nsegs_1mbuf = 0; /* max # of SGL segments in any one mbuf */ @@ -861,7 +839,10 @@ t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) nsegs += n; plen += m->m_len; - /* This mbuf sent us _over_ the nsegs limit, return */ + /* + * This mbuf would send us _over_ the nsegs limit. + * Suspend tx because the PDU can't be sent out. + */ if (plen > max_imm && nsegs > max_nsegs) { toep->flags |= TPF_TX_SUSPENDED; return; @@ -869,30 +850,35 @@ t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) if (max_nsegs_1mbuf < n) max_nsegs_1mbuf = n; - - /* This mbuf put us right at the max_nsegs limit */ - if (plen > max_imm && nsegs == max_nsegs) { - toep->flags |= TPF_TX_SUSPENDED; - return; - } - } - - shove = m == NULL && !(tp->t_flags & TF_MORETOCOME); - /* nothing to send */ - if (plen == 0) { - KASSERT(m == NULL, - ("%s: nothing to send, but m != NULL", __func__)); - break; } if (__predict_false(toep->flags & TPF_FIN_SENT)) panic("%s: excess tx.", __func__); - ulp_len = plen + ulp_extra_len(sndptr, &ulp_mode); + /* + * We have a PDU to send. All of it goes out in one WR so 'm' + * is NULL. A PDU's length is always a multiple of 4. + */ + MPASS(m == NULL); + MPASS((plen & 3) == 0); + MPASS(sndptr->m_pkthdr.len == plen); + + shove = !(tp->t_flags & TF_MORETOCOME); + ulp_submode = mbuf_ulp_submode(sndptr); + MPASS(ulp_submode < nitems(ulp_extra_len)); + + /* + * plen doesn't include header and data digests, which are + * generated and inserted in the right places by the TOE, but + * they do occupy TCP sequence space and need to be accounted + * for. + */ + adjusted_plen = plen + ulp_extra_len[ulp_submode]; if (plen <= max_imm) { /* Immediate data tx */ - wr = alloc_wrqe(roundup(sizeof(*txwr) + plen, 16), + + wr = alloc_wrqe(roundup2(sizeof(*txwr) + plen, 16), toep->ofld_txq); if (wr == NULL) { /* XXX: how will we recover from this? */ @@ -901,16 +887,17 @@ t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) } txwr = wrtod(wr); credits = howmany(wr->wr_len, 16); - write_tx_wr(txwr, toep, plen, ulp_len, credits, shove, - ulp_mode, 0); + write_tx_wr(txwr, toep, plen, adjusted_plen, credits, + shove, ulp_submode, sc->tt.tx_align); m_copydata(sndptr, 0, plen, (void *)(txwr + 1)); + nsegs = 0; } else { int wr_len; /* DSGL tx */ wr_len = sizeof(*txwr) + sizeof(struct ulptx_sgl) + ((3 * (nsegs - 1)) / 2 + ((nsegs - 1) & 1)) * 8; - wr = alloc_wrqe(roundup(wr_len, 16), toep->ofld_txq); + wr = alloc_wrqe(roundup2(wr_len, 16), toep->ofld_txq); if (wr == NULL) { /* XXX: how will we recover from this? */ toep->flags |= TPF_TX_SUSPENDED; @@ -918,8 +905,8 @@ t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) } txwr = wrtod(wr); credits = howmany(wr_len, 16); - write_tx_wr(txwr, toep, 0, ulp_len, credits, shove, - ulp_mode, 0); + write_tx_wr(txwr, toep, 0, adjusted_plen, credits, + shove, ulp_submode, sc->tt.tx_align); write_tx_sgl(txwr + 1, sndptr, m, nsegs, max_nsegs_1mbuf); if (wr_len & 0xf) { @@ -932,28 +919,26 @@ t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) KASSERT(toep->tx_credits >= credits, ("%s: not enough credits", __func__)); + m = mbufq_dequeue(pduq); + MPASS(m == sndptr); + mbufq_enqueue(&toep->ulp_pdu_reclaimq, m); + toep->tx_credits -= credits; toep->tx_nocompl += credits; toep->plen_nocompl += plen; if (toep->tx_credits <= toep->tx_total * 3 / 8 && - toep->tx_nocompl >= toep->tx_total / 4) - compl = 1; - - if (compl) { + toep->tx_nocompl >= toep->tx_total / 4) { txwr->op_to_immdlen |= htobe32(F_FW_WR_COMPL); toep->tx_nocompl = 0; toep->plen_nocompl = 0; } - tp->snd_nxt += ulp_len; - tp->snd_max += ulp_len; - /* goto next mbuf */ - sndptr = m = t4_queue_iscsi_callback(so, toep, 2, &qlen); + tp->snd_nxt += adjusted_plen; + tp->snd_max += adjusted_plen; toep->flags |= TPF_TX_DATA_SENT; - if (toep->tx_credits < MIN_OFLD_TX_CREDITS) { + if (toep->tx_credits < MIN_OFLD_TX_CREDITS) toep->flags |= TPF_TX_SUSPENDED; - } KASSERT(toep->txsd_avail > 0, ("%s: no txsd", __func__)); txsd->plen = plen; @@ -966,10 +951,10 @@ t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int drop) toep->txsd_avail--; t4_l2t_send(sc, wr, toep->l2te); - } while (m != NULL); + } - /* Send a FIN if requested, but only if there's no more data to send */ - if (m == NULL && toep->flags & TPF_SEND_FIN) + /* Send a FIN if requested, but only if there are no more PDUs to send */ + if (mbufq_first(pduq) == NULL && toep->flags & TPF_SEND_FIN) close_conn(sc, toep); } @@ -987,7 +972,10 @@ t4_tod_output(struct toedev *tod, struct tcpcb *tp) ("%s: inp %p dropped.", __func__, inp)); KASSERT(toep != NULL, ("%s: toep is NULL", __func__)); - t4_push_frames(sc, toep, 0); + if (toep->ulp_mode == ULP_MODE_ISCSI) + t4_push_pdus(sc, toep, 0); + else + t4_push_frames(sc, toep, 0); return (0); } @@ -1009,7 +997,7 @@ t4_send_fin(struct toedev *tod, struct tcpcb *tp) toep->flags |= TPF_SEND_FIN; if (tp->t_state >= TCPS_ESTABLISHED) { if (toep->ulp_mode == ULP_MODE_ISCSI) - t4_ulp_push_frames(sc, toep, 0); + t4_push_pdus(sc, toep, 0); else t4_push_frames(sc, toep, 0); } @@ -1250,91 +1238,6 @@ abort_status_to_errno(struct tcpcb *tp, unsigned int abort_reason) } } -int -cpl_not_handled(struct sge_iq *, const struct rss_header *, struct mbuf *); -/* - * tom_cpl_iscsi_callback - - * iscsi and tom would share the following cpl messages, so when any of these - * message is received, after tom is done with processing it, the messages - * needs to be forwarded to iscsi for further processing: - * - CPL_SET_TCB_RPL - * - CPL_RX_DATA_DDP - */ -void (*tom_cpl_iscsi_callback)(struct tom_data *, struct socket *, void *, - unsigned int); - -struct mbuf *(*tom_queue_iscsi_callback)(struct socket *, unsigned int, int *); -/* - * Check if the handler function is set for a given CPL - * return 0 if the function is NULL or cpl_not_handled, 1 otherwise. - */ -int -t4tom_cpl_handler_registered(struct adapter *sc, unsigned int opcode) -{ - - MPASS(opcode < nitems(sc->cpl_handler)); - - return (sc->cpl_handler[opcode] && - sc->cpl_handler[opcode] != cpl_not_handled); -} - -/* - * set the tom_cpl_iscsi_callback function, this function should be used - * whenever both toe and iscsi need to process the same cpl msg. - */ -void -t4tom_register_cpl_iscsi_callback(void (*fp)(struct tom_data *, struct socket *, - void *, unsigned int)) -{ - - tom_cpl_iscsi_callback = fp; -} - -void -t4tom_register_queue_iscsi_callback(struct mbuf *(*fp)(struct socket *, - unsigned int, int *qlen)) -{ - - tom_queue_iscsi_callback = fp; -} - -int -t4_cpl_iscsi_callback(struct tom_data *td, struct toepcb *toep, void *m, - unsigned int opcode) -{ - struct socket *so; - - if (opcode == CPL_FW4_ACK) - so = toep->inp->inp_socket; - else { - INP_WLOCK(toep->inp); - so = toep->inp->inp_socket; - INP_WUNLOCK(toep->inp); - } - - if (tom_cpl_iscsi_callback && so) { - if (toep->ulp_mode == ULP_MODE_ISCSI) { - tom_cpl_iscsi_callback(td, so, m, opcode); - return (0); - } - } - - return (1); -} - -struct mbuf * -t4_queue_iscsi_callback(struct socket *so, struct toepcb *toep, - unsigned int cmd, int *qlen) -{ - - if (tom_queue_iscsi_callback && so) { - if (toep->ulp_mode == ULP_MODE_ISCSI) - return (tom_queue_iscsi_callback(so, cmd, qlen)); - } - - return (NULL); -} - /* * TCP RST from the peer, timeout, or some other such critical error. */ @@ -1733,21 +1636,34 @@ do_fw4_ack(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) toep->tx_credits >= toep->tx_total / 4) { toep->flags &= ~TPF_TX_SUSPENDED; if (toep->ulp_mode == ULP_MODE_ISCSI) - t4_ulp_push_frames(sc, toep, plen); + t4_push_pdus(sc, toep, plen); else t4_push_frames(sc, toep, plen); } else if (plen > 0) { struct sockbuf *sb = &so->so_snd; + int sbu; - if (toep->ulp_mode == ULP_MODE_ISCSI) - t4_cpl_iscsi_callback(toep->td, toep, &plen, - CPL_FW4_ACK); - else { - SOCKBUF_LOCK(sb); + SOCKBUF_LOCK(sb); + sbu = sbused(sb); + if (toep->ulp_mode == ULP_MODE_ISCSI) { + + if (__predict_false(sbu > 0)) { + /* + * The data trasmitted before the tid's ULP mode + * changed to ISCSI is still in so_snd. + * Incoming credits should account for so_snd + * first. + */ + sbdrop_locked(sb, min(sbu, plen)); + plen -= min(sbu, plen); + } + sowwakeup_locked(so); /* unlocks so_snd */ + rqdrop_locked(&toep->ulp_pdu_reclaimq, plen); + } else { sbdrop_locked(sb, plen); - sowwakeup_locked(so); - SOCKBUF_UNLOCK_ASSERT(sb); + sowwakeup_locked(so); /* unlocks so_snd */ } + SOCKBUF_UNLOCK_ASSERT(sb); } INP_WUNLOCK(inp); @@ -1771,14 +1687,21 @@ do_set_tcb_rpl(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) if (is_ftid(sc, tid)) return (t4_filter_rpl(iq, rss, m)); /* TCB is a filter */ - else { - struct toepcb *toep = lookup_tid(sc, tid); - t4_cpl_iscsi_callback(toep->td, toep, m, CPL_SET_TCB_RPL); - return (0); - } + /* + * TOM and/or other ULPs don't request replies for CPL_SET_TCB or + * CPL_SET_TCB_FIELD requests. This can easily change and when it does + * the dispatch code will go here. + */ +#ifdef INVARIANTS + panic("%s: Unexpected CPL_SET_TCB_RPL for tid %u on iq %p", __func__, + tid, iq); +#else + log(LOG_ERR, "%s: Unexpected CPL_SET_TCB_RPL for tid %u on iq %p\n", + __func__, tid, iq); +#endif - CXGBE_UNIMPLEMENTED(__func__); + return (0); } void diff --git a/sys/dev/cxgbe/tom/t4_ddp.c b/sys/dev/cxgbe/tom/t4_ddp.c index 244287f..2df02577 100644 --- a/sys/dev/cxgbe/tom/t4_ddp.c +++ b/sys/dev/cxgbe/tom/t4_ddp.c @@ -513,7 +513,6 @@ do_rx_data_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) unsigned int tid = GET_TID(cpl); uint32_t vld; struct toepcb *toep = lookup_tid(sc, tid); - struct tom_data *td = toep->td; KASSERT(m == NULL, ("%s: wasn't expecting payload", __func__)); KASSERT(toep->tid == tid, ("%s: toep tid/atid mismatch", __func__)); @@ -525,16 +524,11 @@ do_rx_data_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) panic("%s: DDP error 0x%x (tid %d, toep %p)", __func__, vld, tid, toep); } + if (toep->ulp_mode == ULP_MODE_ISCSI) { - m = m_get(M_NOWAIT, MT_DATA); - if (m == NULL) - CXGBE_UNIMPLEMENTED("mbuf alloc failure"); - memcpy(mtod(m, unsigned char *), cpl, - sizeof(struct cpl_rx_data_ddp)); - if (!t4_cpl_iscsi_callback(td, toep, m, CPL_RX_DATA_DDP)) - return (0); - m_freem(m); - } + sc->cpl_handler[CPL_RX_ISCSI_DDP](iq, rss, m); + return (0); + } handle_ddp_data(toep, cpl->u.ddp_report, cpl->seq, be16toh(cpl->len)); diff --git a/sys/dev/cxgbe/tom/t4_tom.c b/sys/dev/cxgbe/tom/t4_tom.c index bcf322e..c136834 100644 --- a/sys/dev/cxgbe/tom/t4_tom.c +++ b/sys/dev/cxgbe/tom/t4_tom.c @@ -37,6 +37,7 @@ __FBSDID("$FreeBSD$"); #include <sys/kernel.h> #include <sys/ktr.h> #include <sys/lock.h> +#include <sys/limits.h> #include <sys/module.h> #include <sys/protosw.h> #include <sys/domain.h> @@ -158,6 +159,8 @@ alloc_toepcb(struct vi_info *vi, int txqid, int rxqid, int flags) toep->ofld_txq = &sc->sge.ofld_txq[txqid]; toep->ofld_rxq = &sc->sge.ofld_rxq[rxqid]; toep->ctrlq = &sc->sge.ctrlq[pi->port_id]; + mbufq_init(&toep->ulp_pduq, INT_MAX); + mbufq_init(&toep->ulp_pdu_reclaimq, INT_MAX); toep->txsd_total = txsd_total; toep->txsd_avail = txsd_total; toep->txsd_pidx = 0; @@ -273,6 +276,14 @@ release_offload_resources(struct toepcb *toep) CTR5(KTR_CXGBE, "%s: toep %p (tid %d, l2te %p, ce %p)", __func__, toep, tid, toep->l2te, toep->ce); + /* + * These queues should have been emptied at approximately the same time + * that a normal connection's socket's so_snd would have been purged or + * drained. Do _not_ clean up here. + */ + MPASS(mbufq_len(&toep->ulp_pduq) == 0); + MPASS(mbufq_len(&toep->ulp_pdu_reclaimq) == 0); + if (toep->ulp_mode == ULP_MODE_TCPDDP) release_ddp_resources(toep); @@ -380,6 +391,7 @@ final_cpl_received(struct toepcb *toep) toep->inp = NULL; toep->flags &= ~TPF_CPL_PENDING; + mbufq_drain(&toep->ulp_pdu_reclaimq); if (!(toep->flags & TPF_ATTACHED)) release_offload_resources(toep); diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index f24e147..f61888d 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2012 Chelsio Communications, Inc. + * Copyright (c) 2012, 2015 Chelsio Communications, Inc. * All rights reserved. * Written by: Navdeep Parhar <np@FreeBSD.org> * @@ -115,6 +115,10 @@ struct toepcb { int rx_credits; /* rx credits (in bytes) to be returned to hw */ u_int ulp_mode; /* ULP mode */ + void *ulpcb; + void *ulpcb2; + struct mbufq ulp_pduq; /* PDUs waiting to be sent out. */ + struct mbufq ulp_pdu_reclaimq; u_int ddp_flags; struct ddp_buffer *db[2]; @@ -220,6 +224,22 @@ td_adapter(struct tom_data *td) return (td->tod.tod_softc); } +static inline void +set_mbuf_ulp_submode(struct mbuf *m, uint8_t ulp_submode) +{ + + M_ASSERTPKTHDR(m); + m->m_pkthdr.PH_per.eight[0] = ulp_submode; +} + +static inline uint8_t +mbuf_ulp_submode(struct mbuf *m) +{ + + M_ASSERTPKTHDR(m); + return (m->m_pkthdr.PH_per.eight[0]); +} + /* t4_tom.c */ struct toepcb *alloc_toepcb(struct vi_info *, int, int, int); void free_toepcb(struct toepcb *); @@ -275,6 +295,7 @@ int t4_send_rst(struct toedev *, struct tcpcb *); void t4_set_tcb_field(struct adapter *, struct toepcb *, int, uint16_t, uint64_t, uint64_t); void t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop); +void t4_push_pdus(struct adapter *sc, struct toepcb *toep, int drop); /* t4_ddp.c */ void t4_init_ddp(struct adapter *, struct tom_data *); @@ -287,19 +308,4 @@ void handle_ddp_close(struct toepcb *, struct tcpcb *, struct sockbuf *, uint32_t); void insert_ddp_data(struct toepcb *, uint32_t); -/* ULP related */ -#define CXGBE_ISCSI_MBUF_TAG 50 -int t4tom_cpl_handler_registered(struct adapter *, unsigned int); -void t4tom_register_cpl_iscsi_callback(void (*fp)(struct tom_data *, - struct socket *, void *, unsigned int)); -void t4tom_register_queue_iscsi_callback(struct mbuf *(*fp)(struct socket *, - unsigned int, int *)); -void t4_ulp_push_frames(struct adapter *sc, struct toepcb *toep, int); -int t4_cpl_iscsi_callback(struct tom_data *, struct toepcb *, void *, uint32_t); -struct mbuf *t4_queue_iscsi_callback(struct socket *, struct toepcb *, uint32_t, - int *); -extern void (*tom_cpl_iscsi_callback)(struct tom_data *, struct socket *, - void *, unsigned int); -extern struct mbuf *(*tom_queue_iscsi_callback)(struct socket*, unsigned int, - int *); #endif diff --git a/sys/dev/e1000/if_igb.c b/sys/dev/e1000/if_igb.c index 3fc1c1f..4988178 100644 --- a/sys/dev/e1000/if_igb.c +++ b/sys/dev/e1000/if_igb.c @@ -336,6 +336,12 @@ SYSCTL_INT(_hw_igb, OID_AUTO, rx_process_limit, CTLFLAG_RDTUN, &igb_rx_process_limit, 0, "Maximum number of received packets to process at a time, -1 means unlimited"); +/* How many packets txeof tries to clean at a time */ +static int igb_tx_process_limit = -1; +SYSCTL_INT(_hw_igb, OID_AUTO, tx_process_limit, CTLFLAG_RDTUN, + &igb_tx_process_limit, 0, + "Maximum number of sent packets to process at a time, -1 means unlimited"); + #ifdef DEV_NETMAP /* see ixgbe.c for details */ #include <dev/netmap/if_igb_netmap.h> #endif /* DEV_NETMAP */ @@ -453,11 +459,15 @@ igb_attach(device_t dev) e1000_get_bus_info(&adapter->hw); - /* Sysctl for limiting the amount of work done in the taskqueue */ + /* Sysctls for limiting the amount of work done in the taskqueues */ igb_set_sysctl_value(adapter, "rx_processing_limit", "max number of rx packets to process", &adapter->rx_process_limit, igb_rx_process_limit); + igb_set_sysctl_value(adapter, "tx_processing_limit", + "max number of tx packets to process", + &adapter->tx_process_limit, igb_tx_process_limit); + /* * Validate number of transmit and receive descriptors. It * must not exceed hardware maximum, and must be multiple @@ -3971,7 +3981,7 @@ igb_txeof(struct tx_ring *txr) struct ifnet *ifp = adapter->ifp; #endif /* DEV_NETMAP */ u32 work, processed = 0; - u16 limit = txr->process_limit; + int limit = adapter->tx_process_limit; struct igb_tx_buf *buf; union e1000_adv_tx_desc *txd; diff --git a/sys/dev/e1000/if_igb.h b/sys/dev/e1000/if_igb.h index 7a232c2..c147788 100644 --- a/sys/dev/e1000/if_igb.h +++ b/sys/dev/e1000/if_igb.h @@ -355,7 +355,6 @@ struct tx_ring { volatile u16 tx_avail; u16 next_avail_desc; u16 next_to_clean; - u16 process_limit; u16 num_desc; enum { IGB_QUEUE_IDLE = 1, @@ -534,6 +533,7 @@ struct adapter { int has_manage; int wol; int rx_process_limit; + int tx_process_limit; u16 vf_ifp; /* a VF interface */ bool in_detach; /* Used only in igb_ioctl */ diff --git a/sys/dev/etherswitch/arswitch/arswitch.c b/sys/dev/etherswitch/arswitch/arswitch.c index 3b1d575..4388819 100644 --- a/sys/dev/etherswitch/arswitch/arswitch.c +++ b/sys/dev/etherswitch/arswitch/arswitch.c @@ -52,7 +52,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_7240.c b/sys/dev/etherswitch/arswitch/arswitch_7240.c index f3750bf..3d2543a7 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_7240.c +++ b/sys/dev/etherswitch/arswitch/arswitch_7240.c @@ -50,7 +50,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_8216.c b/sys/dev/etherswitch/arswitch/arswitch_8216.c index 72b2327..bd5c2cb 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_8216.c +++ b/sys/dev/etherswitch/arswitch/arswitch_8216.c @@ -50,7 +50,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_8226.c b/sys/dev/etherswitch/arswitch/arswitch_8226.c index 43d2281..5a86627 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_8226.c +++ b/sys/dev/etherswitch/arswitch/arswitch_8226.c @@ -50,7 +50,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_8316.c b/sys/dev/etherswitch/arswitch/arswitch_8316.c index 91dd72d..d646e18 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_8316.c +++ b/sys/dev/etherswitch/arswitch/arswitch_8316.c @@ -50,7 +50,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_8327.c b/sys/dev/etherswitch/arswitch/arswitch_8327.c index 96c2282..6e221a8 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_8327.c +++ b/sys/dev/etherswitch/arswitch/arswitch_8327.c @@ -50,7 +50,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_9340.c b/sys/dev/etherswitch/arswitch/arswitch_9340.c index 7754335..89f6caa 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_9340.c +++ b/sys/dev/etherswitch/arswitch/arswitch_9340.c @@ -50,7 +50,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_phy.c b/sys/dev/etherswitch/arswitch/arswitch_phy.c index 43abaa1..ceeb307 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_phy.c +++ b/sys/dev/etherswitch/arswitch/arswitch_phy.c @@ -48,7 +48,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/arswitch/arswitch_reg.c b/sys/dev/etherswitch/arswitch/arswitch_reg.c index d6797a8..420e516 100644 --- a/sys/dev/etherswitch/arswitch/arswitch_reg.c +++ b/sys/dev/etherswitch/arswitch/arswitch_reg.c @@ -49,7 +49,7 @@ #include <dev/iicbus/iicbus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/etherswitch/e6000sw/e6000sw.c b/sys/dev/etherswitch/e6000sw/e6000sw.c index e598607..ca4ec3f 100644 --- a/sys/dev/etherswitch/e6000sw/e6000sw.c +++ b/sys/dev/etherswitch/e6000sw/e6000sw.c @@ -54,7 +54,7 @@ __FBSDID("$FreeBSD$"); #include <arm/mv/mvvar.h> #include <dev/etherswitch/etherswitch.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> #include <dev/mge/if_mgevar.h> diff --git a/sys/dev/etherswitch/ip17x/ip17x.c b/sys/dev/etherswitch/ip17x/ip17x.c index 2f51217..43cdddb 100644 --- a/sys/dev/etherswitch/ip17x/ip17x.c +++ b/sys/dev/etherswitch/ip17x/ip17x.c @@ -51,7 +51,7 @@ #include <machine/bus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> #include <dev/etherswitch/ip17x/ip17x_phy.h> diff --git a/sys/dev/etherswitch/mdio_if.m b/sys/dev/etherswitch/mdio_if.m deleted file mode 100644 index 9aedd92..0000000 --- a/sys/dev/etherswitch/mdio_if.m +++ /dev/null @@ -1,24 +0,0 @@ -# $FreeBSD$ - -#include <sys/bus.h> - -INTERFACE mdio; - -# -# Read register from device on MDIO bus -# -METHOD int readreg { - device_t dev; - int phy; - int reg; -}; - -# -# Write register to device on MDIO bus -# -METHOD int writereg { - device_t dev; - int phy; - int reg; - int val; -}; diff --git a/sys/dev/etherswitch/ukswitch/ukswitch.c b/sys/dev/etherswitch/ukswitch/ukswitch.c index fa65455..876aa9d 100644 --- a/sys/dev/etherswitch/ukswitch/ukswitch.c +++ b/sys/dev/etherswitch/ukswitch/ukswitch.c @@ -50,7 +50,7 @@ #include <machine/bus.h> #include <dev/mii/mii.h> #include <dev/mii/miivar.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/etherswitch.h> diff --git a/sys/dev/hyperv/include/hyperv.h b/sys/dev/hyperv/include/hyperv.h index 6727503..b5600ba 100644 --- a/sys/dev/hyperv/include/hyperv.h +++ b/sys/dev/hyperv/include/hyperv.h @@ -759,7 +759,6 @@ typedef struct hv_vmbus_channel { hv_vmbus_ring_buffer_info inbound; struct mtx inbound_lock; - hv_vmbus_handle control_work_queue; hv_vmbus_pfn_channel_callback on_channel_callback; void* channel_callback_context; diff --git a/sys/dev/hyperv/vmbus/hv_channel_mgmt.c b/sys/dev/hyperv/vmbus/hv_channel_mgmt.c index 783f6bc..1a4763c 100644 --- a/sys/dev/hyperv/vmbus/hv_channel_mgmt.c +++ b/sys/dev/hyperv/vmbus/hv_channel_mgmt.c @@ -31,13 +31,6 @@ #include "hv_vmbus_priv.h" -typedef void (*hv_pfn_channel_msg_handler)(hv_vmbus_channel_msg_header* msg); - -typedef struct hv_vmbus_channel_msg_table_entry { - hv_vmbus_channel_msg_type messageType; - hv_pfn_channel_msg_handler messageHandler; -} hv_vmbus_channel_msg_table_entry; - /* * Internal functions */ @@ -49,38 +42,46 @@ static void vmbus_channel_on_gpadl_created(hv_vmbus_channel_msg_header* hdr); static void vmbus_channel_on_gpadl_torndown(hv_vmbus_channel_msg_header* hdr); static void vmbus_channel_on_offers_delivered(hv_vmbus_channel_msg_header* hdr); static void vmbus_channel_on_version_response(hv_vmbus_channel_msg_header* hdr); -static void vmbus_channel_process_offer(void *context); -struct hv_vmbus_channel* - vmbus_select_outgoing_channel(struct hv_vmbus_channel *promary); /** * Channel message dispatch table */ hv_vmbus_channel_msg_table_entry g_channel_message_table[HV_CHANNEL_MESSAGE_COUNT] = { - { HV_CHANNEL_MESSAGE_INVALID, NULL }, - { HV_CHANNEL_MESSAGE_OFFER_CHANNEL, vmbus_channel_on_offer }, + { HV_CHANNEL_MESSAGE_INVALID, + 0, NULL }, + { HV_CHANNEL_MESSAGE_OFFER_CHANNEL, + 0, vmbus_channel_on_offer }, { HV_CHANNEL_MESSAGE_RESCIND_CHANNEL_OFFER, - vmbus_channel_on_offer_rescind }, - { HV_CHANNEL_MESSAGE_REQUEST_OFFERS, NULL }, + 0, vmbus_channel_on_offer_rescind }, + { HV_CHANNEL_MESSAGE_REQUEST_OFFERS, + 0, NULL }, { HV_CHANNEL_MESSAGE_ALL_OFFERS_DELIVERED, - vmbus_channel_on_offers_delivered }, - { HV_CHANNEL_MESSAGE_OPEN_CHANNEL, NULL }, + 1, vmbus_channel_on_offers_delivered }, + { HV_CHANNEL_MESSAGE_OPEN_CHANNEL, + 0, NULL }, { HV_CHANNEL_MESSAGE_OPEN_CHANNEL_RESULT, - vmbus_channel_on_open_result }, - { HV_CHANNEL_MESSAGE_CLOSE_CHANNEL, NULL }, - { HV_CHANNEL_MESSAGEL_GPADL_HEADER, NULL }, - { HV_CHANNEL_MESSAGE_GPADL_BODY, NULL }, + 1, vmbus_channel_on_open_result }, + { HV_CHANNEL_MESSAGE_CLOSE_CHANNEL, + 0, NULL }, + { HV_CHANNEL_MESSAGEL_GPADL_HEADER, + 0, NULL }, + { HV_CHANNEL_MESSAGE_GPADL_BODY, + 0, NULL }, { HV_CHANNEL_MESSAGE_GPADL_CREATED, - vmbus_channel_on_gpadl_created }, - { HV_CHANNEL_MESSAGE_GPADL_TEARDOWN, NULL }, + 1, vmbus_channel_on_gpadl_created }, + { HV_CHANNEL_MESSAGE_GPADL_TEARDOWN, + 0, NULL }, { HV_CHANNEL_MESSAGE_GPADL_TORNDOWN, - vmbus_channel_on_gpadl_torndown }, - { HV_CHANNEL_MESSAGE_REL_ID_RELEASED, NULL }, - { HV_CHANNEL_MESSAGE_INITIATED_CONTACT, NULL }, + 1, vmbus_channel_on_gpadl_torndown }, + { HV_CHANNEL_MESSAGE_REL_ID_RELEASED, + 0, NULL }, + { HV_CHANNEL_MESSAGE_INITIATED_CONTACT, + 0, NULL }, { HV_CHANNEL_MESSAGE_VERSION_RESPONSE, - vmbus_channel_on_version_response }, - { HV_CHANNEL_MESSAGE_UNLOAD, NULL } + 1, vmbus_channel_on_version_response }, + { HV_CHANNEL_MESSAGE_UNLOAD, + 0, NULL } }; @@ -208,15 +209,6 @@ hv_queue_work_item( return (taskqueue_enqueue(wq->queue, &w->work)); } -/** - * @brief Rescind the offer by initiating a device removal - */ -static void -vmbus_channel_process_rescind_offer(void *context) -{ - hv_vmbus_channel* channel = (hv_vmbus_channel*) context; - hv_vmbus_child_device_unregister(channel->device); -} /** * @brief Allocate and initialize a vmbus channel object @@ -239,14 +231,6 @@ hv_vmbus_allocate_channel(void) TAILQ_INIT(&channel->sc_list_anchor); - channel->control_work_queue = hv_work_queue_create("control"); - - if (channel->control_work_queue == NULL) { - mtx_destroy(&channel->inbound_lock); - free(channel, M_DEVBUF); - return (NULL); - } - return (channel); } @@ -257,7 +241,6 @@ static inline void ReleaseVmbusChannel(void *context) { hv_vmbus_channel* channel = (hv_vmbus_channel*) context; - hv_work_queue_close(channel->control_work_queue); free(channel, M_DEVBUF); } @@ -283,14 +266,12 @@ hv_vmbus_free_vmbus_channel(hv_vmbus_channel* channel) * associated with this offer */ static void -vmbus_channel_process_offer(void *context) +vmbus_channel_process_offer(hv_vmbus_channel *new_channel) { - hv_vmbus_channel* new_channel; boolean_t f_new; hv_vmbus_channel* channel; int ret; - new_channel = (hv_vmbus_channel*) context; f_new = TRUE; channel = NULL; @@ -523,11 +504,7 @@ vmbus_channel_on_offer(hv_vmbus_channel_msg_header* hdr) new_channel->monitor_group = (uint8_t) offer->monitor_id / 32; new_channel->monitor_bit = (uint8_t) offer->monitor_id % 32; - /* TODO: Make sure the offer comes from our parent partition */ - hv_queue_work_item( - new_channel->control_work_queue, - vmbus_channel_process_offer, - new_channel); + vmbus_channel_process_offer(new_channel); } /** @@ -548,8 +525,7 @@ vmbus_channel_on_offer_rescind(hv_vmbus_channel_msg_header* hdr) if (channel == NULL) return; - hv_queue_work_item(channel->control_work_queue, - vmbus_channel_process_rescind_offer, channel); + hv_vmbus_child_device_unregister(channel->device); } /** diff --git a/sys/dev/hyperv/vmbus/hv_vmbus_drv_freebsd.c b/sys/dev/hyperv/vmbus/hv_vmbus_drv_freebsd.c index f9432c8..201f1c9 100644 --- a/sys/dev/hyperv/vmbus/hv_vmbus_drv_freebsd.c +++ b/sys/dev/hyperv/vmbus/hv_vmbus_drv_freebsd.c @@ -76,8 +76,12 @@ vmbus_msg_swintr(void *arg) { int cpu; void* page_addr; + hv_vmbus_channel_msg_header *hdr; + hv_vmbus_channel_msg_table_entry *entry; + hv_vmbus_channel_msg_type msg_type; hv_vmbus_message* msg; hv_vmbus_message* copied; + static bool warned = false; cpu = (int)(long)arg; KASSERT(cpu <= mp_maxid, ("VMBUS: vmbus_msg_swintr: " @@ -87,9 +91,24 @@ vmbus_msg_swintr(void *arg) msg = (hv_vmbus_message*) page_addr + HV_VMBUS_MESSAGE_SINT; for (;;) { - if (msg->header.message_type == HV_MESSAGE_TYPE_NONE) { + if (msg->header.message_type == HV_MESSAGE_TYPE_NONE) break; /* no message */ - } else { + + hdr = (hv_vmbus_channel_msg_header *)msg->u.payload; + msg_type = hdr->message_type; + + if (msg_type >= HV_CHANNEL_MESSAGE_COUNT && !warned) { + warned = true; + printf("VMBUS: unknown message type = %d\n", msg_type); + goto handled; + } + + entry = &g_channel_message_table[msg_type]; + + if (entry->handler_no_sleep) + entry->messageHandler(hdr); + else { + copied = malloc(sizeof(hv_vmbus_message), M_DEVBUF, M_NOWAIT); KASSERT(copied != NULL, @@ -97,11 +116,13 @@ vmbus_msg_swintr(void *arg) " hv_vmbus_message!")); if (copied == NULL) continue; + memcpy(copied, msg, sizeof(hv_vmbus_message)); hv_queue_work_item(hv_vmbus_g_connection.work_queue, - hv_vmbus_on_channel_message, copied); - } - + hv_vmbus_on_channel_message, + copied); + } +handled: msg->header.message_type = HV_MESSAGE_TYPE_NONE; /* diff --git a/sys/dev/hyperv/vmbus/hv_vmbus_priv.h b/sys/dev/hyperv/vmbus/hv_vmbus_priv.h index faa6dec..0503d06 100644 --- a/sys/dev/hyperv/vmbus/hv_vmbus_priv.h +++ b/sys/dev/hyperv/vmbus/hv_vmbus_priv.h @@ -586,6 +586,16 @@ typedef enum { extern hv_vmbus_context hv_vmbus_g_context; extern hv_vmbus_connection hv_vmbus_g_connection; +typedef void (*vmbus_msg_handler)(hv_vmbus_channel_msg_header *msg); + +typedef struct hv_vmbus_channel_msg_table_entry { + hv_vmbus_channel_msg_type messageType; + + bool handler_no_sleep; /* true: the handler doesn't sleep */ + vmbus_msg_handler messageHandler; +} hv_vmbus_channel_msg_table_entry; + +extern hv_vmbus_channel_msg_table_entry g_channel_message_table[]; /* * Private, VM Bus functions diff --git a/sys/dev/isci/scil/scic_sds_phy.h b/sys/dev/isci/scil/scic_sds_phy.h index d8a3c73..501a724 100644 --- a/sys/dev/isci/scil/scic_sds_phy.h +++ b/sys/dev/isci/scil/scic_sds_phy.h @@ -201,7 +201,7 @@ enum SCIC_SDS_PHY_PROTOCOL * @struct SCIC_SDS_PHY * * @brief This structure contains or references all of the data necessary to - * represent the core phy object and SCU harware protocol engine. + * represent the core phy object and SCU hardware protocol engine. */ typedef struct SCIC_SDS_PHY { diff --git a/sys/dev/iscsi/iscsi.c b/sys/dev/iscsi/iscsi.c index 826afd2..d792746 100644 --- a/sys/dev/iscsi/iscsi.c +++ b/sys/dev/iscsi/iscsi.c @@ -1329,6 +1329,7 @@ iscsi_ioctl_daemon_handoff(struct iscsi_softc *sc, struct iscsi_daemon_handoff *handoff) { struct iscsi_session *is; + struct icl_conn *ic; int error; sx_slock(&sc->sc_lock); @@ -1345,6 +1346,7 @@ iscsi_ioctl_daemon_handoff(struct iscsi_softc *sc, return (ESRCH); } ISCSI_SESSION_LOCK(is); + ic = is->is_conn; if (is->is_conf.isc_discovery || is->is_terminating) { ISCSI_SESSION_UNLOCK(is); sx_sunlock(&sc->sc_lock); @@ -1369,18 +1371,24 @@ iscsi_ioctl_daemon_handoff(struct iscsi_softc *sc, is->is_statsn = handoff->idh_statsn; is->is_initial_r2t = handoff->idh_initial_r2t; is->is_immediate_data = handoff->idh_immediate_data; - is->is_max_data_segment_length = handoff->idh_max_data_segment_length; + + /* + * Cap MaxRecvDataSegmentLength obtained from the target to the maximum + * size supported by our ICL module. + */ + is->is_max_data_segment_length = min(ic->ic_max_data_segment_length, + handoff->idh_max_data_segment_length); is->is_max_burst_length = handoff->idh_max_burst_length; is->is_first_burst_length = handoff->idh_first_burst_length; if (handoff->idh_header_digest == ISCSI_DIGEST_CRC32C) - is->is_conn->ic_header_crc32c = true; + ic->ic_header_crc32c = true; else - is->is_conn->ic_header_crc32c = false; + ic->ic_header_crc32c = false; if (handoff->idh_data_digest == ISCSI_DIGEST_CRC32C) - is->is_conn->ic_data_crc32c = true; + ic->ic_data_crc32c = true; else - is->is_conn->ic_data_crc32c = false; + ic->ic_data_crc32c = false; is->is_cmdsn = 0; is->is_expcmdsn = 0; @@ -1399,7 +1407,7 @@ iscsi_ioctl_daemon_handoff(struct iscsi_softc *sc, /* * Handoff without using ICL proxy. */ - error = icl_conn_handoff(is->is_conn, handoff->idh_socket); + error = icl_conn_handoff(ic, handoff->idh_socket); if (error != 0) { sx_sunlock(&sc->sc_lock); iscsi_session_terminate(is); diff --git a/sys/dev/isp/isp.c b/sys/dev/isp/isp.c index d394011..d5c393b 100644 --- a/sys/dev/isp/isp.c +++ b/sys/dev/isp/isp.c @@ -108,12 +108,12 @@ static void isp_fibre_init(ispsoftc_t *); static void isp_fibre_init_2400(ispsoftc_t *); static void isp_clear_portdb(ispsoftc_t *, int); static void isp_mark_portdb(ispsoftc_t *, int); -static int isp_plogx(ispsoftc_t *, int, uint16_t, uint32_t, int, int); +static int isp_plogx(ispsoftc_t *, int, uint16_t, uint32_t, int); static int isp_port_login(ispsoftc_t *, uint16_t, uint32_t); static int isp_port_logout(ispsoftc_t *, uint16_t, uint32_t); -static int isp_getpdb(ispsoftc_t *, int, uint16_t, isp_pdb_t *, int); -static int isp_gethandles(ispsoftc_t *, int, uint16_t *, int *, int, int); -static void isp_dump_chip_portdb(ispsoftc_t *, int, int); +static int isp_getpdb(ispsoftc_t *, int, uint16_t, isp_pdb_t *); +static int isp_gethandles(ispsoftc_t *, int, uint16_t *, int *, int); +static void isp_dump_chip_portdb(ispsoftc_t *, int); static uint64_t isp_get_wwn(ispsoftc_t *, int, int, int); static int isp_fclink_test(ispsoftc_t *, int, int); static int isp_pdb_sync(ispsoftc_t *, int); @@ -2142,19 +2142,41 @@ isp_fibre_init_2400(ispsoftc_t *isp) if ((icbp->icb_fwoptions3 & ICB2400_OPT3_RSPSZ_MASK) == 0) { icbp->icb_fwoptions3 |= ICB2400_OPT3_RSPSZ_24; } - icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_AUTO; if (isp->isp_confopts & ISP_CFG_1GB) { + icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_MASK; icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_1GB; } else if (isp->isp_confopts & ISP_CFG_2GB) { + icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_MASK; icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_2GB; } else if (isp->isp_confopts & ISP_CFG_4GB) { + icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_MASK; icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_4GB; } else if (isp->isp_confopts & ISP_CFG_8GB) { + icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_MASK; icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_8GB; } else if (isp->isp_confopts & ISP_CFG_16GB) { + icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_MASK; icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_16GB; } else { - icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_AUTO; + switch (icbp->icb_fwoptions3 & ICB2400_OPT3_RATE_MASK) { + case ICB2400_OPT3_RATE_4GB: + case ICB2400_OPT3_RATE_8GB: + case ICB2400_OPT3_RATE_16GB: + case ICB2400_OPT3_RATE_AUTO: + break; + case ICB2400_OPT3_RATE_2GB: + if (isp->isp_type <= ISP_HA_FC_2500) + break; + /*FALLTHROUGH*/ + case ICB2400_OPT3_RATE_1GB: + if (isp->isp_type <= ISP_HA_FC_2400) + break; + /*FALLTHROUGH*/ + default: + icbp->icb_fwoptions3 &= ~ICB2400_OPT3_RATE_MASK; + icbp->icb_fwoptions3 |= ICB2400_OPT3_RATE_AUTO; + break; + } } icbp->icb_logintime = ICB_LOGIN_TOV; @@ -2322,6 +2344,161 @@ isp_fibre_init_2400(ispsoftc_t *isp) isp->isp_state = ISP_RUNSTATE; } +static int +isp_fc_enable_vp(ispsoftc_t *isp, int chan) +{ + fcparam *fcp = FCPARAM(isp, chan); + vp_modify_t vp; + void *reqp; + uint8_t resp[QENTRY_LEN]; + + /* Build a VP MODIFY command in memory */ + ISP_MEMZERO(&vp, sizeof(vp)); + vp.vp_mod_hdr.rqs_entry_type = RQSTYPE_VP_MODIFY; + vp.vp_mod_hdr.rqs_entry_count = 1; + vp.vp_mod_cnt = 1; + vp.vp_mod_idx0 = chan; + vp.vp_mod_cmd = VP_MODIFY_ENA; + vp.vp_mod_ports[0].options = ICB2400_VPOPT_ENABLED | + ICB2400_VPOPT_ENA_SNSLOGIN; + if (fcp->role & ISP_ROLE_INITIATOR) + vp.vp_mod_ports[0].options |= ICB2400_VPOPT_INI_ENABLE; + if ((fcp->role & ISP_ROLE_TARGET) == 0) + vp.vp_mod_ports[0].options |= ICB2400_VPOPT_TGT_DISABLE; + if (fcp->isp_loopid < LOCAL_LOOP_LIM) { + vp.vp_mod_ports[0].loopid = fcp->isp_loopid; + if (isp->isp_confopts & ISP_CFG_OWNLOOPID) + vp.vp_mod_ports[0].options |= ICB2400_VPOPT_HARD_ADDRESS; + else + vp.vp_mod_ports[0].options |= ICB2400_VPOPT_PREV_ADDRESS; + } + MAKE_NODE_NAME_FROM_WWN(vp.vp_mod_ports[0].wwpn, fcp->isp_wwpn); + MAKE_NODE_NAME_FROM_WWN(vp.vp_mod_ports[0].wwnn, fcp->isp_wwnn); + + /* Prepare space for response in memory */ + memset(resp, 0xff, sizeof(resp)); + vp.vp_mod_hdl = isp_allocate_handle(isp, resp, ISP_HANDLE_CTRL); + if (vp.vp_mod_hdl == 0) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_MODIFY of Chan %d out of handles", __func__, chan); + return (EIO); + } + + /* Send request and wait for response. */ + reqp = isp_getrqentry(isp); + if (reqp == NULL) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_MODIFY of Chan %d out of rqent", __func__, chan); + isp_destroy_handle(isp, vp.vp_mod_hdl); + return (EIO); + } + isp_put_vp_modify(isp, &vp, (vp_modify_t *)reqp); + ISP_SYNC_REQUEST(isp); + if (msleep(resp, &isp->isp_lock, 0, "VP_MODIFY", 5*hz) == EWOULDBLOCK) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_MODIFY of Chan %d timed out", __func__, chan); + isp_destroy_handle(isp, vp.vp_mod_hdl); + return (EIO); + } + isp_get_vp_modify(isp, (vp_modify_t *)resp, &vp); + + if (vp.vp_mod_hdr.rqs_flags != 0 || vp.vp_mod_status != VP_STS_OK) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_MODIFY of Chan %d failed with flags %x status %d", + __func__, chan, vp.vp_mod_hdr.rqs_flags, vp.vp_mod_status); + return (EIO); + } + return (0); +} + +static int +isp_fc_disable_vp(ispsoftc_t *isp, int chan) +{ + vp_ctrl_info_t vp; + void *reqp; + uint8_t resp[QENTRY_LEN]; + + /* Build a VP CTRL command in memory */ + ISP_MEMZERO(&vp, sizeof(vp)); + vp.vp_ctrl_hdr.rqs_entry_type = RQSTYPE_VP_CTRL; + vp.vp_ctrl_hdr.rqs_entry_count = 1; + if (ISP_CAP_VP0(isp)) { + vp.vp_ctrl_status = 1; + } else { + vp.vp_ctrl_status = 0; + chan--; /* VP0 can not be controlled in this case. */ + } + vp.vp_ctrl_command = VP_CTRL_CMD_DISABLE_VP_LOGO_ALL; + vp.vp_ctrl_vp_count = 1; + vp.vp_ctrl_idmap[chan / 16] |= (1 << chan % 16); + + /* Prepare space for response in memory */ + memset(resp, 0xff, sizeof(resp)); + vp.vp_ctrl_handle = isp_allocate_handle(isp, resp, ISP_HANDLE_CTRL); + if (vp.vp_ctrl_handle == 0) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_CTRL of Chan %d out of handles", __func__, chan); + return (EIO); + } + + /* Send request and wait for response. */ + reqp = isp_getrqentry(isp); + if (reqp == NULL) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_CTRL of Chan %d out of rqent", __func__, chan); + isp_destroy_handle(isp, vp.vp_ctrl_handle); + return (EIO); + } + isp_put_vp_ctrl_info(isp, &vp, (vp_ctrl_info_t *)reqp); + ISP_SYNC_REQUEST(isp); + if (msleep(resp, &isp->isp_lock, 0, "VP_CTRL", 5*hz) == EWOULDBLOCK) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_CTRL of Chan %d timed out", __func__, chan); + isp_destroy_handle(isp, vp.vp_ctrl_handle); + return (EIO); + } + isp_get_vp_ctrl_info(isp, (vp_ctrl_info_t *)resp, &vp); + + if (vp.vp_ctrl_hdr.rqs_flags != 0 || vp.vp_ctrl_status != 0) { + isp_prt(isp, ISP_LOGERR, + "%s: VP_CTRL of Chan %d failed with flags %x status %d %d", + __func__, chan, vp.vp_ctrl_hdr.rqs_flags, + vp.vp_ctrl_status, vp.vp_ctrl_index_fail); + return (EIO); + } + return (0); +} + +static int +isp_fc_change_role(ispsoftc_t *isp, int chan, int new_role) +{ + fcparam *fcp = FCPARAM(isp, chan); + int i, was, res = 0; + + if (chan >= isp->isp_nchan) { + isp_prt(isp, ISP_LOGWARN, "%s: bad channel %d", __func__, chan); + return (ENXIO); + } + if (fcp->role == new_role) + return (0); + for (was = 0, i = 0; i < isp->isp_nchan; i++) { + if (FCPARAM(isp, i)->role != ISP_ROLE_NONE) + was++; + } + if (was == 0 || (was == 1 && fcp->role != ISP_ROLE_NONE)) { + fcp->role = new_role; + return (isp_reinit(isp, 0)); + } + if (fcp->role != ISP_ROLE_NONE) { + res = isp_fc_disable_vp(isp, chan); + isp_clear_portdb(isp, chan); + } + fcp->role = new_role; + if (fcp->role != ISP_ROLE_NONE) + res = isp_fc_enable_vp(isp, chan); + return (res); +} + static void isp_clear_portdb(ispsoftc_t *isp, int chan) { @@ -2373,13 +2550,11 @@ isp_mark_portdb(ispsoftc_t *isp, int chan) * or via FABRIC LOGIN/FABRIC LOGOUT for other cards. */ static int -isp_plogx(ispsoftc_t *isp, int chan, uint16_t handle, uint32_t portid, int flags, int gs) +isp_plogx(ispsoftc_t *isp, int chan, uint16_t handle, uint32_t portid, int flags) { - mbreg_t mbs; - uint8_t q[QENTRY_LEN]; - isp_plogx_t *plp; - fcparam *fcp; - uint8_t *scp; + isp_plogx_t pl; + void *reqp; + uint8_t resp[QENTRY_LEN]; uint32_t sst, parm1; int rval, lev; const char *msg; @@ -2399,63 +2574,58 @@ isp_plogx(ispsoftc_t *isp, int chan, uint16_t handle, uint32_t portid, int flags } } - ISP_MEMZERO(q, QENTRY_LEN); - plp = (isp_plogx_t *) q; - plp->plogx_header.rqs_entry_count = 1; - plp->plogx_header.rqs_entry_type = RQSTYPE_LOGIN; - plp->plogx_handle = 0xffffffff; - plp->plogx_nphdl = handle; - plp->plogx_vphdl = chan; - plp->plogx_portlo = portid; - plp->plogx_rspsz_porthi = (portid >> 16) & 0xff; - plp->plogx_flags = flags; + ISP_MEMZERO(&pl, sizeof(pl)); + pl.plogx_header.rqs_entry_count = 1; + pl.plogx_header.rqs_entry_type = RQSTYPE_LOGIN; + pl.plogx_nphdl = handle; + pl.plogx_vphdl = chan; + pl.plogx_portlo = portid; + pl.plogx_rspsz_porthi = (portid >> 16) & 0xff; + pl.plogx_flags = flags; - if (isp->isp_dblev & ISP_LOGDEBUG1) { - isp_print_bytes(isp, "IOCB LOGX", QENTRY_LEN, plp); - } - - if (gs == 0) { - if (FC_SCRATCH_ACQUIRE(isp, chan)) { - isp_prt(isp, ISP_LOGERR, sacq); - return (-1); - } + /* Prepare space for response in memory */ + memset(resp, 0xff, sizeof(resp)); + pl.plogx_handle = isp_allocate_handle(isp, resp, ISP_HANDLE_CTRL); + if (pl.plogx_handle == 0) { + isp_prt(isp, ISP_LOGERR, + "%s: PLOGX of Chan %d out of handles", __func__, chan); + return (-1); } - fcp = FCPARAM(isp, chan); - scp = fcp->isp_scratch; - isp_put_plogx(isp, plp, (isp_plogx_t *) scp); - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 500000); - mbs.param[1] = QENTRY_LEN; - mbs.param[2] = DMA_WD1(fcp->isp_scdma); - mbs.param[3] = DMA_WD0(fcp->isp_scdma); - mbs.param[6] = DMA_WD3(fcp->isp_scdma); - mbs.param[7] = DMA_WD2(fcp->isp_scdma); - MEMORYBARRIER(isp, SYNC_SFORDEV, 0, QENTRY_LEN, chan); - isp_mboxcmd(isp, &mbs); - if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { - rval = mbs.param[0]; - goto out; + /* Send request and wait for response. */ + reqp = isp_getrqentry(isp); + if (reqp == NULL) { + isp_prt(isp, ISP_LOGERR, + "%s: PLOGX of Chan %d out of rqent", __func__, chan); + isp_destroy_handle(isp, pl.plogx_handle); + return (-1); } - MEMORYBARRIER(isp, SYNC_SFORCPU, QENTRY_LEN, QENTRY_LEN, chan); - scp += QENTRY_LEN; - isp_get_plogx(isp, (isp_plogx_t *) scp, plp); - if (isp->isp_dblev & ISP_LOGDEBUG1) { - isp_print_bytes(isp, "IOCB LOGX response", QENTRY_LEN, plp); + if (isp->isp_dblev & ISP_LOGDEBUG1) + isp_print_bytes(isp, "IOCB LOGX", QENTRY_LEN, &pl); + isp_put_plogx(isp, &pl, (isp_plogx_t *)reqp); + ISP_SYNC_REQUEST(isp); + if (msleep(resp, &isp->isp_lock, 0, "PLOGX", 3 * ICB_LOGIN_TOV * hz) + == EWOULDBLOCK) { + isp_prt(isp, ISP_LOGERR, + "%s: PLOGX of Chan %d timed out", __func__, chan); + isp_destroy_handle(isp, pl.plogx_handle); + return (-1); } + isp_get_plogx(isp, (isp_plogx_t *)resp, &pl); + if (isp->isp_dblev & ISP_LOGDEBUG1) + isp_print_bytes(isp, "IOCB LOGX response", QENTRY_LEN, &pl); - if (plp->plogx_status == PLOGX_STATUS_OK) { - rval = 0; - goto out; - } else if (plp->plogx_status != PLOGX_STATUS_IOCBERR) { + if (pl.plogx_status == PLOGX_STATUS_OK) { + return (0); + } else if (pl.plogx_status != PLOGX_STATUS_IOCBERR) { isp_prt(isp, ISP_LOGWARN, "status 0x%x on port login IOCB channel %d", - plp->plogx_status, chan); - rval = -1; - goto out; + pl.plogx_status, chan); + return (-1); } - sst = plp->plogx_ioparm[0].lo16 | (plp->plogx_ioparm[0].hi16 << 16); - parm1 = plp->plogx_ioparm[1].lo16 | (plp->plogx_ioparm[1].hi16 << 16); + sst = pl.plogx_ioparm[0].lo16 | (pl.plogx_ioparm[0].hi16 << 16); + parm1 = pl.plogx_ioparm[1].lo16 | (pl.plogx_ioparm[1].hi16 << 16); rval = -1; lev = ISP_LOGERR; @@ -2516,17 +2686,13 @@ isp_plogx(ispsoftc_t *isp, int chan, uint16_t handle, uint32_t portid, int flags msg = "no FLOGI_ACC"; break; default: - ISP_SNPRINTF(buf, sizeof (buf), "status %x from %x", plp->plogx_status, flags); + ISP_SNPRINTF(buf, sizeof (buf), "status %x from %x", pl.plogx_status, flags); msg = buf; break; } if (msg) { isp_prt(isp, ISP_LOGERR, "Chan %d PLOGX PortID 0x%06x to N-Port handle 0x%x: %s", chan, portid, handle, msg); } -out: - if (gs == 0) { - FC_SCRATCH_RELEASE(isp, chan); - } return (rval); } @@ -2596,7 +2762,7 @@ isp_port_logout(ispsoftc_t *isp, uint16_t handle, uint32_t portid) } static int -isp_getpdb(ispsoftc_t *isp, int chan, uint16_t id, isp_pdb_t *pdb, int dolock) +isp_getpdb(ispsoftc_t *isp, int chan, uint16_t id, isp_pdb_t *pdb) { fcparam *fcp = FCPARAM(isp, chan); mbreg_t mbs; @@ -2620,18 +2786,14 @@ isp_getpdb(ispsoftc_t *isp, int chan, uint16_t id, isp_pdb_t *pdb, int dolock) mbs.param[3] = DMA_WD0(fcp->isp_scdma); mbs.param[6] = DMA_WD3(fcp->isp_scdma); mbs.param[7] = DMA_WD2(fcp->isp_scdma); - if (dolock) { - if (FC_SCRATCH_ACQUIRE(isp, chan)) { - isp_prt(isp, ISP_LOGERR, sacq); - return (-1); - } + if (FC_SCRATCH_ACQUIRE(isp, chan)) { + isp_prt(isp, ISP_LOGERR, sacq); + return (-1); } MEMORYBARRIER(isp, SYNC_SFORDEV, 0, sizeof (un), chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { - if (dolock) { - FC_SCRATCH_RELEASE(isp, chan); - } + FC_SCRATCH_RELEASE(isp, chan); return (mbs.param[0] | (mbs.param[1] << 16)); } if (IS_24XX(isp)) { @@ -2647,9 +2809,7 @@ isp_getpdb(ispsoftc_t *isp, int chan, uint16_t id, isp_pdb_t *pdb, int dolock) un.bill.pdb_curstate); if (un.bill.pdb_curstate < PDB2400_STATE_PLOGI_DONE || un.bill.pdb_curstate > PDB2400_STATE_LOGGED_IN) { mbs.param[0] = MBOX_NOT_LOGGED_IN; - if (dolock) { - FC_SCRATCH_RELEASE(isp, chan); - } + FC_SCRATCH_RELEASE(isp, chan); return (mbs.param[0]); } } else { @@ -2662,15 +2822,12 @@ isp_getpdb(ispsoftc_t *isp, int chan, uint16_t id, isp_pdb_t *pdb, int dolock) isp_prt(isp, ISP_LOGDEBUG1, "Chan %d handle 0x%x Port 0x%06x", chan, id, pdb->portid); } - if (dolock) { - FC_SCRATCH_RELEASE(isp, chan); - } + FC_SCRATCH_RELEASE(isp, chan); return (0); } static int -isp_gethandles(ispsoftc_t *isp, int chan, uint16_t *handles, int *num, - int dolock, int loop) +isp_gethandles(ispsoftc_t *isp, int chan, uint16_t *handles, int *num, int loop) { fcparam *fcp = FCPARAM(isp, chan); mbreg_t mbs; @@ -2696,18 +2853,14 @@ isp_gethandles(ispsoftc_t *isp, int chan, uint16_t *handles, int *num, mbs.param[3] = DMA_WD3(fcp->isp_scdma); mbs.param[6] = DMA_WD2(fcp->isp_scdma); } - if (dolock) { - if (FC_SCRATCH_ACQUIRE(isp, chan)) { - isp_prt(isp, ISP_LOGERR, sacq); - return (-1); - } + if (FC_SCRATCH_ACQUIRE(isp, chan)) { + isp_prt(isp, ISP_LOGERR, sacq); + return (-1); } MEMORYBARRIER(isp, SYNC_SFORDEV, 0, ISP_FC_SCRLEN, chan); isp_mboxcmd(isp, &mbs); if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { - if (dolock) { - FC_SCRATCH_RELEASE(isp, chan); - } + FC_SCRATCH_RELEASE(isp, chan); return (mbs.param[0] | (mbs.param[1] << 16)); } elp1 = fcp->isp_scratch; @@ -2735,13 +2888,12 @@ isp_gethandles(ispsoftc_t *isp, int chan, uint16_t *handles, int *num, handles[j++] = h; } *num = j; - if (dolock) - FC_SCRATCH_RELEASE(isp, chan); + FC_SCRATCH_RELEASE(isp, chan); return (0); } static void -isp_dump_chip_portdb(ispsoftc_t *isp, int chan, int dolock) +isp_dump_chip_portdb(ispsoftc_t *isp, int chan) { isp_pdb_t pdb; uint16_t lim, nphdl; @@ -2753,7 +2905,7 @@ isp_dump_chip_portdb(ispsoftc_t *isp, int chan, int dolock) lim = NPH_MAX; } for (nphdl = 0; nphdl != lim; nphdl++) { - if (isp_getpdb(isp, chan, nphdl, &pdb, dolock)) { + if (isp_getpdb(isp, chan, nphdl, &pdb)) { continue; } isp_prt(isp, ISP_LOG_SANCFG|ISP_LOGINFO, "Chan %d Handle 0x%04x " @@ -2903,7 +3055,7 @@ isp_fclink_test(ispsoftc_t *isp, int chan, int usdelay) if (fcp->isp_topo == TOPO_F_PORT || fcp->isp_topo == TOPO_FL_PORT) { nphdl = IS_24XX(isp) ? NPH_FL_ID : FL_ID; - r = isp_getpdb(isp, chan, nphdl, &pdb, 1); + r = isp_getpdb(isp, chan, nphdl, &pdb); if (r != 0 || pdb.portid == 0) { if (IS_2100(isp)) { fcp->isp_topo = TOPO_NL_PORT; @@ -3018,7 +3170,7 @@ isp_pdb_sync(ispsoftc_t *isp, int chan) lp->portid, PLOGX_FLG_CMD_LOGO | PLOGX_FLG_IMPLICIT | - PLOGX_FLG_FREE_NPHDL, 0); + PLOGX_FLG_FREE_NPHDL); } /* * Note that we might come out of this with our state @@ -3145,7 +3297,7 @@ isp_fix_portids(ispsoftc_t *isp, int chan) if (VALID_PORT(lp->portid)) continue; - r = isp_getpdb(isp, chan, lp->handle, &pdb, 1); + r = isp_getpdb(isp, chan, lp->handle, &pdb); if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) return; if (r != 0) { @@ -3174,7 +3326,7 @@ isp_scan_loop(ispsoftc_t *isp, int chan) fcparam *fcp = FCPARAM(isp, chan); int idx, lim, r; isp_pdb_t pdb; - uint16_t handles[LOCAL_LOOP_LIM]; + uint16_t *handles; uint16_t handle; if (fcp->isp_loopstate < LOOP_LTEST_DONE) @@ -3196,8 +3348,9 @@ isp_scan_loop(ispsoftc_t *isp, int chan) return (0); } - lim = LOCAL_LOOP_LIM; - r = isp_gethandles(isp, chan, handles, &lim, 1, 1); + handles = (uint16_t *)fcp->isp_scanscratch; + lim = ISP_FC_SCRLEN / 2; + r = isp_gethandles(isp, chan, handles, &lim, 1); if (r != 0) { isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Getting list of handles failed with %x", chan, r); @@ -3247,7 +3400,7 @@ abort: /* * Get the port database entity for this index. */ - r = isp_getpdb(isp, chan, handle, &pdb, 1); + r = isp_getpdb(isp, chan, handle, &pdb); if (fcp->isp_loopstate < LOOP_SCANNING_LOOP) goto abort; if (r != 0) { @@ -3276,20 +3429,10 @@ abort: * * For the 24XX card, we have to use CT-Pass through run via the Execute IOCB * mailbox command. - * - * The net result is to leave the list of Port IDs setting untranslated in - * offset IGPOFF of the FC scratch area, whereupon we'll canonicalize it to - * host order at OGPOFF. */ - -/* - * Take half of our scratch area to store Port IDs - */ -#define GIDLEN (ISP_FC_SCRLEN >> 1) +#define GIDLEN (ISP_FC_SCRLEN - (3 * QENTRY_LEN)) #define NGENT ((GIDLEN - 16) >> 2) -#define IGPOFF (0) -#define OGPOFF (ISP_FC_SCRLEN >> 1) #define XTXOFF (ISP_FC_SCRLEN - (3 * QENTRY_LEN)) /* CT request */ #define CTXOFF (ISP_FC_SCRLEN - (2 * QENTRY_LEN)) /* Request IOCB */ #define ZTXOFF (ISP_FC_SCRLEN - (1 * QENTRY_LEN)) /* Response IOCB */ @@ -3306,21 +3449,25 @@ isp_gid_ft_sns(ispsoftc_t *isp, int chan) uint8_t *scp = fcp->isp_scratch; mbreg_t mbs; - isp_prt(isp, ISP_LOGDEBUG0, "Chan %d scanning fabric (GID_FT) via SNS", chan); + isp_prt(isp, ISP_LOGDEBUG0, "Chan %d requesting GID_FT via SNS", chan); + if (FC_SCRATCH_ACQUIRE(isp, chan)) { + isp_prt(isp, ISP_LOGERR, sacq); + return (-1); + } ISP_MEMZERO(rq, SNS_GID_FT_REQ_SIZE); rq->snscb_rblen = GIDLEN >> 1; - rq->snscb_addr[RQRSP_ADDR0015] = DMA_WD0(fcp->isp_scdma + IGPOFF); - rq->snscb_addr[RQRSP_ADDR1631] = DMA_WD1(fcp->isp_scdma + IGPOFF); - rq->snscb_addr[RQRSP_ADDR3247] = DMA_WD2(fcp->isp_scdma + IGPOFF); - rq->snscb_addr[RQRSP_ADDR4863] = DMA_WD3(fcp->isp_scdma + IGPOFF); + rq->snscb_addr[RQRSP_ADDR0015] = DMA_WD0(fcp->isp_scdma); + rq->snscb_addr[RQRSP_ADDR1631] = DMA_WD1(fcp->isp_scdma); + rq->snscb_addr[RQRSP_ADDR3247] = DMA_WD2(fcp->isp_scdma); + rq->snscb_addr[RQRSP_ADDR4863] = DMA_WD3(fcp->isp_scdma); rq->snscb_sblen = 6; rq->snscb_cmd = SNS_GID_FT; rq->snscb_mword_div_2 = NGENT; rq->snscb_fc4_type = FC4_SCSI; isp_put_gid_ft_request(isp, rq, (sns_gid_ft_req_t *)&scp[CTXOFF]); - MEMORYBARRIER(isp, SYNC_SFORDEV, 0, SNS_GID_FT_REQ_SIZE, chan); + MEMORYBARRIER(isp, SYNC_SFORDEV, CTXOFF, SNS_GID_FT_REQ_SIZE, chan); MBSINIT(&mbs, MBOX_SEND_SNS, MBLOGALL, 10000000); mbs.param[0] = MBOX_SEND_SNS; @@ -3337,6 +3484,12 @@ isp_gid_ft_sns(ispsoftc_t *isp, int chan) return (-1); } } + MEMORYBARRIER(isp, SYNC_SFORCPU, 0, GIDLEN, chan); + if (isp->isp_dblev & ISP_LOGDEBUG1) + isp_print_bytes(isp, "CT response", GIDLEN, scp); + isp_get_gid_ft_response(isp, (sns_gid_ft_rsp_t *)scp, + (sns_gid_ft_rsp_t *)fcp->isp_scanscratch, NGENT); + FC_SCRATCH_RELEASE(isp, chan); return (0); } @@ -3355,7 +3508,11 @@ isp_gid_ft_ct_passthru(ispsoftc_t *isp, int chan) uint32_t *rp; uint8_t *scp = fcp->isp_scratch; - isp_prt(isp, ISP_LOGDEBUG0, "Chan %d scanning fabric (GID_FT) via CT", chan); + isp_prt(isp, ISP_LOGDEBUG0, "Chan %d requesting GID_FT via CT", chan); + if (FC_SCRATCH_ACQUIRE(isp, chan)) { + isp_prt(isp, ISP_LOGERR, sacq); + return (-1); + } /* * Build a Passthrough IOCB in memory. @@ -3368,15 +3525,15 @@ isp_gid_ft_ct_passthru(ispsoftc_t *isp, int chan) pt->ctp_nphdl = fcp->isp_sns_hdl; pt->ctp_cmd_cnt = 1; pt->ctp_vpidx = ISP_GET_VPIDX(isp, chan); - pt->ctp_time = 30; + pt->ctp_time = 10; pt->ctp_rsp_cnt = 1; pt->ctp_rsp_bcnt = GIDLEN; pt->ctp_cmd_bcnt = sizeof (*ct) + sizeof (uint32_t); pt->ctp_dataseg[0].ds_base = DMA_LO32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_basehi = DMA_HI32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_count = sizeof (*ct) + sizeof (uint32_t); - pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma+IGPOFF); - pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma+IGPOFF); + pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma); + pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma); pt->ctp_dataseg[1].ds_count = GIDLEN; if (isp->isp_dblev & ISP_LOGDEBUG1) { isp_print_bytes(isp, "ct IOCB", QENTRY_LEN, pt); @@ -3404,7 +3561,8 @@ isp_gid_ft_ct_passthru(ispsoftc_t *isp, int chan) sizeof (*ct) + sizeof (uint32_t), &scp[XTXOFF]); } ISP_MEMZERO(&scp[ZTXOFF], QENTRY_LEN); - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 500000); + MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, + MBCMD_DEFAULT_TIMEOUT + pt->ctp_time * 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); @@ -3415,7 +3573,7 @@ isp_gid_ft_ct_passthru(ispsoftc_t *isp, int chan) if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { return (-1); } - MEMORYBARRIER(isp, SYNC_SFORCPU, ZTXOFF, QENTRY_LEN, chan); + MEMORYBARRIER(isp, SYNC_SFORCPU, 0, ISP_FC_SCRLEN, chan); pt = &un.plocal; isp_get_ct_pt(isp, (isp_ct_pt_t *) &scp[ZTXOFF], pt); if (isp->isp_dblev & ISP_LOGDEBUG1) { @@ -3424,14 +3582,15 @@ isp_gid_ft_ct_passthru(ispsoftc_t *isp, int chan) if (pt->ctp_status && pt->ctp_status != RQCS_DATA_UNDERRUN) { isp_prt(isp, ISP_LOGWARN, - "Chan %d ISP GID FT CT Passthrough returned 0x%x", + "Chan %d GID_FT CT Passthrough returned 0x%x", chan, pt->ctp_status); return (-1); } - MEMORYBARRIER(isp, SYNC_SFORCPU, IGPOFF, GIDLEN, chan); - if (isp->isp_dblev & ISP_LOGDEBUG1) { - isp_print_bytes(isp, "CT response", GIDLEN, &scp[IGPOFF]); - } + if (isp->isp_dblev & ISP_LOGDEBUG1) + isp_print_bytes(isp, "CT response", GIDLEN, scp); + isp_get_gid_ft_response(isp, (sns_gid_ft_rsp_t *)scp, + (sns_gid_ft_rsp_t *)fcp->isp_scanscratch, NGENT); + FC_SCRATCH_RELEASE(isp, chan); return (0); } @@ -3444,7 +3603,7 @@ isp_scan_fabric(ispsoftc_t *isp, int chan) uint16_t nphdl; isp_pdb_t pdb; int portidx, portlim, r; - sns_gid_ft_rsp_t *rs0, *rs1; + sns_gid_ft_rsp_t *rs; if (fcp->isp_loopstate < LOOP_LSCAN_DONE) return (-1); @@ -3460,13 +3619,6 @@ isp_scan_fabric(ispsoftc_t *isp, int chan) return (0); } - if (FC_SCRATCH_ACQUIRE(isp, chan)) { - isp_prt(isp, ISP_LOGERR, sacq); -fail: - isp_prt(isp, ISP_LOG_SANCFG, - "Chan %d FC fabric scan done (bad)", chan); - return (-1); - } if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) { abort: FC_SCRATCH_RELEASE(isp, chan); @@ -3479,14 +3631,16 @@ abort: * Make sure we still are logged into the fabric controller. */ nphdl = IS_24XX(isp) ? NPH_FL_ID : FL_ID; - r = isp_getpdb(isp, chan, nphdl, &pdb, 0); + r = isp_getpdb(isp, chan, nphdl, &pdb); if ((r & 0xffff) == MBOX_NOT_LOGGED_IN) { - isp_dump_chip_portdb(isp, chan, 0); + isp_dump_chip_portdb(isp, chan); } if (r) { fcp->isp_loopstate = LOOP_LTEST_DONE; - FC_SCRATCH_RELEASE(isp, chan); - goto fail; +fail: + isp_prt(isp, ISP_LOG_SANCFG, + "Chan %d FC fabric scan done (bad)", chan); + return (-1); } /* Get list of port IDs from SNS. */ @@ -3498,42 +3652,36 @@ abort: goto abort; if (r > 0) { fcp->isp_loopstate = LOOP_FSCAN_DONE; - FC_SCRATCH_RELEASE(isp, chan); return (-1); } else if (r < 0) { fcp->isp_loopstate = LOOP_LTEST_DONE; /* try again */ - FC_SCRATCH_RELEASE(isp, chan); return (-1); } - MEMORYBARRIER(isp, SYNC_SFORCPU, IGPOFF, GIDLEN, chan); - rs0 = (sns_gid_ft_rsp_t *) ((uint8_t *)fcp->isp_scratch+IGPOFF); - rs1 = (sns_gid_ft_rsp_t *) ((uint8_t *)fcp->isp_scratch+OGPOFF); - isp_get_gid_ft_response(isp, rs0, rs1, NGENT); + rs = (sns_gid_ft_rsp_t *) fcp->isp_scanscratch; if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; - if (rs1->snscb_cthdr.ct_cmd_resp != LS_ACC) { + if (rs->snscb_cthdr.ct_cmd_resp != LS_ACC) { int level; - if (rs1->snscb_cthdr.ct_reason == 9 && rs1->snscb_cthdr.ct_explanation == 7) { + if (rs->snscb_cthdr.ct_reason == 9 && rs->snscb_cthdr.ct_explanation == 7) { level = ISP_LOG_SANCFG; } else { level = ISP_LOGWARN; } isp_prt(isp, level, "Chan %d Fabric Nameserver rejected GID_FT" " (Reason=0x%x Expl=0x%x)", chan, - rs1->snscb_cthdr.ct_reason, - rs1->snscb_cthdr.ct_explanation); - FC_SCRATCH_RELEASE(isp, chan); + rs->snscb_cthdr.ct_reason, + rs->snscb_cthdr.ct_explanation); fcp->isp_loopstate = LOOP_FSCAN_DONE; return (-1); } /* Check our buffer was big enough to get the full list. */ for (portidx = 0; portidx < NGENT-1; portidx++) { - if (rs1->snscb_ports[portidx].control & 0x80) + if (rs->snscb_ports[portidx].control & 0x80) break; } - if ((rs1->snscb_ports[portidx].control & 0x80) == 0) { + if ((rs->snscb_ports[portidx].control & 0x80) == 0) { isp_prt(isp, ISP_LOGWARN, "fabric too big for scratch area: increase ISP_FC_SCRLEN"); } @@ -3546,24 +3694,24 @@ abort: int npidx; portid = - ((rs1->snscb_ports[portidx].portid[0]) << 16) | - ((rs1->snscb_ports[portidx].portid[1]) << 8) | - ((rs1->snscb_ports[portidx].portid[2])); + ((rs->snscb_ports[portidx].portid[0]) << 16) | + ((rs->snscb_ports[portidx].portid[1]) << 8) | + ((rs->snscb_ports[portidx].portid[2])); for (npidx = portidx + 1; npidx < portlim; npidx++) { uint32_t new_portid = - ((rs1->snscb_ports[npidx].portid[0]) << 16) | - ((rs1->snscb_ports[npidx].portid[1]) << 8) | - ((rs1->snscb_ports[npidx].portid[2])); + ((rs->snscb_ports[npidx].portid[0]) << 16) | + ((rs->snscb_ports[npidx].portid[1]) << 8) | + ((rs->snscb_ports[npidx].portid[2])); if (new_portid == portid) { break; } } if (npidx < portlim) { - rs1->snscb_ports[npidx].portid[0] = 0; - rs1->snscb_ports[npidx].portid[1] = 0; - rs1->snscb_ports[npidx].portid[2] = 0; + rs->snscb_ports[npidx].portid[0] = 0; + rs->snscb_ports[npidx].portid[1] = 0; + rs->snscb_ports[npidx].portid[2] = 0; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d removing duplicate PortID 0x%06x entry from list", chan, portid); } } @@ -3584,9 +3732,9 @@ abort: */ isp_mark_portdb(isp, chan); for (portidx = 0; portidx < portlim; portidx++) { - portid = ((rs1->snscb_ports[portidx].portid[0]) << 16) | - ((rs1->snscb_ports[portidx].portid[1]) << 8) | - ((rs1->snscb_ports[portidx].portid[2])); + portid = ((rs->snscb_ports[portidx].portid[0]) << 16) | + ((rs->snscb_ports[portidx].portid[1]) << 8) | + ((rs->snscb_ports[portidx].portid[2])); isp_prt(isp, ISP_LOG_SANCFG, "Chan %d Checking fabric port 0x%06x", chan, portid); if (portid == 0) { @@ -3608,7 +3756,6 @@ abort: "Chan %d Port 0x%06x@0x%04x [%d] is not probational (0x%x)", chan, lp->portid, lp->handle, FC_PORTDB_TGT(isp, chan, lp), lp->state); - FC_SCRATCH_RELEASE(isp, chan); isp_dump_portdb(isp, chan); goto fail; } @@ -3628,7 +3775,7 @@ abort: * database entry for somebody further along to * decide what to do (policy choice). */ - r = isp_getpdb(isp, chan, lp->handle, &pdb, 0); + r = isp_getpdb(isp, chan, lp->handle, &pdb); if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; if (r != 0) { @@ -3662,7 +3809,6 @@ relogin: if (fcp->isp_loopstate < LOOP_SCANNING_FABRIC) goto abort; - FC_SCRATCH_RELEASE(isp, chan); fcp->isp_loopstate = LOOP_FSCAN_DONE; isp_prt(isp, ISP_LOG_SANCFG, "Chan %d FC fabric scan done", chan); return (0); @@ -3689,7 +3835,7 @@ isp_login_device(ispsoftc_t *isp, int chan, uint32_t portid, isp_pdb_t *p, uint1 return (-1); /* Check if this handle is free. */ - r = isp_getpdb(isp, chan, handle, p, 0); + r = isp_getpdb(isp, chan, handle, p); if (r == 0) { if (p->portid != portid) { /* This handle is busy, try next one. */ @@ -3704,7 +3850,7 @@ isp_login_device(ispsoftc_t *isp, int chan, uint32_t portid, isp_pdb_t *p, uint1 /* * Now try and log into the device */ - r = isp_plogx(isp, chan, handle, portid, PLOGX_FLG_CMD_PLOGI, 1); + r = isp_plogx(isp, chan, handle, portid, PLOGX_FLG_CMD_PLOGI); if (r == 0) { break; } else if ((r & 0xffff) == MBOX_PORT_ID_USED) { @@ -3713,12 +3859,12 @@ isp_login_device(ispsoftc_t *isp, int chan, uint32_t portid, isp_pdb_t *p, uint1 * handle. We need to break that association. We used to try and just substitute the handle, but then * failed to get any data via isp_getpdb (below). */ - if (isp_plogx(isp, chan, r >> 16, portid, PLOGX_FLG_CMD_LOGO | PLOGX_FLG_IMPLICIT | PLOGX_FLG_FREE_NPHDL, 1)) { + if (isp_plogx(isp, chan, r >> 16, portid, PLOGX_FLG_CMD_LOGO | PLOGX_FLG_IMPLICIT | PLOGX_FLG_FREE_NPHDL)) { isp_prt(isp, ISP_LOGERR, "baw... logout of %x failed", r >> 16); } if (FCPARAM(isp, chan)->isp_loopstate != LOOP_SCANNING_FABRIC) return (-1); - r = isp_plogx(isp, chan, handle, portid, PLOGX_FLG_CMD_PLOGI, 1); + r = isp_plogx(isp, chan, handle, portid, PLOGX_FLG_CMD_PLOGI); if (r != 0) i = lim; break; @@ -3742,7 +3888,7 @@ isp_login_device(ispsoftc_t *isp, int chan, uint32_t portid, isp_pdb_t *p, uint1 * so we can crosscheck that it is still what we think it * is and that we also have the role it plays */ - r = isp_getpdb(isp, chan, handle, p, 0); + r = isp_getpdb(isp, chan, handle, p); if (r != 0) { isp_prt(isp, ISP_LOGERR, "Chan %d new device 0x%06x@0x%x disappeared", chan, portid, handle); return (-1); @@ -3839,15 +3985,15 @@ isp_register_fc4_type_24xx(ispsoftc_t *isp, int chan) pt->ctp_nphdl = fcp->isp_sns_hdl; pt->ctp_cmd_cnt = 1; pt->ctp_vpidx = ISP_GET_VPIDX(isp, chan); - pt->ctp_time = 1; + pt->ctp_time = 4; pt->ctp_rsp_cnt = 1; pt->ctp_rsp_bcnt = sizeof (ct_hdr_t); pt->ctp_cmd_bcnt = sizeof (rft_id_t); pt->ctp_dataseg[0].ds_base = DMA_LO32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_basehi = DMA_HI32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_count = sizeof (rft_id_t); - pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma+IGPOFF); - pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma+IGPOFF); + pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma); + pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma); pt->ctp_dataseg[1].ds_count = sizeof (ct_hdr_t); isp_put_ct_pt(isp, pt, (isp_ct_pt_t *) &scp[CTXOFF]); if (isp->isp_dblev & ISP_LOGDEBUG1) { @@ -3878,7 +4024,8 @@ isp_register_fc4_type_24xx(ispsoftc_t *isp, int chan) ISP_MEMZERO(&scp[ZTXOFF], sizeof (ct_hdr_t)); - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 1000000); + MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, + MBCMD_DEFAULT_TIMEOUT + pt->ctp_time * 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); @@ -3904,7 +4051,7 @@ isp_register_fc4_type_24xx(ispsoftc_t *isp, int chan) return (1); } - isp_get_ct_hdr(isp, (ct_hdr_t *) &scp[IGPOFF], ct); + isp_get_ct_hdr(isp, (ct_hdr_t *) scp, ct); FC_SCRATCH_RELEASE(isp, chan); if (ct->ct_cmd_resp == LS_RJT) { @@ -3950,15 +4097,15 @@ isp_register_fc4_features_24xx(ispsoftc_t *isp, int chan) pt->ctp_nphdl = fcp->isp_sns_hdl; pt->ctp_cmd_cnt = 1; pt->ctp_vpidx = ISP_GET_VPIDX(isp, chan); - pt->ctp_time = 1; + pt->ctp_time = 4; pt->ctp_rsp_cnt = 1; pt->ctp_rsp_bcnt = sizeof (ct_hdr_t); pt->ctp_cmd_bcnt = sizeof (rff_id_t); pt->ctp_dataseg[0].ds_base = DMA_LO32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_basehi = DMA_HI32(fcp->isp_scdma+XTXOFF); pt->ctp_dataseg[0].ds_count = sizeof (rff_id_t); - pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma+IGPOFF); - pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma+IGPOFF); + pt->ctp_dataseg[1].ds_base = DMA_LO32(fcp->isp_scdma); + pt->ctp_dataseg[1].ds_basehi = DMA_HI32(fcp->isp_scdma); pt->ctp_dataseg[1].ds_count = sizeof (ct_hdr_t); isp_put_ct_pt(isp, pt, (isp_ct_pt_t *) &scp[CTXOFF]); if (isp->isp_dblev & ISP_LOGDEBUG1) { @@ -3994,7 +4141,8 @@ isp_register_fc4_features_24xx(ispsoftc_t *isp, int chan) ISP_MEMZERO(&scp[ZTXOFF], sizeof (ct_hdr_t)); - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 1000000); + MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, + MBCMD_DEFAULT_TIMEOUT + pt->ctp_time * 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma + CTXOFF); mbs.param[3] = DMA_WD0(fcp->isp_scdma + CTXOFF); @@ -4020,7 +4168,7 @@ isp_register_fc4_features_24xx(ispsoftc_t *isp, int chan) return (1); } - isp_get_ct_hdr(isp, (ct_hdr_t *) &scp[IGPOFF], ct); + isp_get_ct_hdr(isp, (ct_hdr_t *) scp, ct); FC_SCRATCH_RELEASE(isp, chan); if (ct->ct_cmd_resp == LS_RJT) { @@ -4090,7 +4238,7 @@ int isp_start(XS_T *xs) { ispsoftc_t *isp; - uint32_t handle, cdblen; + uint32_t cdblen; uint8_t local[QENTRY_LEN]; ispreq_t *reqp; void *cdbp, *qep; @@ -4381,21 +4529,18 @@ isp_start(XS_T *xs) } ISP_MEMCPY(cdbp, XS_CDBP(xs), cdblen); - *tptr = XS_TIME(xs) / 1000; - if (*tptr == 0 && XS_TIME(xs)) { - *tptr = 1; - } + *tptr = (XS_TIME(xs) + 999) / 1000; if (IS_24XX(isp) && *tptr > 0x1999) { *tptr = 0x1999; } - if (isp_allocate_xs(isp, xs, &handle)) { + /* Whew. Thankfully the same for type 7 requests */ + reqp->req_handle = isp_allocate_handle(isp, xs, ISP_HANDLE_INITIATOR); + if (reqp->req_handle == 0) { isp_prt(isp, ISP_LOG_WARN1, "out of xflist pointers"); XS_SETERR(xs, HBA_BOTCH); return (CMD_EAGAIN); } - /* Whew. Thankfully the same for type 7 requests */ - reqp->req_handle = handle; /* * Set up DMA and/or do any platform dependent swizzling of the request entry @@ -4405,7 +4550,7 @@ isp_start(XS_T *xs) */ dmaresult = ISP_DMASETUP(isp, xs, reqp); if (dmaresult != CMD_QUEUED) { - isp_destroy_handle(isp, handle); + isp_destroy_handle(isp, reqp->req_handle); /* * dmasetup sets actual error in packet, and * return what we were given to return. @@ -4490,13 +4635,14 @@ isp_control(ispsoftc_t *isp, ispctl_t ctl, ...) tmf->tmf_header.rqs_entry_count = 1; tmf->tmf_nphdl = lp->handle; tmf->tmf_delay = 2; - tmf->tmf_timeout = 2; + tmf->tmf_timeout = 4; tmf->tmf_flags = ISP24XX_TMF_TARGET_RESET; tmf->tmf_tidlo = lp->portid; tmf->tmf_tidhi = lp->portid >> 16; tmf->tmf_vpidx = ISP_GET_VPIDX(isp, chan); isp_prt(isp, ISP_LOGALL, "Chan %d Reset N-Port Handle 0x%04x @ Port 0x%06x", chan, lp->handle, lp->portid); - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 5000000); + MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, + MBCMD_DEFAULT_TIMEOUT + tmf->tmf_timeout * 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); @@ -4710,7 +4856,7 @@ isp_control(ispsoftc_t *isp, ispctl_t ctl, ...) tgt = va_arg(ap, int); pdb = va_arg(ap, isp_pdb_t *); va_end(ap); - return (isp_getpdb(isp, chan, tgt, pdb, 1)); + return (isp_getpdb(isp, chan, tgt, pdb)); } break; @@ -4758,11 +4904,11 @@ isp_control(ispsoftc_t *isp, ispctl_t ctl, ...) va_end(ap); if ((p->flags & PLOGX_FLG_CMD_MASK) != PLOGX_FLG_CMD_PLOGI || (p->handle != NIL_HANDLE)) { - return (isp_plogx(isp, p->channel, p->handle, p->portid, p->flags, 0)); + return (isp_plogx(isp, p->channel, p->handle, p->portid, p->flags)); } do { isp_next_handle(isp, &p->handle); - r = isp_plogx(isp, p->channel, p->handle, p->portid, p->flags, 0); + r = isp_plogx(isp, p->channel, p->handle, p->portid, p->flags); if ((r & 0xffff) == MBOX_PORT_ID_USED) { p->handle = r >> 16; r = 0; @@ -5133,12 +5279,6 @@ again: } } - if (!ISP_VALID_HANDLE(isp, sp->req_handle)) { - isp_prt(isp, ISP_LOGERR, "bad request handle 0x%x (iocb type 0x%x)", sp->req_handle, etype); - ISP_MEMZERO(hp, QENTRY_LEN); /* PERF */ - last_etype = etype; - continue; - } xs = isp_find_xs(isp, sp->req_handle); if (xs == NULL) { uint8_t ts = completion_status & 0xff; @@ -5952,6 +6092,8 @@ isp_handle_other_response(ispsoftc_t *isp, int type, isphdr_t *hp, uint32_t *opt { isp_ridacq_t rid; int chan, c; + uint32_t hdl; + void *ptr; switch (type) { case RQSTYPE_STATUS_CONT: @@ -5993,6 +6135,17 @@ isp_handle_other_response(ispsoftc_t *isp, int type, isphdr_t *hp, uint32_t *opt } } return (1); + case RQSTYPE_VP_MODIFY: + case RQSTYPE_VP_CTRL: + case RQSTYPE_LOGIN: + ISP_IOXGET_32(isp, (uint32_t *)(hp + 1), hdl); + ptr = isp_find_xs(isp, hdl); + if (ptr != NULL) { + isp_destroy_handle(isp, hdl); + memcpy(ptr, hp, QENTRY_LEN); + wakeup(ptr); + } + return (1); case RQSTYPE_ATIO: case RQSTYPE_CTIO: case RQSTYPE_ENABLE_LUN: @@ -6879,7 +7032,7 @@ static const uint32_t mbpfc[] = { ISP_FC_OPMAP(0x01, 0x07), /* 0x1f: MBOX_GET_FIRMWARE_STATUS */ ISP_FC_OPMAP_HALF(0x2, 0x01, 0x7e, 0xcf), /* 0x20: MBOX_GET_LOOP_ID */ ISP_FC_OPMAP(0x00, 0x00), /* 0x21: */ - ISP_FC_OPMAP(0x01, 0x07), /* 0x22: MBOX_GET_RETRY_COUNT */ + ISP_FC_OPMAP(0x03, 0x4b), /* 0x22: MBOX_GET_TIMEOUT_PARAMS */ ISP_FC_OPMAP(0x00, 0x00), /* 0x23: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x24: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x25: */ @@ -6895,7 +7048,7 @@ static const uint32_t mbpfc[] = { ISP_FC_OPMAP(0x00, 0x00), /* 0x2f: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x30: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x31: */ - ISP_FC_OPMAP(0x07, 0x07), /* 0x32: MBOX_SET_RETRY_COUNT */ + ISP_FC_OPMAP(0x4b, 0x4b), /* 0x32: MBOX_SET_TIMEOUT_PARAMS */ ISP_FC_OPMAP(0x00, 0x00), /* 0x33: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x34: */ ISP_FC_OPMAP(0x00, 0x00), /* 0x35: */ @@ -7632,6 +7785,7 @@ isp_setdfltfcparm(ispsoftc_t *isp, int chan) fcp->isp_fwoptions |= ICB2400_OPT1_FULL_DUPLEX; } fcp->isp_fwoptions |= ICB2400_OPT1_BOTH_WWNS; + fcp->isp_zfwoptions |= ICB2400_OPT3_RATE_AUTO; } else { fcp->isp_fwoptions |= ICBOPT_FAIRNESS; fcp->isp_fwoptions |= ICBOPT_PDBCHANGE_AE; @@ -7644,6 +7798,7 @@ isp_setdfltfcparm(ispsoftc_t *isp, int chan) * extended options from NVRAM */ fcp->isp_fwoptions &= ~ICBOPT_EXTENDED; + fcp->isp_zfwoptions |= ICBZOPT_RATE_AUTO; } diff --git a/sys/dev/isp/isp_freebsd.c b/sys/dev/isp/isp_freebsd.c index 35ca7fe..e38ae45 100644 --- a/sys/dev/isp/isp_freebsd.c +++ b/sys/dev/isp/isp_freebsd.c @@ -632,7 +632,7 @@ ispioctl(struct cdev *dev, u_long c, caddr_t addr, int flags, struct thread *td) tmf->tmf_header.rqs_entry_count = 1; tmf->tmf_nphdl = lp->handle; tmf->tmf_delay = 2; - tmf->tmf_timeout = 2; + tmf->tmf_timeout = 4; tmf->tmf_tidlo = lp->portid; tmf->tmf_tidhi = lp->portid >> 16; tmf->tmf_vpidx = ISP_GET_VPIDX(isp, chan); @@ -668,7 +668,8 @@ ispioctl(struct cdev *dev, u_long c, caddr_t addr, int flags, struct thread *td) ISP_UNLOCK(isp); break; } - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 5000000); + MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, + MBCMD_DEFAULT_TIMEOUT + tmf->tmf_timeout * 1000000); mbs.param[1] = QENTRY_LEN; mbs.param[2] = DMA_WD1(fcp->isp_scdma); mbs.param[3] = DMA_WD0(fcp->isp_scdma); @@ -1403,7 +1404,7 @@ isp_target_start_ctio(ispsoftc_t *isp, union ccb *ccb, enum Start_Ctio_How how) cto->ct_iid_hi = atp->portid >> 16; cto->ct_oxid = atp->oxid; cto->ct_vpidx = ISP_GET_VPIDX(isp, XS_CHANNEL(ccb)); - cto->ct_timeout = 120; + cto->ct_timeout = (XS_TIME(ccb) + 999) / 1000; cto->ct_flags = atp->tattr << CT7_TASK_ATTR_SHIFT; /* @@ -1555,7 +1556,7 @@ isp_target_start_ctio(ispsoftc_t *isp, union ccb *ccb, enum Start_Ctio_How how) cto->ct_lun = ccb->ccb_h.target_lun; } } - cto->ct_timeout = 10; + cto->ct_timeout = (XS_TIME(ccb) + 999) / 1000; cto->ct_rxid = cso->tag_id; /* @@ -1693,7 +1694,8 @@ isp_target_start_ctio(ispsoftc_t *isp, union ccb *ccb, enum Start_Ctio_How how) TAILQ_INSERT_HEAD(&tptr->waitq, &ccb->ccb_h, periph_links.tqe); break; } - if (isp_allocate_xs_tgt(isp, ccb, &handle)) { + handle = isp_allocate_handle(isp, ccb, ISP_HANDLE_TARGET); + if (handle == 0) { ISP_PATH_PRT(isp, ISP_LOGWARN, ccb->ccb_h.path, "No XFLIST pointers for %s\n", __func__); TAILQ_INSERT_HEAD(&tptr->waitq, &ccb->ccb_h, periph_links.tqe); isp_free_pcmd(isp, ccb); @@ -1722,7 +1724,7 @@ isp_target_start_ctio(ispsoftc_t *isp, union ccb *ccb, enum Start_Ctio_How how) dmaresult = ISP_DMASETUP(isp, cso, (ispreq_t *) local); if (dmaresult != CMD_QUEUED) { - isp_destroy_tgt_handle(isp, handle); + isp_destroy_handle(isp, handle); isp_free_pcmd(isp, ccb); if (dmaresult == CMD_EAGAIN) { TAILQ_INSERT_HEAD(&tptr->waitq, &ccb->ccb_h, periph_links.tqe); @@ -2379,12 +2381,12 @@ isp_handle_platform_ctio(ispsoftc_t *isp, void *arg) uint32_t handle, moved_data = 0, data_requested; handle = ((ct2_entry_t *)arg)->ct_syshandle; - ccb = isp_find_xs_tgt(isp, handle); + ccb = isp_find_xs(isp, handle); if (ccb == NULL) { isp_print_bytes(isp, "null ccb in isp_handle_platform_ctio", QENTRY_LEN, arg); return; } - isp_destroy_tgt_handle(isp, handle); + isp_destroy_handle(isp, handle); data_requested = PISP_PCMD(ccb)->datalen; isp_free_pcmd(isp, ccb); if (isp->isp_nactive) { @@ -3320,7 +3322,7 @@ isp_loop_dead(ispsoftc_t *isp, int chan) for (i = 0; i < isp->isp_maxcmds; i++) { struct ccb_scsiio *xs; - if (!ISP_VALID_HANDLE(isp, isp->isp_xflist[i].handle)) { + if (ISP_H2HT(isp->isp_xflist[i].handle) != ISP_HANDLE_INITIATOR) { continue; } if ((xs = isp->isp_xflist[i].cmd) == NULL) { diff --git a/sys/dev/isp/isp_freebsd.h b/sys/dev/isp/isp_freebsd.h index 9a9093a..7702ee6 100644 --- a/sys/dev/isp/isp_freebsd.h +++ b/sys/dev/isp/isp_freebsd.h @@ -225,8 +225,7 @@ struct isp_fc { struct cam_path *path; struct ispsoftc *isp; struct proc *kproc; - bus_dma_tag_t tdmat; - bus_dmamap_t tdmap; + bus_dmamap_t scmap; uint64_t def_wwpn; uint64_t def_wwnn; time_t loop_down_time; @@ -285,13 +284,18 @@ struct isposinfo { const struct firmware * fw; /* - * DMA related sdtuff + * DMA related stuff */ struct resource * regs; struct resource * regs2; bus_dma_tag_t dmat; - bus_dma_tag_t cdmat; - bus_dmamap_t cdmap; + bus_dma_tag_t reqdmat; + bus_dma_tag_t respdmat; + bus_dma_tag_t atiodmat; + bus_dma_tag_t scdmat; + bus_dmamap_t reqmap; + bus_dmamap_t respmap; + bus_dmamap_t atiomap; /* * Command and transaction related related stuff @@ -406,62 +410,60 @@ struct isposinfo { #define MEMORYBARRIER(isp, type, offset, size, chan) \ switch (type) { \ +case SYNC_REQUEST: \ + bus_dmamap_sync(isp->isp_osinfo.reqdmat, \ + isp->isp_osinfo.reqmap, BUS_DMASYNC_PREWRITE); \ + break; \ +case SYNC_RESULT: \ + bus_dmamap_sync(isp->isp_osinfo.respdmat, \ + isp->isp_osinfo.respmap, BUS_DMASYNC_POSTREAD); \ + break; \ case SYNC_SFORDEV: \ { \ struct isp_fc *fc = ISP_FC_PC(isp, chan); \ - bus_dmamap_sync(fc->tdmat, fc->tdmap, \ + bus_dmamap_sync(isp->isp_osinfo.scdmat, fc->scmap, \ BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); \ break; \ } \ -case SYNC_REQUEST: \ - bus_dmamap_sync(isp->isp_osinfo.cdmat, \ - isp->isp_osinfo.cdmap, \ - BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); \ - break; \ case SYNC_SFORCPU: \ { \ struct isp_fc *fc = ISP_FC_PC(isp, chan); \ - bus_dmamap_sync(fc->tdmat, fc->tdmap, \ + bus_dmamap_sync(isp->isp_osinfo.scdmat, fc->scmap, \ BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); \ break; \ } \ -case SYNC_RESULT: \ - bus_dmamap_sync(isp->isp_osinfo.cdmat, \ - isp->isp_osinfo.cdmap, \ - BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); \ - break; \ case SYNC_REG: \ bus_barrier(isp->isp_osinfo.regs, offset, size, \ BUS_SPACE_BARRIER_READ | BUS_SPACE_BARRIER_WRITE); \ break; \ +case SYNC_ATIOQ: \ + bus_dmamap_sync(isp->isp_osinfo.atiodmat, \ + isp->isp_osinfo.atiomap, BUS_DMASYNC_POSTREAD); \ + break; \ default: \ break; \ } #define MEMORYBARRIERW(isp, type, offset, size, chan) \ switch (type) { \ +case SYNC_REQUEST: \ + bus_dmamap_sync(isp->isp_osinfo.reqdmat, \ + isp->isp_osinfo.reqmap, BUS_DMASYNC_PREWRITE); \ + break; \ case SYNC_SFORDEV: \ { \ struct isp_fc *fc = ISP_FC_PC(isp, chan); \ - bus_dmamap_sync(fc->tdmat, fc->tdmap, \ + bus_dmamap_sync(isp->isp_osinfo.scdmat, fc->scmap, \ BUS_DMASYNC_PREWRITE); \ break; \ } \ -case SYNC_REQUEST: \ - bus_dmamap_sync(isp->isp_osinfo.cdmat, \ - isp->isp_osinfo.cdmap, BUS_DMASYNC_PREWRITE); \ - break; \ case SYNC_SFORCPU: \ { \ struct isp_fc *fc = ISP_FC_PC(isp, chan); \ - bus_dmamap_sync(fc->tdmat, fc->tdmap, \ + bus_dmamap_sync(isp->isp_osinfo.scdmat, fc->scmap, \ BUS_DMASYNC_POSTWRITE); \ break; \ } \ -case SYNC_RESULT: \ - bus_dmamap_sync(isp->isp_osinfo.cdmat, \ - isp->isp_osinfo.cdmap, BUS_DMASYNC_POSTWRITE); \ - break; \ case SYNC_REG: \ bus_barrier(isp->isp_osinfo.regs, offset, size, \ BUS_SPACE_BARRIER_WRITE); \ diff --git a/sys/dev/isp/isp_library.c b/sys/dev/isp/isp_library.c index ec99244..78c7f6f 100644 --- a/sys/dev/isp/isp_library.c +++ b/sys/dev/isp/isp_library.c @@ -247,28 +247,26 @@ copy_and_sync: return (CMD_QUEUED); } -int -isp_allocate_xs(ispsoftc_t *isp, XS_T *xs, uint32_t *handlep) +uint32_t +isp_allocate_handle(ispsoftc_t *isp, void *xs, int type) { isp_hdl_t *hdp; hdp = isp->isp_xffree; - if (hdp == NULL) { - return (-1); - } + if (hdp == NULL) + return (ISP_HANDLE_FREE); isp->isp_xffree = hdp->cmd; hdp->cmd = xs; hdp->handle = (hdp - isp->isp_xflist); - hdp->handle |= (ISP_HANDLE_INITIATOR << ISP_HANDLE_USAGE_SHIFT); + hdp->handle |= (type << ISP_HANDLE_USAGE_SHIFT); hdp->handle |= (isp->isp_seqno++ << ISP_HANDLE_SEQ_SHIFT); - *handlep = hdp->handle; - return (0); + return (hdp->handle); } -XS_T * +void * isp_find_xs(ispsoftc_t *isp, uint32_t handle) { - if (!ISP_VALID_INI_HANDLE(isp, handle)) { + if (!ISP_VALID_HANDLE(isp, handle)) { isp_prt(isp, ISP_LOGERR, "%s: bad handle 0x%x", __func__, handle); return (NULL); } @@ -276,7 +274,7 @@ isp_find_xs(ispsoftc_t *isp, uint32_t handle) } uint32_t -isp_find_handle(ispsoftc_t *isp, XS_T *xs) +isp_find_handle(ispsoftc_t *isp, void *xs) { uint32_t i, foundhdl = ISP_HANDLE_FREE; @@ -292,21 +290,10 @@ isp_find_handle(ispsoftc_t *isp, XS_T *xs) return (foundhdl); } -uint32_t -isp_handle_index(ispsoftc_t *isp, uint32_t handle) -{ - if (!ISP_VALID_HANDLE(isp, handle)) { - isp_prt(isp, ISP_LOGERR, "%s: bad handle 0x%x", __func__, handle); - return (ISP_BAD_HANDLE_INDEX); - } else { - return (handle & ISP_HANDLE_CMD_MASK); - } -} - void isp_destroy_handle(ispsoftc_t *isp, uint32_t handle) { - if (!ISP_VALID_INI_HANDLE(isp, handle)) { + if (!ISP_VALID_HANDLE(isp, handle)) { isp_prt(isp, ISP_LOGERR, "%s: bad handle 0x%x", __func__, handle); } else { isp->isp_xflist[(handle & ISP_HANDLE_CMD_MASK)].handle = ISP_HANDLE_FREE; @@ -573,168 +560,6 @@ isp_fc_toponame(fcparam *fcp) } } -static int -isp_fc_enable_vp(ispsoftc_t *isp, int chan) -{ - fcparam *fcp = FCPARAM(isp, chan); - mbreg_t mbs; - vp_modify_t *vp; - uint8_t qe[QENTRY_LEN], *scp; - - ISP_MEMZERO(qe, QENTRY_LEN); - if (FC_SCRATCH_ACQUIRE(isp, chan)) { - return (EBUSY); - } - scp = fcp->isp_scratch; - - /* - * Build a VP MODIFY command in memory - */ - vp = (vp_modify_t *) qe; - vp->vp_mod_hdr.rqs_entry_type = RQSTYPE_VP_MODIFY; - vp->vp_mod_hdr.rqs_entry_count = 1; - vp->vp_mod_cnt = 1; - vp->vp_mod_idx0 = chan; - vp->vp_mod_cmd = VP_MODIFY_ENA; - vp->vp_mod_ports[0].options = ICB2400_VPOPT_ENABLED | - ICB2400_VPOPT_ENA_SNSLOGIN; - if (fcp->role & ISP_ROLE_INITIATOR) { - vp->vp_mod_ports[0].options |= ICB2400_VPOPT_INI_ENABLE; - } - if ((fcp->role & ISP_ROLE_TARGET) == 0) { - vp->vp_mod_ports[0].options |= ICB2400_VPOPT_TGT_DISABLE; - } - if (fcp->isp_loopid < LOCAL_LOOP_LIM) { - vp->vp_mod_ports[0].loopid = fcp->isp_loopid; - if (isp->isp_confopts & ISP_CFG_OWNLOOPID) - vp->vp_mod_ports[0].options |= - ICB2400_VPOPT_HARD_ADDRESS; - else - vp->vp_mod_ports[0].options |= - ICB2400_VPOPT_PREV_ADDRESS; - } - MAKE_NODE_NAME_FROM_WWN(vp->vp_mod_ports[0].wwpn, fcp->isp_wwpn); - MAKE_NODE_NAME_FROM_WWN(vp->vp_mod_ports[0].wwnn, fcp->isp_wwnn); - isp_put_vp_modify(isp, vp, (vp_modify_t *) scp); - - /* - * Build a EXEC IOCB A64 command that points to the VP MODIFY command - */ - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 0); - mbs.param[1] = QENTRY_LEN; - mbs.param[2] = DMA_WD1(fcp->isp_scdma); - mbs.param[3] = DMA_WD0(fcp->isp_scdma); - mbs.param[6] = DMA_WD3(fcp->isp_scdma); - mbs.param[7] = DMA_WD2(fcp->isp_scdma); - MEMORYBARRIER(isp, SYNC_SFORDEV, 0, 2 * QENTRY_LEN, chan); - isp_control(isp, ISPCTL_RUN_MBOXCMD, &mbs); - if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { - FC_SCRATCH_RELEASE(isp, chan); - return (EIO); - } - MEMORYBARRIER(isp, SYNC_SFORCPU, QENTRY_LEN, QENTRY_LEN, chan); - isp_get_vp_modify(isp, (vp_modify_t *)&scp[QENTRY_LEN], vp); - - FC_SCRATCH_RELEASE(isp, chan); - - if (vp->vp_mod_status != VP_STS_OK) { - isp_prt(isp, ISP_LOGERR, "%s: VP_MODIFY of Chan %d failed with status %d", __func__, chan, vp->vp_mod_status); - return (EIO); - } - return (0); -} - -static int -isp_fc_disable_vp(ispsoftc_t *isp, int chan) -{ - fcparam *fcp = FCPARAM(isp, chan); - mbreg_t mbs; - vp_ctrl_info_t *vp; - uint8_t qe[QENTRY_LEN], *scp; - - ISP_MEMZERO(qe, QENTRY_LEN); - if (FC_SCRATCH_ACQUIRE(isp, chan)) { - return (EBUSY); - } - scp = fcp->isp_scratch; - - /* - * Build a VP CTRL command in memory - */ - vp = (vp_ctrl_info_t *) qe; - vp->vp_ctrl_hdr.rqs_entry_type = RQSTYPE_VP_CTRL; - vp->vp_ctrl_hdr.rqs_entry_count = 1; - if (ISP_CAP_VP0(isp)) { - vp->vp_ctrl_status = 1; - } else { - vp->vp_ctrl_status = 0; - chan--; /* VP0 can not be controlled in this case. */ - } - vp->vp_ctrl_command = VP_CTRL_CMD_DISABLE_VP_LOGO_ALL; - vp->vp_ctrl_vp_count = 1; - vp->vp_ctrl_idmap[chan / 16] |= (1 << chan % 16); - isp_put_vp_ctrl_info(isp, vp, (vp_ctrl_info_t *) scp); - - /* - * Build a EXEC IOCB A64 command that points to the VP CTRL command - */ - MBSINIT(&mbs, MBOX_EXEC_COMMAND_IOCB_A64, MBLOGALL, 0); - mbs.param[1] = QENTRY_LEN; - mbs.param[2] = DMA_WD1(fcp->isp_scdma); - mbs.param[3] = DMA_WD0(fcp->isp_scdma); - mbs.param[6] = DMA_WD3(fcp->isp_scdma); - mbs.param[7] = DMA_WD2(fcp->isp_scdma); - MEMORYBARRIER(isp, SYNC_SFORDEV, 0, 2 * QENTRY_LEN, chan); - isp_control(isp, ISPCTL_RUN_MBOXCMD, &mbs); - if (mbs.param[0] != MBOX_COMMAND_COMPLETE) { - FC_SCRATCH_RELEASE(isp, chan); - return (EIO); - } - MEMORYBARRIER(isp, SYNC_SFORCPU, QENTRY_LEN, QENTRY_LEN, chan); - isp_get_vp_ctrl_info(isp, (vp_ctrl_info_t *)&scp[QENTRY_LEN], vp); - - FC_SCRATCH_RELEASE(isp, chan); - - if (vp->vp_ctrl_status != 0) { - isp_prt(isp, ISP_LOGERR, - "%s: VP_CTRL of Chan %d failed with status %d %d", - __func__, chan, vp->vp_ctrl_status, vp->vp_ctrl_index_fail); - return (EIO); - } - return (0); -} - -/* - * Change Roles - */ -int -isp_fc_change_role(ispsoftc_t *isp, int chan, int new_role) -{ - fcparam *fcp = FCPARAM(isp, chan); - int i, was, res = 0; - - if (chan >= isp->isp_nchan) { - isp_prt(isp, ISP_LOGWARN, "%s: bad channel %d", __func__, chan); - return (ENXIO); - } - if (fcp->role == new_role) - return (0); - for (was = 0, i = 0; i < isp->isp_nchan; i++) { - if (FCPARAM(isp, i)->role != ISP_ROLE_NONE) - was++; - } - if (was == 0 || (was == 1 && fcp->role != ISP_ROLE_NONE)) { - fcp->role = new_role; - return (isp_reinit(isp, 0)); - } - if (fcp->role != ISP_ROLE_NONE) - res = isp_fc_disable_vp(isp, chan); - fcp->role = new_role; - if (fcp->role != ISP_ROLE_NONE) - res = isp_fc_enable_vp(isp, chan); - return (res); -} - void isp_clear_commands(ispsoftc_t *isp) { @@ -745,46 +570,49 @@ isp_clear_commands(ispsoftc_t *isp) #endif for (tmp = 0; isp->isp_xflist && tmp < isp->isp_maxcmds; tmp++) { - XS_T *xs; hdp = &isp->isp_xflist[tmp]; - if (hdp->handle == ISP_HANDLE_FREE) { - continue; - } - xs = hdp->cmd; - if (XS_XFRLEN(xs)) { - ISP_DMAFREE(isp, xs, hdp->handle); - XS_SET_RESID(xs, XS_XFRLEN(xs)); - } else { - XS_SET_RESID(xs, 0); + switch (ISP_H2HT(hdp->handle)) { + case ISP_HANDLE_INITIATOR: { + XS_T *xs = hdp->cmd; + if (XS_XFRLEN(xs)) { + ISP_DMAFREE(isp, xs, hdp->handle); + XS_SET_RESID(xs, XS_XFRLEN(xs)); + } else { + XS_SET_RESID(xs, 0); + } + isp_destroy_handle(isp, hdp->handle); + XS_SETERR(xs, HBA_BUSRESET); + isp_done(xs); + break; } - hdp->handle = 0; - hdp->cmd = NULL; - XS_SETERR(xs, HBA_BUSRESET); - isp_done(xs); - } #ifdef ISP_TARGET_MODE - for (tmp = 0; isp->isp_tgtlist && tmp < isp->isp_maxcmds; tmp++) { - uint8_t local[QENTRY_LEN]; - hdp = &isp->isp_tgtlist[tmp]; - if (hdp->handle == ISP_HANDLE_FREE) { - continue; + case ISP_HANDLE_TARGET: { + uint8_t local[QENTRY_LEN]; + ISP_DMAFREE(isp, hdp->cmd, hdp->handle); + ISP_MEMZERO(local, QENTRY_LEN); + if (IS_24XX(isp)) { + ct7_entry_t *ctio = (ct7_entry_t *) local; + ctio->ct_syshandle = hdp->handle; + ctio->ct_nphdl = CT_HBA_RESET; + ctio->ct_header.rqs_entry_type = RQSTYPE_CTIO7; + } else { + ct2_entry_t *ctio = (ct2_entry_t *) local; + ctio->ct_syshandle = hdp->handle; + ctio->ct_status = CT_HBA_RESET; + ctio->ct_header.rqs_entry_type = RQSTYPE_CTIO2; + } + isp_async(isp, ISPASYNC_TARGET_ACTION, local); + break; } - ISP_DMAFREE(isp, hdp->cmd, hdp->handle); - ISP_MEMZERO(local, QENTRY_LEN); - if (IS_24XX(isp)) { - ct7_entry_t *ctio = (ct7_entry_t *) local; - ctio->ct_syshandle = hdp->handle; - ctio->ct_nphdl = CT_HBA_RESET; - ctio->ct_header.rqs_entry_type = RQSTYPE_CTIO7; - } else { - ct2_entry_t *ctio = (ct2_entry_t *) local; - ctio->ct_syshandle = hdp->handle; - ctio->ct_status = CT_HBA_RESET; - ctio->ct_header.rqs_entry_type = RQSTYPE_CTIO2; +#endif + case ISP_HANDLE_CTRL: + wakeup(hdp->cmd); + isp_destroy_handle(isp, hdp->handle); + break; } - isp_async(isp, ISPASYNC_TARGET_ACTION, local); } +#ifdef ISP_TARGET_MODE for (tmp = 0; tmp < isp->isp_nchan; tmp++) { ISP_MEMZERO(¬ify, sizeof (isp_notify_t)); notify.nt_ncode = NT_HBA_RESET; @@ -2385,69 +2213,6 @@ isp_send_tgt_cmd(ispsoftc_t *isp, void *fqe, void *segp, uint32_t nsegs, uint32_ return (CMD_QUEUED); } -int -isp_allocate_xs_tgt(ispsoftc_t *isp, void *xs, uint32_t *handlep) -{ - isp_hdl_t *hdp; - - hdp = isp->isp_tgtfree; - if (hdp == NULL) { - return (-1); - } - isp->isp_tgtfree = hdp->cmd; - hdp->cmd = xs; - hdp->handle = (hdp - isp->isp_tgtlist); - hdp->handle |= (ISP_HANDLE_TARGET << ISP_HANDLE_USAGE_SHIFT); - /* - * Target handles for SCSI cards are only 16 bits, so - * sequence number protection will be ommitted. - */ - if (IS_FC(isp)) { - hdp->handle |= (isp->isp_seqno++ << ISP_HANDLE_SEQ_SHIFT); - } - *handlep = hdp->handle; - return (0); -} - -void * -isp_find_xs_tgt(ispsoftc_t *isp, uint32_t handle) -{ - if (!ISP_VALID_TGT_HANDLE(isp, handle)) { - isp_prt(isp, ISP_LOGERR, "%s: bad handle 0x%x", __func__, handle); - return (NULL); - } - return (isp->isp_tgtlist[(handle & ISP_HANDLE_CMD_MASK)].cmd); -} - -uint32_t -isp_find_tgt_handle(ispsoftc_t *isp, void *xs) -{ - uint32_t i, foundhdl = ISP_HANDLE_FREE; - - if (xs != NULL) { - for (i = 0; i < isp->isp_maxcmds; i++) { - if (isp->isp_tgtlist[i].cmd != xs) { - continue; - } - foundhdl = isp->isp_tgtlist[i].handle; - break; - } - } - return (foundhdl); -} - -void -isp_destroy_tgt_handle(ispsoftc_t *isp, uint32_t handle) -{ - if (!ISP_VALID_TGT_HANDLE(isp, handle)) { - isp_prt(isp, ISP_LOGERR, "%s: bad handle 0x%x", __func__, handle); - } else { - isp->isp_tgtlist[(handle & ISP_HANDLE_CMD_MASK)].handle = ISP_HANDLE_FREE; - isp->isp_tgtlist[(handle & ISP_HANDLE_CMD_MASK)].cmd = isp->isp_tgtfree; - isp->isp_tgtfree = &isp->isp_tgtlist[(handle & ISP_HANDLE_CMD_MASK)]; - } -} - #endif /* diff --git a/sys/dev/isp/isp_library.h b/sys/dev/isp/isp_library.h index efe4e0e..922a98b 100644 --- a/sys/dev/isp/isp_library.h +++ b/sys/dev/isp/isp_library.h @@ -43,10 +43,9 @@ int isp_send_cmd(ispsoftc_t *, void *, void *, uint32_t, uint32_t, isp_ddir_t, i * * These handles are associate with a command. */ -int isp_allocate_xs(ispsoftc_t *, XS_T *, uint32_t *); -XS_T * isp_find_xs(ispsoftc_t *, uint32_t); -uint32_t isp_find_handle(ispsoftc_t *, XS_T *); -uint32_t isp_handle_index(ispsoftc_t *, uint32_t); +uint32_t isp_allocate_handle(ispsoftc_t *, void *, int); +void *isp_find_xs(ispsoftc_t *, uint32_t); +uint32_t isp_find_handle(ispsoftc_t *, void *); void isp_destroy_handle(ispsoftc_t *, uint32_t); /* @@ -72,9 +71,6 @@ const char *isp_fc_fw_statename(int); const char *isp_fc_loop_statename(int); const char *isp_fc_toponame(fcparam *); -int isp_fc_change_role(ispsoftc_t *, int, int); - - /* * Cleanup */ @@ -165,11 +161,6 @@ void isp_put_fcp_rsp_iu(ispsoftc_t *isp, fcp_rsp_iu_t *, fcp_rsp_iu_t *); #endif int isp_send_tgt_cmd(ispsoftc_t *, void *, void *, uint32_t, uint32_t, isp_ddir_t, void *, uint32_t); - -int isp_allocate_xs_tgt(ispsoftc_t *, void *, uint32_t *); -void *isp_find_xs_tgt(ispsoftc_t *, uint32_t); -uint32_t isp_find_tgt_handle(ispsoftc_t *, void *); -void isp_destroy_tgt_handle(ispsoftc_t *, uint32_t); #endif int isp_find_pdb_empty(ispsoftc_t *, int, fcportdb_t **); int isp_find_pdb_by_wwpn(ispsoftc_t *, int, uint64_t, fcportdb_t **); diff --git a/sys/dev/isp/isp_pci.c b/sys/dev/isp/isp_pci.c index d30f6cb..11386ee 100644 --- a/sys/dev/isp/isp_pci.c +++ b/sys/dev/isp/isp_pci.c @@ -1539,78 +1539,17 @@ isp_pci_wr_reg_2600(ispsoftc_t *isp, int regoff, uint32_t val) struct imush { - ispsoftc_t *isp; - caddr_t vbase; - int chan; + bus_addr_t maddr; int error; }; -static void imc(void *, bus_dma_segment_t *, int, int); -static void imc1(void *, bus_dma_segment_t *, int, int); - static void imc(void *arg, bus_dma_segment_t *segs, int nseg, int error) { struct imush *imushp = (struct imush *) arg; - isp_ecmd_t *ecmd; - - if (error) { - imushp->error = error; - return; - } - if (nseg != 1) { - imushp->error = EINVAL; - return; - } - isp_prt(imushp->isp, ISP_LOGDEBUG0, "request/result area @ 0x%jx/0x%jx", (uintmax_t) segs->ds_addr, (uintmax_t) segs->ds_len); - - imushp->isp->isp_rquest = imushp->vbase; - imushp->isp->isp_rquest_dma = segs->ds_addr; - segs->ds_addr += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(imushp->isp)); - imushp->vbase += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(imushp->isp)); - - imushp->isp->isp_result_dma = segs->ds_addr; - imushp->isp->isp_result = imushp->vbase; - segs->ds_addr += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(imushp->isp)); - imushp->vbase += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(imushp->isp)); - - if (imushp->isp->isp_type >= ISP_HA_FC_2200) { - imushp->isp->isp_osinfo.ecmd_dma = segs->ds_addr; - imushp->isp->isp_osinfo.ecmd_free = (isp_ecmd_t *)imushp->vbase; - imushp->isp->isp_osinfo.ecmd_base = imushp->isp->isp_osinfo.ecmd_free; - for (ecmd = imushp->isp->isp_osinfo.ecmd_free; ecmd < &imushp->isp->isp_osinfo.ecmd_free[N_XCMDS]; ecmd++) { - if (ecmd == &imushp->isp->isp_osinfo.ecmd_free[N_XCMDS - 1]) { - ecmd->next = NULL; - } else { - ecmd->next = ecmd + 1; - } - } - } -#ifdef ISP_TARGET_MODE - segs->ds_addr += (N_XCMDS * XCMD_SIZE); - imushp->vbase += (N_XCMDS * XCMD_SIZE); - if (IS_24XX(imushp->isp)) { - imushp->isp->isp_atioq_dma = segs->ds_addr; - imushp->isp->isp_atioq = imushp->vbase; - } -#endif -} -static void -imc1(void *arg, bus_dma_segment_t *segs, int nseg, int error) -{ - struct imush *imushp = (struct imush *) arg; - if (error) { - imushp->error = error; - return; - } - if (nseg != 1) { - imushp->error = EINVAL; - return; - } - isp_prt(imushp->isp, ISP_LOGDEBUG0, "scdma @ 0x%jx/0x%jx", (uintmax_t) segs->ds_addr, (uintmax_t) segs->ds_len); - FCPARAM(imushp->isp, imushp->chan)->isp_scdma = segs->ds_addr; - FCPARAM(imushp->isp, imushp->chan)->isp_scratch = imushp->vbase; + if (!(imushp->error = error)) + imushp->maddr = segs[0].ds_addr; } static int @@ -1623,6 +1562,7 @@ isp_pci_mbxdma(ispsoftc_t *isp) bus_addr_t llim; /* low limit of unavailable dma */ bus_addr_t hlim; /* high limit of unavailable dma */ struct imush im; + isp_ecmd_t *ecmd; /* * Already been here? If so, leave... @@ -1684,97 +1624,129 @@ isp_pci_mbxdma(ispsoftc_t *isp) isp->isp_xflist[len].cmd = &isp->isp_xflist[len+1]; } isp->isp_xffree = isp->isp_xflist; -#ifdef ISP_TARGET_MODE - len = sizeof (isp_hdl_t) * isp->isp_maxcmds; - isp->isp_tgtlist = (isp_hdl_t *) malloc(len, M_DEVBUF, M_WAITOK | M_ZERO); - if (isp->isp_tgtlist == NULL) { - free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); - free(isp->isp_xflist, M_DEVBUF); - ISP_LOCK(isp); - isp_prt(isp, ISP_LOGERR, "cannot alloc tgtlist array"); - return (1); - } - for (len = 0; len < isp->isp_maxcmds - 1; len++) { - isp->isp_tgtlist[len].cmd = &isp->isp_tgtlist[len+1]; - } - isp->isp_tgtfree = isp->isp_tgtlist; -#endif /* - * Allocate and map the request and result queues (and ATIO queue - * if we're a 2400 supporting target mode), and a region for - * external dma addressable command/status structures (23XX and - * later). + * Allocate and map the request queue and a region for external + * DMA addressable command/status structures (22XX and later). */ len = ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); - len += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); -#ifdef ISP_TARGET_MODE - if (IS_24XX(isp)) { - len += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); - } -#endif - if (isp->isp_type >= ISP_HA_FC_2200) { + if (isp->isp_type >= ISP_HA_FC_2200) len += (N_XCMDS * XCMD_SIZE); + if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, slim, + BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, + len, 1, len, 0, &isp->isp_osinfo.reqdmat)) { + isp_prt(isp, ISP_LOGERR, "cannot create request DMA tag"); + goto bad1; + } + if (bus_dmamem_alloc(isp->isp_osinfo.reqdmat, (void **)&base, + BUS_DMA_COHERENT, &isp->isp_osinfo.reqmap) != 0) { + isp_prt(isp, ISP_LOGERR, "cannot allocate request DMA memory"); + bus_dma_tag_destroy(isp->isp_osinfo.reqdmat); + goto bad1; + } + isp->isp_rquest = base; + im.error = 0; + if (bus_dmamap_load(isp->isp_osinfo.reqdmat, isp->isp_osinfo.reqmap, + base, len, imc, &im, 0) || im.error) { + isp_prt(isp, ISP_LOGERR, "error loading request DMA map %d", im.error); + goto bad1; + } + isp_prt(isp, ISP_LOGDEBUG0, "request area @ 0x%jx/0x%jx", + (uintmax_t)im.maddr, (uintmax_t)len); + isp->isp_rquest_dma = im.maddr; + base += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); + im.maddr += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); + if (isp->isp_type >= ISP_HA_FC_2200) { + isp->isp_osinfo.ecmd_dma = im.maddr; + isp->isp_osinfo.ecmd_free = (isp_ecmd_t *)base; + isp->isp_osinfo.ecmd_base = isp->isp_osinfo.ecmd_free; + for (ecmd = isp->isp_osinfo.ecmd_free; + ecmd < &isp->isp_osinfo.ecmd_free[N_XCMDS]; ecmd++) { + if (ecmd == &isp->isp_osinfo.ecmd_free[N_XCMDS - 1]) + ecmd->next = NULL; + else + ecmd->next = ecmd + 1; + } } /* - * Create a tag for the control spaces. We don't always need this - * to be 32 bits, but we do this for simplicity and speed's sake. + * Allocate and map the result queue. */ - if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, slim, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, len, 1, slim, 0, &isp->isp_osinfo.cdmat)) { - isp_prt(isp, ISP_LOGERR, "cannot create a dma tag for control spaces"); - free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); - free(isp->isp_xflist, M_DEVBUF); -#ifdef ISP_TARGET_MODE - free(isp->isp_tgtlist, M_DEVBUF); -#endif - ISP_LOCK(isp); - return (1); + len = ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); + if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, slim, + BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, + len, 1, len, 0, &isp->isp_osinfo.respdmat)) { + isp_prt(isp, ISP_LOGERR, "cannot create response DMA tag"); + goto bad1; + } + if (bus_dmamem_alloc(isp->isp_osinfo.respdmat, (void **)&base, + BUS_DMA_COHERENT, &isp->isp_osinfo.respmap) != 0) { + isp_prt(isp, ISP_LOGERR, "cannot allocate response DMA memory"); + bus_dma_tag_destroy(isp->isp_osinfo.respdmat); + goto bad1; + } + isp->isp_result = base; + im.error = 0; + if (bus_dmamap_load(isp->isp_osinfo.respdmat, isp->isp_osinfo.respmap, + base, len, imc, &im, 0) || im.error) { + isp_prt(isp, ISP_LOGERR, "error loading response DMA map %d", im.error); + goto bad1; } + isp_prt(isp, ISP_LOGDEBUG0, "response area @ 0x%jx/0x%jx", + (uintmax_t)im.maddr, (uintmax_t)len); + isp->isp_result_dma = im.maddr; - if (bus_dmamem_alloc(isp->isp_osinfo.cdmat, (void **)&base, BUS_DMA_NOWAIT | BUS_DMA_COHERENT, &isp->isp_osinfo.cdmap) != 0) { - isp_prt(isp, ISP_LOGERR, "cannot allocate %d bytes of CCB memory", len); - bus_dma_tag_destroy(isp->isp_osinfo.cdmat); - free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); - free(isp->isp_xflist, M_DEVBUF); #ifdef ISP_TARGET_MODE - free(isp->isp_tgtlist, M_DEVBUF); -#endif - ISP_LOCK(isp); - return (1); - } - - im.isp = isp; - im.chan = 0; - im.vbase = base; - im.error = 0; - - bus_dmamap_load(isp->isp_osinfo.cdmat, isp->isp_osinfo.cdmap, base, len, imc, &im, 0); - if (im.error) { - isp_prt(isp, ISP_LOGERR, "error %d loading dma map for control areas", im.error); - goto bad; + /* + * Allocate and map ATIO queue on 24xx with target mode. + */ + if (IS_24XX(isp)) { + len = ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); + if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, slim, + BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, + len, 1, len, 0, &isp->isp_osinfo.atiodmat)) { + isp_prt(isp, ISP_LOGERR, "cannot create ATIO DMA tag"); + goto bad1; + } + if (bus_dmamem_alloc(isp->isp_osinfo.atiodmat, (void **)&base, + BUS_DMA_COHERENT, &isp->isp_osinfo.atiomap) != 0) { + isp_prt(isp, ISP_LOGERR, "cannot allocate ATIO DMA memory"); + bus_dma_tag_destroy(isp->isp_osinfo.atiodmat); + goto bad1; + } + isp->isp_atioq = base; + im.error = 0; + if (bus_dmamap_load(isp->isp_osinfo.atiodmat, isp->isp_osinfo.atiomap, + base, len, imc, &im, 0) || im.error) { + isp_prt(isp, ISP_LOGERR, "error loading ATIO DMA map %d", im.error); + goto bad; + } + isp_prt(isp, ISP_LOGDEBUG0, "ATIO area @ 0x%jx/0x%jx", + (uintmax_t)im.maddr, (uintmax_t)len); + isp->isp_atioq_dma = im.maddr; } +#endif if (IS_FC(isp)) { + if (isp_dma_tag_create(isp->isp_osinfo.dmat, 64, slim, + BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, + ISP_FC_SCRLEN, 1, ISP_FC_SCRLEN, 0, &isp->isp_osinfo.scdmat)) { + goto bad; + } for (cmap = 0; cmap < isp->isp_nchan; cmap++) { struct isp_fc *fc = ISP_FC_PC(isp, cmap); - if (isp_dma_tag_create(isp->isp_osinfo.dmat, 64, slim, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, ISP_FC_SCRLEN, 1, slim, 0, &fc->tdmat)) { - goto bad; - } - if (bus_dmamem_alloc(fc->tdmat, (void **)&base, BUS_DMA_NOWAIT | BUS_DMA_COHERENT, &fc->tdmap) != 0) { - bus_dma_tag_destroy(fc->tdmat); + if (bus_dmamem_alloc(isp->isp_osinfo.scdmat, + (void **)&base, BUS_DMA_COHERENT, &fc->scmap) != 0) goto bad; - } - im.isp = isp; - im.chan = cmap; - im.vbase = base; + FCPARAM(isp, cmap)->isp_scratch = base; im.error = 0; - bus_dmamap_load(fc->tdmat, fc->tdmap, base, ISP_FC_SCRLEN, imc1, &im, 0); - if (im.error) { - bus_dmamem_free(fc->tdmat, base, fc->tdmap); - bus_dma_tag_destroy(fc->tdmat); + if (bus_dmamap_load(isp->isp_osinfo.scdmat, fc->scmap, + base, ISP_FC_SCRLEN, imc, &im, 0) || im.error) { + bus_dmamem_free(isp->isp_osinfo.scdmat, + base, fc->scmap); goto bad; } + FCPARAM(isp, cmap)->isp_scdma = im.maddr; if (!IS_2100(isp)) { for (i = 0; i < INITIAL_NEXUS_COUNT; i++) { struct isp_nexus *n = malloc(sizeof (struct isp_nexus), M_DEVBUF, M_NOWAIT | M_ZERO); @@ -1815,25 +1787,52 @@ isp_pci_mbxdma(ispsoftc_t *isp) return (0); bad: - while (--cmap >= 0) { - struct isp_fc *fc = ISP_FC_PC(isp, cmap); - bus_dmamap_unload(fc->tdmat, fc->tdmap); - bus_dmamem_free(fc->tdmat, base, fc->tdmap); - bus_dma_tag_destroy(fc->tdmat); - while (fc->nexus_free_list) { - struct isp_nexus *n = fc->nexus_free_list; - fc->nexus_free_list = n->next; - free(n, M_DEVBUF); + if (IS_FC(isp)) { + while (--cmap >= 0) { + struct isp_fc *fc = ISP_FC_PC(isp, cmap); + bus_dmamap_unload(isp->isp_osinfo.scdmat, fc->scmap); + bus_dmamem_free(isp->isp_osinfo.scdmat, base, fc->scmap); + while (fc->nexus_free_list) { + struct isp_nexus *n = fc->nexus_free_list; + fc->nexus_free_list = n->next; + free(n, M_DEVBUF); + } } + bus_dma_tag_destroy(isp->isp_osinfo.scdmat); + } +bad1: + if (isp->isp_rquest_dma != 0) { + bus_dmamap_unload(isp->isp_osinfo.reqdmat, + isp->isp_osinfo.reqmap); + } + if (isp->isp_rquest != NULL) { + bus_dmamem_free(isp->isp_osinfo.reqdmat, isp->isp_rquest, + isp->isp_osinfo.reqmap); + bus_dma_tag_destroy(isp->isp_osinfo.reqdmat); + } + if (isp->isp_result_dma != 0) { + bus_dmamap_unload(isp->isp_osinfo.respdmat, + isp->isp_osinfo.respmap); + } + if (isp->isp_result != NULL) { + bus_dmamem_free(isp->isp_osinfo.respdmat, isp->isp_result, + isp->isp_osinfo.respmap); + bus_dma_tag_destroy(isp->isp_osinfo.respdmat); } - if (isp->isp_rquest_dma != 0) - bus_dmamap_unload(isp->isp_osinfo.cdmat, isp->isp_osinfo.cdmap); - bus_dmamem_free(isp->isp_osinfo.cdmat, base, isp->isp_osinfo.cdmap); - bus_dma_tag_destroy(isp->isp_osinfo.cdmat); - free(isp->isp_xflist, M_DEVBUF); #ifdef ISP_TARGET_MODE - free(isp->isp_tgtlist, M_DEVBUF); + if (IS_24XX(isp)) { + if (isp->isp_atioq_dma != 0) { + bus_dmamap_unload(isp->isp_osinfo.atiodmat, + isp->isp_osinfo.atiomap); + } + if (isp->isp_atioq != NULL) { + bus_dmamem_free(isp->isp_osinfo.reqdmat, isp->isp_atioq, + isp->isp_osinfo.atiomap); + bus_dma_tag_destroy(isp->isp_osinfo.atiodmat); + } + } #endif + free(isp->isp_xflist, M_DEVBUF); free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); isp->isp_rquest = NULL; ISP_LOCK(isp); diff --git a/sys/dev/isp/isp_sbus.c b/sys/dev/isp/isp_sbus.c index 2abfc64..f1ca83c 100644 --- a/sys/dev/isp/isp_sbus.c +++ b/sys/dev/isp/isp_sbus.c @@ -413,7 +413,7 @@ isp_sbus_wr_reg(ispsoftc_t *isp, int regoff, uint32_t val) } struct imush { - ispsoftc_t *isp; + bus_addr_t maddr; int error; }; @@ -423,16 +423,9 @@ static void imc(void *arg, bus_dma_segment_t *segs, int nseg, int error) { struct imush *imushp = (struct imush *) arg; - if (error) { - imushp->error = error; - } else { - ispsoftc_t *isp =imushp->isp; - bus_addr_t addr = segs->ds_addr; - isp->isp_rquest_dma = addr; - addr += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); - isp->isp_result_dma = addr; - } + if (!(imushp->error = error)) + imushp->maddr = segs[0].ds_addr; } static int @@ -479,40 +472,62 @@ isp_sbus_mbxdma(ispsoftc_t *isp) BUS_SPACE_MAXADDR_32BIT, NULL, NULL, BUS_SPACE_MAXSIZE_32BIT, ISP_NSEG_MAX, BUS_SPACE_MAXADDR_24BIT, 0, &isp->isp_osinfo.dmat)) { isp_prt(isp, ISP_LOGERR, "could not create master dma tag"); - free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); - free(isp->isp_xflist, M_DEVBUF); - ISP_LOCK(isp); - return(1); + goto bad; } /* - * Allocate and map the request, result queues, plus FC scratch area. + * Allocate and map the request queue. */ len = ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); - len += ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); - - if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, - BUS_SPACE_MAXADDR_24BIT+1, BUS_SPACE_MAXADDR_32BIT, - BUS_SPACE_MAXADDR_32BIT, NULL, NULL, len, 1, - BUS_SPACE_MAXADDR_24BIT, 0, &isp->isp_osinfo.cdmat)) { - isp_prt(isp, ISP_LOGERR, - "cannot create a dma tag for control spaces"); - free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); - free(isp->isp_xflist, M_DEVBUF); - ISP_LOCK(isp); - return (1); + if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, BUS_SPACE_MAXADDR_24BIT+1, + BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, + len, 1, len, 0, &isp->isp_osinfo.reqdmat)) { + isp_prt(isp, ISP_LOGERR, "cannot create request DMA tag"); + goto bad; + } + if (bus_dmamem_alloc(isp->isp_osinfo.reqdmat, (void **)&base, + BUS_DMA_COHERENT, &isp->isp_osinfo.reqmap) != 0) { + isp_prt(isp, ISP_LOGERR, "cannot allocate request DMA memory"); + bus_dma_tag_destroy(isp->isp_osinfo.reqdmat); + goto bad; + } + im.error = 0; + if (bus_dmamap_load(isp->isp_osinfo.reqdmat, isp->isp_osinfo.reqmap, + base, len, imc, &im, 0) || im.error) { + isp_prt(isp, ISP_LOGERR, "error loading request DMA map %d", im.error); + goto bad; } + isp_prt(isp, ISP_LOGDEBUG0, "request area @ 0x%jx/0x%jx", + (uintmax_t)im.maddr, (uintmax_t)len); + isp->isp_rquest = base; + isp->isp_rquest_dma = im.maddr; - if (bus_dmamem_alloc(isp->isp_osinfo.cdmat, (void **)&base, BUS_DMA_NOWAIT | BUS_DMA_COHERENT, - &isp->isp_osinfo.cdmap) != 0) { - isp_prt(isp, ISP_LOGERR, - "cannot allocate %d bytes of CCB memory", len); - bus_dma_tag_destroy(isp->isp_osinfo.cdmat); - free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); - free(isp->isp_xflist, M_DEVBUF); - ISP_LOCK(isp); - return (1); + /* + * Allocate and map the result queue. + */ + len = ISP_QUEUE_SIZE(RESULT_QUEUE_LEN(isp)); + if (isp_dma_tag_create(isp->isp_osinfo.dmat, QENTRY_LEN, BUS_SPACE_MAXADDR_24BIT+1, + BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, + len, 1, len, 0, &isp->isp_osinfo.respdmat)) { + isp_prt(isp, ISP_LOGERR, "cannot create response DMA tag"); + goto bad; } + if (bus_dmamem_alloc(isp->isp_osinfo.respdmat, (void **)&base, + BUS_DMA_COHERENT, &isp->isp_osinfo.respmap) != 0) { + isp_prt(isp, ISP_LOGERR, "cannot allocate response DMA memory"); + bus_dma_tag_destroy(isp->isp_osinfo.respdmat); + goto bad; + } + im.error = 0; + if (bus_dmamap_load(isp->isp_osinfo.respdmat, isp->isp_osinfo.respmap, + base, len, imc, &im, 0) || im.error) { + isp_prt(isp, ISP_LOGERR, "error loading response DMA map %d", im.error); + goto bad; + } + isp_prt(isp, ISP_LOGDEBUG0, "response area @ 0x%jx/0x%jx", + (uintmax_t)im.maddr, (uintmax_t)len); + isp->isp_result = base; + isp->isp_result_dma = im.maddr; for (i = 0; i < isp->isp_maxcmds; i++) { struct isp_pcmd *pcmd = &isp->isp_osinfo.pcmd_pool[i]; @@ -534,25 +549,28 @@ isp_sbus_mbxdma(ispsoftc_t *isp) } } isp->isp_osinfo.pcmd_free = &isp->isp_osinfo.pcmd_pool[0]; - - im.isp = isp; - im.error = 0; - bus_dmamap_load(isp->isp_osinfo.cdmat, isp->isp_osinfo.cdmap, base, len, imc, &im, 0); - if (im.error) { - isp_prt(isp, ISP_LOGERR, - "error %d loading dma map for control areas", im.error); - goto bad; - } - - isp->isp_rquest = base; - base += ISP_QUEUE_SIZE(RQUEST_QUEUE_LEN(isp)); - isp->isp_result = base; ISP_LOCK(isp); return (0); bad: - bus_dmamem_free(isp->isp_osinfo.cdmat, base, isp->isp_osinfo.cdmap); - bus_dma_tag_destroy(isp->isp_osinfo.cdmat); + if (isp->isp_rquest_dma != 0) { + bus_dmamap_unload(isp->isp_osinfo.reqdmat, + isp->isp_osinfo.reqmap); + } + if (isp->isp_rquest != NULL) { + bus_dmamem_free(isp->isp_osinfo.reqdmat, isp->isp_rquest, + isp->isp_osinfo.reqmap); + bus_dma_tag_destroy(isp->isp_osinfo.reqdmat); + } + if (isp->isp_result_dma != 0) { + bus_dmamap_unload(isp->isp_osinfo.respdmat, + isp->isp_osinfo.respmap); + } + if (isp->isp_result != NULL) { + bus_dmamem_free(isp->isp_osinfo.respdmat, isp->isp_result, + isp->isp_osinfo.respmap); + bus_dma_tag_destroy(isp->isp_osinfo.respdmat); + } free(isp->isp_xflist, M_DEVBUF); free(isp->isp_osinfo.pcmd_pool, M_DEVBUF); isp->isp_rquest = NULL; diff --git a/sys/dev/isp/isp_target.c b/sys/dev/isp/isp_target.c index c6e48fe..c6af888 100644 --- a/sys/dev/isp/isp_target.c +++ b/sys/dev/isp/isp_target.c @@ -1094,7 +1094,7 @@ isp_handle_ctio2(ispsoftc_t *isp, ct2_entry_t *ct) char *fmsg = NULL; if (ct->ct_syshandle) { - xs = isp_find_xs_tgt(isp, ct->ct_syshandle); + xs = isp_find_xs(isp, ct->ct_syshandle); if (xs == NULL) { pl = ISP_LOGALL; } @@ -1249,7 +1249,7 @@ isp_handle_ctio7(ispsoftc_t *isp, ct7_entry_t *ct) char *fmsg = NULL; if (ct->ct_syshandle) { - xs = isp_find_xs_tgt(isp, ct->ct_syshandle); + xs = isp_find_xs(isp, ct->ct_syshandle); if (xs == NULL) { pl = ISP_LOGALL; } diff --git a/sys/dev/isp/ispmbox.h b/sys/dev/isp/ispmbox.h index a89228b..5a1306f 100644 --- a/sys/dev/isp/ispmbox.h +++ b/sys/dev/isp/ispmbox.h @@ -1086,7 +1086,7 @@ typedef struct { #define ICB_DFLT_RDELAY 5 #define ICB_DFLT_RCOUNT 3 -#define ICB_LOGIN_TOV 30 +#define ICB_LOGIN_TOV 10 #define ICB_LUN_ENABLE_TOV 15 diff --git a/sys/dev/isp/ispvar.h b/sys/dev/isp/ispvar.h index 0c42642..a7184e2 100644 --- a/sys/dev/isp/ispvar.h +++ b/sys/dev/isp/ispvar.h @@ -77,7 +77,7 @@ struct ispmdvec { */ #define MAX_TARGETS 16 #ifndef MAX_FC_TARG -#define MAX_FC_TARG 256 +#define MAX_FC_TARG 1024 #endif #define ISP_MAX_TARGETS(isp) (IS_FC(isp)? MAX_FC_TARG : MAX_TARGETS) #define ISP_MAX_LUNS(isp) (isp)->isp_maxluns @@ -315,21 +315,16 @@ typedef struct { # define ISP_HANDLE_NONE 0 # define ISP_HANDLE_INITIATOR 1 # define ISP_HANDLE_TARGET 2 +# define ISP_HANDLE_CTRL 3 #define ISP_HANDLE_SEQ_MASK 0xffff0000 #define ISP_HANDLE_SEQ_SHIFT 16 #define ISP_H2SEQ(hdl) ((hdl & ISP_HANDLE_SEQ_MASK) >> ISP_HANDLE_SEQ_SHIFT) -#define ISP_VALID_INI_HANDLE(c, hdl) \ - (ISP_H2HT(hdl) == ISP_HANDLE_INITIATOR && (hdl & ISP_HANDLE_CMD_MASK) < (c)->isp_maxcmds && \ - ISP_H2SEQ(hdl) == ISP_H2SEQ((c)->isp_xflist[hdl & ISP_HANDLE_CMD_MASK].handle)) -#ifdef ISP_TARGET_MODE -#define ISP_VALID_TGT_HANDLE(c, hdl) \ - (ISP_H2HT(hdl) == ISP_HANDLE_TARGET && (hdl & ISP_HANDLE_CMD_MASK) < (c)->isp_maxcmds && \ - ISP_H2SEQ(hdl) == ISP_H2SEQ((c)->isp_tgtlist[hdl & ISP_HANDLE_CMD_MASK].handle)) #define ISP_VALID_HANDLE(c, hdl) \ - (ISP_VALID_INI_HANDLE((c), hdl) || ISP_VALID_TGT_HANDLE((c), hdl)) -#else -#define ISP_VALID_HANDLE ISP_VALID_INI_HANDLE -#endif + ((ISP_H2HT(hdl) == ISP_HANDLE_INITIATOR || \ + ISP_H2HT(hdl) == ISP_HANDLE_TARGET || \ + ISP_H2HT(hdl) == ISP_HANDLE_CTRL) && \ + ((hdl) & ISP_HANDLE_CMD_MASK) < (c)->isp_maxcmds && \ + (hdl) == ((c)->isp_xflist[(hdl) & ISP_HANDLE_CMD_MASK].handle)) #define ISP_BAD_HANDLE_INDEX 0xffffffff @@ -477,6 +472,8 @@ typedef struct { */ void * isp_scratch; XS_DMA_ADDR_T isp_scdma; + + uint8_t isp_scanscratch[ISP_FC_SCRLEN]; } fcparam; #define FW_CONFIG_WAIT 0 @@ -598,14 +595,6 @@ struct ispsoftc { isp_hdl_t *isp_xflist; isp_hdl_t *isp_xffree; -#ifdef ISP_TARGET_MODE - /* - * Active target commands are stored here, indexed by handle functions. - */ - isp_hdl_t *isp_tgtlist; - isp_hdl_t *isp_tgtfree; -#endif - /* * request/result queue pointers and DMA handles for them. */ diff --git a/sys/dev/ixgbe/README b/sys/dev/ixgbe/README deleted file mode 100644 index 8a1c358..0000000 --- a/sys/dev/ixgbe/README +++ /dev/null @@ -1,319 +0,0 @@ -FreeBSD Driver for Intel(R) Ethernet 10 Gigabit PCI Express Server Adapters -============================================================================ -/*$FreeBSD$*/ - -Jun 18, 2013 - - -Contents -======== - -- Overview -- Supported Adapters -- Building and Installation -- Additional Configurations and Tuning -- Known Limitations - - -Overview -======== - -This file describes the FreeBSD* driver for the -Intel(R) Ethernet 10 Gigabit Family of Adapters. - -For questions related to hardware requirements, refer to the documentation -supplied with your Intel 10GbE adapter. All hardware requirements listed -apply to use with FreeBSD. - - -Supported Adapters -================== - -The driver in this release is compatible with 82598 and 82599-based Intel -Network Connections. - -SFP+ Devices with Pluggable Optics ----------------------------------- - -82599-BASED ADAPTERS - -NOTE: If your 82599-based Intel(R) Ethernet Network Adapter came with Intel -optics, or is an Intel(R) Ethernet Server Adapter X520-2, then it only supports -Intel optics and/or the direct attach cables listed below. - -When 82599-based SFP+ devices are connected back to back, they should be set to -the same Speed setting. Results may vary if you mix speed settings. - -Supplier Type Part Numbers - -SR Modules -Intel DUAL RATE 1G/10G SFP+ SR (bailed) FTLX8571D3BCV-IT -Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDZ-IN2 -Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDDZ-IN1 -LR Modules -Intel DUAL RATE 1G/10G SFP+ LR (bailed) FTLX1471D3BCV-IT -Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDZ-IN2 -Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDDZ-IN1 - -The following is a list of 3rd party SFP+ modules and direct attach cables that -have received some testing. Not all modules are applicable to all devices. - -Supplier Type Part Numbers - -Finisar SFP+ SR bailed, 10g single rate FTLX8571D3BCL -Avago SFP+ SR bailed, 10g single rate AFBR-700SDZ -Finisar SFP+ LR bailed, 10g single rate FTLX8571D3BCV-IT - -Finisar DUAL RATE 1G/10G SFP+ SR (No Bail) FTLX8571D3QCV-IT -Avago DUAL RATE 1G/10G SFP+ SR (No Bail) AFBR-703SDZ-IN1 -Finisar DUAL RATE 1G/10G SFP+ LR (No Bail) FTLX1471D3QCV-IT -Avago DUAL RATE 1G/10G SFP+ LR (No Bail) AFCT-701SDZ-IN1 -Finistar 1000BASE-T SFP FCLF8522P2BTL -Avago 1000BASE-T SFP ABCU-5710RZ - -NOTE: As of driver version 2.5.13 it is possible to allow the operation -of unsupported modules by setting the static variable 'allow_unsupported_sfp' -to TRUE and rebuilding the driver. If problems occur please assure that they -can be reproduced with fully supported optics first. - -82599-based adapters support all passive and active limiting direct attach -cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. - -Laser turns off for SFP+ when ifconfig down --------------------------------------------------------- -"ifconfig down" turns off the laser for 82599-based SFP+ fiber adapters. -"ifconfig up" turns on the later. - -82598-BASED ADAPTERS - -NOTES for 82598-Based Adapters: -- Intel(R) Ethernet Network Adapters that support removable optical modules - only support their original module type (i.e., the Intel(R) 10 Gigabit SR - Dual Port Express Module only supports SR optical modules). If you plug - in a different type of module, the driver will not load. -- Hot Swapping/hot plugging optical modules is not supported. -- Only single speed, 10 gigabit modules are supported. -- LAN on Motherboard (LOMs) may support DA, SR, or LR modules. Other module - types are not supported. Please see your system documentation for details. - -The following is a list of 3rd party SFP+ modules and direct attach cables that have -received some testing. Not all modules are applicable to all devices. - -Supplier Type Part Numbers - -Finisar SFP+ SR bailed, 10g single rate FTLX8571D3BCL -Avago SFP+ SR bailed, 10g single rate AFBR-700SDZ -Finisar SFP+ LR bailed, 10g single rate FTLX1471D3BCL - -82598-based adapters support all passive direct attach cables that comply -with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct attach -cables are not supported. - -Third party optic modules and cables referred to above are listed only for the -purpose of highlighting third party specifications and potential compatibility, -and are not recommendations or endorsements or sponsorship of any third party's -product by Intel. Intel is not endorsing or promoting products made by any -third party and the third party reference is provided only to share information -regarding certain optic modules and cables with the above specifications. There -may be other manufacturers or suppliers, producing or supplying optic modules -and cables with similar or matching descriptions. Customers must use their own -discretion and diligence to purchase optic modules and cables from any third -party of their choice. Customer are solely responsible for assessing the -suitability of the product and/or devices and for the selection of the vendor -for purchasing any product. INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL -DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF -SUCH THIRD PARTY PRODUCTS OR SELECTION OF VENDOR BY CUSTOMERS. - -Configuration and Tuning -======================== - -The driver supports Transmit/Receive Checksum Offload and Jumbo Frames on -all 10 Gigabit adapters. - - Jumbo Frames - ------------ - To enable Jumbo Frames, use the ifconfig utility to increase the MTU - beyond 1500 bytes. - - NOTES: - - - The Jumbo Frames setting on the switch must be set to at least - 22 bytes larger than that of the adapter. - - - There are known performance issues with this driver when running - UDP traffic with Jumbo Frames. - - The Jumbo Frames MTU range for Intel Adapters is 1500 to 16114. The default - MTU range is 1500. To modify the setting, enter the following: - - ifconfig ix<interface_num> <hostname or IP address> mtu 9000 - - To confirm an interface's MTU value, use the ifconfig command. To confirm - the MTU used between two specific devices, use: - - route get <destination_IP_address> - - VLANs - ----- - To create a new VLAN pseudo-interface: - - ifconfig <vlan_name> create - - To associate the VLAN pseudo-interface with a physical interface and - assign a VLAN ID, IP address, and netmask: - - ifconfig <vlan_name> <ip_address> netmask <subnet_mask> vlan - <vlan_id> vlandev <physical_interface> - - Example: - - ifconfig vlan10 10.0.0.1 netmask 255.255.255.0 vlan 10 vlandev ixgbe0 - - In this example, all packets will be marked on egress with 802.1Q VLAN - tags, specifying a VLAN ID of 10. - - To remove a VLAN pseudo-interface: - - ifconfig <vlan_name> destroy - - - Checksum Offload - ---------------- - - Checksum offloading supports both TCP and UDP packets and is - supported for both transmit and receive. - - Checksum offloading can be enabled or disabled using ifconfig. - Both transmit and receive offloading will be either enabled or - disabled together. You cannot enable/disable one without the other. - - To enable checksum offloading: - - ifconfig <interface_num> rxcsum - - To disable checksum offloading: - - ifconfig <interface_num> -rxcsum - - To confirm the current setting: - - ifconfig <interface_num> - - - TSO - --- - - TSO is enabled by default. - - To disable: - - ifconfig <interface_num> -tso - - To re-enable: - - ifconfig <interface_num> tso - - LRO - --- - - Large Receive Offload is available in the driver; it is on by default. - It can be disabled by using: - ifconfig <interface_num> -lro - To enable: - ifconfig <interface_num> lro - - - Important system configuration changes: - --------------------------------------- - - When there is a choice run on a 64bit OS rather than 32, it makes a - significant difference in improvement. - - The interface can generate a high number of interrupts. To avoid running - into the limit set by the kernel, adjust hw.intr_storm_threshold - setting using sysctl: - - sysctl hw.intr_storm_threshold=9000 (the default is 1000) - - For this change to take effect on boot, edit /etc/sysctl.conf and add the - line: - hw.intr_storm_threshold=9000 - - If you still see Interrupt Storm detected messages, increase the limit to a - higher number, or the detection can be disabled by setting it to 0. - - The default number of descriptors is 2048, increasing or descreasing - may improve performance in some workloads, but change carefully. - - -Known Limitations -================= - -For known hardware and troubleshooting issues, refer to the following website. - - http://support.intel.com/support/go/network/adapter/home.htm - -Either select the link for your adapter or perform a search for the adapter -number. The adapter's page lists many issues. For a complete list of hardware -issues download your adapter's user guide and read the Release Notes. - - UDP stress test with 10GbE driver - --------------------------------- - Under small packets UDP stress test with 10GbE driver, the FreeBSD system - will drop UDP packets due to the fullness of socket buffers. You may want - to change the driver's Flow Control variables to the minimum value for - controlling packet reception. - - Attempting to configure larger MTUs with a large numbers of processors may - generate the error message "ix0:could not setup receive structures" - -------------------------------------------------------------------------- - When using the ixgbe driver with RSS autoconfigured based on the number of - cores (the default setting) and that number is larger than 4, increase the - memory resources allocated for the mbuf pool as follows: - - Add to the sysctl.conf file for the system: - - kern.ipc.nmbclusters=262144 - kern.ipc.nmbjumbop=262144 - - Lower than expected performance on dual port 10GbE devices - ---------------------------------------------------------- - Some PCI-E x8 slots are actually configured as x4 slots. These slots have - insufficient bandwidth for full 10Gbe line rate with dual port 10GbE devices. - The driver will detect this situation and will write the following message in - the system log: "PCI-Express bandwidth available for this card is not - sufficient for optimal performance. For optimal performance a x8 PCI-Express - slot is required." - - If this error occurs, moving your adapter to a true x8 slot will resolve the - issue. - - - -Support -======= - -For general information and support, go to the Intel support website at: - - www.intel.com/support/ - -If an issue is identified with the released source code on the supported -kernel with a supported adapter, email the specific information related to -the issue to freebsd@intel.com - - - -License -======= - -This software program is released under the terms of a license agreement -between you ('Licensee') and Intel. Do not use or load this software or any -associated materials (collectively, the 'Software') until you have carefully -read the full terms and conditions of the LICENSE located in this software -package. By loading or using the Software, you agree to the terms of this -Agreement. If you do not agree with the terms of this Agreement, do not -install or use the Software. - -* Other names and brands may be claimed as the property of others. - - diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index edde736..c7ac593 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -47,14 +47,10 @@ #endif /********************************************************************* - * Set this to one to display debug statistics - *********************************************************************/ -int ixgbe_display_debug_stats = 0; - -/********************************************************************* * Driver version *********************************************************************/ -char ixgbe_driver_version[] = "3.1.0"; +char ixgbe_driver_version[] = "3.1.13-k"; + /********************************************************************* * PCI Device ID Table @@ -95,9 +91,11 @@ static ixgbe_vendor_info_t ixgbe_vendor_info_array[] = {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X540T, 0, 0, 0}, {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X540T1, 0, 0, 0}, {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X550T, 0, 0, 0}, + {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X550T1, 0, 0, 0}, {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X550EM_X_KR, 0, 0, 0}, {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X550EM_X_KX4, 0, 0, 0}, {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X550EM_X_10G_T, 0, 0, 0}, + {IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X550EM_X_SFP, 0, 0, 0}, /* required last entry */ {0, 0, 0, 0, 0} }; @@ -131,7 +129,7 @@ static void ixgbe_media_status(struct ifnet *, struct ifmediareq *); static int ixgbe_media_change(struct ifnet *); static void ixgbe_identify_hardware(struct adapter *); static int ixgbe_allocate_pci_resources(struct adapter *); -static void ixgbe_get_slot_info(struct ixgbe_hw *); +static void ixgbe_get_slot_info(struct adapter *); static int ixgbe_allocate_msix(struct adapter *); static int ixgbe_allocate_legacy(struct adapter *); static int ixgbe_setup_msix(struct adapter *); @@ -142,7 +140,6 @@ static void ixgbe_config_gpie(struct adapter *); static void ixgbe_config_dmac(struct adapter *); static void ixgbe_config_delay_values(struct adapter *); static void ixgbe_config_link(struct adapter *); -static void ixgbe_check_eee_support(struct adapter *); static void ixgbe_check_wol_support(struct adapter *); static int ixgbe_setup_low_power_mode(struct adapter *); static void ixgbe_rearm_queues(struct adapter *, u64); @@ -151,6 +148,7 @@ static void ixgbe_initialize_transmit_units(struct adapter *); static void ixgbe_initialize_receive_units(struct adapter *); static void ixgbe_enable_rx_drop(struct adapter *); static void ixgbe_disable_rx_drop(struct adapter *); +static void ixgbe_initialize_rss_mapping(struct adapter *); static void ixgbe_enable_intr(struct adapter *); static void ixgbe_disable_intr(struct adapter *); @@ -171,19 +169,24 @@ static void ixgbe_add_hw_stats(struct adapter *); /* Sysctl handlers */ static void ixgbe_set_sysctl_value(struct adapter *, const char *, - const char *, int *, int); + const char *, int *, int); static int ixgbe_set_flowcntl(SYSCTL_HANDLER_ARGS); static int ixgbe_set_advertise(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_thermal_test(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_dmac(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_phy_temp(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_phy_overtemp_occurred(SYSCTL_HANDLER_ARGS); +#ifdef IXGBE_DEBUG +static int ixgbe_sysctl_power_state(SYSCTL_HANDLER_ARGS); +static int ixgbe_sysctl_print_rss_config(SYSCTL_HANDLER_ARGS); +#endif static int ixgbe_sysctl_wol_enable(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_wufc(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_eee_enable(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_eee_negotiated(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_eee_rx_lpi_status(SYSCTL_HANDLER_ARGS); static int ixgbe_sysctl_eee_tx_lpi_status(SYSCTL_HANDLER_ARGS); +static int ixgbe_sysctl_eee_tx_lpi_delay(SYSCTL_HANDLER_ARGS); /* Support for pluggable optic modules */ static bool ixgbe_sfp_probe(struct adapter *); @@ -439,7 +442,7 @@ ixgbe_attach(device_t dev) /* Allocate, clear, and link in our adapter structure */ adapter = device_get_softc(dev); - adapter->dev = adapter->osdep.dev = dev; + adapter->dev = dev; hw = &adapter->hw; #ifdef DEV_NETMAP @@ -529,18 +532,18 @@ ixgbe_attach(device_t dev) adapter->sfp_probe = TRUE; error = 0; } else if (error == IXGBE_ERR_SFP_NOT_SUPPORTED) { - device_printf(dev,"Unsupported SFP+ module detected!\n"); + device_printf(dev, "Unsupported SFP+ module detected!\n"); error = EIO; goto err_late; } else if (error) { - device_printf(dev,"Unable to initialize the shared code\n"); + device_printf(dev, "Unable to initialize the shared code\n"); error = EIO; goto err_late; } /* Make sure we have a good EEPROM before we read from it */ if (ixgbe_validate_eeprom_checksum(&adapter->hw, &csum) < 0) { - device_printf(dev,"The EEPROM Checksum Is Not Valid\n"); + device_printf(dev, "The EEPROM Checksum Is Not Valid\n"); error = EIO; goto err_late; } @@ -550,24 +553,21 @@ ixgbe_attach(device_t dev) case IXGBE_ERR_EEPROM_VERSION: device_printf(dev, "This device is a pre-production adapter/" "LOM. Please be aware there may be issues associated " - "with your hardware.\n If you are experiencing problems " + "with your hardware.\nIf you are experiencing problems " "please contact your Intel or hardware representative " "who provided you with this hardware.\n"); break; case IXGBE_ERR_SFP_NOT_SUPPORTED: - device_printf(dev,"Unsupported SFP+ Module\n"); + device_printf(dev, "Unsupported SFP+ Module\n"); error = EIO; goto err_late; case IXGBE_ERR_SFP_NOT_PRESENT: - device_printf(dev,"No SFP+ Module found\n"); + device_printf(dev, "No SFP+ Module found\n"); /* falls thru */ default: break; } - /* Detect and set physical type */ - ixgbe_setup_optics(adapter); - if ((adapter->msix > 1) && (ixgbe_enable_msix)) error = ixgbe_allocate_msix(adapter); else @@ -589,11 +589,12 @@ ixgbe_attach(device_t dev) ixgbe_unregister_vlan, adapter, EVENTHANDLER_PRI_FIRST); /* Check PCIE slot type/speed/width */ - ixgbe_get_slot_info(hw); + ixgbe_get_slot_info(adapter); - - /* Set an initial default flow control value */ + /* Set an initial default flow control & dmac value */ adapter->fc = ixgbe_fc_full; + adapter->dmac = 0; + adapter->eee_enabled = 0; #ifdef PCI_IOV if ((hw->mac.type != ixgbe_mac_82598EB) && (adapter->msix > 1)) { @@ -619,7 +620,6 @@ ixgbe_attach(device_t dev) /* Check for certain supported features */ ixgbe_check_wol_support(adapter); - ixgbe_check_eee_support(adapter); /* Add sysctls */ ixgbe_add_device_sysctls(adapter); @@ -680,6 +680,7 @@ ixgbe_detach(device_t dev) } #endif /* PCI_IOV */ + ether_ifdetach(adapter->ifp); /* Stop the adapter */ IXGBE_CORE_LOCK(adapter); ixgbe_setup_low_power_mode(adapter); @@ -721,7 +722,6 @@ ixgbe_detach(device_t dev) if (adapter->vlan_detach != NULL) EVENTHANDLER_DEREGISTER(vlan_unconfig, adapter->vlan_detach); - ether_ifdetach(adapter->ifp); callout_drain(&adapter->timer); #ifdef DEV_NETMAP netmap_detach(adapter->ifp); @@ -776,10 +776,6 @@ ixgbe_suspend(device_t dev) error = ixgbe_setup_low_power_mode(adapter); - /* Save state and power down */ - pci_save_state(dev); - pci_set_powerstate(dev, PCI_POWERSTATE_D3); - IXGBE_CORE_UNLOCK(adapter); return (error); @@ -797,9 +793,6 @@ ixgbe_resume(device_t dev) IXGBE_CORE_LOCK(adapter); - pci_set_powerstate(dev, PCI_POWERSTATE_D0); - pci_restore_state(dev); - /* Read & clear WUS register */ wus = IXGBE_READ_REG(hw, IXGBE_WUS); if (wus) @@ -818,7 +811,6 @@ ixgbe_resume(device_t dev) IXGBE_CORE_UNLOCK(adapter); - INIT_DEBUGOUT("ixgbe_resume: end"); return (0); } @@ -862,7 +854,7 @@ ixgbe_ioctl(struct ifnet * ifp, u_long command, caddr_t data) ifp->if_flags |= IFF_UP; if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) ixgbe_init(adapter); -#if defined(INET) +#ifdef INET if (!(ifp->if_flags & IFF_NOARP)) arp_ifinit(ifp, ifa); #endif @@ -920,10 +912,21 @@ ixgbe_ioctl(struct ifnet * ifp, u_long command, caddr_t data) break; case SIOCSIFCAP: { - int mask = ifr->ifr_reqcap ^ ifp->if_capenable; IOCTL_DEBUGOUT("ioctl: SIOCSIFCAP (Set Capabilities)"); - if (mask & IFCAP_HWCSUM) - ifp->if_capenable ^= IFCAP_HWCSUM; + + int mask = ifr->ifr_reqcap ^ ifp->if_capenable; + if (!mask) + break; + + /* HW cannot turn these on/off separately */ + if (mask & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6)) { + ifp->if_capenable ^= IFCAP_RXCSUM; + ifp->if_capenable ^= IFCAP_RXCSUM_IPV6; + } + if (mask & IFCAP_TXCSUM) + ifp->if_capenable ^= IFCAP_TXCSUM; + if (mask & IFCAP_TXCSUM_IPV6) + ifp->if_capenable ^= IFCAP_TXCSUM_IPV6; if (mask & IFCAP_TSO4) ifp->if_capenable ^= IFCAP_TSO4; if (mask & IFCAP_TSO6) @@ -936,6 +939,7 @@ ixgbe_ioctl(struct ifnet * ifp, u_long command, caddr_t data) ifp->if_capenable ^= IFCAP_VLAN_HWFILTER; if (mask & IFCAP_VLAN_HWTSO) ifp->if_capenable ^= IFCAP_VLAN_HWTSO; + if (ifp->if_drv_flags & IFF_DRV_RUNNING) { IXGBE_CORE_LOCK(adapter); ixgbe_init_locked(adapter); @@ -979,6 +983,42 @@ ixgbe_ioctl(struct ifnet * ifp, u_long command, caddr_t data) return (error); } +/* + * Set the various hardware offload abilities. + * + * This takes the ifnet's if_capenable flags (e.g. set by the user using + * ifconfig) and indicates to the OS via the ifnet's if_hwassist field what + * mbuf offload flags the driver will understand. + */ +static void +ixgbe_set_if_hwassist(struct adapter *adapter) +{ + struct ifnet *ifp = adapter->ifp; + + ifp->if_hwassist = 0; +#if __FreeBSD_version >= 1000000 + if (ifp->if_capenable & IFCAP_TSO4) + ifp->if_hwassist |= CSUM_IP_TSO; + if (ifp->if_capenable & IFCAP_TSO6) + ifp->if_hwassist |= CSUM_IP6_TSO; + if (ifp->if_capenable & IFCAP_TXCSUM) + ifp->if_hwassist |= (CSUM_IP | CSUM_IP_UDP | CSUM_IP_TCP | + CSUM_IP_SCTP); + if (ifp->if_capenable & IFCAP_TXCSUM_IPV6) + ifp->if_hwassist |= (CSUM_IP6_UDP | CSUM_IP6_TCP | + CSUM_IP6_SCTP); +#else + if (ifp->if_capenable & IFCAP_TSO) + ifp->if_hwassist |= CSUM_TSO; + if (ifp->if_capenable & IFCAP_TXCSUM) { + ifp->if_hwassist |= (CSUM_TCP | CSUM_UDP); + struct ixgbe_hw *hw = &adapter->hw; + if (hw->mac.type != ixgbe_mac_82598EB) + ifp->if_hwassist |= CSUM_SCTP; + } +#endif +} + /********************************************************************* * Init entry point * @@ -1001,6 +1041,7 @@ ixgbe_init_locked(struct adapter *adapter) struct rx_ring *rxr; u32 txdctl, mhadd; u32 rxdctl, rxctrl; + int err = 0; #ifdef PCI_IOV enum ixgbe_iov_mode mode; #endif @@ -1029,17 +1070,8 @@ ixgbe_init_locked(struct adapter *adapter) ixgbe_set_rar(hw, 0, hw->mac.addr, adapter->pool, 1); hw->addr_ctrl.rar_used_count = 1; - /* Set the various hardware offload abilities */ - ifp->if_hwassist = 0; - if (ifp->if_capenable & IFCAP_TSO) - ifp->if_hwassist |= CSUM_TSO; - if (ifp->if_capenable & IFCAP_TXCSUM) { - ifp->if_hwassist |= (CSUM_TCP | CSUM_UDP); -#if __FreeBSD_version >= 800000 - if (hw->mac.type != ixgbe_mac_82598EB) - ifp->if_hwassist |= CSUM_SCTP; -#endif - } + /* Set hardware offload abilities from ifnet flags */ + ixgbe_set_if_hwassist(adapter); /* Prepare transmit descriptors and buffers */ if (ixgbe_setup_transmit_structures(adapter)) { @@ -1057,10 +1089,7 @@ ixgbe_init_locked(struct adapter *adapter) /* Setup Multicast table */ ixgbe_set_multi(adapter); - /* - ** Determine the correct mbuf pool - ** for doing jumbo frames - */ + /* Determine the correct mbuf pool, based on frame size */ if (adapter->max_frame_size <= MCLBYTES) adapter->rx_mbuf_sz = MCLBYTES; else @@ -1196,7 +1225,7 @@ ixgbe_init_locked(struct adapter *adapter) * need to be kick-started */ if (hw->phy.type == ixgbe_phy_none) { - int err = hw->phy.ops.identify(hw); + err = hw->phy.ops.identify(hw); if (err == IXGBE_ERR_SFP_NOT_SUPPORTED) { device_printf(dev, "Unsupported SFP+ module type was detected.\n"); @@ -1208,7 +1237,11 @@ ixgbe_init_locked(struct adapter *adapter) IXGBE_WRITE_REG(hw, IXGBE_EITR(adapter->vector), IXGBE_LINK_ITR); /* Configure Energy Efficient Ethernet for supported devices */ - ixgbe_setup_eee(hw, adapter->eee_enabled); + if (hw->mac.ops.setup_eee) { + err = hw->mac.ops.setup_eee(hw, adapter->eee_enabled); + if (err) + device_printf(dev, "Error setting up EEE: %d\n", err); + } /* Config/Enable Link */ ixgbe_config_link(adapter); @@ -1278,7 +1311,7 @@ ixgbe_config_gpie(struct adapter *adapter) /* * Thermal Failure Detection (X540) - * Link Detection (X557) + * Link Detection (X552 SFP+, X552/X557-AT) */ if (hw->mac.type == ixgbe_mac_X540 || hw->device_id == IXGBE_DEV_ID_X550EM_X_SFP || @@ -1586,6 +1619,9 @@ ixgbe_msix_link(void *arg) ++adapter->link_irq; + /* Pause other interrupts */ + IXGBE_WRITE_REG(hw, IXGBE_EIMC, IXGBE_EIMC_OTHER); + /* First get the cause */ reg_eicr = IXGBE_READ_REG(hw, IXGBE_EICS); /* Be sure the queue bits are not cleared */ @@ -1594,8 +1630,10 @@ ixgbe_msix_link(void *arg) IXGBE_WRITE_REG(hw, IXGBE_EICR, reg_eicr); /* Link status change */ - if (reg_eicr & IXGBE_EICR_LSC) + if (reg_eicr & IXGBE_EICR_LSC) { + IXGBE_WRITE_REG(hw, IXGBE_EIMC, IXGBE_EIMC_LSC); taskqueue_enqueue(adapter->tq, &adapter->link_task); + } if (adapter->hw.mac.type != ixgbe_mac_82598EB) { #ifdef IXGBE_FDIR @@ -1609,14 +1647,14 @@ ixgbe_msix_link(void *arg) } else #endif if (reg_eicr & IXGBE_EICR_ECC) { - device_printf(adapter->dev, "\nCRITICAL: ECC ERROR!! " + device_printf(adapter->dev, "CRITICAL: ECC ERROR!! " "Please Reboot!!\n"); IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_ECC); } /* Check for over temp condition */ if (reg_eicr & IXGBE_EICR_TS) { - device_printf(adapter->dev, "\nCRITICAL: OVER TEMP!! " + device_printf(adapter->dev, "CRITICAL: OVER TEMP!! " "PHY IS SHUT DOWN!!\n"); device_printf(adapter->dev, "System shutdown required!\n"); IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_TS); @@ -1658,6 +1696,7 @@ ixgbe_msix_link(void *arg) taskqueue_enqueue(adapter->tq, &adapter->phy_task); } + /* Re-enable other interrupts */ IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMS, IXGBE_EIMS_OTHER); return; } @@ -1751,6 +1790,7 @@ ixgbe_media_status(struct ifnet * ifp, struct ifmediareq * ifmr) ** XXX: These need to use the proper media types once ** they're added. */ +#ifndef IFM_ETH_XTYPE if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_KR) switch (adapter->link_speed) { case IXGBE_LINK_SPEED_10GB_FULL: @@ -1776,6 +1816,33 @@ ixgbe_media_status(struct ifnet * ifp, struct ifmediareq * ifmr) ifmr->ifm_active |= IFM_1000_CX | IFM_FDX; break; } +#else + if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_KR) + switch (adapter->link_speed) { + case IXGBE_LINK_SPEED_10GB_FULL: + ifmr->ifm_active |= IFM_10G_KR | IFM_FDX; + break; + case IXGBE_LINK_SPEED_2_5GB_FULL: + ifmr->ifm_active |= IFM_2500_KX | IFM_FDX; + break; + case IXGBE_LINK_SPEED_1GB_FULL: + ifmr->ifm_active |= IFM_1000_KX | IFM_FDX; + break; + } + else if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_KX4 + || layer & IXGBE_PHYSICAL_LAYER_1000BASE_KX) + switch (adapter->link_speed) { + case IXGBE_LINK_SPEED_10GB_FULL: + ifmr->ifm_active |= IFM_10G_KX4 | IFM_FDX; + break; + case IXGBE_LINK_SPEED_2_5GB_FULL: + ifmr->ifm_active |= IFM_2500_KX | IFM_FDX; + break; + case IXGBE_LINK_SPEED_1GB_FULL: + ifmr->ifm_active |= IFM_1000_KX | IFM_FDX; + break; + } +#endif /* If nothing is recognized... */ if (IFM_SUBTYPE(ifmr->ifm_active) == 0) @@ -1818,13 +1885,14 @@ ixgbe_media_change(struct ifnet * ifp) return (EINVAL); if (hw->phy.media_type == ixgbe_media_type_backplane) - return (EPERM); + return (ENODEV); /* ** We don't actually need to check against the supported ** media types of the adapter; ifmedia will take care of ** that for us. */ +#ifndef IFM_ETH_XTYPE switch (IFM_SUBTYPE(ifm->ifm_media)) { case IFM_AUTO: case IFM_10G_T: @@ -1850,6 +1918,33 @@ ixgbe_media_change(struct ifnet * ifp) default: goto invalid; } +#else + switch (IFM_SUBTYPE(ifm->ifm_media)) { + case IFM_AUTO: + case IFM_10G_T: + speed |= IXGBE_LINK_SPEED_100_FULL; + case IFM_10G_LRM: + case IFM_10G_KR: + case IFM_10G_LR: + case IFM_10G_KX4: + speed |= IXGBE_LINK_SPEED_1GB_FULL; + case IFM_10G_TWINAX: + speed |= IXGBE_LINK_SPEED_10GB_FULL; + break; + case IFM_1000_T: + speed |= IXGBE_LINK_SPEED_100_FULL; + case IFM_1000_LX: + case IFM_1000_SX: + case IFM_1000_KX: + speed |= IXGBE_LINK_SPEED_1GB_FULL; + break; + case IFM_100_TX: + speed |= IXGBE_LINK_SPEED_100_FULL; + break; + default: + goto invalid; + } +#endif hw->mac.autotry_restart = TRUE; hw->mac.ops.setup_link(hw, speed, TRUE); @@ -2384,7 +2479,7 @@ ixgbe_allocate_msix(struct adapter *adapter) return (error); } #if __FreeBSD_version >= 800504 - bus_describe_intr(dev, que->res, que->tag, "que %d", i); + bus_describe_intr(dev, que->res, que->tag, "q%d", i); #endif que->msix = vector; adapter->active_queues |= (u64)(1 << que->msix); @@ -2435,8 +2530,8 @@ ixgbe_allocate_msix(struct adapter *adapter) device_get_nameunit(adapter->dev), cpu_id); #else - taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que", - device_get_nameunit(adapter->dev)); + taskqueue_start_threads(&que->tq, 1, PI_NET, "%s:q%d", + device_get_nameunit(adapter->dev), i); #endif } @@ -2514,7 +2609,7 @@ ixgbe_setup_msix(struct adapter *adapter) } /* Figure out a reasonable auto config value */ - queues = (mp_ncpus > (msgs-1)) ? (msgs-1) : mp_ncpus; + queues = (mp_ncpus > (msgs - 1)) ? (msgs - 1) : mp_ncpus; #ifdef RSS /* If we're doing RSS, clamp at the number of RSS buckets */ @@ -2524,6 +2619,9 @@ ixgbe_setup_msix(struct adapter *adapter) if (ixgbe_num_queues != 0) queues = ixgbe_num_queues; + /* Set max queues to 8 when autoconfiguring */ + else if ((ixgbe_num_queues == 0) && (queues > 8)) + queues = 8; /* reflect correct sysctl value */ ixgbe_num_queues = queues; @@ -2560,12 +2658,12 @@ msi: rid, adapter->msix_mem); adapter->msix_mem = NULL; } - msgs = 1; - if (pci_alloc_msi(dev, &msgs) == 0) { - device_printf(adapter->dev,"Using an MSI interrupt\n"); + msgs = 1; + if (pci_alloc_msi(dev, &msgs) == 0) { + device_printf(adapter->dev, "Using an MSI interrupt\n"); return (msgs); } - device_printf(adapter->dev,"Using a Legacy interrupt\n"); + device_printf(adapter->dev, "Using a Legacy interrupt\n"); return (0); } @@ -2581,22 +2679,24 @@ ixgbe_allocate_pci_resources(struct adapter *adapter) &rid, RF_ACTIVE); if (!(adapter->pci_mem)) { - device_printf(dev,"Unable to allocate bus resource: memory\n"); + device_printf(dev, "Unable to allocate bus resource: memory\n"); return (ENXIO); } + /* Save bus_space values for READ/WRITE_REG macros */ adapter->osdep.mem_bus_space_tag = rman_get_bustag(adapter->pci_mem); adapter->osdep.mem_bus_space_handle = rman_get_bushandle(adapter->pci_mem); + /* Set hw values for shared code */ adapter->hw.hw_addr = (u8 *) &adapter->osdep.mem_bus_space_handle; + adapter->hw.back = adapter; - /* Legacy defaults */ + /* Default to 1 queue if MSI-X setup fails */ adapter->num_queues = 1; - adapter->hw.back = &adapter->osdep; /* - ** Now setup MSI or MSI/X, should + ** Now setup MSI or MSI-X, should ** return us the number of supported ** vectors. (Will be 1 for MSI) */ @@ -2721,13 +2821,22 @@ ixgbe_setup_interface(device_t dev, struct adapter *adapter) */ ifp->if_hdrlen = sizeof(struct ether_vlan_header); - ifp->if_capabilities |= IFCAP_HWCSUM | IFCAP_TSO | IFCAP_VLAN_HWCSUM; - ifp->if_capabilities |= IFCAP_JUMBO_MTU; - ifp->if_capabilities |= IFCAP_LRO; - ifp->if_capabilities |= IFCAP_VLAN_HWTAGGING + /* Set capability flags */ + ifp->if_capabilities |= IFCAP_RXCSUM + | IFCAP_TXCSUM + | IFCAP_RXCSUM_IPV6 + | IFCAP_TXCSUM_IPV6 + | IFCAP_TSO4 + | IFCAP_TSO6 + | IFCAP_LRO + | IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_HWTSO + | IFCAP_VLAN_HWCSUM + | IFCAP_JUMBO_MTU | IFCAP_VLAN_MTU | IFCAP_HWSTATS; + + /* Enable the above capabilities by default */ ifp->if_capenable = ifp->if_capabilities; /* @@ -2747,9 +2856,10 @@ ixgbe_setup_interface(device_t dev, struct adapter *adapter) ifmedia_init(&adapter->media, IFM_IMASK, ixgbe_media_change, ixgbe_media_status); + adapter->phy_layer = ixgbe_get_supported_physical_layer(&adapter->hw); ixgbe_add_media_types(adapter); - /* Autoselect media by default */ + /* Set autoselect media by default */ ifmedia_set(&adapter->media, IFM_ETHER | IFM_AUTO); return (0); @@ -2762,7 +2872,7 @@ ixgbe_add_media_types(struct adapter *adapter) device_t dev = adapter->dev; int layer; - layer = adapter->phy_layer = ixgbe_get_supported_physical_layer(hw); + layer = adapter->phy_layer; /* Media types with matching FreeBSD media defines */ if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_T) @@ -2776,20 +2886,28 @@ ixgbe_add_media_types(struct adapter *adapter) layer & IXGBE_PHYSICAL_LAYER_SFP_ACTIVE_DA) ifmedia_add(&adapter->media, IFM_ETHER | IFM_10G_TWINAX, 0, NULL); - if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_LR) + if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_LR) { ifmedia_add(&adapter->media, IFM_ETHER | IFM_10G_LR, 0, NULL); - if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_SR) + if (hw->phy.multispeed_fiber) + ifmedia_add(&adapter->media, IFM_ETHER | IFM_1000_LX, 0, NULL); + } + if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_SR) { ifmedia_add(&adapter->media, IFM_ETHER | IFM_10G_SR, 0, NULL); + if (hw->phy.multispeed_fiber) + ifmedia_add(&adapter->media, IFM_ETHER | IFM_1000_SX, 0, NULL); + } else if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_SX) + ifmedia_add(&adapter->media, IFM_ETHER | IFM_1000_SX, 0, NULL); if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_CX4) ifmedia_add(&adapter->media, IFM_ETHER | IFM_10G_CX4, 0, NULL); - if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_SX) - ifmedia_add(&adapter->media, IFM_ETHER | IFM_1000_SX, 0, NULL); - /* - ** Other (no matching FreeBSD media type): - ** To workaround this, we'll assign these completely - ** inappropriate media types. - */ +#ifdef IFM_ETH_XTYPE + if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_KR) + ifmedia_add(&adapter->media, IFM_ETHER | IFM_10G_KR, 0, NULL); + if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_KX4) + ifmedia_add(&adapter->media, IFM_ETHER | IFM_10G_KX4, 0, NULL); + if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_KX) + ifmedia_add(&adapter->media, IFM_ETHER | IFM_1000_KX, 0, NULL); +#else if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_KR) { device_printf(dev, "Media supported: 10GbaseKR\n"); device_printf(dev, "10GbaseKR mapped to 10GbaseSR\n"); @@ -2805,10 +2923,9 @@ ixgbe_add_media_types(struct adapter *adapter) device_printf(dev, "1000baseKX mapped to 1000baseCX\n"); ifmedia_add(&adapter->media, IFM_ETHER | IFM_1000_CX, 0, NULL); } - if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_BX) { - /* Someday, someone will care about you... */ +#endif + if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_BX) device_printf(dev, "Media supported: 1000baseBX\n"); - } if (hw->device_id == IXGBE_DEV_ID_82598AT) { ifmedia_add(&adapter->media, @@ -2869,7 +2986,6 @@ ixgbe_initialize_transmit_units(struct adapter *adapter) struct ixgbe_hw *hw = &adapter->hw; /* Setup the Base and Length of the Tx Descriptor Ring */ - for (int i = 0; i < adapter->num_queues; i++, txr++) { u64 tdba = txr->txdma.dma_paddr; u32 txctrl = 0; @@ -2889,12 +3005,15 @@ ixgbe_initialize_transmit_units(struct adapter *adapter) txr->tail = IXGBE_TDT(j); /* Disable Head Writeback */ + /* + * Note: for X550 series devices, these registers are actually + * prefixed with TPH_ isntead of DCA_, but the addresses and + * fields remain the same. + */ switch (hw->mac.type) { case ixgbe_mac_82598EB: txctrl = IXGBE_READ_REG(hw, IXGBE_DCA_TXCTRL(j)); break; - case ixgbe_mac_82599EB: - case ixgbe_mac_X540: default: txctrl = IXGBE_READ_REG(hw, IXGBE_DCA_TXCTRL_82599(j)); break; @@ -2904,8 +3023,6 @@ ixgbe_initialize_transmit_units(struct adapter *adapter) case ixgbe_mac_82598EB: IXGBE_WRITE_REG(hw, IXGBE_DCA_TXCTRL(j), txctrl); break; - case ixgbe_mac_82599EB: - case ixgbe_mac_X540: default: IXGBE_WRITE_REG(hw, IXGBE_DCA_TXCTRL_82599(j), txctrl); break; @@ -2938,7 +3055,7 @@ ixgbe_initialize_transmit_units(struct adapter *adapter) } static void -ixgbe_initialise_rss_mapping(struct adapter *adapter) +ixgbe_initialize_rss_mapping(struct adapter *adapter) { struct ixgbe_hw *hw = &adapter->hw; u32 reta = 0, mrqc, rss_key[10]; @@ -3073,7 +3190,6 @@ ixgbe_initialize_receive_units(struct adapter *adapter) u32 bufsz, fctrl, srrctl, rxcsum; u32 hlreg; - /* * Make sure receives are disabled while * setting up the descriptor ring @@ -3158,7 +3274,7 @@ ixgbe_initialize_receive_units(struct adapter *adapter) rxcsum = IXGBE_READ_REG(hw, IXGBE_RXCSUM); - ixgbe_initialise_rss_mapping(adapter); + ixgbe_initialize_rss_mapping(adapter); if (adapter->num_queues > 1) { /* RSS and RX IPP Checksum are mutually exclusive */ @@ -3168,6 +3284,7 @@ ixgbe_initialize_receive_units(struct adapter *adapter) if (ifp->if_capenable & IFCAP_RXCSUM) rxcsum |= IXGBE_RXCSUM_PCSD; + /* This is useful for calculating UDP/IP fragment checksums */ if (!(rxcsum & IXGBE_RXCSUM_PCSD)) rxcsum |= IXGBE_RXCSUM_IPPCSE; @@ -3389,9 +3506,10 @@ ixgbe_disable_intr(struct adapter *adapter) ** the slot this adapter is plugged into. */ static void -ixgbe_get_slot_info(struct ixgbe_hw *hw) +ixgbe_get_slot_info(struct adapter *adapter) { - device_t dev = ((struct ixgbe_osdep *)hw->back)->dev; + device_t dev = adapter->dev; + struct ixgbe_hw *hw = &adapter->hw; struct ixgbe_mac_info *mac = &hw->mac; u16 link; u32 offset; @@ -3598,12 +3716,12 @@ ixgbe_sfp_probe(struct adapter *adapter) goto out; ret = hw->phy.ops.reset(hw); if (ret == IXGBE_ERR_SFP_NOT_SUPPORTED) { - device_printf(dev,"Unsupported SFP+ module detected!"); - printf(" Reload driver with supported module.\n"); + device_printf(dev, "Unsupported SFP+ module detected!"); + device_printf(dev, "Reload driver with supported module.\n"); adapter->sfp_probe = FALSE; goto out; } else - device_printf(dev,"SFP+ module detected!\n"); + device_printf(dev, "SFP+ module detected!\n"); /* We now have supported optics */ adapter->sfp_probe = FALSE; /* Set the optics type so system reports correctly */ @@ -3622,10 +3740,14 @@ static void ixgbe_handle_link(void *context, int pending) { struct adapter *adapter = context; + struct ixgbe_hw *hw = &adapter->hw; - ixgbe_check_link(&adapter->hw, + ixgbe_check_link(hw, &adapter->link_speed, &adapter->link_up, 0); ixgbe_update_link_status(adapter); + + /* Re-enable link interrupts */ + IXGBE_WRITE_REG(hw, IXGBE_EIMS, IXGBE_EIMS_LSC); } /* @@ -3667,13 +3789,9 @@ ixgbe_handle_msf(void *context, int pending) struct ixgbe_hw *hw = &adapter->hw; u32 autoneg; bool negotiate; - int err; - err = hw->phy.ops.identify_sfp(hw); - if (!err) { - ixgbe_setup_optics(adapter); - INIT_DEBUGOUT1("ixgbe_sfp_probe: flags: %X\n", adapter->optics); - } + /* get_supported_phy_layer will call hw->phy.ops.identify_sfp() */ + adapter->phy_layer = ixgbe_get_supported_physical_layer(hw); autoneg = hw->phy.autoneg_advertised; if ((!autoneg) && (hw->mac.ops.get_link_capabilities)) @@ -3681,6 +3799,7 @@ ixgbe_handle_msf(void *context, int pending) if (hw->mac.ops.setup_link) hw->mac.ops.setup_link(hw, autoneg, TRUE); + /* Adjust media types shown in ifconfig */ ifmedia_removeall(&adapter->media); ixgbe_add_media_types(adapter); return; @@ -3760,18 +3879,6 @@ ixgbe_config_dmac(struct adapter *adapter) } /* - * Checks whether the adapter supports Energy Efficient Ethernet - * or not, based on device ID. - */ -static void -ixgbe_check_eee_support(struct adapter *adapter) -{ - struct ixgbe_hw *hw = &adapter->hw; - - adapter->eee_enabled = !!(hw->mac.ops.setup_eee); -} - -/* * Checks whether the adapter's ports are capable of * Wake On LAN by reading the adapter's NVM. * @@ -3789,8 +3896,8 @@ ixgbe_check_wol_support(struct adapter *adapter) ixgbe_get_device_caps(hw, &dev_caps); if ((dev_caps & IXGBE_DEVICE_CAPS_WOL_PORT0_1) || ((dev_caps & IXGBE_DEVICE_CAPS_WOL_PORT0) && - hw->bus.func == 0)) - adapter->wol_support = hw->wol_enabled = 1; + hw->bus.func == 0)) + adapter->wol_support = hw->wol_enabled = 1; /* Save initial wake up filter configuration */ adapter->wufc = IXGBE_READ_REG(hw, IXGBE_WUFC); @@ -4137,14 +4244,24 @@ ixgbe_add_device_sysctls(struct adapter *adapter) CTLTYPE_INT | CTLFLAG_RW, adapter, 0, ixgbe_sysctl_thermal_test, "I", "Thermal Test"); - /* for X550 devices */ +#ifdef IXGBE_DEBUG + /* testing sysctls (for all devices) */ + SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "power_state", + CTLTYPE_INT | CTLFLAG_RW, adapter, 0, + ixgbe_sysctl_power_state, "I", "PCI Power State"); + + SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "print_rss_config", + CTLTYPE_STRING | CTLFLAG_RD, adapter, 0, + ixgbe_sysctl_print_rss_config, "A", "Prints RSS Configuration"); +#endif + /* for X550 series devices */ if (hw->mac.type >= ixgbe_mac_X550) SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "dmac", CTLTYPE_INT | CTLFLAG_RW, adapter, 0, ixgbe_sysctl_dmac, "I", "DMA Coalesce"); - /* for X550T and X550EM backplane devices */ - if (hw->mac.ops.setup_eee) { + /* for X552 backplane devices */ + if (hw->device_id == IXGBE_DEV_ID_X550EM_X_KR) { struct sysctl_oid *eee_node; struct sysctl_oid_list *eee_list; @@ -4172,11 +4289,15 @@ ixgbe_add_device_sysctls(struct adapter *adapter) CTLTYPE_INT | CTLFLAG_RD, adapter, 0, ixgbe_sysctl_eee_rx_lpi_status, "I", "Whether or not RX link is in LPI state"); + + SYSCTL_ADD_PROC(ctx, eee_list, OID_AUTO, "tx_lpi_delay", + CTLTYPE_INT | CTLFLAG_RD, adapter, 0, + ixgbe_sysctl_eee_tx_lpi_delay, "I", + "TX LPI entry delay in microseconds"); } - /* for certain 10GBaseT devices */ - if (hw->device_id == IXGBE_DEV_ID_X550T || - hw->device_id == IXGBE_DEV_ID_X550EM_X_10G_T) { + /* for WoL-capable devices */ + if (hw->device_id == IXGBE_DEV_ID_X550EM_X_10G_T) { SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "wol_enable", CTLTYPE_INT | CTLFLAG_RW, adapter, 0, ixgbe_sysctl_wol_enable, "I", @@ -4188,7 +4309,7 @@ ixgbe_add_device_sysctls(struct adapter *adapter) "Enable/Disable Wake Up Filters"); } - /* for X550EM 10GBaseT devices */ + /* for X552/X557-AT devices */ if (hw->device_id == IXGBE_DEV_ID_X550EM_X_10G_T) { struct sysctl_oid *phy_node; struct sysctl_oid_list *phy_list; @@ -4544,6 +4665,10 @@ ixgbe_set_advertise(SYSCTL_HANDLER_ARGS) if ((error) || (req->newptr == NULL)) return (error); + /* No speed changes for backplane media */ + if (hw->phy.media_type == ixgbe_media_type_backplane) + return (ENODEV); + /* Checks to validate new value */ if (adapter->advertise == requested) /* no change */ return (0); @@ -4585,7 +4710,7 @@ ixgbe_set_advertise(SYSCTL_HANDLER_ARGS) } /* - * The following two sysctls are for X550 BaseT devices; + * The following two sysctls are for X552/X557-AT devices; * they deal with the external PHY used in them. */ static int @@ -4684,31 +4809,22 @@ static int ixgbe_sysctl_dmac(SYSCTL_HANDLER_ARGS) { struct adapter *adapter = (struct adapter *) arg1; - struct ixgbe_hw *hw = &adapter->hw; struct ifnet *ifp = adapter->ifp; int error; - u16 oldval; + u32 newval; - oldval = adapter->dmac; - error = sysctl_handle_int(oidp, &adapter->dmac, 0, req); + newval = adapter->dmac; + error = sysctl_handle_int(oidp, &newval, 0, req); if ((error) || (req->newptr == NULL)) return (error); - switch (hw->mac.type) { - case ixgbe_mac_X550: - case ixgbe_mac_X550EM_x: - break; - default: - device_printf(adapter->dev, - "DMA Coalescing is only supported on X550 devices\n"); - return (ENODEV); - } - - switch (adapter->dmac) { + switch (newval) { case 0: /* Disabled */ + adapter->dmac = 0; break; - case 1: /* Enable and use default */ + case 1: + /* Enable and use default */ adapter->dmac = 1000; break; case 50: @@ -4720,10 +4836,10 @@ ixgbe_sysctl_dmac(SYSCTL_HANDLER_ARGS) case 5000: case 10000: /* Legal values - allow */ + adapter->dmac = newval; break; default: /* Do nothing, illegal value */ - adapter->dmac = oldval; return (EINVAL); } @@ -4734,6 +4850,42 @@ ixgbe_sysctl_dmac(SYSCTL_HANDLER_ARGS) return (0); } +#ifdef IXGBE_DEBUG +/** + * Sysctl to test power states + * Values: + * 0 - set device to D0 + * 3 - set device to D3 + * (none) - get current device power state + */ +static int +ixgbe_sysctl_power_state(SYSCTL_HANDLER_ARGS) +{ + struct adapter *adapter = (struct adapter *) arg1; + device_t dev = adapter->dev; + int curr_ps, new_ps, error = 0; + + curr_ps = new_ps = pci_get_powerstate(dev); + + error = sysctl_handle_int(oidp, &new_ps, 0, req); + if ((error) || (req->newptr == NULL)) + return (error); + + if (new_ps == curr_ps) + return (0); + + if (new_ps == 3 && curr_ps == 0) + error = DEVICE_SUSPEND(dev); + else if (new_ps == 0 && curr_ps == 3) + error = DEVICE_RESUME(dev); + else + return (EINVAL); + + device_printf(dev, "New state: %d\n", pci_get_powerstate(dev)); + + return (error); +} +#endif /* * Sysctl to enable/disable the WoL capability, if supported by the adapter. * Values: @@ -4752,13 +4904,14 @@ ixgbe_sysctl_wol_enable(SYSCTL_HANDLER_ARGS) error = sysctl_handle_int(oidp, &new_wol_enabled, 0, req); if ((error) || (req->newptr == NULL)) return (error); + new_wol_enabled = !!(new_wol_enabled); if (new_wol_enabled == hw->wol_enabled) return (0); if (new_wol_enabled > 0 && !adapter->wol_support) return (ENODEV); else - hw->wol_enabled = !!(new_wol_enabled); + hw->wol_enabled = new_wol_enabled; return (0); } @@ -4782,13 +4935,14 @@ ixgbe_sysctl_eee_enable(SYSCTL_HANDLER_ARGS) error = sysctl_handle_int(oidp, &new_eee_enabled, 0, req); if ((error) || (req->newptr == NULL)) return (error); + new_eee_enabled = !!(new_eee_enabled); if (new_eee_enabled == adapter->eee_enabled) return (0); if (new_eee_enabled > 0 && !hw->mac.ops.setup_eee) return (ENODEV); else - adapter->eee_enabled = !!(new_eee_enabled); + adapter->eee_enabled = new_eee_enabled; /* Re-initialize hardware if it's already running */ if (ifp->if_drv_flags & IFF_DRV_RUNNING) @@ -4846,6 +5000,21 @@ ixgbe_sysctl_eee_tx_lpi_status(SYSCTL_HANDLER_ARGS) } /* + * Read-only sysctl indicating TX Link LPI delay + */ +static int +ixgbe_sysctl_eee_tx_lpi_delay(SYSCTL_HANDLER_ARGS) +{ + struct adapter *adapter = (struct adapter *) arg1; + struct ixgbe_hw *hw = &adapter->hw; + u32 reg; + + reg = IXGBE_READ_REG(hw, IXGBE_EEE_SU); + + return (sysctl_handle_int(oidp, 0, reg >> 26, req)); +} + +/* * Sysctl to enable/disable the types of packets that the * adapter will wake up on upon receipt. * WUFC - Wake Up Filter Control @@ -4888,6 +5057,58 @@ ixgbe_sysctl_wufc(SYSCTL_HANDLER_ARGS) return (0); } +#ifdef IXGBE_DEBUG +static int +ixgbe_sysctl_print_rss_config(SYSCTL_HANDLER_ARGS) +{ + struct adapter *adapter = (struct adapter *)arg1; + struct ixgbe_hw *hw = &adapter->hw; + device_t dev = adapter->dev; + int error = 0, reta_size; + struct sbuf *buf; + u32 reg; + + buf = sbuf_new_for_sysctl(NULL, NULL, 128, req); + if (!buf) { + device_printf(dev, "Could not allocate sbuf for output.\n"); + return (ENOMEM); + } + + // TODO: use sbufs to make a string to print out + /* Set multiplier for RETA setup and table size based on MAC */ + switch (adapter->hw.mac.type) { + case ixgbe_mac_X550: + case ixgbe_mac_X550EM_x: + reta_size = 128; + break; + default: + reta_size = 32; + break; + } + + /* Print out the redirection table */ + sbuf_cat(buf, "\n"); + for (int i = 0; i < reta_size; i++) { + if (i < 32) { + reg = IXGBE_READ_REG(hw, IXGBE_RETA(i)); + sbuf_printf(buf, "RETA(%2d): 0x%08x\n", i, reg); + } else { + reg = IXGBE_READ_REG(hw, IXGBE_ERETA(i - 32)); + sbuf_printf(buf, "ERETA(%2d): 0x%08x\n", i - 32, reg); + } + } + + // TODO: print more config + + error = sbuf_finish(buf); + if (error) + device_printf(dev, "Error finishing sbuf: %d\n", error); + + sbuf_delete(buf); + return (0); +} +#endif /* IXGBE_DEBUG */ + /* ** Enable the hardware to drop packets when the buffer is ** full. This is useful when multiqueue,so that no single diff --git a/sys/dev/ixgbe/if_ixv.c b/sys/dev/ixgbe/if_ixv.c index d58541e0..0b147d4 100644 --- a/sys/dev/ixgbe/if_ixv.c +++ b/sys/dev/ixgbe/if_ixv.c @@ -43,7 +43,7 @@ /********************************************************************* * Driver version *********************************************************************/ -char ixv_driver_version[] = "1.4.0"; +char ixv_driver_version[] = "1.4.6-k"; /********************************************************************* * PCI Device ID Table @@ -292,7 +292,7 @@ ixv_attach(device_t dev) /* Allocate, clear, and link in our adapter structure */ adapter = device_get_softc(dev); - adapter->dev = adapter->osdep.dev = dev; + adapter->dev = dev; hw = &adapter->hw; #ifdef DEV_NETMAP @@ -322,7 +322,7 @@ ixv_attach(device_t dev) /* Do base PCI setup - map BAR0 */ if (ixv_allocate_pci_resources(adapter)) { - device_printf(dev, "Allocation of PCI resources failed\n"); + device_printf(dev, "ixv_allocate_pci_resources() failed!\n"); error = ENXIO; goto err_out; } @@ -353,6 +353,7 @@ ixv_attach(device_t dev) /* Allocate our TX/RX Queues */ if (ixgbe_allocate_queues(adapter)) { + device_printf(dev, "ixgbe_allocate_queues() failed!\n"); error = ENOMEM; goto err_out; } @@ -363,7 +364,7 @@ ixv_attach(device_t dev) */ error = ixgbe_init_shared_code(hw); if (error) { - device_printf(dev,"Shared Code Initialization Failure\n"); + device_printf(dev, "ixgbe_init_shared_code() failed!\n"); error = EIO; goto err_late; } @@ -371,23 +372,37 @@ ixv_attach(device_t dev) /* Setup the mailbox */ ixgbe_init_mbx_params_vf(hw); - ixgbe_reset_hw(hw); + /* Reset mbox api to 1.0 */ + error = ixgbe_reset_hw(hw); + if (error == IXGBE_ERR_RESET_FAILED) + device_printf(dev, "ixgbe_reset_hw() failure: Reset Failed!\n"); + else if (error) + device_printf(dev, "ixgbe_reset_hw() failed with error %d\n", error); + if (error) { + error = EIO; + goto err_late; + } - /* Get the Mailbox API version */ - device_printf(dev,"MBX API %d negotiation: %d\n", - ixgbe_mbox_api_11, - ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_11)); + /* Negotiate mailbox API version */ + error = ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_11); + if (error) { + device_printf(dev, "MBX API 1.1 negotiation failed! Error %d\n", error); + error = EIO; + goto err_late; + } error = ixgbe_init_hw(hw); if (error) { - device_printf(dev,"Hardware Initialization Failure\n"); + device_printf(dev, "ixgbe_init_hw() failed!\n"); error = EIO; goto err_late; } error = ixv_allocate_msix(adapter); - if (error) + if (error) { + device_printf(dev, "ixv_allocate_msix() failed!\n"); goto err_late; + } /* If no mac address was assigned, make a random one */ if (!ixv_check_ether_addr(hw->mac.addr)) { @@ -447,7 +462,7 @@ ixv_detach(device_t dev) /* Make sure VLANS are not using driver */ if (adapter->ifp->if_vlantrunk != NULL) { - device_printf(dev,"Vlan in use, detach first\n"); + device_printf(dev, "Vlan in use, detach first\n"); return (EBUSY); } @@ -556,13 +571,13 @@ ixv_ioctl(struct ifnet * ifp, u_long command, caddr_t data) #endif case SIOCSIFMTU: IOCTL_DEBUGOUT("ioctl: SIOCSIFMTU (Set Interface MTU)"); - if (ifr->ifr_mtu > IXGBE_MAX_FRAME_SIZE - ETHER_HDR_LEN) { + if (ifr->ifr_mtu > IXGBE_MAX_FRAME_SIZE - IXGBE_MTU_HDR) { error = EINVAL; } else { IXGBE_CORE_LOCK(adapter); ifp->if_mtu = ifr->ifr_mtu; adapter->max_frame_size = - ifp->if_mtu + ETHER_HDR_LEN + ETHER_CRC_LEN; + ifp->if_mtu + IXGBE_MTU_HDR; ixv_init_locked(adapter); IXGBE_CORE_UNLOCK(adapter); } @@ -643,9 +658,9 @@ ixv_init_locked(struct adapter *adapter) struct ifnet *ifp = adapter->ifp; device_t dev = adapter->dev; struct ixgbe_hw *hw = &adapter->hw; - u32 mhadd, gpie; + int error = 0; - INIT_DEBUGOUT("ixv_init: begin"); + INIT_DEBUGOUT("ixv_init_locked: begin"); mtx_assert(&adapter->core_mtx, MA_OWNED); hw->adapter_stopped = FALSE; ixgbe_stop_adapter(hw); @@ -662,12 +677,17 @@ ixv_init_locked(struct adapter *adapter) /* Prepare transmit descriptors and buffers */ if (ixgbe_setup_transmit_structures(adapter)) { - device_printf(dev,"Could not setup transmit structures\n"); + device_printf(dev, "Could not setup transmit structures\n"); ixv_stop(adapter); return; } + /* Reset VF and renegotiate mailbox API version */ ixgbe_reset_hw(hw); + error = ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_11); + if (error) + device_printf(dev, "MBX API 1.1 negotiation failed! Error %d\n", error); + ixv_initialize_transmit_units(adapter); /* Setup Multicast table */ @@ -684,7 +704,7 @@ ixv_init_locked(struct adapter *adapter) /* Prepare receive descriptors and buffers */ if (ixgbe_setup_receive_structures(adapter)) { - device_printf(dev,"Could not setup receive structures\n"); + device_printf(dev, "Could not setup receive structures\n"); ixv_stop(adapter); return; } @@ -692,12 +712,6 @@ ixv_init_locked(struct adapter *adapter) /* Configure RX settings */ ixv_initialize_receive_units(adapter); - /* Enable Enhanced MSIX mode */ - gpie = IXGBE_READ_REG(&adapter->hw, IXGBE_GPIE); - gpie |= IXGBE_GPIE_MSIX_MODE | IXGBE_GPIE_EIAME; - gpie |= IXGBE_GPIE_PBA_SUPPORT | IXGBE_GPIE_OCD; - IXGBE_WRITE_REG(hw, IXGBE_GPIE, gpie); - /* Set the various hardware offload abilities */ ifp->if_hwassist = 0; if (ifp->if_capenable & IFCAP_TSO4) @@ -709,19 +723,9 @@ ixv_init_locked(struct adapter *adapter) #endif } - /* Set MTU size */ - if (ifp->if_mtu > ETHERMTU) { - mhadd = IXGBE_READ_REG(hw, IXGBE_MHADD); - mhadd &= ~IXGBE_MHADD_MFS_MASK; - mhadd |= adapter->max_frame_size << IXGBE_MHADD_MFS_SHIFT; - IXGBE_WRITE_REG(hw, IXGBE_MHADD, mhadd); - } - /* Set up VLAN offload and filter */ ixv_setup_vlan_support(adapter); - callout_reset(&adapter->timer, hz, ixv_local_timer, adapter); - /* Set up MSI/X routing */ ixv_configure_ivars(adapter); @@ -737,6 +741,9 @@ ixv_init_locked(struct adapter *adapter) /* Config/Enable Link */ ixv_config_link(adapter); + /* Start watchdog */ + callout_reset(&adapter->timer, hz, ixv_local_timer, adapter); + /* And now turn on interrupts */ ixv_enable_intr(adapter); @@ -1414,7 +1421,7 @@ ixv_allocate_pci_resources(struct adapter *adapter) &rid, RF_ACTIVE); if (!(adapter->pci_mem)) { - device_printf(dev,"Unable to allocate bus resource: memory\n"); + device_printf(dev, "Unable to allocate bus resource: memory\n"); return (ENXIO); } @@ -1422,12 +1429,11 @@ ixv_allocate_pci_resources(struct adapter *adapter) rman_get_bustag(adapter->pci_mem); adapter->osdep.mem_bus_space_handle = rman_get_bushandle(adapter->pci_mem); - adapter->hw.hw_addr = (u8 *) &adapter->osdep.mem_bus_space_handle; + adapter->hw.hw_addr = (u8 *)&adapter->osdep.mem_bus_space_handle; /* Pick up the tuneable queues */ adapter->num_queues = ixv_num_queues; - - adapter->hw.back = &adapter->osdep; + adapter->hw.back = adapter; /* ** Now setup MSI/X, should @@ -1535,7 +1541,7 @@ ixv_setup_interface(device_t dev, struct adapter *adapter) ether_ifattach(ifp, adapter->hw.mac.addr); adapter->max_frame_size = - ifp->if_mtu + ETHER_HDR_LEN + ETHER_CRC_LEN; + ifp->if_mtu + IXGBE_MTU_HDR_VLAN; /* * Tell the upper layer(s) we support long frames. @@ -1556,7 +1562,6 @@ ixv_setup_interface(device_t dev, struct adapter *adapter) */ ifmedia_init(&adapter->media, IFM_IMASK, ixv_media_change, ixv_media_status); - ifmedia_add(&adapter->media, IFM_ETHER | IFM_FDX, 0, NULL); ifmedia_add(&adapter->media, IFM_ETHER | IFM_AUTO, 0, NULL); ifmedia_set(&adapter->media, IFM_ETHER | IFM_AUTO); @@ -1567,19 +1572,11 @@ static void ixv_config_link(struct adapter *adapter) { struct ixgbe_hw *hw = &adapter->hw; - u32 autoneg, err = 0; + u32 autoneg; if (hw->mac.ops.check_link) - err = hw->mac.ops.check_link(hw, &autoneg, + hw->mac.ops.check_link(hw, &autoneg, &adapter->link_up, FALSE); - if (err) - goto out; - - if (hw->mac.ops.setup_link) - err = hw->mac.ops.setup_link(hw, - autoneg, adapter->link_up); -out: - return; } @@ -1646,7 +1643,6 @@ ixv_initialize_receive_units(struct adapter *adapter) struct ixgbe_hw *hw = &adapter->hw; struct ifnet *ifp = adapter->ifp; u32 bufsz, rxcsum, psrtype; - int max_frame; if (ifp->if_mtu > ETHERMTU) bufsz = 4096 >> IXGBE_SRRCTL_BSIZEPKT_SHIFT; @@ -1659,9 +1655,8 @@ ixv_initialize_receive_units(struct adapter *adapter) IXGBE_WRITE_REG(hw, IXGBE_VFPSRTYPE, psrtype); - /* Tell PF our expected packet-size */ - max_frame = ifp->if_mtu + IXGBE_MTU_HDR; - ixgbevf_rlpml_set_vf(hw, max_frame); + /* Tell PF our max_frame size */ + ixgbevf_rlpml_set_vf(hw, adapter->max_frame_size); for (int i = 0; i < adapter->num_queues; i++, rxr++) { u64 rdba = rxr->rxdma.dma_paddr; @@ -1763,7 +1758,7 @@ ixv_setup_vlan_support(struct adapter *adapter) { struct ixgbe_hw *hw = &adapter->hw; u32 ctrl, vid, vfta, retry; - + struct rx_ring *rxr; /* ** We get here thru init_locked, meaning @@ -1779,6 +1774,12 @@ ixv_setup_vlan_support(struct adapter *adapter) ctrl = IXGBE_READ_REG(hw, IXGBE_VFRXDCTL(i)); ctrl |= IXGBE_RXDCTL_VME; IXGBE_WRITE_REG(hw, IXGBE_VFRXDCTL(i), ctrl); + /* + * Let Rx path know that it needs to store VLAN tag + * as part of extra mbuf info. + */ + rxr = &adapter->rx_rings[i]; + rxr->vtag_strip = TRUE; } /* @@ -1794,7 +1795,7 @@ ixv_setup_vlan_support(struct adapter *adapter) ** based on the bits set in each ** of the array ints. */ - for ( int j = 0; j < 32; j++) { + for (int j = 0; j < 32; j++) { retry = 0; if ((vfta & (1 << j)) == 0) continue; @@ -1821,10 +1822,10 @@ ixv_register_vlan(void *arg, struct ifnet *ifp, u16 vtag) struct adapter *adapter = ifp->if_softc; u16 index, bit; - if (ifp->if_softc != arg) /* Not our event */ + if (ifp->if_softc != arg) /* Not our event */ return; - if ((vtag == 0) || (vtag > 4095)) /* Invalid */ + if ((vtag == 0) || (vtag > 4095)) /* Invalid */ return; IXGBE_CORE_LOCK(adapter); diff --git a/sys/dev/ixgbe/ix_txrx.c b/sys/dev/ixgbe/ix_txrx.c index 7f262f9..00e5dc8 100644 --- a/sys/dev/ixgbe/ix_txrx.c +++ b/sys/dev/ixgbe/ix_txrx.c @@ -81,27 +81,6 @@ static bool ixgbe_rsc_enable = FALSE; static int atr_sample_rate = 20; #endif -/* Shared PCI config read/write */ -inline u16 -ixgbe_read_pci_cfg(struct ixgbe_hw *hw, u32 reg) -{ - u16 value; - - value = pci_read_config(((struct ixgbe_osdep *)hw->back)->dev, - reg, 2); - - return (value); -} - -inline void -ixgbe_write_pci_cfg(struct ixgbe_hw *hw, u32 reg, u16 value) -{ - pci_write_config(((struct ixgbe_osdep *)hw->back)->dev, - reg, value, 2); - - return; -} - /********************************************************************* * Local Function prototypes *********************************************************************/ @@ -189,8 +168,8 @@ ixgbe_start(struct ifnet *ifp) #else /* ! IXGBE_LEGACY_TX */ /* -** Multiqueue Transmit driver -** +** Multiqueue Transmit Entry Point +** (if_transmit function) */ int ixgbe_mq_start(struct ifnet *ifp, struct mbuf *m) @@ -213,10 +192,14 @@ ixgbe_mq_start(struct ifnet *ifp, struct mbuf *m) if (M_HASHTYPE_GET(m) != M_HASHTYPE_NONE) { #ifdef RSS if (rss_hash2bucket(m->m_pkthdr.flowid, - M_HASHTYPE_GET(m), &bucket_id) == 0) - /* TODO: spit out something if bucket_id > num_queues? */ + M_HASHTYPE_GET(m), &bucket_id) == 0) { i = bucket_id % adapter->num_queues; - else +#ifdef IXGBE_DEBUG + if (bucket_id > adapter->num_queues) + if_printf(ifp, "bucket_id (%d) > num_queues " + "(%d)\n", bucket_id, adapter->num_queues); +#endif + } else #endif i = m->m_pkthdr.flowid % adapter->num_queues; } else @@ -448,6 +431,7 @@ retry: } #endif + olinfo_status |= IXGBE_ADVTXD_CC; i = txr->next_avail_desc; for (j = 0; j < nsegs; j++) { bus_size_t seglen; @@ -742,8 +726,12 @@ ixgbe_tx_ctx_setup(struct tx_ring *txr, struct mbuf *mp, struct adapter *adapter = txr->adapter; struct ixgbe_adv_tx_context_desc *TXD; struct ether_vlan_header *eh; +#ifdef INET struct ip *ip; +#endif +#ifdef INET6 struct ip6_hdr *ip6; +#endif u32 vlan_macip_lens = 0, type_tucmd_mlhl = 0; int ehdrlen, ip_hlen = 0; u16 etype; @@ -751,9 +739,11 @@ ixgbe_tx_ctx_setup(struct tx_ring *txr, struct mbuf *mp, int offload = TRUE; int ctxd = txr->next_avail_desc; u16 vtag = 0; + caddr_t l3d; + /* First check if TSO is to be used */ - if (mp->m_pkthdr.csum_flags & CSUM_TSO) + if (mp->m_pkthdr.csum_flags & (CSUM_IP_TSO|CSUM_IP6_TSO)) return (ixgbe_tso_setup(txr, mp, cmd_type_len, olinfo_status)); if ((mp->m_pkthdr.csum_flags & CSUM_OFFLOAD) == 0) @@ -796,20 +786,38 @@ ixgbe_tx_ctx_setup(struct tx_ring *txr, struct mbuf *mp, if (offload == FALSE) goto no_offloads; + /* + * If the first mbuf only includes the ethernet header, jump to the next one + * XXX: This assumes the stack splits mbufs containing headers on header boundaries + * XXX: And assumes the entire IP header is contained in one mbuf + */ + if (mp->m_len == ehdrlen && mp->m_next) + l3d = mtod(mp->m_next, caddr_t); + else + l3d = mtod(mp, caddr_t) + ehdrlen; + switch (etype) { +#ifdef INET case ETHERTYPE_IP: - ip = (struct ip *)(mp->m_data + ehdrlen); + ip = (struct ip *)(l3d); ip_hlen = ip->ip_hl << 2; ipproto = ip->ip_p; type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_IPV4; + /* Insert IPv4 checksum into data descriptors */ + if (mp->m_pkthdr.csum_flags & CSUM_IP) { + ip->ip_sum = 0; + *olinfo_status |= IXGBE_TXD_POPTS_IXSM << 8; + } break; +#endif +#ifdef INET6 case ETHERTYPE_IPV6: - ip6 = (struct ip6_hdr *)(mp->m_data + ehdrlen); + ip6 = (struct ip6_hdr *)(l3d); ip_hlen = sizeof(struct ip6_hdr); - /* XXX-BZ this will go badly in case of ext hdrs. */ ipproto = ip6->ip6_nxt; type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_IPV6; break; +#endif default: offload = FALSE; break; @@ -817,29 +825,32 @@ ixgbe_tx_ctx_setup(struct tx_ring *txr, struct mbuf *mp, vlan_macip_lens |= ip_hlen; + /* No support for offloads for non-L4 next headers */ switch (ipproto) { case IPPROTO_TCP: - if (mp->m_pkthdr.csum_flags & CSUM_TCP) + if (mp->m_pkthdr.csum_flags & (CSUM_IP_TCP | CSUM_IP6_TCP)) type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_TCP; + else + offload = false; break; - case IPPROTO_UDP: - if (mp->m_pkthdr.csum_flags & CSUM_UDP) + if (mp->m_pkthdr.csum_flags & (CSUM_IP_UDP | CSUM_IP6_UDP)) type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_UDP; + else + offload = false; break; - -#if __FreeBSD_version >= 800000 case IPPROTO_SCTP: - if (mp->m_pkthdr.csum_flags & CSUM_SCTP) + if (mp->m_pkthdr.csum_flags & (CSUM_IP_SCTP | CSUM_IP6_SCTP)) type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_SCTP; + else + offload = false; break; -#endif default: - offload = FALSE; + offload = false; break; } - if (offload) /* For the TX descriptor setup */ + if (offload) /* Insert L4 checksum into data descriptors */ *olinfo_status |= IXGBE_TXD_POPTS_TXSM << 8; no_offloads: @@ -884,7 +895,6 @@ ixgbe_tso_setup(struct tx_ring *txr, struct mbuf *mp, #endif struct tcphdr *th; - /* * Determine where frame payload starts. * Jump over vlan headers if already present @@ -1041,7 +1051,7 @@ ixgbe_txeof(struct tx_ring *txr) BUS_DMASYNC_POSTREAD); do { - union ixgbe_adv_tx_desc *eop= buf->eop; + union ixgbe_adv_tx_desc *eop = buf->eop; if (eop == NULL) /* No work */ break; @@ -1283,6 +1293,7 @@ ixgbe_setup_hw_rsc(struct rx_ring *rxr) rxr->hw_rsc = TRUE; } + /********************************************************************* * * Refresh mbuf buffers for RX descriptor rings @@ -1418,7 +1429,6 @@ fail: return (error); } - static void ixgbe_free_receive_ring(struct rx_ring *rxr) { @@ -1438,7 +1448,6 @@ ixgbe_free_receive_ring(struct rx_ring *rxr) } } - /********************************************************************* * * Initialize a receive ring and its buffers. @@ -1916,13 +1925,17 @@ ixgbe_rxeof(struct ix_queue *que) sendmp->m_pkthdr.flowid = le32toh(cur->wb.lower.hi_dword.rss); switch (pkt_info & IXGBE_RXDADV_RSSTYPE_MASK) { + case IXGBE_RXDADV_RSSTYPE_IPV4: + M_HASHTYPE_SET(sendmp, + M_HASHTYPE_RSS_IPV4); + break; case IXGBE_RXDADV_RSSTYPE_IPV4_TCP: M_HASHTYPE_SET(sendmp, M_HASHTYPE_RSS_TCP_IPV4); break; - case IXGBE_RXDADV_RSSTYPE_IPV4: + case IXGBE_RXDADV_RSSTYPE_IPV6: M_HASHTYPE_SET(sendmp, - M_HASHTYPE_RSS_IPV4); + M_HASHTYPE_RSS_IPV6); break; case IXGBE_RXDADV_RSSTYPE_IPV6_TCP: M_HASHTYPE_SET(sendmp, @@ -1932,14 +1945,11 @@ ixgbe_rxeof(struct ix_queue *que) M_HASHTYPE_SET(sendmp, M_HASHTYPE_RSS_IPV6_EX); break; - case IXGBE_RXDADV_RSSTYPE_IPV6: - M_HASHTYPE_SET(sendmp, - M_HASHTYPE_RSS_IPV6); - break; case IXGBE_RXDADV_RSSTYPE_IPV6_TCP_EX: M_HASHTYPE_SET(sendmp, M_HASHTYPE_RSS_TCP_IPV6_EX); break; +#if __FreeBSD_version > 1100000 case IXGBE_RXDADV_RSSTYPE_IPV4_UDP: M_HASHTYPE_SET(sendmp, M_HASHTYPE_RSS_UDP_IPV4); @@ -1952,6 +1962,7 @@ ixgbe_rxeof(struct ix_queue *que) M_HASHTYPE_SET(sendmp, M_HASHTYPE_RSS_UDP_IPV6_EX); break; +#endif default: M_HASHTYPE_SET(sendmp, M_HASHTYPE_OPAQUE); @@ -2021,34 +2032,28 @@ ixgbe_rx_checksum(u32 staterr, struct mbuf * mp, u32 ptype) { u16 status = (u16) staterr; u8 errors = (u8) (staterr >> 24); - bool sctp = FALSE; + bool sctp = false; if ((ptype & IXGBE_RXDADV_PKTTYPE_ETQF) == 0 && (ptype & IXGBE_RXDADV_PKTTYPE_SCTP) != 0) - sctp = TRUE; + sctp = true; + /* IPv4 checksum */ if (status & IXGBE_RXD_STAT_IPCS) { - if (!(errors & IXGBE_RXD_ERR_IPE)) { - /* IP Checksum Good */ - mp->m_pkthdr.csum_flags = CSUM_IP_CHECKED; - mp->m_pkthdr.csum_flags |= CSUM_IP_VALID; - - } else - mp->m_pkthdr.csum_flags = 0; + mp->m_pkthdr.csum_flags |= CSUM_L3_CALC; + /* IP Checksum Good */ + if (!(errors & IXGBE_RXD_ERR_IPE)) + mp->m_pkthdr.csum_flags |= CSUM_L3_VALID; } + /* TCP/UDP/SCTP checksum */ if (status & IXGBE_RXD_STAT_L4CS) { - u64 type = (CSUM_DATA_VALID | CSUM_PSEUDO_HDR); -#if __FreeBSD_version >= 800000 - if (sctp) - type = CSUM_SCTP_VALID; -#endif + mp->m_pkthdr.csum_flags |= CSUM_L4_CALC; if (!(errors & IXGBE_RXD_ERR_TCPE)) { - mp->m_pkthdr.csum_flags |= type; + mp->m_pkthdr.csum_flags |= CSUM_L4_VALID; if (!sctp) mp->m_pkthdr.csum_data = htons(0xffff); - } + } } - return; } /******************************************************************** diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h index e44f7d2..243f37b 100644 --- a/sys/dev/ixgbe/ixgbe.h +++ b/sys/dev/ixgbe/ixgbe.h @@ -141,12 +141,6 @@ #define DBA_ALIGN 128 /* - * This parameter controls the maximum no of times the driver will loop in - * the isr. Minimum Value = 1 - */ -#define MAX_LOOP 10 - -/* * This is the max watchdog interval, ie. the time that can * pass between any two TX clean operations, such only happening * when the TX hardware is functioning. @@ -162,9 +156,11 @@ /* These defines are used in MTU calculations */ #define IXGBE_MAX_FRAME_SIZE 9728 -#define IXGBE_MTU_HDR (ETHER_HDR_LEN + ETHER_CRC_LEN + \ +#define IXGBE_MTU_HDR (ETHER_HDR_LEN + ETHER_CRC_LEN) +#define IXGBE_MTU_HDR_VLAN (ETHER_HDR_LEN + ETHER_CRC_LEN + \ ETHER_VLAN_ENCAP_LEN) #define IXGBE_MAX_MTU (IXGBE_MAX_FRAME_SIZE - IXGBE_MTU_HDR) +#define IXGBE_MAX_MTU_VLAN (IXGBE_MAX_FRAME_SIZE - IXGBE_MTU_HDR_VLAN) /* Flow control constants */ #define IXGBE_FC_PAUSE 0xFFFF @@ -181,6 +177,9 @@ * modern Intel CPUs, results in 40 bytes wasted and a significant drop * in observed efficiency of the optimization, 97.9% -> 81.8%. */ +#if __FreeBSD_version < 1002000 +#define MPKTHSIZE (sizeof(struct m_hdr) + sizeof(struct pkthdr)) +#endif #define IXGBE_RX_COPY_HDR_PADDED ((((MPKTHSIZE - 1) / 32) + 1) * 32) #define IXGBE_RX_COPY_LEN (MSIZE - IXGBE_RX_COPY_HDR_PADDED) #define IXGBE_RX_COPY_ALIGN (IXGBE_RX_COPY_HDR_PADDED - MPKTHSIZE) @@ -211,7 +210,6 @@ #define MSIX_82598_BAR 3 #define MSIX_82599_BAR 4 #define IXGBE_TSO_SIZE 262140 -#define IXGBE_TX_BUFFER_SIZE ((u32) 1514) #define IXGBE_RX_HDR 128 #define IXGBE_VFTA_SIZE 128 #define IXGBE_BR_SIZE 4096 @@ -221,8 +219,12 @@ #define IXV_EITR_DEFAULT 128 -/* Offload bits in mbuf flag */ -#if __FreeBSD_version >= 800000 +/* Supported offload bits in mbuf flag */ +#if __FreeBSD_version >= 1000000 +#define CSUM_OFFLOAD (CSUM_IP_TSO|CSUM_IP6_TSO|CSUM_IP| \ + CSUM_IP_UDP|CSUM_IP_TCP|CSUM_IP_SCTP| \ + CSUM_IP6_UDP|CSUM_IP6_TCP|CSUM_IP6_SCTP) +#elif __FreeBSD_version >= 800000 #define CSUM_OFFLOAD (CSUM_IP|CSUM_TCP|CSUM_UDP|CSUM_SCTP) #else #define CSUM_OFFLOAD (CSUM_IP|CSUM_TCP|CSUM_UDP) @@ -243,7 +245,11 @@ #define IXGBE_LOW_LATENCY 128 #define IXGBE_AVE_LATENCY 400 #define IXGBE_BULK_LATENCY 1200 -#define IXGBE_LINK_ITR 2000 + +/* Using 1FF (the max value), the interval is ~1.05ms */ +#define IXGBE_LINK_ITR_QUANTA 0x1FF +#define IXGBE_LINK_ITR ((IXGBE_LINK_ITR_QUANTA << 3) & \ + IXGBE_EITR_ITR_INT_MASK) /* MAC type macros */ #define IXGBE_IS_X550VF(_adapter) \ @@ -449,11 +455,11 @@ struct ixgbe_vf { /* Our adapter structure */ struct adapter { - struct ifnet *ifp; struct ixgbe_hw hw; - struct ixgbe_osdep osdep; + struct device *dev; + struct ifnet *ifp; struct resource *pci_mem; struct resource *msix_mem; diff --git a/sys/dev/ixgbe/ixgbe_82598.c b/sys/dev/ixgbe/ixgbe_82598.c index 46e64c5..c10e23a 100644 --- a/sys/dev/ixgbe/ixgbe_82598.c +++ b/sys/dev/ixgbe/ixgbe_82598.c @@ -660,7 +660,7 @@ static s32 ixgbe_check_mac_link_82598(struct ixgbe_hw *hw, hw->phy.ops.read_reg(hw, 0xC00C, IXGBE_TWINAX_DEV, &adapt_comp_reg); if (link_up_wait_to_complete) { - for (i = 0; i < IXGBE_LINK_UP_TIME; i++) { + for (i = 0; i < hw->mac.max_link_up_time; i++) { if ((link_reg & 1) && ((adapt_comp_reg & 1) == 0)) { *link_up = TRUE; @@ -689,7 +689,7 @@ static s32 ixgbe_check_mac_link_82598(struct ixgbe_hw *hw, links_reg = IXGBE_READ_REG(hw, IXGBE_LINKS); if (link_up_wait_to_complete) { - for (i = 0; i < IXGBE_LINK_UP_TIME; i++) { + for (i = 0; i < hw->mac.max_link_up_time; i++) { if (links_reg & IXGBE_LINKS_UP) { *link_up = TRUE; break; diff --git a/sys/dev/ixgbe/ixgbe_82599.c b/sys/dev/ixgbe/ixgbe_82599.c index b38620f..109ed95 100644 --- a/sys/dev/ixgbe/ixgbe_82599.c +++ b/sys/dev/ixgbe/ixgbe_82599.c @@ -382,8 +382,8 @@ s32 ixgbe_init_ops_82599(struct ixgbe_hw *hw) mac->max_tx_queues = IXGBE_82599_MAX_TX_QUEUES; mac->max_msix_vectors = ixgbe_get_pcie_msix_count_generic(hw); - mac->arc_subsystem_valid = (IXGBE_READ_REG(hw, IXGBE_FWSM) & - IXGBE_FWSM_MODE_MASK) ? TRUE : FALSE; + mac->arc_subsystem_valid = !!(IXGBE_READ_REG(hw, IXGBE_FWSM_BY_MAC(hw)) + & IXGBE_FWSM_MODE_MASK); hw->mbx.ops.init_params = ixgbe_init_mbx_params_pf; @@ -1370,7 +1370,7 @@ s32 ixgbe_init_fdir_perfect_82599(struct ixgbe_hw *hw, u32 fdirctrl, * Continue setup of fdirctrl register bits: * Turn perfect match filtering on * Report hash in RSS field of Rx wb descriptor - * Initialize the drop queue + * Initialize the drop queue to queue 127 * Move the flexible bytes to use the ethertype - shift 6 words * Set the maximum length per hash bucket to 0xA filters * Send interrupt when 64 (0x4 * 16) filters are left @@ -1381,6 +1381,9 @@ s32 ixgbe_init_fdir_perfect_82599(struct ixgbe_hw *hw, u32 fdirctrl, (0x6 << IXGBE_FDIRCTRL_FLEX_SHIFT) | (0xA << IXGBE_FDIRCTRL_MAX_LENGTH_SHIFT) | (4 << IXGBE_FDIRCTRL_FULL_THRESH_SHIFT); + if ((hw->mac.type == ixgbe_mac_X550) || + (hw->mac.type == ixgbe_mac_X550EM_x)) + fdirctrl |= IXGBE_FDIRCTRL_DROP_NO_MATCH; if (cloud_mode) fdirctrl |=(IXGBE_FDIRCTRL_FILTERMODE_CLOUD << @@ -1392,6 +1395,39 @@ s32 ixgbe_init_fdir_perfect_82599(struct ixgbe_hw *hw, u32 fdirctrl, return IXGBE_SUCCESS; } +/** + * ixgbe_set_fdir_drop_queue_82599 - Set Flow Director drop queue + * @hw: pointer to hardware structure + * @dropqueue: Rx queue index used for the dropped packets + **/ +void ixgbe_set_fdir_drop_queue_82599(struct ixgbe_hw *hw, u8 dropqueue) +{ + u32 fdirctrl; + + DEBUGFUNC("ixgbe_set_fdir_drop_queue_82599"); + /* Clear init done bit and drop queue field */ + fdirctrl = IXGBE_READ_REG(hw, IXGBE_FDIRCTRL); + fdirctrl &= ~(IXGBE_FDIRCTRL_DROP_Q_MASK | IXGBE_FDIRCTRL_INIT_DONE); + + /* Set drop queue */ + fdirctrl |= (dropqueue << IXGBE_FDIRCTRL_DROP_Q_SHIFT); + if ((hw->mac.type == ixgbe_mac_X550) || + (hw->mac.type == ixgbe_mac_X550EM_x)) + fdirctrl |= IXGBE_FDIRCTRL_DROP_NO_MATCH; + + IXGBE_WRITE_REG(hw, IXGBE_FDIRCMD, + (IXGBE_READ_REG(hw, IXGBE_FDIRCMD) | + IXGBE_FDIRCMD_CLEARHT)); + IXGBE_WRITE_FLUSH(hw); + IXGBE_WRITE_REG(hw, IXGBE_FDIRCMD, + (IXGBE_READ_REG(hw, IXGBE_FDIRCMD) & + ~IXGBE_FDIRCMD_CLEARHT)); + IXGBE_WRITE_FLUSH(hw); + + /* write hashes and fdirctrl register, poll for completion */ + ixgbe_fdir_enable_82599(hw, fdirctrl); +} + /* * These defines allow us to quickly generate all of the necessary instructions * in the function below by simply calling out IXGBE_COMPUTE_SIG_HASH_ITERATION @@ -1492,16 +1528,15 @@ u32 ixgbe_atr_compute_sig_hash_82599(union ixgbe_atr_hash_dword input, * Note that the tunnel bit in input must not be set when the hardware * tunneling support does not exist. **/ -s32 ixgbe_fdir_add_signature_filter_82599(struct ixgbe_hw *hw, - union ixgbe_atr_hash_dword input, - union ixgbe_atr_hash_dword common, - u8 queue) +void ixgbe_fdir_add_signature_filter_82599(struct ixgbe_hw *hw, + union ixgbe_atr_hash_dword input, + union ixgbe_atr_hash_dword common, + u8 queue) { u64 fdirhashcmd; u8 flow_type; bool tunnel; u32 fdircmd; - s32 err; DEBUGFUNC("ixgbe_fdir_add_signature_filter_82599"); @@ -1523,7 +1558,7 @@ s32 ixgbe_fdir_add_signature_filter_82599(struct ixgbe_hw *hw, break; default: DEBUGOUT(" Error on flow type input\n"); - return IXGBE_ERR_CONFIG; + return; } /* configure FDIRCMD register */ @@ -1542,15 +1577,9 @@ s32 ixgbe_fdir_add_signature_filter_82599(struct ixgbe_hw *hw, fdirhashcmd |= ixgbe_atr_compute_sig_hash_82599(input, common); IXGBE_WRITE_REG64(hw, IXGBE_FDIRHASH, fdirhashcmd); - err = ixgbe_fdir_check_cmd_complete(hw, &fdircmd); - if (err) { - DEBUGOUT("Flow Director command did not complete!\n"); - return err; - } - DEBUGOUT2("Tx Queue=%x hash=%x\n", queue, (u32)fdirhashcmd); - return IXGBE_SUCCESS; + return; } #define IXGBE_COMPUTE_BKT_HASH_ITERATION(_n) \ diff --git a/sys/dev/ixgbe/ixgbe_api.c b/sys/dev/ixgbe/ixgbe_api.c index 9784e3c..0f4d973 100644 --- a/sys/dev/ixgbe/ixgbe_api.c +++ b/sys/dev/ixgbe/ixgbe_api.c @@ -35,8 +35,10 @@ #include "ixgbe_api.h" #include "ixgbe_common.h" +#define IXGBE_EMPTY_PARAM + static const u32 ixgbe_mvals_base[IXGBE_MVALS_IDX_LIMIT] = { - IXGBE_MVALS_INIT() + IXGBE_MVALS_INIT(IXGBE_EMPTY_PARAM) }; static const u32 ixgbe_mvals_X540[IXGBE_MVALS_IDX_LIMIT] = { @@ -113,6 +115,7 @@ s32 ixgbe_init_shared_code(struct ixgbe_hw *hw) status = IXGBE_ERR_DEVICE_NOT_SUPPORTED; break; } + hw->mac.max_link_up_time = IXGBE_LINK_UP_TIME; return status; } @@ -187,6 +190,7 @@ s32 ixgbe_set_mac_type(struct ixgbe_hw *hw) hw->mvals = ixgbe_mvals_X540; break; case IXGBE_DEV_ID_X550T: + case IXGBE_DEV_ID_X550T1: hw->mac.type = ixgbe_mac_X550; hw->mvals = ixgbe_mvals_X550; break; diff --git a/sys/dev/ixgbe/ixgbe_api.h b/sys/dev/ixgbe/ixgbe_api.h index 8c2c4a8..24d5070 100644 --- a/sys/dev/ixgbe/ixgbe_api.h +++ b/sys/dev/ixgbe/ixgbe_api.h @@ -148,10 +148,10 @@ s32 ixgbe_reinit_fdir_tables_82599(struct ixgbe_hw *hw); s32 ixgbe_init_fdir_signature_82599(struct ixgbe_hw *hw, u32 fdirctrl); s32 ixgbe_init_fdir_perfect_82599(struct ixgbe_hw *hw, u32 fdirctrl, bool cloud_mode); -s32 ixgbe_fdir_add_signature_filter_82599(struct ixgbe_hw *hw, - union ixgbe_atr_hash_dword input, - union ixgbe_atr_hash_dword common, - u8 queue); +void ixgbe_fdir_add_signature_filter_82599(struct ixgbe_hw *hw, + union ixgbe_atr_hash_dword input, + union ixgbe_atr_hash_dword common, + u8 queue); s32 ixgbe_fdir_set_input_mask_82599(struct ixgbe_hw *hw, union ixgbe_atr_input *input_mask, bool cloud_mode); s32 ixgbe_fdir_write_perfect_filter_82599(struct ixgbe_hw *hw, @@ -180,6 +180,7 @@ s32 ixgbe_read_i2c_combined_unlocked(struct ixgbe_hw *hw, u8 addr, u16 reg, u16 *val); s32 ixgbe_write_i2c_byte(struct ixgbe_hw *hw, u8 byte_offset, u8 dev_addr, u8 data); +void ixgbe_set_fdir_drop_queue_82599(struct ixgbe_hw *hw, u8 dropqueue); s32 ixgbe_write_i2c_byte_unlocked(struct ixgbe_hw *hw, u8 byte_offset, u8 dev_addr, u8 data); s32 ixgbe_write_i2c_combined(struct ixgbe_hw *hw, u8 addr, u16 reg, u16 val); diff --git a/sys/dev/ixgbe/ixgbe_common.c b/sys/dev/ixgbe/ixgbe_common.c index f0a0776..d67e680 100644 --- a/sys/dev/ixgbe/ixgbe_common.c +++ b/sys/dev/ixgbe/ixgbe_common.c @@ -70,7 +70,7 @@ s32 ixgbe_init_ops_generic(struct ixgbe_hw *hw) { struct ixgbe_eeprom_info *eeprom = &hw->eeprom; struct ixgbe_mac_info *mac = &hw->mac; - u32 eec = IXGBE_READ_REG(hw, IXGBE_EEC); + u32 eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); DEBUGFUNC("ixgbe_init_ops_generic"); @@ -188,6 +188,7 @@ bool ixgbe_device_supports_autoneg_fc(struct ixgbe_hw *hw) case IXGBE_DEV_ID_X540T1: case IXGBE_DEV_ID_X540_BYPASS: case IXGBE_DEV_ID_X550T: + case IXGBE_DEV_ID_X550T1: case IXGBE_DEV_ID_X550EM_X_10G_T: supported = TRUE; break; @@ -1038,7 +1039,7 @@ void ixgbe_set_lan_id_multi_port_pcie(struct ixgbe_hw *hw) bus->lan_id = bus->func; /* check for a port swap */ - reg = IXGBE_READ_REG(hw, IXGBE_FACTPS); + reg = IXGBE_READ_REG(hw, IXGBE_FACTPS_BY_MAC(hw)); if (reg & IXGBE_FACTPS_LFS) bus->func ^= 0x1; } @@ -1164,7 +1165,7 @@ s32 ixgbe_init_eeprom_params_generic(struct ixgbe_hw *hw) * Check for EEPROM present first. * If not present leave as none */ - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); if (eec & IXGBE_EEC_PRES) { eeprom->type = ixgbe_eeprom_spi; @@ -1725,14 +1726,14 @@ static s32 ixgbe_acquire_eeprom(struct ixgbe_hw *hw) status = IXGBE_ERR_SWFW_SYNC; if (status == IXGBE_SUCCESS) { - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); /* Request EEPROM Access */ eec |= IXGBE_EEC_REQ; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); for (i = 0; i < IXGBE_EEPROM_GRANT_ATTEMPTS; i++) { - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); if (eec & IXGBE_EEC_GNT) break; usec_delay(5); @@ -1741,7 +1742,7 @@ static s32 ixgbe_acquire_eeprom(struct ixgbe_hw *hw) /* Release if grant not acquired */ if (!(eec & IXGBE_EEC_GNT)) { eec &= ~IXGBE_EEC_REQ; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); DEBUGOUT("Could not acquire EEPROM grant\n"); hw->mac.ops.release_swfw_sync(hw, IXGBE_GSSR_EEP_SM); @@ -1752,7 +1753,7 @@ static s32 ixgbe_acquire_eeprom(struct ixgbe_hw *hw) if (status == IXGBE_SUCCESS) { /* Clear CS and SK */ eec &= ~(IXGBE_EEC_CS | IXGBE_EEC_SK); - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); } @@ -1782,7 +1783,7 @@ static s32 ixgbe_get_eeprom_semaphore(struct ixgbe_hw *hw) * If the SMBI bit is 0 when we read it, then the bit will be * set and we have the semaphore */ - swsm = IXGBE_READ_REG(hw, IXGBE_SWSM); + swsm = IXGBE_READ_REG(hw, IXGBE_SWSM_BY_MAC(hw)); if (!(swsm & IXGBE_SWSM_SMBI)) { status = IXGBE_SUCCESS; break; @@ -1807,7 +1808,7 @@ static s32 ixgbe_get_eeprom_semaphore(struct ixgbe_hw *hw) * If the SMBI bit is 0 when we read it, then the bit will be * set and we have the semaphore */ - swsm = IXGBE_READ_REG(hw, IXGBE_SWSM); + swsm = IXGBE_READ_REG(hw, IXGBE_SWSM_BY_MAC(hw)); if (!(swsm & IXGBE_SWSM_SMBI)) status = IXGBE_SUCCESS; } @@ -1815,17 +1816,17 @@ static s32 ixgbe_get_eeprom_semaphore(struct ixgbe_hw *hw) /* Now get the semaphore between SW/FW through the SWESMBI bit */ if (status == IXGBE_SUCCESS) { for (i = 0; i < timeout; i++) { - swsm = IXGBE_READ_REG(hw, IXGBE_SWSM); + swsm = IXGBE_READ_REG(hw, IXGBE_SWSM_BY_MAC(hw)); /* Set the SW EEPROM semaphore bit to request access */ swsm |= IXGBE_SWSM_SWESMBI; - IXGBE_WRITE_REG(hw, IXGBE_SWSM, swsm); + IXGBE_WRITE_REG(hw, IXGBE_SWSM_BY_MAC(hw), swsm); /* * If we set the bit successfully then we got the * semaphore. */ - swsm = IXGBE_READ_REG(hw, IXGBE_SWSM); + swsm = IXGBE_READ_REG(hw, IXGBE_SWSM_BY_MAC(hw)); if (swsm & IXGBE_SWSM_SWESMBI) break; @@ -1922,15 +1923,15 @@ static void ixgbe_standby_eeprom(struct ixgbe_hw *hw) DEBUGFUNC("ixgbe_standby_eeprom"); - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); /* Toggle CS to flush commands */ eec |= IXGBE_EEC_CS; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); eec &= ~IXGBE_EEC_CS; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); } @@ -1950,7 +1951,7 @@ static void ixgbe_shift_out_eeprom_bits(struct ixgbe_hw *hw, u16 data, DEBUGFUNC("ixgbe_shift_out_eeprom_bits"); - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); /* * Mask is used to shift "count" bits of "data" out to the EEPROM @@ -1971,7 +1972,7 @@ static void ixgbe_shift_out_eeprom_bits(struct ixgbe_hw *hw, u16 data, else eec &= ~IXGBE_EEC_DI; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); @@ -1988,7 +1989,7 @@ static void ixgbe_shift_out_eeprom_bits(struct ixgbe_hw *hw, u16 data, /* We leave the "DI" bit set to "0" when we leave this routine. */ eec &= ~IXGBE_EEC_DI; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); IXGBE_WRITE_FLUSH(hw); } @@ -2011,7 +2012,7 @@ static u16 ixgbe_shift_in_eeprom_bits(struct ixgbe_hw *hw, u16 count) * the value of the "DO" bit. During this "shifting in" process the * "DI" bit should always be clear. */ - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); eec &= ~(IXGBE_EEC_DO | IXGBE_EEC_DI); @@ -2019,7 +2020,7 @@ static u16 ixgbe_shift_in_eeprom_bits(struct ixgbe_hw *hw, u16 count) data = data << 1; ixgbe_raise_eeprom_clk(hw, &eec); - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); eec &= ~(IXGBE_EEC_DI); if (eec & IXGBE_EEC_DO) @@ -2045,7 +2046,7 @@ static void ixgbe_raise_eeprom_clk(struct ixgbe_hw *hw, u32 *eec) * (setting the SK bit), then delay */ *eec = *eec | IXGBE_EEC_SK; - IXGBE_WRITE_REG(hw, IXGBE_EEC, *eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), *eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); } @@ -2064,7 +2065,7 @@ static void ixgbe_lower_eeprom_clk(struct ixgbe_hw *hw, u32 *eec) * delay */ *eec = *eec & ~IXGBE_EEC_SK; - IXGBE_WRITE_REG(hw, IXGBE_EEC, *eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), *eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); } @@ -2079,19 +2080,19 @@ static void ixgbe_release_eeprom(struct ixgbe_hw *hw) DEBUGFUNC("ixgbe_release_eeprom"); - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); eec |= IXGBE_EEC_CS; /* Pull CS high */ eec &= ~IXGBE_EEC_SK; /* Lower SCK */ - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); IXGBE_WRITE_FLUSH(hw); usec_delay(1); /* Stop requesting EEPROM access */ eec &= ~IXGBE_EEC_REQ; - IXGBE_WRITE_REG(hw, IXGBE_EEC, eec); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), eec); hw->mac.ops.release_swfw_sync(hw, IXGBE_GSSR_EEP_SM); @@ -3147,6 +3148,9 @@ s32 ixgbe_disable_pcie_master(struct ixgbe_hw *hw) DEBUGOUT("GIO Master Disable bit didn't clear - requesting resets\n"); hw->mac.flags |= IXGBE_FLAGS_DOUBLE_RESET_REQUIRED; + if (hw->mac.type >= ixgbe_mac_X550) + goto out; + /* * Before proceeding, make sure that the PCIe block does not have * transactions pending. @@ -4069,7 +4073,7 @@ s32 ixgbe_check_mac_link_generic(struct ixgbe_hw *hw, ixgbe_link_speed *speed, } if (link_up_wait_to_complete) { - for (i = 0; i < IXGBE_LINK_UP_TIME; i++) { + for (i = 0; i < hw->mac.max_link_up_time; i++) { if (links_reg & IXGBE_LINKS_UP) { *link_up = TRUE; break; @@ -4715,7 +4719,7 @@ bool ixgbe_mng_present(struct ixgbe_hw *hw) if (hw->mac.type < ixgbe_mac_82599EB) return FALSE; - fwsm = IXGBE_READ_REG(hw, IXGBE_FWSM); + fwsm = IXGBE_READ_REG(hw, IXGBE_FWSM_BY_MAC(hw)); fwsm &= IXGBE_FWSM_MODE_MASK; return fwsm == IXGBE_FWSM_FW_MODE_PT; } @@ -4730,7 +4734,7 @@ bool ixgbe_mng_enabled(struct ixgbe_hw *hw) { u32 fwsm, manc, factps; - fwsm = IXGBE_READ_REG(hw, IXGBE_FWSM); + fwsm = IXGBE_READ_REG(hw, IXGBE_FWSM_BY_MAC(hw)); if ((fwsm & IXGBE_FWSM_MODE_MASK) != IXGBE_FWSM_FW_MODE_PT) return FALSE; @@ -4739,7 +4743,7 @@ bool ixgbe_mng_enabled(struct ixgbe_hw *hw) return FALSE; if (hw->mac.type <= ixgbe_mac_X540) { - factps = IXGBE_READ_REG(hw, IXGBE_FACTPS); + factps = IXGBE_READ_REG(hw, IXGBE_FACTPS_BY_MAC(hw)); if (factps & IXGBE_FACTPS_MNGCG) return FALSE; } diff --git a/sys/dev/ixgbe/ixgbe_dcb.c b/sys/dev/ixgbe/ixgbe_dcb.c index 3659d17..437d336 100644 --- a/sys/dev/ixgbe/ixgbe_dcb.c +++ b/sys/dev/ixgbe/ixgbe_dcb.c @@ -148,6 +148,11 @@ s32 ixgbe_dcb_calculate_tc_credits_cee(struct ixgbe_hw *hw, /* Calculate credit refill ratio using multiplier */ credit_refill = min(link_percentage * min_multiplier, (u32)IXGBE_DCB_MAX_CREDIT_REFILL); + + /* Refill at least minimum credit */ + if (credit_refill < min_credit) + credit_refill = min_credit; + p->data_credits_refill = (u16)credit_refill; /* Calculate maximum credit for the TC */ @@ -158,7 +163,7 @@ s32 ixgbe_dcb_calculate_tc_credits_cee(struct ixgbe_hw *hw, * of a TC is too small, the maximum credit may not be * enough to send out a jumbo frame in data plane arbitration. */ - if (credit_max && (credit_max < min_credit)) + if (credit_max < min_credit) credit_max = min_credit; if (direction == IXGBE_DCB_TX_CONFIG) { diff --git a/sys/dev/ixgbe/LICENSE b/sys/dev/ixgbe/ixgbe_osdep.c index 394b094..6a4e2fc 100644 --- a/sys/dev/ixgbe/LICENSE +++ b/sys/dev/ixgbe/ixgbe_osdep.c @@ -31,3 +31,58 @@ ******************************************************************************/ /*$FreeBSD$*/ + +#include "ixgbe_osdep.h" +#include "ixgbe.h" + +inline device_t +ixgbe_dev_from_hw(struct ixgbe_hw *hw) +{ + return ((struct adapter *)hw->back)->dev; +} + +inline u16 +ixgbe_read_pci_cfg(struct ixgbe_hw *hw, u32 reg) +{ + return pci_read_config(((struct adapter *)hw->back)->dev, + reg, 2); +} + +inline void +ixgbe_write_pci_cfg(struct ixgbe_hw *hw, u32 reg, u16 value) +{ + pci_write_config(((struct adapter *)hw->back)->dev, + reg, value, 2); +} + +inline u32 +ixgbe_read_reg(struct ixgbe_hw *hw, u32 reg) +{ + return bus_space_read_4(((struct adapter *)hw->back)->osdep.mem_bus_space_tag, + ((struct adapter *)hw->back)->osdep.mem_bus_space_handle, + reg); +} + +inline void +ixgbe_write_reg(struct ixgbe_hw *hw, u32 reg, u32 val) +{ + bus_space_write_4(((struct adapter *)hw->back)->osdep.mem_bus_space_tag, + ((struct adapter *)hw->back)->osdep.mem_bus_space_handle, + reg, val); +} + +inline u32 +ixgbe_read_reg_array(struct ixgbe_hw *hw, u32 reg, u32 offset) +{ + return bus_space_read_4(((struct adapter *)hw->back)->osdep.mem_bus_space_tag, + ((struct adapter *)hw->back)->osdep.mem_bus_space_handle, + reg + (offset << 2)); +} + +inline void +ixgbe_write_reg_array(struct ixgbe_hw *hw, u32 reg, u32 offset, u32 val) +{ + bus_space_write_4(((struct adapter *)hw->back)->osdep.mem_bus_space_tag, + ((struct adapter *)hw->back)->osdep.mem_bus_space_handle, + reg + (offset << 2), val); +} diff --git a/sys/dev/ixgbe/ixgbe_osdep.h b/sys/dev/ixgbe/ixgbe_osdep.h index 95f6ed5c..b767e73 100644 --- a/sys/dev/ixgbe/ixgbe_osdep.h +++ b/sys/dev/ixgbe/ixgbe_osdep.h @@ -61,7 +61,7 @@ #define usec_delay(x) DELAY(x) #define msec_delay(x) DELAY(1000*(x)) -#define DBG 0 +#define DBG 0 #define MSGOUT(S, A, B) printf(S "\n", A, B) #define DEBUGFUNC(F) DEBUGOUT(F); #if DBG @@ -165,7 +165,7 @@ void prefetch(void *x) * non-overlapping regions and 32-byte padding on both src and dst. */ static __inline int -ixgbe_bcopy(void *_src, void *_dst, int l) +ixgbe_bcopy(void *restrict _src, void *restrict _dst, int l) { uint64_t *src = _src; uint64_t *dst = _dst; @@ -183,11 +183,13 @@ struct ixgbe_osdep { bus_space_tag_t mem_bus_space_tag; bus_space_handle_t mem_bus_space_handle; - struct device *dev; }; -/* These routines are needed by the shared code */ +/* These routines need struct ixgbe_hw declared */ struct ixgbe_hw; +device_t ixgbe_dev_from_hw(struct ixgbe_hw *hw); + +/* These routines are needed by the shared code */ extern u16 ixgbe_read_pci_cfg(struct ixgbe_hw *, u32); #define IXGBE_READ_PCIE_WORD ixgbe_read_pci_cfg @@ -196,26 +198,18 @@ extern void ixgbe_write_pci_cfg(struct ixgbe_hw *, u32, u16); #define IXGBE_WRITE_FLUSH(a) IXGBE_READ_REG(a, IXGBE_STATUS) -#define IXGBE_READ_REG(a, reg) (\ - bus_space_read_4( ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_tag, \ - ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_handle, \ - reg)) - -#define IXGBE_WRITE_REG(a, reg, value) (\ - bus_space_write_4( ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_tag, \ - ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_handle, \ - reg, value)) - +extern u32 ixgbe_read_reg(struct ixgbe_hw *, u32); +#define IXGBE_READ_REG(a, reg) ixgbe_read_reg(a, reg) -#define IXGBE_READ_REG_ARRAY(a, reg, offset) (\ - bus_space_read_4( ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_tag, \ - ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_handle, \ - (reg + ((offset) << 2)))) +extern void ixgbe_write_reg(struct ixgbe_hw *, u32, u32); +#define IXGBE_WRITE_REG(a, reg, val) ixgbe_write_reg(a, reg, val) -#define IXGBE_WRITE_REG_ARRAY(a, reg, offset, value) (\ - bus_space_write_4( ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_tag, \ - ((struct ixgbe_osdep *)(a)->back)->mem_bus_space_handle, \ - (reg + ((offset) << 2)), value)) +extern u32 ixgbe_read_reg_array(struct ixgbe_hw *, u32, u32); +#define IXGBE_READ_REG_ARRAY(a, reg, offset) \ + ixgbe_read_reg_array(a, reg, offset) +extern void ixgbe_write_reg_array(struct ixgbe_hw *, u32, u32, u32); +#define IXGBE_WRITE_REG_ARRAY(a, reg, offset, val) \ + ixgbe_write_reg_array(a, reg, offset, val) #endif /* _IXGBE_OS_H_ */ diff --git a/sys/dev/ixgbe/ixgbe_phy.c b/sys/dev/ixgbe/ixgbe_phy.c index 88206c7..6bff172 100644 --- a/sys/dev/ixgbe/ixgbe_phy.c +++ b/sys/dev/ixgbe/ixgbe_phy.c @@ -508,7 +508,9 @@ enum ixgbe_phy_type ixgbe_get_phy_type_from_id(u32 phy_id) case TN1010_PHY_ID: phy_type = ixgbe_phy_tn; break; - case X550_PHY_ID: + case X550_PHY_ID1: + case X550_PHY_ID2: + case X550_PHY_ID3: case X540_PHY_ID: phy_type = ixgbe_phy_aq; break; @@ -830,7 +832,7 @@ s32 ixgbe_setup_phy_link_generic(struct ixgbe_hw *hw) if (hw->mac.type == ixgbe_mac_X550) { if (speed & IXGBE_LINK_SPEED_5GB_FULL) { - /* Set or unset auto-negotiation 1G advertisement */ + /* Set or unset auto-negotiation 5G advertisement */ hw->phy.ops.read_reg(hw, IXGBE_MII_AUTONEG_VENDOR_PROVISION_1_REG, IXGBE_MDIO_AUTO_NEG_DEV_TYPE, @@ -848,7 +850,7 @@ s32 ixgbe_setup_phy_link_generic(struct ixgbe_hw *hw) } if (speed & IXGBE_LINK_SPEED_2_5GB_FULL) { - /* Set or unset auto-negotiation 1G advertisement */ + /* Set or unset auto-negotiation 2.5G advertisement */ hw->phy.ops.read_reg(hw, IXGBE_MII_AUTONEG_VENDOR_PROVISION_1_REG, IXGBE_MDIO_AUTO_NEG_DEV_TYPE, @@ -950,54 +952,70 @@ s32 ixgbe_setup_phy_link_speed_generic(struct ixgbe_hw *hw, hw->phy.autoneg_advertised |= IXGBE_LINK_SPEED_100_FULL; /* Setup link based on the new speed settings */ - hw->phy.ops.setup_link(hw); + ixgbe_setup_phy_link(hw); return IXGBE_SUCCESS; } /** - * ixgbe_get_copper_link_capabilities_generic - Determines link capabilities - * @hw: pointer to hardware structure - * @speed: pointer to link speed - * @autoneg: boolean auto-negotiation value + * ixgbe_get_copper_speeds_supported - Get copper link speeds from phy + * @hw: pointer to hardware structure * - * Determines the supported link capabilities by reading the PHY auto - * negotiation register. + * Determines the supported link capabilities by reading the PHY auto + * negotiation register. **/ -s32 ixgbe_get_copper_link_capabilities_generic(struct ixgbe_hw *hw, - ixgbe_link_speed *speed, - bool *autoneg) +static s32 ixgbe_get_copper_speeds_supported(struct ixgbe_hw *hw) { s32 status; u16 speed_ability; - DEBUGFUNC("ixgbe_get_copper_link_capabilities_generic"); - - *speed = 0; - *autoneg = TRUE; - status = hw->phy.ops.read_reg(hw, IXGBE_MDIO_PHY_SPEED_ABILITY, IXGBE_MDIO_PMA_PMD_DEV_TYPE, &speed_ability); + if (status) + return status; - if (status == IXGBE_SUCCESS) { - if (speed_ability & IXGBE_MDIO_PHY_SPEED_10G) - *speed |= IXGBE_LINK_SPEED_10GB_FULL; - if (speed_ability & IXGBE_MDIO_PHY_SPEED_1G) - *speed |= IXGBE_LINK_SPEED_1GB_FULL; - if (speed_ability & IXGBE_MDIO_PHY_SPEED_100M) - *speed |= IXGBE_LINK_SPEED_100_FULL; + if (speed_ability & IXGBE_MDIO_PHY_SPEED_10G) + hw->phy.speeds_supported |= IXGBE_LINK_SPEED_10GB_FULL; + if (speed_ability & IXGBE_MDIO_PHY_SPEED_1G) + hw->phy.speeds_supported |= IXGBE_LINK_SPEED_1GB_FULL; + if (speed_ability & IXGBE_MDIO_PHY_SPEED_100M) + hw->phy.speeds_supported |= IXGBE_LINK_SPEED_100_FULL; + + switch (hw->mac.type) { + case ixgbe_mac_X550: + hw->phy.speeds_supported |= IXGBE_LINK_SPEED_2_5GB_FULL; + hw->phy.speeds_supported |= IXGBE_LINK_SPEED_5GB_FULL; + break; + case ixgbe_mac_X550EM_x: + hw->phy.speeds_supported &= ~IXGBE_LINK_SPEED_100_FULL; + break; + default: + break; } - /* Internal PHY does not support 100 Mbps */ - if (hw->mac.type == ixgbe_mac_X550EM_x) - *speed &= ~IXGBE_LINK_SPEED_100_FULL; + return status; +} - if (hw->mac.type == ixgbe_mac_X550) { - *speed |= IXGBE_LINK_SPEED_2_5GB_FULL; - *speed |= IXGBE_LINK_SPEED_5GB_FULL; - } +/** + * ixgbe_get_copper_link_capabilities_generic - Determines link capabilities + * @hw: pointer to hardware structure + * @speed: pointer to link speed + * @autoneg: boolean auto-negotiation value + **/ +s32 ixgbe_get_copper_link_capabilities_generic(struct ixgbe_hw *hw, + ixgbe_link_speed *speed, + bool *autoneg) +{ + s32 status = IXGBE_SUCCESS; + + DEBUGFUNC("ixgbe_get_copper_link_capabilities_generic"); + + *autoneg = TRUE; + if (!hw->phy.speeds_supported) + status = ixgbe_get_copper_speeds_supported(hw); + *speed = hw->phy.speeds_supported; return status; } diff --git a/sys/dev/ixgbe/ixgbe_phy.h b/sys/dev/ixgbe/ixgbe_phy.h index fad31bd..3713e28 100644 --- a/sys/dev/ixgbe/ixgbe_phy.h +++ b/sys/dev/ixgbe/ixgbe_phy.h @@ -92,16 +92,23 @@ #define IXGBE_CS4227_GLOBAL_ID_LSB 0 #define IXGBE_CS4227_SCRATCH 2 #define IXGBE_CS4227_GLOBAL_ID_VALUE 0x03E5 -#define IXGBE_CS4227_SCRATCH_VALUE 0x5aa5 -#define IXGBE_CS4227_RETRIES 5 +#define IXGBE_CS4227_RESET_PENDING 0x1357 +#define IXGBE_CS4227_RESET_COMPLETE 0x5AA5 +#define IXGBE_CS4227_RETRIES 15 +#define IXGBE_CS4227_EFUSE_STATUS 0x0181 #define IXGBE_CS4227_LINE_SPARE22_MSB 0x12AD /* Reg to program speed */ #define IXGBE_CS4227_LINE_SPARE24_LSB 0x12B0 /* Reg to program EDC */ #define IXGBE_CS4227_HOST_SPARE22_MSB 0x1AAD /* Reg to program speed */ #define IXGBE_CS4227_HOST_SPARE24_LSB 0x1AB0 /* Reg to program EDC */ +#define IXGBE_CS4227_EEPROM_STATUS 0x5001 +#define IXGBE_CS4227_EEPROM_LOAD_OK 0x0001 +#define IXGBE_CS4227_SPEED_1G 0x8000 +#define IXGBE_CS4227_SPEED_10G 0 #define IXGBE_CS4227_EDC_MODE_CX1 0x0002 #define IXGBE_CS4227_EDC_MODE_SR 0x0004 +#define IXGBE_CS4227_EDC_MODE_DIAG 0x0008 #define IXGBE_CS4227_RESET_HOLD 500 /* microseconds */ -#define IXGBE_CS4227_RESET_DELAY 500 /* milliseconds */ +#define IXGBE_CS4227_RESET_DELAY 450 /* milliseconds */ #define IXGBE_CS4227_CHECK_DELAY 30 /* milliseconds */ #define IXGBE_PE 0xE0 /* Port expander address */ #define IXGBE_PE_OUTPUT 1 /* Output register offset */ diff --git a/sys/dev/ixgbe/ixgbe_type.h b/sys/dev/ixgbe/ixgbe_type.h index 2a53952..d76cde2 100644 --- a/sys/dev/ixgbe/ixgbe_type.h +++ b/sys/dev/ixgbe/ixgbe_type.h @@ -131,6 +131,7 @@ #define IXGBE_DEV_ID_X540_BYPASS 0x155C #define IXGBE_DEV_ID_X540T1 0x1560 #define IXGBE_DEV_ID_X550T 0x1563 +#define IXGBE_DEV_ID_X550T1 0x15D1 #define IXGBE_DEV_ID_X550EM_X_KX4 0x15AA #define IXGBE_DEV_ID_X550EM_X_KR 0x15AB #define IXGBE_DEV_ID_X550EM_X_SFP 0x15AC @@ -1544,7 +1545,9 @@ struct ixgbe_dmac_config { #define TN1010_PHY_ID 0x00A19410 #define TNX_FW_REV 0xB #define X540_PHY_ID 0x01540200 -#define X550_PHY_ID 0x01540220 +#define X550_PHY_ID1 0x01540220 +#define X550_PHY_ID2 0x01540223 +#define X550_PHY_ID3 0x01540221 #define X557_PHY_ID 0x01540240 #define AQ_FW_REV 0x20 #define QT2022_PHY_ID 0x0043A400 @@ -1937,6 +1940,7 @@ enum { * FIP (0x8914): Filter 4 * LLDP (0x88CC): Filter 5 * LACP (0x8809): Filter 6 + * FC (0x8808): Filter 7 */ #define IXGBE_ETQF_FILTER_EAPOL 0 #define IXGBE_ETQF_FILTER_FCOE 2 @@ -1944,6 +1948,7 @@ enum { #define IXGBE_ETQF_FILTER_FIP 4 #define IXGBE_ETQF_FILTER_LLDP 5 #define IXGBE_ETQF_FILTER_LACP 6 +#define IXGBE_ETQF_FILTER_FC 7 /* VLAN Control Bit Masks */ #define IXGBE_VLNCTRL_VET 0x0000FFFF /* bits 0-15 */ #define IXGBE_VLNCTRL_CFI 0x10000000 /* bit 28 */ @@ -2803,7 +2808,9 @@ enum ixgbe_fdir_pballoc_type { #define IXGBE_FDIRCTRL_REPORT_STATUS 0x00000020 #define IXGBE_FDIRCTRL_REPORT_STATUS_ALWAYS 0x00000080 #define IXGBE_FDIRCTRL_DROP_Q_SHIFT 8 +#define IXGBE_FDIRCTRL_DROP_Q_MASK 0x00007F00 #define IXGBE_FDIRCTRL_FLEX_SHIFT 16 +#define IXGBE_FDIRCTRL_DROP_NO_MATCH 0x00008000 #define IXGBE_FDIRCTRL_FILTERMODE_SHIFT 21 #define IXGBE_FDIRCTRL_FILTERMODE_MACVLAN 0x0001 /* bit 23:21, 001b */ #define IXGBE_FDIRCTRL_FILTERMODE_CLOUD 0x0002 /* bit 23:21, 010b */ @@ -2907,6 +2914,11 @@ enum ixgbe_fdir_pballoc_type { #define FW_DISABLE_RXEN_CMD 0xDE #define FW_DISABLE_RXEN_LEN 0x1 #define FW_PHY_MGMT_REQ_CMD 0x20 +#define FW_INT_PHY_REQ_CMD 0xB +#define FW_INT_PHY_REQ_LEN 10 +#define FW_INT_PHY_REQ_READ 0 +#define FW_INT_PHY_REQ_WRITE 1 + /* Host Interface Command Structures */ struct ixgbe_hic_hdr { @@ -2975,6 +2987,21 @@ struct ixgbe_hic_disable_rxen { u16 pad3; }; +struct ixgbe_hic_internal_phy_req { + struct ixgbe_hic_hdr hdr; + u8 port_number; + u8 command_type; + u16 address; + u16 rsv1; + u32 write_data; + u16 pad; +}; + +struct ixgbe_hic_internal_phy_resp { + struct ixgbe_hic_hdr hdr; + u32 read_data; +}; + /* Transmit Descriptor - Legacy */ struct ixgbe_legacy_tx_desc { @@ -3310,6 +3337,7 @@ union ixgbe_atr_hash_dword { IXGBE_CAT(SRAMREL, m), \ IXGBE_CAT(FACTPS, m), \ IXGBE_CAT(SWSM, m), \ + IXGBE_CAT(SWFW_SYNC, m), \ IXGBE_CAT(FWSM, m), \ IXGBE_CAT(SDP0_GPIEN, m), \ IXGBE_CAT(SDP1_GPIEN, m), \ @@ -3792,6 +3820,7 @@ struct ixgbe_mac_info { u8 flags; struct ixgbe_dmac_config dmac_config; bool set_lben; + u32 max_link_up_time; }; struct ixgbe_phy_info { @@ -3806,6 +3835,7 @@ struct ixgbe_phy_info { u32 phy_semaphore_mask; bool reset_disable; ixgbe_autoneg_advertised autoneg_advertised; + ixgbe_link_speed speeds_supported; enum ixgbe_smart_speed smart_speed; bool smart_speed_active; bool multispeed_fiber; @@ -3918,15 +3948,15 @@ struct ixgbe_hw { #define IXGBE_FUSES0_300MHZ (1 << 5) #define IXGBE_FUSES0_REV1 (1 << 6) -#define IXGBE_KRM_PORT_CAR_GEN_CTRL(P) ((P == 0) ? (0x4010) : (0x8010)) -#define IXGBE_KRM_LINK_CTRL_1(P) ((P == 0) ? (0x420C) : (0x820C)) -#define IXGBE_KRM_AN_CNTL_1(P) ((P == 0) ? (0x422C) : (0x822C)) -#define IXGBE_KRM_DSP_TXFFE_STATE_4(P) ((P == 0) ? (0x4634) : (0x8634)) -#define IXGBE_KRM_DSP_TXFFE_STATE_5(P) ((P == 0) ? (0x4638) : (0x8638)) -#define IXGBE_KRM_RX_TRN_LINKUP_CTRL(P) ((P == 0) ? (0x4B00) : (0x8B00)) -#define IXGBE_KRM_PMD_DFX_BURNIN(P) ((P == 0) ? (0x4E00) : (0x8E00)) -#define IXGBE_KRM_TX_COEFF_CTRL_1(P) ((P == 0) ? (0x5520) : (0x9520)) -#define IXGBE_KRM_RX_ANA_CTL(P) ((P == 0) ? (0x5A00) : (0x9A00)) +#define IXGBE_KRM_PORT_CAR_GEN_CTRL(P) ((P) ? 0x8010 : 0x4010) +#define IXGBE_KRM_LINK_CTRL_1(P) ((P) ? 0x820C : 0x420C) +#define IXGBE_KRM_AN_CNTL_1(P) ((P) ? 0x822C : 0x422C) +#define IXGBE_KRM_DSP_TXFFE_STATE_4(P) ((P) ? 0x8634 : 0x4634) +#define IXGBE_KRM_DSP_TXFFE_STATE_5(P) ((P) ? 0x8638 : 0x4638) +#define IXGBE_KRM_RX_TRN_LINKUP_CTRL(P) ((P) ? 0x8B00 : 0x4B00) +#define IXGBE_KRM_PMD_DFX_BURNIN(P) ((P) ? 0x8E00 : 0x4E00) +#define IXGBE_KRM_TX_COEFF_CTRL_1(P) ((P) ? 0x9520 : 0x5520) +#define IXGBE_KRM_RX_ANA_CTL(P) ((P) ? 0x9A00 : 0x5A00) #define IXGBE_KRM_PORT_CAR_GEN_CTRL_NELB_32B (1 << 9) #define IXGBE_KRM_PORT_CAR_GEN_CTRL_NELB_KRPCS (1 << 11) @@ -3960,15 +3990,6 @@ struct ixgbe_hw { #define IXGBE_KRM_TX_COEFF_CTRL_1_CZERO_EN (1 << 3) #define IXGBE_KRM_TX_COEFF_CTRL_1_OVRRD_EN (1 << 31) -#define IXGBE_KX4_LINK_CNTL_1 0x4C -#define IXGBE_KX4_LINK_CNTL_1_TETH_AN_CAP_KX (1 << 16) -#define IXGBE_KX4_LINK_CNTL_1_TETH_AN_CAP_KX4 (1 << 17) -#define IXGBE_KX4_LINK_CNTL_1_TETH_EEE_CAP_KX (1 << 24) -#define IXGBE_KX4_LINK_CNTL_1_TETH_EEE_CAP_KX4 (1 << 25) -#define IXGBE_KX4_LINK_CNTL_1_TETH_AN_ENABLE (1 << 29) -#define IXGBE_KX4_LINK_CNTL_1_TETH_FORCE_LINK_UP (1 << 30) -#define IXGBE_KX4_LINK_CNTL_1_TETH_AN_RESTART (1 << 31) - #define IXGBE_SB_IOSF_INDIRECT_CTRL 0x00011144 #define IXGBE_SB_IOSF_INDIRECT_DATA 0x00011148 @@ -3985,8 +4006,6 @@ struct ixgbe_hw { #define IXGBE_SB_IOSF_CTRL_BUSY_SHIFT 31 #define IXGBE_SB_IOSF_CTRL_BUSY (1 << IXGBE_SB_IOSF_CTRL_BUSY_SHIFT) #define IXGBE_SB_IOSF_TARGET_KR_PHY 0 -#define IXGBE_SB_IOSF_TARGET_KX4_PHY 1 -#define IXGBE_SB_IOSF_TARGET_KX4_PCS 2 #define IXGBE_NW_MNG_IF_SEL 0x00011178 #define IXGBE_NW_MNG_IF_SEL_INT_PHY_MODE (1 << 24) diff --git a/sys/dev/ixgbe/ixgbe_vf.c b/sys/dev/ixgbe/ixgbe_vf.c index a00b8be..2ce4d32 100644 --- a/sys/dev/ixgbe/ixgbe_vf.c +++ b/sys/dev/ixgbe/ixgbe_vf.c @@ -225,8 +225,6 @@ s32 ixgbe_reset_hw_vf(struct ixgbe_hw *hw) if (ret_val) return ret_val; - msgbuf[0] &= ~IXGBE_VT_MSGTYPE_CTS; - if (msgbuf[0] != (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_ACK) && msgbuf[0] != (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_NACK)) return IXGBE_ERR_INVALID_MAC_ADDR; diff --git a/sys/dev/ixgbe/ixgbe_x540.c b/sys/dev/ixgbe/ixgbe_x540.c index ddf0674..96478ea 100644 --- a/sys/dev/ixgbe/ixgbe_x540.c +++ b/sys/dev/ixgbe/ixgbe_x540.c @@ -138,8 +138,8 @@ s32 ixgbe_init_ops_X540(struct ixgbe_hw *hw) * ARC supported; valid only if manageability features are * enabled. */ - mac->arc_subsystem_valid = (IXGBE_READ_REG(hw, IXGBE_FWSM) & - IXGBE_FWSM_MODE_MASK) ? TRUE : FALSE; + mac->arc_subsystem_valid = !!(IXGBE_READ_REG(hw, IXGBE_FWSM_BY_MAC(hw)) + & IXGBE_FWSM_MODE_MASK); hw->mbx.ops.init_params = ixgbe_init_mbx_params_pf; @@ -356,7 +356,7 @@ s32 ixgbe_init_eeprom_params_X540(struct ixgbe_hw *hw) eeprom->semaphore_delay = 10; eeprom->type = ixgbe_flash; - eec = IXGBE_READ_REG(hw, IXGBE_EEC); + eec = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); eeprom_size = (u16)((eec & IXGBE_EEC_SIZE) >> IXGBE_EEC_SIZE_SHIFT); eeprom->word_size = 1 << (eeprom_size + @@ -681,8 +681,8 @@ s32 ixgbe_update_flash_X540(struct ixgbe_hw *hw) goto out; } - flup = IXGBE_READ_REG(hw, IXGBE_EEC) | IXGBE_EEC_FLUP; - IXGBE_WRITE_REG(hw, IXGBE_EEC, flup); + flup = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)) | IXGBE_EEC_FLUP; + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), flup); status = ixgbe_poll_flash_update_done_X540(hw); if (status == IXGBE_SUCCESS) @@ -691,11 +691,11 @@ s32 ixgbe_update_flash_X540(struct ixgbe_hw *hw) DEBUGOUT("Flash update time out\n"); if (hw->mac.type == ixgbe_mac_X540 && hw->revision_id == 0) { - flup = IXGBE_READ_REG(hw, IXGBE_EEC); + flup = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); if (flup & IXGBE_EEC_SEC1VAL) { flup |= IXGBE_EEC_FLUP; - IXGBE_WRITE_REG(hw, IXGBE_EEC, flup); + IXGBE_WRITE_REG(hw, IXGBE_EEC_BY_MAC(hw), flup); } status = ixgbe_poll_flash_update_done_X540(hw); @@ -724,7 +724,7 @@ static s32 ixgbe_poll_flash_update_done_X540(struct ixgbe_hw *hw) DEBUGFUNC("ixgbe_poll_flash_update_done_X540"); for (i = 0; i < IXGBE_FLUDONE_ATTEMPTS; i++) { - reg = IXGBE_READ_REG(hw, IXGBE_EEC); + reg = IXGBE_READ_REG(hw, IXGBE_EEC_BY_MAC(hw)); if (reg & IXGBE_EEC_FLUDONE) { status = IXGBE_SUCCESS; break; @@ -775,10 +775,11 @@ s32 ixgbe_acquire_swfw_sync_X540(struct ixgbe_hw *hw, u32 mask) if (ixgbe_get_swfw_sync_semaphore(hw)) return IXGBE_ERR_SWFW_SYNC; - swfw_sync = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC); + swfw_sync = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw)); if (!(swfw_sync & (fwmask | swmask | hwmask))) { swfw_sync |= swmask; - IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC, swfw_sync); + IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw), + swfw_sync); ixgbe_release_swfw_sync_semaphore(hw); msec_delay(5); return IXGBE_SUCCESS; @@ -805,10 +806,10 @@ s32 ixgbe_acquire_swfw_sync_X540(struct ixgbe_hw *hw, u32 mask) */ if (ixgbe_get_swfw_sync_semaphore(hw)) return IXGBE_ERR_SWFW_SYNC; - swfw_sync = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC); + swfw_sync = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw)); if (swfw_sync & (fwmask | hwmask)) { swfw_sync |= swmask; - IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC, swfw_sync); + IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw), swfw_sync); ixgbe_release_swfw_sync_semaphore(hw); msec_delay(5); return IXGBE_SUCCESS; @@ -852,9 +853,9 @@ void ixgbe_release_swfw_sync_X540(struct ixgbe_hw *hw, u32 mask) swmask |= mask & IXGBE_GSSR_I2C_MASK; ixgbe_get_swfw_sync_semaphore(hw); - swfw_sync = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC); + swfw_sync = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw)); swfw_sync &= ~swmask; - IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC, swfw_sync); + IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw), swfw_sync); ixgbe_release_swfw_sync_semaphore(hw); msec_delay(5); @@ -881,7 +882,7 @@ static s32 ixgbe_get_swfw_sync_semaphore(struct ixgbe_hw *hw) * If the SMBI bit is 0 when we read it, then the bit will be * set and we have the semaphore */ - swsm = IXGBE_READ_REG(hw, IXGBE_SWSM); + swsm = IXGBE_READ_REG(hw, IXGBE_SWSM_BY_MAC(hw)); if (!(swsm & IXGBE_SWSM_SMBI)) { status = IXGBE_SUCCESS; break; @@ -892,7 +893,7 @@ static s32 ixgbe_get_swfw_sync_semaphore(struct ixgbe_hw *hw) /* Now get the semaphore between SW/FW through the REGSMP bit */ if (status == IXGBE_SUCCESS) { for (i = 0; i < timeout; i++) { - swsm = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC); + swsm = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw)); if (!(swsm & IXGBE_SWFW_REGSMP)) break; @@ -932,13 +933,13 @@ static void ixgbe_release_swfw_sync_semaphore(struct ixgbe_hw *hw) /* Release both semaphores by writing 0 to the bits REGSMP and SMBI */ - swsm = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC); + swsm = IXGBE_READ_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw)); swsm &= ~IXGBE_SWFW_REGSMP; - IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC, swsm); + IXGBE_WRITE_REG(hw, IXGBE_SWFW_SYNC_BY_MAC(hw), swsm); - swsm = IXGBE_READ_REG(hw, IXGBE_SWSM); + swsm = IXGBE_READ_REG(hw, IXGBE_SWSM_BY_MAC(hw)); swsm &= ~IXGBE_SWSM_SMBI; - IXGBE_WRITE_REG(hw, IXGBE_SWSM, swsm); + IXGBE_WRITE_REG(hw, IXGBE_SWSM_BY_MAC(hw), swsm); IXGBE_WRITE_FLUSH(hw); } diff --git a/sys/dev/ixgbe/ixgbe_x550.c b/sys/dev/ixgbe/ixgbe_x550.c index 65daa17..d62035d 100644 --- a/sys/dev/ixgbe/ixgbe_x550.c +++ b/sys/dev/ixgbe/ixgbe_x550.c @@ -115,119 +115,6 @@ static s32 ixgbe_write_cs4227(struct ixgbe_hw *hw, u16 reg, u16 value) } /** - * ixgbe_get_cs4227_status - Return CS4227 status - * @hw: pointer to hardware structure - * - * Returns error if CS4227 not successfully initialized - **/ -static s32 ixgbe_get_cs4227_status(struct ixgbe_hw *hw) -{ - s32 status; - u16 value = 0; - u16 reg_slice, reg_val; - u8 retry; - - for (retry = 0; retry < IXGBE_CS4227_RETRIES; ++retry) { - status = ixgbe_read_cs4227(hw, IXGBE_CS4227_GLOBAL_ID_LSB, - &value); - if (status != IXGBE_SUCCESS) - return status; - if (value == IXGBE_CS4227_GLOBAL_ID_VALUE) - break; - msec_delay(IXGBE_CS4227_CHECK_DELAY); - } - if (value != IXGBE_CS4227_GLOBAL_ID_VALUE) - return IXGBE_ERR_PHY; - - status = ixgbe_read_cs4227(hw, IXGBE_CS4227_SCRATCH, &value); - if (status != IXGBE_SUCCESS) - return status; - - /* If this is the first time after power-on, check the ucode. - * Otherwise, this will disrupt link on all ports. Because we - * can only do this the first time, we must check all ports, - * not just our own. - */ - if (value != IXGBE_CS4227_SCRATCH_VALUE) { - reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB; - reg_val = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 0x1; - status = ixgbe_write_cs4227(hw, reg_slice, - reg_val); - if (status != IXGBE_SUCCESS) - return status; - - reg_slice = IXGBE_CS4227_HOST_SPARE24_LSB; - reg_val = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 0x1; - status = ixgbe_write_cs4227(hw, reg_slice, - reg_val); - if (status != IXGBE_SUCCESS) - return status; - - reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB + (1 << 12); - reg_val = (IXGBE_CS4227_EDC_MODE_SR << 1) | 0x1; - status = ixgbe_write_cs4227(hw, reg_slice, - reg_val); - if (status != IXGBE_SUCCESS) - return status; - - reg_slice = IXGBE_CS4227_HOST_SPARE24_LSB + (1 << 12); - reg_val = (IXGBE_CS4227_EDC_MODE_SR << 1) | 0x1; - status = ixgbe_write_cs4227(hw, reg_slice, - reg_val); - if (status != IXGBE_SUCCESS) - return status; - - msec_delay(10); - } - - /* Verify that the ucode is operational on all ports. */ - reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB; - reg_val = 0xFFFF; - status = ixgbe_read_cs4227(hw, reg_slice, ®_val); - if (status != IXGBE_SUCCESS) - return status; - if (reg_val != 0) - return IXGBE_ERR_PHY; - - reg_slice = IXGBE_CS4227_HOST_SPARE24_LSB; - reg_val = 0xFFFF; - status = ixgbe_read_cs4227(hw, reg_slice, ®_val); - if (status != IXGBE_SUCCESS) - return status; - if (reg_val != 0) - return IXGBE_ERR_PHY; - - reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB + (1 << 12); - reg_val = 0xFFFF; - status = ixgbe_read_cs4227(hw, reg_slice, ®_val); - if (status != IXGBE_SUCCESS) - return status; - if (reg_val != 0) - return IXGBE_ERR_PHY; - - reg_slice = IXGBE_CS4227_HOST_SPARE24_LSB + (1 << 12); - reg_val = 0xFFFF; - status = ixgbe_read_cs4227(hw, reg_slice, ®_val); - if (status != IXGBE_SUCCESS) - return status; - if (reg_val != 0) - return IXGBE_ERR_PHY; - - /* Set scratch for next time. */ - status = ixgbe_write_cs4227(hw, IXGBE_CS4227_SCRATCH, - IXGBE_CS4227_SCRATCH_VALUE); - if (status != IXGBE_SUCCESS) - return status; - status = ixgbe_read_cs4227(hw, IXGBE_CS4227_SCRATCH, &value); - if (status != IXGBE_SUCCESS) - return status; - if (value != IXGBE_CS4227_SCRATCH_VALUE) - return IXGBE_ERR_PHY; - - return IXGBE_SUCCESS; -} - -/** * ixgbe_read_pe - Read register from port expander * @hw: pointer to hardware structure * @reg: register number to read @@ -269,13 +156,17 @@ static s32 ixgbe_write_pe(struct ixgbe_hw *hw, u8 reg, u8 value) * ixgbe_reset_cs4227 - Reset CS4227 using port expander * @hw: pointer to hardware structure * + * This function assumes that the caller has acquired the proper semaphore. * Returns error code **/ static s32 ixgbe_reset_cs4227(struct ixgbe_hw *hw) { s32 status; + u32 retry; + u16 value; u8 reg; + /* Trigger hard reset. */ status = ixgbe_read_pe(hw, IXGBE_PE_OUTPUT, ®); if (status != IXGBE_SUCCESS) return status; @@ -310,7 +201,29 @@ static s32 ixgbe_reset_cs4227(struct ixgbe_hw *hw) if (status != IXGBE_SUCCESS) return status; + /* Wait for the reset to complete. */ msec_delay(IXGBE_CS4227_RESET_DELAY); + for (retry = 0; retry < IXGBE_CS4227_RETRIES; retry++) { + status = ixgbe_read_cs4227(hw, IXGBE_CS4227_EFUSE_STATUS, + &value); + if (status == IXGBE_SUCCESS && + value == IXGBE_CS4227_EEPROM_LOAD_OK) + break; + msec_delay(IXGBE_CS4227_CHECK_DELAY); + } + if (retry == IXGBE_CS4227_RETRIES) { + ERROR_REPORT1(IXGBE_ERROR_INVALID_STATE, + "CS4227 reset did not complete."); + return IXGBE_ERR_PHY; + } + + status = ixgbe_read_cs4227(hw, IXGBE_CS4227_EEPROM_STATUS, &value); + if (status != IXGBE_SUCCESS || + !(value & IXGBE_CS4227_EEPROM_LOAD_OK)) { + ERROR_REPORT1(IXGBE_ERROR_INVALID_STATE, + "CS4227 EEPROM did not load successfully."); + return IXGBE_ERR_PHY; + } return IXGBE_SUCCESS; } @@ -321,29 +234,75 @@ static s32 ixgbe_reset_cs4227(struct ixgbe_hw *hw) **/ static void ixgbe_check_cs4227(struct ixgbe_hw *hw) { + s32 status = IXGBE_SUCCESS; u32 swfw_mask = hw->phy.phy_semaphore_mask; - s32 status; + u16 value = 0; u8 retry; for (retry = 0; retry < IXGBE_CS4227_RETRIES; retry++) { status = hw->mac.ops.acquire_swfw_sync(hw, swfw_mask); if (status != IXGBE_SUCCESS) { ERROR_REPORT2(IXGBE_ERROR_CAUTION, - "semaphore failed with %d\n", status); - return; + "semaphore failed with %d", status); + msec_delay(IXGBE_CS4227_CHECK_DELAY); + continue; } - status = ixgbe_get_cs4227_status(hw); - if (status == IXGBE_SUCCESS) { - hw->mac.ops.release_swfw_sync(hw, swfw_mask); - msec_delay(hw->eeprom.semaphore_delay); + + /* Get status of reset flow. */ + status = ixgbe_read_cs4227(hw, IXGBE_CS4227_SCRATCH, &value); + + if (status == IXGBE_SUCCESS && + value == IXGBE_CS4227_RESET_COMPLETE) + goto out; + + if (status != IXGBE_SUCCESS || + value != IXGBE_CS4227_RESET_PENDING) + break; + + /* Reset is pending. Wait and check again. */ + hw->mac.ops.release_swfw_sync(hw, swfw_mask); + msec_delay(IXGBE_CS4227_CHECK_DELAY); + } + + /* If still pending, assume other instance failed. */ + if (retry == IXGBE_CS4227_RETRIES) { + status = hw->mac.ops.acquire_swfw_sync(hw, swfw_mask); + if (status != IXGBE_SUCCESS) { + ERROR_REPORT2(IXGBE_ERROR_CAUTION, + "semaphore failed with %d", status); return; } - ixgbe_reset_cs4227(hw); - hw->mac.ops.release_swfw_sync(hw, swfw_mask); - msec_delay(hw->eeprom.semaphore_delay); } - ERROR_REPORT2(IXGBE_ERROR_CAUTION, - "Unable to initialize CS4227, err=%d\n", status); + + /* Reset the CS4227. */ + status = ixgbe_reset_cs4227(hw); + if (status != IXGBE_SUCCESS) { + ERROR_REPORT2(IXGBE_ERROR_INVALID_STATE, + "CS4227 reset failed: %d", status); + goto out; + } + + /* Reset takes so long, temporarily release semaphore in case the + * other driver instance is waiting for the reset indication. + */ + ixgbe_write_cs4227(hw, IXGBE_CS4227_SCRATCH, + IXGBE_CS4227_RESET_PENDING); + hw->mac.ops.release_swfw_sync(hw, swfw_mask); + msec_delay(10); + status = hw->mac.ops.acquire_swfw_sync(hw, swfw_mask); + if (status != IXGBE_SUCCESS) { + ERROR_REPORT2(IXGBE_ERROR_CAUTION, + "semaphore failed with %d", status); + return; + } + + /* Record completion for next time. */ + status = ixgbe_write_cs4227(hw, IXGBE_CS4227_SCRATCH, + IXGBE_CS4227_RESET_COMPLETE); + +out: + hw->mac.ops.release_swfw_sync(hw, swfw_mask); + msec_delay(hw->eeprom.semaphore_delay); } /** @@ -452,8 +411,11 @@ s32 ixgbe_init_ops_X550EM(struct ixgbe_hw *hw) hw->bus.type = ixgbe_bus_type_internal; mac->ops.get_bus_info = ixgbe_get_bus_info_X550em; - mac->ops.read_iosf_sb_reg = ixgbe_read_iosf_sb_reg_x550; - mac->ops.write_iosf_sb_reg = ixgbe_write_iosf_sb_reg_x550; + if (hw->mac.type == ixgbe_mac_X550EM_x) { + mac->ops.read_iosf_sb_reg = ixgbe_read_iosf_sb_reg_x550; + mac->ops.write_iosf_sb_reg = ixgbe_write_iosf_sb_reg_x550; + } + mac->ops.get_media_type = ixgbe_get_media_type_X550em; mac->ops.setup_sfp = ixgbe_setup_sfp_modules_X550em; mac->ops.get_link_capabilities = ixgbe_get_link_capabilities_X550em; @@ -679,7 +641,7 @@ s32 ixgbe_setup_eee_X550(struct ixgbe_hw *hw, bool enable_eee) if (enable_eee) { eeer |= (IXGBE_EEER_TX_LPI_EN | IXGBE_EEER_RX_LPI_EN); - if (hw->device_id == IXGBE_DEV_ID_X550T) { + if (hw->mac.type == ixgbe_mac_X550) { /* Advertise EEE capability */ hw->phy.ops.read_reg(hw, IXGBE_MDIO_AUTO_NEG_EEE_ADVT, IXGBE_MDIO_AUTO_NEG_DEV_TYPE, &autoneg_eee_reg); @@ -717,7 +679,7 @@ s32 ixgbe_setup_eee_X550(struct ixgbe_hw *hw, bool enable_eee) } else { eeer &= ~(IXGBE_EEER_TX_LPI_EN | IXGBE_EEER_RX_LPI_EN); - if (hw->device_id == IXGBE_DEV_ID_X550T) { + if (hw->mac.type == ixgbe_mac_X550) { /* Disable advertised EEE capability */ hw->phy.ops.read_reg(hw, IXGBE_MDIO_AUTO_NEG_EEE_ADVT, IXGBE_MDIO_AUTO_NEG_DEV_TYPE, &autoneg_eee_reg); @@ -816,7 +778,7 @@ void ixgbe_set_ethertype_anti_spoofing_X550(struct ixgbe_hw *hw, **/ static s32 ixgbe_iosf_wait(struct ixgbe_hw *hw, u32 *ctrl) { - u32 i, command; + u32 i, command = 0; /* Check every 10 usec to see if the address cycle completed. * The SB IOSF BUSY bit will clear when the operation is @@ -1444,8 +1406,6 @@ static s32 ixgbe_setup_kr_speed_x550em(struct ixgbe_hw *hw, return status; reg_val |= IXGBE_KRM_LINK_CTRL_1_TETH_AN_ENABLE; - reg_val &= ~(IXGBE_KRM_LINK_CTRL_1_TETH_AN_FEC_REQ | - IXGBE_KRM_LINK_CTRL_1_TETH_AN_CAP_FEC); reg_val &= ~(IXGBE_KRM_LINK_CTRL_1_TETH_AN_CAP_KR | IXGBE_KRM_LINK_CTRL_1_TETH_AN_CAP_KX); @@ -1492,14 +1452,10 @@ s32 ixgbe_init_phy_ops_X550em(struct ixgbe_hw *hw) * to determine internal PHY mode. */ phy->nw_mng_if_sel = IXGBE_READ_REG(hw, IXGBE_NW_MNG_IF_SEL); - - /* If internal PHY mode is KR, then initialize KR link */ if (phy->nw_mng_if_sel & IXGBE_NW_MNG_IF_SEL_INT_PHY_MODE) { speed = IXGBE_LINK_SPEED_10GB_FULL | IXGBE_LINK_SPEED_1GB_FULL; - ret_val = ixgbe_setup_kr_speed_x550em(hw, speed); } - phy->ops.identify_sfp = ixgbe_identify_sfp_module_X550em; } @@ -1516,7 +1472,7 @@ s32 ixgbe_init_phy_ops_X550em(struct ixgbe_hw *hw) /* Set functions pointers based on phy type */ switch (hw->phy.type) { case ixgbe_phy_x550em_kx4: - phy->ops.setup_link = ixgbe_setup_kx4_x550em; + phy->ops.setup_link = NULL; phy->ops.read_reg = ixgbe_read_phy_reg_x550em; phy->ops.write_reg = ixgbe_write_phy_reg_x550em; break; @@ -1543,7 +1499,11 @@ s32 ixgbe_init_phy_ops_X550em(struct ixgbe_hw *hw) ret_val = ixgbe_setup_kr_speed_x550em(hw, speed); } - phy->ops.enter_lplu = ixgbe_enter_lplu_t_x550em; + /* setup SW LPLU only for first revision */ + if (!(IXGBE_FUSES0_REV1 & IXGBE_READ_REG(hw, + IXGBE_FUSES0_GROUP(0)))) + phy->ops.enter_lplu = ixgbe_enter_lplu_t_x550em; + phy->ops.handle_lasi = ixgbe_handle_lasi_ext_t_x550em; phy->ops.reset = ixgbe_reset_phy_t_X550em; break; @@ -1580,9 +1540,14 @@ s32 ixgbe_reset_hw_X550em(struct ixgbe_hw *hw) /* flush pending Tx transactions */ ixgbe_clear_tx_pending(hw); - /* PHY ops must be identified and initialized prior to reset */ + if (hw->device_id == IXGBE_DEV_ID_X550EM_X_10G_T) { + /* Config MDIO clock speed before the first MDIO PHY access */ + hlreg0 = IXGBE_READ_REG(hw, IXGBE_HLREG0); + hlreg0 &= ~IXGBE_HLREG0_MDCSPD; + IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg0); + } - /* Identify PHY and related function pointers */ + /* PHY ops must be identified and initialized prior to reset */ status = hw->phy.ops.init(hw); if (status == IXGBE_ERR_SFP_NOT_SUPPORTED) @@ -1659,13 +1624,6 @@ mac_reset_top: hw->mac.num_rar_entries = 128; hw->mac.ops.init_rx_addrs(hw); - if (hw->device_id == IXGBE_DEV_ID_X550EM_X_10G_T) { - /* Config MDIO clock speed. */ - hlreg0 = IXGBE_READ_REG(hw, IXGBE_HLREG0); - hlreg0 &= ~IXGBE_HLREG0_MDCSPD; - IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg0); - } - if (hw->device_id == IXGBE_DEV_ID_X550EM_X_SFP) ixgbe_setup_mux_ctl(hw); @@ -1727,43 +1685,6 @@ s32 ixgbe_setup_kr_x550em(struct ixgbe_hw *hw) } /** - * ixgbe_setup_kx4_x550em - Configure the KX4 PHY. - * @hw: pointer to hardware structure - * - * Configures the integrated KX4 PHY. - **/ -s32 ixgbe_setup_kx4_x550em(struct ixgbe_hw *hw) -{ - s32 status; - u32 reg_val; - - status = ixgbe_read_iosf_sb_reg_x550(hw, IXGBE_KX4_LINK_CNTL_1, - IXGBE_SB_IOSF_TARGET_KX4_PCS, ®_val); - if (status) - return status; - - reg_val &= ~(IXGBE_KX4_LINK_CNTL_1_TETH_AN_CAP_KX4 | - IXGBE_KX4_LINK_CNTL_1_TETH_AN_CAP_KX); - - reg_val |= IXGBE_KX4_LINK_CNTL_1_TETH_AN_ENABLE; - - /* Advertise 10G support. */ - if (hw->phy.autoneg_advertised & IXGBE_LINK_SPEED_10GB_FULL) - reg_val |= IXGBE_KX4_LINK_CNTL_1_TETH_AN_CAP_KX4; - - /* Advertise 1G support. */ - if (hw->phy.autoneg_advertised & IXGBE_LINK_SPEED_1GB_FULL) - reg_val |= IXGBE_KX4_LINK_CNTL_1_TETH_AN_CAP_KX; - - /* Restart auto-negotiation. */ - reg_val |= IXGBE_KX4_LINK_CNTL_1_TETH_AN_RESTART; - status = ixgbe_write_iosf_sb_reg_x550(hw, IXGBE_KX4_LINK_CNTL_1, - IXGBE_SB_IOSF_TARGET_KX4_PCS, reg_val); - - return status; -} - -/** * ixgbe_setup_mac_link_sfp_x550em - Setup internal/external the PHY for SFP * @hw: pointer to hardware structure * @@ -1791,38 +1712,53 @@ s32 ixgbe_setup_mac_link_sfp_x550em(struct ixgbe_hw *hw, if (ret_val != IXGBE_SUCCESS) return ret_val; - /* Configure CS4227 for LINE connection rate then type. */ - reg_slice = IXGBE_CS4227_LINE_SPARE22_MSB + (hw->bus.lan_id << 12); - reg_val = (speed & IXGBE_LINK_SPEED_10GB_FULL) ? 0 : 0x8000; - ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, - reg_val); - - reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB + (hw->bus.lan_id << 12); - if (setup_linear) - reg_val = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 0x1; - else - reg_val = (IXGBE_CS4227_EDC_MODE_SR << 1) | 0x1; - ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, - reg_val); - - /* Configure CS4227 for HOST connection rate then type. */ - reg_slice = IXGBE_CS4227_HOST_SPARE22_MSB + (hw->bus.lan_id << 12); - reg_val = (speed & IXGBE_LINK_SPEED_10GB_FULL) ? 0 : 0x8000; - ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, - reg_val); - - reg_slice = IXGBE_CS4227_HOST_SPARE24_LSB + (hw->bus.lan_id << 12); - if (setup_linear) - reg_val = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 0x1; - else + if (!(hw->phy.nw_mng_if_sel & IXGBE_NW_MNG_IF_SEL_INT_PHY_MODE)) { + /* Configure CS4227 LINE side to 10G SR. */ + reg_slice = IXGBE_CS4227_LINE_SPARE22_MSB + + (hw->bus.lan_id << 12); + reg_val = IXGBE_CS4227_SPEED_10G; + ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, + reg_val); + + reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB + + (hw->bus.lan_id << 12); reg_val = (IXGBE_CS4227_EDC_MODE_SR << 1) | 0x1; - ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, - reg_val); + ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, + reg_val); + + /* Configure CS4227 for HOST connection rate then type. */ + reg_slice = IXGBE_CS4227_HOST_SPARE22_MSB + + (hw->bus.lan_id << 12); + reg_val = (speed & IXGBE_LINK_SPEED_10GB_FULL) ? + IXGBE_CS4227_SPEED_10G : IXGBE_CS4227_SPEED_1G; + ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, + reg_val); + + reg_slice = IXGBE_CS4227_HOST_SPARE24_LSB + + (hw->bus.lan_id << 12); + if (setup_linear) + reg_val = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 0x1; + else + reg_val = (IXGBE_CS4227_EDC_MODE_SR << 1) | 0x1; + ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, + reg_val); - /* If internal link mode is XFI, then setup XFI internal link. */ - if (!(hw->phy.nw_mng_if_sel & IXGBE_NW_MNG_IF_SEL_INT_PHY_MODE)) + /* Setup XFI internal link. */ ret_val = ixgbe_setup_ixfi_x550em(hw, &speed); - + } else { + /* Configure internal PHY for KR/KX. */ + ixgbe_setup_kr_speed_x550em(hw, speed); + + /* Configure CS4227 LINE side to proper mode. */ + reg_slice = IXGBE_CS4227_LINE_SPARE24_LSB + + (hw->bus.lan_id << 12); + if (setup_linear) + reg_val = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 0x1; + else + reg_val = (IXGBE_CS4227_EDC_MODE_SR << 1) | 0x1; + ret_val = ixgbe_write_i2c_combined(hw, IXGBE_CS4227, reg_slice, + reg_val); + } return ret_val; } @@ -2743,6 +2679,10 @@ s32 ixgbe_enter_lplu_t_x550em(struct ixgbe_hw *hw) u32 save_autoneg; bool link_up; + /* SW LPLU not required on later HW revisions. */ + if (IXGBE_FUSES0_REV1 & IXGBE_READ_REG(hw, IXGBE_FUSES0_GROUP(0))) + return IXGBE_SUCCESS; + /* If blocked by MNG FW, then don't restart AN */ if (ixgbe_check_reset_blocked(hw)) return IXGBE_SUCCESS; @@ -2924,7 +2864,7 @@ s32 ixgbe_setup_fc_X550em(struct ixgbe_hw *hw) goto out; } - if (hw->phy.media_type == ixgbe_media_type_backplane) { + if (hw->device_id == IXGBE_DEV_ID_X550EM_X_KR) { ret_val = ixgbe_read_iosf_sb_reg_x550(hw, IXGBE_KRM_AN_CNTL_1(hw->bus.lan_id), IXGBE_SB_IOSF_TARGET_KR_PHY, ®_val); @@ -2940,9 +2880,8 @@ s32 ixgbe_setup_fc_X550em(struct ixgbe_hw *hw) IXGBE_KRM_AN_CNTL_1(hw->bus.lan_id), IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); - /* Not all devices fully support AN. */ - if (hw->device_id == IXGBE_DEV_ID_X550EM_X_KR) - hw->fc.disable_fc_autoneg = TRUE; + /* This device does not fully support AN. */ + hw->fc.disable_fc_autoneg = TRUE; } out: diff --git a/sys/dev/ixgbe/ixgbe_x550.h b/sys/dev/ixgbe/ixgbe_x550.h index 8a544ec..b590bb7 100644 --- a/sys/dev/ixgbe/ixgbe_x550.h +++ b/sys/dev/ixgbe/ixgbe_x550.h @@ -69,7 +69,7 @@ void ixgbe_set_ethertype_anti_spoofing_X550(struct ixgbe_hw *hw, s32 ixgbe_write_iosf_sb_reg_x550(struct ixgbe_hw *hw, u32 reg_addr, u32 device_type, u32 data); s32 ixgbe_read_iosf_sb_reg_x550(struct ixgbe_hw *hw, u32 reg_addr, - u32 device_type, u32 *data); + u32 device_type, u32 *data); void ixgbe_disable_mdd_X550(struct ixgbe_hw *hw); void ixgbe_enable_mdd_X550(struct ixgbe_hw *hw); void ixgbe_mdd_event_X550(struct ixgbe_hw *hw, u32 *vf_bitmap); @@ -82,7 +82,6 @@ void ixgbe_init_mac_link_ops_X550em(struct ixgbe_hw *hw); s32 ixgbe_reset_hw_X550em(struct ixgbe_hw *hw); s32 ixgbe_init_phy_ops_X550em(struct ixgbe_hw *hw); s32 ixgbe_setup_kr_x550em(struct ixgbe_hw *hw); -s32 ixgbe_setup_kx4_x550em(struct ixgbe_hw *hw); s32 ixgbe_init_ext_t_x550em(struct ixgbe_hw *hw); s32 ixgbe_setup_internal_phy_t_x550em(struct ixgbe_hw *hw); s32 ixgbe_setup_phy_loopback_x550em(struct ixgbe_hw *hw); diff --git a/sys/dev/etherswitch/mdio.c b/sys/dev/mdio/mdio.c index 1d1d370..5798c74 100644 --- a/sys/dev/etherswitch/mdio.c +++ b/sys/dev/mdio/mdio.c @@ -29,8 +29,9 @@ #include <sys/param.h> #include <sys/bus.h> #include <sys/systm.h> +#include <sys/module.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include "mdio_if.h" @@ -82,6 +83,21 @@ mdio_writereg(device_t dev, int phy, int reg, int val) return (MDIO_WRITEREG(device_get_parent(dev), phy, reg, val)); } +static int +mdio_readextreg(device_t dev, int phy, int devad, int reg) +{ + + return (MDIO_READEXTREG(device_get_parent(dev), phy, devad, reg)); +} + +static int +mdio_writeextreg(device_t dev, int phy, int devad, int reg, + int val) +{ + + return (MDIO_WRITEEXTREG(device_get_parent(dev), phy, devad, reg, val)); +} + static void mdio_hinted_child(device_t dev, const char *name, int unit) { @@ -104,6 +120,8 @@ static device_method_t mdio_methods[] = { /* MDIO access */ DEVMETHOD(mdio_readreg, mdio_readreg), DEVMETHOD(mdio_writereg, mdio_writereg), + DEVMETHOD(mdio_readextreg, mdio_readextreg), + DEVMETHOD(mdio_writeextreg, mdio_writeextreg), DEVMETHOD_END }; @@ -115,3 +133,5 @@ driver_t mdio_driver = { }; devclass_t mdio_devclass; + +MODULE_VERSION(mdio, 1); diff --git a/sys/dev/etherswitch/mdio.h b/sys/dev/mdio/mdio.h index 82578e0..c38ff48 100644 --- a/sys/dev/etherswitch/mdio.h +++ b/sys/dev/mdio/mdio.h @@ -26,10 +26,12 @@ * $FreeBSD$ */ -#ifndef __DEV_ETHERSWITCH_MDIO_H__ -#define __DEV_ETHERSWITCH_MDIO_H__ +#ifndef __DEV_MDIO_MDIO_H__ +#define __DEV_MDIO_MDIO_H__ extern driver_t mdio_driver; extern devclass_t mdio_devclass; -#endif /* __DEV_ETHERSWITCH_MDIO_H__ */ +#define MDIO_DEVADDR_NONE -1 /**< Use clause 22 register access */ + +#endif /* __DEV_MDIO_MDIO_H__ */ diff --git a/sys/dev/mdio/mdio_if.m b/sys/dev/mdio/mdio_if.m new file mode 100644 index 0000000..83308bc --- /dev/null +++ b/sys/dev/mdio/mdio_if.m @@ -0,0 +1,91 @@ +# $FreeBSD$ + +#include <sys/bus.h> + +INTERFACE mdio; + +CODE { + #include <dev/mdio/mdio.h> + + static int + mdio_null_readextreg(device_t dev, int phy, int devad, int reg) + { + if (devad == MDIO_DEVADDR_NONE) + return (MDIO_READREG(dev, phy, reg)); + return (~0U); + } + + static int + mdio_null_writeextreg(device_t dev, int phy, int devad, int reg, + int val) + { + if (devad == MDIO_DEVADDR_NONE) + return (MDIO_WRITEREG(dev, phy, reg, val)); + + return (EINVAL); + } +} + +/** + * @brief Read register from device on MDIO bus. + * + * @param dev MDIO bus device. + * @param phy PHY address. + * @param reg The PHY register offset. + */ +METHOD int readreg { + device_t dev; + int phy; + int reg; +}; + +/** + * @brief Write register to device on MDIO bus. + * + * @param dev MDIO bus device. + * @param phy PHY address. + * @param reg The PHY register offset. + * @param val The value to write at offset @p reg. + */ +METHOD int writereg { + device_t dev; + int phy; + int reg; + int val; +}; + + +/** + * @brief Read extended register from device on MDIO bus. + * + * @param dev MDIO bus device. + * @param phy PHY address. + * @param devad The MDIO IEEE 802.3 Clause 45 device address, or + * MDIO_DEVADDR_NONE to request Clause 22 register addressing. + * @param reg The PHY register offset. + */ +METHOD int readextreg { + device_t dev; + int phy; + int devad; + int reg; +} DEFAULT mdio_null_readextreg; + + +/** + * @brief Write extended register to device on MDIO bus. + * + * @param dev MDIO bus device. + * @param phy PHY address. + * @param devad The MDIO IEEE 802.3 Clause 45 device address, or + * MDIO_DEVADDR_NONE to request Clause 22 register addressing. + * @param reg The PHY register offset. + * @param val The value to write at offset @p reg. + */ +METHOD int writeextreg { + device_t dev; + int phy; + int devad; + int reg; + int val; +} DEFAULT mdio_null_writeextreg; diff --git a/sys/dev/mge/if_mge.c b/sys/dev/mge/if_mge.c index 61579a9..e0801d8 100644 --- a/sys/dev/mge/if_mge.c +++ b/sys/dev/mge/if_mge.c @@ -74,7 +74,7 @@ __FBSDID("$FreeBSD$"); #include <dev/fdt/fdt_common.h> #include <dev/ofw/ofw_bus.h> #include <dev/ofw/ofw_bus_subr.h> -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/mge/if_mgevar.h> #include <arm/mv/mvreg.h> diff --git a/sys/dev/mlx5/device.h b/sys/dev/mlx5/device.h index b9a8eec..88ca731 100644 --- a/sys/dev/mlx5/device.h +++ b/sys/dev/mlx5/device.h @@ -1042,6 +1042,7 @@ enum { MLX5_ESW_VPORT_ADMIN_STATE_UP = 0x1, MLX5_ESW_VPORT_ADMIN_STATE_AUTO = 0x2, }; + /* MLX5 DEV CAPs */ /* TODO: EAT.ME */ @@ -1219,4 +1220,36 @@ struct mlx5_ifc_mcia_reg_bits { }; #define MLX5_CMD_OP_QUERY_EEPROM 0x93c + +struct mlx5_mini_cqe8 { + union { + u32 rx_hash_result; + u32 checksum; + struct { + u16 wqe_counter; + u8 s_wqe_opcode; + u8 reserved; + } s_wqe_info; + }; + u32 byte_cnt; +}; + +enum { + MLX5_NO_INLINE_DATA, + MLX5_INLINE_DATA32_SEG, + MLX5_INLINE_DATA64_SEG, + MLX5_COMPRESSED, +}; + +enum mlx5_exp_cqe_zip_recv_type { + MLX5_CQE_FORMAT_HASH, + MLX5_CQE_FORMAT_CSUM, +}; + +#define MLX5E_CQE_FORMAT_MASK 0xc +static inline int mlx5_get_cqe_format(const struct mlx5_cqe64 *cqe) +{ + return (cqe->op_own & MLX5E_CQE_FORMAT_MASK) >> 2; +} + #endif /* MLX5_DEVICE_H */ diff --git a/sys/dev/mlx5/mlx5_en/en.h b/sys/dev/mlx5/mlx5_en/en.h index 23e4fd0..bb779d2 100644 --- a/sys/dev/mlx5/mlx5_en/en.h +++ b/sys/dev/mlx5/mlx5_en/en.h @@ -370,6 +370,7 @@ struct mlx5e_params { u16 tx_cq_moderation_pkts; u16 min_rx_wqes; bool hw_lro_en; + bool cqe_zipping_en; u32 lro_wqe_sz; u16 rx_hash_log_tbl_sz; }; @@ -390,7 +391,8 @@ struct mlx5e_params { m(+1, u64 tx_coalesce_usecs, "tx_coalesce_usecs", "Limit in usec for joining tx packets") \ m(+1, u64 tx_coalesce_pkts, "tx_coalesce_pkts", "Maximum number of tx packets to join") \ m(+1, u64 tx_coalesce_mode, "tx_coalesce_mode", "0: EQE mode 1: CQE mode") \ - m(+1, u64 hw_lro, "hw_lro", "set to enable hw_lro") + m(+1, u64 hw_lro, "hw_lro", "set to enable hw_lro") \ + m(+1, u64 cqe_zipping, "cqe_zipping", "0 : CQE zipping disabled") #define MLX5E_PARAMS_NUM (0 MLX5E_PARAMS(MLX5E_STATS_COUNT)) diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c b/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c index eede82f..17e4461 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c @@ -58,13 +58,17 @@ mlx5e_ethtool_handler(SYSCTL_HANDLER_ARGS) PRIV_LOCK(priv); value = priv->params_ethtool.arg[arg2]; - error = sysctl_handle_64(oidp, &value, 0, req); - if (error || req->newptr == NULL || - value == priv->params_ethtool.arg[arg2]) - goto done; + if (req != NULL) { + error = sysctl_handle_64(oidp, &value, 0, req); + if (error || req->newptr == NULL || + value == priv->params_ethtool.arg[arg2]) + goto done; - /* assign new value */ - priv->params_ethtool.arg[arg2] = value; + /* assign new value */ + priv->params_ethtool.arg[arg2] = value; + } else { + error = 0; + } /* check if device is gone */ if (priv->gone) { @@ -200,6 +204,18 @@ mlx5e_ethtool_handler(SYSCTL_HANDLER_ARGS) priv->params.hw_lro_en = false; } + if (&priv->params_ethtool.arg[arg2] == + &priv->params_ethtool.cqe_zipping) { + if (priv->params_ethtool.cqe_zipping && + MLX5_CAP_GEN(priv->mdev, cqe_compression)) { + priv->params.cqe_zipping_en = true; + priv->params_ethtool.cqe_zipping = 1; + } else { + priv->params.cqe_zipping_en = false; + priv->params_ethtool.cqe_zipping = 0; + } + } + if (was_opened) mlx5e_open_locked(priv->ifp); done: @@ -319,34 +335,33 @@ mlx5e_get_eeprom(struct mlx5e_priv *priv, struct mlx5e_eeprom *ee) static void mlx5e_print_eeprom(struct mlx5e_eeprom *eeprom) { - int i, j = 0; - int row = 0; + int row; + int index_in_row; + int byte_to_write = 0; + int line_length = 16; printf("\nOffset\t\tValues\n"); - printf("------\t\t------\n"); - while (row < eeprom->len) { - printf("0x%04x\t\t", row); - for (i = 0; i < 16; i++) { - printf("%02x ", ((u8 *)eeprom->data)[j]); - j++; - row++; + printf("------\t\t------"); + while (byte_to_write < eeprom->len) { + printf("\n0x%04X\t\t", byte_to_write); + for (index_in_row = 0; index_in_row < line_length; index_in_row++) { + printf("%02X ", ((u8 *)eeprom->data)[byte_to_write]); + byte_to_write++; } - printf("\n"); } if (eeprom->page_valid) { row = MLX5E_EEPROM_HIGH_PAGE_OFFSET; - printf("\nUpper Page 0x03\n"); + printf("\n\nUpper Page 0x03\n"); printf("\nOffset\t\tValues\n"); - printf("------\t\t------\n"); + printf("------\t\t------"); while (row < MLX5E_EEPROM_PAGE_LENGTH) { - printf("0x%04x\t\t", row); - for (i = 0; i < 16; i++) { - printf("%02x ", ((u8 *)eeprom->data)[j]); - j++; + printf("\n0x%04X\t\t", row); + for (index_in_row = 0; index_in_row < line_length; index_in_row++) { + printf("%02X ", ((u8 *)eeprom->data)[byte_to_write]); + byte_to_write++; row++; } - printf("\n"); } } } @@ -469,6 +484,7 @@ mlx5e_create_ethtool(struct mlx5e_priv *priv) priv->params_ethtool.tx_coalesce_usecs = priv->params.tx_cq_moderation_usec; priv->params_ethtool.tx_coalesce_pkts = priv->params.tx_cq_moderation_pkts; priv->params_ethtool.hw_lro = priv->params.hw_lro_en; + priv->params_ethtool.cqe_zipping = priv->params.cqe_zipping_en; /* create root node */ node = SYSCTL_ADD_NODE(&priv->sysctl_ctx, @@ -484,10 +500,30 @@ mlx5e_create_ethtool(struct mlx5e_priv *priv) CTLFLAG_MPSAFE, priv, x, &mlx5e_ethtool_handler, "QU", mlx5e_params_desc[2 * x + 1]); } else { +#if (__FreeBSD_version < 1100000) + char path[64]; +#endif + /* + * NOTE: In FreeBSD-11 and newer the + * CTLFLAG_RWTUN flag will take care of + * loading default sysctl value from the + * kernel environment, if any: + */ SYSCTL_ADD_PROC(&priv->sysctl_ctx, SYSCTL_CHILDREN(node), OID_AUTO, mlx5e_params_desc[2 * x], CTLTYPE_U64 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, priv, x, &mlx5e_ethtool_handler, "QU", mlx5e_params_desc[2 * x + 1]); + +#if (__FreeBSD_version < 1100000) + /* compute path for sysctl */ + snprintf(path, sizeof(path), "dev.mce.%d.conf.%s", + device_get_unit(priv->mdev->pdev->dev.bsddev), + mlx5e_params_desc[2 * x]); + + /* try to fetch tunable, if any */ + if (TUNABLE_QUAD_FETCH(path, &priv->params_ethtool.arg[x])) + mlx5e_ethtool_handler(NULL, priv, x, NULL); +#endif } } diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c index 77aaf7b..63db417 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c @@ -1604,6 +1604,16 @@ mlx5e_build_rx_cq_param(struct mlx5e_priv *priv, { void *cqc = param->cqc; + + /* + * TODO The sysctl to control on/off is a bool value for now, which means + * we only support CSUM, once HASH is implemnted we'll need to address that. + */ + if (priv->params.cqe_zipping_en) { + MLX5_SET(cqc, cqc, mini_cqe_res_format, MLX5_CQE_FORMAT_CSUM); + MLX5_SET(cqc, cqc, cqe_compression_en, 1); + } + MLX5_SET(cqc, cqc, log_cq_size, priv->params.log_rq_size); MLX5_SET(cqc, cqc, cq_period, priv->params.rx_cq_moderation_usec); MLX5_SET(cqc, cqc, cq_max_count, priv->params.rx_cq_moderation_pkts); @@ -2571,6 +2581,8 @@ mlx5e_build_ifp_priv(struct mlx5_core_dev *mdev, priv->params.hw_lro_en = false; priv->params.lro_wqe_sz = MLX5E_PARAMS_DEFAULT_LRO_WQE_SZ; + priv->params.cqe_zipping_en = !!MLX5_CAP_GEN(mdev, cqe_compression); + priv->mdev = mdev; priv->params.num_channels = num_comp_vectors; priv->order_base_2_num_channels = order_base_2(num_comp_vectors); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c b/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c index 87fc524..059b5ef 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c @@ -248,6 +248,69 @@ mlx5e_build_rx_mbuf(struct mlx5_cqe64 *cqe, } } +static inline void +mlx5e_read_cqe_slot(struct mlx5e_cq *cq, u32 cc, void *data) +{ + memcpy(data, mlx5_cqwq_get_wqe(&cq->wq, (cc & cq->wq.sz_m1)), + sizeof(struct mlx5_cqe64)); +} + +static inline void +mlx5e_write_cqe_slot(struct mlx5e_cq *cq, u32 cc, void *data) +{ + memcpy(mlx5_cqwq_get_wqe(&cq->wq, cc & cq->wq.sz_m1), + data, sizeof(struct mlx5_cqe64)); +} + +static inline void +mlx5e_decompress_cqe(struct mlx5e_cq *cq, struct mlx5_cqe64 *title, + struct mlx5_mini_cqe8 *mini, + u16 wqe_counter, int i) +{ + title->byte_cnt = mini->byte_cnt; + title->wqe_counter = cpu_to_be16((wqe_counter + i) & cq->wq.sz_m1); + title->check_sum = mini->checksum; + title->op_own = (title->op_own & 0xf0) | + (((cq->wq.cc + i) >> cq->wq.log_sz) & 1); +} + +#define MLX5E_MINI_ARRAY_SZ 8 +/* Make sure structs are not packet differently */ +CTASSERT(sizeof(struct mlx5_cqe64) == + sizeof(struct mlx5_mini_cqe8) * MLX5E_MINI_ARRAY_SZ); +static void +mlx5e_decompress_cqes(struct mlx5e_cq *cq) +{ + struct mlx5_mini_cqe8 mini_array[MLX5E_MINI_ARRAY_SZ]; + struct mlx5_cqe64 title; + u32 cqe_count; + u32 i = 0; + u16 title_wqe_counter; + + mlx5e_read_cqe_slot(cq, cq->wq.cc, &title); + title_wqe_counter = be16_to_cpu(title.wqe_counter); + cqe_count = be32_to_cpu(title.byte_cnt); + + /* Make sure we won't overflow */ + KASSERT(cqe_count <= cq->wq.sz_m1, + ("%s: cqe_count %u > cq->wq.sz_m1 %u", __func__, + cqe_count, cq->wq.sz_m1)); + + mlx5e_read_cqe_slot(cq, cq->wq.cc + 1, mini_array); + while (true) { + mlx5e_decompress_cqe(cq, &title, + &mini_array[i % MLX5E_MINI_ARRAY_SZ], + title_wqe_counter, i); + mlx5e_write_cqe_slot(cq, cq->wq.cc + i, &title); + i++; + + if (i == cqe_count) + break; + if (i % MLX5E_MINI_ARRAY_SZ == 0) + mlx5e_read_cqe_slot(cq, cq->wq.cc + i, mini_array); + } +} + static int mlx5e_poll_rx_cq(struct mlx5e_rq *rq, int budget) { @@ -268,6 +331,11 @@ mlx5e_poll_rx_cq(struct mlx5e_rq *rq, int budget) if (!cqe) break; + if (mlx5_get_cqe_format(cqe) == MLX5_COMPRESSED) + mlx5e_decompress_cqes(&rq->cq); + + mlx5_cqwq_pop(&rq->cq.wq); + wqe_counter_be = cqe->wqe_counter; wqe_counter = be16_to_cpu(wqe_counter_be); wqe = mlx5_wq_ll_get_wqe(&rq->wq, wqe_counter); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c b/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c index cb08727..483a7e1 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c @@ -383,6 +383,8 @@ mlx5e_poll_tx_cq(struct mlx5e_sq *sq, int budget) if (!cqe) break; + mlx5_cqwq_pop(&sq->cq.wq); + ci = sqcc & sq->wq.sz_m1; mb = sq->mbuf[ci].mbuf; sq->mbuf[ci].mbuf = NULL; /* Safety clear */ diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_txrx.c b/sys/dev/mlx5/mlx5_en/mlx5_en_txrx.c index 7836bfe..f46639d 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_txrx.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_txrx.c @@ -37,8 +37,6 @@ mlx5e_get_cqe(struct mlx5e_cq *cq) if ((cqe->op_own ^ mlx5_cqwq_get_wrap_cnt(&cq->wq)) & MLX5_CQE_OWNER_MASK) return (NULL); - mlx5_cqwq_pop(&cq->wq); - /* ensure cqe content is read after cqe ownership bit */ rmb(); diff --git a/sys/dev/netmap/netmap.c b/sys/dev/netmap/netmap.c index 56bbf12..f1b8610 100644 --- a/sys/dev/netmap/netmap.c +++ b/sys/dev/netmap/netmap.c @@ -589,7 +589,7 @@ netmap_set_all_rings(struct netmap_adapter *na, int stopped) /* * Convenience function used in drivers. Waits for current txsync()s/rxsync()s * to finish and prevents any new one from starting. Call this before turning - * netmap mode off, or before removing the harware rings (e.g., on module + * netmap mode off, or before removing the hardware rings (e.g., on module * onload). As a rule of thumb for linux drivers, this should be placed near * each napi_disable(). */ diff --git a/sys/dev/ofw/ofw_subr.c b/sys/dev/ofw/ofw_subr.c new file mode 100644 index 0000000..e046406 --- /dev/null +++ b/sys/dev/ofw/ofw_subr.c @@ -0,0 +1,176 @@ +/*- + * Copyright (c) 2015 Ian Lepore <ian@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * The initial ofw_reg_to_paddr() implementation has been copied from powerpc + * ofw_machdep.c OF_decode_addr(). It was added by Marcel Moolenaar, who did not + * assert copyright with the addition but still deserves credit for the work. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/bus.h> +#include <sys/libkern.h> + +#include <machine/bus.h> + +#include <dev/ofw/openfirm.h> +#include <dev/ofw/ofw_pci.h> +#include <dev/ofw/ofw_subr.h> + +static void +get_addr_props(phandle_t node, uint32_t *addrp, uint32_t *sizep, int *pcip) +{ + char type[64]; + uint32_t addr, size; + int pci, res; + + res = OF_getencprop(node, "#address-cells", &addr, sizeof(addr)); + if (res == -1) + addr = 2; + res = OF_getencprop(node, "#size-cells", &size, sizeof(size)); + if (res == -1) + size = 1; + pci = 0; + if (addr == 3 && size == 2) { + res = OF_getprop(node, "device_type", type, sizeof(type)); + if (res != -1) { + type[sizeof(type) - 1] = '\0'; + pci = (strcmp(type, "pci") == 0) ? 1 : 0; + } + } + if (addrp != NULL) + *addrp = addr; + if (sizep != NULL) + *sizep = size; + if (pcip != NULL) + *pcip = pci; +} + +int +ofw_reg_to_paddr(phandle_t dev, int regno, bus_addr_t *paddr, + bus_size_t *psize, pcell_t *ppci_hi) +{ + pcell_t cell[32], pci_hi; + bus_addr_t addr, raddr, baddr; + bus_size_t size, rsize; + uint32_t c, nbridge, naddr, nsize; + phandle_t bridge, parent; + u_int spc, rspc; + int pci, pcib, res; + + /* Sanity checking. */ + if (dev == 0) + return (EINVAL); + bridge = OF_parent(dev); + if (bridge == 0) + return (EINVAL); + if (regno < 0) + return (EINVAL); + if (paddr == NULL || psize == NULL) + return (EINVAL); + + get_addr_props(bridge, &naddr, &nsize, &pci); + res = OF_getencprop(dev, (pci) ? "assigned-addresses" : "reg", + cell, sizeof(cell)); + if (res == -1) + return (ENXIO); + if (res % sizeof(cell[0])) + return (ENXIO); + res /= sizeof(cell[0]); + regno *= naddr + nsize; + if (regno + naddr + nsize > res) + return (EINVAL); + pci_hi = pci ? cell[regno] : OFW_PADDR_NOT_PCI; + spc = pci_hi & OFW_PCI_PHYS_HI_SPACEMASK; + addr = 0; + for (c = 0; c < naddr; c++) + addr = ((uint64_t)addr << 32) | cell[regno++]; + size = 0; + for (c = 0; c < nsize; c++) + size = ((uint64_t)size << 32) | cell[regno++]; + /* + * Map the address range in the bridge's decoding window as given + * by the "ranges" property. If a node doesn't have such property + * or the property is empty, we assume an identity mapping. The + * standard says a missing property indicates no possible mapping. + * This code is more liberal since the intended use is to get a + * console running early, and a printf to warn of malformed data + * is probably futile before the console is fully set up. + */ + parent = OF_parent(bridge); + while (parent != 0) { + get_addr_props(parent, &nbridge, NULL, &pcib); + res = OF_getencprop(bridge, "ranges", cell, sizeof(cell)); + if (res < 1) + goto next; + if (res % sizeof(cell[0])) + return (ENXIO); + /* Capture pci_hi if we just transitioned onto a PCI bus. */ + if (pcib && pci_hi == OFW_PADDR_NOT_PCI) { + pci_hi = cell[0]; + spc = pci_hi & OFW_PCI_PHYS_HI_SPACEMASK; + } + res /= sizeof(cell[0]); + regno = 0; + while (regno < res) { + rspc = (pci ? cell[regno] : OFW_PADDR_NOT_PCI) & + OFW_PCI_PHYS_HI_SPACEMASK; + if (rspc != spc) { + regno += naddr + nbridge + nsize; + continue; + } + raddr = 0; + for (c = 0; c < naddr; c++) + raddr = ((uint64_t)raddr << 32) | cell[regno++]; + rspc = (pcib) + ? cell[regno] & OFW_PCI_PHYS_HI_SPACEMASK + : OFW_PADDR_NOT_PCI; + baddr = 0; + for (c = 0; c < nbridge; c++) + baddr = ((uint64_t)baddr << 32) | cell[regno++]; + rsize = 0; + for (c = 0; c < nsize; c++) + rsize = ((uint64_t)rsize << 32) | cell[regno++]; + if (addr < raddr || addr >= raddr + rsize) + continue; + addr = addr - raddr + baddr; + if (rspc != OFW_PADDR_NOT_PCI) + spc = rspc; + } + next: + bridge = parent; + parent = OF_parent(bridge); + get_addr_props(bridge, &naddr, &nsize, &pci); + } + + *paddr = addr; + *psize = size; + if (ppci_hi != NULL) + *ppci_hi = pci_hi; + + return (0); +} diff --git a/sys/dev/ofw/ofw_subr.h b/sys/dev/ofw/ofw_subr.h new file mode 100644 index 0000000..8fa64b2 --- /dev/null +++ b/sys/dev/ofw/ofw_subr.h @@ -0,0 +1,49 @@ +/*- + * Copyright (c) 2015 Ian Lepore <ian@freebsd.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _DEV_OFW_OFW_SUBR_H_ +#define _DEV_OFW_OFW_SUBR_H_ + +/* + * Translate an address from the Nth tuple of a device node's reg properties to + * a physical memory address, by applying the range mappings from all ancestors. + * This assumes that all ancestor ranges are simple numerical offsets for which + * addition and subtraction operations will perform the required mapping (the + * bit-options in the high word of standard PCI properties are also handled). + * After the call, *pci_hi (if non-NULL) contains the phys.hi cell of the + * device's parent PCI bus, or OFW_PADDR_NOT_PCI if no PCI bus is involved. + * + * This is intended to be a helper function called by the platform-specific + * implementation of OF_decode_addr(), and not for direct use by device drivers. + */ +#define OFW_PADDR_NOT_PCI (~0) + +int ofw_reg_to_paddr(phandle_t _dev, int _regno, bus_addr_t *_paddr, + bus_size_t *_size, pcell_t *_pci_hi); + +#endif diff --git a/sys/dev/ofw/openfirm.h b/sys/dev/ofw/openfirm.h index d3967a4..acfd2d2 100644 --- a/sys/dev/ofw/openfirm.h +++ b/sys/dev/ofw/openfirm.h @@ -61,6 +61,7 @@ #define _DEV_OPENFIRM_H_ #include <sys/types.h> +#include <machine/_bus.h> /* * Prototypes for Open Firmware Interface Routines @@ -167,5 +168,16 @@ void OF_exit(void) __attribute__((noreturn)); /* User interface functions */ int OF_interpret(const char *cmd, int nreturns, ...); +/* + * Decode the Nth register property of the given device node and create a bus + * space tag and handle for accessing it. This is for use in setting up things + * like early console output before newbus is available. The implementation is + * machine-dependent, and sparc uses a different function signature as well. + */ +#ifndef __sparc64__ +int OF_decode_addr(phandle_t dev, int regno, bus_space_tag_t *ptag, + bus_space_handle_t *phandle); +#endif + #endif /* _KERNEL */ #endif /* _DEV_OPENFIRM_H_ */ diff --git a/sys/dev/pci/pci.c b/sys/dev/pci/pci.c index f5e8470..261aa0c 100644 --- a/sys/dev/pci/pci.c +++ b/sys/dev/pci/pci.c @@ -187,6 +187,8 @@ static device_method_t pci_methods[] = { DEVMETHOD(pci_release_msi, pci_release_msi_method), DEVMETHOD(pci_msi_count, pci_msi_count_method), DEVMETHOD(pci_msix_count, pci_msix_count_method), + DEVMETHOD(pci_msix_pba_bar, pci_msix_pba_bar_method), + DEVMETHOD(pci_msix_table_bar, pci_msix_table_bar_method), DEVMETHOD(pci_get_rid, pci_get_rid_method), DEVMETHOD(pci_child_added, pci_child_added_method), #ifdef PCI_IOV @@ -1851,6 +1853,28 @@ pci_msix_count_method(device_t dev, device_t child) return (0); } +int +pci_msix_pba_bar_method(device_t dev, device_t child) +{ + struct pci_devinfo *dinfo = device_get_ivars(child); + struct pcicfg_msix *msix = &dinfo->cfg.msix; + + if (pci_do_msix && msix->msix_location != 0) + return (msix->msix_pba_bar); + return (-1); +} + +int +pci_msix_table_bar_method(device_t dev, device_t child) +{ + struct pci_devinfo *dinfo = device_get_ivars(child); + struct pcicfg_msix *msix = &dinfo->cfg.msix; + + if (pci_do_msix && msix->msix_location != 0) + return (msix->msix_table_bar); + return (-1); +} + /* * HyperTransport MSI mapping control */ diff --git a/sys/dev/pci/pci_if.m b/sys/dev/pci/pci_if.m index aeeff2c..7049b9f 100644 --- a/sys/dev/pci/pci_if.m +++ b/sys/dev/pci/pci_if.m @@ -36,7 +36,13 @@ CODE { { return (0); } - + + static int + null_msix_bar(device_t dev, device_t child) + { + return (-1); + } + static device_t null_create_iov_child(device_t bus, device_t pf, uint16_t rid, uint16_t vid, uint16_t did) @@ -192,6 +198,16 @@ METHOD int msix_count { device_t child; } DEFAULT null_msi_count; +METHOD int msix_pba_bar { + device_t dev; + device_t child; +} DEFAULT null_msix_bar; + +METHOD int msix_table_bar { + device_t dev; + device_t child; +} DEFAULT null_msix_bar; + METHOD uint16_t get_rid { device_t dev; device_t child; diff --git a/sys/dev/pci/pci_private.h b/sys/dev/pci/pci_private.h index cc45156..7795747 100644 --- a/sys/dev/pci/pci_private.h +++ b/sys/dev/pci/pci_private.h @@ -102,6 +102,8 @@ int pci_remap_msix_method(device_t dev, device_t child, int pci_release_msi_method(device_t dev, device_t child); int pci_msi_count_method(device_t dev, device_t child); int pci_msix_count_method(device_t dev, device_t child); +int pci_msix_pba_bar_method(device_t dev, device_t child); +int pci_msix_table_bar_method(device_t dev, device_t child); struct resource *pci_alloc_resource(device_t dev, device_t child, int type, int *rid, u_long start, u_long end, u_long count, u_int flags); diff --git a/sys/dev/pci/pcivar.h b/sys/dev/pci/pcivar.h index 8b1d6a9..d8df42f 100644 --- a/sys/dev/pci/pcivar.h +++ b/sys/dev/pci/pcivar.h @@ -516,6 +516,18 @@ pci_msix_count(device_t dev) return (PCI_MSIX_COUNT(device_get_parent(dev), dev)); } +static __inline int +pci_msix_pba_bar(device_t dev) +{ + return (PCI_MSIX_PBA_BAR(device_get_parent(dev), dev)); +} + +static __inline int +pci_msix_table_bar(device_t dev) +{ + return (PCI_MSIX_TABLE_BAR(device_get_parent(dev), dev)); +} + static __inline uint16_t pci_get_rid(device_t dev) { diff --git a/sys/dev/puc/pucdata.c b/sys/dev/puc/pucdata.c index 4646fa4..b953146 100644 --- a/sys/dev/puc/pucdata.c +++ b/sys/dev/puc/pucdata.c @@ -64,7 +64,6 @@ static puc_config_f puc_config_timedia; static puc_config_f puc_config_titan; const struct puc_cfg puc_pci_devices[] = { - { 0x0009, 0x7168, 0xffff, 0, "Sunix SUN1889", DEFAULT_RCLK * 8, @@ -179,6 +178,55 @@ const struct puc_cfg puc_pci_devices[] = { .config_function = puc_config_amc }, + /* + * The following members of the Digi International Neo series are + * based on Exar PCI chips, f. e. the 8 port variants on XR17V258IV. + * Accordingly, the PCIe versions of these cards incorporate a PLX + * PCIe-PCI-bridge. + */ + + { 0x114f, 0x00b0, 0xffff, 0, + "Digi Neo PCI 4 Port", + DEFAULT_RCLK * 8, + PUC_PORT_4S, 0x10, 0, -1, + .config_function = puc_config_exar + }, + + { 0x114f, 0x00b1, 0xffff, 0, + "Digi Neo PCI 8 Port", + DEFAULT_RCLK * 8, + PUC_PORT_8S, 0x10, 0, -1, + .config_function = puc_config_exar + }, + + { 0x114f, 0x00f0, 0xffff, 0, + "Digi Neo PCIe 8 Port", + DEFAULT_RCLK * 8, + PUC_PORT_8S, 0x10, 0, -1, + .config_function = puc_config_exar + }, + + { 0x114f, 0x00f1, 0xffff, 0, + "Digi Neo PCIe 4 Port", + DEFAULT_RCLK * 8, + PUC_PORT_4S, 0x10, 0, -1, + .config_function = puc_config_exar + }, + + { 0x114f, 0x00f2, 0xffff, 0, + "Digi Neo PCIe 4 Port RJ45", + DEFAULT_RCLK * 8, + PUC_PORT_4S, 0x10, 0, -1, + .config_function = puc_config_exar + }, + + { 0x114f, 0x00f3, 0xffff, 0, + "Digi Neo PCIe 8 Port RJ45", + DEFAULT_RCLK * 8, + PUC_PORT_8S, 0x10, 0, -1, + .config_function = puc_config_exar + }, + { 0x11fe, 0x8010, 0xffff, 0, "Comtrol RocketPort 550/8 RJ11 part A", DEFAULT_RCLK * 4, @@ -763,6 +811,12 @@ const struct puc_cfg puc_pci_devices[] = { PUC_PORT_4S, 0x10, 0, 8, }, + { 0x1415, 0x950a, 0x131f, 0x2061, + "SIIG Cyber 2SP1 PCIe", + DEFAULT_RCLK * 10, + PUC_PORT_2S, 0x10, 0, 8, + }, + { 0x1415, 0x950a, 0xffff, 0, "Oxford Semiconductor OX16PCI954 UARTs", DEFAULT_RCLK, @@ -840,9 +894,9 @@ const struct puc_cfg puc_pci_devices[] = { */ { 0x1415, 0xc11b, 0xffff, 0, - "Oxford Semiconductor OXPCIe952 1S1P", - DEFAULT_RCLK * 0x22, - PUC_PORT_NONSTANDARD, 0x10, 0, -1, + "Oxford Semiconductor OXPCIe952 1S1P", + DEFAULT_RCLK * 0x22, + PUC_PORT_NONSTANDARD, 0x10, 0, -1, .config_function = puc_config_oxford_pcie }, @@ -1202,9 +1256,10 @@ const struct puc_cfg puc_pci_devices[] = { }; static int -puc_config_amc(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, +puc_config_amc(struct puc_softc *sc __unused, enum puc_cfg_cmd cmd, int port, intptr_t *res) { + switch (cmd) { case PUC_CFG_GET_OFS: *res = 8 * (port & 1); @@ -1236,9 +1291,10 @@ puc_config_diva(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, } static int -puc_config_exar(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, - intptr_t *res) +puc_config_exar(struct puc_softc *sc __unused, enum puc_cfg_cmd cmd, + int port, intptr_t *res) { + if (cmd == PUC_CFG_GET_OFS) { *res = port * 0x200; return (0); @@ -1247,9 +1303,10 @@ puc_config_exar(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, } static int -puc_config_exar_pcie(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, - intptr_t *res) +puc_config_exar_pcie(struct puc_softc *sc __unused, enum puc_cfg_cmd cmd, + int port, intptr_t *res) { + if (cmd == PUC_CFG_GET_OFS) { *res = port * 0x400; return (0); @@ -1258,9 +1315,10 @@ puc_config_exar_pcie(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, } static int -puc_config_icbook(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, - intptr_t *res) +puc_config_icbook(struct puc_softc *sc __unused, enum puc_cfg_cmd cmd, + int port __unused, intptr_t *res) { + if (cmd == PUC_CFG_GET_ILR) { *res = PUC_ILR_DIGI; return (0); @@ -1272,10 +1330,11 @@ static int puc_config_moxa(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, intptr_t *res) { - if (cmd == PUC_CFG_GET_OFS) { - const struct puc_cfg *cfg = sc->sc_cfg; + const struct puc_cfg *cfg = sc->sc_cfg; - if (port == 3 && (cfg->device == 0x1045 || cfg->device == 0x1144)) + if (cmd == PUC_CFG_GET_OFS) { + if (port == 3 && (cfg->device == 0x1045 || + cfg->device == 0x1144)) port = 7; *res = port * 0x200; @@ -1285,8 +1344,8 @@ puc_config_moxa(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, } static int -puc_config_quatech(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, - intptr_t *res) +puc_config_quatech(struct puc_softc *sc, enum puc_cfg_cmd cmd, + int port __unused, intptr_t *res) { const struct puc_cfg *cfg = sc->sc_cfg; struct puc_bar *bar; @@ -1374,8 +1433,8 @@ puc_config_quatech(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, case PUC_CFG_GET_ILR: v0 = (sc->sc_cfg_data >> 8) & 0xff; v1 = sc->sc_cfg_data & 0xff; - *res = (v0 == 0 && v1 == 0x80 + -cfg->clock) - ? PUC_ILR_NONE : PUC_ILR_QUATECH; + *res = (v0 == 0 && v1 == 0x80 + -cfg->clock) ? + PUC_ILR_NONE : PUC_ILR_QUATECH; return (0); default: break; @@ -1687,9 +1746,10 @@ puc_config_sunix(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, } static int -puc_config_titan(struct puc_softc *sc, enum puc_cfg_cmd cmd, int port, - intptr_t *res) +puc_config_titan(struct puc_softc *sc __unused, enum puc_cfg_cmd cmd, + int port, intptr_t *res) { + switch (cmd) { case PUC_CFG_GET_OFS: *res = (port < 3) ? 0 : (port - 2) << 3; diff --git a/sys/dev/random/build.sh b/sys/dev/random/build.sh index 2b4052d..326f8a6 100755 --- a/sys/dev/random/build.sh +++ b/sys/dev/random/build.sh @@ -42,7 +42,6 @@ cc -g -O0 -pthread \ hash.c \ ../../crypto/rijndael/rijndael-api-fst.c \ ../../crypto/rijndael/rijndael-alg-fst.c \ - ../../crypto/sha2/sha2.c \ ../../crypto/sha2/sha256c.c \ -lz \ -o yunit_test @@ -53,7 +52,6 @@ cc -g -O0 -pthread \ hash.c \ ../../crypto/rijndael/rijndael-api-fst.c \ ../../crypto/rijndael/rijndael-alg-fst.c \ - ../../crypto/sha2/sha2.c \ ../../crypto/sha2/sha256c.c \ -lz \ -o funit_test diff --git a/sys/dev/random/fortuna.c b/sys/dev/random/fortuna.c index 4ae006b..800c397 100644 --- a/sys/dev/random/fortuna.c +++ b/sys/dev/random/fortuna.c @@ -50,7 +50,7 @@ __FBSDID("$FreeBSD$"); #include <machine/cpu.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> #include <dev/random/randomdev.h> @@ -68,7 +68,7 @@ __FBSDID("$FreeBSD$"); #include "unit_test.h" #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> #include <dev/random/randomdev.h> diff --git a/sys/dev/random/hash.c b/sys/dev/random/hash.c index 284057c..fe52cd6 100644 --- a/sys/dev/random/hash.c +++ b/sys/dev/random/hash.c @@ -43,7 +43,7 @@ __FBSDID("$FreeBSD$"); #endif /* _KERNEL */ #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> diff --git a/sys/dev/random/other_algorithm.c b/sys/dev/random/other_algorithm.c index 740e879..0c73ef5 100644 --- a/sys/dev/random/other_algorithm.c +++ b/sys/dev/random/other_algorithm.c @@ -55,7 +55,7 @@ __FBSDID("$FreeBSD$"); #include <machine/cpu.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> #include <dev/random/randomdev.h> diff --git a/sys/dev/random/randomdev.c b/sys/dev/random/randomdev.c index f20a462..978a564 100644 --- a/sys/dev/random/randomdev.c +++ b/sys/dev/random/randomdev.c @@ -50,7 +50,7 @@ __FBSDID("$FreeBSD$"); #include <sys/unistd.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> #include <dev/random/randomdev.h> diff --git a/sys/dev/random/unit_test.c b/sys/dev/random/unit_test.c index f682d17..aec2f1d2 100644 --- a/sys/dev/random/unit_test.c +++ b/sys/dev/random/unit_test.c @@ -36,7 +36,7 @@ cc -g -O0 -pthread -DRANDOM_<alg> -I../.. -lstdthreads -Wall \ hash.c \ ../../crypto/rijndael/rijndael-api-fst.c \ ../../crypto/rijndael/rijndael-alg-fst.c \ - ../../crypto/sha2/sha2.c \ + ../../crypto/sha2/sha256c.c \ -lz \ -o unit_test ./unit_test diff --git a/sys/dev/random/yarrow.c b/sys/dev/random/yarrow.c index 5cb1bce..8d5f389 100644 --- a/sys/dev/random/yarrow.c +++ b/sys/dev/random/yarrow.c @@ -41,7 +41,7 @@ __FBSDID("$FreeBSD$"); #include <machine/cpu.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> #include <dev/random/randomdev.h> @@ -60,7 +60,7 @@ __FBSDID("$FreeBSD$"); #include "unit_test.h" #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #include <dev/random/hash.h> #include <dev/random/randomdev.h> diff --git a/sys/dev/rt/if_rt.c b/sys/dev/rt/if_rt.c index b10b581..4deb8de 100644 --- a/sys/dev/rt/if_rt.c +++ b/sys/dev/rt/if_rt.c @@ -1,4 +1,6 @@ /*- + * Copyright (c) 2015, Stanislav Galabov + * Copyright (c) 2014, Aleksandr A. Mityaev * Copyright (c) 2011, Aleksandr Rybalko * based on hard work * by Alexander Egorenkov <egorenar@gmail.com> @@ -56,6 +58,15 @@ __FBSDID("$FreeBSD$"); #include <sys/bus.h> #include <sys/rman.h> +#include "opt_platform.h" +#include "opt_rt305x.h" + +#ifdef FDT +#include <dev/ofw/openfirm.h> +#include <dev/ofw/ofw_bus.h> +#include <dev/ofw/ofw_bus_subr.h> +#endif + #include <dev/mii/mii.h> #include <dev/mii/miivar.h> @@ -78,6 +89,23 @@ __FBSDID("$FreeBSD$"); #define RT_TX_WATCHDOG_TIMEOUT 5 +#define RT_CHIPID_RT3050 0x3050 +#define RT_CHIPID_RT3052 0x3052 +#define RT_CHIPID_RT5350 0x5350 +#define RT_CHIPID_RT6855 0x6855 +#define RT_CHIPID_MT7620 0x7620 + +#ifdef FDT +/* more specific and new models should go first */ +static const struct ofw_compat_data rt_compat_data[] = { + { "ralink,rt6855-eth", (uintptr_t)RT_CHIPID_RT6855 }, + { "ralink,rt5350-eth", (uintptr_t)RT_CHIPID_RT5350 }, + { "ralink,rt3052-eth", (uintptr_t)RT_CHIPID_RT3052 }, + { "ralink,rt305x-eth", (uintptr_t)RT_CHIPID_RT3050 }, + { NULL, (uintptr_t)NULL } +}; +#endif + /* * Static function prototypes */ @@ -96,16 +124,18 @@ static int rt_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data); static void rt_periodic(void *arg); static void rt_tx_watchdog(void *arg); static void rt_intr(void *arg); +static void rt_rt5350_intr(void *arg); static void rt_tx_coherent_intr(struct rt_softc *sc); static void rt_rx_coherent_intr(struct rt_softc *sc); static void rt_rx_delay_intr(struct rt_softc *sc); static void rt_tx_delay_intr(struct rt_softc *sc); -static void rt_rx_intr(struct rt_softc *sc); +static void rt_rx_intr(struct rt_softc *sc, int qid); static void rt_tx_intr(struct rt_softc *sc, int qid); static void rt_rx_done_task(void *context, int pending); static void rt_tx_done_task(void *context, int pending); static void rt_periodic_task(void *context, int pending); -static int rt_rx_eof(struct rt_softc *sc, int limit); +static int rt_rx_eof(struct rt_softc *sc, + struct rt_softc_rx_ring *ring, int limit); static void rt_tx_eof(struct rt_softc *sc, struct rt_softc_tx_ring *ring); static void rt_update_stats(struct rt_softc *sc); @@ -115,7 +145,7 @@ static void rt_intr_enable(struct rt_softc *sc, uint32_t intr_mask); static void rt_intr_disable(struct rt_softc *sc, uint32_t intr_mask); static int rt_txrx_enable(struct rt_softc *sc); static int rt_alloc_rx_ring(struct rt_softc *sc, - struct rt_softc_rx_ring *ring); + struct rt_softc_rx_ring *ring, int qid); static void rt_reset_rx_ring(struct rt_softc *sc, struct rt_softc_rx_ring *ring); static void rt_free_rx_ring(struct rt_softc *sc, @@ -147,8 +177,29 @@ SYSCTL_INT(_hw_rt, OID_AUTO, debug, CTLFLAG_RWTUN, &rt_debug, 0, static int rt_probe(device_t dev) { - device_set_desc(dev, "Ralink RT305XF onChip Ethernet MAC"); - return (BUS_PROBE_NOWILDCARD); + struct rt_softc *sc = device_get_softc(dev); + char buf[80]; +#ifdef FDT + const struct ofw_compat_data * cd; + + cd = ofw_bus_search_compatible(dev, rt_compat_data); + if (cd->ocd_data == (uintptr_t)NULL) + return (ENXIO); + + sc->rt_chipid = (unsigned int)(cd->ocd_data); +#else +#if defined(MT7620) + sc->rt_chipid = RT_CHIPID_MT7620; +#elif defined(RT5350) + sc->rt_chipid = RT_CHIPID_RT5350; +#else + sc->rt_chipid = RT_CHIPID_RT3050; +#endif +#endif + snprintf(buf, sizeof(buf), "Ralink RT%x onChip Ethernet driver", + sc->rt_chipid); + device_set_desc_copy(dev, buf); + return (BUS_PROBE_GENERIC); } /* @@ -240,7 +291,7 @@ ether_request_mac(device_t dev, uint8_t *mac) * "ethaddr" is passed via envp on RedBoot platforms * "kmac" is passed via argv on RouterBOOT platforms */ -#if defined(__U_BOOT__) || defined(__REDBOOT__) || defined(__ROUTERBOOT__) +#if defined(RT305X_UBOOT) || defined(__REDBOOT__) || defined(__ROUTERBOOT__) if ((var = kern_getenv("ethaddr")) != NULL || (var = kern_getenv("kmac")) != NULL ) { @@ -288,6 +339,16 @@ ether_request_mac(device_t dev, uint8_t *mac) return (0); } +/* + * Reset hardware + */ +static void +reset_freng(struct rt_softc *sc) +{ + /* XXX hard reset kills everything so skip it ... */ + return; +} + static int rt_attach(device_t dev) { @@ -331,23 +392,80 @@ rt_attach(device_t dev) "debug", CTLFLAG_RW, &sc->debug, 0, "rt debug level"); #endif - device_printf(dev, "RT305XF Ethernet MAC (rev 0x%08x)\n", - sc->mac_rev); - /* Reset hardware */ - RT_WRITE(sc, GE_PORT_BASE + FE_RST_GLO, PSE_RESET); - - RT_WRITE(sc, GDMA1_BASE + GDMA_FWD_CFG, - ( - GDM_ICS_EN | /* Enable IP Csum */ - GDM_TCS_EN | /* Enable TCP Csum */ - GDM_UCS_EN | /* Enable UDP Csum */ - GDM_STRPCRC | /* Strip CRC from packet */ - GDM_DST_PORT_CPU << GDM_UFRC_P_SHIFT | /* Forward UCast to CPU */ - GDM_DST_PORT_CPU << GDM_BFRC_P_SHIFT | /* Forward BCast to CPU */ - GDM_DST_PORT_CPU << GDM_MFRC_P_SHIFT | /* Forward MCast to CPU */ - GDM_DST_PORT_CPU << GDM_OFRC_P_SHIFT /* Forward Other to CPU */ - )); + reset_freng(sc); + + /* Fill in soc-specific registers map */ + switch(sc->rt_chipid) { + case RT_CHIPID_MT7620: + case RT_CHIPID_RT5350: + device_printf(dev, "RT%x Ethernet MAC (rev 0x%08x)\n", + sc->rt_chipid, sc->mac_rev); + /* RT5350: No GDMA, PSE, CDMA, PPE */ + RT_WRITE(sc, GE_PORT_BASE + 0x0C00, // UDPCS, TCPCS, IPCS=1 + RT_READ(sc, GE_PORT_BASE + 0x0C00) | (0x7<<16)); + sc->delay_int_cfg=RT5350_PDMA_BASE+RT5350_DELAY_INT_CFG; + sc->fe_int_status=RT5350_FE_INT_STATUS; + sc->fe_int_enable=RT5350_FE_INT_ENABLE; + sc->pdma_glo_cfg=RT5350_PDMA_BASE+RT5350_PDMA_GLO_CFG; + sc->pdma_rst_idx=RT5350_PDMA_BASE+RT5350_PDMA_RST_IDX; + for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) { + sc->tx_base_ptr[i]=RT5350_PDMA_BASE+RT5350_TX_BASE_PTR(i); + sc->tx_max_cnt[i]=RT5350_PDMA_BASE+RT5350_TX_MAX_CNT(i); + sc->tx_ctx_idx[i]=RT5350_PDMA_BASE+RT5350_TX_CTX_IDX(i); + sc->tx_dtx_idx[i]=RT5350_PDMA_BASE+RT5350_TX_DTX_IDX(i); + } + sc->rx_ring_count=2; + sc->rx_base_ptr[0]=RT5350_PDMA_BASE+RT5350_RX_BASE_PTR0; + sc->rx_max_cnt[0]=RT5350_PDMA_BASE+RT5350_RX_MAX_CNT0; + sc->rx_calc_idx[0]=RT5350_PDMA_BASE+RT5350_RX_CALC_IDX0; + sc->rx_drx_idx[0]=RT5350_PDMA_BASE+RT5350_RX_DRX_IDX0; + sc->rx_base_ptr[1]=RT5350_PDMA_BASE+RT5350_RX_BASE_PTR1; + sc->rx_max_cnt[1]=RT5350_PDMA_BASE+RT5350_RX_MAX_CNT1; + sc->rx_calc_idx[1]=RT5350_PDMA_BASE+RT5350_RX_CALC_IDX1; + sc->rx_drx_idx[1]=RT5350_PDMA_BASE+RT5350_RX_DRX_IDX1; + sc->int_rx_done_mask=RT5350_INT_RXQ0_DONE; + sc->int_tx_done_mask=RT5350_INT_TXQ0_DONE; + break; + case RT_CHIPID_RT6855: + device_printf(dev, "RT6855 Ethernet MAC (rev 0x%08x)\n", + sc->mac_rev); + break; + default: + device_printf(dev, "RT305XF Ethernet MAC (rev 0x%08x)\n", + sc->mac_rev); + RT_WRITE(sc, GDMA1_BASE + GDMA_FWD_CFG, + ( + GDM_ICS_EN | /* Enable IP Csum */ + GDM_TCS_EN | /* Enable TCP Csum */ + GDM_UCS_EN | /* Enable UDP Csum */ + GDM_STRPCRC | /* Strip CRC from packet */ + GDM_DST_PORT_CPU << GDM_UFRC_P_SHIFT | /* fwd UCast to CPU */ + GDM_DST_PORT_CPU << GDM_BFRC_P_SHIFT | /* fwd BCast to CPU */ + GDM_DST_PORT_CPU << GDM_MFRC_P_SHIFT | /* fwd MCast to CPU */ + GDM_DST_PORT_CPU << GDM_OFRC_P_SHIFT /* fwd Other to CPU */ + )); + + sc->delay_int_cfg=PDMA_BASE+DELAY_INT_CFG; + sc->fe_int_status=GE_PORT_BASE+FE_INT_STATUS; + sc->fe_int_enable=GE_PORT_BASE+FE_INT_ENABLE; + sc->pdma_glo_cfg=PDMA_BASE+PDMA_GLO_CFG; + sc->pdma_glo_cfg=PDMA_BASE+PDMA_GLO_CFG; + sc->pdma_rst_idx=PDMA_BASE+PDMA_RST_IDX; + for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) { + sc->tx_base_ptr[i]=PDMA_BASE+TX_BASE_PTR(i); + sc->tx_max_cnt[i]=PDMA_BASE+TX_MAX_CNT(i); + sc->tx_ctx_idx[i]=PDMA_BASE+TX_CTX_IDX(i); + sc->tx_dtx_idx[i]=PDMA_BASE+TX_DTX_IDX(i); + } + sc->rx_ring_count=1; + sc->rx_base_ptr[0]=PDMA_BASE+RX_BASE_PTR0; + sc->rx_max_cnt[0]=PDMA_BASE+RX_MAX_CNT0; + sc->rx_calc_idx[0]=PDMA_BASE+RX_CALC_IDX0; + sc->rx_drx_idx[0]=PDMA_BASE+RX_DRX_IDX0; + sc->int_rx_done_mask=INT_RX_DONE; + sc->int_tx_done_mask=INT_TXQ0_DONE; + }; /* allocate Tx and Rx rings */ for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) { @@ -360,11 +478,12 @@ rt_attach(device_t dev) } sc->tx_ring_mgtqid = 5; - - error = rt_alloc_rx_ring(sc, &sc->rx_ring); - if (error != 0) { - device_printf(dev, "could not allocate Rx ring\n"); - goto fail; + for (i = 0; i < sc->rx_ring_count; i++) { + error = rt_alloc_rx_ring(sc, &sc->rx_ring[i], i); + if (error != 0) { + device_printf(dev, "could not allocate Rx ring\n"); + goto fail; + } } callout_init(&sc->periodic_ch, 0); @@ -434,7 +553,9 @@ rt_attach(device_t dev) /* set up interrupt */ error = bus_setup_intr(dev, sc->irq, INTR_TYPE_NET | INTR_MPSAFE, - NULL, rt_intr, sc, &sc->irqh); + NULL, (sc->rt_chipid == RT_CHIPID_RT5350 || + sc->rt_chipid == RT_CHIPID_MT7620) ? rt_rt5350_intr : rt_intr, + sc, &sc->irqh); if (error != 0) { printf("%s: could not set up interrupt\n", device_get_nameunit(dev)); @@ -451,7 +572,8 @@ fail: for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) rt_free_tx_ring(sc, &sc->tx_ring[i]); - rt_free_rx_ring(sc, &sc->rx_ring); + for (i = 0; i < sc->rx_ring_count; i++) + rt_free_rx_ring(sc, &sc->rx_ring[i]); mtx_destroy(&sc->lock); @@ -567,8 +689,8 @@ rt_detach(device_t dev) /* free Tx and Rx rings */ for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) rt_free_tx_ring(sc, &sc->tx_ring[i]); - - rt_free_rx_ring(sc, &sc->rx_ring); + for (i = 0; i < sc->rx_ring_count; i++) + rt_free_rx_ring(sc, &sc->rx_ring[i]); RT_SOFTC_UNLOCK(sc); @@ -658,11 +780,12 @@ rt_init_locked(void *priv) RT_SOFTC_ASSERT_LOCKED(sc); /* hardware reset */ - RT_WRITE(sc, GE_PORT_BASE + FE_RST_GLO, PSE_RESET); - rt305x_sysctl_set(SYSCTL_RSTCTRL, SYSCTL_RSTCTRL_FRENG); + //RT_WRITE(sc, GE_PORT_BASE + FE_RST_GLO, PSE_RESET); + //rt305x_sysctl_set(SYSCTL_RSTCTRL, SYSCTL_RSTCTRL_FRENG); /* Fwd to CPU (uni|broad|multi)cast and Unknown */ - RT_WRITE(sc, GDMA1_BASE + GDMA_FWD_CFG, + if(sc->rt_chipid == RT_CHIPID_RT3050 || sc->rt_chipid == RT_CHIPID_RT3052) + RT_WRITE(sc, GDMA1_BASE + GDMA_FWD_CFG, ( GDM_ICS_EN | /* Enable IP Csum */ GDM_TCS_EN | /* Enable TCP Csum */ @@ -675,12 +798,12 @@ rt_init_locked(void *priv) )); /* disable DMA engine */ - RT_WRITE(sc, PDMA_BASE + PDMA_GLO_CFG, 0); - RT_WRITE(sc, PDMA_BASE + PDMA_RST_IDX, 0xffffffff); + RT_WRITE(sc, sc->pdma_glo_cfg, 0); + RT_WRITE(sc, sc->pdma_rst_idx, 0xffffffff); /* wait while DMA engine is busy */ for (ntries = 0; ntries < 100; ntries++) { - tmp = RT_READ(sc, PDMA_BASE + PDMA_GLO_CFG); + tmp = RT_READ(sc, sc->pdma_glo_cfg); if (!(tmp & (FE_TX_DMA_BUSY | FE_RX_DMA_BUSY))) break; DELAY(1000); @@ -698,7 +821,7 @@ rt_init_locked(void *priv) FE_RST_DTX_IDX1 | FE_RST_DTX_IDX0; - RT_WRITE(sc, PDMA_BASE + PDMA_RST_IDX, tmp); + RT_WRITE(sc, sc->pdma_rst_idx, tmp); /* XXX switch set mac address */ for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) @@ -706,36 +829,52 @@ rt_init_locked(void *priv) for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) { /* update TX_BASE_PTRx */ - RT_WRITE(sc, PDMA_BASE + TX_BASE_PTR(i), + RT_WRITE(sc, sc->tx_base_ptr[i], sc->tx_ring[i].desc_phys_addr); - RT_WRITE(sc, PDMA_BASE + TX_MAX_CNT(i), + RT_WRITE(sc, sc->tx_max_cnt[i], RT_SOFTC_TX_RING_DESC_COUNT); - RT_WRITE(sc, PDMA_BASE + TX_CTX_IDX(i), 0); + RT_WRITE(sc, sc->tx_ctx_idx[i], 0); } /* init Rx ring */ - rt_reset_rx_ring(sc, &sc->rx_ring); - - /* update RX_BASE_PTR0 */ - RT_WRITE(sc, PDMA_BASE + RX_BASE_PTR0, - sc->rx_ring.desc_phys_addr); - RT_WRITE(sc, PDMA_BASE + RX_MAX_CNT0, - RT_SOFTC_RX_RING_DATA_COUNT); - RT_WRITE(sc, PDMA_BASE + RX_CALC_IDX0, - RT_SOFTC_RX_RING_DATA_COUNT - 1); + for (i = 0; i < sc->rx_ring_count; i++) + rt_reset_rx_ring(sc, &sc->rx_ring[i]); + + /* update RX_BASE_PTRx */ + for (i = 0; i < sc->rx_ring_count; i++) { + RT_WRITE(sc, sc->rx_base_ptr[i], + sc->rx_ring[i].desc_phys_addr); + RT_WRITE(sc, sc->rx_max_cnt[i], + RT_SOFTC_RX_RING_DATA_COUNT); + RT_WRITE(sc, sc->rx_calc_idx[i], + RT_SOFTC_RX_RING_DATA_COUNT - 1); + } /* write back DDONE, 16byte burst enable RX/TX DMA */ - RT_WRITE(sc, PDMA_BASE + PDMA_GLO_CFG, - FE_TX_WB_DDONE | FE_DMA_BT_SIZE16 | FE_RX_DMA_EN | FE_TX_DMA_EN); + tmp = FE_TX_WB_DDONE | FE_DMA_BT_SIZE16 | FE_RX_DMA_EN | FE_TX_DMA_EN; + if (sc->rt_chipid == RT_CHIPID_MT7620) + tmp |= (1<<31); + RT_WRITE(sc, sc->pdma_glo_cfg, tmp); /* disable interrupts mitigation */ - RT_WRITE(sc, PDMA_BASE + DELAY_INT_CFG, 0); + RT_WRITE(sc, sc->delay_int_cfg, 0); /* clear pending interrupts */ - RT_WRITE(sc, GE_PORT_BASE + FE_INT_STATUS, 0xffffffff); + RT_WRITE(sc, sc->fe_int_status, 0xffffffff); /* enable interrupts */ - tmp = CNT_PPE_AF | + if (sc->rt_chipid == RT_CHIPID_RT5350 || + sc->rt_chipid == RT_CHIPID_MT7620) + tmp = RT5350_INT_TX_COHERENT | + RT5350_INT_RX_COHERENT | + RT5350_INT_TXQ3_DONE | + RT5350_INT_TXQ2_DONE | + RT5350_INT_TXQ1_DONE | + RT5350_INT_TXQ0_DONE | + RT5350_INT_RXQ1_DONE | + RT5350_INT_RXQ0_DONE; + else + tmp = CNT_PPE_AF | CNT_GDM_AF | PSE_P2_FC | GDM_CRC_DROP | @@ -754,7 +893,7 @@ rt_init_locked(void *priv) sc->intr_enable_mask = tmp; - RT_WRITE(sc, GE_PORT_BASE + FE_INT_ENABLE, tmp); + RT_WRITE(sc, sc->fe_int_enable, tmp); if (rt_txrx_enable(sc) != 0) goto fail; @@ -824,12 +963,15 @@ rt_stop_locked(void *priv) RT_SOFTC_LOCK(sc); /* disable interrupts */ - RT_WRITE(sc, GE_PORT_BASE + FE_INT_ENABLE, 0); - - /* reset adapter */ - RT_WRITE(sc, GE_PORT_BASE + FE_RST_GLO, PSE_RESET); + RT_WRITE(sc, sc->fe_int_enable, 0); + + if(sc->rt_chipid == RT_CHIPID_RT5350 || + sc->rt_chipid == RT_CHIPID_MT7620) { + } else { + /* reset adapter */ + RT_WRITE(sc, GE_PORT_BASE + FE_RST_GLO, PSE_RESET); - RT_WRITE(sc, GDMA1_BASE + GDMA_FWD_CFG, + RT_WRITE(sc, GDMA1_BASE + GDMA_FWD_CFG, ( GDM_ICS_EN | /* Enable IP Csum */ GDM_TCS_EN | /* Enable TCP Csum */ @@ -840,6 +982,7 @@ rt_stop_locked(void *priv) GDM_DST_PORT_CPU << GDM_MFRC_P_SHIFT | /* Forward MCast to CPU */ GDM_DST_PORT_CPU << GDM_OFRC_P_SHIFT /* Forward Other to CPU */ )); + } } static void @@ -930,8 +1073,16 @@ rt_tx_data(struct rt_softc *sc, struct mbuf *m, int qid) /* set up Tx descs */ for (i = 0; i < ndmasegs; i += 2) { - /* Set destenation */ - desc->dst = (TXDSCR_DST_PORT_GDMA1); + + /* TODO: this needs to be refined as MT7620 for example has + * a different word3 layout than RT305x and RT5350 (the last + * one doesn't use word3 at all). + */ + + /* Set destination */ + if (sc->rt_chipid != RT_CHIPID_MT7620) + desc->dst = (TXDSCR_DST_PORT_GDMA1); + if ((ifp->if_capenable & IFCAP_TXCSUM) != 0) desc->dst |= (TXDSCR_IP_CSUM_GEN|TXDSCR_UDP_CSUM_GEN| TXDSCR_TCP_CSUM_GEN); @@ -986,7 +1137,7 @@ rt_tx_data(struct rt_softc *sc, struct mbuf *m, int qid) ring->data_cur = (ring->data_cur + 1) % RT_SOFTC_TX_RING_DATA_COUNT; /* kick Tx */ - RT_WRITE(sc, PDMA_BASE + TX_CTX_IDX(qid), ring->desc_cur); + RT_WRITE(sc, sc->tx_ctx_idx[qid], ring->desc_cur); return (0); } @@ -1272,8 +1423,8 @@ rt_intr(void *arg) ifp = sc->ifp; /* acknowledge interrupts */ - status = RT_READ(sc, GE_PORT_BASE + FE_INT_STATUS); - RT_WRITE(sc, GE_PORT_BASE + FE_INT_STATUS, status); + status = RT_READ(sc, sc->fe_int_status); + RT_WRITE(sc, sc->fe_int_status, status); RT_DPRINTF(sc, RT_DEBUG_INTR, "interrupt: status=0x%08x\n", status); @@ -1326,7 +1477,7 @@ rt_intr(void *arg) rt_tx_delay_intr(sc); if (status & INT_RX_DONE) - rt_rx_intr(sc); + rt_rx_intr(sc, 0); if (status & INT_TXQ3_DONE) rt_tx_intr(sc, 3); @@ -1341,6 +1492,56 @@ rt_intr(void *arg) rt_tx_intr(sc, 0); } +/* + * rt_rt5350_intr - main ISR for Ralink 5350 SoC + */ +static void +rt_rt5350_intr(void *arg) +{ + struct rt_softc *sc; + struct ifnet *ifp; + uint32_t status; + + sc = arg; + ifp = sc->ifp; + + /* acknowledge interrupts */ + status = RT_READ(sc, sc->fe_int_status); + RT_WRITE(sc, sc->fe_int_status, status); + + RT_DPRINTF(sc, RT_DEBUG_INTR, "interrupt: status=0x%08x\n", status); + + if (status == 0xffffffff || /* device likely went away */ + status == 0) /* not for us */ + return; + + sc->interrupts++; + + if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) + return; + + if (status & RT5350_INT_TX_COHERENT) + rt_tx_coherent_intr(sc); + if (status & RT5350_INT_RX_COHERENT) + rt_rx_coherent_intr(sc); + if (status & RT5350_RX_DLY_INT) + rt_rx_delay_intr(sc); + if (status & RT5350_TX_DLY_INT) + rt_tx_delay_intr(sc); + if (status & RT5350_INT_RXQ1_DONE) + rt_rx_intr(sc, 1); + if (status & RT5350_INT_RXQ0_DONE) + rt_rx_intr(sc, 0); + if (status & RT5350_INT_TXQ3_DONE) + rt_tx_intr(sc, 3); + if (status & RT5350_INT_TXQ2_DONE) + rt_tx_intr(sc, 2); + if (status & RT5350_INT_TXQ1_DONE) + rt_tx_intr(sc, 1); + if (status & RT5350_INT_TXQ0_DONE) + rt_tx_intr(sc, 0); +} + static void rt_tx_coherent_intr(struct rt_softc *sc) { @@ -1352,19 +1553,19 @@ rt_tx_coherent_intr(struct rt_softc *sc) sc->tx_coherent_interrupts++; /* restart DMA engine */ - tmp = RT_READ(sc, PDMA_BASE + PDMA_GLO_CFG); + tmp = RT_READ(sc, sc->pdma_glo_cfg); tmp &= ~(FE_TX_WB_DDONE | FE_TX_DMA_EN); - RT_WRITE(sc, PDMA_BASE + PDMA_GLO_CFG, tmp); + RT_WRITE(sc, sc->pdma_glo_cfg, tmp); for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) rt_reset_tx_ring(sc, &sc->tx_ring[i]); for (i = 0; i < RT_SOFTC_TX_RING_COUNT; i++) { - RT_WRITE(sc, PDMA_BASE + TX_BASE_PTR(i), + RT_WRITE(sc, sc->tx_base_ptr[i], sc->tx_ring[i].desc_phys_addr); - RT_WRITE(sc, PDMA_BASE + TX_MAX_CNT(i), + RT_WRITE(sc, sc->tx_max_cnt[i], RT_SOFTC_TX_RING_DESC_COUNT); - RT_WRITE(sc, PDMA_BASE + TX_CTX_IDX(i), 0); + RT_WRITE(sc, sc->tx_ctx_idx[i], 0); } rt_txrx_enable(sc); @@ -1377,24 +1578,29 @@ static void rt_rx_coherent_intr(struct rt_softc *sc) { uint32_t tmp; + int i; RT_DPRINTF(sc, RT_DEBUG_INTR, "Rx coherent interrupt\n"); sc->rx_coherent_interrupts++; /* restart DMA engine */ - tmp = RT_READ(sc, PDMA_BASE + PDMA_GLO_CFG); + tmp = RT_READ(sc, sc->pdma_glo_cfg); tmp &= ~(FE_RX_DMA_EN); - RT_WRITE(sc, PDMA_BASE + PDMA_GLO_CFG, tmp); + RT_WRITE(sc, sc->pdma_glo_cfg, tmp); /* init Rx ring */ - rt_reset_rx_ring(sc, &sc->rx_ring); - RT_WRITE(sc, PDMA_BASE + RX_BASE_PTR0, - sc->rx_ring.desc_phys_addr); - RT_WRITE(sc, PDMA_BASE + RX_MAX_CNT0, - RT_SOFTC_RX_RING_DATA_COUNT); - RT_WRITE(sc, PDMA_BASE + RX_CALC_IDX0, - RT_SOFTC_RX_RING_DATA_COUNT - 1); + for (i = 0; i < sc->rx_ring_count; i++) + rt_reset_rx_ring(sc, &sc->rx_ring[i]); + + for (i = 0; i < sc->rx_ring_count; i++) { + RT_WRITE(sc, sc->rx_base_ptr[i], + sc->rx_ring[i].desc_phys_addr); + RT_WRITE(sc, sc->rx_max_cnt[i], + RT_SOFTC_RX_RING_DATA_COUNT); + RT_WRITE(sc, sc->rx_calc_idx[i], + RT_SOFTC_RX_RING_DATA_COUNT - 1); + } rt_txrx_enable(sc); } @@ -1403,19 +1609,22 @@ rt_rx_coherent_intr(struct rt_softc *sc) * rt_rx_intr - a packet received */ static void -rt_rx_intr(struct rt_softc *sc) +rt_rx_intr(struct rt_softc *sc, int qid) { + KASSERT(qid >= 0 && qid < sc->rx_ring_count, + ("%s: Rx interrupt: invalid qid=%d\n", + device_get_nameunit(sc->dev), qid)); RT_DPRINTF(sc, RT_DEBUG_INTR, "Rx interrupt\n"); - sc->rx_interrupts++; + sc->rx_interrupts[qid]++; RT_SOFTC_LOCK(sc); - if (!(sc->intr_disable_mask & INT_RX_DONE)) { - rt_intr_disable(sc, INT_RX_DONE); + if (!(sc->intr_disable_mask & (sc->int_rx_done_mask << qid))) { + rt_intr_disable(sc, (sc->int_rx_done_mask << qid)); taskqueue_enqueue(sc->taskqueue, &sc->rx_done_task); } - sc->intr_pending_mask |= INT_RX_DONE; + sc->intr_pending_mask |= (sc->int_rx_done_mask << qid); RT_SOFTC_UNLOCK(sc); } @@ -1451,12 +1660,12 @@ rt_tx_intr(struct rt_softc *sc, int qid) sc->tx_interrupts[qid]++; RT_SOFTC_LOCK(sc); - if (!(sc->intr_disable_mask & (INT_TXQ0_DONE << qid))) { - rt_intr_disable(sc, (INT_TXQ0_DONE << qid)); + if (!(sc->intr_disable_mask & (sc->int_tx_done_mask << qid))) { + rt_intr_disable(sc, (sc->int_tx_done_mask << qid)); taskqueue_enqueue(sc->taskqueue, &sc->tx_done_task); } - sc->intr_pending_mask |= (INT_TXQ0_DONE << qid); + sc->intr_pending_mask |= (sc->int_tx_done_mask << qid); RT_SOFTC_UNLOCK(sc); } @@ -1478,18 +1687,18 @@ rt_rx_done_task(void *context, int pending) if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) return; - sc->intr_pending_mask &= ~INT_RX_DONE; + sc->intr_pending_mask &= ~sc->int_rx_done_mask; - again = rt_rx_eof(sc, sc->rx_process_limit); + again = rt_rx_eof(sc, &sc->rx_ring[0], sc->rx_process_limit); RT_SOFTC_LOCK(sc); - if ((sc->intr_pending_mask & INT_RX_DONE) || again) { + if ((sc->intr_pending_mask & sc->int_rx_done_mask) || again) { RT_DPRINTF(sc, RT_DEBUG_RX, "Rx done task: scheduling again\n"); taskqueue_enqueue(sc->taskqueue, &sc->rx_done_task); } else { - rt_intr_enable(sc, INT_RX_DONE); + rt_intr_enable(sc, sc->int_rx_done_mask); } RT_SOFTC_UNLOCK(sc); @@ -1515,8 +1724,8 @@ rt_tx_done_task(void *context, int pending) return; for (i = RT_SOFTC_TX_RING_COUNT - 1; i >= 0; i--) { - if (sc->intr_pending_mask & (INT_TXQ0_DONE << i)) { - sc->intr_pending_mask &= ~(INT_TXQ0_DONE << i); + if (sc->intr_pending_mask & (sc->int_tx_done_mask << i)) { + sc->intr_pending_mask &= ~(sc->int_tx_done_mask << i); rt_tx_eof(sc, &sc->tx_ring[i]); } } @@ -1525,7 +1734,15 @@ rt_tx_done_task(void *context, int pending) ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; - intr_mask = ( + if(sc->rt_chipid == RT_CHIPID_RT5350 || + sc->rt_chipid == RT_CHIPID_MT7620) + intr_mask = ( + RT5350_INT_TXQ3_DONE | + RT5350_INT_TXQ2_DONE | + RT5350_INT_TXQ1_DONE | + RT5350_INT_TXQ0_DONE); + else + intr_mask = ( INT_TXQ3_DONE | INT_TXQ2_DONE | INT_TXQ1_DONE | @@ -1584,10 +1801,10 @@ rt_periodic_task(void *context, int pending) * network subsystem. */ static int -rt_rx_eof(struct rt_softc *sc, int limit) +rt_rx_eof(struct rt_softc *sc, struct rt_softc_rx_ring *ring, int limit) { struct ifnet *ifp; - struct rt_softc_rx_ring *ring; +/* struct rt_softc_rx_ring *ring; */ struct rt_rxdesc *desc; struct rt_softc_rx_data *data; struct mbuf *m, *mnew; @@ -1597,12 +1814,12 @@ rt_rx_eof(struct rt_softc *sc, int limit) int error, nsegs, len, nframes; ifp = sc->ifp; - ring = &sc->rx_ring; +/* ring = &sc->rx_ring[0]; */ nframes = 0; while (limit != 0) { - index = RT_READ(sc, PDMA_BASE + RX_DRX_IDX0); + index = RT_READ(sc, sc->rx_drx_idx[0]); if (ring->cur == index) break; @@ -1728,10 +1945,10 @@ skip: } if (ring->cur == 0) - RT_WRITE(sc, PDMA_BASE + RX_CALC_IDX0, + RT_WRITE(sc, sc->rx_calc_idx[0], RT_SOFTC_RX_RING_DATA_COUNT - 1); else - RT_WRITE(sc, PDMA_BASE + RX_CALC_IDX0, + RT_WRITE(sc, sc->rx_calc_idx[0], ring->cur - 1); RT_DPRINTF(sc, RT_DEBUG_RX, "Rx eof: nframes=%d\n", nframes); @@ -1760,7 +1977,7 @@ rt_tx_eof(struct rt_softc *sc, struct rt_softc_tx_ring *ring) nframes = 0; for (;;) { - index = RT_READ(sc, PDMA_BASE + TX_DTX_IDX(ring->qid)); + index = RT_READ(sc, sc->tx_dtx_idx[ring->qid]); if (ring->desc_next == index) break; @@ -1834,12 +2051,13 @@ rt_watchdog(struct rt_softc *sc) #ifdef notyet int ntries; #endif + if(sc->rt_chipid != RT_CHIPID_RT5350 && + sc->rt_chipid != RT_CHIPID_MT7620) { + tmp = RT_READ(sc, PSE_BASE + CDMA_OQ_STA); - tmp = RT_READ(sc, PSE_BASE + CDMA_OQ_STA); - - RT_DPRINTF(sc, RT_DEBUG_WATCHDOG, "watchdog: PSE_IQ_STA=0x%08x\n", - tmp); - + RT_DPRINTF(sc, RT_DEBUG_WATCHDOG, + "watchdog: PSE_IQ_STA=0x%08x\n", tmp); + } /* XXX: do not reset */ #ifdef notyet if (((tmp >> P0_IQ_PCNT_SHIFT) & 0xff) != 0) { @@ -1896,7 +2114,7 @@ rt_intr_enable(struct rt_softc *sc, uint32_t intr_mask) sc->intr_disable_mask &= ~intr_mask; tmp = sc->intr_enable_mask & ~sc->intr_disable_mask; - RT_WRITE(sc, GE_PORT_BASE + FE_INT_ENABLE, tmp); + RT_WRITE(sc, sc->fe_int_enable, tmp); } static void @@ -1906,7 +2124,7 @@ rt_intr_disable(struct rt_softc *sc, uint32_t intr_mask) sc->intr_disable_mask |= intr_mask; tmp = sc->intr_enable_mask & ~sc->intr_disable_mask; - RT_WRITE(sc, GE_PORT_BASE + FE_INT_ENABLE, tmp); + RT_WRITE(sc, sc->fe_int_enable, tmp); } /* @@ -1923,7 +2141,7 @@ rt_txrx_enable(struct rt_softc *sc) /* enable Tx/Rx DMA engine */ for (ntries = 0; ntries < 200; ntries++) { - tmp = RT_READ(sc, PDMA_BASE + PDMA_GLO_CFG); + tmp = RT_READ(sc, sc->pdma_glo_cfg); if (!(tmp & (FE_TX_DMA_BUSY | FE_RX_DMA_BUSY))) break; @@ -1938,7 +2156,7 @@ rt_txrx_enable(struct rt_softc *sc) DELAY(50); tmp |= FE_TX_WB_DDONE | FE_RX_DMA_EN | FE_TX_DMA_EN; - RT_WRITE(sc, PDMA_BASE + PDMA_GLO_CFG, tmp); + RT_WRITE(sc, sc->pdma_glo_cfg, tmp); /* XXX set Rx filter */ return (0); @@ -1948,7 +2166,7 @@ rt_txrx_enable(struct rt_softc *sc) * rt_alloc_rx_ring - allocate RX DMA ring buffer */ static int -rt_alloc_rx_ring(struct rt_softc *sc, struct rt_softc_rx_ring *ring) +rt_alloc_rx_ring(struct rt_softc *sc, struct rt_softc_rx_ring *ring, int qid) { struct rt_rxdesc *desc; struct rt_softc_rx_data *data; @@ -2041,6 +2259,7 @@ rt_alloc_rx_ring(struct rt_softc *sc, struct rt_softc_rx_ring *ring) bus_dmamap_sync(ring->desc_dma_tag, ring->desc_dma_map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + ring->qid = qid; return (0); fail: @@ -2352,7 +2571,7 @@ rt_sysctl_attach(struct rt_softc *sc) "Rx coherent interrupts"); SYSCTL_ADD_ULONG(ctx, SYSCTL_CHILDREN(stats), OID_AUTO, - "rx_interrupts", CTLFLAG_RD, &sc->rx_interrupts, + "rx_interrupts", CTLFLAG_RD, &sc->rx_interrupts[0], "Rx interrupts"); SYSCTL_ADD_ULONG(ctx, SYSCTL_CHILDREN(stats), OID_AUTO, @@ -2606,6 +2825,10 @@ static driver_t rt_driver = static devclass_t rt_dev_class; DRIVER_MODULE(rt, nexus, rt_driver, rt_dev_class, 0, 0); +#ifdef FDT +DRIVER_MODULE(rt, simplebus, rt_driver, rt_dev_class, 0, 0); +#endif + MODULE_DEPEND(rt, ether, 1, 1, 1); MODULE_DEPEND(rt, miibus, 1, 1, 1); diff --git a/sys/dev/rt/if_rtreg.h b/sys/dev/rt/if_rtreg.h index 65b4a8c..56047ef 100644 --- a/sys/dev/rt/if_rtreg.h +++ b/sys/dev/rt/if_rtreg.h @@ -62,6 +62,7 @@ #define FC_DROP_CNT_SHIFT 16 #define PSE_RESET (1<<0) +/* RT305x interrupt registers */ #define FE_INT_STATUS 0x10 #define CNT_PPE_AF (1<<31) #define CNT_GDM_AF (1<<29) @@ -82,6 +83,21 @@ #define TX_DLY_INT (1<<1) /* TXQ[0|1]_DONE with delay */ #define RX_DLY_INT (1<<0) /* RX_DONE with delay */ #define FE_INT_ENABLE 0x14 + +/* RT5350 interrupt registers */ +#define RT5350_FE_INT_STATUS (RT5350_PDMA_BASE + 0x220) +#define RT5350_INT_RX_COHERENT (1<<31) +#define RT5350_RX_DLY_INT (1<<30) +#define RT5350_INT_TX_COHERENT (1<<29) +#define RT5350_TX_DLY_INT (1<<28) +#define RT5350_INT_RXQ1_DONE (1<<17) +#define RT5350_INT_RXQ0_DONE (1<<16) +#define RT5350_INT_TXQ3_DONE (1<<3) +#define RT5350_INT_TXQ2_DONE (1<<2) +#define RT5350_INT_TXQ1_DONE (1<<1) +#define RT5350_INT_TXQ0_DONE (1<<0) +#define RT5350_FE_INT_ENABLE (RT5350_PDMA_BASE + 0x228) + #define MDIO_CFG2 0x18 #define FOE_TS_T 0x1c #define PSE_FQ_PCNT_MASK 0xff000000 @@ -208,7 +224,9 @@ #define P0_IQ_PCNT_SHIFT 0 #define PDMA_BASE 0x0100 +#define RT5350_PDMA_BASE 0x0800 #define PDMA_GLO_CFG 0x00 +#define RT5350_PDMA_GLO_CFG 0x204 #define FE_TX_WB_DDONE (1<<6) #define FE_DMA_BT_SIZE4 (0<<4) #define FE_DMA_BT_SIZE8 (1<<4) @@ -218,6 +236,7 @@ #define FE_TX_DMA_BUSY (1<<1) #define FE_TX_DMA_EN (1<<0) #define PDMA_RST_IDX 0x04 +#define RT5350_PDMA_RST_IDX 0x208 #define FE_RST_DRX_IDX0 (1<<16) #define FE_RST_DTX_IDX3 (1<<3) #define FE_RST_DTX_IDX2 (1<<2) @@ -225,7 +244,9 @@ #define FE_RST_DTX_IDX0 (1<<0) #define PDMA_SCH_CFG 0x08 +#define RT5350_PDMA_SCH_CFG 0x280 #define DELAY_INT_CFG 0x0C +#define RT5350_DELAY_INT_CFG 0x20C #define TXDLY_INT_EN (1<<31) #define TXMAX_PINT_SHIFT 24 #define TXMAX_PTIME_SHIFT 16 @@ -263,6 +284,41 @@ #define TX_CTX_IDX(qid) (((qid>1)?(0x28):(0x18)) + (qid) * 16) #define TX_DTX_IDX(qid) (((qid>1)?(0x2c):(0x1c)) + (qid) * 16) +#define RT5350_TX_BASE_PTR0 0x000 +#define RT5350_TX_MAX_CNT0 0x004 +#define RT5350_TX_CTX_IDX0 0x008 +#define RT5350_TX_DTX_IDX0 0x00C + +#define RT5350_TX_BASE_PTR1 0x010 +#define RT5350_TX_MAX_CNT1 0x014 +#define RT5350_TX_CTX_IDX1 0x018 +#define RT5350_TX_DTX_IDX1 0x01C + +#define RT5350_TX_BASE_PTR2 0x020 +#define RT5350_TX_MAX_CNT2 0x024 +#define RT5350_TX_CTX_IDX2 0x028 +#define RT5350_TX_DTX_IDX2 0x02C + +#define RT5350_TX_BASE_PTR3 0x030 +#define RT5350_TX_MAX_CNT3 0x034 +#define RT5350_TX_CTX_IDX3 0x038 +#define RT5350_TX_DTX_IDX3 0x03C + +#define RT5350_RX_BASE_PTR0 0x100 +#define RT5350_RX_MAX_CNT0 0x104 +#define RT5350_RX_CALC_IDX0 0x108 +#define RT5350_RX_DRX_IDX0 0x10C + +#define RT5350_RX_BASE_PTR1 0x110 +#define RT5350_RX_MAX_CNT1 0x114 +#define RT5350_RX_CALC_IDX1 0x118 +#define RT5350_RX_DRX_IDX1 0x11C + +#define RT5350_TX_BASE_PTR(qid) ((qid) * 0x10 + 0x000) +#define RT5350_TX_MAX_CNT(qid) ((qid) * 0x10 + 0x004) +#define RT5350_TX_CTX_IDX(qid) ((qid) * 0x10 + 0x008) +#define RT5350_TX_DTX_IDX(qid) ((qid) * 0x10 + 0x00C) + #define PPE_BASE 0x0200 #define CNTR_BASE 0x0400 diff --git a/sys/dev/rt/if_rtvar.h b/sys/dev/rt/if_rtvar.h index bd21158..f46574f 100644 --- a/sys/dev/rt/if_rtvar.h +++ b/sys/dev/rt/if_rtvar.h @@ -68,6 +68,7 @@ mtx_assert(&(ring)->lock, MA_OWNED) #define RT_SOFTC_TX_RING_COUNT 4 +#define RT_SOFTC_RX_RING_COUNT 4 #ifndef IF_RT_RING_DATA_COUNT #define IF_RT_RING_DATA_COUNT 128 @@ -151,6 +152,7 @@ struct rt_softc_rx_ring bus_dmamap_t spare_dma_map; struct rt_softc_rx_data data[RT_SOFTC_RX_RING_DATA_COUNT]; int cur; + int qid; }; struct rt_softc_tx_data @@ -216,7 +218,7 @@ struct rt_softc unsigned long periodic_round; struct taskqueue *taskqueue; - struct rt_softc_rx_ring rx_ring; + struct rt_softc_rx_ring rx_ring[RT_SOFTC_RX_RING_COUNT]; struct rt_softc_tx_ring tx_ring[RT_SOFTC_TX_RING_COUNT]; int tx_ring_mgtqid; @@ -227,7 +229,7 @@ struct rt_softc unsigned long interrupts; unsigned long tx_coherent_interrupts; unsigned long rx_coherent_interrupts; - unsigned long rx_interrupts; + unsigned long rx_interrupts[RT_SOFTC_RX_RING_COUNT]; unsigned long rx_delay_interrupts; unsigned long tx_interrupts[RT_SOFTC_TX_RING_COUNT]; unsigned long tx_delay_interrupts; @@ -257,6 +259,25 @@ struct rt_softc #ifdef IF_RT_DEBUG int debug; #endif + + uint32_t rt_chipid; + /* chip specific registers config */ + int rx_ring_count; + uint32_t int_rx_done_mask; + uint32_t int_tx_done_mask; + uint32_t delay_int_cfg; + uint32_t fe_int_status; + uint32_t fe_int_enable; + uint32_t pdma_glo_cfg; + uint32_t pdma_rst_idx; + uint32_t tx_base_ptr[RT_SOFTC_TX_RING_COUNT]; + uint32_t tx_max_cnt[RT_SOFTC_TX_RING_COUNT]; + uint32_t tx_ctx_idx[RT_SOFTC_TX_RING_COUNT]; + uint32_t tx_dtx_idx[RT_SOFTC_TX_RING_COUNT]; + uint32_t rx_base_ptr[RT_SOFTC_RX_RING_COUNT]; + uint32_t rx_max_cnt[RT_SOFTC_RX_RING_COUNT]; + uint32_t rx_calc_idx[RT_SOFTC_RX_RING_COUNT]; + uint32_t rx_drx_idx[RT_SOFTC_RX_RING_COUNT]; }; #ifdef IF_RT_DEBUG @@ -274,7 +295,7 @@ enum #define RT_DPRINTF(sc, m, fmt, ...) \ do { if ((sc)->debug & (m)) \ - device_printf(sc->dev, fmt, __VA_ARGS__); } while (0) + device_printf(sc->dev, fmt, ## __VA_ARGS__); } while (0) #else #define RT_DPRINTF(sc, m, fmt, ...) #endif /* #ifdef IF_RT_DEBUG */ diff --git a/sys/dev/usb/controller/xhci.h b/sys/dev/usb/controller/xhci.h index ac21c0f..4857450 100644 --- a/sys/dev/usb/controller/xhci.h +++ b/sys/dev/usb/controller/xhci.h @@ -465,6 +465,7 @@ struct xhci_softc { struct usb_device *sc_devices[XHCI_MAX_DEVICES]; struct resource *sc_io_res; struct resource *sc_irq_res; + struct resource *sc_msix_res; void *sc_intr_hdl; bus_size_t sc_io_size; diff --git a/sys/dev/usb/controller/xhci_pci.c b/sys/dev/usb/controller/xhci_pci.c index e6f937f..0a8a95b 100644 --- a/sys/dev/usb/controller/xhci_pci.c +++ b/sys/dev/usb/controller/xhci_pci.c @@ -148,6 +148,8 @@ xhci_pci_probe(device_t self) static int xhci_use_msi = 1; TUNABLE_INT("hw.usb.xhci.msi", &xhci_use_msi); +static int xhci_use_msix = 1; +TUNABLE_INT("hw.usb.xhci.msix", &xhci_use_msix); static void xhci_interrupt_poll(void *_sc) @@ -188,7 +190,7 @@ static int xhci_pci_attach(device_t self) { struct xhci_softc *sc = device_get_softc(self); - int count, err, rid; + int count, err, msix_table, rid; uint8_t usemsi = 1; uint8_t usedma32 = 0; @@ -240,7 +242,27 @@ xhci_pci_attach(device_t self) usb_callout_init_mtx(&sc->sc_callout, &sc->sc_bus.bus_mtx, 0); rid = 0; - if (xhci_use_msi && usemsi) { + if (xhci_use_msix && (msix_table = pci_msix_table_bar(self)) >= 0) { + sc->sc_msix_res = bus_alloc_resource_any(self, SYS_RES_MEMORY, + &msix_table, RF_ACTIVE); + if (sc->sc_msix_res == NULL) { + /* May not be enabled */ + device_printf(self, + "Unable to map MSI-X table \n"); + } else { + count = 1; + if (pci_alloc_msix(self, &count) == 0) { + if (bootverbose) + device_printf(self, "MSI-X enabled\n"); + rid = 1; + } else { + bus_release_resource(self, SYS_RES_MEMORY, + msix_table, sc->sc_msix_res); + sc->sc_msix_res = NULL; + } + } + } + if (rid == 0 && xhci_use_msi && usemsi) { count = 1; if (pci_alloc_msi(self, &count) == 0) { if (bootverbose) @@ -341,6 +363,11 @@ xhci_pci_detach(device_t self) sc->sc_io_res); sc->sc_io_res = NULL; } + if (sc->sc_msix_res) { + bus_release_resource(self, SYS_RES_MEMORY, + rman_get_rid(sc->sc_msix_res), sc->sc_msix_res); + sc->sc_msix_res = NULL; + } xhci_uninit(sc); diff --git a/sys/dev/usb/usb_lookup.c b/sys/dev/usb/usb_lookup.c index f3dc551..86e1a82 100644 --- a/sys/dev/usb/usb_lookup.c +++ b/sys/dev/usb/usb_lookup.c @@ -154,28 +154,10 @@ usbd_lookup_id_by_uaa(const struct usb_device_id *id, usb_size_t sizeof_id, /*------------------------------------------------------------------------* * Export the USB device ID format we use to userspace tools. *------------------------------------------------------------------------*/ -#if BYTE_ORDER == BIG_ENDIAN -#define U16_XOR "8" -#define U32_XOR "12" -#define U64_XOR "56" -#define U8_BITFIELD_XOR "7" -#define U16_BITFIELD_XOR "15" -#define U32_BITFIELD_XOR "31" -#define U64_BITFIELD_XOR "63" -#else +#if BYTE_ORDER == LITTLE_ENDIAN #define U16_XOR "0" -#define U32_XOR "0" -#define U64_XOR "0" -#define U8_BITFIELD_XOR "0" -#define U16_BITFIELD_XOR "0" -#define U32_BITFIELD_XOR "0" -#define U64_BITFIELD_XOR "0" -#endif - -#if USB_HAVE_COMPAT_LINUX -#define MFL_SIZE "1" #else -#define MFL_SIZE "0" +#define U16_XOR "8" #endif #if defined(KLD_MODULE) && (USB_HAVE_ID_SECTION != 0) @@ -189,20 +171,19 @@ static const char __section("bus_autoconf_format") __used usb_id_format[] = { /* List size of fields in the usb_device_id structure */ -#if ULONG_MAX >= 0xFFFFFFFFUL - "unused{0,8}" - "unused{0,8}" - "unused{0,8}" - "unused{0,8}" -#if ULONG_MAX >= 0xFFFFFFFFFFFFFFFFULL - "unused{0,8}" - "unused{0,8}" - "unused{0,8}" - "unused{0,8}" -#endif -#else -#error "Please update code." -#endif + "mf_vendor{" U16_XOR ",1}" + "mf_product{" U16_XOR ",1}" + "mf_dev_lo{" U16_XOR ",1}" + "mf_dev_hi{" U16_XOR ",1}" + + "mf_dev_class{" U16_XOR ",1}" + "mf_dev_subclass{" U16_XOR ",1}" + "mf_dev_protocol{" U16_XOR ",1}" + "mf_int_class{" U16_XOR ",1}" + + "mf_int_subclass{" U16_XOR ",1}" + "mf_int_protocol{" U16_XOR ",1}" + "unused{" U16_XOR ",6}" "idVendor[0]{" U16_XOR ",8}" "idVendor[1]{" U16_XOR ",8}" @@ -220,38 +201,20 @@ static const char __section("bus_autoconf_format") __used usb_id_format[] = { "bInterfaceSubClass{0,8}" "bInterfaceProtocol{0,8}" - "mf_vendor{" U8_BITFIELD_XOR ",1}" - "mf_product{" U8_BITFIELD_XOR ",1}" - "mf_dev_lo{" U8_BITFIELD_XOR ",1}" - "mf_dev_hi{" U8_BITFIELD_XOR ",1}" - - "mf_dev_class{" U8_BITFIELD_XOR ",1}" - "mf_dev_subclass{" U8_BITFIELD_XOR ",1}" - "mf_dev_protocol{" U8_BITFIELD_XOR ",1}" - "mf_int_class{" U8_BITFIELD_XOR ",1}" - - "mf_int_subclass{" U8_BITFIELD_XOR ",1}" - "mf_int_protocol{" U8_BITFIELD_XOR ",1}" - "unused{" U8_BITFIELD_XOR ",6}" - - "mfl_vendor{" U16_XOR "," MFL_SIZE "}" - "mfl_product{" U16_XOR "," MFL_SIZE "}" - "mfl_dev_lo{" U16_XOR "," MFL_SIZE "}" - "mfl_dev_hi{" U16_XOR "," MFL_SIZE "}" - - "mfl_dev_class{" U16_XOR "," MFL_SIZE "}" - "mfl_dev_subclass{" U16_XOR "," MFL_SIZE "}" - "mfl_dev_protocol{" U16_XOR "," MFL_SIZE "}" - "mfl_int_class{" U16_XOR "," MFL_SIZE "}" - - "mfl_int_subclass{" U16_XOR "," MFL_SIZE "}" - "mfl_int_protocol{" U16_XOR "," MFL_SIZE "}" - "unused{" U16_XOR "," MFL_SIZE "}" - "unused{" U16_XOR "," MFL_SIZE "}" - - "unused{" U16_XOR "," MFL_SIZE "}" - "unused{" U16_XOR "," MFL_SIZE "}" - "unused{" U16_XOR "," MFL_SIZE "}" - "unused{" U16_XOR "," MFL_SIZE "}" +#if USB_HAVE_COMPAT_LINUX + "mfl_vendor{" U16_XOR ",1}" + "mfl_product{" U16_XOR ",1}" + "mfl_dev_lo{" U16_XOR ",1}" + "mfl_dev_hi{" U16_XOR ",1}" + + "mfl_dev_class{" U16_XOR ",1}" + "mfl_dev_subclass{" U16_XOR ",1}" + "mfl_dev_protocol{" U16_XOR ",1}" + "mfl_int_class{" U16_XOR ",1}" + + "mfl_int_subclass{" U16_XOR ",1}" + "mfl_int_protocol{" U16_XOR ",1}" + "unused{" U16_XOR ",6}" +#endif }; #endif diff --git a/sys/dev/usb/usbdi.h b/sys/dev/usb/usbdi.h index 74453ab..ab7cc1a 100644 --- a/sys/dev/usb/usbdi.h +++ b/sys/dev/usb/usbdi.h @@ -267,7 +267,7 @@ struct usb_config { struct usb_device_id { /* Select which fields to match against */ -#if _BYTE_ORDER == _LITTLE_ENDIAN +#if BYTE_ORDER == LITTLE_ENDIAN uint16_t match_flag_vendor:1, match_flag_product:1, @@ -315,13 +315,6 @@ struct usb_device_id { uint8_t bInterfaceSubClass; uint8_t bInterfaceProtocol; - /* Hook for driver specific information */ - unsigned long driver_info; - -/* - * XXX can't currently participate in auto driver loading - * XXX making it a union with the match_flag_* above messes up init - */ #if USB_HAVE_COMPAT_LINUX /* which fields to match against */ uint16_t match_flags; @@ -336,6 +329,9 @@ struct usb_device_id { #define USB_DEVICE_ID_MATCH_INT_SUBCLASS 0x0100 #define USB_DEVICE_ID_MATCH_INT_PROTOCOL 0x0200 #endif + + /* Hook for driver specific information */ + unsigned long driver_info; } __aligned(32); #define USB_STD_PNP_INFO "M16:mask;U16:vendor;U16:product;L16:product;G16:product;" \ diff --git a/sys/fs/cuse/cuse.c b/sys/fs/cuse/cuse.c index 7284b81..075b2d9 100644 --- a/sys/fs/cuse/cuse.c +++ b/sys/fs/cuse/cuse.c @@ -108,6 +108,7 @@ struct cuse_server { TAILQ_HEAD(, cuse_client) hcli; struct cv cv; struct selinfo selinfo; + pid_t pid; int is_closing; int refs; }; @@ -510,7 +511,7 @@ cuse_client_is_closing(struct cuse_client *pcc) static void cuse_client_send_command_locked(struct cuse_client_command *pccmd, - unsigned long data_ptr, unsigned long arg, int fflags, int ioflag) + uintptr_t data_ptr, unsigned long arg, int fflags, int ioflag) { unsigned long cuse_fflags = 0; struct cuse_server *pcs; @@ -691,6 +692,10 @@ cuse_server_open(struct cdev *dev, int fflags, int devtype, struct thread *td) free(pcs, M_CUSE); return (ENOMEM); } + + /* store current process ID */ + pcs->pid = curproc->p_pid; + TAILQ_INIT(&pcs->head); TAILQ_INIT(&pcs->hdev); TAILQ_INIT(&pcs->hcli); @@ -1132,7 +1137,7 @@ cuse_server_ioctl(struct cdev *dev, unsigned long cmd, if (pccmd != NULL) { pcc = pccmd->client; for (n = 0; n != CUSE_CMD_MAX; n++) { - pcc->cmds[n].sub.per_file_handle = *(unsigned long *)data; + pcc->cmds[n].sub.per_file_handle = *(uintptr_t *)data; } } else { error = ENXIO; @@ -1357,9 +1362,15 @@ cuse_client_open(struct cdev *dev, int fflags, int devtype, struct thread *td) if (pcsd != NULL) { pcs = pcsd->server; pcd = pcsd->user_dev; + /* + * Check that the refcount didn't wrap and that the + * same process is not both client and server. This + * can easily lead to deadlocks when destroying the + * CUSE character device nodes: + */ pcs->refs++; - if (pcs->refs < 0) { - /* overflow */ + if (pcs->refs < 0 || pcs->pid == curproc->p_pid) { + /* overflow or wrong PID */ pcs->refs--; pcsd = NULL; } @@ -1536,7 +1547,7 @@ cuse_client_read(struct cdev *dev, struct uio *uio, int ioflag) cuse_lock(); cuse_client_send_command_locked(pccmd, - (unsigned long)uio->uio_iov->iov_base, + (uintptr_t)uio->uio_iov->iov_base, (unsigned long)(unsigned int)len, pcc->fflags, ioflag); error = cuse_client_receive_command_locked(pccmd, 0, 0); @@ -1596,7 +1607,7 @@ cuse_client_write(struct cdev *dev, struct uio *uio, int ioflag) cuse_lock(); cuse_client_send_command_locked(pccmd, - (unsigned long)uio->uio_iov->iov_base, + (uintptr_t)uio->uio_iov->iov_base, (unsigned long)(unsigned int)len, pcc->fflags, ioflag); error = cuse_client_receive_command_locked(pccmd, 0, 0); diff --git a/sys/fs/cuse/cuse_ioctl.h b/sys/fs/cuse/cuse_ioctl.h index 8e1867c..ff0393a 100644 --- a/sys/fs/cuse/cuse_ioctl.h +++ b/sys/fs/cuse/cuse_ioctl.h @@ -40,8 +40,8 @@ struct cuse_dev; struct cuse_data_chunk { - unsigned long local_ptr; - unsigned long peer_ptr; + uintptr_t local_ptr; + uintptr_t peer_ptr; unsigned long length; }; @@ -53,8 +53,8 @@ struct cuse_alloc_info { struct cuse_command { struct cuse_dev *dev; unsigned long fflags; - unsigned long per_file_handle; - unsigned long data_pointer; + uintptr_t per_file_handle; + uintptr_t data_pointer; unsigned long argument; unsigned long command; /* see CUSE_CMD_XXX */ }; @@ -76,7 +76,7 @@ struct cuse_create_dev { #define CUSE_IOCTL_GET_SIG _IOR('C', 4, int) #define CUSE_IOCTL_ALLOC_MEMORY _IOW('C', 5, struct cuse_alloc_info) #define CUSE_IOCTL_FREE_MEMORY _IOW('C', 6, struct cuse_alloc_info) -#define CUSE_IOCTL_SET_PFH _IOW('C', 7, unsigned long) +#define CUSE_IOCTL_SET_PFH _IOW('C', 7, uintptr_t) #define CUSE_IOCTL_CREATE_DEV _IOW('C', 8, struct cuse_create_dev) #define CUSE_IOCTL_DESTROY_DEV _IOW('C', 9, struct cuse_dev *) #define CUSE_IOCTL_ALLOC_UNIT _IOR('C',10, int) diff --git a/sys/fs/devfs/devfs_vnops.c b/sys/fs/devfs/devfs_vnops.c index 5c45bd8..eb2f98a 100644 --- a/sys/fs/devfs/devfs_vnops.c +++ b/sys/fs/devfs/devfs_vnops.c @@ -557,7 +557,9 @@ devfs_access(struct vop_access_args *ap) return (error); } -/* ARGSUSED */ +_Static_assert(((FMASK | FCNTLFLAGS) & (FLASTCLOSE | FREVOKE)) == 0, + "devfs-only flag reuse failed"); + static int devfs_close(struct vop_close_args *ap) { @@ -566,7 +568,7 @@ devfs_close(struct vop_close_args *ap) struct proc *p; struct cdev *dev = vp->v_rdev; struct cdevsw *dsw; - int vp_locked, error, ref; + int dflags, error, ref, vp_locked; /* * XXX: Don't call d_close() if we were called because of @@ -621,9 +623,11 @@ devfs_close(struct vop_close_args *ap) dsw = dev_refthread(dev, &ref); if (dsw == NULL) return (ENXIO); + dflags = 0; VI_LOCK(vp); if (vp->v_iflag & VI_DOOMED) { /* Forced close. */ + dflags |= FREVOKE | FNONBLOCK; } else if (dsw->d_flags & D_TRACKCLOSE) { /* Keep device updated on status. */ } else if (count_dev(dev) > 1) { @@ -631,13 +635,15 @@ devfs_close(struct vop_close_args *ap) dev_relthread(dev, ref); return (0); } + if (count_dev(dev) == 1) + dflags |= FLASTCLOSE; vholdl(vp); VI_UNLOCK(vp); vp_locked = VOP_ISLOCKED(vp); VOP_UNLOCK(vp, 0); KASSERT(dev->si_refcount > 0, ("devfs_close() on un-referenced struct cdev *(%s)", devtoname(dev))); - error = dsw->d_close(dev, ap->a_fflag, S_IFCHR, td); + error = dsw->d_close(dev, ap->a_fflag | dflags, S_IFCHR, td); dev_relthread(dev, ref); vn_lock(vp, vp_locked | LK_RETRY); vdrop(vp); @@ -1560,11 +1566,15 @@ devfs_setattr(struct vop_setattr_args *ap) return (EINVAL); } + error = devfs_populate_vp(vp); + if (error != 0) + return (error); + de = vp->v_data; if (vp->v_type == VDIR) de = de->de_dir; - error = c = 0; + c = 0; if (vap->va_uid == (uid_t)VNOVAL) uid = de->de_uid; else @@ -1577,8 +1587,8 @@ devfs_setattr(struct vop_setattr_args *ap) if ((ap->a_cred->cr_uid != de->de_uid) || uid != de->de_uid || (gid != de->de_gid && !groupmember(gid, ap->a_cred))) { error = priv_check(td, PRIV_VFS_CHOWN); - if (error) - return (error); + if (error != 0) + goto ret; } de->de_uid = uid; de->de_gid = gid; @@ -1588,8 +1598,8 @@ devfs_setattr(struct vop_setattr_args *ap) if (vap->va_mode != (mode_t)VNOVAL) { if (ap->a_cred->cr_uid != de->de_uid) { error = priv_check(td, PRIV_VFS_ADMIN); - if (error) - return (error); + if (error != 0) + goto ret; } de->de_mode = vap->va_mode; c = 1; @@ -1598,7 +1608,7 @@ devfs_setattr(struct vop_setattr_args *ap) if (vap->va_atime.tv_sec != VNOVAL || vap->va_mtime.tv_sec != VNOVAL) { error = vn_utimes_perm(vp, vap, ap->a_cred, td); if (error != 0) - return (error); + goto ret; if (vap->va_atime.tv_sec != VNOVAL) { if (vp->v_type == VCHR) vp->v_rdev->si_atime = vap->va_atime; @@ -1620,7 +1630,10 @@ devfs_setattr(struct vop_setattr_args *ap) else vfs_timestamp(&de->de_mtime); } - return (0); + +ret: + sx_xunlock(&VFSTODEVFS(vp->v_mount)->dm_lock); + return (error); } #ifdef MAC diff --git a/sys/fs/ext2fs/ext2fs.h b/sys/fs/ext2fs/ext2fs.h index ec19300..756803f 100644 --- a/sys/fs/ext2fs/ext2fs.h +++ b/sys/fs/ext2fs/ext2fs.h @@ -187,6 +187,7 @@ struct csum { #define EXT2F_INCOMPAT_COMP 0x0001 #define EXT2F_INCOMPAT_FTYPE 0x0002 +#define EXT2F_INCOMPAT_RECOVER 0x0004 #define EXT2F_INCOMPAT_META_BG 0x0010 #define EXT2F_INCOMPAT_EXTENTS 0x0040 #define EXT2F_INCOMPAT_64BIT 0x0080 @@ -208,6 +209,7 @@ struct csum { * * We do not support these EXT4 features but they are irrelevant * for read-only support: + * - EXT2F_INCOMPAT_RECOVER * - EXT2F_INCOMPAT_FLEX_BG * - EXT2F_INCOMPAT_META_BG */ @@ -216,6 +218,7 @@ struct csum { EXT2F_ROCOMPAT_EXTRA_ISIZE) #define EXT2F_INCOMPAT_SUPP EXT2F_INCOMPAT_FTYPE #define EXT4F_RO_INCOMPAT_SUPP (EXT2F_INCOMPAT_EXTENTS | \ + EXT2F_INCOMPAT_RECOVER | \ EXT2F_INCOMPAT_FLEX_BG | \ EXT2F_INCOMPAT_META_BG ) diff --git a/sys/geom/bde/g_bde.c b/sys/geom/bde/g_bde.c index 93d7733..7eda17f 100644 --- a/sys/geom/bde/g_bde.c +++ b/sys/geom/bde/g_bde.c @@ -44,7 +44,7 @@ #include <sys/sysctl.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha512.h> #include <geom/geom.h> #include <geom/bde/g_bde.h> #define BDE_CLASS_NAME "BDE" diff --git a/sys/geom/bde/g_bde_crypt.c b/sys/geom/bde/g_bde_crypt.c index c5de0b9..c59b85a 100644 --- a/sys/geom/bde/g_bde_crypt.c +++ b/sys/geom/bde/g_bde_crypt.c @@ -47,7 +47,7 @@ #include <sys/md5.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha512.h> #include <geom/geom.h> #include <geom/bde/g_bde.h> diff --git a/sys/geom/bde/g_bde_lock.c b/sys/geom/bde/g_bde_lock.c index 855987e..462b92b 100644 --- a/sys/geom/bde/g_bde_lock.c +++ b/sys/geom/bde/g_bde_lock.c @@ -58,7 +58,7 @@ #endif #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha512.h> #include <geom/geom.h> #include <geom/bde/g_bde.h> diff --git a/sys/geom/bde/g_bde_work.c b/sys/geom/bde/g_bde_work.c index fceb865..7c21d3f 100644 --- a/sys/geom/bde/g_bde_work.c +++ b/sys/geom/bde/g_bde_work.c @@ -71,7 +71,7 @@ #include <sys/kthread.h> #include <crypto/rijndael/rijndael-api-fst.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha512.h> #include <geom/geom.h> #include <geom/bde/g_bde.h> diff --git a/sys/geom/eli/g_eli.h b/sys/geom/eli/g_eli.h index 36ef555..e4dbee6 100644 --- a/sys/geom/eli/g_eli.h +++ b/sys/geom/eli/g_eli.h @@ -32,7 +32,8 @@ #include <sys/endian.h> #include <sys/errno.h> #include <sys/malloc.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> +#include <crypto/sha2/sha512.h> #include <opencrypto/cryptodev.h> #ifdef _KERNEL #include <sys/bio.h> diff --git a/sys/geom/geom_map.c b/sys/geom/geom_map.c index bfd6fd8..cfd886b 100644 --- a/sys/geom/geom_map.c +++ b/sys/geom/geom_map.c @@ -259,7 +259,7 @@ g_map_parse_part(struct g_class *mp, struct g_provider *pp, } if (find_marker(cp, value, &end) != 0) { if (bootverbose) { - printf("MAP: \"%s\" can't parse/use start value\n", + printf("MAP: \"%s\" can't parse/use end value\n", name); } return (1); diff --git a/sys/geom/part/g_part.c b/sys/geom/part/g_part.c index 52a554d..a751398 100644 --- a/sys/geom/part/g_part.c +++ b/sys/geom/part/g_part.c @@ -78,6 +78,19 @@ struct g_part_alias_list { { "apple-tv-recovery", G_PART_ALIAS_APPLE_TV_RECOVERY }, { "apple-ufs", G_PART_ALIAS_APPLE_UFS }, { "bios-boot", G_PART_ALIAS_BIOS_BOOT }, + { "chromeos-firmware", G_PART_ALIAS_CHROMEOS_FIRMWARE }, + { "chromeos-kernel", G_PART_ALIAS_CHROMEOS_KERNEL }, + { "chromeos-reserved", G_PART_ALIAS_CHROMEOS_RESERVED }, + { "chromeos-root", G_PART_ALIAS_CHROMEOS_ROOT }, + { "dragonfly-ccd", G_PART_ALIAS_DFBSD_CCD }, + { "dragonfly-hammer", G_PART_ALIAS_DFBSD_HAMMER }, + { "dragonfly-hammer2", G_PART_ALIAS_DFBSD_HAMMER2 }, + { "dragonfly-label32", G_PART_ALIAS_DFBSD }, + { "dragonfly-label64", G_PART_ALIAS_DFBSD64 }, + { "dragonfly-legacy", G_PART_ALIAS_DFBSD_LEGACY }, + { "dragonfly-swap", G_PART_ALIAS_DFBSD_SWAP }, + { "dragonfly-ufs", G_PART_ALIAS_DFBSD_UFS }, + { "dragonfly-vinum", G_PART_ALIAS_DFBSD_VINUM }, { "ebr", G_PART_ALIAS_EBR }, { "efi", G_PART_ALIAS_EFI }, { "fat16", G_PART_ALIAS_MS_FAT16 }, @@ -97,28 +110,22 @@ struct g_part_alias_list { { "ms-basic-data", G_PART_ALIAS_MS_BASIC_DATA }, { "ms-ldm-data", G_PART_ALIAS_MS_LDM_DATA }, { "ms-ldm-metadata", G_PART_ALIAS_MS_LDM_METADATA }, + { "ms-recovery", G_PART_ALIAS_MS_RECOVERY }, { "ms-reserved", G_PART_ALIAS_MS_RESERVED }, - { "ntfs", G_PART_ALIAS_MS_NTFS }, + { "ms-spaces", G_PART_ALIAS_MS_SPACES }, { "netbsd-ccd", G_PART_ALIAS_NETBSD_CCD }, { "netbsd-cgd", G_PART_ALIAS_NETBSD_CGD }, { "netbsd-ffs", G_PART_ALIAS_NETBSD_FFS }, { "netbsd-lfs", G_PART_ALIAS_NETBSD_LFS }, { "netbsd-raid", G_PART_ALIAS_NETBSD_RAID }, { "netbsd-swap", G_PART_ALIAS_NETBSD_SWAP }, + { "ntfs", G_PART_ALIAS_MS_NTFS }, + { "openbsd-data", G_PART_ALIAS_OPENBSD_DATA }, + { "prep-boot", G_PART_ALIAS_PREP_BOOT }, + { "vmware-reserved", G_PART_ALIAS_VMRESERVED }, { "vmware-vmfs", G_PART_ALIAS_VMFS }, { "vmware-vmkdiag", G_PART_ALIAS_VMKDIAG }, - { "vmware-reserved", G_PART_ALIAS_VMRESERVED }, { "vmware-vsanhdr", G_PART_ALIAS_VMVSANHDR }, - { "dragonfly-label32", G_PART_ALIAS_DFBSD }, - { "dragonfly-label64", G_PART_ALIAS_DFBSD64 }, - { "dragonfly-swap", G_PART_ALIAS_DFBSD_SWAP }, - { "dragonfly-ufs", G_PART_ALIAS_DFBSD_UFS }, - { "dragonfly-vinum", G_PART_ALIAS_DFBSD_VINUM }, - { "dragonfly-ccd", G_PART_ALIAS_DFBSD_CCD }, - { "dragonfly-legacy", G_PART_ALIAS_DFBSD_LEGACY }, - { "dragonfly-hammer", G_PART_ALIAS_DFBSD_HAMMER }, - { "dragonfly-hammer2", G_PART_ALIAS_DFBSD_HAMMER2 }, - { "prep-boot", G_PART_ALIAS_PREP_BOOT }, }; SYSCTL_DECL(_kern_geom); diff --git a/sys/geom/part/g_part.h b/sys/geom/part/g_part.h index 39b7f95..14db6a0 100644 --- a/sys/geom/part/g_part.h +++ b/sys/geom/part/g_part.h @@ -37,12 +37,28 @@ enum g_part_alias { G_PART_ALIAS_APPLE_BOOT, /* An Apple boot partition entry. */ + G_PART_ALIAS_APPLE_CORE_STORAGE,/* An Apple Core Storage partition. */ G_PART_ALIAS_APPLE_HFS, /* An HFS+ file system entry. */ G_PART_ALIAS_APPLE_LABEL, /* An Apple label partition entry. */ G_PART_ALIAS_APPLE_RAID, /* An Apple RAID partition entry. */ G_PART_ALIAS_APPLE_RAID_OFFLINE,/* An Apple RAID (offline) part entry.*/ G_PART_ALIAS_APPLE_TV_RECOVERY, /* An Apple TV recovery part entry. */ G_PART_ALIAS_APPLE_UFS, /* An Apple UFS partition entry. */ + G_PART_ALIAS_BIOS_BOOT, /* A GRUB 2 boot partition entry. */ + G_PART_ALIAS_CHROMEOS_FIRMWARE, /* A ChromeOS firmware part. entry. */ + G_PART_ALIAS_CHROMEOS_KERNEL, /* A ChromeOS Kernel part. entry. */ + G_PART_ALIAS_CHROMEOS_RESERVED, /* ChromeOS. Reserved for future use. */ + G_PART_ALIAS_CHROMEOS_ROOT, /* A ChromeOS root part. entry. */ + G_PART_ALIAS_DFBSD, /* A DfBSD label32 partition entry */ + G_PART_ALIAS_DFBSD64, /* A DfBSD label64 partition entry */ + G_PART_ALIAS_DFBSD_CCD, /* A DfBSD CCD partition entry */ + G_PART_ALIAS_DFBSD_HAMMER, /* A DfBSD HAMMER FS partition entry */ + G_PART_ALIAS_DFBSD_HAMMER2, /* A DfBSD HAMMER2 FS partition entry */ + G_PART_ALIAS_DFBSD_LEGACY, /* A DfBSD legacy partition entry */ + G_PART_ALIAS_DFBSD_SWAP, /* A DfBSD swap partition entry */ + G_PART_ALIAS_DFBSD_UFS, /* A DfBSD UFS partition entry */ + G_PART_ALIAS_DFBSD_VINUM, /* A DfBSD Vinum partition entry */ + G_PART_ALIAS_EBR, /* A EBR partition entry. */ G_PART_ALIAS_EFI, /* A EFI system partition entry. */ G_PART_ALIAS_FREEBSD, /* A BSD labeled partition entry. */ G_PART_ALIAS_FREEBSD_BOOT, /* A FreeBSD boot partition entry. */ @@ -51,41 +67,32 @@ enum g_part_alias { G_PART_ALIAS_FREEBSD_UFS, /* A UFS/UFS2 file system entry. */ G_PART_ALIAS_FREEBSD_VINUM, /* A Vinum partition entry. */ G_PART_ALIAS_FREEBSD_ZFS, /* A ZFS file system entry. */ - G_PART_ALIAS_MBR, /* A MBR (extended) partition entry. */ G_PART_ALIAS_LINUX_DATA, /* A Linux data partition entry. */ G_PART_ALIAS_LINUX_LVM, /* A Linux LVM partition entry. */ G_PART_ALIAS_LINUX_RAID, /* A Linux RAID partition entry. */ G_PART_ALIAS_LINUX_SWAP, /* A Linux swap partition entry. */ + G_PART_ALIAS_MBR, /* A MBR (extended) partition entry. */ G_PART_ALIAS_MS_BASIC_DATA, /* A Microsoft Data part. entry. */ + G_PART_ALIAS_MS_FAT16, /* A Microsoft FAT16 partition entry. */ + G_PART_ALIAS_MS_FAT32, /* A Microsoft FAT32 partition entry. */ G_PART_ALIAS_MS_LDM_DATA, /* A Microsoft LDM Data part. entry. */ G_PART_ALIAS_MS_LDM_METADATA, /* A Microsoft LDM Metadata entry. */ - G_PART_ALIAS_MS_RESERVED, /* A Microsoft Reserved part. entry. */ G_PART_ALIAS_MS_NTFS, /* A Microsoft NTFS partition entry */ + G_PART_ALIAS_MS_RECOVERY, /* A Microsoft recovery part. entry. */ + G_PART_ALIAS_MS_RESERVED, /* A Microsoft Reserved part. entry. */ + G_PART_ALIAS_MS_SPACES, /* A Microsoft Spaces part. entry. */ G_PART_ALIAS_NETBSD_CCD, /* A NetBSD CCD partition entry. */ G_PART_ALIAS_NETBSD_CGD, /* A NetBSD CGD partition entry. */ G_PART_ALIAS_NETBSD_FFS, /* A NetBSD FFS partition entry. */ + G_PART_ALIAS_NETBSD_LFS, /* A NetBSD LFS partition entry. */ G_PART_ALIAS_NETBSD_RAID, /* A NetBSD RAID partition entry. */ G_PART_ALIAS_NETBSD_SWAP, /* A NetBSD swap partition entry. */ - G_PART_ALIAS_NETBSD_LFS, /* A NetBSD LFS partition entry. */ - G_PART_ALIAS_EBR, /* A EBR partition entry. */ - G_PART_ALIAS_MS_FAT16, /* A Microsoft FAT16 partition entry. */ - G_PART_ALIAS_MS_FAT32, /* A Microsoft FAT32 partition entry. */ - G_PART_ALIAS_BIOS_BOOT, /* A GRUB 2 boot partition entry. */ + G_PART_ALIAS_OPENBSD_DATA, /* An OpenBSD data partition entry. */ + G_PART_ALIAS_PREP_BOOT, /* A PREP/CHRP boot partition entry. */ G_PART_ALIAS_VMFS, /* A VMware VMFS partition entry */ G_PART_ALIAS_VMKDIAG, /* A VMware vmkDiagnostic partition entry */ G_PART_ALIAS_VMRESERVED, /* A VMware reserved partition entry */ G_PART_ALIAS_VMVSANHDR, /* A VMware vSAN header partition entry */ - G_PART_ALIAS_DFBSD, /* A DfBSD label32 partition entry */ - G_PART_ALIAS_DFBSD64, /* A DfBSD label64 partition entry */ - G_PART_ALIAS_DFBSD_SWAP, /* A DfBSD swap partition entry */ - G_PART_ALIAS_DFBSD_UFS, /* A DfBSD UFS partition entry */ - G_PART_ALIAS_DFBSD_VINUM, /* A DfBSD Vinum partition entry */ - G_PART_ALIAS_DFBSD_CCD, /* A DfBSD CCD partition entry */ - G_PART_ALIAS_DFBSD_LEGACY, /* A DfBSD legacy partition entry */ - G_PART_ALIAS_DFBSD_HAMMER, /* A DfBSD HAMMER FS partition entry */ - G_PART_ALIAS_DFBSD_HAMMER2, /* A DfBSD HAMMER2 FS partition entry */ - G_PART_ALIAS_PREP_BOOT, /* A PREP/CHRP boot partition entry. */ - G_PART_ALIAS_APPLE_CORE_STORAGE,/* An Apple Core Storage partition. */ /* Keep the following last */ G_PART_ALIAS_COUNT }; diff --git a/sys/geom/part/g_part_gpt.c b/sys/geom/part/g_part_gpt.c index fa9973c..465bb5e 100644 --- a/sys/geom/part/g_part_gpt.c +++ b/sys/geom/part/g_part_gpt.c @@ -155,6 +155,19 @@ static struct uuid gpt_uuid_apple_raid_offline = GPT_ENT_TYPE_APPLE_RAID_OFFLINE static struct uuid gpt_uuid_apple_tv_recovery = GPT_ENT_TYPE_APPLE_TV_RECOVERY; static struct uuid gpt_uuid_apple_ufs = GPT_ENT_TYPE_APPLE_UFS; static struct uuid gpt_uuid_bios_boot = GPT_ENT_TYPE_BIOS_BOOT; +static struct uuid gpt_uuid_chromeos_firmware = GPT_ENT_TYPE_CHROMEOS_FIRMWARE; +static struct uuid gpt_uuid_chromeos_kernel = GPT_ENT_TYPE_CHROMEOS_KERNEL; +static struct uuid gpt_uuid_chromeos_reserved = GPT_ENT_TYPE_CHROMEOS_RESERVED; +static struct uuid gpt_uuid_chromeos_root = GPT_ENT_TYPE_CHROMEOS_ROOT; +static struct uuid gpt_uuid_dfbsd_ccd = GPT_ENT_TYPE_DRAGONFLY_CCD; +static struct uuid gpt_uuid_dfbsd_hammer = GPT_ENT_TYPE_DRAGONFLY_HAMMER; +static struct uuid gpt_uuid_dfbsd_hammer2 = GPT_ENT_TYPE_DRAGONFLY_HAMMER2; +static struct uuid gpt_uuid_dfbsd_label32 = GPT_ENT_TYPE_DRAGONFLY_LABEL32; +static struct uuid gpt_uuid_dfbsd_label64 = GPT_ENT_TYPE_DRAGONFLY_LABEL64; +static struct uuid gpt_uuid_dfbsd_legacy = GPT_ENT_TYPE_DRAGONFLY_LEGACY; +static struct uuid gpt_uuid_dfbsd_swap = GPT_ENT_TYPE_DRAGONFLY_SWAP; +static struct uuid gpt_uuid_dfbsd_ufs1 = GPT_ENT_TYPE_DRAGONFLY_UFS1; +static struct uuid gpt_uuid_dfbsd_vinum = GPT_ENT_TYPE_DRAGONFLY_VINUM; static struct uuid gpt_uuid_efi = GPT_ENT_TYPE_EFI; static struct uuid gpt_uuid_freebsd = GPT_ENT_TYPE_FREEBSD; static struct uuid gpt_uuid_freebsd_boot = GPT_ENT_TYPE_FREEBSD_BOOT; @@ -167,32 +180,26 @@ static struct uuid gpt_uuid_linux_data = GPT_ENT_TYPE_LINUX_DATA; static struct uuid gpt_uuid_linux_lvm = GPT_ENT_TYPE_LINUX_LVM; static struct uuid gpt_uuid_linux_raid = GPT_ENT_TYPE_LINUX_RAID; static struct uuid gpt_uuid_linux_swap = GPT_ENT_TYPE_LINUX_SWAP; -static struct uuid gpt_uuid_vmfs = GPT_ENT_TYPE_VMFS; -static struct uuid gpt_uuid_vmkdiag = GPT_ENT_TYPE_VMKDIAG; -static struct uuid gpt_uuid_vmreserved = GPT_ENT_TYPE_VMRESERVED; -static struct uuid gpt_uuid_vmvsanhdr = GPT_ENT_TYPE_VMVSANHDR; +static struct uuid gpt_uuid_mbr = GPT_ENT_TYPE_MBR; static struct uuid gpt_uuid_ms_basic_data = GPT_ENT_TYPE_MS_BASIC_DATA; -static struct uuid gpt_uuid_ms_reserved = GPT_ENT_TYPE_MS_RESERVED; static struct uuid gpt_uuid_ms_ldm_data = GPT_ENT_TYPE_MS_LDM_DATA; static struct uuid gpt_uuid_ms_ldm_metadata = GPT_ENT_TYPE_MS_LDM_METADATA; +static struct uuid gpt_uuid_ms_recovery = GPT_ENT_TYPE_MS_RECOVERY; +static struct uuid gpt_uuid_ms_reserved = GPT_ENT_TYPE_MS_RESERVED; +static struct uuid gpt_uuid_ms_spaces = GPT_ENT_TYPE_MS_SPACES; static struct uuid gpt_uuid_netbsd_ccd = GPT_ENT_TYPE_NETBSD_CCD; static struct uuid gpt_uuid_netbsd_cgd = GPT_ENT_TYPE_NETBSD_CGD; static struct uuid gpt_uuid_netbsd_ffs = GPT_ENT_TYPE_NETBSD_FFS; static struct uuid gpt_uuid_netbsd_lfs = GPT_ENT_TYPE_NETBSD_LFS; static struct uuid gpt_uuid_netbsd_raid = GPT_ENT_TYPE_NETBSD_RAID; static struct uuid gpt_uuid_netbsd_swap = GPT_ENT_TYPE_NETBSD_SWAP; -static struct uuid gpt_uuid_mbr = GPT_ENT_TYPE_MBR; -static struct uuid gpt_uuid_unused = GPT_ENT_TYPE_UNUSED; -static struct uuid gpt_uuid_dfbsd_swap = GPT_ENT_TYPE_DRAGONFLY_SWAP; -static struct uuid gpt_uuid_dfbsd_ufs1 = GPT_ENT_TYPE_DRAGONFLY_UFS1; -static struct uuid gpt_uuid_dfbsd_vinum = GPT_ENT_TYPE_DRAGONFLY_VINUM; -static struct uuid gpt_uuid_dfbsd_ccd = GPT_ENT_TYPE_DRAGONFLY_CCD; -static struct uuid gpt_uuid_dfbsd_legacy = GPT_ENT_TYPE_DRAGONFLY_LEGACY; -static struct uuid gpt_uuid_dfbsd_hammer = GPT_ENT_TYPE_DRAGONFLY_HAMMER; -static struct uuid gpt_uuid_dfbsd_hammer2 = GPT_ENT_TYPE_DRAGONFLY_HAMMER2; -static struct uuid gpt_uuid_dfbsd_label32 = GPT_ENT_TYPE_DRAGONFLY_LABEL32; -static struct uuid gpt_uuid_dfbsd_label64 = GPT_ENT_TYPE_DRAGONFLY_LABEL64; +static struct uuid gpt_uuid_openbsd_data = GPT_ENT_TYPE_OPENBSD_DATA; static struct uuid gpt_uuid_prep_boot = GPT_ENT_TYPE_PREP_BOOT; +static struct uuid gpt_uuid_unused = GPT_ENT_TYPE_UNUSED; +static struct uuid gpt_uuid_vmfs = GPT_ENT_TYPE_VMFS; +static struct uuid gpt_uuid_vmkdiag = GPT_ENT_TYPE_VMKDIAG; +static struct uuid gpt_uuid_vmreserved = GPT_ENT_TYPE_VMRESERVED; +static struct uuid gpt_uuid_vmvsanhdr = GPT_ENT_TYPE_VMVSANHDR; static struct g_part_uuid_alias { struct uuid *uuid; @@ -208,6 +215,19 @@ static struct g_part_uuid_alias { { &gpt_uuid_apple_tv_recovery, G_PART_ALIAS_APPLE_TV_RECOVERY, 0 }, { &gpt_uuid_apple_ufs, G_PART_ALIAS_APPLE_UFS, 0 }, { &gpt_uuid_bios_boot, G_PART_ALIAS_BIOS_BOOT, 0 }, + { &gpt_uuid_chromeos_firmware, G_PART_ALIAS_CHROMEOS_FIRMWARE, 0 }, + { &gpt_uuid_chromeos_kernel, G_PART_ALIAS_CHROMEOS_KERNEL, 0 }, + { &gpt_uuid_chromeos_reserved, G_PART_ALIAS_CHROMEOS_RESERVED, 0 }, + { &gpt_uuid_chromeos_root, G_PART_ALIAS_CHROMEOS_ROOT, 0 }, + { &gpt_uuid_dfbsd_ccd, G_PART_ALIAS_DFBSD_CCD, 0 }, + { &gpt_uuid_dfbsd_hammer, G_PART_ALIAS_DFBSD_HAMMER, 0 }, + { &gpt_uuid_dfbsd_hammer2, G_PART_ALIAS_DFBSD_HAMMER2, 0 }, + { &gpt_uuid_dfbsd_label32, G_PART_ALIAS_DFBSD, 0xa5 }, + { &gpt_uuid_dfbsd_label64, G_PART_ALIAS_DFBSD64, 0xa5 }, + { &gpt_uuid_dfbsd_legacy, G_PART_ALIAS_DFBSD_LEGACY, 0 }, + { &gpt_uuid_dfbsd_swap, G_PART_ALIAS_DFBSD_SWAP, 0 }, + { &gpt_uuid_dfbsd_ufs1, G_PART_ALIAS_DFBSD_UFS, 0 }, + { &gpt_uuid_dfbsd_vinum, G_PART_ALIAS_DFBSD_VINUM, 0 }, { &gpt_uuid_efi, G_PART_ALIAS_EFI, 0xee }, { &gpt_uuid_freebsd, G_PART_ALIAS_FREEBSD, 0xa5 }, { &gpt_uuid_freebsd_boot, G_PART_ALIAS_FREEBSD_BOOT, 0 }, @@ -220,31 +240,25 @@ static struct g_part_uuid_alias { { &gpt_uuid_linux_lvm, G_PART_ALIAS_LINUX_LVM, 0 }, { &gpt_uuid_linux_raid, G_PART_ALIAS_LINUX_RAID, 0 }, { &gpt_uuid_linux_swap, G_PART_ALIAS_LINUX_SWAP, 0 }, - { &gpt_uuid_vmfs, G_PART_ALIAS_VMFS, 0 }, - { &gpt_uuid_vmkdiag, G_PART_ALIAS_VMKDIAG, 0 }, - { &gpt_uuid_vmreserved, G_PART_ALIAS_VMRESERVED, 0 }, - { &gpt_uuid_vmvsanhdr, G_PART_ALIAS_VMVSANHDR, 0 }, { &gpt_uuid_mbr, G_PART_ALIAS_MBR, 0 }, { &gpt_uuid_ms_basic_data, G_PART_ALIAS_MS_BASIC_DATA, 0x0b }, { &gpt_uuid_ms_ldm_data, G_PART_ALIAS_MS_LDM_DATA, 0 }, { &gpt_uuid_ms_ldm_metadata, G_PART_ALIAS_MS_LDM_METADATA, 0 }, + { &gpt_uuid_ms_recovery, G_PART_ALIAS_MS_RECOVERY, 0 }, { &gpt_uuid_ms_reserved, G_PART_ALIAS_MS_RESERVED, 0 }, + { &gpt_uuid_ms_spaces, G_PART_ALIAS_MS_SPACES, 0 }, { &gpt_uuid_netbsd_ccd, G_PART_ALIAS_NETBSD_CCD, 0 }, { &gpt_uuid_netbsd_cgd, G_PART_ALIAS_NETBSD_CGD, 0 }, { &gpt_uuid_netbsd_ffs, G_PART_ALIAS_NETBSD_FFS, 0 }, { &gpt_uuid_netbsd_lfs, G_PART_ALIAS_NETBSD_LFS, 0 }, { &gpt_uuid_netbsd_raid, G_PART_ALIAS_NETBSD_RAID, 0 }, { &gpt_uuid_netbsd_swap, G_PART_ALIAS_NETBSD_SWAP, 0 }, - { &gpt_uuid_dfbsd_swap, G_PART_ALIAS_DFBSD_SWAP, 0 }, - { &gpt_uuid_dfbsd_ufs1, G_PART_ALIAS_DFBSD_UFS, 0 }, - { &gpt_uuid_dfbsd_vinum, G_PART_ALIAS_DFBSD_VINUM, 0 }, - { &gpt_uuid_dfbsd_ccd, G_PART_ALIAS_DFBSD_CCD, 0 }, - { &gpt_uuid_dfbsd_legacy, G_PART_ALIAS_DFBSD_LEGACY, 0 }, - { &gpt_uuid_dfbsd_hammer, G_PART_ALIAS_DFBSD_HAMMER, 0 }, - { &gpt_uuid_dfbsd_hammer2, G_PART_ALIAS_DFBSD_HAMMER2, 0 }, - { &gpt_uuid_dfbsd_label32, G_PART_ALIAS_DFBSD, 0xa5 }, - { &gpt_uuid_dfbsd_label64, G_PART_ALIAS_DFBSD64, 0xa5 }, + { &gpt_uuid_openbsd_data, G_PART_ALIAS_OPENBSD_DATA, 0 }, { &gpt_uuid_prep_boot, G_PART_ALIAS_PREP_BOOT, 0x41 }, + { &gpt_uuid_vmfs, G_PART_ALIAS_VMFS, 0 }, + { &gpt_uuid_vmkdiag, G_PART_ALIAS_VMKDIAG, 0 }, + { &gpt_uuid_vmreserved, G_PART_ALIAS_VMRESERVED, 0 }, + { &gpt_uuid_vmvsanhdr, G_PART_ALIAS_VMVSANHDR, 0 }, { NULL, 0, 0 } }; diff --git a/sys/i386/i386/initcpu.c b/sys/i386/i386/initcpu.c index a4a49ea..214c6f6 100644 --- a/sys/i386/i386/initcpu.c +++ b/sys/i386/i386/initcpu.c @@ -81,39 +81,7 @@ SYSCTL_INT(_hw, OID_AUTO, instruction_sse, CTLFLAG_RD, */ static int hw_clflush_disable = -1; -int cpu; /* Are we 386, 386sx, 486, etc? */ -u_int cpu_feature; /* Feature flags */ -u_int cpu_feature2; /* Feature flags */ -u_int amd_feature; /* AMD feature flags */ -u_int amd_feature2; /* AMD feature flags */ -u_int amd_pminfo; /* AMD advanced power management info */ -u_int via_feature_rng; /* VIA RNG features */ -u_int via_feature_xcrypt; /* VIA ACE features */ -u_int cpu_high; /* Highest arg to CPUID */ -u_int cpu_exthigh; /* Highest arg to extended CPUID */ -u_int cpu_id; /* Stepping ID */ -u_int cpu_procinfo; /* HyperThreading Info / Brand Index / CLFUSH */ -u_int cpu_procinfo2; /* Multicore info */ -char cpu_vendor[20]; /* CPU Origin code */ -u_int cpu_vendor_id; /* CPU vendor ID */ -#ifdef CPU_ENABLE_SSE -u_int cpu_fxsr; /* SSE enabled */ -u_int cpu_mxcsr_mask; /* Valid bits in mxcsr */ -#endif -u_int cpu_clflush_line_size = 32; -u_int cpu_stdext_feature; -u_int cpu_stdext_feature2; -u_int cpu_max_ext_state_size; -u_int cpu_mon_mwait_flags; /* MONITOR/MWAIT flags (CPUID.05H.ECX) */ -u_int cpu_mon_min_size; /* MONITOR minimum range size, bytes */ -u_int cpu_mon_max_size; /* MONITOR minimum range size, bytes */ u_int cyrix_did; /* Device ID of Cyrix CPU */ -u_int cpu_maxphyaddr; /* Max phys addr width in bits */ - -SYSCTL_UINT(_hw, OID_AUTO, via_feature_rng, CTLFLAG_RD, - &via_feature_rng, 0, "VIA RNG feature available in CPU"); -SYSCTL_UINT(_hw, OID_AUTO, via_feature_xcrypt, CTLFLAG_RD, - &via_feature_xcrypt, 0, "VIA xcrypt feature available in CPU"); #ifdef I486_CPU /* diff --git a/sys/i386/include/cputypes.h b/sys/i386/include/cputypes.h index 718741a..fb0fa47 100644 --- a/sys/i386/include/cputypes.h +++ b/sys/i386/include/cputypes.h @@ -30,6 +30,8 @@ #ifndef _MACHINE_CPUTYPES_H_ #define _MACHINE_CPUTYPES_H_ +#include <x86/cputypes.h> + /* * Classes of processor. */ @@ -61,25 +63,4 @@ #define CPU_P4 16 /* Intel Pentium 4 */ #define CPU_GEODE1100 17 /* NS Geode SC1100 */ -/* - * Vendors of processor. - */ -#define CPU_VENDOR_NSC 0x100b /* NSC */ -#define CPU_VENDOR_IBM 0x1014 /* IBM */ -#define CPU_VENDOR_AMD 0x1022 /* AMD */ -#define CPU_VENDOR_SIS 0x1039 /* SiS */ -#define CPU_VENDOR_UMC 0x1060 /* UMC */ -#define CPU_VENDOR_NEXGEN 0x1074 /* Nexgen */ -#define CPU_VENDOR_CYRIX 0x1078 /* Cyrix */ -#define CPU_VENDOR_IDT 0x111d /* Centaur/IDT/VIA */ -#define CPU_VENDOR_TRANSMETA 0x1279 /* Transmeta */ -#define CPU_VENDOR_INTEL 0x8086 /* Intel */ -#define CPU_VENDOR_RISE 0xdead2bad /* Rise */ -#define CPU_VENDOR_CENTAUR CPU_VENDOR_IDT - -#ifndef LOCORE -extern int cpu; -extern int cpu_class; -#endif - #endif /* !_MACHINE_CPUTYPES_H_ */ diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c index f9ef458..8a7a11a 100644 --- a/sys/kern/imgact_elf.c +++ b/sys/kern/imgact_elf.c @@ -799,6 +799,11 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *imgp) error = ENOEXEC; goto ret; } + if (interp != NULL) { + uprintf("Multiple PT_INTERP headers\n"); + error = ENOEXEC; + goto ret; + } interp_name_len = phdr[i].p_filesz; if (phdr[i].p_offset > PAGE_SIZE || interp_name_len > PAGE_SIZE - phdr[i].p_offset) { @@ -997,7 +1002,9 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *imgp) if (error == 0) have_interp = TRUE; } - if (!have_interp && newinterp != NULL) { + if (!have_interp && newinterp != NULL && + (brand_info->interp_path == NULL || + strcmp(interp, brand_info->interp_path) == 0)) { error = __elfN(load_file)(imgp->proc, newinterp, &addr, &imgp->entry_addr, sv->sv_pagesize); if (error == 0) @@ -1009,7 +1016,8 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *imgp) } vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY); if (error != 0) { - uprintf("ELF interpreter %s not found\n", interp); + uprintf("ELF interpreter %s not found, error %d\n", + interp, error); goto ret; } } else diff --git a/sys/kern/kern_mbuf.c b/sys/kern/kern_mbuf.c index 0648a5d..a864c94 100644 --- a/sys/kern/kern_mbuf.c +++ b/sys/kern/kern_mbuf.c @@ -32,11 +32,14 @@ __FBSDID("$FreeBSD$"); #include <sys/param.h> #include <sys/malloc.h> +#include <sys/types.h> #include <sys/systm.h> #include <sys/mbuf.h> #include <sys/domain.h> #include <sys/eventhandler.h> #include <sys/kernel.h> +#include <sys/lock.h> +#include <sys/mutex.h> #include <sys/protosw.h> #include <sys/smp.h> #include <sys/sysctl.h> @@ -272,6 +275,12 @@ uma_zone_t zone_jumbo16; uma_zone_t zone_ext_refcnt; /* + * Callout to assist us in freeing mbufs. + */ +static struct callout mb_reclaim_callout; +static struct mtx mb_reclaim_callout_mtx; + +/* * Local prototypes. */ static int mb_ctor_mbuf(void *, int, void *, int); @@ -285,6 +294,7 @@ static void mb_zfini_pack(void *, int); static void mb_reclaim(void *); static void *mbuf_jumbo_alloc(uma_zone_t, vm_size_t, uint8_t *, int); +static void mb_maxaction(uma_zone_t); /* Ensure that MSIZE is a power of 2. */ CTASSERT((((MSIZE - 1) ^ MSIZE) + 1) >> 1 == MSIZE); @@ -310,6 +320,7 @@ mbuf_init(void *dummy) if (nmbufs > 0) nmbufs = uma_zone_set_max(zone_mbuf, nmbufs); uma_zone_set_warning(zone_mbuf, "kern.ipc.nmbufs limit reached"); + uma_zone_set_maxaction(zone_mbuf, mb_maxaction); zone_clust = uma_zcreate(MBUF_CLUSTER_MEM_NAME, MCLBYTES, mb_ctor_clust, mb_dtor_clust, @@ -322,6 +333,7 @@ mbuf_init(void *dummy) if (nmbclusters > 0) nmbclusters = uma_zone_set_max(zone_clust, nmbclusters); uma_zone_set_warning(zone_clust, "kern.ipc.nmbclusters limit reached"); + uma_zone_set_maxaction(zone_clust, mb_maxaction); zone_pack = uma_zsecond_create(MBUF_PACKET_MEM_NAME, mb_ctor_pack, mb_dtor_pack, mb_zinit_pack, mb_zfini_pack, zone_mbuf); @@ -338,6 +350,7 @@ mbuf_init(void *dummy) if (nmbjumbop > 0) nmbjumbop = uma_zone_set_max(zone_jumbop, nmbjumbop); uma_zone_set_warning(zone_jumbop, "kern.ipc.nmbjumbop limit reached"); + uma_zone_set_maxaction(zone_jumbop, mb_maxaction); zone_jumbo9 = uma_zcreate(MBUF_JUMBO9_MEM_NAME, MJUM9BYTES, mb_ctor_clust, mb_dtor_clust, @@ -351,6 +364,7 @@ mbuf_init(void *dummy) if (nmbjumbo9 > 0) nmbjumbo9 = uma_zone_set_max(zone_jumbo9, nmbjumbo9); uma_zone_set_warning(zone_jumbo9, "kern.ipc.nmbjumbo9 limit reached"); + uma_zone_set_maxaction(zone_jumbo9, mb_maxaction); zone_jumbo16 = uma_zcreate(MBUF_JUMBO16_MEM_NAME, MJUM16BYTES, mb_ctor_clust, mb_dtor_clust, @@ -364,6 +378,7 @@ mbuf_init(void *dummy) if (nmbjumbo16 > 0) nmbjumbo16 = uma_zone_set_max(zone_jumbo16, nmbjumbo16); uma_zone_set_warning(zone_jumbo16, "kern.ipc.nmbjumbo16 limit reached"); + uma_zone_set_maxaction(zone_jumbo16, mb_maxaction); zone_ext_refcnt = uma_zcreate(MBUF_EXTREFCNT_MEM_NAME, sizeof(u_int), NULL, NULL, @@ -372,6 +387,11 @@ mbuf_init(void *dummy) /* uma_prealloc() goes here... */ + /* Initialize the mb_reclaim() callout. */ + mtx_init(&mb_reclaim_callout_mtx, "mb_reclaim_callout_mtx", NULL, + MTX_DEF); + callout_init(&mb_reclaim_callout, 1); + /* * Hook event handler for low-memory situation, used to * drain protocols and push data back to the caches (UMA @@ -678,3 +698,61 @@ mb_reclaim(void *junk) if (pr->pr_drain != NULL) (*pr->pr_drain)(); } + +/* + * This is the function called by the mb_reclaim_callout, which is + * used when we hit the maximum for a zone. + * + * (See mb_maxaction() below.) + */ +static void +mb_reclaim_timer(void *junk __unused) +{ + + mtx_lock(&mb_reclaim_callout_mtx); + + /* + * Avoid running this function extra times by skipping this invocation + * if the callout has already been rescheduled. + */ + if (callout_pending(&mb_reclaim_callout) || + !callout_active(&mb_reclaim_callout)) { + mtx_unlock(&mb_reclaim_callout_mtx); + return; + } + mtx_unlock(&mb_reclaim_callout_mtx); + + mb_reclaim(NULL); + + mtx_lock(&mb_reclaim_callout_mtx); + callout_deactivate(&mb_reclaim_callout); + mtx_unlock(&mb_reclaim_callout_mtx); +} + +/* + * This function is called when we hit the maximum for a zone. + * + * At that point, we want to call the protocol drain routine to free up some + * mbufs. However, we will use the callout routines to schedule this to + * occur in another thread. (The thread calling this function holds the + * zone lock.) + */ +static void +mb_maxaction(uma_zone_t zone __unused) +{ + + /* + * If we can't immediately obtain the lock, either the callout + * is currently running, or another thread is scheduling the + * callout. + */ + if (!mtx_trylock(&mb_reclaim_callout_mtx)) + return; + + /* If not already scheduled/running, schedule the callout. */ + if (!callout_active(&mb_reclaim_callout)) { + callout_reset(&mb_reclaim_callout, 1, mb_reclaim_timer, NULL); + } + + mtx_unlock(&mb_reclaim_callout_mtx); +} diff --git a/sys/kern/kern_resource.c b/sys/kern/kern_resource.c index 4edbcba..cbcd680 100644 --- a/sys/kern/kern_resource.c +++ b/sys/kern/kern_resource.c @@ -80,8 +80,6 @@ static int donice(struct thread *td, struct proc *chgp, int n); static struct uidinfo *uilookup(uid_t uid); static void ruxagg_locked(struct rusage_ext *rux, struct thread *td); -static __inline int lim_shared(struct plimit *limp); - /* * Resource controls and accounting. */ @@ -1109,13 +1107,6 @@ lim_hold(struct plimit *limp) return (limp); } -static __inline int -lim_shared(struct plimit *limp) -{ - - return (limp->pl_refcnt > 1); -} - void lim_fork(struct proc *p1, struct proc *p2) { @@ -1146,7 +1137,7 @@ void lim_copy(struct plimit *dst, struct plimit *src) { - KASSERT(!lim_shared(dst), ("lim_copy to shared limit")); + KASSERT(dst->pl_refcnt <= 1, ("lim_copy to shared limit")); bcopy(src->pl_rlimit, dst->pl_rlimit, sizeof(src->pl_rlimit)); } diff --git a/sys/kern/kern_time.c b/sys/kern/kern_time.c index 6ae0fb1..7705d9c 100644 --- a/sys/kern/kern_time.c +++ b/sys/kern/kern_time.c @@ -398,7 +398,8 @@ kern_clock_settime(struct thread *td, clockid_t clock_id, struct timespec *ats) return (error); if (clock_id != CLOCK_REALTIME) return (EINVAL); - if (ats->tv_nsec < 0 || ats->tv_nsec >= 1000000000) + if (ats->tv_nsec < 0 || ats->tv_nsec >= 1000000000 || + ats->tv_sec < 0) return (EINVAL); /* XXX Don't convert nsec->usec and back */ TIMESPEC_TO_TIMEVAL(&atv, ats); @@ -618,7 +619,8 @@ kern_settimeofday(struct thread *td, struct timeval *tv, struct timezone *tzp) return (error); /* Verify all parameters before changing time. */ if (tv) { - if (tv->tv_usec < 0 || tv->tv_usec >= 1000000) + if (tv->tv_usec < 0 || tv->tv_usec >= 1000000 || + tv->tv_sec < 0) return (EINVAL); error = settime(td, tv); } diff --git a/sys/kern/link_elf.c b/sys/kern/link_elf.c index d73c12d..bf2dfd0 100644 --- a/sys/kern/link_elf.c +++ b/sys/kern/link_elf.c @@ -605,7 +605,7 @@ parse_dynamic(elf_file_t ef) static int parse_dpcpu(elf_file_t ef) -{ +{ int count; int error; @@ -636,7 +636,7 @@ parse_dpcpu(elf_file_t ef) #ifdef VIMAGE static int parse_vnet(elf_file_t ef) -{ +{ int count; int error; @@ -916,7 +916,7 @@ link_elf_load_file(linker_class_t cls, const char* filename, */ base_offset = trunc_page(segs[0]->p_offset); base_vaddr = trunc_page(segs[0]->p_vaddr); - base_vlimit = round_page(segs[nsegs - 1]->p_vaddr + + base_vlimit = round_page(segs[nsegs - 1]->p_vaddr + segs[nsegs - 1]->p_memsz); mapsize = base_vlimit - base_vaddr; @@ -1481,7 +1481,7 @@ link_elf_each_function_name(linker_file_t file, elf_file_t ef = (elf_file_t)file; const Elf_Sym *symp; int i, error; - + /* Exhaustive search */ for (i = 0, symp = ef->ddbsymtab; i < ef->ddbsymcnt; i++, symp++) { if (symp->st_value != 0 && @@ -1652,7 +1652,7 @@ link_elf_symtab_get(linker_file_t lf, const Elf_Sym **symtab) return (ef->ddbsymcnt); } - + static long link_elf_strtab_get(linker_file_t lf, caddr_t *strtab) { diff --git a/sys/mips/atheros/if_arge.c b/sys/mips/atheros/if_arge.c index 6a22f3f..792ca5d 100644 --- a/sys/mips/atheros/if_arge.c +++ b/sys/mips/atheros/if_arge.c @@ -78,7 +78,7 @@ __FBSDID("$FreeBSD$"); #include "opt_arge.h" #if defined(ARGE_MDIO) -#include <dev/etherswitch/mdio.h> +#include <dev/mdio/mdio.h> #include <dev/etherswitch/miiproxy.h> #include "mdio_if.h" #endif diff --git a/sys/mips/cavium/cvmx_config.h b/sys/mips/cavium/cvmx_config.h index 1aeabc2..aed3f73 100644 --- a/sys/mips/cavium/cvmx_config.h +++ b/sys/mips/cavium/cvmx_config.h @@ -63,7 +63,7 @@ #define CVMX_ENABLE_PKO_FUNCTIONS /* Define to enable the use of simple executive helper functions. These -** include many harware setup functions. See cvmx-helper.[ch] for +** include many hardware setup functions. See cvmx-helper.[ch] for ** details. */ #define CVMX_ENABLE_HELPER_FUNCTIONS diff --git a/sys/mips/conf/MT7620 b/sys/mips/conf/MT7620 new file mode 100644 index 0000000..4c4c18a --- /dev/null +++ b/sys/mips/conf/MT7620 @@ -0,0 +1,149 @@ +# MT7620 -- Kernel configuration file for FreeBSD/mips for Ralink MT7620 systems +# +# For more information on this file, please read the handbook section on +# Kernel Configuration Files: +# +# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +ident MT7620 + +machine mips mipsel +makeoptions MIPS_LITTLE_ENDIAN=defined +makeoptions KERNLOADADDR=0x80010000 + +# Don't build any modules yet. +#makeoptions MODULES_OVERRIDE="wlan_xauth wlan_wep wlan_tkip wlan_acl wlan_amrr wlan_ccmp wlan_rssadapt if_bridge bridgestp msdosfs md ipfw dummynet libalias geom/geom_label ufs usb/uplcom usb/u3g usb/umodem usb/umass usb/ucom cam zlib" +makeoptions MODULES_OVERRIDE="" +makeoptions MT7620 + +include "../rt305x/std.rt305x" + +hints "MT7620.hints" #Default places to look for devices. + +#makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols + +options MT7620 +options RT305X_UBOOT + +# Debugging for use in -current +options DEADLKRES #Enable the deadlock resolver +options INVARIANTS #Enable calls of extra sanity checking +options INVARIANT_SUPPORT #Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS #Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN #Don't run witness on spinlocks for speed +#options DIAGNOSTIC +#options DEBUG_LOCKS +#options DEBUG_VFS_LOCKS +#options GDB +options DDB +options KDB + +options SCHED_ULE +#options SCHED_4BSD #4BSD scheduler +#options COMPAT_43 +options INET #InterNETworking +options NFSCL #Network Filesystem Client +options NFS_ROOT #NFS usable as /, requires NFSCL +options PSEUDOFS #Pseudo-filesystem framework +#options _KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions + +#options BOOTP +#options BOOTP_NFSROOT +#options BOOTP_NFSV3 +#options BOOTP_WIRED_TO=rt0 +#options BOOTP_COMPAT +#options CD9660 # ISO 9660 Filesystem +#options ROOTDEVNAME=\"cd9660:/dev/map/rootfs.uncompress\" +#options TMPFS # TMP Memory Filesystem + +options FFS #Berkeley Fast Filesystem +#options SOFTUPDATES #Enable FFS soft updates support +#options UFS_ACL #Support for access control lists +#options UFS_DIRHASH #Improve performance on big directories +#options ROOTDEVNAME=\"nfs:10.0.0.1:/mnt/bsd\" + +# Options for making kernel less hangry +#makeoptions INLINE_LIMIT=1024 +#options MAXUSERS=3 +#options MAXFILES=512 +#options NSFBUFS=256 +#options SHMALL=128 +#options MSGBUF_SIZE=65536 + +# Options for making kernel smallest +#options NO_SYSCTL_DESCR # No description string of sysctl +#options NO_FFS_SNAPSHOT # Disable Snapshot supporting +#options SCSI_NO_SENSE_STRINGS +#options SCSI_NO_OP_STRINGS +#options RWLOCK_NOINLINE +#options SX_NOINLINE +#options NO_SWAPPING +options MROUTING # Multicast routing +options IPFIREWALL_DEFAULT_TO_ACCEPT + +options GEOM_UNCOMPRESS +options MD_ROOT +options ROOTDEVNAME=\"ufs:da0s1\" + +device md + +device random +device loop +# RT3050F, RT3052F have only pseudo PHYs, so mii not required +device rt + +device spibus +device mx25l + +device ether +device miibus +device bpf # Berkeley packet filter +device vlan +device lagg +device if_bridge +device uart +nodevice uart_ns8250 +#device tun # Packet tunnel. + +#device wlan + + +#device gpio +#device gpioled + +#device cfi # Detect Flash memmory +#device cfid + +#device nvram2env + +device usb +device ehci +#device ohci +#device dwcotg # DWC like USB OTG Controller driver +#device u3g +#device umodem +#device uplcom +device cdce +device umass +device da +device pass +device scbus +options SCSI_DELAY=1000 # Delay (in ms) before probing SCSI + +#options USB_EHCI_BIG_ENDIAN_DESC # handle big-endian byte order +#options USB_DEBUG +#options USB_REQ_DEBUG + +device pci diff --git a/sys/mips/conf/MT7620.hints b/sys/mips/conf/MT7620.hints new file mode 100644 index 0000000..dd35bbd --- /dev/null +++ b/sys/mips/conf/MT7620.hints @@ -0,0 +1,143 @@ +# $FreeBSD$ +# device.hints +hint.obio.0.at="nexus0" +hint.obio.0.maddr=0x10000000 +hint.obio.0.msize=0x10000000 + +hint.pcib.0.at="nexus0" +hint.pcib.0.maddr=0x10140000 +hint.pcib.0.msize=0x30000 + +hint.mx25l.0.at="spibus0" + +#hint.nvram.0.sig=0xe5e60a74 +#hint.nvram.0.base=0x1f030000 +#hint.nvram.0.maxsize=0x2000 +#hint.nvram.0.flags=3 # 1 = No check, 2 = Format Generic +#hint.nvram.1.sig=0x5a045e94 +#hint.nvram.1.base=0x1f032000 +#hint.nvram.1.maxsize=0x4000 +#hint.nvram.1.flags=3 # 1 = No check, 2 = Format Generic + +# on-board Ralink Frame Engine +hint.rt.0.at="nexus0" +hint.rt.0.maddr=0x10100000 +hint.rt.0.msize=0x10000 +hint.rt.0.irq=3 +# macaddr can be statically set +#hint.rt.0.macaddr="xx:xx:xx:xx:xx:xx" + +# on-board Ralink 2872 802.11n core +hint.rt2860.0.at="nexus0" +hint.rt2860.0.maddr=0x10180000 +hint.rt2860.0.msize=0x40000 +hint.rt2860.0.irq=4 + +# uart0 +#hint.uart.0.at="obio0" +#hint.uart.0.maddr=0x10000C00 +#hint.uart.0.msize=0x100 +#hint.uart.0.irq=12 +#hint.uart.0.flags="0x30" + +# uart1 +#hint.uart.1.at="obio0" +#hint.uart.1.maddr=0x10000500 +#hint.uart.1.msize=0x100 +#hint.uart.1.irq=5 +#hint.uart.1.flags="0x30" + + +# gpio +# GPIO0 - WPS BTN IN II IO +#hint.gpiobutton.0.at="gpiobus0" +#hint.gpiobutton.0.pins="0x01" +#hint.gpiobutton.0.name="wps" +#hint.gpiobutton.0.flags="0x0581" + +# GPIO7 - MODE SW AP IN II IO +#hint.gpiobutton.1.at="gpiobus0" +#hint.gpiobutton.1.pins="0x80" +#hint.gpiobutton.1.name="mode_ap" +#hint.gpiobutton.1.flags="0x0581" + +# GPIO8 - ST LEDRED OUT /* 2pin BiDir RED/BLUE LED */ +# GPIO9 - ST LEDBLUE OUT +#hint.gpioled.0.at="gpiobus0" +#hint.gpioled.0.pins="0x100" +#hint.gpioled.0.name="status_red" +#hint.gpioled.0.flags="0x0002" +#hint.gpioled.1.at="gpiobus0" +#hint.gpioled.1.pins="0x200" +#hint.gpioled.1.name="status_blue" +#hint.gpioled.1.name="status" +#hint.gpioled.1.flags="0x0002" + +# GPIO10 - RST BTN IN II IO +#hint.gpiobutton.2.at="gpiobus0" +#hint.gpiobutton.2.pins="0x400" +##hint.gpiobutton.2.name="reset" +#hint.gpiobutton.2.flags="0x0581" + +# GPIO11 - MODE SW CL IN II IO +#hint.gpiobutton.3.at="gpiobus0" +#hint.gpiobutton.3.pins="0x800" +#hint.gpiobutton.3.name="mode_wlan_client" +#hint.gpiobutton.3.flags="0x0581" + +# GPIO14 - WPS LED OUT II IO +#hint.gpioled.2.at="gpiobus0" +#hint.gpioled.2.pins="0x4000" +#hint.gpioled.2.name="wps" +#hint.gpioled.2.flags="0x0182" + + + +#0x00000000-0x00030000 : "Bootloader" +#0x00030000-0x00040000 : "Factory" +#0x00040000-0x00070000 : "Config" +#0x00070000-0x000b0000 : "Language" +#0x000b0000-0x001a0000 : "Kernel" +#0x001a0000-0x01000000 : "RootFS" + +#hint.map.0.at="cfid0" +#hint.map.0.start=0x00000000 +#hint.map.0.end=0x00030000 +#hint.map.0.name="bootloader" +#hint.map.0.readonly=1 + +#hint.map.1.at="cfid0" +#hint.map.1.start=0x00030000 +#hint.map.1.end=0x00040000 +#hint.map.1.name="factory" + +#hint.map.2.at="cfid0" +#hint.map.2.start=0x00040000 +#hint.map.2.end=0x00800000 +#hint.map.2.name="upgrade" + +#hint.map.3.at="cfid0" +#hint.map.3.start=0x00040000 +#hint.map.3.end=0x00050000 +#hint.map.3.name="config" + +#hint.map.4.at="cfid0" +#hint.map.4.start=0x00000000 +#hint.map.4.end=0x00000000 +#hint.map.4.name="language" + +#hint.map.5.at="cfid0" +#hint.map.5.start=0x00050000 +#hint.map.5.end=0x00150000 +#hint.map.5.name="kernel" + +#hint.map.6.at="cfid0" +#hint.map.6.start=0x00150000 +#hint.map.6.end=0x00800000 +#hint.map.6.name="rootfs" + + +#hint.rt.0.phymask=0x1f +#hint.rt.0.media=100 +#hint.rt.0.fduplex=1 + diff --git a/sys/mips/conf/RT305X b/sys/mips/conf/RT305X index a8b6626..e5e26f5 100644 --- a/sys/mips/conf/RT305X +++ b/sys/mips/conf/RT305X @@ -33,6 +33,9 @@ hints "RT305X.hints" #Default places to look for devices. #makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols +options RT3052F +options RT305X_UBOOT + # Debugging for use in -current #options DEADLKRES #Enable the deadlock resolver #options INVARIANTS #Enable calls of extra sanity checking @@ -115,7 +118,7 @@ device cfid device nvram2env device usb -#device dotg # DWC like USB OTG Controller driver +#device dwcotg # DWC like USB OTG Controller driver #device u3g #device umodem #device uplcom diff --git a/sys/mips/conf/RT5350 b/sys/mips/conf/RT5350 new file mode 100644 index 0000000..b6ec20b --- /dev/null +++ b/sys/mips/conf/RT5350 @@ -0,0 +1,125 @@ +# RT5350 -- Kernel configuration file for FreeBSD/mips for Ralink RT5350 systems +# +# For more information on this file, please read the handbook section on +# Kernel Configuration Files: +# +# http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +ident RT5350 + +machine mips mipsel +makeoptions MIPS_LITTLE_ENDIAN=defined +makeoptions KERNLOADADDR=0x80001000 + +# Don't build any modules yet. +makeoptions MODULES_OVERRIDE="wlan_xauth wlan_wep wlan_tkip wlan_acl wlan_amrr wlan_ccmp wlan_rssadapt if_bridge bridgestp msdosfs md ipfw dummynet libalias geom/geom_label ufs usb/uplcom usb/u3g usb/umodem usb/umass usb/ucom cam zlib" +makeoptions RT5350 + +include "../rt305x/std.rt305x" + +hints "RT5350.hints" #Default places to look for devices. + +#makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols + +options RT5350 +options RT305X_UBOOT + +# Debugging for use in -current +options DEADLKRES #Enable the deadlock resolver +options INVARIANTS #Enable calls of extra sanity checking +options INVARIANT_SUPPORT #Extra sanity checks of internal structures, required by INVARIANTS +options WITNESS #Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN #Don't run witness on spinlocks for speed +#options DIAGNOSTIC +#options DEBUG_LOCKS +#options DEBUG_VFS_LOCKS +#options GDB +options DDB +options KDB + +options SCHED_ULE +options INET #InterNETworking +#options NFSCL #Network Filesystem Client +#options NFS_ROOT #NFS usable as /, requires NFSCL +options PSEUDOFS #Pseudo-filesystem framework +#options _KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions + +#options BOOTP +#options BOOTP_NFSROOT +#options BOOTP_NFSV3 +#options BOOTP_WIRED_TO=rt0 +#options BOOTP_COMPAT + +options TMPFS # TMP Memory Filesystem + +options FFS #Berkeley Fast Filesystem +#options ROOTDEVNAME=\"nfs:193.178.153.200:/bsdmips\" + +#device geom_uncompress +#options GEOM_UNCOMPRESS +#options MD_ROOT +#options ROOTDEVNAME=\"ufs:md0.uncompress\" + +# Options for making kernel less hangry +makeoptions INLINE_LIMIT=1024 +options MAXUSERS=3 +options MAXFILES=512 +options NSFBUFS=256 +options SHMALL=128 +options MSGBUF_SIZE=65536 + +# Options for making kernel smallest +options NO_SYSCTL_DESCR # No description string of sysctl +#options NO_FFS_SNAPSHOT # Disable Snapshot supporting +options SCSI_NO_SENSE_STRINGS +options SCSI_NO_OP_STRINGS +options RWLOCK_NOINLINE +options SX_NOINLINE +options NO_SWAPPING +options MROUTING # Multicast routing +options IPFIREWALL_DEFAULT_TO_ACCEPT + +#device md +device random +device loop +# RT3050F, RT3052F have only pseudo PHYs, so mii not required +device rt + +device ether +device bpf # Berkeley packet filter +device vlan +#device lagg +#device if_bridge +device uart +nodevice uart_ns8250 +device tun # Packet tunnel. + +device wlan + +#device gpio +#device gpioled + +#device nvram2env + +device spibus +device mx25l + +device usb +device ehci +options SCSI_DELAY=1000 # Delay (in ms) before probing SCSI + +#options USB_EHCI_BIG_ENDIAN_DESC # handle big-endian byte order +#options USB_DEBUG +#options USB_REQ_DEBUG diff --git a/sys/mips/conf/RT5350.hints b/sys/mips/conf/RT5350.hints new file mode 100644 index 0000000..61220fe --- /dev/null +++ b/sys/mips/conf/RT5350.hints @@ -0,0 +1,35 @@ +# $FreeBSD$ +# device.hints +hint.obio.0.at="nexus0" +hint.obio.0.maddr=0x10000000 +hint.obio.0.msize=0x10000000 + +hint.mx25l.0.at="spibus0" + +#hint.nvram.0.sig=0xe5e60a74 +#hint.nvram.0.base=0x1f030000 +#hint.nvram.0.maxsize=0x2000 +#hint.nvram.0.flags=3 # 1 = No check, 2 = Format Generic +#hint.nvram.1.sig=0x5a045e94 +#hint.nvram.1.base=0x1f032000 +#hint.nvram.1.maxsize=0x4000 +#hint.nvram.1.flags=3 # 1 = No check, 2 = Format Generic + +# on-board Ralink Frame Engine +hint.rt.0.at="nexus0" +hint.rt.0.maddr=0x10100000 +hint.rt.0.msize=0x10000 +hint.rt.0.irq=3 +# macaddr can be statically set +#hint.rt.0.macaddr="xx:xx:xx:xx:xx:xx" + +# on-board Ralink 2872 802.11n core +hint.rt2860.0.at="nexus0" +hint.rt2860.0.maddr=0x10180000 +hint.rt2860.0.msize=0x40000 +hint.rt2860.0.irq=4 + +hint.rt.0.phymask=0x1f +hint.rt.0.media=100 +hint.rt.0.fduplex=1 + diff --git a/sys/mips/include/cpufunc.h b/sys/mips/include/cpufunc.h index d47f9aa..6ffb0ba 100644 --- a/sys/mips/include/cpufunc.h +++ b/sys/mips/include/cpufunc.h @@ -248,7 +248,7 @@ MIPS_RW32_COP0_SEL(config5, MIPS_COP_0_CONFIG, 5); #if defined(CPU_NLM) || defined(BERI_LARGE_TLB) MIPS_RW32_COP0_SEL(config6, MIPS_COP_0_CONFIG, 6); #endif -#ifdef CPU_NLM +#if defined(CPU_NLM) || defined(CPU_MIPS1004KC) MIPS_RW32_COP0_SEL(config7, MIPS_COP_0_CONFIG, 7); #endif MIPS_RW32_COP0(count, MIPS_COP_0_COUNT); @@ -259,6 +259,7 @@ MIPS_RW32_COP0(cause, MIPS_COP_0_CAUSE); MIPS_RW32_COP0(excpc, MIPS_COP_0_EXC_PC); #endif MIPS_RW32_COP0(status, MIPS_COP_0_STATUS); +MIPS_RW32_COP0_SEL(cmgcrbase, 15, 3); /* XXX: Some of these registers are specific to MIPS32. */ #if !defined(__mips_n64) diff --git a/sys/mips/include/cpuinfo.h b/sys/mips/include/cpuinfo.h index baf3039..deeb93b 100644 --- a/sys/mips/include/cpuinfo.h +++ b/sys/mips/include/cpuinfo.h @@ -54,6 +54,7 @@ struct mips_cpuinfo { u_int8_t cpu_rev; u_int8_t cpu_impl; u_int8_t tlb_type; + u_int32_t tlb_pgmask; u_int16_t tlb_nentries; u_int8_t icache_virtual; boolean_t cache_coherent_dma; diff --git a/sys/mips/include/cpuregs.h b/sys/mips/include/cpuregs.h index a39f6a6..976321a 100644 --- a/sys/mips/include/cpuregs.h +++ b/sys/mips/include/cpuregs.h @@ -154,6 +154,11 @@ #define MIPS_CCA_CACHED 0x03 #endif +#if defined(CPU_MIPS1004KC) +#define MIPS_CCA_UNCACHED 0x02 +#define MIPS_CCA_CACHED 0x05 +#endif + #ifndef MIPS_CCA_UNCACHED #define MIPS_CCA_UNCACHED MIPS_CCA_UC #endif @@ -209,7 +214,7 @@ #define COP0_SYNC .word 0xc0 /* ehb */ #elif defined(CPU_SB1) #define COP0_SYNC ssnop; ssnop; ssnop; ssnop; ssnop; ssnop; ssnop; ssnop; ssnop -#elif defined(CPU_MIPS74KC) +#elif defined(CPU_MIPS74KC) || defined(CPU_MIPS1004KC) #define COP0_SYNC .word 0xc0 /* ehb */ #else /* @@ -557,6 +562,8 @@ #define MIPS_CONFIG2_SS_SHIFT 8 /* Secondary cache sets per way */ #define MIPS_CONFIG2_SS_MASK 0xf +#define MIPS_CONFIG3_CMGCR_MASK (1 << 29) /* Coherence manager present */ + #define MIPS_CONFIG4_MMUSIZEEXT 0x000000FF /* bits 7.. 0 MMU Size Extension */ #define MIPS_CONFIG4_MMUEXTDEF 0x0000C000 /* bits 15.14 MMU Extension Definition */ #define MIPS_CONFIG4_MMUEXTDEF_MMUSIZEEXT 0x00004000 /* This values denotes CONFIG4 bits */ @@ -634,4 +641,8 @@ #define MIPS_OPCODE_SHIFT 26 #define MIPS_OPCODE_C1 0x11 +/* Coherence manager constants */ +#define MIPS_CMGCRB_BASE 11 +#define MIPS_CMGCRF_BASE (~((1 << MIPS_CMGCRB_BASE) - 1)) + #endif /* _MIPS_CPUREGS_H_ */ diff --git a/sys/mips/include/ofw_machdep.h b/sys/mips/include/ofw_machdep.h index 35afce1..f14cfdf 100644 --- a/sys/mips/include/ofw_machdep.h +++ b/sys/mips/include/ofw_machdep.h @@ -32,7 +32,6 @@ #include <sys/types.h> #include <sys/rman.h> #include <sys/bus.h> -#include <dev/ofw/openfirm.h> typedef uint32_t cell_t; struct mem_region { @@ -40,8 +39,6 @@ struct mem_region { vm_size_t mr_size; }; - -int OF_decode_addr(phandle_t, int, bus_space_tag_t *, bus_space_handle_t *); void OF_getetheraddr(device_t dev, u_char *addr); void OF_initial_setup(void *fdt_ptr, void *junk, int (*openfirm)(void *)); diff --git a/sys/mips/include/pte.h b/sys/mips/include/pte.h index 2f2f995..4b628db 100644 --- a/sys/mips/include/pte.h +++ b/sys/mips/include/pte.h @@ -188,4 +188,17 @@ typedef pt_entry_t *pd_entry_t; #endif #endif /* LOCORE */ + +/* PageMask Register (CP0 Register 5, Select 0) Values */ +#define MIPS3_PGMASK_MASKX 0x00001800 +#define MIPS3_PGMASK_4K 0x00000000 +#define MIPS3_PGMASK_16K 0x00006000 +#define MIPS3_PGMASK_64K 0x0001e000 +#define MIPS3_PGMASK_256K 0x0007e000 +#define MIPS3_PGMASK_1M 0x001fe000 +#define MIPS3_PGMASK_4M 0x007fe000 +#define MIPS3_PGMASK_16M 0x01ffe000 +#define MIPS3_PGMASK_64M 0x07ffe000 +#define MIPS3_PGMASK_256M 0x1fffe000 + #endif /* !_MACHINE_PTE_H_ */ diff --git a/sys/mips/mips/cpu.c b/sys/mips/mips/cpu.c index 469ab81..5abd5b6 100644 --- a/sys/mips/mips/cpu.c +++ b/sys/mips/mips/cpu.c @@ -190,6 +190,14 @@ mips_get_identity(struct mips_cpuinfo *cpuinfo) cpuinfo->l1.dc_size = cpuinfo->l1.dc_linesize * cpuinfo->l1.dc_nsets * cpuinfo->l1.dc_nways; + /* + * Probe PageMask register to see what sizes of pages are supported + * by writing all one's and then reading it back. + */ + mips_wr_pagemask(~0); + cpuinfo->tlb_pgmask = mips_rd_pagemask(); + mips_wr_pagemask(MIPS3_PGMASK_4K); + #ifndef CPU_CNMIPS /* L2 cache */ if (!(cfg1 & MIPS_CONFIG_CM)) { @@ -289,8 +297,31 @@ cpu_identify(void) } else if (cpuinfo.tlb_type == MIPS_MMU_FIXED) { printf("Fixed mapping"); } - printf(", %d entries\n", cpuinfo.tlb_nentries); + printf(", %d entries ", cpuinfo.tlb_nentries); + } + + if (cpuinfo.tlb_pgmask) { + printf("("); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_MASKX) + printf("1K "); + printf("4K "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_16K) + printf("16K "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_64K) + printf("64K "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_256K) + printf("256K "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_1M) + printf("1M "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_16M) + printf("16M "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_64M) + printf("64M "); + if (cpuinfo.tlb_pgmask & MIPS3_PGMASK_256M) + printf("256M "); + printf("pg sizes)"); } + printf("\n"); printf(" L1 i-cache: "); if (cpuinfo.l1.ic_linesize == 0) { @@ -318,6 +349,18 @@ cpu_identify(void) cpuinfo.l1.dc_nsets, cpuinfo.l1.dc_linesize); } + printf(" L2 cache: "); + if (cpuinfo.l2.dc_linesize == 0) { + printf("disabled\n"); + } else { + printf("%d ways of %d sets, %d bytes per line, " + "%d KiB total size\n", + cpuinfo.l2.dc_nways, + cpuinfo.l2.dc_nsets, + cpuinfo.l2.dc_linesize, + cpuinfo.l2.dc_size / 1024); + } + cfg0 = mips_rd_config(); /* If config register selection 1 does not exist, exit. */ if (!(cfg0 & MIPS_CONFIG_CM)) @@ -335,6 +378,7 @@ cpu_identify(void) * Config2 contains no useful information other then Config3 * existence flag */ + printf(" Config2=0x%08x\n", cfg2); /* If config register selection 3 does not exist, exit. */ if (!(cfg2 & MIPS_CONFIG_CM)) diff --git a/sys/mips/mips/ofw_machdep.c b/sys/mips/mips/ofw_machdep.c new file mode 100644 index 0000000..82e2959 --- /dev/null +++ b/sys/mips/mips/ofw_machdep.c @@ -0,0 +1,70 @@ +/*- + * Copyright (c) 2015 Ian Lepore <ian@freebsd.org> + * All rights excluded. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/bus.h> + +#include <machine/bus.h> +#include <machine/fdt.h> + +#include <dev/ofw/openfirm.h> +#include <dev/ofw/ofw_subr.h> + +int +OF_decode_addr(phandle_t dev, int regno, bus_space_tag_t *tag, + bus_space_handle_t *handle) +{ + bus_addr_t addr; + bus_size_t size; + pcell_t pci_hi; + int flags, res; + + res = ofw_reg_to_paddr(dev, regno, &addr, &size, &pci_hi); + if (res < 0) + return (res); + + /* + * Nothing special to do for PCI busses right now. + * This may need to be handled per-platform when it does come up. + */ +#ifdef notyet + if (pci_hi == OFW_PADDR_NOT_PCI) { + *tag = fdtbus_bs_tag; + flags = 0; + } else { + *tag = fdtbus_bs_tag; + flags = (pci_hi & OFW_PCI_PHYS_HI_PREFETCHABLE) ? + BUS_SPACE_MAP_PREFETCHABLE: 0; + } +#else + *tag = fdtbus_bs_tag; + flags = 0; +#endif + return (bus_space_map(*tag, addr, size, flags, handle)); +} diff --git a/sys/mips/rt305x/files.rt305x b/sys/mips/rt305x/files.rt305x index 5f4d0ea..94152d3 100644 --- a/sys/mips/rt305x/files.rt305x +++ b/sys/mips/rt305x/files.rt305x @@ -9,6 +9,11 @@ mips/rt305x/rt305x_gpio.c optional gpio mips/rt305x/uart_bus_rt305x.c optional uart mips/rt305x/uart_cpu_rt305x.c optional uart mips/rt305x/uart_dev_rt305x.c optional uart +mips/rt305x/rt305x_dotg.c optional dwcotg +mips/rt305x/rt305x_ehci.c optional ehci +mips/rt305x/rt305x_ohci.c optional ohci +mips/rt305x/rt305x_spi.c optional spibus +mips/rt305x/rt305x_pci.c optional pci mips/mips/intr_machdep.c standard mips/mips/tick.c standard dev/rt/if_rt.c optional rt diff --git a/sys/mips/rt305x/obio.c b/sys/mips/rt305x/obio.c index fd9132a..ff7a32c 100644 --- a/sys/mips/rt305x/obio.c +++ b/sys/mips/rt305x/obio.c @@ -221,10 +221,10 @@ obio_attach(device_t dev) obio_add_res_child(dev, "pcm", 0, PCM_BASE, (PCM_END - PCM_BASE + 1), IC_PCM); +#endif obio_add_res_child(dev, "uart", 0, UART_BASE, (UART_END - UART_BASE + 1), IC_UART); -#endif obio_add_res_child(dev, "gpio", 0, PIO_BASE, (PIO_END - PIO_BASE + 1), IC_PIO); @@ -241,19 +241,28 @@ obio_attach(device_t dev) obio_add_res_child(dev, "i2s", 0, I2S_BASE, (I2S_END - I2S_BASE + 1), IC_I2S); +#endif obio_add_res_child(dev, "spi", 0, SPI_BASE, (SPI_END - SPI_BASE + 1), -1); -#endif obio_add_res_child(dev, "uart", 1, UARTLITE_BASE, (UARTLITE_END - UARTLITE_BASE + 1), IC_UARTLITE); +#if !defined(RT5350) && !defined(MT7620) obio_add_res_child(dev, "cfi", 0, FLASH_BASE, (FLASH_END - FLASH_BASE + 1), -1); - obio_add_res_child(dev, "dotg", 0, + obio_add_res_child(dev, "dwcotg", 0, USB_OTG_BASE, (USB_OTG_END - USB_OTG_BASE + 1), IC_OTG); +#else + obio_add_res_child(dev, "ehci", 0, + USB_OTG_BASE, (USB_OTG_END - USB_OTG_BASE + 1), + IC_OTG); + obio_add_res_child(dev, "ohci", 0, + USB_OHCI_BASE, (USB_OHCI_END - USB_OHCI_BASE + 1), + IC_OTG); +#endif obio_add_res_child(dev, "switch", 0, ETHSW_BASE, (ETHSW_END - ETHSW_BASE + 1), IC_ETHSW); diff --git a/sys/mips/rt305x/rt305x_dotg.c b/sys/mips/rt305x/rt305x_dotg.c index 1159158..607b215 100644 --- a/sys/mips/rt305x/rt305x_dotg.c +++ b/sys/mips/rt305x/rt305x_dotg.c @@ -2,6 +2,7 @@ __FBSDID("$FreeBSD$"); /*- + * Copyright (c) 2015 Stanislav Galabov. All rights reserved. * Copyright (c) 2010,2011 Aleksandr Rybalko. All rights reserved. * Copyright (c) 2007-2008 Hans Petter Selasky. All rights reserved. * @@ -58,7 +59,7 @@ __FBSDID("$FreeBSD$"); #include <dev/usb/usb_controller.h> #include <dev/usb/usb_bus.h> -#include <dev/usb/controller/dotg.h> +#include <dev/usb/controller/dwc_otg.h> #include <mips/rt305x/rt305xreg.h> #include <mips/rt305x/rt305x_sysctlvar.h> @@ -68,10 +69,6 @@ static device_probe_t dotg_obio_probe; static device_attach_t dotg_obio_attach; static device_detach_t dotg_obio_detach; -struct dotg_obio_softc { - struct dotg_softc sc_dci; /* must be first */ -}; - static int dotg_obio_probe(device_t dev) { @@ -82,61 +79,63 @@ dotg_obio_probe(device_t dev) static int dotg_obio_attach(device_t dev) { - struct dotg_obio_softc *sc = device_get_softc(dev); - int err; + struct dwc_otg_softc *sc = device_get_softc(dev); + uint32_t tmp; + int err, rid; /* setup controller interface softc */ /* initialise some bus fields */ - sc->sc_dci.sc_dev = dev; - sc->sc_dci.sc_bus.parent = dev; - sc->sc_dci.sc_bus.devices = sc->sc_dci.sc_devices; - sc->sc_dci.sc_bus.devices_max = DOTG_MAX_DEVICES; - sc->sc_dci.sc_bus.dma_bits = 32; + sc->sc_mode = DWC_MODE_HOST; + sc->sc_bus.parent = dev; + sc->sc_bus.devices = sc->sc_devices; + sc->sc_bus.devices_max = DWC_OTG_MAX_DEVICES; + sc->sc_bus.dma_bits = 32; /* get all DMA memory */ - if (usb_bus_mem_alloc_all(&sc->sc_dci.sc_bus, + if (usb_bus_mem_alloc_all(&sc->sc_bus, USB_GET_DMA_TAG(dev), NULL)) { printf("No mem\n"); return (ENOMEM); } - sc->sc_dci.sc_mem_rid = 0; - sc->sc_dci.sc_mem_res = - bus_alloc_resource_any(dev, SYS_RES_MEMORY, &sc->sc_dci.sc_irq_rid, - RF_ACTIVE); - if (!(sc->sc_dci.sc_mem_res)) { + rid = 0; + sc->sc_io_res = + bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE); + if (!(sc->sc_io_res)) { printf("Can`t alloc MEM\n"); goto error; } - sc->sc_dci.sc_bst = rman_get_bustag(sc->sc_dci.sc_mem_res); - sc->sc_dci.sc_bsh = rman_get_bushandle(sc->sc_dci.sc_mem_res); - - sc->sc_dci.sc_irq_rid = 0; - sc->sc_dci.sc_irq_res = bus_alloc_resource_any(dev, SYS_RES_IRQ, - &sc->sc_dci.sc_irq_rid, RF_SHAREABLE| RF_ACTIVE); - if (!(sc->sc_dci.sc_irq_res)) { + sc->sc_io_tag = rman_get_bustag(sc->sc_io_res); + sc->sc_io_hdl = rman_get_bushandle(sc->sc_io_res); + sc->sc_io_size = rman_get_size(sc->sc_io_res); + + rid = 0; + sc->sc_irq_res = bus_alloc_resource_any(dev, SYS_RES_IRQ, + &rid, RF_ACTIVE); + if (!(sc->sc_irq_res)) { printf("Can`t alloc IRQ\n"); goto error; } - sc->sc_dci.sc_bus.bdev = device_add_child(dev, "usbus", -1); - if (!(sc->sc_dci.sc_bus.bdev)) { + sc->sc_bus.bdev = device_add_child(dev, "usbus", -1); + if (!(sc->sc_bus.bdev)) { printf("Can`t add usbus\n"); goto error; } - device_set_ivars(sc->sc_dci.sc_bus.bdev, &sc->sc_dci.sc_bus); + device_set_ivars(sc->sc_bus.bdev, &sc->sc_bus); #if (__FreeBSD_version >= 700031) - err = bus_setup_intr(dev, sc->sc_dci.sc_irq_res, - INTR_TYPE_BIO | INTR_MPSAFE, NULL, (driver_intr_t *)dotg_interrupt, - sc, &sc->sc_dci.sc_intr_hdl); + err = bus_setup_intr(dev, sc->sc_irq_res, + INTR_TYPE_TTY | INTR_MPSAFE, dwc_otg_filter_interrupt, + dwc_otg_interrupt, sc, &sc->sc_intr_hdl); #else - err = bus_setup_intr(dev, sc->sc_dci.sc_irq_res, - INTR_TYPE_BIO | INTR_MPSAFE, (driver_intr_t *)dotg_interrupt, - sc, &sc->sc_dci.sc_intr_hdl); + #error error + err = bus_setup_intr(dev, sc->sc_irq_res, + INTR_TYPE_BIO | INTR_MPSAFE,(driver_intr_t*)dwc_otg_interrupt, + sc, &sc->sc_intr_hdl); #endif if (err) { - sc->sc_dci.sc_intr_hdl = NULL; + sc->sc_intr_hdl = NULL; printf("Can`t set IRQ handle\n"); goto error; } @@ -144,14 +143,21 @@ dotg_obio_attach(device_t dev) /* Run clock for OTG core */ rt305x_sysctl_set(SYSCTL_CLKCFG1, rt305x_sysctl_get(SYSCTL_CLKCFG1) | SYSCTL_CLKCFG1_OTG_CLK_EN); - rt305x_sysctl_set(SYSCTL_RSTCTRL, SYSCTL_RSTCTRL_OTG); + tmp = rt305x_sysctl_get(SYSCTL_RSTCTRL); + rt305x_sysctl_set(SYSCTL_RSTCTRL, tmp | SYSCTL_RSTCTRL_OTG); DELAY(100); - - err = dotg_init(&sc->sc_dci); + /* + * Docs say that RSTCTRL bits for RT305x are W1C, so there should + * be no need for the below, but who really knows? + */ +// rt305x_sysctl_set(SYSCTL_RSTCTRL, tmp & ~SYSCTL_RSTCTRL_OTG); +// DELAY(100); + + err = dwc_otg_init(sc); if (err) printf("dotg_init fail\n"); if (!err) { - err = device_probe_and_attach(sc->sc_dci.sc_bus.bdev); - if (err) printf("device_probe_and_attach fail\n"); + err = device_probe_and_attach(sc->sc_bus.bdev); + if (err) printf("device_probe_and_attach fail %d\n", err); } if (err) { goto error; @@ -166,44 +172,44 @@ error: static int dotg_obio_detach(device_t dev) { - struct dotg_obio_softc *sc = device_get_softc(dev); + struct dwc_otg_softc *sc = device_get_softc(dev); device_t bdev; int err; - if (sc->sc_dci.sc_bus.bdev) { - bdev = sc->sc_dci.sc_bus.bdev; + if (sc->sc_bus.bdev) { + bdev = sc->sc_bus.bdev; device_detach(bdev); device_delete_child(dev, bdev); } /* during module unload there are lots of children leftover */ device_delete_children(dev); - if (sc->sc_dci.sc_irq_res && sc->sc_dci.sc_intr_hdl) { + if (sc->sc_irq_res && sc->sc_intr_hdl) { /* * only call dotg_obio_uninit() after dotg_obio_init() */ - dotg_uninit(&sc->sc_dci); + dwc_otg_uninit(sc); /* Stop OTG clock */ rt305x_sysctl_set(SYSCTL_CLKCFG1, rt305x_sysctl_get(SYSCTL_CLKCFG1) & ~SYSCTL_CLKCFG1_OTG_CLK_EN); - err = bus_teardown_intr(dev, sc->sc_dci.sc_irq_res, - sc->sc_dci.sc_intr_hdl); - sc->sc_dci.sc_intr_hdl = NULL; + err = bus_teardown_intr(dev, sc->sc_irq_res, + sc->sc_intr_hdl); + sc->sc_intr_hdl = NULL; } - if (sc->sc_dci.sc_irq_res) { + if (sc->sc_irq_res) { bus_release_resource(dev, SYS_RES_IRQ, 0, - sc->sc_dci.sc_irq_res); - sc->sc_dci.sc_irq_res = NULL; + sc->sc_irq_res); + sc->sc_irq_res = NULL; } - if (sc->sc_dci.sc_mem_res) { + if (sc->sc_io_res) { bus_release_resource(dev, SYS_RES_MEMORY, 0, - sc->sc_dci.sc_mem_res); - sc->sc_dci.sc_mem_res = NULL; + sc->sc_io_res); + sc->sc_io_res = NULL; } - usb_bus_mem_free_all(&sc->sc_dci.sc_bus, NULL); + usb_bus_mem_free_all(&sc->sc_bus, NULL); return (0); } @@ -221,11 +227,11 @@ static device_method_t dotg_obio_methods[] = { }; static driver_t dotg_obio_driver = { - .name = "dotg", + .name = "dwcotg", .methods = dotg_obio_methods, - .size = sizeof(struct dotg_obio_softc), + .size = sizeof(struct dwc_otg_softc), }; static devclass_t dotg_obio_devclass; -DRIVER_MODULE(dotg, obio, dotg_obio_driver, dotg_obio_devclass, 0, 0); +DRIVER_MODULE(dwcotg, obio, dotg_obio_driver, dotg_obio_devclass, 0, 0); diff --git a/sys/mips/rt305x/rt305x_ehci.c b/sys/mips/rt305x/rt305x_ehci.c new file mode 100644 index 0000000..9b2fa22 --- /dev/null +++ b/sys/mips/rt305x/rt305x_ehci.c @@ -0,0 +1,245 @@ +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +/*- + * Copyright (c) 2015 Stanislav Galabov. All rights reserved. + * Copyright (c) 2010,2011 Aleksandr Rybalko. All rights reserved. + * Copyright (c) 2007-2008 Hans Petter Selasky. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/stdint.h> +#include <sys/stddef.h> +#include <sys/param.h> +#include <sys/queue.h> +#include <sys/types.h> +#include <sys/systm.h> +#include <sys/kernel.h> +#include <sys/bus.h> +#include <sys/module.h> +#include <sys/lock.h> +#include <sys/mutex.h> +#include <sys/condvar.h> +#include <sys/sysctl.h> +#include <sys/sx.h> +#include <sys/unistd.h> +#include <sys/callout.h> +#include <sys/malloc.h> +#include <sys/priv.h> +#include <sys/rman.h> + +#include <dev/usb/usb.h> +#include <dev/usb/usbdi.h> + +#include <dev/usb/usb_core.h> +#include <dev/usb/usb_busdma.h> +#include <dev/usb/usb_process.h> +#include <dev/usb/usb_util.h> + +#include <dev/usb/usb_controller.h> +#include <dev/usb/usb_bus.h> + +#include <dev/usb/controller/ehci.h> + +#include <mips/rt305x/rt305xreg.h> +#include <mips/rt305x/rt305x_sysctlvar.h> + +#define EHCI_HC_DEVSTR "Ralink integrated USB 2.0 controller" + +static device_probe_t ehci_obio_probe; +static device_attach_t ehci_obio_attach; +static device_detach_t ehci_obio_detach; + +static int +ehci_obio_probe(device_t self) +{ + device_set_desc(self, EHCI_HC_DEVSTR); + + return (BUS_PROBE_DEFAULT); +} + +static int +ehci_obio_attach(device_t self) +{ + ehci_softc_t *sc = device_get_softc(self); + uint32_t reg; + int err; + int rid; + + /* setup controller interface softc */ + reg = rt305x_sysctl_get(SYSCTL_SYSCFG1); + reg |= SYSCTL_SYSCFG1_USB0_HOST_MODE; + rt305x_sysctl_set(SYSCTL_SYSCFG1, reg); + + reg = rt305x_sysctl_get(SYSCTL_CLKCFG1); + reg |= SYSCTL_CLKCFG1_UPHY0_CLK_EN; +#ifdef MT7620 + reg |= SYSCTL_CLKCFG1_UPHY1_CLK_EN; +#endif + rt305x_sysctl_set(SYSCTL_CLKCFG1, reg); + + reg = rt305x_sysctl_get(SYSCTL_RSTCTRL); + reg |= SYSCTL_RSTCTRL_UPHY0 | SYSCTL_RSTCTRL_UPHY1; + rt305x_sysctl_set(SYSCTL_RSTCTRL, reg); + reg &= ~(SYSCTL_RSTCTRL_UPHY0 | SYSCTL_RSTCTRL_UPHY1); + DELAY(100000); + rt305x_sysctl_set(SYSCTL_RSTCTRL, reg); + DELAY(100000); + + /* initialise some bus fields */ + sc->sc_bus.parent = self; + sc->sc_bus.devices = sc->sc_devices; + sc->sc_bus.devices_max = EHCI_MAX_DEVICES; + sc->sc_bus.dma_bits = 32; + + /* get all DMA memory */ + if (usb_bus_mem_alloc_all(&sc->sc_bus, + USB_GET_DMA_TAG(self), &ehci_iterate_hw_softc)) { + printf("No mem\n"); + return (ENOMEM); + } + + rid = 0; + sc->sc_io_res = bus_alloc_resource_any(self, SYS_RES_MEMORY, &rid, + RF_ACTIVE); + if (!sc->sc_io_res) { + device_printf(self, "Could not map memory\n"); + goto error; + } + sc->sc_io_tag = rman_get_bustag(sc->sc_io_res); + sc->sc_io_hdl = rman_get_bushandle(sc->sc_io_res); + sc->sc_io_size = rman_get_size(sc->sc_io_res); + + rid = 0; + sc->sc_irq_res = bus_alloc_resource_any(self, SYS_RES_IRQ, &rid, + RF_SHAREABLE | RF_ACTIVE); + if (sc->sc_irq_res == NULL) { + device_printf(self, "Could not allocate irq\n"); + goto error; + } + + sc->sc_bus.bdev = device_add_child(self, "usbus", -1); + if (!(sc->sc_bus.bdev)) { + device_printf(self, "Could not add USB device\n"); + goto error; + } + device_set_ivars(sc->sc_bus.bdev, &sc->sc_bus); + device_set_desc(sc->sc_bus.bdev, EHCI_HC_DEVSTR); + + sprintf(sc->sc_vendor, "Ralink"); + + err = bus_setup_intr(self, sc->sc_irq_res, INTR_TYPE_BIO | INTR_MPSAFE, + NULL, (driver_intr_t *)ehci_interrupt, sc, &sc->sc_intr_hdl); + if (err) { + device_printf(self, "Could not setup irq, %d\n", err); + sc->sc_intr_hdl = NULL; + goto error; + } + + err = ehci_init(sc); + if (!err) { + err = device_probe_and_attach(sc->sc_bus.bdev); + } + if (err) { + device_printf(self, "USB init failed err=%d\n", err); + goto error; + } + return (0); + +error: + ehci_obio_detach(self); + return (ENXIO); +} + +static int +ehci_obio_detach(device_t self) +{ + ehci_softc_t *sc = device_get_softc(self); + device_t bdev; + int err; + + if (sc->sc_bus.bdev) { + bdev = sc->sc_bus.bdev; + device_detach(bdev); + device_delete_child(self, bdev); + } + /* during module unload there are lots of children leftover */ + device_delete_children(self); + + if (sc->sc_irq_res && sc->sc_intr_hdl) { + /* + * only call ehci_detach() after ehci_init() + */ + ehci_detach(sc); + + /* Stop EHCI clock */ + rt305x_sysctl_set(SYSCTL_CLKCFG1, + rt305x_sysctl_get(SYSCTL_CLKCFG1) & + ~(SYSCTL_CLKCFG1_UPHY0_CLK_EN +#ifdef MT7620 + | SYSCTL_CLKCFG1_UPHY1_CLK_EN +#endif + )); + + err = bus_teardown_intr(self, sc->sc_irq_res, sc->sc_intr_hdl); + if (err) + device_printf(self, "Could not tear down irq, %d\n", + err); + sc->sc_intr_hdl = NULL; + } + if (sc->sc_irq_res) { + bus_release_resource(self, SYS_RES_IRQ, 0, + sc->sc_irq_res); + sc->sc_irq_res = NULL; + } + if (sc->sc_io_res) { + bus_release_resource(self, SYS_RES_MEMORY, 0, + sc->sc_io_res); + sc->sc_io_res = NULL; + } + usb_bus_mem_free_all(&sc->sc_bus, &ehci_iterate_hw_softc); + + return (0); +} + +static device_method_t ehci_obio_methods[] = { + /* Device interface */ + DEVMETHOD(device_probe, ehci_obio_probe), + DEVMETHOD(device_attach, ehci_obio_attach), + DEVMETHOD(device_detach, ehci_obio_detach), + DEVMETHOD(device_suspend, bus_generic_suspend), + DEVMETHOD(device_resume, bus_generic_resume), + DEVMETHOD(device_shutdown, bus_generic_shutdown), + + DEVMETHOD_END +}; + +static driver_t ehci_obio_driver = { + .name = "ehci", + .methods = ehci_obio_methods, + .size = sizeof(ehci_softc_t), +}; + +static devclass_t ehci_obio_devclass; + +DRIVER_MODULE(ehci, obio, ehci_obio_driver, ehci_obio_devclass, 0, 0); diff --git a/sys/mips/rt305x/rt305x_machdep.c b/sys/mips/rt305x/rt305x_machdep.c index dab2ce6..54e7906 100644 --- a/sys/mips/rt305x/rt305x_machdep.c +++ b/sys/mips/rt305x/rt305x_machdep.c @@ -71,6 +71,7 @@ __FBSDID("$FreeBSD$"); #include <machine/vmparam.h> #include <mips/rt305x/rt305xreg.h> +#include <mips/rt305x/rt305x_sysctlvar.h> extern int *edata; extern int *end; @@ -87,11 +88,17 @@ static void mips_init(void) { int i; + char *memsize; printf("entry: mips_init()\n"); + if ((memsize = kern_getenv("memsize")) != NULL) + realmem = btoc(strtol(memsize, NULL, 0) << 20); + else + realmem = btoc(32 << 20); + + bootverbose = 1; - realmem = btoc(32 << 20); for (i = 0; i < 10; i++) { phys_avail[i] = 0; @@ -120,8 +127,13 @@ void platform_reset(void) { +#if !defined(MT7620) && !defined(RT5350) __asm __volatile("li $25, 0xbf000000"); __asm __volatile("j $25"); +#else + rt305x_sysctl_set(SYSCTL_RSTCTRL, 1); + while (1); +#endif } void @@ -144,6 +156,8 @@ platform_start(__register_t a0 __unused, __register_t a1 __unused, /* Initialize pcpu stuff */ mips_pcpu0_init(); + mips_timer_early_init(platform_counter_freq / 2); + /* initialize console so that we have printf */ boothowto |= (RB_SERIAL | RB_MULTIPLE); /* Use multiple consoles */ boothowto |= (RB_VERBOSE); @@ -173,10 +187,12 @@ platform_start(__register_t a0 __unused, __register_t a1 __unused, printf("Environment:\n"); - for (i = 0; envp[i] ; i++) { + for (i = 0; envp[i] && MIPS_IS_VALID_PTR(envp[i]); i++) { char *n, *arg; arg = (char *)(intptr_t)MIPS_PHYS_TO_KSEG0(envp[i]); + if (! MIPS_IS_VALID_PTR(arg)) + continue; printf("\t%s\n", arg); n = strsep(&arg, "="); if (arg == NULL) diff --git a/sys/mips/rt305x/rt305x_ohci.c b/sys/mips/rt305x/rt305x_ohci.c new file mode 100644 index 0000000..e726e60 --- /dev/null +++ b/sys/mips/rt305x/rt305x_ohci.c @@ -0,0 +1,245 @@ +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +/*- + * Copyright (c) 2015 Stanislav Galabov. All rights reserved. + * Copyright (c) 2010,2011 Aleksandr Rybalko. All rights reserved. + * Copyright (c) 2007-2008 Hans Petter Selasky. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/stdint.h> +#include <sys/stddef.h> +#include <sys/param.h> +#include <sys/queue.h> +#include <sys/types.h> +#include <sys/systm.h> +#include <sys/kernel.h> +#include <sys/bus.h> +#include <sys/module.h> +#include <sys/lock.h> +#include <sys/mutex.h> +#include <sys/condvar.h> +#include <sys/sysctl.h> +#include <sys/sx.h> +#include <sys/unistd.h> +#include <sys/callout.h> +#include <sys/malloc.h> +#include <sys/priv.h> +#include <sys/rman.h> + +#include <dev/usb/usb.h> +#include <dev/usb/usbdi.h> + +#include <dev/usb/usb_core.h> +#include <dev/usb/usb_busdma.h> +#include <dev/usb/usb_process.h> +#include <dev/usb/usb_util.h> + +#include <dev/usb/usb_controller.h> +#include <dev/usb/usb_bus.h> + +#include <dev/usb/controller/ohci.h> + +#include <mips/rt305x/rt305xreg.h> +#include <mips/rt305x/rt305x_sysctlvar.h> + +#define OHCI_HC_DEVSTR "Ralink integrated USB controller" + +static device_probe_t ohci_obio_probe; +static device_attach_t ohci_obio_attach; +static device_detach_t ohci_obio_detach; + +static int +ohci_obio_probe(device_t self) +{ + device_set_desc(self, OHCI_HC_DEVSTR); + + return (BUS_PROBE_DEFAULT); +} + +static int +ohci_obio_attach(device_t self) +{ + ohci_softc_t *sc = device_get_softc(self); + uint32_t reg; + int err; + int rid; + + /* setup controller interface softc */ + reg = rt305x_sysctl_get(SYSCTL_SYSCFG1); + reg |= SYSCTL_SYSCFG1_USB0_HOST_MODE; + rt305x_sysctl_set(SYSCTL_SYSCFG1, reg); + + reg = rt305x_sysctl_get(SYSCTL_CLKCFG1); + reg |= SYSCTL_CLKCFG1_UPHY0_CLK_EN; +#ifdef MT7620 + reg |= SYSCTL_CLKCFG1_UPHY1_CLK_EN; +#endif + rt305x_sysctl_set(SYSCTL_CLKCFG1, reg); + + reg = rt305x_sysctl_get(SYSCTL_RSTCTRL); + reg |= SYSCTL_RSTCTRL_UPHY0 | SYSCTL_RSTCTRL_UPHY1; + rt305x_sysctl_set(SYSCTL_RSTCTRL, reg); + reg &= ~(SYSCTL_RSTCTRL_UPHY0 | SYSCTL_RSTCTRL_UPHY1); + DELAY(100000); + rt305x_sysctl_set(SYSCTL_RSTCTRL, reg); + DELAY(100000); + + /* initialise some bus fields */ + sc->sc_bus.parent = self; + sc->sc_bus.devices = sc->sc_devices; + sc->sc_bus.devices_max = OHCI_MAX_DEVICES; + sc->sc_bus.dma_bits = 32; + + /* get all DMA memory */ + if (usb_bus_mem_alloc_all(&sc->sc_bus, + USB_GET_DMA_TAG(self), &ohci_iterate_hw_softc)) { + printf("No mem\n"); + return (ENOMEM); + } + + rid = 0; + sc->sc_io_res = bus_alloc_resource_any(self, SYS_RES_MEMORY, &rid, + RF_ACTIVE); + if (!sc->sc_io_res) { + device_printf(self, "Could not map memory\n"); + goto error; + } + sc->sc_io_tag = rman_get_bustag(sc->sc_io_res); + sc->sc_io_hdl = rman_get_bushandle(sc->sc_io_res); + sc->sc_io_size = rman_get_size(sc->sc_io_res); + + rid = 0; + sc->sc_irq_res = bus_alloc_resource_any(self, SYS_RES_IRQ, &rid, + RF_SHAREABLE | RF_ACTIVE); + if (sc->sc_irq_res == NULL) { + device_printf(self, "Could not allocate irq\n"); + goto error; + } + + sc->sc_bus.bdev = device_add_child(self, "usbus", -1); + if (!(sc->sc_bus.bdev)) { + device_printf(self, "Could not add USB device\n"); + goto error; + } + device_set_ivars(sc->sc_bus.bdev, &sc->sc_bus); + device_set_desc(sc->sc_bus.bdev, OHCI_HC_DEVSTR); + + sprintf(sc->sc_vendor, "Ralink"); + + err = bus_setup_intr(self, sc->sc_irq_res, INTR_TYPE_BIO | INTR_MPSAFE, + NULL, (driver_intr_t *)ohci_interrupt, sc, &sc->sc_intr_hdl); + if (err) { + device_printf(self, "Could not setup irq, %d\n", err); + sc->sc_intr_hdl = NULL; + goto error; + } + + err = ohci_init(sc); + if (!err) { + err = device_probe_and_attach(sc->sc_bus.bdev); + } + if (err) { + device_printf(self, "USB init failed err=%d\n", err); + goto error; + } + return (0); + +error: + ohci_obio_detach(self); + return (ENXIO); +} + +static int +ohci_obio_detach(device_t self) +{ + ohci_softc_t *sc = device_get_softc(self); + device_t bdev; + int err; + + if (sc->sc_bus.bdev) { + bdev = sc->sc_bus.bdev; + device_detach(bdev); + device_delete_child(self, bdev); + } + /* during module unload there are lots of children leftover */ + device_delete_children(self); + + if (sc->sc_irq_res && sc->sc_intr_hdl) { + /* + * only call ohci_detach() after ohci_init() + */ + ohci_detach(sc); + + /* Stop OHCI clock */ + rt305x_sysctl_set(SYSCTL_CLKCFG1, + rt305x_sysctl_get(SYSCTL_CLKCFG1) & + ~(SYSCTL_CLKCFG1_UPHY0_CLK_EN +#ifdef MT7620 + | SYSCTL_CLKCFG1_UPHY1_CLK_EN +#endif + )); + + err = bus_teardown_intr(self, sc->sc_irq_res, sc->sc_intr_hdl); + if (err) + device_printf(self, "Could not tear down irq, %d\n", + err); + sc->sc_intr_hdl = NULL; + } + if (sc->sc_irq_res) { + bus_release_resource(self, SYS_RES_IRQ, 0, + sc->sc_irq_res); + sc->sc_irq_res = NULL; + } + if (sc->sc_io_res) { + bus_release_resource(self, SYS_RES_MEMORY, 0, + sc->sc_io_res); + sc->sc_io_res = NULL; + } + usb_bus_mem_free_all(&sc->sc_bus, &ohci_iterate_hw_softc); + + return (0); +} + +static device_method_t ohci_obio_methods[] = { + /* Device interface */ + DEVMETHOD(device_probe, ohci_obio_probe), + DEVMETHOD(device_attach, ohci_obio_attach), + DEVMETHOD(device_detach, ohci_obio_detach), + DEVMETHOD(device_suspend, bus_generic_suspend), + DEVMETHOD(device_resume, bus_generic_resume), + DEVMETHOD(device_shutdown, bus_generic_shutdown), + + DEVMETHOD_END +}; + +static driver_t ohci_obio_driver = { + .name = "ohci", + .methods = ohci_obio_methods, + .size = sizeof(ohci_softc_t), +}; + +static devclass_t ohci_obio_devclass; + +DRIVER_MODULE(ohci, obio, ohci_obio_driver, ohci_obio_devclass, 0, 0); diff --git a/sys/mips/rt305x/rt305x_pci.c b/sys/mips/rt305x/rt305x_pci.c new file mode 100644 index 0000000..c681dc6 --- /dev/null +++ b/sys/mips/rt305x/rt305x_pci.c @@ -0,0 +1,955 @@ +/*- + * Copyright (c) 2015 Stanislav Galabov. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * This is based on the pci allocator code from sys/dev/arm/mv/: + * + * Copyright (c) 2008 MARVELL INTERNATIONAL LTD. + * Copyright (c) 2010 The FreeBSD Foundation + * Copyright (c) 2010-2012 Semihalf + * All rights reserved. + * + * Developed by Semihalf. + */ +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/systm.h> + +#include <sys/bus.h> +#include <sys/interrupt.h> +#include <sys/malloc.h> +#include <sys/kernel.h> +#include <sys/module.h> +#include <sys/rman.h> +#include <sys/lock.h> +#include <sys/mutex.h> +#include <sys/endian.h> + +#include <vm/vm.h> +#include <vm/pmap.h> +#include <vm/vm_extern.h> + +#include <machine/bus.h> +#include <machine/cpu.h> +#include <machine/intr_machdep.h> +#include <machine/pmap.h> + +#include <dev/pci/pcivar.h> +#include <dev/pci/pcireg.h> + +#include <dev/pci/pcib_private.h> +#include "pcib_if.h" + +#include <mips/rt305x/rt305xreg.h> +#include <mips/rt305x/rt305x_pcireg.h> +#include <mips/rt305x/rt305x_sysctlvar.h> + +struct mtx rt305x_pci_mtx; +MTX_SYSINIT(rt305x_pci_mtx, &rt305x_pci_mtx, "rt305x PCI/PCIe mutex", MTX_SPIN); + +struct rt305x_pci_softc { + device_t sc_dev; + + bus_space_tag_t sc_bst; + bus_space_handle_t sc_bsh; + + int sc_busno; + + struct rman sc_mem_rman; + struct rman sc_io_rman; + struct rman sc_irq_rman; + + bus_addr_t sc_mem_base; + bus_addr_t sc_mem_size; + uint32_t sc_mem_map[(256*1024*1024) / + (PCI_MIN_MEM_ALLOC * BITS_PER_UINT32)]; + + bus_addr_t sc_io_base; + bus_addr_t sc_io_size; + uint32_t sc_io_map[(16*1024*1024) / + (PCI_MIN_IO_ALLOC * BITS_PER_UINT32)]; + + struct intr_event *sc_eventstab[RT305X_PCI_NIRQS]; + mips_intrcnt_t sc_intr_counter[RT305X_PCI_NIRQS]; + + int pcie_link_status; +}; + +static void rt305x_pci_phy_init(device_t); +static void rt305x_pci_init(device_t); +static int rt305x_pcib_init(device_t, int, int); +static int rt305x_pci_intr(void *); + +static void rt305x_pci_dump_regs(device_t); + +static struct rt305x_pci_softc *rt_sc = NULL; + +static int +rt305x_pci_probe(device_t dev) +{ + + return (BUS_PROBE_NOWILDCARD); +} + +static int +rt305x_pci_attach(device_t dev) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + + rt_sc = sc; + + sc->sc_dev = dev; + sc->sc_mem_base = PCIE_MEM_BASE; + sc->sc_mem_size = 0x10000000; + sc->sc_io_base = PCIE_IO_BASE; + sc->sc_io_size = 0x10000; + + sc->sc_bsh = MIPS_PHYS_TO_KSEG1(PCIE_BASE); + sc->sc_bst = mips_bus_space_generic; + + sc->sc_mem_rman.rm_type = RMAN_ARRAY; + sc->sc_mem_rman.rm_descr = "rt305x pci memory window"; + if (rman_init(&sc->sc_mem_rman) != 0 || + rman_manage_region(&sc->sc_mem_rman, sc->sc_mem_base, + sc->sc_mem_base + sc->sc_mem_size - 1) != 0) { + panic("%s: failed to set up memory rman", __FUNCTION__); + } + + sc->sc_io_rman.rm_type = RMAN_ARRAY; + sc->sc_io_rman.rm_descr = "rt305x pci io window"; + if (rman_init(&sc->sc_io_rman) != 0 || + rman_manage_region(&sc->sc_io_rman, sc->sc_io_base, + sc->sc_io_base + sc->sc_io_size - 1) != 0) { + panic("%s: failed to set up io rman", __FUNCTION__); + } + + sc->sc_irq_rman.rm_type = RMAN_ARRAY; + sc->sc_irq_rman.rm_descr = "rt305x pci irqs"; + if (rman_init(&sc->sc_irq_rman) != 0 || + rman_manage_region(&sc->sc_irq_rman, RT305X_PCIE0_IRQ, + RT305X_PCIE0_IRQ) != 0) { + panic("%s: failed to set up irq rman", __FUNCTION__); + } + + cpu_establish_hardintr("pci", rt305x_pci_intr, NULL, sc, + RT305X_PCI_INTR_PIN, INTR_TYPE_MISC | INTR_EXCL, NULL); + + rt305x_pci_phy_init(dev); + + rt305x_pci_init(dev); + + rt305x_pci_dump_regs(dev); + + rt305x_pcib_init(dev, 0, PCI_SLOTMAX); + + device_add_child(dev, "pci", -1); + + return (bus_generic_attach(dev)); +} + +static int +rt305x_pci_read_ivar(device_t dev, device_t child, int which, + uintptr_t *result) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + + switch (which) { + case PCIB_IVAR_DOMAIN: + *result = device_get_unit(dev); + return (0); + case PCIB_IVAR_BUS: + *result = sc->sc_busno; + return (0); + } + + return (ENOENT); +} + +static int +rt305x_pci_write_ivar(device_t dev, device_t child, int which, + uintptr_t result) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + + switch (which) { + case PCIB_IVAR_BUS: + sc->sc_busno = result; + return (0); + } + + return (ENOENT); +} + +static struct resource * +rt305x_pci_alloc_resource(device_t bus, device_t child, int type, int *rid, + u_long start, u_long end, u_long count, u_int flags) +{ + struct rt305x_pci_softc *sc = device_get_softc(bus); + struct resource *rv; + struct rman *rm; + vm_offset_t va; + + switch (type) { + case SYS_RES_IRQ: + rm = &sc->sc_irq_rman; + break; + case SYS_RES_IOPORT: + rm = &sc->sc_io_rman; + break; + case SYS_RES_MEMORY: + rm = &sc->sc_mem_rman; + break; + default: + return (NULL); + } + + rv = rman_reserve_resource(rm, start, end, count, flags, child); + + if (rv == NULL) + return (NULL); + + rman_set_rid(rv, *rid); + + if (type != SYS_RES_IRQ) { + if (type == SYS_RES_MEMORY) { + va = (vm_offset_t)pmap_mapdev(start, count); + } else if (type == SYS_RES_IOPORT){ + va = (vm_offset_t)MIPS_PHYS_TO_KSEG1(start); + } + rman_set_bushandle(rv, va); + rman_set_virtual(rv, (void *)va); + rman_set_bustag(rv, mips_bus_space_generic); + } + + if (flags & RF_ACTIVE) { + if (bus_activate_resource(child, type, *rid, rv)) { + rman_release_resource(rv); + return (NULL); + } + } + + return (rv); +} + +static int +rt305x_pci_activate_resource(device_t bus, device_t child, int type, int rid, + struct resource *r) +{ + + return rman_activate_resource(r); +} + +static inline int +rt305x_idx_to_irq(int idx) +{ + + return ((idx == 0) ? RT305X_PCIE0_IRQ : + (idx == 1) ? RT305X_PCIE1_IRQ : + (idx == 2) ? RT305X_PCIE2_IRQ : -1); +} + +static inline int +rt305x_irq_to_idx(int irq) +{ + + return ((irq == RT305X_PCIE0_IRQ) ? 0 : + (irq == RT305X_PCIE1_IRQ) ? 1 : + (irq == RT305X_PCIE2_IRQ) ? 2 : -1); +} + +static void +rt305x_pci_mask_irq(void *source) +{ + + RT_WRITE32(rt_sc, RT305X_PCI_PCIENA, + RT_READ32(rt_sc, RT305X_PCI_PCIENA) & ~(1<<((int)source))); +} + +static void +rt305x_pci_unmask_irq(void *source) +{ + + RT_WRITE32(rt_sc, RT305X_PCI_PCIENA, + RT_READ32(rt_sc, RT305X_PCI_PCIENA) | (1<<((int)source))); +} + +static int +rt305x_pci_setup_intr(device_t bus, device_t child, struct resource *ires, + int flags, driver_filter_t *filt, driver_intr_t *handler, + void *arg, void **cookiep) +{ + struct rt305x_pci_softc *sc = device_get_softc(bus); + struct intr_event *event; + int irq, error, irqidx; + + irq = rman_get_start(ires); + + if ((irqidx = rt305x_irq_to_idx(irq)) == -1) + panic("%s: bad irq %d", __FUNCTION__, irq); + + event = sc->sc_eventstab[irqidx]; + if (event == NULL) { + error = intr_event_create(&event, (void *)irq, 0, irq, + rt305x_pci_mask_irq, rt305x_pci_unmask_irq, NULL, NULL, + "pci intr%d:", irq); + + if (error == 0) { + sc->sc_eventstab[irqidx] = event; + sc->sc_intr_counter[irqidx] = + mips_intrcnt_create(event->ie_name); + } + else + return (error); + } + + intr_event_add_handler(event, device_get_nameunit(child), filt, + handler, arg, intr_priority(flags), flags, cookiep); + + mips_intrcnt_setname(sc->sc_intr_counter[irqidx], event->ie_fullname); + + rt305x_pci_unmask_irq((void*)irq); + + return (0); +} + +static int +rt305x_pci_teardown_intr(device_t dev, device_t child, struct resource *ires, + void *cookie) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + int irq, result, irqidx; + + irq = rman_get_start(ires); + if ((irqidx = rt305x_irq_to_idx(irq)) == -1) + panic("%s: bad irq %d", __FUNCTION__, irq); + + if (sc->sc_eventstab[irqidx] == NULL) + panic("Trying to teardown unoccupied IRQ"); + + rt305x_pci_mask_irq((void*)irq); + + result = intr_event_remove_handler(cookie); + if (!result) + sc->sc_eventstab[irqidx] = NULL; + + return (result); +} + +static inline uint32_t +rt305x_pci_make_addr(int bus, int slot, int func, int reg) +{ + uint32_t addr; + + addr = (((reg & 0xf00) >> 8) << 24) | (bus << 16) | (slot << 11) | + (func << 8) | (reg & 0xfc) | (1 << 31); + + return (addr); +} + +static int +rt305x_pci_maxslots(device_t dev) +{ + + return (PCI_SLOTMAX); +} + +static uint32_t +rt305x_pci_read_config(device_t dev, u_int bus, u_int slot, u_int func, + u_int reg, int bytes) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + uint32_t addr = 0, data = 0; + + if (bus == 0 && (sc->pcie_link_status & (1<<slot)) == 0) + return (~0U); + + mtx_lock_spin(&rt305x_pci_mtx); + addr = rt305x_pci_make_addr(bus, slot, func, (reg & ~3)); + RT_WRITE32(sc, RT305X_PCI_CFGADDR, addr); + switch (bytes % 4) { + case 0: + data = RT_READ32(sc, RT305X_PCI_CFGDATA); + break; + case 1: + data = RT_READ8(sc, RT305X_PCI_CFGDATA + (reg & 0x3)); + break; + case 2: + data = RT_READ16(sc, RT305X_PCI_CFGDATA + (reg & 0x3)); + break; + default: + panic("%s(): Wrong number of bytes (%d) requested!\n", + __FUNCTION__, bytes % 4); + } + mtx_unlock_spin(&rt305x_pci_mtx); + + return (data); +} + +static void +rt305x_pci_write_config(device_t dev, u_int bus, u_int slot, u_int func, + u_int reg, uint32_t val, int bytes) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + uint32_t addr = 0, data = val; + + if (bus == 0 && (sc->pcie_link_status & (1<<slot)) == 0) + return; + + mtx_lock_spin(&rt305x_pci_mtx); + addr = rt305x_pci_make_addr(bus, slot, func, (reg & ~3)); + RT_WRITE32(sc, RT305X_PCI_CFGADDR, addr); + switch (bytes % 4) { + case 0: + RT_WRITE32(sc, RT305X_PCI_CFGDATA, data); + break; + case 1: + RT_WRITE8(sc, RT305X_PCI_CFGDATA + (reg & 0x3), data); + break; + case 2: + RT_WRITE16(sc, RT305X_PCI_CFGDATA + (reg & 0x3), data); + break; + default: + panic("%s(): Wrong number of bytes (%d) requested!\n", + __FUNCTION__, bytes % 4); + } + mtx_unlock_spin(&rt305x_pci_mtx); +} + +static int +rt305x_pci_route_interrupt(device_t pcib, device_t device, int pin) +{ + //struct rt305x_pci_softc *sc = device_get_softc(pcib); + int bus, sl; + + bus = pci_get_bus(device); + sl = pci_get_slot(device); + + if (bus != 0) + panic("Unexpected bus number %d\n", bus); + + //printf("%s: not done yet.\n", __FUNCTION__); + + switch (sl) { + case 0: return RT305X_PCIE0_IRQ; + default: return (-1); + } + + return (-1); +} + +static device_method_t rt305x_pci_methods[] = { + /* Device interface */ + DEVMETHOD(device_probe, rt305x_pci_probe), + DEVMETHOD(device_attach, rt305x_pci_attach), + DEVMETHOD(device_shutdown, bus_generic_shutdown), + DEVMETHOD(device_suspend, bus_generic_suspend), + DEVMETHOD(device_resume, bus_generic_resume), + + /* Bus interface */ + DEVMETHOD(bus_read_ivar, rt305x_pci_read_ivar), + DEVMETHOD(bus_write_ivar, rt305x_pci_write_ivar), + DEVMETHOD(bus_alloc_resource, rt305x_pci_alloc_resource), + DEVMETHOD(bus_release_resource, bus_generic_release_resource), + DEVMETHOD(bus_activate_resource, rt305x_pci_activate_resource), + DEVMETHOD(bus_deactivate_resource, bus_generic_deactivate_resource), + DEVMETHOD(bus_setup_intr, rt305x_pci_setup_intr), + DEVMETHOD(bus_teardown_intr, rt305x_pci_teardown_intr), + + /* pcib interface */ + DEVMETHOD(pcib_maxslots, rt305x_pci_maxslots), + DEVMETHOD(pcib_read_config, rt305x_pci_read_config), + DEVMETHOD(pcib_write_config, rt305x_pci_write_config), + DEVMETHOD(pcib_route_interrupt, rt305x_pci_route_interrupt), + + DEVMETHOD_END +}; + +static driver_t rt305x_pci_driver = { + "pcib", + rt305x_pci_methods, + sizeof(struct rt305x_pci_softc), +}; + +static devclass_t rt305x_pci_devclass; + +DRIVER_MODULE(rt305x_pci, nexus, rt305x_pci_driver, rt305x_pci_devclass, 0, 0); + +static void +rt305x_pci_dump_regs(device_t dev) +{ +#if 0 + struct rt305x_pci_softc *sc = device_get_softc(dev); + uint32_t reg = 0; + + reg = 0x0000; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x0008; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x000c; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x0020; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x0024; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x0028; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x002c; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2010; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2014; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2018; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2030; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2034; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2038; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2050; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2060; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); + + reg = 0x2064; + printf("0x%04x: 0x%08x\n", reg, RT_READ32(sc, reg)); +#endif +} + +static void +rt305x_pci_init(device_t dev) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + uint32_t tmp; + + rt305x_sysctl_set(SYSCTL_SYSCFG1, + rt305x_sysctl_get(SYSCTL_SYSCFG1) | (1 << 8)); + + rt305x_sysctl_set(SYSCTL_GPIOMODE, + rt305x_sysctl_get(SYSCTL_GPIOMODE) & ~(0x3 << 16)); + rt305x_sysctl_set(SYSCTL_RSTCTRL, + rt305x_sysctl_get(SYSCTL_RSTCTRL) & ~(1<<26)); + rt305x_sysctl_set(SYSCTL_CLKCFG1, + rt305x_sysctl_get(SYSCTL_CLKCFG1) | (1<<26)); + + tmp = rt305x_sysctl_get(SYSCTL_PPLL_CFG1); + if ((tmp & (1<<23)) == 0) { + device_printf(dev, "PPLL not locked\n"); + return; + } + + tmp = rt305x_sysctl_get(SYSCTL_PPLL_DRV); + tmp |= (1<<19); + rt305x_sysctl_set(SYSCTL_PPLL_DRV, tmp); + tmp &= ~(1<<18); + rt305x_sysctl_set(SYSCTL_PPLL_DRV, tmp); + tmp &= ~(1<<17); + rt305x_sysctl_set(SYSCTL_PPLL_DRV, tmp); + tmp|= (1<<31); + rt305x_sysctl_set(SYSCTL_PPLL_DRV, tmp); + + RT_WRITE32(sc, RT305X_PCI_MEMBASE, sc->sc_mem_base); + RT_WRITE32(sc, RT305X_PCI_IOBASE, sc->sc_io_base); + + RT_WRITE32(sc, RT305X_PCI_PCICFG, RT_READ32(sc, 0) & ~(1<<1)); + DELAY(500000); + if ((RT_READ32(sc, RT305X_PCI_PCIE0_STATUS) & 0x1) == 1) + sc->pcie_link_status = 1; + else + sc->pcie_link_status = 0; + + RT_WRITE32(sc, RT305X_PCI_PCIE0_BAR0SETUP, 0x7FFF0001); + RT_WRITE32(sc, RT305X_PCI_PCIE0_BAR1SETUP, 0x00000000); + RT_WRITE32(sc, RT305X_PCI_PCIE0_IMBASEBAR0, 0x00000000); + RT_WRITE32(sc, RT305X_PCI_PCIE0_CLASS, 0x06040001); + + tmp = rt305x_pci_read_config(dev, 0, 0, 0, 4, 4); + rt305x_pci_write_config(dev, 0, 0, 0, 4, tmp | 0x7, 4); + tmp = rt305x_pci_read_config(dev, 0, 0, 0, 0x70c, 4); + tmp &= ~(0xff)<<8; + tmp |= 0x50<<8; + rt305x_pci_write_config(dev, 0, 0, 0, 0x70c, tmp, 4); + tmp = rt305x_pci_read_config(dev, 0, 0, 0, 0x70c, 4); + + rt305x_pci_write_config(dev, 0, 0, 0, PCIR_BAR(0), 0, 4); +} + +static inline uint32_t +pcib_bit_get(uint32_t *map, uint32_t bit) +{ + uint32_t n = bit / BITS_PER_UINT32; + + bit = bit % BITS_PER_UINT32; + return (map[n] & (1 << bit)); +} + +static inline void +pcib_bit_set(uint32_t *map, uint32_t bit) +{ + uint32_t n = bit / BITS_PER_UINT32; + + bit = bit % BITS_PER_UINT32; + map[n] |= (1 << bit); +} + +static inline uint32_t +pcib_map_check(uint32_t *map, uint32_t start, uint32_t bits) +{ + uint32_t i; + + for (i = start; i < start + bits; i++) + if (pcib_bit_get(map, i)) + return (0); + + return (1); +} + +static inline void +pcib_map_set(uint32_t *map, uint32_t start, uint32_t bits) +{ + uint32_t i; + + for (i = start; i < start + bits; i++) + pcib_bit_set(map, i); +} + +static bus_addr_t +pcib_alloc(device_t dev, uint32_t smask) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + uint32_t bits, bits_limit, i, *map, min_alloc, size; + bus_addr_t addr = 0; + bus_addr_t base; + + if (smask & 1) { + base = sc->sc_io_base; + min_alloc = PCI_MIN_IO_ALLOC; + bits_limit = sc->sc_io_size / min_alloc; + map = sc->sc_io_map; + smask &= ~0x3; + } else { + base = sc->sc_mem_base; + min_alloc = PCI_MIN_MEM_ALLOC; + bits_limit = sc->sc_mem_size / min_alloc; + map = sc->sc_mem_map; + smask &= ~0xF; + } + + size = ~smask + 1; + bits = size / min_alloc; + + for (i = 0; i + bits <= bits_limit; i+= bits) + if (pcib_map_check(map, i, bits)) { + pcib_map_set(map, i, bits); + addr = base + (i * min_alloc); + return (addr); + } + + return (addr); +} + +static int +rt305x_pcib_init_bar(device_t dev, int bus, int slot, int func, int barno) +{ + uint32_t addr, bar; + int reg, width; + + reg = PCIR_BAR(barno); + + rt305x_pci_write_config(dev, bus, slot, func, reg, ~0, 4); + bar = rt305x_pci_read_config(dev, bus, slot, func, reg, 4); + if (bar == 0) + return (1); + + /* Calculate BAR size: 64 or 32 bit (in 32-bit units) */ + width = ((bar & 7) == 4) ? 2 : 1; + + addr = pcib_alloc(dev, bar); + if (!addr) + return (-1); + + if (bootverbose) + printf("PCI %u:%u:%u: reg %x: smask=%08x: addr=%08x\n", + bus, slot, func, reg, bar, addr); + + rt305x_pci_write_config(dev, bus, slot, func, reg, addr, 4); + if (width == 2) + rt305x_pci_write_config(dev, bus, slot, func, reg + 4, 0, 4); + + return (width); +} + +static int +rt305x_pcib_init_all_bars(device_t dev, int bus, int slot, int func, + int hdrtype) +{ + int maxbar, bar, i; + + maxbar = (hdrtype & PCIM_HDRTYPE) ? 0 : 6; + bar = 0; + + while (bar < maxbar) { + i = rt305x_pcib_init_bar(dev, bus, slot, func, bar); + bar += i; + if (i < 0) { + device_printf(dev, "PCI IO/Memory space exhausted\n"); + return (ENOMEM); + } + } + + return (0); +} + +static inline int +rt305x_pci_slot_has_link(device_t dev, int slot) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + + return !!(sc->pcie_link_status & (1<<slot)); +} + +static int cur_secbus = 0; + +static void +rt305x_pcib_init_bridge(device_t dev, int bus, int slot, int func) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + bus_addr_t io_base, mem_base; + uint32_t io_limit, mem_limit; + int secbus; + + if (bus == 0 && !rt305x_pci_slot_has_link(dev, slot)) { + device_printf(dev, "Skip bus %d due to no link\n",++cur_secbus); + return; + } + + io_base = sc->sc_io_base; + io_limit = io_base + sc->sc_io_size - 1; + mem_base = sc->sc_mem_base; + mem_limit = mem_base + sc->sc_mem_size - 1; + + rt305x_pci_write_config(dev, bus, slot, func, PCIR_IOBASEL_1, + io_base >> 8, 1); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_IOBASEH_1, + io_base >> 16, 2); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_IOLIMITL_1, + io_limit >> 8, 1); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_IOLIMITH_1, + io_limit >> 16, 2); + + rt305x_pci_write_config(dev, bus, slot, func, PCIR_MEMBASE_1, + mem_base >> 16, 2); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_MEMLIMIT_1, + mem_limit >> 16, 2); + + rt305x_pci_write_config(dev, bus, slot, func, PCIR_PMBASEL_1, + 0x10, 2); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_PMBASEH_1, + 0x0, 4); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_PMLIMITL_1, + 0xF, 2); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_PMLIMITH_1, + 0x0, 4); + + secbus = rt305x_pci_read_config(dev, bus, slot, func, PCIR_SECBUS_1, 1); + + if (secbus == 0) { + rt305x_pci_write_config(dev, bus, slot, func, PCIR_SECBUS_1, + ++cur_secbus, 1); + rt305x_pci_write_config(dev, bus, slot, func, PCIR_SUBBUS_1, + cur_secbus, 1); + secbus = cur_secbus; + } + + rt305x_pcib_init(dev, secbus, PCI_SLOTMAX); +} + +static int +rt305x_pcib_init(device_t dev, int bus, int maxslot) +{ + int slot, func, maxfunc, error; + uint8_t hdrtype, command, class, subclass; + + for (slot = 0; slot <= maxslot; slot++) { + maxfunc = 0; + for (func = 0; func <= maxfunc; func++) { + hdrtype = rt305x_pci_read_config(dev, bus, slot, func, + PCIR_HDRTYPE, 1); + + if ((hdrtype & PCIM_HDRTYPE) > PCI_MAXHDRTYPE) + continue; + + if (func == 0 && (hdrtype & PCIM_MFDEV)) + maxfunc = PCI_FUNCMAX; + + command = rt305x_pci_read_config(dev, bus, slot, func, + PCIR_COMMAND, 1); + command &= ~(PCIM_CMD_MEMEN | PCIM_CMD_PORTEN); + rt305x_pci_write_config(dev, bus, slot, func, + PCIR_COMMAND, command, 1); + + error = rt305x_pcib_init_all_bars(dev, bus, slot, func, + hdrtype); + + if (error) + return (error); + + command |= PCIM_CMD_BUSMASTEREN | PCIM_CMD_MEMEN | + PCIM_CMD_PORTEN; + rt305x_pci_write_config(dev, bus, slot, func, + PCIR_COMMAND, command, 1); + + rt305x_pci_write_config(dev, bus, slot, func, + PCIR_CACHELNSZ, 16, 1); + + class = rt305x_pci_read_config(dev, bus, slot, func, + PCIR_CLASS, 1); + subclass = rt305x_pci_read_config(dev, bus, slot, func, + PCIR_SUBCLASS, 1); + + if (class != PCIC_BRIDGE || subclass != PCIS_BRIDGE_PCI) + continue; + + rt305x_pcib_init_bridge(dev, bus, slot, func); + } + } + + return (0); +} + +#define BUSY 0x80000000 +#define WAITRETRY_MAX 10 +#define WRITE_MODE (1<<23) +#define DATA_SHIFT 0 +#define ADDR_SHIFT 8 + +static int +rt305x_wait_pci_phy_busy(struct rt305x_pci_softc *sc) +{ + uint32_t reg_value = 0x0, retry = 0; + + while (1) { + reg_value = RT_READ32(sc, RT305X_PCI_PHY0_CFG); + if (reg_value & BUSY) + DELAY(100000); + else + break; + if (retry++ > WAITRETRY_MAX) { + printf("PHY retry failed\n"); + return (-1); + } + } + return (0); +} + +static uint32_t +rt305x_pci_phy(struct rt305x_pci_softc *sc, char rwmode, uint32_t addr, + uint32_t val) +{ + uint32_t reg_value = 0x0; + + rt305x_wait_pci_phy_busy(sc); + if (rwmode == 'w') { + reg_value |= WRITE_MODE; + reg_value |= (val) << DATA_SHIFT; + } + reg_value |= (addr) << ADDR_SHIFT; + + RT_WRITE32(sc, RT305X_PCI_PHY0_CFG, reg_value); + DELAY(1000); + + rt305x_wait_pci_phy_busy(sc); + + if (rwmode == 'r') { + reg_value = RT_READ32(sc, RT305X_PCI_PHY0_CFG); + return (reg_value); + } + + return (0); +} + +static void +rt305x_pci_phy_init(device_t dev) +{ + struct rt305x_pci_softc *sc = device_get_softc(dev); + uint32_t tmp; + + rt305x_pci_phy(sc, 'w', 0x00, 0x80); + rt305x_pci_phy(sc, 'w', 0x01, 0x04); + rt305x_pci_phy(sc, 'w', 0x68, 0x84); + + rt305x_sysctl_set(SYSCTL_RSTCTRL, + rt305x_sysctl_get(SYSCTL_RSTCTRL) | (1<<26)); + rt305x_sysctl_set(SYSCTL_CLKCFG1, + rt305x_sysctl_get(SYSCTL_CLKCFG1) & ~(1<<26)); + + tmp = rt305x_sysctl_get(SYSCTL_PPLL_CFG1); + tmp &= ~(1<<19); + rt305x_sysctl_set(SYSCTL_PPLL_CFG1, tmp); + tmp |= (1<<31); + rt305x_sysctl_set(SYSCTL_PPLL_CFG1, tmp); +} + +static int +rt305x_pci_intr(void *arg) +{ + struct rt305x_pci_softc *sc = arg; + struct intr_event *event; + uint32_t reg, irq, irqidx; + + reg = RT_READ32(sc, RT305X_PCI_PCIINT); + + for (irqidx = 0; irqidx < RT305X_PCI_NIRQS; irqidx++) { + irq = rt305x_idx_to_irq(irqidx); + if (reg & (1<<irq)) { + event = sc->sc_eventstab[irqidx]; + if (!event || TAILQ_EMPTY(&event->ie_handlers)) { + if (irq != 0) + printf("Stray PCI IRQ %d\n", irq); + continue; + } + + intr_event_handle(event, NULL); + mips_intrcnt_inc(sc->sc_intr_counter[irqidx]); + } + } + + return (FILTER_HANDLED); +} diff --git a/sys/mips/rt305x/rt305x_pcireg.h b/sys/mips/rt305x/rt305x_pcireg.h new file mode 100644 index 0000000..cc6a62a --- /dev/null +++ b/sys/mips/rt305x/rt305x_pcireg.h @@ -0,0 +1,76 @@ +/*- + * Copyright (c) 2015 Stanislav Galabov. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ +#ifndef __RT305X_PCIREG_H__ +#define __RT305X_PCIREG_H__ + +#define RT305X_PCI_NIRQS 1 +#define RT305X_PCI_BASESLOT 0 + +#define RT305X_PCI_PCICFG 0x0000 +#define RT305X_PCI_PCIINT 0x0008 +#define RT305X_PCI_PCIENA 0x000C +#define RT305X_PCI_CFGADDR 0x0020 +#define RT305X_PCI_CFGDATA 0x0024 +#define RT305X_PCI_MEMBASE 0x0028 +#define RT305X_PCI_IOBASE 0x002C +#define RT305X_PCI_PHY0_CFG 0x0090 + +#define RT305X_PCI_PCIE0_BAR0SETUP 0x2010 +#define RT305X_PCI_PCIE0_BAR1SETUP 0x2014 +#define RT305X_PCI_PCIE0_IMBASEBAR0 0x2018 +#define RT305X_PCI_PCIE0_ID 0x2030 +#define RT305X_PCI_PCIE0_CLASS 0x2034 +#define RT305X_PCI_PCIE0_SUBID 0x2038 +#define RT305X_PCI_PCIE0_STATUS 0x2050 +#define RT305X_PCI_PCIE0_DLECR 0x2060 +#define RT305X_PCI_PCIE0_ECRC 0x2064 + +#define RT305X_PCIE0_IRQ 20 +#define RT305X_PCIE1_IRQ 21 +#define RT305X_PCIE2_IRQ 22 + +#define RT305X_PCI_INTR_PIN 2 + +#define PCI_MIN_IO_ALLOC 4 +#define PCI_MIN_MEM_ALLOC 16 +#define BITS_PER_UINT32 (NBBY * sizeof(uint32_t)) + +#define RT_WRITE32(sc, off, val) \ + bus_space_write_4((sc)->sc_bst, (sc)->sc_bsh, (off), (val)) +#define RT_WRITE16(sc, off, val) \ + bus_space_write_2((sc)->sc_bst, (sc)->sc_bsh, (off), (val)) +#define RT_WRITE8(sc, off, val) \ + bus_space_write_1((sc)->sc_bst, (sc)->sc_bsh, (off), (val)) +#define RT_READ32(sc, off) \ + bus_space_read_4((sc)->sc_bst, (sc)->sc_bsh, (off)) +#define RT_READ16(sc, off) \ + bus_space_read_2((sc)->sc_bst, (sc)->sc_bsh, (off)) +#define RT_READ8(sc, off) \ + bus_space_read_1((sc)->sc_bst, (sc)->sc_bsh, (off)) + +#endif /* __RT305X_PCIREG_H__ */ diff --git a/sys/mips/rt305x/rt305x_spi.c b/sys/mips/rt305x/rt305x_spi.c new file mode 100644 index 0000000..6dd2d74 --- /dev/null +++ b/sys/mips/rt305x/rt305x_spi.c @@ -0,0 +1,350 @@ +/*- + * Copyright (c) 2009, Oleksandr Tymoshenko <gonzo@FreeBSD.org> + * Copyright (c) 2011, Aleksandr Rybalko <ray@FreeBSD.org> + * Copyright (c) 2013, Alexander A. Mityaev <sansan@adm.ua> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice unmodified, this list of conditions, and the following + * disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/systm.h> +#include <sys/bus.h> + +#include <sys/kernel.h> +#include <sys/module.h> +#include <sys/rman.h> +#include <sys/lock.h> +#include <sys/mutex.h> + +#include <machine/bus.h> +#include <machine/cpu.h> +//#include <machine/pmap.h> + +#include <dev/spibus/spi.h> +#include <dev/spibus/spibusvar.h> +#include "spibus_if.h" + +#include "opt_platform.h" + +#ifdef FDT +#include <dev/ofw/openfirm.h> +#include <dev/ofw/ofw_bus.h> +#include <dev/ofw/ofw_bus_subr.h> +#endif + +#include <mips/rt305x/rt305xreg.h> +#include <dev/flash/mx25lreg.h> + +#undef RT305X_SPI_DEBUG +#ifdef RT305X_SPI_DEBUG +#define dprintf printf +#else +#define dprintf(x, arg...) +#endif + +/* + * register space access macros + */ +#define SPI_WRITE(sc, reg, val) do { \ + bus_write_4(sc->sc_mem_res, (reg), (val)); \ + } while (0) + +#define SPI_READ(sc, reg) bus_read_4(sc->sc_mem_res, (reg)) + +#define SPI_SET_BITS(sc, reg, bits) \ + SPI_WRITE(sc, reg, SPI_READ(sc, (reg)) | (bits)) + +#define SPI_CLEAR_BITS(sc, reg, bits) \ + SPI_WRITE(sc, reg, SPI_READ(sc, (reg)) & ~(bits)) + +struct rt305x_spi_softc { + device_t sc_dev; + struct resource *sc_mem_res; +}; + +static int rt305x_spi_probe(device_t); +static int rt305x_spi_attach(device_t); +static int rt305x_spi_detach(device_t); +static int rt305x_spi_wait(struct rt305x_spi_softc *); +static void rt305x_spi_chip_activate(struct rt305x_spi_softc *); +static void rt305x_spi_chip_deactivate(struct rt305x_spi_softc *); +static uint8_t rt305x_spi_txrx(struct rt305x_spi_softc *, uint8_t *, int); +static int rt305x_spi_transfer(device_t, device_t, struct spi_command *); +#ifdef FDT +static phandle_t rt305x_spi_get_node(device_t, device_t); +#endif + +static int +rt305x_spi_probe(device_t dev) +{ +#ifdef FDT + if (!ofw_bus_is_compatible(dev, "ralink,rt305x-spi")) + return(ENXIO); +#endif + device_set_desc(dev, "RT305X SPI"); + return (0); +} + +static int +rt305x_spi_attach(device_t dev) +{ + struct rt305x_spi_softc *sc = device_get_softc(dev); + int rid; + + sc->sc_dev = dev; + rid = 0; + sc->sc_mem_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, + RF_ACTIVE); + if (!sc->sc_mem_res) { + device_printf(dev, "Could not map memory\n"); + return (ENXIO); + } + + if (rt305x_spi_wait(sc)) { + bus_release_resource(dev, SYS_RES_MEMORY, 0, sc->sc_mem_res); + return (EBUSY); + } + + SPI_WRITE(sc, RT305X_SPICFG, MSBFIRST | SPICLKPOL | TX_ON_CLK_FALL | + SPI_CLK_DIV8); /* XXX: make it configurable */ + /* + * W25Q64CV max 104MHz, bus 120-192 MHz, so divide by 2. + * Update: divide by 4, DEV2 to fast for flash. + */ + + device_add_child(dev, "spibus", 0); + return (bus_generic_attach(dev)); +} + +static int +rt305x_spi_detach(device_t dev) +{ + struct rt305x_spi_softc *sc = device_get_softc(dev); + + SPI_SET_BITS(sc, RT305X_SPICTL, HIZSMOSI | CS_HIGH); + + if (sc->sc_mem_res) + bus_release_resource(dev, SYS_RES_MEMORY, 0, sc->sc_mem_res); + + return (0); +} + +static void +rt305x_spi_chip_activate(struct rt305x_spi_softc *sc) +{ +// printf("%s\n", __func__); + rt305x_spi_wait(sc); + /* + * Put all CSx to low + */ + SPI_CLEAR_BITS(sc, RT305X_SPICTL, CS_HIGH | HIZSMOSI); +} + +static void +rt305x_spi_chip_deactivate(struct rt305x_spi_softc *sc) +{ +// printf("%s\n", __func__); + rt305x_spi_wait(sc); + /* + * Put all CSx to high + */ + SPI_SET_BITS(sc, RT305X_SPICTL, CS_HIGH | HIZSMOSI); +} + +static int +rt305x_spi_wait(struct rt305x_spi_softc *sc) +{ + int i = 1000; + + while (i--) { + if (!SPI_READ(sc, RT305X_SPIBUSY)) + break; + } + if (i == 0) { + printf("busy\n"); + return (1); + } + + return (0); +} + +static uint8_t +rt305x_spi_txrx(struct rt305x_spi_softc *sc, uint8_t *data, int write) +{ + + if (rt305x_spi_wait(sc)) + return (EBUSY); + + if (write == RT305X_SPI_WRITE) { + SPI_WRITE(sc, RT305X_SPIDATA, *data); + SPI_SET_BITS(sc, RT305X_SPICTL, START_WRITE); + //printf("%s(W:%d)\n", __func__, *data); + } else {/* RT305X_SPI_READ */ + SPI_SET_BITS(sc, RT305X_SPICTL, START_READ); + if (rt305x_spi_wait(sc)) + return (EBUSY); + + *data = SPI_READ(sc, RT305X_SPIDATA) & 0xff; + //printf("%s(R:%d)\n", __func__, *data); + } + return (0); +} + +static int +rt305x_spi_transfer(device_t dev, device_t child, struct spi_command *cmd) +{ + struct rt305x_spi_softc *sc; + uint8_t *buf, byte, *tx_buf; + struct spibus_ivar *devi = SPIBUS_IVAR(child); + int i, sz, error = 0, write = 0; + + sc = device_get_softc(dev); + + if (devi->cs != 0) + /* Only 1 CS */ + return (ENXIO); + + /* There is always a command to transfer. */ + tx_buf = (uint8_t *)(cmd->tx_cmd); + + /* Perform some fixup because RT305X dont support duplex SPI */ + switch(tx_buf[0]) { + case CMD_READ_IDENT: + cmd->tx_cmd_sz = 1; + cmd->rx_cmd_sz = 3; + break; + case CMD_WRITE_ENABLE: + case CMD_WRITE_DISABLE: + cmd->tx_cmd_sz = 1; + cmd->rx_cmd_sz = 0; + break; + case CMD_READ_STATUS: + cmd->tx_cmd_sz = 1; + cmd->rx_cmd_sz = 1; + break; + case CMD_READ: + cmd->tx_cmd_sz = 4; + case CMD_FAST_READ: + cmd->tx_cmd_sz = 5; + cmd->rx_cmd_sz = cmd->tx_data_sz = 0; + break; + case CMD_SECTOR_ERASE: + cmd->tx_cmd_sz = 4; + cmd->rx_cmd_sz = cmd->tx_data_sz = 0; + break; + case CMD_PAGE_PROGRAM: + cmd->tx_cmd_sz = 4; + cmd->rx_cmd_sz = cmd->rx_data_sz = 0; + break; + } + + rt305x_spi_chip_activate(sc); + + if (cmd->tx_cmd_sz + cmd->rx_cmd_sz) { + buf = (uint8_t *)(cmd->rx_cmd); + tx_buf = (uint8_t *)(cmd->tx_cmd); + sz = cmd->tx_cmd_sz + cmd->rx_cmd_sz; + + for (i = 0; i < sz; i++) { + if(i < cmd->tx_cmd_sz) { + byte = tx_buf[i]; + error = rt305x_spi_txrx(sc, &byte, + RT305X_SPI_WRITE); + if (error) + goto rt305x_spi_transfer_fail; + continue; + } + error = rt305x_spi_txrx(sc, &byte, + RT305X_SPI_READ); + if (error) + goto rt305x_spi_transfer_fail; + buf[i] = byte; + } + } + + /* + * Transfer/Receive data + */ + + if (cmd->tx_data_sz + cmd->rx_data_sz) { + write = (cmd->tx_data_sz > 0)?1:0; + buf = (uint8_t *)(write ? cmd->tx_data : cmd->rx_data); + sz = write ? cmd->tx_data_sz : cmd->rx_data_sz; + + for (i = 0; i < sz; i++) { + byte = buf[i]; + error = rt305x_spi_txrx(sc, &byte, + write?RT305X_SPI_WRITE:RT305X_SPI_READ); + if (error) + goto rt305x_spi_transfer_fail; + buf[i] = byte; + } + } +rt305x_spi_transfer_fail: + rt305x_spi_chip_deactivate(sc); + + return (error); +} + +#ifdef FDT +static phandle_t +rt305x_spi_get_node(device_t bus, device_t dev) +{ + + /* We only have one child, the SPI bus, which needs our own node. */ + return (ofw_bus_get_node(bus)); +} +#endif + +static device_method_t rt305x_spi_methods[] = { + /* Device interface */ + DEVMETHOD(device_probe, rt305x_spi_probe), + DEVMETHOD(device_attach, rt305x_spi_attach), + DEVMETHOD(device_detach, rt305x_spi_detach), + + DEVMETHOD(spibus_transfer, rt305x_spi_transfer), + +#ifdef FDT + /* ofw_bus interface */ + DEVMETHOD(ofw_bus_get_node, rt305x_spi_get_node), +#endif + + DEVMETHOD_END +}; + +static driver_t rt305x_spi_driver = { + .name = "spi", + .methods = rt305x_spi_methods, + .size = sizeof(struct rt305x_spi_softc), +}; + +static devclass_t rt305x_spi_devclass; + +DRIVER_MODULE(rt305x_spi, obio, rt305x_spi_driver, rt305x_spi_devclass, 0, 0); +#ifdef FDT +DRIVER_MODULE(rt305x_spi, simplebus, rt305x_spi_driver, rt305x_spi_devclass, 0, 0); +#endif diff --git a/sys/mips/rt305x/rt305x_sysctl.c b/sys/mips/rt305x/rt305x_sysctl.c index 7c0d36d..077ccad 100644 --- a/sys/mips/rt305x/rt305x_sysctl.c +++ b/sys/mips/rt305x/rt305x_sysctl.c @@ -70,6 +70,7 @@ rt305x_sysctl_dump_config(device_t dev) (val >> 24) & 0xff); DUMPREG(SYSCTL_SYSCFG); +#if !defined(RT5350) && !defined(MT7620) if ( val & SYSCTL_SYSCFG_INIC_EE_SDRAM) printf("\tGet SDRAM config from EEPROM\n"); if ( val & SYSCTL_SYSCFG_INIC_8MB_SDRAM) @@ -123,6 +124,7 @@ rt305x_sysctl_dump_config(device_t dev) ((val & SYSCTL_CLKCFG1_PCM_CLK_DIV_MASK) >> SYSCTL_CLKCFG1_PCM_CLK_DIV_SHIFT)); DUMPREG(SYSCTL_GPIOMODE); +#endif #undef DUMPREG return; diff --git a/sys/mips/rt305x/rt305xreg.h b/sys/mips/rt305x/rt305xreg.h index 42a6c86..458e1c9 100644 --- a/sys/mips/rt305x/rt305xreg.h +++ b/sys/mips/rt305x/rt305xreg.h @@ -1,4 +1,5 @@ /*- + * Copyright (c) 2015 Stanislav Galabov. * Copyright (c) 2010 Aleksandr Rybalko. * All rights reserved. * @@ -29,12 +30,7 @@ #ifndef _RT305XREG_H_ #define _RT305XREG_H_ -/* XXX: must move to config */ -#define RT305X 1 -#define RT305XF 1 -#define RT3052F 1 -#define __U_BOOT__ 1 -/* XXX: must move to config */ +#include "opt_rt305x.h" #ifdef RT3052F #define PLATFORM_COUNTER_FREQ (384 * 1000 * 1000) @@ -42,12 +38,19 @@ #ifdef RT3050F #define PLATFORM_COUNTER_FREQ (320 * 1000 * 1000) #endif +#ifdef MT7620 +#define PLATFORM_COUNTER_FREQ (580 * 1000 * 1000) +#endif +#ifdef RT5350 +#define PLATFORM_COUNTER_FREQ (360 * 1000 * 1000) +#endif #ifndef PLATFORM_COUNTER_FREQ -#error "Nor RT3052F nor RT3050F defined" +#error "No platform selected" #endif -#define SYSTEM_CLOCK (PLATFORM_COUNTER_FREQ/3) +#ifndef MT7620 +#define SYSTEM_CLOCK (PLATFORM_COUNTER_FREQ/3) #define SDRAM_BASE 0x00000000 #define SDRAM_END 0x03FFFFFF @@ -60,16 +63,26 @@ #define INTCTL_END 0x100002FF #define MEMCTRL_BASE 0x10000300 #define MEMCTRL_END 0x100003FF /* SDRAM & Flash/SRAM */ +#ifndef RT5350 #define PCM_BASE 0x10000400 #define PCM_END 0x100004FF +#else +#define PCM_BASE 0x10002000 +#define PCM_END 0x100027FF +#endif #define UART_BASE 0x10000500 #define UART_END 0x100005FF #define PIO_BASE 0x10000600 #define PIO_END 0x100006FF +#ifndef RT5350 #define GDMA_BASE 0x10000700 #define GDMA_END 0x100007FF /* Generic DMA */ #define NANDFC_BASE 0x10000800 #define NANDFC_END 0x100008FF /* NAND Flash Controller */ +#else +#define GDMA_BASE 0x10002800 +#define GDMA_END 0x10002FFF +#endif #define I2C_BASE 0x10000900 #define I2C_END 0x100009FF #define I2S_BASE 0x10000A00 @@ -87,23 +100,110 @@ #define ROM_END 0x10119FFF #define WLAN_BASE 0x10180000 #define WLAN_END 0x101BFFFF /* 802.11n MAC/BBP */ +#ifndef RT5350 #define USB_OTG_BASE 0x101C0000 #define USB_OTG_END 0x101FFFFF +#else +#define USB_OTG_BASE 0x101C0000 +#define USB_OTG_END 0x101C0FFF +#define USB_OHCI_BASE 0x101C1000 +#define USB_OHCI_END 0x101C1FFF +#endif #define EMEM_BASE 0x1B000000 #define EMEM_END 0x1BFFFFFF /* External SRAM/Flash */ +#ifdef RT5350 +#define BOOT_ROM_BASE 0x1C000000 +#define BOOT_ROM_END 0x1C003FFF +#endif +#ifndef RT5350 #define FLASH_BASE 0x1F000000 #define FLASH_END 0x1FFFFFFF /* Flash window */ +#endif #define OBIO_MEM_BASE SYSCTL_BASE #define OBIO_MEM_START OBIO_MEM_BASE +#ifndef RT5350 #define OBIO_MEM_END FLASH_END +#else +#define OBIO_MEM_END BOOT_ROM_END +#endif + +#else /* MT7620 */ +#define SYSTEM_CLOCK (40 * 1000 * 1000) +#define SDRAM_BASE 0x00000000 +#define SDRAM_END 0x0FFFFFFF + +#define SYSCTL_BASE 0x10000000 +#define SYSCTL_END 0x100000FF +#define TIMER_BASE 0x10000100 +#define TIMER_END 0x100001FF +#define INTCTL_BASE 0x10000200 +#define INTCTL_END 0x100002FF +#define MEMCTRL_BASE 0x10000300 +#define MEMCTRL_END 0x100003FF /* SDRAM & Flash/SRAM */ +#define PCM_BASE 0x10002000 +#define PCM_END 0x100027FF +#define UART_BASE 0x10000500 +#define UART_END 0x100005FF +#define PIO_BASE 0x10000600 +#define PIO_END 0x100006FF +#define GDMA_BASE 0x10002800 +#define GDMA_END 0x10002FFF /* Generic DMA */ +#define NANDFC_BASE 0x10000800 +#define NANDFC_END 0x100008FF /* NAND Flash Controller */ +#define I2C_BASE 0x10000900 +#define I2C_END 0x100009FF +#define I2S_BASE 0x10000A00 +#define I2S_END 0x10000AFF +#define SPI_BASE 0x10000B00 +#define SPI_END 0x10000BFF +#define UARTLITE_BASE 0x10000C00 +#define UARTLITE_END 0x10000CFF + +#define FRENG_BASE 0x10100000 +#define FRENG_END 0x1010FFFF /* Frame Engine */ +#define ETHSW_BASE 0x10110000 +#define ETHSW_END 0x10117FFF /* Ethernet Switch */ +#define ROM_BASE 0x10118000 +#define ROM_END 0x1011FFFF +#define WLAN_BASE 0x10180000 +#define WLAN_END 0x101BFFFF /* 802.11n MAC/BBP */ +#define USB_OTG_BASE 0x101C0000 +#define USB_OTG_END 0x101C0FFF +#define USB_OHCI_BASE 0x101C1000 +#define USB_OHCI_END 0x101C1FFF +#define PCIE_BASE 0x10140000 +#define PCIE_END 0x1017FFFF +#define SDHC_BASE 0x10130000 +#define SDHC_END 0x10133FFF + +#define PCIE_IO_BASE 0x10160000 +#define PCIE_IO_END 0x1016FFFF +#define PCIE_MEM_BASE 0x20000000 +#define PCIE_MEM_END 0x2FFFFFFF + +// TODO: fix below mappings? +#define EMEM_BASE 0x1B000000 +#define EMEM_END 0x1BFFFFFF /* External SRAM/Flash */ +#define FLASH_BASE 0x1F000000 +#define FLASH_END 0x1FFFFFFF /* Flash window */ + +#define OBIO_MEM_BASE SYSCTL_BASE +#define OBIO_MEM_START OBIO_MEM_BASE +#define OBIO_MEM_END FLASH_END +#endif /* System Control */ -#define SYSCTL_CHIPID0_3 0x00 /* 'R''T''3''0' */ -#define SYSCTL_CHIPID4_7 0x04 /* '5''2'' '' ' */ +#define SYSCTL_CHIPID0_3 0x00 +#define SYSCTL_CHIPID4_7 0x04 +#ifdef RT5350 +#define SYSCTL_REVID 0x0C +#endif + #define SYSCTL_SYSCFG 0x10 +#if !defined(RT5350) && !defined(MT7620) #define SYSCTL_SYSCFG_INIC_EE_SDRAM (1<<29) #define SYSCTL_SYSCFG_INIC_8MB_SDRAM (1<<28) #define SYSCTL_SYSCFG_GE0_MODE_MASK 0x03000000 @@ -129,6 +229,18 @@ #define SYSCTL_SYSCFG_SRAM_CS_MODE_WDOG_RST 1 #define SYSCTL_SYSCFG_SRAM_CS_MODE_BT_COEX 2 #define SYSCTL_SYSCFG_SDRAM_CLK_DRV (1<<0) /* 8mA/12mA */ +#endif +#ifdef RT5350 +#define SYSCTL1_SYSCFG_PULL_EN (1<<26) +#define SYSCTL1_SYSCFG_SDR_PAD_DRV_MASK 0x0700000 +#define SYSCTL1_SYSCFG_SDR_PAD_DRV_SHIFT 20 +#define SYSCTL1_SYSCFG_SDR_PAD_DRV_0 0 +#define SYSCTL1_SYSCFG_SDR_PAD_DRV_1 1 +#define SYSCTL1_SYSCFG_SDR_PAD_DRV_2 2 +#endif + +#define SYSCTL_SYSCFG1 0x14 +#define SYSCTL_SYSCFG1_USB0_HOST_MODE (1 << 10) #define SYSCTL_TESTSTAT 0x18 #define SYSCTL_TESTSTAT2 0x1C @@ -142,7 +254,10 @@ #define SYSCTL_CLKCFG0_SDRAM_CLK_SKEW_3NS_DELAY 3 #define SYSCTL_CLKCFG1 0x30 +#if !defined(RT5350) #define SYSCTL_CLKCFG1_PBUS_DIV_CLK_BY2 (1<<30) +#define SYSCTL_CLKCFG1_UPHY0_CLK_EN (1<<25) +#define SYSCTL_CLKCFG1_UPHY1_CLK_EN (1<<22) #define SYSCTL_CLKCFG1_OTG_CLK_EN (1<<18) #define SYSCTL_CLKCFG1_I2S_CLK_EN (1<<15) #define SYSCTL_CLKCFG1_I2S_CLK_SEL_EXT (1<<14) @@ -152,10 +267,21 @@ #define SYSCTL_CLKCFG1_PCM_CLK_SEL_EXT (1<<6) #define SYSCTL_CLKCFG1_PCM_CLK_DIV_MASK 0x0000003f #define SYSCTL_CLKCFG1_PCM_CLK_DIV_SHIFT 0 +#endif +#ifdef RT5350 +#define SYSCTL_CLKCFG1_SYSTICK_EN (1<<29) +#define SYSCTL_CLKCFG1_PDMA_CSR_CLK_GATE_BYP (1<<23) +#define SYSCTL_CLKCFG1_UPHY0_CLK_EN (1<<18) +#endif #define SYSCTL_RSTCTRL 0x34 #define SYSCTL_RSTCTRL_ETHSW (1<<23) +#if !defined(MT7620) && !defined(RT5350) #define SYSCTL_RSTCTRL_OTG (1<<22) +#else +#define SYSCTL_RSTCTRL_UPHY0 (1<<25) +#define SYSCTL_RSTCTRL_UPHY1 (1<<22) +#endif #define SYSCTL_RSTCTRL_FRENG (1<<21) #define SYSCTL_RSTCTRL_WLAN (1<<20) #define SYSCTL_RSTCTRL_UARTL (1<<19) @@ -196,6 +322,9 @@ #define SYSCTL_MEMO0 0x68 #define SYSCTL_MEMO1 0x6C +#define SYSCTL_PPLL_CFG1 0x9C +#define SYSCTL_PPLL_DRV 0xA0 + /* Timer */ #define TIMER_TMRSTAT 0x00 #define TIMER_TMRSTAT_TMR1RST (1<<5) @@ -249,7 +378,10 @@ #define IC_OTG 18 #define IC_ETHSW 17 +#define IC_R2P 15 +#define IC_SDHC 14 #define IC_UARTLITE 12 +#define IC_SPI 11 #define IC_I2S 10 #define IC_PERFC 9 #define IC_NAND 8 @@ -362,7 +494,39 @@ */ #define GDMACT0_SWMODE (1<<0) - - +/* SPI controller interface */ + +#define RT305X_SPISTAT 0x00 +/* SPIBUSY is alias for SPIBUSY, because SPISTAT have only BUSY bit*/ +#define RT305X_SPIBUSY RT305X_SPISTAT + +#define RT305X_SPICFG 0x10 +#define MSBFIRST (1<<8) +#define SPICLKPOL (1<<6) +#define CAPT_ON_CLK_FALL (1<<5) +#define TX_ON_CLK_FALL (1<<4) +#define HIZSPI (1<<3) /* Set SPI pins to Tri-state */ +#define SPI_CLK_SHIFT 0 /* SPI clock divide control */ +#define SPI_CLK_MASK 0x00000007 +#define SPI_CLK_DIV2 0 +#define SPI_CLK_DIV4 1 +#define SPI_CLK_DIV8 2 +#define SPI_CLK_DIV16 3 +#define SPI_CLK_DIV32 4 +#define SPI_CLK_DIV64 5 +#define SPI_CLK_DIV128 6 +#define SPI_CLK_DISABLED 7 + +#define RT305X_SPICTL 0x14 +#define HIZSMOSI (1<<3) +#define START_WRITE (1<<2) +#define START_READ (1<<1) +#define CS_HIGH (1<<0) + +#define RT305X_SPIDATA 0x20 +#define SPIDATA_MASK 0x000000ff + +#define RT305X_SPI_WRITE 1 +#define RT305X_SPI_READ 0 #endif /* _RT305XREG_H_ */ diff --git a/sys/mips/rt305x/uart_bus_rt305x.c b/sys/mips/rt305x/uart_bus_rt305x.c index 9cd1df7..8b10080 100644 --- a/sys/mips/rt305x/uart_bus_rt305x.c +++ b/sys/mips/rt305x/uart_bus_rt305x.c @@ -75,21 +75,13 @@ static driver_t uart_rt305x_driver = { sizeof(struct uart_softc), }; -extern SLIST_HEAD(uart_devinfo_list, uart_devinfo) uart_sysdevs; - static int uart_rt305x_probe(device_t dev) { struct uart_softc *sc; sc = device_get_softc(dev); - sc->sc_sysdev = SLIST_FIRST(&uart_sysdevs); sc->sc_class = &uart_rt305x_uart_class; - bcopy(&sc->sc_sysdev->bas, &sc->sc_bas, sizeof(sc->sc_bas)); - sc->sc_sysdev->bas.regshft = 2; - sc->sc_sysdev->bas.bst = mips_bus_space_generic; - sc->sc_sysdev->bas.bsh = - MIPS_PHYS_TO_KSEG1(device_get_unit(dev)?UARTLITE_BASE:UART_BASE); sc->sc_bas.regshft = 2; sc->sc_bas.bst = mips_bus_space_generic; sc->sc_bas.bsh = diff --git a/sys/mips/rt305x/uart_cpu_rt305x.c b/sys/mips/rt305x/uart_cpu_rt305x.c index b9225a1..345db46 100644 --- a/sys/mips/rt305x/uart_cpu_rt305x.c +++ b/sys/mips/rt305x/uart_cpu_rt305x.c @@ -77,6 +77,10 @@ uart_cpu_getdev(int devtype, struct uart_devinfo *di) uart_bus_space_io = NULL; uart_bus_space_mem = mips_bus_space_generic; +#ifdef RT305X_USE_UART + di->bas.bsh = MIPS_PHYS_TO_KSEG1(UART_BASE); +#else di->bas.bsh = MIPS_PHYS_TO_KSEG1(UARTLITE_BASE); +#endif return (0); } diff --git a/sys/modules/Makefile b/sys/modules/Makefile index dc95750..1b3b7eb 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -220,6 +220,7 @@ SUBDIR= \ malo \ mcd \ md \ + mdio \ mem \ mfi \ mii \ diff --git a/sys/modules/crypto/Makefile b/sys/modules/crypto/Makefile index 4506cb3..b476571 100644 --- a/sys/modules/crypto/Makefile +++ b/sys/modules/crypto/Makefile @@ -16,7 +16,7 @@ SRCS += cast.c cryptodeflate.c rmd160.c rijndael-alg-fst.c rijndael-api.c rijnda SRCS += skipjack.c bf_enc.c bf_ecb.c bf_skey.c SRCS += camellia.c camellia-api.c SRCS += des_ecb.c des_enc.c des_setkey.c -SRCS += sha1.c sha2.c sha256c.c +SRCS += sha1.c sha256c.c sha512c.c SRCS += siphash.c SRCS += gmac.c gfmult.c SRCS += opt_param.h cryptodev_if.h bus_if.h device_if.h diff --git a/sys/modules/cxgbe/Makefile b/sys/modules/cxgbe/Makefile index a46850c..23a823f 100644 --- a/sys/modules/cxgbe/Makefile +++ b/sys/modules/cxgbe/Makefile @@ -11,9 +11,11 @@ SUBDIR+= t4_firmware SUBDIR+= t5_firmware SUBDIR+= ${_tom} SUBDIR+= ${_iw_cxgbe} +SUBDIR+= ${_cxgbei} .if ${MACHINE_CPUARCH} == "amd64" _tom= tom +_cxgbei= cxgbei .if ${MK_OFED} != "no" || defined(ALL_MODULES) _iw_cxgbe= iw_cxgbe .endif diff --git a/sys/modules/cxgbe/cxgbei/Makefile b/sys/modules/cxgbe/cxgbei/Makefile new file mode 100644 index 0000000..898cc3f --- /dev/null +++ b/sys/modules/cxgbe/cxgbei/Makefile @@ -0,0 +1,23 @@ +# $FreeBSD$ + +CXGBE = ${.CURDIR}/../../../dev/cxgbe +.PATH: ${CXGBE}/cxgbei + +KMOD= cxgbei + +SRCS= cxgbei.c +SRCS+= cxgbei_ulp2_ddp.c +SRCS+= icl_cxgbei.c +SRCS+= bus_if.h +SRCS+= device_if.h +SRCS+= opt_inet.h +SRCS+= opt_inet6.h +SRCS+= pci_if.h +SRCS+= opt_cam.h +SRCS+= icl_conn_if.h + +CFLAGS+= -I${CXGBE} + +MFILES= kern/bus_if.m kern/device_if.m dev/iscsi/icl_conn_if.m dev/pci/pci_if.m + +.include <bsd.kmod.mk> diff --git a/sys/modules/geom/geom_bde/Makefile b/sys/modules/geom/geom_bde/Makefile index 5c5a590..bcb4dbc 100644 --- a/sys/modules/geom/geom_bde/Makefile +++ b/sys/modules/geom/geom_bde/Makefile @@ -6,6 +6,6 @@ KMOD= geom_bde SRCS= g_bde.c g_bde_crypt.c g_bde_lock.c g_bde_work.c -SRCS+= rijndael-alg-fst.c rijndael-api-fst.c sha2.c sha256c.c +SRCS+= rijndael-alg-fst.c rijndael-api-fst.c sha256c.c sha512c.c .include <bsd.kmod.mk> diff --git a/sys/modules/ix/Makefile b/sys/modules/ix/Makefile index 1f30cb0..7984c1f 100644 --- a/sys/modules/ix/Makefile +++ b/sys/modules/ix/Makefile @@ -5,7 +5,7 @@ KMOD = if_ix SRCS = device_if.h bus_if.h pci_if.h SRCS += opt_inet.h opt_inet6.h opt_rss.h -SRCS += if_ix.c ix_txrx.c +SRCS += if_ix.c ix_txrx.c ixgbe_osdep.c # Shared source SRCS += ixgbe_common.c ixgbe_api.c ixgbe_phy.c ixgbe_mbx.c ixgbe_vf.c SRCS += ixgbe_dcb.c ixgbe_dcb_82598.c ixgbe_dcb_82599.c diff --git a/sys/modules/ixv/Makefile b/sys/modules/ixv/Makefile index f8ce347..3fcab90 100644 --- a/sys/modules/ixv/Makefile +++ b/sys/modules/ixv/Makefile @@ -5,7 +5,7 @@ KMOD = if_ixv SRCS = device_if.h bus_if.h pci_if.h SRCS += opt_inet.h opt_inet6.h opt_rss.h -SRCS += if_ixv.c ix_txrx.c +SRCS += if_ixv.c ix_txrx.c ixgbe_osdep.c # Shared source SRCS += ixgbe_common.c ixgbe_api.c ixgbe_phy.c ixgbe_mbx.c ixgbe_vf.c SRCS += ixgbe_dcb.c ixgbe_dcb_82598.c ixgbe_dcb_82599.c diff --git a/sys/modules/mdio/Makefile b/sys/modules/mdio/Makefile new file mode 100644 index 0000000..0d7ed59 --- /dev/null +++ b/sys/modules/mdio/Makefile @@ -0,0 +1,10 @@ +# $FreeBSD$ + +.PATH: ${.CURDIR}/../../dev/mdio + +KMOD= mdio +SRCS= mdio.c +SRCS+= mdio_if.c mdio_if.h +SRCS+= device_if.h bus_if.h + +.include <bsd.kmod.mk> diff --git a/sys/modules/tcp/fastpath/Makefile b/sys/modules/tcp/fastpath/Makefile index 526b39a..a6be6a6 100644 --- a/sys/modules/tcp/fastpath/Makefile +++ b/sys/modules/tcp/fastpath/Makefile @@ -7,6 +7,9 @@ KMOD= fastpath SRCS= fastpath.c +SRCS+= opt_ipfw.h opt_inet.h opt_inet6.h opt_ipsec.h opt_kdtrace.h +SRCS+= opt_tcpdebug.h + # # Enable full debugging # diff --git a/sys/modules/usb/rsufw/Makefile.inc b/sys/modules/usb/rsufw/Makefile.inc index 03779b9..3792b33 100644 --- a/sys/modules/usb/rsufw/Makefile.inc +++ b/sys/modules/usb/rsufw/Makefile.inc @@ -9,7 +9,7 @@ CLEANFILES+= ${_FIRM} FIRMWS= ${_FIRM}:${KMOD}:120 -FIRMWARE_LICENSE= realtek +# FIRMWARE_LICENSE= realtek ${_FIRM}: ${.CURDIR}/../../../../contrib/dev/rsu/${_FIRM}.uu uudecode -p $? > ${.TARGET} diff --git a/sys/modules/usb/urtwnfw/Makefile.inc b/sys/modules/usb/urtwnfw/Makefile.inc index fc03cf4..17c78be 100644 --- a/sys/modules/usb/urtwnfw/Makefile.inc +++ b/sys/modules/usb/urtwnfw/Makefile.inc @@ -9,7 +9,7 @@ CLEANFILES+= ${_FIRM} FIRMWS= ${_FIRM}:${KMOD}:111 -FIRMWARE_LICENSE= realtek +# FIRMWARE_LICENSE= realtek ${_FIRM}: ${.CURDIR}/../../../../contrib/dev/urtwn/${_FIRM}.uu uudecode -p $? > ${.TARGET} diff --git a/sys/modules/zfs/Makefile b/sys/modules/zfs/Makefile index ffbd9ba..a824422 100644 --- a/sys/modules/zfs/Makefile +++ b/sys/modules/zfs/Makefile @@ -68,7 +68,7 @@ SRCS+= zmod_subr.c SRCS+= zutil.c .PATH: ${SYSDIR}/crypto/sha2 -SRCS+= sha2.c sha256c.c +SRCS+= sha256c.c sha512c.c .PATH: ${SUNW}/common/zfs .include "${SUNW}/uts/common/Makefile.files" diff --git a/sys/net/bpf.c b/sys/net/bpf.c index 985208b..37edbe8 100644 --- a/sys/net/bpf.c +++ b/sys/net/bpf.c @@ -2551,7 +2551,7 @@ bpfattach2(struct ifnet *ifp, u_int dlt, u_int hdrlen, struct bpf_if **driverp) bp->bif_hdrlen = hdrlen; - if (bootverbose) + if (bootverbose && IS_DEFAULT_VNET(curvnet)) if_printf(ifp, "bpf attached\n"); } diff --git a/sys/net/if.c b/sys/net/if.c index b88c05e..39656ef 100644 --- a/sys/net/if.c +++ b/sys/net/if.c @@ -173,7 +173,7 @@ static int if_getgroup(struct ifgroupreq *, struct ifnet *); static int if_getgroupmembers(struct ifgroupreq *); static void if_delgroups(struct ifnet *); static void if_attach_internal(struct ifnet *, int, struct if_clone *); -static void if_detach_internal(struct ifnet *, int, struct if_clone **); +static int if_detach_internal(struct ifnet *, int, struct if_clone **); #ifdef INET6 /* @@ -884,7 +884,7 @@ if_detach(struct ifnet *ifp) CURVNET_RESTORE(); } -static void +static int if_detach_internal(struct ifnet *ifp, int vmove, struct if_clone **ifcp) { struct ifaddr *ifa; @@ -906,11 +906,19 @@ if_detach_internal(struct ifnet *ifp, int vmove, struct if_clone **ifcp) #endif IFNET_WUNLOCK(); if (!found) { + /* + * While we would want to panic here, we cannot + * guarantee that the interface is indeed still on + * the list given we don't hold locks all the way. + */ + return (ENOENT); +#if 0 if (vmove) panic("%s: ifp=%p not on the ifnet tailq %p", __func__, ifp, &V_ifnet); else return; /* XXX this should panic as well? */ +#endif } /* Check if this is a cloned interface or not. */ @@ -993,6 +1001,8 @@ if_detach_internal(struct ifnet *ifp, int vmove, struct if_clone **ifcp) (*dp->dom_ifdetach)(ifp, ifp->if_afdata[dp->dom_family]); } + + return (0); } #ifdef VIMAGE @@ -1007,12 +1017,16 @@ void if_vmove(struct ifnet *ifp, struct vnet *new_vnet) { struct if_clone *ifc; + int rc; /* * Detach from current vnet, but preserve LLADDR info, do not * mark as dead etc. so that the ifnet can be reattached later. + * If we cannot find it, we lost the race to someone else. */ - if_detach_internal(ifp, 1, &ifc); + rc = if_detach_internal(ifp, 1, &ifc); + if (rc != 0) + return; /* * Unlink the ifnet from ifindex_table[] in current vnet, and shrink diff --git a/sys/net/if_llatbl.c b/sys/net/if_llatbl.c index f2749e5..b0450b3 100644 --- a/sys/net/if_llatbl.c +++ b/sys/net/if_llatbl.c @@ -62,11 +62,10 @@ __FBSDID("$FreeBSD$"); MALLOC_DEFINE(M_LLTABLE, "lltable", "link level address tables"); -static VNET_DEFINE(SLIST_HEAD(, lltable), lltables); +static VNET_DEFINE(SLIST_HEAD(, lltable), lltables) = + SLIST_HEAD_INITIALIZER(lltables); #define V_lltables VNET(lltables) -static void vnet_lltable_init(void); - struct rwlock lltable_rwlock; RW_SYSINIT(lltable_rwlock, &lltable_rwlock, "lltable_rwlock"); @@ -740,15 +739,6 @@ lla_rt_output(struct rt_msghdr *rtm, struct rt_addrinfo *info) return (error); } -static void -vnet_lltable_init() -{ - - SLIST_INIT(&V_lltables); -} -VNET_SYSINIT(vnet_lltable_init, SI_SUB_PSEUDO, SI_ORDER_FIRST, - vnet_lltable_init, NULL); - #ifdef DDB struct llentry_sa { struct llentry base; diff --git a/sys/net/sff8436.h b/sys/net/sff8436.h index 97abe10..3399cce 100644 --- a/sys/net/sff8436.h +++ b/sys/net/sff8436.h @@ -131,7 +131,7 @@ enum { SFF_8436_CONNECTOR = 130, /* Connector type (Table 32) */ SFF_8436_TRANS_START = 131, /* Electric or Optical Compatibility * (Table 33) */ - SFF_8436_CODE_E1040G = 131, /* 10/40G Ethernet Compliance Code */ + SFF_8436_CODE_E1040100G = 131, /* 10/40/100G Ethernet Compliance Code */ SFF_8436_CODE_SONET = 132, /* SONET Compliance codes */ SFF_8436_CODE_SATA = 133, /* SAS/SATA compliance codes */ SFF_8436_CODE_E1G = 134, /* Gigabit Ethernet Compliant codes */ diff --git a/sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c b/sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c index d044ad9..57628b0 100644 --- a/sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c +++ b/sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c @@ -708,8 +708,15 @@ static int ng_btsocket_l2cap_process_l2ca_enc_change(struct ng_mesg *msg, ng_bts op = (ng_l2cap_l2ca_enc_chg_op *)(msg->data); + mtx_lock(&ng_btsocket_l2cap_sockets_mtx); + pcb = ng_btsocket_l2cap_pcb_by_cid(&rt->src, op->lcid, op->idtype); + if (pcb == NULL) { + mtx_unlock(&ng_btsocket_l2cap_sockets_mtx); + return (ENOENT); + } + mtx_lock(&pcb->pcb_mtx); pcb->encryption = op->result; @@ -729,6 +736,7 @@ static int ng_btsocket_l2cap_process_l2ca_enc_change(struct ng_mesg *msg, ng_bts } } mtx_unlock(&pcb->pcb_mtx); + mtx_unlock(&ng_btsocket_l2cap_sockets_mtx); return 0; } diff --git a/sys/netinet/if_ether.c b/sys/netinet/if_ether.c index 931883d..4c0c586b 100644 --- a/sys/netinet/if_ether.c +++ b/sys/netinet/if_ether.c @@ -599,7 +599,7 @@ arpintr(struct mbuf *m) layer = "firewire"; /* - * Restrict too long harware addresses. + * Restrict too long hardware addresses. * Currently we are capable of handling 20-byte * addresses ( sizeof(lle->ll_addr) ) */ @@ -608,8 +608,8 @@ arpintr(struct mbuf *m) break; default: ARP_LOG(LOG_NOTICE, - "packet with unknown harware format 0x%02d received on %s\n", - ntohs(ar->ar_hrd), if_name(ifp)); + "packet with unknown hardware format 0x%02d received on " + "%s\n", ntohs(ar->ar_hrd), if_name(ifp)); m_freem(m); return; } diff --git a/sys/netinet/in_rmx.c b/sys/netinet/in_rmx.c index bb98de2..283c450 100644 --- a/sys/netinet/in_rmx.c +++ b/sys/netinet/in_rmx.c @@ -133,7 +133,7 @@ int in_detachhead(void **head, int off) { - return (1); + return (rn_detachhead(head)); } #endif diff --git a/sys/netinet/sctp_indata.c b/sys/netinet/sctp_indata.c index b0d34f2..75c1ac5 100644 --- a/sys/netinet/sctp_indata.c +++ b/sys/netinet/sctp_indata.c @@ -3994,6 +3994,7 @@ again: op_err = sctp_generate_cause(SCTP_CAUSE_USER_INITIATED_ABT, ""); stcb->sctp_ep->last_abort_code = SCTP_FROM_SCTP_INDATA + SCTP_LOC_26; sctp_abort_an_association(stcb->sctp_ep, stcb, op_err, SCTP_SO_NOT_LOCKED); + return; } else { struct sctp_nets *netp; diff --git a/sys/netinet/sctp_os_bsd.h b/sys/netinet/sctp_os_bsd.h index 0d99f54..777e0a7 100644 --- a/sys/netinet/sctp_os_bsd.h +++ b/sys/netinet/sctp_os_bsd.h @@ -104,7 +104,7 @@ __FBSDID("$FreeBSD$"); #include <netinet/ip_options.h> #include <crypto/sha1.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> #ifndef in6pcb #define in6pcb inpcb diff --git a/sys/netinet/sctp_output.c b/sys/netinet/sctp_output.c index c131edf..95ac97c 100644 --- a/sys/netinet/sctp_output.c +++ b/sys/netinet/sctp_output.c @@ -7219,7 +7219,7 @@ one_more_time: } /* Whack down the size */ atomic_subtract_int(&stcb->asoc.total_output_queue_size, sp->length); - if ((stcb->sctp_socket != NULL) && \ + if ((stcb->sctp_socket != NULL) && ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || (stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL))) { atomic_subtract_int(&stcb->sctp_socket->so_snd.sb_cc, sp->length); @@ -7238,9 +7238,6 @@ one_more_time: } } some_taken = sp->some_taken; - if (stcb->asoc.state & SCTP_STATE_CLOSED_SOCKET) { - sp->msg_is_complete = 1; - } re_look: length = sp->length; if (sp->msg_is_complete) { diff --git a/sys/netinet/tcp.h b/sys/netinet/tcp.h index 294e994..bfc8073 100644 --- a/sys/netinet/tcp.h +++ b/sys/netinet/tcp.h @@ -97,6 +97,10 @@ struct tcphdr { #define TCPOLEN_TSTAMP_APPA (TCPOLEN_TIMESTAMP+2) /* appendix A */ #define TCPOPT_SIGNATURE 19 /* Keyed MD5: RFC 2385 */ #define TCPOLEN_SIGNATURE 18 +#define TCPOPT_FAST_OPEN 34 +#define TCPOLEN_FAST_OPEN_EMPTY 2 +#define TCPOLEN_FAST_OPEN_MIN 6 +#define TCPOLEN_FAST_OPEN_MAX 18 /* Miscellaneous constants */ #define MAX_SACK_BLKS 6 /* Max # SACK blocks stored at receiver side */ @@ -165,6 +169,7 @@ struct tcphdr { #define TCP_KEEPIDLE 256 /* L,N,X start keeplives after this period */ #define TCP_KEEPINTVL 512 /* L,N interval between keepalives */ #define TCP_KEEPCNT 1024 /* L,N number of keepalives before close */ +#define TCP_FASTOPEN 1025 /* enable TFO / was created via TFO */ #define TCP_PCAP_OUT 2048 /* number of output packets to keep */ #define TCP_PCAP_IN 4096 /* number of input packets to keep */ #define TCP_FUNCTION_BLK 8192 /* Set the tcp function pointers to the specified stack */ diff --git a/sys/netinet/tcp_fastopen.c b/sys/netinet/tcp_fastopen.c new file mode 100644 index 0000000..482320e --- /dev/null +++ b/sys/netinet/tcp_fastopen.c @@ -0,0 +1,442 @@ +/*- + * Copyright (c) 2015 Patrick Kelsey + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* + * This is a server-side implementation of TCP Fast Open (TFO) [RFC7413]. + * + * This implementation is currently considered to be experimental and is not + * included in kernel builds by default. To include this code, add the + * following line to your kernel config: + * + * options TCP_RFC7413 + * + * The generated TFO cookies are the 64-bit output of + * SipHash24(<16-byte-key><client-ip>). Multiple concurrent valid keys are + * supported so that time-based rolling cookie invalidation policies can be + * implemented in the system. The default number of concurrent keys is 2. + * This can be adjusted in the kernel config as follows: + * + * options TCP_RFC7413_MAX_KEYS=<num-keys> + * + * + * The following TFO-specific sysctls are defined: + * + * net.inet.tcp.fastopen.acceptany (RW, default 0) + * When non-zero, all client-supplied TFO cookies will be considered to + * be valid. + * + * net.inet.tcp.fastopen.autokey (RW, default 120) + * When this and net.inet.tcp.fastopen.enabled are non-zero, a new key + * will be automatically generated after this many seconds. + * + * net.inet.tcp.fastopen.enabled (RW, default 0) + * When zero, no new TFO connections can be created. On the transition + * from enabled to disabled, all installed keys are removed. On the + * transition from disabled to enabled, if net.inet.tcp.fastopen.autokey + * is non-zero and there are no keys installed, a new key will be + * generated immediately. The transition from enabled to disabled does + * not affect any TFO connections in progress; it only prevents new ones + * from being made. + * + * net.inet.tcp.fastopen.keylen (RO) + * The key length in bytes. + * + * net.inet.tcp.fastopen.maxkeys (RO) + * The maximum number of keys supported. + * + * net.inet.tcp.fastopen.numkeys (RO) + * The current number of keys installed. + * + * net.inet.tcp.fastopen.setkey (WO) + * Install a new key by writing net.inet.tcp.fastopen.keylen bytes to this + * sysctl. + * + * + * In order for TFO connections to be created via a listen socket, that + * socket must have the TCP_FASTOPEN socket option set on it. This option + * can be set on the socket either before or after the listen() is invoked. + * Clearing this option on a listen socket after it has been set has no + * effect on existing TFO connections or TFO connections in progress; it + * only prevents new TFO connections from being made. + * + * For passively-created sockets, the TCP_FASTOPEN socket option can be + * queried to determine whether the connection was established using TFO. + * Note that connections that are established via a TFO SYN, but that fall + * back to using a non-TFO SYN|ACK will have the TCP_FASTOPEN socket option + * set. + * + * Per the RFC, this implementation limits the number of TFO connections + * that can be in the SYN_RECEIVED state on a per listen-socket basis. + * Whenever this limit is exceeded, requests for new TFO connections are + * serviced as non-TFO requests. Without such a limit, given a valid TFO + * cookie, an attacker could keep the listen queue in an overflow condition + * using a TFO SYN flood. This implementation sets the limit at half the + * configured listen backlog. + * + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include "opt_inet.h" + +#include <sys/param.h> +#include <sys/kernel.h> +#include <sys/limits.h> +#include <sys/lock.h> +#include <sys/rmlock.h> +#include <sys/socketvar.h> +#include <sys/sysctl.h> +#include <sys/systm.h> + +#include <crypto/siphash/siphash.h> + +#include <net/vnet.h> + +#include <netinet/in.h> +#include <netinet/in_pcb.h> +#include <netinet/tcp_fastopen.h> +#include <netinet/tcp_var.h> + + +#define TCP_FASTOPEN_KEY_LEN SIPHASH_KEY_LENGTH + +#if !defined(TCP_RFC7413_MAX_KEYS) || (TCP_RFC7413_MAX_KEYS < 1) +#define TCP_FASTOPEN_MAX_KEYS 2 +#else +#define TCP_FASTOPEN_MAX_KEYS TCP_RFC7413_MAX_KEYS +#endif + +struct tcp_fastopen_keylist { + unsigned int newest; + uint8_t key[TCP_FASTOPEN_MAX_KEYS][TCP_FASTOPEN_KEY_LEN]; +}; + +struct tcp_fastopen_callout { + struct callout c; + struct vnet *v; +}; + +SYSCTL_NODE(_net_inet_tcp, OID_AUTO, fastopen, CTLFLAG_RW, 0, "TCP Fast Open"); + +static VNET_DEFINE(int, tcp_fastopen_acceptany) = 0; +#define V_tcp_fastopen_acceptany VNET(tcp_fastopen_acceptany) +SYSCTL_INT(_net_inet_tcp_fastopen, OID_AUTO, acceptany, + CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(tcp_fastopen_acceptany), 0, + "Accept any non-empty cookie"); + +static VNET_DEFINE(unsigned int, tcp_fastopen_autokey) = 120; +#define V_tcp_fastopen_autokey VNET(tcp_fastopen_autokey) +static int sysctl_net_inet_tcp_fastopen_autokey(SYSCTL_HANDLER_ARGS); +SYSCTL_PROC(_net_inet_tcp_fastopen, OID_AUTO, autokey, + CTLFLAG_VNET | CTLTYPE_UINT | CTLFLAG_RW, NULL, 0, + &sysctl_net_inet_tcp_fastopen_autokey, "IU", + "Number of seconds between auto-generation of a new key; zero disables"); + +VNET_DEFINE(unsigned int, tcp_fastopen_enabled) = 0; +static int sysctl_net_inet_tcp_fastopen_enabled(SYSCTL_HANDLER_ARGS); +SYSCTL_PROC(_net_inet_tcp_fastopen, OID_AUTO, enabled, + CTLFLAG_VNET | CTLTYPE_UINT | CTLFLAG_RW, NULL, 0, + &sysctl_net_inet_tcp_fastopen_enabled, "IU", + "Enable/disable TCP Fast Open processing"); + +SYSCTL_INT(_net_inet_tcp_fastopen, OID_AUTO, keylen, + CTLFLAG_RD, SYSCTL_NULL_INT_PTR, TCP_FASTOPEN_KEY_LEN, + "Key length in bytes"); + +SYSCTL_INT(_net_inet_tcp_fastopen, OID_AUTO, maxkeys, + CTLFLAG_RD, SYSCTL_NULL_INT_PTR, TCP_FASTOPEN_MAX_KEYS, + "Maximum number of keys supported"); + +static VNET_DEFINE(unsigned int, tcp_fastopen_numkeys) = 0; +#define V_tcp_fastopen_numkeys VNET(tcp_fastopen_numkeys) +SYSCTL_UINT(_net_inet_tcp_fastopen, OID_AUTO, numkeys, + CTLFLAG_VNET | CTLFLAG_RD, &VNET_NAME(tcp_fastopen_numkeys), 0, + "Number of keys installed"); + +static int sysctl_net_inet_tcp_fastopen_setkey(SYSCTL_HANDLER_ARGS); +SYSCTL_PROC(_net_inet_tcp_fastopen, OID_AUTO, setkey, + CTLFLAG_VNET | CTLTYPE_OPAQUE | CTLFLAG_WR, NULL, 0, + &sysctl_net_inet_tcp_fastopen_setkey, "", + "Install a new key"); + +static VNET_DEFINE(struct rmlock, tcp_fastopen_keylock); +#define V_tcp_fastopen_keylock VNET(tcp_fastopen_keylock) + +#define TCP_FASTOPEN_KEYS_RLOCK(t) rm_rlock(&V_tcp_fastopen_keylock, (t)) +#define TCP_FASTOPEN_KEYS_RUNLOCK(t) rm_runlock(&V_tcp_fastopen_keylock, (t)) +#define TCP_FASTOPEN_KEYS_WLOCK() rm_wlock(&V_tcp_fastopen_keylock) +#define TCP_FASTOPEN_KEYS_WUNLOCK() rm_wunlock(&V_tcp_fastopen_keylock) + +static VNET_DEFINE(struct tcp_fastopen_keylist, tcp_fastopen_keys); +#define V_tcp_fastopen_keys VNET(tcp_fastopen_keys) + +static VNET_DEFINE(struct tcp_fastopen_callout, tcp_fastopen_autokey_ctx); +#define V_tcp_fastopen_autokey_ctx VNET(tcp_fastopen_autokey_ctx) + +static VNET_DEFINE(uma_zone_t, counter_zone); +#define V_counter_zone VNET(counter_zone) + +void +tcp_fastopen_init(void) +{ + V_counter_zone = uma_zcreate("tfo", sizeof(unsigned int), + NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE); + rm_init(&V_tcp_fastopen_keylock, "tfo_keylock"); + callout_init_rm(&V_tcp_fastopen_autokey_ctx.c, + &V_tcp_fastopen_keylock, 0); + V_tcp_fastopen_keys.newest = TCP_FASTOPEN_MAX_KEYS - 1; +} + +void +tcp_fastopen_destroy(void) +{ + callout_drain(&V_tcp_fastopen_autokey_ctx.c); + rm_destroy(&V_tcp_fastopen_keylock); + uma_zdestroy(V_counter_zone); +} + +unsigned int * +tcp_fastopen_alloc_counter(void) +{ + unsigned int *counter; + counter = uma_zalloc(V_counter_zone, M_NOWAIT); + if (counter) + *counter = 1; + return (counter); +} + +void +tcp_fastopen_decrement_counter(unsigned int *counter) +{ + if (*counter == 1) + uma_zfree(V_counter_zone, counter); + else + atomic_subtract_int(counter, 1); +} + +static void +tcp_fastopen_addkey_locked(uint8_t *key) +{ + + V_tcp_fastopen_keys.newest++; + if (V_tcp_fastopen_keys.newest == TCP_FASTOPEN_MAX_KEYS) + V_tcp_fastopen_keys.newest = 0; + memcpy(V_tcp_fastopen_keys.key[V_tcp_fastopen_keys.newest], key, + TCP_FASTOPEN_KEY_LEN); + if (V_tcp_fastopen_numkeys < TCP_FASTOPEN_MAX_KEYS) + V_tcp_fastopen_numkeys++; +} + +static void +tcp_fastopen_autokey_locked(void) +{ + uint8_t newkey[TCP_FASTOPEN_KEY_LEN]; + + arc4rand(newkey, TCP_FASTOPEN_KEY_LEN, 0); + tcp_fastopen_addkey_locked(newkey); +} + +static void +tcp_fastopen_autokey_callout(void *arg) +{ + struct tcp_fastopen_callout *ctx = arg; + + CURVNET_SET(ctx->v); + tcp_fastopen_autokey_locked(); + callout_reset(&ctx->c, V_tcp_fastopen_autokey * hz, + tcp_fastopen_autokey_callout, ctx); + CURVNET_RESTORE(); +} + + +static uint64_t +tcp_fastopen_make_cookie(uint8_t key[SIPHASH_KEY_LENGTH], struct in_conninfo *inc) +{ + SIPHASH_CTX ctx; + uint64_t siphash; + + SipHash24_Init(&ctx); + SipHash_SetKey(&ctx, key); + switch (inc->inc_flags & INC_ISIPV6) { +#ifdef INET + case 0: + SipHash_Update(&ctx, &inc->inc_faddr, sizeof(inc->inc_faddr)); + break; +#endif +#ifdef INET6 + case INC_ISIPV6: + SipHash_Update(&ctx, &inc->inc6_faddr, sizeof(inc->inc6_faddr)); + break; +#endif + } + SipHash_Final((u_int8_t *)&siphash, &ctx); + + return (siphash); +} + + +/* + * Return values: + * -1 the cookie is invalid and no valid cookie is available + * 0 the cookie is invalid and the latest cookie has been returned + * 1 the cookie is valid and the latest cookie has been returned + */ +int +tcp_fastopen_check_cookie(struct in_conninfo *inc, uint8_t *cookie, + unsigned int len, uint64_t *latest_cookie) +{ + struct rm_priotracker tracker; + unsigned int i, key_index; + uint64_t cur_cookie; + + if (V_tcp_fastopen_acceptany) { + *latest_cookie = 0; + return (1); + } + + if (len != TCP_FASTOPEN_COOKIE_LEN) { + if (V_tcp_fastopen_numkeys > 0) { + *latest_cookie = + tcp_fastopen_make_cookie( + V_tcp_fastopen_keys.key[V_tcp_fastopen_keys.newest], + inc); + return (0); + } + return (-1); + } + + /* + * Check against each available key, from newest to oldest. + */ + TCP_FASTOPEN_KEYS_RLOCK(&tracker); + key_index = V_tcp_fastopen_keys.newest; + for (i = 0; i < V_tcp_fastopen_numkeys; i++) { + cur_cookie = + tcp_fastopen_make_cookie(V_tcp_fastopen_keys.key[key_index], + inc); + if (i == 0) + *latest_cookie = cur_cookie; + if (memcmp(cookie, &cur_cookie, TCP_FASTOPEN_COOKIE_LEN) == 0) { + TCP_FASTOPEN_KEYS_RUNLOCK(&tracker); + return (1); + } + if (key_index == 0) + key_index = TCP_FASTOPEN_MAX_KEYS - 1; + else + key_index--; + } + TCP_FASTOPEN_KEYS_RUNLOCK(&tracker); + + return (0); +} + +static int +sysctl_net_inet_tcp_fastopen_autokey(SYSCTL_HANDLER_ARGS) +{ + int error; + unsigned int new; + + new = V_tcp_fastopen_autokey; + error = sysctl_handle_int(oidp, &new, 0, req); + if (error == 0 && req->newptr) { + if (new > (INT_MAX / hz)) + return (EINVAL); + + TCP_FASTOPEN_KEYS_WLOCK(); + if (V_tcp_fastopen_enabled) { + if (V_tcp_fastopen_autokey && !new) + callout_stop(&V_tcp_fastopen_autokey_ctx.c); + else if (new) + callout_reset(&V_tcp_fastopen_autokey_ctx.c, + new * hz, tcp_fastopen_autokey_callout, + &V_tcp_fastopen_autokey_ctx); + } + V_tcp_fastopen_autokey = new; + TCP_FASTOPEN_KEYS_WUNLOCK(); + } + + return (error); +} + +static int +sysctl_net_inet_tcp_fastopen_enabled(SYSCTL_HANDLER_ARGS) +{ + int error; + unsigned int new; + + new = V_tcp_fastopen_enabled; + error = sysctl_handle_int(oidp, &new, 0, req); + if (error == 0 && req->newptr) { + if (V_tcp_fastopen_enabled && !new) { + /* enabled -> disabled */ + TCP_FASTOPEN_KEYS_WLOCK(); + V_tcp_fastopen_numkeys = 0; + V_tcp_fastopen_keys.newest = TCP_FASTOPEN_MAX_KEYS - 1; + if (V_tcp_fastopen_autokey) + callout_stop(&V_tcp_fastopen_autokey_ctx.c); + V_tcp_fastopen_enabled = 0; + TCP_FASTOPEN_KEYS_WUNLOCK(); + } else if (!V_tcp_fastopen_enabled && new) { + /* disabled -> enabled */ + TCP_FASTOPEN_KEYS_WLOCK(); + if (V_tcp_fastopen_autokey && + (V_tcp_fastopen_numkeys == 0)) { + tcp_fastopen_autokey_locked(); + callout_reset(&V_tcp_fastopen_autokey_ctx.c, + V_tcp_fastopen_autokey * hz, + tcp_fastopen_autokey_callout, + &V_tcp_fastopen_autokey_ctx); + } + V_tcp_fastopen_enabled = 1; + TCP_FASTOPEN_KEYS_WUNLOCK(); + } + } + return (error); +} + +static int +sysctl_net_inet_tcp_fastopen_setkey(SYSCTL_HANDLER_ARGS) +{ + int error; + uint8_t newkey[TCP_FASTOPEN_KEY_LEN]; + + if (req->oldptr != NULL || req->oldlen != 0) + return (EINVAL); + if (req->newptr == NULL) + return (EPERM); + if (req->newlen != sizeof(newkey)) + return (EINVAL); + error = SYSCTL_IN(req, newkey, sizeof(newkey)); + if (error) + return (error); + + TCP_FASTOPEN_KEYS_WLOCK(); + tcp_fastopen_addkey_locked(newkey); + TCP_FASTOPEN_KEYS_WUNLOCK(); + + return (0); +} diff --git a/sys/netinet/tcp_fastopen.h b/sys/netinet/tcp_fastopen.h new file mode 100644 index 0000000..c64ba2c --- /dev/null +++ b/sys/netinet/tcp_fastopen.h @@ -0,0 +1,47 @@ +/*- + * Copyright (c) 2015 Patrick Kelsey + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _TCP_FASTOPEN_H_ +#define _TCP_FASTOPEN_H_ + +#ifdef _KERNEL + +#define TCP_FASTOPEN_COOKIE_LEN 8 /* tied to SipHash24 64-bit output */ + +VNET_DECLARE(unsigned int, tcp_fastopen_enabled); +#define V_tcp_fastopen_enabled VNET(tcp_fastopen_enabled) + +void tcp_fastopen_init(void); +void tcp_fastopen_destroy(void); +unsigned int *tcp_fastopen_alloc_counter(void); +void tcp_fastopen_decrement_counter(unsigned int *counter); +int tcp_fastopen_check_cookie(struct in_conninfo *inc, uint8_t *cookie, + unsigned int len, uint64_t *latest_cookie); +#endif /* _KERNEL */ + +#endif /* _TCP_FASTOPEN_H_ */ diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c index 140cee7..a763e46 100644 --- a/sys/netinet/tcp_input.c +++ b/sys/netinet/tcp_input.c @@ -98,6 +98,9 @@ __FBSDID("$FreeBSD$"); #include <netinet6/in6_pcb.h> #include <netinet6/ip6_var.h> #include <netinet6/nd6.h> +#ifdef TCP_RFC7413 +#include <netinet/tcp_fastopen.h> +#endif #include <netinet/tcp_fsm.h> #include <netinet/tcp_seq.h> #include <netinet/tcp_timer.h> @@ -916,9 +919,10 @@ findpcb: */ if (inp->inp_ip_minttl != 0) { #ifdef INET6 - if (isipv6 && inp->inp_ip_minttl > ip6->ip6_hlim) - goto dropunlock; - else + if (isipv6) { + if (inp->inp_ip_minttl > ip6->ip6_hlim) + goto dropunlock; + } else #endif if (inp->inp_ip_minttl > ip->ip_ttl) goto dropunlock; @@ -999,7 +1003,8 @@ relocked: INP_INFO_RLOCK_ASSERT(&V_tcbinfo); #endif if (!((tp->t_state == TCPS_ESTABLISHED && (thflags & TH_SYN) == 0) || - (tp->t_state == TCPS_LISTEN && (thflags & TH_SYN)))) { + (tp->t_state == TCPS_LISTEN && (thflags & TH_SYN) && + !(tp->t_flags & TF_FASTOPEN)))) { if (ti_locked == TI_UNLOCKED) { if (INP_INFO_TRY_RLOCK(&V_tcbinfo) == 0) { in_pcbref(inp); @@ -1091,6 +1096,9 @@ relocked: rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } +#ifdef TCP_RFC7413 +new_tfo_socket: +#endif if (so == NULL) { /* * We completed the 3-way handshake @@ -1353,7 +1361,12 @@ relocked: #endif TCP_PROBE3(debug__input, tp, th, mtod(m, const char *)); tcp_dooptions(&to, optp, optlen, TO_SYN); +#ifdef TCP_RFC7413 + if (syncache_add(&inc, &to, th, inp, &so, m, NULL, NULL)) + goto new_tfo_socket; +#else syncache_add(&inc, &to, th, inp, &so, m, NULL, NULL); +#endif /* * Entry added to syncache and mbuf consumed. * Only the listen socket is unlocked by syncache_add(). @@ -1468,7 +1481,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, struct in_conninfo *inc; struct mbuf *mfree; struct tcpopt to; - + int tfo_syn; + #ifdef TCPDEBUG /* * The size of tcp_saveipgen must be the size of the max ip header, @@ -1921,6 +1935,28 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, rstreason = BANDLIM_RST_OPENPORT; goto dropwithreset; } +#ifdef TCP_RFC7413 + if (tp->t_flags & TF_FASTOPEN) { + /* + * When a TFO connection is in SYN_RECEIVED, the + * only valid packets are the initial SYN, a + * retransmit/copy of the initial SYN (possibly with + * a subset of the original data), a valid ACK, a + * FIN, or a RST. + */ + if ((thflags & (TH_SYN|TH_ACK)) == (TH_SYN|TH_ACK)) { + rstreason = BANDLIM_RST_OPENPORT; + goto dropwithreset; + } else if (thflags & TH_SYN) { + /* non-initial SYN is ignored */ + if ((tcp_timer_active(tp, TT_DELACK) || + tcp_timer_active(tp, TT_REXMT))) + goto drop; + } else if (!(thflags & (TH_ACK|TH_FIN|TH_RST))) { + goto drop; + } + } +#endif break; /* @@ -2136,7 +2172,8 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, * RFC5961 Section 4.2 * Send challenge ACK for any SYN in synchronized state. */ - if ((thflags & TH_SYN) && tp->t_state != TCPS_SYN_SENT) { + if ((thflags & TH_SYN) && tp->t_state != TCPS_SYN_SENT && + tp->t_state != TCPS_SYN_RECEIVED) { KASSERT(ti_locked == TI_RLOCKED, ("tcp_do_segment: TH_SYN ti_locked %d", ti_locked)); INP_INFO_RLOCK_ASSERT(&V_tcbinfo); @@ -2330,9 +2367,16 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, */ if ((thflags & TH_ACK) == 0) { if (tp->t_state == TCPS_SYN_RECEIVED || - (tp->t_flags & TF_NEEDSYN)) + (tp->t_flags & TF_NEEDSYN)) { +#ifdef TCP_RFC7413 + if (tp->t_state == TCPS_SYN_RECEIVED && + tp->t_flags & TF_FASTOPEN) { + tp->snd_wnd = tiwin; + cc_conn_init(tp); + } +#endif goto step6; - else if (tp->t_flags & TF_ACKNOW) + } else if (tp->t_flags & TF_ACKNOW) goto dropafterack; else goto drop; @@ -2371,7 +2415,27 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, struct socket *so, tcp_state_change(tp, TCPS_ESTABLISHED); TCP_PROBE5(accept__established, NULL, tp, mtod(m, const char *), tp, th); - cc_conn_init(tp); +#ifdef TCP_RFC7413 + if (tp->t_tfo_pending) { + tcp_fastopen_decrement_counter(tp->t_tfo_pending); + tp->t_tfo_pending = NULL; + + /* + * Account for the ACK of our SYN prior to + * regular ACK processing below. + */ + tp->snd_una++; + } + /* + * TFO connections call cc_conn_init() during SYN + * processing. Calling it again here for such + * connections is not harmless as it would undo the + * snd_cwnd reduction that occurs when a TFO SYN|ACK + * is retransmitted. + */ + if (!(tp->t_flags & TF_FASTOPEN)) +#endif + cc_conn_init(tp); tcp_timer_activate(tp, TT_KEEP, TP_KEEPIDLE(tp)); } /* @@ -2919,7 +2983,9 @@ dodata: /* XXX */ * case PRU_RCVD). If a FIN has already been received on this * connection then we just ignore the text. */ - if ((tlen || (thflags & TH_FIN)) && + tfo_syn = ((tp->t_state == TCPS_SYN_RECEIVED) && + (tp->t_flags & TF_FASTOPEN)); + if ((tlen || (thflags & TH_FIN) || tfo_syn) && TCPS_HAVERCVDFIN(tp->t_state) == 0) { tcp_seq save_start = th->th_seq; m_adj(m, drop_hdrlen); /* delayed header drop */ @@ -2937,8 +3003,9 @@ dodata: /* XXX */ */ if (th->th_seq == tp->rcv_nxt && LIST_EMPTY(&tp->t_segq) && - TCPS_HAVEESTABLISHED(tp->t_state)) { - if (DELAY_ACK(tp, tlen)) + (TCPS_HAVEESTABLISHED(tp->t_state) || + tfo_syn)) { + if (DELAY_ACK(tp, tlen) || tfo_syn) tp->t_flags |= TF_DELACK; else tp->t_flags |= TF_ACKNOW; @@ -3293,6 +3360,21 @@ tcp_dooptions(struct tcpopt *to, u_char *cp, int cnt, int flags) to->to_sacks = cp + 2; TCPSTAT_INC(tcps_sack_rcv_blocks); break; +#ifdef TCP_RFC7413 + case TCPOPT_FAST_OPEN: + if ((optlen != TCPOLEN_FAST_OPEN_EMPTY) && + (optlen < TCPOLEN_FAST_OPEN_MIN) && + (optlen > TCPOLEN_FAST_OPEN_MAX)) + continue; + if (!(flags & TO_SYN)) + continue; + if (!V_tcp_fastopen_enabled) + continue; + to->to_flags |= TOF_FASTOPEN; + to->to_tfo_len = optlen - 2; + to->to_tfo_cookie = to->to_tfo_len ? cp + 2 : NULL; + break; +#endif default: continue; } diff --git a/sys/netinet/tcp_output.c b/sys/netinet/tcp_output.c index d295dcf..482ead5 100644 --- a/sys/netinet/tcp_output.c +++ b/sys/netinet/tcp_output.c @@ -68,6 +68,9 @@ __FBSDID("$FreeBSD$"); #include <netinet/ip6.h> #include <netinet6/ip6_var.h> #endif +#ifdef TCP_RFC7413 +#include <netinet/tcp_fastopen.h> +#endif #define TCPOUTFLAGS #include <netinet/tcp_fsm.h> #include <netinet/tcp_seq.h> @@ -204,6 +207,17 @@ tcp_output(struct tcpcb *tp) return (tcp_offload_output(tp)); #endif +#ifdef TCP_RFC7413 + /* + * For TFO connections in SYN_RECEIVED, only allow the initial + * SYN|ACK and those sent by the retransmit timer. + */ + if ((tp->t_flags & TF_FASTOPEN) && + (tp->t_state == TCPS_SYN_RECEIVED) && + SEQ_GT(tp->snd_max, tp->snd_una) && /* inital SYN|ACK sent */ + (tp->snd_nxt != tp->snd_una)) /* not a retransmit */ + return (0); +#endif /* * Determine length of data that should be transmitted, * and flags that will be used. @@ -390,6 +404,15 @@ after_sack_rexmit: if ((flags & TH_SYN) && SEQ_GT(tp->snd_nxt, tp->snd_una)) { if (tp->t_state != TCPS_SYN_RECEIVED) flags &= ~TH_SYN; +#ifdef TCP_RFC7413 + /* + * When sending additional segments following a TFO SYN|ACK, + * do not include the SYN bit. + */ + if ((tp->t_flags & TF_FASTOPEN) && + (tp->t_state == TCPS_SYN_RECEIVED)) + flags &= ~TH_SYN; +#endif off--, len++; } @@ -403,6 +426,17 @@ after_sack_rexmit: flags &= ~TH_FIN; } +#ifdef TCP_RFC7413 + /* + * When retransmitting SYN|ACK on a passively-created TFO socket, + * don't include data, as the presence of data may have caused the + * original SYN|ACK to have been dropped by a middlebox. + */ + if ((tp->t_flags & TF_FASTOPEN) && + (((tp->t_state == TCPS_SYN_RECEIVED) && (tp->t_rxtshift > 0)) || + (flags & TH_RST))) + len = 0; +#endif if (len <= 0) { /* * If FIN has been sent but not acked, @@ -725,6 +759,22 @@ send: tp->snd_nxt = tp->iss; to.to_mss = tcp_mssopt(&tp->t_inpcb->inp_inc); to.to_flags |= TOF_MSS; +#ifdef TCP_RFC7413 + /* + * Only include the TFO option on the first + * transmission of the SYN|ACK on a + * passively-created TFO socket, as the presence of + * the TFO option may have caused the original + * SYN|ACK to have been dropped by a middlebox. + */ + if ((tp->t_flags & TF_FASTOPEN) && + (tp->t_state == TCPS_SYN_RECEIVED) && + (tp->t_rxtshift == 0)) { + to.to_tfo_len = TCP_FASTOPEN_COOKIE_LEN; + to.to_tfo_cookie = (u_char *)&tp->t_tfo_cookie; + to.to_flags |= TOF_FASTOPEN; + } +#endif } /* Window scaling. */ if ((flags & TH_SYN) && (tp->t_flags & TF_REQ_SCALE)) { @@ -1004,7 +1054,7 @@ send: * give data to the user when a buffer fills or * a PUSH comes in.) */ - if (off + len == sbused(&so->so_snd)) + if ((off + len == sbused(&so->so_snd)) && !(flags & TH_SYN)) flags |= TH_PUSH; SOCKBUF_UNLOCK(&so->so_snd); } else { @@ -1711,6 +1761,25 @@ tcp_addoptions(struct tcpopt *to, u_char *optp) TCPSTAT_INC(tcps_sack_send_blocks); break; } +#ifdef TCP_RFC7413 + case TOF_FASTOPEN: + { + int total_len; + + /* XXX is there any point to aligning this option? */ + total_len = TCPOLEN_FAST_OPEN_EMPTY + to->to_tfo_len; + if (TCP_MAXOLEN - optlen < total_len) + continue; + *optp++ = TCPOPT_FAST_OPEN; + *optp++ = total_len; + if (to->to_tfo_len > 0) { + bcopy(to->to_tfo_cookie, optp, to->to_tfo_len); + optp += to->to_tfo_len; + } + optlen += total_len; + break; + } +#endif default: panic("%s: unknown TCP option type", __func__); break; diff --git a/sys/netinet/tcp_subr.c b/sys/netinet/tcp_subr.c index 00869a6..c2e0696 100644 --- a/sys/netinet/tcp_subr.c +++ b/sys/netinet/tcp_subr.c @@ -84,6 +84,9 @@ __FBSDID("$FreeBSD$"); #include <netinet6/nd6.h> #endif +#ifdef TCP_RFC7413 +#include <netinet/tcp_fastopen.h> +#endif #include <netinet/tcp_fsm.h> #include <netinet/tcp_seq.h> #include <netinet/tcp_timer.h> @@ -593,10 +596,6 @@ tcp_init(void) if (hhook_head_register(HHOOK_TYPE_TCP, HHOOK_TCP_EST_OUT, &V_tcp_hhh[HHOOK_TCP_EST_OUT], HHOOK_NOWAIT|HHOOK_HEADISINVNET) != 0) printf("%s: WARNING: unable to register helper hook\n", __func__); - /* Setup the tcp function block list */ - TAILQ_INIT(&t_functions); - rw_init_flags(&tcp_function_lock, "tcp_func_lock" , 0); - register_tcp_functions(&tcp_def_funcblk, M_WAITOK); hashsize = TCBHASHSIZE; TUNABLE_INT_FETCH(tcbhash_tuneable, &hashsize); if (hashsize == 0) { @@ -613,7 +612,7 @@ tcp_init(void) */ if (hashsize < 512) hashsize = 512; - if (bootverbose) + if (bootverbose && IS_DEFAULT_VNET(curvnet)) printf("%s: %s auto tuned to %d\n", __func__, tcbhash_tuneable, hashsize); } @@ -675,6 +674,10 @@ tcp_init(void) tcp_rexmit_slop = TCPTV_CPU_VAR; tcp_finwait2_timeout = TCPTV_FINWAIT2_TIMEOUT; tcp_tcbhashsize = hashsize; + /* Setup the tcp function block list */ + TAILQ_INIT(&t_functions); + rw_init_flags(&tcp_function_lock, "tcp_func_lock" , 0); + register_tcp_functions(&tcp_def_funcblk, M_WAITOK); if (tcp_soreceive_stream) { #ifdef INET @@ -704,6 +707,10 @@ tcp_init(void) #ifdef TCPPCAP tcp_pcap_init(); #endif + +#ifdef TCP_RFC7413 + tcp_fastopen_init(); +#endif } #ifdef VIMAGE @@ -712,6 +719,9 @@ tcp_destroy(void) { int error; +#ifdef TCP_RFC7413 + tcp_fastopen_destroy(); +#endif tcp_hc_destroy(); syncache_destroy(); tcp_tw_destroy(); @@ -1439,6 +1449,17 @@ tcp_close(struct tcpcb *tp) if (tp->t_state == TCPS_LISTEN) tcp_offload_listen_stop(tp); #endif +#ifdef TCP_RFC7413 + /* + * This releases the TFO pending counter resource for TFO listen + * sockets as well as passively-created TFO sockets that transition + * from SYN_RECEIVED to CLOSED. + */ + if (tp->t_tfo_pending) { + tcp_fastopen_decrement_counter(tp->t_tfo_pending); + tp->t_tfo_pending = NULL; + } +#endif in_pcbdrop(inp); TCPSTAT_INC(tcps_closed); KASSERT(inp->inp_socket != NULL, ("tcp_close: inp_socket NULL")); diff --git a/sys/netinet/tcp_syncache.c b/sys/netinet/tcp_syncache.c index 8ad1b22..940d3de 100644 --- a/sys/netinet/tcp_syncache.c +++ b/sys/netinet/tcp_syncache.c @@ -81,6 +81,9 @@ __FBSDID("$FreeBSD$"); #include <netinet6/in6_pcb.h> #endif #include <netinet/tcp.h> +#ifdef TCP_RFC7413 +#include <netinet/tcp_fastopen.h> +#endif #include <netinet/tcp_fsm.h> #include <netinet/tcp_seq.h> #include <netinet/tcp_timer.h> @@ -1083,6 +1086,39 @@ failed: return (0); } +#ifdef TCP_RFC7413 +static void +syncache_tfo_expand(struct syncache *sc, struct socket **lsop, struct mbuf *m, + uint64_t response_cookie) +{ + struct inpcb *inp; + struct tcpcb *tp; + unsigned int *pending_counter; + + /* + * Global TCP locks are held because we manipulate the PCB lists + * and create a new socket. + */ + INP_INFO_RLOCK_ASSERT(&V_tcbinfo); + + pending_counter = intotcpcb(sotoinpcb(*lsop))->t_tfo_pending; + *lsop = syncache_socket(sc, *lsop, m); + if (*lsop == NULL) { + TCPSTAT_INC(tcps_sc_aborted); + atomic_subtract_int(pending_counter, 1); + } else { + inp = sotoinpcb(*lsop); + tp = intotcpcb(inp); + tp->t_flags |= TF_FASTOPEN; + tp->t_tfo_cookie = response_cookie; + tp->snd_max = tp->iss; + tp->snd_nxt = tp->iss; + tp->t_tfo_pending = pending_counter; + TCPSTAT_INC(tcps_sc_completed); + } +} +#endif /* TCP_RFC7413 */ + /* * Given a LISTEN socket and an inbound SYN request, add * this to the syn cache, and send back a segment: @@ -1095,8 +1131,15 @@ failed: * DoS attack, an attacker could send data which would eventually * consume all available buffer space if it were ACKed. By not ACKing * the data, we avoid this DoS scenario. + * + * The exception to the above is when a SYN with a valid TCP Fast Open (TFO) + * cookie is processed, V_tcp_fastopen_enabled set to true, and the + * TCP_FASTOPEN socket option is set. In this case, a new socket is created + * and returned via lsop, the mbuf is not freed so that tcp_input() can + * queue its data to the socket, and 1 is returned to indicate the + * TFO-socket-creation path was taken. */ -void +int syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, struct inpcb *inp, struct socket **lsop, struct mbuf *m, void *tod, void *todctx) @@ -1109,6 +1152,7 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, u_int ltflags; int win, sb_hiwat, ip_ttl, ip_tos; char *s; + int rv = 0; #ifdef INET6 int autoflowlabel = 0; #endif @@ -1117,6 +1161,11 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, #endif struct syncache scs; struct ucred *cred; +#ifdef TCP_RFC7413 + uint64_t tfo_response_cookie; + int tfo_cookie_valid = 0; + int tfo_response_cookie_valid = 0; +#endif INP_WLOCK_ASSERT(inp); /* listen socket */ KASSERT((th->th_flags & (TH_RST|TH_ACK|TH_SYN)) == TH_SYN, @@ -1141,6 +1190,29 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, sb_hiwat = so->so_rcv.sb_hiwat; ltflags = (tp->t_flags & (TF_NOOPT | TF_SIGNATURE)); +#ifdef TCP_RFC7413 + if (V_tcp_fastopen_enabled && (tp->t_flags & TF_FASTOPEN) && + (tp->t_tfo_pending != NULL) && (to->to_flags & TOF_FASTOPEN)) { + /* + * Limit the number of pending TFO connections to + * approximately half of the queue limit. This prevents TFO + * SYN floods from starving the service by filling the + * listen queue with bogus TFO connections. + */ + if (atomic_fetchadd_int(tp->t_tfo_pending, 1) <= + (so->so_qlimit / 2)) { + int result; + + result = tcp_fastopen_check_cookie(inc, + to->to_tfo_cookie, to->to_tfo_len, + &tfo_response_cookie); + tfo_cookie_valid = (result > 0); + tfo_response_cookie_valid = (result >= 0); + } else + atomic_subtract_int(tp->t_tfo_pending, 1); + } +#endif + /* By the time we drop the lock these should no longer be used. */ so = NULL; tp = NULL; @@ -1152,7 +1224,10 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, } else mac_syncache_create(maclabel, inp); #endif - INP_WUNLOCK(inp); +#ifdef TCP_RFC7413 + if (!tfo_cookie_valid) +#endif + INP_WUNLOCK(inp); /* * Remember the IP options, if any. @@ -1181,6 +1256,10 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, sc = syncache_lookup(inc, &sch); /* returns locked entry */ SCH_LOCK_ASSERT(sch); if (sc != NULL) { +#ifdef TCP_RFC7413 + if (tfo_cookie_valid) + INP_WUNLOCK(inp); +#endif TCPSTAT_INC(tcps_sc_dupsyn); if (ipopts) { /* @@ -1223,6 +1302,14 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, goto done; } +#ifdef TCP_RFC7413 + if (tfo_cookie_valid) { + bzero(&scs, sizeof(scs)); + sc = &scs; + goto skip_alloc; + } +#endif + sc = uma_zalloc(V_tcp_syncache.zone, M_NOWAIT | M_ZERO); if (sc == NULL) { /* @@ -1246,7 +1333,13 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, } } } - + +#ifdef TCP_RFC7413 +skip_alloc: + if (!tfo_cookie_valid && tfo_response_cookie_valid) + sc->sc_tfo_cookie = &tfo_response_cookie; +#endif + /* * Fill in the syncache values. */ @@ -1354,6 +1447,15 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, #endif SCH_UNLOCK(sch); +#ifdef TCP_RFC7413 + if (tfo_cookie_valid) { + syncache_tfo_expand(sc, lsop, m, tfo_response_cookie); + /* INP_WUNLOCK(inp) will be performed by the called */ + rv = 1; + goto tfo_done; + } +#endif + /* * Do a standard 3-way handshake. */ @@ -1371,17 +1473,20 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *to, struct tcphdr *th, } done: + if (m) { + *lsop = NULL; + m_freem(m); + } +#ifdef TCP_RFC7413 +tfo_done: +#endif if (cred != NULL) crfree(cred); #ifdef MAC if (sc == &scs) mac_syncache_destroy(&maclabel); #endif - if (m) { - - *lsop = NULL; - m_freem(m); - } + return (rv); } static int @@ -1533,6 +1638,16 @@ syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked) } } #endif + +#ifdef TCP_RFC7413 + if (sc->sc_tfo_cookie) { + to.to_flags |= TOF_FASTOPEN; + to.to_tfo_len = TCP_FASTOPEN_COOKIE_LEN; + to.to_tfo_cookie = sc->sc_tfo_cookie; + /* don't send cookie again when retransmitting response */ + sc->sc_tfo_cookie = NULL; + } +#endif optlen = tcp_addoptions(&to, (u_char *)(th + 1)); /* Adjust headers by option size. */ diff --git a/sys/netinet/tcp_syncache.h b/sys/netinet/tcp_syncache.h index c0c854d..9fcbbec 100644 --- a/sys/netinet/tcp_syncache.h +++ b/sys/netinet/tcp_syncache.h @@ -41,7 +41,7 @@ void syncache_destroy(void); void syncache_unreach(struct in_conninfo *, struct tcphdr *); int syncache_expand(struct in_conninfo *, struct tcpopt *, struct tcphdr *, struct socket **, struct mbuf *); -void syncache_add(struct in_conninfo *, struct tcpopt *, +int syncache_add(struct in_conninfo *, struct tcpopt *, struct tcphdr *, struct inpcb *, struct socket **, struct mbuf *, void *, void *); void syncache_chkrst(struct in_conninfo *, struct tcphdr *); @@ -74,7 +74,9 @@ struct syncache { #endif struct label *sc_label; /* MAC label reference */ struct ucred *sc_cred; /* cred cache for jail checks */ - +#ifdef TCP_RFC7413 + void *sc_tfo_cookie; /* for TCP Fast Open response */ +#endif void *sc_pspare; /* TCP_SIGNATURE */ u_int32_t sc_spare[2]; /* UTO */ }; diff --git a/sys/netinet/tcp_timer.c b/sys/netinet/tcp_timer.c index 6a24ce7..9767eb7 100644 --- a/sys/netinet/tcp_timer.c +++ b/sys/netinet/tcp_timer.c @@ -643,7 +643,8 @@ tcp_timer_rexmt(void * xtp) } else tp->t_flags &= ~TF_PREVVALID; TCPSTAT_INC(tcps_rexmttimeo); - if (tp->t_state == TCPS_SYN_SENT) + if ((tp->t_state == TCPS_SYN_SENT) || + (tp->t_state == TCPS_SYN_RECEIVED)) rexmt = TCPTV_RTOBASE * tcp_syn_backoff[tp->t_rxtshift]; else rexmt = TCP_REXMTVAL(tp) * tcp_backoff[tp->t_rxtshift]; diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c index 42e2ea7..3435668 100644 --- a/sys/netinet/tcp_usrreq.c +++ b/sys/netinet/tcp_usrreq.c @@ -82,6 +82,9 @@ __FBSDID("$FreeBSD$"); #include <netinet6/ip6_var.h> #include <netinet6/scope6_var.h> #endif +#ifdef TCP_RFC7413 +#include <netinet/tcp_fastopen.h> +#endif #include <netinet/tcp_fsm.h> #include <netinet/tcp_seq.h> #include <netinet/tcp_timer.h> @@ -405,6 +408,10 @@ tcp_usr_listen(struct socket *so, int backlog, struct thread *td) } SOCK_UNLOCK(so); +#ifdef TCP_RFC7413 + if (tp->t_flags & TF_FASTOPEN) + tp->t_tfo_pending = tcp_fastopen_alloc_counter(); +#endif out: TCPDEBUG2(PRU_LISTEN); TCP_PROBE2(debug__user, tp, PRU_LISTEN); @@ -451,6 +458,10 @@ tcp6_usr_listen(struct socket *so, int backlog, struct thread *td) } SOCK_UNLOCK(so); +#ifdef TCP_RFC7413 + if (tp->t_flags & TF_FASTOPEN) + tp->t_tfo_pending = tcp_fastopen_alloc_counter(); +#endif out: TCPDEBUG2(PRU_LISTEN); TCP_PROBE2(debug__user, tp, PRU_LISTEN); @@ -805,6 +816,18 @@ tcp_usr_rcvd(struct socket *so, int flags) } tp = intotcpcb(inp); TCPDEBUG1(); +#ifdef TCP_RFC7413 + /* + * For passively-created TFO connections, don't attempt a window + * update while still in SYN_RECEIVED as this may trigger an early + * SYN|ACK. It is preferable to have the SYN|ACK be sent along with + * application response data, or failing that, when the DELACK timer + * expires. + */ + if ((tp->t_flags & TF_FASTOPEN) && + (tp->t_state == TCPS_SYN_RECEIVED)) + goto out; +#endif #ifdef TCP_OFFLOAD if (tp->t_flags & TF_TOE) tcp_offload_rcvd(tp); @@ -1674,6 +1697,29 @@ unlock_and_done: goto unlock_and_done; #endif +#ifdef TCP_RFC7413 + case TCP_FASTOPEN: + INP_WUNLOCK(inp); + if (!V_tcp_fastopen_enabled) + return (EPERM); + + error = sooptcopyin(sopt, &optval, sizeof optval, + sizeof optval); + if (error) + return (error); + + INP_WLOCK_RECHECK(inp); + if (optval) { + tp->t_flags |= TF_FASTOPEN; + if ((tp->t_state == TCPS_LISTEN) && + (tp->t_tfo_pending == NULL)) + tp->t_tfo_pending = + tcp_fastopen_alloc_counter(); + } else + tp->t_flags &= ~TF_FASTOPEN; + goto unlock_and_done; +#endif + default: INP_WUNLOCK(inp); error = ENOPROTOOPT; @@ -1753,6 +1799,14 @@ unlock_and_done: error = sooptcopyout(sopt, &optval, sizeof optval); break; #endif + +#ifdef TCP_RFC7413 + case TCP_FASTOPEN: + optval = tp->t_flags & TF_FASTOPEN; + INP_WUNLOCK(inp); + error = sooptcopyout(sopt, &optval, sizeof optval); + break; +#endif default: INP_WUNLOCK(inp); error = ENOPROTOOPT; @@ -2076,6 +2130,10 @@ db_print_tflags(u_int t_flags) db_printf("%sTF_ECN_PERMIT", comma ? ", " : ""); comma = 1; } + if (t_flags & TF_FASTOPEN) { + db_printf("%sTF_FASTOPEN", comma ? ", " : ""); + comma = 1; + } } static void diff --git a/sys/netinet/tcp_var.h b/sys/netinet/tcp_var.h index 2b3954a..8d76912 100644 --- a/sys/netinet/tcp_var.h +++ b/sys/netinet/tcp_var.h @@ -253,10 +253,20 @@ struct tcpcb { u_int t_tsomaxsegsize; /* TSO maximum segment size in bytes */ u_int t_pmtud_saved_maxopd; /* pre-blackhole MSS */ u_int t_flags2; /* More tcpcb flags storage */ +#if defined(_KERNEL) && defined(TCP_RFC7413) + uint32_t t_ispare[6]; /* 5 UTO, 1 TBD */ + uint64_t t_tfo_cookie; /* TCP Fast Open cookie */ +#else uint32_t t_ispare[8]; /* 5 UTO, 3 TBD */ +#endif struct tcp_function_block *t_fb;/* TCP function call block */ void *t_fb_ptr; /* Pointer to t_fb specific data */ +#if defined(_KERNEL) && defined(TCP_RFC7413) + unsigned int *t_tfo_pending; /* TCP Fast Open pending counter */ + void *t_pspare2[1]; /* 1 TCP_SIGNATURE */ +#else void *t_pspare2[2]; /* 1 TCP_SIGNATURE, 1 TBD */ +#endif #if defined(_KERNEL) && defined(TCPPCAP) struct mbufq t_inpkts; /* List of saved input packets. */ struct mbufq t_outpkts; /* List of saved output packets. */ @@ -302,6 +312,7 @@ struct tcpcb { #define TF_ECN_SND_ECE 0x10000000 /* ECN ECE in queue */ #define TF_CONGRECOVERY 0x20000000 /* congestion recovery mode */ #define TF_WASCRECOVERY 0x40000000 /* was in congestion recovery */ +#define TF_FASTOPEN 0x80000000 /* TCP Fast Open indication */ #define IN_FASTRECOVERY(t_flags) (t_flags & TF_FASTRECOVERY) #define ENTER_FASTRECOVERY(t_flags) t_flags |= TF_FASTRECOVERY @@ -361,14 +372,17 @@ struct tcpopt { #define TOF_TS 0x0010 /* timestamp */ #define TOF_SIGNATURE 0x0040 /* TCP-MD5 signature option (RFC2385) */ #define TOF_SACK 0x0080 /* Peer sent SACK option */ -#define TOF_MAXOPT 0x0100 +#define TOF_FASTOPEN 0x0100 /* TCP Fast Open (TFO) cookie */ +#define TOF_MAXOPT 0x0200 u_int32_t to_tsval; /* new timestamp */ u_int32_t to_tsecr; /* reflected timestamp */ u_char *to_sacks; /* pointer to the first SACK blocks */ u_char *to_signature; /* pointer to the TCP-MD5 signature */ + u_char *to_tfo_cookie; /* pointer to the TFO cookie */ u_int16_t to_mss; /* maximum segment size */ u_int8_t to_wscale; /* window scaling */ u_int8_t to_nsacks; /* number of SACK blocks */ + u_int8_t to_tfo_len; /* TFO cookie length */ u_int32_t to_spare; /* UTO */ }; diff --git a/sys/netinet/udp_usrreq.c b/sys/netinet/udp_usrreq.c index e200513..f474a54 100644 --- a/sys/netinet/udp_usrreq.c +++ b/sys/netinet/udp_usrreq.c @@ -128,7 +128,6 @@ SYSCTL_INT(_net_inet_udp, OID_AUTO, blackhole, CTLFLAG_VNET | CTLFLAG_RW, "Do not send port unreachables for refused connects"); u_long udp_sendspace = 9216; /* really max datagram size */ - /* 40 1K datagrams */ SYSCTL_ULONG(_net_inet_udp, UDPCTL_MAXDGRAM, maxdgram, CTLFLAG_RW, &udp_sendspace, 0, "Maximum outgoing UDP datagram size"); @@ -138,7 +137,7 @@ u_long udp_recvspace = 40 * (1024 + #else sizeof(struct sockaddr_in) #endif - ); + ); /* 40 1K datagrams */ SYSCTL_ULONG(_net_inet_udp, UDPCTL_RECVSPACE, recvspace, CTLFLAG_RW, &udp_recvspace, 0, "Maximum space for incoming UDP datagrams"); diff --git a/sys/netinet6/in6.c b/sys/netinet6/in6.c index aedaf7f..0f8fb67 100644 --- a/sys/netinet6/in6.c +++ b/sys/netinet6/in6.c @@ -2008,6 +2008,7 @@ in6_if2idlen(struct ifnet *ifp) case IFT_PROPVIRTUAL: /* XXX: no RFC. treat it as ether */ case IFT_L2VLAN: /* ditto */ case IFT_IEEE80211: /* ditto */ + case IFT_BRIDGE: /* bridge(4) only does Ethernet-like links */ case IFT_INFINIBAND: return (64); case IFT_FDDI: /* RFC2467 */ diff --git a/sys/netinet6/in6_rmx.c b/sys/netinet6/in6_rmx.c index 34065c5..97e12162 100644 --- a/sys/netinet6/in6_rmx.c +++ b/sys/netinet6/in6_rmx.c @@ -266,7 +266,7 @@ in6_detachhead(void **head, int off) { callout_drain(&V_rtq_mtutimer); - return (1); + return (rn_detachhead(head)); } #endif diff --git a/sys/opencrypto/xform.h b/sys/opencrypto/xform.h index e0b6397..185e203 100644 --- a/sys/opencrypto/xform.h +++ b/sys/opencrypto/xform.h @@ -33,7 +33,9 @@ #include <sys/md5.h> #include <crypto/sha1.h> -#include <crypto/sha2/sha2.h> +#include <crypto/sha2/sha256.h> +#include <crypto/sha2/sha384.h> +#include <crypto/sha2/sha512.h> #include <opencrypto/rmd160.h> #include <opencrypto/gmac.h> diff --git a/sys/powerpc/booke/pmap.c b/sys/powerpc/booke/pmap.c index 2539c33e..6ea98b7 100644 --- a/sys/powerpc/booke/pmap.c +++ b/sys/powerpc/booke/pmap.c @@ -1024,12 +1024,13 @@ pte_find(mmu_t mmu, pmap_t pmap, vm_offset_t va) static void mmu_booke_bootstrap(mmu_t mmu, vm_offset_t start, vm_offset_t kernelend) { - vm_offset_t phys_kernelend; + vm_paddr_t phys_kernelend; struct mem_region *mp, *mp1; int cnt, i, j; - u_int s, e, sz; + vm_paddr_t s, e, sz; + vm_paddr_t physsz, hwphyssz; u_int phys_avail_count; - vm_size_t physsz, hwphyssz, kstack0_sz; + vm_size_t kstack0_sz; vm_offset_t kernel_pdir, kstack0, va; vm_paddr_t kstack0_phys; void *dpcpu; @@ -1163,7 +1164,7 @@ mmu_booke_bootstrap(mmu_t mmu, vm_offset_t start, vm_offset_t kernelend) for (mp = availmem_regions; mp->mr_size; mp++) { s = mp->mr_start; e = mp->mr_start + mp->mr_size; - debugf(" %08x-%08x -> ", s, e); + debugf(" %09jx-%09jx -> ", (uintmax_t)s, (uintmax_t)e); /* Check whether this region holds all of the kernel. */ if (s < kernload && e > phys_kernelend) { availmem_regions[cnt].mr_start = phys_kernelend; @@ -1188,7 +1189,8 @@ mmu_booke_bootstrap(mmu_t mmu, vm_offset_t start, vm_offset_t kernelend) if (e < s) e = s; sz = e - s; - debugf("%08x-%08x = %x\n", s, e, sz); + debugf("%09jx-%09jx = %jx\n", + (uintmax_t)s, (uintmax_t)e, (uintmax_t)sz); /* Check whether some memory is left here. */ if (sz == 0) { @@ -1237,10 +1239,10 @@ mmu_booke_bootstrap(mmu_t mmu, vm_offset_t start, vm_offset_t kernelend) for (i = 0, j = 0; i < availmem_regions_sz; i++, j += 2) { debugf(" region: 0x%jx - 0x%jx (0x%jx)\n", - availmem_regions[i].mr_start, - availmem_regions[i].mr_start + + (uintmax_t)availmem_regions[i].mr_start, + (uintmax_t)availmem_regions[i].mr_start + availmem_regions[i].mr_size, - availmem_regions[i].mr_size); + (uintmax_t)availmem_regions[i].mr_size); if (hwphyssz != 0 && (physsz + availmem_regions[i].mr_size) >= hwphyssz) { diff --git a/sys/powerpc/include/ofw_machdep.h b/sys/powerpc/include/ofw_machdep.h index 0ee75f4..8376e1a 100644 --- a/sys/powerpc/include/ofw_machdep.h +++ b/sys/powerpc/include/ofw_machdep.h @@ -37,7 +37,6 @@ typedef uint32_t cell_t; -int OF_decode_addr(phandle_t, int, bus_space_tag_t *, bus_space_handle_t *); void OF_getetheraddr(device_t dev, u_char *addr); void OF_initial_setup(void *fdt_ptr, void *junk, int (*openfirm)(void *)); diff --git a/sys/powerpc/include/param.h b/sys/powerpc/include/param.h index df5a7be..453915c 100644 --- a/sys/powerpc/include/param.h +++ b/sys/powerpc/include/param.h @@ -98,7 +98,7 @@ #define PAGE_SHIFT 12 #define PAGE_SIZE (1L << PAGE_SHIFT) /* Page size */ -#define PAGE_MASK (vm_offset_t)(PAGE_SIZE - 1) +#define PAGE_MASK (PAGE_SIZE - 1) #define NPTEPG (PAGE_SIZE/(sizeof (pt_entry_t))) #define MAXPAGESIZES 1 /* maximum number of supported page sizes */ @@ -116,15 +116,17 @@ /* * Mach derived conversion macros */ -#define trunc_page(x) ((unsigned long)(x) & ~(PAGE_MASK)) +#define trunc_page(x) ((x) & ~(PAGE_MASK)) #define round_page(x) (((x) + PAGE_MASK) & ~PAGE_MASK) -#define atop(x) ((unsigned long)(x) >> PAGE_SHIFT) -#define ptoa(x) ((unsigned long)(x) << PAGE_SHIFT) +#define atop(x) ((x) >> PAGE_SHIFT) +#define ptoa(x) ((x) << PAGE_SHIFT) -#define powerpc_btop(x) ((unsigned long)(x) >> PAGE_SHIFT) -#define powerpc_ptob(x) ((unsigned long)(x) << PAGE_SHIFT) +#define powerpc_btop(x) ((x) >> PAGE_SHIFT) +#define powerpc_ptob(x) ((x) << PAGE_SHIFT) #define pgtok(x) ((x) * (PAGE_SIZE / 1024UL)) +#define btoc(x) ((vm_offset_t)(((x)+PAGE_MASK)>>PAGE_SHIFT)) + #endif /* !_POWERPC_INCLUDE_PARAM_H_ */ diff --git a/sys/powerpc/include/platform.h b/sys/powerpc/include/platform.h index 854c0be..28f83cc 100644 --- a/sys/powerpc/include/platform.h +++ b/sys/powerpc/include/platform.h @@ -39,8 +39,8 @@ #include <machine/pcpu.h> struct mem_region { - vm_offset_t mr_start; - vm_size_t mr_size; + uint64_t mr_start; + uint64_t mr_size; }; void mem_regions(struct mem_region **, int *, struct mem_region **, int *); diff --git a/sys/powerpc/include/pmap.h b/sys/powerpc/include/pmap.h index dc8649e..97c5c79 100644 --- a/sys/powerpc/include/pmap.h +++ b/sys/powerpc/include/pmap.h @@ -250,7 +250,7 @@ boolean_t pmap_mmu_install(char *name, int prio); * For more Ram increase the lmb or this value. */ -extern vm_offset_t phys_avail[PHYS_AVAIL_SZ]; +extern vm_paddr_t phys_avail[PHYS_AVAIL_SZ]; extern vm_offset_t virtual_avail; extern vm_offset_t virtual_end; diff --git a/sys/powerpc/ofw/ofw_machdep.c b/sys/powerpc/ofw/ofw_machdep.c index d085af9..d2bb77b 100644 --- a/sys/powerpc/ofw/ofw_machdep.c +++ b/sys/powerpc/ofw/ofw_machdep.c @@ -52,6 +52,7 @@ __FBSDID("$FreeBSD$"); #include <dev/ofw/openfirm.h> #include <dev/ofw/ofw_pci.h> #include <dev/ofw/ofw_bus.h> +#include <dev/ofw/ofw_subr.h> #include <vm/vm.h> #include <vm/vm_param.h> @@ -171,8 +172,8 @@ parse_ofw_memory(phandle_t node, const char *prop, struct mem_region *output) i = 0; j = 0; while (i < sz/sizeof(cell_t)) { - #ifndef __powerpc64__ - /* On 32-bit PPC, ignore regions starting above 4 GB */ + #if !defined(__powerpc64__) && !defined(BOOKE) + /* On 32-bit PPC (OEA), ignore regions starting above 4 GB */ if (address_cells > 1 && OFmem[i] > 0) { i += address_cells + size_cells; continue; @@ -181,21 +182,18 @@ parse_ofw_memory(phandle_t node, const char *prop, struct mem_region *output) output[j].mr_start = OFmem[i++]; if (address_cells == 2) { - #ifdef __powerpc64__ output[j].mr_start <<= 32; - #endif output[j].mr_start += OFmem[i++]; } output[j].mr_size = OFmem[i++]; if (size_cells == 2) { - #ifdef __powerpc64__ output[j].mr_size <<= 32; - #endif output[j].mr_size += OFmem[i++]; } - #ifndef __powerpc64__ + #if !defined(__powerpc64__) && !defined(BOOKE) + /* Book-E can support 36-bit addresses. */ /* * Check for memory regions extending above 32-bit * memory space, and restrict them to stay there. @@ -565,135 +563,28 @@ OF_getetheraddr(device_t dev, u_char *addr) * register in the address space of its parent and recursively walk * the device tree upward this way. */ -static void -OF_get_addr_props(phandle_t node, uint32_t *addrp, uint32_t *sizep, int *pcip) -{ - char type[64]; - uint32_t addr, size; - int pci, res; - - res = OF_getencprop(node, "#address-cells", &addr, sizeof(addr)); - if (res == -1) - addr = 2; - res = OF_getencprop(node, "#size-cells", &size, sizeof(size)); - if (res == -1) - size = 1; - pci = 0; - if (addr == 3 && size == 2) { - res = OF_getprop(node, "device_type", type, sizeof(type)); - if (res != -1) { - type[sizeof(type) - 1] = '\0'; - pci = (strcmp(type, "pci") == 0) ? 1 : 0; - } - } - if (addrp != NULL) - *addrp = addr; - if (sizep != NULL) - *sizep = size; - if (pcip != NULL) - *pcip = pci; -} - int OF_decode_addr(phandle_t dev, int regno, bus_space_tag_t *tag, bus_space_handle_t *handle) { - uint32_t cell[32]; - bus_addr_t addr, raddr, baddr; - bus_size_t size, rsize; - uint32_t c, nbridge, naddr, nsize; - phandle_t bridge, parent; - u_int spc, rspc, prefetch; - int pci, pcib, res; - - /* Sanity checking. */ - if (dev == 0) - return (EINVAL); - bridge = OF_parent(dev); - if (bridge == 0) - return (EINVAL); - if (regno < 0) - return (EINVAL); - if (tag == NULL || handle == NULL) - return (EINVAL); - - /* Assume big-endian unless we find a PCI device */ - *tag = &bs_be_tag; - - /* Get the requested register. */ - OF_get_addr_props(bridge, &naddr, &nsize, &pci); - if (pci) + bus_addr_t addr; + bus_size_t size; + pcell_t pci_hi; + int flags, res; + + res = ofw_reg_to_paddr(dev, regno, &addr, &size, &pci_hi); + if (res < 0) + return (res); + + if (pci_hi == OFW_PADDR_NOT_PCI) { + *tag = &bs_be_tag; + flags = 0; + } else { *tag = &bs_le_tag; - res = OF_getencprop(dev, (pci) ? "assigned-addresses" : "reg", - cell, sizeof(cell)); - if (res == -1) - return (ENXIO); - if (res % sizeof(cell[0])) - return (ENXIO); - res /= sizeof(cell[0]); - regno *= naddr + nsize; - if (regno + naddr + nsize > res) - return (EINVAL); - spc = (pci) ? cell[regno] & OFW_PCI_PHYS_HI_SPACEMASK : ~0; - prefetch = (pci) ? cell[regno] & OFW_PCI_PHYS_HI_PREFETCHABLE : 0; - addr = 0; - for (c = 0; c < naddr; c++) - addr = ((uint64_t)addr << 32) | cell[regno++]; - size = 0; - for (c = 0; c < nsize; c++) - size = ((uint64_t)size << 32) | cell[regno++]; - - /* - * Map the address range in the bridge's decoding window as given - * by the "ranges" property. If a node doesn't have such property - * then no mapping is done. - */ - parent = OF_parent(bridge); - while (parent != 0) { - OF_get_addr_props(parent, &nbridge, NULL, &pcib); - if (pcib) - *tag = &bs_le_tag; - res = OF_getencprop(bridge, "ranges", cell, sizeof(cell)); - if (res == -1) - goto next; - if (res % sizeof(cell[0])) - return (ENXIO); - res /= sizeof(cell[0]); - regno = 0; - while (regno < res) { - rspc = (pci) - ? cell[regno] & OFW_PCI_PHYS_HI_SPACEMASK - : ~0; - if (rspc != spc) { - regno += naddr + nbridge + nsize; - continue; - } - raddr = 0; - for (c = 0; c < naddr; c++) - raddr = ((uint64_t)raddr << 32) | cell[regno++]; - rspc = (pcib) - ? cell[regno] & OFW_PCI_PHYS_HI_SPACEMASK - : ~0; - baddr = 0; - for (c = 0; c < nbridge; c++) - baddr = ((uint64_t)baddr << 32) | cell[regno++]; - rsize = 0; - for (c = 0; c < nsize; c++) - rsize = ((uint64_t)rsize << 32) | cell[regno++]; - if (addr < raddr || addr >= raddr + rsize) - continue; - addr = addr - raddr + baddr; - if (rspc != ~0) - spc = rspc; - } - - next: - bridge = parent; - parent = OF_parent(bridge); - OF_get_addr_props(bridge, &naddr, &nsize, &pci); + flags = (pci_hi & OFW_PCI_PHYS_HI_PREFETCHABLE) ? + BUS_SPACE_MAP_PREFETCHABLE: 0; } - return (bus_space_map(*tag, addr, size, - prefetch ? BUS_SPACE_MAP_PREFETCHABLE : 0, handle)); + return (bus_space_map(*tag, addr, size, flags, handle)); } diff --git a/sys/powerpc/powerpc/cpu.c b/sys/powerpc/powerpc/cpu.c index 7145505..43190ae 100644 --- a/sys/powerpc/powerpc/cpu.c +++ b/sys/powerpc/powerpc/cpu.c @@ -175,9 +175,12 @@ static const struct cputab models[] = { { "Freescale e500v2 core", FSL_E500v2, REVFMT_MAJMIN, 0, cpu_booke_setup }, { "Freescale e500mc core", FSL_E500mc, REVFMT_MAJMIN, - 0, cpu_booke_setup }, + PPC_FEATURE_HAS_FPU, cpu_booke_setup }, { "Freescale e5500 core", FSL_E5500, REVFMT_MAJMIN, - 0, cpu_booke_setup }, + PPC_FEATURE_64 | PPC_FEATURE_HAS_FPU, cpu_booke_setup }, + { "Freescale e6500 core", FSL_E6500, REVFMT_MAJMIN, + PPC_FEATURE_64 | PPC_FEATURE_HAS_ALTIVEC | PPC_FEATURE_HAS_FPU, + cpu_booke_setup }, { "IBM Cell Broadband Engine", IBMCELLBE, REVFMT_MAJMIN, PPC_FEATURE_64 | PPC_FEATURE_HAS_ALTIVEC | PPC_FEATURE_HAS_FPU, NULL}, diff --git a/sys/powerpc/powerpc/machdep.c b/sys/powerpc/powerpc/machdep.c index 781a3f6..dea3461 100644 --- a/sys/powerpc/powerpc/machdep.c +++ b/sys/powerpc/powerpc/machdep.c @@ -176,8 +176,8 @@ cpu_startup(void *dummy) #ifdef PERFMON perfmon_init(); #endif - printf("real memory = %lu (%lu MB)\n", ptoa(physmem), - ptoa(physmem) / 1048576); + printf("real memory = %ju (%ju MB)\n", ptoa((uintmax_t)physmem), + ptoa((uintmax_t)physmem) / 1048576); realmem = physmem; if (bootverbose) @@ -193,23 +193,25 @@ cpu_startup(void *dummy) printf("Physical memory chunk(s):\n"); for (indx = 0; phys_avail[indx + 1] != 0; indx += 2) { - vm_offset_t size1 = + vm_paddr_t size1 = phys_avail[indx + 1] - phys_avail[indx]; #ifdef __powerpc64__ - printf("0x%016lx - 0x%016lx, %ld bytes (%ld pages)\n", + printf("0x%016jx - 0x%016jx, %jd bytes (%jd pages)\n", #else - printf("0x%08x - 0x%08x, %u bytes (%lu pages)\n", + printf("0x%09jx - 0x%09jx, %ju bytes (%ju pages)\n", #endif - phys_avail[indx], phys_avail[indx + 1] - 1, size1, - size1 / PAGE_SIZE); + (uintmax_t)phys_avail[indx], + (uintmax_t)phys_avail[indx + 1] - 1, + (uintmax_t)size1, (uintmax_t)size1 / PAGE_SIZE); } } vm_ksubmap_init(&kmi); - printf("avail memory = %lu (%lu MB)\n", ptoa(vm_cnt.v_free_count), - ptoa(vm_cnt.v_free_count) / 1048576); + printf("avail memory = %ju (%ju MB)\n", + ptoa((uintmax_t)vm_cnt.v_free_count), + ptoa((uintmax_t)vm_cnt.v_free_count) / 1048576); /* * Set up buffers, so they can be used to read disk labels. diff --git a/sys/powerpc/powerpc/platform.c b/sys/powerpc/powerpc/platform.c index 4698ed2..8168758 100644 --- a/sys/powerpc/powerpc/platform.c +++ b/sys/powerpc/powerpc/platform.c @@ -86,8 +86,8 @@ static void memr_merge(struct mem_region *from, struct mem_region *to) { vm_offset_t end; - end = ulmax(to->mr_start + to->mr_size, from->mr_start + from->mr_size); - to->mr_start = ulmin(from->mr_start, to->mr_start); + end = uqmax(to->mr_start + to->mr_size, from->mr_start + from->mr_size); + to->mr_start = uqmin(from->mr_start, to->mr_start); to->mr_size = end - to->mr_start; } diff --git a/sys/powerpc/powerpc/pmap_dispatch.c b/sys/powerpc/powerpc/pmap_dispatch.c index 4e31a53..2082ba0 100644 --- a/sys/powerpc/powerpc/pmap_dispatch.c +++ b/sys/powerpc/powerpc/pmap_dispatch.c @@ -74,7 +74,7 @@ struct msgbuf *msgbufp; vm_offset_t msgbuf_phys; vm_offset_t kernel_vm_end; -vm_offset_t phys_avail[PHYS_AVAIL_SZ]; +vm_paddr_t phys_avail[PHYS_AVAIL_SZ]; vm_offset_t virtual_avail; vm_offset_t virtual_end; diff --git a/sys/security/audit/bsm_domain.c b/sys/security/audit/bsm_domain.c index cb5939f..8119956 100644 --- a/sys/security/audit/bsm_domain.c +++ b/sys/security/audit/bsm_domain.c @@ -25,8 +25,6 @@ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. - * - * P4: //depot/projects/trustedbsd/openbsm/libbsm/bsm_domain.c#3 */ #include <sys/cdefs.h> diff --git a/sys/security/audit/bsm_errno.c b/sys/security/audit/bsm_errno.c index e2e1961..bb746b5 100644 --- a/sys/security/audit/bsm_errno.c +++ b/sys/security/audit/bsm_errno.c @@ -25,8 +25,6 @@ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. - * - * P4: //depot/projects/trustedbsd/openbsm/libbsm/bsm_errno.c#22 */ #include <sys/cdefs.h> diff --git a/sys/security/audit/bsm_fcntl.c b/sys/security/audit/bsm_fcntl.c index 1e7e68f..5d27a3a 100644 --- a/sys/security/audit/bsm_fcntl.c +++ b/sys/security/audit/bsm_fcntl.c @@ -25,8 +25,6 @@ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. - * - * P4: //depot/projects/trustedbsd/openbsm/libbsm/bsm_fcntl.c#2 */ #include <sys/cdefs.h> diff --git a/sys/security/audit/bsm_socket_type.c b/sys/security/audit/bsm_socket_type.c index c049179..94c361c 100644 --- a/sys/security/audit/bsm_socket_type.c +++ b/sys/security/audit/bsm_socket_type.c @@ -25,8 +25,6 @@ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. - * - * P4: //depot/projects/trustedbsd/openbsm/libbsm/bsm_socket_type.c#1 */ #include <sys/cdefs.h> diff --git a/sys/security/audit/bsm_token.c b/sys/security/audit/bsm_token.c index 763d597..84f4fce 100644 --- a/sys/security/audit/bsm_token.c +++ b/sys/security/audit/bsm_token.c @@ -29,8 +29,6 @@ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. - * - * P4: //depot/projects/trustedbsd/openbsm/libbsm/bsm_token.c#99 */ #include <sys/cdefs.h> diff --git a/sys/sys/fcntl.h b/sys/sys/fcntl.h index 5793229..0467b2a 100644 --- a/sys/sys/fcntl.h +++ b/sys/sys/fcntl.h @@ -138,6 +138,11 @@ typedef __pid_t pid_t; */ #ifdef _KERNEL + +/* Only for devfs d_close() flags. */ +#define FLASTCLOSE O_DIRECTORY +#define FREVOKE O_VERIFY + /* convert from open() flags to/from fflags; convert O_RD/WR to FREAD/FWRITE */ #define FFLAGS(oflags) ((oflags) & O_EXEC ? (oflags) : (oflags) + 1) #define OFLAGS(fflags) ((fflags) & O_EXEC ? (fflags) : (fflags) - 1) diff --git a/sys/sys/gpt.h b/sys/sys/gpt.h index e62a803..3ef76b9 100644 --- a/sys/sys/gpt.h +++ b/sys/sys/gpt.h @@ -107,14 +107,18 @@ struct gpt_ent { * advantage might be. I can see how sharing swap partitions is advantageous * though. */ -#define GPT_ENT_TYPE_MS_RESERVED \ - {0xe3c9e316,0x0b5c,0x4db8,0x81,0x7d,{0xf9,0x2d,0xf0,0x02,0x15,0xae}} #define GPT_ENT_TYPE_MS_BASIC_DATA \ {0xebd0a0a2,0xb9e5,0x4433,0x87,0xc0,{0x68,0xb6,0xb7,0x26,0x99,0xc7}} -#define GPT_ENT_TYPE_MS_LDM_METADATA \ - {0x5808c8aa,0x7e8f,0x42e0,0x85,0xd2,{0xe1,0xe9,0x04,0x34,0xcf,0xb3}} #define GPT_ENT_TYPE_MS_LDM_DATA \ {0xaf9b60a0,0x1431,0x4f62,0xbc,0x68,{0x33,0x11,0x71,0x4a,0x69,0xad}} +#define GPT_ENT_TYPE_MS_LDM_METADATA \ + {0x5808c8aa,0x7e8f,0x42e0,0x85,0xd2,{0xe1,0xe9,0x04,0x34,0xcf,0xb3}} +#define GPT_ENT_TYPE_MS_RECOVERY \ + {0xde94bba4,0x06d1,0x4d40,0xa1,0x6a,{0xbf,0xd5,0x01,0x79,0xd6,0xac}} +#define GPT_ENT_TYPE_MS_RESERVED \ + {0xe3c9e316,0x0b5c,0x4db8,0x81,0x7d,{0xf9,0x2d,0xf0,0x02,0x15,0xae}} +#define GPT_ENT_TYPE_MS_SPACES \ + {0xe75caf8f,0xf680,0x4cee,0xaf,0xa3,{0xb0,0x01,0xe5,0x6e,0xfc,0x2d}} #define GPT_ENT_TYPE_LINUX_DATA \ {0x0fc63daf,0x8483,0x4772,0x8e,0x79,{0x3d,0x69,0xd8,0x47,0x7d,0xe4}} @@ -185,6 +189,18 @@ struct gpt_ent { #define GPT_ENT_TYPE_DRAGONFLY_HAMMER2 \ {0x5cbb9ad1,0x862d,0x11dc,0xa9,0x4d,{0x01,0x30,0x1b,0xb8,0xa9,0xf5}} +#define GPT_ENT_TYPE_CHROMEOS_FIRMWARE \ + {0xcab6e88e,0xabf3,0x4102,0xa0,0x7a,{0xd4,0xbb,0x9b,0xe3,0xc1,0xd3}} +#define GPT_ENT_TYPE_CHROMEOS_KERNEL \ + {0xfe3a2a5d,0x4f32,0x41a7,0xb7,0x25,{0xac,0xcc,0x32,0x85,0xa3,0x09}} +#define GPT_ENT_TYPE_CHROMEOS_RESERVED \ + {0x2e0a753d,0x9e48,0x43b0,0x83,0x37,{0xb1,0x51,0x92,0xcb,0x1b,0x5e}} +#define GPT_ENT_TYPE_CHROMEOS_ROOT \ + {0x3cb8e202,0x3b7e,0x47dd,0x8a,0x3c,{0x7f,0xf2,0xa1,0x3c,0xfc,0xec}} + +#define GPT_ENT_TYPE_OPENBSD_DATA \ + {0x824cc7a0,0x36a8,0x11e3,0x89,0x0a,{0x95,0x25,0x19,0xad,0x3f,0x61}} + /* * Boot partition used by GRUB 2. */ diff --git a/sys/sys/libkern.h b/sys/sys/libkern.h index 7484cdc..efbaa4a 100644 --- a/sys/sys/libkern.h +++ b/sys/sys/libkern.h @@ -61,6 +61,8 @@ static __inline u_int max(u_int a, u_int b) { return (a > b ? a : b); } static __inline u_int min(u_int a, u_int b) { return (a < b ? a : b); } static __inline quad_t qmax(quad_t a, quad_t b) { return (a > b ? a : b); } static __inline quad_t qmin(quad_t a, quad_t b) { return (a < b ? a : b); } +static __inline u_quad_t uqmax(u_quad_t a, u_quad_t b) { return (a > b ? a : b); } +static __inline u_quad_t uqmin(u_quad_t a, u_quad_t b) { return (a < b ? a : b); } static __inline u_long ulmax(u_long a, u_long b) { return (a > b ? a : b); } static __inline u_long ulmin(u_long a, u_long b) { return (a < b ? a : b); } static __inline off_t omax(off_t a, off_t b) { return (a > b ? a : b); } diff --git a/sys/sys/param.h b/sys/sys/param.h index f196dd6..69eb391 100644 --- a/sys/sys/param.h +++ b/sys/sys/param.h @@ -58,7 +58,7 @@ * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 1100091 /* Master, propagated to newvers */ +#define __FreeBSD_version 1100092 /* Master, propagated to newvers */ /* * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD, diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index f6ee661..04ea39c 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -13301,43 +13301,43 @@ softdep_ast_cleanup_proc(void) bool req; td = curthread; - mp = td->td_su; - if (mp == NULL) - return; - td->td_su = NULL; - error = vfs_busy(mp, MBF_NOWAIT); - vfs_rel(mp); - if (error != 0) - return; - if (ffs_own_mount(mp) && MOUNTEDSOFTDEP(mp)) { - ump = VFSTOUFS(mp); - for (;;) { - req = false; - ACQUIRE_LOCK(ump); - if (softdep_excess_items(ump, D_INODEDEP)) { - req = true; - request_cleanup(mp, FLUSH_INODES); - } - if (softdep_excess_items(ump, D_DIRREM)) { - req = true; - request_cleanup(mp, FLUSH_BLOCKS); - } - FREE_LOCK(ump); - if (softdep_excess_items(ump, D_NEWBLK) || - softdep_excess_items(ump, D_ALLOCDIRECT) || - softdep_excess_items(ump, D_ALLOCINDIR)) { - error = vn_start_write(NULL, &mp, V_WAIT); - if (error == 0) { + while ((mp = td->td_su) != NULL) { + td->td_su = NULL; + error = vfs_busy(mp, MBF_NOWAIT); + vfs_rel(mp); + if (error != 0) + return; + if (ffs_own_mount(mp) && MOUNTEDSOFTDEP(mp)) { + ump = VFSTOUFS(mp); + for (;;) { + req = false; + ACQUIRE_LOCK(ump); + if (softdep_excess_items(ump, D_INODEDEP)) { + req = true; + request_cleanup(mp, FLUSH_INODES); + } + if (softdep_excess_items(ump, D_DIRREM)) { req = true; - VFS_SYNC(mp, MNT_WAIT); - vn_finished_write(mp); + request_cleanup(mp, FLUSH_BLOCKS); } + FREE_LOCK(ump); + if (softdep_excess_items(ump, D_NEWBLK) || + softdep_excess_items(ump, D_ALLOCDIRECT) || + softdep_excess_items(ump, D_ALLOCINDIR)) { + error = vn_start_write(NULL, &mp, + V_WAIT); + if (error == 0) { + req = true; + VFS_SYNC(mp, MNT_WAIT); + vn_finished_write(mp); + } + } + if ((td->td_pflags & TDP_KTHREAD) != 0 || !req) + break; } - if ((td->td_pflags & TDP_KTHREAD) != 0 || !req) - break; } + vfs_unbusy(mp); } - vfs_unbusy(mp); } /* diff --git a/sys/vm/uma.h b/sys/vm/uma.h index d3e0658..d218e60 100644 --- a/sys/vm/uma.h +++ b/sys/vm/uma.h @@ -521,6 +521,19 @@ int uma_zone_get_max(uma_zone_t zone); void uma_zone_set_warning(uma_zone_t zone, const char *warning); /* + * Sets a function to run when limit is reached + * + * Arguments: + * zone The zone to which this applies + * fx The function ro run + * + * Returns: + * Nothing + */ +typedef void (*uma_maxaction_t)(uma_zone_t); +void uma_zone_set_maxaction(uma_zone_t zone, uma_maxaction_t); + +/* * Obtains the approximate current number of items allocated from a zone * * Arguments: diff --git a/sys/vm/uma_core.c b/sys/vm/uma_core.c index 3a0a799..4600589 100644 --- a/sys/vm/uma_core.c +++ b/sys/vm/uma_core.c @@ -431,6 +431,13 @@ zone_log_warning(uma_zone_t zone) printf("[zone: %s] %s\n", zone->uz_name, zone->uz_warning); } +static inline void +zone_maxaction(uma_zone_t zone) +{ + if (zone->uz_maxaction) + (*zone->uz_maxaction)(zone); +} + static void zone_foreach_keg(uma_zone_t zone, void (*kegfn)(uma_keg_t)) { @@ -1578,6 +1585,7 @@ zone_ctor(void *mem, int size, void *udata, int flags) zone->uz_flags = 0; zone->uz_warning = NULL; timevalclear(&zone->uz_ratecheck); + zone->uz_maxaction = NULL; keg = arg->keg; ZONE_LOCK_INIT(zone, (arg->flags & UMA_ZONE_MTXCLASS)); @@ -2382,6 +2390,7 @@ keg_fetch_slab(uma_keg_t keg, uma_zone_t zone, int flags) if ((zone->uz_flags & UMA_ZFLAG_MULTI) == 0) { zone->uz_flags |= UMA_ZFLAG_FULL; zone_log_warning(zone); + zone_maxaction(zone); } if (flags & M_NOWAIT) break; @@ -2501,6 +2510,7 @@ zone_fetch_slab_multi(uma_zone_t zone, uma_keg_t last, int rflags) zone->uz_flags |= UMA_ZFLAG_FULL; zone->uz_sleeps++; zone_log_warning(zone); + zone_maxaction(zone); msleep(zone, zone->uz_lockptr, PVM, "zonelimit", hz/100); zone->uz_flags &= ~UMA_ZFLAG_FULL; @@ -3007,6 +3017,16 @@ uma_zone_set_warning(uma_zone_t zone, const char *warning) } /* See uma.h */ +void +uma_zone_set_maxaction(uma_zone_t zone, uma_maxaction_t maxaction) +{ + + ZONE_LOCK(zone); + zone->uz_maxaction = maxaction; + ZONE_UNLOCK(zone); +} + +/* See uma.h */ int uma_zone_get_cur(uma_zone_t zone) { diff --git a/sys/vm/uma_int.h b/sys/vm/uma_int.h index ad2a405..5d7ecd3 100644 --- a/sys/vm/uma_int.h +++ b/sys/vm/uma_int.h @@ -303,10 +303,12 @@ struct uma_zone { uint16_t uz_count; /* Amount of items in full bucket */ uint16_t uz_count_min; /* Minimal amount of items there */ - /* The next three fields are used to print a rate-limited warnings. */ + /* The next two fields are used to print a rate-limited warnings. */ const char *uz_warning; /* Warning to print on failure */ struct timeval uz_ratecheck; /* Warnings rate-limiting */ + uma_maxaction_t uz_maxaction; /* Function to run when at limit */ + /* * This HAS to be the last item because we adjust the zone size * based on NCPU and then allocate the space for the zones. diff --git a/sys/vm/vnode_pager.c b/sys/vm/vnode_pager.c index ff30f4d..66dd29d 100644 --- a/sys/vm/vnode_pager.c +++ b/sys/vm/vnode_pager.c @@ -806,6 +806,7 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_page_t *m, int count, * than a page size, then use special small filesystem code. */ if (pagesperblock == 0) { + relpbuf(bp, freecnt); for (i = 0; i < count; i++) { PCPU_INC(cnt.v_vnodein); PCPU_INC(cnt.v_vnodepgsin); diff --git a/sys/x86/include/cputypes.h b/sys/x86/include/cputypes.h new file mode 100644 index 0000000..ca6ce83 --- /dev/null +++ b/sys/x86/include/cputypes.h @@ -0,0 +1,54 @@ +/*- + * Copyright (c) 1993 Christopher G. Demetriou + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _X86_CPUTYPES_H_ +#define _X86_CPUTYPES_H_ + +/* + * Vendors of processor. + */ +#define CPU_VENDOR_NSC 0x100b /* NSC */ +#define CPU_VENDOR_IBM 0x1014 /* IBM */ +#define CPU_VENDOR_AMD 0x1022 /* AMD */ +#define CPU_VENDOR_SIS 0x1039 /* SiS */ +#define CPU_VENDOR_UMC 0x1060 /* UMC */ +#define CPU_VENDOR_NEXGEN 0x1074 /* Nexgen */ +#define CPU_VENDOR_CYRIX 0x1078 /* Cyrix */ +#define CPU_VENDOR_IDT 0x111d /* Centaur/IDT/VIA */ +#define CPU_VENDOR_TRANSMETA 0x1279 /* Transmeta */ +#define CPU_VENDOR_INTEL 0x8086 /* Intel */ +#define CPU_VENDOR_RISE 0xdead2bad /* Rise */ +#define CPU_VENDOR_CENTAUR CPU_VENDOR_IDT + +#ifndef LOCORE +extern int cpu; +extern int cpu_class; +#endif + +#endif /* !_X86_CPUTYPES_H_ */ diff --git a/sys/x86/include/specialreg.h b/sys/x86/include/specialreg.h index e88a4d7..b5274b2 100644 --- a/sys/x86/include/specialreg.h +++ b/sys/x86/include/specialreg.h @@ -342,15 +342,20 @@ #define CPUID_STDEXT_RTM 0x00000800 #define CPUID_STDEXT_MPX 0x00004000 #define CPUID_STDEXT_AVX512F 0x00010000 +#define CPUID_STDEXT_AVX512DQ 0x00020000 #define CPUID_STDEXT_RDSEED 0x00040000 #define CPUID_STDEXT_ADX 0x00080000 #define CPUID_STDEXT_SMAP 0x00100000 +#define CPUID_STDEXT_AVX512IFMA 0x00200000 +#define CPUID_STDEXT_PCOMMIT 0x00400000 #define CPUID_STDEXT_CLFLUSHOPT 0x00800000 +#define CPUID_STDEXT_CLWB 0x01000000 #define CPUID_STDEXT_PROCTRACE 0x02000000 #define CPUID_STDEXT_AVX512PF 0x04000000 #define CPUID_STDEXT_AVX512ER 0x08000000 #define CPUID_STDEXT_AVX512CD 0x10000000 #define CPUID_STDEXT_SHA 0x20000000 +#define CPUID_STDEXT_AVX512BW 0x40000000 /* * CPUID manufacturers identifiers diff --git a/sys/x86/x86/identcpu.c b/sys/x86/x86/identcpu.c index ae72121..902af6b 100644 --- a/sys/x86/x86/identcpu.c +++ b/sys/x86/x86/identcpu.c @@ -67,6 +67,10 @@ __FBSDID("$FreeBSD$"); #include <x86/vmware.h> #ifdef __i386__ +#if !defined(CPU_DISABLE_SSE) && defined(I686_CPU) +#define CPU_ENABLE_SSE +#endif + #define IDENTBLUE_CYRIX486 0 #define IDENTBLUE_IBMCPU 1 #define IDENTBLUE_CYRIXM2 2 @@ -83,9 +87,43 @@ static void print_svm_info(void); static void print_via_padlock_info(void); static void print_vmx_info(void); +int cpu; /* Are we 386, 386sx, 486, etc? */ int cpu_class; +u_int cpu_feature; /* Feature flags */ +u_int cpu_feature2; /* Feature flags */ +u_int amd_feature; /* AMD feature flags */ +u_int amd_feature2; /* AMD feature flags */ +u_int amd_pminfo; /* AMD advanced power management info */ +u_int via_feature_rng; /* VIA RNG features */ +u_int via_feature_xcrypt; /* VIA ACE features */ +u_int cpu_high; /* Highest arg to CPUID */ +u_int cpu_exthigh; /* Highest arg to extended CPUID */ +u_int cpu_id; /* Stepping ID */ +u_int cpu_procinfo; /* HyperThreading Info / Brand Index / CLFUSH */ +u_int cpu_procinfo2; /* Multicore info */ +char cpu_vendor[20]; /* CPU Origin code */ +u_int cpu_vendor_id; /* CPU vendor ID */ +#if defined(__amd64__) || defined(CPU_ENABLE_SSE) +u_int cpu_fxsr; /* SSE enabled */ +u_int cpu_mxcsr_mask; /* Valid bits in mxcsr */ +#endif +u_int cpu_clflush_line_size = 32; +u_int cpu_stdext_feature; +u_int cpu_stdext_feature2; +u_int cpu_max_ext_state_size; +u_int cpu_mon_mwait_flags; /* MONITOR/MWAIT flags (CPUID.05H.ECX) */ +u_int cpu_mon_min_size; /* MONITOR minimum range size, bytes */ +u_int cpu_mon_max_size; /* MONITOR minimum range size, bytes */ +u_int cpu_maxphyaddr; /* Max phys addr width in bits */ char machine[] = MACHINE; +SYSCTL_UINT(_hw, OID_AUTO, via_feature_rng, CTLFLAG_RD, + &via_feature_rng, 0, + "VIA RNG feature available in CPU"); +SYSCTL_UINT(_hw, OID_AUTO, via_feature_xcrypt, CTLFLAG_RD, + &via_feature_xcrypt, 0, + "VIA xcrypt feature available in CPU"); + #ifdef __amd64__ #ifdef SCTL_MASK32 extern int adaptive_machine_arch; @@ -910,18 +948,23 @@ printcpuinfo(void) "\017MPX" /* AVX512 Foundation */ "\021AVX512F" + "\022AVX512DQ" /* Enhanced NRBG */ "\023RDSEED" /* ADCX + ADOX */ "\024ADX" /* Supervisor Mode Access Prevention */ "\025SMAP" + "\026AVX512IFMA" + "\027PCOMMIT" "\030CLFLUSHOPT" + "\031CLWB" "\032PROCTRACE" "\033AVX512PF" "\034AVX512ER" "\035AVX512CD" "\036SHA" + "\037AVX512BW" ); } @@ -930,6 +973,7 @@ printcpuinfo(void) cpu_stdext_feature2, "\020" "\001PREFETCHWT1" + "\002AVX512VBMI" "\004PKU" "\005OSPKE" ); diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c index 50d2e76..58eb736 100644 --- a/sys/x86/xen/pv.c +++ b/sys/x86/xen/pv.c @@ -96,13 +96,8 @@ static int xen_pv_start_all_aps(void); /*---------------------------- Extern Declarations ---------------------------*/ #ifdef SMP /* Variables used by amd64 mp_machdep to start APs */ -extern struct mtx ap_boot_mtx; -extern void *bootstacks[]; extern char *doublefault_stack; extern char *nmi_stack; -extern void *dpcpu; -extern int bootAP; -extern char *bootSTK; #endif /* diff --git a/sys/x86/xen/xen_apic.c b/sys/x86/xen/xen_apic.c index a65f3b5..9400378 100644 --- a/sys/x86/xen/xen_apic.c +++ b/sys/x86/xen/xen_apic.c @@ -74,12 +74,6 @@ static driver_filter_t xen_cpususpend_handler; static driver_filter_t xen_cpustophard_handler; #endif -/*---------------------------- Extern Declarations ---------------------------*/ -/* Variables used by mp_machdep to perform the MMU related IPIs */ -#ifdef __amd64__ -extern int pmap_pcid_enabled; -#endif - /*---------------------------------- Macros ----------------------------------*/ #define IPI_TO_IDX(ipi) ((ipi) - APIC_IPI_INTS) diff --git a/tests/sys/Makefile b/tests/sys/Makefile index 3adf81e..702f664 100644 --- a/tests/sys/Makefile +++ b/tests/sys/Makefile @@ -10,6 +10,7 @@ TESTS_SUBDIRS+= fifo TESTS_SUBDIRS+= file TESTS_SUBDIRS+= kern TESTS_SUBDIRS+= kqueue +TESTS_SUBDIRS+= mac TESTS_SUBDIRS+= mqueue TESTS_SUBDIRS+= netinet TESTS_SUBDIRS+= opencrypto diff --git a/tests/sys/aio/aio_kqueue_test.c b/tests/sys/aio/aio_kqueue_test.c index 14e4729..97c2c38f 100644 --- a/tests/sys/aio/aio_kqueue_test.c +++ b/tests/sys/aio/aio_kqueue_test.c @@ -62,7 +62,10 @@ main (int argc, char *argv[]) struct kevent ke, kq_returned; struct timespec ts; char buffer[32768]; - int cancel, error, failed = 0, fd, kq, pending, result, run; +#ifdef DEBUG + int cancel, error; +#endif + int failed = 0, fd, kq, pending, result, run; int tmp_file = 0; unsigned i, j; @@ -96,19 +99,19 @@ main (int argc, char *argv[]) if (iocb[i] == NULL) err(1, "calloc"); } - - pending = 0; + + pending = 0; for (i = 0; i < nitems(iocb); i++) { pending++; iocb[i]->aio_nbytes = sizeof(buffer); iocb[i]->aio_buf = buffer; iocb[i]->aio_fildes = fd; iocb[i]->aio_offset = iocb[i]->aio_nbytes * i * run; - + iocb[i]->aio_sigevent.sigev_notify_kqueue = kq; iocb[i]->aio_sigevent.sigev_value.sival_ptr = iocb[i]; iocb[i]->aio_sigevent.sigev_notify = SIGEV_KEVENT; - + result = aio_write(iocb[i]); if (result != 0) { perror("aio_write"); @@ -133,7 +136,9 @@ main (int argc, char *argv[]) } } } +#ifdef DEBUG cancel = nitems(iocb) - pending; +#endif i = 0; while (pending) { @@ -144,34 +149,36 @@ main (int argc, char *argv[]) bzero(&kq_returned, sizeof(ke)); ts.tv_sec = 0; ts.tv_nsec = 1; - result = kevent(kq, NULL, 0, + result = kevent(kq, NULL, 0, &kq_returned, 1, &ts); +#ifdef DEBUG error = errno; +#endif if (result < 0) perror("kevent error: "); kq_iocb = kq_returned.udata; #ifdef DEBUG printf("kevent %d %d errno %d return.ident %p " - "return.data %p return.udata %p %p\n", - i, result, error, - kq_returned.ident, kq_returned.data, - kq_returned.udata, + "return.data %p return.udata %p %p\n", + i, result, error, + kq_returned.ident, kq_returned.data, + kq_returned.udata, kq_iocb); #endif - + if (kq_iocb) break; #ifdef DEBUG printf("Try again left %d out of %d %d\n", pending, nitems(iocb), cancel); #endif - } - + } + for (j = 0; j < nitems(iocb) && iocb[j] != kq_iocb; j++) ; #ifdef DEBUG printf("kq_iocb %p\n", kq_iocb); - + printf("Error Result for %d is %d pending %d\n", j, result, pending); #endif @@ -192,7 +199,7 @@ main (int argc, char *argv[]) iocb[j] = NULL; pending--; i++; - } + } for (i = 0; i < nitems(iocb); i++) free(iocb[i]); diff --git a/tests/sys/aio/lio_kqueue_test.c b/tests/sys/aio/lio_kqueue_test.c index 5cc87b3..e69b9c9 100644 --- a/tests/sys/aio/lio_kqueue_test.c +++ b/tests/sys/aio/lio_kqueue_test.c @@ -42,6 +42,7 @@ #include <sys/time.h> #include <aio.h> #include <fcntl.h> +#include <err.h> #include <errno.h> #include <stdio.h> #include <stdlib.h> @@ -57,26 +58,26 @@ #define MAX_RUNS 300 int -main(int argc, char *argv[]){ +main(int argc, char *argv[]) +{ int fd; struct aiocb *iocb[MAX_IOCBS]; - struct aiocb **lio[LIO_MAX], **lio_element, **kq_lio; + struct aiocb **lio[LIO_MAX], **kq_lio; int i, result, run, error, j, k; char buffer[32768]; - int kq = kqueue(); + int kq; struct kevent ke, kq_returned; struct timespec ts; struct sigevent sig; time_t time1, time2; - char *file, pathname[sizeof(PATH_TEMPLATE)-1]; + char *file, pathname[sizeof(PATH_TEMPLATE)]; int tmp_file = 0, failed = 0; PLAIN_REQUIRE_KERNEL_MODULE("aio", 0); - if (kq < 0) { - perror("No kqeueue\n"); - exit(1); - } + kq = kqueue(); + if (kq < 0) + err(1, "kqeueue(2) failed"); if (argc == 1) { strcpy(pathname, PATH_TEMPLATE); @@ -87,34 +88,29 @@ main(int argc, char *argv[]){ file = argv[1]; fd = open(file, O_RDWR|O_CREAT, 0666); } - if (fd < 0){ - fprintf(stderr, "Can't open %s\n", argv[1]); - perror(""); - exit(1); - } + if (fd < 0) + err(1, "can't open %s", argv[1]); #ifdef DEBUG printf("Hello kq %d fd %d\n", kq, fd); #endif - for (run = 0; run < MAX_RUNS; run++){ + for (run = 0; run < MAX_RUNS; run++) { #ifdef DEBUG printf("Run %d\n", run); #endif for (j = 0; j < LIO_MAX; j++) { - lio[j] = (struct aiocb **) + lio[j] = malloc(sizeof(struct aiocb *) * MAX_IOCBS/LIO_MAX); - for(i = 0; i < MAX_IOCBS / LIO_MAX; i++) { + for (i = 0; i < MAX_IOCBS / LIO_MAX; i++) { k = (MAX_IOCBS / LIO_MAX * j) + i; - lio_element = lio[j]; - lio[j][i] = iocb[k] = (struct aiocb *) - malloc(sizeof(struct aiocb)); - bzero(iocb[k], sizeof(struct aiocb)); + lio[j][i] = iocb[k] = + calloc(1, sizeof(struct aiocb)); iocb[k]->aio_nbytes = sizeof(buffer); iocb[k]->aio_buf = buffer; iocb[k]->aio_fildes = fd; - iocb[k]->aio_offset - = iocb[k]->aio_nbytes * k * (run + 1); + iocb[k]->aio_offset + = iocb[k]->aio_nbytes * k * (run + 1); #ifdef DEBUG printf("hello iocb[k] %d\n", @@ -131,27 +127,26 @@ main(int argc, char *argv[]){ error = errno; time(&time2); #ifdef DEBUG - printf("Time %d %d %d result -> %d\n", + printf("Time %d %d %d result -> %d\n", time1, time2, time2-time1, result); #endif if (result != 0) { errno = error; - perror("list_listio"); - printf("FAIL: Result %d iteration %d\n",result, j); - exit(1); + err(1, "FAIL: Result %d iteration %d\n", + result, j); } #ifdef DEBUG printf("write %d is at %p\n", j, lio[j]); #endif } - for(i = 0; i < LIO_MAX; i++) { - for(j = LIO_MAX - 1; j >=0; j--) { + for (i = 0; i < LIO_MAX; i++) { + for (j = LIO_MAX - 1; j >=0; j--) { if (lio[j]) break; } - for(;;) { + for (;;) { bzero(&ke, sizeof(ke)); bzero(&kq_returned, sizeof(ke)); ts.tv_sec = 0; @@ -159,9 +154,9 @@ main(int argc, char *argv[]){ #ifdef DEBUG printf("FOO lio %d -> %p\n", j, lio[j]); #endif - EV_SET(&ke, (uintptr_t)lio[j], + EV_SET(&ke, (uintptr_t)lio[j], EVFILT_LIO, EV_ONESHOT, 0, 0, iocb[j]); - result = kevent(kq, NULL, 0, + result = kevent(kq, NULL, 0, &kq_returned, 1, &ts); error = errno; if (result < 0) { @@ -170,14 +165,14 @@ main(int argc, char *argv[]){ kq_lio = kq_returned.udata; #ifdef DEBUG printf("kevent %d %d errno %d return.ident %p " - "return.data %p return.udata %p %p\n", - i, result, error, - kq_returned.ident, kq_returned.data, - kq_returned.udata, + "return.data %p return.udata %p %p\n", + i, result, error, + kq_returned.ident, kq_returned.data, + kq_returned.udata, lio[j]); #endif - if(kq_lio) + if (kq_lio) break; #ifdef DEBUG printf("Try again\n"); @@ -189,25 +184,21 @@ main(int argc, char *argv[]){ #endif for (j = 0; j < LIO_MAX; j++) { - if (lio[j] == kq_lio) { + if (lio[j] == kq_lio) break; - } - } - if (j == LIO_MAX) { - printf("FAIL:\n"); - exit(1); } + if (j == LIO_MAX) + errx(1, "FAIL: "); #ifdef DEBUG printf("Error Result for %d is %d\n", j, result); #endif if (result < 0) { printf("FAIL: run %d, operation %d result %d \n", run, LIO_MAX - i -1, result); - failed = 1; - } else { + failed++; + } else printf("PASS: run %d, operation %d result %d \n", run, LIO_MAX - i -1, result); - } - for(k = 0; k < MAX_IOCBS / LIO_MAX; k++){ + for (k = 0; k < MAX_IOCBS / LIO_MAX; k++) { result = aio_return(kq_lio[k]); #ifdef DEBUG printf("Return Resulto for %d %d is %d\n", j, k, result); @@ -224,9 +215,8 @@ main(int argc, char *argv[]){ printf("\n"); #endif - for(k = 0; k < MAX_IOCBS / LIO_MAX; k++) { + for (k = 0; k < MAX_IOCBS / LIO_MAX; k++) free(lio[j][k]); - } free(lio[j]); lio[j] = NULL; } @@ -235,15 +225,12 @@ main(int argc, char *argv[]){ printf("Done\n"); #endif - if (tmp_file) { + if (tmp_file) unlink(pathname); - } - if (failed) { - printf("FAIL: Atleast one\n"); - exit(1); - } else { - printf("PASS: All\n"); - exit(0); - } + if (failed) + errx(1, "FAIL: %d testcases failed", failed); + else + errx(0, "PASS: All\n"); + } diff --git a/tests/sys/file/flock_test.sh b/tests/sys/file/flock_test.sh index f963cde..ead4ff1 100755 --- a/tests/sys/file/flock_test.sh +++ b/tests/sys/file/flock_test.sh @@ -43,10 +43,11 @@ for n in `seq 1 $last_testcase`; do todomsg=" # TODO: racy testcase" fi - $(dirname $0)/flock_helper . $n | grep -q SUCCEED - if [ $? -eq 0 ]; then + output=$($(dirname $0)/flock_helper . $n) + if echo "$output" | grep -q SUCCEED; then echo "ok $n$todomsg" else echo "not ok $n$todomsg" + echo "$output" >&2 fi done diff --git a/tests/sys/kern/pipe/pipe_overcommit1_test.c b/tests/sys/kern/pipe/pipe_overcommit1_test.c index f8f881d..4e40be7 100644 --- a/tests/sys/kern/pipe/pipe_overcommit1_test.c +++ b/tests/sys/kern/pipe/pipe_overcommit1_test.c @@ -40,12 +40,11 @@ int main(void) { - int pipes[10000], returnval; + int pipes[10000]; unsigned int i; - for (i = 0; i < nitems(pipes); i++) { - returnval = pipe(&pipes[i]); - } + for (i = 0; i < nitems(pipes); i++) + (void)pipe(&pipes[i]); printf("PASS\n"); exit(0); diff --git a/tests/sys/kern/unix_seqpacket_test.c b/tests/sys/kern/unix_seqpacket_test.c index 986b70e..7305ab3 100644 --- a/tests/sys/kern/unix_seqpacket_test.c +++ b/tests/sys/kern/unix_seqpacket_test.c @@ -47,7 +47,7 @@ static void do_socketpair(int *sv) { int s; - + s = socketpair(PF_LOCAL, SOCK_SEQPACKET, 0, sv); ATF_REQUIRE_EQ(0, s); ATF_REQUIRE(sv[0] >= 0); @@ -59,7 +59,7 @@ static void do_socketpair_nonblocking(int *sv) { int s; - + s = socketpair(PF_LOCAL, SOCK_SEQPACKET, 0, sv); ATF_REQUIRE_EQ(0, s); ATF_REQUIRE(sv[0] >= 0); @@ -69,7 +69,7 @@ do_socketpair_nonblocking(int *sv) ATF_REQUIRE(-1 != fcntl(sv[1], F_SETFL, O_NONBLOCK)); } -/* +/* * Returns a pair of sockets made the hard way: bind, listen, connect & accept * @return const char* The path to the socket */ @@ -100,7 +100,7 @@ mk_pair_of_sockets(int *sv) perror("connect"); atf_tc_fail("connect(2) failed"); } - + /* Accept it */ s1 = accept(s, NULL, NULL); if (s1 == -1) { @@ -239,7 +239,7 @@ test_pipe_simulator(size_t sndbufsize, size_t rcvbufsize) memset(sndbuf, num_sent, pktsize); ssize = send(sv[0], sndbuf, pktsize, MSG_EOR); if (ssize < 0) { - /* + /* * XXX: This is bug-compatible with the kernel. * The kernel returns EMSGSIZE when it should * return EAGAIN @@ -275,7 +275,7 @@ test_pipe_simulator(size_t sndbufsize, size_t rcvbufsize) pktsize, rsize); memset(comparebuf, num_received, pktsize); ATF_CHECK_EQ_MSG(0, memcmp(comparebuf, rcvbuf, - pktsize), + pktsize), "Received data miscompare"); num_received++; } @@ -333,7 +333,7 @@ test_pipe_reader(void* args) "expected %zd=send(...) but got %zd", td->pktsize, rsize); d = memcmp(comparebuf, rcvbuf, td->pktsize); - ATF_CHECK_EQ_MSG(0, d, + ATF_CHECK_EQ_MSG(0, d, "Received data miscompare on packet %d", i); } return (0); @@ -369,7 +369,7 @@ test_pipe(size_t sndbufsize, size_t rcvbufsize) reader_data.so = sv[1]; ATF_REQUIRE_EQ(0, pthread_create(&writer, NULL, test_pipe_writer, (void*)&writer_data)); - /* + /* * Give the writer time to start writing, and hopefully block, before * starting the reader. This increases the likelihood of the test case * failing due to PR kern/185812 @@ -561,7 +561,7 @@ ATF_TC_BODY(resize_buffers, tc) ATF_CHECK_EQ(0, getsockopt(s, SOL_SOCKET, SO_SNDBUF, &xs, &sl)); ATF_CHECK_EQ(0, getsockopt(s, SOL_SOCKET, SO_RCVBUF, &xr, &sl)); printf("After changing SNDBUF | %7d | %7d |\n", xs, xr); - + if (setsockopt(s, SOL_SOCKET, SO_RCVBUF, &rcvbuf, sizeof(rcvbuf)) != 0){ perror("setsockopt"); atf_tc_fail("setsockopt(SO_RCVBUF) failed"); @@ -671,7 +671,9 @@ ATF_TC_BODY(send_recv, tc) ATF_TC_WITHOUT_HEAD(sendto_recvfrom); ATF_TC_BODY(sendto_recvfrom, tc) { +#ifdef TEST_SEQ_PACKET_SOURCE_ADDRESS const char* path; +#endif struct sockaddr_storage from; int sv[2]; const int bufsize = 64; @@ -682,7 +684,10 @@ ATF_TC_BODY(sendto_recvfrom, tc) socklen_t fromlen; /* setup the socket pair */ - path = mk_pair_of_sockets(sv); +#ifdef TEST_SEQ_PACKET_SOURCE_ADDRESS + path = +#endif + mk_pair_of_sockets(sv); /* send and receive a small packet */ datalen = strlen(data) + 1; /* +1 for the null */ @@ -703,19 +708,21 @@ ATF_TC_BODY(sendto_recvfrom, tc) } ATF_CHECK_EQ(datalen, rsize); - /* +#ifdef TEST_SEQ_PACKET_SOURCE_ADDRESS + /* * FreeBSD does not currently provide the source address for SEQ_PACKET * AF_UNIX sockets, and POSIX does not require it, so these two checks * are disabled. If FreeBSD gains that feature in the future, then * these checks may be reenabled */ - /* ATF_CHECK_EQ(PF_LOCAL, from.ss_family); */ - /* ATF_CHECK_STREQ(path, ((struct sockaddr_un*)&from)->sun_path); */ + ATF_CHECK_EQ(PF_LOCAL, from.ss_family); + ATF_CHECK_STREQ(path, ((struct sockaddr_un*)&from)->sun_path); +#endif close(sv[0]); close(sv[1]); } -/* +/* * send(2) and recv(2) a single short record with sockets created the * traditional way, involving bind, listen, connect, and accept */ @@ -795,7 +802,6 @@ ATF_TC_BODY(shutdown_send_sigpipe, tc) /* ATF's isolation mechanisms will guarantee uniqueness of this file */ const char *path = "sock"; const char *data = "data"; - ssize_t ssize; int s, err, s2; s = socket(PF_LOCAL, SOCK_SEQPACKET, 0); @@ -820,7 +826,7 @@ ATF_TC_BODY(shutdown_send_sigpipe, tc) ATF_CHECK_EQ(0, shutdown(s2, SHUT_RDWR)); ATF_REQUIRE(SIG_ERR != signal(SIGPIPE, shutdown_send_sigpipe_handler)); - ssize = send(s2, data, sizeof(data), MSG_EOR); + (void)send(s2, data, sizeof(data), MSG_EOR); ATF_CHECK_EQ(1, got_sigpipe); close(s); close(s2); @@ -861,7 +867,7 @@ ATF_TC_BODY(send_recv_nonblocking, tc) close(sv[1]); } -/* +/* * We should get EMSGSIZE if we try to send a message larger than the socket * buffer, with blocking sockets */ @@ -890,7 +896,7 @@ ATF_TC_BODY(emsgsize, tc) close(sv[1]); } -/* +/* * We should get EMSGSIZE if we try to send a message larger than the socket * buffer, with nonblocking sockets */ @@ -920,7 +926,7 @@ ATF_TC_BODY(emsgsize_nonblocking, tc) } -/* +/* * We should get EAGAIN if we try to send a message larger than the socket * buffer, with nonblocking sockets. Test with several different sockbuf sizes */ @@ -946,7 +952,7 @@ ATF_TC_BODY(eagain_128k_128k, tc) } -/* +/* * nonblocking send(2) and recv(2) of several records, which should collectively * fill up the send buffer but not the receive buffer */ @@ -971,7 +977,7 @@ ATF_TC_BODY(rcvbuf_oversized, tc) ATF_REQUIRE_EQ(0, setsockopt(sv[1], SOL_SOCKET, SO_RCVBUF, &rcvbufsize, sizeof(rcvbufsize))); - /* + /* * Send and receive packets that are collectively greater than the send * buffer, but less than the receive buffer */ @@ -999,7 +1005,7 @@ ATF_TC_BODY(rcvbuf_oversized, tc) "expected %zd=send(...) but got %zd", pktsize, rsize); /* Verify the contents */ - ATF_CHECK_EQ_MSG(0, memcmp(sndbuf, recv_buf, pktsize), + ATF_CHECK_EQ_MSG(0, memcmp(sndbuf, recv_buf, pktsize), "Received data miscompare"); } @@ -1011,7 +1017,7 @@ ATF_TC_BODY(rcvbuf_oversized, tc) close(sv[1]); } -/* +/* * Simulate the behavior of a blocking pipe. The sender will send until his * buffer fills up, then we'll simulate a scheduler switch that will allow the * receiver to read until his buffer empties. Repeat the process until the @@ -1042,7 +1048,7 @@ ATF_TC_BODY(pipe_simulator_128k_128k, tc) test_pipe_simulator(131072, 131072); } -/* +/* * Test blocking I/O by passing data between two threads. The total amount of * data will be >> buffer size to force blocking. Repeat the test with multiple * send and receive buffer sizes diff --git a/tests/sys/mac/Makefile b/tests/sys/mac/Makefile new file mode 100644 index 0000000..ae2c491 --- /dev/null +++ b/tests/sys/mac/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +TESTSDIR= ${TESTSBASE}/sys/mac + +TESTS_SUBDIRS+= bsdextended +TESTS_SUBDIRS+= portacl + +.include <bsd.test.mk> diff --git a/tests/sys/mac/bsdextended/Makefile b/tests/sys/mac/bsdextended/Makefile new file mode 100644 index 0000000..9d0b6f6 --- /dev/null +++ b/tests/sys/mac/bsdextended/Makefile @@ -0,0 +1,13 @@ +# $FreeBSD$ + +TESTSDIR= ${TESTSBASE}/sys/mac/bsdextended + +TAP_TESTS_C+= ugidfw_test +TAP_TESTS_SH+= matches_test + +LIBADD.ugidfw_test+= ugidfw + +TEST_METADATA.matches_test+= required_user="root" +TEST_METADATA.ugidfw_test+= required_user="root" + +.include <bsd.test.mk> diff --git a/tests/sys/mac/bsdextended/matches_test.sh b/tests/sys/mac/bsdextended/matches_test.sh new file mode 100644 index 0000000..5aff413 --- /dev/null +++ b/tests/sys/mac/bsdextended/matches_test.sh @@ -0,0 +1,353 @@ +#!/bin/sh +# +# $FreeBSD$ +# + +uidrange="60000:100000" +gidrange="60000:100000" +uidinrange="nobody" +uidoutrange="daemon" +gidinrange="nobody" # We expect $uidinrange in this group +gidoutrange="daemon" # We expect $uidinrange in this group + +test_num=1 +pass() +{ + echo "ok $test_num # $@" + : $(( test_num += 1 )) +} + +fail() +{ + echo "not ok $test_num # $@" + : $(( test_num += 1 )) +} + +# +# Setup +# + +: ${TMPDIR=/tmp} +if [ $(id -u) -ne 0 ]; then + echo "1..0 # SKIP test must be run as root" + exit 0 +fi +if ! sysctl -N security.mac.bsdextended >/dev/null 2>&1; then + echo "1..0 # SKIP mac_bsdextended(4) support isn't available" + exit 0 +fi +if ! playground=$(mktemp -d $TMPDIR/tmp.XXXXXXX); then + echo "1..0 # SKIP failed to create temporary directory" + exit 0 +fi +trap "rmdir $playground" EXIT INT TERM +if ! mdmfs -s 25m md $playground; then + echo "1..0 # SKIP failed to mount md device" + exit 0 +fi +chmod a+rwx $playground +md_device=$(mount -p | grep "$playground" | awk '{ gsub(/^\/dev\//, "", $1); print $1 }') +trap "umount -f $playground; mdconfig -d -u $md_device; rmdir $playground" EXIT INT TERM +if [ -z "$md_device" ]; then + mount -p | grep $playground + echo "1..0 # SKIP md device not properly attached to the system" +fi + +ugidfw remove 1 + +file1=$playground/test-$uidinrange +file2=$playground/test-$uidoutrange +cat > $playground/test-script.sh <<'EOF' +#!/bin/sh +: > $1 +EOF +if [ $? -ne 0 ]; then + echo "1..0 # SKIP failed to create test script" + exit 0 +fi +echo "1..30" + +command1="sh $playground/test-script.sh $file1" +command2="sh $playground/test-script.sh $file2" + +desc="$uidinrange file" +if su -m $uidinrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +chown "$uidinrange":"$gidinrange" $file1 +chmod a+w $file1 + +desc="$uidoutrange file" +if $command2; then + pass $desc +else + fail $desc +fi + +chown "$uidoutrange":"$gidoutrange" $file2 +chmod a+w $file2 + +# +# No rules +# +desc="no rules $uidinrange" +if su -fm $uidinrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +desc="no rules $uidoutrange" +if su -fm $uidoutrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +# +# Subject Match on uid +# +ugidfw set 1 subject uid $uidrange object mode rasx +desc="subject uid in range" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +desc="subject uid out range" +if su -fm $uidoutrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +# +# Subject Match on gid +# +ugidfw set 1 subject gid $gidrange object mode rasx + +desc="subject gid in range" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +desc="subject gid out range" +if su -fm $uidoutrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +# +# Subject Match on jail +# +rm -f $playground/test-jail + +desc="subject matching jailid" +jailid=`jail -i / localhost 127.0.0.1 /usr/sbin/daemon -f /bin/sh -c "(sleep 5; touch $playground/test-jail) &"` +ugidfw set 1 subject jailid $jailid object mode rasx +sleep 10 + +if [ -f $playground/test-jail ]; then + fail "TODO $desc: this testcase fails (see bug # 205481)" +else + pass $desc +fi + +rm -f $playground/test-jail +desc="subject nonmatching jailid" +jailid=`jail -i / localhost 127.0.0.1 /usr/sbin/daemon -f /bin/sh -c "(sleep 5; touch $playground/test-jail) &"` +sleep 10 +if [ -f $playground/test-jail ]; then + pass $desc +else + fail $desc +fi + +# +# Object uid +# +ugidfw set 1 subject object uid $uidrange mode rasx + +desc="object uid in range" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +desc="object uid out range" +if su -fm $uidinrange -c "$command2"; then + pass $desc +else + fail $desc +fi +ugidfw set 1 subject object uid $uidrange mode rasx + +desc="object uid in range (different subject)" +if su -fm $uidoutrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +desc="object uid out range (different subject)" +if su -fm $uidoutrange -c "$command2"; then + pass $desc +else + fail $desc +fi + +# +# Object gid +# +ugidfw set 1 subject object gid $uidrange mode rasx + +desc="object gid in range" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +desc="object gid out range" +if su -fm $uidinrange -c "$command2"; then + pass $desc +else + fail $desc +fi +desc="object gid in range (different subject)" +if su -fm $uidoutrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +desc="object gid out range (different subject)" +if su -fm $uidoutrange -c "$command2"; then + pass $desc +else + fail $desc +fi + +# +# Object filesys +# +ugidfw set 1 subject uid $uidrange object filesys / mode rasx +desc="object out of filesys" +if su -fm $uidinrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +ugidfw set 1 subject uid $uidrange object filesys $playground mode rasx +desc="object in filesys" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +# +# Object suid +# +ugidfw set 1 subject uid $uidrange object suid mode rasx +desc="object notsuid" +if su -fm $uidinrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +chmod u+s $file1 +desc="object suid" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi +chmod u-s $file1 + +# +# Object sgid +# +ugidfw set 1 subject uid $uidrange object sgid mode rasx +desc="object notsgid" +if su -fm $uidinrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +chmod g+s $file1 +desc="object sgid" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi +chmod g-s $file1 + +# +# Object uid matches subject +# +ugidfw set 1 subject uid $uidrange object uid_of_subject mode rasx + +desc="object uid notmatches subject" +if su -fm $uidinrange -c "$command2"; then + pass $desc +else + fail $desc +fi + +desc="object uid matches subject" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +# +# Object gid matches subject +# +ugidfw set 1 subject uid $uidrange object gid_of_subject mode rasx + +desc="object gid notmatches subject" +if su -fm $uidinrange -c "$command2"; then + pass $desc +else + fail $desc +fi + +desc="object gid matches subject" +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi + +# +# Object type +# +desc="object not type" +ugidfw set 1 subject uid $uidrange object type dbclsp mode rasx +if su -fm $uidinrange -c "$command1"; then + pass $desc +else + fail $desc +fi + +desc="object type" +ugidfw set 1 subject uid $uidrange object type r mode rasx +if su -fm $uidinrange -c "$command1"; then + fail $desc +else + pass $desc +fi diff --git a/tools/regression/mac/mac_bsdextended/test_ugidfw.c b/tests/sys/mac/bsdextended/ugidfw_test.c index 63e25f0..aab8553 100644 --- a/tools/regression/mac/mac_bsdextended/test_ugidfw.c +++ b/tests/sys/mac/bsdextended/ugidfw_test.c @@ -33,24 +33,19 @@ #include <security/mac_bsdextended/mac_bsdextended.h> #include <err.h> +#include <errno.h> #include <grp.h> #include <pwd.h> -#include <ugidfw.h> #include <stdio.h> #include <stdlib.h> #include <string.h> +#include <ugidfw.h> +#include <unistd.h> /* * Starting point for a regression test for mac_bsdextended(4) and the * supporting libugidfw(3). */ -void -usage(void) -{ - - fprintf(stderr, "test_ugidfw\n"); - exit(-1); -} /* * This section of the regression test passes some test cases through the @@ -68,7 +63,6 @@ static const char *test_users[] = { "operator", "bin", }; -static const int test_users_len = sizeof(test_users) / sizeof(char *); static const char *test_groups[] = { "wheel", @@ -76,7 +70,8 @@ static const char *test_groups[] = { "operator", "bin", }; -static const int test_groups_len = sizeof(test_groups) / sizeof(char *); + +int test_num; /* * List of test strings that must go in (and come out) of libugidfw intact. @@ -147,7 +142,6 @@ static const char *test_strings[] = { "object ! uid root:daemon gid daemon filesys / suid sgid uid_of_subject gid_of_subject ! type r " "mode rsx", }; -static const int test_strings_len = sizeof(test_strings) / sizeof(char *); static void test_libugidfw_strings(void) @@ -155,69 +149,83 @@ test_libugidfw_strings(void) struct mac_bsdextended_rule rule; char errorstr[256]; char rulestr[256]; - int i, error; + int error, i; - for (i = 0; i < test_users_len; i++) { + for (i = 0; i < nitems(test_users); i++, test_num++) { if (getpwnam(test_users[i]) == NULL) - err(-1, "test_libugidfw_strings: getpwnam: %s", - test_users[i]); + printf("not ok %d # test_libugidfw_strings: getpwnam(%s) " + "failed: %s\n", test_num, test_users[i], strerror(errno)); + else + printf("ok %d\n", test_num); } - for (i = 0; i < test_groups_len; i++) { + for (i = 0; i < nitems(test_groups); i++, test_num++) { if (getgrnam(test_groups[i]) == NULL) - err(-1, "test_libugidfw_strings: getgrnam: %s", - test_groups[i]); + printf("not ok %d # test_libugidfw_strings: getgrnam(%s) " + "failed: %s\n", test_num, test_groups[i], strerror(errno)); + else + printf("ok %d\n", test_num); } - for (i = 0; i < test_strings_len; i++) { + for (i = 0; i < nitems(test_strings); i++) { error = bsde_parse_rule_string(test_strings[i], &rule, sizeof(errorstr), errorstr); if (error == -1) - errx(-1, "bsde_parse_rule_string: '%s' (%d): %s", - test_strings[i], i, errorstr); + printf("not ok %d # bsde_parse_rule_string: '%s' (%d) " + "failed: %s\n", test_num, test_strings[i], i, errorstr); + else + printf("ok %d\n", test_num); + test_num++; + error = bsde_rule_to_string(&rule, rulestr, sizeof(rulestr)); if (error < 0) - errx(-1, "bsde_rule_to_string: rule for '%s' " - "returned %d", test_strings[i], error); + printf("not ok %d # bsde_rule_to_string: rule for '%s' " + "returned %d\n", test_num, test_strings[i], error); + else + printf("ok %d\n", test_num); + test_num++; if (strcmp(test_strings[i], rulestr) != 0) - errx(-1, "test_libugidfw: '%s' in, '%s' out", - test_strings[i], rulestr); + printf("not ok %d # test_libugidfw: '%s' in, '%s' " + "out\n", test_num, test_strings[i], rulestr); + else + printf("ok %d\n", test_num); + test_num++; } } int -main(int argc, char *argv[]) +main(void) { char errorstr[256]; int count, slots; - if (argc != 1) - usage(); + test_num = 1; /* Print an error if a non-root user attemps to run the tests. */ if (getuid() != 0) { - fprintf(stderr, "Error! Only root may run this utility\n"); - return (EXIT_FAILURE); + printf("1..0 # SKIP you must be root\n"); + return (0); } - /* - * We can test some parts of the library without the MAC Framework - * and policy loaded, so run those tests before calling - * mac_is_present(). - */ - test_libugidfw_strings(); - switch (mac_is_present("bsdextended")) { case -1: - err(-1, "mac_is_present"); + printf("1..0 # SKIP mac_is_present failed: %s\n", + strerror(errno)); + return (0); case 1: break; case 0: default: - errx(-1, "mac_bsdextended not loaded"); + printf("1..0 # SKIP mac_bsdextended not loaded\n"); + return (0); } + printf("1..%lu\n", nitems(test_users) + nitems(test_groups) + + 3 * nitems(test_strings) + 2); + + test_libugidfw_strings(); + /* * Some simple up-front checks to see if we're able to query the * policy for basic state. We want the rule count to be 0 before @@ -226,13 +234,19 @@ main(int argc, char *argv[]) */ count = bsde_get_rule_count(sizeof(errorstr), errorstr); if (count == -1) - errx(-1, "bsde_get_rule_count: %s", errorstr); - if (count != 0) - errx(-1, "bsde_get_rule_count: %d rules", count); + printf("not ok %d # bsde_get_rule_count: %s\n", test_num, + errorstr); + else + printf("ok %d\n", test_num); + + test_num++; slots = bsde_get_rule_slots(sizeof(errorstr), errorstr); if (slots == -1) - errx(-1, "bsde_get_rule_slots: %s", errorstr); + printf("not ok %d # bsde_get_rule_slots: %s\n", test_num, + errorstr); + else + printf("ok %d\n", test_num); return (0); } diff --git a/tools/regression/mac/mac_portacl/LICENSE b/tests/sys/mac/portacl/LICENSE index c95c149..c95c149 100644 --- a/tools/regression/mac/mac_portacl/LICENSE +++ b/tests/sys/mac/portacl/LICENSE diff --git a/tests/sys/mac/portacl/Makefile b/tests/sys/mac/portacl/Makefile new file mode 100644 index 0000000..129d486 --- /dev/null +++ b/tests/sys/mac/portacl/Makefile @@ -0,0 +1,16 @@ +# $FreeBSD$ + +TESTSDIR= ${TESTSBASE}/sys/mac/portacl +BINDIR= ${TESTSDIR} + +FILES+= misc.sh + +TAP_TESTS_SH+= nobody_test +TAP_TESTS_SH+= root_test + +.for t in ${TAP_TESTS_SH} +TEST_METADATA.$t+= required_user="root" +TEST_METADATA.$t+= timeout="450" +.endfor + +.include <bsd.test.mk> diff --git a/tools/regression/mac/mac_portacl/misc.sh b/tests/sys/mac/portacl/misc.sh index a1f152b..5a9e67b 100755 --- a/tools/regression/mac/mac_portacl/misc.sh +++ b/tests/sys/mac/portacl/misc.sh @@ -6,10 +6,18 @@ if [ $? -ne 0 ]; then echo "1..0 # SKIP MAC_PORTACL is unavailable." exit 0 fi +if [ $(id -u) -ne 0 ]; then + echo "1..0 # SKIP testcases must be run as root" + exit 0 +fi ntest=1 check_bind() { + local host idtype name proto port udpflag + + host="127.0.0.1" + idtype=${1} name=${2} proto=${3} @@ -17,10 +25,10 @@ check_bind() { [ "${proto}" = "udp" ] && udpflag="-u" - out=`( + out=$( case "${idtype}" in uid|gid) - ( echo -n | su -m ${name} -c "nc ${udpflag} -o -l 127.0.0.1 $port" 2>&1 ) & + ( echo -n | su -m ${name} -c "nc ${udpflag} -l -w 10 $host $port" 2>&1 ) & ;; jail) kill $$ @@ -29,9 +37,9 @@ check_bind() { kill $$ esac sleep 0.3 - echo | nc ${udpflag} -o 127.0.0.1 $port >/dev/null 2>&1 + echo | nc ${udpflag} -w 10 $host $port >/dev/null 2>&1 wait - )` + ) case "${out}" in "nc: Permission denied"*|"nc: Operation not permitted"*) echo fl @@ -46,6 +54,8 @@ check_bind() { } bind_test() { + local expect_without_rule expect_with_rule idtype name proto port + expect_without_rule=${1} expect_with_rule=${2} idtype=${3} @@ -54,40 +64,40 @@ bind_test() { port=${6} sysctl security.mac.portacl.rules= >/dev/null - out=`check_bind ${idtype} ${name} ${proto} ${port}` + out=$(check_bind ${idtype} ${name} ${proto} ${port}) if [ "${out}" = "${expect_without_rule}" ]; then echo "ok ${ntest}" elif [ "${out}" = "ok" -o "${out}" = "fl" ]; then - echo "not ok ${ntest}" + echo "not ok ${ntest} # '${out}' != '${expect_without_rule}'" else - echo "not ok ${ntest} # ${out}" + echo "not ok ${ntest} # unexpected output: '${out}'" fi - ntest=$((ntest+1)) + : $(( ntest += 1 )) if [ "${idtype}" = "uid" ]; then - idstr=`id -u ${name}` + idstr=$(id -u ${name}) elif [ "${idtype}" = "gid" ]; then - idstr=`id -g ${name}` + idstr=$(id -g ${name}) else idstr=${name} fi sysctl security.mac.portacl.rules=${idtype}:${idstr}:${proto}:${port} >/dev/null - out=`check_bind ${idtype} ${name} ${proto} ${port}` + out=$(check_bind ${idtype} ${name} ${proto} ${port}) if [ "${out}" = "${expect_with_rule}" ]; then echo "ok ${ntest}" elif [ "${out}" = "ok" -o "${out}" = "fl" ]; then - echo "not ok ${ntest}" + echo "not ok ${ntest} # '${out}' != '${expect_with_rule}'" else - echo "not ok ${ntest} # ${out}" + echo "not ok ${ntest} # unexpected output: '${out}'" fi - ntest=$((ntest+1)) + : $(( ntest += 1 )) sysctl security.mac.portacl.rules= >/dev/null } -reserved_high=`sysctl -n net.inet.ip.portrange.reservedhigh` -suser_exempt=`sysctl -n security.mac.portacl.suser_exempt` -port_high=`sysctl -n security.mac.portacl.port_high` +reserved_high=$(sysctl -n net.inet.ip.portrange.reservedhigh) +suser_exempt=$(sysctl -n security.mac.portacl.suser_exempt) +port_high=$(sysctl -n security.mac.portacl.port_high) restore_settings() { sysctl -n net.inet.ip.portrange.reservedhigh=${reserved_high} >/dev/null diff --git a/tools/regression/mac/mac_portacl/nobody.t b/tests/sys/mac/portacl/nobody_test.sh index c0754eb..7d8dbd6 100755 --- a/tools/regression/mac/mac_portacl/nobody.t +++ b/tests/sys/mac/portacl/nobody_test.sh @@ -10,6 +10,8 @@ echo "1..64" # behaviour. # mac_portacl has no impact on ports <= net.inet.ip.portrange.reservedhigh. +trap restore_settings EXIT INT TERM + sysctl security.mac.portacl.suser_exempt=1 >/dev/null sysctl net.inet.ip.portrange.reservedhigh=78 >/dev/null @@ -63,5 +65,3 @@ bind_test fl ok gid nobody tcp 77 bind_test ok ok gid nobody tcp 7777 bind_test fl ok gid nobody udp 77 bind_test ok ok gid nobody udp 7777 - -restore_settings diff --git a/tools/regression/mac/mac_portacl/root.t b/tests/sys/mac/portacl/root_test.sh index 626bdfa..9ed452f 100755 --- a/tools/regression/mac/mac_portacl/root.t +++ b/tests/sys/mac/portacl/root_test.sh @@ -8,6 +8,8 @@ echo "1..48" # Verify if security.mac.portacl.suser_exempt=1 really exempts super-user. +trap restore_settings EXIT INT TERM + sysctl security.mac.portacl.suser_exempt=1 >/dev/null bind_test ok ok uid root tcp 77 @@ -47,5 +49,3 @@ bind_test fl ok gid root tcp 77 bind_test fl ok gid root tcp 7777 bind_test fl ok gid root udp 77 bind_test fl ok gid root udp 7777 - -restore_settings diff --git a/tools/build/mk/OptionalObsoleteFiles.inc b/tools/build/mk/OptionalObsoleteFiles.inc index 779f596..cb51987 100644 --- a/tools/build/mk/OptionalObsoleteFiles.inc +++ b/tools/build/mk/OptionalObsoleteFiles.inc @@ -189,6 +189,7 @@ OLD_DIRS+=usr/share/examples/bhyve .if ${MK_BINUTILS} == no OLD_FILES+=usr/bin/as OLD_FILES+=usr/bin/ld +OLD_FILES+=usr/bin/ld.bfd .if ${MK_ELFCOPY_AS_OBJCOPY} == no OLD_FILES+=usr/bin/objcopy .endif @@ -1052,87 +1053,87 @@ OLD_FILES+=usr/bin/clang++ OLD_FILES+=usr/bin/clang-cpp OLD_FILES+=usr/bin/clang-tblgen OLD_FILES+=usr/bin/tblgen -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/allocator_interface.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/asan_interface.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/common_interface_defs.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/coverage_interface.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/dfsan_interface.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/linux_syscall_hooks.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/lsan_interface.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/msan_interface.h -OLD_FILES+=usr/lib/clang/3.7.0/include/sanitizer/tsan_interface_atomic.h -OLD_DIRS+=usr/lib/clang/3.7.0/include/sanitizer -OLD_FILES+=usr/lib/clang/3.7.0/include/__stddef_max_align_t.h -OLD_FILES+=usr/lib/clang/3.7.0/include/__wmmintrin_aes.h -OLD_FILES+=usr/lib/clang/3.7.0/include/__wmmintrin_pclmul.h -OLD_FILES+=usr/lib/clang/3.7.0/include/adxintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/altivec.h -OLD_FILES+=usr/lib/clang/3.7.0/include/ammintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/arm_acle.h -OLD_FILES+=usr/lib/clang/3.7.0/include/arm_neon.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx2intrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512bwintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512cdintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512dqintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512erintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512fintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512vlbwintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512vldqintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avx512vlintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/avxintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/bmi2intrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/bmiintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/cpuid.h -OLD_FILES+=usr/lib/clang/3.7.0/include/cuda_builtin_vars.h -OLD_FILES+=usr/lib/clang/3.7.0/include/emmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/f16cintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/fma4intrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/fmaintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/fxsrintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/htmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/htmxlintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/ia32intrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/immintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/lzcntintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/mm3dnow.h -OLD_FILES+=usr/lib/clang/3.7.0/include/mm_malloc.h -OLD_FILES+=usr/lib/clang/3.7.0/include/mmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/module.modulemap -OLD_FILES+=usr/lib/clang/3.7.0/include/nmmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/pmmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/popcntintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/prfchwintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/rdseedintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/rtmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/s390intrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/shaintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/smmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/tbmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/tmmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/vadefs.h -OLD_FILES+=usr/lib/clang/3.7.0/include/vecintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/wmmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/x86intrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/xmmintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/xopintrin.h -OLD_FILES+=usr/lib/clang/3.7.0/include/xtestintrin.h -OLD_DIRS+=usr/lib/clang/3.7.0/include -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan-i386.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan-x86_64.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan_cxx-i386.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.asan_cxx-x86_64.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.profile-arm.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.profile-i386.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.profile-x86_64.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.san-i386.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.san-x86_64.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan-i386.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan-x86_64.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan_cxx-i386.a -OLD_FILES+=usr/lib/clang/3.7.0/lib/freebsd/libclang_rt.ubsan_cxx-x86_64.a -OLD_DIRS+=usr/lib/clang/3.7.0/lib/freebsd -OLD_DIRS+=usr/lib/clang/3.7.0/lib -OLD_DIRS+=usr/lib/clang/3.7.0 +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/allocator_interface.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/asan_interface.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/common_interface_defs.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/coverage_interface.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/dfsan_interface.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/linux_syscall_hooks.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/lsan_interface.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/msan_interface.h +OLD_FILES+=usr/lib/clang/3.7.1/include/sanitizer/tsan_interface_atomic.h +OLD_DIRS+=usr/lib/clang/3.7.1/include/sanitizer +OLD_FILES+=usr/lib/clang/3.7.1/include/__stddef_max_align_t.h +OLD_FILES+=usr/lib/clang/3.7.1/include/__wmmintrin_aes.h +OLD_FILES+=usr/lib/clang/3.7.1/include/__wmmintrin_pclmul.h +OLD_FILES+=usr/lib/clang/3.7.1/include/adxintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/altivec.h +OLD_FILES+=usr/lib/clang/3.7.1/include/ammintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/arm_acle.h +OLD_FILES+=usr/lib/clang/3.7.1/include/arm_neon.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx2intrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512bwintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512cdintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512dqintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512erintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512fintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512vlbwintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512vldqintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avx512vlintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/avxintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/bmi2intrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/bmiintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/cpuid.h +OLD_FILES+=usr/lib/clang/3.7.1/include/cuda_builtin_vars.h +OLD_FILES+=usr/lib/clang/3.7.1/include/emmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/f16cintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/fma4intrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/fmaintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/fxsrintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/htmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/htmxlintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/ia32intrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/immintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/lzcntintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/mm3dnow.h +OLD_FILES+=usr/lib/clang/3.7.1/include/mm_malloc.h +OLD_FILES+=usr/lib/clang/3.7.1/include/mmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/module.modulemap +OLD_FILES+=usr/lib/clang/3.7.1/include/nmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/pmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/popcntintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/prfchwintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/rdseedintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/rtmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/s390intrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/shaintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/smmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/tbmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/tmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/vadefs.h +OLD_FILES+=usr/lib/clang/3.7.1/include/vecintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/wmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/x86intrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/xmmintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/xopintrin.h +OLD_FILES+=usr/lib/clang/3.7.1/include/xtestintrin.h +OLD_DIRS+=usr/lib/clang/3.7.1/include +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.asan-i386.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.asan-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.asan_cxx-i386.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.asan_cxx-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.profile-arm.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.profile-i386.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.profile-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.safestack-i386.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.safestack-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.ubsan_standalone-i386.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.ubsan_standalone-x86_64.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.ubsan_standalone_cxx-i386.a +OLD_FILES+=usr/lib/clang/3.7.1/lib/freebsd/libclang_rt.ubsan_standalone_cxx-x86_64.a +OLD_DIRS+=usr/lib/clang/3.7.1/lib/freebsd +OLD_DIRS+=usr/lib/clang/3.7.1/lib +OLD_DIRS+=usr/lib/clang/3.7.1 OLD_DIRS+=usr/lib/clang OLD_FILES+=usr/share/doc/llvm/clang/LICENSE.TXT OLD_DIRS+=usr/share/doc/llvm/clang diff --git a/tools/regression/lib/msun/Makefile b/tools/regression/lib/msun/Makefile deleted file mode 100644 index 8b301cb..0000000 --- a/tools/regression/lib/msun/Makefile +++ /dev/null @@ -1,14 +0,0 @@ -# $FreeBSD$ - -TESTS= test-ctrig \ - test-exponential test-fma \ - test-lround test-nearbyint test-next test-rem test-trig -CFLAGS+= -O0 -lm -Wno-unknown-pragmas - -.PHONY: tests -tests: ${TESTS} - for p in ${TESTS}; do ${.OBJDIR}/$$p; done - -.PHONY: clean -clean: - -rm -f ${TESTS} diff --git a/tools/regression/lib/msun/test-exponential.t b/tools/regression/lib/msun/test-exponential.t deleted file mode 100644 index 8bdfd03..0000000 --- a/tools/regression/lib/msun/test-exponential.t +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# $FreeBSD$ - -cd `dirname $0` - -executable=`basename $0 .t` - -make $executable 2>&1 > /dev/null - -exec ./$executable diff --git a/tools/regression/lib/msun/test-fma.t b/tools/regression/lib/msun/test-fma.t deleted file mode 100644 index 8bdfd03..0000000 --- a/tools/regression/lib/msun/test-fma.t +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# $FreeBSD$ - -cd `dirname $0` - -executable=`basename $0 .t` - -make $executable 2>&1 > /dev/null - -exec ./$executable diff --git a/tools/regression/lib/msun/test-invtrig.t b/tools/regression/lib/msun/test-invtrig.t deleted file mode 100644 index 8bdfd03..0000000 --- a/tools/regression/lib/msun/test-invtrig.t +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# $FreeBSD$ - -cd `dirname $0` - -executable=`basename $0 .t` - -make $executable 2>&1 > /dev/null - -exec ./$executable diff --git a/tools/regression/lib/msun/test-lround.t b/tools/regression/lib/msun/test-lround.t deleted file mode 100644 index 8bdfd03..0000000 --- a/tools/regression/lib/msun/test-lround.t +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# $FreeBSD$ - -cd `dirname $0` - -executable=`basename $0 .t` - -make $executable 2>&1 > /dev/null - -exec ./$executable diff --git a/tools/regression/mac/mac_bsdextended/Makefile b/tools/regression/mac/mac_bsdextended/Makefile deleted file mode 100644 index aaa060a..0000000 --- a/tools/regression/mac/mac_bsdextended/Makefile +++ /dev/null @@ -1,7 +0,0 @@ -# $FreeBSD$ - -PROG= test_ugidfw -LIBADD= ugidfw -MAN= - -.include <bsd.prog.mk> diff --git a/tools/regression/mac/mac_bsdextended/test_matches.sh b/tools/regression/mac/mac_bsdextended/test_matches.sh deleted file mode 100644 index 99d6b62..0000000 --- a/tools/regression/mac/mac_bsdextended/test_matches.sh +++ /dev/null @@ -1,167 +0,0 @@ -#!/bin/sh -# -# $FreeBSD$ -# - -uidrange="60000:100000" -gidrange="60000:100000" -uidinrange="nobody" -uidoutrange="daemon" -gidinrange="nobody" # We expect $uidinrange in this group -gidoutrange="daemon" # We expect $uidinrange in this group - -playground="/stuff/nobody/" # Must not be on root fs - -# -# Setup -# -rm -f $playground/test* -ugidfw remove 1 - -file1=$playground/test-$uidinrange -file2=$playground/test-$uidoutrange -cat <<EOF> $playground/test-script.pl -if (open(F, ">" . shift)) { exit 0; } else { exit 1; } -EOF -command1="perl $playground/test-script.pl $file1" -command2="perl $playground/test-script.pl $file2" - -echo -n "$uidinrange file: " -su -m $uidinrange -c "$command1 && echo good" -chown "$uidinrange":"$gidinrange" $file1 -chmod a+w $file1 - -echo -n "$uidoutrange file: " -$command2 && echo good -chown "$uidoutrange":"$gidoutrange" $file2 -chmod a+w $file2 - -# -# No rules -# -echo -n "no rules $uidinrange: " -su -fm $uidinrange -c "$command1 && echo good" -echo -n "no rules $uidoutrange: " -su -fm $uidoutrange -c "$command1 && echo good" - -# -# Subject Match on uid -# -ugidfw set 1 subject uid $uidrange object mode rasx -echo -n "subject uid in range: " -su -fm $uidinrange -c "$command1 || echo good" -echo -n "subject uid out range: " -su -fm $uidoutrange -c "$command1 && echo good" - -# -# Subject Match on gid -# -ugidfw set 1 subject gid $gidrange object mode rasx -echo -n "subject gid in range: " -su -fm $uidinrange -c "$command1 || echo good" -echo -n "subject gid out range: " -su -fm $uidoutrange -c "$command1 && echo good" - -# -# Subject Match on jail -# -echo -n "subject matching jailid: " -rm -f $playground/test-jail -jailid=`jail -i / localhost 127.0.0.1 /usr/sbin/daemon -f /bin/sh -c "(sleep 3; touch $playground/test-jail) &"` -ugidfw set 1 subject jailid $jailid object mode rasx -sleep 6 -if [ ! -f $playground/test-jail ] ; then echo good ; fi - -echo -n "subject nonmatching jailid: " -rm -f $playground/test-jail -jailid=`jail -i / localhost 127.0.0.1 /usr/sbin/daemon -f /bin/sh -c "(sleep 3; touch $playground/test-jail) &"` -sleep 6 -if [ -f $playground/test-jail ] ; then echo good ; fi - -# -# Object uid -# -ugidfw set 1 subject object uid $uidrange mode rasx -echo -n "object uid in range: " -su -fm $uidinrange -c "$command1 || echo good" -echo -n "object uid out range: " -su -fm $uidinrange -c "$command2 && echo good" -ugidfw set 1 subject object uid $uidrange mode rasx -echo -n "object uid in range (differennt subject): " -su -fm $uidoutrange -c "$command1 || echo good" -echo -n "object uid out range (differennt subject): " -su -fm $uidoutrange -c "$command2 && echo good" - -# -# Object gid -# -ugidfw set 1 subject object gid $uidrange mode rasx -echo -n "object gid in range: " -su -fm $uidinrange -c "$command1 || echo good" -echo -n "object gid out range: " -su -fm $uidinrange -c "$command2 && echo good" -echo -n "object gid in range (differennt subject): " -su -fm $uidoutrange -c "$command1 || echo good" -echo -n "object gid out range (differennt subject): " -su -fm $uidoutrange -c "$command2 && echo good" - -# -# Object filesys -# -ugidfw set 1 subject uid $uidrange object filesys / mode rasx -echo -n "object out of filesys: " -su -fm $uidinrange -c "$command1 && echo good" -ugidfw set 1 subject uid $uidrange object filesys $playground mode rasx -echo -n "object in filesys: " -su -fm $uidinrange -c "$command1 || echo good" - -# -# Object suid -# -ugidfw set 1 subject uid $uidrange object suid mode rasx -echo -n "object notsuid: " -su -fm $uidinrange -c "$command1 && echo good" -chmod u+s $file1 -echo -n "object suid: " -su -fm $uidinrange -c "$command1 || echo good" -chmod u-s $file1 - -# -# Object sgid -# -ugidfw set 1 subject uid $uidrange object sgid mode rasx -echo -n "object notsgid: " -su -fm $uidinrange -c "$command1 && echo good" -chmod g+s $file1 -echo -n "object sgid: " -su -fm $uidinrange -c "$command1 || echo good" -chmod g-s $file1 - -# -# Object uid matches subject -# -ugidfw set 1 subject uid $uidrange object uid_of_subject mode rasx -echo -n "object uid notmatches subject: " -su -fm $uidinrange -c "$command2 && echo good" -echo -n "object uid matches subject: " -su -fm $uidinrange -c "$command1 || echo good" - -# -# Object gid matches subject -# -ugidfw set 1 subject uid $uidrange object gid_of_subject mode rasx -echo -n "object gid notmatches subject: " -su -fm $uidinrange -c "$command2 && echo good" -echo -n "object gid matches subject: " -su -fm $uidinrange -c "$command1 || echo good" - -# -# Object type -# -ugidfw set 1 subject uid $uidrange object type dbclsp mode rasx -echo -n "object not type: " -su -fm $uidinrange -c "$command1 && echo good" -ugidfw set 1 subject uid $uidrange object type r mode rasx -echo -n "object type: " -su -fm $uidinrange -c "$command1 || echo good" - diff --git a/tools/regression/sockets/unix_passfd/unix_passfd.c b/tools/regression/sockets/unix_passfd/unix_passfd.c index 07ef589..59c4c29 100644 --- a/tools/regression/sockets/unix_passfd/unix_passfd.c +++ b/tools/regression/sockets/unix_passfd/unix_passfd.c @@ -1,5 +1,6 @@ /*- * Copyright (c) 2005 Robert N. M. Watson + * Copyright (c) 2015 Mark Johnston * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -146,7 +147,7 @@ sendfd_payload(const char *test, int sockfd, int sendfd, static void sendfd(const char *test, int sockfd, int sendfd) { - char ch; + char ch = 0; return (sendfd_payload(test, sockfd, sendfd, &ch, sizeof(ch))); } @@ -199,7 +200,7 @@ recvfd_payload(const char *test, int sockfd, int *recvfd, static void recvfd(const char *test, int sockfd, int *recvfd) { - char ch; + char ch = 0; return (recvfd_payload(test, sockfd, recvfd, &ch, sizeof(ch))); } @@ -369,8 +370,8 @@ main(void) err(-1, "%s: sysctlbyname(net.local.stream.sendspace)", test); - if ((buf = malloc(sendspace)) == NULL) - err(-1, "%s: malloc", test); + if ((buf = calloc(1, sendspace)) == NULL) + err(-1, "%s: calloc", test); domainsocketpair(test, fd); if (setsockopt(fd[1], 0, LOCAL_CREDS, &on, sizeof(on)) < 0) @@ -384,6 +385,6 @@ main(void) } printf("%s passed\n", test); - + return (0); } diff --git a/tools/tools/locale/tools/cldr2def.pl b/tools/tools/locale/tools/cldr2def.pl index d26cb2e..fae7c91 100755 --- a/tools/tools/locale/tools/cldr2def.pl +++ b/tools/tools/locale/tools/cldr2def.pl @@ -67,6 +67,7 @@ my %callback = ( mdorder => \&callback_mdorder, altmon => \&callback_altmon, cformat => \&callback_cformat, + dtformat => \&callback_dtformat, cbabmon => \&callback_abmon, data => undef, ); @@ -184,7 +185,7 @@ if ($TYPE eq "timedef") { "c_fmt" => "<cformat<d_t_fmt<s", "am_pm" => "as", "d_fmt" => "s", - "d_t_fmt" => "s", + "d_t_fmt" => "<dtformat<d_t_fmt<s", "altmon" => "<altmon<mon<as", "md_order" => "<mdorder<d_fmt<s", "t_fmt_ampm" => "s", @@ -201,6 +202,16 @@ sub callback_cformat { return $s; }; +sub callback_dtformat { + my $s = shift; + my $nl = $callback{data}{l} . "_" . $callback{data}{c}; + + if ($nl eq 'ja_JP') { + $s =~ s/(> )(%H)/$1%A $2/; + } + return $s; +}; + sub callback_mdorder { my $s = shift; return undef if (!defined $s); diff --git a/tools/tools/nanobsd/embedded/common b/tools/tools/nanobsd/embedded/common index 0a79b6a..6347911 100644 --- a/tools/tools/nanobsd/embedded/common +++ b/tools/tools/nanobsd/embedded/common @@ -86,7 +86,7 @@ NANO_CFG_BASE=$(pwd) NANO_CFG_BASE=$(realpath ${NANO_CFG_BASE}/..) NANO_SRC=$(realpath ${NANO_CFG_BASE}/../../..) #### XXX share obj -NANO_OBJ=$(realpath ${NANO_SRC}/../$NANO_NAME/obj) +NANO_OBJ=${NANO_SRC}/../$NANO_NAME/obj # Where cust_pkg() finds packages to install #XXX: Is this the right place? #NANO_PORTS=$(realpath ${NANO_SRC}/../ports) @@ -100,21 +100,22 @@ unset MAKEOBJDIRPREFIX NANO_PORTS=${NANO_PORTS:-/usr/ports} mkdir -p ${NANO_OBJ} +NANO_OBJ=$(realpath ${NANO_OBJ}) + +NANO_FAT_DIR=${NANO_OBJ}/_.fat customize_cmd cust_allow_ssh_root -add_etc_make_conf() -{ +add_etc_make_conf ( ) ( touch ${NANO_WORLDDIR}/etc/make.conf -} +) customize_cmd add_etc_make_conf -cust_install_machine_files() -{ +cust_install_machine_files ( ) ( echo "cd ${NANO_CFG_BASE}/Files" cd ${NANO_CFG_BASE}/Files find . -print | grep -Ev '/(CVS|\.svn|\.hg|\.git)' | cpio -dumpv ${NANO_WORLDDIR} -} +) customize_cmd cust_install_files customize_cmd cust_install_machine_files @@ -178,8 +179,7 @@ WITHOUT_RCS=true NANO_PACKAGE_ONLY=1 # install a package from a pre-built binary -do_add_pkg () -{ +do_add_pkg ( ) ( # Need to create ${NANO_OBJ}/ports in this add_pkg_${port} function set -x mkdir -p ${NANO_OBJ}/ports/distfiles @@ -197,11 +197,10 @@ do_add_pkg () rmdir ${NANO_WORLDDIR}/usr/ports/distfiles rmdir ${NANO_WORLDDIR}/usr/ports set +x -} +) # Build a port (with the side effect of creating a package) -do_add_port () -{ +do_add_port ( ) ( local port_path port_path=$1 shift @@ -242,11 +241,12 @@ do_add_port () umount ${NANO_WORLDDIR}/usr/ports umount ${NANO_WORLDDIR}/usr/src set +x -} +) # Need to check if this function works with cross-compiling architecture!!!! # Recursive complex fonction: Generate one function for each ports -add_port () { +# writes shell functions called later, so don't do in subshell. +add_port ( ) { local port_path=$1 local port=`echo $1 | sed -e 's/\//_/'` shift @@ -259,7 +259,7 @@ add_port () { if [ -f ${NANO_OBJ}/ports/packages/All/${PKG_NAME}.txz ]; then # Pkg file found: Generate add_pkg_NAME function eval " - add_pkg_${port} () { + add_pkg_${port} ( ) { do_add_pkg ${PKG_NAME} } customize_cmd add_pkg_${port} @@ -267,7 +267,7 @@ add_port () { else # No pkg file: Generate add_port_NAME function eval " - add_port_${port} () { + add_port_${port} ( ) { do_add_port ${port_path} $* } customize_cmd add_port_${port} @@ -297,12 +297,15 @@ create_diskimage_mbr ( ) ( done # Populate the FAT partition, if needed - if [-n "${NANO_SLICE_FAT}" ]; then + if [ -n "${NANO_SLICE_FAT}" ]; then echo Creating MSDOS partition for kernel newfs_msdos -C ${NANO_SLICE_FAT_SIZE} -F 16 -L ${NANO_NAME} \ ${NANO_OBJ}/_.${NANO_SLICE_FAT} - # Need to copy files from ${NANO_FATDIR} with mtools, or use - # makefs -t msdos once that's supported + if [ -d ${NANO_FAT_DIR} ]; then + # Need to copy files from ${NANO_FATDIR} with mtools, or use + # makefs -t msdos once that's supported + mcopy -i ${NANO_OBJ}/_.${NANO_SLICE_FAT} ${NANO_FAT_DIR}/* :: + fi fi # Populate the Powerpc boot image, if needed @@ -323,7 +326,7 @@ create_diskimage_mbr ( ) ( if [ -z "${NANO_CFGDIR}" ]; then echo "Faking cfg dir, it's empty" NANO_CFGDIR=${NANO_OBJ}/_.empty - mkdir ${NANO_CFGDIR} + mkdir -p ${NANO_CFGDIR} fi # XXX -F cfg-mtree eval "${NANO_MAKEFS_UFS}" -s ${NANO_SLICE_CFG_SIZE} \ @@ -371,8 +374,7 @@ create_diskimage_mbr ( ) ( ) > ${NANO_OBJ}/_.di 2>&1 ) -die() -{ +die( ) { echo "$*" exit 1 } @@ -429,19 +431,35 @@ $var=$val" fi done -save_build ( ) -{ +typical_embedded ( ) ( + # Need to create rc.conf before we copy over /etc to /conf/base/etc + # so now's a good time. + + local rc=${NANO_WORLDDIR}/etc/rc.conf + + echo "hostname=nanobsd-${NANO_NAME}" > $rc + echo "growfs_enable=YES" >> $rc + echo "growfs_type=nanobsd-pingpong" >> $rc + echo "ntpdate_enable=YES" >> $rc + echo "ifconfig_DEFAULT=DHCP" >> $rc + echo "ntpdate_hosts='0.freebsd.pool.ntp.org 1.freebsd.pool.ntp.org'" >> $rc + # Make sure that firstboot scripts run so growfs works. + # Note: still some issues remvoing this XXX + touch ${NANO_WORLDDIR}/firstboot +) +customize_cmd typical_embedded + +save_build ( ) ( VERSION_FILE=${NANO_WORLDDIR}/etc/version if [ "${SVNREVISION}" = "${REVISION}" ]; then echo "${NANO_NAME}" > "${VERSION_FILE}" else echo "${NANO_NAME} (${SVNREVISION})" > "${VERSION_FILE}" fi -} +) customize_cmd save_build -shrink_md_fbsize() -{ +shrink_md_fbsize ( ) ( # We have a lot of little files on our memory disks. Let's decrease # the block and frag size to fit more little files on them (this # halves our space requirement by ~50% on /etc and /var on 8.x -- @@ -449,13 +467,48 @@ shrink_md_fbsize() # are 4 times larger). sed -i '' -e 's,-S -i 4096,-S -i 4096 -b 4096 -f 512,' \ ${NANO_WORLDDIR}/etc/rc.initdiskless -} +) customize_cmd shrink_md_fbsize customize_cmd cust_comconsole -product_custom() -{ +dos_boot_part ( ) ( + local d=/usr/local/share/u-boot/${NANO_BOOT_PKG} + local f=${NANO_FAT_DIR} + + # For now, just copy all the files. However, for iMX6 and Allwinner, + # we'll need to put a special boot block at a fixed location + # on the disk as well. + rm -rf $f + mkdir $f + chdir $f + cp ${d}/* . + + # Also copy ubldr. u-boot will load it and it will load the kernel + # from the ufs partition + cp ${NANO_WORLDDIR}/boot/ubldr . + cp ${NANO_WORLDDIR}/boot/ubldr.bin . + + # We have to touch the saveenv file + touch uEnv.txt + + # Now we need to copy over dtb files from the build. + cp ${NANO_WORLDDIR}/boot/dtb/*.dtb . +) + +if [ -n "$NANO_BOOT_PKG" ]; then + if [ ! -d ${d} ]; then + echo ${NANO_BOOT_PKG} not installed. Sadly, it must be. + exit 1 + fi + if [ ! -x /usr/local/bin/mcopy ]; then + echo mtools not installed. Sadly, we gotta have it. + exit 1 + fi + customize_cmd dos_boot_part +fi + +product_custom ( ) ( # not quie ready to tweak these in nopriv build if [ -z ${NANO_NOPRIV_BUILD} ]; then # Last second tweaks -- generally not needed @@ -467,7 +520,7 @@ product_custom() chown root:wheel ${NANO_WORLDDIR}/ chown root:wheel ${NANO_WORLDDIR}/usr fi -} +) late_customize_cmd product_custom # @@ -478,7 +531,8 @@ late_customize_cmd product_custom # For each architecture, we have a standard set of settings to the extent # it makes sense. For architectures that don't have a standard environment # cfg files need to either define a lot of settings, provide their own disk -# imaging, or set which of the variants we support. +# imaging, or set which of the variants we support. No subshells, since we +# set global variables # std_aarch64 ( ) { diff --git a/tools/tools/nanobsd/embedded/rpi2.cfg b/tools/tools/nanobsd/embedded/rpi2.cfg index 8415e35..233f3f4 100644 --- a/tools/tools/nanobsd/embedded/rpi2.cfg +++ b/tools/tools/nanobsd/embedded/rpi2.cfg @@ -31,6 +31,6 @@ NANO_KERNEL=RPI2 NANO_DRIVE=mmcsd0 NANO_NAME=rpi2 NANO_BOOT_PKG=u-boot-rpi2 -NANO_CPUTYPE=cortex-a7 +# NANO_CPUTYPE=cortex-a7 . common # Pull in common definitions, keep last diff --git a/usr.bin/bmake/Makefile b/usr.bin/bmake/Makefile index 59c2a6a..7b18da6 100644 --- a/usr.bin/bmake/Makefile +++ b/usr.bin/bmake/Makefile @@ -14,10 +14,10 @@ CFLAGS+= -I${.CURDIR} CLEANDIRS+= FreeBSD CLEANFILES+= bootstrap -# $Id: Makefile,v 1.48 2015/12/02 00:36:42 sjg Exp $ +# $Id: Makefile,v 1.49 2015/12/20 22:54:40 sjg Exp $ # Base version on src date -MAKE_VERSION= 20151201 +MAKE_VERSION= 20151220 PROG?= ${.CURDIR:T} diff --git a/usr.bin/column/column.c b/usr.bin/column/column.c index b092deb..eb8ca66 100644 --- a/usr.bin/column/column.c +++ b/usr.bin/column/column.c @@ -244,7 +244,7 @@ maketbl(void) p = NULL) if (++coloff == maxcols) { if (!(cols = realloc(cols, ((u_int)maxcols + - DEFCOLS) * sizeof(char *))) || + DEFCOLS) * sizeof(wchar_t *))) || !(lens = realloc(lens, ((u_int)maxcols + DEFCOLS) * sizeof(int)))) err(1, NULL); diff --git a/usr.bin/dtc/Makefile b/usr.bin/dtc/Makefile index a6c722a..a834f62 100644 --- a/usr.bin/dtc/Makefile +++ b/usr.bin/dtc/Makefile @@ -6,7 +6,7 @@ MAN= dtc.1 WARNS?= 3 -CXXFLAGS+= -std=c++11 +CXXFLAGS+= -std=c++11 -fno-rtti -fno-exceptions NO_SHARED?=NO diff --git a/usr.bin/dtc/checking.cc b/usr.bin/dtc/checking.cc index 70731ce..26cbbe2 100644 --- a/usr.bin/dtc/checking.cc +++ b/usr.bin/dtc/checking.cc @@ -51,7 +51,7 @@ namespace struct address_cells_checker : public checker { address_cells_checker(const char *name) : checker(name) {} - virtual bool check_node(device_tree *tree, const node_ptr &n) + virtual bool check_node(device_tree *, const node_ptr &n) { // If this has no children, it trivially meets the // conditions. @@ -151,7 +151,7 @@ property_checker::check_property(device_tree *tree, const node_ptr &n, property_ } bool -property_size_checker::check(device_tree *tree, const node_ptr &n, property_ptr p) +property_size_checker::check(device_tree *, const node_ptr &, property_ptr p) { uint32_t psize = 0; for (property::value_iterator i=p->begin(),e=p->end() ; i!=e ; ++i) diff --git a/usr.bin/dtc/checking.hh b/usr.bin/dtc/checking.hh index 34d28c3..e3b3d45 100644 --- a/usr.bin/dtc/checking.hh +++ b/usr.bin/dtc/checking.hh @@ -86,7 +86,7 @@ class checker * Method for checking that a node is valid. The root class version * does nothing, subclasses should override this. */ - virtual bool check_node(device_tree *tree, const node_ptr &n) + virtual bool check_node(device_tree *, const node_ptr &) { return true; } @@ -94,7 +94,7 @@ class checker * Method for checking that a property is valid. The root class * version does nothing, subclasses should override this. */ - virtual bool check_property(device_tree *tree, const node_ptr &n, property_ptr p) + virtual bool check_property(device_tree *, const node_ptr &, property_ptr ) { return true; } @@ -160,7 +160,7 @@ struct property_type_checker <property_value::EMPTY> : public property_checker { property_type_checker(const char* name, string property_name) : property_checker(name, property_name) {} - virtual bool check(device_tree *tree, const node_ptr &n, property_ptr p) + virtual bool check(device_tree *, const node_ptr &, property_ptr p) { return p->begin() == p->end(); } @@ -175,7 +175,7 @@ struct property_type_checker <property_value::STRING> : public property_checker { property_type_checker(const char* name, string property_name) : property_checker(name, property_name) {} - virtual bool check(device_tree *tree, const node_ptr &n, property_ptr p) + virtual bool check(device_tree *, const node_ptr &, property_ptr p) { return (p->begin() + 1 == p->end()) && p->begin()->is_string(); } @@ -190,7 +190,7 @@ struct property_type_checker <property_value::STRING_LIST> : { property_type_checker(const char* name, string property_name) : property_checker(name, property_name) {} - virtual bool check(device_tree *tree, const node_ptr &n, property_ptr p) + virtual bool check(device_tree *, const node_ptr &, property_ptr p) { for (property::value_iterator i=p->begin(),e=p->end() ; i!=e ; ++i) @@ -213,7 +213,7 @@ struct property_type_checker <property_value::PHANDLE> : public property_checker { property_type_checker(const char* name, string property_name) : property_checker(name, property_name) {} - virtual bool check(device_tree *tree, const node_ptr &n, property_ptr p) + virtual bool check(device_tree *tree, const node_ptr &, property_ptr p) { return (p->begin() + 1 == p->end()) && (tree->referenced_node(*p->begin()) != 0); diff --git a/usr.bin/dtc/dtb.hh b/usr.bin/dtc/dtb.hh index a246e96..184369b 100644 --- a/usr.bin/dtc/dtb.hh +++ b/usr.bin/dtc/dtb.hh @@ -186,11 +186,11 @@ class binary_writer : public output_writer * The binary format does not support labels, so this method * does nothing. */ - virtual void write_label(string name) {} + virtual void write_label(string) {} /** * Comments are ignored by the binary writer. */ - virtual void write_comment(string name) {} + virtual void write_comment(string) {} virtual void write_string(string name); virtual void write_data(uint8_t v); virtual void write_data(uint32_t v); diff --git a/usr.bin/dtc/fdt.cc b/usr.bin/dtc/fdt.cc index 3908e0e..23476ae 100644 --- a/usr.bin/dtc/fdt.cc +++ b/usr.bin/dtc/fdt.cc @@ -264,24 +264,6 @@ property::parse_string(input_buffer &input) void property::parse_cells(input_buffer &input, int cell_size) { - unsigned long long cell_max; - switch (cell_size) - { - case 8: - cell_max = UINT8_MAX; - break; - case 16: - cell_max = UINT16_MAX; - break; - case 32: - cell_max = UINT32_MAX; - break; - case 64: - cell_max = UINT64_MAX; - break; - default: - assert(0 && "Invalid cell size!"); - } assert(input[0] == '<'); ++input; property_value v; @@ -327,19 +309,12 @@ property::parse_cells(input_buffer &input, int cell_size) //FIXME: We should support labels in the middle //of these, but we don't. unsigned long long val; - if (!input.consume_integer(val)) + if (!input.consume_integer_expression(val)) { input.parse_error("Expected numbers in array of cells"); valid = false; return; } - if (val > cell_max) - { - fprintf(stderr, "%lld > %lld\n", val, cell_max); - input.parse_error("Value out of range"); - valid = false; - return; - } switch (cell_size) { case 8: @@ -685,6 +660,16 @@ node::parse_name(input_buffer &input, bool &is_property, const char *error) return n; } +void +node::visit(std::function<void(node&)> fn) +{ + fn(*this); + for (auto &&c : children) + { + c->visit(fn); + } +} + node::node(input_buffer &structs, input_buffer &strings) : valid(true) { const char *name_start = (const char*)structs; @@ -742,7 +727,7 @@ node::node(input_buffer &structs, input_buffer &strings) : valid(true) valid = false; return; } - properties.push_back(prop); + props.push_back(prop); break; } break; @@ -806,7 +791,7 @@ node::node(input_buffer &input, string n, string l, string a, define_map *define } else { - properties.push_back(p); + props.push_back(p); } } else if (!is_property && input[0] == ('{')) @@ -824,7 +809,7 @@ node::node(input_buffer &input, string n, string l, string a, define_map *define } else if (input.consume(';')) { - properties.push_back(property_ptr(new property(child_name, child_label))); + props.push_back(property_ptr(new property(child_name, child_label))); } else { @@ -857,9 +842,9 @@ node::sort() { std::sort(property_begin(), property_end(), cmp_properties); std::sort(child_begin(), child_end(), cmp_children); - for (child_iterator i=child_begin(), e=child_end() ; i!=e ; ++i) + for (auto &c : child_nodes()) { - (*i)->sort(); + c->sort(); } } @@ -892,7 +877,7 @@ node::parse_dtb(input_buffer &structs, input_buffer &strings) property_ptr node::get_property(string key) { - for (auto &i : properties) + for (auto &i : props) { if (i->get_key() == key) { @@ -914,14 +899,14 @@ node::merge_node(node_ptr other) // large numbers of properties, but for typical usage the // entire vector will fit (easily) into cache, so iterating // over it repeatedly isn't that expensive. - for (auto &p : other->properties) + for (auto &p : other->properties()) { bool found = false; - for (auto i=property_begin(), e=property_end() ; i!=e ; ++i) + for (auto &mp : properties()) { - if ((*i)->get_key() == p->get_key()) + if (mp->get_key() == p->get_key()) { - *i = p; + mp = p; found = true; break; } @@ -964,13 +949,13 @@ node::write(dtb::output_writer &writer, dtb::string_table &strings) writer.write_comment(name); writer.write_data(name_buffer); writer.write_data((uint8_t)0); - for (auto i=property_begin(), e=property_end() ; i!=e ; ++i) + for (auto p : properties()) { - (*i)->write(writer, strings); + p->write(writer, strings); } - for (child_iterator i=child_begin(), e=child_end() ; i!=e ; ++i) + for (auto &c : child_nodes()) { - (*i)->write(writer, strings); + c->write(writer, strings); } writer.write_token(dtb::FDT_END_NODE); } @@ -999,13 +984,13 @@ node::write_dts(FILE *file, int indent) unit_address.print(file); } fputs(" {\n\n", file); - for (auto i=property_begin(), e=property_end() ; i!=e ; ++i) + for (auto p : properties()) { - (*i)->write_dts(file, indent+1); + p->write_dts(file, indent+1); } - for (child_iterator i=child_begin(), e=child_end() ; i!=e ; ++i) + for (auto &c : child_nodes()) { - (*i)->write_dts(file, indent+1); + c->write_dts(file, indent+1); } for (int i=0 ; i<indent ; i++) { @@ -1039,30 +1024,30 @@ device_tree::collect_names_recursive(node_ptr &n, node_path &path) fprintf(stderr, ". References to this label will not be resolved."); } } - for (node::child_iterator i=n->child_begin(), e=n->child_end() ; i!=e ; ++i) + for (auto &c : n->child_nodes()) { - collect_names_recursive(*i, path); + collect_names_recursive(c, path); } path.pop_back(); // Now we collect the phandles and properties that reference // other nodes. - for (auto i=n->property_begin(), e=n->property_end() ; i!=e ; ++i) + for (auto &p : n->properties()) { - for (property::value_iterator p=(*i)->begin(),pe=(*i)->end() ; p!=pe ; ++p) + for (auto &v : *p) { - if (p->is_phandle()) + if (v.is_phandle()) { - phandles.push_back(&*p); + phandles.push_back(&v); } - if (p->is_cross_reference()) + if (v.is_cross_reference()) { - cross_references.push_back(&*p); + cross_references.push_back(&v); } } - if ((*i)->get_key() == string("phandle") || - (*i)->get_key() == string("linux,phandle")) + if (p->get_key() == string("phandle") || + p->get_key() == string("linux,phandle")) { - if ((*i)->begin()->byte_data.size() != 4) + if (p->begin()->byte_data.size() != 4) { fprintf(stderr, "Invalid phandle value for node "); n->name.dump(); @@ -1071,7 +1056,7 @@ device_tree::collect_names_recursive(node_ptr &n, node_path &path) } else { - uint32_t phandle = (*i)->begin()->get_as_uint32(); + uint32_t phandle = p->begin()->get_as_uint32(); used_phandles.insert(std::make_pair(phandle, n.get())); } } @@ -1104,12 +1089,33 @@ device_tree::resolve_cross_references() { pv->byte_data.push_back('@'); p->second.push_to_buffer(pv->byte_data); + pv->byte_data.push_back(0); } } } - uint32_t phandle = 1; + std::unordered_set<property_value*> phandle_set; for (auto &i : phandles) { + phandle_set.insert(i); + } + std::vector<property_value*> sorted_phandles; + root->visit([&](node &n) { + for (auto &p : n.properties()) + { + for (auto &v : *p) + { + if (phandle_set.count(&v)) + { + sorted_phandles.push_back(&v); + } + } + } + }); + assert(sorted_phandles.size() == phandles.size()); + + uint32_t phandle = 1; + for (auto &i : sorted_phandles) + { string target_name = i->string_data; node *target = node_names[target_name]; if (target == 0) @@ -1163,94 +1169,111 @@ device_tree::resolve_cross_references() } } -void -device_tree::parse_file(input_buffer &input, +bool +device_tree::parse_include(input_buffer &input, const std::string &dir, std::vector<node_ptr> &roots, FILE *depfile, bool &read_header) { - input.next_token(); - // Read the header - if (input.consume("/dts-v1/;")) + if (!input.consume("/include/")) { - read_header = true; + return false; } - input.next_token(); - while(input.consume("/include/")) + bool reallyInclude = true; + if (input.consume("if ")) { - bool reallyInclude = true; - if (input.consume("if ")) - { - input.next_token(); - string name = string::parse_property_name(input); - // XXX: Error handling - if (defines.find(name) == defines.end()) - { - reallyInclude = false; - } - input.consume('/'); - } input.next_token(); - if (!input.consume('"')) + string name = string::parse_property_name(input); + // XXX: Error handling + if (defines.find(name) == defines.end()) { - input.parse_error("Expected quoted filename"); - valid = false; - return; + reallyInclude = false; } - int length = 0; - while (input[length] != '"') length++; + input.consume('/'); + } + input.next_token(); + if (!input.consume('"')) + { + input.parse_error("Expected quoted filename"); + valid = false; + return false; + } + int length = 0; + while (input[length] != '"') length++; - std::string file((const char*)input, length); - std::string include_file = dir + '/' + file; - assert(input.consume(file.c_str())); + std::string file((const char*)input, length); + std::string include_file = dir + '/' + file; + input.consume(file.c_str()); + if (!reallyInclude) + { input.consume('"'); input.next_token(); - if (!reallyInclude) - { - continue; - } + return true; + } - input_buffer *include_buffer = buffer_for_file(include_file.c_str()); + input_buffer *include_buffer = buffer_for_file(include_file.c_str(), false); - if (include_buffer == 0) + if (include_buffer == 0) + { + for (auto i : include_paths) { - for (auto i : include_paths) + include_file = i + '/' + file; + include_buffer = buffer_for_file(include_file.c_str()); + if (include_buffer != 0) { - include_file = i + '/' + file; - include_buffer = buffer_for_file(include_file.c_str()); - if (include_buffer != 0) - { - break; - } + break; } } - if (depfile != 0) - { - putc(' ', depfile); - fputs(include_file.c_str(), depfile); - } - if (include_buffer == 0) - { - valid = false; - return; - } - parse_file(*include_buffer, dir, roots, depfile, read_header); } + if (depfile != 0) + { + putc(' ', depfile); + fputs(include_file.c_str(), depfile); + } + if (include_buffer == 0) + { + input.parse_error("Unable to locate input file"); + input.consume('"'); + input.next_token(); + valid = false; + return true; + } + input.consume('"'); + input.next_token(); + parse_file(*include_buffer, dir, roots, depfile, read_header); + return true; +} + +void +device_tree::parse_file(input_buffer &input, + const std::string &dir, + std::vector<node_ptr> &roots, + FILE *depfile, + bool &read_header) +{ + input.next_token(); + // Read the header + if (input.consume("/dts-v1/;")) + { + read_header = true; + } + input.next_token(); input.next_token(); if (!read_header) { input.parse_error("Expected /dts-v1/; version string"); } + while(parse_include(input, dir, roots, depfile, read_header)) {} // Read any memory reservations while(input.consume("/memreserve/")) { unsigned long long start, len; input.next_token(); // Read the start and length. - if (!(input.consume_integer(start) && + if (!(input.consume_integer_expression(start) && (input.next_token(), - input.consume_integer(len)))) + input.consume_integer_expression(len)))) { input.parse_error("Expected size on /memreserve/ node."); } @@ -1259,6 +1282,7 @@ device_tree::parse_file(input_buffer &input, reservations.push_back(reservation(start, len)); } input.next_token(); + while(parse_include(input, dir, roots, depfile, read_header)) {} while (valid && !input.finished()) { node_ptr n; @@ -1287,11 +1311,12 @@ device_tree::parse_file(input_buffer &input, valid = false; } input.next_token(); + while(parse_include(input, dir, roots, depfile, read_header)) {} } } input_buffer* -device_tree::buffer_for_file(const char *path) +device_tree::buffer_for_file(const char *path, bool warn) { if (string(path) == string("-")) { @@ -1306,7 +1331,10 @@ device_tree::buffer_for_file(const char *path) int source = open(path, O_RDONLY); if (source == -1) { - fprintf(stderr, "Unable to open file '%s'. %s\n", path, strerror(errno)); + if (warn) + { + fprintf(stderr, "Unable to open file '%s'. %s\n", path, strerror(errno)); + } return 0; } struct stat st; @@ -1447,7 +1475,7 @@ device_tree::write_dts(int fd) } void -device_tree::parse_dtb(const char *fn, FILE *depfile) +device_tree::parse_dtb(const char *fn, FILE *) { input_buffer *in = buffer_for_file(fn); if (in == 0) diff --git a/usr.bin/dtc/fdt.hh b/usr.bin/dtc/fdt.hh index 7340a73..5f53f5a 100644 --- a/usr.bin/dtc/fdt.hh +++ b/usr.bin/dtc/fdt.hh @@ -36,6 +36,7 @@ #include <unordered_set> #include <memory> #include <string> +#include <functional> #include "util.hh" #include "string.hh" @@ -402,11 +403,37 @@ class node * The type for the property vector. */ typedef std::vector<property_ptr> property_vector; + /** + * Iterator type for child nodes. + */ + typedef std::vector<node_ptr>::iterator child_iterator; private: /** + * Adaptor to use children in range-based for loops. + */ + struct child_range + { + child_range(node &nd) : n(nd) {} + child_iterator begin() { return n.child_begin(); } + child_iterator end() { return n.child_end(); } + private: + node &n; + }; + /** + * Adaptor to use properties in range-based for loops. + */ + struct property_range + { + property_range(node &nd) : n(nd) {} + property_vector::iterator begin() { return n.property_begin(); } + property_vector::iterator end() { return n.property_end(); } + private: + node &n; + }; + /** * The properties contained within this node. */ - property_vector properties; + property_vector props; /** * The children of this node. */ @@ -458,10 +485,6 @@ class node */ void sort(); /** - * Iterator type for child nodes. - */ - typedef std::vector<node_ptr>::iterator child_iterator; - /** * Returns an iterator for the first child of this node. */ inline child_iterator child_begin() @@ -475,19 +498,27 @@ class node { return children.end(); } + inline child_range child_nodes() + { + return child_range(*this); + } + inline property_range properties() + { + return property_range(*this); + } /** * Returns an iterator after the last property of this node. */ inline property_vector::iterator property_begin() { - return properties.begin(); + return props.begin(); } /** * Returns an iterator for the first property of this node. */ inline property_vector::iterator property_end() { - return properties.end(); + return props.end(); } /** * Factory method for constructing a new node. Attempts to parse a @@ -519,7 +550,7 @@ class node */ inline void add_property(property_ptr &p) { - properties.push_back(p); + props.push_back(p); } /** * Merges a node into this one. Any properties present in both are @@ -539,6 +570,10 @@ class node * with this number of tabs. */ void write_dts(FILE *file, int indent); + /** + * Recursively visit this node and then its children. + */ + void visit(std::function<void(node&)>); }; /** @@ -683,6 +718,14 @@ class device_tree */ void resolve_cross_references(); /** + * Parse a top-level include directive. + */ + bool parse_include(input_buffer &input, + const std::string &dir, + std::vector<node_ptr> &roots, + FILE *depfile, + bool &read_header); + /** * Parses a dts file in the given buffer and adds the roots to the parsed * set. The `read_header` argument indicates whether the header has * already been read. Some dts files place the header in an include, @@ -698,7 +741,7 @@ class device_tree * object then keeps a reference to it, ensuring that it is not * deallocated until the device tree is destroyed. */ - input_buffer *buffer_for_file(const char *path); + input_buffer *buffer_for_file(const char *path, bool warn=true); /** * Template function that writes a dtb blob using the specified writer. * The writer defines the output format (assembly, blob). diff --git a/usr.bin/dtc/input_buffer.cc b/usr.bin/dtc/input_buffer.cc index fe8c402..6d48adc 100644 --- a/usr.bin/dtc/input_buffer.cc +++ b/usr.bin/dtc/input_buffer.cc @@ -37,6 +37,10 @@ #include <stdio.h> #include <stdlib.h> #include <string.h> +#include <functional> +#ifndef NDEBUG +#include <iostream> +#endif #include <sys/stat.h> @@ -49,7 +53,6 @@ namespace dtc { - void input_buffer::skip_spaces() { @@ -131,6 +134,563 @@ input_buffer::consume_integer(unsigned long long &outInt) return true; } +namespace { + +/** + * Convenience typedef for the type that we use for all values. + */ +typedef unsigned long long valty; + +/** + * Expression tree currently being parsed. + */ +struct expression +{ + /** + * Evaluate this node, taking into account operator precedence. + */ + virtual valty operator()() = 0; + /** + * Returns the precedence of this node. Lower values indicate higher + * precedence. + */ + virtual int precedence() = 0; + virtual ~expression() {} +#ifndef NDEBUG + /** + * Dumps this expression to `std::cerr`, appending a newline if `nl` is + * `true`. + */ + void dump(bool nl=false) + { + if (this == nullptr) + { + std::cerr << "{nullptr}\n"; + return; + } + dump_impl(); + if (nl) + { + std::cerr << '\n'; + } + } + private: + /** + * Method that sublcasses override to implement the behaviour of `dump()`. + */ + virtual void dump_impl() = 0; +#endif +}; + +/** + * Expression wrapping a single integer. Leaf nodes in the expression tree. + */ +class terminal_expr : public expression +{ + /** + * The value that this wraps. + */ + valty val; + /** + * Evaluate. Trivially returns the value that this class wraps. + */ + valty operator()() override + { + return val; + } + int precedence() override + { + return 0; + } + public: + /** + * Constructor. + */ + terminal_expr(valty v) : val(v) {} +#ifndef NDEBUG + void dump_impl() override { std::cerr << val; } +#endif +}; + +/** + * Parenthetical expression. Exists to make the contents opaque. + */ +struct paren_expression : public expression +{ + /** + * The expression within the parentheses. + */ + expression_ptr subexpr; + /** + * Constructor. Takes the child expression as the only argument. + */ + paren_expression(expression_ptr p) : subexpr(std::move(p)) {} + int precedence() override + { + return 0; + } + /** + * Evaluate - just forwards to the underlying expression. + */ + valty operator()() override + { + return (*subexpr)(); + } +#ifndef NDEBUG + void dump_impl() override + { + std::cerr << " ("; + subexpr->dump(); + std::cerr << ") "; + } +#endif +}; + +/** + * Template class for unary operators. The `OpChar` template parameter is + * solely for debugging and makes it easy to print the expression. The `Op` + * template parameter is a function object that implements the operator that + * this class provides. Most of these are provided by the `<functional>` + * header. + */ +template<char OpChar, class Op> +class unary_operator : public expression +{ + /** + * The subexpression for this unary operator. + */ + expression_ptr subexpr; + valty operator()() override + { + Op op; + return op((*subexpr)()); + } + /** + * All unary operators have the same precedence. They are all evaluated + * before binary expressions, but after parentheses. + */ + int precedence() override + { + return 3; + } + public: + unary_operator(expression_ptr p) : subexpr(std::move(p)) {} +#ifndef NDEBUG + void dump_impl() override + { + std::cerr << OpChar; + subexpr->dump(); + } +#endif +}; + +/** + * Abstract base class for binary operators. Allows the tree to be modified + * without knowing what the operations actually are. + */ +struct binary_operator_base : public expression +{ + /** + * The left side of the expression. + */ + expression_ptr lhs; + /** + * The right side of the expression. + */ + expression_ptr rhs; + /** + * Insert a node somewhere down the path of left children, until it would + * be preempting something that should execute first. + */ + void insert_left(binary_operator_base *new_left) + { + if (lhs->precedence() < new_left->precedence()) + { + new_left->rhs = std::move(lhs); + lhs.reset(new_left); + } + else + { + static_cast<binary_operator_base*>(lhs.get())->insert_left(new_left); + } + } +}; + +/** + * Template class for binary operators. The precedence and the operation are + * provided as template parameters. + */ +template<int Precedence, class Op> +struct binary_operator : public binary_operator_base +{ + valty operator()() override + { + Op op; + return op((*lhs)(), (*rhs)()); + } + int precedence() override + { + return Precedence; + } +#ifdef NDEBUG + /** + * Constructor. Takes the name of the operator as an argument, for + * debugging. Only stores it in debug mode. + */ + binary_operator(const char *) {} +#else + const char *opName; + binary_operator(const char *o) : opName(o) {} + void dump_impl() override + { + lhs->dump(); + std::cerr << opName; + rhs->dump(); + } +#endif +}; + +/** + * Ternary conditional operators (`cond ? true : false`) are a special case - + * there are no other ternary operators. + */ +class ternary_conditional_operator : public expression +{ + /** + * The condition for the clause. + */ + expression_ptr cond; + /** + * The expression that this evaluates to if the condition is true. + */ + expression_ptr lhs; + /** + * The expression that this evaluates to if the condition is false. + */ + expression_ptr rhs; + valty operator()() override + { + return (*cond)() ? (*lhs)() : (*rhs)(); + } + int precedence() override + { + // The actual precedence of a ternary conditional operator is 15, but + // its associativity is the opposite way around to the other operators, + // so we fudge it slightly. + return 3; + } +#ifndef NDEBUG + void dump_impl() override + { + cond->dump(); + std::cerr << " ? "; + lhs->dump(); + std::cerr << " : "; + rhs->dump(); + } +#endif + public: + ternary_conditional_operator(expression_ptr c, + expression_ptr l, + expression_ptr r) : + cond(std::move(c)), lhs(std::move(l)), rhs(std::move(r)) {} +}; + +template<typename T> +struct lshift +{ + constexpr T operator()(const T &lhs, const T &rhs) const + { + return lhs << rhs; + } +}; +template<typename T> +struct rshift +{ + constexpr T operator()(const T &lhs, const T &rhs) const + { + return lhs >> rhs; + } +}; +template<typename T> +struct unary_plus +{ + constexpr T operator()(const T &val) const + { + return +val; + } +}; +// TODO: Replace with std::bit_not once we can guarantee C++14 as a baseline. +template<typename T> +struct bit_not +{ + constexpr T operator()(const T &val) const + { + return ~val; + } +}; + +} // anonymous namespace + + +expression_ptr input_buffer::parse_binary_expression(expression_ptr lhs) +{ + next_token(); + binary_operator_base *expr = nullptr; + char op = ((*this)[0]); + switch (op) + { + default: + return lhs; + case '+': + expr = new binary_operator<6, std::plus<valty>>("+"); + break; + case '-': + expr = new binary_operator<6, std::minus<valty>>("-"); + break; + case '%': + expr = new binary_operator<5, std::modulus<valty>>("%"); + break; + case '*': + expr = new binary_operator<5, std::multiplies<valty>>("*"); + break; + case '/': + expr = new binary_operator<5, std::divides<valty>>("/"); + break; + case '<': + cursor++; + switch ((*this)[0]) + { + default: + parse_error("Invalid operator"); + return nullptr; + case ' ': + case '(': + case '0'...'9': + cursor--; + expr = new binary_operator<8, std::less<valty>>("<"); + break; + case '=': + expr = new binary_operator<8, std::less_equal<valty>>("<="); + break; + case '<': + expr = new binary_operator<7, lshift<valty>>("<<"); + break; + } + break; + case '>': + cursor++; + switch ((*this)[0]) + { + default: + parse_error("Invalid operator"); + return nullptr; + case '(': + case ' ': + case '0'...'9': + cursor--; + expr = new binary_operator<8, std::greater<valty>>(">"); + break; + case '=': + expr = new binary_operator<8, std::greater_equal<valty>>(">="); + break; + case '>': + expr = new binary_operator<7, rshift<valty>>(">>"); + break; + return lhs; + } + break; + case '=': + if ((*this)[1] != '=') + { + parse_error("Invalid operator"); + return nullptr; + } + consume('='); + expr = new binary_operator<9, std::equal_to<valty>>("=="); + break; + case '!': + if ((*this)[1] != '=') + { + parse_error("Invalid operator"); + return nullptr; + } + cursor++; + expr = new binary_operator<9, std::not_equal_to<valty>>("!="); + break; + case '&': + if ((*this)[1] == '&') + { + expr = new binary_operator<13, std::logical_and<valty>>("&&"); + } + else + { + expr = new binary_operator<10, std::bit_and<valty>>("&"); + } + break; + case '|': + if ((*this)[1] == '|') + { + expr = new binary_operator<12, std::logical_or<valty>>("||"); + } + else + { + expr = new binary_operator<14, std::bit_or<valty>>("|"); + } + break; + case '?': + { + consume('?'); + expression_ptr true_case = parse_expression(); + next_token(); + if (!true_case || !consume(':')) + { + parse_error("Expected : in ternary conditional operator"); + return nullptr; + } + expression_ptr false_case = parse_expression(); + if (!false_case) + { + parse_error("Expected false condition for ternary operator"); + return nullptr; + } + return expression_ptr(new ternary_conditional_operator(std::move(lhs), + std::move(true_case), std::move(false_case))); + } + } + cursor++; + next_token(); + expression_ptr e(expr); + expression_ptr rhs(parse_expression()); + if (!rhs) + { + return nullptr; + } + expr->lhs = std::move(lhs); + if (rhs->precedence() < expr->precedence()) + { + expr->rhs = std::move(rhs); + } + else + { + // If we're a normal left-to-right expression, then we need to insert + // this as the far-left child node of the rhs expression + binary_operator_base *rhs_op = + static_cast<binary_operator_base*>(rhs.get()); + rhs_op->insert_left(expr); + e.release(); + return rhs; + } + return e; +} + +expression_ptr input_buffer::parse_expression(bool stopAtParen) +{ + next_token(); + unsigned long long leftVal; + expression_ptr lhs; + switch ((*this)[0]) + { + case '0'...'9': + if (!consume_integer(leftVal)) + { + return nullptr; + } + lhs.reset(new terminal_expr(leftVal)); + break; + case '(': + { + consume('('); + expression_ptr &&subexpr = parse_expression(); + if (!subexpr) + { + return nullptr; + } + lhs.reset(new paren_expression(std::move(subexpr))); + if (!consume(')')) + { + return nullptr; + } + if (stopAtParen) + { + return lhs; + } + break; + } + case '+': + { + consume('+'); + expression_ptr &&subexpr = parse_expression(); + if (!subexpr) + { + return nullptr; + } + lhs.reset(new unary_operator<'+', unary_plus<valty>>(std::move(subexpr))); + break; + } + case '-': + { + consume('-'); + expression_ptr &&subexpr = parse_expression(); + if (!subexpr) + { + return nullptr; + } + lhs.reset(new unary_operator<'-', std::negate<valty>>(std::move(subexpr))); + break; + } + case '!': + { + consume('!'); + expression_ptr &&subexpr = parse_expression(); + if (!subexpr) + { + return nullptr; + } + lhs.reset(new unary_operator<'!', std::logical_not<valty>>(std::move(subexpr))); + break; + } + case '~': + { + consume('~'); + expression_ptr &&subexpr = parse_expression(); + if (!subexpr) + { + return nullptr; + } + lhs.reset(new unary_operator<'~', bit_not<valty>>(std::move(subexpr))); + break; + } + } + if (!lhs) + { + return nullptr; + } + return parse_binary_expression(std::move(lhs)); +} + +bool +input_buffer::consume_integer_expression(unsigned long long &outInt) +{ + switch ((*this)[0]) + { + case '(': + { + expression_ptr e(parse_expression(true)); + if (!e) + { + return false; + } + outInt = (*e)(); + return true; + } + case '0'...'9': + return consume_integer(outInt); + default: + return false; + } +} + bool input_buffer::consume_hex_byte(uint8_t &outByte) { @@ -222,12 +782,14 @@ input_buffer::parse_error(const char *msg) putc('^', stderr); putc('\n', stderr); } +#ifndef NDEBUG void input_buffer::dump() { fprintf(stderr, "Current cursor: %d\n", cursor); fwrite(&buffer[cursor], size-cursor, 1, stderr); } +#endif mmap_input_buffer::mmap_input_buffer(int fd) : input_buffer(0, 0) { diff --git a/usr.bin/dtc/input_buffer.hh b/usr.bin/dtc/input_buffer.hh index 5b1f5d6..1a87911 100644 --- a/usr.bin/dtc/input_buffer.hh +++ b/usr.bin/dtc/input_buffer.hh @@ -38,6 +38,11 @@ namespace dtc { +namespace { +struct expression; +typedef std::unique_ptr<expression> expression_ptr; +} + /** * Class encapsulating the input file. Can be used as a const char*, but has * range checking. Attempting to access anything out of range will return a 0 @@ -62,6 +67,17 @@ class input_buffer int size; private: /** + * Parse an expression. If `stopAtParen` is set, then only parse a number + * or a parenthetical expression, otherwise assume that either is the + * left-hand side of a binary expression and try to parse the right-hand + * side. + */ + expression_ptr parse_expression(bool stopAtParen=false); + /** + * Parse a binary expression, having already parsed the right-hand side. + */ + expression_ptr parse_binary_expression(expression_ptr lhs); + /** * The current place in the buffer where we are reading. This class * keeps a separate size, pointer, and cursor so that we can move * forwards and backwards and still have checks that we haven't fallen @@ -187,6 +203,11 @@ class input_buffer */ bool consume_integer(unsigned long long &outInt); /** + * Reads an arithmetic expression (containing any of the normal C + * operators), evaluates it, and returns the result. + */ + bool consume_integer_expression(unsigned long long &outInt); + /** * Template function that consumes a binary value in big-endian format * from the input stream. Returns true and advances the cursor if * there is a value of the correct size. This function assumes that @@ -233,12 +254,14 @@ class input_buffer * Prints a message indicating the location of a parse error. */ void parse_error(const char *msg); +#ifndef NDEBUG /** * Dumps the current cursor value and the unconsumed values in the * input buffer to the standard error. This method is intended solely * for debugging. */ void dump(); +#endif }; /** * Explicit specialisation for reading a single byte. diff --git a/usr.bin/kdump/Makefile b/usr.bin/kdump/Makefile index 52c0a09..f149e80 100644 --- a/usr.bin/kdump/Makefile +++ b/usr.bin/kdump/Makefile @@ -6,7 +6,7 @@ .PATH: ${.CURDIR}/../ktrace PROG= kdump -SRCS= kdump_subr.c kdump_subr.h kdump.c ioctl.c subr.c +SRCS= kdump_subr.c kdump_subr.h kdump.c subr.c CFLAGS+= -I${.CURDIR}/../ktrace -I${.CURDIR} -I${.CURDIR}/../.. -I. LIBADD= sysdecode @@ -15,15 +15,9 @@ LIBADD+= capsicum CFLAGS+=-DHAVE_LIBCAPSICUM .endif -.if ${MK_PF} != "no" -CFLAGS+=-DPF -.endif - NO_WERROR?= YES -CLEANFILES= ioctl.c kdump_subr.c kdump_subr.h - -beforedepend: ioctl.c +CLEANFILES= kdump_subr.c kdump_subr.h .if (${MACHINE_ARCH} == "amd64" || ${MACHINE_ARCH} == "i386") beforedepend: linux_syscalls.c @@ -44,10 +38,6 @@ linux32_syscalls.c: linux32_syscalls.conf ${.CURDIR}/../../sys/${MACHINE_ARCH}/linux32/syscalls.master ${.CURDIR}/linux32_syscalls.conf .endif -ioctl.c: mkioctls - env MACHINE=${MACHINE} CPP="${CPP}" \ - sh ${.CURDIR}/mkioctls print ${DESTDIR}${INCLUDEDIR} > ${.TARGET} - kdump_subr.h: mksubr sh ${.CURDIR}/mksubr ${DESTDIR}${INCLUDEDIR} | \ sed -n 's/^\([a-z].*)\)$$/void \1;/p' >${.TARGET} diff --git a/usr.bin/kdump/kdump.c b/usr.bin/kdump/kdump.c index 8165bc7..7af99fb 100644 --- a/usr.bin/kdump/kdump.c +++ b/usr.bin/kdump/kdump.c @@ -116,7 +116,6 @@ void ktrfault(struct ktr_fault *); void ktrfaultend(struct ktr_faultend *); void limitfd(int fd); void usage(void); -void ioctlname(unsigned long, int); #define TIMESTAMP_NONE 0x0 #define TIMESTAMP_ABSOLUTE 0x1 @@ -693,6 +692,20 @@ dumpheader(struct ktr_header *kth) #undef KTRACE int nsyscalls = sizeof (syscallnames) / sizeof (syscallnames[0]); +static void +ioctlname(unsigned long val) +{ + const char *str; + + str = sysdecode_ioctlname(val); + if (str != NULL) + printf("%s", str); + else if (decimal) + printf("%lu", val); + else + printf("%#lx", val); +} + void ktrsyscall(struct ktr_syscall *ktr, u_int flags) { @@ -741,7 +754,7 @@ ktrsyscall(struct ktr_syscall *ktr, u_int flags) case SYS_ioctl: { print_number(ip, narg, c); putchar(c); - ioctlname(*ip, decimal); + ioctlname(*ip); c = ','; ip++; narg--; diff --git a/usr.bin/locate/locate/util.c b/usr.bin/locate/locate/util.c index 3ac69b5..9130e37 100644 --- a/usr.bin/locate/locate/util.c +++ b/usr.bin/locate/locate/util.c @@ -93,7 +93,7 @@ colon(dbv, path, dot) char **pv; if (dbv == NULL) { - if ((dbv = malloc(sizeof(char **))) == NULL) + if ((dbv = malloc(sizeof(char *))) == NULL) err(1, "malloc"); *dbv = NULL; } @@ -123,7 +123,7 @@ colon(dbv, path, dot) *(p + slen) = '\0'; } /* increase dbv with element p */ - if ((dbv = realloc(dbv, sizeof(char **) * (vlen + 2))) + if ((dbv = realloc(dbv, sizeof(char *) * (vlen + 2))) == NULL) err(1, "realloc"); *(dbv + vlen) = p; diff --git a/usr.bin/nfsstat/Makefile b/usr.bin/nfsstat/Makefile index fc92008..e45b5e2 100644 --- a/usr.bin/nfsstat/Makefile +++ b/usr.bin/nfsstat/Makefile @@ -4,6 +4,4 @@ PROG= nfsstat CFLAGS+=-DNFS -LIBADD= kvm - .include <bsd.prog.mk> diff --git a/usr.bin/nfsstat/nfsstat.c b/usr.bin/nfsstat/nfsstat.c index 32fbc96..896a05b 100644 --- a/usr.bin/nfsstat/nfsstat.c +++ b/usr.bin/nfsstat/nfsstat.c @@ -60,7 +60,6 @@ static const char rcsid[] = #include <fcntl.h> #include <ctype.h> #include <errno.h> -#include <kvm.h> #include <limits.h> #include <nlist.h> #include <unistd.h> diff --git a/usr.bin/systat/vmstat.c b/usr.bin/systat/vmstat.c index 258e357..ca8b6bc 100644 --- a/usr.bin/systat/vmstat.c +++ b/usr.bin/systat/vmstat.c @@ -108,6 +108,7 @@ static struct Info { u_int v_active_count; /* number of pages active */ u_int v_inactive_count; /* number of pages inactive */ u_int v_cache_count; /* number of pages on buffer cache queue */ + u_long v_kmem_map_size; /* Current kmem allocation size */ struct vmtotal Total; struct nchstats nchstats; long nchcount; @@ -118,6 +119,8 @@ static struct Info { long freevnodes; int numdirtybuffers; } s, s1, s2, z; +static u_long kmem_size; +static u_int v_page_count; struct statinfo cur, last, run; @@ -278,6 +281,8 @@ initkre(void) allocinfo(&s2); allocinfo(&z); } + GETSYSCTL("vm.kmem_size", kmem_size); + GETSYSCTL("vm.stats.vm.v_page_count", v_page_count); getinfo(&s2); copyinfo(&s2, &s1); return(1); @@ -307,7 +312,8 @@ labelkre(void) clear(); mvprintw(STATROW, STATCOL + 6, "users Load"); - mvprintw(MEMROW, MEMCOL, "Mem:KB REAL VIRTUAL"); + mvprintw(STATROW + 1, STATCOL + 3, "Mem usage: %%Phy %%Kmem"); + mvprintw(MEMROW, MEMCOL, "Mem: KB REAL VIRTUAL"); mvprintw(MEMROW + 1, MEMCOL, " Tot Share Tot Share"); mvprintw(MEMROW + 2, MEMCOL, "Act"); mvprintw(MEMROW + 3, MEMCOL, "All"); @@ -478,6 +484,11 @@ showkre(void) putfloat(avenrun[2], STATROW, STATCOL + 32, 5, 2, 0); mvaddstr(STATROW, STATCOL + 55, buf); #define pgtokb(pg) ((pg) * (s.v_page_size / 1024)) + putfloat(100.0 * (v_page_count - total.t_free) / v_page_count, + STATROW + 1, STATCOL + 15, 2, 0, 1); + putfloat(100.0 * s.v_kmem_map_size / kmem_size, + STATROW + 1, STATCOL + 22, 2, 0, 1); + putint(pgtokb(total.t_arm), MEMROW + 2, MEMCOL + 4, 7); putint(pgtokb(total.t_armshr), MEMROW + 2, MEMCOL + 12, 7); putint(pgtokb(total.t_avm), MEMROW + 2, MEMCOL + 20, 8); @@ -790,6 +801,7 @@ getinfo(struct Info *ls) GETSYSCTL("vfs.freevnodes", ls->freevnodes); GETSYSCTL("vfs.cache.nchstats", ls->nchstats); GETSYSCTL("vfs.numdirtybuffers", ls->numdirtybuffers); + GETSYSCTL("vm.kmem_map_size", ls->v_kmem_map_size); getsysctl("hw.intrcnt", ls->intrcnt, nintr * sizeof(u_long)); size = sizeof(ls->Total); diff --git a/usr.bin/truss/Makefile b/usr.bin/truss/Makefile index a300da1..a6e4524 100644 --- a/usr.bin/truss/Makefile +++ b/usr.bin/truss/Makefile @@ -2,16 +2,11 @@ NO_WERROR= PROG= truss -SRCS= cloudabi.c ioctl.c main.c setup.c syscalls.c +SRCS= cloudabi.c main.c setup.c syscalls.c LIBADD= sysdecode CFLAGS+= -I${.CURDIR} -I. -I${.CURDIR}/../../sys -CLEANFILES= ioctl.c - -ioctl.c: ${.CURDIR}/../kdump/mkioctls - env MACHINE=${MACHINE} CPP="${CPP}" \ - /bin/sh ${.CURDIR}/../kdump/mkioctls return ${DESTDIR}${INCLUDEDIR} > ${.TARGET} # Define where to generate syscalls for each ABI. ABI_SYSPATH.freebsd= sys/kern diff --git a/usr.bin/truss/syscalls.c b/usr.bin/truss/syscalls.c index 990403e6..8ff57db 100644 --- a/usr.bin/truss/syscalls.c +++ b/usr.bin/truss/syscalls.c @@ -1315,7 +1315,7 @@ print_arg(struct syscall_args *sc, unsigned long *args, long *retval, unsigned long cmd; cmd = args[sc->offset]; - temp = ioctlname(cmd); + temp = sysdecode_ioctlname(cmd); if (temp) fputs(temp, fp); else { diff --git a/usr.bin/vmstat/vmstat.c b/usr.bin/vmstat/vmstat.c index 67b5c0c..03f4f96 100644 --- a/usr.bin/vmstat/vmstat.c +++ b/usr.bin/vmstat/vmstat.c @@ -886,9 +886,9 @@ printhdr(int maxid, u_long cpumask) num_shown = (num_selected < maxshowdevs) ? num_selected : maxshowdevs; if (hflag) { - xo_emit("{T:procs} {T:memory} ${T:/page%*s}", 19, ""); + xo_emit("{T:procs} {T:memory} {T:/page%*s}", 19, ""); } else { - xo_emit("{T:procs} {T:memory} ${T:/page%*s}", 19, ""); + xo_emit("{T:procs} {T:memory} {T:/page%*s}", 19, ""); } if (num_shown > 1) xo_emit(" {T:/disks %*s}", num_shown * 4 - 7, ""); diff --git a/usr.bin/xargs/xargs.c b/usr.bin/xargs/xargs.c index c56ab60..955711c 100644 --- a/usr.bin/xargs/xargs.c +++ b/usr.bin/xargs/xargs.c @@ -234,7 +234,7 @@ main(int argc, char *argv[]) * NULL. */ linelen = 1 + argc + nargs + 1; - if ((av = bxp = malloc(linelen * sizeof(char **))) == NULL) + if ((av = bxp = malloc(linelen * sizeof(char *))) == NULL) errx(1, "malloc failed"); /* @@ -471,7 +471,7 @@ prerun(int argc, char *argv[]) * Allocate memory to hold the argument list, and * a NULL at the tail. */ - tmp = malloc((argc + 1) * sizeof(char**)); + tmp = malloc((argc + 1) * sizeof(char *)); if (tmp == NULL) { warnx("malloc failed"); xexit(*argv, 1); diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 3bec616..16feb68 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -46,6 +46,7 @@ SUBDIR= adduser \ mixer \ mlxcontrol \ mountd \ + mount_smbfs \ mpsutil \ mptutil \ mtest \ diff --git a/usr.sbin/Makefile.amd64 b/usr.sbin/Makefile.amd64 index 3f40974..0fdccaf 100644 --- a/usr.sbin/Makefile.amd64 +++ b/usr.sbin/Makefile.amd64 @@ -25,7 +25,6 @@ SUBDIR+= hyperv .endif SUBDIR+= kgmon SUBDIR+= lptcontrol -SUBDIR+= mount_smbfs SUBDIR+= mptable .if ${MK_NDIS} != "no" SUBDIR+= ndiscvt diff --git a/usr.sbin/Makefile.arm b/usr.sbin/Makefile.arm index 590b8cd..84cff4a 100644 --- a/usr.sbin/Makefile.arm +++ b/usr.sbin/Makefile.arm @@ -1,5 +1,4 @@ # $FreeBSD$ SUBDIR+= kgmon -SUBDIR+= mount_smbfs SUBDIR+= ofwdump diff --git a/usr.sbin/Makefile.i386 b/usr.sbin/Makefile.i386 index 772c11d..6c99daf 100644 --- a/usr.sbin/Makefile.i386 +++ b/usr.sbin/Makefile.i386 @@ -12,7 +12,6 @@ SUBDIR+= cpucontrol SUBDIR+= kgmon SUBDIR+= kgzip SUBDIR+= lptcontrol -SUBDIR+= mount_smbfs SUBDIR+= mptable .if ${MK_NDIS} != "no" SUBDIR+= ndiscvt diff --git a/usr.sbin/Makefile.powerpc b/usr.sbin/Makefile.powerpc index 8833a27..131eb57 100644 --- a/usr.sbin/Makefile.powerpc +++ b/usr.sbin/Makefile.powerpc @@ -1,5 +1,4 @@ # $FreeBSD$ -SUBDIR+= mount_smbfs SUBDIR+= nvram SUBDIR+= ofwdump diff --git a/usr.sbin/Makefile.sparc64 b/usr.sbin/Makefile.sparc64 index 632b3a8..81f7a9b 100644 --- a/usr.sbin/Makefile.sparc64 +++ b/usr.sbin/Makefile.sparc64 @@ -1,5 +1,4 @@ # $FreeBSD$ SUBDIR+= eeprom -SUBDIR+= mount_smbfs SUBDIR+= ofwdump diff --git a/usr.sbin/bsdinstall/scripts/entropy b/usr.sbin/bsdinstall/scripts/entropy index add6f30..d3938f1 100755 --- a/usr.sbin/bsdinstall/scripts/entropy +++ b/usr.sbin/bsdinstall/scripts/entropy @@ -26,4 +26,9 @@ # # $FreeBSD$ -dd if=/dev/random of=$BSDINSTALL_CHROOT/entropy bs=4096 count=1 +umask 077 +for i in /entropy /boot/entropy; do + i="$BSDINSTALL_CHROOT/$i" + dd if=/dev/random of="$i" bs=4096 count=1 + chown 0:0 "$i" +done diff --git a/usr.sbin/camdd/Makefile b/usr.sbin/camdd/Makefile index 4007ca7..64ee61b 100644 --- a/usr.sbin/camdd/Makefile +++ b/usr.sbin/camdd/Makefile @@ -3,7 +3,7 @@ PROG= camdd SRCS= camdd.c SDIR= ${.CURDIR}/../../sys -LIBADD= cam mt sbuf bsdxml util pthread +LIBADD= cam mt util pthread NO_WTHREAD_SAFETY= 1 MAN= camdd.8 diff --git a/usr.sbin/cron/cron/do_command.c b/usr.sbin/cron/cron/do_command.c index 8dfd4f6..7a7ac57 100644 --- a/usr.sbin/cron/cron/do_command.c +++ b/usr.sbin/cron/cron/do_command.c @@ -161,8 +161,10 @@ child_process(e, u) /* create some pipes to talk to our future child */ - pipe(stdin_pipe); /* child's stdin */ - pipe(stdout_pipe); /* child's stdout */ + if (pipe(stdin_pipe) != 0 || pipe(stdout_pipe) != 0) { + log_it("CRON", getpid(), "error", "can't pipe"); + exit(ERROR_EXIT); + } /* since we are a forked process, we can diddle the command string * we were passed -- nobody else is going to use it again, right? diff --git a/usr.sbin/cron/cron/popen.c b/usr.sbin/cron/cron/popen.c index 428de2e..57b1f77 100644 --- a/usr.sbin/cron/cron/popen.c +++ b/usr.sbin/cron/cron/popen.c @@ -82,9 +82,8 @@ cron_popen(program, type, e) if (!pids) { if ((fds = getdtablesize()) <= 0) return(NULL); - if (!(pids = (PID_T *)malloc((u_int)(fds * sizeof(PID_T))))) + if (!(pids = calloc(fds, sizeof(PID_T)))) return(NULL); - bzero((char *)pids, fds * sizeof(PID_T)); } if (pipe(pdes) < 0) return(NULL); diff --git a/usr.sbin/cron/crontab/crontab.c b/usr.sbin/cron/crontab/crontab.c index 1ac81b0..f2e97a7 100644 --- a/usr.sbin/cron/crontab/crontab.c +++ b/usr.sbin/cron/crontab/crontab.c @@ -558,7 +558,7 @@ replace_cmd() { case FALSE: e = load_entry(tmp, check_error, pw, envp); if (e) - free(e); + free_entry(e); break; case TRUE: break; diff --git a/usr.sbin/fstyp/Makefile b/usr.sbin/fstyp/Makefile index e82dd04..5eba12b 100644 --- a/usr.sbin/fstyp/Makefile +++ b/usr.sbin/fstyp/Makefile @@ -19,6 +19,8 @@ WARNS?= 2 SUBDIR+= tests .endif +CFLAGS+=-I${.CURDIR}/../../sys + .if ${MK_ZFS} != "no" IGNORE_PRAGMA= YES @@ -34,8 +36,6 @@ CFLAGS+= -I${.CURDIR}/../../sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${.CURDIR}/../../cddl/contrib/opensolaris/head .endif -CFLAGS+=-I${.CURDIR}/../../sys - LIBADD= geom md .if ${MK_ZFS} != "no" diff --git a/usr.sbin/fstyp/fstyp.c b/usr.sbin/fstyp/fstyp.c index cd0bbf9..cde179f 100644 --- a/usr.sbin/fstyp/fstyp.c +++ b/usr.sbin/fstyp/fstyp.c @@ -39,7 +39,6 @@ __FBSDID("$FreeBSD$"); #include <errno.h> #include <stdbool.h> #include <stddef.h> -#include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> diff --git a/usr.sbin/fstyp/geli.c b/usr.sbin/fstyp/geli.c index 7f6d9a5..59e8653 100644 --- a/usr.sbin/fstyp/geli.c +++ b/usr.sbin/fstyp/geli.c @@ -29,11 +29,9 @@ __FBSDID("$FreeBSD$"); #include <sys/disk.h> #include <sys/types.h> -#include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> -#include <unistd.h> #include <geom/eli/g_eli.h> diff --git a/usr.sbin/fstyp/zfs.c b/usr.sbin/fstyp/zfs.c index 1c9ca4d..c37a5db 100644 --- a/usr.sbin/fstyp/zfs.c +++ b/usr.sbin/fstyp/zfs.c @@ -29,6 +29,9 @@ __FBSDID("$FreeBSD$"); #include <sys/types.h> +#include <cddl/compat/opensolaris/sys/types.h> +#include <sys/time.h> +#include <cddl/compat/opensolaris/sys/time.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> diff --git a/usr.sbin/jls/Makefile b/usr.sbin/jls/Makefile index d90d094..4c1de74 100644 --- a/usr.sbin/jls/Makefile +++ b/usr.sbin/jls/Makefile @@ -4,7 +4,7 @@ PROG= jls MAN= jls.8 -LIBADD= jail +LIBADD= jail xo .if ${MK_INET6_SUPPORT} != "no" CFLAGS+= -DINET6 diff --git a/usr.sbin/jls/jls.8 b/usr.sbin/jls/jls.8 index 15a80d4..3b83d73 100644 --- a/usr.sbin/jls/jls.8 +++ b/usr.sbin/jls/jls.8 @@ -33,6 +33,7 @@ .Nd "list jails" .Sh SYNOPSIS .Nm +.Op Fl -libxo .Op Fl dhNnqsv .Op Fl j Ar jail .Op Ar parameter ... @@ -62,6 +63,13 @@ and path (path). .Pp The following options are available: .Bl -tag -width indent +.It Fl -libxo +Generate output via +.Xr libxo 3 +in a selection of different human and machine readable formats. +See +.Xr xo_parse_args 3 +for details on command line arguments. .It Fl d List .Va dying @@ -106,7 +114,9 @@ Without this option, all active jails will be listed. .Sh SEE ALSO .Xr jail_get 2 , .Xr jail 8 , -.Xr jexec 8 +.Xr jexec 8 , +.Xr libxo 3 , +.Xr xo_parse_args 3 .Sh HISTORY The .Nm @@ -114,3 +124,5 @@ utility was added in .Fx 5.1 . Extensible jail parameters were introduced in .Fx 8.0 . +libxo support was added in +.Fx 11.0 . diff --git a/usr.sbin/jls/jls.c b/usr.sbin/jls/jls.c index fbd1694..9dd7641 100644 --- a/usr.sbin/jls/jls.c +++ b/usr.sbin/jls/jls.c @@ -2,6 +2,7 @@ * Copyright (c) 2003 Mike Barcroft <mike@FreeBSD.org> * Copyright (c) 2008 Bjoern A. Zeeb <bz@FreeBSD.org> * Copyright (c) 2009 James Gritton <jamie@FreeBSD.org> + * Copyright (c) 2015 Emmanuel Vadot <manu@bocal.org> * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -45,10 +46,13 @@ __FBSDID("$FreeBSD$"); #include <stdlib.h> #include <string.h> #include <unistd.h> +#include <libxo/xo.h> #define JP_USER 0x01000000 #define JP_OPT 0x02000000 +#define JLS_XO_VERSION "1" + #define PRINT_DEFAULT 0x01 #define PRINT_HEADER 0x02 #define PRINT_NAMEVAL 0x04 @@ -73,7 +77,7 @@ static int sort_param(const void *a, const void *b); static char *noname(const char *name); static char *nononame(const char *name); static int print_jail(int pflags, int jflags); -static void quoted_print(char *str); +static void quoted_print(int pflags, char *name, char *value); int main(int argc, char **argv) @@ -81,6 +85,11 @@ main(int argc, char **argv) char *dot, *ep, *jname, *pname; int c, i, jflags, jid, lastjid, pflags, spc; + argc = xo_parse_args(argc, argv); + if (argc < 0) + exit(1); + + xo_set_version(JLS_XO_VERSION); jname = NULL; pflags = jflags = jid = 0; while ((c = getopt(argc, argv, "adj:hNnqsv")) >= 0) @@ -119,7 +128,7 @@ main(int argc, char **argv) PRINT_VERBOSE; break; default: - errx(1, "usage: jls [-dhNnqv] [-j jail] [param ...]"); + xo_errx(1, "usage: jls [-dhNnqv] [-j jail] [param ...]"); } #ifdef INET6 @@ -196,42 +205,48 @@ main(int argc, char **argv) add_param("lastjid", &lastjid, sizeof(lastjid), NULL, 0); /* Print a header line if requested. */ - if (pflags & PRINT_VERBOSE) - printf(" JID Hostname Path\n" - " Name State\n" - " CPUSetID\n" - " IP Address(es)\n"); + if (pflags & PRINT_VERBOSE) { + xo_emit("{T:/%3s}{T:JID}{P: }{T:Hostname}{Pd:/%22s}{T:Path}\n", + "", ""); + xo_emit("{P:/%8s}{T:Name}{Pd:/%26s}{T:State}\n", "", ""); + xo_emit("{P:/%8s}{T:CPUSetID}\n", ""); + xo_emit("{P:/%8s}{T:IP Address(es)}\n", ""); + } else if (pflags & PRINT_DEFAULT) if (pflags & PRINT_JAIL_NAME) - printf(" JID IP Address " - "Hostname Path\n"); + xo_emit("{P: }{T:JID/%-15s}{P: }{T:IP Address/%-15s}" + "{P: }{T:Hostname/%-29s}{P: }{T:Path}\n"); else - printf(" JID IP Address " - "Hostname Path\n"); + xo_emit("{T:JID/%6s}{P: }{T:IP Address}{P:/%6s}" + "{T:Hostname}{P:/%22s}{T:Path}\n", "", ""); else if (pflags & PRINT_HEADER) { for (i = spc = 0; i < nparams; i++) if (params[i].jp_flags & JP_USER) { if (spc) - putchar(' '); + xo_emit("{P: }"); else spc = 1; - fputs(params[i].jp_name, stdout); + xo_emit(params[i].jp_name); } - putchar('\n'); + xo_emit("{P:\n}"); } + xo_open_container("jail-information"); + xo_open_list("jail"); /* Fetch the jail(s) and print the parameters. */ if (jid != 0 || jname != NULL) { if (print_jail(pflags, jflags) < 0) - errx(1, "%s", jail_errmsg); + xo_errx(1, "%s", jail_errmsg); } else { for (lastjid = 0; (lastjid = print_jail(pflags, jflags)) >= 0; ) ; if (errno != 0 && errno != ENOENT) - errx(1, "%s", jail_errmsg); + xo_errx(1, "%s", jail_errmsg); } - + xo_close_list("jail"); + xo_close_container("jail-information"); + xo_finish(); return (0); } @@ -248,7 +263,7 @@ add_param(const char *name, void *value, size_t valuelen, if (!strcmp(name, "all")) { tnparams = jailparam_all(&tparams); if (tnparams < 0) - errx(1, "%s", jail_errmsg); + xo_errx(1, "%s", jail_errmsg); qsort(tparams, (size_t)tnparams, sizeof(struct jailparam), sort_param); for (i = 0; i < tnparams; i++) @@ -263,7 +278,7 @@ add_param(const char *name, void *value, size_t valuelen, if (!strcmp(name, params[i].jp_name)) { if (value != NULL && jailparam_import_raw(params + i, value, valuelen) < 0) - errx(1, "%s", jail_errmsg); + xo_errx(1, "%s", jail_errmsg); params[i].jp_flags |= flags; if (source != NULL) jailparam_free(source, 1); @@ -276,14 +291,14 @@ add_param(const char *name, void *value, size_t valuelen, params = malloc(paramlistsize * sizeof(*params)); param_parent = malloc(paramlistsize * sizeof(*param_parent)); if (params == NULL || param_parent == NULL) - err(1, "malloc"); + xo_err(1, "malloc"); } else if (nparams >= paramlistsize) { paramlistsize *= 2; params = realloc(params, paramlistsize * sizeof(*params)); param_parent = realloc(param_parent, paramlistsize * sizeof(*param_parent)); if (params == NULL || param_parent == NULL) - err(1, "realloc"); + xo_err(1, "realloc"); } /* Look up the parameter. */ @@ -301,7 +316,7 @@ add_param(const char *name, void *value, size_t valuelen, nparams--; return (-1); } - errx(1, "%s", jail_errmsg); + xo_errx(1, "%s", jail_errmsg); } param->jp_flags = flags; return param - params; @@ -332,7 +347,7 @@ noname(const char *name) nname = malloc(strlen(name) + 3); if (nname == NULL) - err(1, "malloc"); + xo_err(1, "malloc"); p = strrchr(name, '.'); if (p != NULL) sprintf(nname, "%.*s.no%s", (int)(p - name), name, p + 1); @@ -351,7 +366,7 @@ nononame(const char *name) return NULL; nname = malloc(strlen(name) - 1); if (nname == NULL) - err(1, "malloc"); + xo_err(1, "malloc"); if (p != NULL) sprintf(nname, "%.*s.%s", (int)(p - name), name, p + 3); else @@ -362,7 +377,7 @@ nononame(const char *name) static int print_jail(int pflags, int jflags) { - char *nname; + char *nname, *xo_nname; char **param_values; int i, ai, jid, count, n, spc; char ipbuf[INET6_ADDRSTRLEN]; @@ -370,18 +385,19 @@ print_jail(int pflags, int jflags) jid = jailparam_get(params, nparams, jflags); if (jid < 0) return jid; + + xo_open_instance("jail"); + if (pflags & PRINT_VERBOSE) { - printf("%6d %-29.29s %.74s\n" - "%6s %-29.29s %.74s\n" - "%6s %-6d\n", + xo_emit("{:jid/%6d}{P: }{:hostname/%-29.29s/%s}{P: }" + "{:path/%.74s/%s}\n", *(int *)params[0].jp_value, (char *)params[1].jp_value, - (char *)params[2].jp_value, - "", + (char *)params[2].jp_value); + xo_emit("{P: }{:name/%-29.29s/%s}{P: }{:state/%.74s}\n", (char *)params[3].jp_value, - *(int *)params[4].jp_value ? "DYING" : "ACTIVE", - "", - *(int *)params[5].jp_value); + *(int *)params[4].jp_value ? "DYING" : "ACTIVE"); + xo_emit("{P: }{:cpusetid/%d}\n", *(int *)params[5].jp_value); n = 6; #ifdef INET if (ip4_ok && !strcmp(params[n].jp_name, "ip4.addr")) { @@ -390,9 +406,10 @@ print_jail(int pflags, int jflags) if (inet_ntop(AF_INET, &((struct in_addr *)params[n].jp_value)[ai], ipbuf, sizeof(ipbuf)) == NULL) - err(1, "inet_ntop"); - else - printf("%6s %-15.15s\n", "", ipbuf); + xo_err(1, "inet_ntop"); + else { + xo_emit("{P: }{l:ipv4_addrs}{P:\n}", ipbuf); + } n++; } #endif @@ -404,20 +421,21 @@ print_jail(int pflags, int jflags) &((struct in6_addr *) params[n].jp_value)[ai], ipbuf, sizeof(ipbuf)) == NULL) - err(1, "inet_ntop"); + xo_err(1, "inet_ntop"); else - printf("%6s %s\n", "", ipbuf); + xo_emit("{P: }{l:ipv6_addrs}{P:\n}", ipbuf); n++; } #endif } else if (pflags & PRINT_DEFAULT) { if (pflags & PRINT_JAIL_NAME) - printf(" %-15s ", (char *)params[0].jp_value); + xo_emit("{P: }{:name/%-15s/%s}{P: }", + (char *)params[0].jp_value); else - printf("%6d ", *(int *)params[0].jp_value); - printf("%-15.15s %-29.29s %.74s\n", + xo_emit("{:jid/%6d}{P: }", *(int *)params[0].jp_value); + xo_emit("{:ipv4/%-15.15s/%s}{P: }{:hostname/%-29.29s/%s}{P: }{:path/%.74s/%s}\n", #ifdef INET - (!ip4_ok || params[1].jp_valuelen == 0) ? "-" + (!ip4_ok || params[1].jp_valuelen == 0) ? "" : inet_ntoa(*(struct in_addr *)params[1].jp_value), (char *)params[2-!ip4_ok].jp_value, (char *)params[3-!ip4_ok].jp_value); @@ -433,7 +451,7 @@ print_jail(int pflags, int jflags) continue; param_values[i] = jailparam_export(params + i); if (param_values[i] == NULL) - errx(1, "%s", jail_errmsg); + xo_errx(1, "%s", jail_errmsg); } for (i = spc = 0; i < nparams; i++) { if (!(params[i].jp_flags & JP_USER)) @@ -446,7 +464,7 @@ print_jail(int pflags, int jflags) JAIL_SYS_NEW))) continue; if (spc) - putchar(' '); + xo_emit("{P: }"); else spc = 1; if (pflags & PRINT_NAMEVAL) { @@ -456,63 +474,82 @@ print_jail(int pflags, int jflags) */ if (params[i].jp_flags & (JP_BOOL | JP_NOBOOL)) { - if (*(int *)params[i].jp_value) - printf("%s", params[i].jp_name); + if (*(int *)params[i].jp_value) { + asprintf(&xo_nname, "{en:%s/true}", params[i].jp_name); + xo_emit(xo_nname); + xo_emit("{d:/%s}", params[i].jp_name); + } else { nname = (params[i].jp_flags & JP_NOBOOL) ? nononame(params[i].jp_name) : noname(params[i].jp_name); - printf("%s", nname); + if (params[i].jp_flags & JP_NOBOOL) { + asprintf(&xo_nname, "{en:%s/true}", params[i].jp_name); + xo_emit(xo_nname); + } else { + asprintf(&xo_nname, "{en:%s/false}", params[i].jp_name); + xo_emit(xo_nname); + } + xo_emit("{d:/%s}", nname); free(nname); } + free(xo_nname); continue; } - printf("%s=", params[i].jp_name); + xo_emit("{d:%s}=", params[i].jp_name); } if (params[i].jp_valuelen == 0) { if (pflags & PRINT_QUOTED) - printf("\"\""); + xo_emit("{P:\"\"}"); else if (!(pflags & PRINT_NAMEVAL)) - putchar('-'); - } else - quoted_print(param_values[i]); + xo_emit("{P:-}"); + } else { + quoted_print(pflags, params[i].jp_name, param_values[i]); + } } - putchar('\n'); + xo_emit("{P:\n}"); for (i = 0; i < nparams; i++) if (params[i].jp_flags & JP_USER) free(param_values[i]); } + + xo_close_instance("jail"); return (jid); } static void -quoted_print(char *str) +quoted_print(int pflags, char *name, char *value) { - int c, qc; - char *p = str; + int qc; + char *p = value; + char *param_name_value; /* An empty string needs quoting. */ if (!*p) { - fputs("\"\"", stdout); + asprintf(¶m_name_value, "{k:%s}{d:%s/\"\"}", name, name); + xo_emit(param_name_value); + free(param_name_value); return; } + asprintf(¶m_name_value, "{:%s/%%s}", name); /* * The value will be surrounded by quotes if it contains spaces * or quotes. */ qc = strchr(p, '\'') ? '"' - : strchr(p, '"') ? '\'' - : strchr(p, ' ') || strchr(p, '\t') ? '"' - : 0; - if (qc) - putchar(qc); - while ((c = *p++)) { - if (c == '\\' || c == qc) - putchar('\\'); - putchar(c); - } - if (qc) - putchar(qc); + : strchr(p, '"') ? '\'' + : strchr(p, ' ') || strchr(p, '\t') ? '"' + : 0; + + if (qc && pflags & PRINT_QUOTED) + xo_emit("{P:/%c}", qc); + + xo_emit(param_name_value, value); + + free(param_name_value); + + if (qc && pflags & PRINT_QUOTED) + xo_emit("{P:/%c}", qc); } diff --git a/usr.sbin/makefs/makefs.c b/usr.sbin/makefs/makefs.c index bca9722..79cffed 100644 --- a/usr.sbin/makefs/makefs.c +++ b/usr.sbin/makefs/makefs.c @@ -66,9 +66,12 @@ typedef struct { } fstype_t; static fstype_t fstypes[] = { - { "ffs", ffs_prep_opts, ffs_parse_opts, ffs_cleanup_opts, ffs_makefs }, - { "cd9660", cd9660_prep_opts, cd9660_parse_opts, cd9660_cleanup_opts, - cd9660_makefs}, +#define ENTRY(name) { \ + # name, name ## _prep_opts, name ## _parse_opts, \ + name ## _cleanup_opts, name ## _makefs \ +} + ENTRY(ffs), + ENTRY(cd9660), { .type = NULL }, }; diff --git a/usr.sbin/mountd/mountd.c b/usr.sbin/mountd/mountd.c index 649c7d5f..535a3f7 100644 --- a/usr.sbin/mountd/mountd.c +++ b/usr.sbin/mountd/mountd.c @@ -422,7 +422,7 @@ main(int argc, char **argv) * list. */ if (nhosts == 0) { - hosts = malloc(sizeof(char**)); + hosts = malloc(sizeof(char *)); if (hosts == NULL) out_of_mem(); hosts[0] = "*"; diff --git a/usr.sbin/mpsutil/mps_cmd.c b/usr.sbin/mpsutil/mps_cmd.c index 876243f..24ed74e 100644 --- a/usr.sbin/mpsutil/mps_cmd.c +++ b/usr.sbin/mpsutil/mps_cmd.c @@ -486,7 +486,7 @@ mps_firmware_get(int fd, unsigned char **firmware, bool bios) } size = reply.ActualImageSize; - *firmware = calloc(1, sizeof(char) * size); + *firmware = calloc(1, sizeof(unsigned char) * size); if (*firmware == NULL) { warn("calloc"); return (-1); diff --git a/usr.sbin/pw/pw_conf.c b/usr.sbin/pw/pw_conf.c index e9606b4..d30c80e 100644 --- a/usr.sbin/pw/pw_conf.c +++ b/usr.sbin/pw/pw_conf.c @@ -313,7 +313,7 @@ read_userconfig(char const * file) ? NULL : newstr(q); break; case _UC_EXTRAGROUPS: - for (i = 0; q != NULL; q = strtok(NULL, toks)) { + while ((q = strtok(NULL, toks)) != NULL) { if (config.groups == NULL) config.groups = sl_init(); sl_add(config.groups, newstr(q)); diff --git a/usr.sbin/pw/pw_group.c b/usr.sbin/pw/pw_group.c index 67beab9..289a4c8 100644 --- a/usr.sbin/pw/pw_group.c +++ b/usr.sbin/pw/pw_group.c @@ -664,6 +664,11 @@ pw_group_mod(int argc, char **argv, char *arg1) grp_add_members(&grp, newmembers); } + if (dryrun) { + print_group(grp, pretty); + return (EXIT_SUCCESS); + } + if ((rc = chggrent(name, grp)) != 0) { if (rc == -1) errx(EX_IOERR, "group '%s' not available (NIS?)", diff --git a/usr.sbin/pw/pw_vpw.c b/usr.sbin/pw/pw_vpw.c index a23c66e..2d1c75b 100644 --- a/usr.sbin/pw/pw_vpw.c +++ b/usr.sbin/pw/pw_vpw.c @@ -70,7 +70,6 @@ vnextpwent(char const *nam, uid_t uid, int doclose) pw = NULL; line = NULL; linecap = 0; - linelen = 0; if (pwd_fp != NULL || (pwd_fp = fopen(getpwpath(_MASTERPASSWD), "r")) != NULL) { while ((linelen = getline(&line, &linecap, pwd_fp)) > 0) { @@ -153,7 +152,6 @@ vnextgrent(char const *nam, gid_t gid, int doclose) gr = NULL; line = NULL; linecap = 0; - linelen = 0; if (grp_fp != NULL || (grp_fp = fopen(getgrpath(_GROUP), "r")) != NULL) { while ((linelen = getline(&line, &linecap, grp_fp)) > 0) { diff --git a/usr.sbin/pwd_mkdb/pwd_mkdb.c b/usr.sbin/pwd_mkdb/pwd_mkdb.c index c382cb5..fe44520 100644 --- a/usr.sbin/pwd_mkdb/pwd_mkdb.c +++ b/usr.sbin/pwd_mkdb/pwd_mkdb.c @@ -352,7 +352,7 @@ main(int argc, char *argv[]) data.size = 1; if ((dp->put)(dp, &key, &data, 0) == -1) error("put"); - if ((dp->put)(sdp, &key, &data, 0) == -1) + if ((sdp->put)(sdp, &key, &data, 0) == -1) error("put"); } ypcnt = 0; diff --git a/usr.sbin/rpc.lockd/lockd.c b/usr.sbin/rpc.lockd/lockd.c index 4f1347e..88b19b7 100644 --- a/usr.sbin/rpc.lockd/lockd.c +++ b/usr.sbin/rpc.lockd/lockd.c @@ -220,7 +220,7 @@ main(int argc, char **argv) * list. */ if (nhosts == 0) { - hosts = malloc(sizeof(char**)); + hosts = malloc(sizeof(char *)); if (hosts == NULL) out_of_mem(); diff --git a/usr.sbin/rpc.statd/statd.c b/usr.sbin/rpc.statd/statd.c index faa8513..256f58d 100644 --- a/usr.sbin/rpc.statd/statd.c +++ b/usr.sbin/rpc.statd/statd.c @@ -150,7 +150,7 @@ main(int argc, char **argv) * list. */ if (nhosts == 0) { - hosts = malloc(sizeof(char**)); + hosts = malloc(sizeof(char *)); if (hosts == NULL) out_of_mem(); diff --git a/usr.sbin/rtsold/rtsold.c b/usr.sbin/rtsold/rtsold.c index b251482..1482798 100644 --- a/usr.sbin/rtsold/rtsold.c +++ b/usr.sbin/rtsold/rtsold.c @@ -888,7 +888,7 @@ autoifprobe(void) warnmsg(LOG_WARNING, __func__, "multiple interfaces found"); - a = (char **)realloc(argv, (n + 1) * sizeof(char **)); + a = realloc(argv, (n + 1) * sizeof(char *)); if (a == NULL) { warnmsg(LOG_ERR, __func__, "realloc"); exit(1); @@ -903,7 +903,7 @@ autoifprobe(void) } if (n) { - a = (char **)realloc(argv, (n + 1) * sizeof(char **)); + a = realloc(argv, (n + 1) * sizeof(char *)); if (a == NULL) { warnmsg(LOG_ERR, __func__, "realloc"); exit(1); diff --git a/usr.sbin/uhsoctl/uhsoctl.c b/usr.sbin/uhsoctl/uhsoctl.c index 686d45f..9dc13ab 100644 --- a/usr.sbin/uhsoctl/uhsoctl.c +++ b/usr.sbin/uhsoctl/uhsoctl.c @@ -452,6 +452,7 @@ set_nameservers(struct ctx *ctx, const char *respath, int ns, ...) free(ctx->ns[i]); } free(ctx->ns); + ctx->ns = NULL; } fd = open(respath, O_RDWR | O_CREAT | O_NOFOLLOW, 0666); |
