Update base system file(1) to 5.22 to address multiple denial of

service issues. [EN-15:06] Improve reliability of ZFS when TRIM/UNMAP and/or L2ARC is used. [EN-15:07] Approved by: so
author: delphij <delphij@FreeBSD.org> 2015-06-09 22:13:25 +0000
committer: delphij <delphij@FreeBSD.org> 2015-06-09 22:13:25 +0000
commit: 301060471bd9c2f62da3293dd334408ae864be70 (patch)
tree: fb96be74f227016a1aec7aac411b2f5a6eafe025
parent: 3bc97baf8a6eb60b69f56e79f0bc47597014d084 (diff)
download: FreeBSD-src-301060471bd9c2f62da3293dd334408ae864be70.zip
FreeBSD-src-301060471bd9c2f62da3293dd334408ae864be70.tar.gz
72 files changed, 2979 insertions, 1032 deletions
diff --git a/UPDATING b/UPDATING
index cea7e63..b2d431c 100644
--- a/UPDATING
+++ b/UPDATING
@@ -16,6 +16,15 @@ from older versions of FreeBSD, try WITHOUT_CLANG to bootstrap to the tip of
 stable/10, and then rebuild without this option. The bootstrap process from
 older version of current is a bit fragile.
 
+20150609:	p29	FreeBSD-EN-15:06.file
+			FreeBSD-EN-15:07.zfs
+
+	Updated base system file(1) to 5.22 to address multiple denial
+	of service issues. [EN-15:06]
+
+	Improved reliability of ZFS when TRIM/UNMAP and/or L2ARC is used.
+	[EN-15:07]
+
 20150513:	p10	FreeBSD-EN-15:04.freebsd-update
 			FreeBSD-EN-15:05.ufs
 
diff --git a/contrib/file/ChangeLog b/contrib/file/ChangeLog
index ed3df66..98cdcc4 100644
--- a/contrib/file/ChangeLog
+++ b/contrib/file/ChangeLog
@@ -1,3 +1,97 @@
+2015-01-02  15:15  Christos Zoulas <christos@zoulas.com>
+
+	* release 5.22
+
+2015-01-01  12:01  Christos Zoulas <christos@zoulas.com>
+
+	* add indirect relative for TIFF/Exif
+
+2014-12-16  18:10  Christos Zoulas <christos@zoulas.com>
+	
+	* restructure elf note printing to avoid repeated messages
+	* add note limit, suggested by Alexander Cherepanov
+
+2014-12-16  16:53  Christos Zoulas <christos@zoulas.com>
+	
+	* Bail out on partial pread()'s (Alexander Cherepanov)
+	* Fix incorrect bounds check in file_printable (Alexander Cherepanov)
+
+2014-12-11  20:01  Christos Zoulas <christos@zoulas.com>
+
+	* PR/405: ignore SIGPIPE from uncompress programs
+	* change printable -> file_printable and use it in
+	  more places for safety
+	* in ELF, instead of "(uses dynamic libraries)" when PT_INTERP
+	  is present print the interpreter name.
+	
+2014-12-10  20:01  Christos Zoulas <christos@zoulas.com>
+
+	* release 5.21
+
+2014-11-27  18:40  Christos Zoulas <christos@zoulas.com>
+
+	* Allow setting more parameters from the command line.
+	* Split name/use and indirect magic recursion limits.
+
+2014-11-27  11:12  Christos Zoulas <christos@zoulas.com>
+
+	* Adjust ELF parameters and the default recursion
+	  level.
+	* Allow setting the recursion level dynamically.
+
+2014-11-24   8:55  Christos Zoulas <christos@zoulas.com>
+
+	* The following fixes resulted from Thomas Jarosch's fuzzing
+	  tests that revealed severe performance issues on pathological
+	  input:
+	    - limit number of elf program and sections processing
+	    - abort elf note processing quickly
+	    - reduce the number of recursion levels from 20 to 10
+	    - preserve error messages in indirect magic handling
+
+	This is tracked as CVE-2014-8116 and CVE-2014-8117
+
+2014-11-12  10:30  Christos Zoulas <christos@zoulas.com>
+
+	* fix bogus free in the user buffer case.
+
+2014-11-11  12:35  Christos Zoulas <christos@zoulas.com>
+
+	* fix out of bounds read for pascal strings
+	* fix memory leak (not freeing the head of each mlist)
+
+2014-11-07  10:25  Christos Zoulas <christos@zoulas.com>
+
+	* When printing strings from a file, convert them to printable
+	  on a byte by byte basis, so that we don't get issues with
+	  locale's trying to interpret random byte streams as UTF-8 and
+	  having printf error out with EILSEQ.
+	  
+2014-10-17  11:48  Christos Zoulas <christos@zoulas.com>
+
+	* fix bounds in note reading (Francisco Alonso / Red Hat)
+
+2014-10-11  15:02  Christos Zoulas <christos@zoulas.com>
+
+	* fix autoconf glue for setlocale and locale_t; some OS's
+	  have locale_t in xlocale.h
+
+2014-10-10  15:01  Christos Zoulas <christos@zoulas.com>
+
+	* release 5.20
+
+2014-08-17  10:01  Christos Zoulas <christos@zoulas.com>
+
+	* recognize encrypted CDF documents
+
+2014-08-04   9:18  Christos Zoulas <christos@zoulas.com>
+
+	* add magic_load_buffers from Brooks Davis
+
+2014-07-24  16:40  Christos Zoulas <christos@zoulas.com>
+
+	* add thumbs.db support
+
 2014-06-12  12:28  Christos Zoulas <christos@zoulas.com>
 
 	* release 5.19
diff --git a/contrib/file/README b/contrib/file/README
index cfc530f..81a5221 100644
--- a/contrib/file/README
+++ b/contrib/file/README
@@ -1,6 +1,6 @@
 ## README for file(1) Command ##
 
-    @(#) $File: README,v 1.48 2014/03/07 13:55:30 christos Exp $
+    @(#) $File: README,v 1.49 2015/01/02 20:23:04 christos Exp $
 
 Mailing List: file@mx.gw.com  
 Mailing List archives: http://mx.gw.com/pipermail/file/  
@@ -25,8 +25,8 @@ A public read-only git repository of the same sources is available at:
 
 	https://github.com/file/file
 
-The major changes for 5.x are CDF file parsing, indirect magic, and
-overhaul in mime and ascii encoding handling.
+The major changes for 5.x are CDF file parsing, indirect magic, name/use
+(recursion) and overhaul in mime and ascii encoding handling.
 
 The major feature of 4.x is the refactoring of the code into a library,
 and the re-write of the file command in terms of that library. The library
@@ -67,33 +67,41 @@ in magic(5) format please, to the maintainer, Christos Zoulas.
 COPYING - read this first.  
 README - read this second (you are currently reading this file).  
 INSTALL - read on how to install
-
 src/apprentice.c - parses /etc/magic to learn magic  
+src/asctime_r.c - replacement for OS's that don't have it.  
 src/apptype.c - used for OS/2 specific application type magic  
 src/asprintf.c - replacement for OS's that don't have it.  
 src/ascmagic.c - third & last set of tests, based on hardwired assumptions.  
-src/asctime_r.c - for systems that don't have it.  
-src/asprintf.c - for systems that don't have it.  
-src/cdf.c - parser for Microsoft Compound Document Files  
+src/asctime_r.c - replacement for OS's that don't have it.  
+src/asprintf.c - replacement for OS's that don't have it.  
+src/cdf.[ch] - parser for Microsoft Compound Document Files  
 src/cdf_time.c - time converter for CDF.  
 src/compress.c - handles decompressing files to look inside.  
-src/ctime_r.c - for systems that don't have it.  
+src/ctime_r.c - replacement for OS's that don't have it.  
+src/elfclass.h - common code for elf 32/64.
 src/encoding.c - handles unicode encodings  
 src/file.c - the main program  
 src/file.h - header file  
+src/file_opts.h - list of options
+src/fmtcheck.c - replacement for OS's that don't have it.  
 src/fsmagic.c - first set of tests the program runs, based on filesystem info  
 src/funcs.c - utilility functions  
-src/getopt_long.c - for systems that don't have it.  
-src/getline.c - for systems that don't have it.  
+src/getline.c - replacement for OS's that don't have it.  
+src/getopt_long.c - replacement for OS's that don't have it.  
 src/is_tar.c, tar.h - knows about tarchives (courtesy John Gilmore).  
 src/names.h - header file for ascmagic.c  
+src/magic.h.in - source file for magic.h
 src/magic.c - the libmagic api  
+src/pread.c - replacement for OS's that don't have it.  
 src/print.c - print results, errors, warnings.  
 src/readcdf.c - CDF wrapper.  
 src/readelf.[ch] - Stand-alone elf parsing code.  
 src/softmagic.c - 2nd set of tests, based on /etc/magic  
-src/strlcat.c - for systems that don't have it.  
-src/strlcpy.c - for systems that don't have it.  
+src/mygetopt.h - replacement for OS's that don't have it.  
+src/strcasestr.c - replacement for OS's that don't have it.  
+src/strlcat.c - replacement for OS's that don't have it.  
+src/strlcpy.c - replacement for OS's that don't have it.  
+src/tar.h - tar file definitions
 src/vasprintf.c - for systems that don't have it.  
 doc/file.man - man page for the command  
 doc/magic.man - man page for the magic file, courtesy Guy Harris.
diff --git a/contrib/file/TODO b/contrib/file/TODO
index f78893f..eef01ed 100644
--- a/contrib/file/TODO
+++ b/contrib/file/TODO
@@ -15,3 +15,5 @@ small amount of C is needed (because fast execution is typically only
 required for soft magic, not the more detailed information given by
 hard-wired routines). In this regard, note that hplip, which is
 BSD-licensed, has a magic reimplementation in Python.
+
+Read the kerberos magic entry for more ideas.
diff --git a/contrib/file/config.h.in b/contrib/file/config.h.in
index 7f22fa8..066bd2f 100644
--- a/contrib/file/config.h.in
+++ b/contrib/file/config.h.in
@@ -44,6 +44,9 @@
 /* Define to 1 if you have the `fork' function. */
 #undef HAVE_FORK
 
+/* Define to 1 if you have the `freelocale' function. */
+#undef HAVE_FREELOCALE
+
 /* Define to 1 if fseeko (and presumably ftello) exists and is declared. */
 #undef HAVE_FSEEKO
 
@@ -95,9 +98,15 @@
 /* Define to 1 if you have a working `mmap' system call. */
 #undef HAVE_MMAP
 
+/* Define to 1 if you have the `newlocale' function. */
+#undef HAVE_NEWLOCALE
+
 /* Define to 1 if you have the `pread' function. */
 #undef HAVE_PREAD
 
+/* Define to 1 if you have the `setlocale' function. */
+#undef HAVE_SETLOCALE
+
 /* Define to 1 if you have the <stddef.h> header file. */
 #undef HAVE_STDDEF_H
 
@@ -182,6 +191,9 @@
 /* Define to 1 if you have the <unistd.h> header file. */
 #undef HAVE_UNISTD_H
 
+/* Define to 1 if you have the `uselocale' function. */
+#undef HAVE_USELOCALE
+
 /* Define to 1 if you have the `utime' function. */
 #undef HAVE_UTIME
 
@@ -219,6 +231,9 @@
 /* Define to 1 if `vfork' works. */
 #undef HAVE_WORKING_VFORK
 
+/* Define to 1 if you have the <xlocale.h> header file. */
+#undef HAVE_XLOCALE_H
+
 /* Define to 1 if you have the <zlib.h> header file. */
 #undef HAVE_ZLIB_H
 
diff --git a/contrib/file/configure b/contrib/file/configure
index 757d143..3cf1d77 100755
--- a/contrib/file/configure
+++ b/contrib/file/configure
@@ -1,6 +1,6 @@
 #! /bin/sh
 # Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.69 for file 5.19.
+# Generated by GNU Autoconf 2.69 for file 5.22.
 #
 # Report bugs to <christos@astron.com>.
 #
@@ -590,8 +590,8 @@ MAKEFLAGS=
 # Identity of this package.
 PACKAGE_NAME='file'
 PACKAGE_TARNAME='file'
-PACKAGE_VERSION='5.19'
-PACKAGE_STRING='file 5.19'
+PACKAGE_VERSION='5.22'
+PACKAGE_STRING='file 5.22'
 PACKAGE_BUGREPORT='christos@astron.com'
 PACKAGE_URL=''
 
@@ -1327,7 +1327,7 @@ if test "$ac_init_help" = "long"; then
   # Omit some internal or obsolete options to make the list less imposing.
   # This message is too long to be a string in the A/UX 3.1 sh.
   cat <<_ACEOF
-\`configure' configures file 5.19 to adapt to many kinds of systems.
+\`configure' configures file 5.22 to adapt to many kinds of systems.
 
 Usage: $0 [OPTION]... [VAR=VALUE]...
 
@@ -1397,7 +1397,7 @@ fi
 
 if test -n "$ac_init_help"; then
   case $ac_init_help in
-     short | recursive ) echo "Configuration of file 5.19:";;
+     short | recursive ) echo "Configuration of file 5.22:";;
    esac
   cat <<\_ACEOF
 
@@ -1507,7 +1507,7 @@ fi
 test -n "$ac_init_help" && exit $ac_status
 if $ac_init_version; then
   cat <<\_ACEOF
-file configure 5.19
+file configure 5.22
 generated by GNU Autoconf 2.69
 
 Copyright (C) 2012 Free Software Foundation, Inc.
@@ -2163,7 +2163,7 @@ cat >config.log <<_ACEOF
 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.
 
-It was created by file $as_me 5.19, which was
+It was created by file $as_me 5.22, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
 
   $ $0 $@
@@ -3029,7 +3029,7 @@ fi
 
 # Define the identity of the package.
  PACKAGE='file'
- VERSION='5.19'
+ VERSION='5.22'
 
 
 cat >>confdefs.h <<_ACEOF
@@ -12785,7 +12785,7 @@ fi
 
 done
 
-for ac_header in getopt.h err.h
+for ac_header in getopt.h err.h xlocale.h
 do :
   as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
 ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
@@ -14191,7 +14191,7 @@ fi
 fi
 
 
-for ac_func in strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof
+for ac_func in strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale
 do :
   as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
 ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -14998,7 +14998,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
 # report actual input values of CONFIG_FILES etc. instead of their
 # values after options handling.
 ac_log="
-This file was extended by file $as_me 5.19, which was
+This file was extended by file $as_me 5.22, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
 
   CONFIG_FILES    = $CONFIG_FILES
@@ -15064,7 +15064,7 @@ _ACEOF
 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
 ac_cs_version="\\
-file config.status 5.19
+file config.status 5.22
 configured by $0, generated by GNU Autoconf 2.69,
   with options \\"\$ac_cs_config\\"
 
diff --git a/contrib/file/configure.ac b/contrib/file/configure.ac
index 3d55c38..1e2457c 100644
--- a/contrib/file/configure.ac
+++ b/contrib/file/configure.ac
@@ -1,5 +1,5 @@
 dnl Process this file with autoconf to produce a configure script.
-AC_INIT([file],[5.19],[christos@astron.com])
+AC_INIT([file],[5.22],[christos@astron.com])
 AM_INIT_AUTOMAKE([subdir-objects foreign])
 m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
 
@@ -82,7 +82,7 @@ AC_HEADER_MAJOR
 AC_HEADER_SYS_WAIT
 AC_CHECK_HEADERS(stdint.h fcntl.h locale.h stdint.h inttypes.h unistd.h)
 AC_CHECK_HEADERS(stddef.h utime.h wchar.h wctype.h limits.h)
-AC_CHECK_HEADERS(getopt.h err.h)
+AC_CHECK_HEADERS(getopt.h err.h xlocale.h)
 AC_CHECK_HEADERS(sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h)
 AC_CHECK_HEADERS(zlib.h)
 
@@ -138,7 +138,7 @@ else
 fi])
 
 dnl Checks for functions
-AC_CHECK_FUNCS(strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof)
+AC_CHECK_FUNCS(strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale)
 
 dnl Provide implementation of some required functions if necessary
 AC_REPLACE_FUNCS(getopt_long asprintf vasprintf strlcpy strlcat getline ctime_r asctime_r pread strcasestr fmtcheck)
diff --git a/contrib/file/doc/file.man b/contrib/file/doc/file.man
index 361188e..109c676 100644
--- a/contrib/file/doc/file.man
+++ b/contrib/file/doc/file.man
@@ -1,5 +1,5 @@
-.\" $File: file.man,v 1.106 2014/03/07 23:11:51 christos Exp $
-.Dd January 30, 2014
+.\" $File: file.man,v 1.111 2014/12/16 23:18:40 christos Exp $
+.Dd December 16, 2014
 .Dt FILE __CSECTION__
 .Os
 .Sh NAME
@@ -16,6 +16,7 @@
 .Op Fl F Ar separator
 .Op Fl f Ar namefile
 .Op Fl m Ar magicfiles
+.Op Fl P Ar name=value
 .Ar
 .Ek
 .Nm
@@ -303,6 +304,16 @@ or
 attempt to preserve the access time of files analyzed, to pretend that
 .Nm
 never read them.
+.It Fl P , Fl Fl parameter Ar name=value
+Set various parameter limits.
+.Bl -column "elf_phnum" "Default" "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -offset indent
+.It Sy "Name" Ta Sy "Default" Ta Sy "Explanation"
+.It Li indir Ta 15 Ta recursion limit for indirect magic
+.It Li name Ta 30 Ta use count limit for name/use magic
+.It Li elf_notes Ta 256 Ta max ELF notes processed
+.It Li elf_phnum Ta 128 Ta max ELF program sections processed
+.It Li elf_shnum Ta 32768 Ta max ELF sections processed
+.El
 .It Fl r , Fl Fl raw
 Don't translate unprintable characters to \eooo.
 Normally
@@ -385,6 +396,7 @@ options.
 .Xr hexdump 1 ,
 .Xr od 1 ,
 .Xr strings 1 ,
+.Xr fstyp 8
 .Sh STANDARDS CONFORMANCE
 This program is believed to exceed the System V Interface Definition
 of FILE(CMD), as near as one can determine from the vague language
diff --git a/contrib/file/doc/libmagic.man b/contrib/file/doc/libmagic.man
index ee9cdb2..3c907ca 100644
--- a/contrib/file/doc/libmagic.man
+++ b/contrib/file/doc/libmagic.man
@@ -1,4 +1,4 @@
-.\" $File: libmagic.man,v 1.28 2014/03/02 14:47:16 christos Exp $
+.\" $File: libmagic.man,v 1.34 2014/12/16 23:18:40 christos Exp $
 .\"
 .\" Copyright (c) Christos Zoulas 2003.
 .\" All Rights Reserved.
@@ -25,7 +25,7 @@
 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 .\" SUCH DAMAGE.
 .\"
-.Dd January 6, 2012
+.Dd December 16, 2014
 .Dt LIBMAGIC 3
 .Os
 .Sh NAME
@@ -40,6 +40,9 @@
 .Nm magic_compile ,
 .Nm magic_list ,
 .Nm magic_load ,
+.Nm magic_load_buffers ,
+.Nm magic_setparam ,
+.Nm magic_getparam ,
 .Nm magic_version
 .Nd Magic number recognition library
 .Sh LIBRARY
@@ -71,6 +74,12 @@
 .Ft int
 .Fn magic_load "magic_t cookie" "const char *filename"
 .Ft int
+.Fn magic_load_buffers "magic_t cookie" "void **buffers" "size_t *sizes" "size_t nbuffers"
+.Ft int
+.Fn magic_getparam "magic_t cookie" "int param" "void *value"
+.Ft int
+.Fn magic_setparam "magic_t cookie" "int param" "const void *value"
+.Ft int
 .Fn magic_version "void"
 .Sh DESCRIPTION
 These functions
@@ -253,6 +262,60 @@ adds
 to the database filename as appropriate.
 .Pp
 The
+.Fn magic_load_buffers
+function takes an array of size
+.Fa nbuffers
+of
+.Fa buffers
+with a respective size for each in the array of
+.Fa sizes
+loaded with the contents of the magic databases from the filesystem.
+This function can be used in environment where the magic library does
+not have direct access to the filesystem, but can access the magic
+database via shared memory or other IPC means.
+.Pp
+The
+.Fn magic_getparam
+and
+.Fn magic_setparam
+allow getting and setting various limits related to the the magic
+library.
+.Bl -column "MAGIC_PARAM_ELF_PHNUM_MAX" "size_t" "Default" -offset indent
+.It Sy "Parameter" Ta Sy "Type" Ta Sy "Default"
+.It Li MAGIC_PARAM_INDIR_MAX Ta size_t Ta 15
+.It Li MAGIC_PARAM_NAME_MAX Ta size_t Ta 30
+.It Li MAGIC_PARAM_ELF_NOTES_MAX Ta size_t Ta 256
+.It Li MAGIC_PARAM_ELF_PHNUM_MAX Ta size_t Ta 128
+.It Li MAGIC_PARAM_ELF_SHNUM_MAX Ta size_t Ta 32768
+.El
+.Pp
+The
+.Dv MAGIC_PARAM_INDIR_RECURSION
+parameter controls how many levels of recursion will be followed for
+indirect magic entries.
+.Pp
+The
+.Dv MAGIC_PARAM_NAME_RECURSION
+parameter controls how many levels of recursion will be followed for
+for name/use calls.
+.Pp
+The
+.Dv MAGIC_PARAM_NAME_MAX
+parameter controls the maximum number of calls for name/use.
+.Pp
+The
+.Dv MAGIC_PARAM_NOTES_MAX
+parameter controls how many ELF notes will be processed.
+.Pp
+The
+.Dv MAGIC_PARAM_PHNUM_MAX
+parameter controls how many ELF program sections will be processed.
+.Pp
+The
+.Dv MAGIC_PARAM_SHNUM_MAX
+parameter controls how many ELF sections will be processed.
+.Pp
+The
 .Fn magic_version
 command returns the version number of this library which is compiled into
 the shared library using the constant
diff --git a/contrib/file/doc/magic.man b/contrib/file/doc/magic.man
index 762da48..b6523f2 100644
--- a/contrib/file/doc/magic.man
+++ b/contrib/file/doc/magic.man
@@ -1,5 +1,5 @@
-.\" $File: magic.man,v 1.84 2014/06/03 19:01:34 christos Exp $
-.Dd June 3, 2014
+.\" $File: magic.man,v 1.85 2015/01/01 17:07:34 christos Exp $
+.Dd January 1, 2015
 .Dt MAGIC __FSECTION__
 .Os
 .\" install as magic.4 on USG, magic.5 on V7, Berkeley and Linux systems.
@@ -200,6 +200,11 @@ interpreted as a UNIX-style date, but interpreted as local time rather
 than UTC.
 .It Dv indirect
 Starting at the given offset, consult the magic database again.
+The offset of th
+.Dv indirect
+magic is by default absolute in the file, but one can specify
+.Dv /r
+to indicate that the offset is relative from the beginning of the entry.
 .It Dv name
 Define a
 .Dq named
diff --git a/contrib/file/magic/Magdir/android b/contrib/file/magic/Magdir/android
index 4a4c3fe..da98b57 100644
--- a/contrib/file/magic/Magdir/android
+++ b/contrib/file/magic/Magdir/android
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------
-# $File: android,v 1.4 2014/06/03 19:01:34 christos Exp $
+# $File: android,v 1.7 2014/11/10 05:08:23 christos Exp $
 # Various android related magic entries
 #------------------------------------------------------------
 
@@ -15,54 +15,11 @@
 >0	regex	dey\n[0-9]{2}\0	Dalvik dex file (optimized for host)
 >4	string	>000			version %s
 
-# http://android.stackexchange.com/questions/23357/\
-# is-there-a-way-to-look-inside-and-modify-an-adb-backup-created-file/\
-# 23608#23608
-0	string	ANDROID\040BACKUP\n	Android Backup
->15	string	1\n			\b, version 1
->17	string	0\n			\b, uncompressed
->17	string	1\n			\b, compressed
->19	string	none\n			\b, unencrypted
->19	string	AES-256\n		\b, encrypted AES-256
-
-# Android bootimg format
-# From https://android.googlesource.com/\
-# platform/system/core/+/master/mkbootimg/bootimg.h
-0		string	ANDROID!	Android bootimg
->8		lelong	>0			\b, kernel
->>12	lelong	>0			\b (0x%x)
->16		lelong	>0			\b, ramdisk
->>20	lelong	>0			\b (0x%x)
->24		lelong	>0			\b, second stage
->>28	lelong	>0			\b (0x%x)
->36		lelong	>0			\b, page size: %d
->38		string	>0			\b, name: %s
->64		string	>0		 	\b, cmdline (%s)
-# Dalvik .dex format. http://retrodev.com/android/dexformat.html
-# From <mkf@google.com> "Mike Fleming"
-# Fixed to avoid regexec 17 errors on some dex files
-# From <diff@lookout.com> "Tim Strazzere"
-0	string	dex\n
->0	regex	dex\n[0-9]{2}\0	Dalvik dex file
->4	string	>000			version %s
-0	string	dey\n
->0	regex	dey\n[0-9]{2}\0	Dalvik dex file (optimized for host)
->4	string	>000			version %s
-
-# http://android.stackexchange.com/questions/23357/\
-# is-there-a-way-to-look-inside-and-modify-an-adb-backup-created-file/\
-# 23608#23608
-0	string	ANDROID\040BACKUP\n	Android Backup
->15	string	1\n			\b, version 1
->17	string	0\n			\b, uncompressed
->17	string	1\n			\b, compressed
->19	string	none\n			\b, unencrypted
->19	string	AES-256\n		\b, encrypted AES-256
-
 # Android bootimg format
 # From https://android.googlesource.com/\
 # platform/system/core/+/master/mkbootimg/bootimg.h
 0		string	ANDROID!	Android bootimg
+>1024	string	LOKI\01		\b, LOKI'd
 >8		lelong	>0			\b, kernel
 >>12	lelong	>0			\b (0x%x)
 >16		lelong	>0			\b, ramdisk
@@ -98,3 +55,85 @@
 #>>>>>&1	regex/1l .*	\b, PBKDF2 rounds: %s
 #>>>>>>&1	regex/1l .*	\b, IV: %s
 #>>>>>>>&1	regex/1l .*	\b, Key: %s
+
+# *.pit files by Joerg Jenderek
+# http://forum.xda-developers.com/showthread.php?p=9122369
+# http://forum.xda-developers.com/showthread.php?t=816449
+# Partition Information Table for Samsung's smartphone with Android
+# used by flash software Odin
+0		ulelong			0x12349876	
+# 1st pit entry marker
+>0x01C	ulequad&0xFFFFFFFCFFFFFFFC	=0x0000000000000000	
+# minimal 13 and maximal 18 PIT entries found
+>>4		ulelong			<128	Partition Information Table for Samsung smartphone
+>>>4		ulelong			x	\b, %d entries
+# 1. pit entry
+>>>4		ulelong			>0	\b; #1
+>>>0x01C	use				PIT-entry
+>>>4		ulelong			>1	\b; #2
+>>>0x0A0	use				PIT-entry
+>>>4		ulelong			>2	\b; #3
+>>>0x124	use				PIT-entry
+>>>4		ulelong			>3	\b; #4
+>>>0x1A8	use				PIT-entry
+>>>4		ulelong			>4	\b; #5
+>>>0x22C	use				PIT-entry
+>>>4		ulelong			>5	\b; #6
+>>>0x2B0	use				PIT-entry
+>>>4		ulelong			>6	\b; #7
+>>>0x334	use				PIT-entry
+>>>4		ulelong			>7 	\b; #8
+>>>0x3B8	use				PIT-entry
+>>>4		ulelong			>8 	\b; #9
+>>>0x43C	use				PIT-entry
+>>>4		ulelong			>9	\b; #10
+>>>0x4C0	use				PIT-entry
+>>>4		ulelong			>10	\b; #11
+>>>0x544	use				PIT-entry
+>>>4		ulelong			>11	\b; #12
+>>>0x5C8	use				PIT-entry
+>>>4		ulelong			>12	\b; #13
+>>>>0x64C	use				PIT-entry
+# 14. pit entry
+>>>4		ulelong			>13	\b; #14
+>>>>0x6D0	use				PIT-entry
+>>>4		ulelong			>14	\b; #15
+>>>0x754	use				PIT-entry
+>>>4		ulelong			>15	\b; #16
+>>>0x7D8	use				PIT-entry
+>>>4		ulelong			>16	\b; #17
+>>>0x85C	use				PIT-entry
+# 18. pit entry
+>>>4		ulelong			>17	\b; #18
+>>>0x8E0	use				PIT-entry
+
+0	name			PIT-entry
+# garbage value implies end of pit entries
+>0x00		ulequad&0xFFFFFFFCFFFFFFFC	=0x0000000000000000	
+# skip empty partition name
+>>0x24		ubyte				!0			
+# partition name
+>>>0x24		string				>\0			%-.32s
+# flags
+>>>0x0C		ulelong&0x00000002		2			\b+RW
+# partition ID:
+# 0~IPL,MOVINAND,GANG;1~PIT,GPT;2~HIDDEN;3~SBL,HIDDEN;4~SBL2,HIDDEN;5~BOOT;6~KENREl,RECOVER,misc;7~RECOVER
+# ;11~MODEM;20~efs;21~PARAM;22~FACTORY,SYSTEM;23~DBDATAFS,USERDATA;24~CACHE;80~BOOTLOADER;81~TZSW
+>>>0x08	ulelong		x			(0x%x)
+# filename
+>>>0x44		string				>\0			"%-.64s"
+#>>>0x18	ulelong				>0			
+# blocksize in 512 byte units ?
+#>>>>0x18	ulelong				x			\b, %db
+# partition size in blocks ?
+#>>>>0x22	ulelong				x			\b*%d
+
+# Android bootimg format
+# From https://android.googlesource.com/\
+# platform/system/core/+/master/libsparse/sparse_format.h
+0		lelong	0xed26ff3a		Android sparse image
+>4		leshort	x			\b, version: %d
+>6		leshort	x			\b.%d
+>16		lelong	x			\b, Total of %d
+>12		lelong	x			\b %d-byte output blocks in
+>20		lelong	x			\b %d input chunks.
diff --git a/contrib/file/magic/Magdir/animation b/contrib/file/magic/Magdir/animation
index 561fe79..0445adc 100644
--- a/contrib/file/magic/Magdir/animation
+++ b/contrib/file/magic/Magdir/animation
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: animation,v 1.53 2014/04/30 21:41:02 christos Exp $
+# $File: animation,v 1.56 2014/10/23 23:12:51 christos Exp $
 # animation:  file(1) magic for animation/movie formats
 #
 # animation formats
@@ -32,43 +32,155 @@
 !:mime	application/x-quicktime-player
 4	string/W	jP		JPEG 2000 image
 !:mime	image/jp2
+# http://www.ftyps.com/ with local additions
 4	string		ftyp		ISO Media
->8	string		isom		\b, MPEG v4 system, version 1
-!:mime	video/mp4
->8	string		iso2		\b, MPEG v4 system, part 12 revision
->8	string		mp41		\b, MPEG v4 system, version 1
-!:mime	video/mp4
->8	string		mp42		\b, MPEG v4 system, version 2
-!:mime	video/mp4
->8	string		mp7t		\b, MPEG v4 system, MPEG v7 XML
->8	string		mp7b		\b, MPEG v4 system, MPEG v7 binary XML
->8	string/W	jp2		\b, JPEG 2000
-!:mime	image/jp2
+>8	string		3g2		\b, MPEG v4 system, 3GPP2
+!:mime	video/3gpp2
+>>11	byte		4		\b v4 (H.263/AMR GSM 6.10)
+>>11	byte		5		\b v5 (H.263/AMR GSM 6.10)
+>>11	byte		6		\b v6 (ITU H.264/AMR GSM 6.10)
+>>11	byte		a		\b C.S0050-0 V1.0
+>>11	byte		b		\b C.S0050-0-A V1.0.0
+>>11	byte		c		\b C.S0050-0-B V1.0
 >8	string		3ge		\b, MPEG v4 system, 3GPP
 !:mime	video/3gpp
+>>11	byte		6		\b, Release 6 MBMS Extended Presentations
+>>11	byte		7		\b, Release 7 MBMS Extended Presentations
 >8	string		3gg		\b, MPEG v4 system, 3GPP
+>11	byte		6		\b, Release 6 General Profile
 !:mime	video/3gpp
 >8	string		3gp		\b, MPEG v4 system, 3GPP
+>11	byte		1		\b, Release %d (non existent)
+>11	byte		2		\b, Release %d (non existent)
+>11	byte		3		\b, Release %d (non existent)
+>11	byte		4		\b, Release %d
+>11	byte		5		\b, Release %d
+>11	byte		6		\b, Release %d
+>11	byte		7		\b, Release %d Streaming Servers
 !:mime	video/3gpp
 >8	string		3gs		\b, MPEG v4 system, 3GPP
+>11	byte		7		\b, Release %d Streaming Servers
 !:mime	video/3gpp
->8	string		3g2		\b, MPEG v4 system, 3GPP2
+>8	string		avc1		\b, MPEG v4 system, 3GPP JVT AVC [ISO 14496-12:2005]
+!:mime	video/mp4
+>8	string/W	qt		\b, Apple QuickTime movie
+!:mime	video/quicktime
+>8	string		CAEP		\b, Canon Digital Camera
+>8	string		caqv		\b, Casio Digital Camera
+>8	string		CDes		\b, Convergent Design
+>8	string		da0a		\b, DMB MAF w/ MPEG Layer II aud, MOT slides, DLS, JPG/PNG/MNG
+>8	string		da0b		\b, DMB MAF, ext DA0A, with 3GPP timed text, DID, TVA, REL, IPMP
+>8	string		da1a		\b, DMB MAF audio with ER-BSAC audio, JPG/PNG/MNG images
+>8	string		da1b		\b, DMB MAF, ext da1a, with 3GPP timed text, DID, TVA, REL, IPMP
+>8	string		da2a		\b, DMB MAF aud w/ HE-AAC v2 aud, MOT slides, DLS, JPG/PNG/MNG
+>8	string		da2b		\b, DMB MAF, ext da2a, with 3GPP timed text, DID, TVA, REL, IPMP
+>8	string		da3a		\b, DMB MAF aud with HE-AAC aud, JPG/PNG/MNG images
+>8	string		da3b		\b, DMB MAF, ext da3a w/ BIFS, 3GPP, DID, TVA, REL, IPMP
+>8	string		dmb1		\b, DMB MAF supporting all the components defined in the spec
+>8	string		dmpf		\b, Digital Media Project
+>8	string		drc1		\b, Dirac (wavelet compression), encap in ISO base media (MP4)
+>8	string		dv1a		\b, DMB MAF vid w/ AVC vid, ER-BSAC aud, BIFS, JPG/PNG/MNG, TS
+>8	string		dv1b		\b, DMB MAF, ext dv1a, with 3GPP timed text, DID, TVA, REL, IPMP
+>8	string		dv2a		\b, DMB MAF vid w/ AVC vid, HE-AAC v2 aud, BIFS, JPG/PNG/MNG, TS
+>8	string		dv2b		\b, DMB MAF, ext dv2a, with 3GPP timed text, DID, TVA, REL, IPMP
+>8	string		dv3a		\b, DMB MAF vid w/ AVC vid, HE-AAC aud, BIFS, JPG/PNG/MNG, TS
+>8	string		dv3b		\b, DMB MAF, ext dv3a, with 3GPP timed text, DID, TVA, REL, IPMP
+>8	string		dvr1		\b, DVB (.DVB) over RTP
+!:mime	video/vnd.dvb.file
+>8	string		dvt1		\b, DVB (.DVB) over MPEG-2 Transport Stream
+!:mime	video/vnd.dvb.file
+>8	string		F4V		\b, Video for Adobe Flash Player 9+ (.F4V)
+!:mime	video/mp4
+>8	string		F4P		\b, Protected Video for Adobe Flash Player 9+ (.F4P)
+!:mime	video/mp4
+>8	string		F4A		\b, Audio for Adobe Flash Player 9+ (.F4A)
+!:mime	audio/mp4
+>8	string		F4B		\b, Audio Book for Adobe Flash Player 9+ (.F4B)
+!:mime	audio/mp4
+>8	string		isc2		\b, ISMACryp 2.0 Encrypted File
+#	?/enc-isoff-generic
+>8	string		iso2		\b, MP4 Base Media v2 [ISO 14496-12:2005]
+!:mime	video/mp4
+>8	string		isom		\b, MP4 Base Media v1 [IS0 14496-12:2003]
+!:mime	video/mp4
+>8	string/W	jp2		\b, JPEG 2000
+!:mime	image/jp2
+>8	string		JP2		\b, JPEG 2000 Image (.JP2) [ISO 15444-1 ?]
+!:mime	image/jp2
+>8	string		JP20		\b, Unknown, from GPAC samples (prob non-existent)
+>8	string		jpm		\b, JPEG 2000 Compound Image (.JPM) [ISO 15444-6]
+!:mime	image/jpm
+>8	string		jpx		\b, JPEG 2000 w/ extensions (.JPX) [ISO 15444-2]
+!:mime	image/jpx
+>8	string		KDDI		\b, 3GPP2 EZmovie for KDDI 3G cellphones
 !:mime	video/3gpp2
->>11	byte		4		\b v4 (H.263/AMR GSM 6.10)
->>11	byte		5		\b v5 (H.263/AMR GSM 6.10)
->>11	byte		6		\b v6 (ITU H.264/AMR GSM 6.10)
+>8	string		M4A 		\b, Apple iTunes ALAC/AAC-LC (.M4A) Audio
+!:mime	audio/x-m4a
+>8	string		M4B 		\b, Apple iTunes ALAC/AAC-LC (.M4B) Audio Book
+!:mime	audio/mp4
+>8	string		M4P 		\b, Apple iTunes ALAC/AAC-LC (.M4P) AES Protected Audio
+!:mime	video/mp4
+>8	string		M4V 		\b, Apple iTunes Video (.M4V) Video
+!:mime	video/x-m4v
+>8	string		M4VH		\b, Apple TV (.M4V)
+!:mime	video/x-m4v
+>8	string		M4VP		\b, Apple iPhone (.M4V)
+!:mime	video/x-m4v
+>8	string		mj2s		\b, Motion JPEG 2000 [ISO 15444-3] Simple Profile
+!:mime	video/mj2
+>8	string		mjp2		\b, Motion JPEG 2000 [ISO 15444-3] General Profile
+!:mime	video/mj2
+>8	string		mmp4		\b, MPEG-4/3GPP Mobile Profile (.MP4 / .3GP) (for NTT)
+!:mime	video/mp4
+>8	string		mobi		\b, MPEG-4, MOBI format
+!:mime	video/mp4
+>8	string		mp21		\b, MPEG-21 [ISO/IEC 21000-9]
+>8	string		mp41		\b, MP4 v1 [ISO 14496-1:ch13]
+!:mime	video/mp4
+>8	string		mp42		\b, MP4 v2 [ISO 14496-14]
+!:mime	video/mp4
+>8	string		mp71		\b, MP4 w/ MPEG-7 Metadata [per ISO 14496-12]
+>8	string		mp7t		\b, MPEG v4 system, MPEG v7 XML
+>8	string		mp7b		\b, MPEG v4 system, MPEG v7 binary XML
 >8	string		mmp4		\b, MPEG v4 system, 3GPP Mobile
 !:mime	video/mp4
->8	string		avc1		\b, MPEG v4 system, 3GPP JVT AVC
-!:mime	video/3gpp
->8	string/W	M4A		\b, MPEG v4 system, iTunes AAC-LC
+>8	string		MPPI		\b, Photo Player, MAF [ISO/IEC 23000-3]
+>8	string		mqt		\b, Sony / Mobile QuickTime (.MQV) US Pat 7,477,830
+!:mime	video/quicktime
+>8	string		MSNV		\b, MPEG-4 (.MP4) for SonyPSP
+!:mime	audio/mp4
+>8	string		NDAS		\b, MP4 v2 [ISO 14496-14] Nero Digital AAC Audio
 !:mime	audio/mp4
->8	string/W	M4V		\b, MPEG v4 system, iTunes AVC-LC
+>8	string		NDSC		\b, MPEG-4 (.MP4) Nero Cinema Profile
 !:mime	video/mp4
->8	string/W	M4P		\b, MPEG v4 system, iTunes AES encrypted
->8	string/W	M4B		\b, MPEG v4 system, iTunes bookmarked
->8	string/W	qt		\b, Apple QuickTime movie
+>8	string		NDSH		\b, MPEG-4 (.MP4) Nero HDTV Profile
+!:mime	video/mp4
+>8	string		NDSM		\b, MPEG-4 (.MP4) Nero Mobile Profile
+!:mime	video/mp4
+>8	string		NDSP		\b, MPEG-4 (.MP4) Nero Portable Profile
+!:mime	video/mp4
+>8	string		NDSS		\b, MPEG-4 (.MP4) Nero Standard Profile
+!:mime	video/mp4
+>8	string		NDXC		\b, H.264/MPEG-4 AVC (.MP4) Nero Cinema Profile
+!:mime	video/mp4
+>8	string		NDXH		\b, H.264/MPEG-4 AVC (.MP4) Nero HDTV Profile
+!:mime	video/mp4
+>8	string		NDXM		\b, H.264/MPEG-4 AVC (.MP4) Nero Mobile Profile
+!:mime	video/mp4
+>8	string		NDXP		\b, H.264/MPEG-4 AVC (.MP4) Nero Portable Profile
+!:mime	video/mp4
+>8	string		NDXS		\b, H.264/MPEG-4 AVC (.MP4) Nero Standard Profile
+!:mime	video/mp4
+>8	string		odcf  		\b, OMA DCF DRM Format 2.0 (OMA-TS-DRM-DCF-V2_0-20060303-A)
+>8	string		opf2 		\b, OMA PDCF DRM Format 2.1 (OMA-TS-DRM-DCF-V2_1-20070724-C)
+>8	string		opx2  		\b, OMA PDCF DRM + XBS ext (OMA-TS-DRM_XBS-V1_0-20070529-C)
+>8	string		pana		\b, Panasonic Digital Camera
+>8	string		qt  		\b, Apple QuickTime (.MOV/QT)
 !:mime	video/quicktime
+>8	string		ROSS		\b, Ross Video
+>8	string		sdv		\b, SD Memory Card Video
+>8	string		ssc1		\b, Samsung stereo, single stream (patent pending)
+>8	string		ssc2		\b, Samsung stereo, dual stream (patent pending)
 
 # MPEG sequences
 # Scans for all common MPEG header start codes
diff --git a/contrib/file/magic/Magdir/archive b/contrib/file/magic/Magdir/archive
index 91fbadd..4ef73a7 100644
--- a/contrib/file/magic/Magdir/archive
+++ b/contrib/file/magic/Magdir/archive
@@ -1,5 +1,5 @@
 #------------------------------------------------------------------------------
-# $File: archive,v 1.87 2014/06/03 19:15:58 christos Exp $
+# $File: archive,v 1.88 2014/08/16 10:42:17 christos Exp $
 # archive:  file(1) magic for archive formats (see also "msdos" for self-
 #           extracting compressed archives)
 #
@@ -954,34 +954,3 @@
 >0xE08	search/7776		\x55\xAA	
 >>&-512	indirect		x		\b; contains 
 
-# Symantec GHOST image by Joerg Jenderek at May 2014
-# http://us.norton.com/ghost/
-# http://www.garykessler.net/library/file_sigs.html
-0		ubelong&0xFFFFf7f0	0xFEEF0100	Norton GHost image
-# *.GHO
->2		ubyte&0x08		0x00		\b, first file
-# *.GHS or *.[0-9] with cns program option
->2		ubyte&0x08		0x08		\b, split file
-# part of split index interesting for *.ghs
->>4		ubyte			x		id=0x%x
-# compression tag minus one equals numeric compression command line switch z[1-9]
->3		ubyte			0		\b, no compression
->3		ubyte			2		\b, fast compression (Z1)
->3		ubyte			3		\b, medium compression (Z2)
->3		ubyte			>3		
->>3		ubyte			<11		\b, compression (Z%d-1)
->2		ubyte&0x08		0x00		
-# ~ 30 byte password field only for *.gho
->>12		ubequad			!0		\b, password protected
->>44		ubyte			!1		
-# 1~Image All, sector-by-sector only for *.gho
->>>10		ubyte			1		\b, sector copy
-# 1~Image Boot track only for *.gho
->>>43		ubyte			1		\b, boot track
-# 1~Image Disc only for *.gho implies Image Boot track and sector copy
->>44		ubyte			1		\b, disc sector copy
-# optional image description only *.gho
->>0xff		string			>\0		"%-.254s"
-# look for DOS sector end sequence
->0xE08	search/7776		\x55\xAA	
->>&-512	indirect		x		\b; contains 
diff --git a/contrib/file/magic/Magdir/blender b/contrib/file/magic/Magdir/blender
index 1814738..5b9c855 100644
--- a/contrib/file/magic/Magdir/blender
+++ b/contrib/file/magic/Magdir/blender
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: blender,v 1.5 2009/09/19 16:28:08 christos Exp $
+# $File: blender,v 1.6 2014/08/30 08:34:17 christos Exp $
 # blender: file(1) magic for Blender 3D related files
 #
 # Native format rule v1.2. For questions use the developers list 
@@ -35,5 +35,5 @@
 >>>0x44		string	=GLOB		\b.
 >>>>0x60	beshort	x		\b%.4d
 
-# Scripts that run in the embeded Python interpreter
+# Scripts that run in the embedded Python interpreter
 0		string	#!BPY		Blender3D BPython script
diff --git a/contrib/file/magic/Magdir/cafebabe b/contrib/file/magic/Magdir/cafebabe
index b3206e7..4c58fc6 100644
--- a/contrib/file/magic/Magdir/cafebabe
+++ b/contrib/file/magic/Magdir/cafebabe
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: cafebabe,v 1.16 2014/04/30 21:41:02 christos Exp $
+# $File: cafebabe,v 1.17 2015/01/01 17:07:00 christos Exp $
 # Cafe Babes unite!
 #
 # Since Java bytecode and Mach-O universal binaries have the same magic number,
@@ -45,7 +45,7 @@
 
 0	name		mach-o		\b [
 >0	use		mach-o-cpu	\b
->&(8.L)	indirect			\b: 
+>(8.L)	indirect			\b: 
 >0	belong		x		\b]
 
 0	belong		0xcafebabe
diff --git a/contrib/file/magic/Magdir/commands b/contrib/file/magic/Magdir/commands
index 20c1058..3d97489 100644
--- a/contrib/file/magic/Magdir/commands
+++ b/contrib/file/magic/Magdir/commands
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: commands,v 1.50 2014/05/30 16:48:44 christos Exp $
+# $File: commands,v 1.51 2014/09/27 00:12:55 christos Exp $
 # commands:  file(1) magic for various shells and interpreters
 #
 #0	string/w	:			shell archive or script for antique kernel text
@@ -56,7 +56,7 @@
 !:mime	text/x-awk
 0	string/wt	#!\ /usr/bin/awk	awk script text executable
 !:mime	text/x-awk
-0	regex/4096	=^\\s{0,100}BEGIN\\s{0,100}[{]	awk script text
+0	regex/4096	=^\\s{0,100}BEGIN\\s{0,100}[{]	awk or perl script text
 
 # AT&T Bell Labs' Plan 9 shell
 0	string/wt	#!\ /bin/rc	Plan 9 rc shell script text executable
diff --git a/contrib/file/magic/Magdir/compress b/contrib/file/magic/Magdir/compress
index 9dc9489..beb8ebe 100644
--- a/contrib/file/magic/Magdir/compress
+++ b/contrib/file/magic/Magdir/compress
@@ -1,5 +1,5 @@
 #------------------------------------------------------------------------------
-# $File: compress,v 1.58 2014/05/07 19:36:59 christos Exp $
+# $File: compress,v 1.62 2014/09/13 14:27:12 christos Exp $
 # compress:  file(1) magic for pure-compression formats (no archives)
 #
 # compress, gzip, pack, compact, huf, squeeze, crunch, freeze, yabba, etc.
@@ -251,3 +251,13 @@
 # http://code.google.com/p/snappy/source/browse/trunk/framing_format.txt
 0	string	\377\006\0\0sNaPpY	snappy framed data
 !:mime	application/x-snappy-framed
+
+# qpress, http://www.quicklz.com/
+0	string	qpress10	qpress compressed data
+!:mime	application/x-qpress
+
+# Zlib https://www.ietf.org/rfc/rfc6713.txt
+0	beshort%31	=0	
+>0	byte&0xf	=8
+>>0	byte&0x80 	=0	zlib compressed data
+!:mime	application/zlib
diff --git a/contrib/file/magic/Magdir/database b/contrib/file/magic/Magdir/database
index b18e256..b00252b 100644
--- a/contrib/file/magic/Magdir/database
+++ b/contrib/file/magic/Magdir/database
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: database,v 1.41 2014/06/03 19:17:27 christos Exp $
+# $File: database,v 1.43 2014/10/28 15:47:39 christos Exp $
 # database:  file(1) magic for various databases
 #
 # extracted from header/code files by Graeme Wilford (eep2gw@ee.surrey.ac.uk)
@@ -9,9 +9,17 @@
 # GDBM magic numbers
 #  Will be maintained as part of the GDBM distribution in the future.
 #  <downsj@teeny.org>
-0	belong	0x13579ace	GNU dbm 1.x or ndbm database, big endian
+0	belong	0x13579acd	GNU dbm 1.x or ndbm database, big endian, 32-bit
 !:mime	application/x-gdbm
-0	lelong	0x13579ace	GNU dbm 1.x or ndbm database, little endian
+0	belong	0x13579ace	GNU dbm 1.x or ndbm database, big endian, old
+!:mime	application/x-gdbm
+0	belong	0x13579acf	GNU dbm 1.x or ndbm database, big endian, 64-bit
+!:mime	application/x-gdbm
+0	lelong	0x13579acd	GNU dbm 1.x or ndbm database, little endian, 32-bit
+!:mime	application/x-gdbm
+0	lelong	0x13579ace	GNU dbm 1.x or ndbm database, little endian, old
+!:mime	application/x-gdbm
+0	lelong	0x13579acf	GNU dbm 1.x or ndbm database, little endian, 64-bit
 !:mime	application/x-gdbm
 0	string	GDBM		GNU dbm 2.x database
 !:mime	application/x-gdbm
@@ -202,27 +210,27 @@
 # for multiple index files (*.MDX) Production flag,tag numbers(<=0x30),tag length(<=0x20), reserverd (NULL) 
 >>>>>>>24	ubelong&0x0133f7ff	>0		
 # test for reserved NULL byte
->>>>>>>>47	ubyte			x		
+>>>>>>>>47	ubyte			0		
 # test for valid TAG key format (0x10 or 0)
 >>>>>>>>>559	ubyte&0xeF		0		
 # test MM <= 12
->>>>>>>>>45	ubeshort		<0x0C20		
->>>>>>>>>>45	ubyte			>0		
->>>>>>>>>>>46	ubyte			<32		
->>>>>>>>>>>>46	ubyte			>0		
+>>>>>>>>>>45	ubeshort		<0x0C20		
+>>>>>>>>>>>45	ubyte			>0		
+>>>>>>>>>>>>46	ubyte			<32		
+>>>>>>>>>>>>>46	ubyte			>0		
 #!:mime	application/x-mdx
->>>>>>>>>>>>>0	use			xbase-type
->>>>>>>>>>>>>0	ubyte			x		\b MDX
->>>>>>>>>>>>>1	ubyte			x		\b, creation-date
->>>>>>>>>>>>>1	use			xbase-date
->>>>>>>>>>>>>44	ubyte			x		\b, update-date
->>>>>>>>>>>>>44	use			xbase-date
+>>>>>>>>>>>>>>0		use		xbase-type
+>>>>>>>>>>>>>>0		ubyte		x		\b MDX
+>>>>>>>>>>>>>>1		ubyte		x		\b, creation-date
+>>>>>>>>>>>>>>1		use		xbase-date
+>>>>>>>>>>>>>>44	ubyte		x		\b, update-date
+>>>>>>>>>>>>>>44	use		xbase-date
 # No.of tags in use (1,2,5,12)
->>>>>>>>>>>>>28	uleshort		x		\b, %d
+>>>>>>>>>>>>>>28	uleshort	x		\b, %d
 # No. of entries in tag (0x30)
->>>>>>>>>>>>>25	ubyte			x		\b/%d tags
+>>>>>>>>>>>>>>25	ubyte		x		\b/%d tags
 #  Length of tag
->>>>>>>>>>>>>26	ubyte			x		* %d
+>>>>>>>>>>>>>>26	ubyte		x		* %d
 # 1st tag name_
 >>>>>>>>>>>>>548	string		x		\b, 1st tag "%.11s"
 # 2nd tag name
@@ -337,60 +345,103 @@
 # dBASE III
 >>>>>>16	ubyte		3		
 # dBASE III DBT
->>>>>>>0	use		xbase-memo-print
-# dBASE IV DBT , FoxPro FPT or many PNG , ZIP , DBF garbage
+>>>>>>>0	use		dbase3-memo-print
+# dBASE III DBT without version, dBASE IV DBT , FoxPro FPT , or many ZIP , DBF garbage
 >>>>>>16	ubyte		0		
-# dBASE IV DBT with DBF name or DBF garbage
->>>>>>>8	ubelong		>0x40000000	
-# skip DBF and catch dBASE IV DBT with DBF name and with non big index of next free block
->>>>>>>>0	ulelong		<0x01010002	
->>>>>>>>>0	use		xbase-memo-print
->>>>>>>8	ubelong		0		
+# unusual dBASE III DBT like angest.dbt, dBASE IV DBT with block size 0 , FoxPro FPT ,  or garbage PCX DBF
+>>>>>>>20	uleshort	0		
+# FoxPro FPT , unusual dBASE III DBT like biblio.dbt or garbage
+>>>>>>>>8	ulong		=0		
+>>>>>>>>>6	ubeshort	>0		
+# skip emacs.PIF
+>>>>>>>>>>4	ushort		0		
+>>>>>>>>>>>0	use		foxpro-memo-print
+# dBASE III DBT , garbage
+>>>>>>>>>6	ubeshort	0		
 # skip MM*DD*.bin by test for for reserved NULL byte
->>>>>>>>508	ubelong		0		
-# real memo files should contain text here
->>>>>>>>>520	ubelong		>0x20202019	
->>>>>>>>>>520	ubelong		<0xFEFEFEFF	
->>>>>>>>>>>0	use		xbase-memo-print
-# garbage PCX , ZIP , JAR , XPI 
->>>>>>>8	default		x		
+>>>>>>>>>>510	ubeshort	0		
+# skip TK-DOS11.img image by looking for memo text
+>>>>>>>>>>>512	ubelong		<0xfeffff03	
+# skip EFI executables by looking for memo text
+>>>>>>>>>>>>512	ubelong		>0x1F202020	
+>>>>>>>>>>>>>513 ubyte		>0		
+# unusual dBASE III DBT like adressen.dbt
+>>>>>>>>>>>>>>0	use		dbase3-memo-print
+# dBASE III DBT like angest.dbt, or garbage PCX DBF
+>>>>>>>>8	ubelong		!0		
+# skip PCX and some DBF by test for for reserved NULL bytes
+>>>>>>>>>510	ubeshort	0		
+# skip some DBF by test of invalid version
+>>>>>>>>>>0	ubyte		>5		
+>>>>>>>>>>>0	ubyte		<48		
+>>>>>>>>>>>>0	use		dbase3-memo-print
+# dBASE IV DBT with positive block size
+>>>>>>>20	uleshort	>0		
+>>>>>>>>0	use		dbase4-memo-print
 
-#		Print the information of dBase DBT or FoxPro FPT memo files 
-0	name				xbase-memo-print
->0	ubyte			x		
-# test version
-# memo file
->>16	ubyte			3		dBase III DBT
->>16	ubyte			0		
->>>512	ubelong			<0x00000003	FoxPro FPT
-# Size of blocks for FoxPro
->>>>6	ubeshort		x		\b, blocks size %u
-# Number of next available block for appending data for FoxPro
->>>>0	ubelong			=0		\b, next free block index %u
->>>>0	ubelong			!0		\b, next free block index %u
->>>512	default			x		dBase IV DBT
-# DBF file name without extension
->>>>8	string			>\0		\b of %-.8s.DBF
-# size of blocks  ; not reliable 0x2020204C
-#>>>>4	ulelong			=0		\b, blocks size %u
->>>>4	ulelong			!0		\b, blocks size %u
-# Block length found 0 , 512
-#>>>>20	uleshort		=0		\b, block length %u
->>>>20	uleshort		!0		\b, block length %u
+#		Print the information of dBase III DBT memo file 
+0	name				dbase3-memo-print
+>0	ubyte			x		dBase III DBT
+# instead 3 as version number 0 for unusual examples like biblio.dbt
+>16	ubyte			!3		\b, version number %u
 # Number of next available block for appending data
->>>>0	ulelong			=0		\b, next free block index %u
->>>>0	ulelong			!0		\b, next free block index %u
->>512	ubelong			x		
->>>512	ubelong			=0xFFFF0800	
->>>>520	string			>\0		\b, 1st used item "%s"
-# FoxPro
->>>512	ubelong			<3		
-# FoxPro memo
->>>>512	ubelong			=1		
->>>>520	string			>\0		\b, 1st used item "%s"
->>>512	default			x		
-# may be deleted memo field
->>>>512	string			>\0		\b, 1st item "%s"
+#>0	lelong			=0		\b, next free block index %u
+>0	lelong			!0		\b, next free block index %u
+# no positiv block length
+#>20	uleshort		=0		\b, block length %u
+>20	uleshort		!0		\b, block length %u
+# dBase III memo field terminated by \032\032
+>512	string			>\0		\b, 1st item "%s"
+#		Print the information of dBase IV DBT memo file 
+0	name				dbase4-memo-print
+>0		lelong		x		dBase IV DBT
+# 8 character shorted main name of coresponding dBASE IV DBF file
+>8		ubelong		>0x20000000	
+# skip unusual like for angest.dbt
+>>20		uleshort	>0	
+>>>8		string		>\0		\b of %-.8s.DBF
+# value 0 implies 512 as size
+#>4		ulelong		=0		\b, blocks size %u
+# size of blocks not reliable like 0x2020204C in angest.dbt
+>4		ulelong		!0		
+>>4		ulelong&0x0000003f	0	\b, blocks size %u
+# dBase IV DBT with positive block length (found 512 , 1024)
+>20		uleshort	>0		\b, block length %u
+# next available block
+#>0		lelong		=0		\b, next free block index %u
+>0		lelong		!0		\b, next free block index %u
+>20		uleshort	>0		
+>>(20.s)	ubelong		x		
+>>>&-4		use		dbase4-memofield-print
+# unusual dBase IV DBT without block length (implies 512 as length)
+>20		uleshort	=0		
+>>512		ubelong		x		
+>>>&-4		use				dbase4-memofield-print
+#		Print the information of dBase IV memo field 
+0	name			dbase4-memofield-print
+# free dBase IV memo field
+>0		ubelong		!0xFFFF0800	
+>>0		lelong		x		\b, next free block %u
+>>4		lelong		x		\b, next used block %u
+# used dBase IV memo field
+>0		ubelong		=0xFFFF0800	
+# length of memo field
+>>4		lelong		x		\b, field length %d
+>>>8		string		>\0		\b, 1st used item "%s"
+#		Print the information of FoxPro FPT memo file 
+0	name				foxpro-memo-print
+>0		belong		x		FoxPro FPT
+# Size of blocks for FoxPro ( 64,256 )
+>6		ubeshort	x		\b, blocks size %u
+# next available block
+#>0		belong		=0		\b, next free block index %u
+>0		belong		!0		\b, next free block index %u
+# field type ( 0~picture, 1~memo, 2~object ) 
+>512		ubelong		<3		\b, field type %u
+# length of memo field
+>512		ubelong		1		
+>>516		belong		>0		\b, field length %d
+>>>520		string		>\0		\b, 1st item "%s"
 
 # TODO: 
 # DBASE index file *.NDX
diff --git a/contrib/file/magic/Magdir/elf b/contrib/file/magic/Magdir/elf
index b007f88..04ee37e 100644
--- a/contrib/file/magic/Magdir/elf
+++ b/contrib/file/magic/Magdir/elf
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: elf,v 1.67 2014/06/12 13:52:48 christos Exp $
+# $File: elf,v 1.68 2014/09/19 19:05:57 christos Exp $
 # elf:  file(1) magic for ELF executables
 #
 # We have to check the byte order flag to see what byte order all the
@@ -257,6 +257,7 @@
 >18	leshort		216		Cognitive Smart Memory,
 >18	leshort		217		iCelero CoolEngine,
 >18	leshort		218		Nanoradio Optimized RISC,
+>18	leshort		243		UCB RISC-V,
 >18	leshort		0x1057		AVR (unofficial),
 >18	leshort		0x1059		MSP430 (unofficial),
 >18	leshort		0x1223		Adapteva Epiphany (unofficial),
diff --git a/contrib/file/magic/Magdir/filesystems b/contrib/file/magic/Magdir/filesystems
index 7225d75..939a092 100644
--- a/contrib/file/magic/Magdir/filesystems
+++ b/contrib/file/magic/Magdir/filesystems
@@ -1,5 +1,5 @@
 #------------------------------------------------------------------------------
-# $File: filesystems,v 1.95 2014/06/03 19:17:27 christos Exp $
+# $File: filesystems,v 1.108 2015/01/01 17:43:47 christos Exp $
 # filesystems:  file(1) magic for different filesystems
 #
 0	name	partid  
@@ -254,19 +254,20 @@
 30		search/481	\x55\xAA	
 # to display DOS/MBR boot sector (40) before old one (strength=50+21),Syslinux bootloader (71),SYSLINUX MBR (37+36),NetBSD mbr (110),AdvanceMAME mbr (111)
 # DOS BPB information (70) and after DOS floppy (120) like in previous file version
-!:strength +72
+!:strength +65
 # for sector sizes < 512 Bytes
 >11		uleshort	<512		
 >>(11.s-2)	uleshort	0xAA55		DOS/MBR boot sector
 # for sector sizes with 512 or more Bytes
 >0x1FE		leshort		0xAA55		DOS/MBR boot sector
+
 # keep old DOS/MBR boot sector as dummy for mbr and bootloader displaying
 # only for sector sizes with 512 or more Bytes
-0x1FE		leshort		0xAA55		
+0x1FE          leshort         0xAA55         	DOS/MBR boot sector
 #
 # to display information (50) before DOS BPB (strength=70) and after DOS floppy (120) like in old file version
-!:strength +21
->2	string	OSBS			\b, OS/BS MBR
+!:strength +65
+>2		string		OSBS		OS/BS MBR
 # added by Joerg Jenderek at Feb 2013 according to http://thestarman.pcministry.com/asm/mbr/
 # and http://en.wikipedia.org/wiki/Master_Boot_Record
 # test for nearly all MS-DOS Master Boot Record initial program loader (IPL) is now done by 
@@ -538,6 +539,8 @@
 >>>514		string		!HdrS			
 # not BeOS
 >>>>422		string		!Be\ Boot\ Loader	
+>>>>>32769	string    CD001
+>>>>>>0	use cdrom
 # jump over BPB instruction implies DOS bootsector or AdvanceMAME mbr 
 >>>>>0		ubelong&0xFD000000	=0xE9000000	
 # AdvanceMAME mbr
@@ -1043,7 +1046,7 @@
 >>>>>>466	ubyte	<0x10			
 >>>>>>>466	ubyte	0x05			\b, extended partition table
 >>>>>>>466	ubyte	0x0F			\b, extended partition table (LBA)
->>>>>>>466	ubyte	0x0			\b, extended partition table (last)	
+>>>>>>>466	ubyte	0x0			\b, extended partition table (last)
 
 # DOS x86 sector separated and moved from "DOS/MBR boot sector" by Joerg Jenderek at May 2011
 
@@ -1236,6 +1239,7 @@
 >>>>>181	search/166		Error\ \0			
 # "a: disk" , "Fn: diskn" or "NetBSD MBR boot"
 >>>>>>&3	string			x				\b,"%s"
+>>>446	use		partition-table
 # Andrea Mazzoleni AdvanceCD mbr loader of http://advancemame.sourceforge.net/boot-readme.html
 # added by Joerg Jenderek at Nov 2012 for versions 1.3 - 1.4
 # assembler instructions: jmp short 0x58;nop;ASCII
@@ -1353,18 +1357,19 @@
 # minimal short forward jump found 0x29 for bootloaders or 0x0
 # maximal short forward jump is 0x7f
 # OEM-ID is empty or contain readable bytes
-0		ulelong&0x804000E9	0x000000E9	
+0		ulelong&0x804000E9	0x000000E9
+!:strength	+60
 # mtools-3.9.8/msdos.h
 # usual values are marked with comments to get only informations of strange FAT systems
 # valid sectorsize must be a power of 2 from 32 to 32768
->11		uleshort&0xf001f	0	
+>11		uleshort&0x001f	0	
 >>11		uleshort	<32769		
 >>>11		uleshort	>31		
 >>>>21		ubyte&0xf0	0xF0		
->>>>>0		ubyte		0xEB		
+>>>>>0		ubyte		0xEB		DOS/MBR boot sector
 >>>>>>1		ubyte		x		\b, code offset 0x%x+2
 >>>>>0		ubyte		0xE9		
->>>>>>1		uleshort	x		\b, code offset 0x%x+2
+>>>>>>1		uleshort	x		\b, code offset 0x%x+3
 >>>>>3		string		>\0		\b, OEM-ID "%-.8s"
 #http://mirror.href.com/thestarman/asm/debug/debug2.htm#IHC
 >>>>>>8		string		IHC		\b cached by Windows 9M
@@ -1373,10 +1378,11 @@
 >>>>>11		uleshort	<512		\b, Bytes/sector %u
 >>>>>13		ubyte		>1		\b, sectors/cluster %u
 #>>>>>13	ubyte		=1		\b, sectors/cluster %u (usual on Floppies)
->>>>>82		string		FAT32		
+# for lazy FAT32 implementation like Transcend digital photo frame PF830
+>>>>>82		string/c	fat32		
 >>>>>>14	uleshort	!32		\b, reserved sectors %u
 #>>>>>>14	uleshort	=32		\b, reserved sectors %u (usual Fat32)
->>>>>82		string		!FAT32		
+>>>>>82		string/c	!fat32		
 >>>>>>14	uleshort	>1		\b, reserved sectors %u
 #>>>>>>14	uleshort	=1		\b, reserved sectors %u (usual FAT12,FAT16)
 #>>>>>>14	uleshort	0		\b, reserved sectors %u (usual NTFS)
@@ -1399,24 +1405,29 @@
 >>>>>26		ubyte		=1		\b, heads %u
 # valid only for sector sizes with more then 32 Bytes
 >>>>>11		uleshort	>32		
-# skip for Digital Research DOS (version 3.41) 1440 kB Bootdisk
->>>>>>38	ubyte		!0x70		
+# http://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#Extended_BIOS_Parameter_Block
+# skip for values 2,2Ah,70h,73h,DFh
+# and continue for extended boot signature values 0,28h,29h,80h
+>>>>>>38	ubyte&0x56	=0		
 >>>>>>>28	ulelong		>0		\b, hidden sectors %u
 #>>>>>>>28	ulelong		=0		\b, hidden sectors %u (usual floppy)
 >>>>>>>32	ulelong		>0		\b, sectors %u (volumes > 32 MB) 
 #>>>>>>>32	ulelong		=0		\b, sectors %u (volumes > 32 MB)
 # FAT<32 bit specific 
->>>>>>>82	string		!FAT32
+>>>>>>>82	string/c	!fat32		
 #>>>>>>>>36	ubyte		0x80		\b, physical drive 0x%x=0x80 (usual harddisk)
 #>>>>>>>>36	ubyte		0		\b, physical drive 0x%x=0 (usual floppy)
 >>>>>>>>36	ubyte		!0x80		
 >>>>>>>>>36	ubyte		!0		\b, physical drive 0x%x
+# VGA-copy CRC or
+# in Windows NT bit 0 is a dirty flag to request chkdsk at boot time. bit 1 requests surface scan too
 >>>>>>>>37	ubyte		>0		\b, reserved 0x%x
 #>>>>>>>>37	ubyte		=0		\b, reserved 0x%x
-# value is 0x80 for NTFS
+# extended boot signatur value is 0x80 for NTFS, 0x28 or 0x29 for others
 >>>>>>>>38	ubyte		!0x29		\b, dos < 4.0 BootSector (0x%x)
->>>>>>>>38	ubyte		=0x29
+>>>>>>>>38	ubyte&0xFE	=0x28
 >>>>>>>>>39	ulelong		x		\b, serial number 0x%x
+>>>>>>>>38	ubyte		=0x29
 >>>>>>>>>43	string		<NO\ NAME	\b, label: "%11.11s"
 >>>>>>>>>43	string		>NO\ NAME	\b, label: "%11.11s"
 >>>>>>>>>43	string		=NO\ NAME	\b, unlabeled
@@ -1426,11 +1437,35 @@
 # if it is small enough FAT is 12 bit, if it is too big enough FAT is 32 bit,
 # otherwise FAT is 16 bit.
 # http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/determining-fat-widths.html
->>>>>>54	string		FAT		\b, FAT
->>>>>>>54	string		FAT12		\b (12 bit)
->>>>>>>54	string		FAT16		\b (16 bit)
+>>>>>82		string/c	!fat32		
+>>>>>>54	string		FAT12		\b, FAT (12 bit)
+>>>>>>54	string		FAT16		\b, FAT (16 bit)
+>>>>>>54	default		x		
+# determinate FAT bit size by media descriptor
+# small floppies implies FAT12
+>>>>>>>21	ubyte		<0xF0		\b, FAT (12 bit by descriptor)
+# with media descriptor F0h floppy or maybe superfloppy with FAT16
+>>>>>>>21	ubyte		=0xF0		
+# superfloppy (many sectors) implies FAT16
+>>>>>>>>32	ulelong		>0xFFFF		\b, FAT (16 bit by descriptor+sectors)
+# no superfloppy with media descriptor F0h implies FAT12
+>>>>>>>>32	default		x		\b, FAT (12 bit by descriptor+sectors)
+# with media descriptor F8h floppy or hard disc with FAT12 or FAT16
+>>>>>>>21	ubyte		=0xF8		
+# 360 KiB with media descriptor F8h, 9 sectors per track ,single sided floppy implies FAT12
+>>>>>>>>19	ubequad	0xd002f80300090001	\b, FAT (12 bit by descriptor+geometry)
+# hard disc with FAT12 or FAT16
+>>>>>>>>19	default		x		\b, FAT (1Y bit by descriptor)
+# with media descriptor FAh floppy, RAM disc with FAT12 or FAT16 or Tandy hard disc
+>>>>>>>21	ubyte		=0xFA		
+# 320 KiB with media descriptor FAh, 8 sectors per track ,single sided floppy implies FAT12
+>>>>>>>>19	ubequad	0x8002fa0200080001	\b, FAT (12 bit by descriptor+geometry)
+# RAM disc with FAT12 or FAT16 or Tandy hard disc
+>>>>>>>>19	default		x		\b, FAT (1Y bit by descriptor)
+# others are floppy
+>>>>>>>21	default		x		\b, FAT (12 bit by descriptor)
 # FAT32 bit specific
->>>>>82		string		FAT32		\b, FAT (32 bit)
+>>>>>82		string/c	fat32		\b, FAT (32 bit)
 >>>>>>36	ulelong		x		\b, sectors/FAT %u
 # http://technet.microsoft.com/en-us/library/cc977221.aspx
 >>>>>>40	uleshort	>0		\b, extension flags 0x%x
@@ -1443,9 +1478,12 @@
 >>>>>>48	uleshort	>1		\b, infoSector %u
 #>>>>>>48	uleshort	=1		\b, infoSector %u (usual)
 >>>>>>48	uleshort	<1		\b, infoSector %u
->>>>>>50	uleshort	>6		\b, Backup boot sector %u
+# 0 or 0xFFFF instead of usual 6 means no backup sector
+>>>>>>50	uleshort	=0xFFFF		\b, no Backup boot sector
+>>>>>>50	uleshort	=0		\b, no Backup boot sector
 #>>>>>>50	uleshort	=6		\b, Backup boot sector %u (usual) 
->>>>>>50	uleshort	<6		\b, Backup boot sector %u
+>>>>>>50	default		x		
+>>>>>>>50	uleshort	x		\b, Backup boot sector %u
 # corrected by Joerg Jenderek at Feb 2011 according to http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO
 >>>>>>52	ulelong		>0		\b, reserved1 0x%x
 >>>>>>56	ulelong		>0		\b, reserved2 0x%x
@@ -1880,12 +1918,11 @@
 # defect IO.SYS+MSDOS.SYS ?
 #>>>>>0x162	use			2xDOS-filename
 
-# CDROM Filesystems
-# Modified for UDF by gerardo.cacciari@gmail.com
-32769	string    CD001     #
-!:mime	application/x-iso9660-image
+0	name				cdrom
 >38913	string   !NSR0      ISO 9660 CD-ROM filesystem data
+!:mime	application/x-iso9660-image
 >38913	string    NSR0      UDF filesystem data
+!:mime	application/x-iso9660-image
 >>38917	string    1         (version 1.0)
 >>38917	string    2         (version 1.5)
 >>38917	string    3         (version 2.0)
@@ -1898,6 +1935,18 @@
 !:mime	application/x-iso9660-image
 32777	string    CDROM     High Sierra CD-ROM filesystem data
 
+# CDROM Filesystems
+# https://en.wikipedia.org/wiki/ISO_9660
+# Modified for UDF by gerardo.cacciari@gmail.com
+32769	string    CD001
+# mime line at that position does not work
+# to display CD-ROM (70=81-11) after MBR (113=40+72+1), partition-table (71=50+21) and before Apple Driver Map (51)
+!:strength -11
+# to display CD-ROM (114=81+33) before MBR (113=40+72+1), partition-table (71=50+21) and Apple Driver Map (51)
+# does not work
+#!:strength +33
+>0	use cdrom
+
 # .cso files
 0    string    CISO	Compressed ISO CD image
 
@@ -2080,6 +2129,7 @@
 #----------------------------------------------------------
 #delta ISO    Daniel Novotny (dnovotny@redhat.com)
 0	string  DISO	Delta ISO data
+!:strength +50
 >4	belong  x	version %d
 
 # VMS backup savesets - gerardo.cacciari@gmail.com
@@ -2131,7 +2181,6 @@
 # which is mapped to VBN 2 of [000000]INDEXF.SYS;1 - gerardo.cacciari@gmail.com
 #
 1008    string          DECFILE11       Files-11 On-Disk Structure
->525    byte            x               Level %d
 >525    byte            x               (ODS-%d);
 >1017   string          A               RSX-11, VAX/VMS or OpenVMS VAX file system;
 >1017   string          B
@@ -2266,14 +2315,31 @@
 
 # UBIfs
 # Linux kernel sources: fs/ubifs/ubifs-media.h
-0       belong  0x31181006
->0x16   short   0               UBIfs image
->0x08   lequad  x               \b, sequence number %llu
->0x10   leshort x               \b, length %u
->0x04   lelong  x               \b, CRC 0x%08x
-
-0       belong  0x55424923
->0x04   short   <2
->0x05   string  \0\0\0
->0x1c   string  \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0
->0x04   short   x       UBI image, version %u
+0	lelong	0x06101831
+>0x16	leshort	0		UBIfs image
+>0x08	lequad	x		\b, sequence number %llu
+>0x10	leshort x		\b, length %u
+>0x04	lelong	x		\b, CRC 0x%08x
+
+0	lelong	0x23494255
+>0x04	leshort	<2
+>0x05	string	\0\0\0
+>0x1c	string	\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0
+>0x04	leshort	x		UBI image, version %u
+
+# NEC PC-88 2D disk image
+# From Fabio R. Schmidlin <sd-snatcher@users.sourceforge.net>
+0x20		ulelong&0xFFFFFEFF	0x2A0
+>0x10		string			\0\0\0\0\0\0\0\0\0\0
+>>0x280		string			\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0
+>>>0x1A		ubyte&0xEF		0
+>>>>0x1B	ubyte&0x8F		0
+>>>>>0x1B	ubyte&70		<0x40
+>>>>>>0x1C	ulelong			>0x21
+>>>>>>>0	regex	[[:print:]]*	NEC PC-88 disk image, name=%s
+>>>>>>>>0x1B	ubyte	0		\b, media=2D
+>>>>>>>>0x1B	ubyte	0x10		\b, media=2DD
+>>>>>>>>0x1B	ubyte	0x20		\b, media=2HD
+>>>>>>>>0x1B	ubyte	0x30		\b, media=1D
+>>>>>>>>0x1B	ubyte	0x40		\b, media=1DD
+>>>>>>>>0x1A	ubyte	0x10		\b, write-protected
diff --git a/contrib/file/magic/Magdir/images b/contrib/file/magic/Magdir/images
index fe04b49b..672dd88 100644
--- a/contrib/file/magic/Magdir/images
+++ b/contrib/file/magic/Magdir/images
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: images,v 1.91 2014/04/30 21:41:02 christos Exp $
+# $File: images,v 1.102 2015/01/02 02:36:35 christos Exp $
 # images:  file(1) magic for image formats (see also "iff", and "c-lang" for
 # XPM bitmaps)
 #
@@ -115,8 +115,153 @@
 # never changed.  The TIFF specification recommends testing for it.
 0	string		MM\x00\x2a	TIFF image data, big-endian
 !:mime	image/tiff
+>(4.L)	use		\^tiff_ifd
 0	string		II\x2a\x00	TIFF image data, little-endian
 !:mime	image/tiff
+>(4.l)	use		tiff_ifd
+
+0	name		tiff_ifd
+>0	leshort		x		\b, direntries=%d
+>2	use		tiff_entry
+
+0	name		tiff_entry
+# NewSubFileType
+>0	leshort		0xfe
+>>12	use		tiff_entry
+>0	leshort		0x100
+>>4	lelong		1
+>>>12	use		tiff_entry
+>>>8	leshort		x		\b, width=%d
+>0	leshort		0x101
+>>4	lelong		1
+>>>8	leshort		x		\b, height=%d
+>>>12	use		tiff_entry
+>0	leshort		0x102
+>>8	leshort		x		\b, bps=%d
+>>12	use		tiff_entry
+>0	leshort		0x103
+>>4	lelong		1		\b, compression=
+>>>8	leshort		1		\bnone
+>>>8	leshort		2		\bhuffman
+>>>8	leshort		3		\bbi-level group 3
+>>>8	leshort		4		\bbi-level group 4
+>>>8	leshort		5		\bLZW
+>>>8	leshort		6		\bJPEG (old)
+>>>8	leshort		7		\bJPEG
+>>>8	leshort		8		\bdeflate
+>>>8	leshort		9		\bJBIG, ITU-T T.85
+>>>8	leshort		0xa		\bJBIG, ITU-T T.43
+>>>8	leshort		0x7ffe		\bNeXT RLE 2-bit
+>>>8	leshort		0x8005		\bPackBits (Macintosh RLE)
+>>>8	leshort		0x8029		\bThunderscan RLE
+>>>8	leshort		0x807f		\bRasterPadding (CT or MP)
+>>>8	leshort		0x8080		\bRLE (Line Work)
+>>>8	leshort		0x8081		\bRLE (High-Res Cont-Tone)
+>>>8	leshort		0x8082		\bRLE (Binary Line Work)
+>>>8	leshort		0x80b2		\bDeflate (PKZIP)
+>>>8	leshort		0x80b3		\bKodak DCS
+>>>8	leshort		0x8765		\bJBIG
+>>>8	leshort		0x8798		\bJPEG2000
+>>>8	leshort		0x8799		\bNikon NEF Compressed
+>>>8	default		x	
+>>>>8	leshort		x		\b(unknown 0x%x)
+>>>12	use		tiff_entry
+>0	leshort		0x106		\b, PhotometricIntepretation=
+>>8	leshort		0		\bWhiteIsZero
+>>8	leshort		1		\bBlackIsZero
+>>8	leshort		2		\bRGB
+>>8	leshort		3		\bRGB Palette
+>>8	leshort		4		\bTransparency Mask
+>>8	leshort		5		\bCMYK
+>>8	leshort		6		\bYCbCr
+>>8	leshort		8		\bCIELab
+>>>8	leshort		x		\b(unknown=0x%x)
+>>12	use		tiff_entry
+# FillOrder
+>0	leshort		0x10a
+>>4	lelong		1
+>>>12	use		tiff_entry
+# DocumentName
+>0	leshort		0x10d
+>>(8.l)	string		x		\b, name=%s
+>>>12	use		tiff_entry
+# ImageDescription
+>0	leshort		0x10e
+>>(8.l)	string		x		\b, description=%s
+>>>12	use		tiff_entry
+# Make
+>0	leshort		0x10f
+>>(8.l)	string		x		\b, manufacturer=%s
+>>>12	use		tiff_entry
+# Model
+>0	leshort		0x110
+>>(8.l)	string		x		\b, model=%s
+>>>12	use		tiff_entry
+# StripOffsets
+>0	leshort		0x111
+>>12	use		tiff_entry
+# Orientation
+>0	leshort		0x112		\b, orientation=
+>>8	leshort		1		\bupper-left
+>>8	leshort		3		\blower-right
+>>8	leshort		6		\bupper-right
+>>8	leshort		8		\blower-left
+>>8	leshort		9		\bundefined
+>>8	default		x
+>>>8	leshort		x		\b[*%d*]
+>>12	use		tiff_entry
+# XResolution
+>0	leshort		0x11a
+>>8	lelong		x		\b, xresolution=%d
+>>12	use		tiff_entry
+# YResolution
+>0	leshort		0x11b
+>>8	lelong		x		\b, yresolution=%d
+>>12	use		tiff_entry
+# ResolutionUnit
+>0	leshort		0x128
+>>8	leshort		x		\b, resolutionunit=%d
+>>12	use		tiff_entry
+# Software
+>0	leshort		0x131
+>>(8.l)	string		x		\b, software=%s
+>>12	use		tiff_entry
+# Datetime
+>0	leshort		0x132
+>>(8.l)	string		x		\b, datetime=%s
+>>12	use		tiff_entry
+# HostComputer
+>0	leshort		0x13c
+>>(8.l)	string		x		\b, hostcomputer=%s
+>>12	use		tiff_entry
+# WhitePoint
+>0	leshort		0x13e
+>>12	use		tiff_entry
+# PrimaryChromaticities
+>0	leshort		0x13f
+>>12	use		tiff_entry
+# YCbCrCoefficients
+>0	leshort		0x211
+>>12	use		tiff_entry
+# YCbCrPositioning
+>0	leshort		0x213
+>>12	use		tiff_entry
+# ReferenceBlackWhite
+>0	leshort		0x214
+>>12	use		tiff_entry
+# Copyright
+>0	leshort		0x8298
+>>(8.l)	string		x		\b, copyright=%s
+>>12	use		tiff_entry
+# ExifOffset
+>0	leshort		0x8769
+>>12	use		tiff_entry
+# GPS IFD
+>0	leshort		0x8825		\b, GPS-Data
+>>12	use		tiff_entry
+
+#>0	leshort		x		\b, unknown=0x%x
+#>>12	use		tiff_entry
 
 0	string		MM\x00\x2b	Big TIFF image data, big-endian
 !:mime	image/tiff
@@ -892,6 +1037,65 @@
 0	string	\x46\x4d\x52\x00	ISO/IEC 19794-2 Format Minutiae Record (FMR)
 
 # WEBP https://developers.google.com/speed/webp/docs/riff_container
-0	string	RIFF
->8	string	WEBP	Web/P image data
->>4	lelong	x	\b, %d bytes
+#0	string	RIFF
+#>8	string	WEBP	Web/P image data
+#>>4	lelong	x	\b, %d bytes
+
+# doc: http://www.shikino.co.jp/eng/products/images/FLOWER.jpg.zip
+# example: http://www.shikino.co.jp/eng/products/images/FLOWER.wdp.zip
+90	bequad		0x574D50484F544F00	JPEG-XR Image
+>98	byte&0x08	=0x08			\b, hard tiling
+>99	byte&0x80	=0x80			\b, tiling present
+>99	byte&0x40	=0x40			\b, codestream present
+>99	byte&0x38	x			\b, spatial xform=
+>99	byte&0x38	0x00			\bTL
+>99	byte&0x38	0x08			\bBL
+>99	byte&0x38	0x10			\bTR
+>99	byte&0x38	0x18			\bBR
+>99	byte&0x38	0x20			\bBT
+>99	byte&0x38	0x28			\bRB
+>99	byte&0x38	0x30			\bLT
+>99	byte&0x38	0x38			\bLB
+>100	byte&0x80	=0x80			\b, short header
+>>102	beshort+1	x			\b, %d
+>>104	beshort+1	x			\bx%d
+>100	byte&0x80	=0x00			\b, long header
+>>102	belong+1	x			\b, %x
+>>106	belong+1	x			\bx%x
+>101	beshort&0xf	x			\b, bitdepth=
+>>101	beshort&0xf	0x0			\b1-WHITE=1
+>>101	beshort&0xf	0x1			\b8
+>>101	beshort&0xf	0x2			\b16
+>>101	beshort&0xf	0x3			\b16-SIGNED
+>>101	beshort&0xf	0x4			\b16-FLOAT
+>>101	beshort&0xf	0x5			\b(reserved 5)
+>>101	beshort&0xf	0x6			\b32-SIGNED
+>>101	beshort&0xf	0x7			\b32-FLOAT
+>>101	beshort&0xf	0x8			\b5
+>>101	beshort&0xf	0x9			\b10
+>>101	beshort&0xf	0xa			\b5-6-5
+>>101	beshort&0xf	0xb			\b(reserved %d)
+>>101	beshort&0xf	0xc			\b(reserved %d)
+>>101	beshort&0xf	0xd			\b(reserved %d)
+>>101	beshort&0xf	0xe			\b(reserved %d)
+>>101	beshort&0xf	0xf			\b1-BLACK=1
+>101	beshort&0xf0	x			\b, colorfmt=
+>>101	beshort&0xf0	0x00			\bYONLY
+>>101	beshort&0xf0	0x10			\bYUV240
+>>101	beshort&0xf0	0x20			\bYWV422
+>>101	beshort&0xf0	0x30			\bYWV444
+>>101	beshort&0xf0	0x40			\bCMYK
+>>101	beshort&0xf0	0x50			\bCMYKDIRECT
+>>101	beshort&0xf0	0x60			\bNCOMPONENT
+>>101	beshort&0xf0	0x70			\bRGB
+>>101	beshort&0xf0	0x80			\bRGBE
+>>101	beshort&0xf0	>0x80			\b(reserved 0x%x)
+
+# From: Johan van der Knijff <johan.vanderknijff@kb.nl>
+#
+# BPG (Better Portable Graphics) format
+# http://bellard.org/bpg/
+# http://fileformats.archiveteam.org/wiki/BPG
+#
+0	string	\x42\x50\x47\xFB	BPG (Better Portable Graphics)
+!:mime  image/bpg
diff --git a/contrib/file/magic/Magdir/jpeg b/contrib/file/magic/Magdir/jpeg
index bc8b342..cfe8973 100644
--- a/contrib/file/magic/Magdir/jpeg
+++ b/contrib/file/magic/Magdir/jpeg
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: jpeg,v 1.19 2013/02/04 15:50:03 christos Exp $
+# $File: jpeg,v 1.25 2015/01/02 16:56:50 christos Exp $
 # JPEG images
 # SunOS 5.5.1 had
 #
@@ -22,132 +22,73 @@
 >>11	byte		x		\b %d.
 >>12	byte		x		\b%02d
 # Next, the resolution or aspect ratio of the image:
-#>>13	byte		0		\b, aspect ratio
-#>>13	byte		1		\b, resolution (DPI)
-#>>13	byte		2		\b, resolution (DPCM)
-#>>4	beshort		x		\b, segment length %d
+>>13	byte		0		\b, aspect ratio
+>>13	byte		1		\b, resolution (DPI)
+>>13	byte		2		\b, resolution (DPCM)
+>>14	beshort		x		\b, density %dx
+>>16	beshort		x		\b%d
+>>4	beshort		x		\b, segment length %d
 # Next, show thumbnail info, if it exists:
 >>18	byte		!0		\b, thumbnail %dx
 >>>19	byte		x		\b%d
+>6	string		Exif		\b, Exif standard: [
+>>12	indirect/r	x
+>>12	string		x		\b]
 
-# EXIF moved down here to avoid reporting a bogus version number,
-# and EXIF version number printing added.
-#   - Patrik R=E5dman <patrik+file-magic@iki.fi>
->6	string		Exif		\b, EXIF standard
-# Look for EXIF IFD offset in IFD 0, and then look for EXIF version tag in EXIF IFD.
-# All possible combinations of entries have to be enumerated, since no looping
-# is possible. And both endians are possible...
-# The combinations included below are from real-world JPEGs.
-# Little-endian
->>12	string		II		
-# IFD 0 Entry #5:
->>>70	leshort		0x8769          
-# EXIF IFD Entry #1:
->>>>(78.l+14)	leshort	0x9000		
->>>>>(78.l+23)	byte	x		%c
->>>>>(78.l+24)	byte	x		\b.%c
->>>>>(78.l+25)	byte	!0x30		\b%c
-# IFD 0 Entry #9:
->>>118	leshort		0x8769          
-# EXIF IFD Entry #3:
->>>>(126.l+38)	leshort	0x9000		
->>>>>(126.l+47)	byte	x		%c
->>>>>(126.l+48)	byte	x		\b.%c
->>>>>(126.l+49)	byte	!0x30		\b%c
-# IFD 0 Entry #10
->>>130	leshort		0x8769          
-# EXIF IFD Entry #3:
->>>>(138.l+38)	leshort	0x9000		
->>>>>(138.l+47)	byte	x		%c
->>>>>(138.l+48)	byte	x		\b.%c
->>>>>(138.l+49)	byte	!0x30		\b%c
-# EXIF IFD Entry #4:
->>>>(138.l+50)	leshort	0x9000		
->>>>>(138.l+59)	byte	x		%c
->>>>>(138.l+60)	byte	x		\b.%c
->>>>>(138.l+61)	byte	!0x30		\b%c
-# EXIF IFD Entry #5:
->>>>(138.l+62)	leshort	0x9000		
->>>>>(138.l+71)	byte	x		%c
->>>>>(138.l+72)	byte	x		\b.%c
->>>>>(138.l+73)	byte	!0x30		\b%c
-# IFD 0 Entry #11
->>>142	leshort		0x8769          
-# EXIF IFD Entry #3:
->>>>(150.l+38)	leshort	0x9000		
->>>>>(150.l+47)	byte	x		%c
->>>>>(150.l+48)	byte	x		\b.%c
->>>>>(150.l+49)	byte	!0x30		\b%c
-# EXIF IFD Entry #4:
->>>>(150.l+50)	leshort	0x9000		
->>>>>(150.l+59)	byte	x		%c
->>>>>(150.l+60)	byte	x		\b.%c
->>>>>(150.l+61)	byte	!0x30		\b%c
-# EXIF IFD Entry #5:
->>>>(150.l+62)	leshort	0x9000		
->>>>>(150.l+71)	byte	x		%c
->>>>>(150.l+72)	byte	x		\b.%c
->>>>>(150.l+73)	byte	!0x30		\b%c
-# Big-endian
->>12	string		MM		
-# IFD 0 Entry #9:
->>>118	beshort		0x8769          
-# EXIF IFD Entry #1:
->>>>(126.L+14)	beshort	0x9000		
->>>>>(126.L+23)	byte	x		%c
->>>>>(126.L+24)	byte	x		\b.%c
->>>>>(126.L+25)	byte	!0x30		\b%c
-# EXIF IFD Entry #3:
->>>>(126.L+38)	beshort	0x9000		
->>>>>(126.L+47)	byte	x		%c
->>>>>(126.L+48)	byte	x		\b.%c
->>>>>(126.L+49)	byte	!0x30		\b%c
-# IFD 0 Entry #10
->>>130	beshort		0x8769          
-# EXIF IFD Entry #3:
->>>>(138.L+38)	beshort	0x9000		
->>>>>(138.L+47)	byte	x		%c
->>>>>(138.L+48)	byte	x		\b.%c
->>>>>(138.L+49)	byte	!0x30		\b%c
-# EXIF IFD Entry #5:
->>>>(138.L+62)	beshort	0x9000		
->>>>>(138.L+71)	byte	x		%c
->>>>>(138.L+72)	byte	x		\b.%c
->>>>>(138.L+73)	byte	!0x30		\b%c
-# IFD 0 Entry #11
->>>142	beshort		0x8769          
-# EXIF IFD Entry #4:
->>>>(150.L+50)	beshort	0x9000		
->>>>>(150.L+59)	byte	x		%c
->>>>>(150.L+60)	byte	x		\b.%c
->>>>>(150.L+61)	byte	!0x30		\b%c
-# Here things get sticky.  We can do ONE MORE marker segment with
-# indirect addressing, and that's all.  It would be great if we could
-# do pointer arithemetic like in an assembler language.  Christos?
-# And if there was some sort of looping construct to do searches, plus a few
-# named accumulators, it would be even more effective...
-# At least we can show a comment if no other segments got inserted before:
->(4.S+5)	byte		0xFE		\b, comment:
->>(4.S+6)	pstring/HJ	x		"%s"
-# Or, we can show the encoding type (I've included only the three most common)
-# and image dimensions if we are lucky and the SOFn (image segment) is here:
->(4.S+5)	byte		0xC0		\b, baseline
->>(4.S+6)	byte		x		\b, precision %d
->>(4.S+7)	beshort		x		\b, %dx
->>(4.S+9)	beshort		x		\b%d
->(4.S+5)	byte		0xC1		\b, extended sequential
->>(4.S+6)	byte		x		\b, precision %d
->>(4.S+7)	beshort		x		\b, %dx
->>(4.S+9)	beshort		x		\b%d
->(4.S+5)	byte		0xC2		\b, progressive
->>(4.S+6)	byte		x		\b, precision %d
->>(4.S+7)	beshort		x		\b, %dx
->>(4.S+9)	beshort		x		\b%d
-# I've commented-out quantisation table reporting.  I doubt anyone cares yet.
-#>(4.S+5)	byte		0xDB		\b, quantisation table
-#>>(4.S+6)	beshort		x		\b length=%d
-#>14	beshort		x		\b, %d x
-#>16	beshort		x		\b %d
+# Jump to the first segment
+>(4.S+4)	use		jpeg_segment
+
+# This uses recursion...
+0		name		jpeg_segment
+>0	beshort		0xFFFE
+>>(2.S+2)	use			jpeg_segment
+>>2	pstring/HJ	x		\b, comment: "%s"
+
+>0	beshort		0xFFC0
+>>(2.S+2)	use			jpeg_segment
+>>4	byte		x		\b, baseline, precision %d
+>>7	beshort		x		\b, %dx
+>>5	beshort		x		\b%d
+>>9	byte		x		\b, frames %d
+
+>0	beshort		0xFFC1		
+>>(2.S+2)	use			jpeg_segment
+>>4	byte		x		\b, extended sequential, precision %d
+>>7	beshort		x		\b, %dx
+>>5	beshort		x		\b%d
+>>9	byte		x		\b, frames %d
+
+>0	beshort		0xFFC2		
+>>(2.S+2)	use			jpeg_segment
+>>4	byte		x		\b, progressive, precision %d
+>>7	beshort		x		\b, %dx
+>>5	beshort		x		\b%d
+>>9	byte		x		\b, frames %d
+
+# Define Huffman Tables
+>0	beshort		0xFFC4
+>>(2.S+2)	use			jpeg_segment
+
+>0	beshort		0xFFE1		
+#>>(2.S+2)	use			jpeg_segment
+>>4	string		Exif		\b, Exif Standard: [
+>>>10	indirect/r	x		
+>>>10	string		x		\b]
+
+# Application specific markers
+>0	beshort&0xFFE0	=0xFFE0
+>>(2.S+2)	use			jpeg_segment
+
+# DB: Define Quantization tables
+# DD: Define Restart interval [XXX: wrong here, it is 4 bytes]
+# D8: Start of image
+# D9: End of image
+# Dn: Restart
+>0	beshort&0xFFD0	=0xFFD0
+>>(2.S+2)	use			jpeg_segment
+
+#>0	beshort		x		unknown 0x%x
+#>>(2.S+2)	use			jpeg_segment
 
 # HSI is Handmade Software's proprietary JPEG encoding scheme
 0	string		hsi1		JPEG image data, HSI proprietary
diff --git a/contrib/file/magic/Magdir/kerberos b/contrib/file/magic/Magdir/kerberos
new file mode 100644
index 0000000..cb07fed
--- /dev/null
+++ b/contrib/file/magic/Magdir/kerberos
@@ -0,0 +1,45 @@
+
+#------------------------------------------------------------------------------
+# $File: kerberos,v 1.1 2014/12/10 18:45:43 christos Exp $
+# kerberos: MIT kerberos file binary formats
+#
+
+# This magic entry is for demonstration purposes and could be improved
+# if the following features were implemented in file:
+#
+# Strings inside [[ .. ]] in the descriptions have special meanings and
+# are not printed.
+#
+# 	- Provide some form of iteration in number of components
+#		[[${counter}=%d]] in the description
+#		then append
+#		[${counter}--] in the offset of the entries
+#	- Provide a way to round the next offset
+#		Add [R:4] after the offset?
+#	- Provide a way to have optional entries
+#		XXX: Syntax:
+#	- Provide a way to "save" entries to print them later.
+#		if the description is [[${name}=%s]], then nothing is
+#		printed and a subsequent entry in the same magic file
+#		can refer to ${name}
+#	- Provide a way to format strings as hex values
+#
+# http://www.gnu.org/software/shishi/manual/html_node/\
+#	The-Keytab-Binary-File-Format.html
+#
+
+0		name		keytab_entry
+#>0		beshort		x		\b, size=%d
+#>2		beshort		x		\b, components=%d
+>4		pstring/H	x		\b, realm=%s
+>>&0		pstring/H	x		\b, principal=%s/
+>>>&0		pstring/H	x		\b%s
+>>>>&0		belong		x		\b, type=%d
+>>>>>&0		bedate		x		\b, date=%s
+>>>>>>&0	byte		x		\b, kvno=%u
+#>>>>>>>&0	pstring/H	x
+#>>>>>>>>&0	belong		x	
+#>>>>>>>>>>&0	use		keytab_entry
+
+0		belong		0x05020000	Kerberos Keytab file
+>4		use		keytab_entry
diff --git a/contrib/file/magic/Magdir/linux b/contrib/file/magic/Magdir/linux
index c2b1762..d3f6a9d 100644
--- a/contrib/file/magic/Magdir/linux
+++ b/contrib/file/magic/Magdir/linux
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: linux,v 1.57 2014/05/20 20:10:17 christos Exp $
+# $File: linux,v 1.59 2014/11/03 21:03:36 christos Exp $
 # linux:  file(1) magic for Linux files
 #
 # Values for Linux/i386 binaries, from Daniel Quinlan <quinlan@yggdrasil.com>
@@ -413,3 +413,22 @@
 >>>>20		belong		>16
 >>>>>36		belong		x	\b, DT structure block size=%d
 
+# glibc locale archive as defined in glibc locale/locarchive.h
+0		lelong		0xde020109	locale archive
+>24		lelong		x		%d strings
+
+# Summary:     Database file for mlocate
+# Description: A database file as used by mlocate, a fast implementation
+#              of locate/updatedb. It uses merging to reuse the existing
+#              database and avoid rereading most of the filesystem. It's
+#              the default version of locate on Arch Linux (and others).
+# File path:   /var/lib/mlocate/mlocate.db by default (but configurable)
+# Site:        https://fedorahosted.org/mlocate/
+# Format docs: http://linux.die.net/man/5/mlocate.db
+# Type: mlocate database file
+# URL:  https://fedorahosted.org/mlocate/
+# From: Wander Nauta <info@wandernauta.nl>
+0		string		\0mlocate	mlocate database
+>12		byte		x		\b, version %d
+>13		byte		1		\b, require visibility
+>16		string		x		\b, root %s
diff --git a/contrib/file/magic/Magdir/macintosh b/contrib/file/magic/Magdir/macintosh
index d86fd8f..3ca2cab 100644
--- a/contrib/file/magic/Magdir/macintosh
+++ b/contrib/file/magic/Magdir/macintosh
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: macintosh,v 1.23 2013/11/19 18:47:58 christos Exp $
+# $File: macintosh,v 1.25 2014/09/03 13:34:16 christos Exp $
 # macintosh description
 #
 # BinHex is the Macintosh ASCII-encoded file format (see also "apple")
@@ -165,7 +165,7 @@
 #>65	string		ZSYS		(Pre-System 7 system file)
 #>65	string		acf3		(Aldus FreeHand)
 #>65	string		cdev		(control panel)
-#>65	string		dfil		(Desk Acessory suitcase)
+#>65	string		dfil		(Desk Accessory suitcase)
 #>65	string		libr		(library)
 #>65	string		nX^d		(WriteNow word processor)
 #>65	string		nX^w		(WriteNow dictionary)
@@ -288,20 +288,38 @@
 >0x412	beshort			x		number of blocks: %d,
 >0x424	pstring			x		volume name: %s
 
+# *.hfs updated by Joerg Jenderek
+# http://en.wikipedia.org/wiki/Hierarchical_File_System
 # "BD" gives many false positives
-#0x400	beshort			0x4244		Macintosh HFS data
-#>0	beshort			0x4C4B		(bootable)
-#>0x40a	beshort			&0x8000		(locked)
-#>0x40a	beshort			^0x0100		(mounted)
-#>0x40a	beshort			&0x0200		(spared blocks)
-#>0x40a	beshort			&0x0800		(unclean)
-#>0x47C	beshort			0x482B		(Embedded HFS+ Volume)
-#>0x402	beldate-0x7C25B080	x		created: %s,
-#>0x406	beldate-0x7C25B080	x		last modified: %s,
-#>0x440	beldate-0x7C25B080	>0		last backup: %s,
-#>0x414	belong			x		block size: %d,
-#>0x412	beshort			x		number of blocks: %d,
-#>0x424	pstring			x		volume name: %s
+0x400	beshort			0x4244		
+# ftp://ftp.mars.org/pub/hfs/hfsutils-3.2.6.tar.gz/hfsutils-3.2.6/libhfs/apple.h
+# first block of volume bit map (always 3)
+>0x40e	ubeshort		0x0003		
+# maximal length of volume name is 27
+>>0x424		ubyte			<28	Macintosh HFS data
+#!:mime	application/octet-stream
+# these mime and apple types are not sure
+!:mime	application/x-apple-diskimage
+#!:apple	hfsdINIT
+#!:apple	MACSdisk
+>>>0		beshort			0x4C4B	(bootable)
+#>>>0		beshort			0x0000	(not bootable)
+>>>0x40a	beshort			&0x8000	(locked)
+>>>0x40a	beshort			^0x0100	(mounted)
+>>>0x40a	beshort			&0x0200	(spared blocks)
+>>>0x40a	beshort			&0x0800	(unclean)
+>>>0x47C	beshort			0x482B	(Embedded HFS+ Volume)
+# http://www.epochconverter.com/
+# 0x7C245F00 seconds	~ 2082758400	~ 01 Jan 2036 00:00:00	~ 66 years to 1970
+# 0x7C25B080 seconds	~ 2082844800	~ 02 Jan 2036 00:00:00
+# construct not working
+#>>>0x402	beldate-0x7C25B080	x	created: %s,
+#>>>0x406	beldate-0x7C25B080	x	last modified: %s,
+#>>>0x440	beldate-0x7C25B080	>0	last backup: %s,
+# found block sizes 200h,1200h,2800h
+>>>0x414	belong			x	block size: %d,
+>>>0x412	beshort			x	number of blocks: %d,
+>>>0x424	pstring			x	volume name: %s
 
 0x400	beshort			0x482B		Macintosh HFS Extended
 >&0	beshort			x		version %d data
@@ -322,43 +340,9 @@
 >&42	belong			x		number of blocks: %d,
 >&46	belong			x		free blocks: %d
 
-# I don't think this is really necessary since it doesn't do much and 
-# anything with a valid driver descriptor will also have a valid
-# partition map
-#0		beshort		0x4552		Apple Device Driver data
-#>&24		beshort		=1		\b, MacOS
-
-# Is that the partition type a cstring or a pstring? Well, IM says "strings 
-# shorter than 32 bytes must be terminated with NULL" so I'll treat it as a 
-# cstring. Of course, partitions can contain more than four entries, but 
-# what're you gonna do?
-# GRR: This magic is too weak, it is just "PM"
-#0x200		beshort		0x504D		Apple Partition data
-#>0x2		beshort		x		(block size: %d):
-#>0x230		string		x		first type: %s,
-#>0x210		string		x		name: %s,
-#>0x254		belong		x		number of blocks: %d,
-#>0x400		beshort		0x504D		
-#>>0x430		string		x		second type: %s,
-#>>0x410		string		x		name: %s,
-#>>0x454		belong		x		number of blocks: %d,
-#>>0x600		beshort		0x504D
-#>>>0x630	string		x		third type: %s,
-#>>>0x610	string		x		name: %s,
-#>>>0x654	belong		x		number of blocks: %d,
-#>>0x800		beshort		0x504D		
-#>>>0x830	string		x		fourth type: %s,
-#>>>0x810	string		x		name: %s,
-#>>>0x854	belong		x		number of blocks: %d,
-#>>>0xa00	beshort		0x504D		
-#>>>>0xa30	string		x		fifth type: %s,
-#>>>>0xa10	string		x		name: %s,
-#>>>>0xa54	belong		x		number of blocks: %d
-#>>>0xc00	beshort		0x504D
-#>>>>0xc30	string		x		sixth type: %s,
-#>>>>0xc10	string		x		name: %s,
-#>>>>0xc54	belong		x		number of blocks: %d
 ## AFAIK, only the signature is different
+# same as Apple Partition Map
+# GRR: This magic is too weak, it is just "TS"
 #0x200		beshort		0x5453		Apple Old Partition data
 #>0x2		beshort		x		block size: %d,
 #>0x230		string		x		first type: %s,
diff --git a/contrib/file/magic/Magdir/rinex b/contrib/file/magic/Magdir/meteorological
index c5f2bcb..541bbbf 100644
--- a/contrib/file/magic/Magdir/rinex
+++ b/contrib/file/magic/Magdir/meteorological
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: rinex,v 1.4 2011/05/03 01:44:17 christos Exp $
+# $File: meteorological,v 1.1 2014/08/04 06:26:16 christos Exp $
 # rinex:  file(1) magic for RINEX files
 # http://igscb.jpl.nasa.gov/igscb/data/format/rinex210.txt
 # ftp://cddis.gsfc.nasa.gov/pub/reports/formats/rinex300.pdf
@@ -42,3 +42,8 @@
 >>&32	string		x		\b, date %15.15s
 >>5	string		x		\b, version %6.6s
 !:mime	rinex/observation
+
+# https://en.wikipedia.org/wiki/GRIB
+0	string	GRIB
+>7	byte	=1	Gridded binary (GRIB) version 1
+>7	byte	=2	Gridded binary (GRIB) version 2
diff --git a/contrib/file/magic/Magdir/msooxml b/contrib/file/magic/Magdir/msooxml
index e5be5b3..059e729 100644
--- a/contrib/file/magic/Magdir/msooxml
+++ b/contrib/file/magic/Magdir/msooxml
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: msooxml,v 1.4 2014/01/06 18:16:24 rrt Exp $
+# $File: msooxml,v 1.5 2014/08/05 07:38:45 christos Exp $
 # msooxml:  file(1) magic for Microsoft Office XML
 # From: Ralf Brown <ralf.brown@gmail.com>
 
@@ -16,7 +16,7 @@
 0		string		PK\003\004
 !:strength +10
 # make sure the first file is correct
->0x1E		regex		\[Content_Types\]\.xml|_rels/\.rels
+>0x1E		regex		\\[Content_Types\\]\\.xml|_rels/\\.rels
 # skip to the second local file header
 # since some documents include a 520-byte extra field following the file
 # header, we need to scan for the next header
diff --git a/contrib/file/magic/Magdir/netbsd b/contrib/file/magic/Magdir/netbsd
index 3234610..aa933ff 100644
--- a/contrib/file/magic/Magdir/netbsd
+++ b/contrib/file/magic/Magdir/netbsd
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: netbsd,v 1.21 2014/03/29 15:40:34 christos Exp $
+# $File: netbsd,v 1.22 2014/12/08 20:53:52 christos Exp $
 # netbsd:  file(1) magic for NetBSD objects
 #
 # All new-style magic numbers are in network byte order.
@@ -247,7 +247,7 @@
 # Kernel core dump format
 0	belong&0x0000ffff 0x00008fca	NetBSD kernel core file
 >0	belong&0x03ff0000 0x00000000	\b, Unknown
->0	belong&0x03ff0000 0x00001000	\b, sun 68010/68020
+>0	belong&0x03ff0000 0x00010000	\b, sun 68010/68020
 >0	belong&0x03ff0000 0x00020000	\b, sun 68020
 >0	belong&0x03ff0000 0x00640000	\b, 386 PC
 >0	belong&0x03ff0000 0x00860000	\b, i386 BSD
@@ -262,20 +262,24 @@
 >0	belong&0x03ff0000 0x008f0000	\b, arm6 BSD
 >0	belong&0x03ff0000 0x00900000	\b, m68k BSD (2K pages)
 >0	belong&0x03ff0000 0x00910000	\b, sh3 BSD
->0	belong&0x03ff0000 0x00920000	\b, ppc BSD (Big Endian)
->0	belong&0x03ff0000 0x00930000	\b, vax BSD (4K pages)
->0	belong&0x03ff0000 0x00940000	\b, mips1 BSD
->0	belong&0x03ff0000 0x00950000	\b, mips2 BSD
->0	belong&0x03ff0000 0x00960000	\b, parisc BSD
->0	belong&0x03ff0000 0x00970000	\b, sh5/64 BSD
->0	belong&0x03ff0000 0x00980000	\b, SPARC/64 BSD
->0	belong&0x03ff0000 0x00990000	\b, amd64 BSD
->0	belong&0x03ff0000 0x009a0000	\b, hp200 (68010) BSD
->0	belong&0x03ff0000 0x009b0000	\b, hp300 (68020+68881) BSD
->0	belong&0x03ff0000 0x009b0000	\b, hp300 (68020+68881) BSD
->0	belong&0x03ff0000 0x00c80000	\b, hp200
->0	belong&0x03ff0000 0x020b0000	\b, hp300 (68020+68881) HP-UX
->0	belong&0x03ff0000 0x020c0000	\b, hp300 (68020+68881) HP-UX
+>0	belong&0x03ff0000 0x00950000	\b, ppc BSD (Big Endian)
+>0	belong&0x03ff0000 0x00960000	\b, vax BSD (4K pages)
+>0	belong&0x03ff0000 0x00970000	\b, mips1 BSD
+>0	belong&0x03ff0000 0x00980000	\b, mips2 BSD
+>0	belong&0x03ff0000 0x00990000	\b, m88k BSD
+>0	belong&0x03ff0000 0x00920000	\b, parisc BSD
+>0	belong&0x03ff0000 0x009b0000	\b, sh5/64 BSD
+>0	belong&0x03ff0000 0x009c0000	\b, SPARC/64 BSD
+>0	belong&0x03ff0000 0x009d0000	\b, amd64 BSD
+>0	belong&0x03ff0000 0x009e0000	\b, sh5/32 BSD
+>0	belong&0x03ff0000 0x009f0000	\b, ia64 BSD
+>0	belong&0x03ff0000 0x00b70000	\b, aarch64 BSD
+>0	belong&0x03ff0000 0x00b80000	\b, or1k BSD
+>0	belong&0x03ff0000 0x00b90000	\b, Risk-V BSD
+>0	belong&0x03ff0000 0x00c80000	\b, hp200 BSD
+>0	belong&0x03ff0000 0x012c0000	\b, hp300 BSD
+>0	belong&0x03ff0000 0x020b0000	\b, hp800 HP-UX
+>0	belong&0x03ff0000 0x020c0000	\b, hp200/hp300 HP-UX
 >0	belong&0xfc000000 0x04000000	\b, CPU
 >0	belong&0xfc000000 0x08000000	\b, DATA
 >0	belong&0xfc000000 0x10000000	\b, STACK
diff --git a/contrib/file/magic/Magdir/pascal b/contrib/file/magic/Magdir/pascal
index a134a47..eebd349 100644
--- a/contrib/file/magic/Magdir/pascal
+++ b/contrib/file/magic/Magdir/pascal
@@ -1,10 +1,10 @@
 #------------------------------------------------------------------------------
-# $File: pascal,v 1.1 2011/12/08 12:12:46 rrt Exp $
+# $File: pascal,v 1.2 2014/07/14 14:21:33 rrt Exp $
 # pascal:  file(1) magic for Pascal source
 #
 0	search/8192	(input,		Pascal source text
 !:mime	text/x-pascal
-0	regex		\^program	Pascal source text
-!:mime	text/x-pascal
-0	regex           	\^record		Pascal source text
-!:mime	text/x-pascal
+#0	regex		\^program	Pascal source text
+#!:mime	text/x-pascal
+#0	regex           	\^record		Pascal source text
+#!:mime	text/x-pascal
diff --git a/contrib/file/magic/Magdir/pgp b/contrib/file/magic/Magdir/pgp
index a8d3c9a..95a6766 100644
--- a/contrib/file/magic/Magdir/pgp
+++ b/contrib/file/magic/Magdir/pgp
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: pgp,v 1.9 2009/09/19 16:28:11 christos Exp $
+# $File: pgp,v 1.11 2014/11/11 21:32:38 christos Exp $
 # pgp:  file(1) magic for Pretty Good Privacy
 # see http://lists.gnupg.org/pipermail/gnupg-devel/1999-September/016052.html
 #
@@ -21,7 +21,449 @@
 
 2	string	---BEGIN\ PGP\ PUBLIC\ KEY\ BLOCK-	PGP public key block
 !:mime	application/pgp-keys
+>10	search/100	\n\n
+>>&0	use		pgp
 0	string	-----BEGIN\040PGP\40MESSAGE-		PGP message
 !:mime	application/pgp
+>10	search/100	\n\n
+>>&0	use		pgp
 0	string	-----BEGIN\040PGP\40SIGNATURE-		PGP signature
 !:mime	application/pgp-signature
+>10	search/100	\n\n
+>>&0	use		pgp
+
+# Decode the type of the packet based on it's base64 encoding.
+# Idea from Mark Martinec
+# The specification is in RFC 4880, section 4.2 and 4.3:
+# http://tools.ietf.org/html/rfc4880#section-4.2
+
+0	name		pgp
+>0	byte		0x67		Reserved (old)
+>0	byte		0x68		Public-Key Encrypted Session Key (old)
+>0	byte		0x69		Signature (old)
+>0	byte		0x6a		Symmetric-Key Encrypted Session Key (old)
+>0	byte		0x6b		One-Pass Signature (old)
+>0	byte		0x6c		Secret-Key (old)
+>0	byte		0x6d		Public-Key (old)
+>0	byte		0x6e		Secret-Subkey (old)
+>0	byte		0x6f		Compressed Data (old)
+>0	byte		0x70		Symmetrically Encrypted Data (old)
+>0	byte		0x71		Marker (old)
+>0	byte		0x72		Literal Data (old)
+>0	byte		0x73		Trust (old)
+>0	byte		0x74		User ID (old)
+>0	byte		0x75		Public-Subkey (old)
+>0	byte		0x76		Unused (old)
+>0	byte		0x77
+>>1	byte&0xc0	0x00		Reserved
+>>1	byte&0xc0	0x40		Public-Key Encrypted Session Key
+>>1	byte&0xc0	0x80		Signature
+>>1	byte&0xc0	0xc0		Symmetric-Key Encrypted Session Key
+>0	byte		0x78
+>>1	byte&0xc0	0x00		One-Pass Signature
+>>1	byte&0xc0	0x40		Secret-Key
+>>1	byte&0xc0	0x80		Public-Key
+>>1	byte&0xc0	0xc0		Secret-Subkey
+>0	byte		0x79
+>>1	byte&0xc0	0x00		Compressed Data
+>>1	byte&0xc0	0x40		Symmetrically Encrypted Data
+>>1	byte&0xc0	0x80		Marker
+>>1	byte&0xc0	0xc0		Literal Data
+>0	byte		0x7a
+>>1	byte&0xc0	0x00		Trust
+>>1	byte&0xc0	0x40		User ID
+>>1	byte&0xc0	0x80		Public-Subkey
+>>1	byte&0xc0	0xc0		Unused [z%x]
+>0	byte		0x30
+>>1	byte&0xc0	0x00		Unused [0%x]
+>>1	byte&0xc0	0x40		User Attribute
+>>1	byte&0xc0	0x80		Sym. Encrypted and Integrity Protected Data 
+>>1	byte&0xc0	0xc0		Modification Detection Code
+
+# magic signatures to detect PGP crypto material (from stef)
+# detects and extracts metadata from:
+#  - symmetric encrypted packet header
+#  - RSA (e=65537) secret (sub-)keys
+
+# 1024b RSA encrypted data
+
+0	string	\x84\x8c\x03		PGP RSA encrypted session key -
+>3	lelong	x			keyid: %X
+>7	lelong	x			%X
+>11	byte	0x01			RSA (Encrypt or Sign) 1024b
+>11	byte	0x02			RSA Encrypt-Only 1024b
+>12	string	\x04\x00
+>12	string	\x03\xff
+>12	string	\x03\xfe
+>12	string	\x03\xfd
+>12	string	\x03\xfc
+>12	string	\x03\xfb
+>12	string	\x03\xfa
+>12	string	\x03\xf9
+>142	byte	0xd2			.
+
+# 2048b RSA encrypted data
+
+0	string	\x85\x01\x0c\x03	PGP RSA encrypted session key -
+>4	lelong	x			keyid: %X
+>8	lelong	x			%X
+>12	byte	0x01			RSA (Encrypt or Sign) 2048b
+>12	byte	0x02			RSA Encrypt-Only 2048b
+>13	string	\x08\x00
+>13	string	\x07\xff
+>13	string	\x07\xfe
+>13	string	\x07\xfd
+>13	string	\x07\xfc
+>13	string	\x07\xfb
+>13	string	\x07\xfa
+>13	string	\x07\xf9
+>271	byte	0xd2			.
+
+# 3072b RSA encrypted data
+
+0	string	\x85\x01\x8c\x03	PGP RSA encrypted session key -
+>4	lelong	x			keyid: %X
+>8	lelong	x			%X
+>12	byte	0x01			RSA (Encrypt or Sign) 3072b
+>12	byte	0x02			RSA Encrypt-Only 3072b
+>13	string	\x0c\x00
+>13	string	\x0b\xff
+>13	string	\x0b\xfe
+>13	string	\x0b\xfd
+>13	string	\x0b\xfc
+>13	string	\x0b\xfb
+>13	string	\x0b\xfa
+>13	string	\x0b\xf9
+>399	byte	0xd2			.
+
+# 3072b RSA encrypted data
+
+0	string	\x85\x02\x0c\x03	PGP RSA encrypted session key -
+>4	lelong	x			keyid: %X
+>8	lelong	x			%X
+>12	byte	0x01			RSA (Encrypt or Sign) 4096b
+>12	byte	0x02			RSA Encrypt-Only 4096b
+>13	string	\x10\x00
+>13	string	\x0f\xff
+>13	string	\x0f\xfe
+>13	string	\x0f\xfd
+>13	string	\x0f\xfc
+>13	string	\x0f\xfb
+>13	string	\x0f\xfa
+>13	string	\x0f\xf9
+>527	byte	0xd2			.
+
+# 4096b RSA encrypted data
+
+0	string	\x85\x04\x0c\x03	PGP RSA encrypted session key -
+>4	lelong	x			keyid: %X
+>8	lelong	x			%X
+>12	byte	0x01			RSA (Encrypt or Sign) 8129b
+>12	byte	0x02			RSA Encrypt-Only 8129b
+>13	string	\x20\x00
+>13	string	\x1f\xff
+>13	string	\x1f\xfe
+>13	string	\x1f\xfd
+>13	string	\x1f\xfc
+>13	string	\x1f\xfb
+>13	string	\x1f\xfa
+>13	string	\x1f\xf9
+>1039	byte	0xd2			.
+
+# crypto algo mapper
+
+0	name	crypto
+>0	byte	0x00			Plaintext or unencrypted data
+>0	byte	0x01			IDEA
+>0	byte	0x02			TripleDES
+>0	byte	0x03			CAST5 (128 bit key)
+>0	byte	0x04			Blowfish (128 bit key, 16 rounds)
+>0	byte	0x07			AES with 128-bit key
+>0	byte	0x08			AES with 192-bit key
+>0	byte	0x09			AES with 256-bit key
+>0	byte	0x0a			Twofish with 256-bit key
+
+# hash algo mapper
+
+0	name	hash
+>0	byte	0x01			MD5
+>0	byte	0x02			SHA-1
+>0	byte	0x03			RIPE-MD/160
+>0	byte	0x08			SHA256
+>0	byte	0x09			SHA384
+>0	byte	0x0a			SHA512
+>0	byte	0x0b			SHA224
+
+# pgp symmetric encrypted data
+
+0	byte	0x8c			PGP symmetric key encrypted data -
+>1	byte	0x0d
+>1	byte	0x0c
+>2	byte	0x04
+>3	use	crypto
+>4	byte	0x01			salted -
+>>5	use	hash
+>>14	byte	0xd2			.
+>>14	byte	0xc9			.
+>4	byte	0x03			salted & iterated -
+>>5	use	hash
+>>15	byte	0xd2			.
+>>15	byte	0xc9			.
+
+# encrypted keymaterial needs s2k & can be checksummed/hashed
+
+0	name	chkcrypto
+>0	use	crypto
+>1	byte	0x00			Simple S2K
+>1	byte	0x01			Salted S2K
+>1	byte	0x03			Salted&Iterated S2K
+>2	use	hash
+
+# all PGP keys start with this prolog
+# containing version, creation date, and purpose
+
+0	name	keyprolog
+>0	byte	0x04
+>1	beldate	x			created on %s -
+>5	byte	0x01			RSA (Encrypt or Sign)
+>5	byte	0x02			RSA Encrypt-Only
+
+# end of secret keys known signature
+# contains e=65537 and the prolog to
+# the encrypted parameters
+
+0	name	keyend
+>0	string	\x00\x11\x01\x00\x01	e=65537
+>5	use	crypto
+>5	byte	0xff			checksummed
+>>6	use	chkcrypto
+>5	byte	0xfe			hashed
+>>6	use	chkcrypto
+
+# PGP secret keys contain also the public parts
+# these vary by bitsize of the key
+
+0	name	x1024
+>0	use	keyprolog
+>6	string	\x03\xfe
+>6	string	\x03\xff
+>6	string	\x04\x00
+>136	use	keyend
+
+0	name	x2048
+>0	use	keyprolog
+>6	string	\x80\x00
+>6	string	\x07\xfe
+>6	string	\x07\xff
+>264	use	keyend
+
+0	name	x3072
+>0	use	keyprolog
+>6	string	\x0b\xfe
+>6	string	\x0b\xff
+>6	string	\x0c\x00
+>392	use	keyend
+
+0	name	x4096
+>0	use	keyprolog
+>6	string	\x10\x00
+>6	string	\x0f\xfe
+>6	string	\x0f\xff
+>520	use	keyend
+
+# \x00|\x1f[\xfe\xff]).{1024})'
+0	name	x8192
+>0	use	keyprolog
+>6	string	\x20\x00
+>6	string	\x1f\xfe
+>6	string	\x1f\xff
+>1032	use	keyend
+
+# depending on the size of the pkt
+# we branch into the proper key size
+# signatures defined as x{keysize}
+
+>0	name	pgpkey
+>0	string	\x01\xd8	1024b
+>>2	use	x1024
+>0	string	\x01\xeb	1024b
+>>2	use	x1024
+>0	string	\x01\xfb	1024b
+>>2	use	x1024
+>0	string	\x01\xfd	1024b
+>>2	use	x1024
+>0	string	\x01\xf3	1024b
+>>2	use	x1024
+>0	string	\x01\xee	1024b
+>>2	use	x1024
+>0	string	\x01\xfe	1024b
+>>2	use	x1024
+>0	string	\x01\xf4	1024b
+>>2	use	x1024
+>0	string	\x02\x0d	1024b
+>>2	use	x1024
+>0	string	\x02\x03	1024b
+>>2	use	x1024
+>0	string	\x02\x05	1024b
+>>2	use	x1024
+>0	string	\x02\x15	1024b
+>>2	use	x1024
+>0	string	\x02\x00	1024b
+>>2	use	x1024
+>0	string	\x02\x10	1024b
+>>2	use	x1024
+>0	string	\x02\x04	1024b
+>>2	use	x1024
+>0	string	\x02\x06	1024b
+>>2	use	x1024
+>0	string	\x02\x16	1024b
+>>2	use	x1024
+>0	string	\x03\x98	2048b
+>>2	use	x2048
+>0	string	\x03\xab	2048b
+>>2	use	x2048
+>0	string	\x03\xbb	2048b
+>>2	use	x2048
+>0	string	\x03\xbd	2048b
+>>2	use	x2048
+>0	string	\x03\xcd	2048b
+>>2	use	x2048
+>0	string	\x03\xb3	2048b
+>>2	use	x2048
+>0	string	\x03\xc3	2048b
+>>2	use	x2048
+>0	string	\x03\xc5	2048b
+>>2	use	x2048
+>0	string	\x03\xd5	2048b
+>>2	use	x2048
+>0	string	\x03\xae	2048b
+>>2	use	x2048
+>0	string	\x03\xbe	2048b
+>>2	use	x2048
+>0	string	\x03\xc0	2048b
+>>2	use	x2048
+>0	string	\x03\xd0	2048b
+>>2	use	x2048
+>0	string	\x03\xb4	2048b
+>>2	use	x2048
+>0	string	\x03\xc4	2048b
+>>2	use	x2048
+>0	string	\x03\xc6	2048b
+>>2	use	x2048
+>0	string	\x03\xd6	2048b
+>>2	use	x2048
+>0	string	\x05X		3072b
+>>2	use	x3072
+>0	string	\x05k		3072b
+>>2	use	x3072
+>0	string	\x05{		3072b
+>>2	use	x3072
+>0	string	\x05}		3072b
+>>2	use	x3072
+>0	string	\x05\x8d	3072b
+>>2	use	x3072
+>0	string	\x05s		3072b
+>>2	use	x3072
+>0	string	\x05\x83	3072b
+>>2	use	x3072
+>0	string	\x05\x85	3072b
+>>2	use	x3072
+>0	string	\x05\x95	3072b
+>>2	use	x3072
+>0	string	\x05n		3072b
+>>2	use	x3072
+>0	string	\x05\x7e	3072b
+>>2	use	x3072
+>0	string	\x05\x80	3072b
+>>2	use	x3072
+>0	string	\x05\x90	3072b
+>>2	use	x3072
+>0	string	\x05t		3072b
+>>2	use	x3072
+>0	string	\x05\x84	3072b
+>>2	use	x3072
+>0	string	\x05\x86	3072b
+>>2	use	x3072
+>0	string	\x05\x96	3072b
+>>2	use	x3072
+>0	string	\x07[		4096b
+>>2	use	x4096
+>0	string	\x07\x18	4096b
+>>2	use	x4096
+>0	string	\x07+		4096b
+>>2	use	x4096
+>0	string	\x07;		4096b
+>>2	use	x4096
+>0	string	\x07=		4096b
+>>2	use	x4096
+>0	string	\x07M		4096b
+>>2	use	x4096
+>0	string	\x073		4096b
+>>2	use	x4096
+>0	string	\x07C		4096b
+>>2	use	x4096
+>0	string	\x07E		4096b
+>>2	use	x4096
+>0	string	\x07U		4096b
+>>2	use	x4096
+>0	string	\x07.		4096b
+>>2	use	x4096
+>0	string	\x07>		4096b
+>>2	use	x4096
+>0	string	\x07@		4096b
+>>2	use	x4096
+>0	string	\x07P		4096b
+>>2	use	x4096
+>0	string	\x074		4096b
+>>2	use	x4096
+>0	string	\x07D		4096b
+>>2	use	x4096
+>0	string	\x07F		4096b
+>>2	use	x4096
+>0	string	\x07V		4096b
+>>2	use	x4096
+>0	string	\x0e[		8192b
+>>2	use	x8192
+>0	string	\x0e\x18	8192b
+>>2	use	x8192
+>0	string	\x0e+		8192b
+>>2	use	x8192
+>0	string	\x0e;		8192b
+>>2	use	x8192
+>0	string	\x0e=		8192b
+>>2	use	x8192
+>0	string	\x0eM		8192b
+>>2	use	x8192
+>0	string	\x0e3		8192b
+>>2	use	x8192
+>0	string	\x0eC		8192b
+>>2	use	x8192
+>0	string	\x0eE		8192b
+>>2	use	x8192
+>0	string	\x0eU		8192b
+>>2	use	x8192
+>0	string	\x0e.		8192b
+>>2	use	x8192
+>0	string	\x0e>		8192b
+>>2	use	x8192
+>0	string	\x0e@		8192b
+>>2	use	x8192
+>0	string	\x0eP		8192b
+>>2	use	x8192
+>0	string	\x0e4		8192b
+>>2	use	x8192
+>0	string	\x0eD		8192b
+>>2	use	x8192
+>0	string	\x0eF		8192b
+>>2	use	x8192
+>0	string	\x0eV		8192b
+>>2	use	x8192
+
+# PGP RSA (e=65537) secret (sub-)key header
+
+0	byte	0x95			PGP	Secret Key -
+>1	use	pgpkey
+0	byte	0x97			PGP	Secret Sub-key -
+>1	use	pgpkey
+0	byte	0x9d			PGP	Secret Sub-key -
+>1	use	pgpkey
diff --git a/contrib/file/magic/Magdir/python b/contrib/file/magic/Magdir/python
index d954ee6..36cdfd8 100644
--- a/contrib/file/magic/Magdir/python
+++ b/contrib/file/magic/Magdir/python
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: python,v 1.25 2014/05/06 16:08:32 christos Exp $
+# $File: python,v 1.26 2014/08/04 05:58:40 christos Exp $
 # python:  file(1) magic for python
 #
 # Outlook puts """ too for urgent messages
@@ -23,6 +23,7 @@
 0	belong		0x4f0c0d0a	python 3.1 byte-compiled
 0	belong		0x6c0c0d0a	python 3.2 byte-compiled
 0	belong		0x9e0c0d0a	python 3.3 byte-compiled
+0	belong		0xee0c0d0a	python 3.4 byte-compiled
 
 0	search/1/w	#!\ /usr/bin/python	Python script text executable
 !:mime text/x-python
diff --git a/contrib/file/magic/Magdir/qt b/contrib/file/magic/Magdir/qt
new file mode 100644
index 0000000..72e6ff3
--- /dev/null
+++ b/contrib/file/magic/Magdir/qt
@@ -0,0 +1,19 @@
+
+#------------------------------------------------------------------------------
+# $File: qt,v 1.2 2014/12/16 19:49:29 christos Exp $
+# qt:  file(1) magic for Qt
+
+# http://doc.qt.io/qt-5/resources.html
+0	string		\<!DOCTYPE\040RCC\>	Qt Resource Collection file
+
+# https://qt.gitorious.org/qt/qtbase/source/\
+# 5367fa356233da4c0f28172a8f817791525f5457:\
+# src/tools/rcc/rcc.cpp#L840
+0	string		qres\0\0		Qt Binary Resource file
+0	search/1024	The\040Resource\040Compiler\040for\040Qt	Qt C-code resource file
+
+# https://qt.gitorious.org/qt/qtbase/source/\
+# 5367fa356233da4c0f28172a8f817791525f5457:\
+# src/corelib/kernel/qtranslator.cpp#L62
+0	string		\x3c\xb8\x64\x18\xca\xef\x9c\x95
+>8	string		\xcd\x21\x1c\xbf\x60\xa1\xbd\xdd	Qt Translation file
diff --git a/contrib/file/magic/Magdir/riff b/contrib/file/magic/Magdir/riff
index d63ba4b..e551292 100644
--- a/contrib/file/magic/Magdir/riff
+++ b/contrib/file/magic/Magdir/riff
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: riff,v 1.27 2014/04/30 21:41:02 christos Exp $
+# $File: riff,v 1.30 2014/09/23 17:02:12 christos Exp $
 # riff:  file(1) magic for RIFF format
 # See
 #
@@ -45,6 +45,26 @@
 >>&(4.l+4)  use riff-walk
 >0  string  fact
 >>&(4.l+4)  use riff-walk
+>0  string  VP8
+>>11		byte		0x9d
+>>>12		byte		0x01
+>>>>13		byte		0x2a	\b, VP8 encoding
+>>>>>14		leshort&0x3fff	x	\b, %d
+>>>>>16		leshort&0x3fff	x	\bx%d, Scaling:
+>>>>>14		leshort&0xc000	0x0000	\b [none]
+>>>>>14		leshort&0xc000	0x1000	\b [5/4]
+>>>>>14		leshort&0xc000	0x2000	\b [5/3]
+>>>>>14		leshort&0xc000	0x3000	\b [2]
+>>>>>14		leshort&0xc000	0x0000	\bx[none]
+>>>>>14		leshort&0xc000	0x1000	\bx[5/4]
+>>>>>14		leshort&0xc000	0x2000	\bx[5/3]
+>>>>>14		leshort&0xc000	0x3000	\bx[2]
+>>>>>15		byte&0x80	=0x00	\b, YUV color
+>>>>>15		byte&0x80	=0x80	\b, bad color specification
+>>>>>15		byte&0x40	=0x40	\b, no clamping required
+>>>>>15		byte&0x40	=0x00	\b, decoders should clamp
+#>0  string  x		we got %s
+#>>&(4.l+4)  use riff-walk
 
 # AVI section extended by Patrik Radman <patrik+file-magic@iki.fi>
 #
@@ -209,6 +229,8 @@
 >8	string		4XMV		\b, 4X Movie file 
 # AMV-type AVI file: http://wiki.multimedia.cx/index.php?title=AMV
 >8	string		AMV\040		\b, AMV 
+>8      string          WEBP            \b, Web/P image
+>>12	use		riff-walk
 
 #
 # XXX - some of the below may only appear in little-endian form.
@@ -284,4 +306,3 @@
 >>&6	leshort		=2		\b, stereo
 >>&6	leshort		>2		\b, %d channels
 >>&8	lelong		>0		%d Hz
-
diff --git a/contrib/file/magic/Magdir/sequent b/contrib/file/magic/Magdir/sequent
index 9ef2aa9..5137c0e 100644
--- a/contrib/file/magic/Magdir/sequent
+++ b/contrib/file/magic/Magdir/sequent
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: sequent,v 1.11 2014/06/02 19:27:54 christos Exp $
+# $File: sequent,v 1.12 2014/08/16 16:07:12 christos Exp $
 # sequent:  file(1) magic for Sequent machines
 #
 # Sequent information updated by Don Dwiggins <atsun!dwiggins>.
@@ -30,9 +30,6 @@
 0	leshort	0x32eb		SYMMETRY i386 executable (invalid @ 0)
 >16	lelong	>0		not stripped
 >124	lelong	>0		version %d
-0	leshort	0x42eb		SYMMETRY i386 standalone executable
->16	lelong	>0		not stripped
->124	lelong	>0		version %d
 # http://en.wikipedia.org/wiki/Sequent_Computer_Systems
 # below test line conflicts with MS-DOS 2.11 floppies and Acronis loader
 #0	leshort	0x42eb		SYMMETRY i386 standalone executable
diff --git a/contrib/file/magic/Magdir/sereal b/contrib/file/magic/Magdir/sereal
index e40f9e2..7fa4503 100644
--- a/contrib/file/magic/Magdir/sereal
+++ b/contrib/file/magic/Magdir/sereal
@@ -1,5 +1,6 @@
+
 #------------------------------------------------------------------------------
-# $File: sereal,v 1.1 2014/05/14 23:04:59 christos Exp $
+# $File: sereal,v 1.2 2014/11/11 20:10:49 christos Exp $
 # sereal: file(1) magic the Sereal binary serialization format
 #
 # From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
@@ -15,10 +16,10 @@
 #
 # See https://github.com/Sereal/Sereal/commit/35372ae01d in the
 # Sereal.git repository for test Sereal data.
-0      string             \=srl   Sereal data
+0	string		\=srl   Sereal data
 !:mime application/sereal
->4     byte&0x0F          x       (version %d,
->4     byte&0xF0          0x00    uncompressed)
->4     byte&0xF0          0x10    compressed with non-incremental Snappy)
->4     byte&0xF0          0x20    compressed with incremental Snappy)
->4     byte&0xF0          >0x20   unknown subformat, flag: %d>>4)
+>4	byte&0x0F	x	(version %d,
+>4	byte&0xF0	0x00	uncompressed)
+>4	byte&0xF0	0x10	compressed with non-incremental Snappy)
+>4	byte&0xF0	0x20	compressed with incremental Snappy)
+>4	byte&0xF0	>0x20	unknown subformat, flag: %d>>4)
diff --git a/contrib/file/magic/Magdir/ssh b/contrib/file/magic/Magdir/ssh
index d867af0..ca64564 100644
--- a/contrib/file/magic/Magdir/ssh
+++ b/contrib/file/magic/Magdir/ssh
@@ -3,9 +3,11 @@
 
 0	string	SSH\ PRIVATE\ KEY	OpenSSH RSA1 private key,
 >28	string	>\0			version %s
+0	string	-----BEGIN\ OPENSSH\ PRIVATE\ KEY-----	OpenSSH private key
 
 0	string	ssh-dss\ 		OpenSSH DSA public key
 0	string	ssh-rsa\ 		OpenSSH RSA public key
 0	string	ecdsa-sha2-nistp256	OpenSSH ECDSA public key
 0	string	ecdsa-sha2-nistp384	OpenSSH ECDSA public key
 0	string	ecdsa-sha2-nistp521	OpenSSH ECDSA public key
+0	string	ssh-ed25519		OpenSSH ED25519 public key
diff --git a/contrib/file/magic/Magdir/vms b/contrib/file/magic/Magdir/vms
index be716b3..4939303 100644
--- a/contrib/file/magic/Magdir/vms
+++ b/contrib/file/magic/Magdir/vms
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: vms,v 1.7 2013/03/09 22:36:00 christos Exp $
+# $File: vms,v 1.9 2014/08/17 13:47:59 christos Exp $
 # vms:  file(1) magic for VMS executables (experimental)
 #
 # VMS .exe formats, both VAX and AXP (Greg Roelofs, newt@uchicago.edu)
diff --git a/contrib/file/magic/Magdir/vorbis b/contrib/file/magic/Magdir/vorbis
index 4406512..d337398 100644
--- a/contrib/file/magic/Magdir/vorbis
+++ b/contrib/file/magic/Magdir/vorbis
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: vorbis,v 1.18 2014/04/30 21:41:02 christos Exp $
+# $File: vorbis,v 1.20 2014/09/23 16:35:08 christos Exp $
 # vorbis:  file(1) magic for Ogg/Vorbis files
 #
 # From Felix von Leitner <leitner@fefe.de>
@@ -23,7 +23,6 @@
 # --- Ogg Framing ---
 #0		search/1000	OggS		Ogg data
 0		string	OggS		Ogg data
-!:mime		application/ogg
 >4		byte		!0		UNKNOWN REVISION %u
 ##>4		byte		0		revision 0
 >4		byte		0
@@ -31,9 +30,12 @@
 # non-Vorbis content: FLAC (Free Lossless Audio Codec, http://flac.sourceforge.net)
 >>28		string		\x7fFLAC	\b, FLAC audio
 # non-Vorbis content: Theora
+!:mime		audio/ogg
 >>28		string		\x80theora	\b, Theora video
+!:mime		video/ogg
 # non-Vorbis content: Kate
->>28		string		\x80kate\0\0\0\0	\b, Kate
+>>28		string		\x80kate\0\0\0\0	\b, Kate (Karaoke and Text)
+!:mime		application/ogg
 >>>37		ubyte		x		v%u
 >>>38		ubyte		x		\b.%u,
 >>>40		byte		0		utf8 encoding,
@@ -44,18 +46,22 @@
 >>>76		string		\0		no category set
 # non-Vorbis content: Skeleton
 >>28		string		fishead\0	\b, Skeleton
+!:mime		video/ogg
 >>>36		short		x		v%u
 >>>40		short		x		\b.%u
 # non-Vorbis content: Speex
 >>28		string		Speex\ \ \ 	\b, Speex audio
+!:mime		audio/ogg
 # non-Vorbis content: OGM
 >>28		string		\x01video\0\0\0	\b, OGM video
+!:mime		video/ogg
 >>>37		string/c	div3		(DivX 3)
 >>>37		string/c	divx		(DivX 4)
 >>>37		string/c	dx50		(DivX 5)
 >>>37		string/c	xvid		(XviD)
 # --- First vorbis packet - general header ---
 >>28		string		\x01vorbis	\b, Vorbis audio,
+!:mime		audio/ogg
 >>>35		lelong		!0		UNKNOWN VERSION %u,
 ##>>>35		lelong		0		version 0,
 >>>35		lelong		0
diff --git a/contrib/file/magic/Magdir/windows b/contrib/file/magic/Magdir/windows
index 1879f29..3f7bded 100644
--- a/contrib/file/magic/Magdir/windows
+++ b/contrib/file/magic/Magdir/windows
@@ -1,6 +1,6 @@
 
 #------------------------------------------------------------------------------
-# $File: windows,v 1.8 2014/04/30 21:41:02 christos Exp $
+# $File: windows,v 1.10 2014/09/24 19:52:46 christos Exp $
 # windows:  file(1) magic for Microsoft Windows
 #
 # This file is mainly reserved for files where programs
@@ -159,8 +159,9 @@
 >&0	string		Version\ 5.00\r\n\r\n	Windows Registry text (Win2K or above)
 
 # Windows *.INF *.INI files updated by Joerg Jenderek at Apr 2013
-# emtpy ,comment , section , unicode line
-0	regex/s		\\`(\r\n|;|[[]|\xFF\xFE)			
+# empty ,comment , section
+# PR/383: remove unicode BOM because it is not portable across regex impls
+0	regex/s		\\`(\\r\\n|;|[[])
 # left bracket in section line
 >&0	search/8192	[						
 # http://en.wikipedia.org/wiki/Autorun.inf
diff --git a/contrib/file/magic/Makefile.am b/contrib/file/magic/Makefile.am
index 27e1181..7bc327d 100644
--- a/contrib/file/magic/Makefile.am
+++ b/contrib/file/magic/Makefile.am
@@ -1,5 +1,5 @@
 #
-# $File: Makefile.am,v 1.98 2014/06/03 18:22:34 christos Exp $
+# $File: Makefile.am,v 1.101 2014/12/12 16:48:39 christos Exp $
 #
 MAGIC_FRAGMENT_BASE = Magdir
 MAGIC_DIR = $(top_srcdir)/magic
@@ -117,6 +117,7 @@ $(MAGIC_FRAGMENT_DIR)/jpeg \
 $(MAGIC_FRAGMENT_DIR)/karma \
 $(MAGIC_FRAGMENT_DIR)/kde \
 $(MAGIC_FRAGMENT_DIR)/keepass \
+$(MAGIC_FRAGMENT_DIR)/kerberos \
 $(MAGIC_FRAGMENT_DIR)/kml \
 $(MAGIC_FRAGMENT_DIR)/lecter \
 $(MAGIC_FRAGMENT_DIR)/lex \
@@ -142,6 +143,7 @@ $(MAGIC_FRAGMENT_DIR)/matroska \
 $(MAGIC_FRAGMENT_DIR)/mcrypt \
 $(MAGIC_FRAGMENT_DIR)/mercurial \
 $(MAGIC_FRAGMENT_DIR)/metastore \
+$(MAGIC_FRAGMENT_DIR)/meteorological \
 $(MAGIC_FRAGMENT_DIR)/mime \
 $(MAGIC_FRAGMENT_DIR)/mips \
 $(MAGIC_FRAGMENT_DIR)/mirage \
@@ -196,9 +198,9 @@ $(MAGIC_FRAGMENT_DIR)/pulsar \
 $(MAGIC_FRAGMENT_DIR)/pwsafe \
 $(MAGIC_FRAGMENT_DIR)/pyramid \
 $(MAGIC_FRAGMENT_DIR)/python \
+$(MAGIC_FRAGMENT_DIR)/qt \
 $(MAGIC_FRAGMENT_DIR)/revision \
 $(MAGIC_FRAGMENT_DIR)/riff \
-$(MAGIC_FRAGMENT_DIR)/rinex \
 $(MAGIC_FRAGMENT_DIR)/rpm \
 $(MAGIC_FRAGMENT_DIR)/rtf \
 $(MAGIC_FRAGMENT_DIR)/ruby \
diff --git a/contrib/file/magic/Makefile.in b/contrib/file/magic/Makefile.in
index e86cdbf..43f42c6 100644
--- a/contrib/file/magic/Makefile.in
+++ b/contrib/file/magic/Makefile.in
@@ -262,7 +262,7 @@ top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
 
 #
-# $File: Makefile.am,v 1.98 2014/06/03 18:22:34 christos Exp $
+# $File: Makefile.am,v 1.101 2014/12/12 16:48:39 christos Exp $
 #
 MAGIC_FRAGMENT_BASE = Magdir
 MAGIC_DIR = $(top_srcdir)/magic
@@ -378,6 +378,7 @@ $(MAGIC_FRAGMENT_DIR)/jpeg \
 $(MAGIC_FRAGMENT_DIR)/karma \
 $(MAGIC_FRAGMENT_DIR)/kde \
 $(MAGIC_FRAGMENT_DIR)/keepass \
+$(MAGIC_FRAGMENT_DIR)/kerberos \
 $(MAGIC_FRAGMENT_DIR)/kml \
 $(MAGIC_FRAGMENT_DIR)/lecter \
 $(MAGIC_FRAGMENT_DIR)/lex \
@@ -403,6 +404,7 @@ $(MAGIC_FRAGMENT_DIR)/matroska \
 $(MAGIC_FRAGMENT_DIR)/mcrypt \
 $(MAGIC_FRAGMENT_DIR)/mercurial \
 $(MAGIC_FRAGMENT_DIR)/metastore \
+$(MAGIC_FRAGMENT_DIR)/meteorological \
 $(MAGIC_FRAGMENT_DIR)/mime \
 $(MAGIC_FRAGMENT_DIR)/mips \
 $(MAGIC_FRAGMENT_DIR)/mirage \
@@ -457,9 +459,9 @@ $(MAGIC_FRAGMENT_DIR)/pulsar \
 $(MAGIC_FRAGMENT_DIR)/pwsafe \
 $(MAGIC_FRAGMENT_DIR)/pyramid \
 $(MAGIC_FRAGMENT_DIR)/python \
+$(MAGIC_FRAGMENT_DIR)/qt \
 $(MAGIC_FRAGMENT_DIR)/revision \
 $(MAGIC_FRAGMENT_DIR)/riff \
-$(MAGIC_FRAGMENT_DIR)/rinex \
 $(MAGIC_FRAGMENT_DIR)/rpm \
 $(MAGIC_FRAGMENT_DIR)/rtf \
 $(MAGIC_FRAGMENT_DIR)/ruby \
diff --git a/contrib/file/src/Makefile.in b/contrib/file/src/Makefile.in
index 970ff0b..41cb9f5 100644
--- a/contrib/file/src/Makefile.in
+++ b/contrib/file/src/Makefile.in
@@ -81,9 +81,9 @@ build_triplet = @build@
 host_triplet = @host@
 bin_PROGRAMS = file$(EXEEXT)
 subdir = src
-DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am pread.c \
-	ctime_r.c getline.c vasprintf.c asprintf.c asctime_r.c \
-	fmtcheck.c strlcpy.c getopt_long.c strcasestr.c strlcat.c \
+DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am ctime_r.c \
+	vasprintf.c asctime_r.c asprintf.c strcasestr.c pread.c \
+	getline.c strlcpy.c strlcat.c fmtcheck.c getopt_long.c \
 	$(top_srcdir)/depcomp $(include_HEADERS)
 ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
 am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \
diff --git a/contrib/file/src/apprentice.c b/contrib/file/src/apprentice.c
index 71e9af6..47b4c87 100644
--- a/contrib/file/src/apprentice.c
+++ b/contrib/file/src/apprentice.c
@@ -32,7 +32,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: apprentice.c,v 1.211 2014/06/03 19:01:34 christos Exp $")
+FILE_RCSID("@(#)$File: apprentice.c,v 1.229 2015/01/01 17:07:34 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -86,6 +86,10 @@ FILE_RCSID("@(#)$File: apprentice.c,v 1.211 2014/06/03 19:01:34 christos Exp $")
 #define ALLOC_CHUNK	(size_t)10
 #define ALLOC_INCR	(size_t)200
 
+#define MAP_TYPE_MMAP	0
+#define MAP_TYPE_MALLOC	1
+#define MAP_TYPE_USER	2
+
 struct magic_entry {
 	struct magic *mp;	
 	uint32_t cont_count;
@@ -101,6 +105,7 @@ struct magic_entry_set {
 struct magic_map {
 	void *p;
 	size_t len;
+	int type;
 	struct magic *magic[MAGIC_SETS];
 	uint32_t nmagic[MAGIC_SETS];
 };
@@ -131,7 +136,10 @@ private uint16_t swap2(uint16_t);
 private uint32_t swap4(uint32_t);
 private uint64_t swap8(uint64_t);
 private char *mkdbname(struct magic_set *, const char *, int);
+private struct magic_map *apprentice_buf(struct magic_set *, struct magic *,
+    size_t);
 private struct magic_map *apprentice_map(struct magic_set *, const char *);
+private int check_buffer(struct magic_set *, struct magic_map *, const char *);
 private void apprentice_unmap(struct magic_map *);
 private int apprentice_compile(struct magic_set *, struct magic_map *,
     const char *);
@@ -396,10 +404,11 @@ add_mlist(struct mlist *mlp, struct magic_map *map, size_t idx)
 {
 	struct mlist *ml;
 
+	mlp->map = idx == 0 ? map : NULL;
 	if ((ml = CAST(struct mlist *, malloc(sizeof(*ml)))) == NULL)
 		return -1;
 
-	ml->map = idx == 0 ? map : NULL;
+	ml->map = NULL;
 	ml->magic = map->magic[idx];
 	ml->nmagic = map->nmagic[idx];
 
@@ -416,13 +425,11 @@ add_mlist(struct mlist *mlp, struct magic_map *map, size_t idx)
 private int
 apprentice_1(struct magic_set *ms, const char *fn, int action)
 {
-#ifndef COMPILE_ONLY
-	struct mlist *ml;
-#endif /* COMPILE_ONLY */
 	struct magic_map *map;
 #ifndef COMPILE_ONLY
+	struct mlist *ml;
 	size_t i;
-#endif /* COMPILE_ONLY */
+#endif
 
 	if (magicsize != FILE_MAGICSIZE) {
 		file_error(ms, 0, "magic element size %lu != %lu",
@@ -451,22 +458,29 @@ apprentice_1(struct magic_set *ms, const char *fn, int action)
 	for (i = 0; i < MAGIC_SETS; i++) {
 		if (add_mlist(ms->mlist[i], map, i) == -1) {
 			file_oomem(ms, sizeof(*ml));
-			apprentice_unmap(map);
-			return -1;
+			goto fail;
 		}
 	}
 
 	if (action == FILE_LIST) {
 		for (i = 0; i < MAGIC_SETS; i++) {
-			printf("Set %zu:\nBinary patterns:\n", i);
+			printf("Set %" SIZE_T_FORMAT "u:\nBinary patterns:\n",
+			    i);
 			apprentice_list(ms->mlist[i], BINTEST);
 			printf("Text patterns:\n");
 			apprentice_list(ms->mlist[i], TEXTTEST);
 		}
 	}
-#endif /* COMPILE_ONLY */
-	
 	return 0;
+fail:
+	for (i = 0; i < MAGIC_SETS; i++) {
+		mlist_free(ms->mlist[i]);
+		ms->mlist[i] = NULL;
+	}
+	return -1;
+#else
+	return 0;
+#endif /* COMPILE_ONLY */
 }
 
 protected void
@@ -510,6 +524,11 @@ file_ms_alloc(int flags)
 		ms->mlist[i] = NULL;
 	ms->file = "unknown";
 	ms->line = 0;
+	ms->indir_max = FILE_INDIR_MAX;
+	ms->name_max = FILE_NAME_MAX;
+	ms->elf_shnum_max = FILE_ELF_SHNUM_MAX;
+	ms->elf_phnum_max = FILE_ELF_PHNUM_MAX;
+	ms->elf_notes_max = FILE_ELF_NOTES_MAX;
 	return ms;
 free:
 	free(ms);
@@ -521,17 +540,21 @@ apprentice_unmap(struct magic_map *map)
 {
 	if (map == NULL)
 		return;
-	if (map->p != NULL) {
+
+	switch (map->type) {
 #ifdef QUICK
-		if (map->len)
+	case MAP_TYPE_MMAP:
+		if (map->p)
 			(void)munmap(map->p, map->len);
-		else
+		break;
 #endif
+	case MAP_TYPE_MALLOC:
 		free(map->p);
-	} else {
-		uint32_t j;
-		for (j = 0; j < MAGIC_SETS; j++)
-			free(map->magic[j]);
+		break;
+	case MAP_TYPE_USER:
+		break;
+	default:
+		abort();
 	}
 	free(map);
 }
@@ -550,20 +573,69 @@ mlist_alloc(void)
 private void
 mlist_free(struct mlist *mlist)
 {
-	struct mlist *ml;
+	struct mlist *ml, *next;
 
 	if (mlist == NULL)
 		return;
 
-	for (ml = mlist->next; ml != mlist;) {
-		struct mlist *next = ml->next;
+	ml = mlist->next;
+	for (ml = mlist->next; (next = ml->next) != NULL; ml = next) {
 		if (ml->map)
 			apprentice_unmap(ml->map);
 		free(ml);
-		ml = next;
+		if (ml == mlist)
+			break;
+	}
+}
+
+#ifndef COMPILE_ONLY
+/* void **bufs: an array of compiled magic files */
+protected int
+buffer_apprentice(struct magic_set *ms, struct magic **bufs,
+    size_t *sizes, size_t nbufs)
+{
+	size_t i, j;
+	struct mlist *ml;
+	struct magic_map *map;
+
+	if (nbufs == 0)
+		return -1;
+
+	if (ms->mlist[0] != NULL)
+		file_reset(ms);
+
+	init_file_tables();
+
+	for (i = 0; i < MAGIC_SETS; i++) {
+		mlist_free(ms->mlist[i]);
+		if ((ms->mlist[i] = mlist_alloc()) == NULL) {
+			file_oomem(ms, sizeof(*ms->mlist[i]));
+			goto fail;
+		}
+	}
+
+	for (i = 0; i < nbufs; i++) {
+		map = apprentice_buf(ms, bufs[i], sizes[i]);
+		if (map == NULL)
+			goto fail;
+
+		for (j = 0; j < MAGIC_SETS; j++) {
+			if (add_mlist(ms->mlist[j], map, j) == -1) {
+				file_oomem(ms, sizeof(*ml));
+				goto fail;
+			}
+		}
+	}
+
+	return 0;
+fail:
+	for (i = 0; i < MAGIC_SETS; i++) {
+		mlist_free(ms->mlist[i]);
+		ms->mlist[i] = NULL;
 	}
-	free(ml);
+	return -1;
 }
+#endif
 
 /* const char *fn: list of magic files and directories */
 protected int
@@ -590,11 +662,9 @@ file_apprentice(struct magic_set *ms, const char *fn, int action)
 		mlist_free(ms->mlist[i]);
 		if ((ms->mlist[i] = mlist_alloc()) == NULL) {
 			file_oomem(ms, sizeof(*ms->mlist[i]));
-			if (i != 0) {
-				--i;
-				do
-					mlist_free(ms->mlist[i]);
-				while (i != 0);
+			while (i-- > 0) {
+				mlist_free(ms->mlist[i]);
+				ms->mlist[i] = NULL;
 			}
 			free(mfn);
 			return -1;
@@ -1317,7 +1387,7 @@ file_signextend(struct magic_set *ms, struct magic *m, uint64_t v)
 		 * the sign extension must have happened.
 		 */
 		case FILE_BYTE:
-			v = (char) v;
+			v = (signed char) v;
 			break;
 		case FILE_SHORT:
 		case FILE_BESHORT:
@@ -1535,6 +1605,145 @@ check_cond(struct magic_set *ms, int cond, uint32_t cont_level)
 }
 #endif /* ENABLE_CONDITIONALS */
 
+private int
+parse_indirect_modifier(struct magic_set *ms, struct magic *m, const char **lp)
+{
+	const char *l = *lp;
+
+	while (!isspace((unsigned char)*++l))
+		switch (*l) {
+		case CHAR_INDIRECT_RELATIVE:
+			m->str_flags |= INDIRECT_RELATIVE;
+			break;
+		default:
+			if (ms->flags & MAGIC_CHECK)
+				file_magwarn(ms, "indirect modifier `%c' "
+					"invalid", *l);
+			*lp = l;
+			return -1;
+		}
+	*lp = l;
+	return 0;
+}
+
+private void
+parse_op_modifier(struct magic_set *ms, struct magic *m, const char **lp,
+    int op)
+{
+	const char *l = *lp;
+	char *t;
+	uint64_t val;
+
+	++l;
+	m->mask_op |= op;
+	val = (uint64_t)strtoull(l, &t, 0);
+	l = t;
+	m->num_mask = file_signextend(ms, m, val);
+	eatsize(&l);
+	*lp = l;
+}
+
+private int
+parse_string_modifier(struct magic_set *ms, struct magic *m, const char **lp)
+{
+	const char *l = *lp;
+	char *t;
+	int have_range = 0;
+
+	while (!isspace((unsigned char)*++l)) {
+		switch (*l) {
+		case '0':  case '1':  case '2':
+		case '3':  case '4':  case '5':
+		case '6':  case '7':  case '8':
+		case '9':
+			if (have_range && (ms->flags & MAGIC_CHECK))
+				file_magwarn(ms, "multiple ranges");
+			have_range = 1;
+			m->str_range = CAST(uint32_t, strtoul(l, &t, 0));
+			if (m->str_range == 0)
+				file_magwarn(ms, "zero range");
+			l = t - 1;
+			break;
+		case CHAR_COMPACT_WHITESPACE:
+			m->str_flags |= STRING_COMPACT_WHITESPACE;
+			break;
+		case CHAR_COMPACT_OPTIONAL_WHITESPACE:
+			m->str_flags |= STRING_COMPACT_OPTIONAL_WHITESPACE;
+			break;
+		case CHAR_IGNORE_LOWERCASE:
+			m->str_flags |= STRING_IGNORE_LOWERCASE;
+			break;
+		case CHAR_IGNORE_UPPERCASE:
+			m->str_flags |= STRING_IGNORE_UPPERCASE;
+			break;
+		case CHAR_REGEX_OFFSET_START:
+			m->str_flags |= REGEX_OFFSET_START;
+			break;
+		case CHAR_BINTEST:
+			m->str_flags |= STRING_BINTEST;
+			break;
+		case CHAR_TEXTTEST:
+			m->str_flags |= STRING_TEXTTEST;
+			break;
+		case CHAR_TRIM:
+			m->str_flags |= STRING_TRIM;
+			break;
+		case CHAR_PSTRING_1_LE:
+#define SET_LENGTH(a) m->str_flags = (m->str_flags & ~PSTRING_LEN) | (a)
+			if (m->type != FILE_PSTRING)
+				goto bad;
+			SET_LENGTH(PSTRING_1_LE);
+			break;
+		case CHAR_PSTRING_2_BE:
+			if (m->type != FILE_PSTRING)
+				goto bad;
+			SET_LENGTH(PSTRING_2_BE);
+			break;
+		case CHAR_PSTRING_2_LE:
+			if (m->type != FILE_PSTRING)
+				goto bad;
+			SET_LENGTH(PSTRING_2_LE);
+			break;
+		case CHAR_PSTRING_4_BE:
+			if (m->type != FILE_PSTRING)
+				goto bad;
+			SET_LENGTH(PSTRING_4_BE);
+			break;
+		case CHAR_PSTRING_4_LE:
+			switch (m->type) {
+			case FILE_PSTRING:
+			case FILE_REGEX:
+				break;
+			default:
+				goto bad;
+			}
+			SET_LENGTH(PSTRING_4_LE);
+			break;
+		case CHAR_PSTRING_LENGTH_INCLUDES_ITSELF:
+			if (m->type != FILE_PSTRING)
+				goto bad;
+			m->str_flags |= PSTRING_LENGTH_INCLUDES_ITSELF;
+			break;
+		default:
+		bad:
+			if (ms->flags & MAGIC_CHECK)
+				file_magwarn(ms, "string modifier `%c' "
+					"invalid", *l);
+			goto out;
+		}
+		/* allow multiple '/' for readability */
+		if (l[1] == '/' && !isspace((unsigned char)l[2]))
+			l++;
+	}
+	if (string_modifier_check(ms, m) == -1)
+		goto out;
+	*lp = l;
+	return 0;
+out:
+	*lp = l;
+	return -1;
+}
+
 /*
  * parse one line from magic file, put into magic[index++] if valid
  */
@@ -1804,118 +2013,27 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line,
 	m->str_range = 0;
 	m->str_flags = m->type == FILE_PSTRING ? PSTRING_1_LE : 0;
 	if ((op = get_op(*l)) != -1) {
-		if (!IS_STRING(m->type)) {
-			uint64_t val;
-			++l;
-			m->mask_op |= op;
-			val = (uint64_t)strtoull(l, &t, 0);
-			l = t;
-			m->num_mask = file_signextend(ms, m, val);
-			eatsize(&l);
-		}
-		else if (op == FILE_OPDIVIDE) {
-			int have_range = 0;
-			while (!isspace((unsigned char)*++l)) {
-				switch (*l) {
-				case '0':  case '1':  case '2':
-				case '3':  case '4':  case '5':
-				case '6':  case '7':  case '8':
-				case '9':
-					if (have_range &&
-					    (ms->flags & MAGIC_CHECK))
-						file_magwarn(ms,
-						    "multiple ranges");
-					have_range = 1;
-					m->str_range = CAST(uint32_t,
-					    strtoul(l, &t, 0));
-					if (m->str_range == 0)
-						file_magwarn(ms,
-						    "zero range");
-					l = t - 1;
-					break;
-				case CHAR_COMPACT_WHITESPACE:
-					m->str_flags |=
-					    STRING_COMPACT_WHITESPACE;
-					break;
-				case CHAR_COMPACT_OPTIONAL_WHITESPACE:
-					m->str_flags |=
-					    STRING_COMPACT_OPTIONAL_WHITESPACE;
-					break;
-				case CHAR_IGNORE_LOWERCASE:
-					m->str_flags |= STRING_IGNORE_LOWERCASE;
-					break;
-				case CHAR_IGNORE_UPPERCASE:
-					m->str_flags |= STRING_IGNORE_UPPERCASE;
-					break;
-				case CHAR_REGEX_OFFSET_START:
-					m->str_flags |= REGEX_OFFSET_START;
-					break;
-				case CHAR_BINTEST:
-					m->str_flags |= STRING_BINTEST;
-					break;
-				case CHAR_TEXTTEST:
-					m->str_flags |= STRING_TEXTTEST;
-					break;
-				case CHAR_TRIM:
-					m->str_flags |= STRING_TRIM;
-					break;
-				case CHAR_PSTRING_1_LE:
-					if (m->type != FILE_PSTRING)
-						goto bad;
-					m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_1_LE;
-					break;
-				case CHAR_PSTRING_2_BE:
-					if (m->type != FILE_PSTRING)
-						goto bad;
-					m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_2_BE;
-					break;
-				case CHAR_PSTRING_2_LE:
-					if (m->type != FILE_PSTRING)
-						goto bad;
-					m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_2_LE;
-					break;
-				case CHAR_PSTRING_4_BE:
-					if (m->type != FILE_PSTRING)
-						goto bad;
-					m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_4_BE;
-					break;
-				case CHAR_PSTRING_4_LE:
-					switch (m->type) {
-					case FILE_PSTRING:
-					case FILE_REGEX:
-						break;
-					default:
-						goto bad;
-					}
-					m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_4_LE;
-					break;
-				case CHAR_PSTRING_LENGTH_INCLUDES_ITSELF:
-					if (m->type != FILE_PSTRING)
-						goto bad;
-					m->str_flags |= PSTRING_LENGTH_INCLUDES_ITSELF;
-					break;
-				default:
-				bad:
-					if (ms->flags & MAGIC_CHECK)
-						file_magwarn(ms,
-						    "string extension `%c' "
-						    "invalid", *l);
-					return -1;
-				}
-				/* allow multiple '/' for readability */
-				if (l[1] == '/' &&
-				    !isspace((unsigned char)l[2]))
-					l++;
+		if (IS_STRING(m->type)) {
+			int r;
+
+			if (op != FILE_OPDIVIDE) {
+				if (ms->flags & MAGIC_CHECK)
+					file_magwarn(ms,
+					    "invalid string/indirect op: "
+					    "`%c'", *t);
+				return -1;
 			}
-			if (string_modifier_check(ms, m) == -1)
+
+			if (m->type == FILE_INDIRECT)
+				r = parse_indirect_modifier(ms, m, &l);
+			else
+				r = parse_string_modifier(ms, m, &l);
+			if (r == -1)
 				return -1;
-		}
-		else {
-			if (ms->flags & MAGIC_CHECK)
-				file_magwarn(ms, "invalid string op: %c", *t);
-			return -1;
-		}
+		} else
+			parse_op_modifier(ms, m, &l, op);
 	}
+
 	/*
 	 * We used to set mask to all 1's here, instead let's just not do
 	 * anything if mask = 0 (unless you have a better idea)
@@ -2069,8 +2187,14 @@ out:
 }
 
 private int
+goodchar(unsigned char x, const char *extra)
+{
+	return (isascii(x) && isalnum(x)) || strchr(extra, x);
+}
+
+private int
 parse_extra(struct magic_set *ms, struct magic_entry *me, const char *line,
-    off_t off, size_t len, const char *name, int nt)
+    off_t off, size_t len, const char *name, const char *extra, int nt)
 {
 	size_t i;
 	const char *l = line;
@@ -2091,9 +2215,7 @@ parse_extra(struct magic_set *ms, struct magic_entry *me, const char *line,
 	}
 
 	EATAB;
-	for (i = 0; *l && ((isascii((unsigned char)*l) &&
-	    isalnum((unsigned char)*l)) || strchr("-+/.", *l)) &&
-	    i < len; buf[i++] = *l++)
+	for (i = 0; *l && i < len && goodchar(*l, extra); buf[i++] = *l++)
 		continue;
 
 	if (i == len && *l) {
@@ -2103,14 +2225,18 @@ parse_extra(struct magic_set *ms, struct magic_entry *me, const char *line,
 			file_magwarn(ms, "%s type `%s' truncated %"
 			    SIZE_T_FORMAT "u", name, line, i);
 	} else {
+		if (!isspace((unsigned char)*l) && !goodchar(*l, extra))
+			file_magwarn(ms, "%s type `%s' has bad char '%c'",
+			    name, line, *l);
 		if (nt)
 			buf[i] = '\0';
 	}
 
 	if (i > 0)
 		return 0;
-	else
-		return -1;
+
+	file_magerror(ms, "Bad magic entry '%s'", line);
+	return -1;
 }
 
 /*
@@ -2123,7 +2249,7 @@ parse_apple(struct magic_set *ms, struct magic_entry *me, const char *line)
 	struct magic *m = &me->mp[0];
 
 	return parse_extra(ms, me, line, offsetof(struct magic, apple),
-	    sizeof(m->apple), "APPLE", 0);
+	    sizeof(m->apple), "APPLE", "!+-./", 0);
 }
 
 /*
@@ -2136,7 +2262,7 @@ parse_mime(struct magic_set *ms, struct magic_entry *me, const char *line)
 	struct magic *m = &me->mp[0];
 
 	return parse_extra(ms, me, line, offsetof(struct magic, mimetype),
-	    sizeof(m->mimetype), "MIME", 1);
+	    sizeof(m->mimetype), "MIME", "+-/.", 1);
 }
 
 private int
@@ -2697,6 +2823,28 @@ eatsize(const char **p)
 }
 
 /*
+ * handle a buffer containing a compiled file.
+ */
+private struct magic_map *
+apprentice_buf(struct magic_set *ms, struct magic *buf, size_t len)
+{
+	struct magic_map *map;
+
+	if ((map = CAST(struct magic_map *, calloc(1, sizeof(*map)))) == NULL) {
+		file_oomem(ms, sizeof(*map));
+		return NULL;
+	}
+	map->len = len;
+	map->p = buf;
+	map->type = MAP_TYPE_USER;
+	if (check_buffer(ms, map, "buffer") != 0) {
+		apprentice_unmap(map);
+		return NULL;
+	}
+	return map;
+}
+
+/*
  * handle a compiled file.
  */
 
@@ -2705,12 +2853,8 @@ apprentice_map(struct magic_set *ms, const char *fn)
 {
 	int fd;
 	struct stat st;
-	uint32_t *ptr;
-	uint32_t version, entries, nentries;
-	int needsbyteswap;
 	char *dbname = NULL;
 	struct magic_map *map;
-	size_t i;
 
 	fd = -1;
 	if ((map = CAST(struct magic_map *, calloc(1, sizeof(*map)))) == NULL) {
@@ -2742,6 +2886,7 @@ apprentice_map(struct magic_set *ms, const char *fn)
 		file_error(ms, errno, "cannot map `%s'", dbname);
 		goto error;
 	}
+	map->type = MAP_TYPE_MMAP;
 #else
 	if ((map->p = CAST(void *, malloc(map->len))) == NULL) {
 		file_oomem(ms, map->len);
@@ -2751,16 +2896,39 @@ apprentice_map(struct magic_set *ms, const char *fn)
 		file_badread(ms);
 		goto error;
 	}
-	map->len = 0;
+	map->type = MAP_TYPE_MALLOC;
 #define RET	1
 #endif
 	(void)close(fd);
 	fd = -1;
+
+	if (check_buffer(ms, map, dbname) != 0)
+		goto error;
+
+	free(dbname);
+	return map;
+
+error:
+	if (fd != -1)
+		(void)close(fd);
+	apprentice_unmap(map);
+	free(dbname);
+	return NULL;
+}
+
+private int
+check_buffer(struct magic_set *ms, struct magic_map *map, const char *dbname)
+{
+	uint32_t *ptr;
+	uint32_t entries, nentries;
+	uint32_t version;
+	int i, needsbyteswap;
+
 	ptr = CAST(uint32_t *, map->p);
 	if (*ptr != MAGICNO) {
 		if (swap4(*ptr) != MAGICNO) {
 			file_error(ms, 0, "bad magic in `%s'", dbname);
-			goto error;
+			return -1;
 		}
 		needsbyteswap = 1;
 	} else
@@ -2773,15 +2941,14 @@ apprentice_map(struct magic_set *ms, const char *fn)
 		file_error(ms, 0, "File %s supports only version %d magic "
 		    "files. `%s' is version %d", VERSION,
 		    VERSIONNO, dbname, version);
-		goto error;
+		return -1;
 	}
-	entries = (uint32_t)(st.st_size / sizeof(struct magic));
-	if ((off_t)(entries * sizeof(struct magic)) != st.st_size) {
-		file_error(ms, 0, "Size of `%s' %" INT64_T_FORMAT "u is not "
+	entries = (uint32_t)(map->len / sizeof(struct magic));
+	if ((entries * sizeof(struct magic)) != map->len) {
+		file_error(ms, 0, "Size of `%s' %" SIZE_T_FORMAT "u is not "
 		    "a multiple of %" SIZE_T_FORMAT "u",
-		    dbname, (unsigned long long)st.st_size,
-		    sizeof(struct magic));
-		goto error;
+		    dbname, map->len, sizeof(struct magic));
+		return -1;
 	}
 	map->magic[0] = CAST(struct magic *, map->p) + 1;
 	nentries = 0;
@@ -2797,20 +2964,12 @@ apprentice_map(struct magic_set *ms, const char *fn)
 	if (entries != nentries + 1) {
 		file_error(ms, 0, "Inconsistent entries in `%s' %u != %u",
 		    dbname, entries, nentries + 1);
-		goto error;
+		return -1;
 	}
 	if (needsbyteswap)
 		for (i = 0; i < MAGIC_SETS; i++)
 			byteswap(map->magic[i], map->nmagic[i]);
-	free(dbname);
-	return map;
-
-error:
-	if (fd != -1)
-		(void)close(fd);
-	apprentice_unmap(map);
-	free(dbname);
-	return NULL;
+	return 0;
 }
 
 /*
diff --git a/contrib/file/src/ascmagic.c b/contrib/file/src/ascmagic.c
index ca26665..78a6dbb 100644
--- a/contrib/file/src/ascmagic.c
+++ b/contrib/file/src/ascmagic.c
@@ -35,7 +35,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: ascmagic.c,v 1.88 2014/02/12 23:20:53 christos Exp $")
+FILE_RCSID("@(#)$File: ascmagic.c,v 1.91 2014/11/28 02:46:39 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -147,7 +147,8 @@ file_ascmagic_with_encoding(struct magic_set *ms, const unsigned char *buf,
 		    == NULL)
 			goto done;
 		if ((rv = file_softmagic(ms, utf8_buf,
-		    (size_t)(utf8_end - utf8_buf), 0, TEXTTEST, text)) == 0)
+		    (size_t)(utf8_end - utf8_buf), 0, NULL,
+		    TEXTTEST, text)) == 0)
 			rv = -1;
 	}
 
diff --git a/contrib/file/src/cdf.c b/contrib/file/src/cdf.c
index 106fc7a..9e3cf9f 100644
--- a/contrib/file/src/cdf.c
+++ b/contrib/file/src/cdf.c
@@ -35,7 +35,7 @@
 #include "file.h"
 
 #ifndef lint
-FILE_RCSID("@(#)$File: cdf.c,v 1.63 2014/06/09 13:04:37 christos Exp $")
+FILE_RCSID("@(#)$File: cdf.c,v 1.69 2014/12/04 15:56:46 christos Exp $")
 #endif
 
 #include <assert.h>
@@ -73,6 +73,8 @@ static union {
 #define CDF_TOLE8(x)	((uint64_t)(NEED_SWAP ? _cdf_tole8(x) : (uint64_t)(x)))
 #define CDF_TOLE4(x)	((uint32_t)(NEED_SWAP ? _cdf_tole4(x) : (uint32_t)(x)))
 #define CDF_TOLE2(x)	((uint16_t)(NEED_SWAP ? _cdf_tole2(x) : (uint16_t)(x)))
+#define CDF_TOLE(x)	(sizeof(x) == 2 ? CDF_TOLE2(x) : (sizeof(x) == 4 ? \
+    CDF_TOLE4(x) : CDF_TOLE8(x)))
 #define CDF_GETUINT32(x, y)	cdf_getuint32(x, y)
 
 
@@ -461,6 +463,12 @@ cdf_count_chain(const cdf_sat_t *sat, cdf_secid_t sid, size_t size)
 	    / sizeof(maxsector));
 
 	DPRINTF(("Chain:"));
+	if (sid == CDF_SECID_END_OF_CHAIN) {
+		/* 0-length chain. */
+		DPRINTF((" empty\n"));
+		return 0;
+	}
+
 	for (j = i = 0; sid >= 0; i++, j++) {
 		DPRINTF((" %d", sid));
 		if (j >= CDF_LOOP_LIMIT) {
@@ -817,13 +825,18 @@ cdf_read_property_info(const cdf_stream_t *sst, const cdf_header_t *h,
 		goto out;
 	for (i = 0; i < sh.sh_properties; i++) {
 		size_t tail = (i << 1) + 1;
+		size_t ofs;
 		if (cdf_check_stream_offset(sst, h, p, tail * sizeof(uint32_t),
 		    __LINE__) == -1)
 			goto out;
-		size_t ofs = CDF_GETUINT32(p, tail);
+		ofs = CDF_GETUINT32(p, tail);
 		q = (const uint8_t *)(const void *)
 		    ((const char *)(const void *)p + ofs
 		    - 2 * sizeof(uint32_t));
+		if (q < p) {
+			DPRINTF(("Wrapped around %p < %p\n", q, p));
+			goto out;
+		}
 		if (q > e) {
 			DPRINTF(("Ran of the end %p > %p\n", q, e));
 			goto out;
@@ -985,6 +998,54 @@ cdf_unpack_summary_info(const cdf_stream_t *sst, const cdf_header_t *h,
 }
 
 
+#define extract_catalog_field(f, l) \
+    memcpy(&ce[i].f, b + (l), sizeof(ce[i].f)); \
+    ce[i].f = CDF_TOLE(ce[i].f)
+
+int
+cdf_unpack_catalog(const cdf_header_t *h, const cdf_stream_t *sst,
+    cdf_catalog_t **cat)
+{
+	size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ?
+	    CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h);
+	const char *b = CAST(const char *, sst->sst_tab);
+	const char *eb = b + ss * sst->sst_len;
+	size_t nr, i, k;
+	cdf_catalog_entry_t *ce;
+	uint16_t reclen;
+	const uint16_t *np;
+
+	for (nr = 0; b < eb; nr++) {
+		memcpy(&reclen, b, sizeof(reclen));
+		reclen = CDF_TOLE2(reclen);
+		if (reclen == 0)
+			break;
+		b += reclen;
+	}
+	*cat = CAST(cdf_catalog_t *,
+	    malloc(sizeof(cdf_catalog_t) + nr * sizeof(*ce)));
+	(*cat)->cat_num = nr;
+	ce = (*cat)->cat_e;
+	b = CAST(const char *, sst->sst_tab);
+	for (i = 0; i < nr; i++) {
+		extract_catalog_field(ce_namlen, 0);
+		extract_catalog_field(ce_num, 2);
+		extract_catalog_field(ce_timestamp, 6);
+		reclen = ce[i].ce_namlen;
+		ce[i].ce_namlen =
+		    sizeof(ce[i].ce_name) / sizeof(ce[i].ce_name[0]) - 1;
+		if (ce[i].ce_namlen > reclen - 14)
+			ce[i].ce_namlen = reclen - 14;
+		np = CAST(const uint16_t *, (b + 16));
+		for (k = 0; k < ce[i].ce_namlen; k++) {
+			ce[i].ce_name[k] = np[k];
+			CDF_TOLE2(ce[i].ce_name[k]);
+		}
+		ce[i].ce_name[ce[i].ce_namlen] = 0;
+		b += reclen;
+	}
+	return 0;
+}
 
 int
 cdf_print_classid(char *buf, size_t buflen, const cdf_classid_t *id)
@@ -1068,6 +1129,15 @@ cdf_print_elapsed_time(char *buf, size_t bufsiz, cdf_timestamp_t ts)
 	return len;
 }
 
+char *
+cdf_u16tos8(char *buf, size_t len, const uint16_t *p)
+{
+	size_t i;
+	for (i = 0; i < len && p[i]; i++)
+		buf[i] = (char)p[i];
+	buf[i] = '\0';
+	return buf;
+}
 
 #ifdef CDF_DEBUG
 void
@@ -1093,7 +1163,7 @@ cdf_dump_header(const cdf_header_t *h)
 	for (i = 0; i < __arraycount(h->h_master_sat); i++) {
 		if (h->h_master_sat[i] == CDF_SECID_FREE)
 			break;
-		(void)fprintf(stderr, "%35.35s[%.3zu] = %d\n",
+		(void)fprintf(stderr, "%35.35s[%.3" SIZE_T_FORMAT "u] = %d\n",
 		    "master_sat", i, h->h_master_sat[i]);
 	}
 }
@@ -1288,7 +1358,7 @@ cdf_dump_summary_info(const cdf_header_t *h, const cdf_stream_t *sst)
 		return;
 	(void)fprintf(stderr, "Endian: %x\n", ssi.si_byte_order);
 	(void)fprintf(stderr, "Os Version %d.%d\n", ssi.si_os_version & 0xff,
-		ssi.si_os_version >> 8);
+	    ssi.si_os_version >> 8);
 	(void)fprintf(stderr, "Os %d\n", ssi.si_os);
 	cdf_print_classid(buf, sizeof(buf), &ssi.si_class);
 	(void)fprintf(stderr, "Class %s\n", buf);
@@ -1297,6 +1367,27 @@ cdf_dump_summary_info(const cdf_header_t *h, const cdf_stream_t *sst)
 	free(info);
 }
 
+
+void
+cdf_dump_catalog(const cdf_header_t *h, const cdf_stream_t *sst)
+{
+	cdf_catalog_t *cat;
+	cdf_unpack_catalog(h, sst, &cat);
+	const cdf_catalog_entry_t *ce = cat->cat_e;
+	struct timespec ts;
+	char tbuf[64], sbuf[256];
+	size_t i;
+
+	printf("Catalog:\n");
+	for (i = 0; i < cat->cat_num; i++) {
+		cdf_timestamp_to_timespec(&ts, ce[i].ce_timestamp);
+		printf("\t%d %s %s", ce[i].ce_num,
+		    cdf_u16tos8(sbuf, ce[i].ce_namlen, ce[i].ce_name),
+		    cdf_ctime(&ts.tv_sec, tbuf));
+	}
+	free(cat);
+}
+
 #endif
 
 #ifdef TEST
@@ -1309,6 +1400,7 @@ main(int argc, char *argv[])
 	cdf_stream_t sst, scn;
 	cdf_dir_t dir;
 	cdf_info_t info;
+	const cdf_directory_t *root;
 
 	if (argc < 2) {
 		(void)fprintf(stderr, "Usage: %s <filename>\n", getprogname());
@@ -1342,7 +1434,8 @@ main(int argc, char *argv[])
 		if (cdf_read_dir(&info, &h, &sat, &dir) == -1)
 			err(1, "Cannot read dir");
 
-		if (cdf_read_short_stream(&info, &h, &sat, &dir, &sst) == -1)
+		if (cdf_read_short_stream(&info, &h, &sat, &dir, &sst, &root)
+		    == -1)
 			err(1, "Cannot read short stream");
 #ifdef CDF_DEBUG
 		cdf_dump_stream(&h, &sst);
@@ -1355,9 +1448,17 @@ main(int argc, char *argv[])
 
 		if (cdf_read_summary_info(&info, &h, &sat, &ssat, &sst, &dir,
 		    &scn) == -1)
-			err(1, "Cannot read summary info");
+			warn("Cannot read summary info");
+#ifdef CDF_DEBUG
+		else
+			cdf_dump_summary_info(&h, &scn);
+#endif
+		if (cdf_read_catalog(&info, &h, &sat, &ssat, &sst, &dir,
+		    &scn) == -1)
+			warn("Cannot read catalog");
 #ifdef CDF_DEBUG
-		cdf_dump_summary_info(&h, &scn);
+		else
+			cdf_dump_catalog(&h, &scn);
 #endif
 
 		(void)close(info.i_fd);
diff --git a/contrib/file/src/cdf.h b/contrib/file/src/cdf.h
index 910fb95..64e3648 100644
--- a/contrib/file/src/cdf.h
+++ b/contrib/file/src/cdf.h
@@ -267,6 +267,19 @@ typedef struct {
 	size_t i_len;
 } cdf_info_t;
 
+
+typedef struct {
+	uint16_t ce_namlen;
+	uint32_t ce_num;
+	uint64_t ce_timestamp; 
+	uint16_t ce_name[256];
+} cdf_catalog_entry_t;
+
+typedef struct {
+	size_t cat_num;
+	cdf_catalog_entry_t cat_e[0];
+} cdf_catalog_t;
+
 struct timespec;
 int cdf_timestamp_to_timespec(struct timespec *, cdf_timestamp_t);
 int cdf_timespec_to_timestamp(cdf_timestamp_t *, const struct timespec *);
@@ -301,11 +314,19 @@ int cdf_read_property_info(const cdf_stream_t *, const cdf_header_t *, uint32_t,
 int cdf_read_user_stream(const cdf_info_t *, const cdf_header_t *,
     const cdf_sat_t *, const cdf_sat_t *, const cdf_stream_t *,
     const cdf_dir_t *, const char *, cdf_stream_t *);
+#define cdf_read_catalog(info, header, sat, ssat, stream, dir, scn) \
+    cdf_read_user_stream(info, header, sat, ssat, stream, dir, "Catalog", \
+    scn)
+#define cdf_read_encrypted_package(info, header, sat, ssat, stream, dir, scn) \
+    cdf_read_user_stream(info, header, sat, ssat, stream, dir, \
+    "EncryptedPackage", scn)
 int cdf_read_summary_info(const cdf_info_t *, const cdf_header_t *,
     const cdf_sat_t *, const cdf_sat_t *, const cdf_stream_t *,
     const cdf_dir_t *, cdf_stream_t *);
 int cdf_unpack_summary_info(const cdf_stream_t *, const cdf_header_t *,
     cdf_summary_info_header_t *, cdf_property_info_t **, size_t *);
+int cdf_unpack_catalog(const cdf_header_t *, const cdf_stream_t *,
+    cdf_catalog_t **);
 int cdf_print_classid(char *, size_t, const cdf_classid_t *);
 int cdf_print_property_name(char *, size_t, uint32_t);
 int cdf_print_elapsed_time(char *, size_t, cdf_timestamp_t);
@@ -313,6 +334,7 @@ uint16_t cdf_tole2(uint16_t);
 uint32_t cdf_tole4(uint32_t);
 uint64_t cdf_tole8(uint64_t);
 char *cdf_ctime(const time_t *, char *);
+char *cdf_u16tos8(char *, size_t, const uint16_t *);
 
 #ifdef CDF_DEBUG
 void cdf_dump_header(const cdf_header_t *);
@@ -323,6 +345,7 @@ void cdf_dump_dir(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *,
     const cdf_sat_t *, const cdf_stream_t *, const cdf_dir_t *);
 void cdf_dump_property_info(const cdf_property_info_t *, size_t);
 void cdf_dump_summary_info(const cdf_header_t *, const cdf_stream_t *);
+void cdf_dump_catalog(const cdf_header_t *, const cdf_stream_t *);
 #endif
 
 
diff --git a/contrib/file/src/compress.c b/contrib/file/src/compress.c
index 45251b1..e968bb4 100644
--- a/contrib/file/src/compress.c
+++ b/contrib/file/src/compress.c
@@ -35,7 +35,7 @@
 #include "file.h"
 
 #ifndef lint
-FILE_RCSID("@(#)$File: compress.c,v 1.73 2014/01/05 15:55:21 christos Exp $")
+FILE_RCSID("@(#)$File: compress.c,v 1.77 2014/12/12 16:33:01 christos Exp $")
 #endif
 
 #include "magic.h"
@@ -45,7 +45,8 @@ FILE_RCSID("@(#)$File: compress.c,v 1.73 2014/01/05 15:55:21 christos Exp $")
 #endif
 #include <string.h>
 #include <errno.h>
-#ifndef __MINGW32__
+#include <signal.h>
+#if !defined(__MINGW32__) && !defined(WIN32)
 #include <sys/ioctl.h>
 #endif
 #ifdef HAVE_SYS_WAIT_H
@@ -103,10 +104,12 @@ file_zmagic(struct magic_set *ms, int fd, const char *name,
 	size_t i, nsz;
 	int rv = 0;
 	int mime = ms->flags & MAGIC_MIME;
+	sig_t osigpipe;
 
 	if ((ms->flags & MAGIC_COMPRESS) == 0)
 		return 0;
 
+	osigpipe = signal(SIGPIPE, SIG_IGN);
 	for (i = 0; i < ncompr; i++) {
 		if (nbytes < compr[i].maglen)
 			continue;
@@ -133,6 +136,7 @@ file_zmagic(struct magic_set *ms, int fd, const char *name,
 		}
 	}
 error:
+	(void)signal(SIGPIPE, osigpipe);
 	free(newbuf);
 	ms->flags |= MAGIC_COMPRESS;
 	return rv;
@@ -377,6 +381,7 @@ uncompressbuf(struct magic_set *ms, int fd, size_t method,
     const unsigned char *old, unsigned char **newch, size_t n)
 {
 	int fdin[2], fdout[2];
+	int status;
 	ssize_t r;
 	pid_t pid;
 
@@ -459,7 +464,17 @@ uncompressbuf(struct magic_set *ms, int fd, size_t method,
 				/*NOTREACHED*/
 
 			default:  /* parent */
-				break;
+				if (wait(&status) == -1) {
+#ifdef DEBUG
+					(void)fprintf(stderr,
+					    "Wait failed (%s)\n",
+					    strerror(errno));
+#endif
+					exit(1);
+				}
+				exit(WIFEXITED(status) ?
+				    WEXITSTATUS(status) : 1);
+				/*NOTREACHED*/
 			}
 			(void) close(fdin[1]);
 			fdin[1] = -1;
@@ -470,7 +485,7 @@ uncompressbuf(struct magic_set *ms, int fd, size_t method,
 			(void)fprintf(stderr, "Malloc failed (%s)\n",
 			    strerror(errno));
 #endif
-			n = 0;
+			n = NODATA;
 			goto err;
 		}
 		if ((r = sread(fdout[0], *newch, HOWMANY, 0)) <= 0) {
@@ -479,7 +494,7 @@ uncompressbuf(struct magic_set *ms, int fd, size_t method,
 			    strerror(errno));
 #endif
 			free(*newch);
-			n = 0;
+			n = NODATA;
 			*newch = NULL;
 			goto err;
 		} else {
@@ -491,12 +506,24 @@ err:
 		if (fdin[1] != -1)
 			(void) close(fdin[1]);
 		(void) close(fdout[0]);
-#ifdef WNOHANG
-		while (waitpid(pid, NULL, WNOHANG) != -1)
-			continue;
-#else
-		(void)wait(NULL);
+		if (wait(&status) == -1) {
+#ifdef DEBUG
+			(void)fprintf(stderr, "Wait failed (%s)\n",
+			    strerror(errno));
+#endif
+			n = NODATA;
+		} else if (!WIFEXITED(status)) {
+#ifdef DEBUG
+			(void)fprintf(stderr, "Child not exited (0x%x)\n",
+			    status);
 #endif
+		} else if (WEXITSTATUS(status) != 0) {
+#ifdef DEBUG
+			(void)fprintf(stderr, "Child exited (0x%d)\n",
+			    WEXITSTATUS(status));
+#endif
+		}
+
 		(void) close(fdin[0]);
 	    
 		return n;
diff --git a/contrib/file/src/elfclass.h b/contrib/file/src/elfclass.h
index 0826ce3..5360b0b 100644
--- a/contrib/file/src/elfclass.h
+++ b/contrib/file/src/elfclass.h
@@ -32,47 +32,51 @@
 	swap = (u.c[sizeof(int32_t) - 1] + 1) != elfhdr.e_ident[EI_DATA];
 
 	type = elf_getu16(swap, elfhdr.e_type);
+	notecount = ms->elf_notes_max;
 	switch (type) {
 #ifdef ELFCORE
 	case ET_CORE:
 		phnum = elf_getu16(swap, elfhdr.e_phnum);
-		if (phnum > MAX_PHNUM)
-			return toomany(ms, "program", phnum);
+		if (phnum > ms->elf_phnum_max)
+			return toomany(ms, "program headers", phnum);
 		flags |= FLAGS_IS_CORE;
 		if (dophn_core(ms, clazz, swap, fd,
 		    (off_t)elf_getu(swap, elfhdr.e_phoff), phnum,
 		    (size_t)elf_getu16(swap, elfhdr.e_phentsize),
-		    fsize, &flags) == -1)
+		    fsize, &flags, &notecount) == -1)
 			return -1;
 		break;
 #endif
 	case ET_EXEC:
 	case ET_DYN:
 		phnum = elf_getu16(swap, elfhdr.e_phnum);
-		if (phnum > MAX_PHNUM)
+		if (phnum > ms->elf_phnum_max)
 			return toomany(ms, "program", phnum);
 		shnum = elf_getu16(swap, elfhdr.e_shnum);
-		if (shnum > MAX_SHNUM)
+		if (shnum > ms->elf_shnum_max)
 			return toomany(ms, "section", shnum);
 		if (dophn_exec(ms, clazz, swap, fd,
 		    (off_t)elf_getu(swap, elfhdr.e_phoff), phnum,
 		    (size_t)elf_getu16(swap, elfhdr.e_phentsize),
-		    fsize, &flags, shnum) == -1)
+		    fsize, shnum, &flags, &notecount) == -1)
 			return -1;
 		/*FALLTHROUGH*/
 	case ET_REL:
 		shnum = elf_getu16(swap, elfhdr.e_shnum);
-		if (shnum > MAX_SHNUM)
-			return toomany(ms, "section", shnum);
+		if (shnum > ms->elf_shnum_max)
+			return toomany(ms, "section headers", shnum);
 		if (doshn(ms, clazz, swap, fd,
 		    (off_t)elf_getu(swap, elfhdr.e_shoff), shnum,
 		    (size_t)elf_getu16(swap, elfhdr.e_shentsize),
-		    fsize, &flags, elf_getu16(swap, elfhdr.e_machine),
-		    (int)elf_getu16(swap, elfhdr.e_shstrndx)) == -1)
+		    fsize, elf_getu16(swap, elfhdr.e_machine),
+		    (int)elf_getu16(swap, elfhdr.e_shstrndx),
+		    &flags, &notecount) == -1)
 			return -1;
 		break;
 
 	default:
 		break;
 	}
+	if (notecount == 0)
+		return toomany(ms, "notes", ms->elf_notes_max);
 	return 1;
diff --git a/contrib/file/src/encoding.c b/contrib/file/src/encoding.c
index b75e0cc..c1b23cc 100644
--- a/contrib/file/src/encoding.c
+++ b/contrib/file/src/encoding.c
@@ -35,7 +35,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: encoding.c,v 1.9 2013/11/19 20:45:50 christos Exp $")
+FILE_RCSID("@(#)$File: encoding.c,v 1.10 2014/09/11 12:08:52 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -97,7 +97,6 @@ file_encoding(struct magic_set *ms, const unsigned char *buf, size_t nbytes, uni
 		*code_mime = "utf-8";
 	} else if (file_looks_utf8(buf, nbytes, *ubuf, ulen) > 1) {
 		DPRINTF(("utf8 %" SIZE_T_FORMAT "u\n", *ulen));
-		*code = "UTF-8 Unicode (with BOM)";
 		*code = "UTF-8 Unicode";
 		*code_mime = "utf-8";
 	} else if ((ucs_type = looks_ucs16(buf, nbytes, *ubuf, ulen)) != 0) {
diff --git a/contrib/file/src/file.c b/contrib/file/src/file.c
index 370da91..546fd8b 100644
--- a/contrib/file/src/file.c
+++ b/contrib/file/src/file.c
@@ -32,7 +32,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: file.c,v 1.153 2014/02/11 15:41:04 christos Exp $")
+FILE_RCSID("@(#)$File: file.c,v 1.160 2014/12/16 23:18:40 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -54,9 +54,6 @@ FILE_RCSID("@(#)$File: file.c,v 1.153 2014/02/11 15:41:04 christos Exp $")
 #ifdef HAVE_UNISTD_H
 #include <unistd.h>	/* for read() */
 #endif
-#ifdef HAVE_LOCALE_H
-#include <locale.h>
-#endif
 #ifdef HAVE_WCHAR_H
 #include <wchar.h>
 #endif
@@ -101,7 +98,7 @@ private const struct option long_options[] = {
 #undef OPT_LONGONLY
     {0, 0, NULL, 0}
 };
-#define OPTSTRING	"bcCde:Ef:F:hiklLm:nNprsvz0"
+#define OPTSTRING	"bcCde:Ef:F:hiklLm:nNpP:rsvz0"
 
 private const struct {
 	const char *name;
@@ -119,6 +116,18 @@ private const struct {
 	{ "tokens",	MAGIC_NO_CHECK_TOKENS }, /* OBSOLETE: ignored for backwards compatibility */
 };
 
+private struct {
+	const char *name;
+	int tag;
+	size_t value;
+} pm[] = {
+	{ "indir",	MAGIC_PARAM_INDIR_MAX, 0 },
+	{ "name",	MAGIC_PARAM_NAME_MAX, 0 },
+	{ "elf_phnum",	MAGIC_PARAM_ELF_PHNUM_MAX, 0 },
+	{ "elf_shnum",	MAGIC_PARAM_ELF_SHNUM_MAX, 0 },
+	{ "elf_notes",	MAGIC_PARAM_ELF_NOTES_MAX, 0 },
+};
+
 private char *progname;		/* used throughout 		*/
 
 private void usage(void);
@@ -128,6 +137,8 @@ private void help(void);
 private int unwrap(struct magic_set *, const char *);
 private int process(struct magic_set *ms, const char *, int);
 private struct magic_set *load(const char *, int);
+private void setparam(const char *);
+private void applyparam(magic_t);
 
 
 /*
@@ -145,7 +156,9 @@ main(int argc, char *argv[])
 	const char *magicfile = NULL;		/* where the magic is	*/
 
 	/* makes islower etc work for other langs */
+#ifdef HAVE_SETLOCALE
 	(void)setlocale(LC_CTYPE, "");
+#endif
 
 #ifdef __EMX__
 	/* sh-like wildcard expansion! Shouldn't hurt at least ... */
@@ -243,9 +256,13 @@ main(int argc, char *argv[])
 			flags |= MAGIC_PRESERVE_ATIME;
 			break;
 #endif
+		case 'P':
+			setparam(optarg);
+			break;
 		case 'r':
 			flags |= MAGIC_RAW;
 			break;
+			break;
 		case 's':
 			flags |= MAGIC_DEVICES;
 			break;
@@ -298,6 +315,8 @@ main(int argc, char *argv[])
 			    strerror(errno));
 			return 1;
 		}
+
+
 		switch(action) {
 		case FILE_CHECK:
 			c = magic_check(magic, magicfile);
@@ -321,7 +340,7 @@ main(int argc, char *argv[])
 		if (magic == NULL)
 			if ((magic = load(magicfile, flags)) == NULL)
 				return 1;
-		break;
+		applyparam(magic);
 	}
 
 	if (optind == argc) {
@@ -351,6 +370,41 @@ main(int argc, char *argv[])
 	return e;
 }
 
+private void
+applyparam(magic_t magic)
+{
+	size_t i;
+
+	for (i = 0; i < __arraycount(pm); i++) {
+		if (pm[i].value == 0)
+			continue;
+		if (magic_setparam(magic, pm[i].tag, &pm[i].value) == -1) {
+			(void)fprintf(stderr, "%s: Can't set %s %s\n", progname,
+				pm[i].name, strerror(errno));
+			exit(1);
+		}
+	}
+}
+
+private void
+setparam(const char *p)
+{
+	size_t i;
+	char *s;
+
+	if ((s = strchr(p, '=')) == NULL)
+		goto badparm;
+
+	for (i = 0; i < __arraycount(pm); i++) {
+		if (strncmp(p, pm[i].name, s - p) != 0)
+			continue;
+		pm[i].value = atoi(s + 1);
+		return;
+	}
+badparm:
+	(void)fprintf(stderr, "%s: Unknown param %s\n", progname, p);
+	exit(1);
+}
 
 private struct magic_set *
 /*ARGSUSED*/
diff --git a/contrib/file/src/file.h b/contrib/file/src/file.h
index 1a652a8..01aa37a 100644
--- a/contrib/file/src/file.h
+++ b/contrib/file/src/file.h
@@ -27,7 +27,7 @@
  */
 /*
  * file.h - definitions for file(1) program
- * @(#)$File: file.h,v 1.152 2014/06/03 19:01:34 christos Exp $
+ * @(#)$File: file.h,v 1.164 2015/01/01 17:07:34 christos Exp $
  */
 
 #ifndef __file_h__
@@ -64,7 +64,9 @@
 #include <regex.h>
 #include <time.h>
 #include <sys/types.h>
+#ifndef WIN32
 #include <sys/param.h>
+#endif
 /* Do this here and now, because struct stat gets re-defined on solaris */
 #include <sys/stat.h>
 #include <stdarg.h>
@@ -232,6 +234,7 @@ struct magic {
 	 (t) == FILE_LESTRING16 || \
 	 (t) == FILE_REGEX || \
 	 (t) == FILE_SEARCH || \
+	 (t) == FILE_INDIRECT || \
 	 (t) == FILE_NAME || \
 	 (t) == FILE_USE)
 
@@ -344,6 +347,8 @@ struct magic {
 #define STRING_IGNORE_CASE		(STRING_IGNORE_LOWERCASE|STRING_IGNORE_UPPERCASE)
 #define STRING_DEFAULT_RANGE		100
 
+#define	INDIRECT_RELATIVE			BIT(0)
+#define	CHAR_INDIRECT_RELATIVE			'r'
 
 /* list of magic entries */
 struct mlist {
@@ -401,6 +406,16 @@ struct magic_set {
 	/* FIXME: Make the string dynamically allocated so that e.g.
 	   strings matched in files can be longer than MAXstring */
 	union VALUETYPE ms_value;	/* either number or string */
+	uint16_t indir_max;
+	uint16_t name_max;
+	uint16_t elf_shnum_max;
+	uint16_t elf_phnum_max;
+	uint16_t elf_notes_max;
+#define	FILE_INDIR_MAX			15
+#define	FILE_NAME_MAX			30
+#define	FILE_ELF_SHNUM_MAX		32768
+#define	FILE_ELF_PHNUM_MAX		128
+#define	FILE_ELF_NOTES_MAX		256
 };
 
 /* Type for Unicode characters */
@@ -440,8 +455,10 @@ protected int file_encoding(struct magic_set *, const unsigned char *, size_t,
     unichar **, size_t *, const char **, const char **, const char **);
 protected int file_is_tar(struct magic_set *, const unsigned char *, size_t);
 protected int file_softmagic(struct magic_set *, const unsigned char *, size_t,
-    size_t, int, int);
+    uint16_t, uint16_t *, int, int);
 protected int file_apprentice(struct magic_set *, const char *, int);
+protected int buffer_apprentice(struct magic_set *, struct magic **,
+    size_t *, size_t);
 protected int file_magicfind(struct magic_set *, const char *, struct mlist *);
 protected uint64_t file_signextend(struct magic_set *, struct magic *,
     uint64_t);
@@ -464,14 +481,26 @@ protected int file_looks_utf8(const unsigned char *, size_t, unichar *,
     size_t *);
 protected size_t file_pstring_length_size(const struct magic *);
 protected size_t file_pstring_get_length(const struct magic *, const char *);
+protected char * file_printable(char *, size_t, const char *);
 #ifdef __EMX__
 protected int file_os2_apptype(struct magic_set *, const char *, const void *,
     size_t);
 #endif /* __EMX__ */
 
+#if defined(HAVE_LOCALE_H)
+#include <locale.h>
+#endif
+#if defined(HAVE_XLOCALE_H)
+#include <xlocale.h>
+#endif
+
 typedef struct {
 	const char *pat;
-	char *old_lc_ctype;
+#if defined(HAVE_NEWLOCALE) && defined(HAVE_USELOCALE) && defined(HAVE_FREELOCALE)
+#define USE_C_LOCALE
+	locale_t old_lc_ctype;
+	locale_t c_lc_ctype;
+#endif
 	int rc;
 	regex_t rx;
 } file_regex_t;
diff --git a/contrib/file/src/file_opts.h b/contrib/file/src/file_opts.h
index db34eb7..3286ac6 100644
--- a/contrib/file/src/file_opts.h
+++ b/contrib/file/src/file_opts.h
@@ -43,6 +43,12 @@ OPT('0', "print0", 0, "               terminate filenames with ASCII NUL\n")
 #if defined(HAVE_UTIME) || defined(HAVE_UTIMES)
 OPT('p', "preserve-date", 0, "        preserve access times on files\n")
 #endif
+OPT('P', "parameter", 0, "            set file engine parameter limits\n"
+    "                               indir        15 recursion limit for indirection\n"
+    "                               name         30 use limit for name/use magic\n"
+    "                               elf_notes   256 max ELF notes processed\n"
+    "                               elf_phnum   128 max ELF prog sections processed\n"
+    "                               elf_shnum 32768 max ELF sections processed\n")
 OPT('r', "raw", 0, "                  don't translate unprintable chars to \\ooo\n")
 OPT('s', "special-files", 0, "        treat special (block/char devices) files as\n"
     "                             ordinary ones\n")
diff --git a/contrib/file/src/fsmagic.c b/contrib/file/src/fsmagic.c
index ef5492f..1e8fd74 100644
--- a/contrib/file/src/fsmagic.c
+++ b/contrib/file/src/fsmagic.c
@@ -32,7 +32,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: fsmagic.c,v 1.73 2014/05/14 23:15:42 christos Exp $")
+FILE_RCSID("@(#)$File: fsmagic.c,v 1.75 2014/12/04 15:56:46 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -75,10 +75,10 @@ bad_link(struct magic_set *ms, int err, char *buf)
 	else if (!mime) {
 		if (ms->flags & MAGIC_ERROR) {
 			file_error(ms, err,
-				   "broken symbolic link to `%s'", buf);
+				   "broken symbolic link to %s", buf);
 			return -1;
 		} 
-		if (file_printf(ms, "broken symbolic link to `%s'", buf) == -1)
+		if (file_printf(ms, "broken symbolic link to %s", buf) == -1)
 			return -1;
 	}
 	return 1;
@@ -129,7 +129,7 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb)
 
 #ifdef WIN32
 	{
-		HANDLE hFile = CreateFile(fn, 0, FILE_SHARE_DELETE |
+		HANDLE hFile = CreateFile((LPCSTR)fn, 0, FILE_SHARE_DELETE |
 		    FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0,
 		    NULL);
 		if (hFile != INVALID_HANDLE_VALUE) {
@@ -352,7 +352,7 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb)
 			if (mime) {
 				if (handle_mime(ms, mime, "symlink") == -1)
 					return -1;
-			} else if (file_printf(ms, "%ssymbolic link to `%s'",
+			} else if (file_printf(ms, "%ssymbolic link to %s",
 			    COMMA, buf) == -1)
 				return -1;
 		}
diff --git a/contrib/file/src/funcs.c b/contrib/file/src/funcs.c
index 7e7d2e7..a60ccaa 100644
--- a/contrib/file/src/funcs.c
+++ b/contrib/file/src/funcs.c
@@ -27,7 +27,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: funcs.c,v 1.72 2014/05/14 23:15:42 christos Exp $")
+FILE_RCSID("@(#)$File: funcs.c,v 1.79 2014/12/16 20:52:49 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -45,9 +45,6 @@ FILE_RCSID("@(#)$File: funcs.c,v 1.72 2014/05/14 23:15:42 christos Exp $")
 #if defined(HAVE_LIMITS_H)
 #include <limits.h>
 #endif
-#if defined(HAVE_LOCALE_H)
-#include <locale.h>
-#endif
 
 #ifndef SIZE_MAX
 #define SIZE_MAX	((size_t)~0)
@@ -230,7 +227,7 @@ file_buffer(struct magic_set *ms, int fd, const char *inname __attribute__ ((unu
 
 	/* try soft magic tests */
 	if ((ms->flags & MAGIC_NO_CHECK_SOFT) == 0)
-		if ((m = file_softmagic(ms, ubuf, nb, 0, BINTEST,
+		if ((m = file_softmagic(ms, ubuf, nb, 0, NULL, BINTEST,
 		    looks_text)) != 0) {
 			if ((ms->flags & MAGIC_DEBUG) != 0)
 				(void)fprintf(stderr, "softmagic %d\n", m);
@@ -455,13 +452,14 @@ out:
 protected int
 file_regcomp(file_regex_t *rx, const char *pat, int flags)
 {
-	rx->old_lc_ctype = setlocale(LC_CTYPE, NULL);
-	assert(rx->old_lc_ctype != NULL);
-	rx->old_lc_ctype = strdup(rx->old_lc_ctype);
+#ifdef USE_C_LOCALE
+	rx->c_lc_ctype = newlocale(LC_CTYPE_MASK, "C", 0);
+	assert(rx->c_lc_ctype != NULL);
+	rx->old_lc_ctype = uselocale(rx->c_lc_ctype);
 	assert(rx->old_lc_ctype != NULL);
+#endif
 	rx->pat = pat;
 
-	(void)setlocale(LC_CTYPE, "C");
 	return rx->rc = regcomp(&rx->rx, pat, flags);
 }
 
@@ -478,8 +476,10 @@ file_regfree(file_regex_t *rx)
 {
 	if (rx->rc == 0)
 		regfree(&rx->rx);
-	(void)setlocale(LC_CTYPE, rx->old_lc_ctype);
-	free(rx->old_lc_ctype);
+#ifdef USE_C_LOCALE
+	(void)uselocale(rx->old_lc_ctype);
+	freelocale(rx->c_lc_ctype);
+#endif
 }
 
 protected void
@@ -531,3 +531,28 @@ file_pop_buffer(struct magic_set *ms, file_pushbuf_t *pb)
 	free(pb);
 	return rbuf;
 }
+
+/*
+ * convert string to ascii printable format.
+ */
+protected char *
+file_printable(char *buf, size_t bufsiz, const char *str)
+{
+	char *ptr, *eptr;
+	const unsigned char *s = (const unsigned char *)str;
+
+	for (ptr = buf, eptr = ptr + bufsiz - 1; ptr < eptr && *s; s++) {
+		if (isprint(*s)) {
+			*ptr++ = *s;
+			continue;
+		}
+		if (ptr >= eptr - 3)
+			break;
+		*ptr++ = '\\';
+		*ptr++ = ((*s >> 6) & 7) + '0';
+		*ptr++ = ((*s >> 3) & 7) + '0';
+		*ptr++ = ((*s >> 0) & 7) + '0';
+	}
+	*ptr = '\0';
+	return buf;
+}
diff --git a/contrib/file/src/getline.c b/contrib/file/src/getline.c
index 99cd150..b00de01 100644
--- a/contrib/file/src/getline.c
+++ b/contrib/file/src/getline.c
@@ -1,4 +1,4 @@
-/*	$NetBSD: fgetln.c,v 1.9 2008/04/29 06:53:03 martin Exp $	*/
+/*	$NetBSD: getline.c,v 1.2 2014/09/16 17:23:50 christos Exp $	*/
 
 /*-
  * Copyright (c) 2011 The NetBSD Foundation, Inc.
@@ -52,10 +52,14 @@ getdelim(char **buf, size_t *bufsiz, int delimiter, FILE *fp)
 	for (ptr = *buf, eptr = *buf + *bufsiz;;) {
 		int c = fgetc(fp);
 		if (c == -1) {
-			if (feof(fp))
-				return ptr == *buf ? -1 : ptr - *buf;
-			else
-				return -1;
+			if (feof(fp)) {
+				ssize_t diff = (ssize_t)(ptr - *buf);
+				if (diff != 0) {
+					*ptr = '\0';
+					return diff;
+				}
+			}
+			return -1;
 		}
 		*ptr++ = c;
 		if (c == delimiter) {
@@ -76,7 +80,7 @@ getdelim(char **buf, size_t *bufsiz, int delimiter, FILE *fp)
 	}
 }
 
-ssize_t
+public ssize_t
 getline(char **buf, size_t *bufsiz, FILE *fp)
 {
 	return getdelim(buf, bufsiz, '\n', fp);
@@ -93,7 +97,7 @@ main(int argc, char *argv[])
 	size_t n = 0;
 
 	while ((len = getline(&p, &n, stdin)) != -1)
-		(void)printf("%zd %s", len, p);
+		(void)printf("%" SIZE_T_FORMAT "d %s", len, p);
 	free(p);
 	return 0;
 }
diff --git a/contrib/file/src/magic.c b/contrib/file/src/magic.c
index e4bd12b..d16f8c6 100644
--- a/contrib/file/src/magic.c
+++ b/contrib/file/src/magic.c
@@ -33,7 +33,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: magic.c,v 1.84 2014/05/14 23:15:42 christos Exp $")
+FILE_RCSID("@(#)$File: magic.c,v 1.91 2014/12/16 23:18:40 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -128,6 +128,7 @@ out:
 #else
 	char *hmagicp;
 	char *tmppath = NULL;
+	LPTSTR dllpath;
 	hmagicpath = NULL;
 
 #define APPENDPATH() \
@@ -173,7 +174,7 @@ out:
 	}
 
 	/* Third, try to get magic file relative to dll location */
-	LPTSTR dllpath = malloc(sizeof(*dllpath) * (MAX_PATH + 1));
+	dllpath = malloc(sizeof(*dllpath) * (MAX_PATH + 1));
 	dllpath[MAX_PATH] = 0;	/* just in case long path gets truncated and not null terminated */
 	if (GetModuleFileNameA(NULL, dllpath, MAX_PATH)){
 		PathRemoveFileSpecA(dllpath);
@@ -257,6 +258,20 @@ magic_load(struct magic_set *ms, const char *magicfile)
 	return file_apprentice(ms, magicfile, FILE_LOAD);
 }
 
+#ifndef COMPILE_ONLY
+/*
+ * Install a set of compiled magic buffers.
+ */
+public int
+magic_load_buffers(struct magic_set *ms, void **bufs, size_t *sizes,
+    size_t nbufs)
+{
+	if (ms == NULL)
+		return -1;
+	return buffer_apprentice(ms, (struct magic **)bufs, sizes, nbufs);
+}
+#endif
+
 public int
 magic_compile(struct magic_set *ms, const char *magicfile)
 {
@@ -522,3 +537,53 @@ magic_version(void)
 {
 	return MAGIC_VERSION;
 }
+
+public int
+magic_setparam(struct magic_set *ms, int param, const void *val)
+{
+	switch (param) {
+	case MAGIC_PARAM_INDIR_MAX:
+		ms->indir_max = *(const size_t *)val;
+		return 0;
+	case MAGIC_PARAM_NAME_MAX:
+		ms->name_max = *(const size_t *)val;
+		return 0;
+	case MAGIC_PARAM_ELF_PHNUM_MAX:
+		ms->elf_phnum_max = *(const size_t *)val;
+		return 0;
+	case MAGIC_PARAM_ELF_SHNUM_MAX:
+		ms->elf_shnum_max = *(const size_t *)val;
+		return 0;
+	case MAGIC_PARAM_ELF_NOTES_MAX:
+		ms->elf_notes_max = *(const size_t *)val;
+		return 0;
+	default:
+		errno = EINVAL;
+		return -1;
+	}
+}
+
+public int
+magic_getparam(struct magic_set *ms, int param, void *val)
+{
+	switch (param) {
+	case MAGIC_PARAM_INDIR_MAX:
+		*(size_t *)val = ms->indir_max;
+		return 0;
+	case MAGIC_PARAM_NAME_MAX:
+		*(size_t *)val = ms->name_max;
+		return 0;
+	case MAGIC_PARAM_ELF_PHNUM_MAX:
+		*(size_t *)val = ms->elf_phnum_max;
+		return 0;
+	case MAGIC_PARAM_ELF_SHNUM_MAX:
+		*(size_t *)val = ms->elf_shnum_max;
+		return 0;
+	case MAGIC_PARAM_ELF_NOTES_MAX:
+		*(size_t *)val = ms->elf_notes_max;
+		return 0;
+	default:
+		errno = EINVAL;
+		return -1;
+	}
+}
diff --git a/contrib/file/src/magic.h b/contrib/file/src/magic.h
index 38e59d7..721712f 100644
--- a/contrib/file/src/magic.h
+++ b/contrib/file/src/magic.h
@@ -75,7 +75,7 @@
 #define	MAGIC_NO_CHECK_FORTRAN	0x000000 /* Don't check ascii/fortran */
 #define	MAGIC_NO_CHECK_TROFF	0x000000 /* Don't check ascii/troff */
 
-#define MAGIC_VERSION		518	/* This implementation */
+#define MAGIC_VERSION		521	/* This implementation */
 
 
 #ifdef __cplusplus
@@ -96,11 +96,22 @@ int magic_setflags(magic_t, int);
 
 int magic_version(void);
 int magic_load(magic_t, const char *);
+int magic_load_buffers(magic_t, void **, size_t *, size_t);
+
 int magic_compile(magic_t, const char *);
 int magic_check(magic_t, const char *);
 int magic_list(magic_t, const char *);
 int magic_errno(magic_t);
 
+#define MAGIC_PARAM_INDIR_MAX		0
+#define MAGIC_PARAM_NAME_MAX		1
+#define MAGIC_PARAM_ELF_PHNUM_MAX	2
+#define MAGIC_PARAM_ELF_SHNUM_MAX	3
+#define MAGIC_PARAM_ELF_NOTES_MAX	4
+
+int magic_setparam(magic_t, int, const void *);
+int magic_getparam(magic_t, int, void *);
+
 #ifdef __cplusplus
 };
 #endif
diff --git a/contrib/file/src/magic.h.in b/contrib/file/src/magic.h.in
index 86fc41b..0d4c5ce 100644
--- a/contrib/file/src/magic.h.in
+++ b/contrib/file/src/magic.h.in
@@ -96,11 +96,22 @@ int magic_setflags(magic_t, int);
 
 int magic_version(void);
 int magic_load(magic_t, const char *);
+int magic_load_buffers(magic_t, void **, size_t *, size_t);
+
 int magic_compile(magic_t, const char *);
 int magic_check(magic_t, const char *);
 int magic_list(magic_t, const char *);
 int magic_errno(magic_t);
 
+#define MAGIC_PARAM_INDIR_MAX		0
+#define MAGIC_PARAM_NAME_MAX		1
+#define MAGIC_PARAM_ELF_PHNUM_MAX	2
+#define MAGIC_PARAM_ELF_SHNUM_MAX	3
+#define MAGIC_PARAM_ELF_NOTES_MAX	4
+
+int magic_setparam(magic_t, int, const void *);
+int magic_getparam(magic_t, int, void *);
+
 #ifdef __cplusplus
 };
 #endif
diff --git a/contrib/file/src/pread.c b/contrib/file/src/pread.c
index 649f609..72d3a6b 100644
--- a/contrib/file/src/pread.c
+++ b/contrib/file/src/pread.c
@@ -1,14 +1,23 @@
 #include "file.h"
 #ifndef lint
-FILE_RCSID("@(#)$File: pread.c,v 1.2 2013/04/02 16:23:07 christos Exp $")
+FILE_RCSID("@(#)$File: pread.c,v 1.3 2014/09/15 19:11:25 christos Exp $")
 #endif  /* lint */
 #include <fcntl.h>
 #include <unistd.h>
 
 ssize_t
 pread(int fd, void *buf, size_t len, off_t off) {
-	if (lseek(fd, off, SEEK_SET) == (off_t)-1)
+	off_t old;
+	ssize_t rv;
+
+	if ((old = lseek(fd, off, SEEK_SET)) == -1)
+		return -1;
+
+	if ((rv = read(fd, buf, len)) == -1)
+		return -1;
+
+	if (lseek(fd, old, SEEK_SET) == -1)
 		return -1;
 
-	return read(fd, buf, len);
+	return rv;
 }
diff --git a/contrib/file/src/readcdf.c b/contrib/file/src/readcdf.c
index 7ced9ea..635a926 100644
--- a/contrib/file/src/readcdf.c
+++ b/contrib/file/src/readcdf.c
@@ -26,7 +26,7 @@
 #include "file.h"
 
 #ifndef lint
-FILE_RCSID("@(#)$File: readcdf.c,v 1.44 2014/05/14 23:22:48 christos Exp $")
+FILE_RCSID("@(#)$File: readcdf.c,v 1.49 2014/12/04 15:56:46 christos Exp $")
 #endif
 
 #include <assert.h>
@@ -35,9 +35,6 @@ FILE_RCSID("@(#)$File: readcdf.c,v 1.44 2014/05/14 23:22:48 christos Exp $")
 #include <string.h>
 #include <time.h>
 #include <ctype.h>
-#if defined(HAVE_LOCALE_H)
-#include <locale.h>
-#endif
 
 #include "cdf.h"
 #include "magic.h"
@@ -75,7 +72,7 @@ static const struct cv {
 	const char *mime;
 } clsid2mime[] = {
 	{
-		{ 0x00000000000c1084LLU, 0x46000000000000c0LLU  },
+		{ 0x00000000000c1084ULL, 0x46000000000000c0ULL  },
 		"x-msi",
 	},
 	{	{ 0,			 0			},
@@ -83,7 +80,7 @@ static const struct cv {
 	},
 }, clsid2desc[] = {
 	{
-		{ 0x00000000000c1084LLU, 0x46000000000000c0LLU  },
+		{ 0x00000000000c1084ULL, 0x46000000000000c0ULL  },
 		"MSI Installer",
 	},
 	{	{ 0,			 0			},
@@ -107,20 +104,23 @@ cdf_app_to_mime(const char *vbuf, const struct nv *nv)
 {
 	size_t i;
 	const char *rv = NULL;
-	char *old_lc_ctype;
+#ifdef USE_C_LOCALE
+	locale_t old_lc_ctype, c_lc_ctype;
 
-	old_lc_ctype = setlocale(LC_CTYPE, NULL);
-	assert(old_lc_ctype != NULL);
-	old_lc_ctype = strdup(old_lc_ctype);
+	c_lc_ctype = newlocale(LC_CTYPE_MASK, "C", 0);
+	assert(c_lc_ctype != NULL);
+	old_lc_ctype = uselocale(c_lc_ctype);
 	assert(old_lc_ctype != NULL);
-	(void)setlocale(LC_CTYPE, "C");
+#endif
 	for (i = 0; nv[i].pattern != NULL; i++)
 		if (strcasestr(vbuf, nv[i].pattern) != NULL) {
 			rv = nv[i].mime;
 			break;
 		}
-	(void)setlocale(LC_CTYPE, old_lc_ctype);
-	free(old_lc_ctype);
+#ifdef USE_C_LOCALE
+	(void)uselocale(old_lc_ctype);
+	freelocale(c_lc_ctype);
+#endif
 	return rv;
 }
 
@@ -241,6 +241,37 @@ cdf_file_property_info(struct magic_set *ms, const cdf_property_info_t *info,
 }
 
 private int
+cdf_file_catalog(struct magic_set *ms, const cdf_header_t *h,
+    const cdf_stream_t *sst)
+{
+	cdf_catalog_t *cat;
+	size_t i;
+	char buf[256];
+	cdf_catalog_entry_t *ce;
+
+        if (NOTMIME(ms)) {
+		if (file_printf(ms, "Microsoft Thumbs.db [") == -1)
+			return -1;
+		if (cdf_unpack_catalog(h, sst, &cat) == -1)
+			return -1;
+		ce = cat->cat_e;
+		/* skip first entry since it has a , or paren */
+		for (i = 1; i < cat->cat_num; i++)
+			if (file_printf(ms, "%s%s",
+			    cdf_u16tos8(buf, ce[i].ce_namlen, ce[i].ce_name),
+			    i == cat->cat_num - 1 ? "]" : ", ") == -1) {
+				free(cat);
+				return -1;
+			}
+		free(cat);
+	} else {
+		if (file_printf(ms, "application/CDFV2") == -1)
+			return -1;
+	}
+	return 1;
+}
+
+private int
 cdf_file_summary_info(struct magic_set *ms, const cdf_header_t *h,
     const cdf_stream_t *sst, const cdf_directory_t *root_storage)
 {
@@ -285,11 +316,12 @@ cdf_file_summary_info(struct magic_set *ms, const cdf_header_t *h,
 		if (root_storage) {
 			str = cdf_clsid_to_mime(root_storage->d_storage_uuid,
 			    clsid2desc);
-			if (str)
+			if (str) {
 				if (file_printf(ms, ", %s", str) == -1)
 					return -2;
 			}
 		}
+	}
 
         m = cdf_file_property_info(ms, info, count, root_storage);
         free(info);
@@ -302,11 +334,11 @@ private char *
 format_clsid(char *buf, size_t len, const uint64_t uuid[2]) {
 	snprintf(buf, len, "%.8" PRIx64 "-%.4" PRIx64 "-%.4" PRIx64 "-%.4" 
 	    PRIx64 "-%.12" PRIx64,
-	    (uuid[0] >> 32) & (uint64_t)0x000000000ffffffffLLU,
-	    (uuid[0] >> 16) & (uint64_t)0x0000000000000ffffLLU,
-	    (uuid[0] >>  0) & (uint64_t)0x0000000000000ffffLLU, 
-	    (uuid[1] >> 48) & (uint64_t)0x0000000000000ffffLLU,
-	    (uuid[1] >>  0) & (uint64_t)0x0000fffffffffffffLLU);
+	    (uuid[0] >> 32) & (uint64_t)0x000000000ffffffffULL,
+	    (uuid[0] >> 16) & (uint64_t)0x0000000000000ffffULL,
+	    (uuid[0] >>  0) & (uint64_t)0x0000000000000ffffULL, 
+	    (uuid[1] >> 48) & (uint64_t)0x0000000000000ffffULL,
+	    (uuid[1] >>  0) & (uint64_t)0x0000fffffffffffffULL);
 	return buf;
 }
 #endif
@@ -323,6 +355,7 @@ file_trycdf(struct magic_set *ms, int fd, const unsigned char *buf,
         int i;
         const char *expn = "";
         const char *corrupt = "corrupt: ";
+        const cdf_directory_t *root_storage;
 
         info.i_fd = fd;
         info.i_buf = buf;
@@ -356,7 +389,6 @@ file_trycdf(struct magic_set *ms, int fd, const unsigned char *buf,
                 goto out2;
         }
 
-        const cdf_directory_t *root_storage;
         if ((i = cdf_read_short_stream(&info, &h, &sat, &dir, &sst,
 	    &root_storage)) == -1) {
                 expn = "Cannot read short stream";
@@ -404,8 +436,24 @@ file_trycdf(struct magic_set *ms, int fd, const unsigned char *buf,
         if ((i = cdf_read_summary_info(&info, &h, &sat, &ssat, &sst, &dir,
             &scn)) == -1) {
                 if (errno == ESRCH) {
-                        corrupt = expn;
-                        expn = "No summary info";
+			if ((i = cdf_read_catalog(&info, &h, &sat, &ssat, &sst,
+			    &dir, &scn)) == -1) {
+				corrupt = expn;
+				if ((i = cdf_read_encrypted_package(&info, &h,
+				    &sat, &ssat, &sst, &dir, &scn)) == -1)
+					expn = "No summary info";
+				else {
+					expn = "Encrypted";
+					i = -1;
+				}
+				goto out4;
+			}
+#ifdef CDF_DEBUG
+			cdf_dump_catalog(&h, &scn);
+#endif
+			if ((i = cdf_file_catalog(ms, &h, &scn))
+			    < 0)
+				expn = "Can't expand catalog";
                 } else {
                         expn = "Cannot read summary info";
                 }
@@ -464,7 +512,8 @@ out0:
 		    if (file_printf(ms, ", %s%s", corrupt, expn) == -1)
 			return -1;
 	    } else {
-		if (file_printf(ms, "application/CDFV2-corrupt") == -1)
+		if (file_printf(ms, "application/CDFV2-%s",
+		    *corrupt ? "corrupt" : "encrypted") == -1)
 		    return -1;
 	    }
 	    i = 1;
diff --git a/contrib/file/src/readelf.c b/contrib/file/src/readelf.c
index 55b7e98..d159620 100644
--- a/contrib/file/src/readelf.c
+++ b/contrib/file/src/readelf.c
@@ -27,7 +27,7 @@
 #include "file.h"
 
 #ifndef lint
-FILE_RCSID("@(#)$File: readelf.c,v 1.103 2014/05/02 02:25:10 christos Exp $")
+FILE_RCSID("@(#)$File: readelf.c,v 1.117 2014/12/16 23:29:42 christos Exp $")
 #endif
 
 #ifdef BUILTIN_ELF
@@ -43,14 +43,14 @@ FILE_RCSID("@(#)$File: readelf.c,v 1.103 2014/05/02 02:25:10 christos Exp $")
 
 #ifdef	ELFCORE
 private int dophn_core(struct magic_set *, int, int, int, off_t, int, size_t,
-    off_t, int *);
+    off_t, int *, uint16_t *);
 #endif
 private int dophn_exec(struct magic_set *, int, int, int, off_t, int, size_t,
-    off_t, int *, int);
+    off_t, int, int *, uint16_t *);
 private int doshn(struct magic_set *, int, int, int, off_t, int, size_t,
-    off_t, int *, int, int);
+    off_t, int, int, int *, uint16_t *);
 private size_t donote(struct magic_set *, void *, size_t, size_t, int,
-    int, size_t, int *);
+    int, size_t, int *, uint16_t *);
 
 #define	ELF_ALIGN(a)	((((a) + align - 1) / align) * align)
 
@@ -60,13 +60,14 @@ private uint16_t getu16(int, uint16_t);
 private uint32_t getu32(int, uint32_t);
 private uint64_t getu64(int, uint64_t);
 
-#define MAX_PHNUM	256
-#define	MAX_SHNUM	1024
+#define MAX_PHNUM	128
+#define	MAX_SHNUM	32768
+#define SIZE_UNKNOWN	((off_t)-1)
 
 private int
 toomany(struct magic_set *ms, const char *name, uint16_t num)
 {
-	if (file_printf(ms, ", too many %s header sections (%u)", name, num
+	if (file_printf(ms, ", too many %s (%u)", name, num
 	    ) == -1)
 		return -1;
 	return 0;
@@ -292,15 +293,19 @@ private const char os_style_names[][8] = {
 	"NetBSD",
 };
 
-#define FLAGS_DID_CORE		0x01
-#define FLAGS_DID_NOTE		0x02
-#define FLAGS_DID_BUILD_ID	0x04
-#define FLAGS_DID_CORE_STYLE	0x08
-#define FLAGS_IS_CORE		0x10
+#define FLAGS_DID_CORE			0x001
+#define FLAGS_DID_OS_NOTE		0x002
+#define FLAGS_DID_BUILD_ID		0x004
+#define FLAGS_DID_CORE_STYLE		0x008
+#define FLAGS_DID_NETBSD_PAX		0x010
+#define FLAGS_DID_NETBSD_MARCH		0x020
+#define FLAGS_DID_NETBSD_CMODEL		0x040
+#define FLAGS_DID_NETBSD_UNKNOWN	0x080
+#define FLAGS_IS_CORE			0x100
 
 private int
 dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
-    int num, size_t size, off_t fsize, int *flags)
+    int num, size_t size, off_t fsize, int *flags, uint16_t *notecount)
 {
 	Elf32_Phdr ph32;
 	Elf64_Phdr ph64;
@@ -318,13 +323,13 @@ dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
 	 * Loop through all the program headers.
 	 */
 	for ( ; num; num--) {
-		if (pread(fd, xph_addr, xph_sizeof, off) == -1) {
+		if (pread(fd, xph_addr, xph_sizeof, off) < (ssize_t)xph_sizeof) {
 			file_badread(ms);
 			return -1;
 		}
 		off += size;
 
-		if (xph_offset > fsize) {
+		if (fsize != SIZE_UNKNOWN && xph_offset > fsize) {
 			/* Perhaps warn here */
 			continue;
 		}
@@ -346,7 +351,7 @@ dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
 			if (offset >= (size_t)bufsize)
 				break;
 			offset = donote(ms, nbuf, offset, (size_t)bufsize,
-			    clazz, swap, 4, flags);
+			    clazz, swap, 4, flags, notecount);
 			if (offset == 0)
 				break;
 
@@ -476,132 +481,127 @@ do_note_freebsd_version(struct magic_set *ms, int swap, void *v)
 	}
 }
 
-private size_t
-donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size,
-    int clazz, int swap, size_t align, int *flags)
+private int
+do_bid_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type,
+    int swap __attribute__((__unused__)), uint32_t namesz, uint32_t descsz,
+    size_t noff, size_t doff, int *flags)
 {
-	Elf32_Nhdr nh32;
-	Elf64_Nhdr nh64;
-	size_t noff, doff;
-#ifdef ELFCORE
-	int os_style = -1;
-#endif
-	uint32_t namesz, descsz;
-	unsigned char *nbuf = CAST(unsigned char *, vbuf);
-
-	if (xnh_sizeof + offset > size) {
-		/*
-		 * We're out of note headers.
-		 */
-		return xnh_sizeof + offset;
-	}
-
-	(void)memcpy(xnh_addr, &nbuf[offset], xnh_sizeof);
-	offset += xnh_sizeof;
-
-	namesz = xnh_namesz;
-	descsz = xnh_descsz;
-	if ((namesz == 0) && (descsz == 0)) {
-		/*
-		 * We're out of note headers.
-		 */
-		return (offset >= size) ? offset : size;
-	}
-
-	if (namesz & 0x80000000) {
-	    (void)file_printf(ms, ", bad note name size 0x%lx",
-		(unsigned long)namesz);
-	    return 0;
-	}
-
-	if (descsz & 0x80000000) {
-	    (void)file_printf(ms, ", bad note description size 0x%lx",
-		(unsigned long)descsz);
-	    return 0;
-	}
-
-
-	noff = offset;
-	doff = ELF_ALIGN(offset + namesz);
-
-	if (offset + namesz > size) {
-		/*
-		 * We're past the end of the buffer.
-		 */
-		return doff;
-	}
-
-	offset = ELF_ALIGN(doff + descsz);
-	if (doff + descsz > size) {
-		/*
-		 * We're past the end of the buffer.
-		 */
-		return (offset >= size) ? offset : size;
+	if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 &&
+	    type == NT_GNU_BUILD_ID && (descsz == 16 || descsz == 20)) {
+		uint8_t desc[20];
+		uint32_t i;
+		*flags |= FLAGS_DID_BUILD_ID;
+		if (file_printf(ms, ", BuildID[%s]=", descsz == 16 ? "md5/uuid" :
+		    "sha1") == -1)
+			return 1;
+		(void)memcpy(desc, &nbuf[doff], descsz);
+		for (i = 0; i < descsz; i++)
+		    if (file_printf(ms, "%02x", desc[i]) == -1)
+			return 1;
+		return 1;
 	}
+	return 0;
+}
 
-	if ((*flags & (FLAGS_DID_NOTE|FLAGS_DID_BUILD_ID)) ==
-	    (FLAGS_DID_NOTE|FLAGS_DID_BUILD_ID))
-		goto core;
-
+private int
+do_os_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type,
+    int swap, uint32_t namesz, uint32_t descsz,
+    size_t noff, size_t doff, int *flags)
+{
 	if (namesz == 5 && strcmp((char *)&nbuf[noff], "SuSE") == 0 &&
-	    xnh_type == NT_GNU_VERSION && descsz == 2) {
+	    type == NT_GNU_VERSION && descsz == 2) {
+	    *flags |= FLAGS_DID_OS_NOTE;
 	    file_printf(ms, ", for SuSE %d.%d", nbuf[doff], nbuf[doff + 1]);
+	    return 1;
 	}
+
 	if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 &&
-	    xnh_type == NT_GNU_VERSION && descsz == 16) {
+	    type == NT_GNU_VERSION && descsz == 16) {
 		uint32_t desc[4];
 		(void)memcpy(desc, &nbuf[doff], sizeof(desc));
 
+		*flags |= FLAGS_DID_OS_NOTE;
 		if (file_printf(ms, ", for GNU/") == -1)
-			return size;
+			return 1;
 		switch (elf_getu32(swap, desc[0])) {
 		case GNU_OS_LINUX:
 			if (file_printf(ms, "Linux") == -1)
-				return size;
+				return 1;
 			break;
 		case GNU_OS_HURD:
 			if (file_printf(ms, "Hurd") == -1)
-				return size;
+				return 1;
 			break;
 		case GNU_OS_SOLARIS:
 			if (file_printf(ms, "Solaris") == -1)
-				return size;
+				return 1;
 			break;
 		case GNU_OS_KFREEBSD:
 			if (file_printf(ms, "kFreeBSD") == -1)
-				return size;
+				return 1;
 			break;
 		case GNU_OS_KNETBSD:
 			if (file_printf(ms, "kNetBSD") == -1)
-				return size;
+				return 1;
 			break;
 		default:
 			if (file_printf(ms, "<unknown>") == -1)
-				return size; 
+				return 1; 
 		}
 		if (file_printf(ms, " %d.%d.%d", elf_getu32(swap, desc[1]),
 		    elf_getu32(swap, desc[2]), elf_getu32(swap, desc[3])) == -1)
-			return size;
-		*flags |= FLAGS_DID_NOTE;
-		return size;
+			return 1;
+		return 1;
 	}
 
-	if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 &&
-	    xnh_type == NT_GNU_BUILD_ID && (descsz == 16 || descsz == 20)) {
-	    uint8_t desc[20];
-	    uint32_t i;
-	    if (file_printf(ms, ", BuildID[%s]=", descsz == 16 ? "md5/uuid" :
-		"sha1") == -1)
-		    return size;
-	    (void)memcpy(desc, &nbuf[doff], descsz);
-	    for (i = 0; i < descsz; i++)
-		if (file_printf(ms, "%02x", desc[i]) == -1)
-		    return size;
-	    *flags |= FLAGS_DID_BUILD_ID;
+	if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) {
+	    	if (type == NT_NETBSD_VERSION && descsz == 4) {
+			*flags |= FLAGS_DID_OS_NOTE;
+			do_note_netbsd_version(ms, swap, &nbuf[doff]);
+			return 1;
+		}
+	}
+
+	if (namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0) {
+	    	if (type == NT_FREEBSD_VERSION && descsz == 4) {
+			*flags |= FLAGS_DID_OS_NOTE;
+			do_note_freebsd_version(ms, swap, &nbuf[doff]);
+			return 1;
+		}
 	}
 
+	if (namesz == 8 && strcmp((char *)&nbuf[noff], "OpenBSD") == 0 &&
+	    type == NT_OPENBSD_VERSION && descsz == 4) {
+		*flags |= FLAGS_DID_OS_NOTE;
+		if (file_printf(ms, ", for OpenBSD") == -1)
+			return 1;
+		/* Content of note is always 0 */
+		return 1;
+	}
+
+	if (namesz == 10 && strcmp((char *)&nbuf[noff], "DragonFly") == 0 &&
+	    type == NT_DRAGONFLY_VERSION && descsz == 4) {
+		uint32_t desc;
+		*flags |= FLAGS_DID_OS_NOTE;
+		if (file_printf(ms, ", for DragonFly") == -1)
+			return 1;
+		(void)memcpy(&desc, &nbuf[doff], sizeof(desc));
+		desc = elf_getu32(swap, desc);
+		if (file_printf(ms, " %d.%d.%d", desc / 100000,
+		    desc / 10000 % 10, desc % 10000) == -1)
+			return 1;
+		return 1;
+	}
+	return 0;
+}
+
+private int
+do_pax_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type,
+    int swap, uint32_t namesz, uint32_t descsz,
+    size_t noff, size_t doff, int *flags)
+{
 	if (namesz == 4 && strcmp((char *)&nbuf[noff], "PaX") == 0 &&
-	    xnh_type == NT_NETBSD_PAX && descsz == 4) {
+	    type == NT_NETBSD_PAX && descsz == 4) {
 		static const char *pax[] = {
 		    "+mprotect",
 		    "-mprotect",
@@ -614,80 +614,32 @@ donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size,
 		size_t i;
 		int did = 0;
 
+		*flags |= FLAGS_DID_NETBSD_PAX;
 		(void)memcpy(&desc, &nbuf[doff], sizeof(desc));
 		desc = elf_getu32(swap, desc);
 
 		if (desc && file_printf(ms, ", PaX: ") == -1)
-			return size;
+			return 1;
 
 		for (i = 0; i < __arraycount(pax); i++) {
 			if (((1 << i) & desc) == 0)
 				continue;
 			if (file_printf(ms, "%s%s", did++ ? "," : "",
 			    pax[i]) == -1)
-				return size;
-		}
-	}
-
-	if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) {
-		switch (xnh_type) {
-		case NT_NETBSD_VERSION:
-			if (descsz == 4) {
-				do_note_netbsd_version(ms, swap, &nbuf[doff]);
-				*flags |= FLAGS_DID_NOTE;
-				return size;
-			}
-			break;
-		case NT_NETBSD_MARCH:
-			if (file_printf(ms, ", compiled for: %.*s", (int)descsz,
-			    (const char *)&nbuf[doff]) == -1)
-				return size;
-			break;
-		case NT_NETBSD_CMODEL:
-			if (file_printf(ms, ", compiler model: %.*s",
-			    (int)descsz, (const char *)&nbuf[doff]) == -1)
-				return size;
-			break;
-		default:
-			if (file_printf(ms, ", note=%u", xnh_type) == -1)
-				return size;
-			break;
-		}
-		return size;
-	}
-
-	if (namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0) {
-	    	if (xnh_type == NT_FREEBSD_VERSION && descsz == 4) {
-			do_note_freebsd_version(ms, swap, &nbuf[doff]);
-			*flags |= FLAGS_DID_NOTE;
-			return size;
+				return 1;
 		}
+		return 1;
 	}
+	return 0;
+}
 
-	if (namesz == 8 && strcmp((char *)&nbuf[noff], "OpenBSD") == 0 &&
-	    xnh_type == NT_OPENBSD_VERSION && descsz == 4) {
-		if (file_printf(ms, ", for OpenBSD") == -1)
-			return size;
-		/* Content of note is always 0 */
-		*flags |= FLAGS_DID_NOTE;
-		return size;
-	}
-
-	if (namesz == 10 && strcmp((char *)&nbuf[noff], "DragonFly") == 0 &&
-	    xnh_type == NT_DRAGONFLY_VERSION && descsz == 4) {
-		uint32_t desc;
-		if (file_printf(ms, ", for DragonFly") == -1)
-			return size;
-		(void)memcpy(&desc, &nbuf[doff], sizeof(desc));
-		desc = elf_getu32(swap, desc);
-		if (file_printf(ms, " %d.%d.%d", desc / 100000,
-		    desc / 10000 % 10, desc % 10000) == -1)
-			return size;
-		*flags |= FLAGS_DID_NOTE;
-		return size;
-	}
-
-core:
+private int
+do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type,
+    int swap, uint32_t namesz, uint32_t descsz,
+    size_t noff, size_t doff, int *flags, size_t size, int clazz)
+{
+#ifdef ELFCORE
+	int os_style = -1;
 	/*
 	 * Sigh.  The 2.0.36 kernel in Debian 2.1, at
 	 * least, doesn't correctly implement name
@@ -716,20 +668,17 @@ core:
 		os_style = OS_STYLE_NETBSD;
 	}
 
-#ifdef ELFCORE
-	if ((*flags & FLAGS_DID_CORE) != 0)
-		return size;
-
 	if (os_style != -1 && (*flags & FLAGS_DID_CORE_STYLE) == 0) {
 		if (file_printf(ms, ", %s-style", os_style_names[os_style])
 		    == -1)
-			return size;
+			return 1;
 		*flags |= FLAGS_DID_CORE_STYLE;
 	}
 
 	switch (os_style) {
 	case OS_STYLE_NETBSD:
-		if (xnh_type == NT_NETBSD_CORE_PROCINFO) {
+		if (type == NT_NETBSD_CORE_PROCINFO) {
+			char sbuf[512];
 			uint32_t signo;
 			/*
 			 * Extract the program name.  It is at
@@ -737,8 +686,9 @@ core:
 			 * including the terminating NUL.
 			 */
 			if (file_printf(ms, ", from '%.31s'",
-			    &nbuf[doff + 0x7c]) == -1)
-				return size;
+			    file_printable(sbuf, sizeof(sbuf),
+			    (const char *)&nbuf[doff + 0x7c])) == -1)
+				return 1;
 			
 			/*
 			 * Extract the signal number.  It is at
@@ -748,14 +698,14 @@ core:
 			    sizeof(signo));
 			if (file_printf(ms, " (signal %u)",
 			    elf_getu32(swap, signo)) == -1)
-				return size;
+				return 1;
 			*flags |= FLAGS_DID_CORE;
-			return size;
+			return 1;
 		}
 		break;
 
 	default:
-		if (xnh_type == NT_PRPSINFO && *flags & FLAGS_IS_CORE) {
+		if (type == NT_PRPSINFO && *flags & FLAGS_IS_CORE) {
 			size_t i, j;
 			unsigned char c;
 			/*
@@ -823,7 +773,7 @@ core:
 				 * Try next offsets, in case this match is
 				 * in the middle of a string.
 				 */
-				for (k = i + 1 ; k < NOFFSETS ; k++) {
+				for (k = i + 1 ; k < NOFFSETS; k++) {
 					size_t no;
 					int adjust = 1;
 					if (prpsoffsets(k) >= prpsoffsets(i))
@@ -848,9 +798,9 @@ core:
 					cp--;
 				if (file_printf(ms, ", from '%.*s'",
 				    (int)(cp - cname), cname) == -1)
-					return size;
+					return 1;
 				*flags |= FLAGS_DID_CORE;
-				return size;
+				return 1;
 
 			tryanother:
 				;
@@ -859,6 +809,129 @@ core:
 		break;
 	}
 #endif
+	return 0;
+}
+
+private size_t
+donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size,
+    int clazz, int swap, size_t align, int *flags, uint16_t *notecount)
+{
+	Elf32_Nhdr nh32;
+	Elf64_Nhdr nh64;
+	size_t noff, doff;
+	uint32_t namesz, descsz;
+	unsigned char *nbuf = CAST(unsigned char *, vbuf);
+
+	if (*notecount == 0)
+		return 0;
+	--*notecount;
+
+	if (xnh_sizeof + offset > size) {
+		/*
+		 * We're out of note headers.
+		 */
+		return xnh_sizeof + offset;
+	}
+
+	(void)memcpy(xnh_addr, &nbuf[offset], xnh_sizeof);
+	offset += xnh_sizeof;
+
+	namesz = xnh_namesz;
+	descsz = xnh_descsz;
+	if ((namesz == 0) && (descsz == 0)) {
+		/*
+		 * We're out of note headers.
+		 */
+		return (offset >= size) ? offset : size;
+	}
+
+	if (namesz & 0x80000000) {
+	    (void)file_printf(ms, ", bad note name size 0x%lx",
+		(unsigned long)namesz);
+	    return 0;
+	}
+
+	if (descsz & 0x80000000) {
+	    (void)file_printf(ms, ", bad note description size 0x%lx",
+		(unsigned long)descsz);
+	    return 0;
+	}
+
+	noff = offset;
+	doff = ELF_ALIGN(offset + namesz);
+
+	if (offset + namesz > size) {
+		/*
+		 * We're past the end of the buffer.
+		 */
+		return doff;
+	}
+
+	offset = ELF_ALIGN(doff + descsz);
+	if (doff + descsz > size) {
+		/*
+		 * We're past the end of the buffer.
+		 */
+		return (offset >= size) ? offset : size;
+	}
+
+	if ((*flags & FLAGS_DID_OS_NOTE) == 0) {
+		if (do_os_note(ms, nbuf, xnh_type, swap,
+		    namesz, descsz, noff, doff, flags))
+			return size;
+	}
+
+	if ((*flags & FLAGS_DID_BUILD_ID) == 0) {
+		if (do_bid_note(ms, nbuf, xnh_type, swap,
+		    namesz, descsz, noff, doff, flags))
+			return size;
+	}
+		
+	if ((*flags & FLAGS_DID_NETBSD_PAX) == 0) {
+		if (do_pax_note(ms, nbuf, xnh_type, swap,
+		    namesz, descsz, noff, doff, flags))
+			return size;
+	}
+
+	if ((*flags & FLAGS_DID_CORE) == 0) {
+		if (do_core_note(ms, nbuf, xnh_type, swap,
+		    namesz, descsz, noff, doff, flags, size, clazz))
+			return size;
+	}
+
+	if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) {
+		if (descsz > 100)
+			descsz = 100;
+		switch (xnh_type) {
+	    	case NT_NETBSD_VERSION:
+			return size;
+		case NT_NETBSD_MARCH:
+			if (*flags & FLAGS_DID_NETBSD_MARCH)
+				return size;
+			*flags |= FLAGS_DID_NETBSD_MARCH;
+			if (file_printf(ms, ", compiled for: %.*s",
+			    (int)descsz, (const char *)&nbuf[doff]) == -1)
+				return size;
+			break;
+		case NT_NETBSD_CMODEL:
+			if (*flags & FLAGS_DID_NETBSD_CMODEL)
+				return size;
+			*flags |= FLAGS_DID_NETBSD_CMODEL;
+			if (file_printf(ms, ", compiler model: %.*s",
+			    (int)descsz, (const char *)&nbuf[doff]) == -1)
+				return size;
+			break;
+		default:
+			if (*flags & FLAGS_DID_NETBSD_UNKNOWN)
+				return size;
+			*flags |= FLAGS_DID_NETBSD_UNKNOWN;
+			if (file_printf(ms, ", note=%u", xnh_type) == -1)
+				return size;
+			break;
+		}
+		return size;
+	}
+
 	return offset;
 }
 
@@ -914,7 +987,8 @@ static const cap_desc_t cap_desc_386[] = {
 
 private int
 doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
-    size_t size, off_t fsize, int *flags, int mach, int strtab)
+    size_t size, off_t fsize, int mach, int strtab, int *flags,
+    uint16_t *notecount)
 {
 	Elf32_Shdr sh32;
 	Elf64_Shdr sh64;
@@ -925,6 +999,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
 	uint64_t cap_hw1 = 0;	/* SunOS 5.x hardware capabilites */
 	uint64_t cap_sf1 = 0;	/* SunOS 5.x software capabilites */
 	char name[50];
+	ssize_t namesize;
 
 	if (size != xsh_sizeof) {
 		if (file_printf(ms, ", corrupted section header size") == -1)
@@ -933,7 +1008,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
 	}
 
 	/* Read offset of name section to be able to read section names later */
-	if (pread(fd, xsh_addr, xsh_sizeof, off + size * strtab) == -1) {
+	if (pread(fd, xsh_addr, xsh_sizeof, off + size * strtab) < (ssize_t)xsh_sizeof) {
 		file_badread(ms);
 		return -1;
 	}
@@ -941,15 +1016,15 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
 
 	for ( ; num; num--) {
 		/* Read the name of this section. */
-		if (pread(fd, name, sizeof(name), name_off + xsh_name) == -1) {
+		if ((namesize = pread(fd, name, sizeof(name) - 1, name_off + xsh_name)) == -1) {
 			file_badread(ms);
 			return -1;
 		}
-		name[sizeof(name) - 1] = '\0';
+		name[namesize] = '\0';
 		if (strcmp(name, ".debug_info") == 0)
 			stripped = 0;
 
-		if (pread(fd, xsh_addr, xsh_sizeof, off) == -1) {
+		if (pread(fd, xsh_addr, xsh_sizeof, off) < (ssize_t)xsh_sizeof) {
 			file_badread(ms);
 			return -1;
 		}
@@ -964,7 +1039,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
 			stripped = 0;
 			break;
 		default:
-			if (xsh_offset > fsize) {
+			if (fsize != SIZE_UNKNOWN && xsh_offset > fsize) {
 				/* Perhaps warn here */
 				continue;
 			}
@@ -979,7 +1054,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
 				    " for note");
 				return -1;
 			}
-			if (pread(fd, nbuf, xsh_size, xsh_offset) == -1) {
+			if (pread(fd, nbuf, xsh_size, xsh_offset) < (ssize_t)xsh_size) {
 				file_badread(ms);
 				free(nbuf);
 				return -1;
@@ -990,7 +1065,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
 				if (noff >= (off_t)xsh_size)
 					break;
 				noff = donote(ms, nbuf, (size_t)noff,
-				    xsh_size, clazz, swap, 4, flags);
+				    xsh_size, clazz, swap, 4, flags, notecount);
 				if (noff == 0)
 					break;
 			}
@@ -1157,13 +1232,15 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num,
  */
 private int
 dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
-    int num, size_t size, off_t fsize, int *flags, int sh_num)
+    int num, size_t size, off_t fsize, int sh_num, int *flags,
+    uint16_t *notecount)
 {
 	Elf32_Phdr ph32;
 	Elf64_Phdr ph64;
 	const char *linking_style = "statically";
-	const char *shared_libraries = "";
+	const char *interp = "";
 	unsigned char nbuf[BUFSIZ];
+	char ibuf[BUFSIZ];
 	ssize_t bufsize;
 	size_t offset, align, len;
 	
@@ -1174,23 +1251,43 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
 	}
 
   	for ( ; num; num--) {
-		if (pread(fd, xph_addr, xph_sizeof, off) == -1) {
+		if (pread(fd, xph_addr, xph_sizeof, off) < (ssize_t)xph_sizeof) {
 			file_badread(ms);
 			return -1;
 		}
 
 		off += size;
+		bufsize = 0;
+		align = 4;
 
 		/* Things we can determine before we seek */
 		switch (xph_type) {
 		case PT_DYNAMIC:
 			linking_style = "dynamically";
 			break;
+		case PT_NOTE:
+			if (sh_num)	/* Did this through section headers */
+				continue;
+			if (((align = xph_align) & 0x80000000UL) != 0 ||
+			    align < 4) {
+				if (file_printf(ms, 
+				    ", invalid note alignment 0x%lx",
+				    (unsigned long)align) == -1)
+					return -1;
+				align = 4;
+			}
+			/*FALLTHROUGH*/
 		case PT_INTERP:
-			shared_libraries = " (uses shared libs)";
+			len = xph_filesz < sizeof(nbuf) ? xph_filesz
+			    : sizeof(nbuf);
+			bufsize = pread(fd, nbuf, len, xph_offset);
+			if (bufsize == -1) {
+				file_badread(ms);
+				return -1;
+			}
 			break;
 		default:
-			if (xph_offset > fsize) {
+			if (fsize != SIZE_UNKNOWN && xph_offset > fsize) {
 				/* Maybe warn here? */
 				continue;
 			}
@@ -1199,34 +1296,25 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
 
 		/* Things we can determine when we seek */
 		switch (xph_type) {
+		case PT_INTERP:
+			if (bufsize && nbuf[0]) {
+				nbuf[bufsize - 1] = '\0';
+				interp = (const char *)nbuf;
+			} else
+				interp = "*empty*";
+			break;
 		case PT_NOTE:
-			if ((align = xph_align) & 0x80000000UL) {
-				if (file_printf(ms, 
-				    ", invalid note alignment 0x%lx",
-				    (unsigned long)align) == -1)
-					return -1;
-				align = 4;
-			}
-			if (sh_num)
-				break;
 			/*
 			 * This is a PT_NOTE section; loop through all the notes
 			 * in the section.
 			 */
-			len = xph_filesz < sizeof(nbuf) ? xph_filesz
-			    : sizeof(nbuf);
-			bufsize = pread(fd, nbuf, len, xph_offset);
-			if (bufsize == -1) {
-				file_badread(ms);
-				return -1;
-			}
 			offset = 0;
 			for (;;) {
 				if (offset >= (size_t)bufsize)
 					break;
 				offset = donote(ms, nbuf, offset,
 				    (size_t)bufsize, clazz, swap, align,
-				    flags);
+				    flags, notecount);
 				if (offset == 0)
 					break;
 			}
@@ -1235,9 +1323,13 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off,
 			break;
 		}
 	}
-	if (file_printf(ms, ", %s linked%s", linking_style, shared_libraries)
+	if (file_printf(ms, ", %s linked", linking_style)
 	    == -1)
-	    return -1;
+		return -1;
+	if (interp[0])
+		if (file_printf(ms, ", interpreter %s",
+		    file_printable(ibuf, sizeof(ibuf), interp)) == -1)
+			return -1;
 	return 0;
 }
 
@@ -1257,7 +1349,7 @@ file_tryelf(struct magic_set *ms, int fd, const unsigned char *buf,
 	int flags = 0;
 	Elf32_Ehdr elf32hdr;
 	Elf64_Ehdr elf64hdr;
-	uint16_t type, phnum, shnum;
+	uint16_t type, phnum, shnum, notecount;
 
 	if (ms->flags & (MAGIC_MIME|MAGIC_APPLE))
 		return 0;
@@ -1283,7 +1375,10 @@ file_tryelf(struct magic_set *ms, int fd, const unsigned char *buf,
   		file_badread(ms);
 		return -1;
 	}
-	fsize = st.st_size;
+	if (S_ISREG(st.st_mode) || st.st_size != 0)
+		fsize = st.st_size;
+	else
+		fsize = SIZE_UNKNOWN;
 
 	clazz = buf[EI_CLASS];
 
diff --git a/contrib/file/src/softmagic.c b/contrib/file/src/softmagic.c
index b0f193d..5e277e3 100644
--- a/contrib/file/src/softmagic.c
+++ b/contrib/file/src/softmagic.c
@@ -32,7 +32,7 @@
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: softmagic.c,v 1.191 2014/06/04 17:36:34 christos Exp $")
+FILE_RCSID("@(#)$File: softmagic.c,v 1.206 2015/01/01 17:07:34 christos Exp $")
 #endif	/* lint */
 
 #include "magic.h"
@@ -41,17 +41,13 @@ FILE_RCSID("@(#)$File: softmagic.c,v 1.191 2014/06/04 17:36:34 christos Exp $")
 #include <ctype.h>
 #include <stdlib.h>
 #include <time.h>
-#if defined(HAVE_LOCALE_H)
-#include <locale.h>
-#endif
-
 
 private int match(struct magic_set *, struct magic *, uint32_t,
-    const unsigned char *, size_t, size_t, int, int, int, int, int *, int *,
-    int *);
+    const unsigned char *, size_t, size_t, int, int, int, uint16_t,
+    uint16_t *, int *, int *, int *);
 private int mget(struct magic_set *, const unsigned char *,
-    struct magic *, size_t, size_t, unsigned int, int, int, int, int, int *,
-    int *, int *);
+    struct magic *, size_t, size_t, unsigned int, int, int, int, uint16_t,
+    uint16_t *, int *, int *, int *);
 private int magiccheck(struct magic_set *, struct magic *);
 private int32_t mprint(struct magic_set *, struct magic *);
 private int32_t moffset(struct magic_set *, struct magic *);
@@ -68,8 +64,6 @@ private void cvt_64(union VALUETYPE *, const struct magic *);
 
 #define OFFSET_OOB(n, o, i)	((n) < (o) || (i) > ((n) - (o)))
 
-#define MAX_RECURSION_LEVEL	10
-
 /*
  * softmagic - lookup one file in parsed, in-memory copy of database
  * Passed the name and FILE * of one file to be typed.
@@ -77,14 +71,21 @@ private void cvt_64(union VALUETYPE *, const struct magic *);
 /*ARGSUSED1*/		/* nbytes passed for regularity, maybe need later */
 protected int
 file_softmagic(struct magic_set *ms, const unsigned char *buf, size_t nbytes,
-    size_t level, int mode, int text)
+    uint16_t indir_level, uint16_t *name_count, int mode, int text)
 {
 	struct mlist *ml;
 	int rv, printed_something = 0, need_separator = 0;
+	uint16_t nc;
+
+	if (name_count == NULL) {
+		nc = 0;
+		name_count = &nc;
+	}
+
 	for (ml = ms->mlist[0]->next; ml != ms->mlist[0]; ml = ml->next)
 		if ((rv = match(ms, ml->magic, ml->nmagic, buf, nbytes, 0, mode,
-		    text, 0, level, &printed_something, &need_separator,
-		    NULL)) != 0)
+		    text, 0, indir_level, name_count,
+		    &printed_something, &need_separator, NULL)) != 0)
 			return rv;
 
 	return 0;
@@ -101,8 +102,8 @@ file_fmtcheck(struct magic_set *ms, const struct magic *m, const char *def,
 	const char *ptr = fmtcheck(m->desc, def);
 	if (ptr == def)
 		file_magerror(ms,
-		    "%s, %zu: format `%s' does not match with `%s'",
-		    file, line, m->desc, def);
+		    "%s, %" SIZE_T_FORMAT "u: format `%s' does not match"
+		    " with `%s'", file, line, m->desc, def);
 	return ptr;
 }
 #else
@@ -139,8 +140,8 @@ file_fmtcheck(struct magic_set *ms, const struct magic *m, const char *def,
 private int
 match(struct magic_set *ms, struct magic *magic, uint32_t nmagic,
     const unsigned char *s, size_t nbytes, size_t offset, int mode, int text,
-    int flip, int recursion_level, int *printed_something, int *need_separator,
-    int *returnval)
+    int flip, uint16_t indir_level, uint16_t *name_count,
+    int *printed_something, int *need_separator, int *returnval)
 {
 	uint32_t magindex = 0;
 	unsigned int cont_level = 0;
@@ -177,8 +178,8 @@ match(struct magic_set *ms, struct magic *magic, uint32_t nmagic,
 
 		/* if main entry matches, print it... */
 		switch (mget(ms, s, m, nbytes, offset, cont_level, mode, text,
-		    flip, recursion_level + 1, printed_something,
-		    need_separator, returnval)) {
+		    flip, indir_level, name_count,
+		    printed_something, need_separator, returnval)) {
 		case -1:
 			return -1;
 		case 0:
@@ -238,9 +239,9 @@ match(struct magic_set *ms, struct magic *magic, uint32_t nmagic,
 		if (file_check_mem(ms, ++cont_level) == -1)
 			return -1;
 
-		while (++magindex < nmagic &&
-		    magic[magindex].cont_level != 0) {
-			m = &magic[magindex];
+		while (magindex + 1 < nmagic &&
+		    magic[magindex + 1].cont_level != 0) {
+			m = &magic[++magindex];
 			ms->line = m->lineno; /* for messages */
 
 			if (cont_level < m->cont_level)
@@ -266,8 +267,8 @@ match(struct magic_set *ms, struct magic *magic, uint32_t nmagic,
 			}
 #endif
 			switch (mget(ms, s, m, nbytes, offset, cont_level, mode,
-			    text, flip, recursion_level + 1, printed_something,
-			    need_separator, returnval)) {
+			    text, flip, indir_level, name_count,
+			    printed_something, need_separator, returnval)) {
 			case -1:
 				return -1;
 			case 0:
@@ -410,7 +411,7 @@ mprint(struct magic_set *ms, struct magic *m)
 	float vf;
 	double vd;
 	int64_t t = 0;
- 	char buf[128], tbuf[26];
+ 	char buf[128], tbuf[26], sbuf[512];
 	union VALUETYPE *p = &ms->ms_value;
 
   	switch (m->type) {
@@ -504,7 +505,9 @@ mprint(struct magic_set *ms, struct magic *m)
   	case FILE_BESTRING16:
   	case FILE_LESTRING16:
 		if (m->reln == '=' || m->reln == '!') {
-			if (file_printf(ms, F(ms, m, "%s"), m->value.s) == -1)
+			if (file_printf(ms, F(ms, m, "%s"), 
+			    file_printable(sbuf, sizeof(sbuf), m->value.s))
+			    == -1)
 				return -1;
 			t = ms->offset + m->vallen;
 		}
@@ -530,7 +533,8 @@ mprint(struct magic_set *ms, struct magic *m)
 				*++last = '\0';
 			}
 
-			if (file_printf(ms, F(ms, m, "%s"), str) == -1)
+			if (file_printf(ms, F(ms, m, "%s"),
+			    file_printable(sbuf, sizeof(sbuf), str)) == -1)
 				return -1;
 
 			if (m->type == FILE_PSTRING)
@@ -634,7 +638,8 @@ mprint(struct magic_set *ms, struct magic *m)
 			file_oomem(ms, ms->search.rm_len);
 			return -1;
 		}
-		rval = file_printf(ms, F(ms, m, "%s"), cp);
+		rval = file_printf(ms, F(ms, m, "%s"),
+		    file_printable(sbuf, sizeof(sbuf), cp));
 		free(cp);
 
 		if (rval == -1)
@@ -648,7 +653,8 @@ mprint(struct magic_set *ms, struct magic *m)
 	}
 
 	case FILE_SEARCH:
-	  	if (file_printf(ms, F(ms, m, "%s"), m->value.s) == -1)
+	  	if (file_printf(ms, F(ms, m, "%s"),
+		    file_printable(sbuf, sizeof(sbuf), m->value.s)) == -1)
 			return -1;
 		if ((m->str_flags & REGEX_OFFSET_START))
 			t = ms->search.offset;
@@ -946,14 +952,17 @@ mconvert(struct magic_set *ms, struct magic *m, int flip)
 		size_t sz = file_pstring_length_size(m);
 		char *ptr1 = p->s, *ptr2 = ptr1 + sz;
 		size_t len = file_pstring_get_length(m, ptr1);
-		if (len >= sizeof(p->s)) {
+		sz = sizeof(p->s) - sz; /* maximum length of string */
+		if (len >= sz) {
 			/*
 			 * The size of the pascal string length (sz)
 			 * is 1, 2, or 4. We need at least 1 byte for NUL
 			 * termination, but we've already truncated the
 			 * string by p->s, so we need to deduct sz.
+			 * Because we can use one of the bytes of the length
+			 * after we shifted as NUL termination.
 			 */ 
-			len = sizeof(p->s) - sz;
+			len = sz;
 		}
 		while (len--)
 			*ptr1++ = *ptr2++;
@@ -1063,7 +1072,7 @@ mconvert(struct magic_set *ms, struct magic *m, int flip)
 private void
 mdebug(uint32_t offset, const char *str, size_t len)
 {
-	(void) fprintf(stderr, "mget/%zu @%d: ", len, offset);
+	(void) fprintf(stderr, "mget/%" SIZE_T_FORMAT "u @%d: ", len, offset);
 	file_showstr(stderr, str, len);
 	(void) fputc('\n', stderr);
 	(void) fputc('\n', stderr);
@@ -1193,8 +1202,8 @@ mcopy(struct magic_set *ms, union VALUETYPE *p, int type, int indir,
 private int
 mget(struct magic_set *ms, const unsigned char *s, struct magic *m,
     size_t nbytes, size_t o, unsigned int cont_level, int mode, int text,
-    int flip, int recursion_level, int *printed_something,
-    int *need_separator, int *returnval)
+    int flip, uint16_t indir_level, uint16_t *name_count,
+    int *printed_something, int *need_separator, int *returnval)
 {
 	uint32_t offset = ms->offset;
 	uint32_t lhs;
@@ -1204,8 +1213,15 @@ mget(struct magic_set *ms, const unsigned char *s, struct magic *m,
 	union VALUETYPE *p = &ms->ms_value;
 	struct mlist ml;
 
-	if (recursion_level >= MAX_RECURSION_LEVEL) {
-		file_error(ms, 0, "recursion nesting exceeded");
+	if (indir_level >= ms->indir_max) {
+		file_error(ms, 0, "indirect recursion nesting (%hu) exceeded",
+		    indir_level);
+		return -1;
+	}
+
+	if (*name_count >= ms->name_max) {
+		file_error(ms, 0, "name use count (%hu) exceeded",
+		    *name_count);
 		return -1;
 	}
 
@@ -1214,8 +1230,11 @@ mget(struct magic_set *ms, const unsigned char *s, struct magic *m,
 		return -1;
 
 	if ((ms->flags & MAGIC_DEBUG) != 0) {
-		fprintf(stderr, "mget(type=%d, flag=%x, offset=%u, o=%zu, "
-		    "nbytes=%zu)\n", m->type, m->flag, offset, o, nbytes);
+		fprintf(stderr, "mget(type=%d, flag=%x, offset=%u, o=%"
+		    SIZE_T_FORMAT "u, " "nbytes=%" SIZE_T_FORMAT
+		    "u, il=%hu, nc=%hu)\n",
+		    m->type, m->flag, offset, o, nbytes,
+		    indir_level, *name_count);
 		mdebug(offset, (char *)(void *)p, sizeof(union VALUETYPE));
 #ifndef COMPILE_ONLY
 		file_mdump(m);
@@ -1646,6 +1665,8 @@ mget(struct magic_set *ms, const unsigned char *s, struct magic *m,
 		break;
 
 	case FILE_INDIRECT:
+		if (m->str_flags & INDIRECT_RELATIVE)
+			offset += o;
 		if (offset == 0)
 			return 0;
 
@@ -1656,7 +1677,7 @@ mget(struct magic_set *ms, const unsigned char *s, struct magic *m,
 			return -1;
 
 		rv = file_softmagic(ms, s + offset, nbytes - offset,
-		    recursion_level, BINTEST, text);
+		    indir_level + 1, name_count, BINTEST, text);
 
 		if ((ms->flags & MAGIC_DEBUG) != 0)
 			fprintf(stderr, "indirect @offs=%u[%d]\n", offset, rv);
@@ -1691,13 +1712,13 @@ mget(struct magic_set *ms, const unsigned char *s, struct magic *m,
 			file_error(ms, 0, "cannot find entry `%s'", rbuf);
 			return -1;
 		}
-
+		(*name_count)++;
 		oneed_separator = *need_separator;
 		if (m->flag & NOSPACE)
 			*need_separator = 0;
 		rv = match(ms, ml.magic, ml.nmagic, s, nbytes, offset + o,
-		    mode, text, flip, recursion_level, printed_something,
-		    need_separator, returnval);
+		    mode, text, flip, indir_level, name_count,
+		    printed_something, need_separator, returnval);
 		if (rv != 1)
 		    *need_separator = oneed_separator;
 		return rv;
@@ -1952,6 +1973,7 @@ magiccheck(struct magic_set *ms, struct magic *m)
 	case FILE_REGEX: {
 		int rc;
 		file_regex_t rx;
+		const char *search;
 
 		if (ms->search.s == NULL)
 			return 0;
@@ -1968,19 +1990,31 @@ magiccheck(struct magic_set *ms, struct magic *m)
 			size_t slen = ms->search.s_len;
 #ifndef REG_STARTEND
 #define	REG_STARTEND	0
-			char c;
-			if (slen != 0)
-				slen--;
-			c = ms->search.s[slen];
-			((char *)(intptr_t)ms->search.s)[slen] = '\0';
+			char *copy;
+			if (slen != 0) {
+			    copy = malloc(slen);
+			    if (copy == NULL)  {
+				file_error(ms, errno,
+				    "can't allocate %" SIZE_T_FORMAT "u bytes",
+				    slen);
+				return -1;
+			    }
+			    memcpy(copy, ms->search.s, slen);
+			    copy[--slen] = '\0';
+			    search = copy;
+			} else {
+			    search = ms->search.s;
+			    copy = NULL;
+			}
 #else
+			search = ms->search.s;
 			pmatch[0].rm_so = 0;
 			pmatch[0].rm_eo = slen;
 #endif
-			rc = file_regexec(&rx, (const char *)ms->search.s,
+			rc = file_regexec(&rx, (const char *)search,
 			    1, pmatch, REG_STARTEND);
 #if REG_STARTEND == 0
-			((char *)(intptr_t)ms->search.s)[l] = c;
+			free(copy);
 #endif
 			switch (rc) {
 			case 0:
diff --git a/contrib/file/src/vasprintf.c b/contrib/file/src/vasprintf.c
index cc43ae3..7a18bed 100644
--- a/contrib/file/src/vasprintf.c
+++ b/contrib/file/src/vasprintf.c
@@ -108,7 +108,7 @@ you use strange formats.
 #include "file.h"
 
 #ifndef	lint
-FILE_RCSID("@(#)$File: vasprintf.c,v 1.12 2014/05/14 23:09:21 christos Exp $")
+FILE_RCSID("@(#)$File: vasprintf.c,v 1.13 2014/12/04 15:56:46 christos Exp $")
 #endif	/* lint */
 
 #include <assert.h>
@@ -633,11 +633,15 @@ int vasprintf(char **ptr, const char *format_string, va_list vargs)
 #ifdef va_copy
   va_copy (s.vargs, vargs);
 #else
-#ifdef __va_copy
+# ifdef __va_copy
   __va_copy (s.vargs, vargs);
-#else
+# else
+#  ifdef WIN32
+  s.vargs = vargs;
+#  else
   memcpy (&s.vargs, &vargs, sizeof (s.va_args));
-#endif /* __va_copy */
+#  endif /* WIN32 */
+# endif /* __va_copy */
 #endif /* va_copy */
   s.maxlen = (size_t)INT_MAX;
 
diff --git a/lib/libmagic/config.h b/lib/libmagic/config.h
index d80926d..47f3347 100644
--- a/lib/libmagic/config.h
+++ b/lib/libmagic/config.h
@@ -1,5 +1,13 @@
 /* $FreeBSD$ */
 
+/* Get __FreeBSD_version. */
+#include <osreldate.h>
+
+/* Only specific versions of FreeBSD support xlocale */
+#if __FreeBSD_version >= 1000002 || (__FreeBSD_version < 1000000 && __FreeBSD_version >= 900506)
+#define FREEBSD_XLOCALE_SUPPORT 1
+#endif
+
 /* config.h.  Generated from config.h.in by configure.  */
 /* config.h.in.  Generated from configure.ac by autoheader.  */
 
@@ -47,6 +55,11 @@
 /* Define to 1 if you have the `fork' function. */
 #define HAVE_FORK 1
 
+/* Define to 1 if you have the `freelocale' function. */
+#ifdef FREEBSD_XLOCALE_SUPPORT
+#define HAVE_FREELOCALE 1
+#endif
+
 /* Define to 1 if fseeko (and presumably ftello) exists and is declared. */
 #define HAVE_FSEEKO 1
 
@@ -98,9 +111,17 @@
 /* Define to 1 if you have a working `mmap' system call. */
 #define HAVE_MMAP 1
 
+/* Define to 1 if you have the `newlocale' function. */
+#ifdef FREEBSD_XLOCALE_SUPPORT
+#define HAVE_NEWLOCALE 1
+#endif
+
 /* Define to 1 if you have the `pread' function. */
 #define HAVE_PREAD 1
 
+/* Define to 1 if you have the `setlocale' function. */
+#define HAVE_SETLOCALE 1
+
 /* Define to 1 if you have the <stddef.h> header file. */
 #define HAVE_STDDEF_H 1
 
@@ -185,6 +206,11 @@
 /* Define to 1 if you have the <unistd.h> header file. */
 #define HAVE_UNISTD_H 1
 
+/* Define to 1 if you have the `uselocale' function. */
+#ifdef FREEBSD_XLOCALE_SUPPORT
+#define HAVE_USELOCALE 1
+#endif
+
 /* Define to 1 if you have the `utime' function. */
 #define HAVE_UTIME 1
 
@@ -222,6 +248,11 @@
 /* Define to 1 if `vfork' works. */
 #define HAVE_WORKING_VFORK 1
 
+/* Define to 1 if you have the <xlocale.h> header file. */
+#ifdef FREEBSD_XLOCALE_SUPPORT
+#define HAVE_XLOCALE_H 1
+#endif
+
 /* Define to 1 if you have the <zlib.h> header file. */
 #define HAVE_ZLIB_H 1
 
@@ -247,7 +278,7 @@
 #define PACKAGE_NAME "file"
 
 /* Define to the full name and version of this package. */
-#define PACKAGE_STRING "file 5.19"
+#define PACKAGE_STRING "file 5.22"
 
 /* Define to the one symbol short name of this package. */
 #define PACKAGE_TARNAME "file"
@@ -256,7 +287,7 @@
 #define PACKAGE_URL ""
 
 /* Define to the version of this package. */
-#define PACKAGE_VERSION "5.19"
+#define PACKAGE_VERSION "5.22"
 
 /* Define to 1 if you have the ANSI C header files. */
 #define STDC_HEADERS 1
@@ -287,7 +318,7 @@
 
 
 /* Version number of package */
-#define VERSION "5.19"
+#define VERSION "5.22"
 
 /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
    significant byte first (like Motorola and SPARC, unlike Intel). */
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
index defddfe..dd552bc 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
@@ -344,6 +344,7 @@ typedef struct arc_stats {
 	kstat_named_t arcstat_l2_evict_lock_retry;
 	kstat_named_t arcstat_l2_evict_reading;
 	kstat_named_t arcstat_l2_free_on_write;
+	kstat_named_t arcstat_l2_cdata_free_on_write;
 	kstat_named_t arcstat_l2_abort_lowmem;
 	kstat_named_t arcstat_l2_cksum_bad;
 	kstat_named_t arcstat_l2_io_error;
@@ -421,6 +422,7 @@ static arc_stats_t arc_stats = {
 	{ "l2_evict_lock_retry",	KSTAT_DATA_UINT64 },
 	{ "l2_evict_reading",		KSTAT_DATA_UINT64 },
 	{ "l2_free_on_write",		KSTAT_DATA_UINT64 },
+	{ "l2_cdata_free_on_write",	KSTAT_DATA_UINT64 },
 	{ "l2_abort_lowmem",		KSTAT_DATA_UINT64 },
 	{ "l2_cksum_bad",		KSTAT_DATA_UINT64 },
 	{ "l2_io_error",		KSTAT_DATA_UINT64 },
@@ -1629,6 +1631,21 @@ arc_buf_add_ref(arc_buf_t *buf, void* tag)
 	    data, metadata, hits);
 }
 
+static void
+arc_buf_free_on_write(void *data, size_t size,
+    void (*free_func)(void *, size_t))
+{
+	l2arc_data_free_t *df;
+
+	df = kmem_alloc(sizeof (l2arc_data_free_t), KM_SLEEP);
+	df->l2df_data = data;
+	df->l2df_size = size;
+	df->l2df_func = free_func;
+	mutex_enter(&l2arc_free_on_write_mtx);
+	list_insert_head(l2arc_free_on_write, df);
+	mutex_exit(&l2arc_free_on_write_mtx);
+}
+
 /*
  * Free the arc data buffer.  If it is an l2arc write in progress,
  * the buffer is placed on l2arc_free_on_write to be freed later.
@@ -1639,14 +1656,7 @@ arc_buf_data_free(arc_buf_t *buf, void (*free_func)(void *, size_t))
 	arc_buf_hdr_t *hdr = buf->b_hdr;
 
 	if (HDR_L2_WRITING(hdr)) {
-		l2arc_data_free_t *df;
-		df = kmem_alloc(sizeof (l2arc_data_free_t), KM_SLEEP);
-		df->l2df_data = buf->b_data;
-		df->l2df_size = hdr->b_size;
-		df->l2df_func = free_func;
-		mutex_enter(&l2arc_free_on_write_mtx);
-		list_insert_head(l2arc_free_on_write, df);
-		mutex_exit(&l2arc_free_on_write_mtx);
+		arc_buf_free_on_write(buf->b_data, hdr->b_size, free_func);
 		ARCSTAT_BUMP(arcstat_l2_free_on_write);
 	} else {
 		free_func(buf->b_data, hdr->b_size);
@@ -1658,6 +1668,23 @@ arc_buf_data_free(arc_buf_t *buf, void (*free_func)(void *, size_t))
  * arc_buf_t off of the the arc_buf_hdr_t's list and free it.
  */
 static void
+arc_buf_l2_cdata_free(arc_buf_hdr_t *hdr)
+{
+	l2arc_buf_hdr_t *l2hdr = hdr->b_l2hdr;
+
+	ASSERT(MUTEX_HELD(&l2arc_buflist_mtx));
+
+	if (l2hdr->b_tmp_cdata == NULL)
+		return;
+
+	ASSERT(HDR_L2_WRITING(hdr));
+	arc_buf_free_on_write(l2hdr->b_tmp_cdata, hdr->b_size,
+	    zio_data_buf_free);
+	ARCSTAT_BUMP(arcstat_l2_cdata_free_on_write);
+	l2hdr->b_tmp_cdata = NULL;
+}
+
+static void
 arc_buf_destroy(arc_buf_t *buf, boolean_t recycle, boolean_t remove)
 {
 	arc_buf_t **bufp;
@@ -1756,6 +1783,7 @@ arc_hdr_destroy(arc_buf_hdr_t *hdr)
 			trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr,
 			    hdr->b_size, 0);
 			list_remove(l2hdr->b_dev->l2ad_buflist, hdr);
+			arc_buf_l2_cdata_free(hdr);
 			ARCSTAT_INCR(arcstat_l2_size, -hdr->b_size);
 			ARCSTAT_INCR(arcstat_l2_asize, -l2hdr->b_asize);
 			vdev_space_update(l2hdr->b_dev->l2ad_vdev,
@@ -3605,6 +3633,7 @@ arc_release(arc_buf_t *buf, void *tag)
 	l2hdr = hdr->b_l2hdr;
 	if (l2hdr) {
 		mutex_enter(&l2arc_buflist_mtx);
+		arc_buf_l2_cdata_free(hdr);
 		hdr->b_l2hdr = NULL;
 		list_remove(l2hdr->b_dev->l2ad_buflist, hdr);
 	}
@@ -4895,6 +4924,11 @@ top:
 				ARCSTAT_INCR(arcstat_l2_asize, -abl2->b_asize);
 				bytes_evicted += abl2->b_asize;
 				ab->b_l2hdr = NULL;
+				/*
+				 * We are destroying l2hdr, so ensure that
+				 * its compressed buffer, if any, is not leaked.
+				 */
+				ASSERT(abl2->b_tmp_cdata == NULL);
 				kmem_free(abl2, sizeof (l2arc_buf_hdr_t));
 				ARCSTAT_INCR(arcstat_l2_size, -ab->b_size);
 			}
@@ -5133,6 +5167,14 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint64_t target_sz,
 		buf_data = l2hdr->b_tmp_cdata;
 		buf_sz = l2hdr->b_asize;
 
+		/*
+		 * If the data has not been compressed, then clear b_tmp_cdata
+		 * to make sure that it points only to a temporary compression
+		 * buffer.
+		 */
+		if (!L2ARC_IS_VALID_COMPRESS(l2hdr->b_compress))
+			l2hdr->b_tmp_cdata = NULL;
+
 		/* Compression may have squashed the buffer to zero length. */
 		if (buf_sz != 0) {
 			uint64_t buf_p_sz;
@@ -5323,15 +5365,18 @@ l2arc_release_cdata_buf(arc_buf_hdr_t *ab)
 {
 	l2arc_buf_hdr_t *l2hdr = ab->b_l2hdr;
 
-	if (l2hdr->b_compress == ZIO_COMPRESS_LZ4) {
+	ASSERT(L2ARC_IS_VALID_COMPRESS(l2hdr->b_compress));
+	if (l2hdr->b_compress != ZIO_COMPRESS_EMPTY) {
 		/*
 		 * If the data was compressed, then we've allocated a
 		 * temporary buffer for it, so now we need to release it.
 		 */
 		ASSERT(l2hdr->b_tmp_cdata != NULL);
 		zio_data_buf_free(l2hdr->b_tmp_cdata, ab->b_size);
+		l2hdr->b_tmp_cdata = NULL;
+	} else {
+		ASSERT(l2hdr->b_tmp_cdata == NULL);
 	}
-	l2hdr->b_tmp_cdata = NULL;
 }
 
 /*
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c
index 08c8418..14f4d60 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c
@@ -155,10 +155,8 @@ trim_map_create(vdev_t *vd)
 {
 	trim_map_t *tm;
 
-	ASSERT(vd->vdev_ops->vdev_op_leaf);
-
-	if (!zfs_trim_enabled)
-		return;
+	ASSERT(zfs_trim_enabled && !vd->vdev_notrim &&
+		vd->vdev_ops->vdev_op_leaf);
 
 	tm = kmem_zalloc(sizeof (*tm), KM_SLEEP);
 	mutex_init(&tm->tm_lock, NULL, MUTEX_DEFAULT, NULL);
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
index 7eac42c..cce9438 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
@@ -1214,6 +1214,7 @@ vdev_open(vdev_t *vd)
 	vd->vdev_stat.vs_aux = VDEV_AUX_NONE;
 	vd->vdev_cant_read = B_FALSE;
 	vd->vdev_cant_write = B_FALSE;
+	vd->vdev_notrim = B_FALSE;
 	vd->vdev_min_asize = vdev_get_min_asize(vd);
 
 	/*
@@ -1283,10 +1284,8 @@ vdev_open(vdev_t *vd)
 	if (vd->vdev_ishole || vd->vdev_ops == &vdev_missing_ops)
 		return (0);
 
-	if (vd->vdev_ops->vdev_op_leaf) {
-		vd->vdev_notrim = B_FALSE;
+	if (zfs_trim_enabled && !vd->vdev_notrim && vd->vdev_ops->vdev_op_leaf)
 		trim_map_create(vd);
-	}
 
 	for (int c = 0; c < vd->vdev_children; c++) {
 		if (vd->vdev_child[c]->vdev_state != VDEV_STATE_HEALTHY) {
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c
index 1caff2c..eaa9dfc 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c
@@ -796,6 +796,8 @@ vdev_disk_io_start(zio_t *zio)
 		return (ZIO_PIPELINE_STOP);
 	}
 
+	ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE);
+
 	vb = kmem_alloc(sizeof (vdev_buf_t), KM_SLEEP);
 
 	vb->vb_io = zio;
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c
index ab492d8..ad21d6e 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c
@@ -129,6 +129,8 @@ skip_open:
 		return (error);
 	}
 
+	vd->vdev_notrim = B_TRUE;
+
 	*max_psize = *psize = vattr.va_size;
 	*logical_ashift = SPA_MINBLOCKSHIFT;
 	*physical_ashift = SPA_MINBLOCKSHIFT;
@@ -185,6 +187,8 @@ vdev_file_io_start(zio_t *zio)
 		return (ZIO_PIPELINE_STOP);
 	}
 
+	ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE);
+
 	zio->io_error = vn_rdwr(zio->io_type == ZIO_TYPE_READ ?
 	    UIO_READ : UIO_WRITE, vp, zio->io_data, zio->io_size,
 	    zio->io_offset, UIO_SYSSPACE, 0, RLIM64_INFINITY, kcred, &resid);
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
index 210968e..4cc0472 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
@@ -832,6 +832,11 @@ vdev_geom_io_start(zio_t *zio)
 		return (ZIO_PIPELINE_STOP);
 	}
 sendreq:
+	ASSERT(zio->io_type == ZIO_TYPE_READ ||
+	    zio->io_type == ZIO_TYPE_WRITE ||
+	    zio->io_type == ZIO_TYPE_FREE ||
+	    zio->io_type == ZIO_TYPE_IOCTL);
+
 	cp = vd->vdev_tsd;
 	if (cp == NULL) {
 		zio->io_error = SET_ERROR(ENXIO);
diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c
index 70c186f..966f2fa 100644
--- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c
+++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c
@@ -713,8 +713,9 @@ vdev_label_init(vdev_t *vd, uint64_t crtxg, vdev_labeltype_t reason)
 	 * Don't TRIM if removing so that we don't interfere with zpool
 	 * disaster recovery.
 	 */
-	if (zfs_trim_enabled && vdev_trim_on_init && (reason == VDEV_LABEL_CREATE ||
-	    reason == VDEV_LABEL_SPARE || reason == VDEV_LABEL_L2CACHE))
+	if (zfs_trim_enabled && vdev_trim_on_init && !vd->vdev_notrim && 
+	    (reason == VDEV_LABEL_CREATE || reason == VDEV_LABEL_SPARE ||
+	    reason == VDEV_LABEL_L2CACHE))
 		zio_wait(zio_trim(NULL, spa, vd, 0, vd->vdev_psize));
 
 	/*
diff --git a/sys/conf/newvers.sh b/sys/conf/newvers.sh
index d371084..691b57f 100644
--- a/sys/conf/newvers.sh
+++ b/sys/conf/newvers.sh
@@ -32,7 +32,7 @@
 
 TYPE="FreeBSD"
 REVISION="10.1"
-BRANCH="RELEASE-p10"
+BRANCH="RELEASE-p11"
 if [ "X${BRANCH_OVERRIDE}" != "X" ]; then
 	BRANCH=${BRANCH_OVERRIDE}
 fi
author	delphij <delphij@FreeBSD.org>	2015-06-09 22:13:25 +0000
committer	delphij <delphij@FreeBSD.org>	2015-06-09 22:13:25 +0000
commit	301060471bd9c2f62da3293dd334408ae864be70 (patch)
tree	fb96be74f227016a1aec7aac411b2f5a6eafe025
parent	3bc97baf8a6eb60b69f56e79f0bc47597014d084 (diff)
download	FreeBSD-src-301060471bd9c2f62da3293dd334408ae864be70.zip FreeBSD-src-301060471bd9c2f62da3293dd334408ae864be70.tar.gz