Expand the use of stat(2) flags to allow storing some Windows/DOS

and CIFS file attributes as BSD stat(2) flags.

This work is intended to be compatible with ZFS, the Solaris CIFS
server's interaction with ZFS, somewhat compatible with MacOS X,
and of course compatible with Windows.

The Windows attributes that are implemented were chosen based on
the attributes that ZFS already supports.

The summary of the flags is as follows:

UF_SYSTEM:	Command line name: "system" or "usystem"
		ZFS name: XAT_SYSTEM, ZFS_SYSTEM
		Windows: FILE_ATTRIBUTE_SYSTEM

		This flag means that the file is used by the
		operating system.  FreeBSD does not enforce any
		special handling when this flag is set.

UF_SPARSE:	Command line name: "sparse" or "usparse"
		ZFS name: XAT_SPARSE, ZFS_SPARSE
		Windows: FILE_ATTRIBUTE_SPARSE_FILE

		This flag means that the file is sparse.  Although
		ZFS may modify this in some situations, there is
		not generally any special handling for this flag.

UF_OFFLINE:	Command line name: "offline" or "uoffline"
		ZFS name: XAT_OFFLINE, ZFS_OFFLINE
		Windows: FILE_ATTRIBUTE_OFFLINE

		This flag means that the file has been moved to
		offline storage.  FreeBSD does not have any special
		handling for this flag.

UF_REPARSE:	Command line name: "reparse" or "ureparse"
		ZFS name: XAT_REPARSE, ZFS_REPARSE
		Windows: FILE_ATTRIBUTE_REPARSE_POINT

		This flag means that the file is a Windows reparse
		point.  ZFS has special handling code for reparse
		points, but we don't currently have the other
		supporting infrastructure for them.

UF_HIDDEN:	Command line name: "hidden" or "uhidden"
		ZFS name: XAT_HIDDEN, ZFS_HIDDEN
		Windows: FILE_ATTRIBUTE_HIDDEN

		This flag means that the file may be excluded from
		a directory listing if the application honors it.
		FreeBSD has no special handling for this flag.

		The name and bit definition for UF_HIDDEN are
		identical to the definition in MacOS X.

UF_READONLY:	Command line name: "urdonly", "rdonly", "readonly"
		ZFS name: XAT_READONLY, ZFS_READONLY
		Windows: FILE_ATTRIBUTE_READONLY

		This flag means that the file may not written or
		appended, but its attributes may be changed.

		ZFS currently enforces this flag, but Illumos
		developers have discussed disabling enforcement.

		The behavior of this flag is different than MacOS X.
		MacOS X uses UF_IMMUTABLE to represent the DOS
		readonly permission, but that flag has a stronger
		meaning than the semantics of DOS readonly permissions.

UF_ARCHIVE:	Command line name: "uarch", "uarchive"
		ZFS_NAME: XAT_ARCHIVE, ZFS_ARCHIVE
		Windows name: FILE_ATTRIBUTE_ARCHIVE

		The UF_ARCHIVED flag means that the file has changed and
		needs to be archived.  The meaning is same as
		the Windows FILE_ATTRIBUTE_ARCHIVE attribute, and
		the ZFS XAT_ARCHIVE and ZFS_ARCHIVE attribute.

		msdosfs and ZFS have special handling for this flag.
		i.e. they will set it when the file changes.

sys/param.h:		Bump __FreeBSD_version to 1000047 for the
			addition of new stat(2) flags.

chflags.1:		Document the new command line flag names
			(e.g. "system", "hidden") available to the
			user.

ls.1:			Reference chflags(1) for a list of file flags
			and their meanings.

strtofflags.c:		Implement the mapping between the new
			command line flag names and new stat(2)
			flags.

chflags.2:		Document all of the new stat(2) flags, and
			explain the intended behavior in a little
			more detail.  Explain how they map to
			Windows file attributes.

			Different filesystems behave differently
			with respect to flags, so warn the
			application developer to take care when
			using them.

zfs_vnops.c:		Add support for getting and setting the
			UF_ARCHIVE, UF_READONLY, UF_SYSTEM, UF_HIDDEN,
			UF_REPARSE, UF_OFFLINE, and UF_SPARSE flags.

			All of these flags are implemented using
			attributes that ZFS already supports, so
			the on-disk format has not changed.

			ZFS currently doesn't allow setting the
			UF_REPARSE flag, and we don't really have
			the other infrastructure to support reparse
			points.

msdosfs_denode.c,
msdosfs_vnops.c:	Add support for getting and setting
			UF_HIDDEN, UF_SYSTEM and UF_READONLY
			in MSDOSFS.

			It supported SF_ARCHIVED, but this has been
			changed to be UF_ARCHIVE, which has the same
			semantics as the DOS archive attribute instead
			of inverse semantics like SF_ARCHIVED.

			After discussion with Bruce Evans, change
			several things in the msdosfs behavior:

			Use UF_READONLY to indicate whether a file
			is writeable instead of file permissions, but
			don't actually enforce it.

			Refuse to change attributes on the root
			directory, because it is special in FAT
			filesystems, but allow most other attribute
			changes on directories.

			Don't set the archive attribute on a directory
			when its modification time is updated.
			Windows and DOS don't set the archive attribute
			in that scenario, so we are now bug-for-bug
			compatible.

smbfs_node.c,
smbfs_vnops.c:		Add support for UF_HIDDEN, UF_SYSTEM,
			UF_READONLY and UF_ARCHIVE in SMBFS.

			This is similar to changes that Apple has
			made in their version of SMBFS (as of
			smb-583.8, posted on opensource.apple.com),
			but not quite the same.

			We map SMB_FA_READONLY to UF_READONLY,
			because UF_READONLY is intended to match
			the semantics of the DOS readonly flag.
			The MacOS X code maps both UF_IMMUTABLE
			and SF_IMMUTABLE to SMB_FA_READONLY, but
			the immutable flags have stronger meaning
			than the DOS readonly bit.

stat.h:			Add definitions for UF_SYSTEM, UF_SPARSE,
			UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY
			and UF_HIDDEN.

			The definition of UF_HIDDEN is the same as
			the MacOS X definition.

			Add commented-out definitions of
			UF_COMPRESSED and UF_TRACKED.  They are
			defined in MacOS X (as of 10.8.2), but we
			do not implement them (yet).

ufs_vnops.c:		Add support for getting and setting
			UF_ARCHIVE, UF_HIDDEN, UF_OFFLINE, UF_READONLY,
			UF_REPARSE, UF_SPARSE, and UF_SYSTEM in UFS.
			Alphabetize the flags that are supported.

			These new flags are only stored, UFS does
			not take any action if the flag is set.

Sponsored by:	Spectra Logic
Reviewed by:	bde (earlier version)

2 files changed, 72 insertions, 14 deletions

diff --git a/lib/libc/gen/strtofflags.c b/lib/libc/gen/strtofflags.c
index 1edaa5a..55e9945 100644
--- a/lib/libc/gen/strtofflags.c
+++ b/lib/libc/gen/strtofflags.c
@@ -62,13 +62,29 @@ static struct {
 #endif
 	{ "nouappnd",		0, UF_APPEND	},
 	{ "nouappend",		0, UF_APPEND	},
+	{ "nouarch", 		0, UF_ARCHIVE	},
+	{ "nouarchive",		0, UF_ARCHIVE	},
+	{ "nohidden",		0, UF_HIDDEN	},
+	{ "nouhidden",		0, UF_HIDDEN	},
 	{ "nouchg",		0, UF_IMMUTABLE	},
 	{ "nouchange",		0, UF_IMMUTABLE	},
 	{ "nouimmutable",	0, UF_IMMUTABLE	},
 	{ "nodump",		1, UF_NODUMP	},
-	{ "noopaque",		0, UF_OPAQUE	},
 	{ "nouunlnk",		0, UF_NOUNLINK	},
-	{ "nouunlink",		0, UF_NOUNLINK	}
+	{ "nouunlink",		0, UF_NOUNLINK	},
+	{ "nooffline",		0, UF_OFFLINE	},
+	{ "nouoffline",		0, UF_OFFLINE	},
+	{ "noopaque",		0, UF_OPAQUE	},
+	{ "nordonly",		0, UF_READONLY	},
+	{ "nourdonly",		0, UF_READONLY	},
+	{ "noreadonly",		0, UF_READONLY	},
+	{ "noureadonly",	0, UF_READONLY	},
+	{ "noreparse",		0, UF_REPARSE	},
+	{ "noureparse",		0, UF_REPARSE	},
+	{ "nosparse",		0, UF_SPARSE	},
+	{ "nousparse",		0, UF_SPARSE	},
+	{ "nosystem",		0, UF_SYSTEM	},
+	{ "nousystem",		0, UF_SYSTEM	}
 };
 #define nmappings	(sizeof(mapping) / sizeof(mapping[0]))
 
diff --git a/lib/libc/sys/chflags.2 b/lib/libc/sys/chflags.2
index 5bc2ba2..9cca54f 100644
--- a/lib/libc/sys/chflags.2
+++ b/lib/libc/sys/chflags.2
@@ -112,26 +112,61 @@ The flags specified are formed by
 the following values
 .Pp
 .Bl -tag -width ".Dv SF_IMMUTABLE" -compact -offset indent
-.It Dv UF_NODUMP
-Do not dump the file.
-.It Dv UF_IMMUTABLE
-The file may not be changed.
-.It Dv UF_APPEND
+.It Dv SF_APPEND
 The file may only be appended to.
-.It Dv UF_NOUNLINK
-The file may not be renamed or deleted.
-.It Dv UF_OPAQUE
-The directory is opaque when viewed through a union stack.
 .It Dv SF_ARCHIVED
-The file may be archived.
+The file has been archived.
+This flag means the opposite of the DOS, Windows and CIFS
+FILE_ATTRIBUTE_ARCHIVE attribute.
+This flag has been deprecated, and may be removed in a future release.
 .It Dv SF_IMMUTABLE
 The file may not be changed.
-.It Dv SF_APPEND
-The file may only be appended to.
 .It Dv SF_NOUNLINK
 The file may not be renamed or deleted.
 .It Dv SF_SNAPSHOT
 The file is a snapshot file.
+.It Dv UF_APPEND
+The file may only be appended to.
+.It Dv UF_ARCHIVE
+The file needs to be archived.
+This flag has the same meaning as the DOS, Windows and CIFS
+FILE_ATTRIBUTE_ARCHIVE attribute.
+Filesystems in FreeBSD may or may not have special handling for this flag.
+For instance, ZFS tracks changes to files and will set this bit when a
+file is updated.
+UFS only stores the flag, and relies on the application to change it when
+needed.
+.It Dv UF_HIDDEN
+The file may be hidden from directory listings at the application's
+discretion.
+The file has the DOS, Windows and CIFS FILE_ATTRIBUTE_HIDDEN attribute.
+.It Dv UF_IMMUTABLE
+The file may not be changed.
+.It Dv UF_NODUMP
+Do not dump the file.
+.It Dv UF_NOUNLINK
+The file may not be renamed or deleted.
+.It Dv UF_OFFLINE
+The file is offline, or has the Windows and CIFS FILE_ATTRIBUTE_OFFLINE
+attribute.
+Filesystems in FreeBSD store and display this flag, but do not provide any
+special handling when it is set.
+.It Dv UF_OPAQUE
+The directory is opaque when viewed through a union stack.
+.It Dv UF_READONLY
+The file is read only, and may not be written or appended.
+Filesystems may use this flag to maintain compatibility with the DOS, Windows
+and CIFS FILE_ATTRIBUTE_READONLY attribute.
+.It Dv UF_REPARSE
+The file contains a Windows reparse point and has the Windows and CIFS
+FILE_ATTRIBUTE_REPARSE_POINT attribute.
+.It Dv UF_SPARSE
+The file has the Windows FILE_ATTRIBUTE_SPARSE_FILE attribute.
+This may also be used by a filesystem to indicate a sparse file.
+.It Dv UF_SYSTEM
+The file has the DOS, Windows and CIFS FILE_ATTRIBUTE_SYSTEM attribute.
+Filesystems in FreeBSD may store and display this flag, but do not provide
+any special handling when it is set.
 .El
 .Pp
 If one of
@@ -162,6 +197,13 @@ the system is in single-user mode.
 .Xr init 8
 for details.)
 .Pp
+The implementation of all flags is filesystem-dependent.
+See the description of the 
+.Dv UF_ARCHIVE
+flag above for one example of the differences in behavior.
+Care should be exercised when writing applications to account for
+support or lack of support of these flags in various filesystems.
+.Pp
 The
 .Dv SF_SNAPSHOT
 flag is maintained by the system and cannot be toggled.