summaryrefslogtreecommitdiffstats
path: root/sys/ufs
Commit message (Collapse)AuthorAgeFilesLines
* Further fixes for deadlock in the presence of multiple snapshots.mckusick2001-05-141-7/+20
| | | | | There are still more to find, but this fix should cover the common cases that folks are hitting.
* If the effective link count is zero when an NFS file handle requestmckusick2001-05-131-1/+3
| | | | | | | | | | | | | | | | comes in for it, the file is really gone, so return ESTALE. The problem arises when the last reference to an FFS file is released because soft-updates may delay the actual freeing of the inode for some time. Since there are no filesystem links or open file descriptors referencing the inode, from the point of view of the system, the file is inaccessible. However, if the filesystem is NFS exported, then the remote client can still access the inode via ufs_fhtovp() until the inode really goes away. To prevent this anomoly, it is necessary to begin returning ESTALE at the same time that the file ceases to be accessible to the local filesystem. Obtained from: Ian Dowse <iedowse@maths.tcd.ie>
* Remove yet another deadlock case.mckusick2001-05-111-3/+6
|
* When running with soft updates, track the number of blocks and filesmckusick2001-05-089-11/+119
| | | | | | | | | | | | | that are committed to being freed and reflect these blocks in the counts returned by statfs (and thus also by the `df' command). This change allows programs such as those that do news expiration to know when to stop if they are trying to create a certain percentage of free space. Note that this change does not solve the much harder problem of making this to-be-freed space available to applications that want it (thus on a nearly full filesystem, you may still encounter out-of-space conditions even though the free space will show up eventually). Hopefully this harder problem will be the subject of a future enhancement.
* Several fixes for units errors:mckusick2001-05-081-10/+19
| | | | | | | | | | | | | | | | | 1) Do not assume that the superblock will be of size fs->fs_bsize. This fixes a panic when taking a snapshot on a filesystem with a block size bigger than 8K. 2) Properly calculate the number of fragments that follow the superblock summary information. This fixes a bug with inconsistent snapshots. 3) When cleaning up a snapshot that is about to be removed, properly calculate the number of blocks that need to be checked. This fixes a bug that created partially allocated inodes. 4) When moving blocks from a snapshot that is about to be removed to another snapshot, properly account for the reduced number of blocks in the snapshot from which they are taken. This fixes a bug in which the number of blocks released from a snapshot did not match the number that it claimed to have.
* When syncing out snapshot metadata, we must temporarily allow recursivemckusick2001-05-081-27/+29
| | | | | buffer locking so as to avoid locking against ourselves if we need to write filesystem metadata.
* Refinement to revision 1.16 of ufs/ffs/ffs_snapshot.c to reducemckusick2001-05-043-120/+227
| | | | | the amount of time that the filesystem must be suspended. The current snapshot is elided as well as the earlier snapshots.
* Use ufs_bmaparray() rather than VOP_BMAP() on our own vnodes.phk2001-05-011-2/+2
|
* Remove blatantly pointless call to VOP_BMAP().phk2001-05-012-9/+3
| | | | Use ufs_bmaparray() rather than VOP_BMAP() on our own vnodes.
* Implement vop_std{get|put}pages() and add them to the default vop[].phk2001-05-013-18/+0
| | | | | Un-copy&paste all the VOP_{GET|PUT}PAGES() functions which do nothing but the default.
* Undo part of the tangle of having sys/lock.h and sys/mutex.h included inmarkm2001-05-012-6/+11
| | | | | | | | | | | other "system" header files. Also help the deprecation of lockmgr.h by making it a sub-include of sys/lock.h and removing sys/lockmgr.h form kernel .c files. Sort sys/*.h includes where possible in affected files. OK'ed by: bde (with reservations)
* VOP_BALLOC was never really a VOP in the first place, so convert itphk2001-04-2911-49/+44
| | | | to UFS_BALLOC like the other "between UFS and FFS function interfaces".
* Add a vop_stdbmap(), and make it part of the default vop vector.phk2001-04-291-25/+1
| | | | | | Make 7 filesystems which don't really know about VOP_BMAP rely on the default vector, rather than more or less complete local vop_nopbmap() implementations.
* Call ufs_bmaparray() directly instead of indirectly via VOP_BMAP().phk2001-04-291-2/+3
|
* Remove two unused arguments from ufs_bmaparray().phk2001-04-292-22/+17
|
* Remove faint traces of blind copy&paste.phk2001-04-291-1/+0
|
* Remove faint traces of non-existant ffs_bmap().phk2001-04-291-2/+0
|
* Revert consequences of changes to mount.h, part 2.grog2001-04-2916-32/+0
| | | | Requested by: bde
* Rather than copying all the indirect blocks of the snapshot,mckusick2001-04-261-35/+19
| | | | | | simply mark them as BLK_NOCOPY. This trick cuts the initial size of the snapshot in half and cuts the time to take a snapshot by a third.
* When closing the last reference to an unlinked file, it is freedmckusick2001-04-253-26/+104
| | | | | | | | | | | | | | | by the inactive routine. Because the freeing causes the filesystem to be modified, the close must be held up during periods when the filesystem is suspended. For snapshots to be consistent across crashes, they must write blocks that they copy and claim those written blocks in their on-disk block pointers before the old blocks that they referenced can be allowed to be written. Close a loophole that allowed unwritten blocks to be skipped when doing ffs_sync with a request to wait for all I/O activity to be completed.
* Move the netexport structure from the fs-specific mountstructurephk2001-04-256-38/+6
| | | | | | | | | | | | | | to struct mount. This makes the "struct netexport *" paramter to the vfs_export and vfs_checkexport interface unneeded. Consequently that all non-stacking filesystems can use vfs_stdcheckexp(). At the same time, make it a pointer to a struct netexport in struct mount, so that we can remove the bogus AF_MAX and #include <net/radix.h> from <sys/mount.h>
* Pre-dirpref versions of fsck may zero out the new superblock fieldsiedowse2001-04-241-0/+6
| | | | | | | | | | | fs_contigdirs, fs_avgfilesize and fs_avgfpdir. This could cause panics if these fields were zeroed while a filesystem was mounted read-only, and then remounted read-write. Add code to ffs_reload() which copies the fs_contigdirs pointer from the previous superblock, and reinitialises fs_avgf* if necessary. Reviewed by: mckusick
* Correct #includes to work with fixed sys/mount.h.grog2001-04-2316-0/+32
|
* This patch removes the VOP_BWRITE() vector.phk2001-04-171-1/+0
| | | | | | | | | | | | | VOP_BWRITE() was a hack which made it possible for NFS client side to use struct buf with non-bio backing. This patch takes a more general approach and adds a bp->b_op vector where more methods can be added. The success of this patch depends on bp->b_op being initialized all relevant places for some value of "relevant" which is not easy to determine. For now the buffers have grown a b_magic element which will make such issues a tiny bit easier to debug.
* Add debugging option to always read/write cylinder groups as fullmckusick2001-04-171-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | sized blocks. To enable this option, use: `sysctl -w debug.bigcgs=1'. Add debugging option to disable background writes of cylinder groups. To enable this option, use: `sysctl -w debug.dobkgrdwrite=0'. These debugging options should be tried on systems that are panicing with corrupted cylinder group maps to see if it makes the problem go away. The set of panics in question are: ffs_clusteralloc: map mismatch ffs_nodealloccg: map corrupted ffs_nodealloccg: block not in map ffs_alloccg: map corrupted ffs_alloccg: block not in map ffs_alloccgblk: cyl groups corrupted ffs_alloccgblk: can't find blk in cyl ffs_checkblk: partially free fragment The following panics are less likely to be related to this problem, but might be helped by these debugging options: ffs_valloc: dup alloc ffs_blkfree: freeing free block ffs_blkfree: freeing free frag ffs_vfree: freeing free inode If you try these options, please report whether they helped reduce your bitmap corruption panics to Kirk McKusick at <mckusick@mckusick.com> and to Matt Dillon <dillon@earth.backplane.com>.
* Background fsck sysctl operations must use vn_start_write andmckusick2001-04-171-8/+14
| | | | | vn_finished_write so that they do not attempt to modify a suspended filesystem.
* In my first reading of POSIX.1e, I misinterpreted handling of therwatson2001-04-172-88/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ACL_USER_OBJ and ACL_GROUP_OBJ fields, believing that modification of the access ACL could be used by privileged processes to change file/directory ownership. In fact, this is incorrect; ACL_*_OBJ (+ ACL_MASK and ACL_OTHER) should have undefined ae_id fields; this commit attempts to correct that misunderstanding. o Modify arguments to vaccess_acl_posix1e() to accept the uid and gid associated with the vnode, as those can no longer be extracted from the ACL passed as an argument. Perform all comparisons against the passed arguments. This actually has the effect of simplifying a number of components of this call, as well as reducing the indent level, but now seperates handling of ACL_GROUP_OBJ from ACL_GROUP. o Modify acl_posix1e_check() to return EINVAL if the ae_id field of any of the ACL_{USER_OBJ,GROUP_OBJ,MASK,OTHER} entries is a value other than ACL_UNDEFINED_ID. As a temporary work-around to allow clean upgrades, set the ae_id field to ACL_UNDEFINED_ID before each check so that this cannot cause a failure in the short term (this work-around will be removed when the userland libraries and utilities are updated to take this change into account). o Modify ufs_sync_acl_from_inode() so that it forces ACL_{USER_OBJ,GROUP_OBJ,MASK,OTHER} ae_id fields to ACL_UNDEFINED_ID when synchronizing the ACL from the inode. o Modify ufs_sync_inode_from_acl to not propagate uid and gid information to the inode from the ACL during ACL update. Also modify the masking of permission bits that may be set from ALLPERMS to (S_IRWXU|S_IRWXG|S_IRWXO), as ACLs currently do not carry none-ACCESSPERMS (S_ISUID, S_ISGID, S_ISTXT). o Modify ufs_getacl() so that when it emulates an access ACL from the inode, it initializes the ae_id fields to ACL_UNDEFINED_ID. o Clean up ufs_setacl() substantially since it is no longer possible to perform chown/chgrp operations using vop_setacl(), so all the access control for that can be eliminated. o Modify ufs_access() so that it passes owner uid and gid information into vaccess_acl_posix1e(). Pointed out by: jedger Obtained from: TrustedBSD Project
* Update to describe use of mdconfig instead of deprecated vnconfig.mckusick2001-04-141-3/+3
| | | | Submitted by: Steve Ames <steve@virtual-voodoo.com>
* This checkin adds support in ufs/ffs for the FS_NEEDSFSCK flag.mckusick2001-04-143-8/+19
| | | | | | | | | | | | | | | | | | | It is described in ufs/ffs/fs.h as follows: /* * Filesystem flags. * * Note that the FS_NEEDSFSCK flag is set and cleared only by the * fsck utility. It is set when background fsck finds an unexpected * inconsistency which requires a traditional foreground fsck to be * run. Such inconsistencies should only be found after an uncorrectable * disk error. A foreground fsck will clear the FS_NEEDSFSCK flag when * it has successfully cleaned up the filesystem. The kernel uses this * flag to enforce that inconsistent filesystems be mounted read-only. */ #define FS_UNCLEAN 0x01 /* filesystem not clean at mount */ #define FS_DOSOFTDEP 0x02 /* filesystem using soft dependencies */ #define FS_NEEDSFSCK 0x04 /* filesystem needs sync fsck before mount */
* Directory layout preference improvements from Grigoriy Orlov <gluk@ptci.ru>.mckusick2001-04-103-21/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | His description of the problem and solution follow. My own tests show speedups on typical filesystem intensive workloads of 5% to 12% which is very impressive considering the small amount of code change involved. ------ One day I noticed that some file operations run much faster on small file systems then on big ones. I've looked at the ffs algorithms, thought about them, and redesigned the dirpref algorithm. First I want to describe the results of my tests. These results are old and I have improved the algorithm after these tests were done. Nevertheless they show how big the perfomance speedup may be. I have done two file/directory intensive tests on a two OpenBSD systems with old and new dirpref algorithm. The first test is "tar -xzf ports.tar.gz", the second is "rm -rf ports". The ports.tar.gz file is the ports collection from the OpenBSD 2.8 release. It contains 6596 directories and 13868 files. The test systems are: 1. Celeron-450, 128Mb, two IDE drives, the system at wd0, file system for test is at wd1. Size of test file system is 8 Gb, number of cg=991, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=35 2. PIII-600, 128Mb, two IBM DTLA-307045 IDE drives at i815e, the system at wd0, file system for test is at wd1. Size of test file system is 40 Gb, number of cg=5324, size of cg is 8m, block size = 8k, fragment size = 1k OpenBSD-current from Dec 2000 with BUFCACHEPERCENT=50 You can get more info about the test systems and methods at: http://www.ptci.ru/gluk/dirpref/old/dirpref.html Test Results tar -xzf ports.tar.gz rm -rf ports mode old dirpref new dirpref speedup old dirprefnew dirpref speedup First system normal 667 472 1.41 477 331 1.44 async 285 144 1.98 130 14 9.29 sync 768 616 1.25 477 334 1.43 softdep 413 252 1.64 241 38 6.34 Second system normal 329 81 4.06 263.5 93.5 2.81 async 302 25.7 11.75 112 2.26 49.56 sync 281 57.0 4.93 263 90.5 2.9 softdep 341 40.6 8.4 284 4.76 59.66 "old dirpref" and "new dirpref" columns give a test time in seconds. speedup - speed increasement in times, ie. old dirpref / new dirpref. ------ Algorithm description The old dirpref algorithm is described in comments: /* * Find a cylinder to place a directory. * * The policy implemented by this algorithm is to select from * among those cylinder groups with above the average number of * free inodes, the one with the smallest number of directories. */ A new directory is allocated in a different cylinder groups than its parent directory resulting in a directory tree that is spreaded across all the cylinder groups. This spreading out results in a non-optimal access to the directories and files. When we have a small filesystem it is not a problem but when the filesystem is big then perfomance degradation becomes very apparent. What I mean by a big file system ? 1. A big filesystem is a filesystem which occupy 20-30 or more percent of total drive space, i.e. first and last cylinder are physically located relatively far from each other. 2. It has a relatively large number of cylinder groups, for example more cylinder groups than 50% of the buffers in the buffer cache. The first results in long access times, while the second results in many buffers being used by metadata operations. Such operations use cylinder group blocks and on-disk inode blocks. The cylinder group block (fs->fs_cblkno) contains struct cg, inode and block bit maps. It is 2k in size for the default filesystem parameters. If new and parent directories are located in different cylinder groups then the system performs more input/output operations and uses more buffers. On filesystems with many cylinder groups, lots of cache buffers are used for metadata operations. My solution for this problem is very simple. I allocate many directories in one cylinder group. I also do some things, so that the new allocation method does not cause excessive fragmentation and all directory inodes will not be located at a location far from its file's inodes and data. The algorithm is: /* * Find a cylinder group to place a directory. * * The policy implemented by this algorithm is to allocate a * directory inode in the same cylinder group as its parent * directory, but also to reserve space for its files inodes * and data. Restrict the number of directories which may be * allocated one after another in the same cylinder group * without intervening allocation of files. * * If we allocate a first level directory then force allocation * in another cylinder group. */ My early versions of dirpref give me a good results for a wide range of file operations and different filesystem capacities except one case: those applications that create their entire directory structure first and only later fill this structure with files. My solution for such and similar cases is to limit a number of directories which may be created one after another in the same cylinder group without intervening file creations. For this purpose, I allocate an array of counters at mount time. This array is linked to the superblock fs->fs_contigdirs[cg]. Each time a directory is created the counter increases and each time a file is created the counter decreases. A 60Gb filesystem with 8mb/cg requires 10kb of memory for the counters array. The maxcontigdirs is a maximum number of directories which may be created without an intervening file creation. I found in my tests that the best performance occurs when I restrict the number of directories in one cylinder group such that all its files may be located in the same cylinder group. There may be some deterioration in performance if all the file inodes are in the same cylinder group as its containing directory, but their data partially resides in a different cylinder group. The maxcontigdirs value is calculated to try to prevent this condition. Since there is no way to know how many files and directories will be allocated later I added two optimization parameters in superblock/tunefs. They are: int32_t fs_avgfilesize; /* expected average file size */ int32_t fs_avgfpdir; /* expected # of files per directory */ These parameters have reasonable defaults but may be tweeked for special uses of a filesystem. They are only necessary in rare cases like better tuning a filesystem being used to store a squid cache. I have been using this algorithm for about 3 months. I have done a lot of testing on filesystems with different capacities, average filesize, average number of files per directory, and so on. I think this algorithm has no negative impact on filesystem perfomance. It works better than the default one in all cases. The new dirpref will greatly improve untarring/removing/coping of big directories, decrease load on cvs servers and much more. The new dirpref doesn't speedup a compilation process, but also doesn't slow it down. Obtained from: Grigoriy Orlov <gluk@ptci.ru>
* o Indent sub-section headings to be consistent with README.extattr.rwatson2001-04-031-2/+2
| | | | Obtained from: TrustedBSD Project
* o Introduce a README file describing briefly how to use access controlrwatson2001-04-031-0/+51
| | | | | | lists, in the style of FFS README files for soft updates and snapshots. Obtained from: TrustedBSD Project
* o Introduce a README file describing briefly how to use extendedrwatson2001-04-031-0/+86
| | | | | | | attributes, in the style of FFS README files for soft updates and snapshots. Obtained from: TrustedBSD Project
* o Change the default from using IO_SYNC on EA set and delete operationsrwatson2001-04-031-9/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to not using IO_SYNC. Expose a sysctl (debug.ufs_extattr_sync) for enabling the use of IO_SYNC. - Use of IO_SYNC substantially degrades ACL performance when a default ACL is set on a directory, as there are four synchronous writes initiated to define both supporting EAs for new sub-directories, and to set the data; two for new files. Later, this may be optimized to two writes for sub-directories, one for new files. - IO_SYNC does not substantially improve consistency properties due to the poor consistency properties of existing permissions (which ACLs are a superset of), due to interaction with soft updates, and due to differences in handling consistency for data and file system meta-data. - In macro-benchmarks, this reduces the overhead of setting default ACLs down to the same overhead as enabling ACLs on a file system and not using them. Enabling ACLs still introduces a small overhead (I measure 7% on a -j 2 buildworld with pre-allocated EA backing store, but this is not rigorous testing, nor in any way optimized). - The sysctl will probably change to another administration method (or at least, a better name) in the near future, but consistency properties of EAs are still being worked out. The toggle is defined right now to allow easier performance analysis and exploration of possible guarantees. Obtained from: TrustedBSD Project
* o Correct an ACL implementation bug that could result in a system panicrwatson2001-04-021-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | under heavy use when default ACLs were bgin inherited by new files or directories. This is done by removing a bug in default ACL reading, and improving error handling for this failure case: - Move the setting of the buffer length (len) variable to above the ACL type (ap->a_type) switch rather than having it only for ACL_TYPE_ACCESS. Otherwise, the len variable is unitialized in the ACL_TYPE_DEFAULT case, which generally worked right, but could result in failure. - Add a check for a short/long read of the ACL_TYPE_DEFAULT type from the underlying EA, resulting in EPERM rather than passing a potentially corrupted ACL back to the caller (resulting "cleaner" failures if the EA is damaged: right now, the caller will almost always panic in the presence of a corrupted EA). This code is similar to code in the ACL_TYPE_ACCESS handling in the previous switch case. - While I'm fixing this code, remove a redundant bzero() of the ACL reader buffer; it need only be initialized above the acl_type switch. Obtained from: TrustedBSD Project
* Introduce support for POSIX.1e ACLs on UFS-based file systems. Thisrwatson2001-03-263-5/+819
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | implementation is still experimental, and while fairly broadly tested, is not yet intended for production use. Support for POSIX.1e ACLs on UFS will not be MFC'd to RELENG_4. This implementation works by providing implementations of VOP_[GS]ETACL() for FFS, as well as modifying the appropriate access control and file creation routines. In this implementation, ACLs are backed into extended attributes; the base ACL (owner, group, other) permissions remain in the inode for performance and compatibility reasons, so only the extended and default ACLs are placed in extended attributes. The logic for ACL evaluation is provided by the fs-independent kern/kern_acl.c. o Introduce UFS_ACL, a compile-time configuration option that enables support for ACLs on FFS (and potentially other UFS-based file systems). o Introduce ufs_getacl(), ufs_setacl(), ufs_aclcheck(), which respectively get, set, and check the ACLs on the passed vnode. o Introduce ufs_sync_acl_from_inode(), ufs_sync_inode_from_acl() to maintain access control information between inode permissions and extended attribute data. o Modify ufs_access() to load a file access ACL and invoke vaccess_acl_posix1e() if ACLs are available on the file system o Modify ufs_mkdir() and ufs_makeinode() to associate ACLs with newly created directories and files, inheriting from the parent directory's default ACL. o Enable these new vnode operations and conditionally compiled code paths if UFS_ACL is defined. A few notes: o This implementation is fairly widely tested, but still should be considered experimental. o Currently, ACLs are not exported via NFS, instead, the summarizing file mode/etc from the inode is. This results in conservative protection behavior, similar to the behavior of ACL-nonaware programs acting locally. o It is possible that underlying binary data formats associated with this implementation may change. Consumers of the implementation should expect to find their local configuration obsoleted in the next few months, resulting in possible loss of ACL data during an upgrade. o The extended attributes interface and implementation is still undergoing modification to address portable interface concerns, as well as performance. o Many applications do not yet correctly handle ACLs. In general, due to the POSIX.1e ACL model, behavior of ACL-unaware applications will be conservative with respects to file protection; some caution is recommended. o Instructions for configuring and maintaining ACLs on UFS will be committed in the near future; in the mean time it is possible to reference the README included in the last UFS ACL distribution placed in the TrustedBSD web site: http://www.TrustedBSD.org/downloads/ Substantial debugging, hardware, travel, or connectivity support for this project was provided by: BSDi, Safeport Network Services, and NAI Labs. Significant coding contributions were made by Chris Faulhaber. Additional support was provided by Brian Feldman, Thomas Moestl, and Ilmar Habibulin. Reviewed by: jedgar, keichii, mckusick, trustedbsd-discuss, freebsd-fs Obtained from: TrustedBSD Project
* Send the remains (such as I have located) of "block major numbers" tophk2001-03-261-1/+0
| | | | the bit-bucket.
* Fix typo ); -> ,asmodai2001-03-241-1/+1
|
* Check that background fsck operation is being done on a ufs filesystem.mckusick2001-03-231-0/+2
| | | | Obtained from: Robert Watson <rwatson@FreeBSD.org>
* o Remove an unnecessary debugging printf from ufs_extattr_lookup(),rwatson2001-03-211-1/+0
| | | | | | | | | which resulted in the output of warning messages at boot if UFS_EXTATTR_AUTOSTART was enabled but ".attribute" and possible sub-directories weren't in a mounted MFS or UFS file systems. Pointed out by: dcs Obtained from: TrustedBSD Project
* Add kernel support for running fsck on active filesystems.mckusick2001-03-216-43/+278
|
* Clear the fs_clean flag only when the FS_UNCLEAN flag is not setmckusick2001-03-211-16/+20
| | | | | | | | | (as is done in unmount). Remove a snapshot inode from the superblock list when its last name goes away rather than when its last reference goes away. That way it will be properly reclaimed by fsck after a crash rather than reenabled when the filesystem is mounted.
* Report the correct inode number when panicing with freeing free inode.mckusick2001-03-211-14/+14
| | | | Report the correct block number when panicing with freeing free block.
* o Enable UFS-based extended attribute support on MFS. Note that this changerwatson2001-03-192-0/+15
| | | | | | | | | is under-tested, and that MFS appears to be in the process of being deprecated in favor of FFS over md. Note also that UFS_EXTATTR_AUTOSTART doesn't make much sense on MFS unless the MFSROOT is compiled in, so manual configuration is generally required. Obtained from: TrustedBSD Project
* o Rename "namespace" argument to "attrnamespace" as namespace is a C++rwatson2001-03-192-41/+41
| | | | | | | reserved word. Submitted by: jkh Obtained from: TrustedBSD Project
* o Change options FFS_EXTATTR and options FFS_EXTATTR_AUTOSTART torwatson2001-03-194-21/+21
| | | | | | | | | | | | | options UFS_EXTATTR and UFS_EXTATTR_AUTOSTART respectively. This change reflects the fact that our EA support is implemented entirely at the UFS layer (modulo FFS start/stop/autostart hooks for mount and unmount events). This also better reflects the fact that [shortly] MFS will also support EAs, as well as possibly IFS. o Consumers of the EA support in FFS are reminded that as a result, they must change kernel config files to reflect the new option names. Obtained from: TrustedBSD Project
* o Caused FFS_EXTATTR_AUTOSTART to scan two sub-directories of ".attribute"rwatson2001-03-182-16/+33
| | | | | | | | | | | | | | | | | | | | | | | | off of the file system root: "user" for user attributes, and "system" for system attributes. When the scan occurs, attribute backing files discovered in those directories will be started in the respective namespaces. This re-introduces support for auto-starting of user attributes, which was removed when the "$" prefix for system attributes was replaced with explicit namespacing. For users of the TrustedBSD UFS POSIX.1e ACL code, you'll need to: mv ${FSROOT}/'$posix1e.acl_access' ${FSROOT}/system/posix1e.acl_access mv ${FSROOT}/'$posix1e.acl_default' ${FSROOT}/system/posix1e.acl_default For users of the TrustedBSD POSIX.1e Capability code, you'll need to: mv ${FSROOT}/'$posix1e.cap' ${FSROOT}/system/posix1e.cap For users of the TrustedBSD MAC code, you'll need to: mv ${FSROOT}/'$freebsd.mac' ${FSROOT}/system/freebsd.mac Updated versions of relevant patches will be released in the near future. Obtained from: TrustedBSD Project
* o Change the API and ABI of the Extended Attribute kernel interfaces torwatson2001-03-152-95/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | introduce a new argument, "namespace", rather than relying on a first- character namespace indicator. This is in line with more recent thinking on EA interfaces on various mailing lists, including the posix1e, Linux acl-devel, and trustedbsd-discuss forums. Two namespaces are defined by default, EXTATTR_NAMESPACE_SYSTEM and EXTATTR_NAMESPACE_USER, where the primary distinction lies in the access control model: user EAs are accessible based on the normal MAC and DAC file/directory protections, and system attributes are limited to kernel-originated or appropriately privileged userland requests. o These API changes occur at several levels: the namespace argument is introduced in the extattr_{get,set}_file() system call interfaces, at the vnode operation level in the vop_{get,set}extattr() interfaces, and in the UFS extended attribute implementation. Changes are also introduced in the VFS extattrctl() interface (system call, VFS, and UFS implementation), where the arguments are modified to include a namespace field, as well as modified to advoid direct access to userspace variables from below the VFS layer (in the style of recent changes to mount by adrian@FreeBSD.org). This required some cleanup and bug fixing regarding VFS locks and the VFS interface, as a vnode pointer may now be optionally submitted to the VFS_EXTATTRCTL() call. Updated documentation for the VFS interface will be committed shortly. o In the near future, the auto-starting feature will be updated to search two sub-directories to the ".attribute" directory in appropriate file systems: "user" and "system" to locate attributes intended for those namespaces, as the single filename is no longer sufficient to indicate what namespace the attribute is intended for. Until this is committed, all attributes auto-started by UFS will be placed in the EXTATTR_NAMESPACE_SYSTEM namespace. o The default POSIX.1e attribute names for ACLs and Capabilities have been updated to no longer include the '$' in their filename. As such, if you're using these features, you'll need to rename the attribute backing files to the same names without '$' symbols in front. o Note that these changes will require changes in userland, which will be committed shortly. These include modifications to the extended attribute utilities, as well as to libutil for new namespace string conversion routines. Once the matching userland changes are committed, a buildworld is recommended to update all the necessary include files and verify that the kernel and userland environments are in sync. Note: If you do not use extended attributes (most people won't), upgrading is not imperative although since the system call API has changed, the new userland extended attribute code will no longer compile with old include files. o Couple of minor cleanups while I'm there: make more code compilation conditional on FFS_EXTATTR, which should recover a bit of space on kernels running without EA's, as well as update copyright dates. Obtained from: TrustedBSD Project
* o In my merge, missed the one-line patch to ufs_vnops.c that removedrwatson2001-03-141-1/+0
| | | | | | | the static prototype for ufs_readdir(). Note that ufs_readdir() was actually already non-static, the prototype was incorrect. Submitted by: jedgar
* o Implement "options FFS_EXTATTR_AUTOSTART", which depends onrwatson2001-03-144-5/+310
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "options FFS_EXTATTR". When extended attribute auto-starting is enabled, FFS will scan the .attribute directory off of the root of each file system, as it is mounted. If .attribute exists, EA support will be started for the file system. If there are files in the directory, FFS will attempt to start them as attribute backing files for attributes baring the same name. All attributes are started before access to the file system is permitted, so this permits race-free enabling of attributes. For attributes backing support for security features, such as ACLs, MAC, Capabilities, this is vital, as it prevents the file system attributes from getting out of sync as a result of file system operations between mount-time and the enabling of the extended attribute. The userland extattrctl tool will still function exactly as previously. Files must be placed directly in .attribute, which must be directly off of the file system root: symbolic links are not permitted. FFS_EXTATTR will continue to be able to function without FFS_EXTATTR_AUTOSTART for sites that do not want/require auto-starting. If you're using the UFS_ACL code available from www.TrustedBSD.org, using FFS_EXTATTR_AUTOSTART is recommended. o This support is implemented by adding an invocation of ufs_extattr_autostart() to ffs_mountfs(). In addition, several new supporting calls are introduced in ufs_extattr.c: ufs_extattr_autostart(): start EAs on the specified mount ufs_extattr_lookup(): given a directory and filename, return the vnode for the file. ufs_extattr_enable_with_open(): invoke ufs_extattr_enable() after doing the equililent of vn_open() on the passed file. ufs_extattr_iterate_directory(): iterate over a directory, invoking ufs_extattr_lookup() and ufs_extattr_enable_with_open() on each entry. o This feature is not widely tested, and therefore may contain bugs, caution is advised. Several changes are in the pipeline for this feature, including breaking out of EA namespaces into subdirectories of .attribute (this is waiting on the updated EA API), as well as a per-filesystem flag indicating whether or not EAs should be auto-started. This is required because administrators may not want .attribute auto-started on all file systems, especially if non-administrators have write access to the root of a file system. Obtained from: TrustedBSD Project
OpenPOWER on IntegriCloud