summaryrefslogtreecommitdiffstats
path: root/cddl/contrib/opensolaris/lib
Commit message (Collapse)AuthorAgeFilesLines
* MFC r332426: allow ZFS pool to have temporary name for duration of current ↵avg2018-05-031-0/+11
| | | | | | | | | | | | | import The change adds -t <name> option to zpool create and -t option to zpool import in its form with an old name and a new name. This allows to import (or create) a pool under a name that's different from its real, permanent name without affecting that name. This is useful when working with VM images or images of other physical systems if they happen to have a ZFS pool with the same name as the host system. Sponsored by: Panzura (porting)
* MFC r331707: MFV r331706:mav2018-04-163-14/+14
| | | | | | | | | | | | | | | | | | | | | | 9235 rename zpool_rewind_policy_t to zpool_load_policy_t illumos/illumos-gate@5dafeea3ebd2dd77affc802bcb90f63faf01589f We want to be able to pass various settings during import/open of a pool, which are not only related to rewind. Instead of adding a new policy and duplicate a bunch of code, we should just rename rewind_policy to a more generic term like load_policy. For instance, we'd like to set spa->spa_import_flags from the nvlist, rather from a flags parameter passed to spa_import as in some cases we want those flags not only for the import case, but also for the open case. One such flag could be ZFS_IMPORT_MISSING_LOG (as used in zdb) which would allow zfs to open a pool when logs are missing. Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Pavel Zakharov <pavel.zakharov@delphix.com>
* MFC r331701: MFV r331695, 331700: 9166 zfs storage pool checkpointmav2018-04-165-2/+148
| | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@8671400134a11c848244896ca51a7db4d0f69da4 The idea of Storage Pool Checkpoint (aka zpool checkpoint) deals with exactly that. It can be thought of as a “pool-wide snapshot” (or a variation of extreme rewind that doesn’t corrupt your data). It remembers the entire state of the pool at the point that it was taken and the user can revert back to it later or discard it. Its generic use case is an administrator that is about to perform a set of destructive actions to ZFS as part of a critical procedure. She takes a checkpoint of the pool before performing the actions, then rewinds back to it if one of them fails or puts the pool into an unexpected state. Otherwise, she discards it. With the assumption that no one else is making modifications to ZFS, she basically wraps all these actions into a “high-level transaction”. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Approved by: Richard Lowe <richlowe@richlowe.net> Author: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
* MFC r331404: MFV r331400:mav2018-04-161-0/+1
| | | | | | | | | | | | | | 8484 Implement aggregate sum and use for arc counters In pursuit of improving performance on multi-core systems, we should implements fanned out counters and use them to improve the performance of some of the arc statistics. These stats are updated extremely frequently, and can consume a significant amount of CPU time. Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: Paul Dagnelie <pcd@delphix.com>
* MFC r329808: MFV r329807:mav2018-04-162-5/+10
| | | | | | | | | | | | | | | | | | | | | | | | | 8940 Sending an intra-pool resumable send stream may result in EXDEV illumos/illumos-gate@544132fce3fa6583f01318f9559adc46614343a7 "zfs send -t <token>" for an incremental send should be able to resume successfully when sending to the same pool: a subtle issue in zfs_iter_children() doesn't currently allow this. Because resuming from a token requires "guid" -> "dataset" mapping (guid_to_name()), we have to walk the whole hierarchy to find the right snapshots to send. When resuming an incremental send both source and destination live in the same pool and have the same guid: this is where zfs_iter_children() gets confused and picks up the wrong snapshot, so we end up trying to send an incremental "destination@snap1 -> source@snap2" stream instead of "source@snap1 -> source@snap2": this fails with an "Invalid cross-device link" (EXDEV) error. Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org> Author: loli10K <ezomori.nozomu@gmail.com>
* MFC r329798: MFV r329793, r329795:mav2018-04-165-4/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 9075 Improve ZFS pool import/load process and corrupted pool recovery illumos/illumos-gate@6f7938128a2c5e23f4b970ea101137eadd1470a1 Some work has been done lately to improve the debugability of the ZFS pool load (and import) process. This includes: https://www.illumos.org/issues/7638: Refactor spa_load_impl into several functions https://www.illumos.org/issues/8961: SPA load/import should tell us why it failed https://www.illumos.org/issues/7277: zdb should be able to print zfs_dbgmsg's To iterate on top of that, there's a few changes that were made to make the import process more resilient and crash free. One of the first tasks during the pool load process is to parse a config provided from userland that describes what devices the pool is composed of. A vdev tree is generated from that config, and then all the vdevs are opened. The Meta Object Set (MOS) of the pool is accessed, and several metadata objects that are necessary to load the pool are read. The exact configuration of the pool is also stored inside the MOS. Since the configuration provided from userland is external and might not accurately describe the vdev tree of the pool at the txg that is being loaded, it cannot be relied upon to safely operate the pool. For that reason, the configuration in the MOS is read early on. In the past, the two configurations were compared together and if there was a mismatch then the load process was aborted and an error was returned. The latter was a good way to ensure a pool does not get corrupted, however it made the pool load process needlessly fragile in cases where the vdev configuration changed or the userland configuration was outdated. Since the MOS is stored in 3 copies, the configuration provided by userland doesn't have to be perfect in order to read its contents. Hence, a new approach has been adopted: The pool is first opened with the untrusted userland configuration just so that the real configuration can be read from the MOS. The trusted MOS configuration is then used to generate a new vdev tree and the pool is re-opened. When the pool is opened with an untrusted configuration, writes are disabled to avoid accidentally damaging it. During reads, some sanity checks are performed on block pointers to see if each DVA points to a known vdev; when the configuration is untrusted, instead of panicking the system if those checks fail we simply avoid issuing reads to the invalid DVAs. This new two-step pool load process now allows rewinding pools accross vdev tree changes such as device replacement, addition, etc. Loading a pool from an external config file in a clustering environment also becomes much safer now since the pool will import even if the config is outdated and didn't, for instance, register a recent device addition. With this code in place, it became relatively easy to implement a long-sought-after feature: the ability to import a pool with missing top level (i.e. non-redundant) devices. Note that since this almost guarantees some loss Of data, this feature is for now restricted to a read-only import. Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Andrew Stormont <andyjstormont@gmail.com> Approved by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org> Author: Pavel Zakharov <pavel.zakharov@delphix.com>
* MFC r329783: 8942 zfs promote .../%recv should be an errormav2018-04-161-0/+7
| | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@add927f8c8d101e16c23eb9cd270be4fd7edf7d5 Reported on the ZFSonLinux https://github.com/zfsonlinux/zfs/issues/4843, fixed by https://github.com/zfsonlinux/zfs/pull/6339: If we are in the middle of an incremental zfs receive, the child .../%recv will exist. If you concurrently run zfs promote .../%recv, it will "work", but then zfs gets confused. For example, there's no obvious way to destroy the containing filesystem (because it is now a clone of its invisible child). Attempting to do this promote should be an error. We could fix this by having zfs_ioc_promote() check if zc_name contains a %, similar to zfs_ioc_rename(). Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: loli10K <ezomori.nozomu@gmail.com>
* MFC r329759:mav2018-04-161-1/+2
| | | | | | | | | | | | | | | | | | | | | 9018 Replace kmem_cache_reap_now() with kmem_cache_reap_soon() illumos/illumos-gate@36a64e62848b51ac5a9a5216e894ec723cfef14e To prevent kmem_cache reaping from blocking other system resources, turn kmem_cache_reap_now() (which blocks) into kmem_cache_reap_soon(). Callers to kmem_cache_reap_soon() should use kmem_cache_reap_active(), which exploits #9017's new taskq_empty(). Reviewed by: Bryan Cantrill <bryan@joyent.com> Reviewed by: Dan McDonald <danmcd@joyent.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Yuri Pankov <yuripv@yuripv.net> Author: Tim Kordas <tim.kordas@joyent.com> FreeBSD does not use taskqueue for kmem caches reaping, so this change is less dramatic then it is on Illumos, just limiting reaping to 1 time per second. It may possibly be improved later, if needed.
* MFC r329732: MFV r329502: 7614 zfs device evacuation/removalmav2018-04-166-20/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@5cabbc6b49070407fb9610cfe73d4c0e0dea3e77 https://www.illumos.org/issues/7614: This project allows top-level vdevs to be removed from the storage pool with “zpool remove”, reducing the total amount of storage in the pool. This operation copies all allocated regions of the device to be removed onto other devices, recording the mapping from old to new location. After the removal is complete, read and free operations to the removed (now “indirect”) vdev must be remapped and performed at the new location on disk. The indirect mapping table is kept in memory whenever the pool is loaded, so there is minimal performance overhead when doing operations on the indirect vdev. The size of the in-memory mapping table will be reduced when its entries become “obsolete” because they are no longer used by any block pointers in the pool. An entry becomes obsolete when all the blocks that use it are freed. An entry can also become obsolete when all the snapshots that reference it are deleted, and the block pointers that reference it have been “remapped” in all filesystems/zvols (and clones). Whenever an indirect block is written, all the block pointers in it will be “remapped” to their new (concrete) locations if possible. This process can be accelerated by using the “zfs remap” command to proactively rewrite all indirect blocks that reference indirect (removed) vdevs. Note that when a device is removed, we do not verify the checksum of the data that is copied. This makes the process much faster, but if it were used on redundant vdevs (i.e. mirror or raidz vdevs), it would be possible to copy the wrong data, when we have the correct data on e.g. the other side of the mirror. Therefore, mirror and raidz devices can not be removed. Reviewed by: Alex Reece <alex@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Richard Laager <rlaager@wiktel.com> Reviewed by: Tim Chase <tim@chase2k.com> Approved by: Garrett D'Amore <garrett@damore.org> Author: Prashanth Sreenivasa <pks@delphix.com>
* MFC r330295: ZFS: fix adding vdevs to very large poolsavg2018-04-061-0/+1
| | | | PR: 226096
* Revert r331551. It is causing perl and tcl port build failures.markj2018-03-281-3/+2
| | | | Reported by: antoine
* MFC r331222:markj2018-03-261-2/+3
| | | | Given hidden visibility to symbols referenced by the DOF section.
* MFC r329691: MFV r322231:mav2018-03-221-0/+1
| | | | | | | | | | | | | | | | | | 8430 dir_is_empty_readdir() doesn't properly handle error from fdopendir() illumos/illumos-gate@ba6e7e6505150388de6dc6a88741164118a421bf https://github.com/illumos/illumos-gate/commit/ba6e7e6505150388de6dc6a88741164118a421bf https://www.illumos.org/issues/8430 we should close dirfd if fdopendir() fails. Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com> Reviewed by: Igor Kozhukhov <igor@dilos.org> Approved by: Robert Mustacchi <rm@joyent.com> Author: Sowrabha Gopal <sowrabha.gopal@delphix.com>
* MFC r329681: MFV r318941: 7446 zpool create should support efi system partitionmav2018-03-222-31/+155
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@7855d95b30fd903e3918bad5a29b777e765db821 https://github.com/illumos/illumos-gate/commit/7855d95b30fd903e3918bad5a29b777e765db821 https://www.illumos.org/issues/7446 Since we support whole-disk configuration for boot pool, we also will need whole disk support with UEFI boot and for this, zpool create should create efi- system partition. I have borrowed the idea from oracle solaris, and introducing zpool create - B switch to provide an way to specify that boot partition should be created. However, there is still an question, how big should the system partition be. For time being, I have set default size 256MB (thats minimum size for FAT32 with 4k blocks). To support custom size, the set on creation "bootsize" property is created and so the custom size can be set as: zpool create B - o bootsize=34MB rpool c0t0d0 After pool is created, the "bootsize" property is read only. When -B switch is not used, the bootsize defaults to 0 and is shown in zpool get output with value ''. Older zfs/zpool implementations are ignoring this property. https://www.illumos.org/rb/r/219/ Reviewed by: Andrew Stormont <andyjstormont@gmail.com> Reviewed by: Yuri Pankov <yuri.pankov@gmail.com> Approved by: Dan McDonald <danmcd@kebe.com> Author: Toomas Soome <tsoome@me.com> This commit makes no sense for FreeBSD, that is why I blocked the option, but it should be good to stay closer to upstream.
* MFC r329668: MFV r316918:mav2018-03-221-7/+4
| | | | | | | | | | | | | | | 7990 libzfs: snapspec_cb() does not need to call zfs_strdup() illumos/illumos-gate@d8584ba6fb7a5e46da1725845b99ae5fab5a4baf https://github.com/illumos/illumos-gate/commit/d8584ba6fb7a5e46da1725845b99ae5fab5a4baf https://www.illumos.org/issues/7990 The snapspec_cb() callback function in libzfs does not need to call zfs_strdup(). Reviewed by: Yuri Pankov <yuri.pankov@gmail.com> Reviewed by: Toomas Soome <tsoome@me.com> Approved by: Matthew Ahrens <mahrens@delphix.com> Author: Marcel Telka <marcel@telka.sk>
* MFC r329667: MFV r316902:mav2018-03-222-3/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 7745 print error if lzc_* is called before libzfs_core_init illumos/illumos-gate@7c13517fff71be473e47531ef4330160c042bedc https://github.com/illumos/illumos-gate/commit/7c13517fff71be473e47531ef4330160c042bedc https://www.illumos.org/issues/7745 The problem is that consumers of `libZFS_Core` that forget to call `libzfs_core_init()` before calling any other function of the library are having a hard time realizing their mistake. The library's internal file descriptor is declared as global static, which is ok, but it is not initialized explicitly; therefore, it defaults to 0, which is a valid file descriptor. If `libzfs_core_init()`, which explicitly initializes the correct fd, is skipped, the ioctl functions return errors that do not have anything to do with `libZFS_Core`, where the problem is actually located. Even though assertions for that existed within `libZFS_Core` for debug builds, they were never enabled because the `-DDEBUG` flag was missing from the compiler flags. This patch applies the following changes: 1. It adds `-DDEBUG` for debug builds of `libZFS_Core` and `libzfs`, to enable their assertions on debug builds. 2. It corrects an assertion within `libzfs`, where a function had been spelled incorrectly (`zpool_prop_unsupported()`) and nobody knew because the `-DDEBUG` flag was missing, and the preprocessor was taking that part of the code away. 3. The library's internal fd is initialized to `-1` and `VERIFY` assertions have been placed to check that the fd is not equal to `-1` before issuing any ioctl. It is important here to note, that the `VERIFY` assertions exist in both debug and non-debug builds. 4. In `libzfs_core_fini` we make sure to never increment the refcount of our fd below 0, and also reset the fd to `-1` when no one refers to it. The reason for this, is for the rare case that the consumer closes all references but then calls one of the library's functions without using `libzfs_core_init()` first, and in the mean time, a previous call to `open()` decided to reuse our previous fd. This scenario would have passed our assertion in Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Serapheim Dimitropoulos <serapheim@delphix.com>
* MFC r329665: MFV r316901:mav2018-03-221-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 7730 libzfs`add_config() leaks config nvl when reading spare/l2cache devices illumos/illumos-gate@105686550ee9cbf5d033166a8a2a5a763667d436 https://github.com/illumos/illumos-gate/commit/105686550ee9cbf5d033166a8a2a5a763667d436 https://www.illumos.org/issues/7730 antares:root:~# mdb /usr/sbin/zpool > ::sysbp _exit > ::run import pool: data id: 2093977168778024605 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: data ONLINE c6t0d0 ONLINE c6t1d0 ONLINE cache c6t2d0 mdb: stop on entry to _exit mdb: target stopped at: 0xfee556ba: nop mdb: You've got symbols! Loading modules: [ ld.so.1 libumem.so.1 libc.so.1 libtopo.so.1 libavl.so.1 libnvpair.so.1 ] > ::findleaks -d BYTES LEAKED VMEM_SEG CALLER 4096 10 fda7b000 MMAP 8192 1 fea8d000 MMAP 8192 1 fe76d000 MMAP 8192 1 fe66e000 MMAP 4096 1 fe570000 MMAP 8192 1 fe470000 MMAP 4096 1 fe372000 MMAP 4096 1 fe273000 MMAP Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Yuri Pankov <yuri.pankov@nexenta.com>
* MFC r329664: MFV r316893:mav2018-03-221-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 7604 if volblocksize property is the default, it displays as "-" rather than 8K illumos/illumos-gate@4d86c0eab246bdfddc2dd52410ba808433bd6266 https://github.com/illumos/illumos-gate/commit/4d86c0eab246bdfddc2dd52410ba808433bd6266 https://www.illumos.org/issues/7604 If a zvol has the default setting for the "volblocksize" property, it is 8KB. However, it is displayed as "-" (not present), rather than "8K". The problem was introduced by: commit 25228e830e86924a41243343b1de9daf2d7dd43a Author: Matthew Ahrens &lt;mahrens@delphix.com&gt; Date: Thu Nov 17 14:37:24 2016 -0800 7571 non-present readonly numeric ZFS props do not have default value which changed changed get_numeric_property() to indicate that readonly default properties are not present. However, zfs_prop_readonly() returns TRUE for both readonly and set-once properties (e.g. volblocksize). Amusingly, that commit essentially reverted: 6900484 default volblocksize is no longer being reported correctly from November 2009. However, that change was not correct either; the correct solution is to only do this check for "truly readonly" (i.e. not setonce) properties. $ zfs list -t volume -o name,volblocksize NAME VOLBLOCK domain0/group-100/appdata_container-101/appdata_windows_timeflow-102/ archive - domain0/group-100/appdata_container-101/appdata_windows_timeflow-102/ datafile - domain0/group-100/appdata_container-101/appdata_windows_timeflow-102/ external - rpool/dump 128K rpool/swap 4K rpool/swap1 =============================================================================== Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Matthew Ahrens <mahrens@delphix.com>
* MFC r329663: MFV r316876: 7542 zfs_unmount failed with EZFS_UNSHARENFSFAILEDmav2018-03-222-27/+18
| | | | | | | | | | | | | | | | | illumos/illumos-gate@09c9e6dc9b69d10b771bb87e01040ec320a0bfd3 https://github.com/illumos/illumos-gate/commit/09c9e6dc9b69d10b771bb87e01040ec320a0bfd3 https://www.illumos.org/issues/7542 libshare keeps a cached copy of the sharetab listing in memory, which can become out of date if shares are destroyed or created while leaving a libzfs handle open. This results in a spurious unmounting failure when an NFS share exists but isn't in the stale libshare cache. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matt Amdur <matt.amdur@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Chris Williamson <chris.williamson@delphix.com>
* MFC r329661: MFV r316875: 7336 vfork and O_CLOEXEC causes zfs_mount EBUSYmav2018-03-221-6/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@873c4903a52d089cd8234b79d24f5a3fc3bccc82 https://github.com/illumos/illumos-gate/commit/873c4903a52d089cd8234b79d24f5a3fc3bccc82 https://www.illumos.org/issues/7336 We can run into a problem where we call into zfs_mount, which in turn calls is_dir_empty, which opens the directory to try and make sure it's empty. The issue with the current approach is that it holds the directory open while it traverses it with readdir, which, due to subtle interaction with the Java JVM, vfork, and exec can cause a tricky race condition resulting in zfs_mount failures. The approach to resolving the issue in this patch is to drop the usage of readdir altogether, and instead rely on the fact that ZFS stores the number of entries contained in a directory using the st_size field of the stat structure. Thus, if the directory in question is a ZFS directory, we can check to see if it's empty by calling stat() and inspecting the st_size field of structure returned. =============================================================================== The root cause appears to be an interesting race between vfork, exec, and zfs_mount's usage of O_CLOEXEC when calling openat. Here's what is going on: 1. We call zfs_mount, and this in turn calls openat to check if the directory is empty, which results in opening the directory we're trying to mount onto, and increment v_count. 2. As we're in the middle of reading the directory, vfork is called by the JVM and proceeds to exec the jspawnhelper utility. As a result of the vfork, we take an additional hold on the directory, which increments v_count a second time. The semantics of vfork mean the parent process will wait for the child process to exit or exec before the parent can continue; at this point the parent is in the middle of zfs_mount, reading the directory to determine if it's empty or not. 3. The child process exec-ing jspawnhelper gets to the relvm call within exec_args (which is called by exec_common). relvm is the function that releases the parent process, allowing the parent to proceed. The problem is, at this point of calling relvm, the child hasn't yet called close_exec which is responsible for closing the file descriptors inherited from the parent process Reviewed by: Matt Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Robert Mustacchi <rm@joyent.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Prakash Surya <prakash.surya@delphix.com>
* MFC r329659: MFV r316873: 7233 dir_is_empty should open directory with CLOEXECmav2018-03-221-2/+10
| | | | | | | | | | | | | | | | illumos/illumos-gate@d420209d9c807f782c1d31f5683be74798142198 https://github.com/illumos/illumos-gate/commit/d420209d9c807f782c1d31f5683be74798142198 https://www.illumos.org/issues/7233 This fixes a race where one thread is executing zfs_mount() while another thread forks and execs. If the fork occurs while the directory is open, the child process will inherit (but not necessarily close immediately) the open fd for the directory, preventing the mount. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Approved by: Richard Lowe <richlowe@richlowe.net> Author: Alex Reece <alex@delphix.com>
* MFC r329508: MFV r324198: 8081 Compiler warnings in zdbmav2018-03-221-1/+1
| | | | | | | | | | | | | | | | illumos/illumos-gate@3f7978d02b206a6ebc5652c91aa9f42da6fbe00c https://github.com/illumos/illumos-gate/commit/3f7978d02b206a6ebc5652c91aa9f42da6fbe00c https://www.illumos.org/issues/8081 zdb(8) is full of minor problems that generate compiler warnings. On FreeBSD, which uses -WError, the only way to build it is to disable all compiler warnings. This makes it much harder to detect newly introduced bugs. We should cleanup all the warnings. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Approved by: Richard Lowe <richlowe@richlowe.net> Author: Alan Somers <asomers@gmail.com>
* MFC r329505: MFV r323911:mav2018-03-221-1/+8
| | | | | | | | | | | | | | | | | | | | | 8502 illumos#7955 broke delegated datasets when libshare is not present illumos/illumos-gate@1c18e8fbd8db41a9fb39bd3ef7a18ee71ece20da https://github.com/illumos/illumos-gate/commit/1c18e8fbd8db41a9fb39bd3ef7a18ee71ece20da https://www.illumos.org/issues/8502 The code in lib/libzfs/common/libzfs_mount.c already basically handles the case when libshare is not installed. We just need to not fail in zfs_init_libshare_impl. I tested this in lx and things work as expected. I also tested there trying to set sharenfs and sharesmb on the delegated dataset. Neither is allowed from within a zone. The spew of msgs from a native zone is not ZFS specific. I see the same spew simply running the share command. Reviewed by: Robert Mustacchi <rm@joyent.com> Reviewed by: Yuri Pankov <yuripv@gmx.com> Approved by: Richard Lowe <richlowe@richlowe.net> Author: Jerry Jelinek <jerry.jelinek@joyent.com>
* MFC r329719: MFV r329718: 8520 7198 lzc_rollback_to should support rolling ↵avg2018-03-071-9/+23
| | | | back to origin
* MFC r328252: MFV r328251: 8652 Tautological comparisons with ZPROP_INVALmav2018-02-181-2/+2
| | | | | | | | | | | | | | | illumos/illumos-gate@4ae5f5f06c6c2d1db8167480f7d9e3b5378ba2f2 https://www.illumos.org/issues/8652: Clang and GCC prefer to use unsigned ints to store enums. With Clang, that causes tautological comparison warnings when comparing a zfs_prop_t or zpool_prop_t variable to the macro ZPROP_INVAL. It's likely that error handling code is being silently removed as a result. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Igor Kozhukhov <igor@dilos.org> Approved by: Gordon Ross <gwr@nexenta.com> Author: Alan Somers <asomers@gmail.com>
* MFC r328250: MFV r328249:mav2018-02-181-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 8641 "zpool clear" and "zinject" don't work on "spare" or "replacing" vdevs illumos/illumos-gate@2ba5f978a4f9b02da9db1b8cdd9ea5498eb00ad9 https://www.illumos.org/issues/8641: "zpool clear" and "zinject -d" can both operate on specific vdevs, either leaf or interior. However, due to an oversight, neither works on a "spare" or "replacing" vdev. For example: sudo zpool create foo raidz1 c1t5000CCA000081D61d0 c1t5000CCA000186235d0 spare c 1t5000CCA000094115d0 sudo zpool replace foo c1t5000CCA000186235d0 c1t5000CCA000094115d0 $ zpool status foo pool: foo state: ONLINE scan: resilvered 81.5K in 0h0m with 0 errors on Fri Sep 8 10:53:03 2017 config: NAME STATE READ WRITE CKSUM foo ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 c1t5000CCA000081D61d0 ONLINE 0 0 0 spare-1 ONLINE 0 0 0 c1t5000CCA000186235d0 ONLINE 0 0 0 c1t5000CCA000094115d0 ONLINE 0 0 0 spares c1t5000CCA000094115d0 INUSE currently in use $ sudo zinject -d spare-1 -A degrade foo cannot find device 'spare-1' in pool 'foo' $ sudo zpool clear foo spare-1 cannot clear errors for spare-1: no such device in pool Even though there was nothing to clear, those commands shouldn't have reported an error. by contrast, trying to clear "raidz1-0" works just fine: $ sudo zpool clear foo raidz1-0 Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Gordon Ross <gwr@nexenta.com> Author: Alan Somers <asomers@gmail.com>
* MFC r328234: MFV r328233:mav2018-02-181-1/+5
| | | | | | | | | | | | | | | | | | | | 8898 creating fs with checksum=skein on the boot pools fails ungracefully illumos/illumos-gate@9fa2266d9a78b8366e1cd2d5f050e8b5e37d558c https://www.illumos.org/issues/8898: # zfs create -o checksum=skein rpool/test internal error: Result too large Abort (core dumped) Not a big deal per se, but should be handled correctly. Reviewed by: Toomas Soome <tsoome@me.com> Reviewed by: Andy Stormont <astormont@racktopsystems.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: Yuri Pankov <yuri.pankov@nexenta.com> PR: 222199
* MFC r328232:mav2018-02-181-5/+4
| | | | | | | | | | | | | | | | | | | | MFV r328231: 8897 zpool online -e fails assertion when run on non-leaf vdevs illumos/illumos-gate@9a551dd645b478816cb11251b19f5034d885bf01 https://www.illumos.org/issues/8897: # zpool online -e test mirror-1 Assertion failed: nvlist_lookup_string(tgt, "path", &pathname) == 0, file ../common/libzfs_pool.c, line 2558, function zpool_vdev_online Abort (core dumped) Not a big deal per se, but should be handled gracefully, same way as 'offline' and 'online' without '-e'. Also reported as: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221408 Reviewed by: Toomas Soome <tsoome@me.com> Reviewed by: Igor Kozhukhov <igor@dilos.org> Approved by: Dan McDonald <danmcd@joyent.com> Author: Yuri Pankov <yuri.pankov@nexenta.com>
* MFC r328224: MFV r328220: 8677 Open-Context Channel Programsmav2018-02-183-14/+45
| | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@a3b2868063897ff0083dea538f55f9873eec981f https://www.illumos.org/issues/8677 We want to be able to run channel programs outside of synching context. This would greatly improve performance of channel program that just gather information, as we won't have to wait for synching context anymore. This feature should introduce the following: - A new command line flag in "zfs program" to specify our intention to run in open context. - A new flag/option within the channel program ioctl which selects the context. - Appropriate error handling whenever we try a channel program in open-context that contains zfs.sync* expressions. - Documentation for the new feature in the manual pages. Reviewed by: Matt Ahrens <mahrens@delphix.com> Reviewed by: Chris Williamson <chris.williamson@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Serapheim Dimitropoulos <serapheim@delphix.com>
* Add DT_VERS_1_13 to libdtrace to unbreak the build.markj2018-02-141-2/+4
| | | | | | This is a direct commit to stable/11. PR: 225889
* MFC r327888, r327972, r327973:markj2018-02-131-2/+6
| | | | Add "jid" and "jailname" variables to DTrace.
* MFC r326177:markj2017-12-041-1/+1
| | | | Fix the type signature for sx(9) DTrace subroutines.
* MFC r326046: dt_modtext: return error on archs lacking an implementationemaste2017-11-281-4/+4
| | | | | | Reported by: mmel Reviewed by: markj Sponsored by: The FreeBSD Foundation
* MFC r325042: libdtrace: replace "DOODAD" with more descriptive stringemaste2017-11-282-9/+18
| | | | | | | | Previously some unimplemented libdtrace routines printed the function, file and line number, followed by "DOODAD." That is not particularly informative, so replace it with a message reporting the actual issue. Sponsored by: The FreeBSD Foundation
* MFC r322854, r323995, r324568, r324991asomers2017-11-282-0/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | r322854: zfsd(8): Close a race condition when onlining a disk paritition When inserting a partitioned disk, devfs and geom will announce the whole disk before they announce the partition. If the partition containing ZFS extends to one of the disk's extents, then zfsd will see a ZFS label on the whole disk and attempt to online it. ZFS is smart enough to activate the partition instead of the whole disk, but only if GEOM has already created the partition's provider. cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c Add a zpool_read_all_labels method. It's similar to zpool_read_label, but it will return the number of labels found. cddl/usr.sbin/zfsd/zfsd_event.cc When processing a DevFS CREATE event, only online a VDEV if we can read all four ZFS labels. Reviewed by: mav Sponsored by: Spectra Logic Corp Differential Revision: https://reviews.freebsd.org/D11920 r323995: Close a memory leak when using zpool_read_all_labels X-MFC-With: 322854 Sponsored by: Spectra Logic Corp r324568: Optimize zpool_read_all_labels with AIO Read all labels in parallel instead of sequentially X-MFC-With: 322854 Sponsored by: Spectra Logic Corp Differential Revision: https://reviews.freebsd.org/D12495 r324991: Fix zpool_read_all_labels when vfs.aio.enable_unsafe=0 Previously, zpool_read_all_labels was trying to do 256KB reads, which are greater than the default MAXPHYS and therefore must go through the slow, unsafe AIO path. Shrink these reads to 112KB so they can use the safe, fast AIO path instead. X-MFC-With: 324568 Sponsored by: Spectra Logic Corp
* MFC r324864, r324865:markj2017-11-231-234/+318
| | | | Cleanups for ctf.5.
* MFC r325035: MFV r325013,r325034: 640 number_to_scaled_string is duplicated ↵avg2017-11-165-73/+379
| | | | | | | | | | | | in several commands FreeBSD note: of all libcmdutils functionality ZFS (and other illumos contrib code) currently uses only nicenum() function (which is similar to humanize_number but has some formatting differences). For this reason I decided to not port the whole library. As a result, nicenum.c from libcmdutils is compiled into libzfs and libzpool. This is a bit ugly, but works. If one day we are forced to create libillumos, then the file should be moved to that library.
* MFC r324197: MFV r323913: 8600 ZFS channel programs - snapshotavg2017-11-081-3/+8
|
* MFC r324163: MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and ↵avg2017-11-087-10/+164
| | | | | | | | | | followups Also MFC-ed are the following fixes: - r324164 Add several new files to the files enabled by ZFS kernel option - r324178 unbreak kernel builds on sparc64 and powerpc - r324194 fix incorrect use of getzfsvfs_impl in r324163 - r324292 really unbreak kernel builds on sparc64 and powerpc64
* MFC r324348: MFV r316934: 7340 receive manual origin should override ↵avg2017-10-301-6/+6
| | | | automatic origin
* MFC r324347: MFV r316933: 5142 libzfs support raidz root pool (loader project)avg2017-10-301-0/+1
| | | | | FreeBSD note: we have long supported this feature, this commit only removes a small difference in libzfs.
* MFC r324346: MFV r316931: 6268 zfs diff confused by moving a file to another ↵avg2017-10-301-12/+1
| | | | directory
* MFC r324345: MFV r316877: 7571 non-present readonly numeric ZFS props do not ↵avg2017-10-301-1/+2
| | | | have default value
* MFC r324220:asomers2017-10-252-1/+59
| | | | | | | | | | | | | | | | | | | | | | MFV r316858 7280 Allow changing global libzpool variables in zdb 7280 Allow changing global libzpool variables in zdb and ztest through command line illumos/illumos-gate@0e60744c982adecd0a1f146f5637475d07ab1069 https://github.com/illumos/illumos-gate/commit/0e60744c982adecd0a1f146f5637475d07ab1069 https://www.illumos.org/issues/7280 zdb is very handy for diagnosing problems with a pool in a safe and quick way. When a pool is in a bad shape, we often want to disable some fail-safes, or adjust some tunables in order to open them. In the kernel, this is done by changing public variables in mdb. The goal of this feature is to add the same capability to zdb and ztest, so that they can change libzpool tuneables from the command line. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Pavel Zakharov <pavel.zakharov@delphix.com>
* MFC r324373:markj2017-10-132-2/+2
| | | | Avoid adding an extra "0x" prefix before pointer formats.
* MFC r323528: MFV r323527: 5815 libzpool's panic function doesn't set global ↵avg2017-10-131-5/+3
| | | | panicstr
* MFC r323525: MFV r323523: 8331 zfs_unshare returns wrong error code for smb ↵avg2017-10-131-4/+6
| | | | unshare failure
* MFC r323524: MFV r316932: 6280 libzfs: unshare_one() could fail with ↵avg2017-10-131-1/+2
| | | | EZFS_SHARENFSFAILED
* MFC r323791: MFV r323790: 8567 Inconsistent return value in zpool_read_labelavg2017-10-041-4/+5
|
* MFC r323355: MFV r323107: 8414 Implemented zpool scrub pause/resumeavg2017-09-263-12/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@1702cce751c5cb7ead878d0205a6c90b027e3de8 https://github.com/illumos/illumos-gate/commit/1702cce751c5cb7ead878d0205a6c90b027e3de8 FreeBSD note: rather than merging the zpool.8 update I copied the zpool scrub section from the illumos zpool.1m to FreeBSD zpool.8 almost verbatim. Now that the illumos page uses the mdoc format, it was an easier option. Perhaps the change is not in perfect compliance with the FreeBSD style, but I think that it is acceptible. https://www.illumos.org/issues/8414 This issue tracks the port of scrub pause from ZoL: https://github.com/zfsonlinux/zfs/pull/6167 Currently, there is no way to pause a scrub. Pausing may be useful when the pool is busy with other I/O to preserve bandwidth. Description This patch adds the ability to pause and resume scrubbing. This is achieved by maintaining a persistent on-disk scrub state. While the state is 'paused' we do not scrub any more blocks. We do however perform regular scan housekeeping such as freeing async destroyed and deadlist blocks while paused. Motivation and Context Scrub pausing can be an I/O intensive operation and people have been asking for the ability to pause a scrub for a while. This allows one to preserve scrub progress while freeing up bandwidth for other I/O. Reviewed by: George Melikov <mail@gmelikov.ru> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed by: Brad Lewis <brad.lewis@delphix.com> Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: Alek Pinchuk <apinchuk@datto.com>
OpenPOWER on IntegriCloud