Fix a bug that caused some /dev entries to continue to exist after

the underlying drive had been hot-unplugged from the system.  Here
is a specific example.  Filesystem code had opened /dev/da1s1e.
Subsequently, the drive was hot-unplugged.  This (correctly) caused
all of the associated /dev/da1* entries to be deleted.  When the
filesystem later realized that the drive was gone it closed the
device, reducing the write-access counts to 0 on the geom providers
for da1s1e, da1s1, and da1.  This caused geom to re-taste the
providers, resulting in the devices being created again.  When the
drive was hot-plugged back in, it resulted in duplicate /dev entries
for da1s1e, da1s1, and da1.

This fix adds a new disk_gone() function which is called by CAM when a
drive goes away.  It orphans all of the providers associated with the
drive, setting an error condition of ENXIO in each one.  In addition,
we prevent a re-taste on last close for writing if an error condition
has been set in the provider.

Sponsored by:   Isilon Systems
Reviewed by:    phk
MFC after:      1 week

3 files changed, 16 insertions, 2 deletions

diff --git a/sys/geom/geom_disk.c b/sys/geom/geom_disk.c
index 42c40bc..75d8fc4 100644
--- a/sys/geom/geom_disk.c
+++ b/sys/geom/geom_disk.c
@@ -419,6 +419,18 @@ disk_destroy(struct disk *dp)
 	g_post_event(g_disk_destroy, dp, M_WAITOK, NULL);
 }
 
+void
+disk_gone(struct disk *dp)
+{
+	struct g_geom *gp;
+	struct g_provider *pp;
+
+	gp = dp->d_geom;
+	if (gp != NULL)
+		LIST_FOREACH(pp, &gp->provider, provider)
+			g_orphan_provider(pp, ENXIO);
+}
+
 static void
 g_kern_disks(void *p, int flag __unused)
 {
diff --git a/sys/geom/geom_disk.h b/sys/geom/geom_disk.h
index 42f5b60..23719bf 100644
--- a/sys/geom/geom_disk.h
+++ b/sys/geom/geom_disk.h
@@ -95,6 +95,7 @@ struct disk {
 struct disk *disk_alloc(void);
 void disk_create(struct disk *disk, int version);
 void disk_destroy(struct disk *disk);
+void disk_gone(struct disk *disk);
 
 #define DISK_VERSION_00		0x58561059
 #define DISK_VERSION		DISK_VERSION_00
diff --git a/sys/geom/geom_subr.c b/sys/geom/geom_subr.c
index 4af0d4a..8bedee9 100644
--- a/sys/geom/geom_subr.c
+++ b/sys/geom/geom_subr.c
@@ -734,11 +734,12 @@ g_access(struct g_consumer *cp, int dcr, int dcw, int dce)
 	if (!error) {
 		/*
 		 * If we open first write, spoil any partner consumers.
-		 * If we close last write, trigger re-taste.
+		 * If we close last write and provider is not errored,
+		 * trigger re-taste.
 		 */
 		if (pp->acw == 0 && dcw != 0)
 			g_spoil(pp, cp);
-		else if (pp->acw != 0 && pp->acw == -dcw && 
+		else if (pp->acw != 0 && pp->acw == -dcw && pp->error == 0 &&
 		    !(pp->geom->flags & G_GEOM_WITHER))
 			g_post_event(g_new_provider_event, pp, M_WAITOK, 
 			    pp, NULL);