Close a race with caching of -ve name lookups in the NFS client.

Specifically, clients only trust -ve cache entries while the directory
remains unchanged and discard any -ve cache entries for a directory when
they notice that the modification time of a directory entry changes.  The
race involves two concurrent lookups as follows:
- Thread A does a lookup for file 'foo' which sends a lookup RPC to the
  server.  The lookup fails and the server replies.
- The 'foo' file is created (either by the same client or a different
  client) updating the modification time on the parent directory of 'foo'.
- Thread B does a lookup for a different file 'bar' which updates the
  cached attributes of the parent directory of 'foo' to reflect the new
  modification time after 'foo' was created.
- Thread A finally resumes execution to parse the reply from the NFS
  server.  It adds a -ve cache entry and sets the cached value of the
  directory's modification time that is used for invalidating -ve cached
  lookups to the new modification time set by thread B.

At this point, future lookups of 'foo' will honor the -ve cached entry
until the cached entry is pushed out of the name cache's LRU or the
modification time of the parent directory is changed again by some other
change.  The fix is to read the directory's modification time before
sending the lookup RPC and use that cached modification time when setting
the directory's cached modification time.  Also, we do not add a -ve cache
entry if another thread has added -ve cache entry that set the directory's
cached modification time to a newer value than the value we read before
sending the lookup RPC.

Reviewed by:	rmacklem
MFC after:	1 week

1 files changed, 32 insertions, 6 deletions

diff --git a/sys/nfsclient/nfs_vnops.c b/sys/nfsclient/nfs_vnops.c
index 7dfd298..dc61927 100644
--- a/sys/nfsclient/nfs_vnops.c
+++ b/sys/nfsclient/nfs_vnops.c
@@ -924,6 +924,7 @@ nfs_lookup(struct vop_lookup_args *ap)
 	struct vnode **vpp = ap->a_vpp;
 	struct mount *mp = dvp->v_mount;
 	struct vattr vattr;
+	time_t dmtime;
 	int flags = cnp->cn_flags;
 	struct vnode *newvp;
 	struct nfsmount *nmp;
@@ -935,7 +936,7 @@ nfs_lookup(struct vop_lookup_args *ap)
 	int error = 0, attrflag, fhsize, ltype;
 	int v3 = NFS_ISV3(dvp);
 	struct thread *td = cnp->cn_thread;
-	
+
 	*vpp = NULLVP;
 	if ((flags & ISLASTCN) && (mp->mnt_flag & MNT_RDONLY) &&
 	    (cnp->cn_nameiop == DELETE || cnp->cn_nameiop == RENAME))
@@ -992,6 +993,19 @@ nfs_lookup(struct vop_lookup_args *ap)
 		np->n_dmtime = 0;
 		mtx_unlock(&np->n_mtx);
 	}
+
+	/*
+	 * Cache the modification time of the parent directory in case
+	 * the lookup fails and results in adding the first negative
+	 * name cache entry for the directory.  Since this is reading
+	 * a single time_t, don't bother with locking.  The
+	 * modification time may be a bit stale, but it must be read
+	 * before performing the lookup RPC to prevent a race where
+	 * another lookup updates the timestamp on the directory after
+	 * the lookup RPC has been performed on the server but before
+	 * n_dmtime is set at the end of this function.
+	 */
+	dmtime = np->n_vattr.va_mtime.tv_sec;
 	error = 0;
 	newvp = NULLVP;
 	nfsstats.lookupcache_misses++;
@@ -1130,13 +1144,25 @@ nfsmout:
 			 * Maintain n_dmtime as the modification time
 			 * of the parent directory when the oldest -ve
 			 * name cache entry for this directory was
-			 * added.
+			 * added.  If a -ve cache entry has already
+			 * been added with a newer modification time
+			 * by a concurrent lookup, then don't bother
+			 * adding a cache entry.  The modification
+			 * time of the directory might have changed
+			 * due to the file this lookup failed to find
+			 * being created.  In that case a subsequent
+			 * lookup would incorrectly use the entry
+			 * added here instead of doing an extra
+			 * lookup.
 			 */
 			mtx_lock(&np->n_mtx);
-			if (np->n_dmtime == 0)
-				np->n_dmtime = np->n_vattr.va_mtime.tv_sec;
-			mtx_unlock(&np->n_mtx);
-			cache_enter(dvp, NULL, cnp);
+			if (np->n_dmtime <= dmtime) {
+				if (np->n_dmtime == 0)
+					np->n_dmtime = dmtime;
+				mtx_unlock(&np->n_mtx);
+				cache_enter(dvp, NULL, cnp);
+			} else
+				mtx_unlock(&np->n_mtx);
 		}
 		return (ENOENT);
 	}