Current implementations of sync(2) and syncer vnode fsync() VOP uses

mnt_noasync counter to temporary remove MNTK_ASYNC mount option, which
is needed to guarantee a synchronous completion of the initiated i/o
before syscall or VOP return.  Global removal of MNTK_ASYNC option is
harmful because not only i/o started from corresponding thread becomes
synchronous, but all i/o is synchronous on the filesystem which is
initiated during sync(2) or syncer activity.

Instead of removing MNTK_ASYNC from mnt_kern_flag, provide a local
thread flag to disable async i/o for current thread only. Use the
opportunity to move DOINGASYNC() macro into sys/vnode.h and
consistently use it through places which tested for MNTK_ASYNC.

Some testing demonstrated 60-70% improvements in run time for the
metadata-intensive operations on async-mounted UFS volumes, but still
with great deviation due to other reasons.

Reviewed by:	mckusick
Tested by:	scottl
MFC after:	2 weeks

1 files changed, 3 insertions, 11 deletions

diff --git a/sys/kern/vfs_syscalls.c b/sys/kern/vfs_syscalls.c
index 31ad276..e460570 100644
--- a/sys/kern/vfs_syscalls.c
+++ b/sys/kern/vfs_syscalls.c
@@ -134,7 +134,7 @@ sys_sync(td, uap)
 	struct sync_args *uap;
 {
 	struct mount *mp, *nmp;
-	int vfslocked;
+	int save, vfslocked;
 
 	mtx_lock(&mountlist_mtx);
 	for (mp = TAILQ_FIRST(&mountlist); mp != NULL; mp = nmp) {
@@ -145,18 +145,10 @@ sys_sync(td, uap)
 		vfslocked = VFS_LOCK_GIANT(mp);
 		if ((mp->mnt_flag & MNT_RDONLY) == 0 &&
 		    vn_start_write(NULL, &mp, V_NOWAIT) == 0) {
-			MNT_ILOCK(mp);
-			mp->mnt_noasync++;
-			mp->mnt_kern_flag &= ~MNTK_ASYNC;
-			MNT_IUNLOCK(mp);
+			save = curthread_pflags_set(TDP_SYNCIO);
 			vfs_msync(mp, MNT_NOWAIT);
 			VFS_SYNC(mp, MNT_NOWAIT);
-			MNT_ILOCK(mp);
-			mp->mnt_noasync--;
-			if ((mp->mnt_flag & MNT_ASYNC) != 0 &&
-			    mp->mnt_noasync == 0)
-				mp->mnt_kern_flag |= MNTK_ASYNC;
-			MNT_IUNLOCK(mp);
+			curthread_pflags_restore(save);
 			vn_finished_write(mp);
 		}
 		VFS_UNLOCK_GIANT(vfslocked);