In the common case, pmap_enter_quick() completes without sleeping.

In such cases, the busying of the page and the unlocking of the
containing object by vm_map_pmap_enter() and vm_fault_prefault() is
unnecessary overhead.  To eliminate this overhead, this change
modifies pmap_enter_quick() so that it expects the object to be locked
on entry and it assumes the responsibility for busying the page and
unlocking the object if it must sleep.  Note: alpha, amd64, i386 and
ia64 are the only implementations optimized by this change; arm,
powerpc, and sparc64 still conservatively busy the page and unlock the
object within every pmap_enter_quick() call.

Additionally, this change is the first case where we synchronize
access to the page's PG_BUSY flag and busy field using the containing
object's lock rather than the global page queues lock.  (Modifications
to the page's PG_BUSY flag and busy field have asserted both locks for
several weeks, enabling an incremental transition.)

1 files changed, 11 insertions, 0 deletions

diff --git a/sys/arm/arm/pmap.c b/sys/arm/arm/pmap.c
index f9dca09..9159787 100644
--- a/sys/arm/arm/pmap.c
+++ b/sys/arm/arm/pmap.c
@@ -3417,8 +3417,19 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot,
 vm_page_t
 pmap_enter_quick(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_page_t mpte)
 {
+
+	vm_page_lock_queues();
+	vm_page_busy(m);
+	vm_page_unlock_queues();
+	VM_OBJECT_UNLOCK(m->object);
+	mtx_lock(&Giant);
 	pmap_enter(pmap, va, m, VM_PROT_READ|VM_PROT_EXECUTE, FALSE);
 	pmap_dcache_wbinv_all(pmap); /* XXX: shouldn't be needed */
+	mtx_unlock(&Giant);
+	VM_OBJECT_LOCK(m->object);
+	vm_page_lock_queues();
+	vm_page_wakeup(m);
+	vm_page_unlock_queues();
 	return (NULL);
 }