Move UPCALL related data structure out of kse, introduce a new

data structure called kse_upcall to manage UPCALL. All KSE binding
and loaning code are gone.

A thread owns an upcall can collect all completed syscall contexts in
its ksegrp, turn itself into UPCALL mode, and takes those contexts back
to userland. Any thread without upcall structure has to export their
contexts and exit at user boundary.

Any thread running in user mode owns an upcall structure, when it enters
kernel, if the kse mailbox's current thread pointer is not NULL, then
when the thread is blocked in kernel, a new UPCALL thread is created and
the upcall structure is transfered to the new UPCALL thread. if the kse
mailbox's current thread pointer is NULL, then when a thread is blocked
in kernel, no UPCALL thread will be created.

Each upcall always has an owner thread. Userland can remove an upcall by
calling kse_exit, when all upcalls in ksegrp are removed, the group is
atomatically shutdown. An upcall owner thread also exits when process is
in exiting state. when an owner thread exits, the upcall it owns is also
removed.

KSE is a pure scheduler entity. it represents a virtual cpu. when a thread
is running, it always has a KSE associated with it. scheduler is free to
assign a KSE to thread according thread priority, if thread priority is changed,
KSE can be moved from one thread to another.

When a ksegrp is created, there is always N KSEs created in the group. the
N is the number of physical cpu in the current system. This makes it is
possible that even an userland UTS is single CPU safe, threads in kernel still
can execute on different cpu in parallel. Userland calls kse_create to add more
upcall structures into ksegrp to increase concurrent in userland itself, kernel
is not restricted by number of upcalls userland provides.

The code hasn't been tested under SMP by author due to lack of hardware.

Reviewed by: julian

5 files changed, 11 insertions, 9 deletions

diff --git a/sys/i386/i386/critical.c b/sys/i386/i386/critical.c
index de91426..a19a35b 100644
--- a/sys/i386/i386/critical.c
+++ b/sys/i386/i386/critical.c
@@ -137,7 +137,7 @@ i386_unpend(void)
 				break;
 			case 1:		/* bit 1 - statclock */
 				mtx_lock_spin(&sched_lock);
-				statclock_process(curthread->td_kse,
+				statclock_process(curthread,
 				    (register_t)i386_unpend, 0);
 				mtx_unlock_spin(&sched_lock);
 				break;
diff --git a/sys/i386/i386/exception.s b/sys/i386/i386/exception.s
index 0ebb661..e1dcc09 100644
--- a/sys/i386/i386/exception.s
+++ b/sys/i386/i386/exception.s
@@ -298,9 +298,12 @@ doreti_ast:
 	 */
 	cli
 	movl	PCPU(CURTHREAD),%eax
+	testl	$TDF_ASTPENDING, TD_FLAGS(%eax)
+	jnz	call_ast
 	movl	TD_KSE(%eax), %eax
 	testl	$KEF_ASTPENDING | KEF_NEEDRESCHED,KE_FLAGS(%eax)
 	je	doreti_exit
+call_ast:
 	sti
 	pushl	%esp			/* pass a pointer to the trapframe */
 	call	ast
diff --git a/sys/i386/i386/genassym.c b/sys/i386/i386/genassym.c
index 92915b6..65f94fd 100644
--- a/sys/i386/i386/genassym.c
+++ b/sys/i386/i386/genassym.c
@@ -92,6 +92,7 @@ ASSYM(TD_INTR_NESTING_LEVEL, offsetof(struct thread, td_intr_nesting_level));
 ASSYM(TD_CRITNEST, offsetof(struct thread, td_critnest));
 ASSYM(TD_SWITCHIN, offsetof(struct thread, td_switchin));
 ASSYM(TD_MD, offsetof(struct thread, td_md));
+ASSYM(TDF_ASTPENDING, TDF_ASTPENDING);
 
 ASSYM(P_MD, offsetof(struct proc, p_md));
 ASSYM(MD_LDT, offsetof(struct mdproc, md_ldt));
diff --git a/sys/i386/i386/trap.c b/sys/i386/i386/trap.c
index 54c4af8..2c21983 100644
--- a/sys/i386/i386/trap.c
+++ b/sys/i386/i386/trap.c
@@ -264,7 +264,7 @@ trap(frame)
 		!(PCPU_GET(curpcb)->pcb_flags & PCB_VM86CALL))) {
 		/* user trap */
 
-		sticks = td->td_kse->ke_sticks;
+		sticks = td->td_sticks;
 		td->td_frame = &frame;
 		if (td->td_ucred != p->p_ucred) 
 			cred_update_thread(td);
@@ -957,7 +957,7 @@ syscall(frame)
 	KASSERT((td->td_kse != NULL), ("syscall: kse/thread UNLINKED"));
 	KASSERT((td->td_kse->ke_thread == td), ("syscall:kse/thread mismatch"));
 
-	sticks = td->td_kse->ke_sticks;
+	sticks = td->td_sticks;
 	td->td_frame = &frame;
 	if (td->td_ucred != p->p_ucred) 
 		cred_update_thread(td);
diff --git a/sys/i386/i386/vm_machdep.c b/sys/i386/i386/vm_machdep.c
index 823cad9..903a2b0 100644
--- a/sys/i386/i386/vm_machdep.c
+++ b/sys/i386/i386/vm_machdep.c
@@ -312,8 +312,6 @@ cpu_set_upcall(struct thread *td, void *pcb)
 {
 	struct pcb *pcb2;
 
-	td->td_flags |= TDF_UPCALLING;
-
 	/* Point the pcb to the top of the stack. */
 	pcb2 = td->td_pcb;
 
@@ -370,7 +368,7 @@ cpu_set_upcall(struct thread *td, void *pcb)
  * in thread_userret() itself can be done as well.
  */
 void
-cpu_set_upcall_kse(struct thread *td, struct kse *ke)
+cpu_set_upcall_kse(struct thread *td, struct kse_upcall *ku)
 {
 
 	/* 
@@ -387,15 +385,15 @@ cpu_set_upcall_kse(struct thread *td, struct kse *ke)
 	 * function.
 	 */
 	td->td_frame->tf_esp =
-	    (int)ke->ke_stack.ss_sp + ke->ke_stack.ss_size - 16;
-	td->td_frame->tf_eip = (int)ke->ke_upcall;
+	    (int)ku->ku_stack.ss_sp + ku->ku_stack.ss_size - 16;
+	td->td_frame->tf_eip = (int)ku->ku_func;
 
 	/*
 	 * Pass the address of the mailbox for this kse to the uts
 	 * function as a parameter on the stack.
 	 */
 	suword((void *)(td->td_frame->tf_esp + sizeof(void *)),
-	    (int)ke->ke_mailbox);
+	    (int)ku->ku_mailbox);
 }
 
 void