Change the scheduler to actually respect the PUSER barrier. It's been

wrong for many years that negative niceness would lower the priority
of a process below PUSER, and once below PUSER, there were conditionals
in the code that are required to test for whether a process was in
the kernel which would break.

The breakage could (and did) cause lock-ups, basically nothing else
but the least nice program being able to run in some conditions.  The
algorithm which adjusts the priority now subtracts PRIO_MIN to do
things properly, and the ESTCPULIM() algorithm was updated to use
PRIO_TOTAL (PRIO_MAX - PRIO_MIN) to calculate the estcpu.

NICE_WEIGHT is now 1 to accomodate the full range of priorities better
(a -20 process with full CPU time has the priority of a +0 process with
no CPU time).  There are now 20 queues (exactly; 80 priorities) for
use in user processes' scheduling, and PUSER has been lowered to 48
to accomplish this.

This means, to the user, that things will be scheduled more correctly
(noticeable), there is no lock-up anymore WRT a niced -20 process
never releasing the CPU time for other processes.  In this fair system,
tsleep()ed < PUSER processes now will get the proper higher priority
than priority >= PUSER user processes.

The detective work of this was done by me, along with part of the
solution.  Luoqi Chen has provided most of the solution, and really
helped me understand what was happening better, to boot :)

Submitted by:   luoqi
Concept reviewed by:    bde

4 files changed, 9 insertions, 6 deletions

diff --git a/sys/kern/kern_synch.c b/sys/kern/kern_synch.c
index 3ae903d..17547de 100644
--- a/sys/kern/kern_synch.c
+++ b/sys/kern/kern_synch.c
@@ -916,7 +916,7 @@ resetpriority(p)
 
 	if (p->p_rtprio.type == RTP_PRIO_NORMAL) {
 		newpriority = PUSER + p->p_estcpu / INVERSE_ESTCPU_WEIGHT +
-		    NICE_WEIGHT * p->p_nice;
+		    NICE_WEIGHT * (p->p_nice - PRIO_MIN);
 		newpriority = min(newpriority, MAXPRI);
 		p->p_usrpri = newpriority;
 	}
diff --git a/sys/sys/param.h b/sys/sys/param.h
index adf4771..52ea28d 100644
--- a/sys/sys/param.h
+++ b/sys/sys/param.h
@@ -111,7 +111,7 @@
 #define	PCONFIG	32
 #define	PLOCK	36
 #define	PPAUSE	40
-#define	PUSER	50
+#define	PUSER	48
 #define	MAXPRI	127		/* Priorities range from 0 through MAXPRI. */
 
 #define	PRIMASK	0x0ff
diff --git a/sys/sys/proc.h b/sys/sys/proc.h
index d75f033..af7f110 100644
--- a/sys/sys/proc.h
+++ b/sys/sys/proc.h
@@ -404,12 +404,12 @@ extern int	whichidqs;	/* Bit mask summary of non-empty Q's. */
  * INVERSE_ESTCPU_WEIGHT is only suitable for statclock() frequencies in
  * the range 100-256 Hz (approximately).
  */
-#define	ESTCPULIM(e) \
-    min((e), INVERSE_ESTCPU_WEIGHT * (NICE_WEIGHT * PRIO_MAX - PPQ) + \
-	     INVERSE_ESTCPU_WEIGHT - 1)
 #define	INVERSE_ESTCPU_WEIGHT	8	/* 1 / (priorities per estcpu level) */
-#define	NICE_WEIGHT	2		/* priorities per nice level */
+#define	NICE_WEIGHT	1		/* priorities per nice level */
 #define	PPQ		(128 / NQS)	/* priorities per queue */
+#define	ESTCPULIM(e) \
+    min((e), INVERSE_ESTCPU_WEIGHT * (NICE_WEIGHT * PRIO_TOTAL - PPQ) + \
+	INVERSE_ESTCPU_WEIGHT - 1)
 
 extern	u_long ps_arg_cache_limit;
 extern	int ps_argsopen;
diff --git a/sys/sys/resource.h b/sys/sys/resource.h
index a1220c0..ed2ecb3 100644
--- a/sys/sys/resource.h
+++ b/sys/sys/resource.h
@@ -42,6 +42,9 @@
  */
 #define	PRIO_MIN	-20
 #define	PRIO_MAX	20
+#ifdef _KERNEL
+#define	PRIO_TOTAL	(PRIO_MAX - PRIO_MIN)
+#endif /* _KERNEL */
 
 #define	PRIO_PROCESS	0
 #define	PRIO_PGRP	1