- Give PIL_PREEMPT the lowest priority just above low/stray interrupts.

  The reason for this is that the SPARC v9 architecture allows nested
  interrupts of higher priority/level than that of the current interrupt
  to occur (and we can't just entirely bypass this model, also, at least
  for tick interrupts, this also wouldn't be wise). However, when a
  preemption interrupt interrupts another interrupt of lower priority,
  f.e. PIL_ITHREAD, and that one in turn is nested by a third interrupt,
  f.e. PIL_TICK, with SCHED_ULE the execution of interrupts higher than
  PIL_PREEMPT may be migrated to another CPU. In particular, tl1_ret(),
  which is responsible for restoring the state of the CPU prior to entry
  to the interrupt based on the (also migrated) trap frame, then is run
  on a CPU which actually didn't receive the interrupt in question,
  causing an inappropriate processor interrupt level to be "restored".
  In turn, this causes interrupts of the first level, i.e. PIL_ITHREAD
  in the above scenario, to be blocked on the target of the migration
  until the correct PIL happens to be restored again on that CPU again.
  Making PIL_PREEMPT the lowest real priority, this effectively prevents
  this scenario from happening, as preemption interrupts no longer can
  interrupt any other interrupt besides stray ones (which is no issue).
  Thanks to attilio@ and especially mav@ for helping me to understand
  this problem at the 201208DevSummit.
- Give PIL_STOP (which is also used for IPI_STOP_HARD, given that there's
  no real equivalent to NMIs on SPARC v9) the highest possible priority
  just below the hardwired PIL_TICK, so it has a chance to interrupt
  more things.

MFC after:	1 week

2 files changed, 10 insertions, 10 deletions

diff --git a/sys/sparc64/include/intr_machdep.h b/sys/sparc64/include/intr_machdep.h
index 6e472f7..8392e06 100644
--- a/sys/sparc64/include/intr_machdep.h
+++ b/sys/sparc64/include/intr_machdep.h
@@ -41,14 +41,14 @@
 #define	IV_SHIFT	6
 
 #define	PIL_LOW		1	/* stray interrupts */
-#define	PIL_ITHREAD	2	/* interrupts that use ithreads */
-#define	PIL_RENDEZVOUS	3	/* smp rendezvous ipi */
-#define	PIL_AST		4	/* ast ipi */
-#define	PIL_STOP	5	/* stop cpu ipi */
-#define	PIL_PREEMPT	6	/* preempt idle thread cpu ipi */
-#define	PIL_HARDCLOCK	7	/* hardclock broadcast */
-#define	PIL_FILTER	12	/* filter interrupts */
-#define	PIL_BRIDGE	13	/* bridge interrupts */
+#define	PIL_PREEMPT	2	/* preempt idle thread CPU IPI */
+#define	PIL_ITHREAD	3	/* interrupts that use ithreads */
+#define	PIL_RENDEZVOUS	4	/* SMP rendezvous IPI */
+#define	PIL_AST		5	/* asynchronous trap IPI */
+#define	PIL_HARDCLOCK	6	/* hardclock broadcast */
+#define	PIL_FILTER	11	/* filter interrupts */
+#define	PIL_BRIDGE	12	/* bridge interrupts */
+#define	PIL_STOP	13	/* stop CPU IPI */
 #define	PIL_TICK	14	/* tick interrupts */
 
 #ifndef LOCORE
diff --git a/sys/sparc64/sparc64/intr_machdep.c b/sys/sparc64/sparc64/intr_machdep.c
index f8eb6d8..dab11c2 100644
--- a/sys/sparc64/sparc64/intr_machdep.c
+++ b/sys/sparc64/sparc64/intr_machdep.c
@@ -92,15 +92,15 @@ static uint16_t intr_stray_count[IV_MAX];
 static const char *const pil_names[] = {
 	"stray",
 	"low",		/* PIL_LOW */
+	"preempt",	/* PIL_PREEMPT */
 	"ithrd",	/* PIL_ITHREAD */
 	"rndzvs",	/* PIL_RENDEZVOUS */
 	"ast",		/* PIL_AST */
-	"stop",		/* PIL_STOP */
-	"preempt",	/* PIL_PREEMPT */
 	"hardclock",	/* PIL_HARDCLOCK */
 	"stray", "stray", "stray", "stray",
 	"filter",	/* PIL_FILTER */
 	"bridge",	/* PIL_BRIDGE */
+	"stop",		/* PIL_STOP */
 	"tick",		/* PIL_TICK */
 };