Rework the support for ABIs to override resource limits (used by 32-bit

processes under 64-bit kernels).  Previously, each 32-bit process overwrote
its resource limits at exec() time.  The problem with this approach is that
the new limits affect all child processes of the 32-bit process, including
if the child process forks and execs a 64-bit process.  To fix this, don't
ovewrite the resource limits during exec().  Instead, sv_fixlimits() is
now replaced with a different function sv_fixlimit() which asks the ABI to
sanitize a single resource limit.  We then use this when querying and
setting resource limits.  Thus, if a 32-bit process sets a limit, then
that new limit will be inherited by future children.  However, if the
32-bit process doesn't change a limit, then a future 64-bit child will
see the "full" 64-bit limit rather than the 32-bit limit.

MFC is tentative since it will break the ABI of old linux.ko modules (no
other modules are affected).

MFC after:	1 week

1 files changed, 4 insertions, 6 deletions

diff --git a/sys/kern/kern_resource.c b/sys/kern/kern_resource.c
index 32c27ed..647cee8 100644
--- a/sys/kern/kern_resource.c
+++ b/sys/kern/kern_resource.c
@@ -701,6 +701,8 @@ kern_setrlimit(td, which, limp)
 			limp->rlim_max = 1;
 		break;
 	}
+	if (td->td_proc->p_sysent->sv_fixlimit != NULL)
+		td->td_proc->p_sysent->sv_fixlimit(limp, which);
 	*alimp = *limp;
 	p->p_limit = newlim;
 	PROC_UNLOCK(p);
@@ -734,12 +736,6 @@ kern_setrlimit(td, which, limp)
 		}
 	}
 
-	/*
-	 * The data size limit may need to be changed to a value
-	 * that makes sense for the 32 bit binary.
-	 */
-	if (p->p_sysent->sv_fixlimits != NULL)
-		p->p_sysent->sv_fixlimits(p);
 	return (0);
 }
 
@@ -1062,6 +1058,8 @@ lim_rlimit(struct proc *p, int which, struct rlimit *rlp)
 	KASSERT(which >= 0 && which < RLIM_NLIMITS,
 	    ("request for invalid resource limit"));
 	*rlp = p->p_limit->pl_rlimit[which];
+	if (p->p_sysent->sv_fixlimit != NULL)
+		p->p_sysent->sv_fixlimit(rlp, which);
 }
 
 /*