From 9ae733819b7cdf0eef51ba1b18d39feb136a9dbf Mon Sep 17 00:00:00 2001 From: kib Date: Mon, 5 Nov 2007 11:36:16 +0000 Subject: Fix for the panic("vm_thread_new: kstack allocation failed") and silent NULL pointer dereference in the i386 and sparc64 pmap_pinit() when the kmem_alloc_nofault() failed to allocate address space. Both functions now return error instead of panicing or dereferencing NULL. As consequence, vmspace_exec() and vmspace_unshare() returns the errno int. struct vmspace arg was added to vm_forkproc() to avoid dealing with failed allocation when most of the fork1() job is already done. The kernel stack for the thread is now set up in the thread_alloc(), that itself may return NULL. Also, allocation of the first process thread is performed in the fork1() to properly deal with stack allocation failure. proc_linkup() is separated into proc_linkup() called from fork1(), and proc_linkup0(), that is used to set up the kernel process (was known as swapper). In collaboration with: Peter Holm Reviewed by: jhb --- sys/kern/imgact_aout.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'sys/kern/imgact_aout.c') diff --git a/sys/kern/imgact_aout.c b/sys/kern/imgact_aout.c index 7f202d9..45b39c2 100644 --- a/sys/kern/imgact_aout.c +++ b/sys/kern/imgact_aout.c @@ -198,9 +198,11 @@ exec_aout_imgact(imgp) /* * Destroy old process VM and create a new one (with a new stack) */ - exec_new_vmspace(imgp, &aout_sysvec); + error = exec_new_vmspace(imgp, &aout_sysvec); vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY, td); + if (error) + return (error); /* * The vm space can be changed by exec_new_vmspace -- cgit v1.1