If the size passed to {,v}s{w,n}printf is larger than INT_MAX+1

(i.e., the return value would overflow), set errno to EOVERFLOW
and return an error.  This improves the chances that buggy
applications -- for instance, ones that pass in a negative integer
as the size due to a bogus calculation -- will fail in safe ways.
Returning an error in these situations is specified by POSIX, but
POSIX appears to have an off-by-one error that isn't duplicated in
this change.

Previously, some of these functions would silently cap the size at
INT_MAX+1, and others would exit with an error after writing more
than INT_MAX characters.

PR:		39256
MFC after:	2 weeks

1 files changed, 6 insertions, 0 deletions

diff --git a/lib/libc/stdio/vswprintf.c b/lib/libc/stdio/vswprintf.c
index 4b1e382..2585888 100644
--- a/lib/libc/stdio/vswprintf.c
+++ b/lib/libc/stdio/vswprintf.c
@@ -39,6 +39,7 @@ __FBSDID("FreeBSD: src/lib/libc/stdio/vasprintf.c,v 1.16 2002/08/21 16:19:57 mik
 __FBSDID("$FreeBSD$");
 
 #include <errno.h>
+#include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <wchar.h>
@@ -61,6 +62,11 @@ vswprintf_l(wchar_t * __restrict s, size_t n, locale_t locale,
 		errno = EINVAL;
 		return (-1);
 	}
+	if (n - 1 > INT_MAX) {
+		errno = EOVERFLOW;
+		*s = L'\0';
+		return (-1);
+	}
 
 	f._flags = __SWR | __SSTR | __SALC;
 	f._bf._base = f._p = (unsigned char *)malloc(128);