summaryrefslogtreecommitdiffstats
path: root/ui/vnc-enc-tight.c
diff options
context:
space:
mode:
authorGonglei <arei.gonglei@huawei.com>2014-05-28 21:21:35 +0800
committerGerd Hoffmann <kraxel@redhat.com>2014-06-02 16:30:52 +0200
commitb52991537c0efe27ee0c1955eb28a4584226d8b5 (patch)
tree7dda35d44b58adf22bb35d8e845dfef961b1c5e7 /ui/vnc-enc-tight.c
parent4006617552892a1fe3a5a1f4d103613404abc409 (diff)
downloadhqemu-b52991537c0efe27ee0c1955eb28a4584226d8b5.zip
hqemu-b52991537c0efe27ee0c1955eb28a4584226d8b5.tar.gz
vnc-enc-tight: Fix divide-by-zero in tight_detect_smooth_image{16,24,32}
Spotted by Coverity: (1) Event assignment: Assigning: "pixels" = "0". (2) Event cond_true: Condition "y < h", taking true branch (3) Event cond_false: Condition "x < w", taking false branch (4) Event loop_end: Reached end of loop (5) Event divide_by_zero: In expression "(stats[0] + stats[1]) * 100U / pixels", division by expression "pixels" which may be zero has undefined behavior. 290 DEFINE_DETECT_FUNCTION(16) 291 DEFINE_DETECT_FUNCTION(32) Signed-off-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Diffstat (limited to 'ui/vnc-enc-tight.c')
-rw-r--r--ui/vnc-enc-tight.c8
1 files changed, 7 insertions, 1 deletions
diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c
index 59b59c0..f02352c 100644
--- a/ui/vnc-enc-tight.c
+++ b/ui/vnc-enc-tight.c
@@ -181,6 +181,10 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
}
}
+ if (pixels == 0) {
+ return 0;
+ }
+
/* 95% smooth or more ... */
if (stats[0] * 33 / pixels >= 95) {
return 0;
@@ -267,7 +271,9 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
y += w; \
} \
} \
- \
+ if (pixels == 0) { \
+ return 0; \
+ } \
if ((stats[0] + stats[1]) * 100 / pixels >= 90) { \
return 0; \
} \
OpenPOWER on IntegriCloud