diff options
| author | Dmitry Vyukov <dvyukov@google.com> | 2017-03-09 16:17:28 -0800 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2017-03-09 17:01:10 -0800 |
| commit | 68fd814a3391c7e017ae6ace8855788748edd981 (patch) | |
| tree | d7e188b5825eb5ee3d78090bed16891b7cf3c077 /mm/kasan/quarantine.c | |
| parent | 40e952f9d687928b32db20226f085ae660a7237c (diff) | |
| download | op-kernel-dev-68fd814a3391c7e017ae6ace8855788748edd981.zip op-kernel-dev-68fd814a3391c7e017ae6ace8855788748edd981.tar.gz | |
kasan: resched in quarantine_remove_cache()
We see reported stalls/lockups in quarantine_remove_cache() on machines
with large amounts of RAM. quarantine_remove_cache() needs to scan
whole quarantine in order to take out all objects belonging to the
cache. Quarantine is currently 1/32-th of RAM, e.g. on a machine with
256GB of memory that will be 8GB. Moreover quarantine scanning is a
walk over uncached linked list, which is slow.
Add cond_resched() after scanning of each non-empty batch of objects.
Batches are specifically kept of reasonable size for quarantine_put().
On a machine with 256GB of RAM we should have ~512 non-empty batches,
each with 16MB of objects.
Link: http://lkml.kernel.org/r/20170308154239.25440-1-dvyukov@google.com
Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
Acked-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'mm/kasan/quarantine.c')
| -rw-r--r-- | mm/kasan/quarantine.c | 9 |
1 files changed, 8 insertions, 1 deletions
diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 6f1ed16..4ac39f2 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -283,8 +283,15 @@ void quarantine_remove_cache(struct kmem_cache *cache) on_each_cpu(per_cpu_remove_cache, cache, 1); spin_lock_irqsave(&quarantine_lock, flags); - for (i = 0; i < QUARANTINE_BATCHES; i++) + for (i = 0; i < QUARANTINE_BATCHES; i++) { + if (qlist_empty(&global_quarantine[i])) + continue; qlist_move_cache(&global_quarantine[i], &to_free, cache); + /* Scanning whole quarantine can take a while. */ + spin_unlock_irqrestore(&quarantine_lock, flags); + cond_resched(); + spin_lock_irqsave(&quarantine_lock, flags); + } spin_unlock_irqrestore(&quarantine_lock, flags); qlist_free_all(&to_free, cache); |
