summaryrefslogtreecommitdiffstats
path: root/sys/kern/kern_malloc.c
Commit message (Collapse)AuthorAgeFilesLines
* MFC r282690:kib2015-05-231-2/+4
| | | | | Call uma_reclaim() from the additional pagedaemon thread to reclaim kmem arena address space.
* MFC r269963+269964:delphij2014-08-291-14/+10
| | | | | | | | | | | Re-instate UMA cached backend for 4K - 64K allocations. New consumers like geli(4) uses malloc(9) to allocate temporary buffers that gets free'ed shortly, causing frequent TLB shootdown as observed in hwpmc supported flame graph. Add a new loader tunable, vm.kmem_zmax which allows a system administrator to limit the maximum allocation size that malloc(9) would consider using the UMA cache allocator as backend.
* MFC r257854 (discussed with alc@)ian2014-05-161-23/+31
| | | | | | | | | | | | | As of r257209, all architectures have defined VM_KMEM_SIZE_SCALE. In other words, every architecture is now auto-sizing the kmem arena. This revision changes kmeminit() so that the definition of VM_KMEM_SIZE_SCALE becomes mandatory and the definition of VM_KMEM_SIZE becomes optional. Replace or eliminate all existing definitions of VM_KMEM_SIZE. With auto-sizing enabled, VM_KMEM_SIZE effectively became an alternate spelling for VM_KMEM_SIZE_MIN on most architectures. Use VM_KMEM_SIZE_MIN for clarity.
* Tidy up kmeminit(): Since r245575, 'nmbclusters' is calculated afteralc2013-10-051-3/+2
| | | | | | | | | | kmeminit() runs, so it contributes nothing to 'vm_kmem_size'; update a comment to reflect that r254025 replaced the kmem submap with the kmem arena. Reviewed by: kib Approved by: re (gjb) Sponsored by: EMC / Isilon Storage Division
* - Disable quantum caches on the kmem_arena. This can make fragmentationjeff2013-08-131-1/+1
| | | | | | | worse on small KVA systems. I had intended to only enable it for debugging. Sponsored by: EMC / Isilon Storage Division
* Instead of just trying to do it for arm, make sure vm_kmem_size is properlycognet2013-08-091-0/+1
| | | | | | aligned in kmeminit(), where it'll work for any arch. Suggested by: alc
* Replace kernel virtual address space allocation with vmem. This providesjeff2013-08-071-25/+41
| | | | | | | | | | | | | transparent layering and better fragmentation. - Normalize functions that allocate memory to use kmem_* - Those that allocate address space are named kva_* - Those that operate on maps are named kmap_* - Implement recursive allocation handling for kmem_arena in vmem. Reviewed by: alc Tested by: pho Sponsored by: EMC / Isilon Storage Division
* Remove unused malloc type.kib2013-08-011-2/+0
| | | | | Requested by: alc MFC after: 1 week
* Try to improve r242655 take III: move these SYSCTLs describing the kernelmarius2013-02-041-11/+0
| | | | | | map, which is defined and initialized in vm/vm_kern.c, to the latter. Submitted by: alc
* Further improve r242655 and supply VM_{MIN,MAX}_KERNEL_ADDRESS as constantmarius2013-02-031-7/+7
| | | | | | values to SYSCTL_ULONG(9) where possible. Submitted by: bde
* Make r242655 build on sparc64. While at it, make vm_{max,min}_kernel_addressmarius2012-11-081-2/+4
| | | | vm_offset_t as they should be.
* export VM_MIN_KERNEL_ADDRESS and VM_MAX_KERNEL_ADDRESS via sysctl.alfred2012-11-061-0/+8
| | | | | On several platforms the are determined by too many nested #defines to be easily discernible. This will aid in development of auto-tuning.
* Fix a bug with memguard(9) on 32-bit architectures without amdf2012-07-151-1/+1
| | | | | | | | | | | | VM_KMEM_MAX_SIZE. The code was not taking into account the size of the kernel_map, which the kmem_map is allocated from, so it could produce a sub-map size too large to fit. The simplest solution is to ignore VM_KMEM_MAX entirely and base the memguard map's size off the kernel_map's size, since this is always relevant and always smaller. Found by: Justin Hibbits
* Honor db_pager_quit in 'show uma' and 'show malloc'.jhb2012-07-021-0/+4
| | | | MFC after: 1 month
* - Change contigmalloc() to use the vm_paddr_t type instead of an unsignedjhb2012-03-011-1/+1
| | | | | | | | | | | | | | | long for specifying a boundary constraint. - Change bus_dma tags to use bus_addr_t instead of bus_size_t for boundary constraints. These allow boundary constraints to be fully expressed for cases where sizeof(bus_addr_t) != sizeof(bus_size_t). Specifically, it allows a driver to properly specify a 4GB boundary in a PAE kernel. Note that this cannot be safely MFC'd without a lot of compat shims due to KBI changes, so I do not intend to merge it. Reviewed by: scottl
* Eliminate stale numbers from a comment.alc2011-12-071-5/+2
|
* Eliminate the possibility of 32-bit arithmetic overflow in the calculationalc2011-12-071-4/+4
| | | | | | | | | | of vm_kmem_size that may occur if the system administrator has specified a vm.vm_kmem_size tunable value that exceeds the hard cap. PR: 162741 Submitted by: Adam McDougall Reviewed by: bde@ MFC after: 3 weeks
* Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.ed2011-11-071-1/+1
| | | | | | The SYSCTL_NODE macro defines a list that stores all child-elements of that node. If there's no SYSCTL_DECL macro anywhere else, there's no reason why it shouldn't be static.
* contigmalloc(9) and contigfree(9) are now implemented in terms of otheralc2011-10-271-0/+37
| | | | | | more general VM system interfaces. So, their implementation can now reside in kern_malloc.c alongside the other functions that are declared in malloc.h.
* Make memguard(9) capable to guard uma(9) allocations.glebius2011-10-121-1/+1
|
* Fix the handling of an empty kmem map by sysctl_kmem_map_free(). Inalc2011-10-081-2/+2
| | | | | | | | the unlikely event that sysctl_kmem_map_free() was performed on an empty kmem map, it would incorrectly report the free space as zero. Discussed with: avg MFC after: 1 week
* Modestly increase the maximum allowed size of the kmem map on i386.alc2011-03-231-5/+0
| | | | | | | | | | | | | | | | | | | | | | | | | Also, express this new maximum as a fraction of the kernel's address space size rather than a constant so that increasing KVA_PAGES will automatically increase this maximum. As a side-effect of this change, kern.maxvnodes will automatically increase by a proportional amount. While I'm here ensure that this change doesn't result in an unintended increase in maxpipekva on i386. Calculate maxpipekva based upon the size of the kernel address space and the amount of physical memory instead of the size of the kmem map. The memory backing pipes is not allocated from the kmem map. It is allocated from its own submap of the kernel map. In short, it has no real connection to the kmem map. (In fact, the commit messages for the maxpipekva auto-sizing talk about using the kernel map size, cf. r117325 and r117391, even though the implementation actually used the kmem map size.) Although the calculation is now done differently, the resulting value for maxpipekva should remain almost the same on i386. However, on amd64, the value will be reduced by 2/3. This is intentional. The recent change to VM_KMEM_SIZE_SCALE on amd64 for the benefit of ZFS also had the unnecessary side-effect of increasing maxpipekva. This change is effectively restoring maxpipekva on amd64 to its prior value. Eliminate init_param3() since it is no longer used.
* Explicitly wire the user buffer rather than doing it implicitly inmdf2011-01-271-0/+6
| | | | | | | | sbuf_new_for_sysctl(9). This allows using an sbuf with a SYSCTL_OUT drain for extremely large amounts of data where the caller knows that appropriate references are held, and sleeping is not an issue. Inspired by: rwatson
* add kmem_map_free sysctl: query largest contiguous free range in kmem_mapavg2010-10-091-0/+17
| | | | | | Suggested by: alc Reviewed by: alc MFC after: 1 week
* vm.kmem_map_size: a sysctl to query current kmem_map->sizeavg2010-10-071-0/+14
| | | | | | | Based on a patch from Sandvine Incorporated via emaste. Reviewed by: emaste MFC after: 1 week
* kmem_size* sysctls: hint that these are also tunablesavg2010-09-301-4/+4
| | | | MFC after: 1 week
* Re-add r212370 now that the LOR in powerpc64 has been resolved:mdf2010-09-161-49/+9
| | | | | | | | | | | | Add a drain function for struct sysctl_req, and use it for a variety of handlers, some of which had to do awkward things to get a large enough SBUF_FIXEDLEN buffer. Note that some sysctl handlers were explicitly outputting a trailing NUL byte. This behaviour was preserved, though it should not be necessary. Reviewed by: phk (original patch)
* Revert r212370, as it causes a LOR on powerpc. powerpc does a fewmdf2010-09-131-9/+49
| | | | | | | unexpected things in copyout(9) and so wiring the user buffer is not sufficient to perform a copyout(9) while holding a random mutex. Requested by: nwhitehorn
* Add a drain function for struct sysctl_req, and use it for a variety ofmdf2010-09-091-49/+9
| | | | | | | | | | handlers, some of which had to do awkward things to get a large enough FIXEDLEN buffer. Note that some sysctl handlers were explicitly outputting a trailing NUL byte. This behaviour was preserved, though it should not be necessary. Reviewed by: phk
* The realloc case for memguard(9) will copy too many bytes whenmdf2010-08-311-9/+2
| | | | | | | | | reallocating to a smaller-sized allocation. Fix this issue. Noticed by: alc Reviewed by: alc Approved by: zml (mentor) MFC after: 3 weeks
* Rework memguard(9) to reserve significantly more KVA to detectmdf2010-08-111-18/+21
| | | | | | | | | | | | | | | | | use-after-free over a longer time. Also release the backing pages of a guarded allocation at free(9) time to reduce the overhead of using memguard(9). Allow setting and varying the malloc type at run-time. Add knobs to allow: - randomly guarding memory - adding un-backed KVA guard pages to detect underflow and overflow - a lower limit on the size of allocations that are guarded Reviewed by: alc Reviewed by: brueffer, Ulrich Spörlein <uqs spoerlein net> (man page) Silence from: -arch Approved by: zml (mentor) MFC after: 1 month
* Add MALLOC_DEBUG_MAXZONES debug malloc(9) option to use multiple umamdf2010-07-281-24/+124
| | | | | | | | | | | | | | | | | | | | | zones for each malloc bucket size. The purpose is to isolate different malloc types into hash classes, so that any buffer overruns or use-after-free will usually only affect memory from malloc types in that hash class. This is purely a debugging tool; by varying the hash function and tracking which hash class was corrupted, the intersection of the hash classes from each instance will point to a single malloc type that is being misused. At this point inspection or memguard(9) can be used to catch the offending code. Add MALLOC_DEBUG_MAXZONES=8 to -current GENERIC configuration files. The suggestion to have this on by default came from Kostik Belousov on -arch. This code is based on work by Ron Steinke at Isilon Systems. Reviewed by: -arch (mostly silence) Reviewed by: zml Approved by: zml (mentor)
* Use ISO C99 integer types in sys/kern where possible.ed2010-06-211-4/+4
| | | | | | There are only about 100 occurences of the BSD-specific u_int*_t datatypes in sys/kern. The ISO C99 integer types are used here more often.
* If we're passed garbage in malloc_init(), panic() rather than expectingbrian2009-06-051-3/+7
| | | | | | | | | | | | | | a KASSERT to handle it. People are likely to turn off INVARIANTS RSN and loading an old module can cause garbage-in here. I saw the issue with an older nvidia driver (x11/nvidia-driver) loading into a new kernel - a crash wasn't seen 'till sysctl_kern_malloc_stats(). I was lucky that mtp->ks_shortdesc was NULL and not something horrible. While I'm here, KASSERT that malloc_uninit() isn't passed something that's not in kmemstatistics. MFC after: 3 weeks
* Retire kern.vm.kmem.size. It was marked as obsolete prior to 5.2, soimp2009-05-091-4/+0
| | | | it can go.
* struct malloc_type has had a 'magic' field statically initialized torwatson2009-04-191-1/+13
| | | | | | | M_MAGIC by MALLOC_DEFINE() for a long time; add assertions that malloc_type's passed to malloc(), free(), etc have that magic set. MFC after: 2 weeks
* Remove even more unneeded variable assignments.ed2009-02-261-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | kern_time.c: - Unused variable `p'. kern_thr.c: - Variable `error' is always caught immediately, so no reason to initialize it. There is no way that error != 0 at the end of create_thread(). kern_sig.c: - Unused variable `code'. kern_synch.c: - `rval' is always assigned in all different cases. kern_rwlock.c: - `v' is always overwritten with RW_UNLOCKED further on. kern_malloc.c: - `size' is always initialized with the proper value before being used. kern_exit.c: - `error' is always caught and returned immediately. abort2() never returns a non-zero value. kern_exec.c: - `len' is always assigned inside the if-statement right below it. tty_info.c: - `td' is always overwritten by FOREACH_THREAD_IN_PROC(). Found by: LLVM's scan-build
* - Make the keg abstraction more complete. Permit a zone to have multiplejeff2009-01-251-4/+1
| | | | | | | | | | | backend kegs so it may source compatible memory from multiple backends. This is useful for cases such as NUMA or different layouts for the same memory type. - Provide a new api for adding new backend kegs to secondary zones. - Provide a new flag for adjusting the layout of zones to stagger allocations better across cache lines. Sponsored by: Nokia
* Enable the creation of a kmem map larger than 4GB.alc2008-07-051-11/+11
| | | | | | | Submitted by: Tz-Huan Huang Make several variables related to kmem map auto-sizing static. Found by: CScout
* Correct an error in the comments for init_param3().alc2008-07-041-1/+1
| | | | Discussed with: silby
* Add support for the DTrace malloc provider which can enable probesjb2008-05-231-0/+63
| | | | on a per-malloc type basis.
* Introduce a new parameter "superpage_align" to kmem_suballoc() that isalc2008-05-101-1/+1
| | | | | | | | | | | used to request superpage alignment for the submap. Request superpage alignment for the kmem_map. Pass VMFS_ANY_SPACE instead of TRUE to vm_map_find(). (They are currently equivalent but VMFS_ANY_SPACE is the new preferred spelling.) Remove a stale comment from kmem_malloc().
* In keeping with style(9)'s recommendations on macros, use a ';'rwatson2008-03-161-1/+1
| | | | | | | | | after each SYSINIT() macro invocation. This makes a number of lightweight C parsers much happier with the FreeBSD kernel source, including cflow's prcc and lxr. MFC after: 1 month Discussed with: imp, rink
* Use vm_offset_t for kmembase and kmemlimit rather than char *, avoidingrwatson2007-06-271-4/+4
| | | | | | | | unnecessary casts, and making it possible to compile kern_malloc.c with strict aliasing. Submitted by: rdivacky Approved by: re (kensmith)
* Spell statistics more correctly in comments.rwatson2007-06-141-1/+1
|
* Revert VMCNT_* operations introduction.attilio2007-05-311-5/+4
| | | | | | | | Probabilly, a general approach is not the better solution here, so we should solve the sched_lock protection problems separately. Requested by: alc Approved by: jeff (mentor)
* Remove #if 0'd check for 0-size allocations, which if enabled, calledrwatson2007-05-271-4/+0
| | | | kdb_enter().
* - define and use VMCNT_{GET,SET,ADD,SUB,PTR} macros for manipulatingjeff2007-05-181-4/+5
| | | | | | | | vmcnts. This can be used to abstract away pcpu details but also changes to use atomics for all counters now. This means sched lock is no longer responsible for protecting counts in the switch routines. Contributed by: Attilio Rao <attilio@FreeBSD.org>
* Add support for specifying a minimal size for vm.kmem_size in the loader viasepotvin2007-04-211-0/+12
| | | | | | | | vm.kmem_size_min. Useful when using ZFS to make sure that vm.kmem size will be at least 256mb (for example) without forcing a particular value via vm.kmem_size. Approved by: njl (mentor) Reviewed by: alc
* Increase usefulness of "show malloc" by moving from displaying the basicrwatson2006-10-261-5/+11
| | | | | | | | | | | | counters of allocs/frees/use for each malloc type to calculating InUse, MemUse, and Requests as displayed by the userspace vmstat -m. This is more useful when debugging malloc(9)-related memory leaks, where the count of allocs/frees may not usefully reflect that current memory allocation (i.e., when highly variable size allocations occur with the same malloc type, such as with contigmalloc). MFC after: 3 days Limitations observed by: scottl
OpenPOWER on IntegriCloud