summaryrefslogtreecommitdiffstats
path: root/sys/kern/uipc_mbuf.c
Commit message (Collapse)AuthorAgeFilesLines
* Change m_uiotombuf so it will accept offset at which data should be copiedemax2005-05-041-2/+5
| | | | | | | | | | to the mbuf. Offset cannot exceed MHLEN bytes. This is currently used to fix Ethernet header alignment problem on alpha and sparc64. Also change all users of m_uiotombuf to pass proper offset. Reviewed by: jmg, sam Tested by: Sten Spans "sten AT blinkenlights DOT nl" MFC after: 1 week
* add m_copyup function.. This can be used to help make our ip stack lessjmg2005-03-171-0/+48
| | | | | | | | | | alignment restrictive, and help performance on some ethernet cards which currently copy the entire packet a couple bytes to get the packet aligned properly... Wordsmithing by: dwhite Obtained from: NetBSD (code only) I'll clean it up later: rwatson
* allow the destination of m_move_pkthdr to have externalsam2005-03-081-3/+3
| | | | | | storage (e.g. a cluster) Glanced at by: rwatson, silby
* The m_ext reference counts are potentially shared and modifiedalc2005-03-061-6/+4
| | | | | | | | | | | asynchronously by different threads. Thus, declare as volatile the reference count that is accessed through m_ext's pointer, ref_cnt. Revert the previous change, revision 1.144, that casts as volatile a single dereference of ref_cnt. Reviewed by: bmilekic, dwhite Problem reported by: kris MFC after: 3 days
* Insert volatile cast to discourage gcc from optimizing the read outsidedwhite2005-03-031-1/+4
| | | | | | | of the while loop. Suggested by: alc MFC after: 1 day
* change m_adj to reclaim unused mbufs instead of zero'ing m_lensam2005-02-241-2/+4
| | | | | | | | when trim'ing space off the back of a chain; this is indirect solution to a potential null ptr deref Noticed by: Coverity Prevent analysis tool (null ptr deref) Reviewed by: dg, rwatson
* remove dead codesam2005-02-231-4/+0
| | | | | Noticed by: Coverity Prevent analysis tool Reviewed by: silby
* Optimize the way reference counting is performed with Mbufs. Webmilekic2005-02-101-21/+37
| | | | | | | | | | | | | | do not need to perform an extra memory fetch in the Packet (Mbuf+Cluster) constructor to initialize the reference counter anymore. The reference counts are located in a separate memory region (in the slab header, because this zone is UMA_ZONE_REFCNT), so the memory fetch resulted very often in a cache miss. Additionally, and perhaps more significantly, optimize the free mbuf+cluster (packet) case, which is very common, to no longer require an atomic operation on free (to verify the reference counter) if the reference on the cluster has never been increased (also very common). Reduces an atomic on mbuf free on average. Original patch submitted by: Gerrit Nagelhout <gnagelhout@sandvine.com>
* Make a bunch of malloc types static.phk2005-02-101-1/+1
| | | | Found by: src/tools/tools/kernxref
* /* -> /*- for copyright notices, minor format tweaks as necessaryimp2005-01-061-1/+1
|
* fix m_append for case where additional mbufs are requiredsam2004-12-151-2/+2
|
* add m_append utility function to be used in forthcoming changessam2004-12-081-0/+46
|
* improve the mbuf m_print function.. Only pull length from pkthdr if therejmg2004-09-281-5/+20
| | | | | | | | | | | is one, detect mbuf loops and stop, add an extra arg so you can only print the first x bytes of the data per mbuf (print all if arg is -1), print flags using %b (bitmask)... No code in the tree appears to use m_print, and it's just a maner of adding -1 as an additional arg to m_print to restore original behavior.. MFC after: 4 days
* Back out just a portion of Alfred's last commit. Remove the MBUF_CHECKbmilekic2004-07-211-2/+0
| | | | | | | | (WITNESS) for code paths that always call uma_zalloc_arg() shortly after where the check was, because uma_zalloc_arg() already does a similar check. No objections from Alfred. Thanks Alfred.
* Make sure we don't call mbuf allocation functions with mutexes held.alfred2004-07-211-0/+8
| | | | Discussed with: rwatson
* Gah! Plug a mbuf leak I introduced in the last commit.bmilekic2004-06-111-2/+3
| | | | | | I don the pointy-hat. Problem reported by: Peter Holm <pho@>
* Plug a race where upon free this scenario could occur:bmilekic2004-06-101-15/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (time grows downward) thread 1 thread 2 ------------|------------ dec ref_cnt | | dec ref_cnt <-- ref_cnt now zero cmpset | free all | return | | alloc again,| reuse prev | ref_cnt | | cmpset, read | already freed | ref_cnt ------------|------------ This should fix that by performing only a single atomic test-and-set that will serve to decrement the ref_cnt, only if it hasn't changed since the earlier read, otherwise it'll loop and re-read. This forces ordering of decrements so that truly the thread which did the LAST decrement is the one that frees. This is how atomic-instruction-based refcnting should probably be handled. Submitted by: Julian Elischer
* Fix a panic happening when m_getm() is called with len < MCLBYTES.mux2004-06-091-1/+1
| | | | | | Reported by: ale Tested by: ale Reviewed by: bosko
* Bring in mbuma to replace mballoc.bmilekic2004-05-311-38/+197
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mbuma is an Mbuf & Cluster allocator built on top of a number of extensions to the UMA framework, all included herein. Extensions to UMA worth noting: - Better layering between slab <-> zone caches; introduce Keg structure which splits off slab cache away from the zone structure and allows multiple zones to be stacked on top of a single Keg (single type of slab cache); perhaps we should look into defining a subset API on top of the Keg for special use by malloc(9), for example. - UMA_ZONE_REFCNT zones can now be added, and reference counters automagically allocated for them within the end of the associated slab structures. uma_find_refcnt() does a kextract to fetch the slab struct reference from the underlying page, and lookup the corresponding refcnt. mbuma things worth noting: - integrates mbuf & cluster allocations with extended UMA and provides caches for commonly-allocated items; defines several zones (two primary, one secondary) and two kegs. - change up certain code paths that always used to do: m_get() + m_clget() to instead just use m_getcl() and try to take advantage of the newly defined secondary Packet zone. - netstat(1) and systat(1) quickly hacked up to do basic stat reporting but additional stats work needs to be done once some other details within UMA have been taken care of and it becomes clearer to how stats will work within the modified framework. From the user perspective, one implication is that the NMBCLUSTERS compile-time option is no longer used. The maximum number of clusters is still capped off according to maxusers, but it can be made unlimited by setting the kern.ipc.nmbclusters boot-time tunable to zero. Work should be done to write an appropriate sysctl handler allowing dynamic tuning of kern.ipc.nmbclusters at runtime. Additional things worth noting/known issues (READ): - One report of 'ips' (ServeRAID) driver acting really slow in conjunction with mbuma. Need more data. Latest report is that ips is equally sucking with and without mbuma. - Giant leak in NFS code sometimes occurs, can't reproduce but currently analyzing; brueffer is able to reproduce but THIS IS NOT an mbuma-specific problem and currently occurs even WITHOUT mbuma. - Issues in network locking: there is at least one code path in the rip code where one or more locks are acquired and we end up in m_prepend() with M_WAITOK, which causes WITNESS to whine from within UMA. Current temporary solution: force all UMA allocations to be M_NOWAIT from within UMA for now to avoid deadlocks unless WITNESS is defined and we can determine with certainty that we're not holding any locks when we're M_WAITOK. - I've seen at least one weird socketbuffer empty-but- mbuf-still-attached panic. I don't believe this to be related to mbuma but please keep your eyes open, turn on debugging, and capture crash dumps. This change removes more code than it adds. A paper is available detailing the change and considering various performance issues, it was presented at BSDCan2004: http://www.unixdaemons.com/~bmilekic/netbuf_bmilekic.pdf Please read the paper for Future Work and implementation details, as well as credits. Testing and Debugging: rwatson, brueffer, Ketrien I. Saihr-Kesenchedra, ... Reviewed by: Lots of people (for different parts)
* constify the last argument of m_copyback.luigi2004-04-181-1/+1
|
* Remove advertising clause from University of California Regent's license,imp2004-04-051-4/+0
| | | | | | per letter dated July 22, 1999. Approved by: core
* Style fixes: don't indent variable names.silby2004-02-051-6/+6
| | | | Submitted by: bde
* Style fixessilby2004-02-041-6/+0
| | | | Submitted by: bde
* Rewrite sendfile's header support so that headers are now sent in the firstsilby2004-02-011-0/+56
| | | | | | | | | | | | packet along with data, instead of in their own packet. When serving files of size (packetsize - headersize) or smaller, this will result in one less packet crossing the network. Quick testing with thttpd and http_load has shown a noticeable performance improvement in this case (350 vs 330 fetches per second.) Included in this commit are two support routines, iov_to_uio, and m_uiotombuf; these routines are used by sendfile to construct the header mbuf chain that will be linked to the rest of the data in the socket buffer.
* Fix another 0 / NULL mixup.silby2003-12-251-1/+1
|
* Catch a few places where NULL (pointer) was used where 0 (integer) waspeter2003-12-231-1/+1
| | | | expected.
* style(9) pass and type fixups.bms2003-12-161-14/+8
| | | | Submitted by: bde
* Push m_apply() and m_getptr() up into the colleciton of standard mbufbms2003-12-151-0/+67
| | | | | | | | routines, and purge them from opencrypto. Reviewed by: sam Obtained from: NetBSD Sponsored by: spc.org
* Implement MBUF_STRESS_TEST mark II.silby2003-09-011-0/+84
| | | | | | | | | | | | | | | | | | | Changes from the original implementation: - Fragmentation is handled by the function m_fragment, which can be called from whereever fragmentation is needed. Note that this function is wrapped in #ifdef MBUF_STRESS_TEST to discourage non-testing use. - m_fragment works slightly differently from the old fragmentation code in that it allocates a seperate mbuf cluster for each fragment. This defeats dma_map_load_mbuf/buffer's feature of coalescing adjacent fragments. While that is a nice feature in practice, it nerfed the usefulness of mbuf_stress_test. - Add two modes of random fragmentation. Chains with fragments all of the same random length and chains with fragments that are each uniquely random in length may now be requested.
* Three fixes:silby2003-07-191-2/+10
| | | | | | | | | | | | | - Make m_prepend use m_gethdr instead of m_get where appropriate - Make m_copym use m_gethdr instead of m_get where appropriate - Add a call to m_fixhdr in m_defrag; m_defrag can't deal with corrupted pkthdr.len counts. MFC after: 3 days
* Hide the m_defrag* statistics under MBUF_STRESS_TEST, there seemssilby2003-06-171-2/+8
| | | | | | | | to be no need to see them in the general case (and they aren't smp-safe anyway.) Suggested by: hmp MFC after: 1 week
* Use __FBSDID().obrien2003-06-111-1/+3
|
* Add another MBUF_STRESS_TEST feature, m_defragrandomfailures.silby2003-04-151-0/+15
| | | | | | | When enabled, this causes m_defrag to randomly return NULL (following its normal failure case so that extra memory leaks are not introduced.) Code similar to this was used to find / fix a few bugs last week.
* Move MAC label storage for mbufs into m_tags from the m_pkthdr structure,rwatson2003-04-141-15/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | returning some additional room in the first mbuf in a chain, and avoiding feature-specific contents in the mbuf header. To do this: - Modify mbuf_to_label() to extract the tag, returning NULL if not found. - Introduce mac_init_mbuf_tag() which does most of the work mac_init_mbuf() used to do, except on an m_tag rather than an mbuf. - Scale back mac_init_mbuf() to perform m_tag allocation and invoke mac_init_mbuf_tag(). - Replace mac_destroy_mbuf() with mac_destroy_mbuf_tag(), since m_tag's are now GC'd deep in the m_tag/mbuf code rather than at a higher level when mbufs are directly free()'d. - Add mac_copy_mbuf_tag() to support m_copy_pkthdr() and related notions. - Generally change all references to mbuf labels so that they use mbuf_to_label() rather than &mbuf->m_pkthdr.label. This required no changes in the MAC policies (yay!). - Tweak mbuf release routines to not call mac_destroy_mbuf(), tag destruction takes care of it for us now. - Remove MAC magic from m_copy_pkthdr() and m_move_pkthdr() -- the existing m_tag support does all this for us. Note that we can no longer just zero the m_tag list on the target mbuf, rather, we have to delete the chain because m_tag's will already be hung off freshly allocated mbuf's. - Tweak m_tag copying routines so that if we're copying a MAC m_tag, we don't do a binary copy, rather, we initialize the new storage and do a deep copy of the label. - Remove use of MAC_FLAG_INITIALIZED in a few bizarre places having to do with mbuf header copies previously. - When an mbuf is copied in ip_input(), we no longer need to explicitly copy the label because it will get handled by the m_tag code now. - No longer any weird handling of MAC labels in if_loop.c during header copies. - Add MPC_LOADTIME_FLAG_LABELMBUFS flag to Biba, MLS, mac_test. In mac_test, handle the label==NULL case, since it can be dynamically loaded. In order to improve performance with this change, introduce the notion of "lazy MAC label allocation" -- only allocate m_tag storage for MAC labels if we're running with a policy that uses MAC labels on mbufs. Policies declare this intent by setting the MPC_LOADTIME_FLAG_LABELMBUFS flag in their load-time flags field during declaration. Note: this opens up the possibility of post-boot policy modules getting back NULL slot entries even though they have policy invariants of non-NULL slot entries, as the policy might have been loaded after the mbuf was allocated, leaving the mbuf without label storage. Policies that cannot handle this case must be declared as NOTLATE, or must be modified. - mac_labelmbufs holds the current cumulative status as to whether any policies require mbuf labeling or not. This is updated whenever the active policy set changes by the function mac_policy_updateflags(). The function iterates the list and checks whether any have the flag set. Write access to this variable is protected by the policy list; read access is currently not protected for performance reasons. This might change if it causes problems. - Add MAC_POLICY_LIST_ASSERT_EXCLUSIVE() to permit the flags update function to assert appropriate locks. - This makes allocation in mac_init_mbuf() conditional on the flag. Reviewed by: sam Obtained from: TrustedBSD Project Sponsored by: DARPA, Network Associates Laboratories
* Use MBTOM() to convert mbuf allocator flags to malloc() flags, ratherrwatson2003-04-141-2/+1
| | | | | | | than using the same compare/substitute in many places. Obtained from: TrustedBSD Project Sponsored by: DARPA, Network Associates Laboratories
* Introduce an M_ASSERTPKTHDR() macro which performs the very common taskdes2003-04-081-4/+3
| | | | | | | of asserting that an mbuf has a packet header. Use it instead of hand- rolled versions wherever applicable. Submitted by: Hiten Pandya <hiten@unixdaemons.com>
* Add the m_defrag routine, as discussed on committers@. Thissilby2003-03-291-0/+83
| | | | | | | incarnation should address the concerns of all in the discussion, and keeps statistics which show how much it is used. MFC after: 2 weeks
* Allow m_dup_pkthdr to accept mbufs with attached clusters assilby2003-03-281-3/+3
| | | | | | targets. Submitted by: bmilekic
* In m_dup_pkthdr(), convert the supplied `how' argument into mallociedowse2003-03-131-1/+2
| | | | | flags when passing it into m_tag_copy_chain(), as m_tag* functions use malloc, not mbuf flags.
* Back out M_* changes, per decision of the TRB.imp2003-02-191-7/+7
| | | | Approved by: trb
* Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.alfred2003-01-211-7/+7
| | | | Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.
* Correct mbuf packet header propagation. Previously, packet headerssam2002-12-301-14/+54
| | | | | | | | | | | | | | | | | | | | | | were sometimes propagated using M_COPY_PKTHDR which actually did something between a "move" and a "copy" operation. This is replaced by M_MOVE_PKTHDR (which copies the pkthdr contents and "removes" it from the source mbuf) and m_dup_pkthdr which copies the packet header contents including any m_tag chain. This corrects numerous problems whereby mbuf tags could be lost during packet manipulations. These changes also introduce arguments to m_tag_copy and m_tag_copy_chain to specify if the tag copy work should potentially block. This introduces an incompatibility with openbsd which we may want to revisit. Note that move/dup of packet headers does not handle target mbufs that have a cluster bound to them. We may want to support this; for now we watch for it with an assert. Finally, M_COPYFLAGS was updated to include M_FIRSTFRAG|M_LASTFRAG. Supported by: Vernier Networks Reviewed by: Robert Watson <rwatson@FreeBSD.org>
* Replace aux mbufs with packet tags:sam2002-10-161-1/+1
| | | | | | | | | | | | | | | | | | | o instead of a list of mbufs use a list of m_tag structures a la openbsd o for netgraph et. al. extend the stock openbsd m_tag to include a 32-bit ABI/module number cookie o for openbsd compatibility define a well-known cookie MTAG_ABI_COMPAT and use this in defining openbsd-compatible m_tag_find and m_tag_get routines o rewrite KAME use of aux mbufs in terms of packet tags o eliminate the most heavily used aux mbufs by adding an additional struct inpcb parameter to ip_output and ip6_output to allow the IPsec code to locate the security policy to apply to outbound packets o bump __FreeBSD_version so code can be conditionalized o fixup ipfilter's call to ip_output based on __FreeBSD_version Reviewed by: julian, luigi (silent), -arch, -net, darren Approved by: julian, silence from everyone else Obtained from: openbsd (mostly) MFC after: 1 month
* While well intentionned the check to see it there is a packetjulian2002-09-191-3/+0
| | | | | | | | | | | header and return that length, was misguided. The check itself didn't take into account the fact that the mbuf pointer pased in may be null, and the function is defined specifically for cases where the caller knows what it wants. Rather than fix the check I'm removing it as phk suggested. Submitted by: phk@freebsd.org
* fix style.. Return in the kernel always has () around the arguments.julian2002-09-191-1/+1
|
* Compiler was correct:julian2002-09-191-1/+1
| | | | m WAS being used uninitialized..
* If M_PKTHDR is set then we don't need to do a loop to find the total length.darrenr2002-09-191-0/+3
|
* style nit: unsigned -> u_int in the kernel, particularly tobmilekic2002-09-181-11/+11
| | | | | | | stay consistent in this file, and keep m_length() and m_fixhdr() consistent with their prototypes in mbuf.h Inspired by: bde
* Make m_length() and m_fixhdr() return unsigned.phk2002-09-181-4/+4
| | | | Suggested by: arr
* Introduce the m_length() function which will return the accumulatedphk2002-09-181-6/+19
| | | | length of an mbuf-chain and optionally a pointer to the last mbuf.
OpenPOWER on IntegriCloud