summaryrefslogtreecommitdiffstats
path: root/sys/net/bpf.c
Commit message (Collapse)AuthorAgeFilesLines
* MFC r292603:bz2016-01-211-1/+1
| | | | | | | | | If bootverbose is enabled every vnet startup and virtual interface creation will print extra lines on the console. We are generally not interested in this (repeated) information for each VNET. Thus only print it for the default VNET. Virtual interfaces on the base system will remain printing information, but e.g. each loopback in each vnet will no longer cause a "bpf attached" line.
* MFC r286260:loos2015-08-171-4/+4
| | | | | | | | | | | | | | | | | | Remove the mtx_sleep() from the kqueue f_event filter. The filter is called from the network hot path and must not sleep. The filter runs with the descriptor lock held and does not manipulate the buffers, so it is not necessary sleep when the hold buffer is in use. Just ignore the hold buffer contents when it is being copied to user space (when hold buffer in use is set). This fix the "Sleeping thread owns a non-sleepable lock" panic when the userland thread is too busy reading the packets from bpf(4). PR: 200323 Sponsored by: Rubicon Communications (Netgate)
* MFC r286140:loos2015-08-171-8/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the sleep from the buffer allocation routine. The buffer must be allocated (or even changed) before the interface is set and thus, there is no need to verify if the buffer is in use. MFC r286142: Remove two unnecessary sleeps from the hot path in bpf(4). The first one never triggers because bpf_canfreebuf() can only be true for zero-copy buffers and zero-copy buffers are not read with read(2). The second also never triggers, because we check the free buffer before calling ROTATE_BUFFERS(). If the hold buffer is in use the free buffer will be NULL and there is nothing else to do besides drop the packet. If the free buffer isn't NULL the hold buffer _is_ free and it is safe to rotate the buffers. Update the comment in ROTATE_BUFFERS macro to match the logic described here. While here fix a few typos in comments. MFC r286243: Add a KASSERT() to make sure we wont rotate the buffers twice (rotate the buffers while the hold buffer is in use). Sponsored by: Rubicon Communications (Netgate)
* MFC r286139:loos2015-08-171-13/+30
| | | | | | | | | | | | | | | | | | Do not allocate the buffers at opening of the descriptor, because once the buffer is allocated we are committed to a particular buffer method (BPF_BUFMODE_BUFFER in this case). If we are using zero-copy buffers, the userland program must register its buffers before set the interface. If we are using kernel memory buffers, we can allocate the buffer at the time that the interface is being set. This fix allows the usage of BIOCSETBUFMODE after r235746. Update the comments to reflect the recent changes. Sponsored by: Rubicon Communications (Netgate)
* MFC r281765:mav2015-05-041-1/+3
| | | | | | | | | | | Activate write-only optimization if bpf device opened with O_WRONLY. dhclient opens bpf as write-only to send packets. It never reads received packets from that descriptor, but processing them in kernel takes time. Especially much time takes packet timestamping on systems with expensive timecounter, such as bhyve guest, where network speed dropped in half. Sponsored by: iXsystems, Inc.
* MFC r273733, r273740 and r273773:hselasky2014-10-301-5/+8
| | | | | | | | | | The SYSCTL data pointers can come from userspace and must not be directly accessed. Although this will work on some platforms, it can throw an exception if the pointer is invalid and then panic the kernel. Add a missing SYSCTL_IN() of "SCTP_BASE_STATS" structure. Sponsored by: Mellanox Technologies
* While waiting for the bpf hold buffer to become idle, checkghelmer2013-05-231-2/+7
| | | | | | | the return value from mtx_sleep() and exit bpfread() on errors such as EINTR. Reviewed by: jhb
* Functions m_getm2() and m_get2() have different order of arguments,glebius2013-03-121-1/+1
| | | | | | | and that can drive someone crazy. While m_get2() is young and not documented yet, change its order of arguments to match m_getm2(). Sorry for churn, but better now than later.
* - Utilize m_get2(), accidentially fixing some signedness bugs.glebius2013-01-241-20/+3
| | | | | - Return EMSGSIZE in both cases if uio_resid is oversized or undersized. - No need to clear rcvif.
* Changes to resolve races in bpfread() and catchpacket() that, at worst,ghelmer2012-12-101-11/+30
| | | | | | | | | | | | cause kernel panics. Add a flag to the bpf descriptor to indicate whether the hold buffer is in use. In bpfread(), set the "hold buffer in use" flag before dropping the descriptor lock during the call to bpf_uiomove(). Everywhere else the hold buffer is used or changed, wait while the hold buffer is in use by bpfread(). Add a KASSERT in bpfread() after re-acquiring the descriptor lock to assist uncovering any additional hold buffer races.
* Mechanically substitute flags from historic mbuf allocator withglebius2012-12-051-4/+4
| | | | | | | | | malloc(9) flags within sys. Exceptions: - sys/contrib not touched - sys/mbuf.h edited manually
* Fix bpf_if structure leak introduced in r235745.melifaro2012-12-021-52/+66
| | | | | | | Move all such structures to delayed-free lists and delete all matching on interface departure event. MFC after: 1 week
* Work around a race in bpfread() by validating the hold buffer pointerghelmer2012-11-061-4/+7
| | | | | before freeing it. Otherwise, we can lose a buffer and cause a panic in catchpacket().
* Fix typo introduced in r236559.melifaro2012-06-091-1/+1
| | | | | Pointed by: bcr Approved by: kib(mentor)
* Fix panic introduced by r235745. Panic occurs after first packet traverse ↵melifaro2012-06-041-2/+22
| | | | | | | | | | | renamed interface. Add several comments on locking Found by: avg Approved by: ae(mentor) Tested by: avg MFC after: 1 week
* Fix style(9) nits, reduce unnecessary type castings, etc., for bpf_setf().jkim2012-05-291-19/+20
|
* - Save the previous filter right before we set new one.jkim2012-05-291-63/+26
| | | | | | - Reduce duplicate code and make it little easier to read. MFC after: 2 weeks
* Fix 32-bit shim for BIOCSETF to drop all packets buffered on the descriptorjkim2012-05-291-2/+12
| | | | | | and reset statistics as it should. MFC after: 3 days
* Fix BPF_JITTER code broken by r235746.melifaro2012-05-291-46/+48
| | | | | | | Pointed by: jkim Reviewed by: jkim (except locking changes) Approved by: (mentor) MFC after: 2 weeks
* Make most BPF ioctls() SMP-safe.melifaro2012-05-211-6/+47
| | | | | Approved by: kib(mentor) MFC in: 4 weeks
* Call bpf_jitter() before acquiring BPF global lock due to malloc() being ↵melifaro2012-05-211-8/+14
| | | | | | | | | | used inside bpf_jitter. Eliminate bpf_buffer_alloc() and allocate BPF buffers on descriptor creation and BIOCSBLEN ioctl. This permits us not to allocate buffers inside bpf_attachd() which is protected by global lock. Approved by: kib(mentor) MFC in: 4 weeks
* Fix old panic when BPF consumer attaches to destroying interface.melifaro2012-05-211-84/+122
| | | | | | | | | | | | | | | | | 'flags' field is added to the end of bpf_if structure. Currently the only flag is BPFIF_FLAG_DYING which is set on bpf detach and checked by bpf_attachd() Problem can be easily triggered on SMP stable/[89] by the following command (sort of): 'while true; do ifconfig vlan222 create vlan 222 vlandev em0 up ; tcpdump -pi vlan222 & ; ifconfig vlan222 destroy ; done' Fix possible use-after-free when BPF detaches itself from interface, freeing bpf_bif memory, while interface is still UP and there can be routes via this interface. Freeing is now delayed till ifnet_departure_event is received via eventhandler(9) api. Convert bpfd rwlock back to mutex due lack of performance gain (currently checking if packet matches filter is done without holding bpfd lock and we have to acquire write lock if packet matches) Approved by: kib(mentor) MFC in: 4 weeks
* Fix panic on attaching to non-existent interface (introduced by r233937, ↵melifaro2012-05-211-42/+136
| | | | | | | | | | | | | | | | pointed by hrs@) Fix panic on tcpdump being attached to interface being removed (introduced by r233937, pointed by hrs@ and adrian@) Protect most of bpf_setf() by BPF global lock Add several forgotten assertions (thanks to adrian@) Document current locking model inside bpf.c Document EVENTHANDLER(9) usage inside BPF. Approved by: kib(mentor) Tested by: gnn MFC in: 4 weeks
* Fix build broken by r233938.melifaro2012-04-061-1/+2
| | | | | | Pointed by: David Wolfskill <david@catwhisker.org> Approved by: kib (mentor) Pointy hat to: melifaro
* - Improve performace for writer-only BPF users.melifaro2012-04-061-6/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Linux and Solaris (at least OpenSolaris) has PF_PACKET socket families to send raw ethernet frames. The only FreeBSD interface that can be used to send raw frames is BPF. As a result, many programs like cdpd, lldpd, various dhcp stuff uses BPF only to send data. This leads us to the situation when software like cdpd, being run on high-traffic-volume interface significantly reduces overall performance since we have to acquire additional locks for every packet. Here we add sysctl that changes BPF behavior in the following way: If program came and opens BPF socket without explicitly specifyin read filter we assume it to be write-only and add it to special writer-only per-interface list. This makes bpf_peers_present() return 0, so no additional overhead is introduced. After filter is supplied, descriptor is added to original per-interface list permitting packets to be captured. Unfortunately, pcap_open_live() sets catch-all filter itself for the purpose of setting snap length. Fortunately, most programs explicitly sets (event catch-all) filter after that. tcpdump(1) is a good example. So a bit hackis approach is taken: we upgrade description only after second BIOCSETF is received. Sysctl is named net.bpf.optimize_writers and is turned off by default. - While here, document all sysctl variables in bpf.4 Sponsored by Yandex LLC Reviewed by: glebius (previous version) Reviewed by: silence on -net@ Approved by: (mentor) MFC after: 4 weeks
* - Improve BPF locking model.melifaro2012-04-061-106/+145
| | | | | | | | | | | | | | | | | | | | | | | Interface locks and descriptor locks are converted from mutex(9) to rwlock(9). This greately improves performance: in most common case we need to acquire 1 reader lock instead of 2 mutexes. - Remove filter(descriptor) (reader) lock in bpf_mtap[2] This was suggested by glebius@. We protect filter by requesting interface writer lock on filter change. - Cover struct bpf_if under BPF_INTERNAL define. This permits including bpf.h without including rwlock stuff. However, this is is temporary solution, struct bpf_if should be made opaque for any external caller. Found by: Dmitrij Tejblum <tejblum@yandex-team.ru> Sponsored by: Yandex LLC Reviewed by: glebius (previous version) Reviewed by: silence on -net@ Approved by: (mentor) MFC after: 3 weeks
* o) Add COMPAT_FREEBSD32 support for MIPS kernels using the n64 ABI with ↵jmallett2012-03-031-7/+7
| | | | | | | | | | | | | | | | | | | | | userlands using the o32 ABI. This mostly follows nwhitehorn's lead in implementing COMPAT_FREEBSD32 on powerpc64. o) Add a new type to the freebsd32 compat layer, time32_t, which is time_t in the 32-bit ABI being used. Since the MIPS port is relatively-new, even the 32-bit ABIs use a 64-bit time_t. o) Because time{spec,val}32 has the same size and layout as time{spec,val} on MIPS with 32-bit compatibility, then, disable some code which assumes otherwise wrongly when built for MIPS. A more general macro to check in this case would seem like a good idea eventually. If someone adds support for using n32 userland with n64 kernels on MIPS, then they will have to add a variety of flags related to each piece of the ABI that can vary. That's probably the right time to generalize further. o) Add MIPS to the list of architectures which use PAD64_REQUIRED in the freebsd32 compat code. Probably this should be generalized at some point. Reviewed by: gonzo
* Consumers of bpfdetach() expect it to remove all bpf_if structs from thelstewart2012-01-101-22/+31
| | | | | | | | | | | | | | | | | | | | | | | bpf_iflist list which reference the specified ifnet. The existing implementation only removes the first matching bpf_if found in the list, effectively leaking list entries if an ifnet has been bpfattach()ed multiple times with different DLTs. Fix the leak by performing the detach logic in a loop, stopping when all bpf_if structs referencing the specified ifnet have been detached and removed from the bpf_iflist list. Whilst here, also: - Remove the unnecessary "bp->bif_ifp == NULL" check, as a bpf_if should never exist in the list with a NULL ifnet pointer. - Except when INVARIANTS is in the kernel config, silently ignore the case where no bpf_if referencing the specified ifnet is found, as it is harmless and does not require user attention. Reviewed by: csjp MFC after: 1 week
* Revert r228986 until it can be reworked to avoid panicing the kernel when thelstewart2011-12-311-183/+71
| | | | | | | same interface is attached multiple times with different DLTs, as is done in net80211 for example. Reported by: adrian
* - Introduce the net.bpf.tscfg sysctl tree and associated code so as to make onelstewart2011-12-301-71/+183
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | aspect of time stamp configuration per interface rather than per BPF descriptor. Prior to this, the order in which BPF devices were opened and the per descriptor time stamp configuration settings could cause non-deterministic and unintended behaviour with respect to time stamping. With the new scheme, a BPF attached interface's tscfg sysctl entry can be set to "default", "none", "fast", "normal" or "external". Setting "default" means use the system default option (set with the net.bpf.tscfg.default sysctl), "none" means do not generate time stamps for tapped packets, "fast" means generate time stamps for tapped packets using a hz granularity system clock read, "normal" means generate time stamps for tapped packets using a full timecounter granularity system clock read and "external" (currently unimplemented) means use the time stamp provided with the packet from an underlying source. - Utilise the recently introduced sysclock_getsnapshot() and sysclock_snap2bintime() KPIs to ensure the system clock is only read once per packet, regardless of the number of BPF descriptors and time stamp formats requested. Use the per BPF attached interface time stamp configuration to control if sysclock_getsnapshot() is called and whether the system clock read is fast or normal. The per BPF descriptor time stamp configuration is then used to control how the system clock snapshot is converted to a bintime by sysclock_snap2bintime(). - Remove all FAST related BPF descriptor flag variants. Performing a "fast" read of the system clock is now controlled per BPF attached interface using the net.bpf.tscfg sysctl tree. - Update the bpf.4 man page. Committed on behalf of Julien Ridoux and Darryl Veitch from the University of Melbourne, Australia, as part of the FreeBSD Foundation funded "Feed-Forward Clock Synchronization Algorithms" project. For more information, see http://www.synclab.org/radclock/ In collaboration with: Julien Ridoux (jridoux at unimelb edu au)
* Revert r227778 in preparation for committing reworked patches in its place.lstewart2011-11-291-143/+11
|
* - When feed-forward clock support is compiled in, change the BPF header tolstewart2011-11-211-11/+143
| | | | | | | | | | | | | | | | | | contain both a regular timestamp obtained from the system clock and the current feed-forward ffcounter value. This enables new possibilities including comparison of timekeeping performance and timestamp correction during post processing. - Add the net.bpf.ffclock_tstamp sysctl to provide a choice between timestamping packets using the feedback or feed-forward system clock. Committed on behalf of Julien Ridoux and Darryl Veitch from the University of Melbourne, Australia, as part of the FreeBSD Foundation funded "Feed-Forward Clock Synchronization Algorithms" project. For more information, see http://www.synclab.org/radclock/ Submitted by: Julien Ridoux (jridoux at unimelb edu au)
* Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.ed2011-11-071-1/+1
| | | | | | The SYSCTL_NODE macro defines a list that stores all child-elements of that node. If there's no SYSCTL_DECL macro anywhere else, there's no reason why it shouldn't be static.
* Fix a deficiency in the selinfo interface:attilio2011-08-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a selinfo object is recorded (via selrecord()) and then it is quickly destroyed, with the waiters missing the opportunity to awake, at the next iteration they will find the selinfo object destroyed, causing a PF#. That happens because the selinfo interface has no way to drain the waiters before to destroy the registered selinfo object. Also this race is quite rare to get in practice, because it would require a selrecord(), a poll request by another thread and a quick destruction of the selrecord()'ed selinfo object. Fix this by adding the seldrain() routine which should be called before to destroy the selinfo objects (in order to avoid such case), and fix the present cases where it might have already been called. Sometimes, the context is safe enough to prevent this type of race, like it happens in device drivers which installs selinfo objects on poll callbacks. There, the destruction of the selinfo object happens at driver detach time, when all the filedescriptors should be already closed, thus there cannot be a race. For this case, mfi(4) device driver can be set as an example, as it implements a full correct logic for preventing this from happening. Sponsored by: Sandvine Incorporated Reported by: rstone Tested by: pluknet Reviewed by: jhb, kib Approved by: re (bz) MFC after: 3 weeks
* Fix a typo in a comment.jkim2010-09-161-1/+1
| | | | Submitted by: afiveg
* Implement flexible BPF timestamping framework.jkim2010-06-151-58/+205
| | | | | | | | | | | | | | | | | | | | | | | | | | | - Allow setting format, resolution and accuracy of BPF time stamps per listener. Previously, we were only able to use microtime(9). Now we can set various resolutions and accuracies with ioctl(2) BIOCSTSTAMP command. Similarly, we can get the current resolution and accuracy with BIOCGTSTAMP command. Document all supported options in bpf(4) and their uses. - Introduce new time stamp 'struct bpf_ts' and header 'struct bpf_xhdr'. The new time stamp has both 64-bit second and fractional parts. bpf_xhdr has this time stamp instead of 'struct timeval' for bh_tstamp. The new structures let us use bh_tstamp of same size on both 32-bit and 64-bit platforms without adding additional shims for 32-bit binaries. On 64-bit platforms, size of BPF header does not change compared to bpf_hdr as its members are already all 64-bit long. On 32-bit platforms, the size may increase by 8 bytes. For backward compatibility, struct bpf_hdr with struct timeval is still the default header unless new time stamp format is explicitly requested. However, the behaviour may change in the future and all relevant code is wrapped around "#ifdef BURN_BRIDGES" for now. - Add experimental support for tagging mbufs with time stamps from a lower layer, e.g., device driver. Currently, mbuf_tags(9) is used to tag mbufs. The time stamps must be uptime in 'struct bintime' format as binuptime(9) and getbinuptime(9) do. Reviewed by: net@
* MFP4: @177254bz2010-04-271-3/+8
| | | | | | | | | Add missing CURVNET_RESTORE() calls for multiple code paths, to stop leaking the currently cached vnet into callers and to the process. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH MFC after: 4 days
* Provide compat32 shims for bpf(4), except zero-copy facilities.kib2010-04-251-3/+165
| | | | | | | | | | bd_compat32 field of struct bpf_d is kept unconditionally to not impose the requirement of including "opt_compat.h" on all numerous users of bpfdesc.h. Submitted by: jhb (version for 6.x) Reviewed and tested by: emaste MFC after: 2 weeks
* Check the pointer to JIT binary filter before its de-allocation.jkim2010-03-291-3/+4
| | | | | Submitted by: Alexander Sack (asack at niksun dot com) MFC after: 3 days
* Fix a style(9) nit.jkim2010-03-121-2/+1
|
* Tidy up callout for select(2) and read timeout.jkim2010-03-121-3/+6
| | | | | | | | | | - Add a missing callout_drain(9) before the descriptor deallocation.[1] - Prefer callout_init_mtx(9) over callout_init(9) and let the callout subsystem handle the mutex for callout function. PR: kern/144453 Submitted by: Alexander Sack (asack at niksun dot com)[1] MFC after: 1 week
* Return partially filled buffer for non-blocking read(2)jkim2010-02-201-6/+15
| | | | | | in non-immediate mode. PR: kern/143855
* Remove unneeded blank line from bpf_drvinit().rwatson2009-10-231-1/+0
| | | | MFC after: 3 days
* Use C99 initialization for struct filterops.rwatson2009-09-121-2/+5
| | | | | | Obtained from: Mac OS X Sponsored by: Apple Inc. MFC after: 3 weeks
* Always embed pointer to BPF JIT function in BPF descriptorjkim2009-08-121-6/+12
| | | | | | | to avoid inconsistency when opt_bpf.h is not included. Reviewed by: rwatson Approved by: re (rwatson)
* Merge the remainder of kern_vimage.c and vimage.h into vnet.c andrwatson2009-08-011-1/+1
| | | | | | | | | | vnet.h, we now use jails (rather than vimages) as the abstraction for virtualization management, and what remained was specific to virtual network stacks. Minor cleanups are done in the process, and comments updated to reflect these changes. Reviewed by: bz Approved by: re (vimage blanket)
* Implement the -z (zero counters) option for the various bpf counters.csjp2009-06-191-1/+45
| | | | | | | | Add necessary changes to the kernel for this (basically introduce a bpf_zero_counters() function). As well, update the man page. MFC after: 1 month Discussed with: rwatson
* Add explicit includes for jail.h to the files that need them andbz2009-06-171-0/+1
| | | | remove the "hidden" one from vimage.h.
* Adapt vfs kqfilter to the shared vnode lock used by zfs write vop. Usekib2009-06-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | vnode interlock to protect the knote fields [1]. The locking assumes that shared vnode lock is held, thus we get exclusive access to knote either by exclusive vnode lock protection, or by shared vnode lock + vnode interlock. Do not use kl_locked() method to assert either lock ownership or the fact that curthread does not own the lock. For shared locks, ownership is not recorded, e.g. VOP_ISLOCKED can return LK_SHARED for the shared lock not owned by curthread, causing false positives in kqueue subsystem assertions about knlist lock. Remove kl_locked method from knlist lock vector, and add two separate assertion methods kl_assert_locked and kl_assert_unlocked, that are supposed to use proper asserts. Change knlist_init accordingly. Add convenience function knlist_init_mtx to reduce number of arguments for typical knlist initialization. Submitted by: jhb [1] Noted by: jhb [2] Reviewed by: jhb Tested by: rnoland
* Move "options MAC" from opt_mac.h to opt_global.h, as it's now in GENERICrwatson2009-06-051-1/+0
| | | | | | | | and used in a large number of files, but also because an increasing number of incorrect uses of MAC calls were sneaking in due to copy-and-paste of MAC-aware code without the associated opt_mac.h include. Discussed with: pjd
OpenPOWER on IntegriCloud