summaryrefslogtreecommitdiffstats
path: root/sys/netinet/tcp_usrreq.c
Commit message (Collapse)AuthorAgeFilesLines
* replace spaces added in last change with tabskmacy2008-05-051-5/+5
|
* add rcv_nxt, snd_nxt, and toe offload id to FreeBSD-specifickmacy2008-05-051-0/+6
| | | | extension fields for tcp_info
* Convert pcbinfo and inpcb mutexes to rwlocks, and modify macros torwatson2008-04-171-68/+68
| | | | | | | | | | | | | | | explicitly select write locking for all use of the inpcb mutex. Update some pcbinfo lock assertions to assert locked rather than write-locked, although in practice almost all uses of the pcbinfo rwlock main exclusive, and all instances of inpcb lock acquisition are exclusive. This change should introduce (ideally) little functional change. However, it lays the groundwork for significantly increased parallelism in the TCP/IP code. MFC after: 3 months Tested by: kris (superset of committered patch)
* tcp_usrreq.c:1.313 removed tcbinfo locking from tcp_usr_accept(), whichrwatson2008-01-231-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | while in principle a good idea, opened us up to a race inherrent to the syncache's direct insertion of incoming TCP connections into the "completed connection" listen queue, as it transpires that the socket is inserted before the inpcb is fully filled in by syncache_expand(). The bug manifested with the occasional returning of 0.0.0.0:0 in the address returned by the accept() system call, which occurred if accept managed to execute tcp_usr_accept() before syncache_expand() had copied the endpoint addresses into inpcb connection state. Re-add tcbinfo locking around the address copyout, which has the effect of delaying the copy until syncache_expand() has finished running, as it is run while the tcbinfo lock is held. This is undesirable in that it increases contention on tcbinfo further, but a more significant change will be required to how the syncache inserts new sockets in order to fix this and keep more granular locking here. In particular, either more state needs to be passed into sonewconn() so that pru_attach() can fill in the fields *before* the socket is inserted, or the socket needs to be inserted in the incomplete connection queue until it is actually ready to be used. Reported by: glebius (and kris) Tested by: glebius
* In tcp_ctloutput(), don't hold the inpcb lock over sooptcopyin(), rather,rwatson2008-01-181-25/+55
| | | | | | | | | | | | drop the lock and then re-acquire it, revalidating TCP connection state assumptions when we do so. This avoids a potential lock order reversal (and potential deadlock, although none have been reported) due to the inpcb lock being held over a page fault. MFC after: 1 week PR: 102752 Reviewed by: bz Reported by: Václav Haisman <v dot haisman at sh dot cvut dot cz>
* Incorporate TCP offload hooks in to core TCP code.kmacy2007-12-181-9/+13
| | | | | | | | | | - Rename output routines tcp_gen_* -> tcp_output_*. - Rename notification routines that turn in to no-ops in the absence of TOE from tcp_gen_* -> tcp_offload_*. - Fix some minor comment nits. - Add a /* FALLTHROUGH */ Reviewed by: Sam Leffler, Robert Watson, and Mike Silbersack
* Pick the smallest possible TCP window scaling factor that will still allowsilby2007-10-191-2/+1
| | | | | | | | | | | | | | | | us to scale up to sb_max, aka kern.ipc.maxsockbuf. We do this because there are broken firewalls that will corrupt the window scale option, leading to the other endpoint believing that our advertised window is unscaled. At scale factors larger than 5 the unscaled window will drop below 1500 bytes, leading to serious problems when traversing these broken firewalls. With the default maxsockbuf of 256K, a scale factor of 3 will be chosen by this algorithm. Those who choose a larger maxsockbuf should watch out for the compatiblity problems mentioned above. Reviewed by: andre
* Add FBSDID to all files in netinet so that people can moresilby2007-10-071-1/+3
| | | | | | easily include file version information in bug reports. Approved by: re (kensmith)
* Two changes:silby2007-09-241-3/+3
| | | | | | | | | | | | | | | | | | - Reintegrate the ANSI C function declaration change from tcp_timer.c rev 1.92 - Reorganize the tcpcb structure so that it has a single pointer to the "tcp_timer" structure which contains all of the tcp timer callouts. This change means that when the single tcp timer change is reintegrated, tcpcb will not change in size, and therefore the ABI between netstat and the kernel will not change. Neither of these changes should have any functional impact. Reviewed by: bmah, rrs Approved by: re (bmah)
* Back out tcp_timer.c:1.93 and associated changes that reimplemented the manyrwatson2007-09-071-7/+4
| | | | | | | | | | | | | | | | | | | | | | | TCP timers as a single timer, but retain the API changes necessary to reintroduce this change. This will back out the source of at least two reported problems: lock leaks in certain timer edge cases, and TCP timers continuing to fire after a connection has closed (a bug previously fixed and then reintroduced with the timer rewrite). In a follow-up commit, some minor restylings and comment changes performed after the TCP timer rewrite will be reapplied, and a further change to allow the TCP timer rewrite to be added back without disturbing the ABI. The new design is believed to be a good thing, but the outstanding issues are leading to significant stability/correctness problems that are holding up 7.0. This patch was generated by silby, but is being committed by proxy due to poor network connectivity for silby this week. Approved by: re (kensmith) Submitted by: silby Tested by: rwatson, kris Problems reported by: peter, kris, others
* Make tcpstates[] static, and make sure TCPSTATES is defined beforedes2007-07-301-4/+0
| | | | | | | | | <netinet/tcp_fsm.h> is included into any compilation unit that needs tcpstates[]. Also remove incorrect extern declarations and TCPDEBUG conditionals. This allows kernels both with and without TCPDEBUG to build, and unbreaks the tinderbox. Approved by: re (rwatson)
* Fix compilation problems- tcpstates is only available if TCPDEBUGmjacob2007-07-291-1/+3
| | | | | | is set. Approved by: re (in spirit)
* Garbage collect some debug code that not only no longer couldmjacob2007-06-151-6/+0
| | | | | work but in fact probably causes a random pointer dereferences. Garbage collect the tp variable too.
* (1) In tcp_usrclosed(), tp can never become NULL, so don't test for NULLrwatson2007-05-311-4/+3
| | | | | | | | | before handling the socket disconnection case. (2) Clean up surrounding comments and formatting. Found with: Coverity Prevent(tm) (1) CID: 2203
* Reduce network stack oddness: implement .pru_sockaddr and .pru_peeraddrrwatson2007-05-111-3/+3
| | | | | | | | protocol entry points using functions named proto_getsockaddr and proto_getpeeraddr rather than proto_setsockaddr and proto_setpeeraddr. While it's true that sockaddrs are allocated and set, the net effect is to retrieve (get) the socket address or peer address from a socket, not set it, so align names to that intent.
* Remove unneeded wrappers for in_setsockaddr() and in_setpeeraddr(), whichrwatson2007-05-111-25/+2
| | | | | used to exist so pcbinfo locks could be acquired, but are no longer required as a result of socket/pcb reference model refinements.
* Move universally to ANSI C function declarations, with relativelyrwatson2007-05-101-0/+2
| | | | consistent style(9)-ish layout.
* Remove unused requested_s_scale from struct tcpcb.andre2007-05-061-2/+2
|
* Use existing TF_SACK_PERMIT flag in struct tcpcb t_flags field instead ofandre2007-05-061-3/+3
| | | | a decdicated sack_enable int for this bool. Change all users accordingly.
* Remove unused pcbinfo arguments to in_setsockaddr() andrwatson2007-05-011-2/+2
| | | | in_setpeeraddr().
* Change the TCP timer system from using the callout system five timesandre2007-04-111-8/+10
| | | | | | | | | | | | | | | | directly to a merged model where only one callout, the next to fire, is registered. Instead of callout_reset(9) and callout_stop(9) the new function tcp_timer_activate() is used which then internally manages the callout. The single new callout is a mutex callout on inpcb simplifying the locking a bit. tcp_timer() is the called function which handles all race conditions in one place and then dispatches the individual timer functions. Reviewed by: rwatson (earlier version)
* ANSIfy function declarations and remove register keywords for variables.andre2007-03-211-22/+9
| | | | Consistently apply style to all function declarations.
* Remove tcp_minmssoverload DoS detection logic. The problem it tried toandre2007-03-211-5/+0
| | | | | | protect us from wasn't really there and it only bloats the code. Should the problem surface in the future we can simply resurrect it from cvs history.
* Reap FIN_WAIT_2 connections marked SOCANTRCVMORE faster. This mitigatemohans2007-02-261-2/+7
| | | | | | | | potential issues where the peer does not close, potentially leaving thousands of connections in FIN_WAIT_2. This is controlled by a new sysctl fast_finwait2_recycle, which is disabled by default. Reviewed by: gnn, silby.
* Add "show inpcb", "show tcpcb" DDB commands, which should come in handyrwatson2007-02-171-1/+321
| | | | for debugging sblock and other network panics.
* Expose smoothed RTT and RTT variance measurements to userland viabms2007-02-021-0/+4
| | | | | | | socket option TCP_INFO. Note that the units used in the original Linux API are in microseconds, so use a 64-bit mantissa to convert FreeBSD's internal measurements from struct tcpcb from ticks.
* Auto sizing TCP socket buffers.andre2007-02-011-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Normally the socket buffers are static (either derived from global defaults or set with setsockopt) and do not adapt to real network conditions. Two things happen: a) your socket buffers are too small and you can't reach the full potential of the network between both hosts; b) your socket buffers are too big and you waste a lot of kernel memory for data just sitting around. With automatic TCP send and receive socket buffers we can start with a small buffer and quickly grow it in parallel with the TCP congestion window to match real network conditions. FreeBSD has a default 32K send socket buffer. This supports a maximal transfer rate of only slightly more than 2Mbit/s on a 100ms RTT trans-continental link. Or at 200ms just above 1Mbit/s. With TCP send buffer auto scaling and the default values below it supports 20Mbit/s at 100ms and 10Mbit/s at 200ms. That's an improvement of factor 10, or 1000%. For the receive side it looks slightly better with a default of 64K buffer size. New sysctls are: net.inet.tcp.sendbuf_auto=1 (enabled) net.inet.tcp.sendbuf_inc=8192 (8K, step size) net.inet.tcp.sendbuf_max=262144 (256K, growth limit) net.inet.tcp.recvbuf_auto=1 (enabled) net.inet.tcp.recvbuf_inc=16384 (16K, step size) net.inet.tcp.recvbuf_max=262144 (256K, growth limit) Tested by: many (on HEAD and RELENG_6) Approved by: re MFC after: 1 month
* Change the way the advertized TCP window scaling is computed. Instead ofandre2007-02-011-2/+7
| | | | | | | | | | | | | | | upper-bounding it to the size of the initial socket buffer lower-bound it to the smallest MSS we accept. Ideally we'd use the actual MSS information here but it is not available yet. For socket buffer auto sizing to be effective we need room to grow the receive window. The window scale shift is determined at connection setup and can't be changed afterwards. The previous, original, method effectively just did a power of two roundup of the socket buffer size at connection setup severely limiting the headroom for larger socket buffers. Tested by: many (as part of the socket buffer auto sizing patch) MFC after: 1 month
* Change error codes returned by protocol operations when an inpcb issam2006-11-221-6/+6
| | | | | | | | | | | | | | | | | | | | marked INP_DROPPED or INP_TIMEWAIT: o return ECONNRESET instead of EINVAL for close, disconnect, shutdown, rcvd, rcvoob, and send operations o return ECONNABORTED instead of EINVAL for accept These changes should reduce confusion in applications since EINVAL is normally interpreted to mean an invalid file descriptor. This change does not conflict with POSIX or other standards I checked. The return of EINVAL has always been possible but rare; it's become more common with recent changes to the socket/inpcb handling and with finer-grained locking and preemption. Note: there are other instances of EINVAL for this state that were left unchanged; they should be reviewed. Reviewed by: rwatson, andre, ru MFC after: 1 month
* Make tcp_usr_send() free the passed mbufs on error in all cases as theandre2006-09-171-0/+4
| | | | | | comment to it claims. Sponsored by: TCP/IP Optimization Fundraise 2005
* Change semantics of socket close and detach. Add a new protocol switchrwatson2006-07-211-55/+92
| | | | | | | | | | | | | | | | | | | function, pru_close, to notify protocols that the file descriptor or other consumer of a socket is closing the socket. pru_abort is now a notification of close also, and no longer detaches. pru_detach is no longer used to notify of close, and will be called during socket tear-down by sofree() when all references to a socket evaporate after an earlier call to abort or close the socket. This means detach is now an unconditional teardown of a socket, whereas previously sockets could persist after detach of the protocol retained a reference. This faciliates sharing mutexes between layers of the network stack as the mutex is required during the checking and removal of references at the head of sofree(). With this change, pru_detach can now assume that the mutex will no longer be required by the socket layer after completion, whereas before this was not necessarily true. Reviewed by: gnn
* Fix race conditions on enumerating pcb lists by moving the initializationups2006-07-181-2/+1
| | | | | | | | | | | | | | | ( and where appropriate the destruction) of the pcb mutex to the init/finit functions of the pcb zones. This allows locking of the pcb entries and race condition free comparison of the generation count. Rearrange locking a bit to avoid extra locking operation to update the generation count in in_pcballoc(). (in_pcballoc now returns the pcb locked) I am planning to convert pcb list handling from a type safe to a reference count model soon. ( As this allows really freeing the PCBs) Reviewed by: rwatson@, mohans@ MFC after: 1 week
* In tcp6_usr_attach(), return immediately if SS_ISDISCONNECTED, torwatson2006-06-261-4/+2
| | | | | | | avoid dereferencing an uninitialized inp variable. Submitted by: Michiel Boland <michiel at boland dot org> MFC after: 1 month
* Push acquisition of pcbinfo lock out of tcp_usr_attach() intorwatson2006-06-041-6/+8
| | | | | | | tcp_attach() after the call to soreserve(), as it doesn't require the global lock. Rearrange inpcb locking here also. MFC after: 1 month
* Instead of calling tcp_usr_detach() from tcp_usr_abort(), break outrwatson2006-04-241-52/+64
| | | | | | | | | | | | | | | common pcb tear-down logic into tcp_detach(), which is called from either. Invoke tcp_drop() from the tcp_usr_abort() path rather than tcp_disconnect(), as we want to drop it immediately not perform a FIN sequence. This is one reason why some people were experiencing panics in sodealloc(), as the netisr and aborting thread were simultaneously trying to tear down the socket. This bug could often be reproduced using repeated runs of the listenclose regression test. MFC after: 3 months PR: 96090 Reported by: Peter Kostouros <kpeter at melbpc dot org dot au>, kris Tested by: Peter Kostouros <kpeter at melbpc dot org dot au>, kris
* Clarify comment on handling of non-timewait TCP states inrwatson2006-04-031-5/+7
| | | | | | tcp_usr_detach(). MFC after: 3 months
* After checking for SO_ISDISCONNECTED in tcp_usr_accept(), returnrwatson2006-04-031-5/+3
| | | | | | | | | | | | | | immediately rather than jumping to the normal output handling, which assumes we've pulled out the inpcb, which hasn't happened at this point (and isn't necessary). Return ECONNABORTED instead of EINVAL when the inpcb has entered INP_TIMEWAIT or INP_DROPPED, as this is the documented error value. This may correct the panic seen by Ganbold. MFC after: 1 month Reported by: Ganbold <ganbold at micom dot mng dot net>
* During reformulation of tcp_usr_detach(), the call to initiate TCPrwatson2006-04-021-7/+18
| | | | | | | | | | disconnect for fully connected sockets was dropped, meaning that if the socket was closed while the connection was alive, it would be leaked. Structure tcp_usr_detach() so that there are two clear parts: initiating disconnect, and reclaiming state, and reintroduce the tcp_disconnect() call in the first part. MFC after: 3 months
* Properly handle an edge case previously not handled correctly: arwatson2006-04-011-2/+5
| | | | | | | | | | | socket can have a tcp connection that has entered time wait attached to it, in the event that shutdown() is called on the socket and the FINs properly exchange before close(). In this case we don't detach or free the inpcb, just leave the tcptw detached and freed, but we must release the inpcb lock (which we didn't previously). MFC after: 3 months
* Update TCP for infrastructural changes to the socket/pcb refcount model,rwatson2006-04-011-187/+382
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | pru_abort(), pru_detach(), and in_pcbdetach(): - Universally support and enforce the invariant that so_pcb is never NULL, converting dozens of unnecessary NULL checks into assertions, and eliminating dozens of unnecessary error handling cases in protocol code. - In some cases, eliminate unnecessary pcbinfo locking, as it is no longer required to ensure so_pcb != NULL. For example, the receive code no longer requires the pcbinfo lock, and the send code only requires it if building a new connection on an otherwise unconnected socket triggered via sendto() with an address. This should significnatly reduce tcbinfo lock contention in the receive and send cases. - In order to support the invariant that so_pcb != NULL, it is now necessary for the TCP code to not discard the tcpcb any time a connection is dropped, but instead leave the tcpcb until the socket is shutdown. This case is handled by setting INP_DROPPED, to substitute for using a NULL so_pcb to indicate that the connection has been dropped. This requires the inpcb lock, but not the pcbinfo lock. - Unlike all other protocols in the tree, TCP may need to retain access to the socket after the file descriptor has been closed. Set SS_PROTOREF in tcp_detach() in order to prevent the socket from being freed, and add a flag, INP_SOCKREF, so that the TCP code knows whether or not it needs to free the socket when the connection finally does close. The typical case where this occurs is if close() is called on a TCP socket before all sent data in the send socket buffer has been transmitted or acknowledged. If INP_SOCKREF is found when the connection is dropped, we release the inpcb, tcpcb, and socket instead of flagging INP_DROPPED. - Abort and detach protocol switch methods no longer return failures, nor attempt to free sockets, as the socket layer does this. - Annotate the existence of a long-standing race in the TCP timer code, in which timers are stopped but not drained when the socket is freed, as waiting for drain may lead to deadlocks, or have to occur in a context where waiting is not permitted. This race has been handled by testing to see if the tcpcb pointer in the inpcb is NULL (and vice versa), which is not normally permitted, but may be true of a inpcb and tcpcb have been freed. Add a counter to test how often this race has actually occurred, and a large comment for each instance where we compare potentially freed memory with NULL. This will have to be fixed in the near future, but requires is to further address how to handle the timer shutdown shutdown issue. - Several TCP calls no longer potentially free the passed inpcb/tcpcb, so no longer need to return a pointer to indicate whether the argument passed in is still valid. - Un-macroize debugging and locking setup for various protocol switch methods for TCP, as it lead to more obscurity, and as locking becomes more customized to the methods, offers less benefit. - Assert copyright on tcp_usrreq.c due to significant modifications that have been made as part of this work. These changes significantly modify the memory management and connection logic of our TCP implementation, and are (as such) High Risk Changes, and likely to contain serious bugs. Please report problems to the current@ mailing list ASAP, ideally with simple test cases, and optionally, packet traces. MFC after: 3 months
* Chance protocol switch method pru_detach() so that it returns voidrwatson2006-04-011-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | rather than an error. Detaches do not "fail", they other occur or the protocol flags SS_PROTOREF to take ownership of the socket. soclose() no longer looks at so_pcb to see if it's NULL, relying entirely on the protocol to decide whether it's time to free the socket or not using SS_PROTOREF. so_pcb is now entirely owned and managed by the protocol code. Likewise, no longer test so_pcb in other socket functions, such as soreceive(), which have no business digging into protocol internals. Protocol detach routines no longer try to free the socket on detach, this is performed in the socket code if the protocol permits it. In rts_detach(), no longer test for rp != NULL in detach, and likewise in other protocols that don't permit a NULL so_pcb, reduce the incidence of testing for it during detach. netinet and netinet6 are not fully updated to this change, which will be in an upcoming commit. In their current state they may leak memory or panic. MFC after: 3 months
* Change protocol switch pru_abort() API so that it returns void ratherrwatson2006-04-011-5/+13
| | | | | | | | | | | | | | than an int, as an error here is not meaningful. Modify soabort() to unconditionally free the socket on the return of pru_abort(), and modify most protocols to no longer conditionally free the socket, since the caller will do this. This commit likely leaves parts of netinet and netinet6 in a situation where they may panic or leak memory, as they have not are not fully updated by this commit. This will be corrected shortly in followup commits to these components. MFC after: 3 months
* Fix a bunch of SYSCTL_INT() that should have been SYSCTL_ULONG() tomux2005-12-141-2/+2
| | | | | | | match the type of the variable they are exporting. Spotted by: Thomas Hurst <tom@hur.st> MFC after: 3 days
* Push the assignment of a new or updated so_qlimit from solisten()rwatson2005-10-301-4/+4
| | | | | | | | | | | | | | following the protocol pru_listen() call to solisten_proto(), so that it occurs under the socket lock acquisition that also sets SO_ACCEPTCONN. This requires passing the new backlog parameter to the protocol, which also allows the protocol to be aware of changes in queue limit should it wish to do something about the new queue limit. This continues a move towards the socket layer acting as a library for the protocol. Bump __FreeBSD_version due to a change in the in-kernel protocol interface. This change has been tested with IPv4 and UNIX domain sockets, but not other protocols.
* Remove unnecessary IPSEC includes.andre2005-08-231-5/+0
| | | | | MFC after: 2 weeks Sponsored by: TCP/IP Optimization Fundraise 2005
* scope cleanup. with this changeume2005-07-251-0/+2
| | | | | | | | | | | | | | | | | | | - most of the kernel code will not care about the actual encoding of scope zone IDs and won't touch "s6_addr16[1]" directly. - similarly, most of the kernel code will not care about link-local scoped addresses as a special case. - scope boundary check will be stricter. For example, the current *BSD code allows a packet with src=::1 and dst=(some global IPv6 address) to be sent outside of the node, if the application do: s = socket(AF_INET6); bind(s, "::1"); sendto(s, some_global_IPv6_addr); This is clearly wrong, since ::1 is only meaningful within a single node, but the current implementation of the *BSD kernel cannot reject this attempt. Submitted by: JINMEI Tatuya <jinmei__at__isl.rdc.toshiba.co.jp> Obtained from: KAME
* When aborting tcp_attach() due to a problem allocating or attaching therwatson2005-06-011-0/+2
| | | | | | | tcpcb, lock the inpcb before calling in_pcbdetach() or in6_pcbdetach(), as they expect the inpcb to be passed locked. MFC after: 7 days
* Assert tcbinfo lock, inpcb lock in tcp_disconnect().rwatson2005-06-011-1/+8
| | | | | | Assert tcbinfo lock, inpcb lock in in tcp_usrclosed(). MFC after: 7 days
* Assert tcbinfo lock in tcp_attach(), as it is required; the callerrwatson2005-06-011-0/+2
| | | | | | (tcp_usr_attach()) currently grabs it. MFC after: 7 days
* Replace t_force with a t_flag (TF_FORCEDATA).ps2005-05-211-2/+2
| | | | | Submitted by: Raja Mukerji. Reviewed by: Mohan, Silby, Andre Opperman.
OpenPOWER on IntegriCloud