| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
| |
malloc(9) flags in sys/dev.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
These are available as t3_tom and t4_tom modules that augment cxgb(4)
and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as
usual with or without these extra features.
- iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the
works and will follow soon.
Build-tested with make universe.
30s overview
============
What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE
Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe
Which connections are offloaded? Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe
Reviewed by: bz, gnn
Sponsored by: Chelsio communications.
MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
|
|
|
|
| |
Approved by: kmacy
|
|
|
|
|
| |
Reviewed by: kmacy
Approved by: gnn (mentor)
|
| |
|
|
|
|
| |
- fix allocation failure handing in refill_fl
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- remove mbuf iovec - useful, but adds too much complexity when isolated to
the driver
- remove driver private caching - insufficient benefit over UMA to justify
the added complexity and maintenance overhead
- remove separate logic for managing multiple transmit queues, with the
new drbr routines the control flow can be made to much more closely resemble
legacy drivers
- remove dedicated service threads, with per-cpu callouts one can get the same
benefit much more simply by registering a callout 1 tick in the future if there
are still buffered packets
- remove embedded mbuf usage - Jeffr's changes will (I hope) soon be integrated
greatly reducing the overhead of using kernel APIs for reference counting
clusters
- add hysteresis to descriptor coalescing logic
- add coalesce threshold sysctls to allow users to decide at run-time
between optimizing for forwarding / UDP or optimizing for TCP
- add once per second watchdog to effectively close the very rare races
occurring from coalescing
- incorporate Navdeep's changes to the initialization path required to
convert port and adapter locks back to ordinary mutexes (silencing BPF
LOR complaints)
- enable prefetches in get_packet and tx cleaning
Reviewed by: navdeep@
MFC after: 2 weeks
|
| |
|
|
|
|
|
|
| |
- don't leak mbuf tags in the non-conditional case
Found by: Navdeep Parhar
|
|
|
|
| |
Found by: kkenn@
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and ifnet functions
- add memory barriers to <machine/atomic.h>
- update drivers to only conditionally define their own
- add lockless producer / consumer ring buffer
- remove ring buffer implementation from cxgb and update its callers
- add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to
allow drivers to efficiently manage multiple hardware queues
(i.e. not serialize all packets through one ifq)
- expose if_qflush to allow drivers to flush any driver managed queues
This work was supported by Bitgravity Inc. and Chelsio Inc.
|
|
|
|
|
|
|
| |
- determine which ext_arg offsets to use based on the version number
Submitted by: Chelsio Inc.
MFC after: 1 day
|
|
|
|
| |
Submitted by: Chelsio Inc.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- add support for T3C
- add DDP support (zero-copy receive)
- fix TOE transmit of large requests
- fix shutdown so that sockets don't remain in CLOSING state indefinitely
- register listeners when an interface is brought up after tom is loaded
- fix setting of multicast filter
- enable link at device attach
- exit tick handler if shutdown is in progress
- add helper for logging TCB
- add sysctls for dumping transmit queues
- note that TOE wxill not be MFC'd until after 7.0 has been finalized
MFC after: 3 days
|
| |
|
|
|
|
|
| |
- disable pcpu cluster cache by default until reference counting is handled
correctly for held clusters - can be re-enable by sysctl
|
|
|
|
|
|
|
| |
m_collapse from cxgb)
Reviewed by: pyun, jhb, kmacy
MFC after: 2 weeks
|
| |
|
|
|
|
|
|
|
|
| |
associated mbufs
- Track packet zone mbufs separately from other mbufs
- free packet zone buffers via m_free rather than trying to manage the refcount
as with clusters - its refcount and management seems to be "special"
|
|
|
|
|
|
| |
appease the tinderbox on 32-bit platforms.
Tested on: amd64, i386
|
|
|
|
|
|
|
|
|
|
|
| |
- increase asserts for mbuf accounting
- track outstanding mbufs (maps very closely to leaked)
- actually only create one thread per port if !multiq
Oddly enough this fixes the use after free
- move txq_segs to stack in t3_encap
- add checks that pidx doesn't move pass cidx
- simplify mbuf free logic in collapse mbufs routine
|
|
|
|
|
|
|
|
|
|
|
|
| |
- move cxgb_tx_common in to cxgb_multiq.c and rename to cxgb_tx
- move cxgb_tx_common dependencies
- further simplify cxgb_dequeue_packet for the non-multiqueue case
- only launch one service thread per port in the non-multiq case
- remove dead cleaning code from cxgb_sge.c
- simplify PIO case substantially in by returning directly from mbuf collapse
and just using m_copydata
- remove gratuitous m_gethdr in the rx path
- clarify freeing of mbufs in collapse
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- return the error from cxgb_tx_common so that when an error is hit we dont
spin forever in the taskq thread
- remove unused rxsd_ref
- simplify header_offset calculation for embedded mbuf headers
- fix memory leak by making sure that mbuf header initialization took place
- disable printf's for stalled queue, don't do offload/ctrl queue restart
when tunnel queue is restarted
- add more diagnostic information about the txq state
- add facility to dump the actual contents of the hardware queue using sysctl
|
| |
|
| |
|
|
|
|
| |
this was missed in the initial import
|
|
|
|
|
|
|
|
|
| |
and t3_push_frames).
- Import latest changes to cxgb_main.c and cxgb_sge.c from toestack p4 branch
- make driver local copy of tcp_subr.c and tcp_usrreq.c and override tcp_usrreqs so
TOE can also functions on versions with unmodified TCP
- add cxgb back to the build
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
revision 4.5.5
- add filter support
- further improvements for T304
- recover gracefully from spurious immediate packets
Approved by: re(blanket)
Supported by: Chelsio
MFC after: 3 days
|
|
|
|
|
|
| |
This will make it possible to build the module out of tree against an older src tree.
MFC after: 3 days
|
| |
|
|
|
|
|
|
|
|
|
|
| |
- upgrade to reflect state of 1.0.0.86
- move from firmware rev 3.2 to 4.0.0
- import driver bits for offload functionality
- remove binary distribution clause from top level files as it
runs counter to the intent of purely supporting the hardware
MFC after: 3 days
|
| |
|
|
|
|
|
| |
Found by: Coverity
Submitted by: jhb
|
| |
|
| |
|
| |
|
|
|
|
| |
kill redundant INVARIANTS check
|
| |
|
|
|
|
|
|
|
|
| |
implement robust version of m_collapse
add support for sf_buf
add fix for m_iovappend
add calls to m_sanity under INVARIANTS
fix m_freem_vec to correctly travese the mbuf iovec chain
|
|
|
|
|
|
|
| |
double free's and an infinite loop.
CID: 1834
Found by: Coverity Prevent (tm)
|
| |
|
|
|
|
| |
for now simply don't compile/use on sparc64
|
|
|
|
| |
change m_collapse to return an error code
|