summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* MFC r317254: bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatchkevans2017-08-167-10/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bugs have been found in the fastmatch implementation as used in bsdgrep. Some have been fixed (r316495) while fixes for others are in review (D10098). In comparison with the fastmatch implementation, Kyle Evans found that: - regex(3)'s performance with literal expressions offers a speed improvement over fastmatch - regex(3)'s performance, both with simple BREs and EREs, seems to be comparable The regex implementation was imported in r226035, and the commit message reports: This is a temporary solution until the whole regex library is not replaced so that BSD grep development can continue and the backported code gets some review and testing. This change only improves scalability slightly, there is no big performance boost yet but several minor bugs have been found and fixed. Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing of both approaches. Regenerate src.conf(5) as per the original commit PR: 175314, 194823 Approved by: emaste (mentor, blanket MFC)
* MFC r303444 (ed): Call basename() in a portable way.kevans2017-08-161-5/+10
| | | | | | | | Pull a copy of the filename string before calling basename(). Change the loop to not return on its own, so we can put a free() statement at the bottom. Approved by: emaste (mentor, blanket MFC)
* MFC r322328:ae2017-08-161-5/+21
| | | | | | | | | | | | | | | | | | | Make user supplied data checks a bit stricter. key_msg2sp() is used for parsing data from setsockopt(IP[V6]_IPSEC_POLICY) call. This socket option is usually used to configure IPsec bypass for socket. Only privileged user can set this socket option. The message syntax is described here http://www.kame.net/newsletter/20021210/ and our libipsec is usually used to create the correct request. Add additional checks: * that sadb_x_ipsecrequest_len is not out of bounds of user supplied buffer * that src/dst's sa_len is the same * that 2*sa_len is not out of bounds of user supplied buffer * that 2*sa_len fits into bounds of sadb_x_ipsecrequest Reported by: Ilja van Sprundel
* MFC r322309, r322316:mav2017-08-164-20/+20
| | | | | | Use "Ibex Peak" codename for "5 Series/3400 Series" chipsets. This is shorter and unifies naming with later chipsets.
* MFC r322308: Add new Intel Lewisburg and Union Point chipset PCI IDs.mav2017-08-164-11/+31
| | | | While there, polish some old AHCI ones, since they are still reused.
* MFC r321899truckman2017-08-164-1/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lower the amd64 shared page, which contains the signal trampoline, from the top of user memory to one page lower on machines with the Ryzen (AMD Family 17h) CPU. This pushes ps_strings and the stack down by one page as well. On Ryzen there is some sort of interaction between code running at the top of user memory address space and interrupts that can cause FreeBSD to either hang or silently reset. This sounds similar to the problem found with DragonFly BSD that was fixed with this commit: https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/b48dd28447fc8ef62fbc963accd301557fd9ac20 but our signal trampoline location was already lower than the address that DragonFly moved their signal trampoline to. It also does not appear to be related to SMT as described here: https://www.phoronix.com/forums/forum/hardware/processors-memory/955368-some-ryzen-linux-users-are-facing-issues-with-heavy-compilation-loads?p=955498#post955498 "Hi, Matt Dillon here. Yes, I did find what I believe to be a hardware issue with Ryzen related to concurrent operations. In a nutshell, for any given hyperthread pair, if one hyperthread is in a cpu-bound loop of any kind (can be in user mode), and the other hyperthread is returning from an interrupt via IRETQ, the hyperthread issuing the IRETQ can stall indefinitely until the other hyperthread with the cpu-bound loop pauses (aka HLT until next interrupt). After this situation occurs, the system appears to destabilize. The situation does not occur if the cpu-bound loop is on a different core than the core doing the IRETQ. The %rip the IRETQ returns to (e.g. userland %rip address) matters a *LOT*. The problem occurs more often with high %rip addresses such as near the top of the user stack, which is where DragonFly's signal trampoline traditionally resides. So a user program taking a signal on one thread while another thread is cpu-bound can cause this behavior. Changing the location of the signal trampoline makes it more difficult to reproduce the problem. I have not been because the able to completely mitigate it. When a cpu-thread stalls in this manner it appears to stall INSIDE the microcode for IRETQ. It doesn't make it to the return pc, and the cpu thread cannot take any IPIs or other hardware interrupts while in this state." since the system instability has been observed on FreeBSD with SMT disabled. Interrupts to appear to play a factor since running a signal-intensive process on the first CPU core, which handles most of the interrupts on my machine, is far more likely to trigger the problem than running such a process on any other core. Also lower sv_maxuser to prevent a malicious user from using mmap() to load and execute code in the top page of user memory that was made available when the shared page was moved down. Make the same changes to the 64-bit Linux emulator. PR: 219399 Reported by: nbe@renzel.net Reviewed by: kib Reviewed by: dchagin (previous version) Tested by: nbe@renzel.net (earlier version) Differential Revision: https://reviews.freebsd.org/D11780
* bsdgrep: Use implied working directory for -r if no directories are passedkevans2017-08-164-2/+94
| | | | | | | | | | | | | | | | | | | | | | MFC r317050: bsdgrep: for -r, use the working directory if none specified This is more sensible than the previous behaviour of grepping stdin, and matches newer GNU grep behaviour. MFC r317300 (ngie): Only expect :grep_r_implied to pass with bsdgrep(1) The test fails with gnu grep from base and ports. MFC r319002 (ngie): :rgrep : use atf-check to check the exit code/save the output of grep -r instead of calling grep -r without it, and saving the output to a file This ensures that any errors thrown via grep -r are caught, not lost, and uses existing atf-sh idioms for saving files. PR: 216307 Approved by: emaste (mentor, blanket MFC) Relnotes: yes
* MFC r317051: bsdgrep: remove output separators between overlapping segmentskevans2017-08-161-6/+27
| | | | | | | | | | | | | Make bsdgrep more sensitive to context overlaps. If it's printing context that either overlaps or is immediately adjacent to another bit of context, don't print a separator. - Non-overlapping segments no longer have two separators between them - Overlapping segments no longer have separators between them with overlapping sections repeated Approved by: emaste (mentor, blanket MFC)
* bsdgrep: Revise tests based on recent fixes and future changeskevans2017-08-161-13/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MFC r317297: Remove the expected failures for :context and :context2 with bsdgrep(1) They're no longer needed after recent fixes made to bsdgrep(1). MFC r317299: Add more sanity tests for grep, egrep, and fgrep The test suite currently lacks basic sanity checks to ensure that egrep, fgrep, and grep are actually matching the right expression types, i.e. passing the right flags to regcomp(3). Amend the test suite to make sure that not only are the individual versions doing the right thing, but also that we don't have some kind of frankenregex situation happening where egrep is accepting a BRE or grep an ERE. I've chosen to not expand the 'basic' test but to add the 'grep_sanity' checks to their own test case since this is testing for more than just 'grep matches things', but actual expression types. MFC r317694: bsdgrep: revise test case which will soon become a failure Work in progress (D10315) is going to make egrep_empty_invalid an actually invalid regex, to be consistent with the equivalent BRE "{" behavior, when using regex(3). Any non-0 exit value is acceptable, depending on how the installed grep interprets the expression. GNU grep interprets it as non-matching, and in the future BSD grep will interpret it is an error. Approved by: emaste (mentor, blanket MFC)
* bsdgrep: add -z/--null-data support and update NLS catalogs accordinglykevans2017-08-1616-21/+32
| | | | | | | | | | | | | | | | MFC r317049: bsdgrep: add -z/--null-data support -z treats input and output data as sequences of lines terminated by a zero byte instead of a newline. This brings it more in line with GNU grep and brings us closer to passing the current tests with BSD grep. MFC r317679: bsdgrep: correct nls usage data after r317049 r317049 added -z/--null-data to BSD grep but missed the update to nls catalogs. Approved by: emaste (mentor, blanket MFC) Relnotes: yes
* MFC r316542: bsdgrep: Handle special case of single-byte NUL patternkevans2017-08-161-1/+5
| | | | | PR: 202022 Approved by: emaste (mentor, blanket MFC)
* MFC r316495: bsdgrep(1): Fix errors with invalid expressionskevans2017-08-161-2/+2
| | | | | | | | Invalid expressions with an ultimate compiled pattern length of 0 (e.g., "grep -E {") were not taken into account and caused a segfault while trying to fill in the good suffix table. Approved by: emaste (mentor, blanket MFC)
* MFC r316492: bsdgrep(1): Rip out "xmalloc" bitskevans2017-08-166-463/+31
| | | | | | | | | | | xmalloc was a debug malloc implementation, but the x{malloc,calloc,free} functions default to calling the malloc(3) equivalents. Instead of relying on this malloc shim, we can devise better ways to debug malloc issues that aren't misleading upon initial inspection. (I.e., using jemalloc's various built-in debugging capabilities.) Approved by: emaste (mentor, blanket MFC)
* MFC r316750 (ngie): Fix expectations for testcases per bsdgrep vs gnu grepkevans2017-08-161-0/+60
| | | | | | | | | | | | | | | | | The following failures occur with various versions of grep: BSD grep: - :context - :context2 GNU grep (base): - :color - :oflag_zerolen GNU grep (ports): - :recurse_symlink Approved by: emaste (mentor, blanket MFC)
* bsdgrep: Fix matching behavior and add regression testskevans2017-08-1619-16/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MFC r316477: bsdgrep: fix matching behaviour - Set REG_NOTBOL if we've already matched beginning of line and we're examining later parts - For each pattern we examine, apply it to the remaining bits of the line rather than (potentially) smaller subsets - Check for REG_NOSUB after we've looked at all patterns initially matching the line - Keep track of the last match we made to later determine if we're simply not matching any longer or if we need to proceed another byte because we hit a zero-length match - Match the earliest and longest bit of each line before moving the beginning of what we match to further in the line, past the end of the longest match; this generally matches how gnugrep(1) seems to behave, and seems like pretty good behavior to me - Finally, bail out of printing any matches if we were set to print all (empty pattern) but -o (output matches) was set MFC r316489: bsdgrep: Initialize vars to avoid a false positive GCC warning MFC r316491: bsdgrep: revert color changes from r316477 r316477 changed the color output to match exactly the in-tree GNU grep, but introduces unnecessary escape sequences. MFC r316536: bsdgrep: create additional tests for coverage on recent fixes Create additional tests to cover regressions that were discovered by PRs linked to reviews D10098, D10102, and D10104. It is worth noting that neither bsdgrep(1) nor gnugrep(1) in the base system currently pass all of these tests, and gnugrep(1) not quite being up to snuff was also noted in at least one of the PRs. MFC r317052: bsdgrep: fix zero-length matches without the -o flag r316477 broke zero-length matches when not using the -o flag, by skipping over them entirely. Add a regression test so that it doesn't break again in the future. PR: 175314, 180990, 181263, 195763, 197531, 197555, 202022, 209116 Approved by: emaste (mentor, blanket MFC) Relnotes: yes
* MFC r322250:hselasky2017-08-151-4/+4
| | | | | | | | | Count drop events due to lack of PCI bandwidth as queue drops and not as input errors in the mlx5en(4) driver. This improves the sysadmin view of physical port errors. Submitted by: gallatin@ Sponsored by: Mellanox Technologies
* MFC r322251:hselasky2017-08-151-2/+4
| | | | | | | Make sure the received IP header gets 32-bit aligned for short packets in the mlx5en(4) driver. Sponsored by: Mellanox Technologies
* MFC r322456:kib2017-08-151-2/+2
| | | | | On i386 with CPUID but without SSE2, set lfence_works to LMB_NONE instead of looping.
* MFC r322426:kib2017-08-151-1/+1
| | | | Fix indent.
* MFC r322256:kib2017-08-151-4/+4
| | | | | | Fix logic error in the the assert, causing the condition to be always true. PR: 217741
* MFC r322248:hselasky2017-08-155-15/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fix for mlx4en(4) to properly call m_defrag(). The m_defrag() function can only defrag mbuf chains which have a valid mbuf packet header. In r291699 when the mlx4en(4) driver was converted into using BUSDMA(9), the call to m_defrag() was moved after the part of the transmit routine which strips the header from the mbuf chain. This effectivly disabled the mbuf defrag mechanism and such packets simply got dropped. This patch removes the stripping of mbufs from a chain and loads all mbufs using busdma. If busdma finds there are no segments, unload the DMA map and free the mbuf right away, because that means all data in the mbuf has been inlined in the TX ring. Else proceed as usual. Add a per-ring rounter for the number of defrag attempts and make sure the oversized_packets counter gets zeroed while at it. The counters are per-ring to avoid excessive cache misses in the TX path. Submitted by: mjoras@ Differential Revision: https://reviews.freebsd.org/D11683 Sponsored by: Mellanox Technologies
* bsdgrep: treat rgrep as grep -r and install rgrep symlinkkevans2017-08-153-1/+10
| | | | | | | | | | | | | MFC r316473: bsdgrep: treat rgrep as grep -r MFC r316484: bsdgrep(1): create rgrep link Create a convenience rgrep link for bsdgrep(1) that observes 'grep -r' behavior. A follow-up to r316473. Approved by: emaste (mentor)
* MFC r322368, r322371:pfg2017-08-152-2/+3
| | | | | | | | | | | | | | fnmatch(3): improve POSIX conformance. In a recent interpretation[1], "\\" shall return a non-zero value (indicating either no match or an error). The fix involves a change over r254091 and now the behavior matches the Sun/IBM/HP closed source implementations and also likely musl libc. Submitted by: Joerg Schilling <joerg at schily.net> [1] http://austingroupbugs.net/view.php?id=806
* MFC r313948: bsdgrep: fix EOF handling with --mmapkevans2017-08-141-5/+11
| | | | | | | | | Rework part of the loop in grep_fgetln to return the rest of the line and ensure that we still advance the buffer by the length of the rest of the line. PR: 165471 Approved by: emaste (mentor)
* MFC r322139gahr2017-08-144-33/+128
| | | | | | | | | Enhance top(1) to filter on multiple usernames Reviewed by: cognet, bapt Approved by: cognet Relnotes: yes Differential Revision: https://reviews.freebsd.org/D11840
* MFC r322306:hselasky2017-08-141-1/+2
| | | | | | | | Print maximum MTU when trying to set invalid MTU in the mlx4en(4) driver. Useful for debugging. Submitted by: Sepherosa Ziehau <sephe@dragonflybsd.org> Sponsored by: Mellanox Technologies
* MFC r322304:hselasky2017-08-141-36/+68
| | | | | | | | Add support for RX and TX statistics when the mlx4en(4) PCI device is in VF or SRIOV mode typically in a virtual machine environment. Submitted by: Sepherosa Ziehau <sephe@dragonflybsd.org> Sponsored by: Mellanox Technologies
* MFC r314878:hselasky2017-08-141-3/+4
| | | | | | | | | Add support for constant pointer constructs to READ_ONCE() in the LinuxKPI. When the type of the argument is constant the temporary variable cannot be assigned after the barrier. Instead assign the temporary variable by initialization. Sponsored by: Mellanox Technologies
* MFC r322305:hselasky2017-08-141-0/+4
| | | | | | | | Increment queue drops in the network statistics when transmitted packets are dropped by the mlx4en(4) driver. Submitted by: Sepherosa Ziehau <sephe@dragonflybsd.org> Sponsored by: Mellanox Technologies
* MFC r322175:kib2017-08-141-15/+35
| | | | | Avoid DI recursion when reclaim_pv_chunk() is called from pmap_advise() or pmap_remove().
* MFC r322171:kib2017-08-141-0/+29
| | | | | Explain why delayed invalidation is not required in pmap_protect() and pmap_remove_pages().
* MFC 322323 by jkimsephe2017-08-144-15/+17
| | | | | | | | | | Split identify_cpu() into two functions for amd64 as we do for i386. This reduces diff between amd64 and i386. Also, it fixes a regression introduced in r322076, i.e., identify_hypervisor() failed to identify some hypervisors. This function assumes cpu_feature2 is already initialized. Reported by: dexuan Tested by: dexuan
* MFC 322299sephe2017-08-143-41/+811
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | hyperv/hn: Implement transparent mode network VF. How network VF works with hn(4) on Hyper-V in transparent mode: - Each network VF has a cooresponding hn(4). - The network VF and the it's cooresponding hn(4) have the same hardware address. - Once the network VF is attached, the cooresponding hn(4) waits several seconds to make sure that the network VF attach routing completes, then: o Set the intersection of the network VF's if_capabilities and the cooresponding hn(4)'s if_capabilities to the cooresponding hn(4)'s if_capabilities. And adjust the cooresponding hn(4) if_capable and if_hwassist accordingly. (*) o Make sure that the cooresponding hn(4)'s TSO parameters meet the constraints posed by both the network VF and the cooresponding hn(4). (*) o The network VF's if_input is overridden. The overriding if_input changes the input packet's rcvif to the cooreponding hn(4). The network layers are tricked into thinking that all packets are neceived by the cooresponding hn(4). o If the cooresponding hn(4) was brought up, bring up the network VF. The transmission dispatched to the cooresponding hn(4) are redispatched to the network VF. o Bringing down the cooresponding hn(4) also brings down the network VF. o All IOCTLs issued to the cooresponding hn(4) are pass-through'ed to the network VF; the cooresponding hn(4) changes its internal state if necessary. o The media status of the cooresponding hn(4) solely relies on the network VF. o If there are multicast filters on the cooresponding hn(4), allmulti will be enabled on the network VF. (**) - Once the network VF is detached. Undo all damages did to the cooresponding hn(4) in the above item. NOTE: No operation should be issued directly to the network VF, if the network VF transparent mode is enabled. The network VF transparent mode can be enabled by setting tunable hw.hn.vf_transparent to 1. The network VF transparent mode is _not_ enabled by default, as of this commit. The benefit of the network VF transparent mode is that the network VF attachment and detachment are transparent to all network layers; e.g. live migration detaches and reattaches the network VF. The major drawbacks of the network VF transparent mode: - The netmap(4) support is lost, even if the VF supports it. - ALTQ does not work, since if_start method cannot be properly supported. (*) These decisions were made so that things will not be messed up too much during the transition period. (**) This does _not_ need to go through the fancy multicast filter management stuffs like what vlan(4) has, at least currently: - As of this write, multicast does not work in Azure. - As of this write, multicast packets go through the cooresponding hn(4). Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D11803
* MFC r322178mckusick2017-08-141-10/+10
| | | | | | | | | Bug 198500 reports bad sysctl values for gjournal cache limit. PR: 198500 Submitted by: Dr. Andreas Longwitz <longwitz@incore.de> Reported by: Eugene Grosbein Discussed with: kib
* MFC r322113:cy2017-08-131-3/+0
| | | | Remove dead target introduced in r178828.
* MFC r320736: acpidump: warn and exit loop on invalid subtable lengthemaste2017-08-131-0/+4
| | | | | Submitted by: Guangyuan Yang <yzgyyang@outlook.com> Sponsored by: The FreeBSD Foundation
* MFC r304000 (maxim): acpidump: move variable initialization out of assert(3)emaste2017-08-131-4/+4
| | | | | | | This fixes acpidump(8) compiled with "WITHOUT_ASSERT_DEBUG=yes" that removes assert(3)'s from the code. Submitted by: Alexander Nedotsukov
* MFC r321298: acpidump: add ACPI NFIT (NVDIMM Firmware Interface Table)emaste2017-08-131-0/+198
| | | | Also MFC r322351 by mav, add two NFIT fields missed in initial commit.
* MFC r215837: uart: add AX99100 chipset supportemaste2017-08-131-0/+2
| | | | PR: 215837
* MFC r322023:ngie2017-08-121-6/+1
| | | | | | | | | | Remove special-case logic for running tests on host machines I'm not sure what process sjg@ was using, but using CHECKDIR=${.OBJDIR} with "make check" on ^/head is the correct thing to do. This unbreaks "make check" for me (unsandboxed, not using CHECKDIR=${.OBJDIR}). While here, fix a whitespace nit with LIBADD.
* MFC: r322386 Update private sqlite3-3.14.1 to sqlite3-3.20.0.peter2017-08-1210-9752/+19549
|
* MFC: r322380 Update subversion 1.9.5 -> 1.9.7peter2017-08-1224-139/+489
|
* MFC r321684:dim2017-08-122-6/+3
| | | | | | | | | | | | | | | | | | | | | | Don't use libc++ when cross-building for gcc arches Since we imported clang 5.0.0, the version check in Makefile.inc1 which checks whether to use libc++ fires even when the compiler for the target architecture is gcc 4.2.1. This is because only X_COMPILER_VERSION is checked. Also check X_COMPILER_TYPE, so it will only use libc++ when an external gcc toolchain is used. Reviewed by: emaste, rpokala Differential Revision: https://reviews.freebsd.org/D11776 MFC r322170: Follow-up to r321684 (Don't use libc++ when cross-building for gcc arches), and handle two more cases where libc++ includes could be incorrectly enabled, in case the host compiler is clang 5.0.0, and the target (cross) compiler is gcc 4.2.1. Noted by: bdrewery
* MFC r322302: Do not loose CCB flags after r320493.mav2017-08-121-1/+2
| | | | There is at least CAM_UNLOCKED that should be kept.
* MFC r322077:kib2017-08-121-9/+21
| | | | Provide more detailed specification for major(), minor() and makedev().
* MFC r322059:kib2017-08-111-2/+2
| | | | | Fix off by one in calculation of the number of buckets for the pc addresses.
* MFC r322050:kib2017-08-111-2/+8
| | | | Relax visibility for some termios symbols.
* MFC r321924:ed2017-08-111-1/+2
| | | | | | | | | | | | Keep top page on CloudABI to work around AMD Ryzen stability issues. Similar to r321899, reduce sv_maxuser by one page inside of CloudABI. This ensures that the stack, the vDSO and any allocations cannot touch the top page of user virtual memory. Considering that CloudABI userspace is completely oblivious to virtual memory layout, don't bother making this conditional based on the CPU of the running system.
* MFC r322124:avos2017-08-111-1/+1
| | | | rfcomm_pppd.8: fix a typo (SPD -> SDP).
* MFC: r322209marius2017-08-115-53/+189
| | | | | | | | | | | | | | | | | | | | | | | | - If available, use TRIM instead of ERASE for implementing BIO_DELETE. This also involves adding a quirk table as TRIM is broken for some Kingston eMMC devices, though. Compared to ERASE (declared "legacy" in the eMMC specification v5.1), TRIM has the advantage of operating on write sectors rather than on erase sectors, which typically are of a much larger size. Thus, employing TRIM, we don't need to fiddle with coalescing BIO_DELETE requests that are also of (write) sector units into erase sectors, which might not even add up in all cases. - For some SanDisk iNAND devices, the CMD38 argument, e. g. ERASE, TRIM etc., has to be specified via EXT_CSD[113], which now is also handled via a quirk. - My initial understanding was that for eMMC partitions, the granularity should be used as erase sector size, e. g. 128 KB for boot partitions. However, rereading the relevant parts of the eMMC specification v5.1, this isn't actually correct. So drop the code which used partition granularities for delmaxsize and stripesize. For the most part, this change is a NOP, though, because a) for ERASE, mmcsd_delete() used the erase sector size unconditionally for all partitions anyway and b) g_disk_limit() doesn't actually take the stripesize into account. - Take some more advantage of mmcsd_errmsg() in mmcsd(4) for making error codes human readable.
OpenPOWER on IntegriCloud