summaryrefslogtreecommitdiffstats
path: root/lib/libfetch
Commit message (Collapse)AuthorAgeFilesLines
* Support Last-Modified behind proxies which return UTC instead of GMT.bdrewery2014-03-111-0/+6
| | | | | | | | | | | | | | | | The standard states that GMT must be used, but that UTC is equivalent. Still parse UTC as otherwise this causes problems for pkg(8). It will refetch the repository every time 'pkg update' or other remote operations are used behind these proxies. RFC2616: "All HTTP date/time stamps MUST be represented in Greenwich Mean Time (GMT), without exception. For the purposes of HTTP, GMT is exactly equal to UTC (Coordinated Universal Time)."" Approved by: bapt (mentor) Reviewed by: des, peter Sponsored by: EMC / Isilon Storage Division MFC after: 1 week
* Bump copyright datesdes2014-01-303-3/+3
|
* r261230 broke the cases where the amount of data to be read is notdes2014-01-292-17/+16
| | | | | | | | | | | | | | | | | | | | | | | known in advance, or where the caller doesn't care and just keeps reading until it hits EOF. In fetch_read(): the socket is non-blocking, so read() will return 0 on EOF, and -1 (errno == EAGAIN) when the connection is still open but there is no data waiting. In the first case, we should immediately return 0. The EINTR case was also broken, although not in a way that matters. In fetch_writev(): use timersub() and timercmp() as in fetch_read(). In http_fillbuf(): set errno to a sensible value when an invalid chunk header is encountered. In http_readfn(): as in fetch_read(), a zero return from down the stack indicates EOF, not an error. Furthermore, when io->error is EINTR, clear it (but no errno) before returning so the caller can retry after dealing with the interrupt. MFC after: 3 days
* Solve http buffering issues and hangs once and for all (hopefully!) bydes2014-01-283-121/+57
| | | | | | | | | | | | | | | | | simply not trying to return exactly what the caller asked for - just return whatever we got and let the caller be the judge of whether it was enough. If an error occurs or the connection times out after we already received some data, return a short read, under the assumption that the next call will fail or time out before we read anything. As it turns out, none of the code that calls fetch_read() assumes an all-or-nothing result anyway, except for a couple of lines where we read the CR LF at the end of a hunk in HTTP hunked encoding, so the changes outside of fetch_read() and http_readfn() are minimal. While there, replace select(2) with poll(2). MFC after: 3 days
* Fix format string.des2014-01-201-1/+1
| | | | | Submitted by: Jörg Sonnenberger <joerg@NetBSD.org> MFC after: 1 week
* Fix build with GCCbdrewery2013-11-191-1/+2
| | | | | | | | | | SSL_set_tlsext_host_name(3) internally does not modify the host buffer pased to it. So it is safe to DECONST the struct url* here. Reported by: gjb Approved by: bapt (implicit) MFC after: 1 week X-MFC-With: r258347
* Support SNI in libfetchbdrewery2013-11-191-0/+9
| | | | | | | | | | | | | SNI is Server Name Indentification which is a protocol for TLS that indicates the host that is being connected to at the start of the handshake. It allows to use Virtual Hosts on HTTPS. Submitted by: sbz Submitted by: Michael Gmelin <freebsd@grem.de> [1] PR: kern/183583 [1] Reviewed by: des Approved by: bapt MFC after: 1 week
* Even though it doesn't really make sense in the context of a CONNECTdes2013-08-221-0/+2
| | | | | | | | | request, RFC 2616 14.23 mandates the presence of the Host: header in all HTTP 1.1 requests. PR: kern/181445 Submitted by: Kimo <kimor79@yahoo.com> MFC after: 3 days
* Include an Accept header in requests.des2013-07-302-3/+18
| | | | | PR: kern/180917 MFC after: 1 week
* Implement certificate verification, and many other SSL-relateddes2013-07-264-16/+632
| | | | | | | | imrovements; complete details in the PR. PR: kern/175514 Submitted by: Michael Gmelin <freebsd@grem.de> MFC after: 1 week
* Use the correct request syntax for proxied (tunneled) HTTPS requests.des2013-07-211-1/+1
| | | | | PR: bin/180666 MFC after: 3 days
* Fix -Wunsequenced warning.kientzle2013-06-291-1/+1
| | | | Submitted by: dt71@gmx.com
* Use the CONNECT method to proxy HTTPS connections through HTTP proxies.des2013-04-121-8/+13
| | | | | PR: bin/80176 Submitted by: Yuichiro NAITO <naito.yuichiro@gmail.com>
* Fix weird indentation.des2012-11-161-4/+4
|
* Implement HTTP 305 redirect handling.eadler2012-10-221-1/+5
| | | | | | | | PR: 172452 Submitted by: gcooper Reviewed by: des Approved by: cperciva MFC after: 1 week
* Don't deny non-temporary redirects if the -A option is set (pereadler2012-10-222-2/+13
| | | | | | | | | | | | the man page) [0] While here add support for draft-reschke-http-status-308-07 PR: 172451 [0] Submitted by: gcooper [0] Reviewed by: des Approved by: cperciva MFC after: 1 week
* Be a bit more lenient in the maximum number of redirects allowed.eadler2012-10-221-1/+1
| | | | | | | | Chrome and Firefox have a limit of 20. IE has a limit of 8. Reviewed by: des Approved by: cperciva MFC after: 3 days
* Use libmd if and only if OpenSSL is not available.des2012-09-142-2/+10
| | | | | PR: bin/171402 MFC after: 3 days
* Adding missing return statements during error conditions.eadler2012-09-141-2/+8
| | | | | | | | PR: kern/171187 Submitted by: Mark Johnston <markjdb@gmail.com> Reviewed by: des Approved by: cperciva MFC after: 2 weeks
* Merge OpenSSL 1.0.1c.jkim2012-07-121-1/+1
| | | | Approved by: benl (maintainer)
* libfetch: Avoid SIGPIPE on network connections.jilles2012-05-281-0/+2
| | | | | | | | | | | | To avoid unexpected process termination from SIGPIPE when writing to a closed network connection, enable SO_NOSIGPIPE on all network connections. The POSIX standard MSG_NOSIGNAL is not used since it requires modifying all send calls to add this flag. This is particularly nasty for SSL connections. Reviewed by: des Tested by: bapt MFC after: 5 days
* Don't reuse credentials if redirected to a different host.des2012-04-301-1/+3
| | | | | Submitted by: Niels Heinen <heinenn@google.com> MFC after: 3 weeks
* Since the socket is non-blocking, it is necessary to use select(2) evendes2012-04-301-17/+13
| | | | | | | | | when there is no timeout, because read(2) will return immediately if there is no data waiting in the TCP buffer, causing fetch_read() to busy-loop on slow connections. MFC after: 3 weeks Noticed by: Yanhui Shen <shen.elf@gmail.com>
* Support percent-encoded user and passwordemaste2012-04-111-6/+45
| | | | | | | | | RFC 1738 specifies that any ":", "@", or "/" within a user name or password in a URL is percent-encoded, to avoid ambiguity with the use of those characters as URL component separators. Reviewed by: rstone@ MFC after: 1 month
* Remove trailing whitespace per mdoc lint warningeadler2012-03-291-1/+1
| | | | | | | Disussed with: gavin No objection from: doc Approved by: joel MFC after: 3 days
* Fix two nits in previous commit pointed out by pjd@.des2012-01-231-2/+1
| | | | MFC after: 3 weeks
* Fix two issues related to the use of SIGINFO in fetch(1) to displaydes2012-01-183-7/+75
| | | | | | | | | | | | | progress information. The first is that fetch_read() (used in the HTTP code but not the FTP code) can enter an infinite loop if it has previously been interrupted by a signal. The second is that when it is interrupted, fetch_read() will discard any data it may have read up to that point. Luckily, both bugs are extremely timing-sensitive and therefore difficult to trigger. PR: bin/153240 Submitted by: Mark <markjdb@gmail.com> MFC after: 3 weeks
* latin1 -> utf8des2011-10-198-9/+9
|
* Update copyright dates and strip my middle name.des2011-09-278-8/+8
|
* Bump date.des2011-09-271-1/+1
|
* Think first, commit second.des2011-09-272-8/+19
| | | | | | 1. Allow the caller to select active mode. 2. Fix the envar logic so it *always* overrides the caller's flags. 3. Document the change from active to passive.
* Long overdue: make passive mode the default for ftp.des2011-09-271-2/+2
|
* Mark all socket and file descriptors close-on-exec.des2011-05-132-0/+4
| | | | | PR: bin/151866 MFC after: 3 weeks
* Builds cleanly at the default WARNS level (WARNS=6).des2011-05-121-1/+0
|
* Increase WARNS to 4.des2011-05-123-3/+3
|
* Mechanical whitespace cleanup.des2011-05-121-100/+100
|
* Increase WARNS to 3.des2011-05-122-2/+2
|
* Fix a bug related to connection caching which could cause a crash afterdes2011-01-171-0/+1
| | | | | | | | a STAT command. PR: kern/153748 (different patch) Submitted by: Mark Johnston <markjdb@gmail.com> MFC after: 2 weeks
* Move variable declarations into the conditional block where they areemaste2010-10-241-1/+1
| | | | | | | used, to fix warning if WITH_SSL is not set. Submitted by: Sean Bruno MFC after: 1 week
* Redo fetch_read() using non-blocking sockets. This is necessary todes2010-07-281-20/+89
| | | | | | | | | avoid a hang in the SSL case if the server sends a close notification before we are done reading. In the non-SSL case, it can provide a minor (but probably not noticeable) performance improvement for small transfers. MFC after: 3 weeks
* Fix a couple of embarrassing mistakes in the previous commit.des2010-07-281-2/+2
| | | | Submitted by: Dimitry Andric <dimitry@andric.com>
* If the A flag is supplied, http_request() will attempt the request onlydes2010-07-011-2/+4
| | | | | | | | | | | once, even if authentication is required, instead of retrying with the proper credentials. Fix this by bumping the countdown if the origin or proxy server requests authentication so that the initial unauthenticated request does not count as an attempt. PR: 148087 Submitted by: Tom Evans <tevans.uk@googlemail.com> MFC after: 2 weeks
* Document HTTP digest authentication support.des2010-01-261-8/+14
| | | | | | Mention some prominent past contributors: Hajimu Umemoto (ipv6), Henry Whincup (https), Jukka Ukkonen (if-modified-since) and Jean-François Dockes (digest auth)
* Add missing DPADD and fix whitespace.des2010-01-191-2/+3
| | | | Submitted by: ru
* Add HTTP digest authentication.des2010-01-192-67/+819
| | | | | Submitted by: Jean-Francois Dockes <jf@dockes.org> Forgotten by: des (repeatedly)
* Fix a socket leak in ftp_request() after that a connection is established.attilio2009-11-251-3/+9
| | | | | | | Submitted by: Sandvine Incorporated Reviewed by: des, emaste Sponsored by: Sandvine Incorporated MFC: 1 week
* PR: 139751fabient2009-10-211-1/+1
| | | | | | Approved by: des Obtained from: Xavier Heiny <xavier.heiny@netasq.com> MFC after: 3 weeks
* Bump the version of all non-symbol-versioned shared libraries inkensmith2009-07-191-1/+1
| | | | | | | | preparation for 8.0-RELEASE. Add the previous version of those libraries to ObsoleteFiles.inc and bump __FreeBSD_Version. Reviewed by: kib Approved by: re (rwatson)
* 1. Update fetch to consistently return 1 on error, as the man page states,murray2008-12-171-8/+0
| | | | | | | | | | | | | | | | | | | | | | rather than usually returning 1 but in a few instances using a sysexits(3) return value. 2. Remove a few unused variables from libfetch. PR: docs/122470 (1, only) Reviewed by: des > Description of fields to fill in above: 76 columns --| > PR: If a GNATS PR is affected by the change. > Submitted by: If someone else sent in the change. > Reviewed by: If someone else reviewed your modification. > Approved by: If you needed approval for this commit. > Obtained from: If the change is from a third party. > MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email. > Security: Vulnerability reference (one per line) or description. > Empty fields above will be automatically removed. M usr.bin/fetch/fetch.c M lib/libfetch/fetch.c
* Add support for HTTP 1.1 If-Modified-Since behavior.murray2008-12-153-4/+50
| | | | | | | | | | | | | | fetch(1) accepts a new argument -i <file> that if specified will cause the file to be downloaded only if it is more recent than the mtime of <file>. libfetch(3) accepts the mtime in the url structure and a flag to indicate when this behavior is desired. PR: bin/87841 Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially) Reviewed by: des, ru MFC after: 3 weeks
OpenPOWER on IntegriCloud