summaryrefslogtreecommitdiffstats
path: root/lib/libfetch/http.c
Commit message (Collapse)AuthorAgeFilesLines
* MFH (r313974,r314596): open .netrc early in case we want to drop privsdes2017-03-241-14/+5
| | | | | | | | MFH (r314396,r315143): fix a crash caused by an incorrect format string MFH (r314701): fix handling of 416 errors when requesting a range MFH (r315455): fix parsing of IP literals (square brackets) PR: 212065, 217723
* MFH (r310823): fix multi-line CONNECT responsesdes2017-01-091-1/+1
| | | | PR: 194483
* r169386 (PR 112515) was incomplete: it treated 307 as an error exceptdes2016-05-311-0/+3
| | | | | | | | | | | | | in verbose mode, and did not handle 308 at all. r241840 (PR 172451) added support for 308, but with the same bug. Correctly handle both by recognizing them as redirects in all places where we check the HTTP result code. PR: 112515 173451 209546 Submitted by: novel@ MFC after: 1 week
* Use strlcpy() instead of strncpy() to copy the string returned bytruckman2016-05-121-1/+1
| | | | | | | | | | | | setlocale() so that static analyzers know that the string is NUL terminated. This was causing a false positive in Coverity even though the longest string returned by setlocale() is ENCODING_LEN (31) and we are copying into a 64 byte buffer. This change is also a bit of an optimization since we don't need the strncpy() feature of padding the rest of the destination buffer with NUL characters. Reported by: Coverity CID: 974654
* Fix double-free error: r289419 moved all error handling in http_connect()des2016-02-111-1/+0
| | | | | | | | | to the end of the function, but did not remove a fetch_close() call which was made redundant by the one in the shared error-handling code. PR: 206774 Submitted by: Christian Heckendorf <heckendorfc@gmail.com> MFC after: 3 days
* As a followup to r292330, standardize on size_t and add a few comments.des2015-12-161-2/+5
|
* Reset bufpos to 0 immediately after refilling the buffer. Otherwise, wedes2015-12-161-3/+2
| | | | | | | | | | risk leaving the connection in an indeterminate state if the server fails to send a chunk delimiter. Depending on the application and on the sizes of the preceding chunks, the result can be anything from missing data to a segfault. With this patch, it will be reported as a protocol error. PR: 204771 MFC after: 1 week
* Fix buildworld after r291453, similar to r284346: url->user and url->pwddim2015-11-291-4/+2
| | | | | | | are arrays, so they can never be NULL. Reported by: many Pointy hat to: des
* Use .netrc for HTTP sites and proxies, not just FTP.des2015-11-291-0/+8
| | | | | | PR: 193740 Submitted by: TEUBEL György <tgyurci@gmail.com> MFC after: 1 week
* Fix two bugs in HTTPS tunnelling:des2015-10-161-4/+32
| | | | | | | | | | - If the proxy returns a non-200 result, set the error code accordingly so the caller / user gets a somewhat meaningful error message. - Consume and discard any HTTP response header following the result line. PR: 194483 Tested by: Fabian Keil <fk@fabiankeil.de> MFC after: 1 week
* Remove unused variable to silence clang warning.araujo2015-07-041-4/+1
| | | | | Differential Revision: D2683 Reviewed by: rodrigc, bapt
* Fix the following clang 3.7.0 warnings in lib/libfetch/http.c:dim2015-06-131-12/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lib/libfetch/http.c:1628:26: error: address of array 'purl->user' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.user = purl->user ? ~~~~~~^~~~ ~ lib/libfetch/http.c:1630:30: error: address of array 'purl->pwd' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.password = purl->pwd? ~~~~~~^~~~ lib/libfetch/http.c:1657:25: error: address of array 'url->user' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.user = url->user ? ~~~~~^~~~ ~ lib/libfetch/http.c:1659:29: error: address of array 'url->pwd' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.password = url->pwd ? ~~~~~^~~ ~ lib/libfetch/http.c:1669:25: error: address of array 'url->user' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.user = url->user ? ~~~~~^~~~ ~ lib/libfetch/http.c:1671:29: error: address of array 'url->pwd' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] aparams.password = url->pwd ? ~~~~~^~~ ~ Since url->user and url->pwd are arrays, they can never be NULL, so the checks can be removed. Reviewed by: bapt MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D2673
* Add support for arbitrary http requestsbapt2014-06-051-2/+32
| | | | | | | Submitted by: Alex Hornung <alex@alexhornung.com> Reviewed by: des Obtained from: Dragonfly MFC after: 3 week
* Remove unnecessary semicolonsbapt2014-06-051-4/+4
| | | | | | | | Patch by Sascha Wildner <saw@online.de> for Dragonfly Reviewed by: des Obtained from: Dragonfly MFC after: 1 week
* Use NULL instead of 0bapt2014-06-051-2/+2
| | | | | | | | Patch by Sascha Wildner <saw@online.de> for Dragonfly Reviewed by: des Obtained from: Dragonfly MFC after: 1 week
* If HTTP_USER_AGENT is defined but empty, don't send User-Agent at all.des2014-06-051-4/+9
| | | | | | PR: 184507 Submitted by: jbeich@tormail.org (with modifications) MFC after: 1 week
* Support Last-Modified behind proxies which return UTC instead of GMT.bdrewery2014-03-111-0/+6
| | | | | | | | | | | | | | | | The standard states that GMT must be used, but that UTC is equivalent. Still parse UTC as otherwise this causes problems for pkg(8). It will refetch the repository every time 'pkg update' or other remote operations are used behind these proxies. RFC2616: "All HTTP date/time stamps MUST be represented in Greenwich Mean Time (GMT), without exception. For the purposes of HTTP, GMT is exactly equal to UTC (Coordinated Universal Time)."" Approved by: bapt (mentor) Reviewed by: des, peter Sponsored by: EMC / Isilon Storage Division MFC after: 1 week
* Bump copyright datesdes2014-01-301-1/+1
|
* r261230 broke the cases where the amount of data to be read is notdes2014-01-291-4/+6
| | | | | | | | | | | | | | | | | | | | | | | known in advance, or where the caller doesn't care and just keeps reading until it hits EOF. In fetch_read(): the socket is non-blocking, so read() will return 0 on EOF, and -1 (errno == EAGAIN) when the connection is still open but there is no data waiting. In the first case, we should immediately return 0. The EINTR case was also broken, although not in a way that matters. In fetch_writev(): use timersub() and timercmp() as in fetch_read(). In http_fillbuf(): set errno to a sensible value when an invalid chunk header is encountered. In http_readfn(): as in fetch_read(), a zero return from down the stack indicates EOF, not an error. Furthermore, when io->error is EINTR, clear it (but no errno) before returning so the caller can retry after dealing with the interrupt. MFC after: 3 days
* Solve http buffering issues and hangs once and for all (hopefully!) bydes2014-01-281-21/+17
| | | | | | | | | | | | | | | | | simply not trying to return exactly what the caller asked for - just return whatever we got and let the caller be the judge of whether it was enough. If an error occurs or the connection times out after we already received some data, return a short read, under the assumption that the next call will fail or time out before we read anything. As it turns out, none of the code that calls fetch_read() assumes an all-or-nothing result anyway, except for a couple of lines where we read the CR LF at the end of a hunk in HTTP hunked encoding, so the changes outside of fetch_read() and http_readfn() are minimal. While there, replace select(2) with poll(2). MFC after: 3 days
* Even though it doesn't really make sense in the context of a CONNECTdes2013-08-221-0/+2
| | | | | | | | | request, RFC 2616 14.23 mandates the presence of the Host: header in all HTTP 1.1 requests. PR: kern/181445 Submitted by: Kimo <kimor79@yahoo.com> MFC after: 3 days
* Include an Accept header in requests.des2013-07-301-1/+7
| | | | | PR: kern/180917 MFC after: 1 week
* Implement certificate verification, and many other SSL-relateddes2013-07-261-1/+1
| | | | | | | | imrovements; complete details in the PR. PR: kern/175514 Submitted by: Michael Gmelin <freebsd@grem.de> MFC after: 1 week
* Use the correct request syntax for proxied (tunneled) HTTPS requests.des2013-07-211-1/+1
| | | | | PR: bin/180666 MFC after: 3 days
* Use the CONNECT method to proxy HTTPS connections through HTTP proxies.des2013-04-121-8/+13
| | | | | PR: bin/80176 Submitted by: Yuichiro NAITO <naito.yuichiro@gmail.com>
* Fix weird indentation.des2012-11-161-4/+4
|
* Implement HTTP 305 redirect handling.eadler2012-10-221-1/+5
| | | | | | | | PR: 172452 Submitted by: gcooper Reviewed by: des Approved by: cperciva MFC after: 1 week
* Don't deny non-temporary redirects if the -A option is set (pereadler2012-10-221-2/+12
| | | | | | | | | | | | the man page) [0] While here add support for draft-reschke-http-status-308-07 PR: 172451 [0] Submitted by: gcooper [0] Reviewed by: des Approved by: cperciva MFC after: 1 week
* Be a bit more lenient in the maximum number of redirects allowed.eadler2012-10-221-1/+1
| | | | | | | | Chrome and Firefox have a limit of 20. IE has a limit of 8. Reviewed by: des Approved by: cperciva MFC after: 3 days
* Use libmd if and only if OpenSSL is not available.des2012-09-141-0/+8
| | | | | PR: bin/171402 MFC after: 3 days
* Don't reuse credentials if redirected to a different host.des2012-04-301-1/+3
| | | | | Submitted by: Niels Heinen <heinenn@google.com> MFC after: 3 weeks
* Fix two issues related to the use of SIGINFO in fetch(1) to displaydes2012-01-181-5/+12
| | | | | | | | | | | | | progress information. The first is that fetch_read() (used in the HTTP code but not the FTP code) can enter an infinite loop if it has previously been interrupted by a signal. The second is that when it is interrupted, fetch_read() will discard any data it may have read up to that point. Luckily, both bugs are extremely timing-sensitive and therefore difficult to trigger. PR: bin/153240 Submitted by: Mark <markjdb@gmail.com> MFC after: 3 weeks
* latin1 -> utf8des2011-10-191-1/+1
|
* Update copyright dates and strip my middle name.des2011-09-271-1/+1
|
* Increase WARNS to 4.des2011-05-121-1/+1
|
* Mechanical whitespace cleanup.des2011-05-121-100/+100
|
* Increase WARNS to 3.des2011-05-121-1/+1
|
* Fix a couple of embarrassing mistakes in the previous commit.des2010-07-281-2/+2
| | | | Submitted by: Dimitry Andric <dimitry@andric.com>
* If the A flag is supplied, http_request() will attempt the request onlydes2010-07-011-2/+4
| | | | | | | | | | | once, even if authentication is required, instead of retrying with the proper credentials. Fix this by bumping the countdown if the origin or proxy server requests authentication so that the initial unauthenticated request does not count as an attempt. PR: 148087 Submitted by: Tom Evans <tevans.uk@googlemail.com> MFC after: 2 weeks
* Add HTTP digest authentication.des2010-01-191-66/+816
| | | | | Submitted by: Jean-Francois Dockes <jf@dockes.org> Forgotten by: des (repeatedly)
* Add support for HTTP 1.1 If-Modified-Since behavior.murray2008-12-151-3/+25
| | | | | | | | | | | | | | fetch(1) accepts a new argument -i <file> that if specified will cause the file to be downloaded only if it is more recent than the mtime of <file>. libfetch(3) accepts the mtime in the url structure and a flag to indicate when this behavior is desired. PR: bin/87841 Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially) Reviewed by: des, ru MFC after: 3 weeks
* Don't fail mistakenly with -r when we already have the whole file.ru2008-10-241-0/+1
| | | | Reviewed by: des
* Use memcpy(3) instead of the BSD-specific bcopy(3).des2008-02-081-1/+1
| | | | | Submitted by: Joerg Sonnenberger <joerg@britannica.bec.de> MFC after: 2 weeks
* Add necessary cast for tolower() argument.des2008-02-061-2/+3
| | | | | Submitted by: Joerg Sonnenberger <joerg@britannica.bec.de> MFC after: 1 week
* As several people pointed out, I did all the ctype casts the wrongdes2007-12-191-13/+15
| | | | | | | way (not for the first time...) Noticed by: bde, ru ++ MFC after: 1 week
* Add support for the NO_PROXY / no_proxy environment variable as used bydes2007-12-181-3/+5
| | | | | | | | | lynx, curl etc. Note that this patch differs significantly from that in the PR, as the submitter refined it after submitting the PR. PR: 110388 Submitted by: Alexander Pohoyda <alexander.pohoyda@gmx.net> MFC after: 3 weeks
* Old patch I had lying around: correctly cast the argument to is*().des2007-12-181-12/+13
| | | | | | IWBNI gcc could warn about this the way it warns about printf() abuse. MFC after: 1 week
* Clean up namespace violations.des2007-12-141-97/+97
| | | | MFC after: 1 week
* Handle temporary redirects (HTTP status code 307)des2007-05-081-0/+2
| | | | | | PR: kern/112515 Submitted by: Ryan C. Gordon <icculus@icculus.org> MFC after: 3 weeks
* Ignore HTTP_PROXY if it is defined but empty. This was already handleddes2005-08-241-1/+1
| | | | | | | | | | correctly in the case of FTP_PROXY, because an empty FTP_PROXY has a specific meaning ("don't use any proxy at all for ftp, even if HTTP_PROXY is defined"), while an empty HTTP_PROXY has no meaning at all. PR: bin/85185 Submitted by: Conall O'Brien <conallob=freebsd@maths.tcd.ie> MFC after: 2 weeks
OpenPOWER on IntegriCloud