Commit Graph

72 Commits

Author SHA1 Message Date
Vincent Sanders 5a2f69388c hoist the Referer header generation logic up to the low level cache
This removes the need for the fetchers to have any interaction with
 the Referer header. It has not been completely removed from the
 fetch interface as fetch.c:fetch_set_cookie() still uses it for
 unverifiable cookie decision logic. (There is an anchient todo here)
2021-03-25 23:36:41 +00:00
Daniel Silverstone 97870122dd
fetch: When aborting, ensure we note this to not multi-fail
When aborting a fetch, it may be cleaned up, in which case we
would otherwise send a spurious failure message too early. Instead
we record that the abort has occurred to suppress such an error.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 10:24:23 +00:00
Daniel Silverstone 083ba385d7
fetch: Automatically handle fetches which fail to finish
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 09:47:52 +00:00
Vincent Sanders 0c34d06494 Keep the complete certificate chain from a fetch
Instead of extracting information from the X509 certificate chain in
the fetcher the entire chain is propagated in Distinguished Encoding
Rules (DER) format.

This allows all the information contained in a certificate chain to be
retained which can subsequently be presented to the user
2020-02-23 16:23:50 +00:00
Vincent Sanders 214478fc15 Improve some content documentation comments 2020-02-23 16:17:51 +00:00
Daniel Silverstone 1cf1ec55bc Support SSL verification through new about: handler
In doing this, also propagate why the certificates were bad
so that the page can display a reason.  We will need FatMessages
for all these.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2019-08-06 13:15:23 +01:00
Daniel Silverstone 8469f4cc8e Reimplement handling of BAD_AUTH inside browser_window
We now handle authentication requests via an `about:` page
which presents a nice form built into the browser window.
In order to do this, we add internal navigation as a concept
to the browser window and we strip the 401login support from all
frontends except monkey.

The 401login callback is now intended for password safe type support
rather than an immediately interactive prompt.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2019-08-06 11:26:47 +01:00
Daniel Silverstone bfb1bb1192 Migrate SSL certificate storage to the browser window
* Fetchers now provide the certificates before headers
* This is propagated all the way to the browser window
* When a query occurs, we retrieve it from there and fire
  the query with those stored certificates.
* The serial number is a bignum, store it as hex.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2019-08-05 18:12:47 +01:00
Vincent Sanders 5c96acd6f1 fix url encoding to be compatible with nsurl API changes.
As part of this fix the form submission error handling and
 reporting has been improved.
2018-09-26 17:21:48 +01:00
Vincent Sanders 1f52b2d514 remove unused fetch acessor 2017-01-31 22:56:45 +00:00
Daniel Silverstone c523bb47a0 Actually use utils/inet.h instead of sys/select.h 2016-06-27 21:58:09 +01:00
Daniel Silverstone be1ff548cb fetch.h needs sys/select.h for the fd_set type et al. 2016-06-27 21:50:54 +01:00
Daniel Silverstone 47ccd9855d Move fetcher_fdset to fetch.h (and rename to fetch_fdset). Maybe not ideal but better 2016-06-27 21:09:39 +01:00
Daniel Silverstone 4d1ef3bac4 Add support for retrying timed-out cURL fetches.
This is an attempt to amelioriate the situation found in #2384 where
we see the cURL connect() failing to complete.  Based on the pcap
from the bug log, we believe that RISC OS is likely failing to signal
the completion of the connection to cURL.  As such, cURL times out.

This change permits retries of timed out connections in the hope that
a fresh socket FD might subsequently function correctly.  The defaults
chosen mean that the previous behaviour of 30 seconds before timeout
is reported will remain the same, but in that time we will make 3 separate
attempts to connect the socket.
2015-11-10 21:51:54 +00:00
Vincent Sanders 73e45ff024 Fix error reporting from fetch_start
Any fetch start error was being reported as "out of memory" which was
clearly insufficient. Foe example bad urls (reported was file:// with
a missing /) were causing a warn_user with out of memory. This change
now at least causes a "bad url" message.
2015-06-24 10:31:13 +01:00
Vincent Sanders 4b2101ba6a clean up the fetcher factory and improve its API 2014-06-19 18:27:24 +01:00
Vincent Sanders 0475693394 create table for fetcher operations and move all operations into it 2014-01-23 00:00:18 +00:00
Vincent Sanders 4987a3a8a7 remove forward refs from content/fetch.c and cleanup doc comments 2014-01-19 21:18:47 +00:00
Daniel Silverstone 581d877576 In theory, store raw filenames and pass them through for file upload. Untested due to no file-upload in GTK frontend just yet 2014-01-04 19:34:18 +00:00
John-Mark Bell 3dff750ae2 Downgrade TLS version support if it turns out the server can't cope with TLSv1.1. 2013-01-04 23:13:23 +00:00
John Mark Bell 07309888d6 return;
Fix bug #3442642: allow scheme-specific fetchers to have a say in whether an URL can be fetched.

svn path=/trunk/netsurf/; revision=13182
2011-11-27 14:14:36 +00:00
John Mark Bell dcbafe6b87 Clean up fetch callback API. Inject some long-needed type safety.
svn path=/trunk/netsurf/; revision=13137
2011-11-08 21:51:42 +00:00
Michael Drake 6cfd37e60f Convert fetchers to nsurl.
svn path=/trunk/netsurf/; revision=12910
2011-09-29 15:31:54 +00:00
Michael Drake e3211bf4fc fetch_can_fetch can take const nsurl.
svn path=/trunk/netsurf/; revision=12902
2011-09-27 14:42:45 +00:00
Michael Drake e1f7a37f15 Port fetch layer to nsurl. Remove unused fetch_get_referer function.
svn path=/trunk/netsurf/; revision=12899
2011-09-27 11:07:32 +00:00
Michael Drake 9992ee9119 Document fetch_add_fetcher.
svn path=/trunk/netsurf/; revision=12892
2011-09-26 21:23:47 +00:00
Michael Drake c94271edf5 Fetchers register with an lwc_string, rather than a string.
svn path=/trunk/netsurf/; revision=12891
2011-09-26 21:07:19 +00:00
John Mark Bell b0f1b7b170 Improve const-correctness of fetch_start
svn path=/trunk/netsurf/; revision=10342
2010-04-10 17:13:53 +00:00
Daniel Silverstone 6c9e4bea8e Purge FETCH_TYPE which is unused throughout NetSurf. Clearing a todo in llcachec.
svn path=/trunk/netsurf/; revision=10300
2010-04-08 12:16:52 +00:00
Daniel Silverstone 2b1146bcb3 Remove parent from fetches, clearing a todo from llcache.c
svn path=/trunk/netsurf/; revision=10299
2010-04-08 11:47:05 +00:00
Daniel Silverstone 270ef59a98 Merge jmb/new-cache; r=dsilvers,rs=vince
svn path=/trunk/netsurf/; revision=10180
2010-03-28 12:56:39 +00:00
Vincent Sanders 6990c2cd04 remove uneccissary includes
svn path=/trunk/netsurf/; revision=9931
2010-01-29 11:32:53 +00:00
John Mark Bell 355799ce0b Merge branches/MarkieB/gtkmain to trunk.
svn path=/trunk/netsurf/; revision=9729
2009-12-17 23:55:02 +00:00
John Mark Bell 5feb7018c5 Replace parent_url with a pointer to the parent content.
svn path=/trunk/netsurf/; revision=8428
2009-07-10 00:26:37 +00:00
John Mark Bell b98f105b24 Purge WITH_AUTH
svn path=/trunk/netsurf/; revision=6600
2009-02-22 15:13:10 +00:00
John Mark Bell 847b776285 Purge NETSURF_USE_SSL and WITH_SSL
svn path=/trunk/netsurf/; revision=6599
2009-02-22 15:07:50 +00:00
John Mark Bell 79b4cee9d3 Make redirects behave as a NOP wrt the verifiability of fetches.
Apparently, the interweb depends on this.

svn path=/trunk/netsurf/; revision=6542
2009-02-16 20:24:54 +00:00
James Bursa 8282253a54 Move struct cache_data from fetch to content as it is no longer needed by fetch. Make it a member instead of pointer in struct content.
svn path=/trunk/netsurf/; revision=4246
2008-06-03 01:10:46 +00:00
James Bursa ae992eab4d Add new fetch callback FETCH_HEADER for headers and move as much header parsing as possible from fetch_curl.c to fetchcache.c. This simplifies fetch_curl.c and will make it possible to store response headers in future.
svn path=/trunk/netsurf/; revision=4226
2008-05-30 04:11:16 +00:00
John Mark Bell cc3481a2ff Make core fetching code handle verifiability of transactions
Make core fetching code be responsible for inserting cookies into the urldb
Provide accessor to a fetch's parent url (this is defined as being the URL of the verifiable fetch which caused this one to occur)
Make fetchcache's 3xx handling use the parent url when spawning new fetches

svn path=/trunk/netsurf/; revision=3809
2008-01-30 19:56:41 +00:00
Vincent Sanders 2f864eeecc Update all source code file headers to reflect GPL version 2 only and contain appropriate licence text
svn path=/trunk/netsurf/; revision=3486
2007-08-08 16:16:03 +00:00
James Bursa 594eb74c4c Fix deadlock when fetching stylesheets when max_fetchers_per_host is reached by splitting fetch_can_be_freed() into fetch_remove_from_queues() and fetch_free().
svn path=/trunk/netsurf/; revision=3380
2007-07-04 18:44:13 +00:00
John Mark Bell 87ca206e13 Warning fixes.
svn path=/trunk/netsurf/; revision=3332
2007-06-10 18:08:22 +00:00
Daniel Silverstone 1dd7e97eb3 Merge scheme switcher branch in.
svn path=/trunk/netsurf/; revision=3330
2007-06-10 17:46:44 +00:00
Daniel Silverstone 6807b4208a Remove the netsurf/ from the include paths and rationalise use of <> vs "" in includes
NetSurf includes are now done with ""s and other system includes with <>s as C intended.
The scandeps tool has been updated to only look for ""ed includes, and to verify that the
files exist in the tree before adding them to the dependency lines. The depend rule has
therefore been augmented to make sure the autogenerated files are built before it is run.

This is untested under self-hosted RISC OS builds. All else tested and works.


svn path=/trunk/netsurf/; revision=3307
2007-05-30 22:39:54 +00:00
John Mark Bell 25213dca24 Fix handling of cookies in unverifiable transactions caused by a redirect from a fetch into a browser window which was varifiable.
svn path=/trunk/netsurf/; revision=3165
2007-02-02 23:08:13 +00:00
James Bursa a0b6661eb6 Make GTK build compile on FreeBSD.
svn path=/trunk/netsurf/; revision=3154
2007-01-29 22:27:15 +00:00
John Mark Bell df4dbaf4cf Handle cookies in unverifiable transactions
svn path=/trunk/netsurf/; revision=3151
2007-01-27 20:58:20 +00:00
James Bursa e76140557d Add fetch_http_code() to retrieve the HTTP status code of a fetch.
svn path=/trunk/netsurf/; revision=3139
2007-01-13 00:19:02 +00:00
Michael Drake ba23e4b693 Update project URL.
svn path=/trunk/netsurf/; revision=3073
2006-11-27 15:35:18 +00:00