Get a clearance for a host from a browser — clearance • safeconduct

Opens a Chrome on the host, lets any browser check run, and returns what the browser then holds: its user-agent string and its cookies, which clearance_handle() puts on a curl request. Whether the server then serves curl is the server's call – see fetch(), which falls back to downloading in the browser when it does not. A host that blocks plain clients without ever checking a browser yields a clearance all the same, with whatever cookies it set; curl will usually not be served with it, since such a block is about the client rather than a cookie. Clearances are cached per host for the R session.

Usage

clearance(
  url,
  refresh = FALSE,
  quiet = FALSE,
  wait = NULL,
  headed = "auto",
  useragent = TRUE
)

Arguments

url: The file's URL.
refresh: Get a new clearance even when a cached one is at hand.
quiet: Suppress progress messages.
wait: Seconds to wait for the check to pass. The default is two minutes when a window is shown, so that a person can click a box if one appears, and thirty seconds headless.
headed: How to run the browser, when one is needed: "auto", the default, tries headless Chrome first and opens a window only when the headless run does not get through, so that a window is shown only when someone may need to click; TRUE opens a window at once and FALSE never does. Without a display "auto" is headless. A host may run a second check at the file's URL, after the one at its root; headless Chrome has passed such a check with nothing clicked, but only after about two minutes, where a window shows a box that gets it through as soon as it is clicked. "auto" waits those minutes and opens a window if they pass without; TRUE skips the wait.
useragent: The user-agent string the browser announces. Headless Chrome's own calls it HeadlessChrome, and some hosts refuse it on that word alone. TRUE, the default, drops the word, so that the string says the Chrome version and platform it is and nothing it is not; NULL leaves Chrome's own string; a string is used as given. Set before the browser's first request, and what clearance() then reports.

Value

An object of class safeconduct_clearance: a list with host, useragent, cookie (one Cookie: header value) and time. Errors as fetch() does.

Examples

if (FALSE) { # \dontrun{
cl <- clearance("https://www12.statcan.gc.ca/")
curl::curl_fetch_memory("https://www12.statcan.gc.ca/", clearance_handle(cl))
} # }