Hi there,
I have looked around on discourse and can’t seem to find a suggested approach that offers best practices for the following:
1 go to a web site
2 login using user and password
3 navigate to a page
4 download a csv file
I have found many ideas as to HOW to do it, but I think it would be nice to have a CURRENT best practices for something that might be a common requirement. If I missed something that exists I am sorry.
From Home · CSV.jl.
There’s not that much to it:
f = CSV.File(HTTP.get(url).body)
If you need authentication, you can pass headers to HTTP.
https://juliaweb.github.io/HTTP.jl/stable/public_interface/
However, if by “go to a website. navigate to page”, you mean building a browser, then that’s a rather bigger job.
I found both those references but that’s not what my observation/question was. I was asking/postulating the need for a SINGLE best practices example that allows someone to go to a website ( by providing an url NOT writing a browser), send a user and password, and download a csv from that page. Thanks for the confirmation on the links though, I had looked at them but thought an example might be a better approach especially if it linked to the current documentation where the approach was outlined. My question/observation stands.
maybe, BUT let’s say that the dataset is one that is proprietary information behind a paywall and the website owners aren’t too keen on having it on a 3rd party hosting site. How about someone in the maker community trying to run the site from their home to cut costs?