# Youtube searches with HTTP.jl
**URL:** https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915
**Category:** New to Julia
**Created:** [July 28, 2019, 4:59pm UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915 "2019-07-28T16:59:05Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@mwolff](https://discourse.julialang.org/u/mwolff)
#### Post date: [July 28, 2019, 4:59pm UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/1 "2019-07-28T16:59:05Z")
Hello, folks,
i am currently experimenting with the HTTP lib in Julia and try to evaluate the youtube search suggestions.
It also works as far as I want to get more search suggestions and not just the first 20. What is the best way to do that?
At the moment i do something like this -
```julia
using HTTP
function getPositions(search::String, txt::String)
vpos=UnitRange{Int32}[]
i=1
while ((pos=findnext(search, txt, i)) != nothing)
push!(vpos, pos)
i=pos[end]+1
end
return vpos
end
function getPositions(search::String, txt::String, vec::Vector{UnitRange{Int32}})
vpos=UnitRange{Int32}[]
for v in vec
pos=findnext(search, txt, v[end]+1)
(pos != nothing) && push!(vpos, pos)
end
return vpos
end
function posText(vRange::Vector{UnitRange{Int32}}, txt::String)
vpos=UnitRange{Int32}[]
for range in vRange
if ((pos=findnext("\"", txt, range[end]+1)) != nothing)
r=range[end]+1:pos[end]-1
push!(vpos, r)
end
end
return vpos
end
r = HTTP.request("GET", "https://www.youtube.com/results?search_query=julia+programming")
println("Status : ", r.status)
txt = String(r.body)
open("youtube.txt", "w") do io
print(io, txt)
end
v=getPositions("div class=\"yt-lockup-content",txt)
vt=getPositions("title=\"", txt, v)
vt=posText(vt, txt)
vhref=getPositions("a href=\"", txt, v)
vhref=posText(vhref, txt)
for (n, pos) in enumerate(vhref)
playLink="https://www.youtube.com" * txt[pos]
linkText=txt[vt[n]]
println("($n) : $linkText")
println("($n) : $playLink")
end
```
this gives me all the html and scipts which i parse through to search for the titles and the video-links. I don’t know much about webdesign and javascript, but is there a link i overseen to the next search suggestions?
best regards
Michael
---
### Author:  [@Nosferican](https://discourse.julialang.org/u/Nosferican)
#### Post date: [July 28, 2019, 10:23pm UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/2 "2019-07-28T22:23:13Z")
YouTube uses JavaScript which means that you wouldn’t be able to see the actual document inspecting the response. You would need to use a WebDriver such as Selenium to actually interact with JavaScript webpages. Julia doesn’t have a WebDriver package at the moment (a few unmaintained projects). For YouTube in particular, you could use their API and that should be doable with HTTP.jl.
---
### Author:  [@mwolff](https://discourse.julialang.org/u/mwolff)
#### Post date: [July 28, 2019, 11:29pm UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/3 "2019-07-28T23:29:21Z")
Okay, thanks. I probably need access to the google youtube api. I would have to login there for a Developer-ID and so on.
best regards
Michael
---
### Author:  [@freemint](https://discourse.julialang.org/u/freemint)
#### Post date: [July 28, 2019, 11:48pm UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/4 "2019-07-28T23:48:59Z")
Depending on what you intend to do calling out to an existing utility like `youtube-dl`might do the trick.
There are more specialized command lines too.
---
### Author:  [@mwolff](https://discourse.julialang.org/u/mwolff)
#### Post date: [July 29, 2019, 9:55am UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/5 "2019-07-29T09:55:30Z")
yeah looks great. I need only something like give me a thumb-list of the newest videos from channels i like before i have to go directly to youtube.
---
### Author:  [@freemint](https://discourse.julialang.org/u/freemint)
#### Post date: [July 29, 2019, 10:12am UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/6 "2019-07-29T10:12:59Z")
[https://stackoverflow.com/questions/39606419/how-can-i-download-just-thumbnails-using-youtube-dl](https://stackoverflow.com/questions/39606419/how-can-i-download-just-thumbnails-using-youtube-dl)
`+`
[https://askubuntu.com/questions/643286/can-i-download-videos-from-a-youtube-search-query-using-youtube-dl](https://askubuntu.com/questions/643286/can-i-download-videos-from-a-youtube-search-query-using-youtube-dl)
`+`
???
`=`
Profit?
---
### Author:  [@mwolff](https://discourse.julialang.org/u/mwolff)
#### Post date: [July 29, 2019, 10:25am UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/7 "2019-07-29T10:25:55Z")
Nice examples. I’ll give it a try. It’s only for private using
---
### Author:  [@mwolff](https://discourse.julialang.org/u/mwolff)
#### Post date: [July 30, 2019, 7:50pm UTC](https://discourse.julialang.org/t/youtube-searches-with-http-jl/26915/8 "2019-07-30T19:50:16Z")
I have now tested youtube-dl, but by chance I saw in the askubuntu forum that you simply enter the page behind the search text and thus you can get more suggestions similar to scrolling down in youtube itself.
```julia
https://www.youtube.com/results?search_query=julia+programming&page=1
```
Then simply change the page to go further, that simulates the scrolling down to get more values in youtube
That goes also with my example solution from above.