# Scraping youtube webcomments with WebDriver.jl

**URL:** https://discourse.julialang.org/t/scraping-youtube-webcomments-with-webdriver-jl/75996
**Category:** General Usage
**Created:** [February 8, 2022, 6:08am UTC](https://discourse.julialang.org/t/scraping-youtube-webcomments-with-webdriver-jl/75996 "2022-02-08T06:08:28Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [February 8, 2022, 6:08am UTC](https://discourse.julialang.org/t/scraping-youtube-webcomments-with-webdriver-jl/75996/1 "2022-02-08T06:08:28Z")

</div>

Hi All,

I am trying to replicate this post [How to Scrape Youtube Comments with Python | by François St-Amant | Towards Data Science](https://towardsdatascience.com/how-to-scrape-youtube-comments-with-python-61ff197115d4) in Julia. The author scrapes comments from yt videos using selenium driver and does some sentiment analysis on them.

The author does something like

```python
import time
from selenium.webdriver import Chrome
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

data=[]
with Chrome(executable_path=r'C:\Program Files\chromedriver.exe') as driver:
    wait = WebDriverWait(driver,15)
    driver.get("https://www.youtube.com/watch?v=kuhhT_cBtFU&t=2s")

    for item in range(200): 
        wait.until(EC.visibility_of_element_located((By.TAG_NAME, "body"))).send_keys(Keys.END)
        time.sleep(15)

    for comment in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "#content"))):
        data.append(comment.text)

```

I want to rewrite it with `WebDriver.jl`, but I have not found an equivalent of `WebDriverWait` and `expected_conditions` in the library. Does anyone tried something like this or has some experience with it?

Thanks for an answer.  
Tomas

---

<div class="post-metadata">

### Author: ![lawless-m](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lawless-m/32/30869_2.png) [@lawless-m](https://discourse.julialang.org/u/lawless-m)
#### Post date: [February 8, 2022, 12:19pm UTC](https://discourse.julialang.org/t/scraping-youtube-webcomments-with-webdriver-jl/75996/2 "2022-02-08T12:19:04Z")

</div>

I’m not familiar with the Module but there is `implicitly_wait` which seems to do something similar

> Sets a sticky timeout to implicitly wait for an element to be found,  
> or a command to complete.

and as this is PyCall you can always call the Python object directly without being forced to use the wrapper. Sadly I leave that as an exrcise 🙂

---

<div class="post-metadata">

### Author: ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [February 8, 2022, 1:25pm UTC](https://discourse.julialang.org/t/scraping-youtube-webcomments-with-webdriver-jl/75996/3 "2022-02-08T13:25:02Z")

</div>

Thanks, every little helps.