Youtube comments scrape r

devtools::install_github("ropensci/RSelenium") # Install from github

library(RSelenium)
library(rvest)
pJS <- phantom(pjs_cmd = "PATH TO phantomjs.exe") # as i am using windows
Sys.sleep(5) # give the binary a moment
remDr <- remoteDriver(browserName = 'phantomjs')
remDr$open()
remDr$navigate("https://www.youtube.com/watch?v=qRC4Vk6kisY")
remDr$getTitle()[[1]] # [1] "YouTube"

# scroll down
for(i in 1:5){      
  remDr$executeScript(paste("scroll(0,",i*10000,");"))
  Sys.sleep(3)    
}

# Get page source and parse it via rvest
page_source <- remDr$getPageSource()
author <- html(page_source[[1]]) %>% html_nodes(".user-name") %>% html_text()
text <- html(page_source[[1]]) %>% html_nodes(".comment-text-content") %>% html_text()

#combine the data in a data.frame
dat <- data.frame(author = author, text = text)

Result:
> head(dat)
              author                                                                                       text
1 Kikyo bunny simpie Omg I love fluffy puff she's so adorable when she was dancing on a rainbow it's so cute!!!
2   Tatjana Celinska                                                                                     Ciao 0
3      Yvette Austin                                                                   GET OUT OF MYÂ  HEAD!!!!
4           Susan II                                                                             Watch narhwals
5        Greg Ginger               who in the entire fandom never watched this, should be ashamed,\n\nPFFFTT!!!
6        Arnav Sinha                                                                 LOL what the hell is this?

Source: stackoverflow.com

Similar codes

6 votes164 views3 answers
Google sheets if
TypeScript
0 vote183 views1 answer
'`pydot` failed to call GraphViz.' OSError: `pydot` failed to call GraphViz.Please install GraphViz (https://www.graphviz.org/) and ensure that its executables are in the $PATH.
TypeScript
0 vote153 views1 answer
Copy elements from one array to another java
TypeScript
0 vote203 views1 answer
How to return a new string with its first and last characters swapped
TypeScript
0 vote175 views1 answer
// running tests Your code should no longer have a p tag around the text asking what level ninja a user is. // tests completed category:423
TypeScript
2 votes153 views1 answer
Media breakpoints bootstrap 4
TypeScript
1 vote157 views1 answer
How to pass children in react typescript
TypeScript
0 vote152 views1 answer
Which of the foolowing ia an element of pallette that holds multiple elements of nspecific purpose
TypeScript
0 vote169 views2 answers
Subplots legend
TypeScript
0 vote194 views1 answer
Angular 8 TS2307: Cannot find module '@angular/cdk/clipboard'.
TypeScript