Salastil/selenium-twitter-scraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

.

Browse Source
This commit is contained in:
Jarrian
2023-09-11 11:50:06 +08:00
parent 0fc1465e4b
commit 7575e20953
1 changed files with 5 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
scraper/twitter_scraper.py
View File

@@ -233,7 +233,7 @@ It may be due to the following:
sleep(3) sleep(3)
pass pass
def get_tweets(self): def get_tweet_cards(self):
self.tweet_cards = self.driver.find_elements( self.tweet_cards = self.driver.find_elements(
"xpath", '//article[@data-testid="tweet"]' "xpath", '//article[@data-testid="tweet"]'
) )
@@ -271,7 +271,7 @@ It may be due to the following:
while self.scroller.scrolling: while self.scroller.scrolling:
try: try:
self.get_tweets() self.get_tweet_cards()
added_tweets = 0 added_tweets = 0
for card in self.tweet_cards[-15:]: for card in self.tweet_cards[-15:]:
@@ -379,3 +379,6 @@ It may be due to the following:
print("CSV Saved: {}".format(file_path)) print("CSV Saved: {}".format(file_path))
pass pass
def get_tweets(self):
return self.data