To get us started, you will need to start a new Python3 project with and install Scrapy (a web scraping and web crawling library for Python). I’m using pipenv for this tutorial, but you can use ...

Selenium. Selenium is a Python library that is generally used for automation purposes. ... Scrapy essentially allows you to do both of these in one library.