chromedriver

写一个基本的爬虫：

from selenium import webdriver
def main(url):
    driver = webdriver.Chrome()
    driver.get(url)
    html = driver.page_source
    driver.close()
    soup = BeautifulSoup(html)
    title = soup.title.string
    description = soup.find(attrs={"name": "description"})['content']
    # keywords = soup.find(attrs={"name": "keywords"})['content']
    print('title:', title)
    print('description:', description)
    # print('keywords:', keywords)

安装 chromedriver

On Ubuntu:

sudo apt install chromium-chromedriver

On Debian:

sudo apt install chromium-driver

On macOS install https://brew.sh/ then do

brew cask install chromedriver