1. # Install
  2. $ pip3 install autoscraper

一分钟上手

  1. from autoscraper import AutoScraper
  2. url = 'https://stackoverflow.com/questions/2081586/web-scraping-with-python'
  3. # We can add one or multiple candidates here.
  4. # You can also put urls here to retrieve urls.
  5. wanted_list = ["How to call an external command?"]
  6. scraper = AutoScraper()
  7. result = scraper.build(url, wanted_list)
  8. print(result)
  • url 是你需要爬取的网页
  • wanted_list 是一个list,其中有你需要的元素目标,注意会爬取相同tag和范围的数据

image.png


更多使用技巧

请关注Github仓库: https://github.com/alirezamika/autoscraper