8-2 公众号搜狗搜索结果文章信息(连内容) - 图1

抓取的结果信息包含:

  • 文章标题
  • 文章内容链接
  • 公众号名字
  • 时间
  • 文章简介·
  • 文章内容

    结果示例图:

    8-2 公众号搜狗搜索结果文章信息(连内容) - 图2

    模板:

    1. {"_id":"gongzhonghao-sogou","startUrl":["https://weixin.sogou.com/weixin?query=%E9%BB%84%E5%85%89%E8%A3%95&_sug_type_=&sut=1548&lkt=1%2C1593229578336%2C1593229578336&s_from=input&_sug_=y&type=2&sst0=1593229578439&page=[1-4]&ie=utf8&w=01019900&dr=1"],"selectors":[{"id":"info","type":"SelectorElement","parentSelectors":
    2. ["_root"],"selector":".news-list li","multiple":true,"delay":0},{"id":"title","type":"SelectorLink","parentSelectors":
    3. ["info"],"selector":"h3 a","multiple":false,"delay":0},{"id":"source","type":"SelectorText","parentSelectors":["info"],"selector":"a.account","multiple":false,"regex":"","delay":0},{"id":"time","type":"SelectorText","parentSelectors":["info"],"selector":"span","multiple":false,"regex":"","delay":0},{"id":"intro","type":"SelectorText","parentSelectors":["info"],"selector":"p","multiple":false,"regex":"","delay":0},{"id":"content","type":"SelectorText","parentSelectors":["title"],"selector":"div.rich_media_content","multiple":false,"regex":"","delay":0}]}

    模板套用步骤:

    (1)进入需要抓取的文章搜索结果页面,例如:https://weixin.sogou.com/weixin?query=%E5%A6%82%E4%BD%95%E5%86%99%E4%BD%9C&sug_type=&sfrom=input&_sug=n&type=2&page=1&ie=utf8
    (2)导入模板
    (3)替换 Start URL为要抓取的网页链接,(抓取多页需修改 Start URL 里的页码数)
    (4)开始抓取

注意:搜狗的反爬比较厉害,抓取多页建议一次只抓 4 页,输入一次验证码,然后再抓 4 页,以此类推