安装

  1. pip install -U spaCy
  2. python -m spacy download en

特性

spaCy的一大特点是在进行tokenize的时候会保留下标，它在进行标记时还会保留原字符串中的空格（分割单词的单个空格不算）。这与NLTK是有很大不同的，每个单词可以通过idx下标轻松获取到对应原字符串中的位置。

  1. nlp = spacy.load("en_core_web_sm")
  2. doc = nlp('Hello  World!')
  3. for token in doc:
  4.     print('"' + token.text + '"', token.idx)
  7. 结果：
  8. "Hello" 0
  9. " " 6
  10. "World" 7
  11. "!" 12

句子检测

  1. nlp = spacy.load("en_core_web_sm")
  2. doc = nlp("These are apples. These are oranges.")
  3. for sent in doc.sents:
  4.     print(sent)
  7. 结果：
  8. These are apples.
  9. These are oranges.

词性标注（POS Tagging）

  1. nlp = spacy.load("en_core_web_sm")
  2. doc = nlp("Next week I'll be in Madrid.")
  3. print([(token.text, token.tag_) for token in doc])
  6. 结果：
  7. [('Next', 'JJ'), ('week', 'NN'), ('I', 'PRP'), ("'ll", 'MD'), ('be', 'VB'), ('in', 'IN'), ('Madrid', 'NNP'), ('.', '.')]

命名实体识别

  1. nlp = spacy.load("en_core_web_sm")
  2. doc = nlp("Next week I'll be in Madrid.")
  3. for ent in doc.ents:
  4.     print(ent.text, ent.label_)
  7. 结果：
  8. Next week DATE
  9. Madrid GPE

使用notebook可以对结果进行可视化：

  1. import spacy
  2. from spacy import displacy
  4. nlp = spacy.load("en_core_web_sm")
  5. doc = nlp("Next week I'll be in Madrid.")
  6. displacy.render(doc, style='ent', jupyter=True)
  7. for ent in doc.ents:
  8.     print(ent.text, ent.label_)

image.png

分词（词组）

  1. nlp = spacy.load("en_core_web_sm")
  2. doc = nlp("Wall Street Journal just published an interesting piece on crypto currencies")
  3. for chunk in doc.noun_chunks:
  4.     print(chunk.text, chunk.label_, chunk.root.text)
  6. # Wall Street Journal NP Journal
  7. # an interesting piece NP piece
  8. # crypto currencies NP currencies

依存分析

  1. nlp = spacy.load("en_core_web_sm")
  2. doc = nlp('Wall Street Journal just published an interesting piece on crypto currencies')
  4. for token in doc:
  5.     print("{0}/{1} <--{2}-- {3}/{4}".format(
  6.         token.text, token.tag_, token.dep_, token.head.text, token.head.tag_))
  8. # Wall/NNP <--compound-- Street/NNP
  9. # Street/NNP <--compound-- Journal/NNP
  10. # Journal/NNP <--nsubj-- published/VBD
  11. # just/RB <--advmod-- published/VBD
  12. # published/VBD <--ROOT-- published/VBD
  13. # an/DT <--det-- piece/NN
  14. # interesting/JJ <--amod-- piece/NN
  15. # piece/NN <--dobj-- published/VBD
  16. # on/IN <--prep-- piece/NN
  17. # crypto/JJ <--compound-- currencies/NNS
  18. # currencies/NNS <--pobj-- on/IN

同样，使用notebook可以进行可视化

  1. from spacy import displacy
  3. nlp = spacy.load("en_core_web_sm")
  4. doc = nlp('Wall Street Journal just published an interesting piece on crypto currencies')
  5. displacy.render(doc, style='dep', jupyter=True, options={'distance': 90})

spaCy - 图2