本节参考 官方文档 检索示例:
导入样本测试数据
https://gitee.com/xlh_blog/common_content/blob/master/es%E6%B5%8B%E8%AF%95%E6%95%B0%E6%8D%AE.jsonPOST /bank/account/_bulk
测试数据样本模板
{"account_number": 1,"balance": 39225,"firstname": "Amber","lastname": "Duke","age": 32,"gender": "M","address": "880 Holmes Lane","employer": "Pyrami","email": "amberduke@pyrami.com","city": "Brogan","state": "IL"}
检索示例介绍
请求接口
GET /bank/_search{"query": {"match_all": {}},"sort": [{"account_number": "asc"}]}// query: 查询条件// sort: 排序条件
结果
{"took" : 537, // 检索毫秒数"timed_out" : false, // 是否超时"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1000, // 查询条数"relation" : "eq"},"max_score" : null,"hits" : [ // hits: 查询到的所有数据{"_index" : "bank","_type" : "account","_id" : "0","_score" : null,"_source" : {"account_number" : 0,"balance" : 16623,"firstname" : "Bradshaw","lastname" : "Mckenzie","age" : 29,"gender" : "F","address" : "244 Columbus Place","employer" : "Euron","email" : "bradshawmckenzie@euron.com","city" : "Hobucken","state" : "CO"},"sort" : [0]},...]}}
响应字段解释
took– 使用Elasticsearch运行查询所用的时间(毫秒)timed_out– 搜索请求是否超时_shards– 搜索了多少个碎片,以及有多少个碎片成功、失败或被跳过。max_score– 最相关文件的得分hits.total.value- 找到了多少匹配的文档hits.sort- 文档的排序位置(当不按相关性得分排序时)hits._score- 文档的相关性评分(在使用Match_All时不适用)响应结果说明
Elasticsearch 默认会分页返回10条数据,不会一下返回所有数据。
请求方式说明
ES支持两种基本方式检索;
通过REST request uri 发送搜索参数 (uri +检索参数);
- 通过REST request body 来发送它们(uri+请求体);
也就是说除了上面示例的请求接口,根据请求体进行检索外;
还可以用GET请求参数的方式检索:
GET bank/_search?q=*&sort=account_number:asc// q=* (查询所有)// sort=account_number:asc (按照account_number进行升序排列)
Query DSL
本小节参考官方文档:Query DSL
Elasticsearch提供了一个可以执行查询的Json风格的DSL。这个被称为Query DSL,该查询语言非常全面。
基本语法格式
一个查询语句的典型结构:
QUERY_NAME:{ARGUMENT:VALUE,ARGUMENT:VALUE,...}
如果针对于某个字段,那么它的结构如下:
{QUERY_NAME:{FIELD_NAME:{ARGUMENT:VALUE,ARGUMENT:VALUE,...}}}
请求示例:
GET /bank/_search{"query": {"match_all": {}},"from": 0,"size": 5,"sort": [{"account_number": {"order": "desc"},"balance": {"order": "asc"}}]}# match_all: 查询类型 [代表查询所有的索引], es中可以在query中组合非常多的查询类型完成复杂查询;# from+size: 限定, 完成分页功能; 从第几条数据开始, 每页有多少数据# sort: 排序, 多字段排序, 会在前序字段相等时后续字段内部排序, 否则以前序为准;
返回部分字段
请求示例:
GET /bank/_search{"query": {"match_all": {}},"from": 0,"size": 5,"sort": [{"account_number": {"order": "desc"}}],"_source": ["balance","firstname"]}# _source: 指定返回结果中包含的字段名
结果示例:
{"took" : 259,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1000,"relation" : "eq"},"max_score" : null,"hits" : [{"_index" : "bank","_type" : "account","_id" : "999","_score" : null,"_source" : {"firstname" : "Dorothy","balance" : 6087},"sort" : [999]}]}}
match-匹配查询
精确查询-基本数据类型(非文本)
GET /bank/_search{"query": {"match": {"account_number": 20}}}# 查找匹配 account_number 为 20 的数据 非文本推荐使用 term
模糊查询-文本字符串
GET /bank/_search{"query": {"match": {"address": "mill lane"}}}# 查找匹配 address 包含 mill 或 lane 的数据
match 即全文检索,对检索字段进行分词匹配,会按照响应的评分 _score 排序,原理是倒排索引。
精确匹配-文本字符串
GET /bank/_search{"query": {"match": {"address.keyword": "288 Mill Street"}}}# 查找 address 为 288 Mill Street 的数据。# 这里的查找是精确查找,只有完全匹配时才会查找出存在的记录,# 如果想模糊查询应该使用match_phrase 短语匹配
match_phrase-短语匹配
将需要匹配的值当成一整个单词(不分词)进行检索
GET /bank/_search{"query": {"match_phrase": {"address": "mill lane"}}}# 这里会检索 address 匹配包含短语 mill lane 的数据
multi_math-多字段匹配
GET /bank/_search{"query": {"multi_match": {"query": "mill","fields": ["city","address"]}}}# 检索 city 或 address 字段匹配包含 mill 的数据, 会对查询条件分词# 类似数据库的 (where city like '%mill%' and address like %mill%)
bool-复合查询
复合语句可以合并,任何其他查询语句,包括符合语句。这也就意味着,复合语句之间可以互相嵌套,可以表达非常复杂的逻辑。
must:必须达到must所列举的所有条件;must中的条件使用AND拼接must_not:必须不匹配must_not所列举的所有条件;对must进行取反should:应该满足should所列举的条件;should中的条件使用OR拼接GET /bank/_search{"query": {"bool": {"must": [{"match": {"gender": "M"}},{"match": {"address": "mill"}}]}}}# 查询 gender 包含 M 且 address 包含 mill 的数据# where gender like '%M%' and address like '%mill%'
filter-结果过滤
并不是所有的查询都需要产生分数,特别是哪些仅用于 filtering 过滤的文档。为了不计算分数,elasticsearch 会自动检查场景并且优化查询的执行。filter 对结果进行过滤,且不计算相关性得分。
GET bank/_search{"query": {"bool": {"must": [{"match": {"address": "mill"}}],"filter": {"range": {"age": {"gte": "18","lte": "30"}}}}}}# 这里先是查询所有匹配 address 包含 mill 的文档,# 然后再根据 age >= 18 and age <= 30 进行过滤查询结果# 过滤不会产生相关性得分
term-精确检索
Avoid using the
termquery for[text](https://www.elastic.co/guide/en/elasticsearch/reference/7.11/text.html)fields. 避免使用 term 查询文本字段 By default, Elasticsearch changes the values oftextfields as part of analysis. This can make finding exact matches fortextfield values difficult. 默认情况下,Elasticsearch 会通过analysis分词将文本字段的值拆分为一部分,这使精确匹配文本字段的值变得困难。 To searchtextfield values, use the[match](https://www.elastic.co/guide/en/elasticsearch/reference/7.11/query-dsl-match-query.html)query instead. 如果要查询文本字段值,请使用 match 查询代替。https://www.elastic.co/guide/en/elasticsearch/reference/7.11/query-dsl-term-query.html
在上面 3.match-匹配查询 中有介绍对于非文本字段的精确查询,Elasticsearch 官方对于这种非文本字段,使用 term 来精确检索是一个推荐的选择
GET bank/_search{"query": {"term": {"age": "28"}}}# 查找 age 为 28 的数据
Aggregation-执行聚合
https://www.elastic.co/guide/en/elasticsearch/reference/7.11/search-aggregations.htmlterms:统计数量avg:平均数max:最大值min:最小值
聚合语法
GET /my-index-000001/_search{"aggs":{"aggs_name":{ # 这次聚合的名字,方便展示在结果集中"AGG_TYPE":{ # 聚合的类型(avg,terms)平均数、统计、最大值...}}}}
示例1-搜索 address 中包含 Mill 的所有人的年龄分布以及平均工资
GET /bank/_search{"query": {"bool": {"must": [{"match": {"address": "Mill"}}]}},"aggs": { // aggs: 聚合条件对象"ageAgg":{ // age_aggs: 自定义聚合结果对象名"terms": {"field": "age", // field: 指定聚合属性"size": 10 // size: 最大显示条数}},"balanceAgg": {"avg": {"field": "balance"}}},"size": 0 // 显示命中结果的条数}
结果
{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 4,"relation" : "eq"},"max_score" : null,"hits" : [ ]},"aggregations" : { // 聚合查询结果"ageAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : 38,"doc_count" : 2},{"key" : 28,"doc_count" : 1},{"key" : 32,"doc_count" : 1}]},"balanceAgg" : {"value" : 25208.0}}}
示例2-按照年龄聚合,并且求这些年龄段的这些人的平均薪资
GET /bank/_search{"query": {"match_all": {}},"aggs": {"ageAgg": {"terms": {"field": "age","size": 100},"aggs": {"ageAvg": {"avg": {"field": "balance"}}}}},"size": 0}
结果
{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1000,"relation" : "eq"},"max_score" : null,"hits" : [ ]},"aggregations" : {"ageAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : 31,"doc_count" : 61,"ageAvg" : {"value" : 28312.918032786885}},{"key" : 39,"doc_count" : 60,"ageAvg" : {"value" : 25269.583333333332}},{"key" : 26,"doc_count" : 59,"ageAvg" : {"value" : 23194.813559322032}},{"key" : 32,"doc_count" : 52,"ageAvg" : {"value" : 23951.346153846152}},{"key" : 35,"doc_count" : 52,"ageAvg" : {"value" : 22136.69230769231}},{"key" : 36,"doc_count" : 52,"ageAvg" : {"value" : 22174.71153846154}},{"key" : 22,"doc_count" : 51,"ageAvg" : {"value" : 24731.07843137255}},{"key" : 28,"doc_count" : 51,"ageAvg" : {"value" : 28273.882352941175}},{"key" : 33,"doc_count" : 50,"ageAvg" : {"value" : 25093.94}},{"key" : 34,"doc_count" : 49,"ageAvg" : {"value" : 26809.95918367347}},{"key" : 30,"doc_count" : 47,"ageAvg" : {"value" : 22841.106382978724}},{"key" : 21,"doc_count" : 46,"ageAvg" : {"value" : 26981.434782608696}},{"key" : 40,"doc_count" : 45,"ageAvg" : {"value" : 27183.17777777778}},{"key" : 20,"doc_count" : 44,"ageAvg" : {"value" : 27741.227272727272}},{"key" : 23,"doc_count" : 42,"ageAvg" : {"value" : 27314.214285714286}},{"key" : 24,"doc_count" : 42,"ageAvg" : {"value" : 28519.04761904762}},{"key" : 25,"doc_count" : 42,"ageAvg" : {"value" : 27445.214285714286}},{"key" : 37,"doc_count" : 42,"ageAvg" : {"value" : 27022.261904761905}},{"key" : 27,"doc_count" : 39,"ageAvg" : {"value" : 21471.871794871793}},{"key" : 38,"doc_count" : 39,"ageAvg" : {"value" : 26187.17948717949}},{"key" : 29,"doc_count" : 35,"ageAvg" : {"value" : 29483.14285714286}}]}}}
示例3-查出所有年龄分布,并且这些年龄段中M的平均薪资和F的平均薪资以及这个年龄段的总体平均薪资
GET /bank/_search{"query": {"match_all": {}},"aggs": {"ageAgg": {"terms": {"field": "age","size": 100},"aggs": {"genderAgg": {"terms": {"field": "gender.keyword","size": 10},"aggs": {"balanceAvg": {"avg": {"field": "balance"}}}},"ageBalanceAvg": {"avg": {"field": "balance"}}}}},"size": 0}
结果
{"took" : 7,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1000,"relation" : "eq"},"max_score" : null,"hits" : [ ]},"aggregations" : {"ageAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : 31,"doc_count" : 61,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 35,"balanceAvg" : {"value" : 29565.628571428573}},{"key" : "F","doc_count" : 26,"balanceAvg" : {"value" : 26626.576923076922}}]},"ageBalanceAvg" : {"value" : 28312.918032786885}},{"key" : 39,"doc_count" : 60,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 38,"balanceAvg" : {"value" : 26348.684210526317}},{"key" : "M","doc_count" : 22,"balanceAvg" : {"value" : 23405.68181818182}}]},"ageBalanceAvg" : {"value" : 25269.583333333332}},{"key" : 26,"doc_count" : 59,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 32,"balanceAvg" : {"value" : 25094.78125}},{"key" : "F","doc_count" : 27,"balanceAvg" : {"value" : 20943.0}}]},"ageBalanceAvg" : {"value" : 23194.813559322032}},{"key" : 32,"doc_count" : 52,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 28,"balanceAvg" : {"value" : 22941.964285714286}},{"key" : "F","doc_count" : 24,"balanceAvg" : {"value" : 25128.958333333332}}]},"ageBalanceAvg" : {"value" : 23951.346153846152}},{"key" : 35,"doc_count" : 52,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 28,"balanceAvg" : {"value" : 24226.321428571428}},{"key" : "F","doc_count" : 24,"balanceAvg" : {"value" : 19698.791666666668}}]},"ageBalanceAvg" : {"value" : 22136.69230769231}},{"key" : 36,"doc_count" : 52,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 31,"balanceAvg" : {"value" : 20884.677419354837}},{"key" : "F","doc_count" : 21,"balanceAvg" : {"value" : 24079.04761904762}}]},"ageBalanceAvg" : {"value" : 22174.71153846154}},{"key" : 22,"doc_count" : 51,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 27,"balanceAvg" : {"value" : 22152.74074074074}},{"key" : "M","doc_count" : 24,"balanceAvg" : {"value" : 27631.708333333332}}]},"ageBalanceAvg" : {"value" : 24731.07843137255}},{"key" : 28,"doc_count" : 51,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 31,"balanceAvg" : {"value" : 27076.8064516129}},{"key" : "M","doc_count" : 20,"balanceAvg" : {"value" : 30129.35}}]},"ageBalanceAvg" : {"value" : 28273.882352941175}},{"key" : 33,"doc_count" : 50,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 26,"balanceAvg" : {"value" : 26437.615384615383}},{"key" : "M","doc_count" : 24,"balanceAvg" : {"value" : 23638.291666666668}}]},"ageBalanceAvg" : {"value" : 25093.94}},{"key" : 34,"doc_count" : 49,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 30,"balanceAvg" : {"value" : 26039.166666666668}},{"key" : "M","doc_count" : 19,"balanceAvg" : {"value" : 28027.0}}]},"ageBalanceAvg" : {"value" : 26809.95918367347}},{"key" : 30,"doc_count" : 47,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 25,"balanceAvg" : {"value" : 25316.16}},{"key" : "M","doc_count" : 22,"balanceAvg" : {"value" : 20028.545454545456}}]},"ageBalanceAvg" : {"value" : 22841.106382978724}},{"key" : 21,"doc_count" : 46,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 24,"balanceAvg" : {"value" : 28210.916666666668}},{"key" : "M","doc_count" : 22,"balanceAvg" : {"value" : 25640.18181818182}}]},"ageBalanceAvg" : {"value" : 26981.434782608696}},{"key" : 40,"doc_count" : 45,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 24,"balanceAvg" : {"value" : 26474.958333333332}},{"key" : "F","doc_count" : 21,"balanceAvg" : {"value" : 27992.571428571428}}]},"ageBalanceAvg" : {"value" : 27183.17777777778}},{"key" : 20,"doc_count" : 44,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 27,"balanceAvg" : {"value" : 29047.444444444445}},{"key" : "F","doc_count" : 17,"balanceAvg" : {"value" : 25666.647058823528}}]},"ageBalanceAvg" : {"value" : 27741.227272727272}},{"key" : 23,"doc_count" : 42,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 24,"balanceAvg" : {"value" : 27730.75}},{"key" : "F","doc_count" : 18,"balanceAvg" : {"value" : 26758.833333333332}}]},"ageBalanceAvg" : {"value" : 27314.214285714286}},{"key" : 24,"doc_count" : 42,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 23,"balanceAvg" : {"value" : 29414.521739130436}},{"key" : "M","doc_count" : 19,"balanceAvg" : {"value" : 27435.052631578947}}]},"ageBalanceAvg" : {"value" : 28519.04761904762}},{"key" : 25,"doc_count" : 42,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 23,"balanceAvg" : {"value" : 29336.08695652174}},{"key" : "F","doc_count" : 19,"balanceAvg" : {"value" : 25156.263157894737}}]},"ageBalanceAvg" : {"value" : 27445.214285714286}},{"key" : 37,"doc_count" : 42,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 23,"balanceAvg" : {"value" : 25015.739130434784}},{"key" : "F","doc_count" : 19,"balanceAvg" : {"value" : 29451.21052631579}}]},"ageBalanceAvg" : {"value" : 27022.261904761905}},{"key" : 27,"doc_count" : 39,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 21,"balanceAvg" : {"value" : 21618.85714285714}},{"key" : "M","doc_count" : 18,"balanceAvg" : {"value" : 21300.38888888889}}]},"ageBalanceAvg" : {"value" : 21471.871794871793}},{"key" : 38,"doc_count" : 39,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "F","doc_count" : 20,"balanceAvg" : {"value" : 27931.65}},{"key" : "M","doc_count" : 19,"balanceAvg" : {"value" : 24350.894736842107}}]},"ageBalanceAvg" : {"value" : 26187.17948717949}},{"key" : 29,"doc_count" : 35,"genderAgg" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 0,"buckets" : [{"key" : "M","doc_count" : 23,"balanceAvg" : {"value" : 29943.17391304348}},{"key" : "F","doc_count" : 12,"balanceAvg" : {"value" : 28601.416666666668}}]},"ageBalanceAvg" : {"value" : 29483.14285714286}}]}}}
