1 search搜索入门
1.1 搜索语法入门
1.1.1 query string search
- 语法:无条件查询所有
GET /index/_search
- 示例:
- 新增数据:
PUT /book/_doc/1{"name": "Bootstrap开发","description": "Bootstrap是由Twitter推出的一个前台页面开发css框架,是一个非常流行的开发框架,此框架集成了多种页面效果。此开发框架包含了大量的CSS、JS程序代码,可以帮助开发者(尤其是不擅长css页面开发的程序人员)轻松的实现一个css,不受浏览器限制的精美界面css效果。","studymodel": "201002","price":38.6,"timestamp":"2019-08-25 19:11:35","pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags": [ "bootstrap", "dev"]}
PUT /book/_doc/2{"name": "java编程思想","description": "java语言是世界第一编程语言,在软件开发领域使用人数最多。","studymodel": "201001","price":68.6,"timestamp":"2019-08-25 19:11:35","pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags": [ "java", "dev"]}
PUT /book/_doc/3{"name": "spring开发基础","description": "spring 在java领域非常流行,java程序员都在用。","studymodel": "201001","price":88.6,"timestamp":"2019-08-24 19:11:35","pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags": [ "spring", "java"]}
- 无条件查询所有:
GET /book/_search
- 返回:
{"took" : 928,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "book","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"name" : "Bootstrap开发","description" : "Bootstrap是由Twitter推出的一个前台页面开发css框架,是一个非常流行的开发框架,此框架集成了多种页面效果。此开发框架包含了大量的CSS、JS程序代码,可以帮助开发者(尤其是不擅长css页面开发的程序人员)轻松的实现一个css,不受浏览器限制的精美界面css效果。","studymodel" : "201002","price" : 38.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["bootstrap","dev"]}},{"_index" : "book","_type" : "_doc","_id" : "2","_score" : 1.0,"_source" : {"name" : "java编程思想","description" : "java语言是世界第一编程语言,在软件开发领域使用人数最多。","studymodel" : "201001","price" : 68.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["java","dev"]}},{"_index" : "book","_type" : "_doc","_id" : "3","_score" : 1.0,"_source" : {"name" : "spring开发基础","description" : "spring 在java领域非常流行,java程序员都在用。","studymodel" : "201001","price" : 88.6,"timestamp" : "2019-08-24 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["spring","java"]}}]}}
- took:耗费了几毫秒。
- timed_out:是否超时,false表示没有超时,true表示超时了。
- _shards:到几个分片搜索。
- successful:成功几个。
- skipped:跳过几个。
- failed:失败几个。
- hits:命中。
- total:查询结果的数量。
- max_score:最大分数。socre的含义,就是document对于一个search的相关度的匹配分数,越相关,就越匹配,分数也越高。
- hits:包含了匹配搜索的document的所有详细数据
1.1.2 传参
- 语法:
GET /index/_search?q=field:xxx&sort=filed2:desc
类似于SQL:SELECT * FROM emp where field like ‘%xxx%’ order by filed2 desc。
- 示例:
- 查询索引book中的name是java的相关数据
GET /book/_search?q=name:java
- 返回:
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 0.9227538,"hits" : [{"_index" : "book","_type" : "_doc","_id" : "2","_score" : 0.9227538,"_source" : {"name" : "java编程思想","description" : "java语言是世界第一编程语言,在软件开发领域使用人数最多。","studymodel" : "201001","price" : 68.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["java","dev"]}}]}}
- 示例:
- 查询索引book根据价格降序
GET /book/_search?sort=price:desc
- 返回:
{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : null,"hits" : [{"_index" : "book","_type" : "_doc","_id" : "3","_score" : null,"_source" : {"name" : "spring开发基础","description" : "spring 在java领域非常流行,java程序员都在用。","studymodel" : "201001","price" : 88.6,"timestamp" : "2019-08-24 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["spring","java"]},"sort" : [88.6]},{"_index" : "book","_type" : "_doc","_id" : "2","_score" : null,"_source" : {"name" : "java编程思想","description" : "java语言是世界第一编程语言,在软件开发领域使用人数最多。","studymodel" : "201001","price" : 68.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["java","dev"]},"sort" : [68.6]},{"_index" : "book","_type" : "_doc","_id" : "1","_score" : null,"_source" : {"name" : "Bootstrap开发","description" : "Bootstrap是由Twitter推出的一个前台页面开发css框架,是一个非常流行的开发框架,此框架集成了多种页面效果。此开发框架包含了大量的CSS、JS程序代码,可以帮助开发者(尤其是不擅长css页面开发的程序人员)轻松的实现一个css,不受浏览器限制的精美界面css效果。","studymodel" : "201002","price" : 38.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["bootstrap","dev"]},"sort" : [38.6]}]}}
1.1.3 timeout超时查询
- 语法:
GET /index/_search?timeout=10s
- 示例:超时查询时间为10秒
GET /book/_search?timeout=10s
- 搜索的时候,请求必定会横跨所有的主分片,当数量太大的时候,搜索完毕,假设每个分配都需要1个小时,那么用户可能会失去耐心,以至于我们丢失业务。
- timeout机制,用来指定每个分片只能在给定的时间内查询数据,能有几条数据就返回给用户几条数据。
- 全局设置:在ES的配置文件elasticsearch.yml中配置
search.default_search_timeout: 100ms,如果不配置此参数,默认情况下不超时。
1.2 multi-index多索引搜索
1.2.1 multi-index搜索模式
- multi-index搜索模式:一次性的搜索多个index和多个type下的数据
- 语法:所有索引下的所有数据都搜索出来(慎用)
GET /_search
- 语法:指定一个索引,搜索其下的所有数据
GET /index/_search
- 语法:同时搜索两个及以上索引下的所有数据
GET /index1,index2/_search
- 语法:按照通配符去匹配多个索引
GET /index*/_search
- 应用场景:生产环境中的日志log索引是以日期分开的,比如log_to_es_2019-11-11,log_to_es_2019-11-12和log_to_es_2019-11-13。
1.2.2 搜索原理
- 客户端发送一个搜索请求,会把请求发送到所有的主分片上进行查询。因为每个主分片上都有一部分数据,每个主分片可以同时搜索;在有副本分片的情况下,会将请求发送到对应的副本分片上进行搜索。
1.3 分页搜索
1.3.1 分页搜索语法
- 语法:类似于MySQL中的
SELECT * FROM book limit 0,10
GET /index/_search?from=0&size=10
- 示例:
GET /book/_search?from=0&size=3
- 示例:
GET /book/_search?from=3&size=3
1.3.2 deep paging(深度分页)
- deep paging(深度分页):根据相关度评分倒排序,如果分页过深,协调节点会将大量数据聚合分析。
- deep paging(深度分页)的性能问题:
消耗网络带宽:搜索过深的话,各个shard要把数据传递给coordinate node(协调节点),这个过程中有大量的数据传递,消耗网络。
消耗内存:各个shard要把数据传递给coordinate node(协调节点),这个传递过来的数据,是被coordinate node(协调节点)保存在内存中的,会大量消耗内存。
消耗CPU:coordinate node(协调节点)要把传递过来的数据进行排序,这个排序过程很耗CPU。
- 总结:鉴于deep paging(深度分页)的性能问题,应该尽量少用之。
1.4 query string基础语法
1.4.1 query string基础语法
- 语法:根据文档中的某给field包含search content的文档信息
GET /index/_search?q=field:search content
- 语法:根据文档中的某给field包含search content的文档信息
GET /index/_search?q=+field:search content
- 语法:根据文档中的某给field不包含search content的文档信息
GET /index/_search?q=-field:search content
- 示例:查询图书名是java的图书信息
GET /book/_search?q=name:java
- 示例:查询图书名不是java的图书信息
GET /book/_search?q=-name:java
1.4.2 _all metadata的原理和作用
- 语法:
GET /index/_search?q=search content
- 原理:
- 上述的语法可以搜索所有的field,只要任意一个field包含指定的关键字都可以搜索出来。那么,在进行搜索的时候,难道需要对文档中的每一个field都进行一次搜索吗?答案,显示不是。
- ES中的all元数据:ES在建立索引的时候,插入一条文档的时候,ES会将所有的field值经过全量分词,把这些分词,放到all field中,在搜索的时候,如果没有指定field,就去
_all中搜索。 - 比如:
{name:"tom",age:13}对应_all:tom,13。
1.5 query DSL入门
1.5.1 什么是DSL
GET /book/_search?from=0&size=10&q=name:java&sort=price:desc
- 当query string后面的参数越来越多的时候,搜索条件越来越复杂,不能满足我们的需求,query DSL应运而生。
- DSL(Domain Specified Language):领域特定语言。
- ES的DSL,可以在请求体中携带条件,功能非常强大。
- 语法:查询全部(query string的写法:
GET /index/_search)
GET /index/_search{"query":{"match_all":{}}}
- 语法:查询某个字段并且按照某个字段排序(query string的写法:
GET /index/_search?q=field:search value&sort=field:desc)
GET /index/_search{"query": {"match": {"field": "search value"}},"sort": [{"field": {"order": "desc/asc"}}]}
- 语法:分页查询(query string的写法:
GET /index/_search?from=0&size=5)
GET /index/_search{"query":{"match_all":{}},from: 0,size: 5}
- 语法:返回指定字段(query string的写法:
GET /index/_search?_source=field1,field2)
GET /index/_search{"query": {"match_all": {}},"_source": ["field1","field2"]}
1.5.2 query DSL语法
- 语法1:
GET /index/_search{QUERY_NAME: {ARGUMENT: VALUE,ARGUMENT: VALUE,...}}
- 语法2:
GET /index/_search{QUERY_NAME: {FIELD_NAME: {ARGUMENT: VALUE,ARGUMENT: VALUE,...}}}
- 示例:
GET /test_index/_search{"query": {"match": {"test_field": "test"}}}
1.5.3 组合多个搜索条件
- 初始化数据:
PUT /website/_doc/1{"title": "my hadoop article","content": "hadoop is very bad","author_id": 111}
PUT /website/_doc/2{"title": "my elasticsearch article","content": "es is very bad","author_id": 112}
PUT /website/_doc/3{"title": "my elasticsearch article","content": "es is very goods","author_id": 111}
- 示例:title必须包含elasticsearch,content可以包含elasticsearch也可以不包含elasticsearch,author_id必须不为111。
GET /website/_search{"query": {"bool": {"must": [{"match": {"title": "elasticsearch"}}],"should": [{"match": {"content": "elasticsearch"}}],"must_not": [{"match": {"author_id": "111"}}]}}}
1.6 DSL语法练习
1.6.1 match_all
- 示例:
GET /book/_search{"query": {"match_all": {}}}
1.6.2 match
- 示例:
GET /book/_search{"query": {"match": {"description": "java"}}}
1.6.3 multi_match
- 示例:
GET /book/_search{"query": {"multi_match": {"query": "java","fields": ["name","description"]}}}
1.6.4 range query范围查询
- 示例:
GET /book/_search{"query": {"range": {"price": {"gte": 50,"lte": 70}}}}
1.6.5 term query
- 示例:term query 在查询的时候不分词。
GET /book/_search{"query": {"term": {"description": {"value": "java程序员"}}}}
1.6.6 terms query
- 示例:
GET /book/_search{"query": {"terms": {"tags": ["java","dev"]}}}
1.6.7 exist query
- 查询某些文档是否存在某个字段。
- 示例:
GET /book/_search{"query": {"exists": {"field":"name"}}}
1.6.8 fuzzy query
- 返回包含和搜索词类似的词的文档,该词由Levenshtein编辑距离度量。
- 包括以下几种情况:
- ①更改角色(box→fox)。
- ②删除字符(aple→apple)。
- ③插入字符(sick→sic)。
- ④调换两个相邻字符(ACT→CAT)
- 示例:
GET /book/_search{"query": {"fuzzy": {"description": {"value": "java"}}}}
1.6.9 IDs
- 示例:
GET /book/_search{"query": {"ids": {"values": ["1","2","3"]}}}
1.6.10 prefix 前缀查询
- 示例:
GET /book/_search{"query": {"prefix": {"description": {"value": "spring"}}}}
1.6.11 regexp query正则查询
- 示例:
GET /book/_search{"query": {"regexp": {"description": {"value": "j.*a","flags": "ALL","max_determinized_states": 10000,"rewrite": "constant_score"}}}}
1.7 Filter
1.7.1 query和filter的应用示例
- 示例:查询“description”中有“java程序员”,并且价格大于50小于90的数据。
GET /book/_search{"query": {"bool": {"must": [{"match": {"description": "java程序员"}},{"range": {"price": {"gte": 50,"lte": 90}}}]}}}
GET /book/_search{"query": {"bool": {"must": [{"match": {"description": "java程序员"}}],"filter": [{"range": {"price": {"gte": 50,"lte": 90}}}]}}}
1.7.2 query和filter的概念对比
- query:会去计算每个文档相对于搜索条件的相关度,并按照相关度进行排序。
- filter:仅仅只是按照搜索条件过滤出需要的数据即可,不计算任何相关度分数,对相关度没有任何影响。
- 应用场景:
- ①一般来说,如果我们是在进行搜索,需要将最匹配搜索条件的数据先返回,那么用query。
- ②如果我们只是根据一些条件筛选出一部分数据,不关注其排序,那么用filter。
1.7.3 query和filter的性能对比
- query:要计算相关度分数,按照分数进行排序,而且无法cache结果。
- filter:不需要计算相关度分数,不需要按照相关度分数进行阿皮序,同时还内置的自动cache最常使用filter的数据。比如:keyword,范围查询。
1.8 定位错误语法
- 验证错误语句:
GET /book/_validate/query?explain{"query": {"mach": {"name": "java"}}}
- 返回:
{"valid" : false,"error" : "ParsingException[unknown query [mach] did you mean [match]?]; nested: NamedObjectNotFoundException[[3:13] unknown field [mach]];; org.elasticsearch.common.xcontent.NamedObjectNotFoundException: [3:13] unknown field [mach]"}
- 正确的语句:
GET /book/_validate/query?explain{"query": {"match": {"description": "java is good"}}}
- 返回:
{"_shards" : {"total" : 1,"successful" : 1,"failed" : 0},"valid" : true,"explanations" : [{"index" : "book","valid" : true,"explanation" : "description:java description:is description:good"}]}
- 一般在那种特别复杂庞大的搜索下,比如:直接写了数百行的搜索,这个时候我们可以使用validate去验证一下,搜索是否合法。
- 合法以后,explain就像MySQL中的执行计划,可以看到搜索的目标等信息。
1.9 定制排序规则
1.9.1 默认排序规则
- 默认情况下,是按照
_score进行降序排序的。 - 但是,某些情况下,可能没有有用的
_score,比如说filter。 - 示例:只过滤
GET /book/_search{"query": {"constant_score": {"filter": {"range": {"price": {"gte": 50,"lte": 90}}}}}}
1.9.2 定制排序规则
- 示例:
GET /book/_search{"query": {"constant_score": {"filter": {"term": {"studymodel": "201001"}}}},"sort": [{"price": {"order": "desc"}}]}
1.10 Text字段排序问题
- 场景:数据库中按照某个字段排序,SQL只需要写
order by 字段名即可。 - 如果在ES对一个text的field进行排序,ES是无法进行排序的。因为文档存储到索引表的时候,是分词存储的,ES无法知道此字段的真实值。
- 通常的解决方案:
- 方案一:设置
fielddata:true。此种方式有弊端,比如下面的title是不计入相关度查询的,而且是按照第一个分词排序的。
PUT /website{"mappings": {"properties": {"title": {"type": "text","fielddata": true},"content": {"type": "text"},"post_date": {"type": "date"},"author_id": {"type": "long"}}}}
- 方案二:将一个text类型的field字段建立两次索引,一个分词,用来进行搜索;一个分词,用来进行排序。
PUT /website{"mappings": {"properties": {"title": {"type": "text","fields": {"keyword":{"type":"keyword"}}},"content": {"type": "text"},"post_date": {"type": "date"},"author_id": {"type": "long"}}}}
PUT /website/_doc/1{"title": "first article","content": "this is my second article","post_date": "2019-01-01","author_id": 110}
PUT /website/_doc/2{"title": "second article","content": "this is my second article","post_date": "2019-01-01","author_id": 110}
PUT /website/_doc/3{"title": "third article","content": "this is my third article","post_date": "2019-01-02","author_id": 110}
GET /website/_search{"query": {"match_all": {}},"sort": [{"title.keyword": {"order": "desc"}}]}
1.11 scroll分批查询
- 场景:下载某一个索引中的一亿条数据到文件或数据库中,不能一下全查出来,容易造成系统内存溢出,可以采用scroll滚动搜索技术,一批一批的查询。
- scroll搜索会在第一次搜索的时候,保存一个当时的视图快照,之后只会基于该视图快照提供数据搜索,如果这个期间数据发生变更,是不会让用户看到的。
- 每次发送scroll请求,需要指定一个scroll参数,指定一个时间窗口,每次搜索请求只要在这个时间窗口内能完成就可以了。
- scroll分批查询的步骤:
搜索:下面的语句表示1分钟查询2条数据
GET /book/_search?scroll=1m{"query": {"match_all": {}},"size": 2}
- 返回:获取的结果中包含
_scroll_id字段。
{"_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFktoNzhPVE1qUUhTaUctdTFmUEpaRkEAAAAAAAAcgxZNVFl2VXhVV1JTcVNFMkdMc1g4X3pn","took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 10,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "book","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"name" : "Bootstrap开发","description" : "Bootstrap是由Twitter推出的一个前台页面开发css框架,是一个非常流行的开发框架,此框架集成了多种页面效果。此开发框架包含了大量的CSS、JS程序代码,可以帮助开发者(尤其是不擅长css页面开发的程序人员)轻松的实现一个css,不受浏览器限制的精美界面css效果。","studymodel" : "201002","price" : 38.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["bootstrap","dev"]}},{"_index" : "book","_type" : "_doc","_id" : "2","_score" : 1.0,"_source" : {"name" : "java编程思想","description" : "java语言是世界第一编程语言,在软件开发领域使用人数最多。","studymodel" : "201001","price" : 68.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["java","dev"]}}]}}
拿第一步获取的
scroll_id字段的值,再发送一次scroll请求:
GET /_search/scroll{"scroll":"1m","scroll_id": "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFktoNzhPVE1qUUhTaUctdTFmUEpaRkEAAAAAAAAc2xZNVFl2VXhVV1JTcVNFMkdMc1g4X3pn"}
重复第二步,直到查不到数据为止。
scroll分批查询和分页查询的区别:
- 分页查询是给用户看得。
- scroll分批查询是系统内部操作,比如批量下载数据,转移数据、零停机改变索引映射(这种方式太麻烦,弃用,ES有更好的方式实现)等等。
2 Java API实现搜索
2.1 match_all
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/*** 搜索全部记录*<pre>* GET /book/_search* {* "query": {* "match_all": {}* }* }* </pre>*/@Testpublic void testMatchAll() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.matchAllQuery());//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.2 match
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "match": {* "name": "java"* }* }* }* </pre>*/@Testpublic void testMatchAll() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.matchQuery("name", "java"));//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.3 multi_match
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "multi_match": {* "query": "java",* "fields": ["name","description"]* }* }* }* </pre>*/@Testpublic void testMatchAll() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery("java", "name","description"));//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.4 分页搜索
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "match_all": {}* },* "from": 0,* "size": 5* }* </pre>*/@Testpublic void testPagingQuery() throws IOException {Integer pageNo = 1;Integer pageSize = 5;//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.matchAllQuery());searchSourceBuilder.from((pageNo - 1) * pageSize);searchSourceBuilder.size(pageSize);//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.5 IDs
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "ids": {* "values": ["1","2","3"]* }* }* }* </pre>*/@Testpublic void testIDs() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.idsQuery().addIds("1", "2", "3"));//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.6 term query
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "term": {* "description": {* "value": "java"* }* }* }* }* </pre>*/@Testpublic void testTermQuery() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.termQuery("description", "java"));//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.7 range query
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "range": {* "price": {* "gte": 50,* "lte": 70* }* }* }* }* </pre>*/@Testpublic void testRangeQuery() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gte(50).lte(70));//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.8 bool query
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.BoolQueryBuilder;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "bool": {* "must": [* {* "multi_match": {* "query": "java程序员",* "fields": ["name","description"]* }* }* ],* "should": [* {* "match": {* "studymodel": "201001"* }* }* ]* }* }* }* </pre>*/@Testpublic void testBoolQuery() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();boolQueryBuilder.must(QueryBuilders.multiMatchQuery("java程序员", "name","description"));boolQueryBuilder.should(QueryBuilders.matchQuery("studymodel", "201001"));SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(boolQueryBuilder);//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.9 Filter
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "constant_score": {* "filter": {* "range": {* "price": {* "gte": 50,* "lte": 90* }* }* }* }* }* }* </pre>*/@Testpublic void testFilter() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.constantScoreQuery(QueryBuilders.rangeQuery("price").gte(50).lte(70)));//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.BoolQueryBuilder;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "bool": {* "must": [* {* "multi_match": {* "query": "java程序员",* "fields": ["name","description"]* }* }* ],* "should": [* {* "match": {* "studymodel": "201001"* }* }* ],* "filter": [* {* "range": {* "price": {* "gte": 50,* "lte": 90* }* }* }* ]* }* }* }* </pre>*/@Testpublic void testFilter() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();boolQueryBuilder.must(QueryBuilders.multiMatchQuery("java程序员", "name","description"));boolQueryBuilder.should(QueryBuilders.matchQuery("studymodel", "201001"));boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(50).lte(90));SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(boolQueryBuilder);//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);float score = searchHit.getScore();System.out.println("score = " + score);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
2.10 排序
- 示例:
package com.sunxaiping.elk;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.elasticsearch.search.sort.SortOrder;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;import java.util.List;import java.util.Map;/*** Query DSL*/@SpringBootTestpublic class ElkApplicationTests {@Autowiredprivate RestHighLevelClient client;/***<pre>* GET /book/_search* {* "query": {* "multi_match": {* "query": "java程序员",* "fields": [* "name",* "description"* ]* }* },* "sort": [* {* "price": {* "order": "desc"* }* }* ]* }* </pre>*/@Testpublic void testSort() throws IOException {//创建请求SearchRequest searchRequest = new SearchRequest("book");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery("java程序员", "name", "description"));searchSourceBuilder.sort("price", SortOrder.DESC);//返回某些字段searchSourceBuilder.fetchSource(List.of("price", "name", "description").toArray(String[]::new), List.of().toArray(String[]::new));searchRequest.source(searchSourceBuilder);//发送请求SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);//返回结果SearchHits hits = response.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit searchHit : searchHits) {String index = searchHit.getIndex();System.out.println("index = " + index);String type = searchHit.getType();System.out.println("type = " + type);String id = searchHit.getId();System.out.println("id = " + id);Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();System.out.println("sourceAsMap = " + sourceAsMap);}}}
