1. sphinx php 学习可以参考 /var/www/to8to/trunk/gb_php/sphinxapi.php
    3. include 'sphinxapi.php';
    4. $sp = new SphinxClient;
    5. $sp->SetServer('127.0.0.1', 9314);
    6. $sp->SetConnectTimeout(5);
    8. $sp->SetLimits(0, 10);//($start, $limit);
    10. $keyword=(isset($_GET['kw'])&& !empty($_GET['kw'])) ?trim($_GET['kw']) : '搜索内容';
    12. //在执行搜索之前，可以加入各种条件
    13. $result=$sp>Query($keyword,'iiyicms');//'*' 'iiyicms:iiyicms_increment'
    15. 学习地址
    16. http://www.21andy.com/new/20100928/1973.html
    18. 遇到问题
    20. 编绎coreseek以支持python的时候，提示：
    22. py_layer.h:16:27: 致命错误： Python.h：没有那个文件或目录
    24. 找不到python.h头文件
    26. 解决方法：
    28. ubuntu下安装python-dev
    30. sudo apt-get install python-dev
    32. http://pecl.php.net/package/sphinx
    34. sphinx php 学习可以参考 /var/www/to8to/trunk/gb_php/sphinxapi.php
    36. include 'sphinxapi.php';
    37. $sp = new SphinxClient;
    38. $sp->SetServer('127.0.0.1', 9314);
    39. $sp->SetConnectTimeout(5);
    41. $sp->SetLimits(0, 10);//($start, $limit);
    43. $keyword=(isset($_GET['kw'])&& !empty($_GET['kw'])) ?trim($_GET['kw']) : '搜索内容';
    45. //在执行搜索之前，可以加入各种条件
    46. $result=$sp>Query($keyword,'iiyicms');//'*' 'iiyicms:iiyicms_increment'
    48. new sphinxapi url
    49. https://code.google.com/p/sphinxsearch/source/browse/trunk/api/
    50. http://code.google.com/p/sphinxsearch/source/browse/trunk/api/sphinxapi.php
    52. 1、PHP+Mysql+Sphinx高效的站内搜索引擎搭建详释
    53. http://jingyan.baidu.com/article/95c9d20d9a7176ec4e756119.html
    55. 2、使用Coreseek-4.1快速搭建Sphinx中文分词 Php-Mysql 全文检索 搜索引擎
    56. http://www.gretheer.com/2014/07/install-coreseek-sphinx-php-mysql.html
    58. 3、sphinx 下载地址
    59. http://sphinxsearch.com/downloads/release/
    61. 找到对应版本
    62. http://sphinxsearch.com/files/sphinxsearch_2.2.6-release-1~wheezy_i386.deb
    64. 安装学习
    65. http://sphinxsearch.com/docs/archives/1.10/installing.html
    67. 两个结合安装
    69. http://blog.atime.me/note/sphinx-coreseek-summary.html
    71. http://github.tiankonguse.com/blog/2014/11/03/coreseek-install-log/
    73. 开发环境
    74. 操作系统: Ubuntu 12.04 x86-64
    75. Coreseek: 4.1测试版(Sphinx-2.0.1)
    76. Python: 2.7
    77. Sphinx/Coreseek简介
    78. Sphinx是一个高性能的全文检索引擎，使用C++语言开发，采用GPL协议发布，可购买商业授权，目前的稳定版本是2.1.7。
    80. Coreseek是基于Sphinx的中文全文检索引擎，使用MMSEG算法进行中文分词，并且提供Python数据源。Coreseek采用GPLv2协议发布，可购买商业授权，目前的稳定版本是3.2.14，基于Sphinx-0.9.9，测试版本是4.1，基于Sphinx-2.0.1。（另外，Coreseek官方论坛在2013年的年末称即将发布5.0版本，不过至今无详细消息）
    82. Sphinx/Coreseek安装
    83. 下载Coreseek-4.1的源代码
    85. wget http://www.coreseek.cn/uploads/csft/4.0/coreseek-4.1-beta.tar.gz
    86. tar xvf coreseek-4.1.beta.tar.gz
    87. cd coreseek-4.1-beta
    88. 解压后发现有三个目录，主要的目录结构如下
    90. coreseek-4.1-beta/
    91. csft-4.1/ coreseek修改sphinx-2.0.1后的代码
    92. api/ sphinx searchd[查询API][6]的实现
    93. mmseg-3.2.14/ libmmseg分词库
    94. testpack/ 测试和配置示例
    95. README.txt 介绍和安装指南
    96. 按照官方的安装指南，依次安装mmseg和csft。如果在configure过程中提示缺少头文件，可通过apt-file查询需要安装的软件包。
    98. 安装mmseg-3.2.14 http://www.coreseek.cn/uploads/csft/3.2/mmseg-3.2.14.tar.gz
    99. 这里完全参考官方的安装指南即可
    101. cd mmseg-3.2.14
    102. ./bootstrap
    103. ./configure --prefix=/usr/local/mmseg3
    104. make && sudo make install
    105. 安装libiconv-1.14
    106. 先安装libiconv，用于字符集编码的转换。
    108. wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.14.tar.gz
    109. tar xvf libiconv-1.14.tar.gz
    110. cd libiconv-1.14
    111. ./configure
    112. make && sudo make install && ldconfig
    113. 如果你的glibc版本在2.16以上，make时很有可能出现如下错误
    115. In file included from progname.c:26:0:
    116. ./stdio.h:1010:1: error: 'gets' undeclared here (not in a function)
    117. _GL_WARN_ON_USE (gets, "gets is a security hole - use fgets instead");
    118. ^
    119. 参考这里的方法，下载patch文件，解压后打上patch即可。
    121. 在libiconv-1.14目录下执行
    123. wget -O - http://blog.atime.me/static/resource/libiconv-glibc-2.16.patch.gz | gzip -d - | patch -p0
    124. 或者考虑直接注释掉srclib/stdio.in.h文件的第698行（应该没问题），即
    126. // _GL_WARN_ON_USE (gets, "gets is a security hole - use fgets instead");
    127. 安装csft-4.1
    128. 这里configure的参数和安装指南上稍有区别，一是添加--with-python选项来支持Python数据源，二是添加LIBS=-liconv来避免最后的链接错误。
    130. cd csft-4.1
    131. sh buildconf.sh
    132. ./configure --prefix=/usr/local/coreseek --without-unixodbc --with-mmseg --with-mmseg-includes=/usr/local/mmseg3/include/mmseg/ --with-mmseg-libs=/usr/local/mmseg3/lib/ --with-mysql --with-python LIBS=-liconv
    133. make -j2 && sudo make install
    134. 如果sh buildconf.sh最后没有生成configure脚本，且提示automake: warnings are treated as errors，可以将configure.ac中的这行
    136. AM_INIT_AUTOMAKE([-Wall -Werror foreign])
    137. 改为
    139. AM_INIT_AUTOMAKE([-Wall foreign])
    140. 即删掉-Werror，然后重新运行sh buildconf.sh。
    142. 如果configure的时候提示没有安装MySQL的头文件，安装libmysql++-dev包即可。
    144. 如果你的gcc版本在4.7以上，编译的时候可能会因为sphinx的一个bug报错
    146. sphinxexpr.cpp:1746:43: error: 'ExprEval' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
    147. 解决方法参考bug报告里的一个patch，在csft-4.1目录下执行
    149. wget -O - http://blog.atime.me/static/resource/sphinxexpr-gcc4.7.patch.gz | gzip -d - | patch -p0
    150. 或者你也可以直接修改src/sphixexpr.cpp文件的1746, 1777和1823行，将三行中的ExprEval改为this->ExprEval。
    152. 安装辅助工具
    153. 将csft-4.1/contrib/scripts目录下的searchd脚本拷贝到/etc/init.d/目录下，即可使用service命令启动和终止searchd服务。
    155. 安装好coreseek后，将/usr/local/coreseek/share/man/目录下的所有文件和目录都拷贝到/usr/local/share/man/目录里，即可使用man命令查看indexer和searchd的使用手册。
    157. Sphinx/Coreseek目录结构
    158. 按照上面的步骤正确安装Coreseek后，在/usr/local/coreseek可看到如下几个文件夹
    160. bin/ sphinx的程序目录
    161. searchd 搜索服务器程序
    162. indexer 索引建立工具
    163. etc/ 配置文件目录
    164. csft.conf 默认配置文件
    165. share/
    166. man/ sphinx的man手册，建议拷贝到系统man目录，方便查询
    167. var/
    168. data/ 默认的索引存放目录
    169. log/ 默认的日志目录和pid文件目录
    170. 实际使用sphinx的流程大概如下:
    172. 使用indexer建立或更新索引，如果searchd已经运行，则需要使用--rotate选项。
    173. 运行searchd
    174. 例如：
    176. cd /usr/local/coreseek
    177. ./bin/indexer --all # 第一次建立索引，使用默认配置文件/usr/local/coreseek/etc/csft.conf
    178. ./bin/searchd # 使用默认配置文件/usr/local/coreseek/etc/csft.conf
    179. Sphinx/Coreseek配置
    180. 配置文件可参考Sphinx的官方文档和配置例子/usr/local/coreseek/etc/sphinx.conf.dist。
    182. searchd
    183. 配置示例
    185. searchd
    186. {
    187. listen = 9312
    188. listen = 9306:mysql41
    189. log = /usr/local/coreseek/var/log/searchd.log
    190. query_log = /usr/local/coreseek/var/log/query.log
    191. read_timeout = 5
    192. max_children = 30
    193. pid_file = /usr/local/coreseek/var/log/searchd.pid
    194. max_matches = 1000
    195. seamless_rotate = 1
    196. preopen_indexes = 1
    197. unlink_old = 1
    198. workers = threads # for RT to work
    199. }
    200. 这里面的诸多配置选项可参考searchd program configuration options。
    202. 其中，通过第二个listen配置listen = 9306:mysql41，你可以使用mysql的client来访问searchd的索引。
    204. mysql -h 127.0.0.1 -P 9306
    205. 然后使用SphinxQL查询语言即可搜索索引。
    207. indexer
    208. 配置示例
    210. indexer {
    211. mem_limit = 1024M
    212. write_buffer = 16M
    213. }
    214. 索引工具indexer的配置相对少一些，参考indexer program configuration options。需要注意的是，mem_limit如果查过2048M会出问题1。
    216. 数据源和索引配置
    217. 参考示例配置文件/usr/local/coreseek/etc/sphinx.conf.dist和官方文档Data source configuration options，Index configuration options即可。
    219. 数据源
    220. 关于数据源，需要注意的是:
    222. 每条数据的document id必须是唯一的正整数(不能为0)。6
    223. Python数据源
    224. Coreseek开发了一个号称万能的Python数据源，使用起来比xmlpipe2要方便一些。其实就是用Python脚本来获取待索引数据，配置文档见这里，接口文档见这里，示例程序见这里。
    226. Xmlpipe2数据源
    227. 这是用Sphinx官方支持的一个"万能"数据源，其实就是将待索引数据按照xmlpipe2的schema写入标准输出中。
    229. 在数据源的配置项中需要设置type为xmlpipe2，另外还要设置一个xmlpipe_command选项，该选项的命令必须输出符合xmlpipe2 schema的xml文档到标准输出流(stdout)里，比如:
    231. source news_src
    232. {
    233. type = xmlpipe2
    234. xmlpipe_command = cat /tmp/xmlpipe2_out.xml
    235. }
    236. 建立索引
    237. 使用indexer命令建立索引
    239. /usr/local/coreseek/bin/indexer --rotate $INDEX_NAME
    240. Sphinx使用indexer工具建立和更新索引，据称indexer的索引速度能达到10~15MB/秒2。实际使用过程中，我尝试过分别用Python数据源和xmlpipe2数据源来建立索引，xmlpipe2稍微快一点点。使用Python数据源索引14G文本，大约50万个文件，最后生成2.3G索引，最快在2.8MB/秒左右，估计是慢在中文分词上。
    242. 自定义中文词库
    243. 见这篇文章。
    245. 查询
    246. Sphinx支持使用SphinxAPI和SphinxQL查询数据。
    248. SphinxAPI
    249. SphinxAPI用于和searchd通信，官方提供PHP, Python和Java的实现，API的文档见此。Coreseek携带的API和示例程序实现都放在csft-4.1/api/目录下。
    251. SphinxQL
    252. SphinxQL是Sphinx提供的SQL方言，用于查询和管理索引，相比SphinxAPI，SphinxQL支持的操作更多，比如删除索引等，文档在此。
    254. 实际应用
    255. 项目简介
    256. 项目的部分需求：
    258. 目前需要做全文检索的数据是html网页文件，总数在1000万左右，文件总大小大概是200GB，每天新增几千个文件左右。将来很可能需要检索pdf和mysql等不同的数据来源。
    259. 提供RESTful风格的搜索接口，返回json格式的查询结果。因为搜索服务主要是内部使用，估计搜索请求的压力不大。
    260. 为缩短开发周期，整个项目采用Python实现，使用coreseek自带的Python数据源建立索引。
    262. 在开发过程中使用了如下的第三方Python packages:
    264. lxml-3.3.4: 解析html文件
    265. tornado-3.2: 异步http服务器，异步socket通信等
    266. 设计考量
    267. 索引
    269. 上面有提到过，indexer是一个单线程的工具，建立中文索引的速度基本上很难超过3MB/秒，因此可以考虑将大的索引拆分成若干小索引，这些小索引可以同时建立，最后再合并成一个完整的索引。
    271. 因为待索引文档的基数很大，但每天更新的数量又比较小，所以建立索引的时候最好使用官方推荐的一种Main + Delta的方式，主(Main)索引只需要最开始建立一次，然后每天重建一次增量(Delta)索引并合并到主索引中，相关文档见Delta index updates。
    273. Python相关
    275. 项目里需要使用Python查找和解析html文件。
    277. 文件查找没有使用Python标准库os的walk函数，当文件数量较多时，walk函数的效率会比较低。有兴趣的可以看下一个叫betterwalk的第三方库，据称比os.walk快不少。实际项目中，因为待索引文件的目录结构固定且很有规律，直接用os.listdir和os.lstat即可解决，os.lstat可以获取文件的最后修改日期，在建立增量索引的时候非常有用。
    279. html文件的解析使用了口碑很给力的lxml库，用lxml解析html文件时通常有多种方法，使用之前最好仔细看一下lxml各个函数的benchmark，了解一下哪种方法更快一些，比如使用xpath查找html节点时，lxml的XPath类比xpath()函数要快好几倍3。
    281. 另外，Python的多线程处理计算密集型(CPU Bound)任务是一个众所周知的大坑，比如多线程解析html文件。这时最好用多进程分别做解析任务，然后将解析好的文件收集起来。
    283. 前面说过indexer比较慢，一般建立索引的时候，速度瓶颈就在indexer上。为了尽量加快整体建立索引的速度，比较靠谱的方法是将文件扫描，文件解析和indexer索引这三步同时进行，由于indexer无法及时索引解析好的文件，因此必须将解析好的文件缓存起来，比如缓存在内存里。然而内存是紧俏资源，必须限量节约使用。
    285. 关于内存的限量使用，在实现时可以为缓存设定一个阀值，缓存满了就先暂停所有的文件扫描和解析进程，等缓存快没了的时候再继续，在Linux上使用SIGSTOP和SIGCONT信号可以很容易就实现这一功能。相比之下，如何准确的获取缓存对象所占用的内存大小倒是比较困难，折中的办法是统计整个进程的内存占用或是间接的方法，或者干脆通过限制缓存对象的数目来做限制（这个比较弱智的感觉）。
    287. 关于内存的节约使用，大家都知道一般的Python对象都会自动创建一个__dict__属性来存储其他的属性，然而不太广为人知的是，Python的内置类型dict是一个内存大户，当Python对象少的时候可能很难发现，如果在内存里存储十万或一百万个Python对象时，用Memory Profiler（比如Heapy）做下profiling你会发现，光是__dict__本身（不包括存在__dict__里的数据）就能吃掉你巨量的内存。
    289. 通过设置类属性__slots__可以禁止__dict__属性的自动创建，其中一个成功故事在这里，这个哥们通过__slots__节约了9G内存。需要说明的是，__slots__会带来一些负面作用，比较明显的一个是，使用version 0版本的pickle协议序列化定义了__slots__属性的对象会有报错，但使用更高级别的pickle协议则没问题4（一般很少用到cPickle的protocol version 0，因为又慢又占空间)。
    291. 另外缓存所使用的数据结构也比较重要，直接用Python的内置类型list肯定不行，因为缓存应该是一个FIFO的队列，而del(list[0])操作是O(n)的复杂度5，用collections.deque比较合适。
    293. 使用中遇到的问题
    294. 索引文件损坏导致searchd崩溃
    295. 测试时发现搜索部分关键词的时候，searchd会因为断言失败后crash并自动重启。经调试和在网上查资料，发现有个比较大的索引（2G左右）很可能在merge的时候发生了损坏，用indextool --check检查对应的索引后，输出大量的FAILED, row not found错误，目前除了升级sphinx和重建损坏的索引，貌似没有别的解决方法。
    297. socket接收数据超时
    298. 使用sphinxapi.py提供的接口和searchd通讯时，如果索引较大，searchd可能响应较慢，此时很有可能会报socket超时的异常。python里阻塞的socket的默认超时时间是1秒，解决的方法比较简单，直接调用sphinxapi里的SetConnectTimeout函数设置超时即可。
    300. 资源和参考资料
    301. Sphinx 2.0.1 Documentation
    302. Coreseek与第四城搜索，有很多性能相关的测试，很详尽。
    303. Coreseek python数据源接口文档
    304. 脚注
    305. Sphinx indexer program configuration options, mem_limit，引用于2014-04-17。
    306. Wikipedia:Sphinx，引用于2014-04-17。
    307. lxml benchmarks and speed, xpath，引用于2014-04-18。
    308. python pickling slots error，引用于2014-04-18。
    309. Python Time Complexity，引用于2014-04-18。
    310. Restrictions on the source data，引用于2014-09-18。
    312. Linux下编译安装Sphinx、中文分词coreseek及PHP的sphinx扩展
    313. http://www.icultivator.com/p/6347.html
    315. 使用 sphinx 需要做以下几件事
    317. 1.有数据；
    319. 2.建立 sphinx 配置文件；
    321. 3.生成索引；
    323. 4.启动 searchd 服务进程，默认是9312
    325. 5.用 PHP 去连接 sphinx 服务
    327. 启动 sphinx
    329. cd /usr/local/coreseek/bin/
    330. ./searchd
    331. 启动命令
    333. searchd 命令参数介绍：
    335. -c 指定配置文件
    337. --stop 停止服务
    339. --pidfile 用来显式指定一个 PID 文件
    341. -p 指定端口
    343. 5、php 安装 sphinx 扩展
    345. sudo pecl install sphinx
    346. 如果出现错误："configure: error: Cannot find libsphinxclient headers"
    348. 解决方法：
    350. cd coreseek-4.1/csft-4.1/api/libsphinxclient/
    351. ./configure --prefix=/usr/local/libsphinxclient
    352. sudo make && make install
    353. 解决完毕！
    355. 回去接着执行
    357. ./configure --with-php-config=/usr/local/php/bin/php-config --with-sphinx=/usr/local/libsphinxclient
    358. sudo make && make install
    359. 出现类似"Installing shared extensions: /usr/lib/php5/20090626/sphinx.so"，表示成功。
    361. 可以进入该目录下会发现生成了一个 sphinx.so 文件
    363. 在 php.ini 中加载该 so 文件
    365. extension=/usr/lib/php5/20090626/sphinx.so
    367. 重启 apache ，phpinfo() 中出现这个表明成功。
    369. sphinx
    371. <?php
    373. //
    374. // $Id: sphinxapi.php 1566 2008-11-17 19:06:44Z shodan $
    375. //
    377. //
    378. // Copyright (c) 2001-2008, Andrew Aksyonoff. All rights reserved.
    379. //
    380. // This program is free software; you can redistribute it and/or modify
    381. // it under the terms of the GNU General Public License. You should have
    382. // received a copy of the GPL license along with this program; if you
    383. // did not, you can find it at http://www.gnu.org/
    384. //
    386. /////////////////////////////////////////////////////////////////////////////
    387. // PHP version of Sphinx searchd client (PHP API)
    388. /////////////////////////////////////////////////////////////////////////////
    390. /// known searchd commands
    391. define ( "SEARCHD_COMMAND_SEARCH", 0 );
    392. define ( "SEARCHD_COMMAND_EXCERPT", 1 );
    393. define ( "SEARCHD_COMMAND_UPDATE", 2 );
    394. define ( "SEARCHD_COMMAND_KEYWORDS",3 );
    395. define ( "SEARCHD_COMMAND_PERSIST", 4 );
    397. /// current client-side command implementation versions
    398. define ( "VER_COMMAND_SEARCH", 0x116 );
    399. define ( "VER_COMMAND_EXCERPT", 0x100 );
    400. define ( "VER_COMMAND_UPDATE", 0x102 );
    401. define ( "VER_COMMAND_KEYWORDS", 0x100 );
    403. /// known searchd status codes
    404. define ( "SEARCHD_OK", 0 );
    405. define ( "SEARCHD_ERROR", 1 );
    406. define ( "SEARCHD_RETRY", 2 );
    407. define ( "SEARCHD_WARNING", 3 );
    409. /// known match modes
    410. define ( "SPH_MATCH_ALL", 0 );
    411. define ( "SPH_MATCH_ANY", 1 );
    412. define ( "SPH_MATCH_PHRASE", 2 );
    413. define ( "SPH_MATCH_BOOLEAN", 3 );
    414. define ( "SPH_MATCH_EXTENDED", 4 );
    415. define ( "SPH_MATCH_FULLSCAN", 5 );
    416. define ( "SPH_MATCH_EXTENDED2", 6 ); // extended engine V2 (TEMPORARY, WILL BE REMOVED)
    418. /// known ranking modes (ext2 only)
    419. define ( "SPH_RANK_PROXIMITY_BM25", 0 ); ///< default mode, phrase proximity major factor and BM25 minor one
    420. define ( "SPH_RANK_BM25", 1 ); ///< statistical mode, BM25 ranking only (faster but worse quality)
    421. define ( "SPH_RANK_NONE", 2 ); ///< no ranking, all matches get a weight of 1
    422. define ( "SPH_RANK_WORDCOUNT", 3 ); ///< simple word-count weighting, rank is a weighted sum of per-field keyword occurence counts
    423. define ( "SPH_RANK_PROXIMITY", 4 );
    424. define ( "SPH_RANK_MATCHANY", 5 );
    426. /// known sort modes
    427. define ( "SPH_SORT_RELEVANCE", 0 );
    428. define ( "SPH_SORT_ATTR_DESC", 1 );
    429. define ( "SPH_SORT_ATTR_ASC", 2 );
    430. define ( "SPH_SORT_TIME_SEGMENTS", 3 );
    431. define ( "SPH_SORT_EXTENDED", 4 );
    432. define ( "SPH_SORT_EXPR", 5 );
    434. /// known filter types
    435. define ( "SPH_FILTER_VALUES", 0 );
    436. define ( "SPH_FILTER_RANGE", 1 );
    437. define ( "SPH_FILTER_FLOATRANGE", 2 );
    439. /// known attribute types
    440. define ( "SPH_ATTR_INTEGER", 1 );
    441. define ( "SPH_ATTR_TIMESTAMP", 2 );
    442. define ( "SPH_ATTR_ORDINAL", 3 );
    443. define ( "SPH_ATTR_BOOL", 4 );
    444. define ( "SPH_ATTR_FLOAT", 5 );
    445. define ( "SPH_ATTR_BIGINT", 6 );
    446. define ( "SPH_ATTR_MULTI", 0x40000000 );
    448. /// known grouping functions
    449. define ( "SPH_GROUPBY_DAY", 0 );
    450. define ( "SPH_GROUPBY_WEEK", 1 );
    451. define ( "SPH_GROUPBY_MONTH", 2 );
    452. define ( "SPH_GROUPBY_YEAR", 3 );
    453. define ( "SPH_GROUPBY_ATTR", 4 );
    454. define ( "SPH_GROUPBY_ATTRPAIR", 5 );
    456. // important properties of PHP's integers:
    457. // - always signed (one bit short of PHP_INT_SIZE)
    458. // - conversion from string to int is saturated
    459. // - float is double
    460. // - div converts arguments to floats
    461. // - mod converts arguments to ints
    463. // the packing code below works as follows:
    464. // - when we got an int, just pack it
    465. // if performance is a problem, this is the branch users should aim for
    466. //
    467. // - otherwise, we got a number in string form
    468. // this might be due to different reasons, but we assume that this is
    469. // because it didn't fit into PHP int
    470. //
    471. // - factor the string into high and low ints for packing
    472. // - if we have bcmath, then it is used
    473. // - if we don't, we have to do it manually (this is the fun part)
    474. //
    475. // - x64 branch does factoring using ints
    476. // - x32 (ab)uses floats, since we can't fit unsigned 32-bit number into an int
    477. //
    478. // unpacking routines are pretty much the same.
    479. // - return ints if we can
    480. // - otherwise format number into a string
    482. /// pack 64-bit signed
    483. function sphPackI64 ( $v )
    484. {
    485. assert ( is_numeric($v) );
    487. // x64
    488. if ( PHP_INT_SIZE>=8 )
    489. {
    490. $v = (int)$v;
    491. return pack ( "NN", $v>>32, $v&0xFFFFFFFF );
    492. }
    494. // x32, int
    495. if ( is_int($v) )
    496. return pack ( "NN", $v < 0 ? -1 : 0, $v );
    498. // x32, bcmath
    499. if ( function_exists("bcmul") )
    500. {
    501. if ( bccomp ( $v, 0 ) == -1 )
    502. $v = bcadd ( "18446744073709551616", $v );
    503. $h = bcdiv ( $v, "4294967296", 0 );
    504. $l = bcmod ( $v, "4294967296" );
    505. return pack ( "NN", (float)$h, (float)$l ); // conversion to float is intentional; int would lose 31st bit
    506. }
    508. // x32, no-bcmath
    509. $p = max(0, strlen($v) - 13);
    510. $lo = abs((float)substr($v, $p));
    511. $hi = abs((float)substr($v, 0, $p));
    513. $m = $lo + $hi*1316134912.0; // (10 ^ 13) % (1 << 32) = 1316134912
    514. $q = floor($m/4294967296.0);
    515. $l = $m - ($q*4294967296.0);
    516. $h = $hi*2328.0 + $q; // (10 ^ 13) / (1 << 32) = 2328
    518. if ( $v<0 )
    519. {
    520. if ( $l==0 )
    521. $h = 4294967296.0 - $h;
    522. else
    523. {
    524. $h = 4294967295.0 - $h;
    525. $l = 4294967296.0 - $l;
    526. }
    527. }
    528. return pack ( "NN", $h, $l );
    529. }
    531. /// pack 64-bit unsigned
    532. function sphPackU64 ( $v )
    533. {
    534. assert ( is_numeric($v) );
    536. // x64
    537. if ( PHP_INT_SIZE>=8 )
    538. {
    539. assert ( $v>=0 );
    541. // x64, int
    542. if ( is_int($v) )
    543. return pack ( "NN", $v>>32, $v&0xFFFFFFFF );
    545. // x64, bcmath
    546. if ( function_exists("bcmul") )
    547. {
    548. $h = bcdiv ( $v, 4294967296, 0 );
    549. $l = bcmod ( $v, 4294967296 );
    550. return pack ( "NN", $h, $l );
    551. }
    553. // x64, no-bcmath
    554. $p = max ( 0, strlen($v) - 13 );
    555. $lo = (int)substr ( $v, $p );
    556. $hi = (int)substr ( $v, 0, $p );
    558. $m = $lo + $hi*1316134912;
    559. $l = $m % 4294967296;
    560. $h = $hi*2328 + (int)($m/4294967296);
    562. return pack ( "NN", $h, $l );
    563. }
    565. // x32, int
    566. if ( is_int($v) )
    567. return pack ( "NN", 0, $v );
    569. // x32, bcmath
    570. if ( function_exists("bcmul") )
    571. {
    572. $h = bcdiv ( $v, "4294967296", 0 );
    573. $l = bcmod ( $v, "4294967296" );
    574. return pack ( "NN", (float)$h, (float)$l ); // conversion to float is intentional; int would lose 31st bit
    575. }
    577. // x32, no-bcmath
    578. $p = max(0, strlen($v) - 13);
    579. $lo = (float)substr($v, $p);
    580. $hi = (float)substr($v, 0, $p);
    582. $m = $lo + $hi*1316134912.0;
    583. $q = floor($m / 4294967296.0);
    584. $l = $m - ($q * 4294967296.0);
    585. $h = $hi*2328.0 + $q;
    587. return pack ( "NN", $h, $l );
    588. }
    590. // unpack 64-bit unsigned
    591. function sphUnpackU64 ( $v )
    592. {
    593. list ( $hi, $lo ) = array_values ( unpack ( "N*N*", $v ) );
    595. if ( PHP_INT_SIZE>=8 )
    596. {
    597. if ( $hi<0 ) $hi += (1<<32); // because php 5.2.2 to 5.2.5 is totally fucked up again
    598. if ( $lo<0 ) $lo += (1<<32);
    600. // x64, int
    601. if ( $hi<=2147483647 )
    602. return ($hi<<32) + $lo;
    604. // x64, bcmath
    605. if ( function_exists("bcmul") )
    606. return bcadd ( $lo, bcmul ( $hi, "4294967296" ) );
    608. // x64, no-bcmath
    609. $C = 100000;
    610. $h = ((int)($hi / $C) << 32) + (int)($lo / $C);
    611. $l = (($hi % $C) << 32) + ($lo % $C);
    612. if ( $l>$C )
    613. {
    614. $h += (int)($l / $C);
    615. $l = $l % $C;
    616. }
    618. if ( $h==0 )
    619. return $l;
    620. return sprintf ( "%d%05d", $h, $l );
    621. }
    623. // x32, int
    624. if ( $hi==0 )
    625. {
    626. if ( $lo>0 )
    627. return $lo;
    628. return sprintf ( "%u", $lo );
    629. }
    631. $hi = sprintf ( "%u", $hi );
    632. $lo = sprintf ( "%u", $lo );
    634. // x32, bcmath
    635. if ( function_exists("bcmul") )
    636. return bcadd ( $lo, bcmul ( $hi, "4294967296" ) );
    638. // x32, no-bcmath
    639. $hi = (float)$hi;
    640. $lo = (float)$lo;
    642. $q = floor($hi/10000000.0);
    643. $r = $hi - $q*10000000.0;
    644. $m = $lo + $r*4967296.0;
    645. $mq = floor($m/10000000.0);
    646. $l = $m - $mq*10000000.0;
    647. $h = $q*4294967296.0 + $r*429.0 + $mq;
    649. $h = sprintf ( "%.0f", $h );
    650. $l = sprintf ( "%07.0f", $l );
    651. if ( $h=="0" )
    652. return sprintf( "%.0f", (float)$l );
    653. return $h . $l;
    654. }
    656. // unpack 64-bit signed
    657. function sphUnpackI64 ( $v )
    658. {
    659. list ( $hi, $lo ) = array_values ( unpack ( "N*N*", $v ) );
    661. // x64
    662. if ( PHP_INT_SIZE>=8 )
    663. {
    664. if ( $hi<0 ) $hi += (1<<32); // because php 5.2.2 to 5.2.5 is totally fucked up again
    665. if ( $lo<0 ) $lo += (1<<32);
    667. return ($hi<<32) + $lo;
    668. }
    670. // x32, int
    671. if ( $hi==0 )
    672. {
    673. if ( $lo>0 )
    674. return $lo;
    675. return sprintf ( "%u", $lo );
    676. }
    677. // x32, int
    678. elseif ( $hi==-1 )
    679. {
    680. if ( $lo<0 )
    681. return $lo;
    682. return sprintf ( "%.0f", $lo - 4294967296.0 );
    683. }
    685. $neg = "";
    686. $c = 0;
    687. if ( $hi<0 )
    688. {
    689. $hi = ~$hi;
    690. $lo = ~$lo;
    691. $c = 1;
    692. $neg = "-";
    693. }
    695. $hi = sprintf ( "%u", $hi );
    696. $lo = sprintf ( "%u", $lo );
    698. // x32, bcmath
    699. if ( function_exists("bcmul") )
    700. return $neg . bcadd ( bcadd ( $lo, bcmul ( $hi, "4294967296" ) ), $c );
    702. // x32, no-bcmath
    703. $hi = (float)$hi;
    704. $lo = (float)$lo;
    706. $q = floor($hi/10000000.0);
    707. $r = $hi - $q*10000000.0;
    708. $m = $lo + $r*4967296.0;
    709. $mq = floor($m/10000000.0);
    710. $l = $m - $mq*10000000.0 + $c;
    711. $h = $q*4294967296.0 + $r*429.0 + $mq;
    713. $h = sprintf ( "%.0f", $h );
    714. $l = sprintf ( "%07.0f", $l );
    715. if ( $h=="0" )
    716. return $neg . sprintf( "%.0f", (float)$l );
    717. return $neg . $h . $l;
    718. }
    720. /// sphinx searchd client class
    721. class SphinxClient
    722. {
    723. var $_host; ///< searchd host (default is "localhost")
    724. var $_port; ///< searchd port (default is 3312)
    725. var $_offset; ///< how many records to seek from result-set start (default is 0)
    726. var $_limit; ///< how many records to return from result-set starting at offset (default is 20)
    727. var $_mode; ///< query matching mode (default is SPH_MATCH_ALL)
    728. var $_weights; ///< per-field weights (default is 1 for all fields)
    729. var $_sort; ///< match sorting mode (default is SPH_SORT_RELEVANCE)
    730. var $_sortby; ///< attribute to sort by (defualt is "")
    731. var $_min_id; ///< min ID to match (default is 0, which means no limit)
    732. var $_max_id; ///< max ID to match (default is 0, which means no limit)
    733. var $_filters; ///< search filters
    734. var $_groupby; ///< group-by attribute name
    735. var $_groupfunc; ///< group-by function (to pre-process group-by attribute value with)
    736. var $_groupsort; ///< group-by sorting clause (to sort groups in result set with)
    737. var $_groupdistinct;///< group-by count-distinct attribute
    738. var $_maxmatches; ///< max matches to retrieve
    739. var $_cutoff; ///< cutoff to stop searching at (default is 0)
    740. var $_retrycount; ///< distributed retries count
    741. var $_retrydelay; ///< distributed retries delay
    742. var $_anchor; ///< geographical anchor point
    743. var $_indexweights; ///< per-index weights
    744. var $_ranker; ///< ranking mode (default is SPH_RANK_PROXIMITY_BM25)
    745. var $_maxquerytime; ///< max query time, milliseconds (default is 0, do not limit)
    746. var $_fieldweights; ///< per-field-name weights
    747. var $_overrides; ///< per-query attribute values overrides
    748. var $_select; ///< select-list (attributes or expressions, with optional aliases)
    750. var $_error; ///< last error message
    751. var $_warning; ///< last warning message
    752. var $_connerror; ///< connection error vs remote error flag
    754. var $_reqs; ///< requests array for multi-query
    755. var $_mbenc; ///< stored mbstring encoding
    756. var $_arrayresult; ///< whether $result["matches"] should be a hash or an array
    757. var $_timeout; ///< connect timeout
    759. /////////////////////////////////////////////////////////////////////////////
    760. // common stuff
    761. /////////////////////////////////////////////////////////////////////////////
    763. /// create a new client object and fill defaults
    764. function SphinxClient ()
    765. {
    766. // per-client-object settings
    767. $this->_host = "localhost";
    768. $this->_port = 3312;
    769. $this->_path = false;
    770. $this->_socket = false;
    772. // per-query settings
    773. $this->_offset = 0;
    774. $this->_limit = 20;
    775. $this->_mode = SPH_MATCH_ALL;
    776. $this->_weights = array ();
    777. $this->_sort = SPH_SORT_RELEVANCE;
    778. $this->_sortby = "";
    779. $this->_min_id = 0;
    780. $this->_max_id = 0;
    781. $this->_filters = array ();
    782. $this->_groupby = "";
    783. $this->_groupfunc = SPH_GROUPBY_DAY;
    784. $this->_groupsort = "@group desc";
    785. $this->_groupdistinct= "";
    786. $this->_maxmatches = 1000;
    787. $this->_cutoff = 0;
    788. $this->_retrycount = 0;
    789. $this->_retrydelay = 0;
    790. $this->_anchor = array ();
    791. $this->_indexweights= array ();
    792. $this->_ranker = SPH_RANK_PROXIMITY_BM25;
    793. $this->_maxquerytime= 0;
    794. $this->_fieldweights= array();
    795. $this->_overrides = array();
    796. $this->_select = "*";
    798. $this->_error = ""; // per-reply fields (for single-query case)
    799. $this->_warning = "";
    800. $this->_connerror = false;
    802. $this->_reqs = array (); // requests storage (for multi-query case)
    803. $this->_mbenc = "";
    804. $this->_arrayresult = false;
    805. $this->_timeout = 0;
    806. }
    808. function __destruct()
    809. {
    810. if ( $this->_socket !== false )
    811. fclose ( $this->_socket );
    812. }
    814. /// get last error message (string)
    815. function GetLastError ()
    816. {
    817. return $this->_error;
    818. }
    820. /// get last warning message (string)
    821. function GetLastWarning ()
    822. {
    823. return $this->_warning;
    824. }
    826. /// get last error flag (to tell network connection errors from searchd errors or broken responses)
    827. function IsConnectError()
    828. {
    829. return $this->_connerror;
    830. }
    832. /// set searchd host name (string) and port (integer)
    833. function SetServer ( $host, $port = 0 )
    834. {
    835. assert ( is_string($host) );
    836. if ( $host[0] == '/')
    837. {
    838. $this->_path = 'unix://' . $host;
    839. return;
    840. }
    841. if ( substr ( $host, 0, 7 )=="unix://" )
    842. {
    843. $this->_path = $host;
    844. return;
    845. }
    847. assert ( is_int($port) );
    848. $this->_host = $host;
    849. $this->_port = $port;
    850. $this->_path = '';
    852. }
    854. /// set server connection timeout (0 to remove)
    855. function SetConnectTimeout ( $timeout )
    856. {
    857. assert ( is_numeric($timeout) );
    858. $this->_timeout = $timeout;
    859. }
    861. function _Send ( $handle, $data, $length )
    862. {
    863. if ( feof($handle) || fwrite ( $handle, $data, $length ) !== $length )
    864. {
    865. $this->_error = 'connection unexpectedly closed (timed out?)';
    866. $this->_connerror = true;
    867. return false;
    868. }
    869. return true;
    870. }
    872. /////////////////////////////////////////////////////////////////////////////
    874. /// enter mbstring workaround mode
    875. function _MBPush ()
    876. {
    877. $this->_mbenc = "";
    878. if ( ini_get ( "mbstring.func_overload" ) & 2 )
    879. {
    880. $this->_mbenc = mb_internal_encoding();
    881. mb_internal_encoding ( "latin1" );
    882. }
    883. }
    885. /// leave mbstring workaround mode
    886. function _MBPop ()
    887. {
    888. if ( $this->_mbenc )
    889. mb_internal_encoding ( $this->_mbenc );
    890. }
    892. /// connect to searchd server
    893. function _Connect ()
    894. {
    895. if ( $this->_socket !== false )
    896. return $this->_socket;
    898. $errno = 0;
    899. $errstr = "";
    900. $this->_connerror = false;
    902. if ( $this->_path )
    903. {
    904. $host = $this->_path;
    905. $port = 0;
    906. }
    907. else
    908. {
    909. $host = $this->_host;
    910. $port = $this->_port;
    911. }
    913. if ( $this->_timeout<=0 )
    914. $fp = @fsockopen ( $host, $port, $errno, $errstr );
    915. else
    916. $fp = @fsockopen ( $host, $port, $errno, $errstr, $this->_timeout );
    918. if ( !$fp )
    919. {
    920. if ( $this->_path )
    921. $location = $this->_path;
    922. else
    923. $location = "{$this->_host}:{$this->_port}";
    925. $errstr = trim ( $errstr );
    926. $this->_error = "connection to $location failed (errno=$errno, msg=$errstr)";
    927. $this->_connerror = true;
    928. return false;
    929. }
    931. // check version
    932. list(,$v) = unpack ( "N*", fread ( $fp, 4 ) );
    933. $v = (int)$v;
    934. if ( $v<1 )
    935. {
    936. fclose ( $fp );
    937. $this->_error = "expected searchd protocol version 1+, got version '$v'";
    938. return false;
    939. }
    941. // all ok, send my version
    942. if ( !$this->_Send ( $fp, pack ( "N", 1 ), 4 ) )
    943. return false;
    944. return $fp;
    945. }
    947. /// get and check response packet from searchd server
    948. function _GetResponse ( $fp, $client_ver )
    949. {
    950. $response = "";
    951. $len = 0;
    953. $header = fread ( $fp, 8 );
    954. if ( strlen($header)==8 )
    955. {
    956. list ( $status, $ver, $len ) = array_values ( unpack ( "n2a/Nb", $header ) );
    957. $left = $len;
    958. while ( $left>0 && !feof($fp) )
    959. {
    960. $chunk = fread ( $fp, $left );
    961. if ( $chunk )
    962. {
    963. $response .= $chunk;
    964. $left -= strlen($chunk);
    965. }
    966. }
    967. }
    968. if ( $this->_socket === false )
    969. fclose ( $fp );
    971. // check response
    972. $read = strlen ( $response );
    973. if ( !$response || $read!=$len )
    974. {
    975. $this->_error = $len
    976. ? "failed to read searchd response (status=$status, ver=$ver, len=$len, read=$read)"
    977. : "received zero-sized searchd response";
    978. return false;
    979. }
    981. // check status
    982. if ( $status==SEARCHD_WARNING )
    983. {
    984. list(,$wlen) = unpack ( "N*", substr ( $response, 0, 4 ) );
    985. $this->_warning = substr ( $response, 4, $wlen );
    986. return substr ( $response, 4+$wlen );
    987. }
    988. if ( $status==SEARCHD_ERROR )
    989. {
    990. $this->_error = "searchd error: " . substr ( $response, 4 );
    991. return false;
    992. }
    993. if ( $status==SEARCHD_RETRY )
    994. {
    995. $this->_error = "temporary searchd error: " . substr ( $response, 4 );
    996. return false;
    997. }
    998. if ( $status!=SEARCHD_OK )
    999. {
    1000. $this->_error = "unknown status code '$status'";
    1001. return false;
    1002. }
    1004. // check version
    1005. if ( $ver<$client_ver )
    1006. {
    1007. $this->_warning = sprintf ( "searchd command v.%d.%d older than client's v.%d.%d, some options might not work",
    1008. $ver>>8, $ver&0xff, $client_ver>>8, $client_ver&0xff );
    1009. }
    1011. return $response;
    1012. }
    1014. /////////////////////////////////////////////////////////////////////////////
    1015. // searching
    1016. /////////////////////////////////////////////////////////////////////////////
    1018. /// set offset and count into result set,
    1019. /// and optionally set max-matches and cutoff limits
    1020. function SetLimits ( $offset, $limit, $max=0, $cutoff=0 )
    1021. {
    1022. assert ( is_int($offset) );
    1023. assert ( is_int($limit) );
    1024. assert ( $offset>=0 );
    1025. assert ( $limit>0 );
    1026. assert ( $max>=0 );
    1027. $this->_offset = $offset;
    1028. $this->_limit = $limit;
    1029. if ( $max>0 )
    1030. $this->_maxmatches = $max;
    1031. if ( $cutoff>0 )
    1032. $this->_cutoff = $cutoff;
    1033. }
    1035. /// set maximum query time, in milliseconds, per-index
    1036. /// integer, 0 means "do not limit"
    1037. function SetMaxQueryTime ( $max )
    1038. {
    1039. assert ( is_int($max) );
    1040. assert ( $max>=0 );
    1041. $this->_maxquerytime = $max;
    1042. }
    1044. /// set matching mode
    1045. function SetMatchMode ( $mode )
    1046. {
    1047. assert ( $mode==SPH_MATCH_ALL
    1048. || $mode==SPH_MATCH_ANY
    1049. || $mode==SPH_MATCH_PHRASE
    1050. || $mode==SPH_MATCH_BOOLEAN
    1051. || $mode==SPH_MATCH_EXTENDED
    1052. || $mode==SPH_MATCH_FULLSCAN
    1053. || $mode==SPH_MATCH_EXTENDED2 );
    1054. $this->_mode = $mode;
    1055. }
    1057. /// set ranking mode
    1058. function SetRankingMode ( $ranker )
    1059. {
    1060. assert ( $ranker==SPH_RANK_PROXIMITY_BM25
    1061. || $ranker==SPH_RANK_BM25
    1062. || $ranker==SPH_RANK_NONE
    1063. || $ranker==SPH_RANK_WORDCOUNT
    1064. || $ranker==SPH_RANK_PROXIMITY );
    1065. $this->_ranker = $ranker;
    1066. }
    1068. /// set matches sorting mode
    1069. function SetSortMode ( $mode, $sortby="" )
    1070. {
    1071. assert (
    1072. $mode==SPH_SORT_RELEVANCE ||
    1073. $mode==SPH_SORT_ATTR_DESC ||
    1074. $mode==SPH_SORT_ATTR_ASC ||
    1075. $mode==SPH_SORT_TIME_SEGMENTS ||
    1076. $mode==SPH_SORT_EXTENDED ||
    1077. $mode==SPH_SORT_EXPR );
    1078. assert ( is_string($sortby) );
    1079. assert ( $mode==SPH_SORT_RELEVANCE || strlen($sortby)>0 );
    1081. $this->_sort = $mode;
    1082. $this->_sortby = $sortby;
    1083. }
    1085. /// bind per-field weights by order
    1086. /// DEPRECATED; use SetFieldWeights() instead
    1087. function SetWeights ( $weights )
    1088. {
    1089. assert ( is_array($weights) );
    1090. foreach ( $weights as $weight )
    1091. assert ( is_int($weight) );
    1093. $this->_weights = $weights;
    1094. }
    1096. /// bind per-field weights by name
    1097. function SetFieldWeights ( $weights )
    1098. {
    1099. assert ( is_array($weights) );
    1100. foreach ( $weights as $name=>$weight )
    1101. {
    1102. assert ( is_string($name) );
    1103. assert ( is_int($weight) );
    1104. }
    1105. $this->_fieldweights = $weights;
    1106. }
    1108. /// bind per-index weights by name
    1109. function SetIndexWeights ( $weights )
    1110. {
    1111. assert ( is_array($weights) );
    1112. foreach ( $weights as $index=>$weight )
    1113. {
    1114. assert ( is_string($index) );
    1115. assert ( is_int($weight) );
    1116. }
    1117. $this->_indexweights = $weights;
    1118. }
    1120. /// set IDs range to match
    1121. /// only match records if document ID is beetwen $min and $max (inclusive)
    1122. function SetIDRange ( $min, $max )
    1123. {
    1124. assert ( is_numeric($min) );
    1125. assert ( is_numeric($max) );
    1126. assert ( $min<=$max );
    1127. $this->_min_id = $min;
    1128. $this->_max_id = $max;
    1129. }
    1131. /// set values set filter
    1132. /// only match records where $attribute value is in given set
    1133. function SetFilter ( $attribute, $values, $exclude=false )
    1134. {
    1135. assert ( is_string($attribute) );
    1136. assert ( is_array($values) );
    1137. assert ( count($values) );
    1139. if ( is_array($values) && count($values) )
    1140. {
    1141. foreach ( $values as $value )
    1142. assert ( is_numeric($value) );
    1144. $this->_filters[] = array ( "type"=>SPH_FILTER_VALUES, "attr"=>$attribute, "exclude"=>$exclude, "values"=>$values );
    1145. }
    1146. }
    1148. /// set range filter
    1149. /// only match records if $attribute value is beetwen $min and $max (inclusive)
    1150. function SetFilterRange ( $attribute, $min, $max, $exclude=false )
    1151. {
    1152. assert ( is_string($attribute) );
    1153. assert ( is_numeric($min) );
    1154. assert ( is_numeric($max) );
    1155. assert ( $min<=$max );
    1157. $this->_filters[] = array ( "type"=>SPH_FILTER_RANGE, "attr"=>$attribute, "exclude"=>$exclude, "min"=>$min, "max"=>$max );
    1158. }
    1160. /// set float range filter
    1161. /// only match records if $attribute value is beetwen $min and $max (inclusive)
    1162. function SetFilterFloatRange ( $attribute, $min, $max, $exclude=false )
    1163. {
    1164. assert ( is_string($attribute) );
    1165. assert ( is_float($min) );
    1166. assert ( is_float($max) );
    1167. assert ( $min<=$max );
    1169. $this->_filters[] = array ( "type"=>SPH_FILTER_FLOATRANGE, "attr"=>$attribute, "exclude"=>$exclude, "min"=>$min, "max"=>$max );
    1170. }
    1172. /// setup anchor point for geosphere distance calculations
    1173. /// required to use @geodist in filters and sorting
    1174. /// latitude and longitude must be in radians
    1175. function SetGeoAnchor ( $attrlat, $attrlong, $lat, $long )
    1176. {
    1177. assert ( is_string($attrlat) );
    1178. assert ( is_string($attrlong) );
    1179. assert ( is_float($lat) );
    1180. assert ( is_float($long) );
    1182. $this->_anchor = array ( "attrlat"=>$attrlat, "attrlong"=>$attrlong, "lat"=>$lat, "long"=>$long );
    1183. }
    1185. /// set grouping attribute and function
    1186. function SetGroupBy ( $attribute, $func, $groupsort="@group desc" )
    1187. {
    1188. assert ( is_string($attribute) );
    1189. assert ( is_string($groupsort) );
    1190. assert ( $func==SPH_GROUPBY_DAY
    1191. || $func==SPH_GROUPBY_WEEK
    1192. || $func==SPH_GROUPBY_MONTH
    1193. || $func==SPH_GROUPBY_YEAR
    1194. || $func==SPH_GROUPBY_ATTR
    1195. || $func==SPH_GROUPBY_ATTRPAIR );
    1197. $this->_groupby = $attribute;
    1198. $this->_groupfunc = $func;
    1199. $this->_groupsort = $groupsort;
    1200. }
    1202. /// set count-distinct attribute for group-by queries
    1203. function SetGroupDistinct ( $attribute )
    1204. {
    1205. assert ( is_string($attribute) );
    1206. $this->_groupdistinct = $attribute;
    1207. }
    1209. /// set distributed retries count and delay
    1210. function SetRetries ( $count, $delay=0 )
    1211. {
    1212. assert ( is_int($count) && $count>=0 );
    1213. assert ( is_int($delay) && $delay>=0 );
    1214. $this->_retrycount = $count;
    1215. $this->_retrydelay = $delay;
    1216. }
    1218. /// set result set format (hash or array; hash by default)
    1219. /// PHP specific; needed for group-by-MVA result sets that may contain duplicate IDs
    1220. function SetArrayResult ( $arrayresult )
    1221. {
    1222. assert ( is_bool($arrayresult) );
    1223. $this->_arrayresult = $arrayresult;
    1224. }
    1226. /// set attribute values override
    1227. /// there can be only one override per attribute
    1228. /// $values must be a hash that maps document IDs to attribute values
    1229. function SetOverride ( $attrname, $attrtype, $values )
    1230. {
    1231. assert ( is_string ( $attrname ) );
    1232. assert ( in_array ( $attrtype, array ( SPH_ATTR_INTEGER, SPH_ATTR_TIMESTAMP, SPH_ATTR_BOOL, SPH_ATTR_FLOAT, SPH_ATTR_BIGINT ) ) );
    1233. assert ( is_array ( $values ) );
    1235. $this->_overrides[$attrname] = array ( "attr"=>$attrname, "type"=>$attrtype, "values"=>$values );
    1236. }
    1238. /// set select-list (attributes or expressions), SQL-like syntax
    1239. function SetSelect ( $select )
    1240. {
    1241. assert ( is_string ( $select ) );
    1242. $this->_select = $select;
    1243. }
    1245. //////////////////////////////////////////////////////////////////////////////
    1247. /// clear all filters (for multi-queries)
    1248. function ResetFilters ()
    1249. {
    1250. $this->_filters = array();
    1251. $this->_anchor = array();
    1252. }
    1254. /// clear groupby settings (for multi-queries)
    1255. function ResetGroupBy ()
    1256. {
    1257. $this->_groupby = "";
    1258. $this->_groupfunc = SPH_GROUPBY_DAY;
    1259. $this->_groupsort = "@group desc";
    1260. $this->_groupdistinct= "";
    1261. }
    1263. /// clear all attribute value overrides (for multi-queries)
    1264. function ResetOverrides ()
    1265. {
    1266. $this->_overrides = array ();
    1267. }
    1269. //////////////////////////////////////////////////////////////////////////////
    1271. /// connect to searchd server, run given search query through given indexes,
    1272. /// and return the search results
    1273. function Query ( $query, $index="*", $comment="" )
    1274. {
    1275. assert ( empty($this->_reqs) );
    1277. $this->AddQuery ( $query, $index, $comment );
    1278. $results = $this->RunQueries ();
    1279. $this->_reqs = array (); // just in case it failed too early
    1281. if ( !is_array($results) )
    1282. return false; // probably network error; error message should be already filled
    1284. $this->_error = $results[0]["error"];
    1285. $this->_warning = $results[0]["warning"];
    1286. if ( $results[0]["status"]==SEARCHD_ERROR )
    1287. return false;
    1288. else
    1289. return $results[0];
    1290. }
    1292. /// helper to pack floats in network byte order
    1293. function _PackFloat ( $f )
    1294. {
    1295. $t1 = pack ( "f", $f ); // machine order
    1296. list(,$t2) = unpack ( "L*", $t1 ); // int in machine order
    1297. return pack ( "N", $t2 );
    1298. }
    1300. /// add query to multi-query batch
    1301. /// returns index into results array from RunQueries() call
    1302. function AddQuery ( $query, $index="*", $comment="" )
    1303. {
    1304. // mbstring workaround
    1305. $this->_MBPush ();
    1307. // build request
    1308. $req = pack ( "NNNNN", $this->_offset, $this->_limit, $this->_mode, $this->_ranker, $this->_sort ); // mode and limits
    1309. $req .= pack ( "N", strlen($this->_sortby) ) . $this->_sortby;
    1310. $req .= pack ( "N", strlen($query) ) . $query; // query itself
    1311. $req .= pack ( "N", count($this->_weights) ); // weights
    1312. foreach ( $this->_weights as $weight )
    1313. $req .= pack ( "N", (int)$weight );
    1314. $req .= pack ( "N", strlen($index) ) . $index; // indexes
    1315. $req .= pack ( "N", 1 ); // id64 range marker
    1316. $req .= sphPackU64 ( $this->_min_id ) . sphPackU64 ( $this->_max_id ); // id64 range
    1318. // filters
    1319. $req .= pack ( "N", count($this->_filters) );
    1320. foreach ( $this->_filters as $filter )
    1321. {
    1322. $req .= pack ( "N", strlen($filter["attr"]) ) . $filter["attr"];
    1323. $req .= pack ( "N", $filter["type"] );
    1324. switch ( $filter["type"] )
    1325. {
    1326. case SPH_FILTER_VALUES:
    1327. $req .= pack ( "N", count($filter["values"]) );
    1328. foreach ( $filter["values"] as $value )
    1329. $req .= sphPackI64 ( $value );
    1330. break;
    1332. case SPH_FILTER_RANGE:
    1333. $req .= sphPackI64 ( $filter["min"] ) . sphPackI64 ( $filter["max"] );
    1334. break;
    1336. case SPH_FILTER_FLOATRANGE:
    1337. $req .= $this->_PackFloat ( $filter["min"] ) . $this->_PackFloat ( $filter["max"] );
    1338. break;
    1340. default:
    1341. assert ( 0 && "internal error: unhandled filter type" );
    1342. }
    1343. $req .= pack ( "N", $filter["exclude"] );
    1344. }
    1346. // group-by clause, max-matches count, group-sort clause, cutoff count
    1347. $req .= pack ( "NN", $this->_groupfunc, strlen($this->_groupby) ) . $this->_groupby;
    1348. $req .= pack ( "N", $this->_maxmatches );
    1349. $req .= pack ( "N", strlen($this->_groupsort) ) . $this->_groupsort;
    1350. $req .= pack ( "NNN", $this->_cutoff, $this->_retrycount, $this->_retrydelay );
    1351. $req .= pack ( "N", strlen($this->_groupdistinct) ) . $this->_groupdistinct;
    1353. // anchor point
    1354. if ( empty($this->_anchor) )
    1355. {
    1356. $req .= pack ( "N", 0 );
    1357. } else
    1358. {
    1359. $a =& $this->_anchor;
    1360. $req .= pack ( "N", 1 );
    1361. $req .= pack ( "N", strlen($a["attrlat"]) ) . $a["attrlat"];
    1362. $req .= pack ( "N", strlen($a["attrlong"]) ) . $a["attrlong"];
    1363. $req .= $this->_PackFloat ( $a["lat"] ) . $this->_PackFloat ( $a["long"] );
    1364. }
    1366. // per-index weights
    1367. $req .= pack ( "N", count($this->_indexweights) );
    1368. foreach ( $this->_indexweights as $idx=>$weight )
    1369. $req .= pack ( "N", strlen($idx) ) . $idx . pack ( "N", $weight );
    1371. // max query time
    1372. $req .= pack ( "N", $this->_maxquerytime );
    1374. // per-field weights
    1375. $req .= pack ( "N", count($this->_fieldweights) );
    1376. foreach ( $this->_fieldweights as $field=>$weight )
    1377. $req .= pack ( "N", strlen($field) ) . $field . pack ( "N", $weight );
    1379. // comment
    1380. $req .= pack ( "N", strlen($comment) ) . $comment;
    1382. // attribute overrides
    1383. $req .= pack ( "N", count($this->_overrides) );
    1384. foreach ( $this->_overrides as $key => $entry )
    1385. {
    1386. $req .= pack ( "N", strlen($entry["attr"]) ) . $entry["attr"];
    1387. $req .= pack ( "NN", $entry["type"], count($entry["values"]) );
    1388. foreach ( $entry["values"] as $id=>$val )
    1389. {
    1390. assert ( is_numeric($id) );
    1391. assert ( is_numeric($val) );
    1393. $req .= sphPackU64 ( $id );
    1394. switch ( $entry["type"] )
    1395. {
    1396. case SPH_ATTR_FLOAT: $req .= $this->_PackFloat ( $val ); break;
    1397. case SPH_ATTR_BIGINT: $req .= sphPackI64 ( $val ); break;
    1398. default: $req .= pack ( "N", $val ); break;
    1399. }
    1400. }
    1401. }
    1403. // select-list
    1404. $req .= pack ( "N", strlen($this->_select) ) . $this->_select;
    1406. // mbstring workaround
    1407. $this->_MBPop ();
    1409. // store request to requests array
    1410. $this->_reqs[] = $req;
    1411. return count($this->_reqs)-1;
    1412. }
    1414. /// connect to searchd, run queries batch, and return an array of result sets
    1415. function RunQueries ()
    1416. {
    1417. if ( empty($this->_reqs) )
    1418. {
    1419. $this->_error = "no queries defined, issue AddQuery() first";
    1420. return false;
    1421. }
    1423. // mbstring workaround
    1424. $this->_MBPush ();
    1426. if (!( $fp = $this->_Connect() ))
    1427. {
    1428. $this->_MBPop ();
    1429. return false;
    1430. }
    1432. ////////////////////////////
    1433. // send query, get response
    1434. ////////////////////////////
    1436. $nreqs = count($this->_reqs);
    1437. $req = join ( "", $this->_reqs );
    1438. $len = 4+strlen($req);
    1439. $req = pack ( "nnNN", SEARCHD_COMMAND_SEARCH, VER_COMMAND_SEARCH, $len, $nreqs ) . $req; // add header
    1441. if ( !( $this->_Send ( $fp, $req, $len+8 ) ) ||
    1442. !( $response = $this->_GetResponse ( $fp, VER_COMMAND_SEARCH ) ) )
    1443. {
    1444. $this->_MBPop ();
    1445. return false;
    1446. }
    1448. $this->_reqs = array ();
    1450. //////////////////
    1451. // parse response
    1452. //////////////////
    1454. $p = 0; // current position
    1455. $max = strlen($response); // max position for checks, to protect against broken responses
    1457. $results = array ();
    1458. for ( $ires=0; $ires<$nreqs && $p<$max; $ires++ )
    1459. {
    1460. $results[] = array();
    1461. $result =& $results[$ires];
    1463. $result["error"] = "";
    1464. $result["warning"] = "";
    1466. // extract status
    1467. list(,$status) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1468. $result["status"] = $status;
    1469. if ( $status!=SEARCHD_OK )
    1470. {
    1471. list(,$len) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1472. $message = substr ( $response, $p, $len ); $p += $len;
    1474. if ( $status==SEARCHD_WARNING )
    1475. {
    1476. $result["warning"] = $message;
    1477. } else
    1478. {
    1479. $result["error"] = $message;
    1480. continue;
    1481. }
    1482. }
    1484. // read schema
    1485. $fields = array ();
    1486. $attrs = array ();
    1488. list(,$nfields) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1489. while ( $nfields-->0 && $p<$max )
    1490. {
    1491. list(,$len) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1492. $fields[] = substr ( $response, $p, $len ); $p += $len;
    1493. }
    1494. $result["fields"] = $fields;
    1496. list(,$nattrs) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1497. while ( $nattrs-->0 && $p<$max )
    1498. {
    1499. list(,$len) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1500. $attr = substr ( $response, $p, $len ); $p += $len;
    1501. list(,$type) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1502. $attrs[$attr] = $type;
    1503. }
    1504. $result["attrs"] = $attrs;
    1506. // read match count
    1507. list(,$count) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1508. list(,$id64) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1510. // read matches
    1511. $idx = -1;
    1512. while ( $count-->0 && $p<$max )
    1513. {
    1514. // index into result array
    1515. $idx++;
    1517. // parse document id and weight
    1518. if ( $id64 )
    1519. {
    1520. $doc = sphUnpackU64 ( substr ( $response, $p, 8 ) ); $p += 8;
    1521. list(,$weight) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1522. }
    1523. else
    1524. {
    1525. list ( $doc, $weight ) = array_values ( unpack ( "N*N*",
    1526. substr ( $response, $p, 8 ) ) );
    1527. $p += 8;
    1529. if ( PHP_INT_SIZE>=8 )
    1530. {
    1531. // x64 route, workaround broken unpack() in 5.2.2+
    1532. if ( $doc<0 ) $doc += (1<<32);
    1533. } else
    1534. {
    1535. // x32 route, workaround php signed/unsigned braindamage
    1536. $doc = sprintf ( "%u", $doc );
    1537. }
    1538. }
    1539. $weight = sprintf ( "%u", $weight );
    1541. // create match entry
    1542. if ( $this->_arrayresult )
    1543. $result["matches"][$idx] = array ( "id"=>$doc, "weight"=>$weight );
    1544. else
    1545. $result["matches"][$doc]["weight"] = $weight;
    1547. // parse and create attributes
    1548. $attrvals = array ();
    1549. foreach ( $attrs as $attr=>$type )
    1550. {
    1551. // handle 64bit ints
    1552. if ( $type==SPH_ATTR_BIGINT )
    1553. {
    1554. $attrvals[$attr] = sphUnpackI64 ( substr ( $response, $p, 8 ) ); $p += 8;
    1555. continue;
    1556. }
    1558. // handle floats
    1559. if ( $type==SPH_ATTR_FLOAT )
    1560. {
    1561. list(,$uval) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1562. list(,$fval) = unpack ( "f*", pack ( "L", $uval ) );
    1563. $attrvals[$attr] = $fval;
    1564. continue;
    1565. }
    1567. // handle everything else as unsigned ints
    1568. list(,$val) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1569. if ( $type & SPH_ATTR_MULTI )
    1570. {
    1571. $attrvals[$attr] = array ();
    1572. $nvalues = $val;
    1573. while ( $nvalues-->0 && $p<$max )
    1574. {
    1575. list(,$val) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1576. $attrvals[$attr][] = sprintf ( "%u", $val );
    1577. }
    1578. } else
    1579. {
    1580. $attrvals[$attr] = sprintf ( "%u", $val );
    1581. }
    1582. }
    1584. if ( $this->_arrayresult )
    1585. $result["matches"][$idx]["attrs"] = $attrvals;
    1586. else
    1587. $result["matches"][$doc]["attrs"] = $attrvals;
    1588. }
    1590. list ( $total, $total_found, $msecs, $words ) =
    1591. array_values ( unpack ( "N*N*N*N*", substr ( $response, $p, 16 ) ) );
    1592. $result["total"] = sprintf ( "%u", $total );
    1593. $result["total_found"] = sprintf ( "%u", $total_found );
    1594. $result["time"] = sprintf ( "%.3f", $msecs/1000 );
    1595. $p += 16;
    1597. while ( $words-->0 && $p<$max )
    1598. {
    1599. list(,$len) = unpack ( "N*", substr ( $response, $p, 4 ) ); $p += 4;
    1600. $word = substr ( $response, $p, $len ); $p += $len;
    1601. list ( $docs, $hits ) = array_values ( unpack ( "N*N*", substr ( $response, $p, 8 ) ) ); $p += 8;
    1602. $result["words"][$word] = array (
    1603. "docs"=>sprintf ( "%u", $docs ),
    1604. "hits"=>sprintf ( "%u", $hits ) );
    1605. }
    1606. }
    1608. $this->_MBPop ();
    1609. return $results;
    1610. }
    1612. /////////////////////////////////////////////////////////////////////////////
    1613. // excerpts generation
    1614. /////////////////////////////////////////////////////////////////////////////
    1616. /// connect to searchd server, and generate exceprts (snippets)
    1617. /// of given documents for given query. returns false on failure,
    1618. /// an array of snippets on success
    1619. function BuildExcerpts ( $docs, $index, $words, $opts=array() )
    1620. {
    1621. assert ( is_array($docs) );
    1622. assert ( is_string($index) );
    1623. assert ( is_string($words) );
    1624. assert ( is_array($opts) );
    1626. $this->_MBPush ();
    1628. if (!( $fp = $this->_Connect() ))
    1629. {
    1630. $this->_MBPop();
    1631. return false;
    1632. }
    1634. /////////////////
    1635. // fixup options
    1636. /////////////////
    1638. if ( !isset($opts["before_match"]) ) $opts["before_match"] = "<b>";
    1639. if ( !isset($opts["after_match"]) ) $opts["after_match"] = "</b>";
    1640. if ( !isset($opts["chunk_separator"]) ) $opts["chunk_separator"] = " ... ";
    1641. if ( !isset($opts["limit"]) ) $opts["limit"] = 256;
    1642. if ( !isset($opts["around"]) ) $opts["around"] = 5;
    1643. if ( !isset($opts["exact_phrase"]) ) $opts["exact_phrase"] = false;
    1644. if ( !isset($opts["single_passage"]) ) $opts["single_passage"] = false;
    1645. if ( !isset($opts["use_boundaries"]) ) $opts["use_boundaries"] = false;
    1646. if ( !isset($opts["weight_order"]) ) $opts["weight_order"] = false;
    1648. /////////////////
    1649. // build request
    1650. /////////////////
    1652. // v.1.0 req
    1653. $flags = 1; // remove spaces
    1654. if ( $opts["exact_phrase"] ) $flags |= 2;
    1655. if ( $opts["single_passage"] ) $flags |= 4;
    1656. if ( $opts["use_boundaries"] ) $flags |= 8;
    1657. if ( $opts["weight_order"] ) $flags |= 16;
    1658. $req = pack ( "NN", 0, $flags ); // mode=0, flags=$flags
    1659. $req .= pack ( "N", strlen($index) ) . $index; // req index
    1660. $req .= pack ( "N", strlen($words) ) . $words; // req words
    1662. // options
    1663. $req .= pack ( "N", strlen($opts["before_match"]) ) . $opts["before_match"];
    1664. $req .= pack ( "N", strlen($opts["after_match"]) ) . $opts["after_match"];
    1665. $req .= pack ( "N", strlen($opts["chunk_separator"]) ) . $opts["chunk_separator"];
    1666. $req .= pack ( "N", (int)$opts["limit"] );
    1667. $req .= pack ( "N", (int)$opts["around"] );
    1669. // documents
    1670. $req .= pack ( "N", count($docs) );
    1671. foreach ( $docs as $doc )
    1672. {
    1673. assert ( is_string($doc) );
    1674. $req .= pack ( "N", strlen($doc) ) . $doc;
    1675. }
    1677. ////////////////////////////
    1678. // send query, get response
    1679. ////////////////////////////
    1681. $len = strlen($req);
    1682. $req = pack ( "nnN", SEARCHD_COMMAND_EXCERPT, VER_COMMAND_EXCERPT, $len ) . $req; // add header
    1683. if ( !( $this->_Send ( $fp, $req, $len+8 ) ) ||
    1684. !( $response = $this->_GetResponse ( $fp, VER_COMMAND_EXCERPT ) ) )
    1685. {
    1686. $this->_MBPop ();
    1687. return false;
    1688. }
    1690. //////////////////
    1691. // parse response
    1692. //////////////////
    1694. $pos = 0;
    1695. $res = array ();
    1696. $rlen = strlen($response);
    1697. for ( $i=0; $i<count($docs); $i++ )
    1698. {
    1699. list(,$len) = unpack ( "N*", substr ( $response, $pos, 4 ) );
    1700. $pos += 4;
    1702. if ( $pos+$len > $rlen )
    1703. {
    1704. $this->_error = "incomplete reply";
    1705. $this->_MBPop ();
    1706. return false;
    1707. }
    1708. $res[] = $len ? substr ( $response, $pos, $len ) : "";
    1709. $pos += $len;
    1710. }
    1712. $this->_MBPop ();
    1713. return $res;
    1714. }
    1716. /////////////////////////////////////////////////////////////////////////////
    1717. // keyword generation
    1718. /////////////////////////////////////////////////////////////////////////////
    1720. /// connect to searchd server, and generate keyword list for a given query
    1721. /// returns false on failure,
    1722. /// an array of words on success
    1723. function BuildKeywords ( $query, $index, $hits )
    1724. {
    1725. assert ( is_string($query) );
    1726. assert ( is_string($index) );
    1727. assert ( is_bool($hits) );
    1729. $this->_MBPush ();
    1731. if (!( $fp = $this->_Connect() ))
    1732. {
    1733. $this->_MBPop();
    1734. return false;
    1735. }
    1737. /////////////////
    1738. // build request
    1739. /////////////////
    1741. // v.1.0 req
    1742. $req = pack ( "N", strlen($query) ) . $query; // req query
    1743. $req .= pack ( "N", strlen($index) ) . $index; // req index
    1744. $req .= pack ( "N", (int)$hits );
    1746. ////////////////////////////
    1747. // send query, get response
    1748. ////////////////////////////
    1750. $len = strlen($req);
    1751. $req = pack ( "nnN", SEARCHD_COMMAND_KEYWORDS, VER_COMMAND_KEYWORDS, $len ) . $req; // add header
    1752. if ( !( $this->_Send ( $fp, $req, $len+8 ) ) ||
    1753. !( $response = $this->_GetResponse ( $fp, VER_COMMAND_KEYWORDS ) ) )
    1754. {
    1755. $this->_MBPop ();
    1756. return false;
    1757. }
    1759. //////////////////
    1760. // parse response
    1761. //////////////////
    1763. $pos = 0;
    1764. $res = array ();
    1765. $rlen = strlen($response);
    1766. list(,$nwords) = unpack ( "N*", substr ( $response, $pos, 4 ) );
    1767. $pos += 4;
    1768. for ( $i=0; $i<$nwords; $i++ )
    1769. {
    1770. list(,$len) = unpack ( "N*", substr ( $response, $pos, 4 ) ); $pos += 4;
    1771. $tokenized = $len ? substr ( $response, $pos, $len ) : "";
    1772. $pos += $len;
    1774. list(,$len) = unpack ( "N*", substr ( $response, $pos, 4 ) ); $pos += 4;
    1775. $normalized = $len ? substr ( $response, $pos, $len ) : "";
    1776. $pos += $len;
    1778. $res[] = array ( "tokenized"=>$tokenized, "normalized"=>$normalized );
    1780. if ( $hits )
    1781. {
    1782. list($ndocs,$nhits) = array_values ( unpack ( "N*N*", substr ( $response, $pos, 8 ) ) );
    1783. $pos += 8;
    1784. $res [$i]["docs"] = $ndocs;
    1785. $res [$i]["hits"] = $nhits;
    1786. }
    1788. if ( $pos > $rlen )
    1789. {
    1790. $this->_error = "incomplete reply";
    1791. $this->_MBPop ();
    1792. return false;
    1793. }
    1794. }
    1796. $this->_MBPop ();
    1797. return $res;
    1798. }
    1800. function EscapeString ( $string )
    1801. {
    1802. $from = array ( '(',')','|','-','!','@','~','"','&', '/', '\\' );
    1803. $to = array ( '\(','\)','\|','\-','\!','\@','\~','\"', '\&', '\/', '\\\\' );
    1805. return str_replace ( $from, $to, $string );
    1806. }
    1808. /////////////////////////////////////////////////////////////////////////////
    1809. // attribute updates
    1810. /////////////////////////////////////////////////////////////////////////////
    1812. /// batch update given attributes in given rows in given indexes
    1813. /// returns amount of updated documents (0 or more) on success, or -1 on failure
    1814. function UpdateAttributes ( $index, $attrs, $values, $mva=false )
    1815. {
    1816. // verify everything
    1817. assert ( is_string($index) );
    1818. assert ( is_bool($mva) );
    1820. assert ( is_array($attrs) );
    1821. foreach ( $attrs as $attr )
    1822. assert ( is_string($attr) );
    1824. assert ( is_array($values) );
    1825. foreach ( $values as $id=>$entry )
    1826. {
    1827. assert ( is_numeric($id) );
    1828. assert ( is_array($entry) );
    1829. assert ( count($entry)==count($attrs) );
    1830. foreach ( $entry as $v )
    1831. {
    1832. if ( $mva )
    1833. {
    1834. assert ( is_array($v) );
    1835. foreach ( $v as $vv )
    1836. assert ( is_int($vv) );
    1837. } else
    1838. assert ( is_int($v) );
    1839. }
    1840. }
    1842. // build request
    1843. $req = pack ( "N", strlen($index) ) . $index;
    1845. $req .= pack ( "N", count($attrs) );
    1846. foreach ( $attrs as $attr )
    1847. {
    1848. $req .= pack ( "N", strlen($attr) ) . $attr;
    1849. $req .= pack ( "N", $mva ? 1 : 0 );
    1850. }
    1852. $req .= pack ( "N", count($values) );
    1853. foreach ( $values as $id=>$entry )
    1854. {
    1855. $req .= sphPackU64 ( $id );
    1856. foreach ( $entry as $v )
    1857. {
    1858. $req .= pack ( "N", $mva ? count($v) : $v );
    1859. if ( $mva )
    1860. foreach ( $v as $vv )
    1861. $req .= pack ( "N", $vv );
    1862. }
    1863. }
    1865. // connect, send query, get response
    1866. if (!( $fp = $this->_Connect() ))
    1867. return -1;
    1869. $len = strlen($req);
    1870. $req = pack ( "nnN", SEARCHD_COMMAND_UPDATE, VER_COMMAND_UPDATE, $len ) . $req; // add header
    1871. if ( !$this->_Send ( $fp, $req, $len+8 ) )
    1872. return -1;
    1874. if (!( $response = $this->_GetResponse ( $fp, VER_COMMAND_UPDATE ) ))
    1875. return -1;
    1877. // parse response
    1878. list(,$updated) = unpack ( "N*", substr ( $response, 0, 4 ) );
    1879. return $updated;
    1880. }
    1882. /////////////////////////////////////////////////////////////////////////////
    1883. // persistent connections
    1884. /////////////////////////////////////////////////////////////////////////////
    1886. function Open()
    1887. {
    1888. if ( $this->_socket !== false )
    1889. {
    1890. $this->_error = 'already connected';
    1891. return false;
    1892. }
    1893. if ( !$fp = $this->_Connect() )
    1894. return false;
    1896. // command, command version = 0, body length = 4, body = 1
    1897. $req = pack ( "nnNN", SEARCHD_COMMAND_PERSIST, 0, 4, 1 );
    1898. if ( !$this->_Send ( $fp, $req, 12 ) )
    1899. return false;
    1901. $this->_socket = $fp;
    1902. return true;
    1903. }
    1905. function Close()
    1906. {
    1907. if ( $this->_socket === false )
    1908. {
    1909. $this->_error = 'not connected';
    1910. return false;
    1911. }
    1913. fclose ( $this->_socket );
    1914. $this->_socket = false;
    1916. return true;
    1917. }
    1918. }
    1920. //
    1921. // $Id: sphinxapi.php 1566 2008-11-17 19:06:44Z shodan $
    1922. //
    1923. ?>