1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-09-21 09:48:47 +02:00
phorge-phorge/src
epriestley f880000eb0 Stem fulltext tokens before filtering them for stopwords
Summary:
Fixes T12596. A query for a token (like "having") which stems to a stopword (like "have") currently survives filtering. Stem it first so it gets caught.

Also, for InnoDB, a custom stopword table can be configured. If it is, read that instead of the default stopword list (I configured it locally, but the default list is reasonable so we never formally recommended installs configure it).

Test Plan:
Queried for words that stem to stopwords, saw them filtered:

{F4915843}

Queried for the original problem query and saw "having" caught with "have" in the stopword list:

{F4915844}

Fiddled with local InnoDB stopword table config and saw the stopword list get loaded correctly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12596

Differential Revision: https://secure.phabricator.com/D17728
2017-04-19 10:02:21 -07:00
..
__tests__ Use PhutilClassMapQuery instead of PhutilSymbolLoader 2015-08-14 07:49:01 +10:00
aphront Make "Range" HTTP header work for Celerity static resource requests 2017-04-18 13:46:27 -07:00
applications Stem fulltext tokens before filtering them for stopwords 2017-04-19 10:02:21 -07:00
docs Suggest use of "usermod" rather than manually editing critical files in /etc 2017-04-10 14:15:38 -07:00
extensions
infrastructure Show users how fulltext search queries are parsed and executed; don't query stopwords or short tokens 2017-04-12 19:07:54 -07:00
view Disable auto-zoom on mobile form UIs 2017-04-17 15:55:05 -07:00
__phutil_library_init__.php
__phutil_library_map__.php Switch File deletion to use ModularTransactions 2017-04-18 13:01:51 -07:00