Tokenize datasource indexes on "(" and ")"

Summary:
Fixes T11955. Milestone names are currently tokenizing and indexing awkwardly. For example, "A (B C D)" becomes the tokens "A", "(B", "C" and "D)".

The token "(B" can't be searched for since "(" is tokenized on the client.

Instead, tokenize "A (B C D)" into "A", "B", "C", "D".

Test Plan:
  - Added unit tests.
  - Used `bin/search index --type project --force` to reindex.
  - Searched for "A", "B", "C", "D", etc., for real examples.
  - Now, found milestones more consistently.
  - Also serached for `viewer()`, `members()`, etc.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11955

Differential Revision: https://secure.phabricator.com/D17012

src/applications/typeahead/datasource/PhabricatorTypeaheadDatasource.php

@@ -141,7 +141,9 @@ abstract class PhabricatorTypeaheadDatasource extends Phobject {
       return array();
     }
 
-    $tokens = preg_split('/[\s\[\]-]+/u', $string);
+    // NOTE: Splitting on "(" and ")" is important for milestones.
+
+    $tokens = preg_split('/[\s\[\]\(\)-]+/u', $string);
     $tokens = array_unique($tokens);
 
     // Make sure we don't return the empty token, as this will boil down to a

src/applications/typeahead/datasource/__tests__/PhabricatorTypeaheadDatasourceTestCase.php

@@ -27,6 +27,18 @@ final class PhabricatorTypeaheadDatasourceTestCase
     $this->assertTokenization(
       '[[ brackets ]] [-] ]-[ tie-fighters',
       array('brackets', 'tie', 'fighters'));
+
+    $this->assertTokenization(
+      'viewer()',
+      array('viewer'));
+
+    $this->assertTokenization(
+      'Work (Done)',
+      array('work', 'done'));
+
+    $this->assertTokenization(
+      'A (B C D)',
+      array('a', 'b', 'c', 'd'));
   }
 
   private function assertTokenization($input, $expect) {