1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-10 00:42:41 +01:00
No description
Find a file
epriestley 152ddf5709 Use unicode mode when tokenizing strings like user realnames
Summary:
Fixes T9732. We currently tokenize strings (like user realnames) in the default non-unicode mode, which can cause patterns like `\s` to work incorrectly.

Use `/u` to use unicode-aware tokenization instead.

Test Plan:
The behavior of "\s" depends upon environmental settings like LC_ALL.

With LC_ALL set to "C", `\xA0` is not considered a whitespace character.
With LC_ALL set to "en_US", it is:

```
$ php -r 'setlocale(LC_ALL, "C"); echo count(preg_split("/\s/", "\xE5\xBF\xA0")) . "\n";'
1
$ php -r 'setlocale(LC_ALL, "en_US"); echo count(preg_split("/\s/", "\xE5\xBF\xA0")) . "\n";'
2
```

To reproduce the original issue, I added an explicit:

```
setlocale(LC_ALL, "en_US");
```

...call before the `preg_split()` call. This caused "忠" to be improperly split.

I then added "/u", and observed proper tokenization.

Reviewers: chad

Reviewed By: chad

Subscribers: qiu8310

Maniphest Tasks: T9732

Differential Revision: https://secure.phabricator.com/D14441
2015-11-08 07:03:09 -08:00
bin Provide bin/garbage for interacting with garbage collection 2015-10-02 09:17:24 -07:00
conf Mark some strings for translation 2015-06-09 23:06:52 +10:00
externals Fix Oblivious skin summary remarkup and partially fix title 2015-11-06 20:24:11 +00:00
resources Add Can Create Policy Capability to Phame Blogs 2015-11-08 07:00:18 -08:00
scripts Various translation improvements 2015-11-03 07:02:46 +11:00
src Use unicode mode when tokenizing strings like user realnames 2015-11-08 07:03:09 -08:00
support Add a "Startup" to DarkConsole 2015-08-21 14:53:29 -07:00
webroot Add Can Create Policy Capability to Phame Blogs 2015-11-08 07:00:18 -08:00
.arcconfig Use the configuration driven unit test engine 2015-08-11 07:57:11 +10:00
.arclint Turn lint TODO comments back on 2015-05-27 10:06:55 -07:00
.arcunit Use the configuration driven unit test engine 2015-08-11 07:57:11 +10:00
.editorconfig Fix text lint issues 2015-02-12 07:00:13 +11:00
.gitignore Add custom Cows and Figlet directories to .gitignore 2015-10-08 20:23:05 -07:00
LICENSE Fix text lint issues 2015-02-12 07:00:13 +11:00
NOTICE Update Phabricator NOTICE file to reflect modern legal circumstances 2014-06-25 13:42:13 -07:00
README.md Remove push to IRC from "readme.md" too 2015-10-24 18:39:16 -07:00

Phabricator is a collection of web applications which help software companies build better software.

Phabricator includes applications for:

  • reviewing and auditing source code;
  • hosting and browsing repositories;
  • tracking bugs;
  • managing projects;
  • conversing with team members;
  • assembling a party to venture forth;
  • writing stuff down and reading it later;
  • hiding stuff from coworkers; and
  • also some other things.

You can learn more about the project (and find links to documentation and resources) at Phabricator.org

Phabricator is developed and maintained by Phacility.


SUPPORT RESOURCES

For resources on filing bugs, requesting features, reporting security issues, and getting other kinds of support, see Support Resources.

NO PULL REQUESTS!

We do not accept pull requests through GitHub. If you would like to contribute code, please read our Contributor's Guide.

LICENSE

Phabricator is released under the Apache 2.0 license except as otherwise noted.