Summary:
Fixes T10228. When we receive a gzipped request (rare, but `git` may send them), decode it before providing it to the application.
This fixes the issue with proxying certain requests described in T10228.
Test Plan:
- Applied this fix in production.
- Cloned a problem repository cleanly.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10228
Differential Revision: https://secure.phabricator.com/D15145
Summary: Ref T8588. It looks like something slow is happening //before// we start DarkConsole. Add some crude reporting to try to narrow it down.
Test Plan: {F743050}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T8588
Differential Revision: https://secure.phabricator.com/D13956
Summary: Use `PhutilClassMaQuery` instead of `PhutilSymbolLoader`, mostly for consistency. Depends on D13588.
Test Plan: Poked around a bunch of pages.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13589
Summary:
Ref T8424. Fixes T7114. This was envisioned as a per-request cache for reusing interpreters, but isn't a good fit for that in modern Phabricator.
In particular, it isn't loaded by the daemons, but they have equal need for per-request caching.
Since I finally need such a cache for Spaces, throw the old stuff away before I built a more modern cache.
Also resolves T7114 by dropping filtering on $_SERVER. I'm pretty sure this is the simplest fix, see D12977 for a bit more discussion.
Test Plan: Called `didFatal()` from somewhere in normal code and verified it was able to use the access log.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7114, T8424
Differential Revision: https://secure.phabricator.com/D13152
Summary: Use `__CLASS__` instead of hard-coding class names. Depends on D12605.
Test Plan: Eyeball it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: hach-que, Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12806
Summary:
Caught this in the production error logs. We can end up with `argv` defined and set to an array in an nginx + php-fpm configuration.
When we later run `ExecFuture` subprocesses, they won't be able to forward the value.
The error this produces looks like this:
```
015/04/27 12:17:35 [error] 10948#0: *674 FastCGI sent in stderr: "PHP message: [2015-04-27 12:17:35] ERROR 8: Array to string conversion at [/core/lib/libphutil/src/future/exec/ExecFuture.php:667]
PHP message: arcanist(head=master, ref.master=805ae12408e8), phabricator(head=master, ref.master=8ce8a761efe9), phutil(head=master, ref.master=fccf03d48e08)
PHP message: #0 ExecFuture::isReady() called at [<phutil>/src/future/Future.php:39]
PHP message: #1 Future::resolve(NULL) called at [<phutil>/src/future/exec/ExecFuture.php:413]
PHP message: #2 ExecFuture::resolvex() called at [<phabricator>/src/applications/diffusion/query/rawdiff/DiffusionGitRawDiffQuery.php:40]
PHP message: #3 DiffusionGitRawDiffQuery::executeQuery() called at [<phabricator>/src/applications/diffusion/query/rawdiff/DiffusionRawDiffQuery.php:17]
PHP message: #4 DiffusionRawDiffQuery::loadRawDiff() called at [<phabricator>/src/applications/diffusion/conduit/DiffusionRawDiffQueryConduitAPIMethod.php:56]
PHP message: #5 DiffusionRawDiffQueryConduitAPIMethod::getResult(ConduitAPIRequest) called at [<phabricator>/src/applications/diffusion/conduit/DiffusionQueryConduitAPIMethod.php:135]
PHP message: #6 DiffusionQueryConduitAPIMethod::execute(ConduitAPIRequest) called at [<phabricator>/src/applications/conduit/method/ConduitAPIMethod.php:90]
PHP message: #7 ConduitAPIMethod::executeMethod(ConduitAPIRequest) called at [<phabricator>/src/applications/conduit/call/ConduitCall.php:134]
PHP message: #8 ConduitCall::executeMethod() called at [<phabricator>/src/applications/conduit/call/ConduitCall.php:84]
PHP message: #9 ConduitCall::execute() called at [<phabricator>/src/applications/diffusion/query/DiffusionQuery.php:81]
PHP message: #10 DiffusionQuery::callConduitWithDiffusionRequest(PhabricatorUser, DiffusionGitRequest, string, array) called at [<phabricator>/src/applications/diffusion/controller/DiffusionController.php:184]
PHP message: #11 DiffusionController::callConduitWithDiffusionRequest(string, array) called at [<phabricat
```
Test Plan: I'm just going to push this to make sure it fixes things, since I can't repro it locally.
Reviewers: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D12571
Summary:
Fixes T7130. Fixes T7041. Fixes T7012.
Major change here is partitioning clients. In the Phacility cluster, being able to get a huge pile of instances on a single server -- without needing to run a process per instance -- is desirable.
To accomplish this, just bucket clients by the path they connect with. This will let us set client URIs to `/instancename/` and then route connections to a small set of servers. This degrades cleanly in the common case and has no effect on installs which don't do instancing.
Also fix two unrelated issues:
- Fix the timeouts, which were incorrectly initializing in `open()` (which is called during reconnect, causing them to reset every time). Instead, initialize in the constructor. Cap timeout at 5 minutes.
- Probably fix subscriptions, which were using a property with an object definition. Since this is by-ref, all concrete instances of the object share the same property, so all users would be subscribed to everything. Probably.
Test Plan:
- Hit notification status page, saw version bump and instance/path name.
- Saw instance/path name in client and server logs.
- Stopped server, saw reconnects after 2, 4, 16, ... seconds.
- Sent test notification; received test notification.
- Didn't explicitly test the subscription thing but it should be obvious by looking at `/notification/status/` shortly after a push.
Reviewers: joshuaspence, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7041, T7012, T7130
Differential Revision: https://secure.phabricator.com/D11769
Summary: Ref T7124. The local version of `this` in the handler of 'end' was incremented rather than the global one.
Test Plan: Sending test notifications did not increment the `messages.in` value before this patch even if it should have. With the patch sending test notifications does increment `messages.in`.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7124
Differential Revision: https://secure.phabricator.com/D11648
Summary: Ref T7126. Dictionaries do not have a `length` property unlike arrays resulting in the `getActiveListenerCount()` function returning undefined results. Using the `length` property on the array of keys will work.
Test Plan: Using `wscat` to generate multiple connections to the server, establish new ones and close others while keeping an eye on the displayed `clients.active` value.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Maniphest Tasks: T7126
Differential Revision: https://secure.phabricator.com/D11647
Summary: Ref T7110. Listeners are now removed when clients close the connection to avoid stacking a never ending number of unused listeners.
Test Plan: Using `wscat` to connect to the Aphlict server; when closing the connection a 'Diconnected.' will appear in the logs and the number of active listeners is decreased by one.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Maniphest Tasks: T7110
Differential Revision: https://secure.phabricator.com/D11634
Summary: Add the logger as soon as possible so that the log file will contain errors if the `ws` module cannot be loaded.
Test Plan: Ran `./bin/aphlict debug` without having the `ws` module installed. Saw errors in the logs.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11457
Summary:
This option prohibits overwriting prototypes of native objects such as `Array`, `Date` and so on.
```lang=js
// jshint freeze:true
Array.prototype.count = function (value) { return 4; };
// -> Warning: Extending prototype of native object: 'Array'.
```
Test Plan: Linted existing JavaScript files, found no violations.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11439
Summary:
In particular, this changes the behavior of NodeJS in the following ways:
- Any attempt to get or modify the global object will result in an error.
- `null` values of `this` will no longer be evaluated to the global object and primitive values of this will not be converted to wrapper objects.
- Writing or deleting properties which have there writeable or configurable attributes set to false will now throw an error instead of failing silently.
- Adding a property to an object whose extensible attribute is false will also throw an error now.
- A functions arguments are not writeable so attempting to change them will now throw an error `arguments = [...]`.
- `with(){}` statements are gone.
- Use of `eval` is effectively banned.
- `eval` and `arguments` are not allowed as variable or function identifiers in any scope.
- The identifiers `implements`, `interface`, `let`, `package`, `private`, `protected`, `public`, `static` and `yield` are all now reserved for future use (roll on ES6).
Test Plan: Verified that Aphlict was still functional.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11430
Summary:
This was broken in D11383. Basically, I had the `ws` module installed globally whilst testing, but the changes made do not work if the `ws` module is installed locally (i.e. in the `./support/aphlict/server/node_modules` directory). After poking around, it seems that this is due to the sandboxing that is done by `JX.require`.
A quick fix is to just //not// use `JX.require`, although you may have a better idea?
The error that is occurring is as follows:
```
<<< UNCAUGHT EXCEPTION! >>>
Error: Cannot find module 'ws'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at extra.require (/usr/src/phabricator/webroot/rsrc/externals/javelin/core/init_node.js:48:16)
at /usr/src/phabricator/support/aphlict/server/lib/AphlictClientServer.js:10:17
at Script.(anonymous function) [as runInNewContext] (vm.js:41:22)
at Object.JX.require (/usr/src/phabricator/webroot/rsrc/externals/javelin/core/init_node.js:58:6)
at Object.<anonymous> (/usr/src/phabricator/support/aphlict/server/aphlict_server.js:102:4)
at Module._compile (module.js:456:26)
>>> Server exited!
```
Test Plan: Now able to start the Aphlict server.
Reviewers: joshuaspence
Reviewed By: joshuaspence
Subscribers: Korvin, epriestley
Maniphest Tasks: T6987
Differential Revision: https://secure.phabricator.com/D11425
Summary: By calling `process.exit(1)` we are forcing the Node.js process to exit prematurely. Specifically, the process is terminated before the error is written to the log file. This can be verified by inspecting the value of `debug._logs[0]._writableState.writing` immediately before the process is terminated.
Test Plan: Ran `./bin/aphlict debug` without the `ws` module being installed. Verified that `<<< UNCAUGHT EXCEPTION! >>>` was echoed to the console as well as the log file. Also verified that the exit status was non-zero.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11426
Summary: Tidy the Aphlict server by splitting the functionality into two main modules, `AphlictClientServer` and `AphlictAdminServer. There is still further tidying that could be done here, but I feel that this puts us in a much better place.
Test Plan: Sent notifications via `/notification/status/`.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11383
Summary: Currently, the Aphlict server created the log file (if it doesn't exist) but then immediately fails with "Unable to open logfile". It seems that we don't set the permissions correctly.
Test Plan: Deleted log file and was able to start the Aphlict server.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11351
Summary:
Yeahhhhhhhh....
- Open a "stream", not a "steam".
- Make error easier for users to understand.
- Write to the log in debug mode so the issue is more apparent.
Test Plan:
- Started server with bad permissions, got usable error message.
- Started server with good permissions, got logfile.
Reviewers: joshuaspence, btrahan
Reviewed By: btrahan
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11339
Summary: If you are running the Aphlict server behind a reverse proxy (such as `nginx`) then there's no need to bind to `0.0.0.0`. Add a `--client-host` flag to `aphlict_server.js` to allow binding to a different hostname. Also changed the other flags for consistency and clarity.
Test Plan: Started, stopped and debug the Aphlict server.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11288
Summary: Fixes T6910. This advice is bad, doesn't work, and was based on me havng an outdated or incorrect understanding of Node and npm.
Test Plan: Read documentation.
Reviewers: richardvanvelzen, btrahan, chad, joshuaspence
Reviewed By: chad, joshuaspence
Subscribers: epriestley
Maniphest Tasks: T6910
Differential Revision: https://secure.phabricator.com/D11285
Summary:
Fixes T6559. No more flash, use Websockets. This is less aggressive than the earlier version, and retains more server logic.
- Support "wss".
- Make the client work.
- Remove "notification.user" entirely.
- Seems ok?
Test Plan:
In Safari, Firefox and Chrome, saw the browsers connect. Made a bunch of comments/updates and saw notifications.
Notable holes in the test plan:
- Haven't tested "wss" yet. I'll do this on secure.
- Notifications are //too fast// now, locally. I get them after I hit submit but before the page reloads.
- There are probably some other rough edges, this is a fairly big patch.
Reviewers: joshuaspence, btrahan
Reviewed By: joshuaspence, btrahan
Subscribers: fabe, btrahan, epriestley
Maniphest Tasks: T6713, T6559
Differential Revision: https://secure.phabricator.com/D11143
Summary: We don't need to handle any request data for the `/status/` route, so we can simplify this code slightly.
Test Plan:
```lang=bash
> curl http://127.0.0.1:22281/status/
{"uptime":2543,"clients.active":0,"clients.total":0,"messages.in":0,"messages.out":0,"log":"/var/log/aphlict.log","version":6}
```
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11145
Summary:
Fixes T6692. Addresses two main issues:
- The write guard would sometimes not get disposed of on exception pathways, generating an unnecessary secondary error which was just a symptom of the original root error.
- This was generally confusing and reduced the quality of reports we received because users would report the symptomatic error sometimes instead of the real error.
- Instead, reflow the handling so that we always dispose of the write guard if we create one.
- If we missed the Controller-level error page generation (normally, a nice page with full CSS, etc), we'd jump straight to Startup-level error page generation (very basic plain text).
- A large class of errors occur too early or too late to be handled by Controller-level pages, but many of these errors are not fundamental, and the plain text page is excessively severe.
- Provide a mid-level simple HTML error page for errors which can't get full CSS, but also aren't so fundamental that we have no recourse but plain text.
Test Plan:
Mid-level errors now produce an intentional-looking error page:
{F259885}
Verified that setup errors still render properly.
@chad, feel free to tweak the exception page -- I just did a rough pass on it. Like the setup error stuff, it doesn't have Celerity, so we can't use `{$colors}` and no other CSS will be loaded.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley, chad
Maniphest Tasks: T6692
Differential Revision: https://secure.phabricator.com/D11126
Summary: Basically, don't assign expressions to a variable if the variable is unused.
Test Plan: `arc lint`
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11072
Summary: Currently, we assume that all JavaScript files are for use in a browser. This is not true for the NodeJS Aphlict server code. Split the current JSHint configuration into `jshint-browser` and `jshint-node`.
Test Plan: `arc lint`
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11071
Summary: Let JSHint know that `Raphael` is a global that can be used anywhere. This is not technically correct, but it silences a few JSHint warnings. See http://jshint.com/docs/ for more information.
Test Plan: `arc lint -- webroot/rsrc/js/application/maniphest/behavior-line-chart.js`
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D11069
Summary:
Fixes T6044. We've had two cases (both the same install, coincidentally) where pages got hung doing too much data fetching.
When pages hang, we don't get a useful stack trace out of them, since nginx, php-fpm, or PHP eventually terminates things in a non-useful way without any diagnostic information.
The second time (the recent Macros issue) I was able to walk the install through removing limits on nginx, php-fpm, php, and eventually getting a profile by letting the page run for several minutes until the request completed. However, this install is exceptionally technically proficient and this was still a big pain for everyone, and this approach would not have worked if the page actually looped rather than just taking a long time.
Provide `debug.time-limit`, which should give us a better tool for reacting to this situation: by setting it to a small value (like 10), we'll kill the page after 10 seconds with a trace, before nginx/php-fpm/php/etc can kill it uselessly. Hopefully that will be enough information to find the issue (generally, getting a trace has been 95% of the problem in the two cases we've encountered).
Test Plan: Set this option to `3` and added a sleep loop, saw a termination after 3 seconds with a useful trace.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: csilvers, joshuaspence, epriestley
Maniphest Tasks: T6044
Differential Revision: https://secure.phabricator.com/D10465
Summary: Fixes T6011. See that task for discussion. We can detect when `memory_limit` will be the limiting factor for drag-and-drop uploads and warn administrators about it.
Test Plan: Fiddled configuration values and hit, then resolved, the issue.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T6011
Differential Revision: https://secure.phabricator.com/D10413
Summary: Ref T5651. Only throw a HTTP 400 if the data is invalid (i.e. the request is bad). If something bad happens when trying to transmit the notification, throw a HTTP 500 instead.
Test Plan: Eye-ball it.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Maniphest Tasks: T5651
Differential Revision: https://secure.phabricator.com/D9968
Summary: Ref T5651. Currently, the Aphlict server returns either `200 OKAY` or `400 Bad Request`. We could return more specific errors in some cases and this may assist with debugging.
Test Plan:
Sent myself a test notification at `/notification/status/` and saw the Aphlict server process the request (running in debug mode). Also poked around with `curl`:
```
> curl http://localhost:22281/
405 Method Not Allowed
> curl http://localhost:22281/ -d ""
400 Bad Request
> curl http://localhost:22281/foobar/
404 Not Found
```
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T5651
Differential Revision: https://secure.phabricator.com/D9967
Summary: Fixes T5651. Sometime we'll send an object to the notification server for `subscribers`, which it will choke on. Use `array_values()` to make sure we're sending an array.
Test Plan: With `(object)` instead, got a consistent error ("no .filter method on object"). With `array_values()`, no error.
Reviewers: joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5651
Differential Revision: https://secure.phabricator.com/D9963
Summary: It is somewhat overkill to use an entire directory for a single file (the configuration file for JSHint). Instead, rename the directory so that it could (theoretically) be used for other linter configuration files.
Test Plan: Ran `arc lint -- webroot/rsrc/js/core/phtize.js` just to make sure everything still works.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9944
Summary: Various fixes as suggested by JSHint.
Test Plan: Eye-balled it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9783
Summary: Ref T5365. Surface disconnects in the UI.
Test Plan:
- Connected, then killed the server.
- Saw disconnected event and appropriate update in the UI.
{F169605}
Reviewers: joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5365
Differential Revision: https://secure.phabricator.com/D9706
Summary:
Fixes T5373. Ref T5281. Several changes:
- The `marshallExceptions` thing is useful if JS throws an exception when invoked from Flash, so set it. The resulting exceptions are a little odd (not escaped correctly, e.g.) but way better than nothing.
- Put connection status in the notification menu.
- When the connection fails, try to provide contextual help where we can.
Test Plan: {F169493}
Reviewers: chad, joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5281, T5373
Differential Revision: https://secure.phabricator.com/D9700
Summary:
Ref T5373. The control flow between `aphlict-listener` and `JX.Aphlict` is pretty weird right now, where the listener (which is the highest-level component) has intimate knowledge of how to put the SWF on the page.
Instead:
- Make `JX.Aphlict` a real singleton.
- Instantiate it sooner.
- Have it handle the flash setup handshake.
Test Plan: Loaded page in debug mode, saw normal flow take place.
Reviewers: joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5373
Differential Revision: https://secure.phabricator.com/D9699
Summary:
Ref T5373. This seems to work pretty much correctly.
Also stop popping bubbles and just use the log, since users find the bubbles confusing/not useful and they're not great for developers either.
Future diffs will expose more user-facing stuff.
Test Plan: Added `throw` to AphlictClient.as, got a log in the parent window.
Reviewers: joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5373
Differential Revision: https://secure.phabricator.com/D9698
Summary: Most scripts detect the relevant workflows automatically. Some scripts, however, use a hardcoded list of workflows.
Test Plan: Ran `./bin/aphlict`, `./bin/cache` and `./bin/phd`.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9564
Summary: The `_activeListenerCount` variable is overkill, we should be able to achieve the same result using `Object.keys(this._listeners).length`.
Test Plan:
Mucked around in a NodeJS shell.
```lang=js
> Object.keys({}).length
0
> Object.keys({foo: 'bar'}).length
1
> Object.keys({1: 'foo', 2: 'bar'}).length
2
```
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D9554
Summary: As pointed out by @epriestley in D9458#62, this call to `debug.log` is missing an argument.
Test Plan: meh..
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D9485
