1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-22 13:30:55 +01:00
Commit graph

48 commits

Author SHA1 Message Date
epriestley
bbb321395a Support Aphlict clustering
Summary:
Ref T6915. This allows multiple notification servers to talk to each other:

  - Every server has a list of every other server, including itself.
  - Every server generates a unique fingerprint at startup, like "XjeHuPKPBKHUmXkB".
  - Every time a server gets a message, it marks it with its personal fingerprint, then sends it to every other server.
  - Servers do not retransmit messages that they've already seen (already marked with their fingerprint).
  - Servers learn other servers' fingerprints after they send them a message, and stop sending them messages they've already seen.

This is pretty crude, and the first message to a cluster will transmit N^2 times, but N is going to be like 3 or 4 in even the most extreme cases for a very long time.

The fingerprinting stops cycles, and stops servers from sending themselves copies of messages.

We don't need to do anything more sophisticated than this because it's fine if some notifications get lost when a server dies. Clients will reconnect after a short period of time and life will continue.

Test Plan:
  - Wrote two server configs.
  - Started two servers.
  - Told Phabricator about all four services.
  - Loaded Chrome and Safari.
  - Saw them connect to different servers.
  - Sent messages in one, got notifications in the other (magic!).
  - Saw the fingerprinting stuff work on the console, no infinite retransmission of messages, etc.

(This pretty much just worked when I ran it the first time so I probably missed something?)

{F1218835}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6915

Differential Revision: https://secure.phabricator.com/D15711
2016-04-14 13:26:30 -07:00
epriestley
07fc8f17cc Support "ssl.chain" in Aphlict configuration
Summary: Fixes T10806. Although browsers don't seem to care about this, it's more correct to support it, and the new test console uses normal `cURL` and does care.

Test Plan:
  - Hit the error case for providing a chain but no key/cert.
  - Used `openssl s_client -connect localhost:22280` to connect to local Aphlict servers.
  - With SSL but no chain, saw `openssl` fail to verify the remote.
  - With SSL and a chain, saw `openssl` verify the identify of the remote.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10806

Differential Revision: https://secure.phabricator.com/D15709
2016-04-14 10:41:21 -07:00
epriestley
2930733ac9 Complete modernization of Aphlict configuration
Summary:
Fixes T10697. This finishes bringing the rest of the config up to cluster power levels.

Phabricator is now given an arbitrarily long list of notification servers.

Each Aphlict server is given an arbitrarily long list of ports to run services on.

Users are free to make them meet in the middle by proxying whatever they want to whatever else they want.

This should also accommodate clustering fairly easily in the future.

Also rewrote the status UI and changed a million other things. 🐗

Test Plan:
{F1217864}

{F1217865}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15703
2016-04-14 04:57:00 -07:00
epriestley
c6b0925954 Move Aphlict logging and PID configuration options to config file
Summary: Ref T10697. Mostly straightforward. Also allow the server to have multiple logs and log options in the future (e.g., different verbosities or separate admin/client logs or whatever). No specific plans for this, but the default log is pretty noisy today.

Test Plan: Set up a couple of logs, started server, saw it log to them.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15702
2016-04-14 04:55:19 -07:00
epriestley
c84dee522b Move server-related Aphlict options to a configuration file
Summary: Ref T10697. This isn't everything but starts generalizing options and moving us toward a cluster-ready state of affairs.

Test Plan: Started server in various configurations, hit most (all?) of the error cases with bad configs, sent test notifications.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15701
2016-04-14 04:54:42 -07:00
epriestley
e32ce529d7 Begin generalizing Aphlict server to prepare for clustering/sensible config file
Summary:
Ref T10697. Currently, `aphlict` takes a ton of command line flags to configure exactly one admin server and exactly one client server.

I want to replace this with a config file. Additionally, I plan to support:

  - arbitrary numbers of listening client ports;
  - arbitrary numbers of listening admin ports;
  - SSL on any port.

For now, just transform the arguments to look like they're a config file. In the future, I'll load from a config file instead.

This greater generality will allow you to do stuff like run separate HTTP and HTTPS admin ports if you really want. I don't think there's a ton of use for this, but it tends to make the code cleaner anyway and there may be some weird cross-datacneter cases for it. Certainly, we undershot with the initial design and lots of users want to terminate SSL in nginx and run only HTTP on this server.

(Some sort-of-plausible use cases are running separate HTTP and HTTPS client servers, if your Phabricator install supports both, or running multiple HTTPS servers with different certificates if you have a bizarre VPN.)

Test Plan: Started Aphlict, connected to it, sent myself test notifications, viewed status page, reviewed logfile.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15700
2016-04-14 04:54:20 -07:00
Joshua Spence
04ee853cec Add the logger earlier in the Aphlict startup process
Summary: Add the logger as soon as possible so that the log file will contain errors if the `ws` module cannot be loaded.

Test Plan: Ran `./bin/aphlict debug` without having the `ws` module installed. Saw errors in the logs.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11457
2015-01-22 07:45:55 +11:00
Joshua Spence
f61846b469 Fix Aphlict exit status
Summary: Fixes T6998. Based on https://groups.google.com/d/msg/nodejs/zF7GEoPccqw/n26c6gPaluwJ.

Test Plan: Ran Aphlict and forced an error. Saw error messages in the logs and also saw a non-zero exit status.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6998

Differential Revision: https://secure.phabricator.com/D11456
2015-01-22 06:46:49 +11:00
Joshua Spence
27422ffe8e Use single quotes in JavaScript files
Summary: Use single quotes to keep JSHint happy.

Test Plan: `arc lint`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11431
2015-01-20 08:53:47 +11:00
Joshua Spence
53834d1471 Enable "strict" mode for NodeJS
Summary:
In particular, this changes the behavior of NodeJS in the following ways:

- Any attempt to get or modify the global object will result in an error.
- `null` values of `this` will no longer be evaluated to the global object and primitive values of this will not be converted to wrapper objects.
- Writing or deleting properties which have there writeable or configurable attributes set to false will now throw an error instead of failing silently.
- Adding a property to an object whose extensible attribute is false will also throw an error now.
- A functions arguments are not writeable so attempting to change them will now throw an error `arguments = [...]`.
- `with(){}` statements are gone.
- Use of `eval` is effectively banned.
- `eval` and `arguments` are not allowed as variable or function identifiers in any scope.
- The identifiers `implements`, `interface`, `let`, `package`, `private`, `protected`, `public`, `static` and `yield` are all now reserved for future use (roll on ES6).

Test Plan: Verified that Aphlict was still functional.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11430
2015-01-20 07:43:10 +11:00
epriestley
4636833f3d Fix module imports in Aphlict server
Summary:
This was broken in D11383. Basically, I had the `ws` module installed globally whilst testing, but the changes made do not work if the `ws` module is installed locally (i.e. in the `./support/aphlict/server/node_modules` directory). After poking around, it seems that this is due to the sandboxing that is done by `JX.require`.

A quick fix is to just //not// use `JX.require`, although you may have a better idea?

The error that is occurring is as follows:

```
<<< UNCAUGHT EXCEPTION! >>>

Error: Cannot find module 'ws'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at extra.require (/usr/src/phabricator/webroot/rsrc/externals/javelin/core/init_node.js:48:16)
    at /usr/src/phabricator/support/aphlict/server/lib/AphlictClientServer.js:10:17
    at Script.(anonymous function) [as runInNewContext] (vm.js:41:22)
    at Object.JX.require (/usr/src/phabricator/webroot/rsrc/externals/javelin/core/init_node.js:58:6)
    at Object.<anonymous> (/usr/src/phabricator/support/aphlict/server/aphlict_server.js:102:4)
    at Module._compile (module.js:456:26)
>>> Server exited!
```

Test Plan: Now able to start the Aphlict server.

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: Korvin, epriestley

Maniphest Tasks: T6987

Differential Revision: https://secure.phabricator.com/D11425
2015-01-19 11:46:14 -08:00
Joshua Spence
2b12f61602 Don't exit from the Aphlict Server prematurely
Summary: By calling `process.exit(1)` we are forcing the Node.js process to exit prematurely. Specifically, the process is terminated before the error is written to the log file. This can be verified by inspecting the value of `debug._logs[0]._writableState.writing` immediately before the process is terminated.

Test Plan: Ran `./bin/aphlict debug` without the `ws` module being installed. Verified that `<<< UNCAUGHT EXCEPTION! >>>` was echoed to the console as well as the log file. Also verified that the exit status was non-zero.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11426
2015-01-20 06:36:51 +11:00
Joshua Spence
9a7ad972cd Refactoring of the Aphlict server
Summary: Tidy the Aphlict server by splitting the functionality into two main modules, `AphlictClientServer` and `AphlictAdminServer. There is still further tidying that could be done here, but I feel that this puts us in a much better place.

Test Plan: Sent notifications via `/notification/status/`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11383
2015-01-19 07:49:50 +11:00
epriestley
0d070c91dc Fix Aphlict logging
Summary:
Yeahhhhhhhh....

  - Open a "stream", not a "steam".
  - Make error easier for users to understand.
  - Write to the log in debug mode so the issue is more apparent.

Test Plan:
  - Started server with bad permissions, got usable error message.
  - Started server with good permissions, got logfile.

Reviewers: joshuaspence, btrahan

Reviewed By: btrahan

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11339
2015-01-12 08:16:08 -08:00
Joshua Spence
638cf20c9d Allow the Aphlict server to bind to localhost
Summary: If you are running the Aphlict server behind a reverse proxy (such as `nginx`) then there's no need to bind to `0.0.0.0`. Add a `--client-host` flag to `aphlict_server.js` to allow binding to a different hostname. Also changed the other flags for consistency and clarity.

Test Plan: Started, stopped and debug the Aphlict server.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11288
2015-01-09 11:10:47 +11:00
epriestley
f0ade1be1c Stop advising users to use npm -g to install websockets
Summary: Fixes T6910. This advice is bad, doesn't work, and was based on me havng an outdated or incorrect understanding of Node and npm.

Test Plan: Read documentation.

Reviewers: richardvanvelzen, btrahan, chad, joshuaspence

Reviewed By: chad, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T6910

Differential Revision: https://secure.phabricator.com/D11285
2015-01-08 14:02:14 -08:00
Joshua Spence
a87f2cd610 Fix --ssl-cert for Aphlict
Summary: `PhabricatorAphlictManagementWorkflow` passes `--ssl-cert` but `aphlict_server.js` expects `--ssl-certificate`.

Test Plan: Tested on a production system.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11283
2015-01-09 08:51:47 +11:00
epriestley
9e0f70e17d Rewrite Aphlict to use Websockets
Summary:
Fixes T6559. No more flash, use Websockets. This is less aggressive than the earlier version, and retains more server logic.

  - Support "wss".
  - Make the client work.
  - Remove "notification.user" entirely.
  - Seems ok?

Test Plan:
In Safari, Firefox and Chrome, saw the browsers connect. Made a bunch of comments/updates and saw notifications.

Notable holes in the test plan:

  - Haven't tested "wss" yet. I'll do this on secure.
  - Notifications are //too fast// now, locally. I get them after I hit submit but before the page reloads.
  - There are probably some other rough edges, this is a fairly big patch.

Reviewers: joshuaspence, btrahan

Reviewed By: joshuaspence, btrahan

Subscribers: fabe, btrahan, epriestley

Maniphest Tasks: T6713, T6559

Differential Revision: https://secure.phabricator.com/D11143
2015-01-08 10:03:00 -08:00
Joshua Spence
87f11a091d Minor improvements to handling Aphlict status code
Summary: Self explanatory.

Test Plan: `curl`ed a few URLs.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11147
2015-01-03 09:11:08 +11:00
Joshua Spence
9f31e023f4 Minor improvements for handling of /status/ for Aphlict
Summary: We don't need to handle any request data for the `/status/` route, so we can simplify this code slightly.

Test Plan:
```lang=bash
> curl http://127.0.0.1:22281/status/
{"uptime":2543,"clients.active":0,"clients.total":0,"messages.in":0,"messages.out":0,"log":"/var/log/aphlict.log","version":6}
```

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11145
2015-01-03 09:10:49 +11:00
Joshua Spence
44198d68a7 Use the correct Content-Type for the Aphlict status route
Summary: The `/status/` route for the Aphlict server returns JSON.

Test Plan:
```lang=bash
> curl -I http://127.0.0.1:22281/status/
HTTP/1.1 200 OK
Content-Type: application/json
Date: Fri, 02 Jan 2015 13:08:34 GMT
Connection: keep-alive
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11144
2015-01-03 09:09:48 +11:00
Joshua Spence
a00efcbfca Fix some functions being used before they are defined
Summary: Minor JSHint fixes.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11073
2014-12-30 22:09:53 +11:00
Joshua Spence
fcc7dbbf15 Fix a few minor JSHint warnings
Summary: Basically, don't assign expressions to a variable if the variable is unused.

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11072
2014-12-30 03:02:58 -08:00
Joshua Spence
f2fee5a84e Return a HTTP 500 instead of a HTTP 400 if an internal error occurs in the Aphlict server
Summary: Ref T5651. Only throw a HTTP 400 if the data is invalid (i.e. the request is bad). If something bad happens when trying to transmit the notification, throw a HTTP 500 instead.

Test Plan: Eye-ball it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5651

Differential Revision: https://secure.phabricator.com/D9968
2014-07-18 09:20:00 +10:00
Joshua Spence
41a8837f78 Make HTTP errors returned from the Aphlict server more specific
Summary: Ref T5651. Currently, the Aphlict server returns either `200 OKAY` or `400 Bad Request`. We could return more specific errors in some cases and this may assist with debugging.

Test Plan:
Sent myself a test notification at `/notification/status/` and saw the Aphlict server process the request (running in debug mode). Also poked around with `curl`:

```
> curl http://localhost:22281/
405 Method Not Allowed

> curl http://localhost:22281/ -d ""
400 Bad Request

> curl http://localhost:22281/foobar/
404 Not Found
```

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5651

Differential Revision: https://secure.phabricator.com/D9967
2014-07-18 09:01:46 +10:00
epriestley
6bf4ec97d5 Fix HTTP 400 from notification server for JSON subscription objects
Summary: Fixes T5651. Sometime we'll send an object to the notification server for `subscribers`, which it will choke on. Use `array_values()` to make sure we're sending an array.

Test Plan: With `(object)` instead, got a consistent error ("no .filter method on object"). With `array_values()`, no error.

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5651

Differential Revision: https://secure.phabricator.com/D9963
2014-07-17 14:48:54 -07:00
Joshua Spence
7304e29dec Various minor JSHint fixes.
Summary: Various fixes as suggested by JSHint.

Test Plan: Eye-balled it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9783
2014-07-01 06:00:12 +10:00
Joshua Spence
e40b18fb75 Fix call to debug.log.
Summary: As pointed out by @epriestley in D9458#62, this call to `debug.log` is missing an argument.

Test Plan: meh..

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D9485
2014-06-14 11:28:28 -07:00
Joshua Spence
84d259cea2 Modify the Aphlict server to transmit messages instead of broadcasting them.
Summary: Ref T4324. Ref T5284. This adds server-side support for keeping track of a set of PHIDs that the Aphlict clients have subscribed to. Instead of broadcasting a notification to all clients (after which the clients can poll `/notification/individual` in order to determine whether or not they are interested in the notification), transmit notifications only to clients that have subscribed to a PHID that is relevant to the notification.

Test Plan:
I opened up two clients on the same host (incognito tabs in Chrome). Here is the output from the server:

```
> sudo ./bin/aphlict debug
Starting Aphlict server in foreground...
Launching server:

    $ 'nodejs' '/usr/src/phabricator/src/applications/aphlict/management/../../../../support/aphlict/server/aphlict_server.js' --port='22280' --admin='22281' --host='localhost' --user='aphlict'

[Wed Jun 11 2014 19:10:27 GMT+0000 (UTC)] Started Server (PID 4546)
[Wed Jun 11 2014 19:10:36 GMT+0000 (UTC)] <FlashPolicy> Policy Request From ::ffff:192.168.1.1
[Wed Jun 11 2014 19:10:37 GMT+0000 (UTC)] <Listener/1> Connected from ::ffff:192.168.1.1
[Wed Jun 11 2014 19:10:37 GMT+0000 (UTC)] <Listener/1> Received data: {"command":"subscribe","data":["PHID-USER-cb5af6p4oepy5tlgqypi"]}
[Wed Jun 11 2014 19:10:37 GMT+0000 (UTC)] <Listener/1> Subscribed to: ["PHID-USER-cb5af6p4oepy5tlgqypi"]
[Wed Jun 11 2014 19:10:39 GMT+0000 (UTC)] <Listener/1> Received data: {"command":"subscribe","data":["PHID-USER-kfohe3ca5oe6ygykmioq"]}
[Wed Jun 11 2014 19:10:39 GMT+0000 (UTC)] <Listener/1> Subscribed to: ["PHID-USER-kfohe3ca5oe6ygykmioq"]
[Wed Jun 11 2014 19:10:42 GMT+0000 (UTC)] notification: {"key":"6023751084283587681","type":"notification","subscribers":["PHID-USER-cb5af6p4oepy5tlgqypi"]}
[Wed Jun 11 2014 19:10:42 GMT+0000 (UTC)] <Listener/1> Wrote Message
```

I verified (using the "Network" tab in Chrome) that an AJAX request to `/notification/individual/` was only made in the tab belonging to the user that triggered the test notification.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5284, T4324

Differential Revision: https://secure.phabricator.com/D9458
2014-06-11 12:17:29 -07:00
epriestley
60d2b743d9 Print out stack traces when Aphlict server dies
Summary: Printing out `err` is less informative than `err.stack`, which has the message, type, //and// a trace.

Test Plan:
Faked an exception, then:

```
$ sudo ./bin/aphlict debug
Starting Aphlict server in foreground...
Launching server:

    $ 'node' '/INSECURE/devtools/phabricator/src/applications/aphlict/management/../../../../support/aphlict/server/aphlict_server.js' --port='22280' --admin='22281' --host='localhost'

[Wed Jun 11 2014 10:20:39 GMT-0700 (PDT)]
<<< UNCAUGHT EXCEPTION! >>>
Error: Example Exception
    at Object.<anonymous> (/INSECURE/devtools/phabricator/support/aphlict/server/aphlict_server.js:73:7)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3
>>> Server exited!
```

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9481
2014-06-11 10:40:04 -07:00
Joshua Spence
ab4324148a Make the Aphlict server more resilient.
Summary:
Currently, the Aphlict server will crash if invalid JSON data is `POST`ed to it. I have fixed this to, instead, return a 400. Also made some minor formatting changes.

Ref T4324. Ref T5284. Also, modify the data structure that is passed around (i.e. `POST`ed to the Aphlict server and broadcast to the Aphlict clients) to include the subscribers. Initially, I figured that we shouldn't expose this information to the clients... however, it is necessary for T4324 that the `AphlictMaster` is able to route a notification to the appropriate clients.

Test Plan:
Making the following `curl` request: `curl --data "{" http://localhost:22281/`.

**Before**
```
sudo ./bin/aphlict debug
Starting Aphlict server in foreground...
Launching server:

    $ 'nodejs' '/usr/src/phabricator/src/applications/aphlict/management/../../../../support/aphlict/server/aphlict_server.js' --port='22280' --admin='22281' --host='localhost' --user='aphlict'

[Wed Jun 11 2014 17:07:51 GMT+0000 (UTC)] Started Server (PID 2033)
[Wed Jun 11 2014 17:07:55 GMT+0000 (UTC)]
<<< UNCAUGHT EXCEPTION! >>>

SyntaxError: Unexpected end of input
>>> Server exited!
```

**After**
(No output... the bad JSON is caught and a 400 is returned)

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4324, T5284

Differential Revision: https://secure.phabricator.com/D9480
2014-06-11 10:17:12 -07:00
Joshua Spence
fe98aa6839 Publish additional context to the Aphlict server.
Summary:
Ref T4324. As well as sending the key for the notification, also publish the notification type and a list of subscribers to the Aphlict server.

The idea here is that the Aphlict server passes anything within the `data` key to the clients, whereas other keys (such as `subscribers`) will be used by the server to determine where the notifications should be routed.

Note that these changes don't do anything useful, but are a prerequisite for further work on T4324.

Test Plan:
Sent myself test notifications at `/notification/status/`. Also inspected the Aphlict server debug output:

```
> sudo ./bin/aphlict --foreground
Starting server in foreground, ignoring pidfile...
Launching server:

    $ node '/usr/src/phabricator/support/aphlict/server/aphlict_server.js' --port='22280' --admin='22281' --host='localhost' --user='aphlict' --log='/var/log/aphlict.log'

[Thu Jun 05 2014 18:38:14 GMT+0000 (UTC)] Started Server (PID 15437)
[Thu Jun 05 2014 18:38:16 GMT+0000 (UTC)] <FlashPolicy> Policy Request From ::ffff:10.0.0.1
[Thu Jun 05 2014 18:38:16 GMT+0000 (UTC)] <Listener/1> Connected from ::ffff:10.0.0.1
[Thu Jun 05 2014 18:38:19 GMT+0000 (UTC)] notification: {"data":{"key":"6021516228036848559","type":"notification"},"subscribers":["PHID-USER-cb5af6p4oepy5tlgqypi"]}
[Thu Jun 05 2014 18:38:19 GMT+0000 (UTC)] <Listener/1> Wrote Message
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T4324

Differential Revision: https://secure.phabricator.com/D9396
2014-06-05 12:09:26 -07:00
Joshua Spence
3202f0f23d Post data to the Aphlict server in JSON encoded form.
Summary:
Ref T4324. Currently, notifications data is `POST`ed to the Aphlict server in the `application/x-www-form-urlencoded` format. This works fine for simple data but is problematic for nested data. For example:

```lang=php
array(
  'data' => array(
    'key'  => '6021329908492455737',
    'type' => 'PhabricatorNotificationAdHocFeedStory',
  ),
  'subscribers' => array(
    'PHID-USER-y7ofqm276ejs62yqghge',
  ),
);
```

Is encoded as `data%5Bkey%5D=6021329908492455737&data%5Btype%5D=PhabricatorNotificationAdHocFeedStory&subscribers%5B0%5D=PHID-USER-y7ofqm276ejs62yqghge`. This string is then (incorrectly) decoded by `querystring.parse` as:

```lang=javascript
> querystring.parse('data%5Bkey%5D=6021329908492455737&data%5Btype%5D=PhabricatorNotificationAdHocFeedStory&subscribers%5B0%5D=PHID-USER-y7ofqm276ejs62yqghge');
{ 'data[key]': '6021329908492455737',
  'data[type]': 'PhabricatorNotificationAdHocFeedStory',
  'subscribers[0]': 'PHID-USER-y7ofqm276ejs62yqghge' }
```

Test Plan: Sent test notifications from `/notification/status/` and verified that the notifications still worked.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4324

Differential Revision: https://secure.phabricator.com/D9386
2014-06-05 09:47:33 -07:00
epriestley
f302bfc8f8 Break Aphlict's flash policy server into a separate class
Summary: Ref T4324. One of the server we start just sends pre-canned XML responses. Separate it out of the main file and hand it all the objects it interacts with in structured, reasonable ways.

Test Plan: Hit "Send Test Notification", saw notification, saw flash policy info in the log.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4324

Differential Revision: https://secure.phabricator.com/D8257
2014-02-17 16:01:09 -08:00
epriestley
28fe44da0a Break some of Aphlict into reasonable classes with sensible responsibilities
Summary:
Ref T4324.

  - Create `Listener` to represent listening clients.
  - Create `ListenerList` to represent the current list of clients.
  - Create `Logfile` to handle logging.

Test Plan: Clicked "Send Test Notification", verified logs, status and notifications all work correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4324

Differential Revision: https://secure.phabricator.com/D8256
2014-02-17 16:00:51 -08:00
epriestley
260eb5344b Allow Aphlict to load Javelin and use Javelin class definitions
Summary:
Ref T4324. The server code is probably going to get a fair amount more complicated, so allow it to load Javelin classes in a mostly-reasonable way.

This integration has a few warts, but should be good enough to let us manage complexity through the next iteration of the server.

(Mostly I just want the concicse Javelin mechanism for defining new classes.)

Version bump is just so I can figure stuff out if this creates any issues for users based on which version of things they're running.

Test Plan: Started server, posted some messages through it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4324

Differential Revision: https://secure.phabricator.com/D8253
2014-02-17 16:00:01 -08:00
epriestley
a1e7a4ccca Version the Aphlict notification server and prompt users to upgrade if they're out of date
Summary: Ref T4324. Add some version information to the server status output, and setup checks to test for an unreachable or out-of-date server.

Test Plan:
  - With server down, hit reasonable setup check.
  - With server up and at a bad version, hit reasonable setup check.
  - Viewed `/notification/status/`.
  - The CSS thing fixes this:

{F114445}

Reviewers: btrahan, chad

Reviewed By: chad

CC: chad, aran

Maniphest Tasks: T4324

Differential Revision: https://secure.phabricator.com/D8251
2014-02-17 15:59:39 -08:00
epriestley
f88a2b735d Remove spurious "+x" from files that shouldn't have it
Summary: We have a bunch of files with +x that aren't actually executable.
Remove +x from PNGs, etc.
2013-10-05 05:18:17 -07:00
epriestley
a7e4b846d6 Fix Aphlict server for newer Node
Summary: Fixes T3630.

Test Plan: Ran server on 0.11.0, got notifications.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3630

Differential Revision: https://secure.phabricator.com/D6588
2013-07-27 16:24:12 -07:00
epriestley
e54a2c1325 Minor, fix obvious typo in Aphlict server error handling. 2013-06-28 14:42:46 -07:00
epriestley
0569218201 Use JsShrink if jsxmin is not available
Summary: If `jsxmin` is not available, use a pure PHP implementation instead (JsShrink).

Test Plan:
  - Ran `arc lint --lintall` on all JS and fixed every relevant warning.
  - Forced minification on and browsed around the site using JS behaviors. Didn't hit anything problematic.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5670
2013-05-18 17:04:22 -07:00
epriestley
2b8adb3a83 Don't hang on /status/ for newer Node
Summary: At some point, Node started requiring us to read data before we can get the 'end' event, it seems. Fixes T2953.

Test Plan: Ran server in `--foreground` mode on node v0.11.0, made request to `/status/`, got response.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2953

Differential Revision: https://secure.phabricator.com/D5666
2013-04-11 15:45:50 -07:00
vrana
776a240870 Let Aphlict listen on the specified host
Summary: We allow user to specify the host to listen to but we listen on 127.0.0.1 instead.

Test Plan:
Hardcoded the path and verified that I can connect to the host from other machine.

Verified that `localhost` still doesn't allow remote connections.

Reviewers: ddfisher, epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4238
2012-12-19 15:44:17 -08:00
epriestley
5e184ee593 Improve debug support for notifications
Summary: Add a `notification.debug` setting that shows debug info in the browser. Also improve some logging/error handling stuff and fix a bug with host names.

Test Plan: {F13098}

Reviewers: jungejason, btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T944

Differential Revision: https://secure.phabricator.com/D2810
2012-06-20 13:20:47 -07:00
epriestley
86040227b0 Improve Aphlict server
Summary:
  - Move to port 22280 by default.
  - Warn when running as non-root.
  - Allow subscription and publish/admin ports to be configured.
  - Allow server to drop root after binding to 843.
  - Allow log path to be configured.
  - Add /status/ admin URI which shows server status.
  - Return HTTP 400 Bad Request for other requests, instead of hanging.
  - Minor formatting cleanup.

Test Plan:
Ran without root:

  $ node aphlict_server.js

...got a good error message. Ran with --user:

  $ sudo node aphlict_server.js --user=epriestley

...verified server dropped permissions. Ran with --port / --admin. Hit /status/ with GET, got status. Hit other URLs with GET, got 400.

Reviewers: allenjohnashton, ddfisher, keebuhm

Reviewed By: ddfisher

CC: aran

Differential Revision: https://secure.phabricator.com/D2737
2012-06-14 06:12:54 -07:00
David Fisher
f8f195b329 Make Notifications Realtime
Summary:
Adds the node.js Aphlict server, the flash Aphlict client, and some
supporting javascript. Built on top of - and requires - D2703 (which is still
in progress).  Will likely work with no modification on top of the final
version, though.

The node server is currently run with

   sudo node support/aphlict/server/aphlict_server.js

Test Plan: tested locally

Reviewers: epriestley

Reviewed By: epriestley

CC: allenjohnashton, keebuhm, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2704
2012-06-11 17:51:12 -07:00
epriestley
f8431bbfee Make Aphlict client somewhat more approachable
Summary: Provide a reasonable JS API for the Aphlict client. Provide an example behavior to invoke it.

Test Plan:
Ran "aphlict_server.js" with:

  $ sudo node aphlict_server.js

Loaded /aphlict/. Opened console. Got "hello" from the server every second.

Got reasonable errors with the server not present ("Security exception", but this is because it can't connect to port 843 to access the policy server).

Reviewers: ddfisher, keebuhm, allenjohnashton, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T944

Differential Revision: https://secure.phabricator.com/D1800
2012-03-06 20:14:03 -08:00
epriestley
84731e8f00 Aphlict, simple notification server
Summary:
This is purely a prototype at the moment, but the basic functionality sort of
works.

I'm not sure how far I want to go with this but I think we might be able to get
somewhere without it being gross.

The idea here is to build a notification server WITHOUT using Comet, since Comet
is extremely difficult and complicated.

Instead, I use Flash on the client. LocalConnection allows flash instances to
talk to each other and connect() can be used as a locking primitive. This allows
all the instances to elect a master instance in a race-safe way. The master is
responsible for opening a single connnection to the server.

On the server, I use Node.js since PHP is pretty unsuitable for this task.

See Github Issue #3: https://github.com/facebook/phabricator/issues/3

One thing I need to figure out next is if I can reasonably do SSL/TSL over Flash
(it looks like I can, in theory, with the as3crypto library) or if the server
needs to just send down version information and trigger a separate Ajax call on
the client.

Test Plan:
Created a client pool and connected it to the server, with election and failover
apparently working correctly.

Reviewed By: aran
Reviewers: Girish, aran, jungejason, tuomaspelkonen, davidrecordon
Commenters: Girish, davidrecordon
CC: aran, epriestley, Girish, davidrecordon
Differential Revision: 284
2011-05-17 10:32:41 -07:00