1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-10 00:42:41 +01:00
Commit graph

12369 commits

Author SHA1 Message Date
epriestley
287e761f19 Make repository synchronization safer when leaders are ambiguous
Summary:
Ref T4292. Right now, repository versions only get marked when a write happens.

This potentially creates a problem: if I pushed all the sync code to `secure` and enabled `secure002` as a repository host, the daemons would create empty copies of all the repositories on that host.

Usually, this would be fine. Most repositories have already received a write on `secure001`, so that working copy has a verison and is a leader.

However, when a write happened to a rarely-used repository (say, rKEYSTORE) that hadn't received any write recently, it might be sent to `secure002` randomly. Now, we'd try to figure out if `secure002` has the most up-to-date copy of the repository or not.

We wouldn't be able to, since we don't have any information about which node has the data on it, since we never got a write before. The old code could guess wrong and decide that `secure002` is a leader, then accept the write. Since this would bump the version on `secure002`, that would //make// it an authoritative leader, and `secure001` would synchronize from it passively (or on the next read or write), which would potentially destroy data.

Instead:

  - Refuse to continue in situations like this.
  - When a repository is on exactly one device, mark it as a leader with version "0".
  - When a repository is created into a cluster service, mark its version as "0" on all devices (they're all leaders, since the repository is empty).

This should mean that we won't lose data no matter how much weird stuff we run into.

Test Plan:
  - In single-node mode, used `repository update` to verify that `0` was written properly.
  - With multiple nodes, used `repository update` to verify that we refuse to continue.
  - Created a new repository, verified versions were initialized correctly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15761
2016-04-19 13:07:02 -07:00
epriestley
6edf181a7e Record which cluster host received a push
Summary: Ref T4292. When we write a push log, also log which node received the request.

Test Plan: {F1230467}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15759
2016-04-19 13:06:30 -07:00
epriestley
d87c500002 Synchronize (hosted, clustered, Git) repositories over Conduit + HTTP
Summary:
Ref T4292. We currently synchronize hosted, clustered, Git repositories when we receive an SSH pull or push.

Additionally:

  - Synchronize before HTTP reads and writes.
  - Synchronize reads before Conduit requests.

We could relax Conduit eventually and allow Diffusion to say "it's OK to give me stale data".

We could also redirect some set of these actions to just go to the up-to-date host instead of connecting to a random host and synchronizing it. However, this potentially won't work as well at scale: if you have a larger number of servers, it sends all of the traffic to the leader immediately following a write. That can cause "thundering herd" issues, and isn't efficient if replicas are in different geographical regions and the write just went to the east coast but most clients are on the west coast. In large-scale cases, it's better to go to the local replica, wait for an update, then serve traffic from it -- particularly given that writes are relatively rare. But we can finesse this later once things are solid.

Test Plan:
  - Pushed and pulled a Git repository over HTTP.
  - Browsed a Git repository from the web UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15758
2016-04-19 13:05:45 -07:00
epriestley
31bc023eff Synchronize (hosted, git, clustered, SSH) repositories prior to reads
Summary:
Ref T4292. Before we write or read a hosted, clustered Git repository over SSH, check if another version of the repository exists on another node that is more up-to-date.

If such a version does exist, fetch that version first. This allows reads and writes of any node to always act on the most up-to-date code.

Test Plan: Faked my way through this and got a fetch via `bin/repository update`; this is difficult to test locally and needs more work before we can put it in production.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15757
2016-04-19 13:05:17 -07:00
epriestley
c70f4815a9 Allow cluster devices to SSH to one another without acting as a user
Summary:
Ref T4292. When you run `git fetch` and connect to, say, `repo001.west.company.com`, we'll look at the current version of the repository in other nodes in the cluster.

If `repo002.east.company.com` has a newer version of the repository, we'll fetch that version first, then respond to your request.

To do this, we need to run `git fetch repo002.east.company.com ...` and have that connect to the other host and be able to fetch data.

This change allows us to run `PHABRICATOR_AS_DEVICE=1 git fetch ...` to use device credentials to do this fetch. (Device credentials are already supported and used, they just always connect as a user right now, but these fetches should be doable without having a user. We will have a valid user when you run `git fetch` yourself, but we won't have one if the daemons notice that a repository is out of date and want to update it, so the update code should not depend on having a user.)

Test Plan:
```
$ PHABRICATOR_AS_DEVICE=1 ./bin/ssh-connect local.phacility.com
Warning: Permanently added 'local.phacility.com' (RSA) to the list of known hosts.
PTY allocation request failed on channel 0
phabricator-ssh-exec: Welcome to Phabricator.

You are logged in as device/daemon.phacility.net.

You haven't specified a command to run. This means you're requesting an interactive shell, but Phabricator does not provide an interactive shell over SSH.

Usually, you should run a command like `git clone` or `hg push` rather than connecting directly with SSH.

Supported commands are: conduit, git-lfs-authenticate, git-receive-pack, git-upload-pack, hg, svnserve.
Connection to local.phacility.com closed.
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15755
2016-04-19 13:04:41 -07:00
epriestley
0db6eaca41 Consolidate handling of SSH usernames
Summary:
Ref T4292. This consolidates code for figuring out which user we should connect to hosts with.

Also narrows a lock window.

Test Plan: Browsed Diffusion, pulled and pushed through an SSH proxy.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15754
2016-04-19 13:04:04 -07:00
Eitan Adler
c9daa2b0ad Consistently refer to 'Projects' as 'Tags'
Summary:
In calendar, dashboard, diffusion, diviner, feed, fund,
maniphest, pholio, ponder, and slowvote use the term 'tags' if possible.

This intenctionally skips diffusion, differential, and the projects application itself.

Ref T10326 Ref T10349

Test Plan: inspection on a running, locally modified, system

Reviewers: avivey, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T10835, T10326, T10349

Differential Revision: https://secure.phabricator.com/D15753
2016-04-19 16:48:21 +00:00
epriestley
c30fe65ee9 Remove the warning about the Git 2GB pathname issue
Summary:
Ref T10832. In practice, `git --version` is not a useful test for this issue:

  - Vendors like Debian have backported the patch into custom versions like `0.0.0.1-debian-lots-of-patches.3232`.
  - Vendors like Ubuntu distribute multiple different versions which report the same string from `git --version`, some of which are patched and some of which are not.

In other cases, we can perform an empirical test for the vulnerability. Here, we can not, because we can't write a 2GB path in a reasonable amount of time.

Since vendors (other than Apple) //generally// seem to be on top of this and any warning we try to raise based on `git --version` will frequently be incorrect, don't raise this warning.

I'll note this in the changelog instead.

Test Plan: Looked at setup issues, no more warning for vulnerable git version.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10832

Differential Revision: https://secure.phabricator.com/D15756
2016-04-19 07:01:45 -07:00
epriestley
575c01373e Extract repository command construction from Repositories
Summary:
Ref T4292. Ref T10366. Depends on D15751. Today, generating repository commands is purely a function of the repository, so they use protocols and credentials based on the repository configuration.

For example, a repository with an SSH "remote URI" always generate SSH "remote commands".

This needs to change in the future:

  - After T10366, repositories won't necessarily just have one type of remote URI. They can only have one at a time still, but the repository itself won't change based on which one is currently active.
  - For T4292, I need to generate intracluster commands, regardless of repository configuration. These will have different protocols and credentials.

Prepare for these cases by separating out command construction, so they'll be able to generate commands in a more flexible way.

Test Plan:
  - Added unit tests.
  - Browsed diffusion.
  - Ran `bin/phd debug pull` to pull a bunch of repos.
  - Ran daemons.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292, T10366

Differential Revision: https://secure.phabricator.com/D15752
2016-04-19 04:51:48 -07:00
Aviv Eyal
a3bb35e9d2 make Trigger Daemon sleep correctly when one-time triggers exist
Summary:
Trigger daemon is trying to find the next event to invoke before sleeping, but the query includes already-elapsed triggers.
It then tries to sleep for 0 seconds.

Test Plan:
On a new instance, schedule a single trigger of type `PhabricatorOneTimeTriggerClock` to a very near time.

Use top to see trigger daemon not going to 100% CPU once the event has elapsed.

Reviewers: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15750
2016-04-18 14:17:10 -07:00
Aviv Eyal
091a64e91b Rename Differential field Projects to Tags
Summary: Users can't find the "Tags" field in the Edit Menu; Added keyword "Tag".

Test Plan: Looked in Edit page; I think this shouldn't change anything else?

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15749
2016-04-18 19:33:40 +00:00
epriestley
f424f9f2d2 Record more details about where a write is taking place while holding a cluster lock
Summary: Ref T4292. This will let the UI and future `bin/repository` tools give administrators more tools to understand problems when reporting or resolving them.

Test Plan:
  - Pushed fully clean repository.
  - Pushed previously-pushed repository.
  - Forced write to abort, inspected useful information in the database.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15748
2016-04-18 11:55:27 -07:00
epriestley
368d2d1ddb Improve robustness of cluster version bookkeeping
Summary:
Ref T4292. Small fixes:

  - There was a bug with the //first// write, where we'd write 1 but expect 0. Fix this.
  - Narrow the window where we hold the `isWriting` lock: we don't need to wait for the client to finish.
  - Release the lock even if something throws.
  - Use a more useful variable name.

Test Plan:
  - Made new writes to a fresh cluster repository.
  - Made sequential writes.
  - Made concurrent writes.
  - Made good writes and bad writes.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15747
2016-04-18 11:54:59 -07:00
epriestley
595f203816 Correct RepositoryURI schema and propagate adjust exit code correctly
Summary:
Fixes T10830.

  - The return code from `storage adjust` did not propagate correct.
  - There was one column issue which I missed the first time around because I had a bunch of unrelated stuff locally.

Test Plan:
  - Ran `bin/storage upgrade -f` with failures, used `echo $?` to make sure it exited nonzero.
  - Got fully clean `bin/storage adjust` by dropping all my extra local tables.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10830

Differential Revision: https://secure.phabricator.com/D15746
2016-04-18 08:11:22 -07:00
epriestley
d844e51127 Warn users about remote code execution in older Git
Summary: Ref T10832. Raise a setup warning for out-of-date versions of `git`.

Test Plan: {F1224632}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10832

Differential Revision: https://secure.phabricator.com/D15745
2016-04-18 07:54:13 -07:00
epriestley
9352ed8abb Add missing RepositoryURI table + run storage adjustments in tests
Summary:
Fixes T10830. Ref T10366. I wasn't writing to this table yet so I didn't build it, but the fact that `bin/storage adjust` would complain slipped my mind.

  - Add the table.
  - Make the tests run `adjust`. This is a little slow (a few extra seconds) but we could eventually move some steps like this to run server-side only.

Test Plan: Ran `bin/storage upgrade -f`, got a clean `adjust`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10366, T10830

Differential Revision: https://secure.phabricator.com/D15744
2016-04-18 07:54:02 -07:00
epriestley
fbfe730452 Support more transactions types in RepositoryEditEngine
Summary:
Ref T10748. This supports more transaction types in the modern editor and improves validation so Conduit benefits.

You can technically create repositories via `diffusion.repository.edit` now, although they aren't very useful.

Test Plan:
  - Used `diffusion.repository.edit` to create and edit repositories.
  - Used `/editpro/` to edit repositories.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10748

Differential Revision: https://secure.phabricator.com/D15740
2016-04-17 16:27:02 -07:00
epriestley
92c50de8aa Rough in the new custom URI panel
Summary: Ref T10748. Ref T10366. No support for editing and no impact on the UI, but get some of the basics in place.

Test Plan: {F1223279}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10366, T10748

Differential Revision: https://secure.phabricator.com/D15742
2016-04-17 16:03:24 -07:00
epriestley
51838f990f Copy repository status to a management panel
Summary: Ref T10748. Pretty straightforward. I'd like to put a little "!" icon in the menu if there's a warning/error eventually, but can deal with that latre.

Test Plan: {F1223096}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10748

Differential Revision: https://secure.phabricator.com/D15741
2016-04-17 16:03:03 -07:00
epriestley
e582e9172b Rough in basics + policies + history repository management panels
Summary:
Ref T10748. This is roughly where I'm headed, if it makes some kind of sense? The "Edit" links in sub-sections don't work yet since I haven't built the thing.

Probably depends on D15736.

Test Plan: Manually navigated to `/manage/`, clicked around.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10748

Differential Revision: https://secure.phabricator.com/D15737
2016-04-17 16:02:35 -07:00
epriestley
adf42db5ea Trivially implement RepositoryEditEngine and API methods
Summary: Ref T10748. Ref T10337. This technically implements this stuff, but it does not do anything useful yet. This skips all the hard stuff.

Test Plan:
  - Technically used `diffusion.repository.search` to get repository information.
  - Technically used `diffusion.repository.edit` to change a repository name.
  - Used `editpro/` to edit a repository name.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10337, T10748

Differential Revision: https://secure.phabricator.com/D15736
2016-04-17 16:02:13 -07:00
epriestley
eef2172161 When a user tries to regsiter while logged in, just send them home
Summary: This error message is pointless and dead-ends logged-in users needlessly if they're sent to the register page by documentation or Advanced Enterprise Sales Funnels.

Test Plan: Visited `/auth/register/` while logged in, was sent home.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15739
2016-04-16 19:15:15 -07:00
epriestley
025b243e27 Document wiki relative link syntax
Summary: Also make `../` work to start relative a link so I don't have to document it as `./../path`.

Test Plan:
  - Used `./`, `../`. `./../`, and normal links (proper title pickup).
  - Used bad links (red).
  - Regenerated documentation:

{F1221692}

Reviewers: hach-que

Reviewed By: hach-que

Differential Revision: https://secure.phabricator.com/D15734
2016-04-16 18:54:55 -07:00
Austin Seipp
b2d2f03dea Tell users to avoid magical CloudFlare nonsense in the CDN documentation
Summary:
Fixes T9716. Doesn't go into too much detail, but will hopefully
save some pain.

Test Plan: Read all the wonderful text.

Reviewers: #blessed_committers, epriestley, #blessed_reviewers

Reviewed By: #blessed_committers, epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T9716

Differential Revision: https://secure.phabricator.com/D15738
2016-04-17 01:50:26 +00:00
June Rhodes
27227b8010 Show missing Phriction documents as red links, invisible documents with a lock
Summary: Ref T7691 (errata).  This shows links to Phriction documents in red if they're missing, and links to Phriction documents in grey with a lock icon if the user doesn't have the correct permissions to see the document.

Test Plan:
Tested a bunch of different configurations:

```
[[ ./../ ]] Back to Main Document
[[ ./../subdocument_2]] Mmmm more documents
[[ ./../invisible_document]] Mmmm more documents

[[ ./../ | Explicit Title ]] Back to Main Document
[[ ./../subdocument_2 | Explicit Title ]] Mmmm more documents
[[ ./../invisible_document | Explicit Title ]] Mmmm more documents

[[ ]] Absolute link
[[ subdocument_2 ]] Absolute link
[[ invisible_document ]] Absolute link

[[ | Explicit Title ]] Absolute link
[[ subdocument_2 | Explicit Title ]] Absolute link
[[ invisible_document | Explicit Title ]] Absolute link
```

Got the expected result:

{F1221106}

Reviewers: epriestley, chad, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T7691

Differential Revision: https://secure.phabricator.com/D15733
2016-04-17 01:49:54 +00:00
June Rhodes
dd1023e5a8 Support relative links in Phriction
Summary:
Resolves T7691.  This turned out more complex than I really wanted, mainly because I needed to feed the slug information through to both the document renderer and the preview window that appears in the edit controller.

After this change, you can now create relative links in Phriction by doing `[[ ./../some/relative/path ]]`.  Relative paths aren't handled anywhere else (they'll still render, but the dots are turned into a literal 'dot' as per existing behaviour).

Test Plan: Created some Phriction documents with relative links, saw them all link correctly.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin

Maniphest Tasks: T7691

Differential Revision: https://secure.phabricator.com/D15732
2016-04-16 03:15:01 +00:00
epriestley
d96b6506d7 Disable repository read/write synchronization for now
Summary:
This nearly works but I didn't have time to get back to it and it isn't stable enough to turn on in the cluster yet.

We have enough other stuff going out this week, so just disable it before `stable` gets cut. Should be ready by next week if things go well.

Test Plan: Fetched a Git SSH repo locally.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15731
2016-04-15 15:39:36 -07:00
epriestley
b2db1ec2ca Make bin/aphlict stop read new config properly
Summary: Ref T10697. I missed this so it isn't reading the new config properly.

Test Plan: Ran `bin/aphlict stop`, saw it read config.

Reviewers: chad, Mnkras

Reviewed By: Mnkras

Subscribers: Mnkras

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15729
2016-04-15 15:15:03 -07:00
epriestley
fe40be7fc9 Allow users to be banished from Conpherence rooms
Summary: Fixes T9348. If you have edit permission, you can kick people out of a room.

Test Plan:
  - Kicked people out of a room.
  - As an unprivileged user, wasn't able to kick people out of a room.
  - Hit most (all?) of the various weird dialog sub-cases.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9348

Differential Revision: https://secure.phabricator.com/D15728
2016-04-15 14:57:29 -07:00
epriestley
f146f4577e Fail explicitly instead of continuing with a warning if "phd.user" is misconfigured
Summary: Fixes T6806. We haven't seen users having issues with `phd.user` in a very long time.

Test Plan:
  - Configured daemons to run as `notepriestley`, got a well-explained exception.
  - Configured daemons to run as `epriestley`, got a clean start.
  - Configured daemons with `phd.user=null`, got a clean start.

Reviewers: chad, areitz

Reviewed By: areitz

Subscribers: areitz

Maniphest Tasks: T6806

Differential Revision: https://secure.phabricator.com/D15726
2016-04-15 14:09:13 -07:00
epriestley
d9dd4d427d Improve daemon console for daemons on multiple hosts
Summary:
Ref T10756. This:

  - Fixes T7307. This UI is now admin-only.
  - Makes the main "running daemons" table more useful for multi-host setups (show where daemons are running).
  - Removes logs from the web UI: these are sometimes vaguely sensitive and shouldn't be visible. The UI tells you how to get them with `bin/phd log`.
  - Minor modernization.

Test Plan:
  - As a non-admin, viewed daemons (access error) and bulk jobs (worked great).
  - Browsed bulk job pages.
  - Ran a bulk job.
  - Viewed daemon console.
  - Viewed task detail / daemon detail / daemon list pages.

{F1220516}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7307, T10756

Differential Revision: https://secure.phabricator.com/D15724
2016-04-15 12:19:58 -07:00
lkassianik
42a8776228 Time controls should format end date value correctly (not 5:0 PM)
Summary: Fixes T9296

Test Plan: Create an event, change start time to `3PM`, end value should update to `4:00 PM`, not `4:0 PM`

Reviewers: chad, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T9296

Differential Revision: https://secure.phabricator.com/D15725
2016-04-15 12:18:12 -07:00
lkassianik
7b27653f57 Fixing UI Example that use badge qualities
Summary: Fixes T10706

Test Plan: Open UI Example of badges. Shouldn't be broken

Reviewers: epriestley, #blessed_reviewers, chad

Reviewed By: #blessed_reviewers, chad

Subscribers: Korvin

Maniphest Tasks: T10706

Differential Revision: https://secure.phabricator.com/D15723
2016-04-15 11:05:45 -07:00
epriestley
cd8491ae93 Fix "daemons running as wrong user" setup issue
Summary:
Fixes T9385. This was accidentally mangled a bit a long time ago by D12797, which was a 1,000-file change which got almost everything right.

Simplify the message and fix all the `%s` conversions and how they map to parameters.

Test Plan: {F1220400}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9385

Differential Revision: https://secure.phabricator.com/D15722
2016-04-15 10:03:15 -07:00
epriestley
20bad9a4ba Reset umask to 022 for all Phabricator processes
Summary:
Fixes T7475. If you do something like:

  $ umask 123
  $ ./bin/phd start

...the daemons might inherit the weird umask, do a `git fetch` with the weird umask, and end up creating files with weird permissions in repositories.

Instead, just normalize the umask to 022 in all cases. This is overwhelmingly the most common setting, and the one we assume things are configured with.

(When we want to force permissions to a certain setting, we do so explicitly.)

Test Plan:
  - Added `var_dump(umask())` to observe umask.
  - Ran `bin/phd`, saw proper umask (`18`, which is decimal of `022` octal).
  - Set `umask 123`, then ran `bin/phd`, saw it correct properly again.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T7475

Differential Revision: https://secure.phabricator.com/D15721
2016-04-15 10:03:01 -07:00
Povilas Balzaravicius Pawka
f05c3e41b9 Fixed localcommits include on getDiffDict
Summary: Ref T10808

Test Plan: Call `differential.querydiffs` method and expect 'local:commits' property be added to the result.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Maniphest Tasks: T10808

Differential Revision: https://secure.phabricator.com/D15710
2016-04-15 18:06:32 +03:00
lkassianik
899856a1d4 Links on badge card should be accessible
Summary: Ref T10710

Test Plan: Open user profile with badge, flip badge card, open awarder link.

Reviewers: epriestley, chad, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: chad, Korvin

Maniphest Tasks: T10710

Differential Revision: https://secure.phabricator.com/D15699
2016-04-15 07:55:31 -07:00
epriestley
686c02d54a Add a "memory.hint" parameter to Aphlict
Summary:
Ref T10696. By default, `node` uses 1.5GB, which is enormous overkill for this service and can crowd out other services if it's running next to things like a database on the same host.

Provide a configuration option to adjust it via `--max-old-space-size` and default to 256MB. It only seems to need about 30M locally, so this should be plenty of headroom.

Test Plan:
Ran `bin/aphlict debug`, things seemed OK.

It takes a long time (days?) to grow to 1.5GB so I can't easily test this locally without a lot of work, but I'll keep an eye on it in production.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10696

Differential Revision: https://secure.phabricator.com/D15720
2016-04-15 07:14:45 -07:00
epriestley
0534002894 Add coverage tooltips in Diffusion file browse mode
Summary: Fixes T10816. The way these work is a little unusual since these chunks of file-rendering code are unusuall performance-sensitive, so the Differential version doesn't adapt directly to Diffusion. Both can possibly be unified at some point in the future, although they do slightly different things.

Test Plan: {F1220170}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10816

Differential Revision: https://secure.phabricator.com/D15719
2016-04-15 06:59:38 -07:00
epriestley
fad9e043c3 Fix bad cluster.database in documentation
Summary: This is mistaken.

Test Plan: Careful inspection, `grep` for other mistakes.

Reviewers: chad, eadler

Reviewed By: eadler

Differential Revision: https://secure.phabricator.com/D15717
2016-04-14 15:00:03 -07:00
epriestley
7852ec1619 Use --master-data, not --dump-slave, in bin/storage dump
Summary: These flags do slightly different things, I actually want --master-data here. My test databases are setup half-weird and work with either statement, which is why I missed this.

Test Plan: Ran a dump against master, got the right CHANGE MASTER statement with no warnings.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15716
2016-04-14 14:56:21 -07:00
epriestley
bbb321395a Support Aphlict clustering
Summary:
Ref T6915. This allows multiple notification servers to talk to each other:

  - Every server has a list of every other server, including itself.
  - Every server generates a unique fingerprint at startup, like "XjeHuPKPBKHUmXkB".
  - Every time a server gets a message, it marks it with its personal fingerprint, then sends it to every other server.
  - Servers do not retransmit messages that they've already seen (already marked with their fingerprint).
  - Servers learn other servers' fingerprints after they send them a message, and stop sending them messages they've already seen.

This is pretty crude, and the first message to a cluster will transmit N^2 times, but N is going to be like 3 or 4 in even the most extreme cases for a very long time.

The fingerprinting stops cycles, and stops servers from sending themselves copies of messages.

We don't need to do anything more sophisticated than this because it's fine if some notifications get lost when a server dies. Clients will reconnect after a short period of time and life will continue.

Test Plan:
  - Wrote two server configs.
  - Started two servers.
  - Told Phabricator about all four services.
  - Loaded Chrome and Safari.
  - Saw them connect to different servers.
  - Sent messages in one, got notifications in the other (magic!).
  - Saw the fingerprinting stuff work on the console, no infinite retransmission of messages, etc.

(This pretty much just worked when I ran it the first time so I probably missed something?)

{F1218835}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T6915

Differential Revision: https://secure.phabricator.com/D15711
2016-04-14 13:26:30 -07:00
epriestley
5a0b7398ca Give bin/storage some replica-aware options
Summary:
Fixes T10758.

  - Adds a "--host" flag. If you specify this, we read your cluster config. This lets you dump from a replica.
  - Adds a "--for-replica" flag to `storage dump`. This makes `mysqldump` include a `CHANGE MASTER ...` statement in the output, which is useful when setting up a replica for the first time.

Test Plan:
  - Dumped master and replica cluster databases.
  - Dumped non-cluster databases.
  - Ran various other commands (help, status, etc).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10758

Differential Revision: https://secure.phabricator.com/D15714
2016-04-14 13:23:35 -07:00
epriestley
1b2b84ce1f Use monospaced font in Passphrase "Reveal Secret" dialog
Summary: Fixes T10812. Make it easier to disambiguate great passwords like `iI|l1oO()thenumber1nospellitout`.

Test Plan: {F1219074}

Reviewers: chad, yelirekim

Reviewed By: yelirekim

Maniphest Tasks: T10812

Differential Revision: https://secure.phabricator.com/D15715
2016-04-14 13:09:52 -07:00
Chad Little
7b16f5d807 Convert Project Pages to new UI
Summary: Updating the subproject and member pages in Projects to new UI

Test Plan: Visit a subproject parent page, visit members pages

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15687
2016-04-14 18:26:43 +00:00
Chad Little
eae82c51f5 Minor quality of life updates to Phurl
Summary: Default to "All" (maybe "Active" in the future). Adds more info to results.

Test Plan: visit /phurl/, see additional information about URL

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15713
2016-04-14 18:25:22 +00:00
epriestley
07fc8f17cc Support "ssl.chain" in Aphlict configuration
Summary: Fixes T10806. Although browsers don't seem to care about this, it's more correct to support it, and the new test console uses normal `cURL` and does care.

Test Plan:
  - Hit the error case for providing a chain but no key/cert.
  - Used `openssl s_client -connect localhost:22280` to connect to local Aphlict servers.
  - With SSL but no chain, saw `openssl` fail to verify the remote.
  - With SSL and a chain, saw `openssl` verify the identify of the remote.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10806

Differential Revision: https://secure.phabricator.com/D15709
2016-04-14 10:41:21 -07:00
epriestley
383ae7621f Fix a "websokket" typo
Summary: Typo fix from D15703 that I overlooked.

Test Plan: Careful inspection.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15708
2016-04-14 10:40:54 -07:00
epriestley
ca6da4c2be When proxying the SVN protocol, don't mutate URIs in protocol frames if we're an intracluster proxy
Summary:
Ref T10809. Currently, both the proxy and target may mutate URIs (rewriting "svn+ssh://x/diffusion/Y/" to a path on disk).

I believe this previously worked by fate/chance/luck since both URI variants contain the repository information, but the algorithms were tightened up recently with callsign removal.

Stop rewriting them if we're the intracluster proxy -- they only need to be rewritten on the target host.

Test Plan:
  - Checked out a proxied SVN repository, with and without a callsign.
  - Checked out an unproxied SVN repository, with and without a callsign.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10809

Differential Revision: https://secure.phabricator.com/D15712
2016-04-14 10:15:23 -07:00
epriestley
d4bf2a147b Make paths and Aphlict instance names less ambiguous
Summary:
Fixes T10783 (what little of it remains). Ref T10697.

Aphlict currently uses request paths for two different things:

  - multi-tenant instancing in the Phacility cluster (each instance gets its own namespace within an Aphlict server);
  - some users configure nginx and apache to do proxying or SSL termination based on the path.

Currently, these can collide.

Put a "~" before the instance name to make it unambiguous. At some point we can possibly just use a GET parameter, but I think there was some reason I didn't do that originally and this sequence of changes is disruptive enough already.

Test Plan: Saw local Aphlict unambiguously recognize "local.phacility.com" as instance "local", with a "~"-style URI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697, T10783

Differential Revision: https://secure.phabricator.com/D15705
2016-04-14 04:57:21 -07:00