1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-23 14:00:56 +01:00
Commit graph

527 commits

Author SHA1 Message Date
Joshua Spence
f8b7f03145 Change "lint save" to not use Arcanist Projects
Summary: Ref T7604. Change `DiffusionLintSaveRunner` to use repositories instead of Arcanist Projects.

Test Plan: Ran the `save_lint.php` script and queried results using the `diffusion.getlintmessages` Conduit endpoint.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T7604

Differential Revision: https://secure.phabricator.com/D12893
2015-05-19 00:07:47 +10:00
Joshua Spence
70c8649142 Use phutil_json_decode instead of json_decode
Summary: Generally, `phutil_json_decode` should be preferred over `json_decode`.

Test Plan: Eyellballed.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D12680
2015-05-05 20:48:55 +10:00
Joshua Spence
1dfe1f49f7 Fix a few issues with the "import symbols" script
Summary: Ref T7977. Fix a few issues that I forgot to fix up.

Test Plan: Run the script.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7977

Differential Revision: https://secure.phabricator.com/D12668
2015-05-04 12:22:11 +10:00
Joshua Spence
2483f6f120 Move symbols to be repository-based
Summary: Fixes T7220. Ref T7977. Changes symbols from being bound to an Arcanist project to being bound to a repository.

Test Plan:
- Added symbols and then applied migrations, symbols seemed to be migrated successfully.
- Tested the `/diffusion/symbol/$SYMBOL_NAME` endpoint.
- Tested the `/diffusion/symbol/$SYMBOL_NAME` endpoint with the `?repositories=$REPOSITORY_PHID` parameter.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: avivey, Korvin, epriestley

Maniphest Tasks: T7977, T7220

Differential Revision: https://secure.phabricator.com/D12608
2015-05-03 13:23:07 +10:00
epriestley
a804f0ab93 Make file policies for emailed files more consistent
Summary:
Fixes T7712. Currently, files sent via email get default policies, like they were dragged and dropped onto the home page.

User expectation is better aligned with giving files more restrictive policies, like they were draggged and dropped directly onto an object.

Make files sent via email have restricted default visibility. Once we identify the sender, set them as the file author. Later, the file will become visible to other users via attachment to a task, revision, etc.

Test Plan: Sent some files via email; verified they got restrictive policies, correct authorship, and appropriate object attachment.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7712

Differential Revision: https://secure.phabricator.com/D12255
2015-04-02 13:41:39 -07:00
epriestley
09f3d0bb7e Pass overseer configuration over stdin
Summary:
Ref T7352. This changes `phd` to pass configuration to overseers over stdin. We still run one overseer per daemon.

The "status" stuff needs some cleanup, but it's mostly just UI/cosmetic.

Test Plan:
  - Ran `phd debug`, `phd launch`, `phd start`, `phd status`, `phd stop`, etc.
  - Verified PID files write in a reasonable format.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11855
2015-02-24 14:50:30 -08:00
epriestley
02b174c2af Allow a different SSH host to be set in Diffusion
Summary:
Ref T6941. In the cluster (and in other reasonable setups) we've separated SSH load balancers from HTTP load balancers.

In particular, ELBs will not let you load balance port 22, so this is likely a reasonable/common issue in larger clusters in AWS.

Allow users to specify an alternate host for SSH traffic.

Test Plan: Set host to someting different, saw it reflected in UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6941

Differential Revision: https://secure.phabricator.com/D11800
2015-02-18 10:51:14 -08:00
epriestley
8e2f054fe4 Remove TERM=dumb, which is causing difficult-to-reproduce hangs
Summary:
Ref T7119. Three users have now reported that this causes their systems to hang, so it's looking like the cure is worse than the disease.

Revert this until we can eventually reproduce and understand the hang.

Test Plan: Users running into issues have reported that this resolves things.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7119

Differential Revision: https://secure.phabricator.com/D11644
2015-02-03 09:54:02 -08:00
Joshua Spence
7982b23eb4 Use PhutilXHPASTBinary methods
Summary: Use `PhutilXHPASTBinary` methods instead of `xhpast_parse` functions. Depends on D11517.

Test Plan: N/A, this is a direct swap.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11612
2015-02-03 06:59:16 +11:00
epriestley
d804598f17 Add some of a billing daemon skeleton
Summary:
Ref T6881. This adds the worker, and a script to make it easier to test. It doesn't actually invoice anything.

I'm intentionally allowing the script to double-bill since it makes testing way easier (by letting you bill the same period over and over again), and provides a tool for recovery if billing screws up.

(This diff isn't very interesting, just trying to avoid a 5K-line diff at the end.)

Test Plan: Used `bin/phortune invoice ...` to get the worker to print out some date ranges which it would theoretically invoice.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6881

Differential Revision: https://secure.phabricator.com/D11577
2015-01-30 11:29:05 -08:00
epriestley
8798083ad9 Proxy VCS SSH requests
Summary: Fixes T7034. Like HTTP, proxy requests to the correct host if a repository has an Almanac service host.

Test Plan: Ran VCS requests through the proxy.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7034

Differential Revision: https://secure.phabricator.com/D11543
2015-01-28 14:41:24 -08:00
epriestley
9b359affe7 Prepare SSH connections for proxying
Summary:
Ref T7034.

In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request.

In order to proxy the request, we need to know which repository the user is interested in accessing.

Split the SSH workflow into two steps:

  # First, identify the repository.
  # Then, execute the operation.

In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit.

This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line.

This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically:

  - The client doesn't tell us what it's after.
  - To get it to tell us, we have to send it a server capabilities string //first//.
  - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository.
  - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame.
  - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do.

The approach here is straightforward, but the implementation is not trivial. Roughly:

  - Start `svnserve`, read the "hello" frame from it.
  - Kill `svnserve`.
  - Send the "hello" to the client.
  - Wait for the client to send us "I want repository X".
  - Save the message it sent us in the "peekBuffer".
  - Return "this is a request for repository X", so we can proxy it.

Then, to continue the request:

  - Start the real `svnserve`.
  - Read the "hello" frame from it and throw it away.
  - Write the data in the "peekBuffer" to it, as though we'd just received it from the client.
  - State of the world is normal again, so we can continue.

Also fixed some other issues:

  - SVN could choke if `repository.default-local-path` contained extra slashes.
  - PHP might emit some complaints when executing the commit hook; silence those.

Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7034

Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 10:18:07 -08:00
epriestley
834079f766 Pass cluster.instance to ssh-exec if it is defined
Summary: Ref T7034. This is a second special case, like commit hooks, where we need some help from Phabricator to make instance identity knowable.

Test Plan: Connected to an instance and ran SSH commands.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7034

Differential Revision: https://secure.phabricator.com/D11539
2015-01-28 10:17:40 -08:00
epriestley
d8550c114d Promote instance identity to the upstream and pass it to commit hooks
Summary:
Fixes T7019. In a cluster environment, pushes currently fail because the commit hook can't identify the instance.

For web processes, the hostname identifies the instance -- but we don't have a hostname in the hook.

For CLI processes, the environment identifies the instance -- but we don't have an environment in the hook under SVN.

Promote the instance identifier into the upstream and pack/unpack it explicitly for hooks. This is probably not useful for anyone but us, but the amount of special-purpose code we're introducing is very small.

I poked at trying to do this in a more general way, but:

  - We MUST know this BEFORE we run code, so the normal subclassing stuff is useless.
  - I couldn't come up with any other parameter which might ever be useful to pass in.

Test Plan: Used `git push` to push code through proxied HTTP, got a clean push.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7019

Differential Revision: https://secure.phabricator.com/D11495
2015-01-27 14:51:48 -08:00
Chad Little
32b85c065e Delete app icons
Summary: Unused at this point

Test Plan: Grep

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11506
2015-01-27 12:36:39 -08:00
Chad Little
6018ef91b8 Remove 1x AppIcons, use FontIcons instead
Summary: Removes the 1x application icons, and uses the fonticons instead. Feed was only known location.

Test Plan:
feed, dashboards, grep for use

{F275636}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11496
2015-01-25 14:14:41 -08:00
epriestley
934df0e735 Add bin/trigger, for testing event triggers
Summary:
Ref T6881. This makes it easier to fire a trigger and make sure it works properly. You can use the `--now` flag to travel through time, and test scheduling conditions with `--last` and `--next`. It will tell you when the trigger would reschedule.

Better than waiting 24 hours to see if things work.

Test Plan: Fired some backups, got useful output which made me think my code probably works correctly.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6881

Differential Revision: https://secure.phabricator.com/D11438
2015-01-20 11:31:32 -08:00
Bob Trahan
648fa2e1bc Repositories - Move scripts/repository/reparse.php to bin/repository reparse
Summary:
Fixes T5966. Accomplishes a few things

 - see title
 - adds a force-autoclose flag and the plumbing for it
 - removes references to some HarborMaster thing that used to key off commits and seems long dead, but forgotten :/

Test Plan:
ran a few commands. These first three had great success:

`./repository reparse --all FIRSTREPO --message --change  --herald --owners`
`./repository reparse --all FIRSTREPO --message --change  --herald --owners --min-date yesterday`
`./repository reparse --all FIRSTREPO --message --change  --herald --owners --min-date yesterday --force-autoclose`

...and these next two showed me some errors as expected:

`./repository reparse --all FIRSTREPO --message --change  --herald --owners --min-date garbagedata`
`./repository reparse --all GARBAGEREPO --message --change  --herald --owners`

Also, made a diff in a repository with autoclose disabled and commited the diff. Later, reparse the diff with force-autoclose. Verified the diff closed and that the reason "why" had the proper message text.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: joshuaspence, epriestley, Korvin

Maniphest Tasks: T5966

Differential Revision: https://secure.phabricator.com/D10492
2015-01-06 11:42:15 -08:00
Joshua Spence
e12f0bb53a Correctly pass worker priority from reparse script
Summary: D11044 broke the `./scripts/repository/reparse.php` script.

Test Plan: `./scripts/repository/reparse.php --message --all X`

Reviewers: btrahan, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11211
2015-01-05 08:34:05 +11:00
Joshua Spence
39ca2fdf64 Use new FutureIterator instead of Futures
Summary: Ref T6829. Deprecate the `Futures()` function.

Test Plan: N/A

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T6829

Differential Revision: https://secure.phabricator.com/D11077
2014-12-30 23:13:38 +11:00
Chad Little
b925f42c55 Remove docs sprite, replace with FontAwesome
Summary: Removes the docs sprite in Conpherence with FontAwesome, adds additional icons. Unsure what happens if someone customized this config option.

Test Plan: Added images and files to a Conpherence, saw new icons.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11028
2014-12-20 20:59:24 -08:00
Chad Little
1e0f1f4936 Remove payments sprite
Summary: Removes unused payments sprite and code, also some unused conpherence generated images. We use images in login (and could use FontAwesome, maybe).

Test Plan: grep codebase, pull up uiexamples icons page

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11025
2014-12-20 09:44:53 -08:00
Chad Little
f07f7bfaa4 Remove unused Conpherence sprites
Summary: These were refactored out a while ago

Test Plan: Grep codebase, use Conpherence on desktop, mobile.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11023
2014-12-20 07:44:50 -08:00
Joshua Spence
54f8aa8cd9 Minor linter fixes
Summary: Self explanatory

Test Plan: `arc lint`

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D10946
2014-12-09 18:37:32 +11:00
epriestley
7e1c312183 Add bin/worker flood, for flooding the task queue with work
Summary: Ref T6615. Ref T3554. We need better tooling around the queue eventually, so start here.

Test Plan: Added 100K+ tasks locally with `bin/worker flood`. Executed some of them with `bin/phd debug taskmaster` (we already have a TestWorker, used in unit tests).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3554, T6615

Differential Revision: https://secure.phabricator.com/D10894
2014-11-24 11:10:15 -08:00
epriestley
bf17b12daf Standardize SSH key storage
Summary:
Ref T5833. This fixes a few weird things with this table:

  - A bunch of columns were nullable for no reason.
  - We stored an MD5 hash of the key (unusual) but never used it and callers were responsible for manually populating it.
  - We didn't perform known-key-text lookups by using an index.

Test Plan:
  - Ran migrations.
  - Faked duplicate keys, saw them clean up correctly.
  - Added new keys.
  - Generated new keys.
  - Used `bin/auth-ssh` and `bin/auth-ssh-key`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5833

Differential Revision: https://secure.phabricator.com/D10805
2014-11-07 15:34:44 -08:00
epriestley
6f0d3b0796 Add a query/policy layer on top of SSH keys for Almanac
Summary:
Ref T5833. Currently, SSH keys are associated only with users, and are a bit un-modern. I want to let Almanac Devices have SSH keys so devices in a cluster can identify to one another.

For example, with hosted installs, initialization will go something like this:

  - A request comes in for `company.phacility.com`.
  - A SiteSource (from D10787) makes a Conduit call to Almanac on the master install to check if `company` is a valid install and pull config if it is.
  - This call can be signed with an SSH key which identifies a trusted Almanac Device.

In the cluster case, a web host can make an authenticated call to a repository host with similar key signing.

To move toward this, put a proper Query class on top of SSH key access (this diff). In following diffs, I'll:

  - Rename `userPHID` to `objectPHID`.
  - Move this to the `auth` database.
  - Provide UI for device/key association.

An alternative approach would be to build some kind of special token layer in Conduit, but I think that would be a lot harder to manage in the hosting case. This gives us a more direct attack on trusting requests from machines and recognizing machines as first (well, sort of second-class) actors without needing things like fake user accounts.

Test Plan:
  - Added and removed SSH keys.
  - Added and removed SSH keys from a bot account.
  - Tried to edit an unonwned SSH key (denied).
  - Ran `bin/ssh-auth`, got sensible output.
  - Ran `bin/ssh-auth-key`, got sensible output.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5833

Differential Revision: https://secure.phabricator.com/D10790
2014-11-06 12:37:02 -08:00
epriestley
917da08417 Fix various MySQL version issues with new charset stuff
Summary:
Ref T1191. Notable stuff:

  - Adds `--disable-utf8mb4` to `bin/storage` to make it easier to test what things will (approximately) do on old MySQL. This isn't 100% perfect but should catch all the major stuff. It basically makes us pretend the server is an old server.
  - Require utf8mb4 to dump a quickstart.
  - Fix some issues with quickstart generation, notably special casing the FULLTEXT handling.
  - Add an `--unsafe` flag to `bin/storage adjust` to let it truncate data to fix schemata.
  - Fix some old patches which don't work if the default table charset is utf8mb4.

Test Plan:
  - Dumped a quickstart.
  - Loaded the quickstart with utf8mb4.
  - Loaded the quickstart with `--disable-utf8mb4` (verified that we get binary columns, etc).
  - Adjusted schema with `--disable-utf8mb4` (got a long adjustment with binary columns, some truncation stuff with weird edge case test data).
  - Adjusted schema with `--disable-utf8mb4 --unsafe` (got truncations and clean adjust).
  - Adjusted schema back without `--disable-utf8mb4` (got a long adjustment with utf8mb4 columns, some invalid data on truncated utf8).
  - Adjusted schema without `--disable-utf8mb4`, but with `--unsafe` (got truncations on the invalid data).

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D10757
2014-10-29 15:49:29 -07:00
Joshua Spence
3cf9a5820f Minor formatting changes
Summary: Apply some autofix linter rules.

Test Plan: `arc lint` and `arc unit`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D10585
2014-10-08 08:39:49 +11:00
James Rhodes
8fbebce501 Implement storage of a host ID and a public key for authorizing Conduit between servers
Summary:
Ref T4209.  This creates storage for public keys against authorized hosts, such that servers can be authorized to make Conduit calls as the omnipotent user.

Servers are registered into this system by running the following command once:

```
bin/almanac register
```

NOTE: This doesn't implement authorization between servers, just the storage of public keys.

Placing this against Almanac seemed like the most sensible place, since I'm imagining in future that the `register` command will accept more information (like the hostname of the server so it can be found in the service directory).

Test Plan: Ran `bin/almanac register` and saw the host (and public key information) appear in the database.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4209

Differential Revision: https://secure.phabricator.com/D10400
2014-10-03 22:52:41 +10:00
Joshua Spence
6f806bd12b Use PhutilConsoleProgressBar for the repository/reparse.php script
Summary: The output from this script is too verbose... all that I care about is the overall progress.

Test Plan:
Ran the script on a large repository.

```lang=bash
./scripts/repository/reparse.php --all XYZ --message
NOTE: This script will queue tasks to reparse the data. Once the tasks have been queued, you need to run Taskmaster daemons to execute them.

QUEUEING TASKS (92,969 Commits):
[                                                                  ]   0.1%
```

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10441
2014-09-09 00:04:25 +10:00
epriestley
6be8d65763 Convert two missed phutil_utf8_shorten() callsites
Summary: Fixes T6006. These didn't get caught by D10392.

Test Plan: Forced migration to re-run; ran SSH commands against Phabricator.

Auditors: btrahan
2014-08-30 07:20:35 -07:00
epriestley
eaacb4a511 Replace ActionHeader minicons with Font icons
Summary:
Ref T5885. See D10276.

Currently, ActionHeaders can only have minicons, and we don't use them anywhere and they probably don't make much sense in the product anymore.

Instead, allow them to have font icons. Remove minicons, which have no callsites and probably won't in the future.

Test Plan:
{F190925}

  - Grepped for `minicons`.
  - Grepped for `setHeaderIcon()`.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5885

Differential Revision: https://secure.phabricator.com/D10277
2014-08-15 11:08:28 -07:00
epriestley
237b1d769b Improve bin/storage upgrade behavior when run out-of-order
Summary:
Fixes T5770. This error occurs if you run `bin/storage upgrade` before you set up MySQL credentials.

This isn't what the setup guide says to do, but it's an easy mistake to make and should be a permitted install path since there's no reason you can't do things in this order.

Specifically, we use a mixture of "standard" (configured) and "administrative" (`--user` and `--password`) credentials, and if the standard ones are bogus bad things happen. We use the standard credentials to make some initialization order stuff easier, and because there's no `--host` flag and adding one would be silly, and because we only need administrative credentials to issue ALTER / CREATE statements.

Test Plan: Ran with bad standard credentials; ran with bad administrative credentials. Ran with good credentials.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5770

Differential Revision: https://secure.phabricator.com/D10199
2014-08-08 15:44:40 -07:00
Mukunda Modell
12aaa942ac Add a CanCDN flag to uploaded files
Summary:
CanCDN flag indicates that a file can be served + cached
via anonymous content distribution networks.

Once D10054 lands, any files that lack the CanCDN flag
will require a one-time-use token and headers will
prohibit cache to protect sensitive files from
unauthorized access.

This diff separates the CanCDN changes from the code that
enforces these restrictions in D10054 so that the changes
can be tested and refined independently.

Test Plan: Work in progress

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: rush898, qgil, epriestley, aklapper, Korvin

Maniphest Tasks: T5685

Differential Revision: https://secure.phabricator.com/D10166
2014-08-07 18:56:20 -07:00
Joshua Spence
9a679bf374 Allow worker tasks to have priorities
Summary: Fixes T5336. Currently, `PhabricatorWorkerLeaseQuery` is basically FIFO. It makes more sense for the queue to be a priority-queue, and to assign higher priorities to alerts (email and SMS).

Test Plan: Created dummy tasks in the queue (with different priorities). Verified that the priority field was set correctly in the DB and that the priority was shown on the `/daemon/` page. Started a `PhabricatorTaskmasterDaemon` and verified that the higher priority tasks were executed before lower priority tasks.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5336

Differential Revision: https://secure.phabricator.com/D9871
2014-07-12 03:02:06 +10:00
Joshua Spence
97bb13d1d7 Discover workflows automatically.
Summary: Most scripts detect the relevant workflows automatically. Some scripts, however, use a hardcoded list of workflows.

Test Plan: Ran `./bin/aphlict`, `./bin/cache` and `./bin/phd`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9564
2014-06-16 09:00:25 +10:00
Joshua Spence
d0128afa29 Applied various linter fixes.
Summary: Applied some more linter fixes that I previously missed because my global `arc` install was out-of-date.

Test Plan: Will run `arc unit` on another host.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9443
2014-06-09 16:04:12 -07:00
Joshua Spence
0a62f13464 Change double quotes to single quotes.
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.

Test Plan: Eyeballed it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9431
2014-06-09 11:36:50 -07:00
epriestley
fcc2a24d63 Update Aphlict documentation for new server management commands
Summary:
The docs are now a little out of date.

Also //possibly// we should call this `bin/notifications` or something, maybe?

Test Plan: read

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9398
2014-06-05 12:40:53 -07:00
Chad Little
dbebfea9cc FontAwesome RemarkupBar
Summary:
Replaces the icons with fonts from FontAwesome. Up in the air about the meme icon. Thoughts?

Also removed the second fullscreen/normal state. Seems obvious what it does, but assume someone complained previously?

Test Plan:
Tested all the icon states and made sure they still worked. Test fullscreen and help.

{F163485}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9385
2014-06-05 10:37:21 -07:00
Chad Little
14dab3f55c Convert ButtonBar to FontAwesome
Summary: Removes the sprites and images, uses fontawesome in examples and calendar

Test Plan:
UIExamples, Calendar

{F163375}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9371
2014-06-04 12:53:32 -07:00
epriestley
5b1262c98b Add a bin/hunks script to manage migrations of hunk data
Summary:
Ref T4045. Ref T5179. While we'll eventually need to force a migration, we can let installs (particularly large installs) do an online migration for now. This moves hunks to the new storage format one at a time.

(Note that nothing writes to the new store yet, so this is the only way to populate it.)

WARNING: Installs, don't run this yet! It won't compress the data. Wait until it can also do compression.

Test Plan: Added a `break;` after migrating one row and moved a few rows over. Spot checked them in the database and viewed the affected diffs.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4045, T5179

Differential Revision: https://secure.phabricator.com/D9291
2014-06-03 18:01:23 -07:00
epriestley
e3d7c16d8c Use "https://" instead of "git://" for anoymous GitHub requests in documentation
Summary: These both work, but "git://" uses a nonstandard and possibly firewalled port, is less familiar to users, and is not promoted in the GitHub UI.

Test Plan: `grep`, reviewed diff.

Reviewers: chad, avive, avivey

Reviewed By: avivey

Subscribers: epriestley, avivey

Differential Revision: https://secure.phabricator.com/D9360
2014-06-03 09:40:16 -07:00
epriestley
63ed126b2a Point github.com/facebook URIs at github.com/phacility insead
Summary: Point everything at the new canonical URI.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9328
2014-05-29 08:33:25 -07:00
Chad Little
ac05fe9c3b Replace action sprite with FontAwesome
Summary: Replaces the action icons in action headers with FontAwesome

Test Plan:
- grep SPRITE_ACTIONS
- grep sprite-actions
- Replace on UIExamples
- Replace on Workboards
- Replace on Dashboards
- Replace on FeedStories

{F157840}
{F157841}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9230
2014-05-21 10:18:43 -07:00
Chad Little
3a81f8c68d Convert rest of SPRITE_STATUS to FontAwesome
Summary:
Updates policy, headers, typeaheads to FA over policy icons

Need advice - can't seem to place where icons come from on Typeahead? Wrong icons and wrong colors.... it is late

Test Plan:
- grepped for SPRITE_STATUS
- grepped for sprite-status
- grepped for setStatus for headers
- grepped individual icons names

Browsed numerous places, checked new dropdowns, see pudgy people.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4739

Differential Revision: https://secure.phabricator.com/D9179
2014-05-18 16:10:54 -07:00
Chad Little
b2f3001ec4 Replace Sprite-Icons with FontAwesome
Summary: The removes the sprite sheet 'icons' and replaces it with FontAwesome fonts.

Test Plan:
- Grep for SPRITE_ICONS and replace
- Grep for sprite-icons and replace
- Grep for PhabricatorActionList and choose all new icons
- Grep for Crumbs and fix icons
- Test/Replace PHUIList Icon support
- Test/Replace ObjectList Icon support (foot, epoch, etc)
- Browse as many pages as I could get to
- Remove sprite-icons and move remarkup to own sheet
- Review this diff in Differential

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9052
2014-05-12 10:08:32 -07:00
Bob Trahan
e96c363eef Add SMS support
Summary:
Provides a working SMS implementation with support for Twilio.

This version doesn't really retry if we get any gruff at all. Future versions should retry.

Test Plan: used bin/sms to send messages and look at them.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: aurelijus, epriestley, Korvin

Maniphest Tasks: T920

Differential Revision: https://secure.phabricator.com/D8930
2014-05-09 12:47:21 -07:00
epriestley
889440ead0 Allow structured destruction of Differential Revisions
Summary:
Ref T4749. Ref T3265. Ref T4909.

  - Remove old "destroy revision" script.
  - Move to structured `bin/remove` destruction.
  - Fix some edge issues.
  - Add transaction destruction support.

Test Plan:
  - Destroyed a bunch of revisions.
  - Saw diffs, changesets, hunks, transactions, edges, and inlines also get wiped out.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4749, T4909, T3265

Differential Revision: https://secure.phabricator.com/D8943
2014-05-01 18:25:30 -07:00
epriestley
827fbb3782 Remove various old things in scripts/
Summary: These have all been obsolete for a reasonable amount of time, or are no longer relevant.

Test Plan: shrug~

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8941
2014-05-01 18:23:48 -07:00
epriestley
2022a70e16 Implement bin/remove, for structured destruction of objects
Summary:
Ref T4749. Ref T3265. Ref T4909. Several goals here:

  - Move user destruction to the CLI to limit the power of rogue admins.
  - Start consolidating all "destroy named object" scripts into a single UI, to make it easier to know how to destroy things.
  - Structure object destruction so we can do a better and more automatic job of cleaning up transactions, edges, search indexes, etc.
  - Log when we destroy objects so there's a record if data goes missing.

Test Plan: Used `bin/remove destroy` to destroy several users.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3265, T4749, T4909

Differential Revision: https://secure.phabricator.com/D8940
2014-05-01 18:23:31 -07:00
Felipe Bessa Coelho
f27f7dce52 Install PHP mbstring extension on RHEL & friends
Summary:
The mbstring extension for PHP is not a dependency to any of the already
listed packages on RHEL-like systems, and is needed by Phabricator
(showing a "install mbstring" message as the first thing if it is not
installed)

RHEL seems to have some extra steps to allow php-mbstring to be installed, though:

http://snippets.roozbehk.com/post/35750940300/php-mbstring-missing-on-red-hat-enterprise-linux-6

PS: disabled lint for this change because of the already >80 chars long "yum install" string

Test Plan:
* Created a new container with docker using both centos:6.4 and fedora:20 images
* Ran install script
* Started httpd and mysqld services
* Browsed to server's address
* Got error message
* Installed php-mbstring & restarted httpd
* Works

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8772
2014-04-14 14:59:27 -07:00
epriestley
0fd12f9cad Remove nonfunctional/obsolete 'reconcile.php'
Summary: Ref T4780. This no longer works and barely ever worked. T4237 is now the relevant task.

Test Plan: Grepped for `reconcile.php`

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4780

Differential Revision: https://secure.phabricator.com/D8739
2014-04-09 16:12:29 -07:00
epriestley
af0b749369 Fix many lies in the "User Roles" document
Summary: Fixes T3047. Update this document and remove some lies ("menu bar is read in admin interfaces"!!!!).

Test Plan:
  - Read text.
  - Searched for "System Agent" in the UI and replaced it with "bot" or "bot/script" or similar.

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3047

Differential Revision: https://secure.phabricator.com/D8675
2014-04-02 12:06:56 -07:00
Joshua Spence
6270114767 Various linter fixes.
Summary:
- Removed trailing newlines.
- Added newline at EOF.
- Removed leading newlines.
- Trimmed trailing whitespace.
- Spelling fix.
- Added newline at EOF

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: hach-que, chad, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8344
2014-02-26 12:44:58 -08:00
Aviv Eyal
361e49dd67 Symbol import: preemptively error on non-utf8 symbols
Summary:
Put a test somewhere we can --ignore it.

Also fix path tests.

Test Plan: insert good/bad values at various positions.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8353
2014-02-26 12:41:55 -08:00
Chad Little
eec593e313 Add ButtonBar icons
Summary: Adds an icon set for icons on button bars.

Test Plan: photoshop

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4394

Differential Revision: https://secure.phabricator.com/D8170
2014-02-09 10:22:12 -08:00
epriestley
0726411cb4 Write a very basic string extractor
Summary: Ref T1139. This has some issues and glitches, but is a reasonable initial attempt that gets some of the big pieces in. We have about 5,200 strings in Phabricator.

Test Plan: {F108261}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D8138
2014-02-05 11:02:41 -08:00
Ricky Elrod
6b4998bf4b Increment copyright year and add 2017 dates to calendar import script
Summary: D4335 except a year later.

Test Plan: Nope.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8106
2014-01-30 09:05:25 -08:00
Eric Stern
14f070a0af Skip anon functions in symbol generation script
Summary:
Filters closures out of symbol generator script, per @epriestley's
comment in T4334

Test Plan:
Before:
  eric@eric-dev ~/phabricator/scripts/symbols: echo 'closure.php' | ./generate_php_symbols.php
    function php  /closure.php
   d function php 10 /closure.php
    function php  /closure.php
   a class php 3 /closure.php
  a b method php 4 /closure.php

After:
  eric@eric-dev ~/phabricator/scripts/symbols: echo 'closure.php' | ./generate_php_symbols.php
   d function php 10 /closure.php
   a class php 3 /closure.php
  a b method php 4 /closure.php

eric@eric-dev ~/phabricator/scripts/symbols: cat closure.php
  <?php

  class a {
    function b() {
      $c = function() { return 1; };
      $c();
    }
  }

  function d() {
    return 2;
  }
  $e = function() {
    return 3;
  };

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: epriestley, Korvin, aran

Differential Revision: https://secure.phabricator.com/D8054
2014-01-23 17:01:11 -08:00
Andrej E Baranov
89bd84986a Fix typo
Summary: 'daeons' to 'daemons'

Test Plan: run `phd help`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: epriestley

CC: epriestley, aran

Differential Revision: https://secure.phabricator.com/D7973
2014-01-15 10:03:50 -08:00
John Mullanaphy
b960c8114b Changed \$callsign to \$argv[1] on commit_hook.php since is undefined causing an error when trying to report an error.
Summary:
Phabricator was going to give me an error message via commit_hook.php, unfortunately said error wasn't being set since
\$callsign was undefined. So, just changed \$callsign to \$argv[1] and now I get the appropriate commit.

Test Plan:
1. Add commit_hook.php to an SVN pre-commit.
2. Set the SVN to be hosted off of Phabricator.
3. Attempt to commit to commit to SVN repository.
Expected: Error message saying that the repository isn't hosted on Phabricator
Results: Error message saying undefined function.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7920
2014-01-09 10:36:01 -08:00
epriestley
972dfa7bfc Add 'hook.d/' directories to SVN and Git repositories for custom hooks
Summary:
Fixes T4189. Ref T4151. Allows repositories to have additional custom hooks for operations which can't be expressed with Herald (one such operation is lint).

This adds only local hook directories, since they're easier to use with existing hooks than global directories. I might add global directories eventually.

This doesn't support Mercurial since we have no demand for it and it's more complicated (we lose compatibility and power by just dropping a `hooks.d/` somewhere).

Test Plan:
  - Pulled hosted SVN and Git repos to verify the hook directories generate correctly.
  - Added a variety of hooks to the hook directories (echo + pass, fail).
  - Pushed commits and verified the hooks fired (output expected info, or failed).
  - Verified push log reflected the correct error code ("3", external) and detail ("nope.sh") when rejecting.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4151, T4189

Differential Revision: https://secure.phabricator.com/D7884
2014-01-03 12:26:10 -08:00
epriestley
2cfc3acf32 Allow Herald pre-commit rules to act on repository projects
Summary:
Fixes T4264. Adds:

  - New "Repository's projects" field to Herald pre-commit rules, so you can write global rules which act based on projects.
  - Allows pre-ref/pre-content rules to bind to projects, and fire for all repositories in that project, so users with limited power can write rules which apply to many repositories.
  - The pre-ref and pre-content classes were starting to share a fair amount of code, so I made them both extend an abstract base class.

Test Plan: Wrote new pre-ref and pre-content rules bound to projects, then pushed commits into repositories in those projects and not in those projects. The "repository projects" field populated, and the rules fired for repositories in the relevant projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4264

Differential Revision: https://secure.phabricator.com/D7883
2014-01-03 12:24:28 -08:00
lesha
8c114394e3 Enable Evil^H^H^H^HJSON for PHP on Ubuntu 13.10
Summary:
Good news. Starting with Ubuntu 13.10, Phabricator can legally be used by evil dictators, mad scientists, and toxic derivative creators.

The JSON implementation prohibiting evil (http://www.phoronix.com/scan.php?page=news_item&px=MTQ0MTY) was ripped out and replaced by the Evil-friendly PHP license: https://github.com/remicollet/pecl-json-c/blob/master/LICENSE

Test Plan: ran the shell script, Phabricator no longer fails with "Call to undefined function json_decode".

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7878
2014-01-02 11:59:12 -08:00
epriestley
2c35532256 Drive all Celerity operations from the new map
Summary:
Ref T4222.

  - Removes the old map and changes the CelerityResourceMap API to be entirely driven by the new map.
  - The new map is about 50% smaller and organized more sensibly.
  - This removes the `/pkg/` URI component. All resources are now required to have unique names, so we can tell if a resource is a package or not by looking at the name.
  - Removes some junky old APIs.
  - Cleans up some other APIs.
  - Added some feedback for `bin/celerity map`.
  - `CelerityResourceMap` is still a singleton which is inextricably bound to the Phabricator map; this will change in the future.

Test Plan:
  - Reloaded pages.
  - Verified packaging works by looking at generated includes.
  - Forced minification on and verified it worked.
  - Forced no-timestamps on and verified it worked.
  - Rebuilt map.
  - Ran old script and verified error message.
  - Checked logs.

Reviewers: btrahan, hach-que

Reviewed By: hach-que

CC: chad, aran

Maniphest Tasks: T4222

Differential Revision: https://secure.phabricator.com/D7872
2013-12-31 18:04:25 -08:00
epriestley
906ac21e54 Begin construction of bin/celerity map
Summary:
Ref T4222. Moves us toward a more modern Celerity CLI, and moves map discovery into the classtree. This is a little bit bulky (and means you can't ship completely standalone celerity maps) but has the advantage of being completely standard, and we could subclass maps into an auto-discovering map later if we have a need for it.

This doesn't affect the existing Celerity stuff. I'm going to build the new stuff in parallel, and then swap us over at the end.

Test Plan: Ran `bin/celerity map`, got reasonable-looking output.

Reviewers: btrahan, hach-que

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T4222

Differential Revision: https://secure.phabricator.com/D7864
2013-12-31 18:02:41 -08:00
epriestley
95a806ada3 Remove celerity.resource-path config
Summary: Ref T4222. This was used by Facebook while developing Releeph, but should no longer be necessary since Releeph is in the upstream. I can't get an answer out of Facebook about whether they still use it or not (see T4227), so nuke it. We're going to replace it with a more general mechanism (see T4222).

Test Plan: Regenerated celerity map. Browsed some pages, still got resources.

Reviewers: btrahan, hach-que

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T4222

Differential Revision: https://secure.phabricator.com/D7863
2013-12-31 18:02:35 -08:00
epriestley
db3228844a Note where to stop/start php-fpm in upgrade example script
Summary: This isn't as explicit as it could be.

Test Plan: Reading.

Reviewers: poop

Reviewed By: poop

CC: aran

Differential Revision: https://secure.phabricator.com/D7861
2013-12-30 16:47:05 -08:00
epriestley
26582eb82d Provide a more helpful message if a user connects via raw SSH
Summary: We currently print a fairly vague, technical message which is ambiguous about indicating success or an error if you aren't familiar with SSH.

Test Plan:
  $ ssh -T dweller@localhost
  phabricator-ssh-exec: Welcome to Phabricator.

  You are logged in as epriestley.

  You haven't specified a command to run. This means you're requesting an interactive shell, but Phabricator does not provide an interactive shell over SSH.

  Usually, you should run a command like `git clone` or `hg push` rather than connecting directly with SSH.

  Supported commands are: conduit, git-receive-pack, git-upload-pack, hg, svnserve.

Reviewers: btrahan, dctrwatson

Reviewed By: dctrwatson

CC: aran

Differential Revision: https://secure.phabricator.com/D7854
2013-12-30 13:21:56 -08:00
epriestley
ce78bf1de4 Make all bin/* scripts locate their workflows dynamically
Summary:
Ref T2015. Not directly related to Drydock, but I bumped into this. All these scripts currently enumerate their workflows explicitly.

Instead, use `PhutilSymbolLoader` to automatically discover workflows. This reduces code duplication and errors (see all the bad `extends` this diff fixes) and lets third parties add new workflows (not clearly valuable?).

Test Plan: Ran `bin/x help` for each modified script.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7840
2013-12-27 13:15:48 -08:00
epriestley
1650874004 Modernize Drydock CLI management of task execution
Summary:
Ref T2015. Currently, Drydock has a `wait-for-lease` workflow which is invoked in the background by the `lease` workflow.

The goal of this mechanism is to allow `bin/drydock lease` to print out logs as the lease is acquired. However, this predates the `runAllTasksInProcess` flags, and they provide a simpler and more robust way (potentially with `--trace` and `PhutilConsole`) to do synchronous execution and debug logging.

Simplify this whole mechanism: just run everything in-process in `bin/drydock lease`, and do logging via `--trace`. We could thread a `PhutilConsole` through things too, but this seems good enough for now.

Also various cleanup/etc.

Test Plan: Ran `bin/drydock lease`. Ran `bin/harbormaster build X --plan Y`, for `Y` being a Drydock-dependent build plan.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7835
2013-12-27 13:15:12 -08:00
epriestley
aad6b57c36 Add bin/harbormaster to make builds easier to debug
Summary:
Ref T1049. Adds `bin/harbormaster` and `bin/harbormaster build` for applying plans from the console. Since this gets `--trace`, it's much easier to debug what's going on.

This doesn't work properly with some of the Drydock steps yet, I need to look at those. I think `setRunAllTasksInProcess` probably obsoletes some of the mechanisms. It might also not work with "Wait for Builds" but I didn't check.

Test Plan: Used `bin/harbormaster` to run a bunch of builds. Ran builds from web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7825
2013-12-26 10:40:52 -08:00
epriestley
d667b12206 Provide a standalone query for resolution of commit author/committer into Phabricator users
Summary:
Ref T4195. To implement the "Author" and "Committer" rules, I need to resolve author/committer strings into Phabricator users.

The code to do this is currently buried in the daemons. Extract it into a standalone query.

I also added `bin/repository lookup-users <commit>` to test this query, both to improve confidence I'm getting this right and to provide a diagnostic command for users, since there's occasionally some confusion over how author/committer strings resolve into valid users.

Test Plan:
I tested this using `bin/repository lookup-users` and `reparse.php --message` on Git, Mercurial and SVN commits. Here's the `lookup-users` output:

  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIS3
  Examining commit rINIS3...
  Raw author string: epriestley
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rPOEMS165b6c54f487c8
  Examining commit rPOEMS165b6c54f487...
  Raw author string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIH6d24c1aee7741e
  Examining commit rINIH6d24c1aee774...
  Raw author string: epriestley <hg@yghe.net>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $

The `reparse.php` output was similar, and all VCSes resolved authors correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1731, T4195

Differential Revision: https://secure.phabricator.com/D7801
2013-12-19 11:05:17 -08:00
epriestley
74251b3636 Support bookmark hook operations in Mercurial
Summary: Ref T4195. Turns bookmark mutations in Mercurial into log objects.

Test Plan:
Pushed a pile of bookmarks and got logs:

{F89313}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7764
2013-12-17 08:34:30 -08:00
epriestley
7f45824984 Fix two issues with creating Conpherence threads via mail on some configurations
Summary:
Ref T4107. Two issues:

  - With strict MySQL settings, we try to insert `null` into the non-nullable `messageCount` field. Add an `initializeNew...` method.
  - If we don't create a new conpherence (for example, because the message body is empty), we fatal on `getPHID()` right now.

Also, make this stuff a little easier to test.

Test Plan: Used `mail_handler.php` to receive empty conpherence mail, and new-thread conpherence mail.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4107

Differential Revision: https://secure.phabricator.com/D7760
2013-12-12 10:59:28 -08:00
epriestley
d846f6508b Fix some repository URI handling issues in Git and Mercurial
Summary:
See <https://github.com/facebook/phabricator/issues/467>. @dctrwatson also ran into an issue where we were trying to `setPass()` a GitURI.

  - For Git and Mercurial, properly generate credential URIs where relevant.
  - Don't try to `setPass()` on Git-style URIs.

This isn't perfect but should clean things up a bit.

Test Plan: Added unit tests. Lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: dctrwatson, aran

Differential Revision: https://secure.phabricator.com/D7759
2013-12-12 09:45:27 -08:00
Chad Little
3ad4be4d93 Multicolor headers
Summary: This adds a handful of 'Main Header' colors to change the look of Phabricator very slightly. I know I would probably set my dev header to a different color.

Test Plan: Tested each css class and color, can add more in the future.

Reviewers: epriestley, btrahan

Reviewed By: btrahan

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7731
2013-12-06 12:08:11 -08:00
epriestley
a74bfe5167 Minor, sort out a change request which was lost in the melee. 2013-12-05 17:37:04 -08:00
epriestley
5ca84589bd Add an SSH access log
Summary: Ref T4107. Ref T4189. This implements an SSH access log, similar to the HTTP access log.

Test Plan:
  [Thu, 05 Dec 2013 13:45:41 -0800]	77841	orbital	::1	dweller	epriestley	epriestley	git-receive-pack	/diffusion/POEMS/	0	324765	402	232
  [Thu, 05 Dec 2013 13:45:48 -0800]	77860	orbital	::1	dweller	epriestley	epriestley	git-receive-pack	/diffusion/POEMS/	0	325634	402	232

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4107, T4189

Differential Revision: https://secure.phabricator.com/D7719
2013-12-05 17:00:48 -08:00
epriestley
e28b848ab2 Store pusher remote address and push protocol in PushLog
Summary: Ref T4195. Stores remote address and protocol in the logs, where possible.

Test Plan: Pushed some stuff, looked at the log, saw data.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7711
2013-12-05 11:59:22 -08:00
epriestley
d2e9aee16d Reject dangerous changes in Git repositories by default
Summary: Ref T4189. This adds a per-repository "dangerous changes" flag, which defaults to off. This flag must be enabled to do non-appending branch mutation (delete branches / rewrite history).

Test Plan:
With flag on and off, performed various safe and dangerous pushes.

  >>> orbital ~/repos/POEMS $ git push origin :blarp
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push deletes the branch 'blarp'.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad, richardvanvelzen

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7689
2013-12-03 10:28:39 -08:00
epriestley
f93c6985ad Support Mercurial pretxnchangegroup hooks
Summary: Ref T4189. Fixes T2066. Mercurial has a //lot// of hooks so I'm not 100% sure this is all we need to install (we may need separate hooks for tags/bookmarks) but it should cover most of what we're after at least.

Test Plan:
  - `bin/repository pull`'d a Mercurial repo and got a hook install.
  - Pushed to a Mercurial repository over SSH and HTTP, with good/bad hooks. Saw hooks fire.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2066, T4189

Differential Revision: https://secure.phabricator.com/D7685
2013-12-02 15:46:03 -08:00
epriestley
017d6ccd07 Support SVN pre-commit hoooks
Summary:
Ref T4189. This adds SVN support, which was a little more messy than I though. Principally, we can not use `PHABRICATOR_USER` for Subversion, because it strips away the entire environment for "security reasons".

Instead, use `--tunnel-user` plus `svnlook author` to figure out the author.

Also fix "ssh://" clone URIs, which needs to be "svn+ssh://".

Test Plan:
  - Made SVN commits through the hook.
  - Made Git commits, too, to make sure I didn't break anything.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7683
2013-12-02 15:45:55 -08:00
epriestley
618b5cbbc4 Install pre-commit hooks in Git repositories
Summary:
Ref T4189. T4189 describes most of the intent here:

  - When updating hosted repositories, sync a pre-commit hook into them instead of doing a `git fetch`.
  - The hook calls into Phabricator. The acting Phabricator user is sent via PHABRICATOR_USER in the environment. The active repository is sent via CLI.
  - The hook doesn't do anything useful yet; it just veifies basic parameters, does a little parsing, and exits 0 to allow the commit.

Test Plan:
  - Performed Git pushes and pulls over SSH and HTTP.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7682
2013-12-02 15:45:36 -08:00
epriestley
d59722321f Handle "-p port" flag in ssh-connect
Summary:
The documentation is explicit that Git does not pass this flag:

  The $GIT_SSH command will be given exactly two arguments: the username@host (or just host) from the URL and the shell command to execute on that remote system.

This isn't true; it does. Accommodate it.

I'll see if I can fix this in the upstream, too.

Test Plan: Ran various `ssh-connect` commands with -p, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7680
2013-12-02 11:25:46 -08:00
epriestley
e19ad8ab8a Warn users about using bin/accountadmin for first time setup
Summary: The "easy setup" is sort of an exaggeration since it basically just amounts to getting you logged in correctly, but it will probably be more true in the future.

Test Plan: Ran `bin/accountadmin` with zero and more than zero accounts.

Reviewers: asherkin, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7678
2013-12-02 11:25:42 -08:00
epriestley
1b026fa629 Unconditionally approve bin/accountadmin accounts
Summary: Some oldschool users create accounts via bin/accountadmin, which
messes up the first-time setup process. Auto-approve these accounts, as this
better aligns with user expectation.

Auditors: btrahan
2013-12-02 06:51:42 -08:00
Eric Stern
45c3a605ee Correctly handle case where no ssh keys are found
Summary: If no ssh keys are in phabricator, bin/ssh-auth errors with undefined `$lines`. This fixes that case and explicitly tells the user no rows were found.

Test Plan: Ran bin/ssh-auth before and after change with no ssh keys in the system. Error goes away after change.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: epriestley

CC: epriestley, aran, Korvin

Differential Revision: https://secure.phabricator.com/D7675
2013-11-30 18:56:23 -08:00
epriestley
58530669a2 Make ssh-connect more robust against known_host issues
I think the daemon is choking on this, prevent it from failing.

Auditors: btrahan
2013-11-22 16:24:24 -08:00
epriestley
6e41016077 Document and remove some scary warnings from repository hosting
Summary: Fixes T2230. This isn't a total walk in the park to configure, but should work for early adopters now.

Test Plan: Read documentation, browsed UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7634
2013-11-22 15:24:27 -08:00
epriestley
d09dd23bd7 Actually push to mirrors
Summary: Fixes T4038. Push repositories to mirrors.

Test Plan: Created a functional mirror of a local: https://github.com/epriestley/poems

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7633
2013-11-22 15:24:09 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
James Rhodes
7c3cb5948c Drydock blueprint for preallocated remote hosts
Summary:
This adds a Drydock blueprint for preallocated, remote hosts.  This will be used by the Harbormaster interface to allow users to specify remote hosts that builds can be run on.

This adds a `canAllocateResource` method to Drydock blueprints; it is used to detect whether a blueprint can allocate a resource for the given type and attributes.

Test Plan:
Ran:

```
bin/drydock lease --type host --attributes remote=true,preallocated=true,host=192.168.56.101,port=22,user=james,keyfile=,path=C:\\Build\\,platform=windows
```

and saw the "C:\Build\<id>" folder appear on the remote Windows machine.  Viewed the lease and resource in Drydock as well.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T4111

Differential Revision: https://secure.phabricator.com/D7593
2013-11-22 14:34:10 -08:00
Nikolajus Krauklis
67b8f9d35d Update manage_cache.php
tagline copied from mail

Reviewed by: epriestley

See: <https://github.com/facebook/phabricator/pull/446>
2013-11-21 12:36:00 -08:00
epriestley
ff8b48979e Simplify Repository remote and local command construction
Summary:
This cleans up some garbage:

  - We were specifying environmental variables with `X=y git ...`, but now have `setEnv()` on both `ExecFuture` and `PhutilExecPassthru`. Use `setEnv()`.
  - We were specifying the working directory with `(cd %s && git ...)`, but now have `setCWD()` on both `ExecFuture` and `PhutilExecPassthru`. Use `setCWD()`.
  - We were specifying the Git credentials with `ssh-agent -c (ssh-add ... && git ...)`. We can do this more cleanly with `GIT_SSH`. Use `GIT_SSH`.
  - Since we have to write a script for `GIT_SSH` anyway, use the same script for Subversion and Mercurial.

This fixes two specific issues:

  - Previously, we were not able to set `-o StrictHostKeyChecking=no` on Git commands, so the first time you cloned a git repo the daemons would generally prompt you to add `github.com` or whatever to `known_hosts`. Since this was non-interactive, things would mysteriously hang, in effect. With `GIT_SSH`, we can specify the flag, reducing the number of ways things can go wrong.
  - This adds `LANG=C`, which probably (?) forces the language to English for all commands. Apparently you need to install special language packs or something, so I don't know that this actually works, but at least two users with non-English languages have claimed it does (see <https://github.com/facebook/arcanist/pull/114> for a similar issue in Arcanist).

At some point in the future I might want to combine the Arcanist code for command execution with the Phabricator code for command execution (they share some stuff like LANG and HGPLAIN). However, credential management is kind of messy, so I'm adopting a "wait and see" approach for now. I expect to split this at least somewhat in the future, for Drydock/Automerge if nothing else.

Also I'm not sure if we use the passthru stuff at all anymore, I may just be able to delete that. I'll check in a future diff.

Test Plan: Browsed and pulled Git, Subversion and Mercurial repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7600
2013-11-20 10:41:35 -08:00
William R. Otte
74d81f9be0 Bugfix for D7596
Summary: Fixed a small bug that caused the catch-all commit to purge previously added symbols in that session.

Test Plan: Re-ran the script, observed corrected behavior.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4117

Differential Revision: https://secure.phabricator.com/D7597
2013-11-17 20:19:49 -08:00
William R. Otte
7d43e59110 Modified script to commit smaller batches of symbols to the database.
Summary:
Modified the import script so it will only try to load a configurable
number of symbols at a time to avoid exhausting memory for large project
imports.

I haven't written a line of PHP in more than a decade, so please forgive
any stylistic or technical errors.

Test Plan: Ran the script on symbol table generated from linux kernel.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4117

Differential Revision: https://secure.phabricator.com/D7596
2013-11-17 16:33:33 -08:00
epriestley
f5ca647d2c Add bin/repository edit for CLI repository editing
Summary:
Ref T4039. This is mostly to deal with that, to prevent the security issues associated with mutable local paths. The next diff will lock them in the web UI.

I also added a confirmation prompt to `bin/repository delete`, which was a little scary without one.

See one comment inline about the `--as` flag. I don't love this, but when I started adding all the stuff we'd need to let this transaction show up as "Administrator" it quickly got pretty big.

Test Plan: Ran `bin/repository edit ...`, saw an edit with a transaction show up on the web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4039

Differential Revision: https://secure.phabricator.com/D7579
2013-11-13 11:26:05 -08:00
epriestley
c0e1a63a63 Implement an approval queue
Summary:
  - Add an option for the queue.
  - By default, enable it.
  - Dump new users into the queue.
  - Send admins an email to approve them.

Test Plan:
  - Registered new accounts with queue on and off.
  - As an admin, approved accounts and disabled the queue from email.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7576
2013-11-13 11:24:56 -08:00
epriestley
7f11e8d740 Improve handling of email verification and "activated" accounts
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:

  - Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
    - Migrate all the existing users.
    - When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
    - Just make the checks look at the `isEmailVerified` field.
  - Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
  - Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
    - When the queue is enabled, registering users are created with `isApproved = false`.
    - Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
    - They go to the web UI and approve the user.
    - Manually-created accounts are auto-approved.
    - The email will have instructions for disabling the queue.

I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.

Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.

Test Plan:
  - Ran migration, verified `isEmailVerified` populated correctly.
  - Created a new user, checked DB for verified (not verified).
  - Verified, checked DB (now verified).
  - Used Conduit, People, Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Differential Revision: https://secure.phabricator.com/D7572
2013-11-12 14:37:04 -08:00
epriestley
85f505465e Support serving SVN repositories over SSH
Summary:
Ref T2230. The SVN protocol has a sensible protocol format with a good spec here:

http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_ra_svn/protocol

Particularly, compare this statement to the clown show that is the Mercurial wire protocol:

> It is possible to parse an item without knowing its type in advance.

WHAT A REASONABLE STATEMENT TO BE ABLE TO MAKE ABOUT A WIRE PROTOCOL

Although it makes substantially more sense than Mercurial, it's much heavier-weight than the Git or Mercurial protocols, since it isn't distributed.

It's also not possible to figure out if a request is a write request (or even which repository it is against) without proxying some of the protocol frames. Finally, several protocol commands embed repository URLs, and we need to reach into the protocol and translate them.

Test Plan: Ran various SVN commands over SSH (`svn log`, `svn up`, `svn commit`, etc).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7556
2013-11-11 12:19:06 -08:00
epriestley
8840f60218 Enable Mercurial reads and writes over SSH
Summary:
Ref T2230. This is substantially more complicated than Git, but mostly because Mercurial's protocol is a like 50 ad-hoc extensions cobbled together. Because we must decode protocol frames in order to determine if a request is read or write, 90% of this is implementing a stream parser for the protocol.

Mercurial's own parser is simpler, but relies on blocking reads. Since we don't even have methods for blocking reads right now and keeping the whole thing non-blocking is conceptually better, I made the parser nonblocking. It ends up being a lot of stuff. I made an effort to cover it reasonably well with unit tests, and to make sure we fail closed (i.e., reject requests) if there are any parts of the protocol I got wrong.

A lot of the complexity is sharable with the HTTP stuff, so it ends up being not-so-bad, just very hard to verify by inspection as clearly correct.

Test Plan:
  - Ran `hg clone` over SSH.
  - Ran `hg fetch` over SSH.
  - Ran `hg push` over SSH, to a read-only repo (error) and a read-write repo (success).

Reviewers: btrahan, asherkin

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7553
2013-11-11 12:18:27 -08:00
James Rhodes
448d8684e8 Quick hack to make symbol lookup work for C#.
Summary: This is kind of a quick hack to make symbol lookup work for C#.  ctags calls C# 'csharp', while pygments recognises it as 'cs' (or at least, I have to put 'cs' in the Arcanist indexed languages for the clickables to appear, while it's 'csharp' in the symbol database).

Test Plan: Tested this in my live install and it makes symbol lookup work.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7497
2013-11-09 15:08:50 -08:00
epriestley
bd29784a32 Add an administrative bin/repository importing command to list importing commits
Summary: Ref T4068. Adds a command to list all commits in an "importing" status. This will allow users to use `reparse.php` to diagnose and repair issues.

Test Plan:
  - Ran `bin/repository importing P`, etc.
  - Used `reparse.php` to reparse some commit stages and saw status update correctly.
  - Ran on a repo with no importing commits.
  - Ran with `... --simple | xargs`, which saves us having to put an `awk` or something in there for users.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4068

Differential Revision: https://secure.phabricator.com/D7515
2013-11-06 11:26:41 -08:00
epriestley
e3a5ab1f8c Add an administrative bin/repository mark-imported command
Summary:
Ref T4068. In some cases like that one, I anticipate a repository not fully importing when a handful of random commits are broken. In the long run we should just deal with that properly, but in the meantime provide an administrative escape hatch so you can mark the repository as imported and get it running normally.

The major reason to do this is that Herald, Feed, Harbormaster, etc., won't activate until a repository is "imported".

Test Plan:
  - Tried to mark an imported repository as imported, got an "already imported" message.
  - Same for not-imported.
  - Marked a repository not-imported.
  - Marked a repository imported.
  - Marked a repository not-imported, then waited for the daemons to mark it imported again automatically.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, kbrownlees

Maniphest Tasks: T4068

Differential Revision: https://secure.phabricator.com/D7514
2013-11-06 11:26:24 -08:00
epriestley
c6665b1907 Serve git writes over SSH
Summary: Looks like this is pretty straightforward; same as the reads except mark it as needing PUSH.

Test Plan: Ran `git push`, pushed over SSH to a hosted repo.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7425
2013-10-29 15:32:41 -07:00
epriestley
bb4904553f Route some VCS connections over SSH
Summary:
  - Add web UI for configuring SSH hosting.
  - Route git reads (`git-upload-pack` over SSH).

Test Plan:
  >>> orbital ~ $ git clone ssh://127.0.0.1/
  Cloning into '127.0.0.1'...
  Exception: Unrecognized repository path "/". Expected a path like "/diffusion/X/".
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  >>> orbital ~ $ git clone ssh://127.0.0.1/diffusion/X/
  Cloning into 'X'...
  Exception: No repository "X" exists!
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  >>> orbital ~ $ git clone ssh://127.0.0.1/diffusion/MT/
  Cloning into 'MT'...
  Exception: This repository is not available over SSH.
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  >>> orbital ~ $ git clone ssh://127.0.0.1/diffusion/P/
  Cloning into 'P'...
  Exception: TODO: Implement serve over SSH.
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7421
2013-10-29 15:32:40 -07:00
epriestley
888b3839e7 Prepare to route VCS connections through SSH
Summary:
Fixes T2229. This sets the stage for a patch similar to D7417, but for SSH. In particular, SSH 6.2 introduced an `AuthorizedKeysCommand` directive, which lets us do this in a mostly-reasonable way without needing users to patch sshd (if they have a recent enough version, at least).

The way the `AuthorizedKeysCommand` works is that it gets run and produces an `authorized_keys`-style file fragment. This isn't ideal, because we have to dump every key into the result, but should be fine for most installs. The earlier patch against `sshd` passes the public key itself, which allows the script to just look up the key. We might use this eventually, since it can scale much better, so I haven't removed it.

Generally, auth is split into two scripts now which mostly do the same thing:

  - `ssh-auth` is the AuthorizedKeysCommand auth, which takes nothing and dumps the whole keyfile.
  - `ssh-auth-key` is the slightly cleaner and more scalable (but patch-dependent) version, which takes the public key and dumps only matching options.

I also reworked the argument parsing to be a bit more sane.

Test Plan:
This is somewhat-intentionally a bit obtuse since I don't really want anyone using it yet, but basically:

  - Copy `phabricator-ssh-hook.sh` to somewhere like `/usr/libexec/openssh/`, chown it `root` and chmod it `500`.
    - This script should probably also do a username check in the future.
  - Create a copy of `sshd_config` and fix the paths/etc. Point the KeyScript at your copy of the hook.
  - Start a copy of sshd (6.2 or newer) with `-f <your config file>` and maybe `-d -d -d` to foreground and debug.
  - Run `ssh -p 2222 localhost` or similar.

Specifically, I did this setup and then ran a bunch of commands like:

  - `ssh host` (denied, no command)
  - `ssh host ls` (denied, not supported)
  - `echo '{}' | ssh host conduit conduit.ping` (works)

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Maniphest Tasks: T2229, T2230

Differential Revision: https://secure.phabricator.com/D7419
2013-10-29 15:32:40 -07:00
Chad Little
1f0b40b61d Projects Icons Series 1
Summary: A set of random icons for use as project identifiers. 42, white.

Test Plan: photoshop, epriestley

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7290
2013-10-12 19:15:38 -07:00
Chad Little
97c690fc0f PHUIPropertyListView
Summary: This builds out and implements PHUIPropertyListView (container) and PHUIPropertyListItemView (section) as well as adding tabs.

Test Plan: Tested each page I edited with the exception of Releeph and Phortune, though those changes look ok to me diff wise. Updated examples page with tabs.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7283
2013-10-11 07:53:56 -07:00
epriestley
1d1ecb5629 Add bin/policy unlock
Summary: Ref T603. We might need a fine-grained CLI tool later on, but here's a bat we can bludgeon things with.

Test Plan:
  - Ran `bin/policy unlock D12` (adjusted policies).
  - Ran `bin/policy unlock rPca85c457ebcb` (got "not mutable" stuff).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7189
2013-10-01 16:01:15 -07:00
epriestley
e2ed527353 Add a very simple bin/policy script for CLI policy administration
Summary:
Ref T603. I want to provide at least a basic CLI tool for fixing policy problems, since there are various ways users can lock themselves out of objects right now. Although I imagine we'll solve most of them in the application eventually, having a workaround in the meantime will probably make support a lot easier.

This implements `bin/policy show <object>`, which shows an object's policy settings. In a future diff, I'll implement something like `bin/policy set --capability view --policy users <object>`, although maybe just `bin/policy unlock <object>` (which sets view and edit to "all users") would be better for now. Whichever way we go, it will be some blanket answer to people showing up in IRC having locked themselves out of objects which unblocks them while we work on preventing the issue in the first place.

Test Plan: See screenshot.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7171
2013-09-29 09:06:41 -07:00
epriestley
ec02ac1806 Tweak static resource package definitions
Summary: Add a couple more resources that we need on most pages.

Test Plan: Regenerated resources, viewed homepage.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7155
2013-09-27 10:50:40 -07:00
epriestley
02ed9f1368 Remove ManiphestTransactionDetailView
Summary: Ref T2217. No remaining callsites. Also nukes associated CSS.

Test Plan: `grep`, looked at some tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T2217

Differential Revision: https://secure.phabricator.com/D7076
2013-09-23 14:30:38 -07:00
Chad Little
e8bb24fd60 Policy, Status in PHUIHeaderView
Summary: The adds the ability to set 'properties' such as state, privacy, due date to the header of objects.

Test Plan: Implemented in Paste, Pholio. Tested various states.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7016
2013-09-17 09:12:37 -07:00
Chad Little
5ba20b8924 Move PhabricatorObjectItem to PHUIObjectItem, add 'plain' setting for lists.
Summary: Adds plain support for object lists that just look like lists

Test Plan: review UIexamples and a number of other applications

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6922
2013-09-09 14:14:34 -07:00
Chad Little
fe2a96e37f Update Form Layouts
Summary:
This attempts some consistency in form layouts. Notably, they all now contain headers and are 16px off the sides and tops of pages. Also updated dialogs to the same look and feel. I think I got 98% of forms with this pass, but it's likely I missed some buried somewhere.

TODO: will take another pass as consolidating these colors and new gradients in another diff.

Test Plan: Played in my sandbox all week. Please play with it too and let me know how they feel.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6806
2013-08-26 11:53:11 -07:00
epriestley
86989c9f98 Provide a more flexible script for administrative management of audits
Summary: Fixes T3679. This comes up every so often and the old script is extremely broad (nuke everything in a repository). Provide a more surgical tool.

Test Plan: Ran a bunch of variations of the script and they all seemed to work OK.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, staticshock

Maniphest Tasks: T3679

Differential Revision: https://secure.phabricator.com/D6678
2013-08-05 10:35:01 -07:00
epriestley
88530ba053 Add bin/phd log <id> to dump all logs to the CLI
Summary: Ref T3557. Make it easier to access full daemon logs from the CLI.

Test Plan: {F51265}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3557

Differential Revision: https://secure.phabricator.com/D6547
2013-07-23 16:58:19 -07:00
epriestley
9e0a299b06 Launch daemons with a full Phabricator environment in the overseers
Summary:
Ref T1670. Prepare for the overseers to talk directly to the database instead of using Conduit. See T1670 for discussion.

This shouldn't impact anything, except it has a very small chance of destabilizing the overseers.

Test Plan:
Ran `phd launch`, `phd debug`, `phd start`.

Ran with `--trace-memory` and verified elevated but mostly steady memory usage (8MB / overseer). This climbed by 0.05KB / sec (4MB / day) but the source of the leaks seems to be the cURL calls we're making over Conduit so this will actually fix that. Disabling `--conduit-uri` reported steady memory usage. I wasn't able to identify anything leaking within code we control. This may be something like a dynamic but capped buffer in cURL, since we haven't seen any issues in the wild.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1670

Differential Revision: https://secure.phabricator.com/D6534
2013-07-23 12:09:45 -07:00
epriestley
d89b8ce2b2 Modernize architecture of phd
Summary:
Ref T1670. Mostly, use PhutilArgumentParser. This breaks up the mismash of functional stuff and PhabriatorDaemonControl into proper argumentparser Workflows.

There are no functional changes, except that I removed the "pingConduit()" call prior to starting daemons, because I intend to remove all Conduit integration.

Test Plan:
- Ran `phd list`.
- Ran `phd status` (running daemons).
- Ran `phd status` (no running daemons).
- Ran `phd stop <pid>` (dead task).
- Ran `phd stop <pid>` (live task).
- Ran `phd stop zebra` (invalid PID).
- Ran `phd stop 1` (bad PID).
- Ran `phd stop`.
- Ran `phd debug zebra` (no match).
- Ran `phd debug e` (ambiguous).
- Ran `phd debug task`.
- Ran `phd launch task`.
- Ran `phd launch 0 task` (invalid arg).
- Ran `phd launch 2 task`.
- Ran `phd help`.
- Ran `phd help list`.
- Ran `phd start`.
- Ran `phd restart`.
- Looked at Repositories (daemon running).
- Looked at Repositories (daemon not running).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1670

Differential Revision: https://secure.phabricator.com/D6490
2013-07-18 15:28:56 -07:00
epriestley
dd76143399 Pholio - make the form maintain client side edits between file uploads
Summary: Fixes T3553. Did it by adding some code that refreshes the File object on keyup events within a given file entry. also fixes an html derp I found trying to fix this.

Test Plan: added cool things like 'bbb' to every field and noted they were maintained when I added more files

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, Korvin, chad

Maniphest Tasks: T3553

Differential Revision: https://secure.phabricator.com/D6488
2013-07-18 15:04:08 -07:00
Chad Little
c9610721b1 Status Icons
Summary: Status icons for next to people names

Test Plan: photoshop

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2064

Differential Revision: https://secure.phabricator.com/D6479
2013-07-17 08:44:11 -07:00
Levi Jackson
d27e7c52b2 Add explicit mysql.port configuration
See: https://github.com/facebook/phabricator/pull/356

Reviewed by: epriestley
2013-07-14 16:06:23 -07:00
epriestley
544a84ebb9 Move outbound mail lists to CLI and enhance details
Summary: Finish off moving all this stuff to the CLI. Ref T3306.

Test Plan:
  PROPERTIES
  ID: 6483
  Status: void
  Retry Count: 0
  Next Retry: 1373494457
  Related PHID: PHID-DREV-5bnb33yeuhuaulyc3exg
  Message: Message has no valid recipients: all To/Cc are disabled, invalid, or configured not to receive this mail.

  PARAMETERS
  from: PHID-USER-lqiz3yd7wmk64ejugvov
  is-html:
  parent-message-id: null
  thread-id: differential-rev-PHID-DREV-5bnb33yeuhuaulyc3exg-req
  is-first-message: null
  is-bulk: 1
  mailtags: ["differential-comment"]
  cc: ["PHID-USER-cluwcdowc35gmperlkbi"]
  subject: D22: quack quack
  subject-prefix: [Differential]
  vary-subject-prefix: [Commented On]
  worker-task: 936546

  HEADERS
  Thread-Topic: D22: quack quack
  X-Herald-Rules: none
  X-Differential-Author: <PHID-USER-lqiz3yd7wmk64ejugvov>
  X-Differential-CC: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Differential-CC: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Differential-CC: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Differential-CC: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Differential-CC: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Differential-CC: <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-CCs: <PHID-USER-ly3pvrtdkw7lbgs72jvr>, <PHID-USER-cluwcdowc35gmperlkbi>, <PHID-MLST-wkxaantg3q6pgdkty5pt>, <PHID-USER-aeabc4ipqbifny3rw4ok>, <PHID-USER-zqxtb3oi4pouwxnxlv3f>, <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-Explicit-CC: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Differential-Explicit-CC: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Differential-Explicit-CC: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Differential-Explicit-CC: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Differential-Explicit-CC: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Differential-Explicit-CC: <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-Explicit-CCs: <PHID-USER-ly3pvrtdkw7lbgs72jvr>, <PHID-USER-cluwcdowc35gmperlkbi>, <PHID-MLST-wkxaantg3q6pgdkty5pt>, <PHID-USER-aeabc4ipqbifny3rw4ok>, <PHID-USER-zqxtb3oi4pouwxnxlv3f>, <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Phabricator-To: <PHID-USER-lqiz3yd7wmk64ejugvov>
  X-Phabricator-Cc: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Phabricator-Cc: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Phabricator-Cc: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Phabricator-Cc: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Phabricator-Cc: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Phabricator-Cc: <PHID-USER-cknqtm2dzw7twnwyiaye>

  RECIPIENTS
  ! dog (dog)
      - This user is disabled; disabled users do not receive mail.

  BODY
  epriestley has commented on the revision "quack quack".

    zxcbzxcb

  REVISION DETAIL
    http://local.aphront.com:8080/D22

  To: epriestley
  Cc: Unknown User, dog, list, duck, epriestley992, asana

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6423
2013-07-10 18:52:22 -07:00
epriestley
e8f679fa88 Move the inbound mail list to the CLI
Summary: Same as D6419. See D6413.

Test Plan:
      13  epriestley        D6                    Re: [local/Differential] [Request, 83 lines] D6: aabd
      14  epriestley        D6                    Re: [local/Differential] [Request, 83 lines] D6: aabd
      15  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      16  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      17  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      18  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      19  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      20  -                 -                     (No subject.)
      21  epriestley        D22                   (No subject.)
      22  epriestley        D12                   (No subject.)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6420
2013-07-10 15:18:37 -07:00
epriestley
6bac7e390e Move "send test" to bin/mail
Summary: Like D6417. See D6413.

Test Plan:
  >>> orbital ~/devtools/phabricator $ echo "hello" | ./bin/mail send-test --to dog --cc duck --from epriestley --subject 'hey there' --attach README --tag blip --tag bloop
  Reading message body from stdin...
  Mail sent! You can view details by running this command:

      phabricator/ $ ./bin/mail show-outbound --id 6480

  >>> orbital ~/devtools/phabricator $ ./bin/mail show-outbound --id 6480
  PROPERTIES
  ID: 6480
  Status: sent
  Retry Count: 0
  Next Retry: 1373493443
  Related PHID:
  Message:

  PARAMETERS
  to: ["PHID-USER-cluwcdowc35gmperlkbi"]
  cc: ["PHID-USER-aeabc4ipqbifny3rw4ok"]
  subject: hey there
  is-html:
  is-bulk:
  mailtags: ["blip","bloop"]
  from: PHID-USER-lqiz3yd7wmk64ejugvov
  worker-task: 936543

  BODY
  hello

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6419
2013-07-10 15:18:24 -07:00
epriestley
7fa2343822 Move mail "Receive Test" from web UI to CLI
Summary:
Ref T3306. Moves this from the web to the CLI, which is a tiny bit clunkier but way better as far as policies go and more repeatable for development.

See discussion in D6413.

Test Plan: Ran `bin/mail receive-test`, verified mail was received. Used and abused various options.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6417
2013-07-10 15:13:24 -07:00
epriestley
fcb56c6371 Move "scripts/sql/probe.php" to "bin/storage probe"
Summary: This makes it namespace/database/connection aware and a little easier to find. Also use pht() / PhutilConsole.

Test Plan: Ran `bin/storage probe`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6341
2013-07-02 16:34:17 -07:00
epriestley
90123dd739 Add DifferentialDiffQuery and change most callsites
Summary:
Ref T603. This introduces a policy-aware DifferentialDiffQuery and converts most callsites.

I've left unusual callsites (mostly: hard to get the viewer, unusual query, queries related to active diffs) alone for now, so this isn't exhaustive but hits 60-80% of sites.

Test Plan: Created diff; created revision; viewed diffs and revisions; made additional conduit calls.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D6338
2013-07-01 12:38:42 -07:00
epriestley
42c0f060d5 Push feed publishing deeper into the task queue
Summary:
Ref T2852. I want to model Asana integration as a response to feed events. Currently, we queue one feed event for each HTTP hook.

Instead, always queue one feed event and then have it queue any necessary followup events (now, http hooks; soon, asana).

Add a script to make it easy to reproducibly fire feed event publishing.

Test Plan:
Republished a feed event and verified it hit configured HTTP hooks correctly.

  $ ./bin/feed republish 5765774156541908292 --trace
  >>> [2] <connect> phabricator2_feed
  <<< [2] <connect> 1,660 us
  >>> [3] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [3] <query> 595 us
  >>> [4] <connect> phabricator2_differential
  <<< [4] <connect> 760 us
  >>> [5] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [5] <query> 478 us
  >>> [6] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [6] <query> 449 us
  >>> [7] <connect> phabricator2_user
  <<< [7] <connect> 1,062 us
  >>> [8] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [8] <query> 540 us
  >>> [9] <connect> phabricator2_file
  <<< [9] <connect> 951 us
  >>> [10] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [10] <query> 498 us
  >>> [11] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [11] <query> 507 us
  Republishing story...
  >>> [12] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [12] <query> 685 us
  >>> [13] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [13] <query> 489 us
  >>> [14] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [14] <query> 512 us
  >>> [15] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [15] <query> 601 us
  >>> [16] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [16] <query> 405 us
  >>> [17] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [17] <query> 551 us
  >>> [18] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [18] <query> 507 us
  >>> [19] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [19] <query> 428 us
  >>> [20] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [20] <query> 419 us
  >>> [21] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [21] <query> 591 us
  >>> [22] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [22] <query> 406 us
  >>> [23] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [23] <query> 593 us
  >>> [24] <http> http://127.0.0.1/derp/
  <<< [24] <http> 746,157 us
  [2013-06-24 20:23:26] EXCEPTION: (HTTPFutureResponseStatusHTTP) [HTTP/500] Internal Server Error

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6291
2013-06-25 16:29:47 -07:00
epriestley
c94ef134e4 Add bin/auth refresh for debugging OAuth token refresh issues
Summary: Ref T2852. Provide a script for inspecting/debugging OAuth token refresh.

Test Plan: Ran `bin/auth refresh` with various arguments, saw token refreshes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6276
2013-06-24 15:55:41 -07:00
epriestley
6a2e27ba8d Put all DifferentialComment loading behind DifferentialCommentQuery
Summary:
Ref T2222.

I'm thinking about how I want to approach the Asana sync, and I want to try to do T2222 first so that we can build it cleanly on top of ApplicationTransactions. I think we can at least walk down this road a little bit and if it turns out to be scary we can take another approach.

I was generally very happy with how the auth migration turned out (seemingly, it was almost completely clean), and want to pursue a similar strategy here. Basically:

  - Wrap the new objects in the old objects for reads/writes.
  - Migrate all the existing data to the new table.
  - Everything hard is done; move things over a piece at a time at a leisurely pace in lots of smallish, relatively-easy-to-understand changes.

This deletes or abstracts all reads of the DifferentialComment table. In particular, these things are **deleted**:

  - The script `undo_commits.php`, which I haven't pointed anyone at in a very long time.
  - The `differential.getrevisionfeedback` Conduit method, which has been marked deprecated for a year or more.
  - The `/stats/` interface in Differential, which should be rebuilt on Fact and has never been exposed in the UI. It does a ton of joins and such which are prohibitively difficult to migrate.

This leaves a small number of reading interfaces, which I replaced with a new `DifferentialCommentQuery`. Some future change will make this actually load transactions and wrap them with DifferentialComment interfaces.

Test Plan: Viewed a revision; made revision comments

Reviewers: btrahan

Reviewed By: btrahan

CC: edward, chad, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6260
2013-06-21 12:51:18 -07:00
epriestley
c0cc7bbfdf Provide bin/auth ldap for LDAP diagnostics
Summary: Ref T1536.

Test Plan: Ran `bin/auth ldap`.

Reviewers: mbishopim3, chad

Reviewed By: mbishopim3

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6218
2013-06-17 13:26:25 -07:00
epriestley
278905543e Add very basic bin/auth tool
Summary: Ref T1536. This script basically exists to restore access if/when users shoot themselves in the foot by disabling all auth providers and can no longer log in.

Test Plan: {F46411}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6205
2013-06-17 10:55:05 -07:00
Chad Little
02b59e685f PHUIButtonView class
Summary: Rough pass at a PHUIButtonView Class. Keeps phutil_tag intact and adds some image features if you use the class.

Test Plan: UIExamples

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6192
2013-06-12 18:23:35 -07:00
Chad Little
8e6bda51f2 Login icons
Summary: Took a stab at some login icons for buttons.

Test Plan: photoshop

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6174
2013-06-11 10:22:09 -07:00
epriestley
5d1f94ac8a Fix some Phabricator lint warnings
Summary: Lint.

Test Plan: Lint.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6127
2013-06-04 15:28:24 -07:00
epriestley
87bc6eb28c Use ObjectItemListView in Diviner
Summary: Ref T988. Nuke the old directory CSS.

Test Plan: {F44796}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6093
2013-05-31 10:51:53 -07:00
epriestley
0f070236bd Add a bin/files purge workflow
Summary:
We can lose file data through various means; one reasonable way is if files get deleted from disk with 'local-disk' storage. If data goes missing,

Ref T3265. Also, reduce some code duplication.

Test Plan:
Ran `bin/files purge`, `bin/files migrate`, `bin/files rebuild` with various args.

Deleted a file with "local-disk" storage, ran `bin/files purge`, made sure it got picked up.

Reviewers: dctrwatson, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3265

Differential Revision: https://secure.phabricator.com/D6068
2013-05-29 06:28:57 -07:00
epriestley
e01ceaa07f Provide 'bin/cache', for managing caches
Summary:
See <https://github.com/facebook/phabricator/issues/323>. We have a very old cache management script which doesn't purge all the modern caches (and does purge some caches which are no longer in use). Update it so it purges all the modern caches (remarkup, general, changeset), no longer purges outdated caches, and is easier to use.

Also delete a lot of "this script has moved" scripts from the last few rounds of similar cleanup, I believe all of these have been in master for at least several months, which should be enough time for users to get used to the new stuff.

Test Plan: Ran `bin/cache` with various arguments. Verified caches were purged.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5978
2013-05-20 10:16:35 -07:00
epriestley
e4525f9de1 Provide some diagnostic tools for examining inbound and outbound mail
Summary: We can't show this stuff on the web UI because it has password reset links and private reply-to addresses, but we can provide easier CLI tools than "root around in the database". Land a rough version of `bin/mail show-inbound` and `bin/mail show-outbound`.

Test Plan: Used both commands to examine mail from the CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: tido, euresti, aran

Differential Revision: https://secure.phabricator.com/D5963
2013-05-20 10:13:42 -07:00
epriestley
a1a46656cb Minor, package more resources and rebuild Celerity map. 2013-05-18 17:16:24 -07:00
epriestley
4fc9b8e785 Apply sprite sheet changes to Phabricator
Summary: Basically a one line change, plus a regen. This should be the last full-rewrite these things get since they'll stop fully rehashing all the time now.

Test Plan: See D5964.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D5965
2013-05-18 10:34:10 -07:00
epriestley
9e5410e6a2 Rename bin/files metadata to bin/files rebuild and let it rebuild MIME information
Summary:
See <https://github.com/facebook/phabricator/issues/320>. Files can end up with a bad MIME type, and we don't update it when uploading another copy of the file since obviously the new copy has the same data and thus the same MIME type.

  - Rename `bin/files metadata` to `bin/files rebuild` to make it a more consistent verb.
  - Let it rebuild MIME types so users who hit issues like this can run `bin/files rebuild --all --rebuild-mime` to straighten things out.

Test Plan: Ran `bin/files` in various modes, examined output.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5951
2013-05-17 10:00:31 -07:00
epriestley
eabe3a4d33 Begin improving the soundness of received mail
Summary:
We/I broke a couple of things here recently (see D5911) and are doing some work here in general (see D5912, etc.).

Generally, this code is pretty oldschool and not especially well architected for modern application-oriented Phabricator. It hardcodes a lot of stuff which should be applications' responsibilites.

Take the first steps toward making it more solid to reduce the risk here. In particular:

  - Factor out the "self mail" and "duplicate mail" checks and add unit tests.
  - Make Message-ID hash handling automatic.

Test Plan: Ran unit tests.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5915
2013-05-13 16:32:19 -07:00
epriestley
cbd3c0b7ac Add action icons to object list views
Summary:
We have a few interfaces where add "Edit", "Delete" or some other action to a list. Currently, this happens via icons, but these are cumbersome and weird, are inconsistent, can't be workflow'd, are hard to hit on desktops and virtually impossible to hit on mobile, and generally just feel iffy to me. Prominent examples are Projects and Flags. I'd like to try adding an "edit" action to Maniphest (to provide quick edit from list views, basically). It looks like some of Releeph would benefit here, as well.

Instead, provide first-class actions:

{F42978}

They produce targets which my meaty ham-fists can plausibly hit on mobile, too:

{F42979}

(We could do some kind of swipe-to-expose thing eventually, but I think putting them by default is OK?)

Test Plan: Added UIExamples. Checked desktop/mobile.

Reviewers: chad, btrahan, edward

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5890
2013-05-10 12:57:01 -07:00
Jakub Vrana
3bfad75764 Put JX.phtize in package 2013-04-23 17:58:56 -07:00