Summary:
Ref T7789. This implements:
- A new table to store the `<objectHash, filePHID>` relationship between Git LFS files and Phabricator file objects.
- A basic response to `batch` commands, which return actions for a list of files.
Test Plan:
Ran `git lfs push origin master`, got a little further than previously:
```
epriestley@orbital ~/dev/scratch/poemslocal $ git lfs push origin master
Git LFS: (2 of 1 files) 174.24 KB / 87.12 KB
Git LFS operation "upload/b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69" is not supported by this server.
Git LFS operation "upload/b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69" is not supported by this server.
```
With `GIT_TRACE=1`, this shows the batch part of the API going through.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7789
Differential Revision: https://secure.phabricator.com/D15489
Summary:
adds the `{{PHID....}}` rule. Should mostly be useful in UI code that refers to Objects.
It doesn't add any mention links/transactions.
Test Plan: Comment with this, see email (plain + html) and comment box.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15488
Summary: This updates (all?) of Diffusion/Audit to new UI, included edit and other extra form pages. It's fairly complete but I don't know all the nooks and crannies so to speak to fully verify I didn't mess anything up.
Test Plan: Tested creating new repositories, browsing, searching, auditing. Need more eyes.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15487
Summary:
Ref T7789. This builds on top of `git-lfs-authenticate` to detect LFS requests, read LFS tokens, and route them to a handler which can do useful things.
This handler promptly drops them on the floor with an error message.
Test Plan:
Here's a transcript showing the parts working together so far:
- `git-lfs` connects to the server with SSH, and gets told how to connect with HTTP to do uploads.
- `git-lfs` uses HTTP, and authenticates with the tokens properly.
- But the server tells it to go away, and that it doesn't support anything, so the operation ultimately fails.
```
$ GIT_TRACE=1 git lfs push origin master
12:45:56.153913 git.c:558 trace: exec: 'git-lfs' 'push' 'origin' 'master'
12:45:56.154376 run-command.c:335 trace: run_command: 'git-lfs' 'push' 'origin' 'master'
trace git-lfs: Upload refs origin to remote [master]
trace git-lfs: run_command: git rev-list --objects master --not --remotes=origin
trace git-lfs: run_command: git cat-file --batch-check
trace git-lfs: run_command: git cat-file --batch
trace git-lfs: run_command: 'git' config -l
trace git-lfs: tq: starting 3 transfer workers
trace git-lfs: tq: running as batched queue, batch size of 100
trace git-lfs: prepare upload: b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69 lfs/dog1.jpg 1/1
trace git-lfs: tq: sending batch of size 1
trace git-lfs: ssh: local@localvault.phacility.com git-lfs-authenticate diffusion/18/poems.git upload
trace git-lfs: api: batch 1 files
trace git-lfs: HTTP: POST http://local.phacility.com/diffusion/POEMS/poems.git/info/lfs/objects/batch
trace git-lfs: HTTP: 404
trace git-lfs: HTTP: {"message":"Git LFS operation \"objects\/batch\" is not supported by this server."}
trace git-lfs: HTTP:
trace git-lfs: api: batch not implemented: 404
trace git-lfs: run_command: 'git' config lfs.batch false
trace git-lfs: tq: batch api not implemented, falling back to individual
trace git-lfs: ssh: local@localvault.phacility.com git-lfs-authenticate diffusion/18/poems.git upload b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69
trace git-lfs: api: uploading (b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69)
trace git-lfs: HTTP: POST http://local.phacility.com/diffusion/POEMS/poems.git/info/lfs/objects
trace git-lfs: HTTP: 404
trace git-lfs: HTTP: {"message":"Git LFS operation \"objects\" is not supported by this server."}
trace git-lfs: HTTP:
trace git-lfs: tq: retrying 1 failed transfers
trace git-lfs: ssh: local@localvault.phacility.com git-lfs-authenticate diffusion/18/poems.git upload b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69
trace git-lfs: api: uploading (b7e0aeb82a03d627c6aa5fc1bbfd454b6789d9d9affc8607d40168fa18cf6c69)
trace git-lfs: HTTP: POST http://local.phacility.com/diffusion/POEMS/poems.git/info/lfs/objects
trace git-lfs: HTTP: 404
trace git-lfs: HTTP: {"message":"Git LFS operation \"objects\" is not supported by this server."}
trace git-lfs: HTTP:
Git LFS: (0 of 1 files) 0 B / 87.12 KB
Git LFS operation "objects" is not supported by this server.
Git LFS operation "objects" is not supported by this server.
```
Reviewers: chad
Reviewed By: chad
Subscribers: eadler
Maniphest Tasks: T7789
Differential Revision: https://secure.phabricator.com/D15485
Summary:
Ref T10604. This uses the new standalone stream reader introduced in D15483 to read request data, instead of putting the logic in PhabricatorStartup.
It also doesn't read request data until it specifically needs to. This supports, e.g., streaming Git LFS PUT requests, and streaming more types of requests in the future.
Test Plan: See D15483. Made various different types of requests and wasn't immediately able to break anything.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10604
Differential Revision: https://secure.phabricator.com/D15484
Summary:
Ref T7789. This implements a (probably) usable "git-lfs-authenticate" on top of the new temporary token infrastructure.
This won't actually do anything yet, since nothing reads the tokens.
Test Plan:
```
$ ./bin/ssh-exec --phabricator-ssh-user admin --ssh-command 'git-lfs-authenticate'
phabricator-ssh-exec: Expected `git-lfs-authenticate <path> <operation>`, but received too few arguments.
```
```
$ ./bin/ssh-exec --phabricator-ssh-user admin --ssh-command 'git-lfs-authenticate x'
phabricator-ssh-exec: Unrecognized repository path "x". Expected a path like "/diffusion/X/" or "/diffusion/123/".
```
```
$ ./bin/ssh-exec --phabricator-ssh-user admin --ssh-command 'git-lfs-authenticate diffusion/22'
Exception: Expected `git-lfs-authenticate <path> <operation>`, but received too few arguments.
```
```
$ ./bin/ssh-exec --phabricator-ssh-user admin --ssh-command 'git-lfs-authenticate diffusion/22 y'
Exception: Git LFS operation "y" is not supported by this server.
```
```
$ ./bin/ssh-exec --phabricator-ssh-user admin --ssh-command 'git-lfs-authenticate diffusion/22 upload'
{"header":{"Authorization":"Basic QGdpdC1sZnM6NmR2bDVreWVsaXNuMmtnNXBtbnZwM3VlaWhubmI1bmI="},"href":"http:\/\/local.phacility.com\/diffusion\/22\/new-callsign-free-repository.git\/info\/lfs"}
```
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T7789
Differential Revision: https://secure.phabricator.com/D15482
Summary:
Ref T10560. Reverts D15460. See that task for discussion: we dug up some more information to explain the behavior, and this key was just sort of sidestepping an analyze/cardinality estimate issue on the index.
With proper cardinality estimates it shouldn't be used, so just nuke it.
Test Plan: Ran `bin/storage adjust`, saw key drop.
Reviewers: eadler, chad
Reviewed By: chad
Maniphest Tasks: T10560
Differential Revision: https://secure.phabricator.com/D15486
Summary: Fixes T10603. This is the last of the ad-hoc temporary tokens.
Test Plan:
- Used a file token.
- Viewed type in {nav Config > Temporary Tokens}.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15481
Summary: Ref T10603. Swap these over and give them nice UI strings.
Test Plan:
- Refreshed a Twitter OAuth link.
- Unlinked and re-linked a Twitter account.
- Viewed the new type in {nav Config > Temporary Tokens}.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15480
Summary:
Ref T10603. We have a couple of sort of ad-hoc tokens, so start formalizing them. First up is MFA tokens.
Also adds a new config module panel for these.
Test Plan:
- Added MFA.
- Added MFA, intentionally fumbled the input, completed the workflow.
- Removed MFA.
- Viewed tokens, saw MFA sync tokens.
- Viewed new module config panel.
{F1177014}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15479
Summary:
Ref T10603. This makes minor updates to temporary tokens:
- Rename `objectPHID` (which is sometimes used to store some other kind of identifier instead of a PHID) to `tokenResource` (i.e., which resource does this token permit access to?).
- Add a `userPHID` column. For LFS tokens and some other types of tokens, I want to bind the token to both a resource (like a repository) and a user.
- Add a `properties` column. This makes tokens more flexible and supports custom behavior (like scoping LFS tokens even more tightly).
Test Plan:
- Ran `bin/storage upgrade -f`, got a clean upgrade.
- Viewed one-time tokens.
- Revoked one token.
- Revoked all tokens.
- Performed a one-time login.
- Performed a password reset.
- Added an MFA token.
- Removed an MFA token.
- Used a file token to view a file.
- Verified file token was removed after viewing file.
- Linked my account to an OAuth1 account (Twitter).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15478
Summary:
Ref T10603. This converts existing hard-codes to modular constants.
Also removes one small piece of code duplication.
Test Plan:
- Performed one-time logins.
- Performed a password reset.
- Verified temporary tokens were revoked properly.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15476
Summary:
Ref T10603. For LFS, we need to issue a new type of temporary token.
This makes the temporary token code modular so applications can add new token types without modifying the Auth application.
(I'm moving slowly here because it impacts authentication.)
Test Plan:
- Used `bin/auth recover` to get a one-time token from the CLI.
- Used "Forgot your password?" to get a one-time token from the web UI.
- Followed the web UI token to initiate a password reset, prompting generation of a password token.
- Viewed these tokens in the web UI:
{F1176908}
- Revoked a token.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10603
Differential Revision: https://secure.phabricator.com/D15475
Summary: Fixes T10591. This was accidentally reverted in 148a50e48b, probably when resolvign a merge/rebase.
Test Plan: Will push to production.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10591
Differential Revision: https://secure.phabricator.com/D15474
Summary: These are a little tight in Differential/Audit
Test Plan: Review spacing, desktop tablet and mobile.
Reviewers: avivey, epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15473
Summary: Updates Drydock to use two column + curtain layouts.
Test Plan: Tested what I could get to, need @epriestley to run this locally for edge cases.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D15467
Summary:
First pass at converting Differential, I likely have some buggy-poos but thought I'd toss this up now in case very bad bugs present.
To do:
- Need to put status back on Hovercards
- "Diff Detail" probably needs a better design
Test Plan: Looking at lots of diffs, admittedly I dont have harbormaster, etc, running locally. Checked Diffusion for Table of Content changes on small and large commits.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15463
Summary: Fixes T10539. When building mail targets, we fail to preserve whether a recipient was originally "To" or "Cc", and just move everyone to "To".
Test Plan:
Added a comment to a task with a "To" user and a "Cc" user, with `metamta.placeholder-to-recipient` set and `metamta.one-mail-per-recipient` set.
Got mail with me Cc'd as the Cc'd user:
{F1172020}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10539
Differential Revision: https://secure.phabricator.com/D15465
Summary: Ref T10537. Currently, when you have at least two cursors, the daemon can poll too frequently when processing the last source because it never hits the end-of-list condition.
Test Plan:
- Ran `bin/phd debug trigger`.
- Observed huge volumes of output before change as triggers fired as fast as possible.
- Observed reasonable poll frequency after change.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10537
Differential Revision: https://secure.phabricator.com/D15464
Summary: See Q335. This might need some tweaking, but the JS works now so we can move/style the node now.
Test Plan:
{F1169168}
- Also dragged stuff around, saw counts update properly.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15462
Summary: Fixes T10546. Some day, decades from now, we can revisit this when we iterate on Phrequent. Just don't regress for no real reason in the meantime, since it's easy enough to keep it working in reasonable shape.
Test Plan: {F1169096}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10546
Differential Revision: https://secure.phabricator.com/D15461
Summary:
Ref T10563. This isn't a complete fix, but should make viewing complex inline threads a little more manageable.
This just tries to put stuff in thread order instead of in pure chronological order. We can likely improve the display treatment -- this is a pretty minimal approach, but should improve clarity.
Test Plan:
T10563 has a "before" shot. Here's the "after":
{F1169018}
This makes it a bit easier to follow the conversations.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10563
Differential Revision: https://secure.phabricator.com/D15459
Summary:
Ref T10493.
- Call this action "Unassigned" instead of "Up For Grabs", since the latter implies that it's OK for anyone to grab it, which is a social/context thing that we probably shouldn't assume.
- Show who a task was previously assigned to in the transaction.
The text is a little clunky, yell if you've got a better wording? Or maybe I'll come up with something.
Test Plan: {F1166299}
Reviewers: chad
Reviewed By: chad
Subscribers: cburroughs
Maniphest Tasks: T10493
Differential Revision: https://secure.phabricator.com/D15454
Summary:
Ref T10560. I don't fully understand what MySQL is doing here, but it looks like this key improves the problematic dataset in practice.
(It makes sense that this key helps, I'm just not sure why the two separate keys and the UNION ALL are so bad.)
This key isn't hugely expensive to add, so we can try it and see if there are still issues.
Test Plan: Ran `bin/storage adjust`, saw key added to table. Used `SHOW CREATE TABLE ...` to verify the key exists. Used `EXPLAIN SELECT ...` to make sure MySQL actually uses it.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10560
Differential Revision: https://secure.phabricator.com/D15460
Summary:
Fixes T10562. I left this behavior sort of ambiguous in the original implementation because I didn't anticipate or stumble across this situation.
It's easy to fix: when you reply to a ghost, just put the reply in the exact same place as the ghost (even if it's a different diff), so they always move/ghost/port/thread together.
Test Plan:
See T10562 for reproduction steps and a "before" picture. Here's the after picture:
{F1168983}
The two comments at the bottom are pre-fix, and exhibit the bug. The comment at the top is post-fix, and appears adjacent to the original correctly.
Reviewers: chad
Reviewed By: chad
Subscribers: eadler
Maniphest Tasks: T10562
Differential Revision: https://secure.phabricator.com/D15458
Summary: I think this works?
Test Plan:
i am wizard
{F1168808}
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15457
Summary: Adds a date with the author name on the Authored By panel in Maniphest. A basic treatment, will see how it feels.
Test Plan: Look at a few tasks
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15456
Summary: This allows setting of full-width content underneath the two column, or full column all by itself. Maybe these names are bad.
Test Plan: Using these in Differential / Diffusion conversions.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15455
Summary: Until we have a new treatment, wrap the really long tags. Fixes T10550
Test Plan: Make a really long tag and assign it to a task.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T10550
Differential Revision: https://secure.phabricator.com/D15453
Summary:
To improve the performance of Herald, we attempt to generate the value for each field (e.g., a task title) only once.
For most field values this is cheap, but for some (like a commit's branches) it can be quite expensive. We only want to pay this cost once, so we cache field values.
However, D12957 accidentally added a check where we bypass the cache and generate the value for every field, before reading the cache. This causes us to generate each field for every rule that uses it, plus one extra time.
Instead, use the cache for this check, too. Also allow the cache to cache `null`, since it can be expensive to generate `null` even though the value isn't too interesting.
The value of this early hit isn't even used (we only care if it throws or not).
Test Plan:
- Wrote a rule like "if any condition matches: branches contain a, branches contain b, branches contain c".
- Put `phlog(new Exception())` in `DiffusionCommitBranchesHeraldField`.
- Before patch, saw `bin/repository reparse --herald <any commit>` compute branches three times.
- After patch, saw only one computation.
- Verified field values in the transcript view
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15451
Summary:
Ref T10538.
This is a tiny fraction of the API. GitHub has 25 primary event types; we currently partially parse 3 of them. GitHub has 17 issue event types; we currently partially parse 12.
Test Plan: Ran `arc unit`.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10538
Differential Revision: https://secure.phabricator.com/D15448
Summary:
Ref T10538. This sets up a Doorkeeper bridge for GitHub issues, and pulls issues from GitHub to create ExternalObject references.
Broadly, does nothing useful.
Test Plan: Put a `var_dump()` in there somewhere and saw it probably do something when running `bin/nuance update --item 44`.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10538
Differential Revision: https://secure.phabricator.com/D15447
Summary:
Ref T10538. The primary GitHub event activity stream does not report minor events (labels, milestones, etc).
GitHub has a second, similar activity stream which does report these events (the "Issues Events API").
Use two separate cursors: one consumes the primary stream; the second consumes the events stream.
One possible issue with this is that we may write events in a different order than they occurred, so GitHub shows "comment, label, close" but we show "comment, close, label" or similar. This is probably OK because the secondary API doesn't seem to have any very important events (e.g., it's probably fine if label changes are out-of-order), but we can conceivably put some buffer stage in between the two if it's an issue.
Test Plan: {F1164894}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10538
Differential Revision: https://secure.phabricator.com/D15446
Summary: Fixes T10545. Converts layout to two column.
Test Plan: Review a few project manage pages, see new layout and flag ability.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T10545
Differential Revision: https://secure.phabricator.com/D15450
Summary: Ref T10545, this brings flags back? and converts the layout to two column w/curtain
Test Plan: View a few manage pages.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T10545
Differential Revision: https://secure.phabricator.com/D15449
Summary:
Ref T10537. This adds an update daemon for pulling item data (e.g., figuring out who the author of a GitHub comment is) and routing items (e.g., sending them to a queue or applying them directly to a task).
Also adds `bin/nuance update --item X` for doing this manually for debugging.
And adds item types, for specializing item behavior. Previously, sources completely dictated item behavior, but I think we want something a little more flexible.
Test Plan:
- This still does nothing.
- Ran `bin/nuance update --item 15`.
- Saw an item route to a default queue.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10537
Differential Revision: https://secure.phabricator.com/D15441
Summary: This inverts colors and icons a bit, so they're not as harsh. So instead of a dark green item with white icon, its now light green with a dark green icon. I've also changed all text and comment boxes to be "grey" visually to separate out the UI from converation/actions. Give it a spin and let me know how this feels. I still need to update the comment UI.
Test Plan:
UIExamples, lots of various tasks and diffs.
{F1163837}
{F1163839}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15442
Summary:
Ref T10537. Ref T10538. This polls the GitHub events API and creates Nuance items from the raw data.
It does nothing useful with them.
Test Plan:
- Polled GitHub.
- Saw some items get created.
- X-Poll-Interval seemed to work.
- ETag seemed to work.
- Recognizing when we hit items we've already seen seemed to work.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10537, T10538
Differential Revision: https://secure.phabricator.com/D15440
Summary: Cleaner UI, moved visit to be button.
Test Plan: Make a phurl about cats, click on it.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D15385
Summary:
Ref T10537. More infrastructure:
- Put a `bin/nuance` in place with `bin/nuance import`. This has no useful behavior yet.
- Allow sources to be searched by substring. This supports `bin/nuance import --source whatever` so you don't have to dig up PHIDs.
Test Plan:
- Applied migrations.
- Ran `bin/nuance import --source ...` (no meaningful effect, but works fine).
- Searched for sources by substring in the UI.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10537
Differential Revision: https://secure.phabricator.com/D15436
Summary:
Ref T10537. Some sources (like the future "GitHub Repository" source) need to poll remotes.
- Provide a mechanism for sources to emit import cursors.
- Hook them into the trigger daemon so they'll fire periodically.
- Provide some storage.
This diff does nothing useful or interesting, and is pure infrastructure.
Test Plan:
- Ran `bin/storage upgrade -f`, no adjustment issues.
- Poked around Nuance.
- Ran the trigger daemon, verified it didn't crash and checked for Nuance stuff to do.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10537
Differential Revision: https://secure.phabricator.com/D15435
Summary: Ref T10537. Converts sources to EditEngine.
Test Plan:
- Created a new source.
- Edited an existing source.
- Submitted a complaint with the complaint form.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10537
Differential Revision: https://secure.phabricator.com/D15434
Summary: See D15432. There, we can use this test to check if the user //could// reassign the task by using "Edit Form" or the stacked actions, so any dedicated "claim" element is consistent with the other permissions.
Test Plan:
- Added a `var_dump($can_reassign)` after the call.
- Saw `true`.
- Edited the edit form, locked and disabled "Assigned To".
- Saw `false`.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D15433
