1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-22 14:52:41 +01:00
Commit graph

16135 commits

Author SHA1 Message Date
epriestley
1d58f14469 Remove WePay support from Phortune, and Restful/Httpful dependencies
Summary: Ref PHI1166. I'm documenting our dependencies, and we have approximately 5,000 lines of external code to support WePay as a Phortune provider. We don't use it, I'm almost certain it doesn't work, and we have no plans to use it in the near future. If we did pursue it, I'd probably just wrap the API in a 100-line `WePayFuture` anyway since 5K lines of dependencies to make a couple method calls is ridiculous.

Test Plan: Grepped for `wepay`, `httpful`, `restful`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: aurelijus

Differential Revision: https://secure.phabricator.com/D20521
2019-05-14 09:14:53 -07:00
epriestley
7a8d489ebd Fix a HunkParser typo, "identcal" -> "identical"
Summary: See rP3f8eccdaec8f85933bfea6ce5348bde25ced0489.

Test Plan: Read carefully.

Reviewers: amckinley, avivey

Reviewed By: avivey

Subscribers: avivey

Differential Revision: https://secure.phabricator.com/D20499
2019-05-07 11:16:55 -07:00
epriestley
5dc4e76eb9 Fix the direction of the commit/revision edge
Summary: Ref T13276. This edge is pointed the wrong way. Point it the right way.

Test Plan: Will verify production works better.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20490
2019-05-02 07:28:49 -07:00
epriestley
aba7d1406a Really fix movable panels inside tab panels by changing the JX.Request serializer
Summary:
Depends on D20475. Ref T13272. Currently, if you `JX.Request` with `data` like `{x: null}`, we submit that as `?x=null`, i.e. as though `null` was the string `"null"`.

This is weird and almost certainly never intended/desiarable. In particular, it causes a bug where panels embedded inside tab panels are incorrectly draggable.

It's possible this breaks something which relied on the buggy behavior, but that seems unlikely.

Test Plan: Tried to drag a panel inside a tab panel, it really truly didn't work.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20476
2019-05-01 15:38:40 -07:00
epriestley
c43618a3a8 Add "Move Left" and "Move Right" to dashboard tab panels
Summary: Depends on D20474. Ref T13272. Provide an easy way to rearrange tabs on a tab panel, by moving them left or right from the context menu.

Test Plan: Moved tabs left and right. Tried to move them off the end of the tab list, no luck.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20475
2019-05-01 15:35:53 -07:00
epriestley
9d716379a3 Add a "Customize Query" action to query panels to make it easier to make minor query adjustments
Summary:
Depends on D20473. Ref T13272. Fixes T7216. If you want to tweak the query a panel uses, you currently have to complete 7 Great Labors.

Instead, add a "Customize Query" action which lets you update the query inline.

Test Plan: {F6402171}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272, T7216

Differential Revision: https://secure.phabricator.com/D20474
2019-05-01 11:01:09 -07:00
epriestley
a15b0063df Add "Revision has passing builds" Herald rules for commit content (pushes) and commits (discovery)
Summary:
Depends on D20469. Ref T13276. See PHI1159. See PHI953. See PHI901.

Allow Herald to detect when "arc land" would (or did) warn users about failed or ongoing builds. This respects the "Warn on Landing" build plan behavior.

To accomplish this:

  - When we close a revision, set a "wrong build state" flag if it lands in the wrong build state.
  - If the revision is closed when we hit Herald, look for the flag.
  - If not (common for push rules, can happen for commit rules if we race against the revision update worker), hit Harbormaster ourselves and check the current state.

Test Plan:

  - Wrote a "Require Green" rule.
  - Ran it against various commits with various build states (good, not good).
  - Fiddled with "Warn on Landing" and saw the effect in rule evaluation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20470
2019-05-01 10:02:47 -07:00
epriestley
a78f141b3f Unify code for parsing "Reverts X" magic, and when something "reverts <hash>", also revert associated revisions
Summary:
Depends on D20468. Ref T13276. See PHI1008.

When a commit or revision "reverts <hash>", and that's the hash of a commit which has a revision, also write a "reverts" edge to the revision.

Test Plan:
Created "reverts X" objects for:

  - a revision;
  - a commit;
  - a commit with a revision (both got marked properly).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20469
2019-05-01 09:47:40 -07:00
epriestley
56f5c2443b Consolidate readers of "differential.revisionID" property
Summary:
Depends on D20467. Ref T13277. Currently, the "MessageParserWorker" writes this property on commits, then Herald and Audit both read it.

Make them share code so this property has one writer and one reader. This property isn't great, but at least now the badness is hidden.

Currently, we can't just use edges because they may not have been written yet. I am likely to just do this, soon:

  - Just write the edges (in "MessageParserWorker").
  - Hide the edges from mail.

However, we'll sort-of lose the "revisionMatchData" explanation thing if I do that. Maybe this is fine? But when commits match because hashes match, it legitimately isn't obvious.

For now, just reduce the amount of harm/badness here.

Test Plan:
  - Ran `bin/repository reparse --publish ...`.
  - Ran a Herald "Audit" rule using the "Accepted Differential revision" field.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20468
2019-05-01 09:46:17 -07:00
epriestley
45f01dc716 Restore "Limit" to dashboard Query panels
Summary: See PHI1220. Ref T13272. I accidentally left the ability to set a query limit behind when updating this.

Test Plan: Edited a query panel, set/removed the limit, tried to set an invalid limit.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20472
2019-05-01 08:41:43 -07:00
epriestley
dfbc4c1cd3 Fix an issue where editing a dashboard panel from a dashboard could duplicate the panel
Summary:
Depends on D20472. Ref T13272. Currently, when you edit a panel from a dashboard, we try to add the panel to the dashboard. This always works since dashboards no longer enforce panel uniqueness, and you can end up with duplicate panels.

Instead, only add panels if we're creating them.

Test Plan:
  - Edited an existing panel, no duplication.
  - Created a new panel, saw it added to the dashboard.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13272

Differential Revision: https://secure.phabricator.com/D20473
2019-04-30 08:27:20 -07:00
epriestley
5462a52b8a Update Quickstart SQL
Summary:
This hasn't been updated in a bit more than a year (last updated in D18594) and we've accumulated a fair number of SQL patches. Update it.

This is mostly automatic (with `bin/storage quickstart`), except:

  - Manual edit to one migration for a missed callsite to `DashboardInstall`.
  - Replaced two InnoDB tables that still have FULLTEXT indexes with MyISAM (see rP6cedd4a95cfc).

This is not really possible to review and more for reference than examination. `bin/storage quickstart` has historically worked correctly.

Test Plan: I have great faith that `bin/storage quickstart` is a script which creates a big `.sql` file.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20480
2019-04-30 08:19:39 -07:00
epriestley
648d5ce190 Set a URI on Auth Messages, so the "Change Details" dialog from the transaction record has a cancel button
Summary:
If you edit an auth message in Auth > Customize Messages, then click "Show Details" in the transaction record, the resulting dialog uses the object's handle's URI to generate a "cancel" button.

Since these handles currently have no URI, the dialog currently has no cancel/done button to close it.

Test Plan: Edited an auth message, clicked "Show Details", was now able to click "Done" to close the dialog.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20471
2019-04-30 06:59:04 -07:00
epriestley
6d13b9fb7a Make very minor generality improvements to the scope selector
Summary:
See PHI985. I think we pretty much need to start applying language-specific rules, but we can apply at least one more relatively language-agnostic rule: don't match lines which are indented 3+ levels.

In C++, we may have symbols like this:

```
class X {
  public:
    int m() { ... }
}
```

..but I believe no mainstream language puts symbol definitions 3+ levels deep.

Also clean up some of the tab handling very slightly.

Test Plan: Tests pass, looked at some C++ code and got slightly better (but still not great) matches.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20479
2019-04-30 06:52:29 -07:00
epriestley
6eae75d8f2 When building a synthetic diff in Differential, adjust diff properties correctly
Summary:
See PHI1218. When rendering "A vs B", we currently show the properties of diff A without modification.

Instead, take properties from the same place we're taking change details.

See T12664 for a followup.

Test Plan:
  - In diff A, removed "+x" from a file.
  - In diff B, changed the file but did not remove "+x" from it.
  - Diffed B vs A.
    - Before change: UI incorrectly shows "+x" removed (both sides incorrect, just showing the change from diff A).
    - After change: UI shows 100644 -> null, which is half right.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20478
2019-04-30 06:51:40 -07:00
epriestley
0d9362355b In ApplicationTransactionEditor, determine new objects with "getID()" instead of "getPHID()"
Summary:
See <https://discourse.phabricator-community.org/t/unable-to-reload-object-that-hasnt-been-loaded/2677>.

When editing "Config" objects, they currently get a PHID set outside of the TransactionEditor. They probably should not, but fixing that is likely an involved change.

This causes us to incorrectly fail to detect `$is_new` correctly and try to `reload()` and object with no ID.

To work around this, test for new objects with `getID()` instead of `getPHID()`.

Test Plan: Edited any config value with the web UI.

Reviewers: amckinley

Differential Revision: https://secure.phabricator.com/D20482
2019-04-29 07:37:03 -07:00
epriestley
c0a4d1de13 Merge the "Herald" and "Owners" daemon workers into a single "Publish" worker
Summary:
Depends on D20466. Ref T13277. Currently:

  - The "Owners" worker writes ownership relationships (e.g., commit X affects package Y, because it touches a path in package Y) -- these are just edges.
  - It also triggers audits.
  - Then it queues a "Herald" worker.
  - This formally publishes the commit and triggers Herald.

These aren't really separate steps and can happen more easily in one shot. Merge them.

Test Plan:
  - Ran `bin/repository reparse --publish` to republish various commits, got sensible behavior.
  - Grepped for "IMPORTED_OWNERS", "IMPORTED_HERALD", "--herald", "--owners", and "--force-local" flags.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20467
2019-04-25 09:45:59 -07:00
epriestley
0fab41ff3c Show "hold reasons" on commit page, not on "Edit" page
Summary:
Depends on D20465. Ref T13277. Currently, when a commit is unpublished, we put a single line about it on the "Edit Commit" page. This is pretty much impossible to find.

Move it to the main page. This treatment is more big/bold than I'd probably like to end up, but we should probably overshoot on the explanatory text until users get used to this behavior.

Also, allow searching for only published / unpublished commits.

Test Plan: {F6395705}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20466
2019-04-25 09:22:49 -07:00
epriestley
8f43c773b8 Remove nearly all remaining references to "Autoclose"
Summary:
Depends on D20464. Ref T13277. Broadly:

  - Move all the "should publish X" and "why aren't we publishing X" stuff to a separate class (`PhabricatorRepositoryPublisher`).
  - Rename things to be more consistent with modern terminology ("Publish", "Permanent Refs").

Test Plan:
This could use some trial-by-fire on `secure`, but:

  - Grepped for all symbols.
  - Viewed various commits.
  - Reparsed commits.
  - Here's a commit with an explanation:

{F6394569}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20465
2019-04-24 08:29:41 -07:00
epriestley
45b9859f02 Remove "--force-autoclose" from "bin/repository reparse"
Summary: Depends on D20463. Ref T13277. This flag was added some time before 2015 and I don't think I've ever used it. Just get rid of it.

Test Plan: Grepped for `force-autoclose`, `forceAutoclose`, `AUTOCLOSE_FORCED`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20464
2019-04-24 08:24:06 -07:00
epriestley
a2d3d8edeb Move "update related object after commit" to a separate worker in the task queue
Summary:
Depends on D20462. Ref T13276. Currently, the "Message" parser also updates related tasks and revisions when a commit is published.

For PHI1165, which ran into a race with message parsing, I originally believed we needed to separate this logic and lock + yield to avoid the race. D20462 provides what is probably a better approach for avoiding the race.

Still, I think separating these "update related revisions" and "updated related tasks" chunks into separate workers is a net improvement. There may still be some value in doing lock + yield in the future to deal with other issues, and when we occasionally run into problems with pulling a diff out of the repository to update the revision (usually because the diff is too big) this isolates the problem better and allows the commit to import.

I think the only thing to watch out for here is that Herald may now run before the revision and commit are attached to one another. This is fine for all current Herald rules, we just need to be mindful in implementing new rules.

Test Plan: Used `bin/repository reparse --message` on various commits, including commits that close revisions and close tasks.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20463
2019-04-24 06:32:27 -07:00
epriestley
b3b1cc64bd When applying transactions, acquire a read lock sooner
Summary:
Depends on D20461. Ref T13276. Ref T13054.

Currently, we acquire the transaction read lock after populating "old values" in transactions and filtering transactions with no effect.

This isn't early enough to prevent all weird chaotic races: if two processes try to apply a "close revision" transaction at the same time, this can happen:

```
PROCESS A             PROCESS B
Old Value = Open      Old Value = Open
Transaction OK: Yes   Transaction OK: Yes
Acquire Read Lock     Acquire Read Lock
Got Read Lock!        Wait...
Apply Transactions    Wait...
New Value = Closed    Wait...
Release Lock          Wait...
                      Got Read Lock!
                      Apply Transactions
                      New Value = Closed
                      Release Lock
```

That's not great: both processes apply an "Open -> Closed" transaction since this was a valid transaction from the viewpoint of each process when it did the checks.

We actually want this:

```
PROCESS A             PROCESS B
Acquire Read Lock     Acquire Read Lock
Got Read Lock!        Wait...
Old Value = Open      Wait...
Transaction OK: Yes   Wait...
Apply Transactions    Wait...
New Value = Closed    Wait...
Release Lock          Wait...
                      Got Read Lock!
                      >>> Old Value = Closed
                      >>> Transaction Has No Effect!
                      >>> Do Nothing / Abort
                      Release Lock
```

Move the "lock" part up so we do that.

This may cause some kind of weird second-order effects, but T13054 went through pretty cleanly and we have to do this to get correct behavior, so we can survive those if/when they arise.

Test Plan:
  - Added a `sleep(10)` before the lock.
  - Ran `bin/repository message --reparse X` in two console windows, where X is a commit that closes revision Y and Y is open.
    - Before patch: both windows closed the revision and added duplicate transactions.
    - After patch: only one of the processes had an effect.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jmeador

Maniphest Tasks: T13276, T13054

Differential Revision: https://secure.phabricator.com/D20462
2019-04-24 05:57:29 -07:00
epriestley
42c02557e4 Remove all remaining readers and writers for TABLE_COMMIT
Summary: Depends on D20459. Ref T13276. I'll file a followup to actually destroy the table.

Test Plan:
- Grepped for `TABLE_COMMIT`.
- Ran `bin/storage upgrade -f`, got a clean bill of health.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20461
2019-04-24 05:55:23 -07:00
epriestley
ec0085fd0c Align "RevisionQuery->needCommitPHIDs()" to use Edges, not the legacy table
Summary:
Depends on D20458. Ref T13276. Although I'm not thrilled about "needCommitPHIDs()", it has a few callers, including custom fields. Allow "need + attach + get" to survive for now since they're reasonably modern, at least.

However, use edges instead of "TABLE_COMMIT" and require `need...()` + `get...()`, removing the direct `load...()`.

Also remove `RevisionQuery->withCommitPHIDs(...)`, which has no callers.

Test Plan:
- Grepped for `loadCommitPHIDs` (only two hits, the private `RevisionQuery` method).
- Called "differential.getrevision", got commits.
- Viewed a revision, saw "Commits: ...".
- Grepped for `withCommitPHIDs()`, no callers on `RevisionQuery` (some other query classes have methods with this name).
- Called "differential.query", got commits.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20459
2019-04-24 05:54:38 -07:00
epriestley
7e8dc0742b Remove all callers to "DifferentialRevision->loadIDsByCommitPHIDs()"
Summary: Depends on D20457. Ref T13276. Kill all remaining callers to this method and delete it.

Test Plan:
- Grepped for `loadIDsByCommitPHIDs`.
- Viewed blame again to make sure I didn't break it.
- Viewed "History" view for commits with revisions.
- Viewed "Graph" view for commits with revisions.
- Viewed "Merged Commits" table for commits with revisions.
- Viewed "Compare" table for commits with revisions.
- Viewed "Repository" main page history table for commits with revisions.
- Grepped for `linkRevision`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20458
2019-04-24 05:53:35 -07:00
epriestley
fc92cf4382 Move BlameController away from ancient "TABLE_COMMIT"
Summary:
Ref T13276. Differential has a pre-edge "TABLE_COMMIT" with about a half-dozen weird callers I'd like to get rid of.

Move blame to use edges instead. (Bonus: this makes blame respect edge edits in the UI.)

Since there are some more callers to clean up this code may move into some "RelatedObjectQueryThing" class or something, but I'm taking it one step at a time for now.

Test Plan: {F6394106}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13276

Differential Revision: https://secure.phabricator.com/D20457
2019-04-24 05:44:40 -07:00
epriestley
70c643c685 Simplify implementation of "pure" Chart functions
Summary:
Depends on D20445. Ref T13279. I'm not sure what the class tree of functions actually looks like, and I suspect it isn't really a tree, so I'm hesitant to start subclassing. Instead, try adding some `isSomethingSomething()` methods.

We have some different types of functions:

  # Some functions can be evaluated anywhere, like "constant(3)", which always evaluates to 3.
  # Some functions can't be evaluated anywhere, but have values everywhere in some domain. This is most interesting functions, like "number of open tasks". These functions also usually have a distinct set of interesting points, and are constant between those points (any count of anything, like "open points in project" or "tasks closed by alice", etc).
  # Some functions can be evaluated almost nowhere and have only discrete values. This is most of the data we actually store, which is just "+1" when a task is opened and "-1" when a task is closed.

Soon, I'd like to be able to show ("all tasks" - "open tasks") and draw a chart of closed tasks. This is somewhat tricky because the two datasets are of the second class of function (straight lines connecting dots) but their "interesting" x values won't be the same (users don't open and close tasks every second, or at the same time).

The "subtract X Y" function will need to be able to know that `subtract "all tasks" 3` and `subtract "all tasks" "closed tasks"` evaluate slightly differently.

To make this worse, the data we actually //store// is of the third class of function (just the "derivative" of the line chart), then we accumulate it in the application after we pull it out of the database. So the code will need to know that `subtract "derivative of all tasks" "derivative of closed tasks"` is meaningless, or the UI needs to make that clear, or it needs to interpret it to mean "accumulate the derivative into a line first".

Anyway, I'll sort that out in future changes. For now, simplify the easy case of functions in class (1), where they're just actual functions.

Add "shift(function, number)" and "scale(function, number)". These are probably like "mul" and "add" but they can't take two functions -- the second value must always be a constant. Maybe these will go away in the future and become `add(function, constant(3))` or something?

Test Plan: {F6382885}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20446
2019-04-23 12:29:45 -07:00
epriestley
edaf17f3fe Make chart function argument parsing modular/flexible with 900 pages of error messages
Summary:
Depends on D20444. Ref T13279. Instead of ad-hoc parsing and messages, formalize chart function arguments.

Also, add a whole lot of extra type checking.

Test Plan: Built and charted various functions with various valid and invalid argument lists, got sensible-seeming errors and results.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20445
2019-04-23 12:26:32 -07:00
epriestley
7b8ac020b5 Change "Revision Close" story to use commit identities only with no fallback to commit data
Summary:
See PHI1213. If we don't have identities for "revision X closed by commit Y" stories, just do the plain non-attribution rendering rather than trying to fall back. Falling back won't work since we don't load the data, which should be OK now since identities seem like they're in generally good shape.

(We could probably just throw out the fallback behavior instead, but we can always clean things up later.)

Test Plan: Forced no commit identity on a revision, loaded it, saw reasonable story rendering.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20460
2019-04-23 11:57:33 -07:00
epriestley
b0b8926c75 Fix "before/after" cursor paging for API call "feed.query"
Summary:
Ref T13266. See <https://discourse.phabricator-community.org/t/undefined-method-setafterid-when-calling-feed-query/2653>.

This older API call needs an update to the newer paging/cursor API.

Test Plan: Called `feed.query` with an "after" parameter.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: Itms

Maniphest Tasks: T13266

Differential Revision: https://secure.phabricator.com/D20456
2019-04-23 11:51:16 -07:00
epriestley
a4a1143b18 Fix an issue where internal paging of notifications could fail if some notifications are not visible
Summary:
Ref T13266. See <https://discourse.phabricator-community.org/t/notification-page-throws-unrecoverable-fatal-error/2651/>.

The "notifications" query currently uses offset paging for no apparent reason (just a legacy issue?), so some of the paging code is only reachable internally.

  - Stop it from using offset paging, since modern cursor paging is fine here (and Feed has used cursor paging for a long time).
  - Fix the non-offset paging to work like Feed.

Also:

  - Remove a couple of stub methods with no callsites after cursor refactoring.

Test Plan:
  - Set things up so I had more than 100 notifications and some in the first 100 were policy filtered, to reproduce the issue (I just made `FeedStory` return `NO_ONE` as a visibility policy).
  - Applied the patch, notifications now page cleanly.
  - Verified that "Next Page" took me to the right place in the result list.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: hskiba

Maniphest Tasks: T13266

Differential Revision: https://secure.phabricator.com/D20455
2019-04-23 11:45:04 -07:00
epriestley
a82a2b8459 Simplify non-bare working copy rules for the new "git fetch" strategy
Summary:
Ref T13280. In D20421, I changed our observe strategy to `git fetch <uri>` in all cases.

This doesn't work in an ancient, non-bare repository if `master` is checked out and `master` is also fetch: `git` refuses to overwrite the local ref unless we pass `--update-head-ok`. Pass this flag.

Also, remove some code which examines branches and tags in a special way for non-bare working copies. The old `git fetch <origin>` code without explicit revsets meant that `refs/remotes/orgin/heads/xyz` got updated instead of `refs/heads/xyz`. We now update our local refs in all cases (bare and non-bare) so we can throw away this special casing.

Test Plan:
  - Replaced a modern bare working copy with a non-bare working copy by explicitly using `git clone` without `--bare`.
  - Ran `bin/repository update`, hit the `--update-head-ok` error. Applied the patch, got a clean update.
  - Used the "repository.branchquery" API method...
    - ...with "contains" to trigger the "git branch" case. Got identical results after removing the special casing.
    - ...without "contains" to trigger the "low level ref" case. Got identical results after removing the special casing.
  - Grepped for `isWorkingCopyBare()`. The only remaining callsites deal with hook paths, and genuinely need to be specialized.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13280

Differential Revision: https://secure.phabricator.com/D20450
2019-04-19 07:06:10 -07:00
epriestley
e1076528ef Copy the "line-chart" behavior to "line-chart-legacy" to keep "Maniphest > Reports" working
Summary:
Ref T13279. Charting changes alter how the "line-chart" behavior works, but the "Burnup Chart" still relies on the old behavior.

Although I'm intending to remove "Maniphest > Reports" once Facts is a minimally sufficient replacement, copy this behavior to keep it working until we're ready to pull the trigger.

Also fix a leftover typo from D20435.

Test Plan: Viewed a legacy Maniphest burnup rate report.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20449
2019-04-19 07:05:37 -07:00
Austin McKinley
de74332639 Improve timeline rendering for old macros
Summary: This always bugs me when I'm going through `secure` looking for the spiciest macros.

Test Plan: Forced a date to be null, saw reasonable text.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20453
2019-04-18 18:25:17 -07:00
epriestley
d8dba26a08 Add a "sin()" function to charts
Summary:
Depends on D20443. Ref T13279. This is probably not terribly useful on its own, but is mostly a function which takes another function as an argument, and a step toward more useful functions like arithmetic and drawing a picture of an owl.

The only structural change here is that functions now read data parameters (domain, sample limit) using a more tailored "ChartDataQuery" instead of reading the actual axis. Mostly, I want a more cohesive representation of query state that can be easily passed to sub-functions, as here.

Test Plan: {F6382432}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20444
2019-04-18 13:19:02 -07:00
epriestley
c4e4448ee6 Add chart functions "x()" and "constant(3)"
Summary:
Depends on D20442. Ref T13279. Add basic support for drawing chart functions that are not based on Facts first-party ETL datasets. Some general goals:

  - This might be useful to draw a line like "goal" or "profitability".
  - This might be useful to pull data from an external source.
  - For composable functions like "add" or "subtract", which are useful in manipulating ETL datasets, these value functions will make testing easier.

Test Plan:
Added a `constant(256)` function:

{F6382408}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20443
2019-04-18 13:16:42 -07:00
epriestley
cd2215fd4a Don't warn that workboard columns need a name when editing milestone columns
Summary: See <https://discourse.phabricator-community.org/t/columns-must-have-a-name-while-editong-point-limit-on-milestone-column/2650>. This check doesn't make sense for proxy columns, including milestone columns.

Test Plan: Added a point limit to a milestone column.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20448
2019-04-18 13:13:10 -07:00
epriestley
23d8f000f7 Select the domain (X-axis range) for charts before pulling data
Summary:
Depends on D20441. Ref T13279. Currently, we pull all the data, then decide what the X-axis should look like.

Since users will reasonably want to do stuff like "show me march-april 2018" in the future, we need to move toward flipping this around so that we can support cases where the domain is specified by the user.

For actual chart functions (like "constant(3)" or "cos(x)"), we must also know the domain before we pull data, since there are an infinite number of places where we can evaluate the function "constant(3)".

See note in T13279 about continunity.

Test Plan: {F6382356}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20442
2019-04-18 13:11:17 -07:00
epriestley
954831f533 Separate chart functions into a class tree
Summary:
Depends on D20440. Ref T13279. Create a class to represent a chartable function: something we can get some data points out of.

Then, make the chart chart two functions.

For now, the only supported function is "fact(key)", which pulls data from the Facts ETL pipeline, identified by "key", and takes no other arguments.

In future changes, I plan to support things like "fact(tasks.open.project, PHID-PROJ-xyz)", "constant(1000)" (e.g. to draw a goal line), "sum(fact(...), fact(...))" (to combine data from several projects), and so on.

The UI may not expose this level of power for a while (or maybe ever) but until we get close enough to the UI that these features are a ton of extra work I'm going to try to keep things fairly flexible/modular.

Test Plan: {F6382286}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20441
2019-04-18 13:05:52 -07:00
epriestley
45b3c23148 Fetch chart data via async request and redraw charts when the window is resized
Summary:
Depends on D20439. Ref T13279. Some day, charts will probably need to reload themselves or do a bunch of defer/request-shaping magic when they're on a dashboard with 900 other charts.

Give the controller separate "HTML placeholder" and "actual data" modes, and make the placeholder fetch the data in a separate request.

Then, make the chart redraw if you resize the window instead of staying at whatever size it started as.

Test Plan:
  - Loaded a chart, saw it load data asynchronously.
  - Resized the window, saw the chart resize.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20440
2019-04-18 07:10:05 -07:00
epriestley
044c6fbc19 Support drawing multiple functions on the same chart
Summary: Depends on D20438. Ref T13279. Widgets produced vs widgets sold, etc.

Test Plan: {F6381609}

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20439
2019-04-18 07:07:33 -07:00
epriestley
02981f0add Fix negative chart values, add storage for charts
Summary:
Ref T13279. I think I'm going to fling some stuff at the wall for a bit here and hope most of it sticks, so this series of changes may not be terribly cohesive or focused. Here:

The range of the chart is locked to "[0, 105% of max]". This is trying to make a pleasing extra margin above the maximum value, but currently just breaks charts with negative values. Later:

    - I'll probably let users customize this.
    - We should likely select 0 as the automatic minimum for charts with no negative values.
    - For charts with positive values, it would be nice to automatically pick a pleasantly round number (25, 100, 1000) as a maximum by default.

We don't have any storage for charts yet. Add some. This works like queries, where every possible configuration gets a short URL slug. Nothing writes or reads this yet.

Rename `fn()` to `css_function()`. This builds CSS functions for D3. The JS is likely to get substantial structural rewrites later on, `fn()` was just particularly offensive.

Test Plan: Viewed a fact series with negative values. Ran `bin/storage upgrade`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13279

Differential Revision: https://secure.phabricator.com/D20438
2019-04-18 06:59:41 -07:00
epriestley
faf0a311ec In Git repositories, use "git symbolic-ref HEAD ..." to select the default branch
Summary:
Depends on D20434. Fixes T5963. Broadly, the issue here is that when:

  - You create a new, empty repository.
  - Then, you work on some branch other than `master`, without ever creating `master`.

...you get a warning on `git clone`:

> warning: remote HEAD refers to nonexistent ref, unable to checkout

To fix this, point the symbolic-ref HEAD at `refs/heads/<default-branch>` after installing commit hooks.

This fixes the warning, and also means that `git clone` will check out the repository default branch by default, which is nice.

There are a few caveats about this behavior (see T5963 for discussion) but nothing too substantial.

The only real issue is that Git prevents deletion of the default branch without a config setting. Just set that settting.

Test Plan:
See T5963.

In a repository, set `HEAD` to point somewhere invalid. Ran `bin/repository update ...`. Saw HEAD pointed back at the repository default branch.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T5963

Differential Revision: https://secure.phabricator.com/D20435
2019-04-18 05:46:40 -07:00
epriestley
7be671fb07 Update "Autoclose" documentation to focus on "Permanent Refs" instead
Summary:
Depends on D20433. Ref T13277. Since "Autoclose" no longer exists, update the documentation.

Currently, this documentation focuses a lot on troubleshooting because users historically had a lot of trouble with figuring out why things were or were not autoclosing. I haven't seen any real confusion about this in years, so I suspect we may have improved the import pipeline and/or UI to make this less of a problem.

It's also possible that this document "fixed" the problem, but usually I expect a documentation fix to not affect the frequency of reports, just make them easier to resolve, so I doubt it.

If unclear things remain //and// documentation really did fix it, maybe we can fix the issues. Or we can just put the troubleshooting documentation back.

Test Plan: Read documentation.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20434
2019-04-18 05:43:15 -07:00
epriestley
c33f544e74 Deprecate "Track Only" in the Diffusion UI
Summary:
Depends on D20432. Ref T13277. Fixes T12967. Removes some "Track Only" hints and warns that the feature is deprecated in favor of "Permanent Refs" and "Fetch Only".

(This "fixes" T12967 by mooting it.)

Test Plan: Viewed "branches" sectino of the manage UI, edited "braches" section of a repository.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277, T12967

Differential Revision: https://secure.phabricator.com/D20433
2019-04-18 05:40:02 -07:00
epriestley
6449a0ecb2 Rename some internal "Autoclose" mentions to "Permanent Refs"
Summary: Depends on D20428. Ref T13277.

Test Plan: Grep / reading.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20432
2019-04-18 05:38:09 -07:00
epriestley
d1223ac577 When a commit appears as an ancestor of a permanent ref for the first time, run all import steps
Summary:
Depends on D20427. Ref T13277. As an optimization, when we discover that a commit which was previously only on a non-permanent ref ("tmp-epriestley-123") is now reachable from a permanent ref ("master"), we currently queue only a new "message" parse step.

This is an optimization because these commits previously got the full treatment (feed, publish, audit, etc) as soon as they were discovered. Now, those steps only happen once a commit is reachable from a permanent ref, so we need to run everything.

Test Plan:
  - Pushed local "tmp-123" branch to remote tag "tmp-123".
  - Updated repository with "bin/repository update", saw commit import as a "not on any permanent ref" commit, with no herald/audit/etc.
  - Merged "tmp-123" tag into "master".
  - Pushed new "master".
  - Updated repository with "bin/repository refs ... --trace --verbose", saw commit detected as now reachable from a permament ref.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20428
2019-04-18 05:37:15 -07:00
epriestley
9107c2e262 Deprecate the "Commit is on autoclose/permanent branch" Herald "Commit" field
Summary:
Depends on D20426. Ref T13277. The new behavior is to fire Herald only once a commit becomes reachable from a permanent ref (previously, an "Autoclose" branch).

That means that every "Commit" Herald rule implicitly has this field as a condition, and it no longer does anything.

Test Plan: Wrote a Herald rule, saw this as an option in the "Deprecated" section.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20427
2019-04-18 05:32:45 -07:00
epriestley
aed755e1d8 Expose repository ref rules via "diffusion.repository.search"
Summary:
Depends on D20425. Ref T13277. See PHI1067. There's currently no way to retrieve branch/ref rules over the API, which makes some management operations against a large number of repositories difficult.

Expose these rules to the API.

Test Plan: Called `diffusion.repository.search`, got rules in the result set.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20426
2019-04-18 05:27:37 -07:00
epriestley
7a4ef2bad8 Move the repository "Publishing" option to the "Basics" panel in repository management
Summary:
Depends on D20424. Ref T13277. Now that the "Actions" panel only has one item ("Publishing"), just move it to the "Basics" panel.

Update the UI to show active/publishing status more clearly and relate them to one another and importing state.

Test Plan: {F6378087}

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13277

Differential Revision: https://secure.phabricator.com/D20425
2019-04-18 05:24:27 -07:00