Summary:
Improve performance of large discovery tasks in Git by using subprocess streaming, like we do for Mercurial.
Basically, we save the cost of running many `git log` commands by running one big `git log` command but only parsing as much of it as we need to. This is pretty complicated, but we more or less need it for mercurial (which has ~100ms of 'hg' overhead instead of ~5ms of 'git' overhead) so we're already committed to most of the complexity costs. The git implementation is much simpler than the hg implementation because we don't need to handle all the weird parent rules (git gives us to them easily).
Test Plan:
Before, `discover --repair` on Phabricator took 35s:
real 0m35.324s
user 0m13.364s
sys 0m21.088s
Now 7s:
real 0m7.236s
user 0m2.436s
sys 0m3.444s
Note that most of the time is spent inserting rows after discover, the actual speedup of the git discovery part is much larger (subjectively, it runs in less than a second now, from ~28 seconds before).
Also ran discover/pull on single new commits in normal cases to verify that nothing broke in the common case.
Reviewers: jungejason, nh, vrana
Reviewed By: vrana
CC: aran
Maniphest Tasks: T1401
Differential Revision: https://secure.phabricator.com/D2851
Summary:
If a repository is missing commits because they mysteriously vanished, there's no reasonable way to get them back right now. Provide a way to ignore the state in the database and rediscover the entire repository unconditionally.
We don't queue any reparses or anything, but when I move reparse into this script we can hook things up or something. This generally shouldn't be too important anyway.
Test Plan: Ran `repository discover --repair` on Phabricator.
Reviewers: jungejason, nh, vrana
Reviewed By: vrana
CC: aran
Differential Revision: https://secure.phabricator.com/D2850
Summary:
Nothing new or exciting here yet, just moving the random scripts/repositories/ things to bin/repository. Also add `repository list`.
(Console stuff comes from D2841.)
Test Plan: Ran `repository list`, `repository pull`, `repository discover`, `repository discover --verbose`, `repository help`.
Reviewers: jungejason, vrana
Reviewed By: vrana
CC: aran
Differential Revision: https://secure.phabricator.com/D2849
Summary: Add verbose logging. This logging is activated by setting "phd.verbose" in the config, running "phd debug", or explicitly in scripts/repository/pull.php and scripst/repository/discover.php
Test Plan:
>>> orbital ~/devtools/phabricator $ ./scripts/repository/discover.php GTEST
Discovering 'GTEST'...
<VERB> PhabricatorRepositoryPullLocalDaemon Discovering commits in repository 'GTEST'...
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '()_+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
<VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
<VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '_+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
<VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
<VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch', at 774c7737b2d560a291697126bf4513204ccf661a.
<VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch-1', at dc97539bee07293f95990d71f4638335a2531d69.
<VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
<VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch-2', at 1acfaec313c46dd3caa90448800181fb91b0270f.
Reviewers: jungejason
Reviewed By: jungejason
CC: aran
Differential Revision: https://secure.phabricator.com/D2843
Summary:
See D2855 for usage.
This has a drawback that inlines without a comment (synthetic comments) are not attached anywhere.
I don't like adding more and more methods so I've chosen this solution.
Plus comments and inline comments are often useful together.
Test Plan: Called the method on a revision with inline comments.
Reviewers: epriestley
Reviewed By: epriestley
CC: vii, aran, Korvin
Differential Revision: https://secure.phabricator.com/D2844
Test Plan:
Displayed repository.
Displayed repository history.
Wondered that we actually have bunch of commits without a revision.
Displayed blame.
Didn't display merge commit.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D2840
Summary: in svn and hg (for now), no branch used.
Test Plan: will test live
Reviewers: epriestley
CC: nh, vrana, aran, Korvin
Differential Revision: https://secure.phabricator.com/D2839
Summary:
This allows translations to specify custom date and time format.
We will need to call `setlocale()` or build system independent on it at some point but I guess it is good for now.
Test Plan: Translated 'D, M j, g:i A', saw it in diff.
Reviewers: btrahan, epriestley
Reviewed By: btrahan
CC: aran, Korvin
Maniphest Tasks: T1139, T1321
Differential Revision: https://secure.phabricator.com/D2809
Summary: I need this much more than commit but it can be useful too.
Test Plan: Displayed blame, clicked on the link.
Reviewers: epriestley
Reviewed By: epriestley
CC: john, aran, Korvin, phunt
Differential Revision: https://secure.phabricator.com/D2820
Summary: People want to create filters checking if they are in CC which is almost impossible with multiplexing in Outlook.
Test Plan: Sent e-mail with multiple CCs, verified headers.
Reviewers: epriestley, nh
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D2829
Summary:
Currently, notifications aren't z-indexed so they can end up underneath some other elements:
{F13144}
Fix this so they float correctly.
Test Plan: {F13166}
Reviewers: jungejason, btrahan
Reviewed By: jungejason
CC: aran
Maniphest Tasks: T1398
Differential Revision: https://secure.phabricator.com/D2834
Summary: it's calling pull daemon's failure. This is actually Nick's fix.
Test Plan: Nick already manually ran it on daemon machine.
Reviewers: vrana, nh
Reviewed By: vrana
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D2828
Summary: documentation changes for D2822
Test Plan:
Generated docs, but failed to read them because no one at Facebook
remembers how to get docs working in our sandbox.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T1233
Differential Revision: https://secure.phabricator.com/D2827
Summary: Installs may set global translation which may not be available before loading libraries.
Test Plan:
$ aphrontpath.php /
Reviewers: nh
Reviewed By: nh
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D2819
Summary: Add a `notification.debug` setting that shows debug info in the browser. Also improve some logging/error handling stuff and fix a bug with host names.
Test Plan: {F13098}
Reviewers: jungejason, btrahan, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T944
Differential Revision: https://secure.phabricator.com/D2810
Summary:
Currently, when rendering the panel we count the number of unread notifications in the last 15, but this means we can never render a number larger than 15. If the user has more unread notifications than that, or unread notifications older than the most recent 15, there will be a flash of the higher number and then it will update to the lower number afterward.
Instead, count all unread notifications. This uses the same method used to render both numbers.
Test Plan: Loaded a page, checked the menu, nothing exploded.
Reviewers: btrahan, jungejason, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T974
Differential Revision: https://secure.phabricator.com/D2811
Summary: loadRunningDaemons loads daemons that may be running (have a file pid). other commands handle this appropriately so just make sure the start command knows whats up
Test Plan: phd start, stop, reload all seemed to work just fine.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T1284
Differential Revision: https://secure.phabricator.com/D2808
Summary:
"differential.creatediff" requires a mostly-parsed diff, but there's no reason we can't make DifferentialDiffs out of raw diffs.
This mainly serves to lower the adoption barrier if getting "arc" distributed is too much of a hassle.
Test Plan: Made a diff out of a raw block of diff text.
Reviewers: ffx, vrana, btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D2751
Summary: D2230 did the heavy lifting spoken of in the comment this diff deletes. Also remove 'macro' => false.
Test Plan: Added an image macro to a wiki document and it showed up
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T1023
Differential Revision: https://secure.phabricator.com/D2668
Summary: This prevents "8 active days" for last week.
Test Plan: `strtotime('1 week ago 24:00')`
Reviewers: john, epriestley
Reviewed By: epriestley
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D2794
Summary: Hopefully this is helpful? Also fixed a thing that wasn't using config.
Test Plan: Read documentation. Sent myself a notification over the server.
Reviewers: jungejason
Reviewed By: jungejason
CC: aran
Maniphest Tasks: T944
Differential Revision: https://secure.phabricator.com/D2804
Summary: The current behavior is that, when the dropdown menu is clicked for a binary file, it navigates to the current revision page directly without showing the dropdown meu. The fix is to do nothing when there is no file displayed.
Test Plan:
- checked a diff with binary file
- checked a diff with normal files
Reviewers: vrana, epriestley
Reviewed By: epriestley
CC: epriestley, aran
Differential Revision: https://secure.phabricator.com/D2801
Summary:
The autoclose logic is currently doing a little too much work. We want to parse each commit at most twice:
# When it first appears in the repository.
# When it first appears on an autoclose branch.
These two events might not be distinct (i.e., it might first appear on an autoclose branch).
Currently, to discover commits initially appearing on autoclose branches, we check each branch, determine if it's an autoclose branch or not, and determine if the HEAD is already a known commit on an autoclose branch. This is correct so far, and allows us to ignore branches which either haven't changed or have commits at HEAD which we've already examined.
However, if an autoclose branch has a new commit, we start working backward through it. Prior to this patch, we only stop when we hit commits that we've already discovered lie on this branch. If the branch is new, none of the commits will be discovered on it (they're discovered in general, and likely discovered on other autoclose branches, but not discovered on this branch), so we'll parse all the way back to the root.
Instead, we want to stop when we hit commits that we've already discovered on //any// autoclose branch.
So do that.
Test Plan: Pushed a new branch, then pushed a new commit at HEAD. Ran discovery, verified we rediscovered only 1 commit, not every commit in history.
Reviewers: vrana, jungejason, aurelijus
Reviewed By: vrana
CC: aran
Maniphest Tasks: T1389
Differential Revision: https://secure.phabricator.com/D2798
Summary:
Support the `--verbose` flag added in D2795 in `phd`. See T1389.
Also simplify argument generation a little bit.
Test Plan: Ran "nice" daemon with debug, daemon + verbose, daemon + no verbose.
Reviewers: vrana, jungejason, edward, aurelijus
Reviewed By: aurelijus
CC: aran
Maniphest Tasks: T1389
Differential Revision: https://secure.phabricator.com/D2797
Summary:
Array-like fields in HerladCommitAdapter should always return things of
type array. If they return array(...) or null, then array_*() functions
sporadically break.
Test Plan:
Ran a PhabricatorRepositoryCommitHeraldWorker for a troublesome commit.
Failed before; fails no more.
Reviewers: epriestley, jungejason
Reviewed By: epriestley
CC: aran, Korvin, vdt
Maniphest Tasks: T1385
Differential Revision: https://secure.phabricator.com/D2793
