1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-24 15:52:41 +01:00
Commit graph

7374 commits

Author SHA1 Message Date
epriestley
e6f6a58f93 Move activity log rendering to a dashboard panel
Summary:
Ref T4986. Swap this in. Two minor notes:

  - I adjusted the SearchEngine to add an additional constraint when the viewer isn't an admin. This mostly stops us from doing a bunch of unnecessary work.
  - I fixed the settings panel to paginate (currently loads all results, slow in production).

Test Plan: Viewed logs; viewed settings panel; created a dashboard panel.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9136
2014-05-15 19:17:02 -07:00
Bob Trahan
6300955661 Dashboards - add layout mode to dashboards
Summary:
This gets us the ability to specify a "layout mode" and which column a panel should appear in at panel add time. Changing the layout mode from a multi column view to a single column view or vice versa will reset all panels to the left most column.

You can also drag and drop where columns appear via the "arrange" mode.

We also have a new dashboard create flow. Create dashboard -> arrange mode. (As opposed to view mode.) This could all possibly use massaging.

Fixes T4996.

Test Plan:
made a dashboard with panels in multiple columns. verified correct widths for various layout modes

re-arranged collumns like whoa.

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4996

Differential Revision: https://secure.phabricator.com/D9031
2014-05-15 19:12:40 -07:00
Chad Little
9d5f9d5c2c Add more information to Workboard Cards
Summary: Adds the taskid/objname and removes the white-space restrictions. Seems still decent.

Test Plan:
Test long and short title cards, test mobile and desktop

{F155552}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4863

Differential Revision: https://secure.phabricator.com/D9148
2014-05-15 18:21:13 -07:00
epriestley
702f073f0a Fix some logic in WaitForPreviousBuildStep
Summary: Fixes T5062. See inlines.

Test Plan: Did not test whatsoever.

Reviewers: hach-que

Reviewed By: hach-que

Subscribers: epriestley

Maniphest Tasks: T5062

Differential Revision: https://secure.phabricator.com/D9132
2014-05-15 07:36:44 -07:00
Aviv Eyal
f2c0e94ea8 Show command transactions in Harbormaster builds
Summary:
Create transaction, editor, etc, and move command generation over to editor.
Show in a timeline in the buildable page.

Also prevent Engine from creating an empty transaction when build starts (Fixes T4885).

Fixes T4886.

Test Plan: Restart builds and buildables, look at timeline.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4885, T4886

Differential Revision: https://secure.phabricator.com/D9110
2014-05-15 07:04:34 -07:00
epriestley
7f22958a82 Use font icons on user approval queue
Summary: Fixes T5066.

Test Plan: {F155058}

Reviewers: chad, btrahan, shadowhand

Reviewed By: shadowhand

Subscribers: shadowhand, epriestley

Maniphest Tasks: T5066

Differential Revision: https://secure.phabricator.com/D9133
2014-05-15 06:51:01 -07:00
epriestley
f2d72ced4b Fix a missed call to setCommit() on DiffusionRequest
See: <https://github.com/facebook/phabricator/issues/608>

Test Plan: Viewed a Subversion diff. Grepped for `setCommit()` again.

Auditors: btrahan
2014-05-15 04:45:53 -07:00
epriestley
15561a27c3 When a conduit method requires a string constant, call it "string-const" not "enum"
Summary: Ref T5058. The use of "enum" is confusing; we mean "choose one of these specific string constants". Make this more clear.

Test Plan: Viewed each call from the web UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5058

Differential Revision: https://secure.phabricator.com/D9127
2014-05-14 21:59:03 -07:00
epriestley
0ad0669916 Allow branch deletions to be pushed in Mercurial
Summary: Fixes T5050. This might not be 100% right in all edge cases, but it worked on everything I tried.

Test Plan:
  - Pushed a branch deletion.
  - Pushed a branch creation.
  - Pushed a brnach creation + deletion.
  - Pushed a brnach deletion + creation.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5050

Differential Revision: https://secure.phabricator.com/D9122
2014-05-14 16:59:15 -07:00
epriestley
fe0c98facc Pass correct number of parameters to pht() when closing a branch in Mercurial
Summary: Ref T5050. This fixes the immediate error (bad pht()) but doesn't fix the other error (can't `--close-branch`) yet.

Test Plan: Pushed a `--close-branch` commit, got a first-level error instead of an error about an error.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5050

Differential Revision: https://secure.phabricator.com/D9119
2014-05-14 16:59:02 -07:00
epriestley
e841270677 Fix "Recently Closed" column for custom task statuses
Summary: Fixes T5056.

Test Plan: Viewed locally, saw numbers.

Reviewers: btrahan, shadowhand

Reviewed By: shadowhand

Subscribers: shadowhand, epriestley

Maniphest Tasks: T5056

Differential Revision: https://secure.phabricator.com/D9126
2014-05-14 16:49:50 -07:00
Eric Stern
358b59ffa7 If PhabricatorIRCBot encounters a 'nick used' error, try adding a number
Summary: Someone stole my bot's name, so the bot couldn't (re)connect. This tries adding some text to the name if it encounters a 433 'nick in use' error

Test Plan: Started a ton of bots: F154744

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9123
2014-05-14 15:30:08 -07:00
Chad Little
6a45b7e670 Update FontAwesome to 4.1.0
Summary: 71 new icons! Pied Piper!

Test Plan: tested new icons on UIExample. Perused a few other pages in Maniphest, Differential. No issues noted.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9125
2014-05-14 14:44:56 -07:00
Bob Trahan
67a5dfa662 Notifications - hide header dropdown if app is uninstalled
Summary: Fixes T5019.

Test Plan: uninstalled notifications and dropdown disappeared too

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5019

Differential Revision: https://secure.phabricator.com/D9121
2014-05-14 10:08:07 -07:00
Bob Trahan
53e95d37cf Calendar - make calendar publicly accessible
Summary: yay public policy. Ref T4830.

Test Plan: browsed about calendar from a logged out browser

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4830

Differential Revision: https://secure.phabricator.com/D9120
2014-05-14 10:00:46 -07:00
epriestley
d111536ae2 Provide more context on what "Lock Permanently" means in the UI
Summary:
Administrators may encounter this UI for the first time when configuring a repository, without ever having used Passphrase before. If they end up here on a workflow like this, the meaning of "Lock Permanently" may not be totally clear. Add some explanatory text.

(See also T4721, which discusses Passphrase documentation more generally.)

Test Plan: {F154209}

Reviewers: lpriestley

Reviewed By: lpriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9102
2014-05-14 09:25:58 -07:00
Bob Trahan
7ab26aef66 Internet Explorer compatibility fixes
Summary:
A few fixes for IE that makes this work in IE9+ and at least you can drag things and have it fail horribly in IE8 now. Ref T5023.

 - try / catch around getBoundingClientRect because IE throws if the object is not attached to the DOM. The correct value can be inferred in these cases as 0,0
 - turns out IE has different values for button here (1,2,4) than other browsers. special case this by using the other IE fact that 'which' will be null

Test Plan: dragged and dropped tasks successfully in IE9, IE10, and IE11 on Windows 7. dragged and dropped tasks unsuccessfully in IE8 on Windows 7 - they always "drop" to the 2nd position or so as I think the ondrag code isn't firing properly or something

Reviewers: chad, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5023

Differential Revision: https://secure.phabricator.com/D9118
2014-05-14 08:53:11 -07:00
Ben Alpert
348ca4207c Use "added a dependency" over "added a revision"
Test Plan: Added a dependency, saw a descriptive description.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9111
2014-05-13 19:53:26 -07:00
epriestley
26b985006d Git did not 3-way merge this quite correctly. 2014-05-13 14:09:00 -07:00
epriestley
436f0563e8 Add a SublimeText-style repository typeahead
Summary:
Allows you to quickly search for files within a repository. Roughly:

  - We build a big tree of everything and ship it to the client.
  - The client implements a bunch of Sublime-ish magic to find paths.

Test Plan: {F154007}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley, zeeg

Differential Revision: https://secure.phabricator.com/D9087
2014-05-13 14:08:21 -07:00
epriestley
82102cd95a Move Push log rendering to SearchEngine
Summary: Ref T4986. Move push logs to a View, then have all the stuff that needs to use it use that View.

Test Plan: Viewed push logs and transaction detail in Diffusion. Created a panel.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9104
2014-05-13 14:00:24 -07:00
epriestley
23ada21d35 Remove commit from DiffusionRequest
Summary: Ref T2683. This field is //almost// entirely redundant with `symbolicCommit`. Improve how some of the diff query stuff works a bit, then remove it.

Test Plan: Browsed around in all interfaces, looked at a bunch of diffs, etc.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9099
2014-05-13 13:53:06 -07:00
epriestley
347252fda8 Improve clarity of commit and symbol handling in DiffusionRequest
Summary:
Ref T2683. Currently, DiffusionRequest has four different "commitey" things:

  - `commit`
  - `rawCommit`
  - `symbolicCommit`
  - `stableCommit`

Of these, only two are actually distinct, useful values: `symbolicCommit` (which holds the value the request originally contained, if one existed) and `stableCommit` (which resolves that value, or the value implied by its omission, into a stable, permanent commit identifier).

  - `rawCommit` is equivalent to `symbolicCommit` and can be simply removed.
  - `commit` has some sketchy magic around it that needs to be pulled out before it can be jettisoned.

Test Plan: Viewed SVN, Git, and Mercurial repositories. Viewed brwose/history/change/tag/branch/etc views.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9098
2014-05-13 13:52:48 -07:00
epriestley
b80b851600 Throw a more tailored exception after failing to resolve a ref
Summary: Ref T2683. Throw a more tailored exception to allow callers to distinguish between bad refs (which are expected, if users try to visit garbage branches) and other types of errors.

Test Plan: Tried to view branch "alksndfklansdf". Viewed branch "master".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9094
2014-05-13 13:52:33 -07:00
epriestley
ce3f9211e4 Let diffusion.readmequery accept a commit
Summary:
Ref T2683. This should probably just be `diffusion.filecontentquery` but keep things as they are for now.

This method uses a commit, so accept one. Soon, this will save a bit of work.

Test Plan: Viewed readmes in main and browse views.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9093
2014-05-13 13:52:20 -07:00
epriestley
112c9e6b5e Rename "commitType" to "symbolicType"
Summary:
Ref T2683. The old name was a bit confusing because it meant "the type of the thing the symbol represents": a "commit type" should logically always be "commit".

(Currently, this is only used to detect when we're looking at a tag.)

Test Plan: Looked at a tag. Looked at some other non-tag things. Browsed around, `grep`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9092
2014-05-13 13:52:03 -07:00
epriestley
9a2c68fd88 Rename "stableCommitName" to "stableCommit"
Summary:
Ref T2683. This is closely related to "symbolicCommit", but has an inconsistent "name" on the end.

Also, `diffusion.searchquery` uses this parameter inconsistently.

Test Plan:
  - `grep`ed for callsites.
  - Ran searches in Git and Mercurial repositories.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9091
2014-05-13 13:51:45 -07:00
epriestley
7cc9720d60 Remove shouldCreateDiffusionRequest from Diffusion conduit methods
Summary:
Ref T2683. This has no callsites, and the functionality is covered by the `initFromConduit` flag.

This simplifies the code and reduces then number of internal `diffusion.resolverefs` calls we make on, e.g., the Git repository page from 7 to 2.

Test Plan: Grepped for these symbols.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9090
2014-05-13 13:51:33 -07:00
William R. Otte
0ab192d245 Add documentation about the script and regex linter to the user guide.
Summary:
The big, gigantic comment about the script and regex linter belongs in a more obvious place.  I think this is a more obvious place. I also cleaned up a couple things.

I'll update D9084 to remove the big comment block and point here instead.

Test Plan: `bin/diviner generate --book src/docs/book/user.book`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9100
2014-05-13 13:51:01 -07:00
Lex
cff721c657 s/Debate Paste Accuracy/Eat Paste/
Summary: Don't get me wrong, "Debate Paste Accuracy" is witty, but we can aspire to better.

Test Plan: Consume glue, obviously.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, chrisbolt, Korvin

Differential Revision: https://secure.phabricator.com/D9106
2014-05-13 12:19:42 -07:00
epriestley
38f5894b58 Move Drydock object list rendering to SearchEngine
Summary: Ref T4986. Allows the Drydock search engines to render as panels.

Test Plan: Viewed affected interfaces in Drydock. Created panels from each engine.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9103
2014-05-13 12:14:33 -07:00
epriestley
4a7499f230 Fix credential upgrade issue after introduction of isLocked column
Summary: Fixes T5035. This migration isn't forward compatible after schema mutation.

Test Plan: Ran locally, will get reporting user to confirm.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: gera, epriestley

Maniphest Tasks: T5035

Differential Revision: https://secure.phabricator.com/D9101
2014-05-13 12:14:27 -07:00
epriestley
2592bfcbb3 Add an "authored commits" prebuilt filter to Audit
Summary: Add a prebuilt filter to show all of the viewer's commits across all repositories. I could go either way on this, but it seems maybe-useful (?), and we have similar prebuilt filters elsewhere.

Test Plan: scoped it out <.< >.>

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8881
2014-05-13 08:06:22 -07:00
Chad Little
0120388a75 Found some missing icons
Summary: Did a more exhaustive grep on setIcon and found 99.9% of the icons.

Test Plan: I verified icon names on UIExamples, but unable to test some of the more complex flows visually. Mostly a read and replace.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9088
2014-05-13 07:45:39 -07:00
Chad Little
5b31a17c96 Fix mobile menus
Summary: The multiple icons are causing layout issues. This hides the unsused div (the real icon gets appended). We should probably clean this up next redesign.

Test Plan: test mobile layouts in Chrome, test main menu and application menu. Go to Differential, check those menus to.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9086
2014-05-12 20:11:45 -07:00
epriestley
77b4c3145a Simplify Diffusion main view
Summary:
Currently, Diffusion has very complex views. After three years I'm not really used to them and rarely use many of these options.

Simplify the browse and history views:

  - Put the browse view on top.
  - Move dates to the right.
  - Remove "History" and "Edit" links from the browse view. You can access these actions by clicking the file/path.
  - Remove "Browse" link from the history view. You can access this action by clicking the commit.
  - Remove "Change Type", which is essentially never useful, from the history view.
  - Add some tweaks for mobile.

Test Plan: {F153931}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley, zeeg

Differential Revision: https://secure.phabricator.com/D9085
2014-05-12 19:57:12 -07:00
epriestley
91eb751d6f Use k in map, not array.indexOf, in Event.js
Summary: See T5023.

Test Plan: Dragged stuff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5023

Differential Revision: https://secure.phabricator.com/D9082
2014-05-12 14:29:58 -07:00
Chad Little
315ac6aab7 Remove white app icons
Summary: Remove white app icons, no longer in use as far as grep/memory serve. These were for list hover states.

Test Plan: Rebuild sprites, celerity. Grep for appIcon use (only feed). Verify all action lists are driven by FontAwesome.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9078
2014-05-12 13:34:00 -07:00
epriestley
f170991897 Put an in-request cache in front of APC
Summary: Most requests examine the same buckets, especially the first bucket. Let them just read it out of request cache.

Test Plan: Observed most bucket fetches resolving in <10us instead of <10ms.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9080
2014-05-12 12:34:04 -07:00
epriestley
c88f5f0759 Vegetables have vitamins. 2014-05-12 11:53:36 -07:00
epriestley
84fc8f0baf Don't try to rebuild a repository graph cache bucket more than once per request
Summary: Ref T2683. This is a small optimization, but it has low complexity: don't rebuild a bucket more than once in the same request, since it will almost always be the same. Bucket rebuilds are pretty cheap, but this saves a few queries.

Test Plan:
  - After discovering (but before parsing) a commit, viewed its browse view. Verified that this patch causes us to perform only one bucket rebuild, and therefore reduces the number of queries we issue.
  - Parsed the commit and viewed the browse view again, got successful rebuild and then fills from cache.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9055
2014-05-12 11:47:32 -07:00
epriestley
cfa265f020 Make sure READMEs can hit the markup cache in Diffusion
Summary: Ref T2683. Normally not a big deal, but if a readme has some codeblocks missing the cache can slow things down.

Test Plan:
  - Verified we hit the cache.
  - Verified TOC still works.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5028, T2683

Differential Revision: https://secure.phabricator.com/D9049
2014-05-12 11:47:31 -07:00
epriestley
53e9df8a02 Slightly reduce the cost of resolving refs
Summary: Ref T2683. By resolving the stable name earlier, we can save a resolve when viewing branch heads. This is ~100ms in Mercurial, and roughly 25% of page weight. It's less bad in Git.

Test Plan: Saw page cost go down in "Services" tab, particularly for Mercurial browse views.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9048
2014-05-12 11:47:30 -07:00
epriestley
e13369d208 Use RepositoryGraphCache to service diffusion.lastmodifiedquery
Summary:
Ref T2683. At least locally, browse views are now nearly instantaneous, even in Mercurial. We also fall back to what we were doing before if we miss or take too long, so this shouldn't make things very much worse even in extreme cases.

For a local `hg` repo, the time we spend pulling browse stuff has dropped from ~3,000ms to ~20ms. This is probably atypical, but not completely crazy or rigged or anything.

Test Plan: Viewed Git, Subversion and Mercurial repositories and observed dramatically better performance in Git and Mercurial as they took advantage of the cache.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, jhurwitz

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9047
2014-05-12 11:47:29 -07:00
epriestley
e34ee684e1 Batch execution of LastModified query
Summary:
Ref T2683. Further reduces query count of last modified loads; we're now at 11 instead of 200+.

(This works in SVN but could be further optimized.)

Test Plan:
Loaded SVN, Mercurial, Git:

{F34864}
{F34865}
{F34866}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, vrana, aran

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D5256
2014-05-12 11:47:28 -07:00
epriestley
e03deb7d4a Always pull extra browse information over Ajax, and batch some of the queries
Summary:
This code is currently quite complicated because we pull history data inline for SVN files, and via ajax for everything else (SVN dirs, everything in Git and Hg).

Always pull over ajax; batch some of the queries.

Test Plan: {F34860}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, vrana, aran

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D5255
2014-05-12 11:47:27 -07:00
epriestley
df59f4b047 Batch all supplementary information in Diffusion browse views
Summary: Ref T2683. Instead of sending one request for each path's history, send one request for all of it. This permits optimizations which are not currently available to us. It degrades the user experience a tiny bit in theory, but on my machine it's actually way faster already.

Test Plan: Loaded a browse page.

Reviewers: vrana, btrahan

Reviewed By: btrahan

Subscribers: epriestley, aran

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D5254
2014-05-12 11:47:26 -07:00
epriestley
ac020bc420 Implement a lint count query
Summary: Ref T2683. This query is currently unbatched and happens inside a view. Leave it inside the view for now, but separate it and make it batchable.

Test Plan: {F34848}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, vrana, aran

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D5252
2014-05-12 11:47:25 -07:00
epriestley
e4ea092f60 Implement a chunked, APC-backed graph cache
Summary:
Ref T2683. This is a refinement and simplification of D5257. In particular:

  - D5257 only cached the commit chain, not path changes. This meant that we had to go issue an awkward query (which was slow on Facebook's install) periodically while reading the cache. This was reasonable locally but killed performance at FB scale. Instead, we can include path information in the cache. It is very rare that this is large except in Subversion, and we do not need to use this cache in Subversion. In other VCSes, the scale of this data is quite small (a handful of bytes per commit on average).
  - D5257 required a large, slow offline computation step. This relies on D9044 to populate parent data so we can build the cache online at will, and let it expire with normal LRU/LFU/whatever semantics. We need this parent data for other reasons anyway.
  - D5257 separated graph chunks per-repository. This change assumes we'll be able to pull stuff from APC most of the time and that the cost of switching chunks is not very large, so we can just build one chunk cache across all repositories. This allows the cache to be simpler.
  - D5257 needed an offline cache, and used a unique cache structure. Since this one can be built online it can mostly use normal cache code.
  - This also supports online appends to the cache.
  - Finally, this has a timeout to guarantee a ceiling on the worst case: the worst case is something like a query for a file that has never existed, in a repository which receives exactly 1 commit every time other repositories receive 4095 commits, on a cold cache. If we hit cases like this we can bail after warming the cache up a bit and fall back to asking the VCS for an answer.

This cache isn't perfect, but I believe it will give us substantial gains in the average case. It can often satisfy "average-looking" queries in 4-8ms, and pathological-ish queries in 20ms on my machine; `hg` usually can't even start up in less than 100ms. The major thing that's attractive about this approach is that it does not require anything external or complicated, and will "just work", even producing reasonble improvements for users without APC.

In followups, I'll modify queries to use this cache and see if it holds up in more realistic workloads.

Test Plan:
  - Used `bin/repository cache` to examine the behavior of this cache.
  - Did some profiling/testing from the web UI using `debug.php`.
  - This //appears// to provide a reasonable fast way to issue this query very quickly in the average case, without the various issues that plagued D5257.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, jhurwitz

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9045
2014-05-12 11:47:23 -07:00
epriestley
95eab2f3b0 Record parent relationships when discovering commits
Summary:
Ref T4455. This adds a `repository_parents` table which stores `<childCommitID, parentCommitID>` relationships.

For new commits, it is populated when commits are discovered.

For older commits, there's a `bin/repository parents` script to rebuild the data.

Right now, there's no UI suggestion that you should run the script. I haven't come up with a super clean way to do this, and this table will only improve performance for now, so it's not important that we get everyone to run the script right away. I'm just leaving it for the moment, and we can figure out how to tell admins to run it later.

The ultimate goal is to solve T2683, but solving T4455 gets us some stuff anyway (for example, we can serve `diffusion.commitparentsquery` faster out of this cache).

Test Plan:
  - Used `bin/repository discover` to discover new commits in Git, SVN and Mercurial repositories.
  - Used `bin/repository parents` to rebuild Git and Mercurial repositories (SVN repos just exit with a message).
  - Verified that the table appears to be sensible.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley

Maniphest Tasks: T4455

Differential Revision: https://secure.phabricator.com/D9044
2014-05-12 11:47:22 -07:00