1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-26 23:40:57 +01:00
Commit graph

400 commits

Author SHA1 Message Date
Jakub Vrana
a9099912bb Handle SVN root changes correctly
Summary: If there is a change in SVN root (perhaps properties change) then we try to list its parent (which doesn't exist) and mark the root itself as deleted.

Test Plan: Parsed SVN commit with property change of root.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5709
2013-04-16 08:28:38 -07:00
Jakub Vrana
e31e998f3b Convert differential.revisionPHID commit detail to edge
Summary: Migration doesn't delete differential.revisionPHID but maybe it should?

Test Plan: Reparsed commit, ran the migration, deleted differential.revisionPHID, looked at task with attached commit with attached revision.

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5634
2013-04-12 22:48:16 -07:00
epriestley
855e085c6f Uninstall Conduit calls when uninstalling applications
Summary: Fixes T2698. When applications are installed, their Conduit calls should drop out. This will also let us land Releeph without exposing Conduit calls.

Test Plan:
  - Viewed Conduit console; uninstalled some applications and verified their calls dropped out.
  - Tried to make an uninstalled call; got an appropriate error.

Reviewers: edward, btrahan

Reviewed By: edward

CC: aran

Maniphest Tasks: T2698

Differential Revision: https://secure.phabricator.com/D5302
2013-03-13 07:09:05 -07:00
James Rhodes
868e79b7a7 Implemented option to allow users to view username of repository URIs.
Test Plan:
Enable the checkbox under the Tracking options for a repository.
View the repository in Diffusion, the username should show.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2575

Differential Revision: https://secure.phabricator.com/D5269
2013-03-07 08:06:09 -08:00
vrana
b3a63a62a2 Introduce PhabricatorEmptyQueryException
Summary: It's dumb to execute a query which we know will return an empty result.

Test Plan: Looked at comment preview with "11", didn't see "1 = 0" in DarkConsole.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5177
2013-03-06 19:22:00 -08:00
epriestley
0a069cb55a Require a viewer to load handles
Summary:
Unmuck almost all of the we-sort-of-have-viewers-some-of-the-time mess.

There are a few notable cases here:

  - I used Omnipotent users when indexing objects for search. I think this is correct; we do policy filtering when showing results.
  - I cheated in a bad way in the Remarkup object rule, but fixing this requires fixing all the PhabricatorRemarkupEngine callsites (there are 85). I'll do that in the next diff.
  - I cheated in a few random places, like when sending mail about package edits. These aren't a big deal.

Test Plan:
  - Grepped for all PhabricatorObjectHandleData references.
  - Gave them viewers.

Reviewers: vrana

Reviewed By: vrana

CC: aran, edward

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D5151
2013-02-28 17:15:09 -08:00
epriestley
e3567b2d4d Fix grossness in Herald daemon
Summary: Do this in a reasonably proper / formal way. Leave a TODO for the policy stuff.

Test Plan: Ran `scripts/repository/reparse.php --herald rPnnnnn`

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D5150
2013-02-28 16:38:40 -08:00
epriestley
3f547d562d Give the fake Herald user a PHID so it counts as logged in and can see commits
See last two commits in this chain.

Auditors: vrana
2013-02-27 11:54:17 -08:00
epriestley
69dcd47751 Fix two cases where we load Commit handles without a viewer
I missed these in testing D5139 because they aren't in the web UIs and I am dumb and can not brain today.

Auditors: vrana
2013-02-27 11:44:52 -08:00
epriestley
fe78944c9d Prepare Diffusion for hovercards
Summary:
Move Diffusion to be hovercard-ready, and expand our ability to resolve commit references.

  - Link unqualified hashes of 7 characters or more which match a commit.
  - Link qualified hashes of 5 characters or more which match a commit.
  - Support `{...}` syntax.

Test Plan: {F33896}

Reviewers: chad, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5121
2013-02-27 08:04:54 -08:00
epriestley
2b10173a03 Don't retain "\r\n" when summarizing commits
Auditors: vrana
2013-02-21 15:12:54 -08:00
epriestley
b32bfb6541 Render commit summaries when rendering handles
Summary:
Fixes T2563. Instead of rendering "rPnnnnnn", render "rPnnnnnn: add feature X". Tweak Audit tables to accommodate.

@vrana / @nh, this migration might take a while. You could safely skip it when deploying and then run it after deployment.

I think I fixed all the other places where these render, but might have missed something.

Test Plan:
  - Ran first schema migration, clicked around to make sure nothing broke.
  - Ran `scripts/repository/reparse.php --message rXyyyyy`, verified summary populated.
  - Ran second migration.
  - Checked task/diffusion/audit/differential for weird rendering.

Reviewers: vrana

Reviewed By: vrana

CC: nh, aran, chrisbolt, allixsenos

Maniphest Tasks: T2563

Differential Revision: https://secure.phabricator.com/D5012
2013-02-21 15:09:35 -08:00
epriestley
a22bea2a74 Apply lint rules to Phabricator
Summary: Mostly applies a new call spacing rule; also a few things that have slipped through via pull requests and such

Test Plan: `find src/ -type f -name '*.php' | xargs -n16 arc lint --output summary --apply-patches`

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D5002
2013-02-19 13:33:10 -08:00
epriestley
ef7f16180c Restore merge of phutil_tag. 2013-02-13 14:51:18 -08:00
epriestley
73cce6e131 Revert "Promote phutil-tag again"
This reverts commit 8fbabdc06d, reversing
changes made to 2dab1c1e42.
2013-02-13 14:08:57 -08:00
vrana
9b8da73765 Convert AphrontTableView to safe HTML
Summary:
Lots of killed `phutil_escape_html()`.

Done by searching for `AphrontTableView` and then `$rows` (usually) backwards.

Test Plan:
Looked at homepage.

  echo id(new AphrontTableView(array(array('<'))))->render();

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4884
2013-02-09 15:11:38 -08:00
vrana
d817dfa8fc Convert some phutil_escape_html() to hsprintf()
Summary: Found by `sgrep_php -e '"...".phutil_escape_html(...)'`.

Test Plan:
/
/D1
/uiexample/
/countdown/1/
/herald/transcript/1/all/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4869
2013-02-08 15:59:02 -08:00
vrana
afc5333bb3 Convert AphrontFormView to safe HTML
Summary: Searched for `AphrontFormView` and then for `appendChild()`.

Test Plan: /login/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4855
2013-02-07 18:01:00 -08:00
vrana
85961c8eca Convert AphrontFormInsetView to safe HTML
Summary: Done by searching for `AphrontFormInsetView` and verifying all method calls.

Test Plan: /repository/edit/1/tracking/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4845
2013-02-07 18:00:59 -08:00
vrana
059920c2da Convert AphrontErrorView to safe HTML
Summary: Done by searching for `AphrontErrorView` and then `appendChild()`.

Test Plan:
Looked at Commit Detail.
Looked at Revision Detail.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4843
2013-02-07 17:26:01 -08:00
vrana
f864d9e611 Fix double escaping in phutil_tag
Summary:
I wasn't able to reproduce the "recursion detected" in real web request but I saw lots of 1073741824 refcounts in `debug_zval_dump()` of $object.
I'm not sure how that happens.

Test Plan: D4807#4

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4839
2013-02-06 15:21:05 -08:00
vrana
be4662e667 Convert setCaption() to safe HTML
Test Plan: /settings/panel/display/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4824
2013-02-05 15:52:43 -08:00
epriestley
ebc2937dc8 render_tag -> tag: final block
Summary:
Convert a final few `render_tag()` calls to `tag()` calls. This leaves us with 36 calls:

  - 9 are in Conpherence, and will be converted after merging. Using Views makes the most sense here, to get access to renderHTMLView() and such.
  - 2 are the definition and its library map entry.
  - 3 are in the documentation.
  - I believe the remaining 22 are too difficult to convert pre-merge. About half are in code which is slated for destruction; the other half are in the base implementations of very common classes (like PhabricatorStandardPageView) and can only be converted by converting the entire codebase.

My plan at this point is:

  - Test the branch thoroughly.
  - Merge to master.
  - Over time, resolve the remaining issues: lint means things shouldn't get any worse.

Test Plan: See inlines.

Reviewers: vrana

Reviewed By: vrana

CC: aran, edward

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4802
2013-02-04 11:38:04 -08:00
epriestley
0f1bdbe147 Merge branch 'master' into phutil_tag
(Sync.)
2013-02-04 06:19:52 -08:00
epriestley
363f292fd8 Partially revert marking of copies as direct
Summary:
The other `true` is correct (and I think we can fix the scaling issues) but this one should be an indirect change. This prevents the branch from appearing in the history of every file.

(I misread this code and gave @vrana some bad advice originally. This is //actually// consistent with Mercurial and Git.)

Test Plan: Partial revert. I'll make this stuff testable.

Reviewers: nh, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4742
2013-01-30 12:35:37 -08:00
epriestley
3093d1663d Add javelin_tag(), convert easy callsites
Summary:
  - Implements `javelin_tag()`, which is `javelin_render_tag()` on top of `phutil_tag()` instead of `phutil_render_tag()`.
  - Manually converts all or almost all of the trivial callsites.

Test Plan:
  - Site does not seem any more broken than before.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4639
2013-01-25 12:57:17 -08:00
epriestley
3e147dd61c Fix some easy phutil_render_tag()
Summary:
  - Grepped for phutil_render_tag().
  - Fixed some easy ones.

Test Plan:
  - Browsed around; site didn't seem more broken than it was before.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4638
2013-01-25 11:39:23 -08:00
vrana
20768d65d5 Convert phutil_render_tag(X, Y, '...') to phutil_tag
Summary:
Created with spatch:

  lang=diff
  - phutil_render_tag
  + phutil_tag
    (X, Y, '...')

Then searched for `&` and `<` in the output and replaced them.

Test Plan: Loaded homepage.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4503
2013-01-24 19:20:27 -08:00
epriestley
2584af2cb8 Clean up repositories with symbols correctly
Summary: We currently try to delete symbols by ID, but the table has a multipart primary key and no `id` column.

Test Plan: Ran query locally; had @JThramer verify fix in his environment.

Reviewers: btrahan

Reviewed By: btrahan

CC: JThramer, aran

Differential Revision: https://secure.phabricator.com/D4626
2013-01-24 17:48:41 -08:00
vrana
760ab135eb Delete license header 2013-01-18 14:45:58 -08:00
epriestley
0e612c910b Sort repositories in Diffusion by name, not creation order
Summary: Ref T2298. This seems like the least complicated reasonable order to implement.

Test Plan: Looked at repositories, saw them ordered by name.

Reviewers: vrana, btrahan, brennantaylor

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2298

Differential Revision: https://secure.phabricator.com/D4395
2013-01-16 10:51:08 -08:00
Ricky Elrod
800f62d0d5 Repository option.
Test Plan: Looked at the option.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4468
2013-01-16 09:35:48 -08:00
Chad Little
fb9a1c38fb Panel background changes for Repositories app.
Summary: Changed some background panels.

Test Plan: Reload page.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4458
2013-01-16 07:49:24 -08:00
Michael Halstead
21ed25019d Prevent exceptions when adding repositories via conduit.
Summary: Simple change to stop exceptions from being thrown based on the diff in D4146.

Test Plan: arc call-conduit repository.create

Reviewers: epriestley, btrahan

Reviewed By: btrahan

CC: vrana, aran, Korvin

Maniphest Tasks: T2268

Differential Revision: https://secure.phabricator.com/D4342
2013-01-10 15:37:56 -08:00
vrana
0f1086fd68 Mark SVN dir copy as direct change for its files
Test Plan: HHVM currently core dumps so I didn't test it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4343
2013-01-07 13:46:26 -08:00
epriestley
f6b1964740 Improve Search architecture
Summary:
The search indexing API has several problems right now:

  - Always runs in-process.
    - It would be nice to push this into the task queue for performance. However, the API currently passses an object all the way through (and some indexers depend on preloaded object attributes), so it can't be dumped into the task queue at any stage since we can't serialize it.
    - Being able to use the task queue will also make rebuilding indexes faster.
    - Instead, make the API phid-oriented.
  - No uniform indexing API.
    - Each "Editor" currently calls SomeCustomIndexer::indexThing(). This won't work with AbstractTransactions. The API is also just weird.
    - Instead, provide a uniform API.
  - No uniform CLI.
    - We have `scripts/search/reindex_everything.php`, but it doesn't actually index everything. Each new document type needs to be separately added to it, leading to stuff like D3839. Third-party applications can't provide indexers.
    - Instead, let indexers expose documents for indexing.
  - Not application-oriented.
    - All the indexers live in search/ right now, which isn't the right organization in an application-orietned view of the world.
    - Instead, move indexers to applications and load them with SymbolLoader.

Test Plan:
  - `bin/search index`
    - Indexed one revision, one task.
    - Indexed `--type TASK`, `--type DREV`, etc., for all types.
    - Indexed `--all`.
  - Added the word "saboteur" to a revision, task, wiki page, and question and then searched for it.
    - Creating users is a pain; searched for a user after indexing.
    - Creating commits is a pain; searched for a commit after indexing.
    - Mocks aren't currently loadable in the result view, so their indexing is moot.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: 20after4, aran

Maniphest Tasks: T1991, T2104

Differential Revision: https://secure.phabricator.com/D4261
2012-12-21 14:21:31 -08:00
vrana
2cc7f82ece Move Conduit methods inside applications
Test Plan:
/conduit/
/conduit/method/arcanist.projectinfo/
Call method

  $ echo '{}' | arc call-conduit user.whoami

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4268
2012-12-21 12:21:59 -08:00
vrana
800be357e8 Order repositories in Diffusion from oldest
Summary: Same to ordering in Repository List.

Test Plan: /diffusion/

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4251
2012-12-20 15:49:37 -08:00
epriestley
db89e23761 Make Repositories partially policy-aware
Summary: Small step toward repository hosting. No user-visible changes.

Test Plan: Looked at repositories in Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D4227
2012-12-19 11:07:06 -08:00
Nick Harper
b8d372c4bc Fix bugs when saving package changes due to commit changes
Summary:
Because of the way PhabricatorOwnersPackage works, the code to save package
changes when parsing commit changes was raising a few undefined index errors,
and was throwing an exception trying to call a method on null (because not
all of the phids related to the package had their handles loaded). This fix
doesn't load the missing handles, it just avoids trying to use them.

Test Plan:
./scripts/repository/reparse.php on a commit with path changes that triggered
a package change

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4236
2012-12-19 10:53:04 -08:00
Nick Harper
4a81ae6d6d Add data information to daemon task view
Summary:
Load the data for daemon worker tasks when viewing them, and present
the information in a useful way. This defaults to printing the json data,
but for some classes of worker it will also link to the corresponding
object, to make debugging problems with workers easier.

Test Plan:
load /daemon/task/NNN for a CommitParserWorker and a MetaMTAWorker, and
see the addition of a data field with useful content and link.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4226
2012-12-17 17:12:55 -08:00
Nick Harper
0ba7d34f97 Allow phabricator to manage repositories
Summary:
This adds a configuration option for repositories to be marked as managed
by phabricator, which is then used by the pullLocal daemon to try to recover
from some errors it runs into.

Test Plan: Set a bogus uri for a repo, saw that the pullLocal daemon fixed it.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3680
2012-12-12 14:43:09 -08:00
vrana
0e53731d1d Expose repository info in arcanist.projectinfo
Summary: This is to reduce number of calls from Arcanist.

Test Plan: Called it from web interface.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4146
2012-12-10 15:19:09 -08:00
Bob Trahan
42a514ec79 make repository tool fail a little less hard if daemons don't interact nicely
Summary: we were catching a specific exception; just catch all exceptions

Test Plan: viewed repository tool home page

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2155

Differential Revision: https://secure.phabricator.com/D4118
2012-12-07 16:12:16 -08:00
epriestley
f306cab653 Use application icons for "Eye" menu and Crumbs
Summary:
Issues here:

  - Need an application-sized "eye", or a "home" icon for "Phabricator Home".
  - Some of the "apps_lb_2x" sliced images are the "_dark_" versions, not the light versions.
  - If you slice an application-sized "logout" (power off) icon and application-sized "help" (questionmark in circle) icon I can replace the current menu icons and nearly get rid of "autosprite".
  - To replace the icons on /applications/, the non-retina size is "4x", so we'd need "8x" for retina. Alternatively I can reduce the icon sizes by 50%.
  - The "Help", "Settings" and "Logout" items currently have a "glowing" hover state, which needs a variant (or we can drop it).
  - The /applications/ icons have a white hover state (or we can drop it).
  - The 1x application (14x14) icons aren't used anywhere right now, should they be? Maybe in the feed in the future, etc?
  - The "apps-2x" and "apps-large" sheets are the same image, but getting them to actually use the same file is a bit tricky, so I just left them separate for now.

Test Plan:
{F26698}
{F26699}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4108
2012-12-07 13:37:28 -08:00
epriestley
20ee3003b5 Replace all instances of AphrontSideNavView with AphrontSideNavFilterView
Summary:
AphrontSideNavView is an old class which required you to do a lot of work; it was obsoleted by AphrontSideNavFilterView. Remove all direct callsites so I can clean it up.

This is a precursor to letting me render a filter menu as a dropdown menu for T1960.

Test Plan:
Examined each interface for correct filter construction and selection:

  - Browsed Diffusion
  - Browsed Differential
  - Browsed Files
  - Browsed Slowvote
  - Browsed Phriction
  - Browsed repo edit interface

Grepped for `AphrontSideNavView`. The only remaining instances are in `AphrontSideNavView` itself and `AphrontSideNavFilterView` (which currently uses it).

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4034
2012-12-07 13:30:31 -08:00
epriestley
1a3bf098ae Correctly parse author and committer identities again
Summary: See D3977, a terrible diff where I made a huge messs.

Test Plan: Ran `reparse.php --message --trace <revision>`, verified correct identification of author.

Reviewers: edward, vrana

Reviewed By: edward

CC: aran

Differential Revision: https://secure.phabricator.com/D4104
2012-12-07 07:06:29 -08:00
vrana
cb45ad67a3 Revive essential commit details
Summary: Deleted by D3977.

Test Plan:
  ./reparse.php --message

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4090
2012-12-06 10:44:16 -08:00
Suman Karumuri
2852b4f5d3 Track path moves in Owner's packages
Summary:
Currently, if a package is moved from one location to another those package paths become stale in Owner's packages.
This diff fixes that by tracking the path moves in diffusion commit messages and updating the owners package paths accordingly.

Test Plan:
Tested the script by running the reparse.php script on the following test commits:
https://phabricator.fb.com/rPHGITd7271a2de212e0b2de33d485b2ff806e75f73d36
https://phabricator.fb.com/rPHGIT0192fb41e7bd3e0faeeb14facdc7f7bea1b716d0

on the following packages:
https://phabricator.fb.com/owners/package/1167/
https://phabricator.fb.com/owners/package/1168/

Reviewers: nh

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D3985
2012-12-05 17:21:45 -08:00
epriestley
fd5beffb99 Remove PhabricatorRepositoryDefaultCommitMessageDetailParser
Summary: See discussion in T1544. This has been obsoleted by simpler/better mechanisms.

Test Plan: Edited a repository; ran a parse task.

Reviewers: edward

Reviewed By: edward

CC: aran

Maniphest Tasks: T1544

Differential Revision: https://secure.phabricator.com/D3977
2012-12-05 15:34:28 -08:00
epriestley
2de879e613 Minor, extend the lease time of Herald task leases. 2012-11-30 11:02:42 -08:00
vrana
846a359d9a Display number of lint messages in Diffusion dir
Test Plan:
Went to https://secure.phabricator.com/diffusion/ARC/browse/.
Saw number of lint messages per dir, not terribly slow.
Went to https://secure.phabricator.com/diffusion/ARC/browse/master/src/difference/.
Saw number of lint messages per file.
Disabled lint, didn't see it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3921
2012-11-08 15:40:48 -08:00
vrana
23a046b3cd Allow saving lint errors to database
Summary: This saves lint errors to the path change of current commit. It requires pushed revision. It doesn't save difference from previous commit mentioned in T2038#comment-4 - I don't plan doing it after all, everything would be much more complicated and the amount of data saved with this approach isn't that bad.

Test Plan: Applied patch, ran script, verified DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3899
2012-11-08 15:39:43 -08:00
vrana
10cf2bb4e2 Mark PhabricatorRepositorySvnCommitChangeParserWorker as final
Test Plan:
  $ git grep PhabricatorRepositorySvnCommitChangeParserWorker

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3897
2012-11-05 23:27:40 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
Bob Trahan
4edf8ae2fc make "browse in diffusion" action work for commits in branches other than master
Summary: we do this by passing the "seenOnBranches" commit data detail through the stack

Test Plan: browse in diffusion link worked for non-master checkins under git

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1949

Differential Revision: https://secure.phabricator.com/D3853
2012-10-31 14:07:25 -07:00
Bob Trahan
5d5f8a7a91 whoops - forgot to remove the third slash on this comment from D3822 2012-10-25 16:38:14 -07:00
Bob Trahan
731a6900bd upgrade repository delete function to full-blown workflow
Summary: fancy title. really just make the delete() method aware of related objects and build a quick workflow which calls delete(). also make commit delete savvy about audit requests.

Test Plan: deleted a repository per the instructions given to me in the web UI

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1416, T1958, T1372

Differential Revision: https://secure.phabricator.com/D3822
2012-10-25 16:23:41 -07:00
Bob Trahan
d9c6e07f2c If users are on the email to Phabricator, do not send them the Phabricator reply.
Summary: When we receive an email, figure out if any of the other tos and ccs are users. If they are, pass their phids through the stach as "exclude phids" and exclude them from getting the email.

Test Plan: used the various applications (audit, differential, maniphest) and noted emails were sent as expected.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin, vrana

Maniphest Tasks: T1676

Differential Revision: https://secure.phabricator.com/D3645
2012-10-10 10:18:23 -07:00
epriestley
cd9d78c107 Allow Fact analysis of commits
Summary: For immutable objects, just use the ID as a cursor.

Test Plan:
  - Analyzed commits from an empty cursor.
  - Checked that cursor was good.
  - Pulled some more commits.
  - Analyzed commits again, verified it only hit the new ones.
  - Verified the graph of "Count of CMIT" looked reasonable.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1866

Differential Revision: https://secure.phabricator.com/D3656
2012-10-08 13:27:06 -07:00
vrana
22cb8f5d08 Require canonical numbers in routes
Summary:
D03646 works, I don't want it to work.
Theoretically, it can cause us some troubles if we use this string in JS number context where 030 is 24.

Test Plan: D03646, D3646

Reviewers: epriestley, edward

Reviewed By: edward

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3646
2012-10-05 18:07:54 -07:00
vrana
fe355f7905 Fix typo in route 2012-10-04 13:52:06 -07:00
Bob Trahan
4c86893108 allow arcanist projects to be deleted via the ui
Summary: tricky part is purging symbols at that time too. override "delete" method to get there, use transactions, etc.

Test Plan: deleted an arcanist project - it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, mbishopim3

Maniphest Tasks: T1738

Differential Revision: https://secure.phabricator.com/D3613
2012-10-04 13:25:58 -07:00
vrana
e84c18d734 Mark revision as closed before attaching commit and loading changes
Summary:
D3555 stopped multiple commenting but it still attaches multiple diffs.

Save earlier to stop doing unnecessary work.

Test Plan: Reparsed the same commit twice at the same time.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3598
2012-10-03 15:47:55 -07:00
epriestley
78784aa1fe Group applications into groups on /applications/
Summary: So they're maybe a little easier to deal with? I'm going to take this formally to "plz @chad plz help" land.

Test Plan: {F20329}

Reviewers: btrahan, vrana, chad

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3609
2012-10-03 15:46:19 -07:00
vrana
b585146532 Display auditors in audit commit list
Summary: Users wonder who is actually doing all these audits.

Test Plan:
/owners/package/1/
/audit/view/packagecommits/?phid=PHID-OPKG-1

Reviewers: epriestley

Reviewed By: epriestley

CC: jungejason, aran, Korvin

Differential Revision: https://secure.phabricator.com/D3597
2012-10-03 10:41:39 -07:00
epriestley
f817d81b12 Add applications for Owners, Repositories, PHID manager, PHPAST
Summary: See D3572.

Test Plan: Viewed /applications/, saw all these applications.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3579
2012-10-01 12:56:33 -07:00
Evan Priestley
7dd8851a61 Merge pull request #207 from vvavrychuk/daemon
use available api to check daemon status
2012-09-27 12:53:54 -07:00
Vasyl Vavrychuk
18ee51ac7b use available api to check daemon status
Summary:
Replace executing 'ps aux' with usage of available api to
control daemons PhabricatorDaemonControl.

This fixes isPullDaemonRunningOnThisMachine returning wrong status
under Fedora & lighttpd & SELinux because SELinux with default
settings blocked getting all processes in 'ps aux'.

Test Plan: start stop daemon and check repository app

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3557
2012-09-27 22:27:51 +03:00
vrana
7ed8b6d7fa Recheck if revision hasn't been already closed in commit parser
Summary:
If attaching a commit or checking if there are any changes takes nonzero time then the revision may be closed by someone else.

Cleaner solution would be to do it inside a transaction and mark the SELECT as FOR UPDATE but it would be blocking.

Test Plan: Patched `$should_close` to be true, reparsed an already closed commit.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3555
2012-09-26 21:12:03 -07:00
vrana
e0e97b08b8 Open editor on first modified line
Test Plan: Created diff, opened the file from Differential, opened the file in Diffusion.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3538
2012-09-24 11:07:59 -07:00
vrana
f7a38df5a5 Add TODO comment to commit attacher 2012-09-19 12:30:27 -07:00
Nick Harper
90b60bc3c9 Load relationships for revision
Summary:
In the commit message parser worker, it tries to get relationships for a
revision without first loading them. (D3444 introduced the change.) This
is where the get happens, so I'm adding the load here - maybe it should be
added in the CommitMessageParserWorker instead?

Test Plan: Made this change, and my taskmaster daemons stopped failing.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3448
2012-09-06 15:46:51 -07:00
epriestley
0736592cff Add didParseCommit() to DifferentialFieldSpecification
Summary:
This is mostly ripped from D2721, but doesn't implement the T945 part.

After we parse a commit message, give DifferentialFieldSpecifications an opportunity to react to the message as well (e.g., by updating related objects).

Test Plan: Impelmented a var_dump() body, ran `reparse.php` on a commit.

Reviewers: vrana, 20after4, btrahan, edward

Reviewed By: 20after4

CC: aran

Maniphest Tasks: T945, T1544

Differential Revision: https://secure.phabricator.com/D3444
2012-09-06 12:13:51 -07:00
vrana
b3d7ed3e8e Respect users with duplicate real names in commit parser
Test Plan:
  $parser->resolveUserPHID('Lei Zhao');
  $parser->resolveUserPHID('Jakub Vrana');

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3437
2012-09-05 12:24:35 -07:00
epriestley
ad409126b1 Fix a bug with owners analysis of commits
Summary: If the commit has a known author but that author is different from the revision author or the revision doesn't exist, we'll throw away the commit author and then raise an audit for "Owners Not Involved". Instead, we should just use the commit author in all cases.

Test Plan:
Debugged this with Zor in IRC, he reported it fixed his issue.

Before:

http://pastie.org/4574680

("Author" is empty.)

After:

http://pastie.org/4574712

("Author" is correctly recognized.)

Reviewers: jungejason, nh, vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3374
2012-08-23 10:12:41 -07:00
epriestley
93b0a501a4 Add local navigation to Differential
Summary:
Adds a flexible navigation menu to diffs that shows you your current position in the diff.

Anticipating some "this is the best thing ever" and some "this is the wosrt thing ever" on this, but let's see how much pushback we get? It seems pretty good to me.

Test Plan: Will attach screenshots.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1633, T1591

Differential Revision: https://secure.phabricator.com/D3355
2012-08-21 15:01:20 -07:00
vrana
0f919ecd3c Update line count after attaching commit diff
Summary: We should probably do this also in `differential.creatediff` but it's not a big deal because we later call `differential.updaterevision` which does this using `DifferentialRevisionEditor`.

Test Plan: None.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3263
2012-08-13 14:43:05 -07:00
epriestley
3e29921f5e Use phutil_utf8_convert() in Phabricator
Summary: See D3252.

Test Plan: This one is nasty to test, I'm going to make some coffee first.

Reviewers: davidreuss, vrana, btrahan

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T452

Differential Revision: https://secure.phabricator.com/D3254
2012-08-12 08:50:19 -07:00
Nick Harper
62b3e4aea3 Remove a check for closing revisions
Summary:
I ran into a case where a commit isn't "new" but hasn't been closed. I
think the check on the status of the differential revision should be
enough and this check isn't needed.

Test Plan:
used the reparse.php script to close a revision that previously wouldn't
close.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3232
2012-08-10 16:15:37 -07:00
vrana
1c98ca7042 Fix doc links 2012-08-06 15:58:28 -07:00
Alan Huang
bcb9de4ea1 Add a context field to symbol objects
Summary:
See T1602.

This is just the minimal functional patch; the scripts will continue
working because of the `DEFAULT ''`.

Test Plan:
Can't fully test this until I get more code working, but
nothing broke horribly yet.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3147
2012-08-06 12:20:45 -07:00
epriestley
b8f4f7c438 Fix an issue with Diffusion's display of copied files in Git
Summary: If a change copies some file `A` to `B` and also edits `A`, we currently record this as an indirect change and don't show the edits to `A` in the diff. Instead, record these as direct changes.

Test Plan: Created two commits, one which copied `A` to `B` without modifying `A` and one which copied `A` to `B` and modified A. Viewed both commits in Diffusion. The unmodified commit did not show `A`, and the modified commit did (with the correct changes).

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: champo, aran

Differential Revision: https://secure.phabricator.com/D3120
2012-08-01 12:32:35 -07:00
vrana
9092994e45 Make old and new files when checking for changes by commit
Summary:
We have some false positives on commit changes checker.
I'm not sure if the reason is a difference between `git diff` and `svn diff` or something else but making this more robust doesn't harm anything.

We couldn't make the files from the whole changeset because I want to ignore context bigger than `$num_lines` to reduce rebase noise.

Test Plan:
Ran the method on diff which had false positive previously.
Ran the method on a diff with real change.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3107
2012-07-30 17:57:30 -07:00
pearj
f2dea87c11 Fix Exception Bad getter call: getURIObject
After this commit: d9296638cd

I started getting this error:

Unhandled Exception ("Exception")
Bad getter call: getURIObject

It turns out that getURIObject just needed to be getRemoteURIObject and then the problem goes away.
2012-07-28 18:27:52 +10:00
epriestley
d9296638cd Fix a "user@domain:path" protocol handling bug
Summary:
In D3063, we stopped converting "user@domain:path" git-style URIs, but this broke the SSH-detection code and I missed that in my test plan because my test case uses natural SSH keys so the omission of SSH flags didn't cause failures.

This code is a bit of a mess anyway. Consolidate and refactor it to be a bit simpler, and add test coverage.

Test Plan: Ran unit tests. Ran "test_connection.php" in SSH and non-SSH modes, verified SSH modes generated appropriate ssh-agent commands around the git remote commands.

Reviewers: vrana, btrahan, tberman

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1529

Differential Revision: https://secure.phabricator.com/D3093
2012-07-27 17:21:33 -07:00
epriestley
514ee3526c Add an event for looking up names from repositories
Summary:
Currently, we have this cumbersome `PhabricatorRepositoryCommitMessageDetailParser` hook. This is really old and outdated; I want to just use the Differential custom field parser. See T945 for a specific application.

However, it allows installs to override author/committer association. Instead, provide an event hook for doing this.

Test Plan: Added a listener, made every commit resolve to "turtle", parsed some commits, verified the events looked sane and they now correctly were all attributed to "turtle".

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1337

Differential Revision: https://secure.phabricator.com/D3040
2012-07-24 11:59:28 -07:00
epriestley
1e9120cdfb Minor, fix an issue in Diffusion/Herald with commits that have no Auditors. 2012-07-24 07:37:41 -07:00
epriestley
8cbd860e31 Include email in Git author/committer lookup
Summary: We currently omit email from Git author/committer lookups, which gives us some bad results when identify commit authors. Include email. Also simplify this block a little bit.

Test Plan: Ran "reparse.php --message" on several commits, verified that the author/committer seemed reasonable with var_dump()s.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1337

Differential Revision: https://secure.phabricator.com/D3039
2012-07-23 13:59:58 -07:00
epriestley
839f3df9c2 Stop trasforming "scp-style" URIs into normal URIs in Git
Summary:
See D3033, T1529. We currently transform "scp-style" `user@host:path` URIs into normal `ssh://user@host/path` URIs. This is undesirable for two reasons:

  - The paths aren't always equivalent. They are for GitHub, which is why I missed this originally, but in the general case the ":path" is resolved relatively and the "/path" is resolved absolutely. So this transformation can break things.
  - It confuses users, who do not think of "git@host:path" URIs as SSH URIs even though the SSH protocol is implied.

So stop using them, and just use the "git@host:path" URIs instead. This is a bit messy since we have some validation built up on top of URIs. Hopefully we can get rid of more of this in the future as we simplify repository management.

Test Plan: Unit tests cover this stuff pretty well. Made a new git repository with a "git@host:path" style URI and did pull/discover on it, verified the right URI was used.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1529

Differential Revision: https://secure.phabricator.com/D3036
2012-07-23 12:05:03 -07:00
epriestley
4dd707b189 Include explicit "Auditors:" in initial Diffusion email
Summary:
See https://groups.google.com/forum/?hl=en_US&fromgroups#!topic/phabricator-dev/ZonbTN-0TGw

When a commit's author explicitly specifies users as "Auditors", send them an email about the commit.

Test Plan: Made a commit with "Auditors: turtle", ran "reparse.php --herald" on it, verified user "turtle" got included in the intial email.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, MarkChuCarroll

Differential Revision: https://secure.phabricator.com/D3032
2012-07-23 09:24:02 -07:00
epriestley
378feb3ffb Centralize rendering of application mail bodies
Summary: This is a minor quality-of-life improvement to prevent D2968 from being as nasty as it is.

Test Plan: Ran unit tests; generated Differential, Maniphest and Diffusion emails and verified the bodies looked sensible.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T931

Differential Revision: https://secure.phabricator.com/D2986
2012-07-16 19:01:43 -07:00
epriestley
9574b91e55 Add options to include patches inline or attached for Diffusion commit emails
Summary:
See D818 for an older attempt at this. Support code has matured to the point where the patch is pretty straightforward.

@tido, this was a long-standing request from Aditya back in the day.

Test Plan: Used `reparse.php --herald` to send myself a bunch of emails with various patch configurations. Confirmed that limits are respected, reasonable errors arise when they're violated, etc. (Timeout is a little funky but that's out of scope here, I think.)

Reviewers: btrahan

Reviewed By: btrahan

CC: tido, aran

Maniphest Tasks: T456

Differential Revision: https://secure.phabricator.com/D2967
2012-07-12 13:33:35 -07:00
epriestley
e2e9aed4fa Fix symbol handling in symbol query and IRC "Where is x?" handler
Summary: If a symbol's project has no linked repository, we currently explode. Instead, decline to generate a URI and fall back gracefully.

Test Plan: https://secure.phabricator.com/chatlog/channel/%23phabricator/?at=22345

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1465

Differential Revision: https://secure.phabricator.com/D2948
2012-07-09 17:51:42 -07:00
epriestley
ddf67fce58 Add an example event listener, improve documentation, and add a commit discovery event
Summary: Improve documentation around Phabricator events.

Test Plan: Generated and read documentation. Ran test script.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran

Maniphest Tasks: T1092

Differential Revision: https://secure.phabricator.com/D2917
2012-07-03 16:46:27 -07:00
epriestley
dcf3a31944 Minor, unlock after discovery exception. 2012-07-03 07:46:38 -07:00
epriestley
f5f88d8698 Minor, fix an issue where %e is empty from git log. 2012-07-02 07:29:15 -07:00
epriestley
1c1fdf0448 Minor, move the "repair" instructions to the right place. 2012-07-02 07:22:56 -07:00
Nick Harper
d70f8dad3f Change lock in PullLocalDaemon to only lock on discovery
Summary:
The locks held by read-only pullLocal daemons were causing our discovery instance
to not get the lock and fail at discovery. We don't need to hold the lock while
pulling (only while discovering), so this moves the lock to the appropriate place.

Test Plan: tested in production

Reviewers: jungejason, epriestley, vrana

Reviewed By: jungejason

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2890
2012-06-29 16:19:32 -07:00
David Reuss
aa95ef2463 Handle encoding if git tells us it's still not UTF-8
Summary:
Even though `--encoding` is passed to the command, git still fails
in some cases to correctly convert the output. Attempt the conversion
ourselves if it's non UTF-8.

Test Plan: Reparsed message in a repository with ISO-8859-1 encoded commit messages.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T452

Differential Revision: https://secure.phabricator.com/D2888
2012-06-29 12:38:09 -07:00
epriestley
46008ce3b3 Record initial branch when recording commits
Summary:
We have a race condition right now, where we may insert a commit without `seenOnBranches`. This means shouldAutocloseCommit will return false (since it's not on any autoclose branches) if the message parser runs fast enough, causing it to associate the commit but not close the revision. This happened for D2851.

Also prompt the user to repair broken repositories.

Test Plan: Ran discovery / repair. Ran discovery on new commits. Verified 'seenOnBranches' value. Deleted some data, verified "repair" error. Repaired repository.

Reviewers: jungejason, nh, vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2858
2012-06-28 14:00:00 -07:00
epriestley
1bd0a1f0d9 Add a global lock for the RepositoryPullDaemon
Summary: Allow multiple daemons to run without contention.

Test Plan: Ran multiple daemons simultaneously in "debug" mode, observed them acquiring (and sometimes failing to acquire) locks.

Reviewers: btrahan, jungejason, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1400

Differential Revision: https://secure.phabricator.com/D2877
2012-06-28 09:54:55 -07:00
Jason Ge
b644a2c3a7 Remove leading "/" so that "git cat-file" works
Summary:
When the absolute path is used for load file (loadFileContent(()), it fails in git. For example:

  /var/repo/page_admin_app
  > git cat-file blob '4d6c03923006d6c444660f2c734fe03e10fd20bd':'/ios/PageAdminApp/Resources/splash/De           fault-Portrait@2x~ipad.png'
  fatal: Not a valid object name 4d6c03923006d6c444660f2c734fe03e10fd20bd:/ios/PageAdminApp/Resources/s           plash/Default-Portrait@2x~ipad.png

This is breaking the auto-closing for about 8 revisions like

https://phabricator.fb.com/rPPA4d6c03923006d6c444660f2c734fe03e10fd20bd ...
https://phabricatorcator.fb.com/rPPA51acb7e482aab0c491b530ed19dddc741d50f673 ...

Test Plan:
- reparsed https://phabricator.fb.com/rPPA4d6c03923006d6c444660f2c734fe03e10fd20bd successfully with corresponding differential revision being closed.

- verified that without leading '/', loadFileContent for svn still works. Both of the following commands worked (note the double '/' right before 'tfb':

  svn cat svn+ssh://svn.vip.facebook.com/svnroot//tfb/trunk/www/flib/intern/cachearchiver/regenerators/wurfl/CacheArchiveWurflRegenerator.php@579700
  svn cat svn+ssh://svn.vip.facebook.com/svnroot/tfb/trunk/www/flib/intern/cachearchiver/regenerators/wurfl/CacheArchiveWurflRegenerator.php@579700

Reviewers: vrana

Reviewed By: vrana

CC: nh, aran, epriestley

Differential Revision: https://secure.phabricator.com/D2847
2012-06-26 11:58:34 -07:00
epriestley
7b50b2fbdc Use subprocess output streaming to improve performance of Git commit discovery
Summary:
Improve performance of large discovery tasks in Git by using subprocess streaming, like we do for Mercurial.

Basically, we save the cost of running many `git log` commands by running one big `git log` command but only parsing as much of it as we need to. This is pretty complicated, but we more or less need it for mercurial (which has ~100ms of 'hg' overhead instead of ~5ms of 'git' overhead) so we're already committed to most of the complexity costs. The git implementation is much simpler than the hg implementation because we don't need to handle all the weird parent rules (git gives us to them easily).

Test Plan:
Before, `discover --repair` on Phabricator took 35s:

  real	0m35.324s
  user	0m13.364s
  sys	0m21.088s

Now 7s:

  real	0m7.236s
  user	0m2.436s
  sys	0m3.444s

Note that most of the time is spent inserting rows after discover, the actual speedup of the git discovery part is much larger (subjectively, it runs in less than a second now, from ~28 seconds before).

Also ran discover/pull on single new commits in normal cases to verify that nothing broke in the common case.

Reviewers: jungejason, nh, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1401

Differential Revision: https://secure.phabricator.com/D2851
2012-06-25 15:21:48 -07:00
epriestley
ca31e3e84b Add a --repair flag to "bin/repository discover"
Summary:
If a repository is missing commits because they mysteriously vanished, there's no reasonable way to get them back right now. Provide a way to ignore the state in the database and rediscover the entire repository unconditionally.

We don't queue any reparses or anything, but when I move reparse into this script we can hook things up or something. This generally shouldn't be too important anyway.

Test Plan: Ran `repository discover --repair` on Phabricator.

Reviewers: jungejason, nh, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2850
2012-06-25 12:35:47 -07:00
epriestley
13d96e6377 Introduce "bin/repository" for repository management
Summary:
Nothing new or exciting here yet, just moving the random scripts/repositories/ things to bin/repository. Also add `repository list`.

(Console stuff comes from D2841.)

Test Plan: Ran `repository list`, `repository pull`, `repository discover`, `repository discover --verbose`, `repository help`.

Reviewers: jungejason, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2849
2012-06-25 12:35:37 -07:00
epriestley
a705f336a3 Add vebose logging to PhutilRepositoryPullDaemon
Summary: Add verbose logging. This logging is activated by setting "phd.verbose" in the config, running "phd debug", or explicitly in scripts/repository/pull.php and scripst/repository/discover.php

Test Plan:
  >>> orbital ~/devtools/phabricator $ ./scripts/repository/discover.php GTEST
  Discovering 'GTEST'...
  <VERB> PhabricatorRepositoryPullLocalDaemon Discovering commits in repository 'GTEST'...
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '()_+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '_+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch', at 774c7737b2d560a291697126bf4513204ccf661a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch-1', at dc97539bee07293f95990d71f4638335a2531d69.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch-2', at 1acfaec313c46dd3caa90448800181fb91b0270f.

Reviewers: jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2843
2012-06-24 15:06:40 -07:00
Jason Ge
de46a2550c Fix auto-close in a repo with missing commits in Phabricator db
Summary:
After adding more error logging: https://secure.phabricator.com/differential/diff/5295/, discover.php prints the output: https://secure.phabricator.com/P427

The problem is that there are gaps in the commits and revisions in Phabricator's DB for the repo. For some git commits, both the phabricator commits and the differential revision are missing (because of a mysterious DB data loss. nharper should be able to recover it); for some other git commits, the phabricator commits are missing (probably also because of the daba loss mentioned above). This can also happen if the git commits were too big to be parsed by phabricator.

Because of the gap, the pull daemon detects them as new autoclose commits on the branch
  (https://secure.phabricator.com/diffusion/P/browse/master/src/applications/repository/daemon/PhabricatorRepositoryPullLocalDaemon.php;382bafa271ec4962$267)
  (https://secure.phabricator.com/diffusion/P/browse/master/src/applications/repository/daemon/PhabricatorRepositoryPullLocalDaemon.php;382bafa271ec4962$592)

Then tries to update it
  (https://secure.phabricator.com/diffusion/P/browse/master/src/applications/repository/daemon/PhabricatorRepositoryPullLocalDaemon.php;382bafa271ec4962$616)
and will fail because the commit doesn't exist in the DB.
  (https://secure.phabricator.com/diffusion/P/browse/master/src/applications/repository/daemon/PhabricatorRepositoryPullLocalDaemon.php;382bafa271ec4962$341)
After the exception is thrown, all the commits just detected are not marked as 'seenOnBranches', so they won't be marked as closed by the CommitMessageParserWorker.

This also explains the weird behavior where the parsing of the commits is always paused for 15 minutes and then gets started again. The parsing pauses because the process listed at https://secure.phabricator.com/P427 takes quite some time to finish before it can record tha tasks. The parsing happens all the time because the process will be executed again and again.

A better fix might be to put all the missing commits in the repository_badcommit table and check against it.

Test Plan: will run 'discover.php FBCODE' to test it.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: nh, vrana, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2846
2012-06-23 09:47:20 -07:00
Jason Ge
969dcc5bae handle svn and hg for auto-close
Summary: in svn and hg (for now), no branch used.

Test Plan: will test live

Reviewers: epriestley

CC: nh, vrana, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2839
2012-06-22 17:05:06 -07:00
Jason Ge
a75da4efac Revert "run record commit if its the first time seen"
Summary: This reverts commit 22e606d49e.

Test Plan: none

Reviewers: epriestley, nh

Reviewed By: nh

CC: nh, vrana, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2833
2012-06-21 20:26:07 -07:00
Jason Ge
22e606d49e run record commit if its the first time seen
Test Plan: test live.

Reviewers: nh

Reviewed By: nh

CC: epriestley, aran

Differential Revision: https://secure.phabricator.com/D2832
2012-06-21 19:47:25 -07:00
Jason Ge
06c976b738 check commit before calling its method
Summary: it's calling pull daemon's failure. This is actually Nick's fix.

Test Plan: Nick already manually ran it on daemon machine.

Reviewers: vrana, nh

Reviewed By: vrana

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2828
2012-06-21 18:36:31 -07:00
epriestley
8ddfdb26e9 Publish feed stories about commits
Summary: When stuff gets committed, publish feed stories about it.

Test Plan: {F13061}

Reviewers: jungejason, vrana, voldern

Reviewed By: jungejason

CC: aran

Maniphest Tasks: T1322

Differential Revision: https://secure.phabricator.com/D2805
2012-06-20 06:03:44 -07:00
epriestley
d277908842 Minor, avoid branch loop nonsense for repositories not set to autoclose. See IRC. 2012-06-19 15:05:22 -07:00
epriestley
e48830eecb Don't rediscover an entire branch when a new commit appears at HEAD
Summary:
The autoclose logic is currently doing a little too much work. We want to parse each commit at most twice:

  # When it first appears in the repository.
  # When it first appears on an autoclose branch.

These two events might not be distinct (i.e., it might first appear on an autoclose branch).

Currently, to discover commits initially appearing on autoclose branches, we check each branch, determine if it's an autoclose branch or not, and determine if the HEAD is already a known commit on an autoclose branch. This is correct so far, and allows us to ignore branches which either haven't changed or have commits at HEAD which we've already examined.

However, if an autoclose branch has a new commit, we start working backward through it. Prior to this patch, we only stop when we hit commits that we've already discovered lie on this branch. If the branch is new, none of the commits will be discovered on it (they're discovered in general, and likely discovered on other autoclose branches, but not discovered on this branch), so we'll parse all the way back to the root.

Instead, we want to stop when we hit commits that we've already discovered on //any// autoclose branch.

So do that.

Test Plan: Pushed a new branch, then pushed a new commit at HEAD. Ran discovery, verified we rediscovered only 1 commit, not every commit in history.

Reviewers: vrana, jungejason, aurelijus

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1389

Differential Revision: https://secure.phabricator.com/D2798
2012-06-19 13:47:01 -07:00
epriestley
14648d6d7a Provide more informative messages when autoclosing revisions
Summary: This has been a point of some confusion, make the messages more explicit.

Test Plan:
Added var_dump() stuff and ran on some commits:

  $ ./scripts/repository/reparse.php --message rP9fc54f4dfb61f7338cb1cfe819bc72d2a3404264
  Running 'PhabricatorRepositoryGitCommitMessageParserWorker'...
  string(58) "Closed by commit rP9fc54f4dfb61 (authored by @epriestley)."
  $ ./scripts/repository/reparse.php --message rP444c634b6c6612fc7b36ddffab8023ef67372ab9
  Running 'PhabricatorRepositoryGitCommitMessageParserWorker'...
  string(83) "Closed by commit rP444c634b6c66 (authored by Ben Rogers, committed by @epriestley)."
  $ ./scripts/repository/reparse.php --message rP22d12fe499e3ecb62392397f2ac2a91768c974aa
  Running 'PhabricatorRepositoryGitCommitMessageParserWorker'...
  string(52) "Closed by commit rP22d12fe499e3 (authored by vrana)."
  $ ./scripts/repository/reparse.php --message rPe51958159483cd0acf00adcff51edf8717b4a23b
  Running 'PhabricatorRepositoryGitCommitMessageParserWorker'...
  string(85) "Closed by commit rPe51958159483 (authored by David Fisher, committed by @epriestley)."

Reviewers: csilvers, vrana

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2765
2012-06-15 17:00:08 -07:00
Ben Rogers
444c634b6c Allows branches to appear in diffusion with out triggering closing things from the commit message
Summary: Implemented it how it was suggested in ticket comments

Test Plan: create a revision in a branch, push that branch up, verify it's visible in diffusion and also that revision is not closed, then merge and push to master, verify that revision closed

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, 20after4

Maniphest Tasks: T1210

Differential Revision: https://secure.phabricator.com/D2706
2012-06-15 14:10:15 -07:00
vrana
892a2d1b61 Make Thread-Topic human readable
Summary:
Some e-mail clients display this header and it needs to be constant.

This is somehow involved but I doubt that there is a simpler solution.

Test Plan:
Applied SQL patch.
Commented on revision, commented on commit, changed package.
Verified that the `Thread-Topic` has constant and human readable value.

Reviewers: epriestley

Reviewed By: epriestley

CC: ola, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2745
2012-06-14 11:36:34 -07:00
epriestley
375c921bb0 Minor, fix issue with defaults: wildcard and repeat default to array and may not have a different default. 2012-06-12 07:05:57 -07:00
vrana
2e484e257d Fix lint errors found by Nemo
Summary:
See also:

- https://github.com/tpyo/amazon-s3-php-class/pull/33
- https://github.com/stripe/stripe-php/pull/13

Test Plan: Ran a script analyzing sources by HPHP.

Reviewers: btrahan, jungejason, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2713
2012-06-11 19:09:42 -07:00
vrana
2793828795 Refactor setting e-mail subjects
Summary:
It seems that Outlook and Mail.app mostly ignores the threading headers and thread primarily by subject.
They are also very picky about the Re: part in the header.
I guess that's because users of these clients often hit Reply when they want to create a new message to the sender of an e-mail.

We need both of these applications to work with the same setting because we don't use multiplexing to prevent sending multiple e-mails to people in lists.
I also believe that the default behavior should just work in most setups.

I've tried several different combinations of putting "Re:" and none of them seems to always work in both clients.

This diff at least adds more abstraction to the code which should prevent copy/paste errors (two fixed by this diff!).

Test Plan: Sent several e-mails with varying subject, verified that they look as before in Outlook and Mail.app.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2709
2012-06-11 19:07:21 -07:00
vrana
a0b57cefb7 Declare that revision was closed by committer, not author
Summary: D2703#13 is confusing - it looks like that @allenjohnashton took the action but it was @epriestley.

Test Plan: I don't have a repro so I tested this block standalone.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2707
2012-06-11 11:30:27 -07:00
epriestley
5fee8c50ee Fail more softly if we can't execute "ps"
Summary: If, e.g., $PATH is broken we may not be able to run "ps". We'll explode pretty hard, currently. Instead, just show a harsher warning.

Test Plan: Changed "ps auxwww" to "psq", which doesn't exist on my system. Loaded page, got warning instead of explosion.

Reviewers: nathanws, vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2624
2012-06-02 14:05:27 -07:00
vrana
6cc196a2e5 Move files in Phabricator one level up
Summary:
- `kill_init.php` said "Moving 1000 files" - I hope that this is not some limit in `FileFinder`.
- [src/infrastructure/celerity] `git mv utils.php map.php; git mv api/utils.php api.php`
- Comment `phutil_libraries` in `.arcconfig` and run `arc liberate`.

NOTE: `arc diff` timed out so I'm pushing it without review.

Test Plan:
/D1234
Browsed around, especially in `applications/repository/worker/commitchangeparser` and `applications/` in general.

Auditors: epriestley

Maniphest Tasks: T1103
2012-06-01 12:32:44 -07:00
vrana
1ebf9186b4 Depend on class autoloading
Test Plan:
Run setup.
/differential/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2612
2012-05-30 16:57:21 -07:00
epriestley
09c8af4de0 Upgrade phabricator to libphutil v2
Summary: Mechanical changes from D2588. No "Class.php" moves yet.

Test Plan: See D2588.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2591
2012-05-30 14:26:29 -07:00
Jason Ge
bbd5c8c187 Add audit entry when code changed after a revision was accepted
Summary: build on top of D2530 and D2540. Add an auditing entry when the code was changed after the revision was accepted.

Test Plan: ran reparse.php manually. It worked at https://phabricator.fb.com/rPHGIT461864d2e09dad04c28505658ef75a979e44d0d3. Look at the latest auditing entry for auditor 'phgit ha directory'.

Reviewers: vrana

Reviewed By: vrana

CC: nh, epriestley, aran

Maniphest Tasks: T201

Differential Revision: https://secure.phabricator.com/D2586
2012-05-29 11:32:09 -07:00
vrana
af6238ca4a Inform about changes made between last revision and commit
Summary:
This adds a link to [Closed] e-mail if it detects some changes.
It compares added and removed lines with 3 lines context.
The subtle form of informing is permissive to false negatives and positives.
I have an e-mail filter for [Closed] e-mails so I wouldn't personally notice this change - we should probably promote this feature a little bit.

Test Plan:
Reparse a diff with a change after last update.
Reparse a diff without a change after last update.

Reviewers: jungejason, epriestley

Reviewed By: jungejason

CC: aran, Koolvin, btrahan

Maniphest Tasks: T201

Differential Revision: https://secure.phabricator.com/D2540
2012-05-25 21:39:58 -07:00
Bob Trahan
65710ee2d2 Fix repository interactions for SVN repositories using the SVN protocol with SASL
Summary: also makes the UI more general for this username + password business.

Test Plan:
- configure a phabricator repository from the svn server @asherwin provided which is configured for svn protocol with SASL
- observed phabricator failing without my patch
- upgraded my SVN client to support SASL (protip for mac users - http://www.wandisco.com/subversion/download#osx)
- applied patch to phabricator
- restarted daemons
- noted daemon success - diffusion populating nicely

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1260

Differential Revision: https://secure.phabricator.com/D2549
2012-05-23 12:37:43 -07:00
Hafsteinn Baldvinsson
a438c87c52 Show the difference between a committer and an author
Summary:
Git and hg (supposedly..) differentiate between an author (who wrote the patch)
and a committer (who applied the patch).

This patch allows Phabricator to note when a patch is committed
by someone other than the Author.

Test Plan:
Created 2 accounts,
 - U (Account with a PHID)
 - U' (Account without a PHID)
and had them create and commit commits

testing if their username/real name would be displayed correctly in Diffusion,
  - BrowserTable
  - HistoryTable
  - Code revision

Teztz,
A(uthor)/C(ommitter)
If it's A/A then Author committed

UL = User link (<a href="/p/username">username</a>)
UN = User name ("Firstname Lastname")

Tezt | Expected in table  | Got
-------------------------------------------
A/A   | UL/UL             | UL/UL
A'/C  | UN/UL             | UN/UL
A/C'  | UL/UN             | UL/UN
A'/C' | UN/UN             | UN/UN

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T688

Differential Revision: https://secure.phabricator.com/D2541
2012-05-23 08:34:54 -07:00
vrana
ccd37afab8 Attach commit diff to its revision
Summary:
This attaches commit diff to its associated revision as any other diff.
The consequence is that the revision page now shows the actual commit instead of the last diff. It may be disturbing but it is desired.
Another consequence is that lint and unit results are displayed as skipped after the revision was committed. I want to fix it somehow.

My next plan is to automatically diff against the last normal diff and include the link to this diff in commit e-mail if non-empty.

Test Plan:
  reparse.php

Diff against last normal diff.

Reviewers: epriestley, jungejason

Reviewed By: jungejason

CC: aran, Koolvin, jungejason

Maniphest Tasks: T201

Differential Revision: https://secure.phabricator.com/D2530
2012-05-22 16:08:16 -07:00
Sigurd Holsen
c2d2f6ded1 bugfix: mercurial commit worker had an array error
It tried to get $path['targetPath'] istead of $change['targetPath']
when $path was a string
2012-05-21 09:04:48 -07:00
Bob Trahan
80f479d948 Make directories with spaces in their names work OK in diffusion end to end
Summary:
we were parsing the git log output slightly incorrectly and over-exploding on spaces. we also needed to escape the path %20 stuff`.

Not sure if there's something fancy to do given folks should reparse their repos if they are impacted by this issue.

Test Plan:
made a directory with spaces and some dummy revisions. observed diffs wouldn't load and links broken.
with patch, ran scripts/reparse.php for pertinent revisions and diffs loaded and links weren't broken.

Reviewers: floatinglomas, epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1252

Differential Revision: https://secure.phabricator.com/D2510
2012-05-20 15:29:36 -07:00
epriestley
3a1ee00335 Select default branches more effectively in Diffusion
Summary: If you have an empty value saved in the "default branch" field, we default to empty string (or null, or whatever) instead of the correct default.

Test Plan:
  - Looked at a Git repo with an empty default branch, got a default to "master" instead of an error.
  - Looked at a Mercurial repo with an empty default branch, got a default to "default" instead of an error.

Reviewers: btrahan, csilvers

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1237

Differential Revision: https://secure.phabricator.com/D2512
2012-05-20 14:50:43 -07:00
Bob Trahan
7a19cf5a55 Fix lines of a large include path
Summary:
D2457 and D2459 made some changes here. This little guy just needed to be touched so it could be arc lint'd to have the proper updated include paths

Sorry to spam reviewers; Evan is in Diablo 3 somewhere. :D

Test Plan: no more fatal on test everything implemented (arc unit src/infrastructure/__tests__/)

Reviewers: epriestley, jungejason, nh, csilvers

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2474
2012-05-15 12:37:34 -07:00
epriestley
820a6d407a Improve performance of repository discovery under Mercurial
Summary: Prelminary, I want to test this a bit more when I'm better rested. Provide a "bulk" import mode for Mercurial so we can do initial discovery more quickly.

Test Plan: Imported the Mercurial repository in 2m45s, blocked on MySQL I/O rather than Mercurial I/O.

Reviewers: csilvers, btrahan

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2457
2012-05-11 18:29:14 -07:00
epriestley
61b728b19d Add a setting to blanket-disable autoclosing revisions in a repository
Summary: For git workflows where developers push personal feature branches to the origin, this is a quick workaround until T1210 is implemented properly. I'll also mention this in D2446.

Test Plan:
  - Committed a revision in an "autoclose" repository; it was autoclosed. Committed a revision in a no-autoclose repository, it was not autoclosed.
  - Edited repositories, saving the "autoclose" setting as enabled/disabled.

Reviewers: aurelijus, btrahan

Reviewed By: aurelijus

CC: aran

Maniphest Tasks: T1210

Differential Revision: https://secure.phabricator.com/D2448
2012-05-10 14:18:56 -07:00
epriestley
1bf68e06a5 Improve Diffusion error messages and UI for partially imported repositories
Summary:
  - When you have an un-cloned repository, we currently throw random-looking Git/Hg exception. Instead, throw a useful error.
  - When you have a cloned but undiscovered repository, we show no commits. This is crazy confusing. Instead, show commits as "importing...".
  - Fix some warnings and errors for empty path table cases, etc.

Test Plan:
  - Wiped database.
  - Added Mercurial repo without running daemons. Viewed in Diffusion, got a good exception.
  - Pulled Mercurial repo without discovering it. Got "Importing...".
  - Discovered Mercurial repo without parsing it. Got "Importing..." plus date information.
  - Parsed Mercurial repo, got everything working properly.
  - Added Git repo without running daemons, did all the stuff above, same results.
  - This doesn't improve SVN much but that's a trickier case since we don't actually make SVN calls and rely only on the parse state.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776

Differential Revision: https://secure.phabricator.com/D2439
2012-05-09 17:28:57 -07:00
epriestley
7e0c06716f Minor, fix obvious typo that I missed in testing somehow.
Auditors: csilvers
2012-05-09 12:43:57 -07:00
epriestley
b800df8c1b Simplify daemon management: "phd start"
Summary:
  - Merge CommitTask daemon into PullLocal daemon. This is another artifact of past instability (and order-dependent parsers). We still publish to the timeline, although this was the last consumer. Long term we'll probably delete timeline and move to webhooks, since everyone who has asked about this stuff has been eager to trade away the durability and ordering of the timeline for the ease of use of webhooks. There's also no reason to timeline this anymore since parsing is no longer order-dependent.
  - Add `phd start` to start all the daemons you need. Add `phd restart` to restart all the daemons you need. So cool~
  - Simplify and improve phd and Diffusion daemon documentation.

Test Plan:
  - Ran `phd start`.
  - Ran `phd restart`.
  - Generated/read documentation.
  - Imported some stuff, got clean parses.

Reviewers: btrahan, csilvers

Reviewed By: csilvers

CC: aran, jungejason, nh

Differential Revision: https://secure.phabricator.com/D2433
2012-05-09 10:29:37 -07:00
epriestley
d2b01aead0 Use one daemon to discover commits in all repositories, not one per repository
Summary:
See D2418. This merges the commit discovery daemon into the same single daemon, and applies all the same rules to it.

There are relatively few implementation changes, but a few things did change:

  - I simplified/improved Mercurial importing, by finding full branch tip hashes with "--debug branches" and using "parents --template {node}" so we don't need to do separate "--debug id" calls.
  - Added a new "--not" flag to exclude repositories, since I switched to real arg parsing anyway.
  - I removed a web UI notification that you need to restart the daemons, this is no longer true.
  - I added a web UI notification that no pull daemon is running on the machine.

NOTE: @makinde, this doesn't change anything from your perspective, but it something breaks this is the likely cause.

This implicitly resolves T792, because discovery no longer runs before pulling.

Test Plan:

  - Swapped databases to a fresh install.
  - Ran "pulllocal" in debug mode. Verified it correctly does nothing (fixed a minor issue with min() on empty array).
  - Added an SVN repository. Verified it cloned and discovered correctly.
  - Added a Mercurial repository. Verified it cloned and discovered correctly.
  - Added a Git repository. Verified it cloned and discovered correctly.
  - Ran with arguments to verify behaviors: "--not MTEST --not STEST", "P --no-discovery", "P".

Reviewers: btrahan, csilvers, Makinde

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T792

Differential Revision: https://secure.phabricator.com/D2430
2012-05-08 12:53:41 -07:00
epriestley
1c62a35710 Run one daemon to pull all working copies, not one daemon per working copy
Summary:
Allow the pull daemon to take a list of repositories. By default, pull all repositories.

Make some effort to respect pull frequencies, although we'll necessarily suffer a bit if running with only one process.

NOTE: We still launch one discovery daemon per working copy, so this only cuts the daemon count in half.

Test Plan:
  - Ran `phd debug pulllocal`, verified behavior.
  - Ran `pull.php P MTEST SVNTEST --trace`, verified it pulled the repos and ran the right commands.
  - Ran `phd repository-launch-master`, verified the right daemons launched, checked daemon console.
  - Ran `phd repository-launch-readonly`, verified the right daemon launched, checked daemon console.

Reviewers: btrahan, csilvers, davidreuss

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2418
2012-05-07 15:01:10 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
epriestley
049048765d Add "repository.create" and "repository.query" methods to Conduit
Summary: Primarily for @csilvers who has 92 million repositories or something. This is a touch hacky, but movitated by pragmatism.

Test Plan:
  - Ran "repository.create" to create repositories, "repository.query" to list them.
  - Tested most or maybe all of the error conditions, probably.

Reviewers: btrahan, vrana, csilvers

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2396
2012-05-04 16:16:22 -07:00
vrana
38ffe45f8e Use committer date instead of author date for Git epoch
Summary:
This is somewhat controversial but push date is usually more useful than commit date (which can be for example a month before other people can see the commit).
We can also store both dates.

Test Plan:
  git log --pretty="%ct %at"

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2319
2012-04-26 16:25:56 -07:00
epriestley
20a5c9b261 Use "closed", not "committed", in Differential
Summary: "Committed" is SVN-specific language, and confusing in Git and Mercurial. Use neutral language instead.

Test Plan: Inspection.

Reviewers: btrahan, Makinde, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T909

Differential Revision: https://secure.phabricator.com/D2087
2012-04-23 17:40:57 -07:00
vrana
1f2cf78c1b Display committed date in Revision Status field
Summary:
This is slightly more complicated for this reason:

- We don't set `dateCommitted` for normal commits, only for markcommitted.
-- We need to add this date to old revisions now.

Test Plan:
Reparse a revision - commit date was set.
Conduit `markcommitted` - commit date was set.
Run SQL script.
Display closed revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2282
2012-04-19 15:05:09 -07:00
epriestley
dec8acd803 Add very basic tag support to Diffusion
Summary: Lists the 25 most recent tags on the "Repository" page.

Test Plan: Looked at a git repository with a tag, saw it. Looked at HG/SVN repos, they didn't break.

Reviewers: davidreuss, 20after4, btrahan, vrana, jungejason

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T1130

Differential Revision: https://secure.phabricator.com/D2255
2012-04-18 08:02:08 -07:00
vrana
d17be1d824 Fix SVN commit change parser for files moved from deleted directory
Summary:
This continues work started at D2215.
Files moved from deleted directory were marked as Copied Here instead of Moved Here.

Test Plan: Reparsed two commits which was previously wrong, now correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Maniphest Tasks: T1114

Differential Revision: https://secure.phabricator.com/D2229
2012-04-14 22:18:27 -07:00