1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-11 08:06:13 +01:00
Commit graph

960 commits

Author SHA1 Message Date
epriestley
e8d272b0da Use standard infrastructure to attach commits to other objects
Summary:
Ref T4896. Now that we have a transaction editor, we can delete a giant block of hacks.

I believe this also resolves the commit/task attachment issues @joshuaspence and @mbishopim3 mentioned.

Test Plan: Attached and detached commits and tasks.

Reviewers: btrahan, joshuaspence, mbishopim3

Reviewed By: mbishopim3

Subscribers: mbishopim3, epriestley, joshuaspence

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10138
2014-08-04 12:03:58 -07:00
epriestley
89b942c183 Move Audit to proper Subscriptions
Summary:
Ref T4896. Currently, subscriptions to commits are stored as auditors with a special "CC" type.

Instead, use normal subscriptions storage, reads and writes.

Test Plan:
  - Ran migration and verified data still looked good.
  - Viewed commits in UI and saw "subscribers".
  - Saw "Automatically Subscribed", clicked Subscribe/Unsubscribe on a non-authored commit, saw subscriptions update.
  - Pushed a commit through Herald rules and saw them trigger subscriptions and auditors.
  - Used "Add CCs".
  - Added CCs with mentions.

Reviewers: btrahan, joshuaspence

Reviewed By: btrahan, joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10103
2014-08-02 00:06:13 -07:00
epriestley
a6698f2ea5 Use ApplicationTransactions when indexing commit/audit comments
Summary:
Ref T4896. Depends on D10056. Moves search indexing to standard infrastructure.

Also, fixes a bug where inline comments would not be indexed.

Test Plan: Used `bin/search index ... --trace` to view index construction of a commit, saw all the comments and inlines get indexed.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10057
2014-07-28 15:02:18 -07:00
epriestley
3fca1b2d2d Fix some missing renames of Application classes
Summary: I think these got caught in the crossfire between Conduit and
Applications. Ref T5655.

Auditors: joshuaspence
2014-07-24 18:03:59 -07:00
epriestley
dc5c87f74c Hide Audit comment table reads behind an API
Summary: Ref T4896. Buries all direct access to the table so we can limit the surface area affected by the migration.

Test Plan:
  - Grepped for `PhabricatorAuditComment`.
  - Grepped for `audit_comment`.
  - Viewed a bunch of comments.
  - Added a comment.
  - Reindexed a commit.
  - Searched for unique term in new comment.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10019
2014-07-24 18:00:07 -07:00
epriestley
8605a1808d Hide direct accesses to Audit inline comment table behind API
Summary: Ref T4896. Move all direct accesses to the inline comment table behind a small amount of API to make it easier to migrate the table.

Test Plan:
  - Grepped for `PhabricatorAuditInlineComment`.
  - Grepped for `audit_inlinecomment`.
  - Created a draft comment.
  - Previewed a draft comment.
  - Reloaded page, still saw draft.
  - Viewed standalone, still saw draft.
  - Made comment, inline published.
  - Added a draft, saw both.
  - Edited inline comment.
  - Reindexed commit.
  - Searched for unique word in published comment, found commit.
  - Searched for unique word in draft comment, no results.

Reviewers: btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T4896

Differential Revision: https://secure.phabricator.com/D10016
2014-07-24 17:59:28 -07:00
Joshua Spence
023dee0d3b Rename Conduit classes
Summary: Ref T5655. Rename Conduit classes and provide a `getAPIMethodName` method to declare the API method.

Test Plan:
```
> echo '{}' | arc --conduit-uri='http://phabricator.joshuaspence.com' call-conduit user.whoami
Waiting for JSON parameters on stdin...
{"error":null,"errorMessage":null,"response":{"phid":"PHID-USER-lioqffnwn6y475mu5ndb","userName":"josh","realName":"Joshua Spence","image":"http:\/\/phabricator.joshuaspence.com\/res\/1404425321T\/phabricator\/3eb28cd9\/rsrc\/image\/avatar.png","uri":"http:\/\/phabricator.joshuaspence.com\/p\/josh\/","roles":["admin","verified","approved","activated"]}}
```

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9991
2014-07-25 10:54:15 +10:00
Joshua Spence
c34de83619 Rename policy capabilities
Summary: Ref T5655. Rename `PhabricatorPolicyCapability` subclasses for consistency.

Test Plan: Browsed a few applications, nothing seemed broken.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D10037
2014-07-25 08:20:39 +10:00
Joshua Spence
97a8700e45 Rename PHIDType classes
Summary: Ref T5655. Rename `PhabricatorPHIDType` subclasses for clarity (see discussion in D9839). I'm not too keen on some of the resulting class names, so feel free to suggest alternatives.

Test Plan: Ran unit tests.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin, hach-que

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9986
2014-07-24 08:05:46 +10:00
Joshua Spence
0c8f487b0f Implement the getName method in PhabricatorApplication subclasses
Summary: Provide an implementation for the `getName` method rather than automagically determining the application name.

Test Plan: Saw reasonable application names in the launcher.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D10027
2014-07-23 23:52:50 +10:00
Joshua Spence
86c399b657 Rename PhabricatorApplication subclasses
Summary: Ref T5655. Some discussion in D9839. Generally speaking, `Phabricator{$name}Application` is clearer than `PhabricatorApplication{$name}`.

Test Plan:
# Pinned and uninstalled some applications.
# Applied patch and performed migrations.
# Verified that the pinned applications were still pinned and that the uninstalled applications were still uninstalled.
# Performed a sanity check on the database contents.

Reviewers: btrahan, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: hach-que, epriestley, Korvin

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9982
2014-07-23 10:03:09 +10:00
Joshua Spence
254542237a Simplify the implementation of PhabricatorPHIDType subclasses
Summary: Instead of implementing the `getTypeConstant` method in all subclasses of `PhabricatorPHIDType`, provide a `final` implementation in the base class which uses reflection. See D9837 for a similar implementation.

Test Plan: Ran `arc unit`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9985
2014-07-22 00:38:23 +10:00
Joshua Spence
76ed7d1a02 Rename PhabricatorDestructableInterface interface
Summary: Ref T5655. The `PhabricatorDestructibleInterface` interface is misspelled as `PhabricatorDestructableInterface`. Fix the spelling mistake.

Test Plan: `grep`. Seeing as this interface is fairly recent, I don't expect that this would cause any widespread breakages.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T5655

Differential Revision: https://secure.phabricator.com/D9988
2014-07-21 23:59:22 +10:00
epriestley
dcc6997793 Modernize "users" typeahead datasource
Summary: Ref T4420. Modernize users.

Test Plan:
- Edited "Commit Authors" on Audit search.
- Edited "Created By" on calendar search.
- Edited "invited" on calendar search.
- Edited "To" on "New conpherence message".
- Edited user on "Add user to conpherence thread".
- Edited "Authors" on countdown search.
- Edited "Author" on differential search.
- Edited "Responsible users" on differential search.
- Edited "Owner" on Diffusion lint search.
- Edited "include users" on Feed search.
- Edited "Authors" on file search.
- Edited "Authors" on Herald rule search.
- Edited a couple of user-selecting Herald fields on rules.
- Edited "user" on legalpad signature exemption.
- Edited "creator" on legalpad search.
- Edited "contributors" on legalpad search.
- Edited "signers" on legalpad signature search.
- Edited "Authors" on macro search.
- Edited "Reassign/claim" on task detail.
- Edited "assigned to" on task edit.
- Edited "assigned to", "users projects", "authors" on task search.
- Edited "creators" on oauthserver.
- Edited "authors" on paste search.
- Edited "actors" and "users" on activity log search.
- Edited "authors" on pholio search.
- Edited "users" on phrequent search.
- Edited "authors", "answered by" on Ponder search.
- Edited "add members" on project membership editor.
- Edited "members" on project search.
- Edited "pushers" on releeph product edit.
- Edited "requestors" on releeph request search.
- Edited "pushers" on diffusion push log.
- Edited "authors", "owners", "subscribers" on global search.
- Edited "authors" on slowvote search.
- Edited users in custom policy.
- Grepped for "common/authors", no hits.
- Grepped for "common/users", no (relevant) hits.
- Grepped for "common/accounts", no (relevant) hits.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4420

Differential Revision: https://secure.phabricator.com/D9885
2014-07-17 15:44:18 -07:00
epriestley
33120e377a Modernize Project/Object edges
Summary: Ref T5245. Updates the project/object edge to use a modern class definition. Moves further toward real edges.

Test Plan: Added projects to some objects, viewed transactions in transaction record.

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9849
2014-07-17 15:42:19 -07:00
epriestley
d4b2bfa2f4 Modernize commit/edge transaction when parsing commit messages
Summary: Ref T5245. With work elsewhere (notably, D9839) we can remove this TODO and use real transactions.

Test Plan: Pushed a `closes Txxx` commit and got a close + transaction.

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9848
2014-07-17 15:42:06 -07:00
epriestley
8cbfb49b4e Remove all edge events
Summary:
Ref T5245. These were a bad idea.

We no longer need actors for edge edits either, so remove those. Generally, edges have fit into the policy model as pure/low-level infrastructure, and they do not have any policy or capability information in and of themselves.

Test Plan: `grep`

Reviewers: chad, btrahan, joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5245

Differential Revision: https://secure.phabricator.com/D9840
2014-07-17 15:41:42 -07:00
James Rhodes
9cb6b2cfcc Remove user-independent date and time functions from Phabricator
Summary: These have been moved into libphutil.

Test Plan: Browsed Phabricator, didn't see a crash.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9907
2014-07-13 12:03:17 +10:00
Joshua Spence
e495ce496b Fix various spelling mistakes
Summary: Minor change, self-explanatory.

Test Plan: Eye-ball it

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9911
2014-07-13 00:45:33 +10:00
epriestley
ae263ddde5 Show a better message for empty repositories and invalid branches
Summary:
Ref T1493.

  - When viewing an invalid branch, show a "there is no such branch" message.
  - When viewing an empty repository, show a "this repository is empty" message.

Test Plan:
  - Viewed empty, bad branch, and nonempty in Git.
  - Viewed empty, bad branch, and nonempty in Mercurial.
  - Viewed empty and nonempty in Subversion.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T1493

Differential Revision: https://secure.phabricator.com/D9912
2014-07-12 07:05:19 -07:00
epriestley
793eced32d Modernize "projects" typeahead datasource
Summary: Ref T4420. Update "projects" source.

Test Plan:
- Edited projects on a Differential revision.
- Edited projects on a commit.
- Edited projects on a repository.
- Edited projects in feed search.
- Edited projects in a Herald rule field.
- Edited projects in a Herald rule action.
- Edited projects in Maniphest batch editor.
- Edited projects on Maniphest task.
- Edited projects in "Associate Projects..." action in Maniphest.
- Edited projects on Maniphest search in "all projects", "any project" and "not projects" fields.
- Edited projects on a Paste.
- Edited projects on a Pholio mock.
- Edited projects on a custom policy rule.
- Edited projects on a Ponder question.
- Edited projects on a Diffusion search query.
- Edited projects on a global search query.
- Edited projects on a slowvote.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4420

Differential Revision: https://secure.phabricator.com/D9884
2014-07-10 17:28:29 -07:00
epriestley
e9dbe747ff Modernize "arcanist project" datasource
Summary: Ref T4420. Do arc projects.

Test Plan:
  - Used Herald typeahead.
  - Used Repositories typehaead.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4420

Differential Revision: https://secure.phabricator.com/D9879
2014-07-10 16:21:10 -07:00
epriestley
a2caea13d6 Modernize "macro" typeahead datasource
Summary: Ref T4420. Make this modern.

Test Plan:
  - Used typeahead in remarkup comment area to select macro "derpdog".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4420

Differential Revision: https://secure.phabricator.com/D9875
2014-07-10 16:18:15 -07:00
epriestley
34628002fd Modernize "repositories" typeahead datasource
Summary:
Ref T4420.

  - Allow tokenizers to accept either a `Datasource` object (new style) or a URI (old style).
  - Read URI and placeholder text from object, if available.
  - Swap the "repositories" datasource (which seemed like the simplest one) over to the new stuff.
  - Tweak/update the repo tokens a little bit.

Test Plan:
  - Used tokenizer in Herald, Differential (search), Differential (edit), Push Logs.
  - Grepped for other callsites.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4420

Differential Revision: https://secure.phabricator.com/D9874
2014-07-10 16:18:04 -07:00
Aviv Eyal
64e2aad46f set localpath in repository.create
Summary: Since there's no way to set it, it defaults to an empty value. Make the conduit call set up sane default.

Test Plan: Call method, repo get's built with expected localpath.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9842
2014-07-09 15:35:25 -07:00
Joshua Spence
8756d82cf6 Remove @group annotations
Summary: I'm pretty sure that `@group` annotations are useless now... see D9855. Also fixed various other minor issues.

Test Plan: Eye-ball it.

Reviewers: #blessed_reviewers, epriestley, chad

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9859
2014-07-10 08:12:48 +10:00
epriestley
455ac8ec4d Use slim tags in ObjectList
Summary: Toss the hard-codes and use slim tags.

Test Plan: Scoped out task list.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9748
2014-06-26 15:23:48 -07:00
epriestley
d541664956 Fixes T4990
Summary: Fix for T4990, using export TERM directly in pre receive hook, tested for git

Test Plan:
pushing into repository over ssh will now not cause remote warning
No entry for terminal type "unknown";
using dumb terminal settings.

Tested with git

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Projects: #diffusion, #repositories

Maniphest Tasks: T4990

Differential Revision: https://secure.phabricator.com/D9744
2014-06-26 09:41:11 -07:00
epriestley
41adc1b4c3 Use standard tag elements instead of "maniphest project tag" to show tags in list views
Summary:
Standardizes tag rendering in Maniphest and Maniphest/Diffusion list views.

(This might need some size/spacing tweaks, I tried to make it look reasonable.)

Test Plan: See screenshots.

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9741
2014-06-26 08:49:44 -07:00
epriestley
40fa4799a3 Recover from null authors in commit parsing
Summary:
Fixes T5445. Some import tools and other unusual situations can leave repositories with commits that don't have authors. This fails on insert.

Instead, explicitly cast the value to a string.

Test Plan: I didn't build a local repro, but see task/GitHub.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5445

Differential Revision: https://secure.phabricator.com/D9684
2014-06-23 07:23:42 -07:00
lkassianik
d5e84cf16b Implement Project Interface in PhabricatorRepository
Summary: T2628, PhabricatoryRepository.php now implements PhabricatorProjectInterface

Test Plan: Verify project tags still work in phabricator repositories

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9683
2014-06-23 07:14:11 -07:00
James Rhodes
f7f8664456 Move build variables into HarbormasterBuildableInterface
Summary: Ref T1049.  This moves the declaration of build variables onto HarbormasterBuildableInterface, allowing new classes implementing HarbormasterBuildableInterface to declare their own variables.

Test Plan: Implemented it on another class, saw the build variables appear.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D9618
2014-06-20 12:58:23 +10:00
Joshua Spence
14e3c727cc Allow Arcanist Projects to be deleted using ./bin/remove destroy.
Summary: Self-explanatory.

Test Plan: Created (and destroyed) an arcanist project. Verified that the deletion actually happened.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Maniphest Tasks: T4749

Differential Revision: https://secure.phabricator.com/D9353
2014-06-16 04:49:51 +10:00
epriestley
d401036bd8 Prevent the use of file:// URIs in Diffusion
Summary:
Via HackerOne. There are two attacks here:

  - Configuring mirroring to a `file://` URI to place files on disk or overwrite another repository. This is not particularly severe.
  - Configuring cloning from a `file://` URI to read repositories you should not have access to. This is more severe.

Historically, repository creation and editing explicitly supported `file://` URIs to deal with use cases where you had something else managing repositories on the same machine. Since there were no permissions, repository management was admin-only, and you couldn't mirror, this was fine.

As we've evolved, this use case is a tiny minority use case and the security implications of `file://` URIs overwhelm the utility it provides. Prevent the use of `file://` URIs. Existing configured repositories won't stop working, you just can't add any new ones.

Also prevent `localPath` from being set via Conduit (see T4039).

Test Plan:
  - Tried to create a `file://` repository.
  - Tried to create a `file://` mirror.
  - Tried to create a `file://` repository via Conduit.
  - Created a non-`file://` repository.
  - Created a non-`file://` mirror.
  - Created a non-`file://` repository via Conduit.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9513
2014-06-13 07:07:00 -07:00
epriestley
b8bc0aa2b0 Allow users to select QueryPanel search engines from a list
Summary: Ref T4986. Instead of requiring users to know the name of an application search engine class, let them select from a list.

Test Plan:
Created a new panel.

{F165468}

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9500
2014-06-12 13:22:20 -07:00
Joshua Spence
0a62f13464 Change double quotes to single quotes.
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.

Test Plan: Eyeballed it.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D9431
2014-06-09 11:36:50 -07:00
Chad Little
41ef6824be Make ObjectItem default as "Card"
Summary: This went smoother than expeced. Makes the rounded Card the default, also tweaked selected state a little.

Test Plan:
Test UIExamples, Maniphest, Home, Differential, Harbormaster, Audit. Everything seems normal

{F163971}

{F163973}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9408
2014-06-07 12:12:11 -07:00
epriestley
40e2a1c800 Write new hunks to the modern hunk store
Summary: Ref T4045. Ref T5179. Send all new writes into the modern store.

Test Plan:
  - Created a diff.
  - Verified it went to the modern store.
  - Destroyed a revision, verified hunks were destroyed.
  - Also unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4045, T5179

Differential Revision: https://secure.phabricator.com/D9293
2014-06-03 18:01:25 -07:00
epriestley
0aa913805d Add an alternate "modern" hunk datastore
Summary:
Ref T4045. Ref T5179. Hunk storage has two major issues:

  - It's utf8, but actual diffs are binary.
  - It's huge and can't be compressed or archived.

This introduces a second datastore which solves these problems: by recording hunk encoding, supporting compression, and supporting alternate storage. There's no actual compression or storage support yet, but there's space in the table for them.

Since nothing actually uses hunk IDs, it's fine to have these tables exist at the same time and use the same IDs. We can migrate data between the tables gradually without requiring downtime or disrupting installs.

Test Plan:
  - There are no writes to the new table yet.
  - The only effect this has is making us issue one extra query when looking for hunks.
  - Observed the query issue, but everything else continue working fine.
  - Created a new diff.
  - Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4045, T5179

Differential Revision: https://secure.phabricator.com/D9290
2014-06-03 18:01:22 -07:00
epriestley
cb58acbe5b Use DiffQuery instead of direct hunk loads in MessageParser daemon
Summary: Ref T5179. Ref T4045. Continue reducing the number of direct hunk loads we perform.

Test Plan: Pushed a closing commit, used `scripts/repository/reparse.php --message ...` to trigger this logic, got a sensible/accurate result.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4045, T5179

Differential Revision: https://secure.phabricator.com/D9288
2014-06-03 18:01:20 -07:00
Joshua Spence
1503840cd9 Batch up SQL operations in the ./bin/repository parents script.
Summary: Fixes T5255. Currently the `./bin/repository parents` workflow is quite slow. Batching up the SQL operations should make the workflow //seem// much faster.

Test Plan: Not yet tested.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5255

Differential Revision: https://secure.phabricator.com/D9361
2014-06-03 11:27:57 -07:00
Joshua Spence
0d03bbe43c Allow repositories to be deleted using ./bin/remove.
Summary: Currently, repositories can be deleted using `./bin/repository delete`. It makes sense to expose this operate to the `./bin/remove` script as well, for consistency.

Test Plan: Deleted a repository with `./bin/remove rTEST`.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D9350
2014-06-02 17:11:58 -07:00
epriestley
8ea9935ea5 Deduplicate parents from PhabricatorGitGraphStream
Summary: Fixes T5226. It's rare (but possible) for a commit to have the same parent more than once in Git.

Test Plan: Ran `bin/repository parents` on a normal repository.

Reviewers: joshuaspence

Reviewed By: joshuaspence

Subscribers: epriestley

Maniphest Tasks: T5226

Differential Revision: https://secure.phabricator.com/D9344
2014-06-02 15:25:28 -07:00
epriestley
81d95cf682 Make default view of "Applications" app a full-page launcher
Summary:
This probably needs some tweaks, but the idea is to make it easier to browse and access applications without necessarily needing them to be on the homepage.

Open to feedback.

Test Plan:
(This screenshot merges "Organization", "Communication" and "Core" into a single "Core" group. We can't actually do this yet because it wrecks the homepage.)

{F160052}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T5176

Differential Revision: https://secure.phabricator.com/D9297
2014-05-29 12:17:54 -07:00
Joshua Spence
2f668493a0 Don't attempt to discover parents commits for untracked branchs.
Summary: Fixes T5195. Currently, the `./bin/repository parents` workflow doesn't respect tracked branches and will attempt to build parents caches for all branches.

Test Plan: For at least one of our repositories, this patch fixes the `Unknown commit` exception. Unfortunately, it doesn't seem to completely solve this problem though, but I suspect that this is due to commits that were overwritten with a `git push --force` or similar.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T5195

Differential Revision: https://secure.phabricator.com/D9322
2014-05-29 12:02:37 -07:00
epriestley
63ed126b2a Point github.com/facebook URIs at github.com/phacility insead
Summary: Point everything at the new canonical URI.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D9328
2014-05-29 08:33:25 -07:00
epriestley
a74545c9da Provide a rough, unstable API for reporting coverage into Diffusion
Summary:
Ref T4994. This stuff works:

  - You can dump a blob of coverage information into `diffusion.updatecoverage`. This wipes existing coverage information and replaces it.
  - It shows up when viewing files.
  - It shows up when viewing commits.

This stuff does not work:

  - When viewing files, the Javascript hover interaction isn't tied in yet.
  - We always show this information, even if you're behind the commit where it was generated.
  - You can't do incremental updates.
  - There's no aggregation at the file (this file has 90% coverage), diff (the changes in this commit are 90% covered), or directory (the code in this directory has 90% coverage) levels yet.
  - This is probably not the final form of the UI, storage, or API, so you should expect occasional changes over time. I've marked the method as "Unstable" for now.

Test Plan:
  - Ran `save_lint.php` to check for collateral damage; it worked fine.
  - Ran `save_lint.php` on a new branch to check creation.
  - Published some fake coverage information.
  - Viewed an affected commit.
  - Viewed an affected file.

{F151915}

{F151916}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley, zeeg

Maniphest Tasks: T5044, T4994

Differential Revision: https://secure.phabricator.com/D9022
2014-05-17 16:10:54 -07:00
epriestley
baa6441668 Remove some needless +x flags. 2014-05-17 15:41:12 -07:00
epriestley
4aea31023d Index inline comments on commits
Summary: Fixes T3044. We currently don't add these to the index.

Test Plan: Made a unique inline comment on a commit, then searched for it.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T3044

Differential Revision: https://secure.phabricator.com/D9170
2014-05-17 15:31:53 -07:00
epriestley
15561a27c3 When a conduit method requires a string constant, call it "string-const" not "enum"
Summary: Ref T5058. The use of "enum" is confusing; we mean "choose one of these specific string constants". Make this more clear.

Test Plan: Viewed each call from the web UI.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5058

Differential Revision: https://secure.phabricator.com/D9127
2014-05-14 21:59:03 -07:00
epriestley
436f0563e8 Add a SublimeText-style repository typeahead
Summary:
Allows you to quickly search for files within a repository. Roughly:

  - We build a big tree of everything and ship it to the client.
  - The client implements a bunch of Sublime-ish magic to find paths.

Test Plan: {F154007}

Reviewers: chad, btrahan

Reviewed By: btrahan

Subscribers: epriestley, zeeg

Differential Revision: https://secure.phabricator.com/D9087
2014-05-13 14:08:21 -07:00
epriestley
82102cd95a Move Push log rendering to SearchEngine
Summary: Ref T4986. Move push logs to a View, then have all the stuff that needs to use it use that View.

Test Plan: Viewed push logs and transaction detail in Diffusion. Created a panel.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9104
2014-05-13 14:00:24 -07:00
epriestley
84fc8f0baf Don't try to rebuild a repository graph cache bucket more than once per request
Summary: Ref T2683. This is a small optimization, but it has low complexity: don't rebuild a bucket more than once in the same request, since it will almost always be the same. Bucket rebuilds are pretty cheap, but this saves a few queries.

Test Plan:
  - After discovering (but before parsing) a commit, viewed its browse view. Verified that this patch causes us to perform only one bucket rebuild, and therefore reduces the number of queries we issue.
  - Parsed the commit and viewed the browse view again, got successful rebuild and then fills from cache.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9055
2014-05-12 11:47:32 -07:00
epriestley
cfa265f020 Make sure READMEs can hit the markup cache in Diffusion
Summary: Ref T2683. Normally not a big deal, but if a readme has some codeblocks missing the cache can slow things down.

Test Plan:
  - Verified we hit the cache.
  - Verified TOC still works.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T5028, T2683

Differential Revision: https://secure.phabricator.com/D9049
2014-05-12 11:47:31 -07:00
epriestley
e13369d208 Use RepositoryGraphCache to service diffusion.lastmodifiedquery
Summary:
Ref T2683. At least locally, browse views are now nearly instantaneous, even in Mercurial. We also fall back to what we were doing before if we miss or take too long, so this shouldn't make things very much worse even in extreme cases.

For a local `hg` repo, the time we spend pulling browse stuff has dropped from ~3,000ms to ~20ms. This is probably atypical, but not completely crazy or rigged or anything.

Test Plan: Viewed Git, Subversion and Mercurial repositories and observed dramatically better performance in Git and Mercurial as they took advantage of the cache.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, jhurwitz

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9047
2014-05-12 11:47:29 -07:00
epriestley
e4ea092f60 Implement a chunked, APC-backed graph cache
Summary:
Ref T2683. This is a refinement and simplification of D5257. In particular:

  - D5257 only cached the commit chain, not path changes. This meant that we had to go issue an awkward query (which was slow on Facebook's install) periodically while reading the cache. This was reasonable locally but killed performance at FB scale. Instead, we can include path information in the cache. It is very rare that this is large except in Subversion, and we do not need to use this cache in Subversion. In other VCSes, the scale of this data is quite small (a handful of bytes per commit on average).
  - D5257 required a large, slow offline computation step. This relies on D9044 to populate parent data so we can build the cache online at will, and let it expire with normal LRU/LFU/whatever semantics. We need this parent data for other reasons anyway.
  - D5257 separated graph chunks per-repository. This change assumes we'll be able to pull stuff from APC most of the time and that the cost of switching chunks is not very large, so we can just build one chunk cache across all repositories. This allows the cache to be simpler.
  - D5257 needed an offline cache, and used a unique cache structure. Since this one can be built online it can mostly use normal cache code.
  - This also supports online appends to the cache.
  - Finally, this has a timeout to guarantee a ceiling on the worst case: the worst case is something like a query for a file that has never existed, in a repository which receives exactly 1 commit every time other repositories receive 4095 commits, on a cold cache. If we hit cases like this we can bail after warming the cache up a bit and fall back to asking the VCS for an answer.

This cache isn't perfect, but I believe it will give us substantial gains in the average case. It can often satisfy "average-looking" queries in 4-8ms, and pathological-ish queries in 20ms on my machine; `hg` usually can't even start up in less than 100ms. The major thing that's attractive about this approach is that it does not require anything external or complicated, and will "just work", even producing reasonble improvements for users without APC.

In followups, I'll modify queries to use this cache and see if it holds up in more realistic workloads.

Test Plan:
  - Used `bin/repository cache` to examine the behavior of this cache.
  - Did some profiling/testing from the web UI using `debug.php`.
  - This //appears// to provide a reasonable fast way to issue this query very quickly in the average case, without the various issues that plagued D5257.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley, jhurwitz

Maniphest Tasks: T2683

Differential Revision: https://secure.phabricator.com/D9045
2014-05-12 11:47:23 -07:00
epriestley
95eab2f3b0 Record parent relationships when discovering commits
Summary:
Ref T4455. This adds a `repository_parents` table which stores `<childCommitID, parentCommitID>` relationships.

For new commits, it is populated when commits are discovered.

For older commits, there's a `bin/repository parents` script to rebuild the data.

Right now, there's no UI suggestion that you should run the script. I haven't come up with a super clean way to do this, and this table will only improve performance for now, so it's not important that we get everyone to run the script right away. I'm just leaving it for the moment, and we can figure out how to tell admins to run it later.

The ultimate goal is to solve T2683, but solving T4455 gets us some stuff anyway (for example, we can serve `diffusion.commitparentsquery` faster out of this cache).

Test Plan:
  - Used `bin/repository discover` to discover new commits in Git, SVN and Mercurial repositories.
  - Used `bin/repository parents` to rebuild Git and Mercurial repositories (SVN repos just exit with a message).
  - Verified that the table appears to be sensible.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: jhurwitz, epriestley

Maniphest Tasks: T4455

Differential Revision: https://secure.phabricator.com/D9044
2014-05-12 11:47:22 -07:00
epriestley
e6aff100f2 Move even more rendering into SearchEngine
Summary: Ref T4986. I think this is the last of the easy ones, there are about 10 not-quite-so-trivial ones left.

Test Plan:
  - Viewed app results.
  - Created panels.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9025
2014-05-09 12:28:02 -07:00
epriestley
78b89711cb Move a bunch more rendering into SearchEngine
Summary: Ref T4986. These are mostly mechanical now, I skipped a couple of slightly tricky ones. Still a bunch to go.

Test Plan:
For each engine:

  - Viewed the application;
  - created a panel to issue the query.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4986

Differential Revision: https://secure.phabricator.com/D9017
2014-05-08 20:04:19 -07:00
epriestley
7145587df7 Lock down some config options
Summary:
This is just a general review of config options, to reduce the amount of damage a rogue administrator (without host access) can do. In particular:

  - Fix some typos.
  - Lock down some options which would potentially let a rogue administrator do something sketchy.
    - Most of the new locks relate to having them register a new service account, then redirect services to their account. This potentially allows them to read email.
    - Lock down some general disk stuff, which could be troublesome in combination with other vulnerabilities.

Test Plan:
  - Read through config options.
  - Tried to think about how to do evil things with each one.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8928
2014-05-01 10:23:49 -07:00
austinkelleher
eec0576793 Make the hard limit on the number of files showing in Herald emails a constant.
See: <https://github.com/facebook/phabricator/pull/587>

Reviewed by: epriestley
2014-05-01 07:37:26 -07:00
epriestley
8fe27800fc Don't show document types in search for uninstalled applications
Summary:
Fixes T4917. Currently, if a user doesn't have access to, e.g., Phriction, they still get a checkbox in the search results to search for Wiki Documents. Those results will be filtered anyway, so this is confusing at best.

Instead, bind PHID types to applications. This is a relatively tailored fix; some areas for potential future work:

  - Go through every PHID type and bind them all to applications. Vaguely nice to have, but doesn't get us anything for now.
  - If no searchable application is installed, we don't show you an error state. This isn't currently possible ("People" is always installed) but in the interest of generality we could throw an exception or something at least.
  - The elasticserach thing could probably constrain types to visible types, but we don't have a viewer there easily right now.

Test Plan: Uninstalled Phriction, saw the checkbox vanish.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4917

Differential Revision: https://secure.phabricator.com/D8904
2014-04-29 15:01:50 -07:00
epriestley
692a28b5b2 Unfatal rendering of repository policy transactions
Summary: Fixes T4919. There's some special casing in Diffusion for CAN_PUSH right now, just accommodate that until things get more general.

Test Plan: Viewed a repository edit screen with a custom policy transaction. Clicked the link to view it.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4919

Differential Revision: https://secure.phabricator.com/D8898
2014-04-29 10:57:32 -07:00
Peng Li
3a0694543d Put a limit on the number of files showing in herald emails
Summary:
Sometimes a commit can be huge (like a branch cut in FB www which could have more than half a million files touched). It will generate some emails with size more than 30M, and it will take quite a while to just sort the files and to send out.
Put a hard limit here to avoid such cases. Probably only matters for FB right now, but still even for a small repo with several thousand files, it is a waste to send them all out. Not sure if there is any cleaner way to do it though.

Test Plan: Tried it in FB installtion.

Reviewers: lifeihuang, epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D8889
2014-04-29 10:38:39 -07:00
Bob Trahan
2ecc04c159 Audit - move over to application search
Summary: ...also kills off "PhabricatorAuditCommitQuery" and "PhabricatorAuditQuery", by moving the work to "DiffusionCommitQuery". Generally cleans up some code around the joint on this too. Also provides policies for audit requests, which is basically the policy for the underlying commit. Fixes T4715. (For the TODO I added about files, I just grabbed T4713.)

Test Plan:
Audit: verified the three default views all showed the correct things, including highligthing. did some custom queries and got the correct results.
Diffusion: verified "blame view" still worked. verified paths were highlighted for packages i owned.
Home: verified audit boxes showed up with proper commits w/ audits
bin/audit: played around with it via --dry-run and got the right audits back

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: chad, epriestley, Korvin

Maniphest Tasks: T4715

Differential Revision: https://secure.phabricator.com/D8805
2014-04-27 09:43:05 -07:00
Chad Little
11fd6afeb1 Move Timeline icons to Fonts
Summary: Throwing this up for testing, swapped out all icons in timeline for their font equivelants. Used better icons where I could as well. We should feel free to use more / be fun with the icons when possible since there is no penalty anymore.

Test Plan: I browsed many, not all, timelines in my sandbox and in IE8. Some of these were just swagged, but I'm expecting we'll do more SB testing before landing.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D8827
2014-04-22 08:25:54 -07:00
austinkelleher
2e5065feb5 Update function name to follow naming convention.
See: <http://github.com/facebook/phabricator/pull/575>

Reviewed by: epriestley
2014-04-20 08:37:37 -07:00
epriestley
c402d7d307 Implement smart waits for rarely updated repositories
Summary:
Ref T4605. When figuring out how long to wait to update a repository, factor in when it was last pushed. For rarely updated repositories, wait longer between updates.

(A slightly funky thing about this is that empty repos update every 15 seconds, but that seems OK for the moment.)

Test Plan:
Ran `bin/phd debug pulllocal` and saw sensible calculations and output:

```
...
<VERB> PhabricatorRepositoryPullLocalDaemon Last commit to repository "rPOEMS" was 1,239,608 seconds ago; considering a wait of 6,198 seconds before update.
>>> [79] <query> SELECT * FROM `repository` r   ORDER BY r.id DESC
<<< [79] <query> 514 us
>>> [80] <query> SELECT * FROM `repository_statusmessage` WHERE statusType = 'needs-update'
<<< [80] <query> 406 us
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIH" is not due for an update for 8,754 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rDUCK" is not due for an update for 14 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rMTESTX" is not due for an update for 21,598 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rQWER" is not due for an update for 14 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rBT" is not due for an update for 13 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rSVNX" is not due for an update for 21,598 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIG" is not due for an update for 13 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rHGTEST" is not due for an update for 21,598 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rBTX" is not due for an update for 14 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rGX" is not due for an update for 13 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rMTX" is currently updating.
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rPOEMS" is not due for an update for 6,198 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rPHU" is currently updating.
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rSVN" is not due for an update for 21,598 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rPHY" is currently updating.
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rGTEST" is not due for an update for 21,598 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIS" is not due for an update for 6,894 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rARCLINT" is not due for an update for 21,599 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rLPHX" is not due for an update for 1,979 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rARC" is not due for an update for 1,824 second(s).
<VERB> PhabricatorRepositoryPullLocalDaemon Repository "rINIHG" is not due for an update for 21,599 second(s).
...
```

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4605

Differential Revision: https://secure.phabricator.com/D8782
2014-04-16 13:01:04 -07:00
epriestley
5671c4b276 Make PullLocal daemon more flexible and transparent about scheduling
Summary:
Ref T4605. Fixes T3466. The major change here is that we now run up to four simultaneous updates. This should ease cases where, e.g., one very slow repository was blocking other repositories. It also tends to increase load; the next diff will introduce smart backoff for cold repositories to ease this.

The rest of this is just a ton of logging so I can IRC debug these things by having users run them in `phd debug pulllocal` mode.

For T3466:

  - You now have to hit four simultaneous hangs to completely block the update process.
  - Importing repository updates are killed after 4 hours.
  - Imported repository updates are killed after 15 minutes.

Test Plan:
  - Ran `phd debug pulllocal` and observed sensible logs and behavior.
  - Interrupted daemon from sleeps and processing with `diffusion.looksoon`.
  - Ran with various `--not`, `--no-discovery` flags.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3466, T4605

Differential Revision: https://secure.phabricator.com/D8785
2014-04-16 13:00:49 -07:00
epriestley
417056932e Make discovery slightly cheaper in the common case
Summary:
Ref T4605. Before discovering branches, try to prefill the cache in bulk. For repositories with large numbers of branches, this allows us to issue dramatically fewer queries.

(Before D8780, this cache was usually held across discovery events, so being able to fill it cheaply was not as relevant.)

Test Plan: Ran discovery on Git, Mercurial and SVN repositories. Observed fewer queries for Git/Mercurial.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4605

Differential Revision: https://secure.phabricator.com/D8781
2014-04-16 13:00:38 -07:00
epriestley
118c696f72 Separate repository updates from the pull daemon
Summary:
Ref T4605. Currently, the PullLocal daemon is responsible for two relatively distinct things:

  - scheduling repository updates; and
  - actually updating repositories.

Move the "actually updating" part into a new `bin/repository update` command, which basically runs the pull, discover, refs and mirror commands. This will let the parent process focus on scheduling in a more understandable way and update multiple repositories at once. It also makes it easier to debug and understand update behavior since the non-scheduling pipeline can be run separately.

Test Plan:
  - Ran `update --trace` on SVN, Mercurial and Git repos.
  - Ran PullLocal daemon for a while without issues.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4605

Differential Revision: https://secure.phabricator.com/D8780
2014-04-16 13:00:29 -07:00
epriestley
cb545856a9 Make task queue more robust against long-running tasks
Summary:
See discussion in D8773. Three small adjustments which should help prevent this kind of issue:

  - When queueing followup tasks, hold them on the worker until we finish the task, then queue them only if the work was successful.
  - Increase the default lease time from 60 seconds to 2 hours. Although most tasks finish in far fewer than 60 seconds, the daemons are generally stable nowadays and these short leases don't serve much of a purpose. I think they also date from an era where lease expiry and failure were less clearly distinguished.
  - Increase the default wait-after-failure from 60 seconds to 5 minutes. This largely dates from the MetaMTA era, where Facebook ran services with high failure rates and it was appropriate to repeatedly hammer them until things went through. In modern infrastructure, such failures are rare.

Test Plan:
  - Verified that tasks queued properly after the main task was updated.
  - Verified that leases default to 7200 seconds.
  - Intentionally failed a task and verified default 300 second wait before retry.
  - Removed all default leases shorter than 7200 seconds (there was only one).
  - Checked all the wait before retry implementations for anything much shorter than 5 minutes (they all seem reasonable).

Reviewers: btrahan, sowedance

Reviewed By: sowedance

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D8774
2014-04-15 08:42:02 -07:00
Peng Li
6a4f126000 Give the commitownersparser a little more time
Summary:
Recently we see issues with huge commits (branch cuts for www) where people received hundreds of emails for the same commit. By checking all the active and archived tasks related to such commits, I saw the following pattern:
 - The commit itself is marked as importStatus = 15 which means all the processing was actually done;
 - In archived tasks, I see one PhabricatorRepositorySvnCommitMessageParserWorker, one PhabricatorRepositorySvnCommitChangeParserWorker, followed by many PhabricatorRepositoryCommitHeraldWorker, which means that the PhabricatorRepositoryCommitOwnersWorker (who schedule those herald tasks) was never done;
 - PhabricatorRepositoryCommitOwnersWorker is always active (for days) with failureCount = 0;
 - In daemon log I see a lot of lease expire exception for PhabricatorRepositoryCommitOwnersWorker.
So to me it looks like the following happened:
 - Everything is fine until we schedule the PhabricatorRepositoryCommitOwnersWorker
 - PhabricatorRepositoryCommitOwnersWorker actually successfully finished but its running time exceed 60s. Before it finishes, it scheduled the PhabricatorRepositoryCommitHeraldWorker task
 - When we try to archive it, the lease expiration exception happened. As a result, it stayed active and will be picked up immediately since it is in the head of the queue
 - The two steps above repeat forever until we kill it
I am not sure why we want to check lease expiration when we are archiving the task. For now I am giving the worker a little more time since parsing half million affected path needs some time..

Test Plan: Patched in our production and it worked.

Reviewers: lifeihuang, JoelB, #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D8773
2014-04-14 15:52:02 -07:00
Bob Trahan
d5ded805b2 Herald - fix change type bug
Summary: wasn't working due to some type issues. Fixes T4756. I also made it display nicer while I was debugging this.

Test Plan: created a herald rule to block changes that added refs. git tag -a "test" -m "test test"; git push origin test got me blocked!

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin

Maniphest Tasks: T4756

Differential Revision: https://secure.phabricator.com/D8724
2014-04-08 11:58:28 -07:00
epriestley
bd0011076b Don't disrupt repository import chain when publishing is disabled
Summary:
Fixes T4736. Currently, we incorrectly skip the `writeImportStatusFlag()` call if publishing is disabled (the `herald-disabled`) check. This means we don't flag the commit as imported, and don't move the pipeline forward correctly.

Instead, we only want to skip the owners stuff, not the pipeline stuff. Move that to a method.

(Also fix a nearby TODO now that we have a permanent failure exception.)

Test Plan:
  - Used `scripts/repository/reparse.php --owners ...` to execute this code, fiddled with things to hit both the disabled and enabled branches and verified the flag stuff is still reached.
  - Faked the exceptions and made sure they raise correctly.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4736

Differential Revision: https://secure.phabricator.com/D8715
2014-04-08 05:13:28 -07:00
Bob Trahan
c6cbff1997 Differential - modernize "Local Commits" table
Summary: ...also link to commits we know about in "Local Commits" and "Revision Update History" tables. Fixes T4585.

Test Plan: made a repo. made a diff (foo) and committed it (bar). made a new diff that was comprised of two local commits. noted links to (bar) in various commit hashes as expected

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: epriestley, Korvin, chad

Maniphest Tasks: T4585

Differential Revision: https://secure.phabricator.com/D8679
2014-04-02 13:18:11 -07:00
epriestley
1aad40b7bf Allow users to receive email about pushes via Herald
Summary:
Fixes T4677. Implements a "send an email" pre-receive action, which sends push summaries.

For use cases where features are often pushed as a large number of commits (e.g., checkpoint commits are retained), using commit emails means users get a ton of email. Instead, this allows you to get an email about a push, which summarizes what changed.

Overall, this is basically the same as commit email, but more suitable for some workflows.

Test Plan:
Wrote some rules, then made a bunch of pushes. Got email like this:

{F134929}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4677

Differential Revision: https://secure.phabricator.com/D8618
2014-03-26 13:51:15 -07:00
epriestley
75c47c6ae0 Provide an "event" page for push logs, which shows details on all events in a given push
Summary:
Ref T4677. This shows a more detailed view of an entire "git push", "hg push", or "svn commit".

This is mostly to give push summary emails a reasonable, stable URI to link to for T4677.

Test Plan:
  - Pushed into SVN, Git and Mercurial.
  - Viewed partial and imported event records.

{F134864}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4677

Differential Revision: https://secure.phabricator.com/D8616
2014-03-26 13:51:09 -07:00
epriestley
a5f55d506f Provide a real object ("PhabricatorRepositoryPushEvent") to represent an entire push transaction
Summary:
Ref T4677. Currently, we record individual actions in a push as PhabricatorRepositoryPushLogs, but tie them together only loosely with a `transactionKey`.

Provide a real PushEvent object, and move some of the denormalized fields to it. This primarily just gives us more robust infrastructure for building, e.g., email about pushes, for T4677, since we can act on real PHIDs rather than passing awkward identifiers around.

Test Plan:
  - Performed migration.
  - Looked at database for consistency.
  - Browsed/queried push logs.
  - Pushed a bunch of stuff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4677

Differential Revision: https://secure.phabricator.com/D8615
2014-03-26 13:51:06 -07:00
epriestley
c828160c22 Minor, fix a constant in PhabricatorRepositoryEditor
See <https://github.com/facebook/phabricator/pull/554>
2014-03-26 10:45:48 -07:00
epriestley
e8e12910a7 Implement a "credential" standard custom field
Summary: Ref T4590. Ref T1049. This is primarily intended to support HTTP auth in Harbormaster.

Test Plan: Added a field, edited it, etc.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T4590, T1049

Differential Revision: https://secure.phabricator.com/D8607
2014-03-25 16:13:27 -07:00
epriestley
1d30354223 Fix issue with ref/fixes in commit messages
Summary: Fixes T4600. If there's also a revision, the variable "$message" gets overwritten. groan~

Test Plan: Pushed a commit with "Fixes T123" and a revision, saw it parse on the first try.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: chrisbolt, aran, epriestley

Maniphest Tasks: T4600

Differential Revision: https://secure.phabricator.com/D8519
2014-03-13 17:47:11 -07:00
epriestley
969d0c3e8d Use "\z" instead of "$" to anchor validating regular expressions
Summary:
Via HackerOne. In regular expressions, "$" matches "end of input, or before terminating newline". This means that the expression `/^A$/` matches two strings: `"A"`, and `"A\n"`.

When we care about this, use `\z` instead, which matches "end of input" only.

This allowed registration of `"username\n"` and similar.

Test Plan:
  - Grepped codebase for all calls to `preg_match()` / `preg_match_all()`.
  - Fixed the ones where this seemed like it could have an impact.
  - Added and executed unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: aran, epriestley

Differential Revision: https://secure.phabricator.com/D8516
2014-03-13 12:42:41 -07:00
epriestley
d27cd5fb99 Disable Herald more aggressively when it's turned off for a repository
Summary:
Currently, disabling Herald only disables feed, notifications and email. Historically, audits didn't really create external effects so it made sense for Herald to only partially disable itself.

With the advent of Harbormaster/Build Plans, it makes more sense for Herald to just stop doing anything. When this option is disabled, stop all audit/build/publish/feed/email actions for the repository.

Test Plan: Ran `scripts/repository/reparse.php --herald`, etc.

Reviewers: dctrwatson, btrahan

Reviewed By: btrahan

Subscribers: aran, epriestley

Differential Revision: https://secure.phabricator.com/D8509
2014-03-12 18:16:50 -07:00
epriestley
a4a4322d7a Add a "tags" field to Diffusion commit
Summary:
  - Fixes T4588.
  - See D8501.
  - Adds a "Tags" field for Herald commit emails.
  - Fixes a bug in `tagsquery` when filtering by commit name.
  - Make `tagsquery` just return nothing instead of fataling against Mercurial/Subversion.

Test Plan: Used `bin/repository/reparse.php --herald` to exercise this code.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: aran, epriestley

Maniphest Tasks: T4588

Differential Revision: https://secure.phabricator.com/D8502
2014-03-12 11:30:52 -07:00
epriestley
611f670720 Add CustomField to Diffusion and add a "branches" field
Summary:
Ref T4588. Request from @zeeg. Adds a "BRANCHES" field to commit emails, so the branches the commit appears on are shown.

I've implemented this with CustomField, but in a very light way.

Test Plan: Used `scripts/repository/reparse.php --herald` to generate mail, got a BRANCHES section where applicable.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: aran, epriestley, zeeg

Maniphest Tasks: T4588

Differential Revision: https://secure.phabricator.com/D8501
2014-03-12 11:30:33 -07:00
epriestley
592591e715 Clean up various pieces of dead/obsolete Differential code
Summary:
Ref T2222.

  - Removes `DifferentialTasksAttacher`, which has had no callsites for a very long time.
  - Moves `differential.getrevisioncomments` off `DifferentialCommentQuery`.
  - Moves Releeph churn field off `DifferentialCommentQuery`.
  - Removes dead code in `DifferentialRevisionViewController`.
  - Removes `DifferentialException` (no references).
  - Removes `DifferentialRevision->loadComments()` (no callsites).
  - Removes `DifferentialRevision->loadReviewedBy()` (all callsites updated).
  - Removes `DifferentialCommentQuery` (all callsites updated).

Test Plan: Mostly a lot of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8476
2014-03-11 13:02:19 -07:00
epriestley
3f67430f46 Use PhabricatorObjectListQuery in Herald worker
Summary: Ref T2222. Straightforward, just breaks a needless dependency.

Test Plan: Pushed and parsed a commit with "Auditors" in it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8473
2014-03-11 13:02:14 -07:00
epriestley
40b471faea Move "close tasks on commit" code out of field specification stuff
Summary: Ref T2222. There's some magic here, just port it forward in a mostly-reasonable way. This could use some refinement eventually.

Test Plan: Pushed commits with "Fixes" and "Ref" language, used `reparse.php` to trigger the new code. Saw expected updates in Maniphest.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8471
2014-03-11 13:02:12 -07:00
Joshua Spence
e11adc4ad7 Added some additional assertion methods.
Summary:
There are quite a few tests in Arcanist, libphutil and Phabricator that do something similar to `$this->assertEqual(false, ...)` or `$this->assertEqual(true, ...)`.

This is unnecessarily verbose and it would be cleaner if we had `assertFalse` and `assertTrue` methods.

Test Plan: I contemplated adding a unit test for the `getCallerInfo` method but wasn't sure if it was required / where it should live.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8460
2014-03-08 19:16:21 -08:00
epriestley
b04f706c0a Use TransactionEditor when closing revisions in response to commits
Summary: Ref T2222. When we discover a commit associated with a revision, close it using modern transactions.

Test Plan: {F123848}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8441
2014-03-07 17:43:58 -08:00
epriestley
a3c1dcb928 Produce a more tailored checkout URI for Subversion
Summary:
For imported SVN repositories with an "Import Only" path, we produce a `/path/to/root/` URI, but should produce `/path/to/root/then/to/import/only/`.

As it is, the URI instructs the user to check out the whole repository.

Also, don't show the "Clone As" fragment in the URI for remote repositories, and prevent it from being edited for nonhosted repositories. This is generally more consistent with user expectation.

Test Plan:
  - Created a remote SVN repository with "Import Only", saw path include it.
  - Verified no "Clone As" options, no "Clone As" in URI.
  - Switched it to hosted, saw "Clone As" options appear and work properly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, staticshock

Differential Revision: https://secure.phabricator.com/D8375
2014-02-28 13:04:41 -08:00
epriestley
d86bb086ca Reduce initial discovery from O(branches * commits) to O(commits)
Summary:
Fixes T4414. Currently, when we discover a new repository, we do something like this:

  foreach (branch) {
    foreach (commit on this branch) {
      do_something();
    }
  }

In cases where there are a lot of branches which mostly just branch `master`, this leads to us doing roughly `O(branches * commits)` work.

We have a `commitCache` to prevent this, but it has two problems:

  - It only fills out of the DB, and we do this whole thing before writing to the DB, which is the entire point.
  - It has a fixed size (2048) and on initial discovery we're usually dealing with way more commits than that.

Instead, also stop doing work if we hit a commit which is known already.

Test Plan:
  - Added `print` on the number of discovered refs and number of unique refs.
  - Ran `bin/repository discover --repair X` on a repo with several branches.
  - Before the patch, got 397 refs and 135 unique refs.
  - After the patch, got 135 refs and 135 unique refs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4414

Differential Revision: https://secure.phabricator.com/D8374
2014-02-28 13:02:14 -08:00
epriestley
e28b78f5eb Fix two issues with ref discovery
Summary:
See IRC. I'm having trouble figuring out what's going on with b4taylor's report, but fix two possible issues:

  # The commit query is missing a `repositoryID`, which could cause issues if you import two copies of the same repository.
  # I think we may try to close commits on untracked branches right now, as long as they aren't excluded by other autoclose rules.

Test Plan: Ran `bin/repository refs` on a few repos.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, brennantaylor

Differential Revision: https://secure.phabricator.com/D8373
2014-02-28 12:56:18 -08:00
Joshua Spence
6270114767 Various linter fixes.
Summary:
- Removed trailing newlines.
- Added newline at EOF.
- Removed leading newlines.
- Trimmed trailing whitespace.
- Spelling fix.
- Added newline at EOF

Test Plan: N/A

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: hach-que, chad, Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8344
2014-02-26 12:44:58 -08:00
epriestley
0f0673b9e5 Remove "dateCommitted" field from DifferentialRevision
Summary: Ref T2222. This is obsolete and no longer used. We could deduce it from transactions or commits in modern Phabricator if we wanted it. We may implement a more general mechanism for T4434.

Test Plan: `grep`

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D8330
2014-02-25 12:36:14 -08:00
epriestley
763a049f74 Possibly fix issue where refs engine finds empty refs?
Summary:
@mbishopim3 reported an issue in IRC:

> mbishopim3: epriestley: "Error updating working copy: Commit "" has not been discovered yet! Run discovery before updating refs." any ideas?

I can't reproduce it and it went away for him, but one theory is that we're getting here and git/hg are spitting out nothing, which we incorrectly parse as `array("")` when we intend `array()`.

Test Plan:
Pushed some new commits, ran `bin/repositoy refs X`, got expected results.

I can't actually reproduce the bug, but this might fix it and appears to make the code more correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: mbishopim3, aran

Differential Revision: https://secure.phabricator.com/D8326
2014-02-24 14:39:44 -08:00
epriestley
a298a79bda Convert Phabricator to handle "%s" / "%B" properly
Summary:
Ref T1191. I believe we only have three meaningful binary fields across all applications:

  - The general cache may contain gzipped content.
  - The file storage blob may contain arbitrary binary content.
  - The Passphrase secret can store arbitrary binary data (although it currently never does).

This adds Lisk config for binary fields, and uses `%B` where necessary.

Test Plan:
  - Added and executed unit tests.
  - Forced file uploads to use MySQL, uploaded binaries.
  - Disabled the CONFIG_BINARY on the file storage blob and tried again, got an appropraite failure.
  - Tried to register with an account containing a G-Clef, and was stopped before the insert.

Reviewers: btrahan, arice

Reviewed By: arice

CC: arice, chad, aran

Maniphest Tasks: T1191

Differential Revision: https://secure.phabricator.com/D8316
2014-02-23 16:20:46 -08:00
epriestley
88227d26bc Allow CustomField to provide ApplicationTransaction change details
Summary:
Ref T3886. Ref T418. For fields like "Summary" and "Test Plan" where changes can't be summarized in one line, allow CustomField to provide a "(Show Details)" link and render a diff.

Also consolidate some of the existing copy/paste, and simplify this featuer slightly now that we've move to dialogs.

Test Plan:
{F115918}

  - Viewed "description"-style field changes in phlux, pholio, legalpad, maniphest, differential, ponder (questions), ponder (answers), and repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3886, T418

Differential Revision: https://secure.phabricator.com/D8284
2014-02-21 11:53:04 -08:00
John Watson
1845798771 Fix call to getRefType on non-object in PhabricatorRepositoryEngine.php
Summary:
Error:
  Fatal error: Call to a member function getRefType() on a non-object in /opt/phabricator/phabricator/src/applications/repository/engine/PhabricatorRepositoryRefEngine.php on line 197

Test Plan: No more error in daemon.log afterwards

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D8278
2014-02-19 11:55:46 -08:00
epriestley
b96ab5aadf Modernize VCS password storage to use shared hash infrastructure
Summary: Fixes T4443. Plug VCS passwords into the shared key stretching. They don't use any real stretching now (I anticipated doing something like T4443 eventually) so we can just migrate them into stretching all at once.

Test Plan:
  - Viewed VCS settings.
  - Used VCS password after migration.
  - Set VCS password.
  - Upgraded VCS password by using it.
  - Used VCS password some more.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4443

Differential Revision: https://secure.phabricator.com/D8272
2014-02-18 14:09:36 -08:00
Chris Colborne
ac09a1a245 Fix Create New Repository link on Repositories
Summary: See <https://github.com/facebook/phabricator/pull/510>.

After the repository rework, the Create New Repository link in Repositories goes straight to creating a phabricator hosted repo(diffusion/create), rather than the chooser create/import (diffusion/new)

I updated it to point to diffusion/new the same as the New Repository link in Diffusion.

As a side note, the Repositories page could probably use the Crumbs treatment.

Reviewed by: epriestley
2014-02-16 20:12:09 -08:00
epriestley
e42a982445 When creating foreign stubs during SVN import, mark them imported
Summary: See <https://github.com/facebook/phabricator/issues/501>. I think the issue here is that we created a foreign stub for commit `X-1`, probably because commit `X` was created by running `svn cp y x`.

Test Plan: I'll write a separate test for this before I land it. Huge pain to test.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8133
2014-02-03 12:21:03 -08:00
Aviv Eyal
5d1489a3ce Search repos by Project
Summary:
Ref T3102

In diffusion, add "In Any Project" to search options.

Test Plan: use it.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T3102

Differential Revision: https://secure.phabricator.com/D8113
2014-01-30 11:45:43 -08:00
epriestley
c41b4cfac0 Allow Git and Mercurial repositories to be cloned with names in the URI
Summary:
Ref T4175. This allows these URIs to all be valid for Git and Mercurial:

  /diffusion/X/
  /diffusion/X/anything.git
  /diffusion/X/anything/

This mostly already works, it just needed a few tweaks.

Test Plan: Cloned git and hg working copies using HTTP and SSH.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4175

Differential Revision: https://secure.phabricator.com/D8098
2014-01-30 11:42:25 -08:00
epriestley
ffeee37810 Add "Clone As" to repositories and generate full clone commands in UI
Summary:
Ref T4175.

  - Add a configurable name for the clone-as directory, so you can have "Bits & Pieces" clone as "bits~n~pieces/" or simliar.
  - By default, use "reasonable" heruistics to choose such a name.
  - Generate a copy/pasteable clone commmand with this directory name.

Test Plan: Looked at some repositories.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4175

Differential Revision: https://secure.phabricator.com/D8097
2014-01-30 11:42:10 -08:00
epriestley
96dd530c44 Distinguish between "Remote URI" and "Clone URI" in Repositories
Summary:
Hosted repositories have muddied this distinction somewhat. In some cases, we only want to use the real remote URI, and the call is only relevant for imported repositories.

In other cases, we want the URI we'd plug into `git clone`.

Move this logic into `PhabricatorRepository` and make the distinction more clear.

Test Plan: Viewed SVN, Git, and Mercurial hosted and remote repositories, all the URIs looked reasonable.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, dctrwatson

Differential Revision: https://secure.phabricator.com/D8096
2014-01-30 11:41:21 -08:00
epriestley
f585f6fddd Enable "Allow Dangerous Changes" config for Mercurial repositories
Summary:
Fixes T4357. The Mercurial commit hooks enforce dangerous change protection, but the UI doesn't actually let you configure it.

This was just an oversight (I never went back and enabled it) -- allow it to be configured in the UI.

Test Plan: Clicked "Edit Repository" on a hosted Mercurial repository, saw option to enable dangerous changes.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T4357

Differential Revision: https://secure.phabricator.com/D8108
2014-01-30 09:45:03 -08:00
epriestley
4a6239d53c Fix an issue which could prevent new repositories from being marked imported. 2014-01-30 09:43:53 -08:00
Richard van Velzen
28fcb5711b Add basic support for mirroring Mercurial repositories
Summary:
Ref T4338. Mercurial exits with exit code 1 and "no changes found" in stdout
when there's no changes. I've split up the `pushRepositoryToMirror` to make
the code a tad more readable.

It isn't perfect, but it works for me.

Test Plan:
pushed some changes to my hosted repo. Saw them appearing in the
mirrored repo

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4338

Differential Revision: https://secure.phabricator.com/D8107
2014-01-30 08:37:12 -08:00
epriestley
e293d39b5f Split the difference on remote URIs.
This is slightly trickier than D8082.

Auditors: btrahan
2014-01-27 19:44:39 -08:00
epriestley
f2bc293a2a Use remote URI, not display URI, to match repositories
Summary:
I derped this up, and it passed my tests because the two URIs are the same for hosted repositories.

Match repositories against their remote URI (like `https://github.com/user/repo.git`), not their display URI (like `/diffusion/X/`).

Test Plan: i r dums

Reviewers: talshiri, btrahan

Reviewed By: talshiri

CC: aran

Differential Revision: https://secure.phabricator.com/D8082
2014-01-27 18:49:35 -08:00
epriestley
014a873773 Update DifferentialDiff: add repositoryPHID, drop parentRevisionID
Summary:
Moves away from ArcanistProjects:

  - Adds storage for diffs to be directly associated with a repository (instead of indirectly, through arcanist projects). Not really populated yet.
  - Drops `parentRevisionID`, which is obsoleted by the "Depends On" edge. This is not exposed in the UI anywhere and doesn't do anything. Resolves TODO.

Test Plan: Ran storage upgrades, browsed around, lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8072
2014-01-26 15:29:22 -08:00
epriestley
e34a44a7ed Allow repository lookup by remote URI
Summary: This helps us move away from arcanist projects in normal use, by allowing us to identify repositories based on the remote URI.

Test Plan: Used `repository.query` to query some repos by remote URI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8069
2014-01-25 14:02:38 -08:00
epriestley
ce45c57057 Add more options to repository.query
Summary: Expose more options on `repository.query`. My broad goal here is to move forward on suppressing Arcanist Projects.

Test Plan: Used Conduit console to execute queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8068
2014-01-25 14:02:28 -08:00
epriestley
31e11a97d2 If repository mirroring fails, keep trying the other mirrors
Summary: Ref T4338. Currently, if you have several mirrors and the first one fails, we won't try the other mirrors (since we'll throw and that will take us out of the mirroring process). Instead, try each mirror even if one fails, and then throw an AggregateException with all the failures.

Test Plan:
  - Ran `bin/repository mirror` normally.
  - Faked an exception, ran again, got the AggregateException I expected.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4338

Differential Revision: https://secure.phabricator.com/D8067
2014-01-25 14:02:09 -08:00
epriestley
dd944f7d83 Separate repository mirroring into an Engine and provide bin/repository mirror
Summary:
Ref T4338. Currently, there's no diagnostic command to execute mirroring (so I can't give users an easy command to run), and it's roughly the last piece of real logic left in the PullLocal daemon.

Separate mirroring out, and provide `bin/repository mirror`.

Test Plan:
  - Ran `bin/repository mirror` to mirror a repository.
  - Ran PullLocalDaemon and verified it also continued mirroring normally.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4338

Differential Revision: https://secure.phabricator.com/D8066
2014-01-25 14:01:58 -08:00
Brecht Van Lommel
f007ed6263 Allow repository mirrors even if the repository is hosted elsewhere.
Summary:
I'm not sure if this is desired functionality, but we happen to need
mirroring of our repository which is not hosted by phabricator, and as far as
I can tell the mirroring code does not depend on the hosting code.

Test Plan:
Setup mirror with SSH credentials to github, pushed changes to
elsewhere hosted repository, commits got mirrored to github.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4338

Differential Revision: https://secure.phabricator.com/D7637
2014-01-25 12:02:50 -08:00
epriestley
c21a8d31dd When repairing Git remote URIs, include credentials
Summary: See IRC. Currently, if a fetch fails, we may repair the remote URI incorrectly, replacing it with one without any credentials. Instead, retain credentials.

Test Plan: Faked it locally, got le-boom to verify on IRC.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8055
2014-01-23 17:23:38 -08:00
epriestley
be59578794 Fix bin/repository importing for CLOSEABLE flag
Summary: After adding the CLOSEABLE flag, `bin/repository importing` reports CLOSEABLE commits with no information. Just don't report these, for consistency with the old behavior. Adding flags to show them might be nice at some point if we run into issues where that output would be useful.

Test Plan: Ran `bin/repositroy importing` on a repository with CLOSEABLE commits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D8024
2014-01-21 14:09:51 -08:00
epriestley
c9a0ffa1cf Verify that SVN repository roots really are repository roots
Summary: Fixes T3238. Ref T4327. Although the instructions are fairly clear on this, it's easy to miss them. Make sure the root the user enters matches the real root.

Test Plan: Added unit tests. Used `bin/repository discover` to hit the check explicitly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3238, T4327

Differential Revision: https://secure.phabricator.com/D8020
2014-01-21 14:02:58 -08:00
epriestley
ef2c9861ef Add yet more unit tests for Subversion
Summary: Ref T4327. This adds a subpath/partial repository where we import only a subdirectory, and adds tests for it.

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8019
2014-01-21 14:02:48 -08:00
epriestley
6c860bf850 Add even more Subversion unit tests
Summary:
Ref T4327. Adds additional tests:

  - Property changes on the root directory (which has special handling).
  - Copying a file from a previous revision (this is `svn cp svn://server.com/root/some_file@123 some_file`).
  - "Multicopying" a file: this is where a file is copied to several places and moved in the same commit.

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8018
2014-01-21 14:02:40 -08:00
epriestley
9dde415884 Add more Subversion unit tests
Summary:
Ref T4327. Adds additional SVN tests to cover:

  - replacing a file with a file;
  - replacing a file with a directory;
  - removing a directory with files in it;
  - adding a directory with files;
  - copying a directory and adding and deleting files inside it.

Test Plan: Ran unit tests. One of these has a slightly irregular parse, see inline.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8017
2014-01-21 14:02:32 -08:00
epriestley
dc74da0abe Add basic test coverage for the Subversion parser
Summary:
Ref T4327. Adds the same basic battery of tests to the SVN parser as the other parsers have.

cowsay {{{ THIS IS AN AMAZING DIFF }}}

Test Plan:
Ran unit tests against SVN change parsing.

bwahaha

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8016
2014-01-20 17:25:04 -08:00
epriestley
c951d7e13b Add unit tests for the Mercurial change parser
Summary: Ref T4327. This adds a set of test cases for the Mercurial parser. These tests are nearly identical to the git cases, except that the Mercurial parser can't figure out symlinks right now.

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8014
2014-01-20 13:14:18 -08:00
epriestley
f63e7571e5 Add unit tests for Git change parsers
Summary: Ref T4327. Adds some basic tests to the Git parser for a set of common operations (add, change, move, copy, directory, symlink, propchange).

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D8012
2014-01-20 13:14:04 -08:00
epriestley
f7a1feea38 Begin making change parsers testable
Summary:
Ref T4327. There are a bunch of other probably-related tasks too, some linked there.

We have some rare/unusual bugs in the change parsers, mostly in Subversion, but it's terrifying to touch them because they're complicated and fragile and have no test coverage.

To fix this stuff, I want to make them more testable. In particular, they basically end with this big INSERT right now. Instead, I'm going to make them return objects representing the data to be inserted, then have the common infrastructure do the insert. This gives us two benefits:

  - Reduced code duplication on the insert;
  - we can stop before the insert and have unit tests examine the objects.

This swaps the Git parser over, but doesn't swap the hg/svn parsers yet. I'll do those separately, the SVN one looks a bit tricky.

Test Plan:
  - Used `scripts/repository/reparse.php` to reparse a Git commit, with `--trace`. Verified it looked the same as before and the SQL that was executed seemed reasonable.
  - Did the same for `hg` / `svn` commits, to make sure I didn't derp anything. These aren't expected to do anything differently.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7980
2014-01-20 13:12:44 -08:00
epriestley
a9c16fbe4e Reduce parse latency for changes pushed to hosted repositories
Summary:
Currently, we can sit in a sleep() for up to 15 seconds (or longer, in some cases), even if there's a parse request.

Try polling the DB every second to see if we should wake up instead. This might or might not produce significant improvements, but seems OK to try and see.

Also a small fix for logging branch names with `%` in them, etc.

Test Plan: Ran `phd debug pulllocal` and pushed commits, saw them parse within a second.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, dctrwatson

Differential Revision: https://secure.phabricator.com/D7998
2014-01-17 16:09:51 -08:00
epriestley
8520d9e070 Move more discovery responsibilities into DiscoveryEngine
Summary:
Ref T4327. This moves the last pieces of discovery responsibility out of the PullLocal daemon and into the DiscoveryEngine.

(This makes it easier to discover repositories in unit tests in the future, since we don't need to build a PullLocal daemon and can just build a DiscoveryEngine.)

Test Plan: Ran `phd debug pulllocal`. Ran `repostory discover`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7997
2014-01-17 16:09:24 -08:00
epriestley
220addb249 Move Git discovery into DiscoveryEngine
Summary:
Ref T4327. Consolidates and simplifies infrastructure:

  - Moves Git discovery into DiscoveryEngine.
  - Collapses a bunch of the Git and Mercurial code related to stream discovery.
  - Removes all cach code from PullLocal daemon (it's no longer called).
  - Adds basic unit tests for Git and Mercurial discovery.

Various cleanup:

  - Makes GitStream and MercurialStream extend a common base.
  - Improves performance of MercurialStream in some cases, by requiring fewer commits be output and parsed.
  - Makes mirroring exceptions easier to debug.
  - Fixes discovery of Mercurial repositories with multiple branch heads.
  - Adds some missing `pht()`.

Test Plan:
I tested this fairly throughly because I think this phase is complete:

  - Made new repositories in multiple VCSes and did full imports.
    - Particularly, I reimported Arcanist to make sure that TODO was resolved. I think it was related to the toposort stuff.
  - Pushed commits to multiple VCSes.
  - Pushed commits to a non-close branch, then pushed a merge commit. Observed commits import initially as non-close, then get flagged for close.
  - Started full daemons and resolved various minor issues that showed up in the daemon log until everything ran cleanly.
  - Basically spent about 30 minutes banging on this in every way I could think of to try to break it. I found and fixed some minor stuff, but it seems solid.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7987
2014-01-17 11:48:53 -08:00
epriestley
618da2265d Remove all the multi-pass autoclose-branch separate-cache / seenOnBranches junk
Summary:
Ref T4327. Simplify the git discovery process so I can move it to the DiscoveryEngine, so I can make change parsing testable.

In particular:

  - As an optimization, we process closeable branches ("master") first, then process uncloseable branches ("epriestley-devel"). This means that in the common case we can insert a commit as closeable immediately when it is discovered, the first pass through the pipeline will get it right, and the "ref update" step will never need to do any meaningful work.
  - Commits which do not initially appear on a closeable branch, but later move to one (via merges or ref moves) will now be caught in the ref update step, have the closeable flag set, and have a message step re-queued.
  - We no longer need to do a separate discovery step on closable branches.
  - We no longer need to keep track of `seenOnBranches`.

Test Plan: Ran discovery on repositories after pushing commits, got reasonable results.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7985
2014-01-17 11:48:53 -08:00
epriestley
cec3f44b90 Provide an alternate, more general "closeable" flag for commits
Summary:
Ref T4327. This provides a more general RefCursor-based way to identify closeable commits, and moves away from the messy `seenOnBranches` stuff. Basically:

  - When a closeable ref (like the branch "master") is updated, query the VCS for all commits which are ancestors of the new ref, but not ancestors of the existing closeable heads we previously knew about. This is basically all the commits which have been merged or moved onto the closeable ref.
  - Take these commits and set the "closeable" flag on the ones which don't have it yet, then queue new tasks to reprocess them.

I haven't removed the old stuff yet, but will do that shortly.

Test Plan:
  - Ran `bin/repository discover` and `bin/repository refs` on a bunch of different VCSes and VCS states. The effects seemed correct (e.g., new commits were marked as closeable).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7984
2014-01-17 11:48:53 -08:00
epriestley
f4b9efe256 Introduce ref cursors for repository parsing
Summary:
Ref T4327. I want to make change parsing testable; one thing which is blocking this is that the Git discovery process is still part of `PullLocal` daemon instead of being part of `DiscoveryEngine`. The unit test stuff which I want to use for change parsing relies on `DiscoveryEngine` to discover repositories during unit tests.

The major reason git discovery isn't part of `DiscoveryEngine` is that it relies on the messy "autoclose" logic, which we never implemented for Mercurial. Generally, I don't like how autoclose was implemented: it's complicated and gross and too hard to figure out and extend.

Instead, I want to do something more similar to what we do for pushes, which is cleaner overall. Basically this means remembering the old branch heads from the last time we parsed a repository, and figuring out what's new by comparing the old and new branch heads. This should give us several advantages:

  - It should be simpler to understand than the autoclose stuff, which is pretty mind-numbing, at least for me.
  - It will let us satisfy branch and tag queries cheaply (from the database) instead of having to go to the repository. We could also satisfy some ref-resolve queries from the database.
  - It should be easier to extend to Mercurial.

This implements the basics -- pretty much a table to store the cursors, which we update only for Git for now.

Test Plan:
  - Ran migration.
  - Ran `bin/repository discover X --trace --verbose` on various repositories with branches and tags, before and after modifying pushes.
  - Pushed commits to a git repo.
  - Looked at database tables.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7982
2014-01-17 11:48:53 -08:00
epriestley
0ac58d7db6 Move repository URI normalization out of PullLocalDaemon
Summary: Ref T4327. Moves us one small step forward toward testable change parsers by separating out this unrelated logic from the PullLocal daemon. We will also probably want to run this logic so we can do remote path lookups to limit the role of Arcanist Projects in the future, which is why I made the URI type (here, only "git") a parameter rather than calling this a `GitURINormalizer` or something.

Test Plan:
  - Ran unit tests.
  - Ran `repository discover` on a correctly-configured remote repository.
  - Ran `repository discover` on an incorrectly-configured remote, got an error message.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4327

Differential Revision: https://secure.phabricator.com/D7981
2014-01-17 11:48:52 -08:00
epriestley
a716fe99f3 Perform search indexing in the worker queue and respect bin/search index --background
Summary: Fixes T3857. Earlier work made this trivial and just left product questions, which I've answered by requiring the daemons to run on reasonable installs.

Test Plan: Ran `bin/search index` and `bin/search index --background`. Observed indexes write in the former case and tasks queue in the latter case. Commented with a unique string on a revision and searched for it a moment later, got exactly one result (that revision), verifying that reindexing works correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3857

Differential Revision: https://secure.phabricator.com/D7966
2014-01-14 13:22:56 -08:00
John Watson
2ee4507486 Add Hosted/Remote filtering to Diffusion
Test Plan: Did searches in Diffusion using all 3 Hosted values

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7927
2014-01-10 09:12:50 -08:00
epriestley
efe187d5be Support "Repository's projects" field in Commit and Differential Revision rules
Summary: This also cleans up some code a little bit. Most of the gymnastics are to make sure we call `needProjectPHIDs()` appropriately.

Test Plan: Created new commit and revision rules with this field. Ran commits and revisions through the test console. Field behavior seemed correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, dctrwatson

Differential Revision: https://secure.phabricator.com/D7923
2014-01-09 15:56:24 -08:00
epriestley
972dfa7bfc Add 'hook.d/' directories to SVN and Git repositories for custom hooks
Summary:
Fixes T4189. Ref T4151. Allows repositories to have additional custom hooks for operations which can't be expressed with Herald (one such operation is lint).

This adds only local hook directories, since they're easier to use with existing hooks than global directories. I might add global directories eventually.

This doesn't support Mercurial since we have no demand for it and it's more complicated (we lose compatibility and power by just dropping a `hooks.d/` somewhere).

Test Plan:
  - Pulled hosted SVN and Git repos to verify the hook directories generate correctly.
  - Added a variety of hooks to the hook directories (echo + pass, fail).
  - Pushed commits and verified the hooks fired (output expected info, or failed).
  - Verified push log reflected the correct error code ("3", external) and detail ("nope.sh") when rejecting.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4151, T4189

Differential Revision: https://secure.phabricator.com/D7884
2014-01-03 12:26:10 -08:00
epriestley
637e3f38f3 Allow repositories to be associated with projects
Summary: Ref T4264. Ref T2628. Ref T3102. Allows you to associate repositories with projects. In the future, you'll be able to write Herald object rules against projects, use Herald fields like "Repository's projects", and search by project.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3102, T4264, T2628

Differential Revision: https://secure.phabricator.com/D7881
2014-01-03 12:24:09 -08:00
iodragon
f981d50f5d Set php PATH for repository's hook
Summary:
~~Set PATH for repository's hook, so the environment.append-paths can used~~
repository's hook may can't find php path if user's profile like bash_profile is not loaded.

Test Plan: check the hook generated is contain the right path

Reviewers: epriestley, #blessed_reviewers

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7743
2014-01-03 09:39:31 -08:00
epriestley
09341be10f Remove repository shortcuts
Summary:
Repositories currently have a no-UI "shortcut" feature which is only used by Facebook (and I'm not sure it's even used). As implemented, this feature is policy-oblivious and kind of nonsensical. Throw it away.

I'm open to reimplementing this, but I want to see some level of interest in it before I do. The new implementation would add shortcuts to each repository, similar to how mirrors work. My original plan was to follow this up with such an implementation (it's half-implemented in my sandbox), but as I worked through it I'm not sure it's really valuable.

Test Plan: Browsed repository list, grep.

Reviewers: btrahan

Reviewed By: btrahan

CC: FacebookPOC, aran

Differential Revision: https://secure.phabricator.com/D7862
2014-01-02 11:59:27 -08:00
epriestley
140c88e971 Implement basic object rules for Herald
Summary:
Ref T4264. Allows you to create "Object" rules, in addition to Global and Personal rules. If you choose to create an Object rule, you'll be prompted to select an object on a new screen. You must be able to edit and object in order to create rules for it.

Ref T3506. This makes "All" the default filter for the transcript view, which should reduce confusion on smaller installs.

Test Plan:
  - Created non-object rules.
  - Created object rules.
  - Triggered object rules against matching and unmatching objects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3506, T4264

Differential Revision: https://secure.phabricator.com/D7853
2013-12-30 16:48:14 -08:00
epriestley
e397103bf2 Extend all "ManagementWorkflow" classes from a base class
Summary:
Ref T2015. Not directly related to Drydock, but I've wanted to do this for a bit.

Introduce a common base class for all the workflows in the scripts in `bin/*`. This slightly reduces code duplication by moving `isExecutable()` to the base, but also provides `getViewer()`. This is a little nicer than `PhabricatorUser::getOmnipotentUser()` and gives us a layer of indirection if we ever want to introduce more general viewer mechanisms in scripts.

Test Plan: Lint; ran some of the scripts.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D7838
2013-12-27 13:15:40 -08:00
epriestley
aad6b57c36 Add bin/harbormaster to make builds easier to debug
Summary:
Ref T1049. Adds `bin/harbormaster` and `bin/harbormaster build` for applying plans from the console. Since this gets `--trace`, it's much easier to debug what's going on.

This doesn't work properly with some of the Drydock steps yet, I need to look at those. I think `setRunAllTasksInProcess` probably obsoletes some of the mechanisms. It might also not work with "Wait for Builds" but I didn't check.

Test Plan: Used `bin/harbormaster` to run a bunch of builds. Ran builds from web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7825
2013-12-26 10:40:52 -08:00
epriestley
81dcf6378d Make repository pull install hooks the first time
Summary:
Ref T4257. Currently, the pull logic looks like this:

  if (new) {
    create();
  } else {
    if (hosted) {
      install_hooks();
    } else {
      update();
    }
  }

This means that the first time you run `repository pull`, hooks aren't installed, which makes debugging trickier. Instead, reorganize the logic:

  if (new) {
    create();
  } else {
    if (!hosted) {
      update();
    }
  }

  if (hosted) {
    install_hooks();
  }

Test Plan: Ran `bin/repository pull` on a new `hg` repo and got hooks installed immediately.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4257

Differential Revision: https://secure.phabricator.com/D7818
2013-12-23 10:43:49 -08:00
epriestley
d9a04fcb53 Minor, make sure we release locks in pull daemon if we fail in unusual ways
Summary: A user is reporting a re-lock in this daemon, which I can't
reproduce, but might be possible if this throws. Stop it from throwing in
a way which evades unlock.

See: <https://github.com/facebook/phabricator/issues/476>

Auditors: btrahan
2013-12-22 08:51:07 -08:00
epriestley
c462713584 Minor cleanup for task rendering in Daemons
Summary:
Fixes two issues:

  - When rendering a task's details, we currently issue a policy-oblivious query. Instead, issue a policy-aware query.
  - The formatting is a little bit weird, with the top half in a box and the bottom half with an older style. Make them consistent.

Test Plan: Looked at the detail pages for several tasks in queue.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D7812
2013-12-20 18:02:32 -08:00
epriestley
9c938701c3 Modernize Diffusion commitparentsquery
Summary: Ref T4195. Ref T2783. We have an old-school implementation of this; move it into a LowLevel query and make callers all run through Conduit. I need the LowLevel query for hooks, to implement an "is merge commit" Herald rule.

Test Plan:
  - Ran query via Conduit for SVN, Mercurial, Git.
  - Parsed a commit which closed a revision, attach/closed worked correctly.
  - Browsed Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195, T2783

Differential Revision: https://secure.phabricator.com/D7808
2013-12-20 12:39:21 -08:00
epriestley
d7c4edab28 Move commit message/metadata field query to a separate class
Summary: Ref T4195. I need to query commit metadata to figure out which revision a commit is associated with. Move this out of the MessageParser so the code can be called from the HookEngine.

Test Plan: Used `reparse.php` to reparse a variety of SVN, Mercurial and Git commits. Used `var_dump()` to verify sensible fields were returned.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7805
2013-12-20 12:38:44 -08:00
epriestley
23332241b2 Move commit hash querying to DiffusionLowLevelCommitQuery
Summary: Ref T4195. I need this for the Herald pre-commit rules, and it generally simplifies things.

Test Plan: Used `reparse.php` plus `var_dump()` to inspect refs in Git, Mercurial and SVN repos. They all looked correct and reparsed correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7804
2013-12-20 12:38:15 -08:00
epriestley
d667b12206 Provide a standalone query for resolution of commit author/committer into Phabricator users
Summary:
Ref T4195. To implement the "Author" and "Committer" rules, I need to resolve author/committer strings into Phabricator users.

The code to do this is currently buried in the daemons. Extract it into a standalone query.

I also added `bin/repository lookup-users <commit>` to test this query, both to improve confidence I'm getting this right and to provide a diagnostic command for users, since there's occasionally some confusion over how author/committer strings resolve into valid users.

Test Plan:
I tested this using `bin/repository lookup-users` and `reparse.php --message` on Git, Mercurial and SVN commits. Here's the `lookup-users` output:

  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIS3
  Examining commit rINIS3...
  Raw author string: epriestley
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rPOEMS165b6c54f487c8
  Examining commit rPOEMS165b6c54f487...
  Raw author string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: epriestley <git@epriestley.com>
  Phabricator user: epriestley (Evan Priestley   )
  >>> orbital ~/devtools/phabricator $ ./bin/repository lookup-users rINIH6d24c1aee7741e
  Examining commit rINIH6d24c1aee774...
  Raw author string: epriestley <hg@yghe.net>
  Phabricator user: epriestley (Evan Priestley   )
  Raw committer string: null
  >>> orbital ~/devtools/phabricator $

The `reparse.php` output was similar, and all VCSes resolved authors correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1731, T4195

Differential Revision: https://secure.phabricator.com/D7801
2013-12-19 11:05:17 -08:00
epriestley
f750d5f8dc Provide a low-level SVN commit query, and merge the VCS query types
Summary: Ref T4195. Even though we use `svnlook` in the hook itself, I need this query elsewhere, so provide it and merge the classes into one which does the right thing.

Test Plan:
  - Used `reparse.php` to reparse messages for Git, SVN and Mercurial commits, using `var_dump()` to examine the commit refs for sanity.
  - Used `reparse.php` to reparse changes for an SVN commit.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7800
2013-12-19 11:05:06 -08:00
epriestley
92bc76aae0 Move mercurial commit metadata parsing into a LowLevel query
Summary: Ref T4195. Same as D7793, but for mercurial. (As usual, SVN needs some goofy nonsense instead, so the next diff will just make this field work.)

Test Plan: Ran `reparse.php` on Git and Mercurial commits, var_dump'd the output and it looked correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7795
2013-12-18 17:48:19 -08:00
epriestley
f048053c75 Move git commit metadata parsing into a LowLevelQuery
Summary: Ref T4195. I need to issue this command from the pre-commit hook to get commit bodies for hooks.

Test Plan: Ran `reparse.php --message --trace` and dumped the $ref, which looked correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7793
2013-12-18 17:48:06 -08:00
William R. Otte
dc43123494 Get the proper Subversion URI for hosted repositories.
Summary: The commit change parser worker was incorrectly getting the remote uri; calling getSubversionPathURI instead behaves properly for hosted repositories.  Fixes T4236.

Test Plan: Observed that my hosted subversion repositories finally had history in Diffusion.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4236

Differential Revision: https://secure.phabricator.com/D7794
2013-12-18 15:01:36 -08:00
epriestley
1ff3ef382d Give Herald rules a standard "Hnnn" object name
Summary: Allow Herald rules to be referred to with `H123`, etc., like other object types are. Herald rules now have proper PHIDs and an increasingly prominent role in triggering application actions. Although I suspect users will rarely use `H123` in Remarkup to mention rules, this can simplify some of the interfaces which relate objects across systems.

Test Plan: Looked at various interfaces and saw `H123` names. Mentioned `H123` in remarkup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7786
2013-12-18 12:00:18 -08:00
epriestley
5863f792a6 Remove many redundant implementations of canLoadNamedObject()
Summary:
These just got copy/pasted like crazy, the base class has the correct default implementation.

(I'm adding "H" for Herald Rules, which is why I was in this code.)

I also documented the existing prefixes at [[ Object Name Prefixes ]].

Test Plan: Verified base implementation. Typed some object names into the jump nav.

Reviewers: btrahan

Reviewed By: btrahan

CC: hach-que, aran

Differential Revision: https://secure.phabricator.com/D7785
2013-12-18 12:00:01 -08:00
epriestley
b0f7e7a6af Work around hg echoing warnings to stdout under --debug
Summary:
Ref T615. Ref T4237. With `--debug`, Mercurial will echo an "ignoring untrusted configuration option" warning **to stdout** if `.hgrc` has the wrong owner.

However, we need `--debug` to make `{parents}` usable, at least until the patches I got into the upstream are widely deployed. So after getting `--debug` output, strip off any leading warnings.

These warnings should always be in English, at least, since we set `LANG` explicitly.

Test Plan: Unit tests. @asherkin, maybe you can confirm this? I can't actually get the warning, but I think my `hg` in PATH is just a bit out of date.

Reviewers: asherkin, btrahan

Reviewed By: asherkin

CC: asherkin, aran

Maniphest Tasks: T615, T4237

Differential Revision: https://secure.phabricator.com/D7784
2013-12-17 18:04:01 -08:00
epriestley
3386920971 Add Herald support for blocking ref changes
Summary: Ref T4195. Allows users to write Herald rules which block ref changes. For example, you can write a rule like `alincoln can not create branches`, or `no one can push to the branch "frozen"`.

Test Plan:
This covers a lot of ground. I created and pushed a bunch of rules, then looked at transcripts, in general. Here are some bits in detail:

Here's a hook-based reject message:

  >>> orbital ~/repos/POEMS $ git push
  Counting objects: 5, done.
  Delta compression using up to 8 threads.
  Compressing objects: 100% (3/3), done.
  Writing objects: 100% (3/3), 274 bytes, done.
  Total 3 (delta 2), reused 0 (delta 0)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: This commit was rejected by Herald pre-commit rule H24.
  remote: Rule: No Branches Called Blarp
  remote: Reason: "blarp" is a bad branch name
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp -> blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Here's a transcript, showing that all the field values populate sensibly:

{F90453}

Here's a rule:

{F90454}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7782
2013-12-17 15:23:55 -08:00
epriestley
f28d3089d7 Assign PHIDs to PushLogs
Summary: Ref T4195. We need these in Herald, since HeraldTranscripts need to refer to a PHID which they acted upon.

Test Plan:
Ran migration, got PHIDs:

  mysql> select phid from repository_pushlog limit 3;
  +--------------------------------+
  | phid                           |
  +--------------------------------+
  | PHID-PSHL-25jnc6cjgzw5rwqgmr7r |
  | PHID-PSHL-2vrvmtslkrj5yv7nxsv2 |
  | PHID-PSHL-34x262zkrwoka6mplony |
  +--------------------------------+
  3 rows in set (0.00 sec)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7780
2013-12-17 15:23:23 -08:00
epriestley
2216a5e6ef Add Subversion ref and content logs to pre-commit hooks
Summary: Ref T4195. SVN has no such thing as refs (I was thinking about writing a quasi-ref anyway like `HEAD: r23 -> r24`, but I'm not sure it would actually be useful). And content is very easy to build.

Test Plan: Pushed some stuff to SVN, got logs from it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7766
2013-12-17 11:11:52 -08:00
epriestley
11b8e57ae0 Remove "pretag" hook support in Mercurial
Summary: Ref T4195. This doesn't actually work like I thought it did: it only fires locally, when you run `hg tag`. Mercurial tags are also weird and basically don't make any sense and everyone should use bookmarks instead. We could implement some flavor of this eventually, but I'd like to see users request it first. They can implement their own with content-based hooks once those work, anyway.

Test Plan: This code didn't do anything.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7765
2013-12-17 09:18:48 -08:00
epriestley
74251b3636 Support bookmark hook operations in Mercurial
Summary: Ref T4195. Turns bookmark mutations in Mercurial into log objects.

Test Plan:
Pushed a pile of bookmarks and got logs:

{F89313}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7764
2013-12-17 08:34:30 -08:00
epriestley
6f3a99eb39 Generate ref updates in Mercurial hooks
Summary: Ref T4195. Mercurial is not my favorite VCS.

Test Plan:
Hit the split branches case:

  >>> orbital ~/repos/INIH $ hg push --force
  pushing to ssh://dweller@local.aphront.com/diffusion/INIH
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 2 changesets with 2 changes to 1 files (+1 heads)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push splits the head of branch 'default' into multiple heads: 802c785c3dd9, e73400db39b0. This is inadvisable and dangerous.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  remote: transaction abort!
  remote: rollback completed
  remote: abort: pretxnchangegroup.phabricator hook exited with status 1

Hit the divergent heads case:

  >>> orbital ~/repos/INIH $ hg push --force
  pushing to ssh://dweller@local.aphront.com/diffusion/INIH
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 1 changesets with 1 changes to 1 files (+1 heads)
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push creates new, divergent heads for the branch 'default': f56af4232aa9. This is inadvisable and dangerous.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  remote: transaction abort!
  remote: rollback completed
  remote: abort: pretxnchangegroup.phabricator hook exited with status 1

Did a bunch of good/bad pushes:

{F89300}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7763
2013-12-17 08:34:15 -08:00
epriestley
2725586baf Restructure HookEngine to use PushLog records for all operations
Summary:
Ref T4195. This pulls the central logic of HookEngine up one level and makes all the git stuff genrate PushLogs.

In future diffs, everything will generate PushLogs and we can hand those off to Herald.

Test Plan:
Pushed a pile of valid/invalid stuff:

{F89256}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7761
2013-12-17 08:32:33 -08:00
epriestley
d846f6508b Fix some repository URI handling issues in Git and Mercurial
Summary:
See <https://github.com/facebook/phabricator/issues/467>. @dctrwatson also ran into an issue where we were trying to `setPass()` a GitURI.

  - For Git and Mercurial, properly generate credential URIs where relevant.
  - Don't try to `setPass()` on Git-style URIs.

This isn't perfect but should clean things up a bit.

Test Plan: Added unit tests. Lots of `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: dctrwatson, aran

Differential Revision: https://secure.phabricator.com/D7759
2013-12-12 09:45:27 -08:00
epriestley
39b384041f Add "r <name>" to jump nav to locate repositories by name
Summary: User request.

Test Plan: Searched for `r ph`, `r poetry`.

Reviewers: btrahan, bigo

Reviewed By: bigo

CC: aran

Differential Revision: https://secure.phabricator.com/D7720
2013-12-05 14:26:38 -08:00
epriestley
2ff5541fc5 Record new commits in the push log
Summary:
Ref T4195. Like the previous diffs, these both create a useful log and give us an object to hand off to Herald.

Surface this information in Diffusion, too, and clean things up a little bit.

Test Plan: {F87565}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7718
2013-12-05 11:59:41 -08:00
epriestley
3f50460149 Allow repository push logs to be filtered by pusher and repository
Summary: Ref T4195. Add UI options to filter push logs by pusher and repository. Add a link from the repository view page to the push logs.

Test Plan: Viewed a hosted repository, clicked logs link, saw logs. Filtered lgos by repo/pusher.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7713
2013-12-05 11:59:33 -08:00
epriestley
caa6fdf56d Add a basic push log for recording repository push events
Summary:
Ref T4195. This log serves two purposes:

  - It's a log, so you can see what happened. Particularly, in Git/Hg, there is no other way to tell:
    - Who //pushed// a change (vs committed / authored)?
    - When was a change pushed?
    - What was the old value of some tag/branch before someone destroyed it?
  - We can hand these objects off to Herald to implement pre-commit rules.

This is a very basic implementation, but gets some data written and has a basic UI for it.

Test Plan: {F87339}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4195

Differential Revision: https://secure.phabricator.com/D7705
2013-12-05 11:56:14 -08:00
epriestley
e77d5012be Fix two issues with shell/config scripts for hosted repositories
Summary: Ref T4151. `-ne` is numeric in some/most/all shells; `exec --` apparently doens't always work.

Test Plan: Will make @zeeg test.

Reviewers: btrahan, zeeg

Reviewed By: zeeg

CC: zeeg, aran

Maniphest Tasks: T4151

Differential Revision: https://secure.phabricator.com/D7702
2013-12-04 16:45:54 -08:00
epriestley
d2e9aee16d Reject dangerous changes in Git repositories by default
Summary: Ref T4189. This adds a per-repository "dangerous changes" flag, which defaults to off. This flag must be enabled to do non-appending branch mutation (delete branches / rewrite history).

Test Plan:
With flag on and off, performed various safe and dangerous pushes.

  >>> orbital ~/repos/POEMS $ git push origin :blarp
  remote: +---------------------------------------------------------------+
  remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
  remote: +---------------------------------------------------------------+
  remote:             \
  remote:              \                    ^    /^
  remote:               \                  / \  // \
  remote:                \   |\___/|      /   \//  .\
  remote:                 \  /V  V  \__  /    //  | \ \           *----*
  remote:                   /     /  \/_/    //   |  \  \          \   |
  remote:                   @___@`    \/_   //    |   \   \         \/\ \
  remote:                  0/0/|       \/_ //     |    \    \         \  \
  remote:              0/0/0/0/|        \///      |     \     \       |  |
  remote:           0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
  remote:        0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
  remote:                    ,-}        _      *-.|.-~-.           .~    ~
  remote:   \     \__/        `/\      /                 ~-. _ .-~      /
  remote:    \____(Oo)           *.   }            {                   /
  remote:    (    (--)          .----~-.\        \-`                 .~
  remote:    //__\\  \ DENIED!  ///.----..<        \             _ -~
  remote:   //    \\               ///-._ _ _ _ _ _ _{^ - - - - ~
  remote:
  remote:
  remote: DANGEROUS CHANGE: The change you're attempting to push deletes the branch 'blarp'.
  remote: Dangerous change protection is enabled for this repository.
  remote: Edit the repository configuration before making dangerous changes.
  remote:
  To ssh://dweller@localhost/diffusion/POEMS/
   ! [remote rejected] blarp (pre-receive hook declined)
  error: failed to push some refs to 'ssh://dweller@localhost/diffusion/POEMS/'

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, chad, richardvanvelzen

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7689
2013-12-03 10:28:39 -08:00
epriestley
50b5a0c8b7 Add explicit "/" when installing hooks
Summary: There's no guarantee that the local path has a trailing "/". We
should probably guarantee that at some point, but just add one
unconditionally for now.

Auditors: btrahan
2013-12-02 16:18:02 -08:00
epriestley
f93c6985ad Support Mercurial pretxnchangegroup hooks
Summary: Ref T4189. Fixes T2066. Mercurial has a //lot// of hooks so I'm not 100% sure this is all we need to install (we may need separate hooks for tags/bookmarks) but it should cover most of what we're after at least.

Test Plan:
  - `bin/repository pull`'d a Mercurial repo and got a hook install.
  - Pushed to a Mercurial repository over SSH and HTTP, with good/bad hooks. Saw hooks fire.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2066, T4189

Differential Revision: https://secure.phabricator.com/D7685
2013-12-02 15:46:03 -08:00
epriestley
017d6ccd07 Support SVN pre-commit hoooks
Summary:
Ref T4189. This adds SVN support, which was a little more messy than I though. Principally, we can not use `PHABRICATOR_USER` for Subversion, because it strips away the entire environment for "security reasons".

Instead, use `--tunnel-user` plus `svnlook author` to figure out the author.

Also fix "ssh://" clone URIs, which needs to be "svn+ssh://".

Test Plan:
  - Made SVN commits through the hook.
  - Made Git commits, too, to make sure I didn't break anything.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7683
2013-12-02 15:45:55 -08:00
epriestley
618b5cbbc4 Install pre-commit hooks in Git repositories
Summary:
Ref T4189. T4189 describes most of the intent here:

  - When updating hosted repositories, sync a pre-commit hook into them instead of doing a `git fetch`.
  - The hook calls into Phabricator. The acting Phabricator user is sent via PHABRICATOR_USER in the environment. The active repository is sent via CLI.
  - The hook doesn't do anything useful yet; it just veifies basic parameters, does a little parsing, and exits 0 to allow the commit.

Test Plan:
  - Performed Git pushes and pulls over SSH and HTTP.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4189

Differential Revision: https://secure.phabricator.com/D7682
2013-12-02 15:45:36 -08:00
epriestley
209861500f Use "en_US.UTF-8", not "C", as the LANG setting
Summary: `LANG=C` is smooshing UTF-8 in some cases. See IRC.

Test Plan: User confirmed this works.

Reviewers: btrahan, asherkin

Reviewed By: asherkin

CC: aran

Differential Revision: https://secure.phabricator.com/D7659
2013-11-26 08:50:36 -08:00
epriestley
de73029e99 Propagate PHABRICATOR_ENV into VCS commands explicitly
Summary: Fixes T4155. See discussion in T4155.

Test Plan: @mbishopim3 confirmed this fixes his issue.

Reviewers: btrahan, chad

Reviewed By: chad

CC: mbishopim3, aran

Maniphest Tasks: T4155

Differential Revision: https://secure.phabricator.com/D7646
2013-11-24 18:11:56 -08:00
epriestley
c978fe5240 Fix an issue where SSH protocols would check legacy credentials
Missed this while grepping. We should use SSH purely based on protocol.

Auditors: btrahan
2013-11-22 16:31:14 -08:00
epriestley
d09dd23bd7 Actually push to mirrors
Summary: Fixes T4038. Push repositories to mirrors.

Test Plan: Created a functional mirror of a local: https://github.com/epriestley/poems

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7633
2013-11-22 15:24:09 -08:00
epriestley
4b91c4f7ae Add UI for defining repository mirrors
Summary:
Ref T4038. This adds everything except the actual pushing part for mirrors.

This isn't the most beautiful or sophisticated UI, but I want get the authoritative repositories self-hosted and get users beta-ing hosting as soon as possible. We can do transactions, etc., later on.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4038

Differential Revision: https://secure.phabricator.com/D7632
2013-11-22 15:23:50 -08:00
epriestley
51fb1ca16d Migrate repositories to use Passphrase for credential management
Summary: Fixes T4122. Ref T2230. Instead of storing credentials on each repository, store them in Passphrase. This allows easy creation/management of many repositories which share credentials.

Test Plan:
  - Upgraded repositories.
  - Created and edited repositories.
  - Pulled HTTP and SSH repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230, T4122

Differential Revision: https://secure.phabricator.com/D7629
2013-11-22 15:23:33 -08:00
Saulius Zemaitaitis
4910a36563 Set reasonable defaults when displaying remote repository URIs.
Summary: Show SSH user on git-over-ssh repositories and hide both username and password for other repos.

Test Plan: View repository details page in diffusion, Clone URI should appear with a username (taken from repo config) and any http(s) repos should be without usernames.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T4147

Differential Revision: https://secure.phabricator.com/D7631
2013-11-22 11:22:39 -08:00
epriestley
e99c53da2e Fix an issue with SVN path construction in the presence of subpath configuration
Summary: D7590 made path construction more consistent, but affected this callsite if a subpath is configured. Currently, we end up with double `@@` in the URI.

Test Plan:
  - Ran unit tests.
  - Ran `bin/repostitory discover`.

Reviewers: staticshock, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7619
2013-11-21 14:41:38 -08:00
epriestley
a07f444f2a Fix git "origin" remote in more circumstances
Summary:
Fixes T4041. We currently detect when "origin" is incorrect, but can do better:

  - When "origin" is missing, we can add it. This happens for Git 1.7.1 -- see T4041.
  - When "origin" is wrong, we can fix it automatically if we control the repository.

We only need to fail when origin exists, is wrong, and we aren't in charge of the repository.

Test Plan: Ran `bin/repository discover X` on a repository with a good origin, no origin, a bad-but-under-control origin, and a bad-out-of-control origin. Got the right behavior in all cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, champo

Maniphest Tasks: T4041

Differential Revision: https://secure.phabricator.com/D7614
2013-11-20 10:41:56 -08:00
epriestley
ff8b48979e Simplify Repository remote and local command construction
Summary:
This cleans up some garbage:

  - We were specifying environmental variables with `X=y git ...`, but now have `setEnv()` on both `ExecFuture` and `PhutilExecPassthru`. Use `setEnv()`.
  - We were specifying the working directory with `(cd %s && git ...)`, but now have `setCWD()` on both `ExecFuture` and `PhutilExecPassthru`. Use `setCWD()`.
  - We were specifying the Git credentials with `ssh-agent -c (ssh-add ... && git ...)`. We can do this more cleanly with `GIT_SSH`. Use `GIT_SSH`.
  - Since we have to write a script for `GIT_SSH` anyway, use the same script for Subversion and Mercurial.

This fixes two specific issues:

  - Previously, we were not able to set `-o StrictHostKeyChecking=no` on Git commands, so the first time you cloned a git repo the daemons would generally prompt you to add `github.com` or whatever to `known_hosts`. Since this was non-interactive, things would mysteriously hang, in effect. With `GIT_SSH`, we can specify the flag, reducing the number of ways things can go wrong.
  - This adds `LANG=C`, which probably (?) forces the language to English for all commands. Apparently you need to install special language packs or something, so I don't know that this actually works, but at least two users with non-English languages have claimed it does (see <https://github.com/facebook/arcanist/pull/114> for a similar issue in Arcanist).

At some point in the future I might want to combine the Arcanist code for command execution with the Phabricator code for command execution (they share some stuff like LANG and HGPLAIN). However, credential management is kind of messy, so I'm adopting a "wait and see" approach for now. I expect to split this at least somewhat in the future, for Drydock/Automerge if nothing else.

Also I'm not sure if we use the passthru stuff at all anymore, I may just be able to delete that. I'll check in a future diff.

Test Plan: Browsed and pulled Git, Subversion and Mercurial repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7600
2013-11-20 10:41:35 -08:00
epriestley
08bdfacff3 Make Subversion URI construction more consistent
Summary:
Ref T2230. SVN has some weird rules about path construction. Particularly, if you're missing a "/" in the remote URI right now, the change parsing step doesn't build the right paths.

Instead, build the right paths more intelligently.

Test Plan: Added and executed unit tests. Imported an SVN repo.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, jpeffer

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7590
2013-11-20 10:41:25 -08:00
Aviv Eyal
dcf909ba56 Land to GitHub + support stuff
Summary:
A usable, Land to GitHub flow.

Still to do:
- Refactor all git/hg stratagies to a sane structure.
- Make the dialogs Workflow + explain why it's disabled.
- Show button and request Link Account if GH is enabled, but user is not linked.
- After refreshing token, user ends up in the settings stage.

Hacked something in LandController to be able to show an arbitrary dialog from a strategy.
It's not very nice, but I want to make some more refactoring to the controller/strategy/ies anyway.

Also made PhabricatorRepository::getRemoteURIObject() public, because it was very useful in getting
the domain and path for the repo.

Test Plan:
Went through these flows:
- load revision in hosted, github-backed, non-github backed repos to see button as needed.
- hit land with weak token - sent to refresh it with the extra scope.
- Land to repo I'm not allowed - got proper error message.
- Successfully landed; Failed to apply patch.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T182

Differential Revision: https://secure.phabricator.com/D7555
2013-11-13 17:25:24 -08:00
epriestley
f5ca647d2c Add bin/repository edit for CLI repository editing
Summary:
Ref T4039. This is mostly to deal with that, to prevent the security issues associated with mutable local paths. The next diff will lock them in the web UI.

I also added a confirmation prompt to `bin/repository delete`, which was a little scary without one.

See one comment inline about the `--as` flag. I don't love this, but when I started adding all the stuff we'd need to let this transaction show up as "Administrator" it quickly got pretty big.

Test Plan: Ran `bin/repository edit ...`, saw an edit with a transaction show up on the web UI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4039

Differential Revision: https://secure.phabricator.com/D7579
2013-11-13 11:26:05 -08:00
epriestley
d4eca25774 Don't implement SVN over HTTP
Summary:
Ref T2230. As far as I can tell, getting SVN working over HTTP is incredibly complicated. It's all DAV-based and doesn't appear to have any kind of binary we can just execute and pass requests through to. Don't support it for now.

  - Disable it in the UI.
  - Make sure all the error messages are reasonable.

Test Plan: Tried to HTTP an SVN repo. Tried to clone a Git repo with SVN, got a good error message.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7562
2013-11-11 16:10:41 -08:00
Jakub Vrana
a29b5b070f Replace some hsprintf() by phutil_tag()
Test Plan: Looked at a diff with inline comment.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Differential Revision: https://secure.phabricator.com/D7549
2013-11-11 09:23:23 -08:00
James Rhodes
7ffea0463e Use herald to trigger builds of revisions and commits.
Summary:
Depends on D7500.

This seemed like a pretty good idea once I thought of it.  Instead of having some custom triggering logic instead Harbormaster, I figured it best to leverage all of Herald's power so that users can create rules to apply builds to commits and differential revisions.  This gives the added advantage that they can trigger off builds for particular types of revisions and commits, which seems like it could be really useful (e.g. run extra tests against revisions that touch sensitive areas of the code).

Test Plan: Ran the usual daemons + the Harbormaster daemon.  Pushed a commit to the repository and saw both the buildable and build get created when the commit worked picked it up.  Submitted a diff and saw both the buildable and build get created when the Herald rules were evaluated for the diff.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley

CC: Korvin, epriestley, aran, hwinkel

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D7501
2013-11-08 16:58:39 -08:00
epriestley
1c73d6cd06 Tighten a regular expression in repository.create
Summary:
See <https://github.com/facebook/phabricator/issues/433>. We were missing a "^" here.

This should be moved over to transactions soon and then we can get rid of the duplication. :/

Test Plan: Tried to create a repository with callsign "9X", got a helpful error about "ALL UPPERCASE LETTERS".

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7531
2013-11-08 11:09:09 -08:00
epriestley
5b873a74de Move Mercurial discovery to PhabricatorRepositoryDiscoveryEngine
Summary: Ref T4068. Partly, this moves discovery to the more unit-testable PhabricatorRepositoryDiscoveryEngine. It also fixes some issues, see inlines.

Test Plan: In a Mercurial repository, ran `bin/repository discover --repair`, verified commits came out topographically sorted. Ran without `--repair` and in various other contexts, like with no commits to discover and some-but-not-all commits to discover.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4068

Differential Revision: https://secure.phabricator.com/D7518
2013-11-06 17:20:22 -08:00
epriestley
bd29784a32 Add an administrative bin/repository importing command to list importing commits
Summary: Ref T4068. Adds a command to list all commits in an "importing" status. This will allow users to use `reparse.php` to diagnose and repair issues.

Test Plan:
  - Ran `bin/repository importing P`, etc.
  - Used `reparse.php` to reparse some commit stages and saw status update correctly.
  - Ran on a repo with no importing commits.
  - Ran with `... --simple | xargs`, which saves us having to put an `awk` or something in there for users.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4068

Differential Revision: https://secure.phabricator.com/D7515
2013-11-06 11:26:41 -08:00
epriestley
e3a5ab1f8c Add an administrative bin/repository mark-imported command
Summary:
Ref T4068. In some cases like that one, I anticipate a repository not fully importing when a handful of random commits are broken. In the long run we should just deal with that properly, but in the meantime provide an administrative escape hatch so you can mark the repository as imported and get it running normally.

The major reason to do this is that Herald, Feed, Harbormaster, etc., won't activate until a repository is "imported".

Test Plan:
  - Tried to mark an imported repository as imported, got an "already imported" message.
  - Same for not-imported.
  - Marked a repository not-imported.
  - Marked a repository imported.
  - Marked a repository not-imported, then waited for the daemons to mark it imported again automatically.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, kbrownlees

Maniphest Tasks: T4068

Differential Revision: https://secure.phabricator.com/D7514
2013-11-06 11:26:24 -08:00
epriestley
71e6386a73 Fix issue with new branch query data getting used incorrectly
See <https://github.com/facebook/phabricator/issues/429>
2013-11-05 09:59:31 -08:00
epriestley
b90e51ab0e Clean up hg --debug branches calls
Summary: Ref T1493. Consolidate these a bit; they might need some more magic once we do `--noupdate` checkouts. Mostly just trying to clean up and centralize this code a bit.

Test Plan: Viewed and `bin/repository discover`'d Mercurial repos with and without any branches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1493

Differential Revision: https://secure.phabricator.com/D7480
2013-11-04 12:15:32 -08:00
epriestley
cd674931fc When creating a repository in Diffusion, prompt for "Create" or "Import" first
Summary:
Ref T2230. This will need some more refinement, but basically it adds a "Create" vs "Import" step before we go through the paged workflow.

  - If you choose "Create", we skip the remote URI / auth stuff, and then set the "hosted" flag.
  - If you choose "Import", we do what we do now.

Test Plan: Created and imported repos.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7475
2013-11-01 17:39:35 -07:00
epriestley
3607bd487c Survive pull/discover for hosted repositories in all VCSes
Summary:
Hosted repositories only sometimes survive the pull/discover phases right now, due to issues like:

  - Pull tries to `git clone`, but should `git init`.
  - Mercurial doesn't handle empty repositories with on branches.
  - SVN tries to connect to an invalid remote.
  - None of them set the INIT repo flag correctly, so status doesn't get updated properly in the UI.

Fix all this stuff.

Test Plan:
  - For each of Git, SVN and Mercurial:
    - Created a new repository from the web UI in a deactivated state.
    - Made it hosted.
    - Manually ran pull/discover.
    - Verified we end up with initialized, empty repositories in consistent states.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7474
2013-11-01 17:36:30 -07:00
epriestley
a0e820ad9a Improve repository hinting and feedback
Summary:
  - Warn about "Read/Write" instead of disabling it, to prevent edits which mutate it after changing a hosted repository to an unhosted one.
  - Warn about authenticated connections with HTTPS auth disabled, and link to the relevant setting.
  - When "Autoclose" is disabled, show that "Autoclose Branches" won't have an effect.
  - For hosted repositories, show the HTTP and SSH clone URIs.
    - Make them easy to copy/paste.
    - Link to credential management.
    - Show if they're read-only.
    - This could be a bit nicer-looking than it is.

Test Plan: Looked at repositories in a bunch of states and made various edits to them.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7471
2013-11-01 17:35:43 -07:00
epriestley
d1649d176f Mark IMPORTED_CHANGE more consistently
Summary:
See <https://github.com/facebook/phabricator/issues/425>. There are some ways that the change parsers may not reach `finishParse()`, but we now need them to in order to mark the commit imported, advance the progress bar, and eventually kick the repository out of IMPORTING status.

Take all the copy/pasted code in the parsers and move it into the parent. Specifically, this is:

  - Printing a status message about starting a parse;
  - checking for bad commits;
  - queueing the next parse stage; and
  - marking the import step complete.

Test Plan: Used `reparse.php --change` to reparse Git, SVN and Mercurial repos.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7470
2013-11-01 12:54:10 -07:00
epriestley
0278b15ceb Implementation of VCS passwords against user.
Summary: This allows users to set their HTTP access passwords via Diffusion interface.

Test Plan: Clicked the "Set HTTP Access Password" link, set a password and saw it appear in the DB.

Reviewers: #blessed_reviewers, hach-que, btrahan

Reviewed By: hach-que

CC: Korvin, epriestley, aran, jamesr

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7462
2013-11-01 08:34:11 -07:00
epriestley
40b0818207 Show additional status information during repository import
Summary:
Ref T2350. Fixes T2231.

  - Adds log flags around discovery.
  - Adds message flags for "needs update". This is basically an out-of-band hint to the daemons that a repository should be pulled sooner than normal. We set the flag when users push a revision, and expose a Conduit method that `arc land` will be able to use.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2350, T2231

Differential Revision: https://secure.phabricator.com/D7467
2013-10-31 15:46:57 -07:00
epriestley
3a39b01233 Add "RepositoryStatusMessage" and detailed information about initilization
Summary:
`RepositoryStatusMessage` is basically a key/value table associated with a repository that I'm using to let the daemons store the most recent event of a given type, so we can easily show it on the status dashboard. I think this will be a lot easier for users to figure out than digging through logfiles.

I'm also going to write the "this needs a pull" status here eventually, for reducing the time lapse between pushes and discovery.

  - Add storage for these messages.
  - Have the pull engine populate the INIT phase. I'll do the FETCH phase next.
  - Update the status readout to show all the various states.

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7461
2013-10-30 16:04:19 -07:00
epriestley
5ef4954a36 Remove "daemons not running" code from Repositories
Summary: This moved into Diffusion in D7458 and is now presented in a much cleaner, more targeted way.

Test Plan: Loaded `/repository/`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7459
2013-10-30 13:15:41 -07:00
epriestley
2d01da14fd Provide detailed status information about repository state/progress
Summary:
Replace the blanket "daemons not running" warning with a lot more specific detail, to try to make it easier for users to figure out how to set up repositories correctly.

The next change here will add some additional status information from the daemons, so this panel can report results in greater detail.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7458
2013-10-30 13:15:32 -07:00
epriestley
d51ae49f61 Clean up Diffusion dedicated tag table view
Summary: Minor cleanup. Make the "imported" check less strict (we don't need owners or herald to show change status). Export the "imported" flag over Conduit.

Test Plan: Viewed tag table. Viewed partially imported repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7455
2013-10-30 13:15:14 -07:00
epriestley
f08908ff35 Raise a setup warning for missing or invalid local repository directory
Summary: I'm planning to add more detailed info to Diffusion itself, but catch the big issue here.

Test Plan: Hit config issue locally, then resolved it.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7439
2013-10-30 13:07:09 -07:00
epriestley
d5bc8197ec Remove "isUnparsed" flag from commits
Summary: The new "importStatus" property provides a much stronger and more consistent version of this flag. The only callsite was removed by D7452.

Test Plan: Used `grep` to check for callsites and found none.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7453
2013-10-30 13:06:57 -07:00
epriestley
d1c4b5081c Clean up Diffusion branch query a bit
Summary:
Ref T2716.

  - Serve from `DiffusionCommitQuery`, not `PhabricatorAuditCommitQuery` (which should probably die).
  - Fix logic for `limit`, which incorrectly failed to display the "Showing %d branches." text.
  - Clean up things a touch.
  - I didn't end up actually needing `needCommitData()`, but left it in there since I think it will be needed soon.
  - Removed a "TODO" because I don't remember what "etc etc" means.

Test Plan: Looked at branches in several repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2716

Differential Revision: https://secure.phabricator.com/D7451
2013-10-30 13:06:28 -07:00
epriestley
3bf372c60c Consolidate querying of things which we can use git for-each-ref for
Summary: Ref T2230. This cleans up D7442, by using `git for-each-ref` everywhere we can, in a basically reasonable way.

Test Plan:
In bare and non-bare repositories:

  - Ran discovery with `bin/repository discover`;
  - listed branches on `/diffusion/X/`;
  - listed tags on `/diffusion/X/`;
  - listed tags, branches and refs on `/diffusion/rXnnnn`.

Reviewers: btrahan, avivey

Reviewed By: avivey

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7447
2013-10-30 13:06:09 -07:00
epriestley
19124554d8 Fix various branch/ref issues with bare repositories in Git
Summary:
Ref T2230. Although all the non-bare commands //run// fine in bare repos, not all of them do exactly the same thing.

This could use further cleanup, but at least get it working again for now.

Test Plan: Ran `bin/repository pull`, `bin/repository discover`, viewed Diffusion (looked at branch table), viewed a commit (looked at "Branches"), for bare and non-bare git repos.

Reviewers: avive, btrahan, avivey

Reviewed By: avivey

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7442
2013-10-29 21:04:38 -07:00
Dean Reilly
62edbb8e21 Treat relocated files as added files in svn worker.
Summary:
Relocated files aren't treated as newly created files by the worker. This
can lead to the worker trying to look up information about deleted files
in the wrong location.

Test Plan: See T4030

Reviewers: #blessed_reviewers, epriestley

Reviewed By: epriestley

CC: epriestley, aran

Maniphest Tasks: T4030

Differential Revision: https://secure.phabricator.com/D7432
2013-10-29 20:11:06 -07:00
epriestley
59922b78b9 Make Phabricator clone bare git repositories
Summary:
This doesn't really impact anything very much, but is a little cleaner than cloning repositories with a working copy. It's somewhat important for allowing pushes, because you can't push to a checked-out branch.

Mercurial has a similar option (`--noupdate`) but leave that alone for now.

The origin stuff was mostly for sanity/explicitness purposes -- I believe it's safe to remove in all non-ridiculous cases. Git fails with it in bare repositories (it automatically creates an `origin`, but doesn't create the local refs for it, or something).

Test Plan: Nuked a repo, re-cloned it, pulled and updated it several times. Browsed both bare and non-bare repos in Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7430
2013-10-29 15:32:41 -07:00
epriestley
d8bda7c66e Add an "importing" state to repositories and clean up the UI
Summary:
Fixes T3217. Ref T776. Ref T1493. Broadly, this introduces a mechanism which works like this:

  - When a repository is created, we set an "importing" flag.
  - After discovery completes, we check if a repository has no importing commits. Basically, this is the first time we catch up to HEAD.
  - If we're caught up, clear the "importing" flag.

This flag lets us fix some issues:

  - T3217. Currently, when you import a new repository and users have rules like "Email me on every commit ever" or "trigger an audit on every commit", we take a bunch of publish actions. Instead, implicitly disable publishing during import.
  - An imported but un-pulled repository currently has an incomprehensible error on `/diffusion/X/`. Fix that.
  - Show more cues in the UI about importing.
  - Made some exceptions more specific.

Test Plan:
This is the new screen for a completely new repo, replacing a giant exception:

{F75443}

  - Created a repository, saw it "importing".
  - Pulled and discovered it.
  - Processed its commits.
  - Ran discovery again, saw import flag clear.
  - Also this repository was empty, which hit some of the other code.

This is the new "parsed empty repository" UI, which isn't good, but is less broken:

{F75446}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, hach-que

Maniphest Tasks: T3607, T1493, T776, T3217

Differential Revision: https://secure.phabricator.com/D7429
2013-10-29 15:32:41 -07:00
epriestley
9125095587 Distinguish between empty and unparsed commits in Diffusion
Summary:
Fixes T3416. Fixes T1733.

  - Adds a flag to the commit table showing whether or not we have parsed it.
  - The flag is set to `0` initially when the commit is discovered.
  - The flag is set to `1` when the changes are parsed.
  - The UI can now use the flag to distinguish between "empty commit" and "commit which we haven't imported changes for yet".
  - Simplify rendering code a little bit.
  - Fix an issue with the Message parser for empty commits.
  - There's a key on the flag so we can do `SELECT * FROM repository_commit WHERE repositoryID = %d AND importStatus = 0 LIMIT 1` soon, to determine if a repository is fully imported or not. This will let us improve the UI (Ref T776, Ref T3217).

Test Plan:
  - Ran `bin/storage upgrade -f`.
  - Created an empty commit.
  - Without the daemons running, ran `bin/repository pull GTEST` and `bin/repository discover GTEST`.
  - Viewed web UI to get the first screenshot ("Still Importing...").
  - Ran the message and change steps with `scripts/repository/reparse.php`.
  - Viewed web UI to get the second screenshot ("Empty Commit").

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776, T1733, T3416, T3217

Differential Revision: https://secure.phabricator.com/D7428
2013-10-29 15:32:41 -07:00
epriestley
0965f0041a Don't pull updates for repositories marked as hosted
Summary:
  - Don't try to pull hosted repos.
  - Also, fix the `--verbose` + `--trace` interaction for `bin/repository`.
  - Also, fix a couple of unit tests which got tweaked earlier.

Test Plan:
  $ ./bin/repository pull GTEST --verbose
  Pulling 'GTEST'...
  Repository "GTEST" is hosted, so Phabricator does not pull updates for it.
  Done.

Reviewers: btrahan, hach-que

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7427
2013-10-29 15:32:41 -07:00
epriestley
c7f23f522a Accept and route VCS HTTP requests
Summary:
Mostly ripped from D7391, with some changes:

  - Serve repositories at `/diffusion/X/`, with no special `/git/` or `/serve/` URI component.
    - This requires a little bit of magic, but I got the magic working for Git, Mercurial and SVN, and it seems reasonable.
    - I think having one URI for everything will make it easier for users to understand.
    - One downside is that git will clone into `X` by default, but I think that's not a big deal, and we can work around that in the future easily enough.
  - Accept HTTP requests for Git, SVN and Mercurial repositories.
  - Auth logic is a little different in order to be more consistent with how other things work.
  - Instead of AphrontBasicAuthResponse, added "VCSResponse". Mercurial can print strings we send it on the CLI if we're careful, so support that. I did a fair amount of digging and didn't have any luck with git or svn.
  - Commands we don't know about are assumed to require "Push" capability by default.

No actual VCS data going over the wire yet.

Test Plan:
Ran a bunch of stuff like this:

  $ hg clone http://local.aphront.com:8080/diffusion/P/
  abort: HTTP Error 403: This repository is not available over HTTP.

...and got pretty reasonable-seeming errors in all cases. All this can do is produce errors for now.

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7417
2013-10-29 15:32:40 -07:00
epriestley
bb35f8ec9f Add hosting, serving, and push policy options to repository edit
Summary:
Basically straight from D7391. The differences are basically:

  - Policy stuff is all application-scope instead of global-scope.
  - Made a few strings a little nicer.
  - Deleted a bit of dead code.
  - Added a big "THIS DOESN'T WORK YET" warning.

Test Plan: See screenshots.

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7416
2013-10-29 15:32:40 -07:00
epriestley
86fe020a97 Add global "push" policy to Repositories
Summary: No editing or view yet, just adds the schema and a policy default. Part of D7391.

Test Plan: `bin/storage upgrade`

Reviewers: hach-que, btrahan

Reviewed By: hach-que

CC: aran

Maniphest Tasks: T2230

Differential Revision: https://secure.phabricator.com/D7415
2013-10-29 15:32:40 -07:00
epriestley
ba63c5e426 Ship "Repositories" create button to new Diffusion workflow
Summary: Ref T2231. Get rid of the old create controller and make the button go to the new stuff instead. This will eventually get cleaned up more, but I don't have a clear plan for Arcanist Projects yet.

Test Plan: Clicked button, hit new workflow.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7414
2013-10-29 15:32:39 -07:00
epriestley
5f6ea9f662 Activate the new Repository creation workflow
Summary:
Ref T2231. This:

  - Activates the new multi-step workflow, and exposes it in the UI.
  - Adds "can create", "default view" and "default edit" capabilities.
  - Provides a default value for `repository.default-local-path` and forces repositories into it by default. It's still editable, but Phabricator gets it correct (for some definition of correct) by default now.

Test Plan: Created some new repositories with the new workflow.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1286, T2231

Differential Revision: https://secure.phabricator.com/D7413
2013-10-29 15:32:39 -07:00
epriestley
c585f97e90 Remove horrible old repository edit controller
Summary:
Ref T2231. I didn't port these options over, so they're still supported but have no edit UI:

  - Pull Frequency (confusing/not useful, I think?)
  - Default Owners Path (probably used only by Facebook and only in the E repository)
  - Show user in public repository URL (probably mostly obsolete with hosting?)

We can add those back if users notice, but they seem like the three least useful options so I'm going to see if we can get away with removing them.

Test Plan: Clicked "Edit" from Repositories, got kicked into the nice new Diffusion edit UI instead of the old one.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7410
2013-10-29 15:32:39 -07:00
epriestley
d1ed816e61 Move "Delete Repository" stuff to Diffusion
Summary: Ref T2231. This just moves the "Delete" dialog from Repositories to Diffusion. This dialog just shows instructions and isn't interesting.

Test Plan: {F75093}

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7412
2013-10-29 12:26:07 -07:00
epriestley
c4cdb5c5f0 Move editing "Local Path" to modern UI/controller/etc
Summary: Fixes T1286. Ref T2231. See previous diffs; same as the others but does "Local Path".

Test Plan: See screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1286, T2231

Differential Revision: https://secure.phabricator.com/D7409
2013-10-29 12:20:26 -07:00
epriestley
b57b72368c Move "Remote URI" to modern repository editor thing
Summary:
Ref T2231. Allows you to edit the remote URI and credentials.

This is a little bit funky because I'm reusing some of the pages on the new (not-yet-hooked-up) create form. Specifically, it had pages like this:

  - Repo Type
  - Name/Callsign/Remote
  - Auth
  - Done

I split "Name/Callsign/Remote" into "Name/Callsign" and "Remote", then when editing the remote I just take you through "Remote" and "Auth" and then back. This lets us reuse the giant pile of protocol/URI sanity checking logic and ends up being pretty clean, although it's a little weird that the "Create" controller does both full-create and edit-remote.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7405
2013-10-25 13:59:02 -07:00
epriestley
52d4e66883 Move repository actions (notify, autoclose) to new UI
Summary: Ref T2231. Brings "Notify/Publish" and "Autoclose" to the new UI.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7402
2013-10-25 13:58:15 -07:00
epriestley
49670e1a56 Move Subversion repository information to the new interface
Summary: Ref T2231. Moves "UUID" and "Subpath/Import Only" to Diffusion.

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7400
2013-10-25 13:58:03 -07:00
epriestley
dcb0b1b64f Add support for branch-related configuration to new Repository edit workflow
Summary: Ref T2231. Modernizes editing "Default Branch", "Track Only", and "Autoclose Only".

Test Plan: See screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D7399
2013-10-25 13:57:14 -07:00
Bob Trahan
da84546058 Add filter by object ability to flag query
Summary: See title. Fixes T1809.

Test Plan:
verified each type that has flaggable interface still can be flagged

verified that new custom query filter works

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, epriestley, aran

Maniphest Tasks: T1809

Differential Revision: https://secure.phabricator.com/D7392
2013-10-25 12:52:00 -07:00
epriestley
2a5c987c71 Lock policy queries to their applications
Summary:
While we mostly have reasonable effective object accessibility when you lock a user out of an application, it's primarily enforced at the controller level. Users can still, e.g., load the handles of objects they can't actually see. Instead, lock the queries to the applications so that you can, e.g., never load a revision if you don't have access to Differential.

This has several parts:

  - For PolicyAware queries, provide an application class name method.
  - If the query specifies a class name and the user doesn't have permission to use it, fail the entire query unconditionally.
  - For handles, simplify query construction and count all the PHIDs as "restricted" so we get a UI full of "restricted" instead of "unknown" handles.

Test Plan:
  - Added a unit test to verify I got all the class names right.
  - Browsed around, logged in/out as a normal user with public policies on and off.
  - Browsed around, logged in/out as a restricted user with public policies on and off. With restrictions, saw all traces of restricted apps removed or restricted.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7367
2013-10-21 17:20:27 -07:00
epriestley
00bf47f973 Fix "Manage herald rules" link by removing it
Summary: Fixes T4001. I broke this some time ago and no one has complained. I don't think it gets much use, and we haven't added it for the newer apps. Just get rid of it rather than adapt the URIs for ApplicationSearch.

Test Plan: Unit tests, sent myself some email.

Reviewers: zeeg, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4001

Differential Revision: https://secure.phabricator.com/D7355
2013-10-21 16:58:56 -07:00
epriestley
7180199d13 Fix daemon auth issue
Summary: I touched this code recently but it needs an unusual special case because we call through with the "omnipotent user" from the daemons. As per the TODO below, this will all get cleaned up at some point.

Test Plan: Will make @poop verify.

Reviewers: btrahan, poop

Reviewed By: poop

CC: poop, aran

Differential Revision: https://secure.phabricator.com/D7356
2013-10-18 18:29:36 -07:00
epriestley
073cb0e78c Make PhabricatorPolicyInterface require a getPHID() method
Summary:
Ref T603. This cleans up an existing callsite in the policy filter, and opens up some stuff in the future.

Some policy objects don't have real PHIDs:

  PhabricatorTokenGiven
  PhabricatorSavedQuery
  PhabricatorNamedQuery
  PhrequentUserTime
  PhabricatorFlag
  PhabricatorDaemonLog
  PhabricatorConduitMethodCallLog
  ConduitAPIMethod
  PhabricatorChatLogEvent
  PhabricatorChatLogChannel

Although it would be reasonable to add real PHIDs to some of these (like `ChatLogChannel`), it probably doesn't make much sense for others (`DaemonLog`, `MethodCallLog`). Just let them return `null`.

Also remove some duplicate `$id` and `$phid` properties. These are declared on `PhabricatorLiskDAO` and do not need to be redeclared.

Test Plan: Ran the `testEverythingImplemented` unit test, which verifies that all classes conform to the interface.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7306
2013-10-14 14:35:47 -07:00
epriestley
0598600476 Always pass handles to tokenizers, not <phid -> name> maps
Summary: Ref T1279. Prerequisite for adding icons or other type information to tokenizers, since we don't currently have enough information to prefill them when rendering things from the server side. By passing handles in, the tokenizer can extract type information.

Test Plan:
- Searched by user in Audit.
- Sent Conpherence from profile page.
- Tried to send an empty conpherence.
- Searched Countdown by user.
- Edited CCs in Differential.
- Edited reviewers in Differential.
- Edited a commit's projects.
- Searched lint by owner.
- Searched feed by owner/project.
- Searched files by owner.
- Searched Herald by owner.
- Searched Legalpad by owner.
- Searched Macro by owner.
- Filtered Maniphest reports by project.
- Edited CCs in Maniphest.
- Searched Owners by owner.
- Edited an Owners package.
- Searched Paste by owner.
- Searched activity logs by owner.
- Searched for mocks by owner.
- Edited a mock's CCs.
- Searched Ponder by owner.
- Searched projects by owner.
- Edited a Releeph project's pushers.
- Searched Releeph by requestor.
- Edited "Uses Symbols" for an Arcanist project.
- Edited all tokenizers in main search.
- Searched Slowvote by user.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7248
2013-10-07 12:51:24 -07:00
epriestley
953ff197bf Allow Herald rules to be disabled, instead of deleted
Summary:
Ref T603. Ref T1279. Further improves transaction and policy support for Herald.

  - Instead of deleting rules (which wipes out history and can't be undone) allow them to be disabled.
  - Track disables with transactions.
  - Gate disables with policy controls.
  - Show policy and status information in the headers.
  - Show transaction history on rule detail screens.
  - Remove the delete controller.
  - Support disabled queries in the ApplicationSearch.

Test Plan:
  - Enabled and disabled rules.
  - Searched for enabled/disabled rules.
  - Verified disabled rules don't activate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279, T603

Differential Revision: https://secure.phabricator.com/D7247
2013-10-06 17:10:29 -07:00
epriestley
2d733f88a1 Split users apart from projects/packages in reviewer and audit UIs
Summary: Ref T1279. Show separate sections for "Reviewers" and "Project Reviewers" (Differential) and for "Auditors" and "Package/Project Auditors" (Diffusion/Audit).

Test Plan:
  - Looked at a commit. Saw separation.
  - Looked at a revision. Saw separation.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D7233
2013-10-05 14:10:49 -07:00
epriestley
13dae05193 Make most file reads policy-aware
Summary: Ref T603. Swaps out most `PhabricatorFile` loads for `PhabricatorFileQuery`.

Test Plan:
  - Viewed Differential changesets.
  - Used `file.info`.
  - Used `file.download`.
  - Viewed a file.
  - Deleted a file.
  - Used `/Fnnnn` to access a file.
  - Uploaded an image, verified a thumbnail generated.
  - Created and edited a macro.
  - Added a meme.
  - Did old-school attach-a-file-to-a-task.
  - Viewed a paste.
  - Viewed a mock.
  - Embedded a mock.
  - Profiled a page.
  - Parsed a commit with image files linked to a revision with image files.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7178
2013-09-30 09:38:13 -07:00
epriestley
7f0d0e4e6c Make more Diffusion controllers/views capability-sensitive
Summary:
Ref T603. I got most of this earlier, but finish it up.

  - Make a couple of controllers public; pretty much everything in Diffusion has implicit policy checks as a result of building a `DiffusionRequest`.
  - Add an "Edit" capability to commits.
  - Swap out the comment thing for commits.
  - Disable actions if the user can't take them.

Test Plan: Viewed a bunch of interfaces while logged out, got appropriate results or roadblocks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7152
2013-09-27 10:49:45 -07:00
epriestley
2e5ac128b3 Explain policy exception rules to users
Summary:
Ref T603. Adds clarifying text which expands on policies and explains exceptions and rules. The goal is to provide an easy way for users to learn about special policy rules, like "task owners can always see a task".

This presentation might be a little aggressive. That's probably OK as we introduce policies, but something a little more tempered might be better down the road.

Test Plan: See screenshot.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7150
2013-09-27 08:43:41 -07:00
epriestley
874a9b7fe3 When creating or updating a revision, infer the repository from the diff
Summary:
Ref T603. When a diff is attached to a revision, try to guess the repository if possible. In cases where we succeed, this automatically gives us intuitive policy behavior (i.e., you can see a revision if you can see the repository the change is against).

I pulled this into a funky little "Lookup" class for two reasons:

  - It's used in two places;
  - I anticipate that we might need to add some sort of `explainWhy()` method if users find the heuristics confusing.

Test Plan: Created and updated revisions, saw them pick up the correct repository association. Ran Herald dry run against associable and nonassociable revisions, saw correct values populate.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7147
2013-09-26 15:28:42 -07:00
epriestley
9b3d7b0dba Make most Differential reads policy-aware
Summary: Ref T603. Makes the majority of reads policy aware (and pretty much all the important ones).

Test Plan:
  - Created a comment with `differential.createcomment`.
  - Created a new revision with `arc diff` in order to exercise `differential.creatediff`.
  - Created an inline comment with `differential.createinline`.
  - Added a comment to a revision.
  - Edited an inline comment.
  - Edited a revision.
  - Wrote "Depends on ..." in a summary, saved, verified link was created.
  - Browsed a file in Diffusion.
  - Got past the code I changed in the Releeph request thing.
  - Edited a Releeph request.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7136
2013-09-26 12:37:19 -07:00
epriestley
79abe6653e Remove PhabricatorRepository::loadAllByPHIDOrCallsign()
Summary: Ref T603. Move to real Query classes.

Test Plan:
  - Ran `phd debug pull X` (where `X` does not match a repository).
  - Ran `phd debug pull Y` (where `Y` does match a repository).
  - Ran `phd debug pull`.
  - Ran `repository pull`.
  - Ran `repository pull X`.
  - Ran `repository pull Y`.
  - Ran `repository discover`.
  - Ran `repository delete`.
  - Ran `grep`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7137
2013-09-26 12:36:24 -07:00
epriestley
c467cc464f Make most repository reads policy-aware
Summary: Ref T603. This swaps almost all queries against the repository table over to be policy aware.

Test Plan:
  - Made an audit comment on a commit.
  - Ran `save_lint.php`.
  - Looked up a commit with `diffusion.getcommits`.
  - Looked up lint messages with `diffusion.getlintmessages`.
  - Clicked an external/submodule in Diffusion.
  - Viewed main lint and repository lint in Diffusion.
  - Completed and validated Owners paths in Owners.
  - Executed dry runs via Herald.
  - Queried for package owners with `owners.query`.
  - Viewed Owners package.
  - Edited Owners package.
  - Viewed Owners package list.
  - Executed `repository.query`.
  - Viewed "Repository" tool repository list.
  - Edited Arcanist project.
  - Hit "Delete" on repository (this just tells you to use the CLI).
  - Created a repository.
  - Edited a repository.
  - Ran `bin/repository list`.
  - Ran `bin/search index rGTESTff45d13dffcfb3ea85b03aac8cc36251cacdf01c`
  - Pushed and parsed a commit.
  - Skipped all the Drydock stuff, as it it's hard to test and isn't normally reachable.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7132
2013-09-25 16:54:48 -07:00
Chad Little
9be7a948f9 Move PHUIFormBoxView to PHUIObjectBoxView
Summary: I'd like to reuse this for other content areas, renaming for now. This might be weird to keep setForm, but I can fix that later if we need.

Test Plan: reload a few forms in maniphest, projects, differential

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7120
2013-09-25 11:23:29 -07:00
epriestley
b928e7f8f5 Fix an issue with Diffusion where paging by commit date would fail
Summary: When loading the cursor repository, we need to load the most recent
commit too if we're paging by commit date. This fixes a fatal for installs
with more than 100 repositories.

Auditors: btrahan
2013-09-23 18:25:06 -07:00
epriestley
d63789e4b2 Allow repository policies to be edited
Summary: Ref T603. Allows permitted users to set view and edit policies for repositories. So far the repository list, repository detail, repository edit, and browse interfaces respect these settings. Most other interfaces will respect stricter settings, but "Public" won't work. Lots of rough edges in the integration still. None of this makes policies any looser than they were already without explicit user intervention, so I just put a warning about it in the UI.

Test Plan: Set a repository to public and browsed it. Verified I could not access non-public repositories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, davidressman

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7061
2013-09-23 12:53:41 -07:00
epriestley
e7a7e43104 Fix a bug where policy queries with cursor-based pagers and non-ID orders can go into infinite loops
Summary:
Ref T603. See inlines for an explanation. The case where I hit this was loading the "Pending Differential Revisions" panel in Diffusion when logged out, after making a repository public.

What happens is that we load 10 revisions (say, D1 .. D10) but the user can't see any of them. We then try to load the next 10, but since the pagination is ordered by date modified, we need to base the next query on the modified date of the last thing we loaded (D10). However, since we use the viewer's policies to load that cursor object, it fails to load, and then we just issue the same query over and over again, loading D1 .. D10 until we run out of execution time.

Test Plan: Interface now loads correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7059
2013-09-21 16:23:44 -07:00
epriestley
63818818cb Allow repositories to be activate/deactivated in a transaction-aware way
Summary: Adds activate/deactivate action plus transactions.

Test Plan: See screenshot.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7057
2013-09-21 16:23:35 -07:00
epriestley
b8cb6ddaa5 Add some keys and policy fields to repositories
Summary:
  - Add some TODO'd keys.
  - Add policy fields.

Test Plan: Viewed repositories; created a new repository and verified it got the right default policy settings.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7056
2013-09-21 16:23:01 -07:00
epriestley
8651e5feba Allow repositories to be filtered by type
Summary: Allows the user to query for repos by VCS type.

Test Plan: See screenshot.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7038
2013-09-19 11:56:48 -07:00
epriestley
7d26252a3f Use PhutilBugtraqParser in Phabricator
Summary: Fixes T3840. Depends on D7021. See task for discussion. Also improved some config/help stuff.

Test Plan: See screenshot.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3840

Differential Revision: https://secure.phabricator.com/D7022
2013-09-18 10:13:00 -07:00
epriestley
256fcf3721 Make it easier to construct multi-column paging clauses from Query classes
Summary:
We currently have two giant messes for paging across multiple columns (usually because one column is not unique), and I'm about to add a third for Maniphest.

Provide a more structured way to build these `A > a OR (A = a AND B > b)` clauses.

Test Plan: Set page size to `2` for Differential and Diffusion and paged forward and backward with a bunch of different orders set. Pages worked as expected.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6971
2013-09-13 11:49:41 -07:00
epriestley
7a39ac43b4 Add a "list<regex>" config option and move regex config to it
Summary:
Fixes T3807. Several issues:

  - Currently, we split config of type `list<string>` on commas, which makes it impossible to enter a regex with a comma in it.
    - Split on newlines only.
  - Some of the examples are confusing (provided in JSON instead of the format you actually have to enter them).
    - Show examples in the same format you should enter text.
  - We didn't validate regexps.
    - Introduce `list<regex>` to validate regexes.

@hlau: Note that the old config format for the bugtraq stuff implied the delimiters on the regular expression. They are no longer implied. The examples show the correct format.

Test Plan: Viewed and edited affected config, hitting error and success cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: hlau, aran

Maniphest Tasks: T3807

Differential Revision: https://secure.phabricator.com/D6969
2013-09-13 11:48:00 -07:00
Bob Trahan
b902005bed Kill PhabricatorObjectDataHandle
Summary: Ref T603. Killing this class is cool because the classes that replace it are policy-aware. Tried to keep my wits about me as I did this and fixed a few random things along the way. (Ones I remember right now are pulling a query outside of a foreach loop in Releeph and fixing the text in UIExample to note that the ace of hearts if "a powerful" card and not the "most powerful" card (Q of spades gets that honor IMO))

Test Plan: tested the first few changes (execute, executeOne X handle, object) then got real mechanical / careful with the other changes.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran, FacebookPOC

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D6941
2013-09-11 12:27:28 -07:00
Bob Trahan
07b8becfc6 Policy - introduce parentQuery and pass around policy configuration from parent to child
Summary: Ref T603. Ref D6941.

Test Plan: Clicked around all over - looked good. I plan to re-test D6941 to make sure the executeOne case works now as intended

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D6944
2013-09-11 12:19:34 -07:00
epriestley
0da6321b2c Provide ordering options in Diffusion application search
Summary: Fixes T2298. Allows repositories to be ordered by name, callsign, commit, or date created. Slightly messy because of cursor paging.

Test Plan: Sorted commits.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2298

Differential Revision: https://secure.phabricator.com/D6919
2013-09-10 15:29:37 -07:00
epriestley
9872d57f87 Allow Diffusion repostories to be filtered by active/inactive status
Summary: Adds a status filter and makes the default query "active" repositories.

Test Plan: Used new filter to execute queries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6918
2013-09-10 15:26:23 -07:00
epriestley
904add9f44 Use ApplicationSearch in Diffusion
Summary:
Ref T2625. Switches Diffusion to ApplicationSearch. Notes:

  - Rendering is a bit rough, I'll clean that up next.
  - Ordering is a bit arbitrary, also coming shortly.

Test Plan: Used `/diffusion/` to execute various searches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6917
2013-09-10 15:26:08 -07:00
epriestley
93c6704059 Move "most recent commit" and "commit count" into DiffusionRepositoryQuery
Summary: Ref T2625. `DiffusionHomeController` currently runs these queries inline. Move them into `DiffusionRepositoryQuery`. Prepareds for ApplicationSearch.

Test Plan: Loaded `/diffusion/`, saw the same content as before.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2625

Differential Revision: https://secure.phabricator.com/D6914
2013-09-10 15:22:41 -07:00
Gareth Evans
fcba0c74d9 Replace all "attach first..." exceptions with assertAttached()
Summary:
Ref T3599
Go through everything, grep a bit, replace some bits.

Test Plan: Navigate around a bit

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3599

Differential Revision: https://secure.phabricator.com/D6871
2013-09-03 06:02:14 -07:00
epriestley
b932c606cb Implement "Add to CC" rule for Audits
Summary:
D6660 accidentally allowed you to build Herald rules for commits that take action "Add to CC", but provided no implementation.

Someone at Facebook then wrote such a rule.

Fix forward since there's no real reason not to allow this.

Test Plan: Used `./scripts/repository/reparse.php --herald rXnnnn` to trigger rules. Observed rule trigger and subsequent subscription.

Reviewers: wez, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6845
2013-08-29 17:03:40 -07:00
Chad Little
bb9be01d55 Update forms to use PHUIFormBoxView
Summary: Some more callsites, let me know if you see others, I think think is 98% of them now.

Test Plan: tested each page

Reviewers: epriestley, btrahan

Reviewed By: btrahan

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6814
2013-08-26 15:45:58 -07:00
Chad Little
fe2a96e37f Update Form Layouts
Summary:
This attempts some consistency in form layouts. Notably, they all now contain headers and are 16px off the sides and tops of pages. Also updated dialogs to the same look and feel. I think I got 98% of forms with this pass, but it's likely I missed some buried somewhere.

TODO: will take another pass as consolidating these colors and new gradients in another diff.

Test Plan: Played in my sandbox all week. Please play with it too and let me know how they feel.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6806
2013-08-26 11:53:11 -07:00
epriestley
f034fd80db Remove getApplicationObjectTypeName from ApplicationTransactions
Summary:
We can get this out of PHIDType reasonably in all cases and simplify implementation here.

None of these translate correctly anyway so they're basically debugging/development strings.

Test Plan: `grep`, browsed some transactions

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6786
2013-08-21 12:32:06 -07:00
epriestley
751cd547c2 Remove dust from page construction
Summary:
  ^\s+(['"])dust\1\s*=>\s*true,?\s*$\n

Test Plan: Looked through the diff.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6769
2013-08-19 18:09:35 -07:00
Bob Trahan
e1d2523efe make commit message worker savvier around revision ids
Summary: Fixes T2836

Test Plan: make a diff, get it approved, arc land, verify things okay. ask users on T2836 to try.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T2836

Differential Revision: https://secure.phabricator.com/D6770
2013-08-19 12:39:20 -07:00
epriestley
a0f0ba6acd Stop using process/filesystem-based checks to determine if daemons are running
Summary:
We currently check if daemons are running using the filesystem and process list. These checks reach the wrong result for a lot of users because their webservers can't read the filesystem or process list. They also reach the wrong result for daemons running on other machines.

Instead, query the active daemon list to see if daemons are running. This should be significantly more reliable.

(We didn't do this before because the running daemon list mechanism didn't exist when the check was written, and at the time it was more complex than doing a simple filesystem/process list thing.)

Test Plan: Viewed `/repositories/` with and without daemons running, saw appropriate warning or lack of warning.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6722
2013-08-12 11:20:22 -07:00
Bob Trahan
7225dc4525 Sort arcanist projects by name
Summary: Fixes T3691.

Test Plan: they be sorted now

Reviewers: epriestley

Reviewed By: epriestley

CC: edward, Korvin, aran

Maniphest Tasks: T3691

Differential Revision: https://secure.phabricator.com/D6714
2013-08-09 18:09:28 -07:00
epriestley
4f49ec1cff Remove HeraldDryRunAdapter
Summary: Ref T2769. This isn't a real adapter and its methods are increasingly hacky messes. Make "dry run" a first-class concept on the HeraldEngine instead and remove the adapter.

Test Plan: Ran Herald via test console and via CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6693
2013-08-07 18:04:40 -07:00
epriestley
b767bd3f2d Move Herald rule querying into HeraldRuleQuery
Summary: Ref T2769. The `HeraldRule` class has some query logic; move it into `HeraldRuleQuery`. Also some minor cleanup.

Test Plan: Ran test console, created a new revision, used `reparse.php --herald`. Verified rules triggered correctly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6689
2013-08-07 18:04:38 -07:00
epriestley
78eb81ffd0 Remove almost all instances of HeraldContentTypeConfig
Summary: Ref T2769. This cleans up almost every use of the HeraldContentTypeConfig class.

Test Plan: Viewed and edited Herald rules.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6662
2013-08-07 18:04:33 -07:00
epriestley
6badb05d64 Make Herald adapters provide content types
Summary:
Ref T2769. Get content types out of hard-coded config and into dynamic adapters.

This removes the "MERGE" and "OWNERS" content types, which were vestigal. These needs are likely better addressed through subscriptions/transactions, and are obsolete, and haven't existed for 2+ years and no one has asked for them to be restored.

Test Plan: Mostly a bunch of grep. Viewed rule list, rule edit. Edited a revision.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2769

Differential Revision: https://secure.phabricator.com/D6656
2013-08-07 18:03:51 -07:00
epriestley
333e377488 Modernize Releeph project create controller
Summary:
Ref T3092.

  - Check for a duplicate key error;
  - do less single loading and use Query classes;
  - use responsive UI elements;
  - add crumbs.

Test Plan: Created a new project, and hit error cases.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3092

Differential Revision: https://secure.phabricator.com/D6629
2013-07-30 12:38:11 -07:00
epriestley
5cc3bbf721 Use application PHIDs for application transactions
Summary: Ref T2715. Ref T3578. Load application transactions through application PHID infrastructure.

Test Plan: Viewed feed, saw successful loads of application transaction objects and rendered feed stories.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715, T3578

Differential Revision: https://secure.phabricator.com/D6617
2013-07-29 12:04:15 -07:00
Bob Trahan
ec61c15cb2 move repository arcanist projects over to new phid stuff
Summary: also submit casual entry for longest class name award with new query class. Ref T2715

Test Plan: phid.query and saw the right arcanist project

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6586
2013-07-26 14:33:31 -07:00
Bob Trahan
1cb0db8755 Move PhabricatorUser to new phid stuff
Summary: Ref T2715. Had to start loading status information in the query class. Debated trying to clean up some of the attach / load stuff but decided to just add status under the new paradigm for now.

Test Plan: phid.query  also made a status and checked that out. also played in conpherence.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6585
2013-07-26 14:05:19 -07:00
epriestley
1c454ea6d9 Add commits/audits to Asana bridge
Summary: Ref T2852. Adds sync for commits/audits.

Test Plan: {F51660}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6573
2013-07-26 10:31:35 -07:00
epriestley
938b5c69c6 Use PhutilProxyException to disambiguate repository pull errors
Summary: Ref T2569. When repository pulls fail, they don't necessarily list identifying information about which repository failed. Use a proxy exception to list that information.

Test Plan: {F51267}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2569

Differential Revision: https://secure.phabricator.com/D6548
2013-07-23 16:58:20 -07:00
epriestley
5f78a6953f Fix a type constant which slipped through grep
Auditors: btrahan
2013-07-22 17:21:20 -07:00
epriestley
d3cf7874ed Use application PHIDs in Repositories
Summary: Ref T2715. Move repositories to new app PHIDs.

Test Plan: Used `phid.lookup`; `phid.query`; browsed Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6529
2013-07-22 12:17:54 -07:00
epriestley
69d5a9af58 Use Application PHIDs in Projects
Summary: Ref T2715. Move Projects to the new stuff.

Test Plan: Used `phid.query` to load projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715

Differential Revision: https://secure.phabricator.com/D6526
2013-07-22 12:17:53 -07:00
epriestley
0e3cb3b393 Use Application PHIDs for commits
Summary: Ref T2716. Ref T2715. Move CMIT to use Application PHIDs. Nothing too special here, but I consolidated some code into DiffusionCommitQuery. Depends on D6514.

Test Plan: Browsed Diffusion. Browsed Differential/Maniphest with linked commits. Used jump nav; used `phid.lookup` and `phid.query`. Used remarkup for Git and SVN repos. Grepped for PHID_TYPE_CMIT.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2715, T2716

Differential Revision: https://secure.phabricator.com/D6515
2013-07-22 12:17:34 -07:00
epriestley
d89b8ce2b2 Modernize architecture of phd
Summary:
Ref T1670. Mostly, use PhutilArgumentParser. This breaks up the mismash of functional stuff and PhabriatorDaemonControl into proper argumentparser Workflows.

There are no functional changes, except that I removed the "pingConduit()" call prior to starting daemons, because I intend to remove all Conduit integration.

Test Plan:
- Ran `phd list`.
- Ran `phd status` (running daemons).
- Ran `phd status` (no running daemons).
- Ran `phd stop <pid>` (dead task).
- Ran `phd stop <pid>` (live task).
- Ran `phd stop zebra` (invalid PID).
- Ran `phd stop 1` (bad PID).
- Ran `phd stop`.
- Ran `phd debug zebra` (no match).
- Ran `phd debug e` (ambiguous).
- Ran `phd debug task`.
- Ran `phd launch task`.
- Ran `phd launch 0 task` (invalid arg).
- Ran `phd launch 2 task`.
- Ran `phd help`.
- Ran `phd help list`.
- Ran `phd start`.
- Ran `phd restart`.
- Looked at Repositories (daemon running).
- Looked at Repositories (daemon not running).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1670

Differential Revision: https://secure.phabricator.com/D6490
2013-07-18 15:28:56 -07:00
Juan Pablo Civile
8e0a975e3f Remove calls to DifferentialRevision::loadRelations from commit workers
Summary:
Remove ocurrences of `loadRelations` in workers.

One was simply unnecesary, no subsequent call to `getReviewers` or `getCCPHIDs` was made.
The other was replaced with the nicer `DifferentialRevisionQuery` using `needRelations` and `needReviewerStatus` (for future upgrade).

Test Plan:
Land a revision into a tracked repository and check the parser worker attached the commit correctly.
For the owners worker I just checked it didn't crash into a hundred tiny pieces.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6467
2013-07-15 18:39:55 -07:00
epriestley
a0084bbb0d Build an early multi-step repository create form
Summary: Ref T2231. Ref T2232. This form doesn't do anything yet and there are no link sto it, but it lets you enter all the data to create a repository in a relatively simple, straightforward way.

Test Plan:
{F49740}
{F49741}
{F49742}
{F49743}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231, T2232

Differential Revision: https://secure.phabricator.com/D6430
2013-07-14 07:37:17 -07:00
Lauri-Henrik Jalonen
93e37e9060 Phabricator event timeline removed
Summary: Removed related files and references

Test Plan: Crossed my fingers

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T2003

Differential Revision: https://secure.phabricator.com/D5744

Conflicts:
	src/__phutil_library_map__.php
	src/infrastructure/storage/patch/PhabricatorBuiltinPatchList.php
2013-07-09 18:07:42 -07:00
epriestley
a0c5a6cdb6 Use %P for all sensitive command construction in Phabricator
Summary: Depends on D6366. Applies %P everywhere.

Test Plan: Ran various daemon commands via scripts, e.g. `bin/repository pull`, `bin/storage dump`.

Reviewers: btrahan, mbishopim3

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6367
2013-07-03 15:13:45 -07:00
Svemir Brkic
b29067e38b Process multiple Bugtraq links in reverse to avoid links overwriting each other
Reviewed by: epriestley
2013-07-02 04:46:40 -07:00
epriestley
328aa383e4 Always provide a viewer when executing DifferentialRevisionQuery
Summary: Ref T603. This query isn't policy-aware yet, but prepare for it to be one day.

Test Plan: Looked at: home page; differential home; differential detail; diffusion browse. Made differential.query conduit call.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D6337
2013-07-01 12:38:27 -07:00
epriestley
efbd3ecc48 Fix the type of some values passed to MySQL
Summary: Ref T3377. MySQL ignores indexes if we hand it mismatched datatypes. This seems colossally dumb, but give it what it expects.

Test Plan: wat

Reviewers: wez, btrahan

Reviewed By: wez

CC: aran

Maniphest Tasks: T3377

Differential Revision: https://secure.phabricator.com/D6201
2013-06-13 18:01:40 -07:00
epriestley
f9f19f2fdc Provide transaction-based edits of repository encoding
Summary: Adds support for the "encoding" field to the new transactional interface.

Test Plan:
{F44189}

{F44190}

Some of the encodings in the second screen are from testing, and can no longer be set.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6035
2013-05-25 06:30:38 -07:00
epriestley
50bc33fc1c Change repository description from plain text to remarkup
Summary:
At one point this was sort of a one-line summary but it isn't really anymore (and doesn't appear on the list view). We could add a summary in the future if we wanted.

  - Change the control from a text area to a remarkup area.
  - Change the display to remarkup.

Test Plan:
{F44183}
{F44184}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6034
2013-05-24 12:38:44 -07:00
epriestley
2fd018ad92 Begin transacitonalizing repository edits and provide a more sensible edit interface
Summary:
Ref T2231, T603. Plan of attack here is pretty much:

  - Built out a new (currently not linked in the UI) edit interface in Diffusion which is transaction-based and has a sensible layout.
  - Build out a new create interface based on PagedForm which dumps into the new edit interface.
  - Throw the old stuff away.
  - Everyone lives happily ever after.

Test Plan:
{F44163}
{F44164}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2231

Differential Revision: https://secure.phabricator.com/D6029
2013-05-24 12:37:42 -07:00
epriestley
c48f64b391 Specify HOME when invoking Git commands
Summary: Fixes T2965, see that task for discussion. This is dumb but seems like our best bet.

Test Plan:
  - Installed newish version of Git.
  - Set HOME on the websever to `/var/root` (or any other unreadable directory).
  - Hit the error described in T2965 when viewing Diffusion.
  - Applied this patch.
  - Diffusion works.

Reviewers: btrahan, joel

Reviewed By: btrahan

CC: aran, chad

Maniphest Tasks: T2965

Differential Revision: https://secure.phabricator.com/D5994
2013-05-21 14:14:31 -07:00
epriestley
80ce5968db Minor, fix a variable name that I derped in recent refactoring.
Auditors: btrahan
2013-05-20 12:41:09 -07:00
epriestley
37e28f3813 Minor, correct more application class name spellings. 2013-05-15 17:01:34 -07:00
epriestley
571d191a90 Fix missing 'user' in PhabricatorOwnerPathQuery
Summary:
See:

  - https://github.com/facebook/phabricator/issues/317
  - https://github.com/facebook/phabricator/issues/319

This callsite was just overlooked in D5928, I think.

Test Plan: Ran `scripts/repository/reparse.php --owners rXnnnn` without hitting an exception.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5930
2013-05-15 08:12:39 -07:00
Bob Trahan
016b62a7ef Diffusion - move some DiffusionRequest queries to occur over Conduit
Summary: Ref T2784. This one was a wee bit complicated. Had to add PhabricatorUser and concept of initFromConduit (or not) to DiffusionRequest.

Test Plan: foreach repo, visited CALLSIGN and clicked a commit and verified they laoded correctly. Hacked code to hit NOT via Conduit and repeated tests to great success.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T2784

Differential Revision: https://secure.phabricator.com/D5928
2013-05-14 15:32:19 -07:00
epriestley
e7fde9a77c Make repository discovery partially testable
Summary:
Ref T2784. Begins pulling discovery into Engines and covering it with tests. In particular:

  - Discovery is currently a one-shot process where we find all the new commits and write them to the database in one go. Split it apart so we find and return the new commits first, then write them to the database separately. This makes things simpler and more testable.
  - This diff only brings SVN into an engine (and only the "find the commits" part), since it's simpler than Git or Mercurial.
  - Creates a base Engine class and moves common functionality there.
  - Restores the `--verbose` flag to `repository pull`.

Test Plan: Added unit tests. Ran `bin/repository discover`. Ran `bin/phd debug pulllocal`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2784

Differential Revision: https://secure.phabricator.com/D5906
2013-05-12 19:08:37 -07:00
epriestley
105dd1899c Make repository pulls testable
Summary:
Ref T2784. This moves us toward being able to test the background and Conduit pipelines for repositories. In particular:

  - Separate the logic for pulling repositories (`git pull`, `hg pull`) out of `PhabricatorRepositoryPullLocalDaemon` and put it in `PhabricatorRepositoryPullEngine`. This allows repositories to be pulled directly without invoking the daemons.
  - Add tests for the engine, including a future-looking base test case.
  - Add basic `PhutilDirectoryFixture`-based repositories.

Next steps:

  # Do the same for repo discovery.
  # Then we can start writing tests against specific Conduit methods.

Test Plan: Ran unit tests. Ran `bin/repository pull` on SVN, Hg and Git repositories. Ran `bin/phd debug pulllocal`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, nh

Maniphest Tasks: T2784

Differential Revision: https://secure.phabricator.com/D5904
2013-05-12 19:05:52 -07:00
Bob Trahan
a42851501f Diffusion - move DiffusionBrowseQuery => Conduit
Summary: see title. Ref T2784.

Test Plan:
In diffusion, for each of SVN, Mercurial, and Git, I loaded up /diffusion/CALLSIGN/. I verified the README was displayed and things looked good. Next I clicked on "browse" on the top-most commit and verified things looked correct. Also clicked through to a file for a good measure and things looked good.
In owners, for each of SVN, Mercurial, and Git, I played around with the path typeahead / validator. It worked correctly!

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T2784

Differential Revision: https://secure.phabricator.com/D5883
2013-05-10 11:02:58 -07:00
deedydas
78a8079f59 Tokens added to Repository Commits and Pastes
Summary: Ref T3023

Test Plan: Tokens visible when awarded to these applications and notifications successful.

Reviewers: epriestley, AnhNhan

Reviewed By: AnhNhan

CC: aran, Korvin, AnhNhan

Maniphest Tasks: T3023

Differential Revision: https://secure.phabricator.com/D5859
2013-05-09 14:21:33 -07:00
Chad Little
43ff24b0f3 Update form styles, implement in many places
Summary:
This creates a common form look and feel across the site. I spent a bit of time working out a number of kinks in our various renderings. Some things:

- Font Styles are correctly applied for form elements now.
- Everything lines up!
- Selects are larger, easier to read, interact.
- Inputs have been squared.
- Consistant CSS applied glow (try it!)
- Improved Mobile Responsiveness
- CSS applied to all form elements, not just Aphront
- Many other minor tweaks.

I tried to hit as many high profile forms as possible in an effort to increase consistency. Stopped for now and will follow up after this lands. I know Evan is not a super fan of the glow, but after working with it for a week, it's way cleaner and responsive than the OS controls. Give it a try.

Test Plan: Tested many applications, forms, mobile and tablet.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5860
2013-05-07 14:07:06 -07:00
Bob Trahan
7573ad9a70 DiffusionBranchQuery => Conduit
Summary: ref T2784. This one had a few fun spots where I had to move data around. Also, is there some common object (or should I add it?) that can do this toDictionary newFromConduit stuff? Also, this assumes D5803 is largely correct at the time of this diff.

Test Plan: browsed mercurial and git repository page. saw the branches i expected.

Reviewers: epriestley

Reviewed By: epriestley

CC: chad, aran, Korvin

Maniphest Tasks: T2784

Differential Revision: https://secure.phabricator.com/D5810
2013-05-01 14:56:36 -07:00
Jakub Vrana
1c507f14af Support Mercurial commands on Windows
Test Plan: Grepped Hg repo.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5774
2013-04-24 17:09:19 -07:00
Jakub Vrana
141233565a Fix whitespace around methods 2013-04-23 09:42:35 -07:00
Jakub Vrana
6a5807eba4 Link external bug trackers
Summary: Fixes T2971.

Test Plan:
/rG1.
Set regexp to one line and URL, then /rG1.
Set regexp to two lines, then /rG1.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2971

Differential Revision: https://secure.phabricator.com/D5676
2013-04-19 21:56:13 -07:00
Jakub Vrana
a9099912bb Handle SVN root changes correctly
Summary: If there is a change in SVN root (perhaps properties change) then we try to list its parent (which doesn't exist) and mark the root itself as deleted.

Test Plan: Parsed SVN commit with property change of root.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5709
2013-04-16 08:28:38 -07:00
Jakub Vrana
e31e998f3b Convert differential.revisionPHID commit detail to edge
Summary: Migration doesn't delete differential.revisionPHID but maybe it should?

Test Plan: Reparsed commit, ran the migration, deleted differential.revisionPHID, looked at task with attached commit with attached revision.

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5634
2013-04-12 22:48:16 -07:00
epriestley
855e085c6f Uninstall Conduit calls when uninstalling applications
Summary: Fixes T2698. When applications are installed, their Conduit calls should drop out. This will also let us land Releeph without exposing Conduit calls.

Test Plan:
  - Viewed Conduit console; uninstalled some applications and verified their calls dropped out.
  - Tried to make an uninstalled call; got an appropriate error.

Reviewers: edward, btrahan

Reviewed By: edward

CC: aran

Maniphest Tasks: T2698

Differential Revision: https://secure.phabricator.com/D5302
2013-03-13 07:09:05 -07:00
James Rhodes
868e79b7a7 Implemented option to allow users to view username of repository URIs.
Test Plan:
Enable the checkbox under the Tracking options for a repository.
View the repository in Diffusion, the username should show.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2575

Differential Revision: https://secure.phabricator.com/D5269
2013-03-07 08:06:09 -08:00
vrana
b3a63a62a2 Introduce PhabricatorEmptyQueryException
Summary: It's dumb to execute a query which we know will return an empty result.

Test Plan: Looked at comment preview with "11", didn't see "1 = 0" in DarkConsole.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5177
2013-03-06 19:22:00 -08:00
epriestley
0a069cb55a Require a viewer to load handles
Summary:
Unmuck almost all of the we-sort-of-have-viewers-some-of-the-time mess.

There are a few notable cases here:

  - I used Omnipotent users when indexing objects for search. I think this is correct; we do policy filtering when showing results.
  - I cheated in a bad way in the Remarkup object rule, but fixing this requires fixing all the PhabricatorRemarkupEngine callsites (there are 85). I'll do that in the next diff.
  - I cheated in a few random places, like when sending mail about package edits. These aren't a big deal.

Test Plan:
  - Grepped for all PhabricatorObjectHandleData references.
  - Gave them viewers.

Reviewers: vrana

Reviewed By: vrana

CC: aran, edward

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D5151
2013-02-28 17:15:09 -08:00
epriestley
e3567b2d4d Fix grossness in Herald daemon
Summary: Do this in a reasonably proper / formal way. Leave a TODO for the policy stuff.

Test Plan: Ran `scripts/repository/reparse.php --herald rPnnnnn`

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D5150
2013-02-28 16:38:40 -08:00
epriestley
3f547d562d Give the fake Herald user a PHID so it counts as logged in and can see commits
See last two commits in this chain.

Auditors: vrana
2013-02-27 11:54:17 -08:00
epriestley
69dcd47751 Fix two cases where we load Commit handles without a viewer
I missed these in testing D5139 because they aren't in the web UIs and I am dumb and can not brain today.

Auditors: vrana
2013-02-27 11:44:52 -08:00
epriestley
fe78944c9d Prepare Diffusion for hovercards
Summary:
Move Diffusion to be hovercard-ready, and expand our ability to resolve commit references.

  - Link unqualified hashes of 7 characters or more which match a commit.
  - Link qualified hashes of 5 characters or more which match a commit.
  - Support `{...}` syntax.

Test Plan: {F33896}

Reviewers: chad, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D5121
2013-02-27 08:04:54 -08:00
epriestley
2b10173a03 Don't retain "\r\n" when summarizing commits
Auditors: vrana
2013-02-21 15:12:54 -08:00
epriestley
b32bfb6541 Render commit summaries when rendering handles
Summary:
Fixes T2563. Instead of rendering "rPnnnnnn", render "rPnnnnnn: add feature X". Tweak Audit tables to accommodate.

@vrana / @nh, this migration might take a while. You could safely skip it when deploying and then run it after deployment.

I think I fixed all the other places where these render, but might have missed something.

Test Plan:
  - Ran first schema migration, clicked around to make sure nothing broke.
  - Ran `scripts/repository/reparse.php --message rXyyyyy`, verified summary populated.
  - Ran second migration.
  - Checked task/diffusion/audit/differential for weird rendering.

Reviewers: vrana

Reviewed By: vrana

CC: nh, aran, chrisbolt, allixsenos

Maniphest Tasks: T2563

Differential Revision: https://secure.phabricator.com/D5012
2013-02-21 15:09:35 -08:00
epriestley
a22bea2a74 Apply lint rules to Phabricator
Summary: Mostly applies a new call spacing rule; also a few things that have slipped through via pull requests and such

Test Plan: `find src/ -type f -name '*.php' | xargs -n16 arc lint --output summary --apply-patches`

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D5002
2013-02-19 13:33:10 -08:00
epriestley
ef7f16180c Restore merge of phutil_tag. 2013-02-13 14:51:18 -08:00
epriestley
73cce6e131 Revert "Promote phutil-tag again"
This reverts commit 8fbabdc06d, reversing
changes made to 2dab1c1e42.
2013-02-13 14:08:57 -08:00
vrana
9b8da73765 Convert AphrontTableView to safe HTML
Summary:
Lots of killed `phutil_escape_html()`.

Done by searching for `AphrontTableView` and then `$rows` (usually) backwards.

Test Plan:
Looked at homepage.

  echo id(new AphrontTableView(array(array('<'))))->render();

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4884
2013-02-09 15:11:38 -08:00
vrana
d817dfa8fc Convert some phutil_escape_html() to hsprintf()
Summary: Found by `sgrep_php -e '"...".phutil_escape_html(...)'`.

Test Plan:
/
/D1
/uiexample/
/countdown/1/
/herald/transcript/1/all/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4869
2013-02-08 15:59:02 -08:00
vrana
afc5333bb3 Convert AphrontFormView to safe HTML
Summary: Searched for `AphrontFormView` and then for `appendChild()`.

Test Plan: /login/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4855
2013-02-07 18:01:00 -08:00
vrana
85961c8eca Convert AphrontFormInsetView to safe HTML
Summary: Done by searching for `AphrontFormInsetView` and verifying all method calls.

Test Plan: /repository/edit/1/tracking/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4845
2013-02-07 18:00:59 -08:00
vrana
059920c2da Convert AphrontErrorView to safe HTML
Summary: Done by searching for `AphrontErrorView` and then `appendChild()`.

Test Plan:
Looked at Commit Detail.
Looked at Revision Detail.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4843
2013-02-07 17:26:01 -08:00
vrana
f864d9e611 Fix double escaping in phutil_tag
Summary:
I wasn't able to reproduce the "recursion detected" in real web request but I saw lots of 1073741824 refcounts in `debug_zval_dump()` of $object.
I'm not sure how that happens.

Test Plan: D4807#4

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4839
2013-02-06 15:21:05 -08:00
vrana
be4662e667 Convert setCaption() to safe HTML
Test Plan: /settings/panel/display/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4824
2013-02-05 15:52:43 -08:00
epriestley
ebc2937dc8 render_tag -> tag: final block
Summary:
Convert a final few `render_tag()` calls to `tag()` calls. This leaves us with 36 calls:

  - 9 are in Conpherence, and will be converted after merging. Using Views makes the most sense here, to get access to renderHTMLView() and such.
  - 2 are the definition and its library map entry.
  - 3 are in the documentation.
  - I believe the remaining 22 are too difficult to convert pre-merge. About half are in code which is slated for destruction; the other half are in the base implementations of very common classes (like PhabricatorStandardPageView) and can only be converted by converting the entire codebase.

My plan at this point is:

  - Test the branch thoroughly.
  - Merge to master.
  - Over time, resolve the remaining issues: lint means things shouldn't get any worse.

Test Plan: See inlines.

Reviewers: vrana

Reviewed By: vrana

CC: aran, edward

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4802
2013-02-04 11:38:04 -08:00
epriestley
0f1bdbe147 Merge branch 'master' into phutil_tag
(Sync.)
2013-02-04 06:19:52 -08:00
epriestley
363f292fd8 Partially revert marking of copies as direct
Summary:
The other `true` is correct (and I think we can fix the scaling issues) but this one should be an indirect change. This prevents the branch from appearing in the history of every file.

(I misread this code and gave @vrana some bad advice originally. This is //actually// consistent with Mercurial and Git.)

Test Plan: Partial revert. I'll make this stuff testable.

Reviewers: nh, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4742
2013-01-30 12:35:37 -08:00
epriestley
3093d1663d Add javelin_tag(), convert easy callsites
Summary:
  - Implements `javelin_tag()`, which is `javelin_render_tag()` on top of `phutil_tag()` instead of `phutil_render_tag()`.
  - Manually converts all or almost all of the trivial callsites.

Test Plan:
  - Site does not seem any more broken than before.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4639
2013-01-25 12:57:17 -08:00
epriestley
3e147dd61c Fix some easy phutil_render_tag()
Summary:
  - Grepped for phutil_render_tag().
  - Fixed some easy ones.

Test Plan:
  - Browsed around; site didn't seem more broken than it was before.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4638
2013-01-25 11:39:23 -08:00
vrana
20768d65d5 Convert phutil_render_tag(X, Y, '...') to phutil_tag
Summary:
Created with spatch:

  lang=diff
  - phutil_render_tag
  + phutil_tag
    (X, Y, '...')

Then searched for `&` and `<` in the output and replaced them.

Test Plan: Loaded homepage.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4503
2013-01-24 19:20:27 -08:00
epriestley
2584af2cb8 Clean up repositories with symbols correctly
Summary: We currently try to delete symbols by ID, but the table has a multipart primary key and no `id` column.

Test Plan: Ran query locally; had @JThramer verify fix in his environment.

Reviewers: btrahan

Reviewed By: btrahan

CC: JThramer, aran

Differential Revision: https://secure.phabricator.com/D4626
2013-01-24 17:48:41 -08:00
vrana
760ab135eb Delete license header 2013-01-18 14:45:58 -08:00
epriestley
0e612c910b Sort repositories in Diffusion by name, not creation order
Summary: Ref T2298. This seems like the least complicated reasonable order to implement.

Test Plan: Looked at repositories, saw them ordered by name.

Reviewers: vrana, btrahan, brennantaylor

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2298

Differential Revision: https://secure.phabricator.com/D4395
2013-01-16 10:51:08 -08:00
Ricky Elrod
800f62d0d5 Repository option.
Test Plan: Looked at the option.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4468
2013-01-16 09:35:48 -08:00
Chad Little
fb9a1c38fb Panel background changes for Repositories app.
Summary: Changed some background panels.

Test Plan: Reload page.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4458
2013-01-16 07:49:24 -08:00
Michael Halstead
21ed25019d Prevent exceptions when adding repositories via conduit.
Summary: Simple change to stop exceptions from being thrown based on the diff in D4146.

Test Plan: arc call-conduit repository.create

Reviewers: epriestley, btrahan

Reviewed By: btrahan

CC: vrana, aran, Korvin

Maniphest Tasks: T2268

Differential Revision: https://secure.phabricator.com/D4342
2013-01-10 15:37:56 -08:00
vrana
0f1086fd68 Mark SVN dir copy as direct change for its files
Test Plan: HHVM currently core dumps so I didn't test it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4343
2013-01-07 13:46:26 -08:00
epriestley
f6b1964740 Improve Search architecture
Summary:
The search indexing API has several problems right now:

  - Always runs in-process.
    - It would be nice to push this into the task queue for performance. However, the API currently passses an object all the way through (and some indexers depend on preloaded object attributes), so it can't be dumped into the task queue at any stage since we can't serialize it.
    - Being able to use the task queue will also make rebuilding indexes faster.
    - Instead, make the API phid-oriented.
  - No uniform indexing API.
    - Each "Editor" currently calls SomeCustomIndexer::indexThing(). This won't work with AbstractTransactions. The API is also just weird.
    - Instead, provide a uniform API.
  - No uniform CLI.
    - We have `scripts/search/reindex_everything.php`, but it doesn't actually index everything. Each new document type needs to be separately added to it, leading to stuff like D3839. Third-party applications can't provide indexers.
    - Instead, let indexers expose documents for indexing.
  - Not application-oriented.
    - All the indexers live in search/ right now, which isn't the right organization in an application-orietned view of the world.
    - Instead, move indexers to applications and load them with SymbolLoader.

Test Plan:
  - `bin/search index`
    - Indexed one revision, one task.
    - Indexed `--type TASK`, `--type DREV`, etc., for all types.
    - Indexed `--all`.
  - Added the word "saboteur" to a revision, task, wiki page, and question and then searched for it.
    - Creating users is a pain; searched for a user after indexing.
    - Creating commits is a pain; searched for a commit after indexing.
    - Mocks aren't currently loadable in the result view, so their indexing is moot.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: 20after4, aran

Maniphest Tasks: T1991, T2104

Differential Revision: https://secure.phabricator.com/D4261
2012-12-21 14:21:31 -08:00
vrana
2cc7f82ece Move Conduit methods inside applications
Test Plan:
/conduit/
/conduit/method/arcanist.projectinfo/
Call method

  $ echo '{}' | arc call-conduit user.whoami

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4268
2012-12-21 12:21:59 -08:00
vrana
800be357e8 Order repositories in Diffusion from oldest
Summary: Same to ordering in Repository List.

Test Plan: /diffusion/

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4251
2012-12-20 15:49:37 -08:00
epriestley
db89e23761 Make Repositories partially policy-aware
Summary: Small step toward repository hosting. No user-visible changes.

Test Plan: Looked at repositories in Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D4227
2012-12-19 11:07:06 -08:00
Nick Harper
b8d372c4bc Fix bugs when saving package changes due to commit changes
Summary:
Because of the way PhabricatorOwnersPackage works, the code to save package
changes when parsing commit changes was raising a few undefined index errors,
and was throwing an exception trying to call a method on null (because not
all of the phids related to the package had their handles loaded). This fix
doesn't load the missing handles, it just avoids trying to use them.

Test Plan:
./scripts/repository/reparse.php on a commit with path changes that triggered
a package change

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4236
2012-12-19 10:53:04 -08:00
Nick Harper
4a81ae6d6d Add data information to daemon task view
Summary:
Load the data for daemon worker tasks when viewing them, and present
the information in a useful way. This defaults to printing the json data,
but for some classes of worker it will also link to the corresponding
object, to make debugging problems with workers easier.

Test Plan:
load /daemon/task/NNN for a CommitParserWorker and a MetaMTAWorker, and
see the addition of a data field with useful content and link.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4226
2012-12-17 17:12:55 -08:00
Nick Harper
0ba7d34f97 Allow phabricator to manage repositories
Summary:
This adds a configuration option for repositories to be marked as managed
by phabricator, which is then used by the pullLocal daemon to try to recover
from some errors it runs into.

Test Plan: Set a bogus uri for a repo, saw that the pullLocal daemon fixed it.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3680
2012-12-12 14:43:09 -08:00
vrana
0e53731d1d Expose repository info in arcanist.projectinfo
Summary: This is to reduce number of calls from Arcanist.

Test Plan: Called it from web interface.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4146
2012-12-10 15:19:09 -08:00
Bob Trahan
42a514ec79 make repository tool fail a little less hard if daemons don't interact nicely
Summary: we were catching a specific exception; just catch all exceptions

Test Plan: viewed repository tool home page

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2155

Differential Revision: https://secure.phabricator.com/D4118
2012-12-07 16:12:16 -08:00
epriestley
f306cab653 Use application icons for "Eye" menu and Crumbs
Summary:
Issues here:

  - Need an application-sized "eye", or a "home" icon for "Phabricator Home".
  - Some of the "apps_lb_2x" sliced images are the "_dark_" versions, not the light versions.
  - If you slice an application-sized "logout" (power off) icon and application-sized "help" (questionmark in circle) icon I can replace the current menu icons and nearly get rid of "autosprite".
  - To replace the icons on /applications/, the non-retina size is "4x", so we'd need "8x" for retina. Alternatively I can reduce the icon sizes by 50%.
  - The "Help", "Settings" and "Logout" items currently have a "glowing" hover state, which needs a variant (or we can drop it).
  - The /applications/ icons have a white hover state (or we can drop it).
  - The 1x application (14x14) icons aren't used anywhere right now, should they be? Maybe in the feed in the future, etc?
  - The "apps-2x" and "apps-large" sheets are the same image, but getting them to actually use the same file is a bit tricky, so I just left them separate for now.

Test Plan:
{F26698}
{F26699}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4108
2012-12-07 13:37:28 -08:00
epriestley
20ee3003b5 Replace all instances of AphrontSideNavView with AphrontSideNavFilterView
Summary:
AphrontSideNavView is an old class which required you to do a lot of work; it was obsoleted by AphrontSideNavFilterView. Remove all direct callsites so I can clean it up.

This is a precursor to letting me render a filter menu as a dropdown menu for T1960.

Test Plan:
Examined each interface for correct filter construction and selection:

  - Browsed Diffusion
  - Browsed Differential
  - Browsed Files
  - Browsed Slowvote
  - Browsed Phriction
  - Browsed repo edit interface

Grepped for `AphrontSideNavView`. The only remaining instances are in `AphrontSideNavView` itself and `AphrontSideNavFilterView` (which currently uses it).

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4034
2012-12-07 13:30:31 -08:00
epriestley
1a3bf098ae Correctly parse author and committer identities again
Summary: See D3977, a terrible diff where I made a huge messs.

Test Plan: Ran `reparse.php --message --trace <revision>`, verified correct identification of author.

Reviewers: edward, vrana

Reviewed By: edward

CC: aran

Differential Revision: https://secure.phabricator.com/D4104
2012-12-07 07:06:29 -08:00
vrana
cb45ad67a3 Revive essential commit details
Summary: Deleted by D3977.

Test Plan:
  ./reparse.php --message

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4090
2012-12-06 10:44:16 -08:00
Suman Karumuri
2852b4f5d3 Track path moves in Owner's packages
Summary:
Currently, if a package is moved from one location to another those package paths become stale in Owner's packages.
This diff fixes that by tracking the path moves in diffusion commit messages and updating the owners package paths accordingly.

Test Plan:
Tested the script by running the reparse.php script on the following test commits:
https://phabricator.fb.com/rPHGITd7271a2de212e0b2de33d485b2ff806e75f73d36
https://phabricator.fb.com/rPHGIT0192fb41e7bd3e0faeeb14facdc7f7bea1b716d0

on the following packages:
https://phabricator.fb.com/owners/package/1167/
https://phabricator.fb.com/owners/package/1168/

Reviewers: nh

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D3985
2012-12-05 17:21:45 -08:00
epriestley
fd5beffb99 Remove PhabricatorRepositoryDefaultCommitMessageDetailParser
Summary: See discussion in T1544. This has been obsoleted by simpler/better mechanisms.

Test Plan: Edited a repository; ran a parse task.

Reviewers: edward

Reviewed By: edward

CC: aran

Maniphest Tasks: T1544

Differential Revision: https://secure.phabricator.com/D3977
2012-12-05 15:34:28 -08:00
epriestley
2de879e613 Minor, extend the lease time of Herald task leases. 2012-11-30 11:02:42 -08:00
vrana
846a359d9a Display number of lint messages in Diffusion dir
Test Plan:
Went to https://secure.phabricator.com/diffusion/ARC/browse/.
Saw number of lint messages per dir, not terribly slow.
Went to https://secure.phabricator.com/diffusion/ARC/browse/master/src/difference/.
Saw number of lint messages per file.
Disabled lint, didn't see it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3921
2012-11-08 15:40:48 -08:00
vrana
23a046b3cd Allow saving lint errors to database
Summary: This saves lint errors to the path change of current commit. It requires pushed revision. It doesn't save difference from previous commit mentioned in T2038#comment-4 - I don't plan doing it after all, everything would be much more complicated and the amount of data saved with this approach isn't that bad.

Test Plan: Applied patch, ran script, verified DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3899
2012-11-08 15:39:43 -08:00
vrana
10cf2bb4e2 Mark PhabricatorRepositorySvnCommitChangeParserWorker as final
Test Plan:
  $ git grep PhabricatorRepositorySvnCommitChangeParserWorker

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3897
2012-11-05 23:27:40 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
Bob Trahan
4edf8ae2fc make "browse in diffusion" action work for commits in branches other than master
Summary: we do this by passing the "seenOnBranches" commit data detail through the stack

Test Plan: browse in diffusion link worked for non-master checkins under git

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1949

Differential Revision: https://secure.phabricator.com/D3853
2012-10-31 14:07:25 -07:00
Bob Trahan
5d5f8a7a91 whoops - forgot to remove the third slash on this comment from D3822 2012-10-25 16:38:14 -07:00
Bob Trahan
731a6900bd upgrade repository delete function to full-blown workflow
Summary: fancy title. really just make the delete() method aware of related objects and build a quick workflow which calls delete(). also make commit delete savvy about audit requests.

Test Plan: deleted a repository per the instructions given to me in the web UI

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1416, T1958, T1372

Differential Revision: https://secure.phabricator.com/D3822
2012-10-25 16:23:41 -07:00
Bob Trahan
d9c6e07f2c If users are on the email to Phabricator, do not send them the Phabricator reply.
Summary: When we receive an email, figure out if any of the other tos and ccs are users. If they are, pass their phids through the stach as "exclude phids" and exclude them from getting the email.

Test Plan: used the various applications (audit, differential, maniphest) and noted emails were sent as expected.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin, vrana

Maniphest Tasks: T1676

Differential Revision: https://secure.phabricator.com/D3645
2012-10-10 10:18:23 -07:00
epriestley
cd9d78c107 Allow Fact analysis of commits
Summary: For immutable objects, just use the ID as a cursor.

Test Plan:
  - Analyzed commits from an empty cursor.
  - Checked that cursor was good.
  - Pulled some more commits.
  - Analyzed commits again, verified it only hit the new ones.
  - Verified the graph of "Count of CMIT" looked reasonable.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1866

Differential Revision: https://secure.phabricator.com/D3656
2012-10-08 13:27:06 -07:00
vrana
22cb8f5d08 Require canonical numbers in routes
Summary:
D03646 works, I don't want it to work.
Theoretically, it can cause us some troubles if we use this string in JS number context where 030 is 24.

Test Plan: D03646, D3646

Reviewers: epriestley, edward

Reviewed By: edward

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3646
2012-10-05 18:07:54 -07:00
vrana
fe355f7905 Fix typo in route 2012-10-04 13:52:06 -07:00
Bob Trahan
4c86893108 allow arcanist projects to be deleted via the ui
Summary: tricky part is purging symbols at that time too. override "delete" method to get there, use transactions, etc.

Test Plan: deleted an arcanist project - it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, mbishopim3

Maniphest Tasks: T1738

Differential Revision: https://secure.phabricator.com/D3613
2012-10-04 13:25:58 -07:00
vrana
e84c18d734 Mark revision as closed before attaching commit and loading changes
Summary:
D3555 stopped multiple commenting but it still attaches multiple diffs.

Save earlier to stop doing unnecessary work.

Test Plan: Reparsed the same commit twice at the same time.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3598
2012-10-03 15:47:55 -07:00
epriestley
78784aa1fe Group applications into groups on /applications/
Summary: So they're maybe a little easier to deal with? I'm going to take this formally to "plz @chad plz help" land.

Test Plan: {F20329}

Reviewers: btrahan, vrana, chad

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3609
2012-10-03 15:46:19 -07:00
vrana
b585146532 Display auditors in audit commit list
Summary: Users wonder who is actually doing all these audits.

Test Plan:
/owners/package/1/
/audit/view/packagecommits/?phid=PHID-OPKG-1

Reviewers: epriestley

Reviewed By: epriestley

CC: jungejason, aran, Korvin

Differential Revision: https://secure.phabricator.com/D3597
2012-10-03 10:41:39 -07:00
epriestley
f817d81b12 Add applications for Owners, Repositories, PHID manager, PHPAST
Summary: See D3572.

Test Plan: Viewed /applications/, saw all these applications.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3579
2012-10-01 12:56:33 -07:00
Evan Priestley
7dd8851a61 Merge pull request #207 from vvavrychuk/daemon
use available api to check daemon status
2012-09-27 12:53:54 -07:00
Vasyl Vavrychuk
18ee51ac7b use available api to check daemon status
Summary:
Replace executing 'ps aux' with usage of available api to
control daemons PhabricatorDaemonControl.

This fixes isPullDaemonRunningOnThisMachine returning wrong status
under Fedora & lighttpd & SELinux because SELinux with default
settings blocked getting all processes in 'ps aux'.

Test Plan: start stop daemon and check repository app

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3557
2012-09-27 22:27:51 +03:00
vrana
7ed8b6d7fa Recheck if revision hasn't been already closed in commit parser
Summary:
If attaching a commit or checking if there are any changes takes nonzero time then the revision may be closed by someone else.

Cleaner solution would be to do it inside a transaction and mark the SELECT as FOR UPDATE but it would be blocking.

Test Plan: Patched `$should_close` to be true, reparsed an already closed commit.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3555
2012-09-26 21:12:03 -07:00
vrana
e0e97b08b8 Open editor on first modified line
Test Plan: Created diff, opened the file from Differential, opened the file in Diffusion.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3538
2012-09-24 11:07:59 -07:00
vrana
f7a38df5a5 Add TODO comment to commit attacher 2012-09-19 12:30:27 -07:00
Nick Harper
90b60bc3c9 Load relationships for revision
Summary:
In the commit message parser worker, it tries to get relationships for a
revision without first loading them. (D3444 introduced the change.) This
is where the get happens, so I'm adding the load here - maybe it should be
added in the CommitMessageParserWorker instead?

Test Plan: Made this change, and my taskmaster daemons stopped failing.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3448
2012-09-06 15:46:51 -07:00
epriestley
0736592cff Add didParseCommit() to DifferentialFieldSpecification
Summary:
This is mostly ripped from D2721, but doesn't implement the T945 part.

After we parse a commit message, give DifferentialFieldSpecifications an opportunity to react to the message as well (e.g., by updating related objects).

Test Plan: Impelmented a var_dump() body, ran `reparse.php` on a commit.

Reviewers: vrana, 20after4, btrahan, edward

Reviewed By: 20after4

CC: aran

Maniphest Tasks: T945, T1544

Differential Revision: https://secure.phabricator.com/D3444
2012-09-06 12:13:51 -07:00
vrana
b3d7ed3e8e Respect users with duplicate real names in commit parser
Test Plan:
  $parser->resolveUserPHID('Lei Zhao');
  $parser->resolveUserPHID('Jakub Vrana');

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3437
2012-09-05 12:24:35 -07:00
epriestley
ad409126b1 Fix a bug with owners analysis of commits
Summary: If the commit has a known author but that author is different from the revision author or the revision doesn't exist, we'll throw away the commit author and then raise an audit for "Owners Not Involved". Instead, we should just use the commit author in all cases.

Test Plan:
Debugged this with Zor in IRC, he reported it fixed his issue.

Before:

http://pastie.org/4574680

("Author" is empty.)

After:

http://pastie.org/4574712

("Author" is correctly recognized.)

Reviewers: jungejason, nh, vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3374
2012-08-23 10:12:41 -07:00
epriestley
93b0a501a4 Add local navigation to Differential
Summary:
Adds a flexible navigation menu to diffs that shows you your current position in the diff.

Anticipating some "this is the best thing ever" and some "this is the wosrt thing ever" on this, but let's see how much pushback we get? It seems pretty good to me.

Test Plan: Will attach screenshots.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1633, T1591

Differential Revision: https://secure.phabricator.com/D3355
2012-08-21 15:01:20 -07:00
vrana
0f919ecd3c Update line count after attaching commit diff
Summary: We should probably do this also in `differential.creatediff` but it's not a big deal because we later call `differential.updaterevision` which does this using `DifferentialRevisionEditor`.

Test Plan: None.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3263
2012-08-13 14:43:05 -07:00
epriestley
3e29921f5e Use phutil_utf8_convert() in Phabricator
Summary: See D3252.

Test Plan: This one is nasty to test, I'm going to make some coffee first.

Reviewers: davidreuss, vrana, btrahan

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T452

Differential Revision: https://secure.phabricator.com/D3254
2012-08-12 08:50:19 -07:00
Nick Harper
62b3e4aea3 Remove a check for closing revisions
Summary:
I ran into a case where a commit isn't "new" but hasn't been closed. I
think the check on the status of the differential revision should be
enough and this check isn't needed.

Test Plan:
used the reparse.php script to close a revision that previously wouldn't
close.

Reviewers: epriestley, vrana

Reviewed By: vrana

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3232
2012-08-10 16:15:37 -07:00
vrana
1c98ca7042 Fix doc links 2012-08-06 15:58:28 -07:00
Alan Huang
bcb9de4ea1 Add a context field to symbol objects
Summary:
See T1602.

This is just the minimal functional patch; the scripts will continue
working because of the `DEFAULT ''`.

Test Plan:
Can't fully test this until I get more code working, but
nothing broke horribly yet.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3147
2012-08-06 12:20:45 -07:00
epriestley
b8f4f7c438 Fix an issue with Diffusion's display of copied files in Git
Summary: If a change copies some file `A` to `B` and also edits `A`, we currently record this as an indirect change and don't show the edits to `A` in the diff. Instead, record these as direct changes.

Test Plan: Created two commits, one which copied `A` to `B` without modifying `A` and one which copied `A` to `B` and modified A. Viewed both commits in Diffusion. The unmodified commit did not show `A`, and the modified commit did (with the correct changes).

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: champo, aran

Differential Revision: https://secure.phabricator.com/D3120
2012-08-01 12:32:35 -07:00
vrana
9092994e45 Make old and new files when checking for changes by commit
Summary:
We have some false positives on commit changes checker.
I'm not sure if the reason is a difference between `git diff` and `svn diff` or something else but making this more robust doesn't harm anything.

We couldn't make the files from the whole changeset because I want to ignore context bigger than `$num_lines` to reduce rebase noise.

Test Plan:
Ran the method on diff which had false positive previously.
Ran the method on a diff with real change.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3107
2012-07-30 17:57:30 -07:00
pearj
f2dea87c11 Fix Exception Bad getter call: getURIObject
After this commit: d9296638cd

I started getting this error:

Unhandled Exception ("Exception")
Bad getter call: getURIObject

It turns out that getURIObject just needed to be getRemoteURIObject and then the problem goes away.
2012-07-28 18:27:52 +10:00
epriestley
d9296638cd Fix a "user@domain:path" protocol handling bug
Summary:
In D3063, we stopped converting "user@domain:path" git-style URIs, but this broke the SSH-detection code and I missed that in my test plan because my test case uses natural SSH keys so the omission of SSH flags didn't cause failures.

This code is a bit of a mess anyway. Consolidate and refactor it to be a bit simpler, and add test coverage.

Test Plan: Ran unit tests. Ran "test_connection.php" in SSH and non-SSH modes, verified SSH modes generated appropriate ssh-agent commands around the git remote commands.

Reviewers: vrana, btrahan, tberman

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1529

Differential Revision: https://secure.phabricator.com/D3093
2012-07-27 17:21:33 -07:00
epriestley
514ee3526c Add an event for looking up names from repositories
Summary:
Currently, we have this cumbersome `PhabricatorRepositoryCommitMessageDetailParser` hook. This is really old and outdated; I want to just use the Differential custom field parser. See T945 for a specific application.

However, it allows installs to override author/committer association. Instead, provide an event hook for doing this.

Test Plan: Added a listener, made every commit resolve to "turtle", parsed some commits, verified the events looked sane and they now correctly were all attributed to "turtle".

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1337

Differential Revision: https://secure.phabricator.com/D3040
2012-07-24 11:59:28 -07:00
epriestley
1e9120cdfb Minor, fix an issue in Diffusion/Herald with commits that have no Auditors. 2012-07-24 07:37:41 -07:00
epriestley
8cbd860e31 Include email in Git author/committer lookup
Summary: We currently omit email from Git author/committer lookups, which gives us some bad results when identify commit authors. Include email. Also simplify this block a little bit.

Test Plan: Ran "reparse.php --message" on several commits, verified that the author/committer seemed reasonable with var_dump()s.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1337

Differential Revision: https://secure.phabricator.com/D3039
2012-07-23 13:59:58 -07:00