1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-28 09:42:41 +01:00
Commit graph

158 commits

Author SHA1 Message Date
epriestley
1c62a35710 Run one daemon to pull all working copies, not one daemon per working copy
Summary:
Allow the pull daemon to take a list of repositories. By default, pull all repositories.

Make some effort to respect pull frequencies, although we'll necessarily suffer a bit if running with only one process.

NOTE: We still launch one discovery daemon per working copy, so this only cuts the daemon count in half.

Test Plan:
  - Ran `phd debug pulllocal`, verified behavior.
  - Ran `pull.php P MTEST SVNTEST --trace`, verified it pulled the repos and ran the right commands.
  - Ran `phd repository-launch-master`, verified the right daemons launched, checked daemon console.
  - Ran `phd repository-launch-readonly`, verified the right daemon launched, checked daemon console.

Reviewers: btrahan, csilvers, davidreuss

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2418
2012-05-07 15:01:10 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
epriestley
049048765d Add "repository.create" and "repository.query" methods to Conduit
Summary: Primarily for @csilvers who has 92 million repositories or something. This is a touch hacky, but movitated by pragmatism.

Test Plan:
  - Ran "repository.create" to create repositories, "repository.query" to list them.
  - Tested most or maybe all of the error conditions, probably.

Reviewers: btrahan, vrana, csilvers

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2396
2012-05-04 16:16:22 -07:00
vrana
38ffe45f8e Use committer date instead of author date for Git epoch
Summary:
This is somewhat controversial but push date is usually more useful than commit date (which can be for example a month before other people can see the commit).
We can also store both dates.

Test Plan:
  git log --pretty="%ct %at"

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2319
2012-04-26 16:25:56 -07:00
epriestley
20a5c9b261 Use "closed", not "committed", in Differential
Summary: "Committed" is SVN-specific language, and confusing in Git and Mercurial. Use neutral language instead.

Test Plan: Inspection.

Reviewers: btrahan, Makinde, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T909

Differential Revision: https://secure.phabricator.com/D2087
2012-04-23 17:40:57 -07:00
vrana
1f2cf78c1b Display committed date in Revision Status field
Summary:
This is slightly more complicated for this reason:

- We don't set `dateCommitted` for normal commits, only for markcommitted.
-- We need to add this date to old revisions now.

Test Plan:
Reparse a revision - commit date was set.
Conduit `markcommitted` - commit date was set.
Run SQL script.
Display closed revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2282
2012-04-19 15:05:09 -07:00
epriestley
dec8acd803 Add very basic tag support to Diffusion
Summary: Lists the 25 most recent tags on the "Repository" page.

Test Plan: Looked at a git repository with a tag, saw it. Looked at HG/SVN repos, they didn't break.

Reviewers: davidreuss, 20after4, btrahan, vrana, jungejason

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T1130

Differential Revision: https://secure.phabricator.com/D2255
2012-04-18 08:02:08 -07:00
vrana
d17be1d824 Fix SVN commit change parser for files moved from deleted directory
Summary:
This continues work started at D2215.
Files moved from deleted directory were marked as Copied Here instead of Moved Here.

Test Plan: Reparsed two commits which was previously wrong, now correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Maniphest Tasks: T1114

Differential Revision: https://secure.phabricator.com/D2229
2012-04-14 22:18:27 -07:00
vrana
ef990703fa Fix SVN commit change parser for directories copied from the same path
Summary: See [[ https://secure.phabricator.com/D2215?id=3773#inline-2451 | D2215#inline-2451 ]].

Test Plan: Reparsed commit which was wrong, now correct.

Reviewers: epriestley, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2232
2012-04-14 21:46:02 -07:00
vrana
17b0277ec5 Fix SVN commit change parser for files moved from deleted directory
Summary: This is not perfect. Moved files are reported as deleted but I'm happy with it.

Test Plan: Reparsed two commits which was previously wrong, now semi-correct.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Maniphest Tasks: T1114

Differential Revision: https://secure.phabricator.com/D2215
2012-04-12 17:47:19 -07:00
epriestley
c458768415 Fix various threading issues, particularly in Gmail
Summary:
  - Add an explicit multiplexing option, and enable it by default. This is necessary for Mail.app to coexist with other clients ("Re:" breaks outlook at the very least, and generally sucks in the common case), and allows users with flexible clients to enable subject variance.
  - Add an option for subject line variance. Default to not varying the subject, so mail no longer says [Committed], [Closed], etc. This is so the defaults thread correctly in Gmail (not entirely sure this actually works).
  - Add a preference to enable subject line variance.
  - Unless all mail is multiplexed, don't enable or respect the "Re" or "vary subject" preferences. These are currently shown and respected in non-multiplex cases, which creates inconsistent results.

NOTE: @jungejason @nh @vrana This changes the default behavior (from non-multiplexing to multiplexing), and might break Facebook's integration. You should be able to keep the same behavior by setting the options appropriately, although if you can get the new defaults working they're probably better.

Test Plan:
Send mail from Maniphest, Differential and Audit. Updated preferences. Enabled/disabled multiplexing. Things seem OK?

NOTE: I haven't actually been able to repro the Gmail threading issue so I'm not totally sure what's going on there, maybe it started respecting "Re:" (or always has), but @cpiro and @20after4 both reported it independently. This fixes a bunch of bugs in any case and gives us more conservative set of defaults.

I'll see if I can buff out the Gmail story a bit but every client is basically a giant black box of mystery. :/

Reviewers: btrahan, vrana, jungejason, nh

Reviewed By: btrahan

CC: cpiro, 20after4, aran

Maniphest Tasks: T1097, T847

Differential Revision: https://secure.phabricator.com/D2206
2012-04-12 09:31:03 -07:00
vrana
e87e1786a6 Fix docs links after D2181
Test Plan:
  diviner .

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2188
2012-04-10 11:33:26 -07:00
epriestley
a5903d2a53 Use head_key() and last_key() to explicitly communicate intent
Summary:
PHP arrays have an internal "current position" marker. (I think because foreach() wasn't introduced until PHP 4 and there was no way to get rid of it by then?)

A few functions affect the position of the marker, like reset(), end(), each(), next(), and prev(). A few functions read the position of the marker, like each(), next(), prev(), current() and key().

For the most part, no one uses any of this because foreach() is vastly easier and more natural. However, we sometimes want to select the first or last key from an array. Since key() returns the key //at the current position//, and you can't guarantee that no one will introduce some next() calls somewhere, the right way to do this is reset() + key(). This is cumbesome, so we introduced head_key() and last_key() (like head() and last()) in D2161.

Switch all the reset()/end() + key() (or omitted reset() since I was feeling like taking risks + key()) calls to head_key() or last_key().

Test Plan: Verified most of these by visiting the affected pages.

Reviewers: btrahan, vrana, jungejason, Koolvin

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2169
2012-04-09 11:08:59 -07:00
Jason Ge
f6748bc190 The existing Audit code forgot to set authorPHID
Summary:
The audit tools has many false positive about Author Not
Matching with Revision. The fix is to set the authorPHID which was
missing in the existing code

Test Plan:
run reparse.php and it doesn't generate false positive result
anymore.

Reviewers: nh, epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2119
2012-04-06 14:16:42 -07:00
mkedia
591d50008f Allow projects to own packages
Summary:
- The UI is pretty straightforward, since Handle just works (tm)
- Added two methods to the owners object to handle the new layer of
  indirection. Then ran git grep PhabricatorOwnersOwner and changed
  callsites as appropriate.

Sending this to get a round of feedback before I test the non-trivial
changes in this diff.

Test Plan:
- owners tool: edit, view, list for basic functionality.
- phlog for the two new methods I added

Reviewers: epriestley, blair, jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2079
2012-04-03 16:20:07 -07:00
vrana
8813c7be0e Use assert_instances_of() everywhere but Differential and Diffusion
Summary: NOTE: This is not produced by a script so there might be errors. Please review carefully.

Test Plan: Browse around.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2091
2012-04-03 14:53:20 -07:00
epriestley
2ef3e69e58 Improve Herald commit emails
Summary:
  - Show the canonical (i.e., shorter) commit identifier in the subject.
  - For commits without a revision, put the commit summary in the subject.

Test Plan: Ran "scripts/repository/reparse.php <commit> --herald" for a number of different commits (with revision, without revision); got more useful email subjects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T1028

Differential Revision: https://secure.phabricator.com/D2004
2012-03-23 11:08:18 -07:00
epriestley
85f19e16dc Fix the "Browse in Diffusion" URI in Differential
Summary: Apparently I spent like a good month copy/pasting slightly different versions of this logic all over the codebase.

Test Plan: Selected "View Options -> Browse in Diffusion" for a chagneset, got a URI with a branch name in it under Git.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1993
2012-03-22 13:55:35 -07:00
epriestley
6d631577f2 Detect changes in merge commits as the diff of the merge and the first parent
Summary:
Currently, we use "git log" to detect the change list for all commits, but this produces no output for merge commits.

Instead, parse them as changes against the first parent (the merge destination). This produces generally sensible/expected behavior, and is consistent with what GitHub does.

We need to special-case the first commit because it doesn't have parents.

NOTE: This is a parser change so you need to run `./scripts/repository/reparse.php --all <callsign> --change` to reparse merge commits in already-imported repositories after updating.

Test Plan: Reparsed a merge commit, a non-merge commit, and the first commit in the Phabricator repository.

Reviewers: btrahan, gschmidt

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T961

Differential Revision: https://secure.phabricator.com/D1985
2012-03-21 17:10:05 -07:00
epriestley
620e936cba Fix symbol URI generation to include default branch name for relevant repositories
Summary: We need to build a request in order to pick up an appropriate default branch name, instead of using the raw static generator.

Test Plan: Clicked a symbol link, got /master/path/blahblah

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1982
2012-03-21 16:58:44 -07:00
epriestley
30ae22bfcf Fix many encoding and architecture problems in Diffusion request and URI handling
Summary:
Diffusion request/uri handling is currently a big, hastily ported mess. In particular, it has:

  - Tons and tons of duplicated code.
  - Bugs with handling unusual branch and file names.
  - An excessively large (and yet insufficiently expressive) API on DiffusionRequest, including a nonsensical concrete base class.
  - Other tools were doing hacky things like passing ":" branch names.

This diff attempts to fix these issues.

  - Make the base class abstract (it was concrete ONLY for "/diffusion/").
  - Move all URI generation to DiffusionRequest. Make the core static. Add unit tests.
  - Delete the 300 copies of URI generation code throughout Diffusion.
  - Move all URI parsing to DiffusionRequest. Make the core static. Add unit tests.
  - Add an appropriate static initializer for other callers.
  - Convert all code calling `newFromAphrontRequestDictionary` outside of Diffusion to the new `newFromDictionary` API.
  - Refactor static initializers to be sensibly-sized.
  - Refactor derived DiffusionRequest classes to remove duplicated code.
  - Properly encode branch names (fixes branches with "/", see <https://github.com/facebook/phabricator/issues/100>).
  - Properly encode path names (fixes issues in D1742).
  - Properly escape delimiter characters ";" and "$" in path names so files like "$100" are not interpreted as "line 100".
  - Fix a couple warnings.
  - Fix a couple lint issues.
  - Fix a bug where we would not parse filenames with spaces in them correctly in the Git browse query.
  - Fix a bug where Git change queries would fail unnecessarily.
  - Provide or improve some documentation.

This thing is pretty gigantic but also kind of hard to split up. If it's unreasonably difficult to review, let me know and I can take a stab at it though.

This supplants D1742.

Test Plan:
  - Used home, repository, branch, browse, change, history, diff (ajax), lastmodified (ajax) views of Diffusion.
  - Used Owners typeaheads and search.
  - Used diffusion.getrecentcommitsbypath method.
  - Pushed a change to an absurdly-named file on an absurdly-named branch, everything worked properly.

{F9185}

Reviewers: nh, vrana, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1921
2012-03-19 19:52:14 -07:00
epriestley
ec736f9c50 Handle "hg pull" return code change between 2.1 and 2.1.1 more gracefully
Summary:
See <https://github.com/facebook/phabricator/issues/102>. Between Feb 1 and Mar 1, the hg released changed the exit code behavior of "hg pull". This broke us mildly (and a bunch of other applications more severely, which is why it was reverted).

Detect the common case of this (english) and don't fail.

Test Plan: @killermonk, can you try applying this? I'll try to do an upgrade to 2.1 and see if I can also do a proper test.

Reviewers: Makinde, btrahan, killermonk

Reviewed By: btrahan

CC: killermonk, aran, epriestley

Differential Revision: https://secure.phabricator.com/D1948
2012-03-19 19:19:48 -07:00
Hafsteinn Baldvinsson
fffc1e51d0 Inset view controller for inset elements of forms.
Summary:
T937 suggests 'inset' could have its own view controller.

It has the following methods:
 - setTitle         for title
 - setRightbutton   if you have to place something (preferably a button)
                    on the right side of the form
 - setDescription   if you want to describe what it does
 - setContent       for the main content
 - addDivAttributes REALLY not sure about this one but it had to be included
                    because of a single controller (see owners/controller/edit/PhabricatorOwnersEditController.php:238)
 - appendChild      works as usual if your form is complex but you still want to remove
                    ->appendChild('<div class..') ->appendChild('</div>');

It might be an overkill so maybe some could be dropped:
 - addDivAttributes() and just rewrite how PhabricatorOwnersEditController.php works
 - setContent() and use appendChild for the main content?

Test Plan:
 - Looked at the controllers in phabricator
 - Changed the controller
 - Opened the page in another tab
 - If something didnd't look the same I fixed it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1926
2012-03-15 17:10:38 -07:00
epriestley
d0af617818 Add "final" to (almost) everything else
Summary: Last of the big final patches. Left a few debatable classes (12 out of about 400) that I'll deal with individually eventually.

Test Plan: Ran testEverythingImplemented.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T795

Differential Revision: https://secure.phabricator.com/D1881
2012-03-13 16:21:04 -07:00
Nick Harper
9b48384415 [svn1.7] Fix matching xml tag when parsing svn commits
Summary:
svn 1.7 changed their xml format slightly, they now have a
 ##<?xml version="1.0" encoding="UTF-8"?>## tag instead of
 ##<?xml version="1.0"?>##. This relaxes matching this tag.

Test Plan: ./scripts/repository/reparse.php rE521979 --change

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1866
2012-03-12 16:32:30 -07:00
epriestley
def19bb8de Add additional protections against local repository misconfigurations
Summary: We've hit a couple of these in the wild, raise better error messages when the local repo is toast / broken / nonsense.

Test Plan: Broke my local repo in all of the different ways we test for, verified I got an error message in each case.

Reviewers: btrahan, abirchall

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T964, T924

Differential Revision: https://secure.phabricator.com/D1855
2012-03-12 10:34:37 -07:00
vrana
d5bf30bb48 Prepare database for UTF-8
Summary: D1830#8

Test Plan:
`scripts/sql/upgrade_schema.php`
Try adding duplicate SSH Public Key - failed.
Try adding new SSH Public Key - succeeded.

Reviewers: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1852
2012-03-09 18:56:22 -08:00
epriestley
b2890eeb0e Add "final" to all Phabricator "Controller" classes
Summary:
These are all unambiguously unextensible. Issues I hit:

  - Maniphest Change/Diff controllers, just consolidated them.
  - Some search controllers incorrectly extend from "Search" but should extend from "SearchBase". This has no runtime effects.
  - D1836 introduced a closure, which we don't handle correctly (somewhat on purpose; we target PHP 5.2). See T962.

Test Plan: Ran "testEverythingImplemented" unit test to identify classes extending from `final` classes. Resolved issues.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T795

Differential Revision: https://secure.phabricator.com/D1843
2012-03-09 15:46:25 -08:00
Edward Speyer
226d0321af Revert "PhabricatorRepository: temporarily change the localPath"
Summary:
Reverts an unwanted commit (579941b866)
that got pushed by mistake.
2012-03-09 10:54:24 -08:00
Edward Speyer
579941b866 PhabricatorRepository: temporarily change the localPath
Summary:
I'm on a host where I have the PhabricatorRepositoryPullLocalDaemons
tracking a remote repo.  In my case, these end up as local git repos in
/var/repo/$name.

I'm working on another daemon that is going to automatically make
changes and commit them back upstream.  I figured it would be best to do
this in a new local repo.  I'll put these in /var/repo-clones/$name.

It's nice to use the exec*() functions in PhabricatorRepository, so the
approach I thought of was to load the PhabricatorRepository object from
the database, then change its localPath to point at the
/var/repo-clones/$name directory instead.

I didn't really want to change the local-path detail with setDetail(),
as that risks committing the change upstream.  It's nice to use the
repo's execLocalCommand() methods though, hence wanting to change the
local path.

Test Plan: None yet.

Reviewers: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1821
2012-03-08 15:15:13 -08:00
epriestley
89dac1cf19 When updating git repositories, use --prune to prune old branches
Summary: We'll keep deleted branches around right now because Git's behavior is not to remove them without --prune.

Test Plan: Ran "git fetch --all --prune" to make sure it at least ostensibly works.

Reviewers: btrahan, 20after4

Reviewed By: 20after4

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1833
2012-03-08 13:24:25 -08:00
epriestley
1c80a4eb58 Add a "properties" table to the Repository view in Diffusion
Summary: Notably, expose the clone URI / remote URI, after stripping credentials.

Test Plan: Looked at a repository.

Reviewers: btrahan, nh

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T91

Differential Revision: https://secure.phabricator.com/D1811
2012-03-08 12:46:19 -08:00
Edward Speyer
17d801a50e GitFetch daemon: more verbose
Summary: A smidgen more messaging about what's going on.

Test Plan:
Ran it, saw this:

  2012-03-07 5:37:30 PM [STDO] >>> [0] <exec> $ /data/phabricator/bin/list_db_services --tier_name 'cdb.phabricator'
  <<< [0] <exec> 47,101 us
  ...
  >>> [9] <exec> $ mkdir -p '/var/repo'
  <<< [9] <exec> 41,374 us
  Creating new directory /var/repo/fbcode for repo FBCode
  >>> [10] <exec> $ git clone --origin origin 'ssh://fbcode.git.vip.facebook.com/data/gitrepos/fbcode.git' '/var/repo/fbcode'

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1819
2012-03-07 19:24:06 -08:00
epriestley
8dfe8e84f0 Improve Diffusion parsing of submoudule changes
Summary: We currently parse these as directory changes and discard them. Instead, parse them as a new "SUBMODULE" type of change.

Test Plan:
  - Reparsed a commit which changes submodules and verified it parses correctly.
  - Reparsed a commit which adds submodules and verified it parses correctly.

Reviewers: btrahan, kdeggelman

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1815
2012-03-07 14:24:43 -08:00
epriestley
76fd9a2d28 Reduce laziness for "Mark Committed"
Summary:
  - Enforce proper workflow rules.
  - Fix a derp-bug with patches.

Test Plan:
  - Tried to mark a revision I didn't own.
  - Tried to mark a revision already marked committed.
  - Tried to mark a revision otherwise not accepted.
  - Verified daemon can override workflow rules and mark from arbitrary states.

Reviewers: btrahan, Makinde

Reviewed By: Makinde

CC: aran, epriestley

Maniphest Tasks: T948

Differential Revision: https://secure.phabricator.com/D1809
2012-03-07 10:20:17 -08:00
epriestley
a95c9873aa Add an "Auditors" field to commit messages which pushes audit requests when present
Summary:
Adds an optional "Auditors" field (like "Reviewers") to commit messages which gives installs a zero-config method for making audit requests.

This field does not appear on templates unless set, and is mostly ignored (but validated and preserved) by Differential.

It is then parsed by the daemons if present, and audit requests are pushed to valid users.

Test Plan: Made an "Auditors" commit and verified it was retained with "arc amend --show". Pushed it and verified the audit was triggered.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904, T880

Differential Revision: https://secure.phabricator.com/D1793
2012-03-06 15:10:35 -08:00
epriestley
d94129b739 Migrate "PhabricatorOwnersPackageCommitRelationship" to "PhabricatorRepositoryAuditRequest"
Summary:
  - Move table to Repository, since we have no Owners joins in the application anymore but would like to do a Repository join.
  - Rename "packagePHID" to "auditorPHID", since this column may contain package, project, or user PHIDs.

Test Plan:
  - Browsed Owners, Audit, and Differential interfaces to the Audit tool.
  - Made comments and state changes.
  - Ran "reparse.php --herald --owners" on several commits.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley, nh, vrana

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1787
2012-03-05 13:17:29 -08:00
epriestley
5590515007 Make Diffusion Herald emails thread with Audit emails
Summary:
  - Users may elect to receive an initial notification about a commit; allow it to be replied to in order to interact with the object.
  - Share thread headers between emails.
  - Add the "REPLY HANDLER ACTIONS" section to both emails.

Test Plan:
  - Used "reparse.php --herald" to trigger herald emails, verified reply-to and email body.
  - Made audit comments, verified body.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1762
2012-03-05 09:54:45 -08:00
epriestley
d7bb686a47 Minor, actually update commit author field when it is parsed by the worker. 2012-02-28 21:25:15 -08:00
epriestley
8a0a00f118 Make PhabricatorRepositoryCommmit schema changes for audit
Summary:
  - Add a proper mailKey field to make these things mailable. Backfill all
existing objects.
  - Denormalize authorPHID to the commit object so we can query by it
efficiently in a future diff. We currently use the search engine to drive
"commits by author" but that's not so good for audit, which needs more
constraints.
  - Add an overall audit status field so we can efficiently query "commits that
needs your attention".
  - Add enough code to convince myself that these fields are basically
reasonable and work correctly.

Test Plan:
  - Ran schema upgrades. Checked database state afterward.
  - Ran "reparse.php --owners --herald" to verify worker changes.
  - Looked at a commit, altered aggregate status via audits / reparse.php,
verified it responded correctly.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, nh

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1706
2012-02-28 21:06:34 -08:00
epriestley
37a1db6fe1 Use HGPLAIN for local hg commands in Phabricator
Summary: See D1707 -- just in case the Phabricator server is configured
suspiciously.

Test Plan: Cursory inspection.

Reviewers: Makinde, btrahan

Reviewed By: Makinde

CC: aran, epriestley

Maniphest Tasks: T922

Differential Revision: https://secure.phabricator.com/D1708
2012-02-27 19:21:03 -08:00
epriestley
d7a7bca85c Enable email for audits
Summary:
When users submit an audit, send email to relevant parties informing them.

Allow email to be replied to. Just basic support so far; no "!raise" stuff and
no threading with the Herald commit notification.

Test Plan: Made comments, got email. Replied to email, got comments.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1698
2012-02-27 12:57:57 -08:00
epriestley
25fade5008 Add audits to search
Summary: Add audit information to the commit search index.

Test Plan: Updated a commit, searched for terms in its comments, got hits.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1696
2012-02-27 09:51:00 -08:00
epriestley
1094527072 Allow Herald to trigger audits for users or projects
Summary:
Allows you to write a commit rule that triggers an audit by a user (personal
rules) or a project (global rules).

Mostly this is trying to make auditing more lightweight and accessible in
environments where setting up Owners packages doesn't make sense.

For instance, Disqus wants a rule like "trigger an audit for everything that
didn't have a Differential revision". While not necessarily scalable, this is a
perfectly reasonable rule for a small company, but a lot of work to implement
with Owners (and you'll get a lot of collateral damage if you don't make every
committer a project owner).

Instead, they can create a project called 'Unreviewed Commits' and write a rule
like:

	- When: Differential revision does not exist
 	- Action: Trigger an Audit for project: "Unreviewed Commits"

Then whoever cares can join that project and they'll see those audits in their
queue, and when they approve/raise on commits their actions will affect the
project audit.

Similarly, if I want to look at all commits that match some other rule (say,
XSS) but only want to do it like once a month, I can just set up an audit rule
and go through the queue when I feel like it.

NOTE: This abuses the 'packagePHID' field to also store user and project PHIDs.
Through the magic of handles, this (apparently) works fine for now; I'll do a
big schema patch soon but have several other edits I want to make at the same
time.

Also:

	- Adds an "active" fiew for /audit/, eventually this will be like the
Differential "active" view (stuff that is relevant to you right now).
	- On commits, highlight triggered audits you are responsible for.

Test Plan: Added personal and global audit triggers to Herald, reparsed some
commits with --herald, got audits. Browsed all audit interfaces to make sure
nothing exploded. Viewed a commit where I was responsible for only some audits.
Performed audits and made sure the triggers I am supposed to be responsible for
updated properly.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1690
2012-02-27 09:36:30 -08:00
epriestley
a15130b47c Add a maintenance script for reconciling repositories to disk state
Summary:
@rguerin ran into an issue in his install where Phabricator appears to have
discovered commits which no longer exist, and thus is failing to proceed with
its repository import.

It's not clear how we got into this state. Previously, it was possible by, e.g.,
parsing a different repository's working copy and then switching them back, but
there are now safeguards against that.

I'm taking a three-pronged approach to try to sort this out:

  - Provide a script to get out of this state (this script) and reconcile
Phabricator's view of a repository with an authoritative copy of it. This
basically "un-discovers" any discovered commits which don't actually exist (any
queued tasks to parse them will fail permanently when they fail to load the
commit object).
  - Add more logging to the discovery daemon so we can figure out where commits
came from.
  - Improve Diffusion's UI when stuff is partially discovered (T776).

(This script should also clean up some nonsense on secure.phabricator.com from a
botched Diviner import.)

Test Plan: Ran "reconcile.php" with bogus commits and bogus differential/commit
links, had them expunged. Will work with @rguerin to see if this resolves
things.

Reviewers: btrahan, rguerin

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1552
2012-02-02 16:03:50 -08:00
Jason Ge
b4053190ec Fix two issues in owners worker
Summary:
Fix two issues in PhabricatorRepositoryCommitOwnersWorker:

- if a commit was reviewed by some owner of the package, it should not be
  marked as needing audit
- do not run herald worker when it is not needed (for example, when the
  worker is executed from reparse.php)

Test Plan:
reparse a commit which is reviewed by the owner of a package
and verify that it is not marked as needing audit, and herald is not
executed.

Reviewers: nh, epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1496
2012-01-26 11:02:42 -08:00
epriestley
5b463e634c Write fewer "applied" rows and clean up excess historical rows
Summary:
  - Only write the <ruleID, phid> row if the rule is a one-time rule.
  - Delete all the rows for rules which aren't one-time.

NOTE: This is probably like several million rows for Facebook and could take a
while.

Test Plan:
Added some one-time and every-time rules, ran them against objects, verified
only relevant rows were inserted.
Ran upgrade script against a database with one-time and every-time "ruleapplied"
rows, got the irrelevant rows removed.

Reviewers: nh, btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1484
2012-01-25 11:53:39 -08:00
epriestley
3142fe4419 Remove massive "rule applied" query
Summary:
Herald rules may be marked as "one-time". We track this by writing a row with
<ruleID, phid> when we apply a rule.

However, the current test for rule application involves loading every <ruleID,
*> pair. We also always write this row even for rules which are not one-time, so
if there are 100 rules, we'll load 1,000,000 rows after processing 10,000
objects.

Instead, load only the <phid, *> pairs, which are guaranteed to be bounded to at
most the number of rules.

I'll follow up with a diff that causes us to write rows only for one-time rules,
and deletes all historic rows which are not associated with one-time rules.

Test Plan:
Grepped for callsites to loadAllByContentTypeWithFullData(). Ran
rules in test console.

Reviewers: nh, btrahan, jungejason

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1483
2012-01-24 19:29:54 -08:00
epriestley
b43eb5aa7c Add transaction-oriented editing to projects
Summary:
  - Make some editing operations transaction-oriented, like Maniphest. (This
seems to be a good model, particularly for extensibility.) I'll move the rest of
the editing operations to transactions in future diffs.
  - Make transaction-oriented operations publish feed stories.

Test Plan:
  - Created a new project.
  - Edited an existing project.
  - Created a new project via quick create flow from Maniphest.
  - Verified feed stories publish correctly.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T681

Differential Revision: https://secure.phabricator.com/D1477
2012-01-24 09:44:35 -08:00
epriestley
27f52efd37 Minor, fix spelling issues detected by linter. 2012-01-20 07:39:55 -08:00