1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-14 19:02:41 +01:00
Commit graph

2446 commits

Author SHA1 Message Date
epriestley
f1eb92a126 Fix a bug where some Maniphest batch edits were incorrectly skipped
Summary: Currently, if you have a task with project "X" and you apply a batch edit to it to remove "X", the action has no effect because we incorrectly skip the edit as a no-op. Instead, don't perform this check for edge edits.

Test Plan: Batch removed a project from several tasks with only one project.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1566

Differential Revision: https://secure.phabricator.com/D3092
2012-07-27 15:25:32 -07:00
epriestley
486f7c1e8e Add aggregated facts to the Facts application
Summary:
Some facts are aggregations of other facts. For example, we may compute how many times each macro is used in each object as a "raw fact":

  Dnnn uses macro "psyduck" 6 times.

But we want to present this data in aggregate form, e.g. "order macros by popularity". We can do this at runtime and it probably won't be too awful a query, but we can also aggregate it cheaply:

  Macro "psyduck" is used 3920 times across all objects.

...and then do a query like "select macros ordered by usage".

"Aggregate" facts support facts like this. The aggregate facts I've implemented are:

  - Count of all objects.
  - Count of objects of type X.
  - Last time facts were updated.

These clearly fit the "aggregate" facts template well. I'm not 100% sure macros do. We can use this table to answer a question like "What are the most popular macros, ordered by use?" We can also use it to answer a question like "What are the most popular macros in the last 6 months?", if we build a specific fact for that. But we can't use it to answer a question like "What are the most popular macros between times X and Y?". Maybe that's important; maybe not.

This seems like a good fit for at least some types of facts.

I'll de-magic the keys a bit in the next diff.

Test Plan: Ran the engines and got some aggregated facts about other facts.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3089
2012-07-27 13:46:01 -07:00
vrana
05bf6bef81 Jump to correct line in Blame previous revision
Test Plan: Jumped on correct line in SVN, Git and HG repos.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3084
2012-07-27 13:38:15 -07:00
epriestley
7c934e4176 Add a basic "fact" application
Summary:
Basic "Fact" application with some storage, part of a daemon, and a control binary.

= Goals =

The general idea is that we have various statistics we'd like to compute, like the frequency of image macros, reviewer responsiveness, task close rates, etc. Computing these on page load is expensive and messy. By building an ETL pipeline and running it in a daemon, we can precompute statistics and just pull them out of "stats" tables.

One way to do this is just to completely hard-code everything, e.g. have a daemon that runs every hour which issues a big-ass query and dumps results into a table per-fact or per fact-group. But this has a bunch of drawbacks: adding new stuff to the pipeline is a pain, various fact aggregators can't share much code, updates are slow and expensive, we can never build generic graphs on top of it, etc.

I'm hoping to build an ETL pipeline which is generic enough that we can use it for most things we're interested in without needing schema changes, and so that installs can use it also without needing schema changes, while still being specific enough that it's fast and we can build useful stuff on top of it. I'm not sure if this will actually work, but it would be cool if it does so I'm starting pretty generally and we'll see how far I get. I haven't built this exact sort of thing before so I might be way off.

I'm basing the whole thing on analyzing entire objects, not analyzing changes to objects. So each part of the pipeline is handed an object and told "analyze this", not handed a change. It pretty much deletes all the old data about that thing and then writes new data. I think this is simpler to implement and understand, and it protects us from all sorts of weird issues where we end up with some kind of garbage in the DB and have to wipe the whole thing.

= Facts =

The general idea is that we extract "facts" out of objects, and then the various view interfaces just report those facts. This change has on type of fact, a "raw fact", which is directly derived from an object. These facts are concerete and relate specifically to the object they are derived from. Some examples of such facts might be:

  D123 has 9 comments.
  D123 uses macro "psyduck" 15 times.
  D123 adds 35 lines.
  D123 has 5 files.
  D123 has 1 object.
  D123 has 1 object of type "DREV".
  D123 was created at epoch timestamp 89812351235.
  D123 was accepted by @alincoln at epoch timestamp 8397981839.

The fact storage looks like this:

  <factType, objectPHID, objectA, valueX, valueY, epoch>

Currently, we supprot one optional secondary key (like a user PHID or macro PHID), two optional integer values, and an optional timestamp. We might add more later. Each fact type can use these fields if it wants. Some facts use them, others don't. For instance, this diff adds a "N:*" fact, which is just the count of total objects in the system. These facts just look like:

  <"N:*", "PHID-xxxx-yyyy", ...>

...where all other fields are ignored. But some of the more complex facts might look like:

  <"DREV:accept", "PHID-DREV-xxxx", "PHID-USER-yyyy", ..., ..., nnnn> # User 'yyyy' accepted at epoch 'nnnn'.
  <"FILE:macro", "PHID-DREV-xxxx", "PHID-MACR-yyyy", 17, ..., ...> # Object 'xxxx' uses macro 'yyyy' 17 times.

Facts have no uniqueness constraints. For @vrana's reviewer responsiveness stuff, we can insert multiple rows for each reviewer, e.g.

  <"DREV:reviewed", "PHID-DREV-xxxx", "PHID-USER-yyyy", nnnn, ..., mmmm> # User 'yyyy' reviewed revision 'xxxx' after 'nnnn' seconds at 'mmmm'.

The second value (valueY) is mostly because we need it if we sample anything (valueX = observed value, valueY = sample rate) but there might be other uses. We might need to add "objectB" at some point too -- currently we can't represent a fact like "User X used macro Y on revision Z", so it would be impossible to compute macro use rates //for a specific user// based on this schema. I think we can start here though and see how far we get.

= Aggregated Facts =

These aren't implemented yet, but the idea is that we can then take the "raw facts" and compute derived/aggregated/rollup facts based on the raw fact table. For example, the "count" fact can be aggregated to arrive at a count of all objects in the system. This stuff will live in a separate table which does have uniqueness constraints, and come in the next diff.

We might need some kind of time series facts too, not sure about that. I think most of our use cases today are covered by raw facts + aggregated facts.

Test Plan: Ran `bin/fact` commands and verified they seemed to do reasonable things.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, majak

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3078
2012-07-27 13:34:21 -07:00
Nick Harper
14e3e5ccfd Provide user option for disabling diffusion symbol cross-references
Summary: Some people don't like these, so they should be able to turn them off.

Test Plan:
Toggled the setting on and off; loaded a page in diffusion and differential
that should have symbol cross-references, and saw that they weren't linked
when I had the setting disabled. I also checked that the symbols are still
linked when the setting hasn't been touched.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3069
2012-07-26 20:25:27 -07:00
vrana
0cfdf2f74f Allow specifying against commit in DiffusionRawDiffQuery
Summary: I will need this for tracking line number in Blame previous revision.

Test Plan:
  $ hg diff --rev 0:1
  $ svn diff -r 64382:64383 README

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3083
2012-07-26 15:13:03 -07:00
Evan Priestley
94445d41e7 Merge pull request #168 from ossareh/support_ldap_search_for_users
Support ldap search for users
2012-07-26 14:54:10 -07:00
Evan Priestley
1cf880ffc6 Merge pull request #167 from CodeBlock/master
D3081
2012-07-26 14:49:44 -07:00
Ricky Elrod
729f246f92 Add a checkbox to allow administrators to re-send the welcome email to users.
Summary:
See title - This simply adds a checkbox to the "Edit User" page in the
admin view, to allow an administrator to re-send the "Welcome to Phabricator"
email.

Test Plan:
Sent myself another welcome email using the checkbox.
Created a new user using the admin panel, to make sure emails still get
sent for new users.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1524

Differential Revision: https://secure.phabricator.com/D3081
2012-07-26 17:41:14 -04:00
Michael Ossareh
a9af5d611d Prevent the ability to scrape for valid usernames
- return the same error message when either bind or the username search
   fails to find a user
 - config variables should use hypen and not underscore
2012-07-26 14:32:51 -07:00
vrana
636877807f Fix typo in comment 2012-07-26 13:45:48 -07:00
Evan Priestley
dcc28a28ec Merge pull request #166 from CodeBlock/master
D3079
2012-07-26 13:35:12 -07:00
vrana
aefc5654c3 Load contents of shielded files with lots of changes
Summary:
We need `$this->old` and `$this->new` in `renderShield()`.

Broken since D2358.

Test Plan: Loaded contents of file with lots of changes.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3080
2012-07-26 13:28:10 -07:00
Ricky Elrod
0fb93e5423 Fix an overlap between two elements of the Image form input.
Summary:
The filename field and the checkbox to select the default image were
overlapping in Firefox on Linux on both the Project Edit page and the
Profile Edit page.

Test Plan: Looked at both of the pages and saw that they rendered better.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3079
2012-07-26 16:25:56 -04:00
epriestley
c614af53bf Add PhutilFactsUpateIterator
Summary: This iterator processes objects that have been updated.

Test Plan:
Ran this test script:

  $cursor = null;
  $table  = new DifferentialRevision();
  while (true) {
    $iterator = new PhabricatorFactsUpdateIterator($table, $cursor);
    foreach ($iterator as $new_cursor => $update) {
      echo "{$new_cursor} => D".$update->getID()."\n";
      $cursor = $new_cursor;
    }
    echo "Zzz...\n";
    sleep(5);
  }

Verified it iterated over every object and then stopped. Made a comment on a differenial revision, verified it iterated over the object after 15 seconds.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3077
2012-07-26 13:09:36 -07:00
epriestley
ff61dba7ac Extend LiskMigrationIterator from PhutilBufferedIterator
Summary: PhutilBufferedIterator now implements all the nonspecific logic here.

Test Plan:
Created a test script like this:

  $iterator = new LiskMigrationIterator(new DifferentialRevision());
  $iterator->setPageSize(3);

  foreach ($iterator as $key => $rev) {
    echo "{$key}: ".$rev->getID()."\n";
  }

Ran it and verified sensible iteration results.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3076
2012-07-26 12:01:57 -07:00
epriestley
fc09bcf0a3 Move qsprintf() test cases from libphutil to Phabricator
Summary: Also move the other tests up so they'll trigger when this stuff is touched.

Test Plan: liberate

Reviewers: nh, btrahan, vrana

Reviewed By: nh

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3074
2012-07-26 12:01:47 -07:00
epriestley
f0ee4946a6 Update documentation for D3045
Summary: Some minor updates for changes in D3045 and clarity.

Test Plan: Read documentation.

Reviewers: dschleimer, bos, btrahan, vrana

Reviewed By: dschleimer

CC: aran, cbiesinger

Maniphest Tasks: T1507, T1546

Differential Revision: https://secure.phabricator.com/D3046
2012-07-26 12:01:39 -07:00
Michael Ossareh
0a0607d2f7 Support searching for users to find their LDAP entry
Summary:
 - the current LDAP auth flow expects a DN to look like
   cn=ossareh,ou=Users,dc=example,dc=com
 - however many LDAP setups have their dn look something like
   cn=Mike Ossareh,ou=Users,dc=example,dc=com

Test Plan:
 Test if logins work with a LDAP setup which has cn=Full Name
 instead of cn=username.

 To test you should ensure you set the properties needed to
 trigger the search before login as detailed in conf/default.conf.php

Reviewers: epriestley

CC: mbeck, aran, Korvin

Differential Revision: https://secure.phabricator.com/D3072
2012-07-25 19:03:10 -07:00
epriestley
94bc4003f1 Show a "Logout?" dialog when the user goes to /logout/
Summary: Currently, the logout link is this awkward form in the footer since we have to CSRF it. Enable a GET + dialog + confirm workflow instead so the logout link can just be a link instead of this weird mess.

Test Plan: Went to /logout/, logged out.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3066
2012-07-25 13:58:49 -07:00
Bob Trahan
321df38ccf Deleting unused private member variable from AphrontRequest
Summary: I was poking around to see how this class worked and noticed this variable does nothing.

Test Plan: Careful inspection and reasoning that unused private member variable can be deleted.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3068
2012-07-25 13:48:46 -07:00
vrana
92a243492e Fix loading owning package
Summary: It is completely broken since D2943.

Test Plan:
  PhabricatorOwnersPackage::loadAffectedPackages($repository, $path);

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3067
2012-07-25 12:52:44 -07:00
epriestley
59949bf811 Add "device" JS/CSS for reactive stuff
Summary:
As we work through @chad's redesign, one thing I want to do is improve the tablet/mobile experience.

Add a "device" behavior which sets a "device-phone", "device-tablet" or "device-desktop" class on the root div. The behavior (device names, width triggers) is mostly based on Bootstrap.

Also adds a preview viewport=meta tag, which makes the iPhone not scale the page like crazy and is a desirable end state, but currently makes the app less usable since things get cut off.

Test Plan:
Added some classes like this:

	.device-desktop {
	  background: blue;
	}
	.device-tablet {
	  background: orange;
	}
	.device-phone {
	  background: yellow;
	}

...and loaded the site on a desktop, iPad and iPhone. Resized the window. Got the right background color in all cases.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D3063
2012-07-25 11:51:27 -07:00
vrana
1fc2dfd54b Load reviewer stats
Summary:
This allows getting stats for any arbitrary period, e.g.

- everything
- last week
- week before last week

Test Plan: Ran the script for last week.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3064
2012-07-25 11:40:16 -07:00
vrana
1970ceefe3 Compute reviewer stats
Summary:
The final goal is to display reviewers response time on homepage.
This is a building block for it.

The algorithm is quite strict - it doesn't count simple comment as response because reviewers would be able to cheat with comments like "I'm overwhelmed right now and will review next week".
We are more liberate in Phabricator where reviewers response with comments without changing the status quite often but I'm not trying to improve response times in Phabricator so this is irrelevant.
Reviewers in Facebook changes status more often (to clean their queue) so I follow this approach.

There is currently no way to track reviewers silently added and removed in Edit Revision but it's not a big deal.
The algorithm doesn't track commandeered revision, there's a TODO for it.

Response times are put in two buckets: `$reviewed` and `$not_reviewed`.
`$reviewed` contains reviewers who took action, `$not_reviewed` contains reviewers who didn't respond on time.
I will probably compute average time from `$reviewed` and raise it for those `$not_reviewed` that are higher than this average.
The idea is to not favor reviewers who were only lucky for being in a group with someone fast.

Test Plan: Passed test.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3062
2012-07-24 23:37:58 -07:00
vrana
02594c2c22 Delete unused code 2012-07-24 15:15:45 -07:00
vrana
1b3da47fb0 Allow querying PHIDs with GET
Summary: I want to link this page from outside.

Test Plan: /phid/?phids=PHID-USER-gsraq7yc66r4stl4c6u3

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3060
2012-07-24 15:08:56 -07:00
epriestley
7ffe802671 Remove queryfx() from phabricator/
Summary: Seee D3057.

Test Plan: Loaded site.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, chad

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3058
2012-07-24 12:34:02 -07:00
epriestley
514ee3526c Add an event for looking up names from repositories
Summary:
Currently, we have this cumbersome `PhabricatorRepositoryCommitMessageDetailParser` hook. This is really old and outdated; I want to just use the Differential custom field parser. See T945 for a specific application.

However, it allows installs to override author/committer association. Instead, provide an event hook for doing this.

Test Plan: Added a listener, made every commit resolve to "turtle", parsed some commits, verified the events looked sane and they now correctly were all attributed to "turtle".

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1337

Differential Revision: https://secure.phabricator.com/D3040
2012-07-24 11:59:28 -07:00
epriestley
17e20bc363 Remove AphrontConnection from Phabricator
Summary: See D3055.

Test Plan: Loaded pages and such.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3056
2012-07-24 11:50:19 -07:00
epriestley
27f6cc3b27 Support PhabricatorOpaqueEnvelope for managing database passwords
Summary: Currently, MySQL/MySQLi connections store passwords in plain text on the object. Allow them to be stored in PhutilOpaqueEnvelopes instead. See D3053.

Test Plan: Loaded site.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3054
2012-07-24 11:13:53 -07:00
epriestley
5d4a6bcf95 Break AphrontDatabaseConnection dependencies on PhabricatorEnv
Summary: We pull "retries" and a doc link from PhabricatorEnv directly. Break these dependencies so the classes can move to libphutil.

Test Plan: Browsed site, triggered a schema exception and verified I still got the useful footer text.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3053
2012-07-24 10:47:27 -07:00
epriestley
be63cb386b Remove AphrontWriteGuard from phabricator/
Summary: See D3051.

Test Plan: Loaded some pages.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3052
2012-07-24 10:46:07 -07:00
epriestley
1e9120cdfb Minor, fix an issue in Diffusion/Herald with commits that have no Auditors. 2012-07-24 07:37:41 -07:00
epriestley
f1270315e9 Allow connections to be closed; close connections for global locks
Summary: Add an explicit close() method to connections and call it in GlobalLock.

Test Plan:
Wrote a script like this:

  $lock = PhabricatorGlobalLock::newLock('test');
  echo "LOCK";
  $lock->lock();
  sleep(10);
  echo "UNLOCK";
  $lock->unlock();
  sleep(9999);

Using `SHOW FULL PROCESSLIST`, verified the connection closed after 10 seconds with both the "MySQL" and "MySQLi" implementations.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1470

Differential Revision: https://secure.phabricator.com/D3035
2012-07-23 19:06:58 -07:00
Alan Huang
afa2dbc3dc Accept GET requests for jump nav
Summary:
The URL /jump/?jump=%s now does the same thing as the POST version,
except that if it couldn't jump to anything, it loads /jump/ with the
query filled in so you can just press enter (we don't save searches
without a CSRF token).

(hsb: Sorry for stealing your task! It hadn't been updated in two months
so I figured you were likely not actively working on it.)

Test Plan:
Loaded given URL with different queries (including various
flavors of nothing). Search worked as expected.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1036

Differential Revision: https://secure.phabricator.com/D3047
2012-07-23 17:41:46 -07:00
vrana
7d9960e522 Display committed date in audits
Summary: Our auditors requested displaying this field and I can image that it can be useful.

Test Plan: /audit/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3044
2012-07-23 16:25:38 -07:00
epriestley
d07934474e Break AphrontWriteGuard dependency on AphrontRequest
Summary:
I want to move queryfx() and family to libphutil, for @chad and others (see T1283). We need to break a few dependencies to do this.

Since AphrontWriteGuard is independently useful, I broke the dependency between it and AphrontRequest rather than between Connection and WriteGuard. I'll move its implementation to libphutil in a future diff.

Test Plan: Loaded site, submitted CSRF form successfully, monkeyed with CSRF token, submitted CSRF form, got error.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3042
2012-07-23 15:17:59 -07:00
epriestley
8213a70f3d Fix "fullName" rules for Maniphest transaction email
Summary: I changed this from `getName` to `getFullName` to make attached revisions, etc., render with "Dnnn", but accidentally made all users render as "username (Full Name)". Be a little more surgical in application of full names.

Test Plan: Created a task and attached a CC, a task and a revision. Verified the task and revision rendered with "Tnn", "Dnn" but the CC rendered as "username".

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3041
2012-07-23 14:01:09 -07:00
epriestley
8cbd860e31 Include email in Git author/committer lookup
Summary: We currently omit email from Git author/committer lookups, which gives us some bad results when identify commit authors. Include email. Also simplify this block a little bit.

Test Plan: Ran "reparse.php --message" on several commits, verified that the author/committer seemed reasonable with var_dump()s.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1337

Differential Revision: https://secure.phabricator.com/D3039
2012-07-23 13:59:58 -07:00
Jonathan Hitchcock
045ca4eca5 Minor typo in "Writing Reviewable Code" 2012-07-23 23:12:17 +03:00
epriestley
c85d6d5e1b Fix Diffusion fatal with symbol highlighting
Summary: This returns a list of PHIDs, not objects which need to be pulled.

Test Plan:
Repro'd fatal locally, verified it was fixed.

Repro steps are:

  - Create an arc project associated with a repository, with indexed language(s) and subprojects.
  - View a file in that repository.

Reviewers: nh, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3038
2012-07-23 12:49:43 -07:00
epriestley
839f3df9c2 Stop trasforming "scp-style" URIs into normal URIs in Git
Summary:
See D3033, T1529. We currently transform "scp-style" `user@host:path` URIs into normal `ssh://user@host/path` URIs. This is undesirable for two reasons:

  - The paths aren't always equivalent. They are for GitHub, which is why I missed this originally, but in the general case the ":path" is resolved relatively and the "/path" is resolved absolutely. So this transformation can break things.
  - It confuses users, who do not think of "git@host:path" URIs as SSH URIs even though the SSH protocol is implied.

So stop using them, and just use the "git@host:path" URIs instead. This is a bit messy since we have some validation built up on top of URIs. Hopefully we can get rid of more of this in the future as we simplify repository management.

Test Plan: Unit tests cover this stuff pretty well. Made a new git repository with a "git@host:path" style URI and did pull/discover on it, verified the right URI was used.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1529

Differential Revision: https://secure.phabricator.com/D3036
2012-07-23 12:05:03 -07:00
epriestley
81fb6aade2 Modernize MetaMTA + daemon documentation
Summary: MetaMTA + daemons used to be pretty hard but @nh landed some patches a while ago that make it way eaiser. Back off the "ooh scary config" text in the documentation, since this option will just work for ~every install now.

Test Plan: Read it.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1525

Differential Revision: https://secure.phabricator.com/D3037
2012-07-23 12:04:53 -07:00
Bob Trahan
bc29a3e8a2 Make inline comment preview work in Diffusion
Summary: created a PhabricatorInlineCommentPreviewController so controllers in Diffusion and Differential respectively just have to handle the URI mapping and data loading like good little controllers.

Test Plan:
left inline comments on commits, deleted inline commits, submitted inline comments -- all worked well
did the same on some diffs

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1176

Differential Revision: https://secure.phabricator.com/D3034
2012-07-23 11:01:28 -07:00
epriestley
49205f977c Add/update some documentation on branches and reviewable code
Summary:
  - The most common workflow complaint I've seen recently is something like "how do I use Differential with a branch full of random code that me and several other developers all commit to"? There are some okay answers ("commandeer") but I think the best answer is "don't do that". Add a document explaining how development works at Facebook (and many other companies) without the use of feature branching, why it's better, and how you can lay the technical groundwork you need to to stop doing this.
  - Add a general "smaller commits are better" and "your commit messsage should provide context" document.
  - Minor updates to other stuff as my understanding of Mercurial has been refined.

Test Plan: Generated and read documentation.

Reviewers: btrahan, vrana, schrockn

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3025
2012-07-23 10:33:53 -07:00
epriestley
4dd707b189 Include explicit "Auditors:" in initial Diffusion email
Summary:
See https://groups.google.com/forum/?hl=en_US&fromgroups#!topic/phabricator-dev/ZonbTN-0TGw

When a commit's author explicitly specifies users as "Auditors", send them an email about the commit.

Test Plan: Made a commit with "Auditors: turtle", ran "reparse.php --herald" on it, verified user "turtle" got included in the intial email.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, MarkChuCarroll

Differential Revision: https://secure.phabricator.com/D3032
2012-07-23 09:24:02 -07:00
epriestley
59db282624 Fix an issue with email "Re:" prefix
Summary: See detailed discussion in T1543.

Test Plan:
  - Enabled multiplexing.
  - Set user A to "enable Re".
  - Created a task owned by user B with user A cc'd.
  - Verified A got no "Re:" before this patch.
  - Applied patch.
  - Verified A got "Re:" after this patch.

Reviewers: nh, btrahan, vrana

Reviewed By: nh

CC: aran

Maniphest Tasks: T1543

Differential Revision: https://secure.phabricator.com/D3031
2012-07-23 09:23:54 -07:00
Bob Trahan
ac46ea39b8 Make it so the diffusion comment panel can be haunted
Summary: ...basically by pounding the DOM a bit to be a little closer to differential. I also make the "add comment" UI show up if and only if the commit is rendering properly.

Test Plan: prezzed "Z" on diffusion and noted it was like differential now

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1177

Differential Revision: https://secure.phabricator.com/D3027
2012-07-23 09:15:10 -07:00
Bob Trahan
96860c5cc1 Add full text search to maniphest.query
Summary: ...also swapped "status" and "order" so "status" is first, as in my testing it was sub-optimal to specifiy status (more of "what i want") after order ("how I want it")

Test Plan: ran various queries on my test instance via conduit console and the results all seem correct

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1381

Differential Revision: https://secure.phabricator.com/D3028
2012-07-23 09:14:45 -07:00
Alan Huang
f5c08f113e Add a formatting reference link to the inline comment editor
Summary:
Various text boxes have a documentation link below them.
Make the Differential inline comment box one of them.

Test Plan:
Loaded Differential revision in sandbox. Played around
with inline comments.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1513

Differential Revision: https://secure.phabricator.com/D3024
2012-07-20 14:04:28 -07:00
epriestley
9be12551a9 Move Task <=> Revision storage to Edges
Summary:
  - Add edges for this relationship.
  - Use edges to store this data.
  - Migrate old data.
  - Fix some warnings with generating feed stories about Aux and Edge transactions.
  - Fix a task-task edge issue with "Create Subtask".

Test Plan:
  - Migrated data, verified reivsions showed up.
  - Attached and detached tasks to revisions and vice versa.
  - Created a new revision with attached tasks.
  - Created a subtask.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3018
2012-07-20 08:59:39 -07:00
Nick Harper
0a3e49cd76 Show cross-references in diffusion
Summary:
We do this in differential. To do this in diffusion, we need to know the
arcanist project (which I do by loading all possible projects for the
repository) and the language.

Test Plan:
load a php file in diffusion to see crossreferences; load a text file and
check darkconsole that it didn't try to crossreference.

Reviewers: epriestley, vrana, jungejason

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3020
2012-07-19 22:01:31 -07:00
vrana
293199b5d3 Fix doc links 2012-07-19 15:04:21 -07:00
epriestley
ba4fb05d91 Fix translations
Summary: Theses are sort of silly anyway since they should all have the actor in them rather than being sentence fragments, but make them work OK for English at least. See D3013.

Test Plan:
Ran:

  echo pht('added %d dependencie(s): %s', 1, 'derp')."\n";
  echo pht('added %d dependencie(s): %s', 2, 'derp, derp')."\n";

Got:

  added dependency: derp
  added dependencies: derp, derp

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3015
2012-07-19 11:45:08 -07:00
vrana
43b94a6f95 Redirect after post when changing Diffusion view
Summary: I spend most time in software development by being lazy.

Test Plan: Changed view, reloaded page, viewed the file without sending the form.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3012
2012-07-19 10:27:53 -07:00
Evan Priestley
e746ccfeeb Merge pull request #163 from floatingLomas/master
Fixed some @{method} links in PhabricatorEdgeEditor
2012-07-19 09:53:38 -07:00
Jonathan Lomas
0be6d87a45 Fixed some @{method} links in the PhabricatorEdgeEditor documentation. 2012-07-19 09:49:10 -07:00
Bob Trahan
108ffb9e0a Fix small bugs regarding Maniphest Task <=> Differential Revision attachments
Summary:
introduced in D3006, D3007. we need a list of phids for revision and now that its attachments its always two way.

without this patch revisions don't show up on maniphest and attaching from either mani or diffu only has the attachment show up where you did it. (since two_way = false)

Test Plan: attached revisions and tasks to one another and verified things were showing up where they should

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3011
2012-07-19 09:44:23 -07:00
Bob Trahan
ae13d33859 Phame - introduce blogs
Summary:
blogs are collections of posts. a blog also has metadata like a name, description and "bloggers" that can edit the metadata of the blog and contribute posts.

changes include the post edit flow where bloggers can now select which blogs to publish to. also made various small tweaks throughout the UI to make things sensical and clean as the concept of blogs is introduced.

there's edges powering this stuff.  bloggers <=> blogs and posts <=> blogs in particular.

Test Plan:
made blogs, deleted blogs, tried to make blogs with no bloggers. all went well.
verified ui to publish only showed up for public posts, published posts to blogs, un-published posts to blogs, re-published posts to blogs, deleted posts and verified they disappeared from blogs.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3003
2012-07-19 09:03:10 -07:00
epriestley
ee709a0543 Use Edges to store dependencies between revisions in Differential
Summary: See D3006. Move this data to the edge store.

Test Plan:
  - Created dependencies, migrated, verified dependencies were preserved.
  - Added new dependencies, they worked.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3007
2012-07-18 20:42:06 -07:00
epriestley
9196a6bd9f Use Edges to store dependencies between tasks in Maniphest
Summary:
  - Use edges to store "X depends on Y" information in Maniphest.
  - Show both "Depends On" and "Dependent Tasks".
  - Migrate all the old edges.

Test Plan:
  - Added some relationships, migrated, verified they were preserved.
  - Added some new valid relationships, verified tasks got updated with sensible transactions and sent reasonable emails.
  - Tried to add a cycle, got an ugly but effective error.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3006
2012-07-18 20:41:42 -07:00
epriestley
409974fbd6 Add getDestinationPHIDs() to PhabricatorEdgeQuery
Summary: This should simplify a bunch of stuff in D3006 and D3003.

Test Plan: Will update D3006.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3010
2012-07-18 20:41:26 -07:00
epriestley
a7bcc532da Add an iterator to make it easier to perform database migrations
Summary: See D3006, D3007. Make it easier to do migrations like that without holding all results in memory.

Test Plan:
Ran this code with an artificially small page size (2):

  foreach (new LiskMigrationIterator(new DifferentialRevision()) as $rev) {
    echo "Revision ".$rev->getID()."\n";
  }

Verified each revision as loaded and processed.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3008
2012-07-18 20:01:23 -07:00
vrana
6e63674b2d Use class_exists() before is_subclass_of()
Summary: HPHP doesn't autoload without it.

Test Plan: /p/vrana/feed/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3009
2012-07-18 19:57:39 -07:00
vrana
00160ab2a3 Remove ambiguous test 2012-07-18 18:31:34 -07:00
vrana
2c9cdff848 Fix test 2012-07-18 18:24:49 -07:00
vrana
91d8b3c8ab Highlight audited paths in commit detail
Summary:
This doesn't indicate which path is part of which package - I think it would be too heavy.
It just highlights the paths in a similar way as audits are highlighted.

Maybe we can use different colors for highlighting different packages and use them also in paths. We can mix the colors if one path is part of more packages :-).

Test Plan:
Viewed commit with 9 files and 4 packages where I am responsible only for one of them.
Verified that the only file in my package is highlighted.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, haugen

Maniphest Tasks: T1226

Differential Revision: https://secure.phabricator.com/D2982
2012-07-18 18:02:05 -07:00
Marcel Beck
823555ce30 Add LDAP Referrals Option
Summary: In order to perform the searches on Windows 2003 Server Active Directory you have to set the LDAP_OPT_REFERRALS option to 0

Test Plan: Test if LDAP works with Windows 2003 AD

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3004
2012-07-18 23:29:40 +02:00
Nick Harper
67c302ae4f Send messages with only a CC
Summary: This keeps people in the correct To or CC field on multiplexed messages.

Test Plan:
with multiplexing on, checked that I received an email with me in the CC
field instead of the To field for a diff I'm CC'd on.

Reviewers: epriestley, jungejason, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2999
2012-07-18 11:45:46 -07:00
mkedia
ce3a3f1d14 Merge branch 'master' of github.com:facebook/phabricator 2012-07-17 20:17:48 -07:00
mkedia
1383683b1b [easy] add phid to repository.query
Summary:
owners.query returns paths as (repo phid, path). That should
get fixed in some non-breaking way, but for now just add phid to
repository.query so I can support non-E repos.

Would like to sneak this in before tomorrow's push

Test Plan: http://phabricator.mkedia.dev4925.snc6.facebook.com/conduit/method/repository.query/

Reviewers: epriestley, nh, vrana

Reviewed By: nh

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3001
2012-07-17 20:17:03 -07:00
Neal Poole
7081923bd7 Adding 'Secure Browsing' config setting to Facebook OAuth.
Summary: The Graph API exposes a new field, security_settings, which allows applications to see whether a user has enabled Secure Browsing. This diff adds a configuration setting to Phabricator which forces users to have Secure Browsing enabled when logging in via Facebook.

Test Plan: With the configuration setting off, verify that secure browsing does not affect the ability to log in. With the configuration setting on and secure browsing off, verify that the login attempts is rejected. Then verify that the login attempt succeeds when secure browsing is enabled.

Reviewers: epriestley

Reviewed By: epriestley

CC: arice, aran, Korvin

Maniphest Tasks: T1487

Differential Revision: https://secure.phabricator.com/D2964
2012-07-17 18:18:16 -07:00
vrana
0d162e15f6 Make Diffusion view sticky
Summary: Blame can be slow.

Test Plan:
Viewed a file with no preference, saw blame.
Changed view, saw it.
Viewed a file, saw the changed view.
Viewed a file as raw document.

Reviewers: Two9A, epriestley

Reviewed By: epriestley

CC: aran, epriestley, Korvin, btrahan

Maniphest Tasks: T1278

Differential Revision: https://secure.phabricator.com/D3000
2012-07-17 17:38:29 -07:00
epriestley
30deacdbaf Fix some more ldap issues
Summary:
  - LDAP import needs to use envelopes.
  - Use ldap_sprintf().

Test Plan: Configured an LDAP server. Added an account. Imported it; logged in with it. Tried to login with accounts like ",", etc., got good errors.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2995
2012-07-17 14:05:26 -07:00
Evan Priestley
a620a172a0 Merge pull request #161 from ojacobson/phd-status-exit-status
Use exit statuses to communicate result of 'phd status' to tools
2012-07-17 12:25:02 -07:00
Owen Jacobson
e098f5d275 Mention non-zero exit from 'phd status' in 'phd help'. 2012-07-17 15:20:53 -04:00
epriestley
dd70c59465 Use OpaqueEnvelopes for all passwords in Phabricator
Summary:
See D2991 / T1526. Two major changes here:

  - PHP just straight-up logs passwords on ldap_bind() failures. Suppress that with "@" and keep them out of DarkConsole by enabling discard mode.
  - Use PhutilOpaqueEnvelope whenever we send a password into a call stack.

Test Plan:
  - Created a new account.
  - Reset password.
  - Changed password.
  - Logged in with valid password.
  - Tried to login with bad password.
  - Changed password via accountadmin.
  - Hit various LDAP errors and made sure nothing appears in the logs.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2993
2012-07-17 12:06:33 -07:00
epriestley
ae2e73ce80 Add "stop on redirect" and "always profile" debugging options
Summary:
Currently, it's hard to debug performance issues on POST pages. Add flags to stop redirects and always collect profiles.

Also fix an issue with "all" profiles. This feature is mostly just for profiling DarkConsole itself and is rarely used, I think it's been broken for some time. There's no way to get to it with the UI.

NOTE: Some JS workflows don't stop on redirect because they use JS/AJAX redirects.

Test Plan: Enabled options, browsed, got stopped on redirects and had profiles generated. Disabled options and verified redirects and profiles work normally.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2990
2012-07-17 12:06:25 -07:00
epriestley
9d26ef4248 Improve differential.createinline
Summary:
  - Share code between `createinline` and `getcomments`
  - Make them both extend the base class.
  - Sync up the parameters (this is more or less nonbreaking since the call is ~6 hours old).

Test Plan: Created and fetched inlines via the API.

Reviewers: alanh, vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1500

Differential Revision: https://secure.phabricator.com/D2988
2012-07-17 12:06:18 -07:00
Owen Jacobson
883e11f761 Retain pid files for dead daemons in 'phd status'.
'phd status' should have a stable result when invoked multiple times.
Automatically removing PID files for dead daemons every time 'phd status' is
invoked prevents tools from noticing that a daemon has died if something
happens to invoke 'phd status' before the tool looks. This affects Puppet
noticably, since it probably runs the status command every half hour.
2012-07-17 13:48:24 -04:00
Owen Jacobson
420d6426f9 'phd status' should exit with non-zero if daemons are not running.
'phd status' may be invoked by tools (such as puppet) which need to make
automated decisions about whether to start/restart the daemon. To enable this,
'phd status' now exits with 0 if all daemons are running, 1 if no daemons are
running, and 2 if some (but not all) daemons are running.
2012-07-17 13:48:15 -04:00
epriestley
02f40fd7ef Allow noncritical blocks in email bodies to be disabled via config
Summary:
See T931. LLVM users would also prefer simpler mail, and have similarly harsh opinions about the current state of affairs.

Allow installs to disable the hint blocks if they don't want them.

Test Plan: Sent myself mail with these settings on/off, got mail with/without the blocks.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T931

Differential Revision: https://secure.phabricator.com/D2968
2012-07-16 19:02:06 -07:00
epriestley
378feb3ffb Centralize rendering of application mail bodies
Summary: This is a minor quality-of-life improvement to prevent D2968 from being as nasty as it is.

Test Plan: Ran unit tests; generated Differential, Maniphest and Diffusion emails and verified the bodies looked sensible.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T931

Differential Revision: https://secure.phabricator.com/D2986
2012-07-16 19:01:43 -07:00
mkedia
508f1b19f9 [easy] Add limit param to recent commits
Summary: as title

Test Plan: https://phabricator.fb.com/conduit/method/diffusion.getrecentcommitsbypath/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2838
2012-07-16 17:46:42 -07:00
vrana
5daf08048b Fix path autocomplete
Summary: Paths autocompleter sometimes omits `/`.

Test Plan:
# Go to https://secure.phabricator.com/owners/new/.
# Write `/specs/h` to path.
# Delete the second slash resulting in `/specsh`.
# Don't find `/specshistorytest.html` in autocomplete.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2981
2012-07-16 13:28:16 -07:00
Alan Huang
8065d72899 Add a differential.createinline method to Conduit
Summary: a silly thing because I was bored

Test Plan: and I said "arc call-conduit", and there were comments

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1500

Differential Revision: https://secure.phabricator.com/D2971
2012-07-16 11:49:06 -07:00
epriestley
22660cff2a OMG
Summary: NOOOO

Test Plan: Image macros work again.

Reviewers: kdeggelman, btrahan, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2979
2012-07-16 10:35:36 -07:00
epriestley
cb8120551f Don't special-case LiskDAO->load(0)
Summary:
Lisk currently behaves in two different ways if you call it like `load("cow")` (throws) versus `load(99999999)` (returns null), where neither ID exists.

This was intended to catch programming errors as distinct from missing data, but in practice the former is very rare and you have to handle the latter in most cases anyway. The case where you pass "0" is particularly confusing. See D2971 for an example.

On the balance, I think this ends up being far more confusing than helpful. Instead, just return NULL if we're sure there's no such object.

Test Plan: Reasoned about program behavior.

Reviewers: alanh, btrahan, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2977
2012-07-16 09:50:23 -07:00
epriestley
8fd97f4352 When users submit a no-op edit in Phriction, don't update the document
Summary:
See T1501. When users mash "save", stop them if they didn't change anything.

Also, don't default-fill the "edit notes" field with the previous notes. This is meant to be more like a commit message for your changes.

Test Plan: Edited a document with no changes, got a dialog. Edited a document with a title change only and a description change only, things worked. Edited a document with a previous "Edit notes", got a blank default fill.

Reviewers: alanh, btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1501

Differential Revision: https://secure.phabricator.com/D2978
2012-07-16 09:40:10 -07:00
vrana
23f2ffb81c Add footer link to report a bug
Summary:
According to http://www.phabricator.com/docs/phabricator/article/Give_Feedback!_Get_Support!.html, we love feedback.
But there is no way to provide it without reading the docs.

Test Plan: Clicked the link.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, ldemailly

Differential Revision: https://secure.phabricator.com/D2975
2012-07-16 09:37:27 -07:00
Jason Ge
e0d4793e74 Show responsiveness' in differential stats
Summary: For "accept" and "reject" action, find the time between the action and last time the revision was updated by a new diff. Show it as the responsiveness row.

Test Plan: view it for a couple of engineers.

Reviewers: epriestley, vii

Reviewed By: epriestley

CC: vrana, nh, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2970
2012-07-13 10:38:27 -07:00
Evan Priestley
f5754ddadc Merge pull request #157 from avish/active-directory-ldap
Add active-directory domain-based ldap authentication support
2012-07-13 08:44:41 -07:00
Avishay Lavie
226cf288e9 Add active-directory domain-based ldap authentication support
Summary: Add active-directory domain-based ldap authentication support

Test Plan: Tested on a live install against Active Directory on a Windows Server

Reviewers: epriestley

CC: aran, epriestley

Maniphest Tasks: T1496

Differential Revision: https://secure.phabricator.com/D2966
2012-07-13 15:19:34 +03:00
epriestley
621bac74c7 Unfatal the "show change" link in Maniphest description edits
Summary: I missed this callsite in D2946. Transition it to the new markup cache.

Test Plan: Clicked "show change" on a description edit transaction, got the change instead of a fatal.

Reviewers: alanh, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1502

Differential Revision: https://secure.phabricator.com/D2972
2012-07-12 21:34:53 -07:00
epriestley
9574b91e55 Add options to include patches inline or attached for Diffusion commit emails
Summary:
See D818 for an older attempt at this. Support code has matured to the point where the patch is pretty straightforward.

@tido, this was a long-standing request from Aditya back in the day.

Test Plan: Used `reparse.php --herald` to send myself a bunch of emails with various patch configurations. Confirmed that limits are respected, reasonable errors arise when they're violated, etc. (Timeout is a little funky but that's out of scope here, I think.)

Reviewers: btrahan

Reviewed By: btrahan

CC: tido, aran

Maniphest Tasks: T456

Differential Revision: https://secure.phabricator.com/D2967
2012-07-12 13:33:35 -07:00
epriestley
2b690372de Fix several issues with Differential exception email
Summary:
  - Assigning $cc_phids clobbers the correct value assigned on line 80.
  - Remove stack trace noise, the trace is always meaningless and well-known.
  - Actually show the original body.

Test Plan: Piped mail to the mail receiver and verified the errors didn't CC revision CCs, no longer had traces, and included the original raw text body.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2969
2012-07-12 13:33:26 -07:00
Evan Priestley
62e039b748 Merge pull request #150 from linead/ldap_dir
Ldap dir
2012-07-11 18:37:36 -07:00
Evan Priestley
f5196e868d Merge pull request #151 from flxf/sever_typo
Fixed 'Sever' Typo
2012-07-11 17:22:18 -07:00
epriestley
6f20809a51 Improve Windows editor documentation with specific examples for popular editors
Summary: See D2955, D2956, D2957.

Test Plan: Read documentation. Ran all these commands from Git Bash and cmd.exe and used the specified editors to edit blocks of text.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1309

Differential Revision: https://secure.phabricator.com/D2958
2012-07-11 17:02:26 -07:00
epriestley
ff4774a970 Don't fatal on missing commit in DiffusionBrowseFileController
Summary:
Still not 100% sure what the repro case here is, but we do something similar on line 438 above. One case could be an SVN repo with a subpath specified, I think.

In any case, don't fatal if we're missing the commit.

Test Plan: Loaded some browse views, although I don't have a repro.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2959
2012-07-11 17:02:15 -07:00
Bob Trahan
dc75e79cb5 Make IRC Bot connect on both successful end of MOTD (376) and non-successful MOTD (422)
Summary: http://www.networksorcery.com/enp/protocol/irc.htm

Test Plan: augment code with an additional debug line (phlog('hi');) so I can see my case was trigged and it will fall through. setup an ill-configured IRC server with ngircd. Configure an ircbot to connect to said ill-configured IRC server. verify ircbot connected to channel. verify in irc bot logs that debug line was invoked.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1452

Differential Revision: https://secure.phabricator.com/D2962
2012-07-11 16:27:41 -07:00
epriestley
fcd04708d2 Add markup cache collection to the GC daemon
Summary: Allow the GC daemon to collect the new markup cache.

Test Plan: Ran gc daemon in "debug" mode, saw it collect cache entries.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2947
2012-07-11 11:40:18 -07:00
epriestley
5d8b75b4da Use the unified markup cache for Maniphest
Summary:
  - See D2945.
  - Drop `cache` field from ManiphestTransaction.
  - Render task descriptions and transactions through PhabricatorMarkupEngine.
  - Also pull the list of macros more lazily.

Test Plan:
  - Verified transactions and transaction preview work correctly and interact with cache correctly.
  - Verified tasks descriptions and task preview work correctly.
  - Verified we don't hit the imagemacro table when we're rendering everything from cache anymore.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2946
2012-07-11 11:40:10 -07:00
Felix Fung
523395590b Fixed 'Sever' Typo 2012-07-10 23:11:42 -07:00
Andrew Gallagher
eac8e0e7f3 Support postponed lint status in creatediff conduit call
Test Plan:
Created diff with a postponed linter and the lint status set to such
via arcanist.  Verified lint status showed as postponed in diff view.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1332

Differential Revision: https://secure.phabricator.com/D2932
2012-07-10 20:02:16 -07:00
epriestley
176b4a68a8 Fix some mercurial edge cases
Summary:
  - Old versions of Mercurial give different output for `hg log -- ''` and `hg log`. Just use `hg log`.
  - Branch names with spaces can't be specified in `--rev`. I talked with hstuart in #mercurial and apparently am not crazy.

Test Plan:
  - Viewed history of a repository.
  - Viewed history of a file.
  - Viewed branch `m m m m m 2:ffffffffffff (inactive)`
  - Learned that you checkout this branch with `hg checkout ':m m m m m 2:ffffffffffff (inactive)'`

Reviewers: dschleimer, btrahan

Reviewed By: dschleimer

CC: aran

Maniphest Tasks: T1268

Differential Revision: https://secure.phabricator.com/D2950
2012-07-10 10:36:33 -07:00
Daniel Fullarton
42dfee38e7 Refactor methods, fix style issues 2012-07-11 00:56:38 +10:00
epriestley
e2e9aed4fa Fix symbol handling in symbol query and IRC "Where is x?" handler
Summary: If a symbol's project has no linked repository, we currently explode. Instead, decline to generate a URI and fall back gracefully.

Test Plan: https://secure.phabricator.com/chatlog/channel/%23phabricator/?at=22345

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1465

Differential Revision: https://secure.phabricator.com/D2948
2012-07-09 17:51:42 -07:00
epriestley
2b0b9a1573 Add a generic multistep Markup cache
Summary:
The immediate issue this addresses is T1366, adding a rendering cache to Phriction. For wiki pages with code blocks especially, rerendering them each time is expensive.

The broader issue is that out markup caches aren't very good right now. They have three major problems:

**Problem 1: the data is stored in the wrong place.** We currently store remarkup caches on objects. This means we're always loading it and passing it around even when we don't need it, can't genericize cache management code (e.g., have one simple script to drop/GC caches), need to update authoritative rows to clear caches, and can't genericize rendering code since each object is different.

To solve this, I created a dedicated cache database that I plan to move all markup caches to use.

**Problem 2: time-variant rules break when cached.** Some rules like `**bold**` are time-invariant and always produce the same output, but some rules like `{Tnnn}` and `@username` are variant and may render differently (because a task was closed or a user is on vacation). Currently, we cache the raw output, so these time-variant rules get locked at whatever values they had when they were first rendered. This is the main reason Phriction doesn't have a cache right now -- I wanted `{Tnnn}` rules to reflect open/closed tasks.

To solve this, I split markup into a "preprocessing" phase (which does all the parsing and evaluates all time-invariant rules) and a "postprocessing" phase (which evaluates time-variant rules only). The preprocessing phase is most of the expense (and, notably, includes syntax highlighting) so this is nearly as good as caching the final output. I did most of the work here in D737 / D738, but we never moved to use it in Phabricator -- we currently just do the two operations serially in all cases.

This diff splits them apart and caches the output of preprocessing only, so we benefit from caching but also get accurate time-variant rendering.

**Problem 3: cache access isn't batched/pipelined optimally.** When we're rendering a list of markup blocks, we should be able to batch datafetching better than we do. D738 helped with this (fetching is batched within a single hunk of markup) and this improves batching on cache access. We could still do better here, but this is at least a step forward.

Also fixes a bug with generating a link in the Phriction history interface ($uri gets clobbered).

I'm using PHP serialization instead of JSON serialization because Remarkup does some stuff with non-ascii characters that might not survive JSON.

Test Plan:
  - Created a Phriction document and verified that previews don't go to cache (no rows appear in the cache table).
  - Verified that published documents come out of cache.
  - Verified that caches generate/regenerate correctly, time-variant rules render properly and old documents hit the right caches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1366

Differential Revision: https://secure.phabricator.com/D2945
2012-07-09 15:20:56 -07:00
epriestley
918ac5755c Limit maximum size of Owners package queries
Summary:
Currently, a change may affect a very large number of paths. When we run the OwnersWorker on it, we'll execute a query which looks up packages for the paths. This may exceed "max_allowed_packet". Instead, break the list of paths into smaller chunks.

This is mostly to unblock r4nt / llvm, I'm going to add a more finessed approach to array_chunk() shortly.

Test Plan: Ran `reparse.php --owners` on a revision which affected packages, verified results are the same before and after the change. Set chunk size to 1, verified query results aggregated properly.

Reviewers: btrahan, jungejason, nh

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2943
2012-07-09 10:51:46 -07:00
epriestley
1089a48d4a Allow edges to be configured to prevent cycles
Summary:
Certain types of things we should be storing in edges (notably, Task X depends on Task Y) should always be acyclic. Allow `PhabricatorEdgeEditor` to enforce this, since we can't correctly enforce it outside of the editor without being vulnerable to races.

Each edge type can be marked acyclic. If an edge type is acyclic, we perform additional steps when writing new edges of that type:

  - We acquire a global lock on the edge type before performing any reads or writes. This ensures we can't produce a cycle as a result of a race where two edits add edges which independently do not produce a cycle, but do produce a cycle when combined.
  - After performing writes but before committing transactions, we load the edge graph for each acyclic type and verify that it is, in fact, acyclic. If we detect cycles, we abort the edit.
  - When we're done, we release the edge type locks.

This is a relatively high-complexity change, but gives us a simple way to flag an edge type as acyclic in a robust way.

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2940
2012-07-09 10:39:38 -07:00
epriestley
bf8cbf55b1 Namespace GlobalLocks to storage namespaces
Summary:
Currently, multiple unit tests that acquire global locks will interfere with each other. Namespace the locks so they don't.

(Possibly we should also rename this to PhabricatorStorageNamespaceLock or something since it's not really global any more, but that's kind of unwieldy...)

Test Plan: Acquired locks with --trace and verified they were namespaced properly.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2939
2012-07-09 10:39:30 -07:00
epriestley
d86c4e0366 Store forced connections in the Lisk connection cache
Summary:
In unit tests which use fixtures, we open transactions on every connection we establish. However, since we don't track connections that are established with "$force_new" (currently, only GlobalLock connections) we never close these transactions normally.

Instead of not tracking these connections, track them using unique keys so we'll never get a cache hit on them.

Test Plan: Built unit tests on top of this, had them stop dying from unclosed transactions.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2938
2012-07-09 10:39:21 -07:00
epriestley
7cf6313be9 Add a generic object for unit tests
Summary:
A later diff adds unit tests against edges, but we need real objects to connect with edges. Add some trivial objects to the Harbormaster database to compliment the similar HarbormasterScratchTable.
On its own, this does nothing interesting.

Test Plan: Built unit tests on this in a followup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2937
2012-07-09 10:39:14 -07:00
epriestley
bb714a2bef Update GitHub oauth docs
Summary: GitHub moved these pages around and made it easier to find your application list.

Test Plan: Read docs, verified links are correct.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2941
2012-07-09 10:38:32 -07:00
epriestley
3e15c3580d Simplify build file from data-or-hash code
Summary:
We have a bit more copy-paste than we need, consolidate a bit.

(Also switch Mercurial to download git diffs, which it handles well; we use them in "arc patch".)

Test Plan:
  - Downloaded a raw diff from Differential.
  - Downloaded a raw change from Diffusion.
  - Downloaded a raw file from Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2942
2012-07-09 10:38:25 -07:00
epriestley
18cfab0c36 Allow configuration of the default monospaced font style
Summary: This is a fairly contentious default that we can easily move to configuration.

Test Plan: Changed the default, changed my user setting, reverted my user setting, verified the "settings" page.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2935
2012-07-06 15:39:43 -07:00
Miles Shang
1b8ed98ddc Multi-line highlighting in Diffusion.
Summary: Currently, Diffusion supports highlighting of line ranges passed in the URI. It would be helpful to be able to highlight multiple line ranges.

Test Plan: Accessed directly via URL in my sandbox. Seems to work. I'm not sure what other components use this functionality, but this change should be backwards compatible.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, Korvin

Differential Revision: https://secure.phabricator.com/D2921
2012-07-05 19:44:12 -07:00
Alan Huang
731e0df2b5 Add a differential.abandon Conduit method.
Summary:
Brazenly copied from differential.close. Ulterior motive
is to be able to automate discarding of useless commits from
arcanist testing.

Test Plan:
Called method in phabricator sandbox. Revisions were
successfully abandoned.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2928
2012-07-05 16:47:04 -07:00
epriestley
78d2f08fcd Add an Aphlict CLI client
Summary: Depends on D2926. Adds a simple CLI client for Aphlict to make it easier to debug stuff.

Test Plan: Ran client, saw debug messages.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2927
2012-07-05 16:04:04 -07:00
epriestley
63be89ba00 Improve error message for error 2006
Summary:
See discussion here:

https://secure.phabricator.com/chatlog/channel/%23phabricator/?at=21186

Basically, MySQL usually raises a good error if we exceed "max_allowed_packet":

  EXCEPTION: (AphrontQueryException) #1153: Got a packet bigger than 'max_allowed_packet' bytes

But sometimes it gives us a #2006 instead. This is documented, at least:

>"With some clients, you may also get a Lost connection to MySQL server during query error if the communication packet is too large."

  http://dev.mysql.com/doc/refman//5.5/en/packet-too-large.html

Try to improve the error message to point at this as a possible explanation.

Test Plan: Faked an error, had it throw, read exception message. See also chatlog.

Reviewers: btrahan, skrul

Reviewed By: skrul

CC: aran

Differential Revision: https://secure.phabricator.com/D2923
2012-07-05 16:03:58 -07:00
epriestley
ae6661ac3f Document event --trace added by D2918
Summary: Also fix a doc typo.

Test Plan: Read.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1092

Differential Revision: https://secure.phabricator.com/D2919
2012-07-05 16:03:49 -07:00
epriestley
ce360926b7 Allow PhabricatorGlobalLock to block
Summary: See D2924.

Test Plan: Ran locks with blocking timeouts.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2925
2012-07-05 16:03:43 -07:00
Bob Trahan
67691c196e Phame - add an "Everyone but Me" view and make published posts truly accessible to the whole world.
Summary:
accessibility covers not only a given post but also the various "published" views.

to keep the code relative clean, this diff also splits up the post list controller logic quite a bit. this also feels like good preparation for some other work around introducing "blogs" which are collections of published posts from bloggers with some fancy features around that.

Test Plan: clicked around various parts of the Phame application as a logged in user, a logged in user with no personal posts, and without any user logged in at all. various views all seemed reasonable.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D2898
2012-07-05 15:43:14 -07:00
epriestley
4dd5bcf1cd Don't fail in Diffusion if .gitmodules is missing
Summary:
See T1448. If this file isn't present, just move on instead of failing, since it's a (sort of) legitimate repository state.

Also fix some silliness a little later that got introduced in refactoring, I think.

Test Plan: Added an external to my test repo and removed ".gitmodules". Verified that the directory is now viewable after this patch.

Reviewers: btrahan, davidreuss, jungejason

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T1448

Differential Revision: https://secure.phabricator.com/D2922
2012-07-05 06:12:35 -07:00
epriestley
bf9cd55577 Minor, fix an issue where fail to extract the author PHID from an edge transaction. 2012-07-05 05:48:30 -07:00
linead
51d2d06e37 Added ldap import controller 2012-07-04 12:10:38 +10:00
epriestley
ddf67fce58 Add an example event listener, improve documentation, and add a commit discovery event
Summary: Improve documentation around Phabricator events.

Test Plan: Generated and read documentation. Ran test script.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran

Maniphest Tasks: T1092

Differential Revision: https://secure.phabricator.com/D2917
2012-07-03 16:46:27 -07:00
Jason Ge
61b79b5359 Use binary_safe_diff from arcanist
Summary:
binary_safe_diff is needed in arcanist too. Moved it over to
arcanist. See D2915.

Test Plan: diffusion page rendered correctly on binary file.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2916
2012-07-03 13:51:37 -07:00
epriestley
dcf3a31944 Minor, unlock after discovery exception. 2012-07-03 07:46:38 -07:00
Jonathan Lomas
c821639e93 Opt-in approve your own Differential revisions
Summary: Did exactly what @epriestley suggested in T1428#2.

Test Plan: Turn it on in your config, post a revision, accept it.  Turn it off in your config, post a revision, can't accept it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2900
2012-07-03 06:30:48 -07:00
epriestley
948259ee92 Minor, restore missing limits from 310cf00fc3 2012-07-03 06:25:03 -07:00
epriestley
bda5c670bc Add useful text descriptions to edge transactions
Summary: See D2906. This just adds text so they render pretty.

Test Plan:
Got pretty emails and rendered transactions.

{F13706}

Reviewers: btrahan, davidreuss

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2907
2012-07-02 15:42:16 -07:00
epriestley
9f4cfd40bc Insert Maniphest transactions when edges are edited
Summary:
  - See D2741.
  - When EdgeEditor performs edits, emit events.
  - Listen for Maniphest edge events and save the changes as transactions.
  - Do all this in a reasonably generic way that won't take too much rewriting as we use edges more generally.

Test Plan: Attached and detached commits from tasks, saw reasonable-looking transactions spring into existence.

Reviewers: btrahan, davidreuss

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2906
2012-07-02 15:42:06 -07:00
epriestley
310cf00fc3 Consolidate feed query code
Summary: Simplify FeedQuery by making it extend from PhabricatorIDPagedPolicyQuery

Test Plan: Looked at feed on home, projects, user profile, and called `feed.query`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2905
2012-07-02 15:41:19 -07:00
epriestley
3a453f2cce Fix a typo
Summary: This should be "files".

Test Plan: Read carefully.

Reviewers: btrahan, nikil

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1435

Differential Revision: https://secure.phabricator.com/D2909
2012-07-02 15:22:35 -07:00
epriestley
a33e84e1e5 Add table markup to Phabricator
Summary: See D2902.

Test Plan: Made tables, generated docs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2903
2012-07-02 14:44:38 -07:00
epriestley
534c0aa326 Minor, move all storage/query/db code to src/infrastructure/storage 2012-07-02 10:49:00 -07:00
epriestley
9ba6ebb97b Minor, move all remarkup code to src/infrastructure/markup/ 2012-07-02 10:44:37 -07:00
epriestley
d49b06c322 Consolidate PhabricatorFeedStory construction code
Summary: This code is duplicated in two places; share it.

Test Plan: Looked at feed and notifications.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2901
2012-07-02 10:37:22 -07:00
epriestley
f5f88d8698 Minor, fix an issue where %e is empty from git log. 2012-07-02 07:29:15 -07:00
epriestley
1c1fdf0448 Minor, move the "repair" instructions to the right place. 2012-07-02 07:22:56 -07:00
epriestley
16c20c92fe Minor, fix an issue where we may fail to select a translation for mail. 2012-07-02 07:17:56 -07:00
epriestley
fc45398ba9 Support basic notification aggregation
Summary: This is both only partially complete (supports Maniphest only) and somewhat overcomplicated (includes support for applying similar algorithms to Feed), but provides runtime aggregation of notifications.

Test Plan: {F13502}

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2884
2012-07-01 11:08:59 -07:00
epriestley
126f8e7240 Document git:branch-unique(*) and arc:bookmark
Summary: Explain how these work. Let me know if there's a clearer way to explain "arc:bookmark".

Test Plan: Generated / viewed documentation.

Reviewers: dschleimer

Reviewed By: dschleimer

CC: aran

Maniphest Tasks: T1233

Differential Revision: https://secure.phabricator.com/D2868
2012-07-01 11:07:05 -07:00
dschleimer
86fa4fd97f [Phabricator] track Mercurial bookmarks for differential diffs
Summary:
This adds all the changes necessary to track the active Mercurial
bookmark for differential diffs.  We render both branch and bookmark
information in the branch field of the Differential revison view, as
seen in
https://secure.phabricator.com/file/data/kzpmu3evfkukxdjyxrfz/PHID-FILE-eqorsqupxvwirqi2s5lo/bookmark_differential.jpg

The Arcanist half of this is https://secure.phabricator.com/D2896

Test Plan:
Mostly D2896.

Additionally, loaded a diff created with a bookmark, as per the link in the summary.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1331

Differential Revision: https://secure.phabricator.com/D2897
2012-06-30 15:41:58 -07:00
epriestley
f666c3d559 Minor, fix wording for some maniphest stories 2012-06-30 06:53:11 -07:00
David Reuss
fd94700d69 Minor, really fix empty reference properties in diffusion
Summary:
Oh man, after the explode/map, the output when no references is actually

  array(1) {
    [0]=> string(0) ""
  }

.. making the falsey check fail to work as expected.

Test Plan: same as D2892, but with a little more scrutiny :p

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2893
2012-06-30 06:43:33 -07:00
David Reuss
19d58564d1 Minor, don't show references in diffusion if there isn't any
Summary:
When there are no other references (and there will be none ususally,
if D2891 is accepted), we would show "References:" with empty contents.

Avoid that by testing for empty output after applying stupid filtering.

Test Plan: Looked in a standard repository with no special refs.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2892
2012-06-30 06:26:36 -07:00