1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-24 06:20:56 +01:00
Commit graph

130 commits

Author SHA1 Message Date
epriestley
63ce372480 Add "DiffusionRawDiffQuery"
Summary:
  - This is only slightly useful for updating Differential, since DiffQuery (vs RawDiffQuery) already gets you most of what you need. The only thing is that DiffQuery returns the diff for one path only right now(and the SVN version is very "special"). Should be easy to fix in the Git/HG cases at least, though (or maybe just use RawDiffQuery to avoid the SVN mess).
  - Added a "download raw diff" link.

Test Plan: Viewed Diffusion and raw commits for SVN, Mercurial and Git repositories.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2350
2012-05-02 13:43:45 -07:00
epriestley
570feee199 Make default database namespace configurable
Summary: Allow the default namespace to be set in configuration, so you can juggle multiple copies of sandbox test data or whatever.

Test Plan: Changed default namespace, verified web UI and "storage" script respect it.

Reviewers: btrahan, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T345

Differential Revision: https://secure.phabricator.com/D2341
2012-04-30 11:56:58 -07:00
epriestley
8ed48a89f4 Use a disk-based default avatar, not a database-based one
Summary:
This is mostly in an effort to simplify D2323. Currently, we load one image into the database by default. This is a weird special case that makes things more complicated than necessary.

Instead, use a disk-based default avatar.

Test Plan: Verified that a user without an image appears with the default avatar as a handle, in profile settings, and on their person page.

Reviewers: btrahan, vrana, edward, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T345

Differential Revision: https://secure.phabricator.com/D2331
2012-04-27 17:44:10 -07:00
epriestley
3ce69b6306 Allow Phabricator to write an access log using PhutilDeferredLog
Summary: Provide a configurable access log.

Test Plan:
Got a sensible-looking log including logged-in, logged-out, conduit, 404, etc:

  [Mon, 23 Apr 2012 20:08:12 -0700]	32599	orbital	-	epriestley	DifferentialCommentPreviewController	-	/differential/comment/preview/42/	http://local.aphront.com:8080/D42	200	65406
  [Mon, 23 Apr 2012 20:08:12 -0700]	32881	orbital	-	epriestley	DifferentialChangesetViewController	-	/differential/changeset/	http://local.aphront.com:8080/D42	200	72669
  [Mon, 23 Apr 2012 20:08:39 -0700]	32882	orbital	127.0.0.1	epriestley	DifferentialRevisionListController	-	/differential/	http://local.aphront.com:8080/D42	200	106444
  [Mon, 23 Apr 2012 20:08:54 -0700]	32867	orbital	127.0.0.1	epriestley	DifferentialRevisionListController	-	/differential/	http://local.aphront.com:8080/differential/	200	112229
  [Mon, 23 Apr 2012 20:09:05 -0700]	32530	orbital	127.0.0.1	epriestley	PhabricatorDirectoryMainController	-	/	http://local.aphront.com:8080/differential/	200	141350
  [Mon, 23 Apr 2012 20:09:10 -0700]	32598	orbital	127.0.0.1	epriestley	PhabricatorDirectoryCategoryViewController	-	/directory/6/	http://local.aphront.com:8080/	200	43474
  [Mon, 23 Apr 2012 20:09:12 -0700]	32880	orbital	127.0.0.1	epriestley	PhabricatorConduitConsoleController	-	/conduit/	http://local.aphront.com:8080/directory/6/	200	139340
  [Mon, 23 Apr 2012 20:09:15 -0700]	32868	orbital	127.0.0.1	epriestley	PhabricatorConduitAPIController	arcanist.projectinfo	/api/arcanist.projectinfo	http://local.aphront.com:8080/conduit/	200	128774
  [Mon, 23 Apr 2012 20:10:04 -0700]	32599	orbital	127.0.0.1	epriestley	Phabricator404Controller	-	/asdbmabdmbsm	-	404	38782
  [Mon, 23 Apr 2012 20:10:04 -0700]	32881	orbital	127.0.0.1	-	CelerityResourceController	-	/res/c9a43002/rsrc/css/aphront/request-failure-view.css	http://local.aphront.com:8080/asdbmabdmbsm	200	25160
  [Mon, 23 Apr 2012 20:10:57 -0700]	32882	orbital	127.0.0.1	epriestley	PhabricatorLogoutController	-	/logout/	http://local.aphront.com:8080/asdbmabdmbsm	200	40810
  [Mon, 23 Apr 2012 20:10:57 -0700]	32867	orbital	127.0.0.1	-	PhabricatorLoginController	-	/login/	http://local.aphront.com:8080/asdbmabdmbsm	200	42526
  [Mon, 23 Apr 2012 20:10:59 -0700]	32919	orbital	127.0.0.1	-	PhabricatorLoginController	-	/login/	http://local.aphront.com:8080/asdbmabdmbsm	200	49052
  [Mon, 23 Apr 2012 20:10:59 -0700]	32880	orbital	127.0.0.1	-	CelerityResourceController	-	/res/c80156c4/rsrc/js/application/core/behavior-dark-console.js	http://local.aphront.com:8080/login/	200	33166
  [Mon, 23 Apr 2012 20:10:59 -0700]	32868	orbital	127.0.0.1	-	CelerityResourceController	-	/res/4965d970/rsrc/css/aphront/dark-console.css	http://local.aphront.com:8080/login/	200	38078
  [Mon, 23 Apr 2012 20:10:59 -0700]	32599	orbital	127.0.0.1	-	CelerityResourceController	-	/res/pkg/8a5de8a3/javelin.pkg.js	http://local.aphront.com:8080/login/	200	40534
  [Mon, 23 Apr 2012 20:10:59 -0700]	32882	orbital	127.0.0.1	-	CelerityResourceController	-	/res/pkg/9c4e265b/core.pkg.css	http://local.aphront.com:8080/login/	200	41262
  [Mon, 23 Apr 2012 20:10:59 -0700]	32881	orbital	127.0.0.1	-	CelerityResourceController	-	/res/pkg/0c96375e/core.pkg.js	http://local.aphront.com:8080/login/	200	43720
  [Mon, 23 Apr 2012 20:10:59 -0700]	32921	orbital	127.0.0.1	-	CelerityResourceController	-	/res/caa86a45/rsrc/js/javelin/core/init.js	http://local.aphront.com:8080/login/	200	47566
  [Mon, 23 Apr 2012 20:10:59 -0700]	32867	orbital	127.0.0.1	-	CelerityResourceController	-	/res/f46289e9/rsrc/js/application/core/behavior-error-log.js	http://local.aphront.com:8080/login/	200	29328
  [Mon, 23 Apr 2012 20:10:59 -0700]	32919	orbital	127.0.0.1	-	CelerityResourceController	-	/res/7e62ff40/rsrc/image/phabricator_logo.png	http://local.aphront.com:8080/login/	200	25583
  [Mon, 23 Apr 2012 20:10:59 -0700]	32880	orbital	127.0.0.1	-	CelerityResourceController	-	/res/8c6200d3/rsrc/image/sprite.png	http://local.aphront.com:8080/login/	200	29829
  [Mon, 23 Apr 2012 20:11:01 -0700]	32868	orbital	127.0.0.1	-	PhabricatorOAuthLoginController	-	/oauth/facebook/login/  http://local.aphront.com:8080/login/	200	855931
  [Mon, 23 Apr 2012 20:11:02 -0700]	32882	orbital	127.0.0.1	epriestley789	PhabricatorLoginValidateController	-	/login/validate/	http://local.aphront.com:8080/login/	200	29793
  [Mon, 23 Apr 2012 20:11:02 -0700]	32881	orbital	127.0.0.1	epriestley789	PhabricatorDirectoryMainController	-	/	http://local.aphront.com:8080/login/	200	91638

Reviewers: jungejason, btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2310
2012-04-25 07:24:08 -07:00
epriestley
bbe2063443 [NO CLUE WHAT I'M DOING] Add an Elasticsearch engine
Summary:
I have no idea what I'm doing, but here's part of an elasticsearch engine. These things work:

  - Indexing stuff (??)
  - Searching for text/type?
  - Reconstructing things??

All the complicated stuff doesn't work. I'm having a hard time figuring out the best way to model things because elasticsearch's documentation is not exactly the most complete or illuminating.

@amckinley, does this look sane-ish so far? Particularly, the /phabricator/<type>/<phid>/ URI scheme and how I've set up the relationships and fields in the documents?

How should I model the relationship and field queries? I want, like, an "equal" query but it seems like I've got "text" or "term" to work with and neither are exact match? And "term" doesn't consider PHIDs to be terms since they have hyphens in them?

I'll keep kind of slogging my way forward here but if you have valuable wisdom to share it would probably get me to a better end state much faster. The whole query construction phase is pretty much black magic to me.

Test Plan: nyancat

Reviewers: amckinley, vrana

Reviewed By: vrana

CC: jungejason, tuomaspelkonen, aran, 20after4, vrana

Differential Revision: https://secure.phabricator.com/D790
2012-04-20 15:33:09 -07:00
epriestley
fbfccf5ddc Improve Policy options
Summary:
  - Add an "Administrators" policy.
  - Allow "Public" to be completely disabled in configuration.
  - Simplify unit tests, and cover the new policies.

Test Plan: Ran unit tests.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D2238
2012-04-17 07:52:10 -07:00
Bob Trahan
51418900f7 Phame V1 - Phabricator blogging software
Summary:
'cuz we need to be phamous!

V1 feature set

- posts
-- standard thing you'd expect - a title and a remarkup-powered body and...
-- "phame" title - a short string that can be used to reference the story. this gets auto-updated when you mess with the title.
-- configuration - for now, do you want Facebook, Disqus or no comments? this is a per-post thing but feeds from an instance-wide configuration

Please do toss out any must have features or changes.

Test Plan: played around with this bad boy like whoa

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, vrana

Maniphest Tasks: T1111

Differential Revision: https://secure.phabricator.com/D2202
2012-04-12 13:09:04 -07:00
epriestley
c458768415 Fix various threading issues, particularly in Gmail
Summary:
  - Add an explicit multiplexing option, and enable it by default. This is necessary for Mail.app to coexist with other clients ("Re:" breaks outlook at the very least, and generally sucks in the common case), and allows users with flexible clients to enable subject variance.
  - Add an option for subject line variance. Default to not varying the subject, so mail no longer says [Committed], [Closed], etc. This is so the defaults thread correctly in Gmail (not entirely sure this actually works).
  - Add a preference to enable subject line variance.
  - Unless all mail is multiplexed, don't enable or respect the "Re" or "vary subject" preferences. These are currently shown and respected in non-multiplex cases, which creates inconsistent results.

NOTE: @jungejason @nh @vrana This changes the default behavior (from non-multiplexing to multiplexing), and might break Facebook's integration. You should be able to keep the same behavior by setting the options appropriately, although if you can get the new defaults working they're probably better.

Test Plan:
Send mail from Maniphest, Differential and Audit. Updated preferences. Enabled/disabled multiplexing. Things seem OK?

NOTE: I haven't actually been able to repro the Gmail threading issue so I'm not totally sure what's going on there, maybe it started respecting "Re:" (or always has), but @cpiro and @20after4 both reported it independently. This fixes a bunch of bugs in any case and gives us more conservative set of defaults.

I'll see if I can buff out the Gmail story a bit but every client is basically a giant black box of mystery. :/

Reviewers: btrahan, vrana, jungejason, nh

Reviewed By: btrahan

CC: cpiro, 20after4, aran

Maniphest Tasks: T1097, T847

Differential Revision: https://secure.phabricator.com/D2206
2012-04-12 09:31:03 -07:00
epriestley
01907bcccc Allow "Test Plan" to be disabled in config
Summary:
This is a somewhat common request, and far more difficult than necessary currently.

I think the field is useful enough to leave it default-enabled, but there's wide diversity in testing philosophy.

Test Plan: Verified "test plan" field appeared. Disabled config. Verified "test plan" field vanished.

Reviewers: btrahan, vrana, jungejason

Reviewed By: vrana

CC: aran, asouza

Differential Revision: https://secure.phabricator.com/D2193
2012-04-10 13:36:05 -07:00
vrana
2c8e6f99bd Standardize mysql.configuration-provider
Summary: NOTE: BC break!

Test Plan: /

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, nh

Differential Revision: https://secure.phabricator.com/D2130
2012-04-08 21:32:15 -07:00
vrana
d4c5761f41 Customizable MySQL implementation
Test Plan:
- /
- upgrade_schema.php
- Setup
- Try disabling mysql_connect.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2133
2012-04-07 10:54:12 -07:00
epriestley
eaa2ff71d3 Minify static resources
Summary: For production servers, minify CSS and JS by stripping comments, whitespace, etc.

Test Plan: Looked at CSS/JS, it was much smaller.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T264

Differential Revision: https://secure.phabricator.com/D2034
2012-03-28 10:13:53 -07:00
epriestley
ae9d1bf9ae Allow installs to add a custom corp/org header link
Summary: A bunch of installs are doing this to varying degrees of success anyway, make it easier and nudge them toward a more consistent approach.

Test Plan: Set a custom logo, viewed normal and admin pages.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T700

Differential Revision: https://secure.phabricator.com/D2019
2012-03-26 09:29:31 -07:00
epriestley
0a4cbdff5e Straighten out Diffusion file integration
Summary:
This is in preparation for getting the "View Options" dropdown working on audits.

  - Use Files to serve raw data so we get all the security benefits of the alternate file domain. Although the difficulty of exploiting this is high (you need commit access to the repo) there's no reason to leave it dangling.
  - Add a "contentHash" to Files so we can lookup files by content rather than adding some weird linker table. We can do other things with this later, potentially.
  - Don't use 'data' URIs since they're crazy and we can just link to the file URI.
  - When showing a binary file or an image, don't give options like "show highlighted text with blame" or "edit in external editor" since they don't make any sense.
  - Use the existing infrastructure to figure out if things are images or binaries instead of an ad-hoc thing in this class.

Test Plan: Looked at text, image and binary files in Diffusion. Verified we reuse existing files if we've already generated them.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1899
2012-03-19 19:52:24 -07:00
epriestley
06ae6194cc Allow ".ico" files to be viewed
Summary: No reason to keep these out of the defaults, can be useful in web stuff.

Test Plan: Uploaded, viewed a .ico file.

Reviewers: btrahan, paularmstrong

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1883
2012-03-13 17:26:30 -07:00
epriestley
1bff43a0c3 Mask a couple more config keys
Summary: This config controls which values are masked in darkconsole. Mask a couple more vaguely-secret values.

Test Plan: Looked at darkconsole "Config" tab.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1845
2012-03-09 15:45:30 -08:00
epriestley
0962980fef Add an option to inline diffs up to a certain size in emails
Summary:
We already generate patches, but currently attach them. Allow them to be inlined instead (optionally, up to a certain size).

Also allow selection between unified and git patches.

Test Plan: Set these options in my local config, sent out a diff.

Reviewers: btrahan, Makinde

Reviewed By: Makinde

CC: aran, epriestley

Maniphest Tasks: T874

Differential Revision: https://secure.phabricator.com/D1759
2012-03-03 11:05:19 -08:00
epriestley
21f0aba701 Use an inline dialog element for inline comments in Differential
Summary:
The current approach of using a modal overlay dialog to create/edit inline
comments is pretty silly. Use an inline textarea instead.

This element isn't perfect and we have some mild modalness issues, but I think
it's better than the silly thing we've got going on right now. We can keep
poking it as people break it.

Test Plan:
  - Created comments; submitted and undid them in empty and nonempty states.
Used undo for nonempty states + cancel.
  - Edited comments; saved and canceled them. Used undo for changed state.
  - Replied to comments; yada yada as above.
  - Deleted comments.
  - Did various modal trickery where I clicked "Reply" on something else with a
dialog already up, this very mildly glitches but I think it's not a big issue.

Reviewers: vrana, btrahan, Makinde, nh

Reviewed By: vrana

CC: aran, epriestley

Maniphest Tasks: T431

Differential Revision: https://secure.phabricator.com/D1716
2012-02-29 14:28:48 -08:00
epriestley
d7a7bca85c Enable email for audits
Summary:
When users submit an audit, send email to relevant parties informing them.

Allow email to be replied to. Just basic support so far; no "!raise" stuff and
no threading with the Herald commit notification.

Test Plan: Made comments, got email. Replied to email, got comments.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1698
2012-02-27 12:57:57 -08:00
Bob Trahan
7a3f33b5c2 OAuth - Phabricator OAuth server and Phabricator client for new Phabricator OAuth Server
Summary:
adds a Phabricator OAuth server, which has three big commands:
 - auth - allows $user to authorize a given client or application.  if $user has already authorized, it hands an authoization code back to $redirect_uri
 - token - given a valid authorization code, this command returns an authorization token
 - whoami - Conduit.whoami, all nice and purdy relative to the oauth server.
Also has a "test" handler, which I used to create some test data.  T850 will
delete this as it adds the ability to create this data in the Phabricator
product.

This diff also adds the corresponding client in Phabricator for the Phabricator
OAuth Server.  (Note that clients are known as "providers" in the Phabricator
codebase but client makes more sense relative to the server nomenclature)

Also, related to make this work well
 - clean up the diagnostics page by variabilizing the provider-specific
information and extending the provider classes as appropriate.
 - augment Conduit.whoami for more full-featured OAuth support, at least where
the Phabricator client is concerned

What's missing here...   See T844, T848, T849, T850, and T852.

Test Plan:
- created a dummy client via the test handler.   setup development.conf to have
have proper variables for this dummy client.  went through authorization and
de-authorization flows
- viewed the diagnostics page for all known oauth providers and saw
provider-specific debugging information

Reviewers: epriestley

CC: aran, epriestley

Maniphest Tasks: T44, T797

Differential Revision: https://secure.phabricator.com/D1595
2012-02-19 14:00:13 -08:00
Nick Harper
2cf26d8036 Remove links to maniphest, phriction in tactical command, jump nav
Summary:
We don't use maniphest or phriction in our install, so the links/references to
them in tactical command and jump nav can be confusing for users. This hides
these elements if they aren't enabled.

Test Plan: loaded the front page of phabricator in my sandbox, saw they went
away

Reviewers: epriestley, btrahan, jungejason

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1636
2012-02-17 16:45:39 -08:00
epriestley
549146bc7c Move ALL files to serve from the alternate file domain, not just files without
"Content-Disposition: attachment"

Summary:
We currently serve some files off the primary domain (with "Content-Disposition:
attachment" + a CSRF check) and some files off the alternate domain (without
either).

This is not sufficient, because some UAs (like the iPad) ignore
"Content-Disposition: attachment". So there's an attack that goes like this:

	- Alice uploads xss.html
	- Alice says to Bob "hey download this file on your iPad"
        - Bob clicks "Download" on Phabricator on his iPad, gets XSS'd.

NOTE: This removes the CSRF check for downloading files. The check is nice to
have but only raises the barrier to entry slightly. Between iPad / sniffing /
flash bytecode attacks, single-domain installs are simply insecure. We could
restore the check at some point in conjunction with a derived authentication
cookie (i.e., a mini-session-token which is only useful for downloading files),
but that's a lot of complexity to drop all at once.

(Because files are now authenticated only by knowing the PHID and secret key,
this also fixes the "no profile pictures in public feed while logged out"
issue.)

Test Plan: Viewed, info'd, and downloaded files

Reviewers: btrahan, arice, alok

Reviewed By: arice

CC: aran, epriestley

Maniphest Tasks: T843

Differential Revision: https://secure.phabricator.com/D1608
2012-02-14 14:52:27 -08:00
vrana
8da4f981fb Always display Branch in revision
Summary:
I, as an author, sometimes forget branch associated with a revision.
Plus setting ##differential.show-host-field## makes a false sense of security
that branch will stay hidden so that I can name it
//finally_solve_this_crap_which_makes_no_sense//. But it is published in
Accepted and Request Changes e-mails anyway.

Test Plan: Display revision with disabled ##differential.show-host-field##.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1602
2012-02-13 11:02:46 -08:00
vrana
18ba5fa0ad Separate field for branch in revision
Summary:
The main purpose of this change is to allow selecting the branch by
triple-click.
Plus it is not perfectly clear that the text in brackets means branch.

Test Plan: Display revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1585
2012-02-06 17:28:46 -08:00
epriestley
36e72639de Reduce visibility of "Host" and "Path" Differential fields by default
Summary:
See discussion in T838. These fields expose information which it isn't necessary
or useful to expose in the general case.

  - Disable fields by default, allow them to be enabled in config (these fields
were useful for me at Facebook when I had access to all the machines).
  - Remove 'sourcePath' from Conduit methods other than differential.query.
  - Condition 'sourcePath' field in Conduit on the caller being the revision
author. This is a bit hacky but not so awful.

Test Plan:
  - Verified fields are gone by default and restored by configuration.
  - Verified Conduit no longer returns these fields other than
differential.query.
  - Verified field presence/absence according to authorship in
differential.query.
  - Grepped around in arcanist to make sure we aren't relying on sourcePath.
There's a workflow in "arc merge" that technically might hit it, but I think
it's unreachable, definitely irrelvant (we never use source path as a
distinguisher under git/hg, and can't 'arc merge' in SVN) and it's going away
Real Soon Now anyway.

Reviewers: btrahan, arice

Reviewed By: arice

CC: aran, epriestley

Maniphest Tasks: T838

Differential Revision: https://secure.phabricator.com/D1582
2012-02-06 12:14:07 -08:00
vrana
339369dc36 Github is actually GitHub
Test Plan: none

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1557
2012-02-02 17:47:04 -08:00
Chris Piro
69562319be add Haskell extensions to ##differential.whitespace-matters## in default.conf.py
Summary: ##/\.l?hs$/## files are sensitive to changes in whitespace

Test Plan: what, me worry?

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1482
2012-01-25 15:08:58 -08:00
Nick Harper
d5eaef9567 Add retry loop when trying to establish db connection, log retries
Summary:
We retried if a db connection was lost when executing a query, but not when
establishing a connection. I've seen a lot of failures establishing connections
in our install (they go away when retrying), so this diff retries when
establishing connections, and logs when we retry.

Test Plan:
- Loaded phabricator in a sandbox
- Temporarily added a check in the try block to throw if there were still
  retries (to test logging, retry logic)

Reviewers: epriestley, blair

Reviewed By: epriestley

CC: aran, btrahan

Differential Revision: https://secure.phabricator.com/D1460
2012-01-20 13:56:36 -08:00
epriestley
27f52efd37 Minor, fix spelling issues detected by linter. 2012-01-20 07:39:55 -08:00
epriestley
d1ee08b2df Drydock Rough Cut
Summary:
Rough cut of Drydock. This is very basic and doesn't do much of use yet (it
//does// allocate EC2 machines as host resources and expose interfaces to them),
but I think the overall structure is more or less reasonable.

== Interfaces

Vision: Applications interact with Drydock resources through DrydockInterfaces,
like **command**, **filesystem** and **httpd** interfaces. Each interface allows
applications to perform some kind of operation on the resource, like executing
commands, reading/writing files, or configuring a web server. Interfaces have a
concrete, specific API:

  // Filesystem Interface
  $fs = $lease->getInterface('filesystem'); // Constants, some day?
  $fs->writeFile('index.html', 'hello world!');

  // Command Interface
  $cmd = $lease->getInterface('command');
  echo $cmd->execx('uptime');

  // HTTPD Interface
  $httpd = $lease->getInterface('httpd');
  $httpd->restart();

Interfaces are mostly just stock, although installs might add new interfaces if
they expose different ways to interact with resources (for instance, a resource
might want to expose a new 'MongoDB' interface or whatever).

Currently: We have like part of a command interface.

== Leases

Vision: Leases keep track of which resources are in use, and what they're being
used for. They allow us to know when we need to allocate more resources (too
many sandcastles on the existing hosts, e.g.) and when we can release resources
(because they are no longer being used). They also give applications something
to hold while resources are being allocated.

  // EXAMPLE: How this should work some day.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('sandcastle');
  $allocator->setAttributes(
    array(
      'diffID' => $diff->getID(),
    ));
  $lease = $allocator->allocate();
  $diff->setSandcastleLeaseID($lease->getID());

  // ...

  if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) {
    $sandcastle_link = $lease->getInterface('httpd')->getURI('/');
  } else {
    $sandcastle_link = 'Still building your sandcastle...';
  }
  echo "Sandcastle for this diff: ".$sandcastle_link;

  // EXAMPLE: How this actually works now.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('host');
  // NOTE: Allocation is currently synchronous but will be task-driven soon.
  $lease = $allocator->allocate();

Leases are completely stock, installs will not define new lease types.

Currently: Leases exist and work but are very very basic.

== Resources

Vision: Resources represent some actual thing we've put somewhere, whether it's
a host, a block of storage, a webroot, or whatever else. Applications interact
through resources by acquiring leases to them, and then getting interfaces
through these leases. The lease acquisition process has a side effect of
allocating new resources if a lease can't be acquired on existing resources
(e.g., the application wants storage but all storage resources are full) and
things are configured to autoscale.

Resources may themselves acquire leases in order to allocate. For instance, a
storage resource might first acquire a lease to a host resource. A 'test
scaffold' resource might lease a storage resource and a mysql resource.

Not all resources are auto-allocate: the entry-level version of Drydock is that
you manually allocate a couple boxes and configure them through the web console.
Then, e.g.,  'storage' / 'webroot' resources allocate on top of them, but the
host pool itself does not autoscale.

Resources are completely stock, they are abstract shells representing any
arbitrary thing.

Currently: Resource exist ('host' only) but are very very basic.

== Blueprints

Vision: Blueprints contain instructions for building interfaces to, (possibly)
allocating, updating, managing, and destroying a specific type of resource in a
specific location. One way to think of them is that they are scripts for
creating and deleting resources. For example, the LocalHost, RemoteHost and
EC2Host blueprints can all manage 'host' resources.

Eventually, we will support more types of resources (storage, webroot,
sandcastle, test scaffold, phacility deployment) and more providers for resource
types, some of which will be in the Phabricator mainline and some of which will
be custom.

Blueprints are very custom and specific to application types, so installs will
define new blueprints if they are making significant use of Drydock.

Currently: They exist but have few capabilities. The stock blueprints do nearly
nothing useful. There is a technically functional blueprint for host allocation
in EC2.

== Allocator

This is just the actual code to execute the lease acquisition process.

Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2,
acquired a lease on it, interfaced with it, and then released the lease. Ran it
again, got a fresh lease on the existing resource.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1454
2012-01-19 21:12:57 -08:00
Dave Ingram
3edf60627d Add support for marking files as "generated" by regexp against path
Summary:
Not all auto-generated files can include the magical
"generated" annotation for one reason or another, but they may follow
path rules. This patch allows files to be marked as automatically
generated by matching the path with a regular expression.

Test Plan:
Alter 'differential.generated-paths' setting in config.
Create a new diff that affects a file matching one of those regular
expressions. Verify that Differential marks it as automatically
generated and therefore probably not worth reviewing (in the same way as
the magical "generated" annotation.

Reviewers: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D1455
2012-01-19 18:30:19 +00:00
epriestley
ad36865e50 Add optional "Re:" prefix to all threaded mail and allow disabling mail about
your own actions

Summary:
  - Mail.app on Lion has cumbersome threading rules, see T782. Add an option to
stick "Re: " in front of all threaded mail so it behaves. This is horrible, but
apparently the least-horrible option.
  - While I was in there, I added an option for T228.

Test Plan:
  - Sent a bunch of threaded and unthreaded mail with varous "Re:" settings,
seemed to get "Re:" in the right places.
  - Disabled email about my stuff, created a task with just me, got voided mail,
added a CC, got mail to just the CC.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, mkjones

Maniphest Tasks: T228, T782

Differential Revision: https://secure.phabricator.com/D1448
2012-01-18 15:20:50 -08:00
epriestley
bfbe6ec594 Prevent login brute forcing with captchas
Summary: If a remote address has too many recent login failures, require they
fill out a captcha before they can attempt to login.

Test Plan: Tried to login a bunch of times, then submitted the CAPTHCA form with
various combinations of valid/invalid passwords and valid/invalid captchas.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, jungejason

Maniphest Tasks: T765

Differential Revision: https://secure.phabricator.com/D1379
2012-01-12 15:22:05 -08:00
epriestley
02fb5fea89 Allow configuration of a minimum password length, unify password reset
interfaces

Summary:
  - We have a hard-coded minimum length of 3 right now (and 1 in the other
interface), which is sort of silly.
  - Provide a more reasonable default, and allow it to be configured.
  - We have two password reset interfaces, one of which no longer actually
requires you to verify you own the account. This is more than a bit derp.
  - Merge the interfaces into one, using either an email token or the account's
current password to let you change the password.

Test Plan:
  - Reset password on an account.
  - Changed password on an account.
  - Created a new account, logged in, set the password.
  - Tried to set a too-short password, got an error.

Reviewers: btrahan, jungejason, nh

Reviewed By: jungejason

CC: aran, jungejason

Maniphest Tasks: T766

Differential Revision: https://secure.phabricator.com/D1374
2012-01-12 07:39:13 -08:00
epriestley
d16454d45d Improve a race condition in session establishment code
Summary:
If you try to establish several sessions quickly (e.g., by running several
copies of "arc" at once, as in "arc x | arc y"), the current logic has a high
chance of making them all pick the same conduit session to refresh (since it's
the oldest one when each process selects the current sessions). This means they
all issue updates against "conduit-3" (or whatever) and one ends up with a bogus
session.

Instead, do an update against the table with the session key we read, so only
one process wins the race. If we don't win the race, try again until we do or
have tried every session slot.

Test Plan:
  - Wiped conduit sessions, ran arc commands to verify the fresh session case.
  - Ran a bunch of arc piped to itself, e.g. "arc list | arc list | arc list |
...". It succeeds up to the session limit, and above that gets failures as
expected.
  - Manually checked the session table to make sure things seemed reasonable
there.
  - Generally ran a bunch of arc commands.
  - Logged out and logged in on the web interface.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T687

Differential Revision: https://secure.phabricator.com/D1329
2012-01-06 11:33:03 -08:00
epriestley
d43dec1d12 Make it harder to miss errors and warnings while developing Phabricator
Summary:
If a page generates warnings or errors, you only get a little red dot in
DarkConsole which is hard to see. DarkConsole is also fairly big and there are
plenty of reasons not to leave it open all the time.

Instead, unconditionally show a big message to developers if there are errors or
warnings.

We could make this more sophisticated eventually, but the value is just that you
see it.

Test Plan: Browsed pages with and without warnings, got the right banner state.

Reviewers: nh, btrahan, jungejason

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T734

Differential Revision: https://secure.phabricator.com/D1307
2012-01-04 10:21:00 -08:00
jungejason
46439ee8a4 Turn on syntax highlight for .arcconfig
Summary: use js format to display .arcconfig

Test Plan: verified that .arcconfig is highlighted

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1284
2011-12-23 23:35:02 -08:00
epriestley
e45ffda55a Move most remaining sha1() calls to HMAC
Summary:
  - For context, see T547. This is the last (maybe?) in a series of diffs that
moves us off raw sha1() calls in order to make it easier to audit the codebase
for correct use of hash functions.
  - This breaks CSRF tokens. Any open forms will generate an error when
submitted, so maybe upgrade off-peak.
  - We now generate HMAC mail keys but accept MAC or HMAC. In a few months, we
can remove the MAC version.
  - The only remaining callsite is Conduit. We can't use HMAC since Arcanist
would need to know the key. {T550} provides a better solution to this, anyway.

Test Plan:
  - Verified CSRF tokens generate properly.
  - Manually changed CSRF to an incorrect value and got an error.
  - Verified mail generates with a new mail hash.
  - Verified Phabricator accepts both old and new mail hashes.
  - Verified Phabricator rejects bad mail hashes.
  - Checked user log, things look OK.

Reviewers: btrahan, jungejason, benmathews

Reviewed By: btrahan

CC: aran, epriestley, btrahan

Maniphest Tasks: T547

Differential Revision: 1237
2011-12-19 08:56:53 -08:00
epriestley
074bf4ed7d Add a script for purging long-lived caches
Summary: See task; installs occasionally need to do this themselves, and a
script is much better than me telling them to truncate tables.

Test Plan:
Ran various flavors of this command:

  - purge_cache.php
  - purge_cache.php derp
  - purge_cache.php --help
  - purge_cache.php --all
  - purge_cache.php --differential
  - purge_cache.php --differential --maniphest

Then I verified the actual behavior:

  - Visited a Differential revision with comments, observed cache update in
'Services' tab.
  - Visited a Maniphest task with comments, observed cache update in 'Services'
tab.
  - Reloaded a diff standalone view, got a cache update.

Reviewers: Makinde, btrahan, jungejason

Reviewed By: jungejason

CC: aran, jungejason

Maniphest Tasks: T676

Differential Revision: 1214
2011-12-16 12:43:31 -08:00
epriestley
fbfb263cd9 Provide a configuration flag to disable silliness in the UI
Summary: See comments. A few installs have remarked that their organizations
would prefer buttons labled "Submit" to buttons labeled "Clowncopterize".

Test Plan:
  - In "serious" mode, verified Differential and Maniphest have serious strings,
tasks can not be closed out of spite, and reset/welcome emails are extremely
serious.
  - In unserious mode, verified Differential and Maniphest have normal strings,
tasks can be closed out of spite, and reset/welcome emails are silly.
  - This does not disable the "fax these changes" message in Arcanist (no
reasonable way for it to read the config value) or the rainbow syntax
highlighter (already removable though configuration).

Reviewers: moskov, jungejason, nh, tuomaspelkonen, aran

Reviewed By: moskov

CC: aran, moskov

Differential Revision: 1081
2011-11-04 15:24:54 -07:00
Marek Sapota
789dc6cb5e Allow anonymus access to Differential.
Summary:
Add possibility for not logged in users to browse and see Differential
revisions.

Test Plan:
Set 'differential.anonymous-access' config option to true, log out, you should
be able to browse Differential without logging back in.

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: aran, epriestley, mareksapota

Differential Revision: 1044
2011-10-25 10:23:08 -07:00
epriestley
0669abc5f0 Use a proper entropy source to generate file keys
Summary:
See T549. Under configurations where files are served from an alternate domain
which does not have cookie credentials, we use random keys to prevent browsing,
similar to how Facebook relies on pseudorandom information in image URIs (we
could some day go farther than this and generate file sessions on the alternate
domain or something, I guess).

Currently, we generate these random keys in a roundabout manner. Instead, use a
real entropy source and store the key on the object. This reduces the number of
sha1() calls in the codebase as per T547.

Test Plan: Ran upgrade scripts, verified database was populated correctly.
Configured alternate file domain, uploaded file, verified secret generated and
worked properly. Changed secret, was given 404.

Reviewers: jungejason, benmathews, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley

Differential Revision: 1036
2011-10-23 14:42:23 -07:00
epriestley
4156cf6bd9 Add an optional configuration option to set 'Precedence: bulk' headers on
transactional mail

Summary: See T571. SES refuses to deliver mail with this header and there are
various reports of other issues on the internet so I'm defaulting it to off.

Test Plan: Set config to true, tried to send mail, SES rejected it because of
"Precedence: bulk" header.

Reviewers: bmaurer, ola, jungejason, nh, aran

Reviewed By: aran

CC: aran, epriestley, bmaurer

Differential Revision: 1032
2011-10-23 14:25:13 -07:00
epriestley
9a4bb3901e Allow bugs@ addresses to blanket-accept tasks
Summary: Allow configuration of a default author for bugs@ emails which don't
correspond to a known system user.

Test Plan: Configured a default author, sent some mails from nonsense addresses,
tasks were created.

Reviewers: davidreuss, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley, ide

Differential Revision: 1013
2011-10-20 14:26:19 -07:00
Marek Sapota
87a2987ad6 Differential mail
Test Plan: EMPTY

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: aran, epriestley, mareksapota

Differential Revision: 1004
2011-10-14 12:12:41 -07:00
epriestley
d625f94c55 Provide a markup protocol whitelist for Phabricator
Summary: See T548 and D996. Makes Phabricator configure the remarkup engine so
http:// and https:// get linked. Also make the "named link" syntax respect the
whitelist.

Test Plan:
  - Whitelisted URIs (they get linked).
  - Other URIs (not linked).
  - Whitelisted, named URIs (linked).
  - Other, named URIs (treated as phriction links).
  - Actual phriction links (work correctly).

Reviewers: jungejason, nh, tuomaspelkonen, aran, benmathews

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 997
2011-10-10 13:12:11 -07:00
epriestley
522e5b4779 Build an event dispatch mechanism into Phabricator
Summary:
This is an attempt to satisfy a lot of the one-off requests a little more
generally, by providing a relatively generic piece of event architecture.

Allow the registation of event listeners which can react to various application
events (currently, task editing).

I'll doc this a bit better but I wanted to see if anyone had massive objections
to doing this or the broad approach. The specific problem I want to address is
that one client wants to do a bunch of routing for tasks via email, so it's
either build a hook, or have them override most of ManiphestReplyHandler, or
something slightly more general like this.

Test Plan: Wrote a silly listener that adds "Quack!" to a task every time it is
edited and edited some tasks. I was justly rewarded.

Reviewers: nh, jungejason, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley

Differential Revision: 881
2011-09-30 12:16:40 -07:00
epriestley
d0b6602e29 Add an option to switch tokenizers to use "ondemand" instead of "preloaded"
datasources

Summary:
The open source Phabricator has like 3,500 user accounts now and it takes a
while to pull/render them. Add an option to switch to ondemand for large
installs.

I'll follow up with a patch at some point to address a couple of name things:

  - Denormalize last names into a keyed column (although this evidences some
bias toward the western world).
  - Force all usernames to lowercase (sorry Girish, Makinde).

Also this patch is so clean it's crazy.

Didn't bother with other object types for now, I'm planning to dedicate a few
days to Projects at some point and I'll flesh out some auxiliary features like
this when I do that.

Test Plan: Switched to ondemand, verified data was queried dynamically. Switched
back, verified data was preloaded.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: nh

CC: aran, epriestley, nh

Differential Revision: 923
2011-09-21 14:22:01 -07:00
epriestley
a42f116749 Allow "!accept" to be enabled through configuration
Summary: For reasons explained in the config I've omitted this from the default
action set, but it's trivial to support it. See D916.

Test Plan: Commented on a revision, was informed I could "!accept" in the email.
Used "!accept" to accept the revision.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 928
2011-09-14 09:52:13 -07:00
epriestley
1620bce842 Add Google as an OAuth2 provider (BETA)
Summary:
This is pretty straightforward, except:

  - We need to request read/write access to the address book to get the account
ID (which we MUST have) and real name, email and account name (which we'd like
to have). This is way more access than we should need, but there's apparently no
"get_loggedin_user_basic_information" type of call in the Google API suite (or,
at least, I couldn't find one).
  - We can't get the profile picture or profile URI since there's no Plus API
access and Google users don't have meaningful public pages otherwise.
  - Google doesn't save the fact that you've authorized the app, so every time
you want to login you need to reaffirm that you want to give us silly amounts of
access. Phabricator sessions are pretty long-duration though so this shouldn't
be a major issue.

Test Plan:
  - Registered, logged out, and logged in with Google.
  - Registered, logged out, and logged in with Facebook / Github to make sure I
didn't break anything.
  - Linked / unlinked Google accounts.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley, Makinde

Differential Revision: 916
2011-09-14 07:32:04 -07:00