1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-16 03:42:41 +01:00
Commit graph

428 commits

Author SHA1 Message Date
vrana
f5f7987013 Revert rP87c60abbd02d, apply D1772
Test Plan:
Apply SQL patch.
Visit /differential/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1781
2012-03-05 11:04:55 -08:00
Bob Trahan
0327a5fc69 OAuthServer polish and random sauce
Summary:
This diff makes the OAuthServer more compliant with the spec by
- making it return well-formatted error codes with error types from the spec.
- making it respect the "state" variable, which is a transparent variable the
client passes and the server passes back
- making it be super, duper compliant with respect to redirect uris
-- if specified in authorization step, check if its valid relative to the client
registered URI and if so save it
-- if specified in authorization step, check if its been specified in the access
step and error if it doesn't match or doesn't exist
-- note we don't make any use of it in the access step which seems strange but
hey, that's what the spec says!
This diff makes the OAuthServer suck less by
- making the "cancel" button do something in the user authorization flow
- making the client list view and client edit view be a bit more usable around
client secrets
- fixing a few bugs I managed to introduce along the way

Test Plan:
- create a test phabricator client, updated my conf, and then linked and
unlinked phabricator to itself
- wrote some tests for PhabricatorOAuthServer -- they pass!
-- these validate the various validate URI checks
- tried a few important authorization calls
--
http://phabricator.dev/oauthserver/auth/?client_id=X&state=test&redirect_uri=http://www.evil.com
--- verified error'd from mismatching redirect uri's
--- verified state parameter in response
--- verified did not redirect to client redirect uri
-- http://phabricator.dev/oauthserver/auth/?client_id=X w/ existing
authorization
--- got redirected to proper client url with error that response_type not
specified
-- http://phabricator.dev/oauthserver/auth/?client_id=X&response_type=code w/
existing authorization
--- got redirected to proper client url with pertinent code!
- tried a few important access calls
-- verified appropriate errors if missing any required parameters
-- verified good access code with appropriate other variables resulted in an
access token
- verified that if redirect_uri set correctly in authorization required for
access and errors if differs at all / only succeeds if exactly the same

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, ajtrichards

Maniphest Tasks: T889, T906, T897

Differential Revision: https://secure.phabricator.com/D1727
2012-03-01 14:46:18 -08:00
epriestley
8a0a00f118 Make PhabricatorRepositoryCommmit schema changes for audit
Summary:
  - Add a proper mailKey field to make these things mailable. Backfill all
existing objects.
  - Denormalize authorPHID to the commit object so we can query by it
efficiently in a future diff. We currently use the search engine to drive
"commits by author" but that's not so good for audit, which needs more
constraints.
  - Add an overall audit status field so we can efficiently query "commits that
needs your attention".
  - Add enough code to convince myself that these fields are basically
reasonable and work correctly.

Test Plan:
  - Ran schema upgrades. Checked database state afterward.
  - Ran "reparse.php --owners --herald" to verify worker changes.
  - Looked at a commit, altered aggregate status via audits / reparse.php,
verified it responded correctly.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, nh

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1706
2012-02-28 21:06:34 -08:00
Bob Trahan
3c4070a168 OAuth Server -- add controllers to RUD client authorizations and CRUD clients
Summary:
beyond the title, this diff tweaks the test console to have a bit more
functionality.  also makes a small change to CSS for AphrontFormControlMarkup,
which IMO fixes a display issue on
https://secure.phabricator.com/settings/page/profile/ where the Profile URI is
all up in the air and whatnot

I think this is missing pagination.  I am getting tired of the size though and
will add later.  See T905.

Test Plan:
viewed, updated and deleted client authorizations.  viewed, created,
updated and deleted clients

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T849, T850, T848

Differential Revision: https://secure.phabricator.com/D1683
2012-02-24 14:56:18 -08:00
Bob Trahan
af295e0b26 OAuth Server enhancements -- more complete access token response and groundwork
for scope

Summary:
this patch makes the access token response "complete" relative to spec by
returning when it expires AND that the token_type is in fact 'Bearer'.

This patch also lays the groundwork for scope by fixing the underlying data
model and adding the first scope checks for "offline_access" relative to expires
and the "whoami" method.   Further, conduit is augmented to open up individual
methods for access via OAuth generally to enable "whoami" access.   There's also
a tidy little scope class to keep track of all the various scopes we plan to
have as well as strings for display (T849 - work undone)

Somewhat of a hack but Conduit methods by default have SCOPE_NOT_ACCESSIBLE.  We
then don't even bother with the OAuth stuff within conduit if we're not supposed
to be accessing the method via Conduit.   Felt relatively clean to me in terms
of additional code complexity, etc.

Next up ends up being T848 (scope in OAuth) and T849 (let user's authorize
clients for specific scopes which kinds of needs T850).  There's also a bunch of
work that needs to be done to return the appropriate, well-formatted error
codes.  All in due time...!

Test Plan:
verified that an access_token with no scope doesn't let me see
anything anymore.  :(  verified that access_tokens made awhile ago expire.  :(

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T888, T848

Differential Revision: https://secure.phabricator.com/D1657
2012-02-21 16:33:06 -08:00
Bob Trahan
7a3f33b5c2 OAuth - Phabricator OAuth server and Phabricator client for new Phabricator OAuth Server
Summary:
adds a Phabricator OAuth server, which has three big commands:
 - auth - allows $user to authorize a given client or application.  if $user has already authorized, it hands an authoization code back to $redirect_uri
 - token - given a valid authorization code, this command returns an authorization token
 - whoami - Conduit.whoami, all nice and purdy relative to the oauth server.
Also has a "test" handler, which I used to create some test data.  T850 will
delete this as it adds the ability to create this data in the Phabricator
product.

This diff also adds the corresponding client in Phabricator for the Phabricator
OAuth Server.  (Note that clients are known as "providers" in the Phabricator
codebase but client makes more sense relative to the server nomenclature)

Also, related to make this work well
 - clean up the diagnostics page by variabilizing the provider-specific
information and extending the provider classes as appropriate.
 - augment Conduit.whoami for more full-featured OAuth support, at least where
the Phabricator client is concerned

What's missing here...   See T844, T848, T849, T850, and T852.

Test Plan:
- created a dummy client via the test handler.   setup development.conf to have
have proper variables for this dummy client.  went through authorization and
de-authorization flows
- viewed the diagnostics page for all known oauth providers and saw
provider-specific debugging information

Reviewers: epriestley

CC: aran, epriestley

Maniphest Tasks: T44, T797

Differential Revision: https://secure.phabricator.com/D1595
2012-02-19 14:00:13 -08:00
epriestley
7200040479 Add a basic chatlog
Summary:
This is pretty simple and unpolished, but it's getting pretty big and it seems
like a reasonable starting point.

  - Log chat in various "channels".
  - Conduit record and query methods.
  - IRCBot integration for IRC logging

Major TODO:

  - Web UI is really unpolished and has no search, paging, anchor-linking, etc.
Basically all presentation stuff, though.
  - I think the bot should have a map of channels to log with channel aliases?
  - The "channels" should probably be in a separate table.
  - The "authors" should probably be correlated to Phabricator accounts somehow,
where possible.

Test Plan: Used phabotlocal to log #phabricator.

Reviewers: kdeggelman, btrahan, Koolvin

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T837

Differential Revision: https://secure.phabricator.com/D1625
2012-02-17 10:21:38 -08:00
epriestley
d7bb1f183c Trim "\n" from mimeType field in phabricator_file.file
Summary: See patch comment, which explains the situation.

Test Plan: Ran upgrade, verified mimeType field is \n-free in database.

Reviewers: davidreuss, btrahan

Reviewed By: davidreuss

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1624
2012-02-16 07:25:56 -08:00
epriestley
35c5852d3f Add a safeguard against multiple patches with the same version
Summary:
I accidentally added two "104" patches. This actually works OK for the most part
but is fundamentally bad and wrong.

Merge the patches (installs applied both as "104", so we can't move one to
"105") and add a safeguard.

Test Plan: Ran upgrade_schema.php with two "104" patches, got error'd. Ran
without, got successs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1614
2012-02-14 16:24:02 -08:00
epriestley
4caa684724 Simplify Project status field
Summary:
This was a sort of speculative feature added by a contributor some time ago and
just serves as a label; for now, simplify it into "active" and "archived" and
remove "archived" projects from the "active" list.

  - Fix a bug where we'd publish a "renamed from X to X" transaction that had no
effect.
  - Publish stories about status changes.
  - Remove the "edit affiliation" controller, which has no links in the UI
(effectively replaced by join/leave links).
  - Add query/conduit support.

Test Plan: Edited the status of several projects.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T681

Differential Revision: https://secure.phabricator.com/D1573
2012-02-07 14:59:38 -08:00
epriestley
a5f8846f47 Use a unique random key to identify queries, not a sequential ID
Summary:
We save search information and then redirect to a "/search/<query_id>/" URI in
order to make search URIs short and bookmarkable, and save query data for
analysis/improvement of search results.

Currently, there's a vague object enumeration security issue with using
sequential IDs to identify searches, where non-admins can see searches other
users have performed. This isn't really too concerning but we lose nothing by
using random keys from a large ID space instead.

  - Drop 'authorPHID', which was unused anyway, so searches can not be
personally identified, even by admins.
  - Identify searches by random hash keys, not sequential IDs.
  - Map old queries' keys to their IDs so we don't break any existing bookmarked
URIs.

Test Plan: Ran several searches, got redirected to URIs with random hashes from
a large ID space rather than sequential integers.

Reviewers: arice, btrahan

Reviewed By: arice

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1587
2012-02-07 14:58:46 -08:00
awyler
56df2bc7be Add basic edit history to herald rules
Summary:
Add a very basic edit history table to herald rules.  This table is updated
whenever saving a herald rule.  The contents of the save are not examined, and
the edit history contains no information about the rule itself *yet*.  Edit
history can be viewed by anyone through /herald/history/<rule id>/.

Task ID: #

Blame Rev:

Test Plan:
Made a test rule, saved some stuff.

Revert Plan:

Tags:

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: zizzy, aran, xela, epriestley

Differential Revision: https://secure.phabricator.com/D1387
2012-01-30 11:52:44 -08:00
epriestley
5b463e634c Write fewer "applied" rows and clean up excess historical rows
Summary:
  - Only write the <ruleID, phid> row if the rule is a one-time rule.
  - Delete all the rows for rules which aren't one-time.

NOTE: This is probably like several million rows for Facebook and could take a
while.

Test Plan:
Added some one-time and every-time rules, ran them against objects, verified
only relevant rows were inserted.
Ran upgrade script against a database with one-time and every-time "ruleapplied"
rows, got the irrelevant rows removed.

Reviewers: nh, btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1484
2012-01-25 11:53:39 -08:00
epriestley
3142fe4419 Remove massive "rule applied" query
Summary:
Herald rules may be marked as "one-time". We track this by writing a row with
<ruleID, phid> when we apply a rule.

However, the current test for rule application involves loading every <ruleID,
*> pair. We also always write this row even for rules which are not one-time, so
if there are 100 rules, we'll load 1,000,000 rows after processing 10,000
objects.

Instead, load only the <phid, *> pairs, which are guaranteed to be bounded to at
most the number of rules.

I'll follow up with a diff that causes us to write rows only for one-time rules,
and deletes all historic rows which are not associated with one-time rules.

Test Plan:
Grepped for callsites to loadAllByContentTypeWithFullData(). Ran
rules in test console.

Reviewers: nh, btrahan, jungejason

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1483
2012-01-24 19:29:54 -08:00
epriestley
b43eb5aa7c Add transaction-oriented editing to projects
Summary:
  - Make some editing operations transaction-oriented, like Maniphest. (This
seems to be a good model, particularly for extensibility.) I'll move the rest of
the editing operations to transactions in future diffs.
  - Make transaction-oriented operations publish feed stories.

Test Plan:
  - Created a new project.
  - Edited an existing project.
  - Created a new project via quick create flow from Maniphest.
  - Verified feed stories publish correctly.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T681

Differential Revision: https://secure.phabricator.com/D1477
2012-01-24 09:44:35 -08:00
epriestley
d1ee08b2df Drydock Rough Cut
Summary:
Rough cut of Drydock. This is very basic and doesn't do much of use yet (it
//does// allocate EC2 machines as host resources and expose interfaces to them),
but I think the overall structure is more or less reasonable.

== Interfaces

Vision: Applications interact with Drydock resources through DrydockInterfaces,
like **command**, **filesystem** and **httpd** interfaces. Each interface allows
applications to perform some kind of operation on the resource, like executing
commands, reading/writing files, or configuring a web server. Interfaces have a
concrete, specific API:

  // Filesystem Interface
  $fs = $lease->getInterface('filesystem'); // Constants, some day?
  $fs->writeFile('index.html', 'hello world!');

  // Command Interface
  $cmd = $lease->getInterface('command');
  echo $cmd->execx('uptime');

  // HTTPD Interface
  $httpd = $lease->getInterface('httpd');
  $httpd->restart();

Interfaces are mostly just stock, although installs might add new interfaces if
they expose different ways to interact with resources (for instance, a resource
might want to expose a new 'MongoDB' interface or whatever).

Currently: We have like part of a command interface.

== Leases

Vision: Leases keep track of which resources are in use, and what they're being
used for. They allow us to know when we need to allocate more resources (too
many sandcastles on the existing hosts, e.g.) and when we can release resources
(because they are no longer being used). They also give applications something
to hold while resources are being allocated.

  // EXAMPLE: How this should work some day.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('sandcastle');
  $allocator->setAttributes(
    array(
      'diffID' => $diff->getID(),
    ));
  $lease = $allocator->allocate();
  $diff->setSandcastleLeaseID($lease->getID());

  // ...

  if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) {
    $sandcastle_link = $lease->getInterface('httpd')->getURI('/');
  } else {
    $sandcastle_link = 'Still building your sandcastle...';
  }
  echo "Sandcastle for this diff: ".$sandcastle_link;

  // EXAMPLE: How this actually works now.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('host');
  // NOTE: Allocation is currently synchronous but will be task-driven soon.
  $lease = $allocator->allocate();

Leases are completely stock, installs will not define new lease types.

Currently: Leases exist and work but are very very basic.

== Resources

Vision: Resources represent some actual thing we've put somewhere, whether it's
a host, a block of storage, a webroot, or whatever else. Applications interact
through resources by acquiring leases to them, and then getting interfaces
through these leases. The lease acquisition process has a side effect of
allocating new resources if a lease can't be acquired on existing resources
(e.g., the application wants storage but all storage resources are full) and
things are configured to autoscale.

Resources may themselves acquire leases in order to allocate. For instance, a
storage resource might first acquire a lease to a host resource. A 'test
scaffold' resource might lease a storage resource and a mysql resource.

Not all resources are auto-allocate: the entry-level version of Drydock is that
you manually allocate a couple boxes and configure them through the web console.
Then, e.g.,  'storage' / 'webroot' resources allocate on top of them, but the
host pool itself does not autoscale.

Resources are completely stock, they are abstract shells representing any
arbitrary thing.

Currently: Resource exist ('host' only) but are very very basic.

== Blueprints

Vision: Blueprints contain instructions for building interfaces to, (possibly)
allocating, updating, managing, and destroying a specific type of resource in a
specific location. One way to think of them is that they are scripts for
creating and deleting resources. For example, the LocalHost, RemoteHost and
EC2Host blueprints can all manage 'host' resources.

Eventually, we will support more types of resources (storage, webroot,
sandcastle, test scaffold, phacility deployment) and more providers for resource
types, some of which will be in the Phabricator mainline and some of which will
be custom.

Blueprints are very custom and specific to application types, so installs will
define new blueprints if they are making significant use of Drydock.

Currently: They exist but have few capabilities. The stock blueprints do nearly
nothing useful. There is a technically functional blueprint for host allocation
in EC2.

== Allocator

This is just the actual code to execute the lease acquisition process.

Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2,
acquired a lease on it, interfaced with it, and then released the lease. Ran it
again, got a fresh lease on the existing resource.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1454
2012-01-19 21:12:57 -08:00
awyler
6080d74112 Created personal vs. global herald rule distingtion
Summary:
A personal rule only has actions targeting the owner.  Likewise, only they can
edit the rule. OTOH, a global may affect any target and is editable by anyone.

There are no new action types.  Instead, type of the rule modifies the available
targets and the messaging in the ui.  This is beneficial because herald rule
adapters don't need to be aware of the difference between emailing the owner of
a personal rule and emailing an arbitrary user.

This diff sets up the logic and ui for creating personal/global rules.  All
existing rules have been defaulted to global.

TODO: Filter all existing rules into personal/global
TODO: Create a UI for surfacing (relevant?) global rules.

Test Plan:
1. Created a personal rule to email myself.  Created a dumby revision satisfying
the conditions of that rule.  Verified that I recieved a herald email.
2. Removed my adminship, change the owner of a personal rule. verified that I
couldn't edit the rule.
3.Changed rule type to global. verified that I could edit the rule.
4. Verified that admins can edit both global and personal rules.

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: aran, zizzy

Differential Revision: https://secure.phabricator.com/D1449
2012-01-19 11:21:49 -08:00
vrana
6472dbe168 Change fileName to filename
Summary: There are lots of callsites to $changeset->getFilename() so it seemed
easier to rename getFileName() to getFilename() even if it includes database
change. Plus I think that getFilename() is better.

Test Plan:
Alter database.
Open revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D1437
2012-01-17 10:50:14 -08:00
vrana
59380582d8 Delete /xhprof/ from directory, mark /mail/ as Admin Only
Summary: /xhprof/ doesn't exist at all, /mail/ is only for admins

Test Plan: Display directory

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, vrana

Differential Revision: https://secure.phabricator.com/D1405
2012-01-15 20:27:55 -08:00
epriestley
95cde81daf Minor, reduce slug column size to 128, see D1391. 2012-01-13 17:16:37 -08:00
epriestley
13bf353f14 Shorten Phriction slug field to 256 to avoid key length issues in UTF-8 default
databases

Summary: This is a stopgap patch until we fix T345/T327 properly, but you can't
add a 512-character key to a UTF-8 column because it creates an index on more
than 767 bytes, which is the key length limit.

Test Plan: Ran schmea upgrade script.

Reviewers: indiefan, btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, xela, jungejason

Maniphest Tasks: T327

Differential Revision: https://secure.phabricator.com/D1391
2012-01-13 15:25:18 -08:00
epriestley
efb0fa739f Make tracked git repositories use an implicit 'origin' remote
Summary:
See T624. I originally wrote this to require an explicit remote, but this
creates an ugly "origin:" in all the URIs and makes T270 more difficult.

Treat all branch names as implying 'origin/'.

Test Plan:
  - Pulled and imported a fresh copy of libphutil without issues.
  - Browsed various git repositories.
  - Browsed Javelin's various branches.
  - Ran upgrade script, got a bunch of clean 'origin/master' -> 'master'
conversions.
  - Tried to specify an explicit remote in a default branch name.
  - Unit tests.

Reviewers: nh, jungejason, btrahan

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T624

Differential Revision: https://secure.phabricator.com/D1269
2011-12-29 08:35:32 -08:00
epriestley
f30f4fa713 Issue CREATE DATABASE before ALTER TABLE in patch 88
Summary:
Sometimes the SQL account may not have CREATE DATABASE privileges, so try to
fail this script immediately rather than having it work halfway if the
permission is missing.

This is just a minor tweak to try to slightly improve some cases, the long term
fix is T345 but that's a good deal more work than this.

Test Plan: Visually inspected patch; trivial change.

Reviewers: jungejason, btrahan, skrul

Reviewed By: skrul

CC: aran, skrul

Differential Revision: https://secure.phabricator.com/D1287
2011-12-27 17:50:23 -08:00
epriestley
bdbe9df65e Remove support for GitHub post-receive notifications
Summary:
  - These never actually did anything.
  - I don't even really remember why I built them, maybe the Open Source team
was pushing for more GitHub integration or something? I really have no idea.
  - Anyway, repository tailers do everything these could do (and much more).

Test Plan:
  - Ran tailers off GitHub for many months without needing post-receive hooks.
  - Grepped for relevant strings, couldn't find any references.
  - Used "Repository" edit interface for a Git repository.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, jungejason

Maniphest Tasks: T706

Differential Revision: https://secure.phabricator.com/D1273
2011-12-24 09:00:08 -08:00
epriestley
21ba07d5bd Provide wiki pages for projects
Summary:
Provide tighter integration between Projects and Phriction. Partly, I have most
of a rewrite for the Projects homepage ready but it's not currently possible to
publish feed stories about a project so all the feeds are empty/boring. This
partly makes them more useful and partly just provides a tool integration point.

  - When you create a project, all the wiki pages in projects/<project_name>/*
are associated with it.
  - Publish updates to those pages as being related to the project so they'll
show up in project feeds.
  - Show a project link on those pages.

This is very "convention over configuration" but I think it's the right
approach. We could provide some sort of, like, "@project=derp" tag to let you
associated arbitrary pages to projects later, but just letting you move pages is
probably far better.

Test Plan:
  - Ran upgrade scripts against stupidly named projects ("der", "  der", "  der
", "der (2)", "  der (2) (2)", etc). Ended up with uniquely named projects.
  - Ran unit tests.
  - Created /projects/ wiki documents and made sure they displayed correctly.
  - Verified feed stories publish as project-related.
  - Edited projects, including perfomring a name-colliding edit.
  - Created projects, including performing a name-colliding create.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley, btrahan

Maniphest Tasks: T681

Differential Revision: 1231
2011-12-20 14:03:12 -08:00
jungejason
c80d1480d5 Add Basic Auditing Functionalities
Summary:
add basic auditing functionalities. For the related commits for a
package, we detect the following conditions which might be suspicious to the
owners of the package:

* no revision specified
* revision not found
* author not match
* reviewedby not match
* owners not involved
* commit author not recognized

The owners of the package can change the status of the audit entries by
accepting it or specify concern.

The owner can turn on/off the auditing for a package.

Test Plan:
*  verified that non-owner cannot see the details of the audit and cannot modify
it
*  verified that all the audit reasons can be detected
*  tested dropdown filtering and package search
*  verified really normal change not detected
*  verified accept/concern a commit
*  tested enable/disable a package for auditing
*  verified one audit applies to all <commit, packages> to the packages the
auditor owns
*  verified that re-parsing a commit won't have effect if there exists a
 relationship for <commit, package> already

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, benmathews, btrahan, mpodobnik, prithvi, TomL, epriestley

Differential Revision: 1242
2011-12-20 13:36:53 -08:00
epriestley
afc2f8526b Allow Phriction documents to be deleted
Summary:
  - Add a "delete" operation. Delete is just a special edit which removes the
page from indexes and shows a notice that the document has been deleted.
  - When a user deletes all the content on a page, treat it as a delete.
  - When a conduit call deletes all the content on a page, treat it as a delete.
  - Add page status to Conduit.
  - Add change type field to history.
  - Added a couple of constants to support a future 'move' change, which would
move content from one document to another.

Test Plan:
  - Verified deleted pages vanish from the document index (and restoring them
puts them back).
  - Verified deleted pages show "This page has been deleted...".
  - Created, edited and deleted a document via Conduit.
  - Deleted pages via "delete" button.
  - Deleted pages via editing content to nothing.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: skrul, aran, btrahan, epriestley

Maniphest Tasks: T680

Differential Revision: 1230
2011-12-17 11:45:25 -08:00
epriestley
93d5d29541 Remove "Former" project members
Summary:
This is a needlessly confusing/complex feature that I originally wrote sort of
speculativley. I think we can better serve what little need may exist here with
project feeds.

I'm probably going to get rid of or deemphasize "role" too and just add "Join
Project" and "Leave Project" buttons.

Test Plan: Viewed project list, project profile. Edited project profile and
affiliation.

Reviewers: btrahan, jungejason, zeeg

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T681

Differential Revision: 1228
2011-12-16 17:46:02 -08:00
jungejason
c13b7da290 Add Related Commits for Owners
Summary:
For each commit, find the affected packages, and provide a way to
search by package.

Test Plan:
create commits that touch and don't touch two packages, and verify
that they display correctly in all the UI pages.

Reviewers: epriestley, blair, nh, tuomaspelkonen

Reviewed By: epriestley

CC: benmathews, aran, epriestley, btrahan, jungejason, mpodobnik, prithvi

Maniphest Tasks: T83

Differential Revision: 1208
2011-12-14 22:48:57 -08:00
Bob Trahan
d739d8ccd5 Paste - upgrade scheme to support queries by authorPHID
Test Plan: ran upgrade_schema.php on my local install.   re-examined SQL query
"explain select * from phabricator_pastebin.pastebin_paste where authorPHID =
"PHID-USER-xee4ju2teq7mflitwfcs" ORDER BY id DESC;" on my local install and
noted fewer rows scanned, index used, etc

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1218
2011-12-14 19:48:47 -08:00
epriestley
4fd81150be Remove "Updated" view from Differential
Summary:
This landed during my review drama embargo and is a generally good idea but had
some implementation issues.

@elynde reports it has been broken for some time, although it still works on
secure.phabricator.com so I'm guessing it's just taking a zillion years to run
at Facebook. It's up to more than a second for me on secure.phabricator.com:

https://secure.phabricator.com/file/view/PHID-FILE-v4ql4c66u3xnkarmrpm4/

The basic problem is that some of the data architecture around this
implementation is hard to scale. I want to pursue a similar feature eventually,
but drive it off notifications that we'll ship through real-time infrastructure
too.

I'm also trying to get rid of DifferentialRevisionListData and this simplifies
that somewhat.

Test Plan:
  - Grepped for table name, table constant, query constant, and class name; no
hits.
  - Applied SQL patch.
  - Verified that Differential no longer shows "Updated".

Reviewers: elynde, btrahan, jungejason

Reviewed By: elynde

CC: aran, elynde

Differential Revision: 1178
2011-12-07 06:55:03 -08:00
epriestley
720af267a7 Add a missing key to the ManiphestTransaction table
Summary:
Caught a bad (8ms) query that didn't have a good key available. Add an
appropriate key.

The query in question is:

   SELECT * FROM `maniphest_transaction` WHERE taskID = 262 ORDER BY id ASC;

...which is used in the task detail view.

Test Plan: Ran EXPLAIN on this query before/after upgrading, it now uses the
newly available key.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1037
2011-10-23 14:43:03 -07:00
epriestley
0669abc5f0 Use a proper entropy source to generate file keys
Summary:
See T549. Under configurations where files are served from an alternate domain
which does not have cookie credentials, we use random keys to prevent browsing,
similar to how Facebook relies on pseudorandom information in image URIs (we
could some day go farther than this and generate file sessions on the alternate
domain or something, I guess).

Currently, we generate these random keys in a roundabout manner. Instead, use a
real entropy source and store the key on the object. This reduces the number of
sha1() calls in the codebase as per T547.

Test Plan: Ran upgrade scripts, verified database was populated correctly.
Configured alternate file domain, uploaded file, verified secret generated and
worked properly. Changed secret, was given 404.

Reviewers: jungejason, benmathews, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley

Differential Revision: 1036
2011-10-23 14:42:23 -07:00
epriestley
ddce177d81 Add a name token table so on-demand typeaheads can match last names
Summary: See T585. We currently don't match middle/last/nth names in on-demand
tokenizers. Build a table so we can match them.

Test Plan:
Ran upgrade script, verified table looks sensible. Searched for "priestley" in a
tokenizer, got a bunch of test account hits.

  mysql> select * from user_nametoken;
  +-------------------+--------+
  | token             | userID |
  +-------------------+--------+
  | evan              |      1 |
  | priestley         |      1 |
  | epriestley        |      1 |
  | epriestley2       |      2 |
  | ducks             |      4 |
  | epriestley3       |      4 |
  | asdf              |      6 |
  | epriestley99      |      6 |
  ...

Reviewers: bh, nh, jungejason, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1034
2011-10-23 14:25:26 -07:00
epriestley
9a4bb3901e Allow bugs@ addresses to blanket-accept tasks
Summary: Allow configuration of a default author for bugs@ emails which don't
correspond to a known system user.

Test Plan: Configured a default author, sent some mails from nonsense addresses,
tasks were created.

Reviewers: davidreuss, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley, ide

Differential Revision: 1013
2011-10-20 14:26:19 -07:00
Marek Sapota
cb0700a99c Allow databases to be created outside upgrade_schema scrpt.
Summary:
Current scripts make it hard to administer Phabricator instance while not having
direct (priviledged) access to the database.  This change allows scenario where
DB administrator creates the databases for you before you run update_schema
script.

Test Plan:
Create the databases before running the update_schema script - it shouldn't
complain that the databases already exist.

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1014
2011-10-17 13:59:56 -07:00
epriestley
254f606e89 Tie all the pieces for symbol cross-references together
Summary:
This makes symbol cross-references work in Differential. You need to do a little
legwork but I'll document that once the change has baked for a little while.

Basically:

  - Projects are annotated with indexed languages, and "shared library" projects
(for example, symbols in Phabricator should be searched for in Arcanist and
libphutil).
  - When we render a changeset, we check if its language is an indexed one. If
it is, we invoke the decorator Javascript.
  - The Javascript takes you to a lookup page, which either gives you a list of
matching symbols (if several match) or redirects you instantly to the
definition.

Test Plan: Clicked class and function symbols in a diff, got jumped into
sensible sorts of places in Diffusion.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 980
2011-10-09 17:58:17 -07:00
epriestley
016b060aea Add a relation table for Revisions to local commit hashes
Summary:
This allows us to performantly query for diffs related to a given local hash.
Immediate applications are:

  - Commit detection in Mercurial and Git-Immutable workflows.
  - Some async unit test stuff @mgummelt was doing.

Test Plan:
Diffed locally under SVN/Git/hg, checked the table, got sensible output.

  mysql> select * from differential_revisionhash;
  +------------+------+------------------------------------------+
  | revisionID | type | hash                                     |
  +------------+------+------------------------------------------+
  |         40 | gtcm | 8c6fb2f95598a50f7aac64a5f4cc6c12b5db42f5 |
  |         40 | gttr | 54710e361a465f4ff39565a93b2a221b6e7dd07c |
  |         41 | hgcm | c29cb69aec14                             |
  |         41 | hgcm | e7309be4eabb                             |
  |         41 | hgcm | 4e885caeff60                             |
  |         41 | hgcm | 213ee1cd30ea                             |
  |         41 | hgcm | b4050fb3490f                             |
  |         41 | hgcm | 72a76bd7ffa2                             |
  |         41 | hgcm | 06c2687e63fb                             |
  |         41 | hgcm | 2b464bde6b48                             |
  +------------+------+------------------------------------------+
  10 rows in set (0.00 sec)

NOTE: Mercurial hashes are short-form but I'll shoot out a separate Arcanist
diff to fix this.

Reviewers: Makinde, fratrik, mgummelt, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 961
2011-09-26 15:02:37 -07:00
epriestley
43a3f4d234 Build an "affected path" index when attaching diffs to revisions
Summary: See T262. This creates the index on the Differential side which we need in order to execute this query efficiently on the Diffusion side.

Also renames "DiffusionGitPathIDQuery" to "DiffusionPathIDQuery", this query object has nothing to do with git.

Test Plan: Attached top-level and sub-level diffs to revisions and verified they populated the table with sensible data.

Reviewers: bmaurer, aravindn, fmoo, jungejason, nh, tuomaspelkonen, aran

CC:

Differential Revision: 931
2011-09-15 07:45:14 -07:00
epriestley
cd05c960ff Add storage for repository symbol tracking
Summary: See T315 for an extensive description of this feature. Adds the
descibed storage table.

Test Plan: Used phpsh to read/write symbol objects.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: tuomaspelkonen

CC: aran, epriestley, tuomaspelkonen

Differential Revision: 897
2011-09-13 08:49:44 -07:00
epriestley
e875c81f6d Remove blameRevision and revertPlan from the DifferentialRevision schema
Summary:
These fields use auxiliary storage now. Migrate the data and get rid of the
columns in the main table.

  - This might take a little while to run, although there are <500k rows so
probably not too long.
  - Maybe grab a backup of the table first, if I screwed something up this will
delete the data in these fields.

Test Plan:
  - Ran migration locally.
  - Browsed Differential.
  - Grepped for "revertPlan" and "blameRevision".

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason, epriestley

Differential Revision: 832
2011-09-04 16:19:12 -07:00
epriestley
69445222f7 Track content sources (email, web, conduit, mobile) for replies
Summary:
When an object is updated, record the content source for the update. This mostly
isn't terribly useful but one concrete thing I want to do with it is let admins
audit via-email replies more easily since there are a bunch of options which let
you do hyjinx if you intentionally configure them insecurely. I think having a
little more auditability around this feature is generally good. At some point
I'm going to turn this into a link admins can click to see details.

It also allows us to see how frequently different mechanisms are used, and lets
you see if someone is at their desk or on a mobile or whatever, at least
indirectly.

The "tablet" and "mobile" sources are currently unused but I figured I'd throw
them in anyway. SMS support should definitely happen at some point.

Not 100% sure about the design for this, I might change it to plain text at some
point.

Test Plan: Updated objects and saw update sources rendered.

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, epriestley, jungejason

Differential Revision: 844
2011-08-30 11:08:27 -07:00
epriestley
dd74903cae Add basic auxiliary field storage for Differential
Summary:
Precursor to building this out to solve T343. This is similar to the Maniphest
fields we landed recently, although I think they're dissimilar enough that it
isn't worth going crazy trying to make them share code, at least for now.

This doesn't really do anything yet, just adds a storage object and a couple of
selector/field indirection classes.

Test Plan: Ran SQL upgrade script, created an aux field.
Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 798
2011-08-14 10:04:21 -07:00
epriestley
879431fb50 Improve GC performance for Herald Transcripts
Summary: This has to table scan a ginormous table right now, give it a fighting
chance with a more usable key.
Test Plan:
  - Launched GC daemon, no errors.
  - Used test console to create a new transcript.
  - Viewed some old transcripts.
  - Ran EXPLAIN on the SELECT and verified it was utilizing the garbageCollected
key.

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, epriestley, jungejason
Differential Revision: 735
2011-07-28 18:50:54 -07:00
hunterbridges
aeae33b7d6 Key Value Store for ManiphestTask
Test Plan: Look at a task detail. Some dummy attributes are automatically added in ManiphestTaskDetailController.
Reviewed By: epriestley

Differential Revision: 730
2011-07-25 19:11:55 -07:00
epriestley
6e08a9215d Move "Preferences" to "Settings"
Summary:
It makes more sense to just make this a settings panel rather than a standalone
app, particularly since setting panels are relatively well separated now.

Also default-disabled the SSH Keys interface since it won't currently be useful
for most installs.

Test Plan: Edited preferences.
Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason, aran
CC: aran, jungejason
Differential Revision: 716
2011-07-24 12:25:43 -07:00
Hua Wang
2a894438ae Add a 'description' field to Phriction
Summary:
Add a new column to PhrictionContent called 'comment' or 'description' or
something
        Add an optional field to the Phriction document editing interface that
allows you to add a comment

Test Plan:
Run the sql patch to modify phriction_content table
           Create a new wiki page in Phriction
           Type in words in description field and save the page
           Visit this page and click "Edit Page" button
           The content in the desciption field is saved

Reviewed By: epriestley
Reviewers: epriestley, hsb, codeblock
Commenters: codeblock
CC: aran, codeblock, hwang, epriestley
Differential Revision: 709
2011-07-23 21:11:42 -07:00
epriestley
8df62d5352 Allow users to associate SSH Public Keys with their accounts
Summary:
With the sshd-vcs thing I hacked together, this will enable Phabricator to host
repositories without requiring users to have SSH accounts.

I also fixed "subporjects" and added an explicit ENGINE to it.

Test Plan: Created, edited and deleted public keys. Attempted to add the same
public key twice. Attempted to add invalid and unnamed public keys.
Reviewed By: aran
Reviewers: jungejason, tuomaspelkonen, aran, cadamo, codeblock
CC: aran, epriestley
Differential Revision: 711
2011-07-23 09:15:20 -07:00
Cristian Adamo
8cca3079fc Add subprojects to a main project.
Summary:
Quite basic subproject attachment.

Test Plan:
Go to some proj
ect and add another as a subproject.
Play around the ui.

Reviewers:	epr
iestley
CC:
2011-07-19 15:50:15 -03:00
Ricky Elrod
95d6e0a3c4 Store parents of forked pastes, and list child pastes if there are any.
Summary:
Added a 'parent' field which stores a PHID of another paste. If it is not NULL show a list of children pastes on view.
Also did some misc. refactoring to clean up the code a bit, specifically in the Create controller.

Test Plan:
- Checked old pastes, they were not affected.
- Added a paste, successfully.
- Forked it, successfully.
- Went to the original paste, saw the child paste listed.
- Forked it again, saw the new one added to the list.

Reviewers:
epriestley

CC:

Differential Revision: 672
2011-07-15 18:42:08 -04:00
epriestley
4c44c44964 Add Phriction to the main nav menu
Summary:
A few minor things:

  - Add Phriction to the homepage.
  - Link to remarkup reference on the edit screen.
  - Add a help tab with a super basic help document.

Test Plan: Cursory inspection.
Reviewed By: codeblock
Reviewers: hsb, codeblock, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, codeblock
Differential Revision: 655
2011-07-12 09:26:51 -07:00
epriestley
5704b2bc70 Basic edit/create workflow for Phriction
Summary:
This is another chunk of D636, I just simplified it a bit and added slugs.

When you go to a page like /w/pokemon/, it allows you to create or edit the
page.

Title vs slug stuff is a little funky but I think mostly-reasonable.

Test Plan: Created and edited /w/, /w/pokemon/, etc.
Reviewed By: hsb
Reviewers: hsb, codeblock, jungejason, aran, tuomaspelkonen
CC: aran, hsb
Differential Revision: 643
2011-07-11 14:47:33 -07:00
epriestley
f95913ec47 Phriction basics
Summary:
Basically a copy/paste of parts of D636, but with two changes:

  - Fully separate the index table ("document") from the content table
("content"). I think this will be a cleaner solution in the long run.
  - Build slugs into the document structure.

This doesn't do anything useful, it just normalizes slugs and lays some
groundwork.

Test Plan:
  - Visited various /w/ pages and saw them normalize correctly.
  - Verified the DAO works by inserting dummy rows.

Reviewed By: codeblock
Reviewers: hsb, codeblock, jungejason, aran, tuomaspelkonen
CC: aran, codeblock, epriestley
Differential Revision: 638
2011-07-11 11:42:50 -07:00
epriestley
467308dd12 Script to selectively convert MyISAM tables to InnoDB
Summary:
A bunch of the .sql patch files don't explicitly specify the table engine, but
we should always use InnoDB with the exception of one table which needs MyISAM
for FULLTEXT.

MySQL doesn't no-op an ALTER TABLE statment that changes the engine back to
itself and converting large tables can be time consuming, so convert only the
required tables.

Test Plan: Ran on secure.phabricator.com and my local box, it fixed all the
issues in about 3 seconds on secure.phabricator.com and <<1 second on my local.
Reviewed By: codeblock
Reviewers: codeblock, tuomaspelkonen, jungejason, aran
CC: aran, epriestley, codeblock
Differential Revision: 641
2011-07-11 11:42:28 -07:00
epriestley
233a2055f1 Add missing keys to some tables
Summary: I wrote a little query plan plugin for "Services", it's not ready yet
but it found a few cases where we're doing table scans.
Test Plan:
https://secure.phabricator.com/file/view/PHID-FILE-374fd7e846a0ce27d5fb/
https://secure.phabricator.com/file/view/PHID-FILE-2f680de9b8c3c3b7089e/
https://secure.phabricator.com/file/view/PHID-FILE-742f645fcf011252ff25/

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 627
2011-07-09 10:55:15 -07:00
epriestley
51c2726a34 Add Differential parse cache to the GC daemon
Summary:
Add the differential parse cache to the GC. This is the largest object in the
system by a wide margin, I think.

This table is potentially gigantic which is why the script truncates it before
doing a schema change.

Test Plan: Ran the GC daemon, it cleaned up some parse caches.
Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason, aran
Commenters: tuomaspelkonen
CC: aran, jungejason, tuomaspelkonen, epriestley
Differential Revision: 620
2011-07-08 17:31:25 -07:00
epriestley
57097c2874 Port the "Slowvote" application
Summary: Port slowvote. This has some style/layout roughness but gets us most of
the way there. I'll followup to fix some of the markup issues.
Test Plan: Created and voted in several different kinds of poll.
Reviewed By: codeblock
Reviewers: codeblock, tomo, jungejason, aran, tuomaspelkonen
Commenters: aran, jungejason
CC: aran, codeblock, jungejason, epriestley
Differential Revision: 613
2011-07-08 12:38:53 -07:00
Ricky Elrod
420235f9c4 Drag-drop file upload.
Summary:
- have files be uploaded by drag+drop instead of browse.
- Files are named by their uploaded filename, the user isn't given a chance to enter a file name. Is this bad?
- Store author PHID now with files
- Allow an ?author=<username> to limit the /files/ list by author.
- If one file is uploaded, the user is taken to its info page.
- If several are uploaded, they are taken to a list of their files.

Test Plan:
- Quickly tested everything and it still worked, I'd recommend some people try this out before it gets committed though. It's a rather huge revision.

Reviewers:
epriestley, Ttech

CC:

Differential Revision: 612
2011-07-08 15:20:57 -04:00
tuomaspelkonen
49310391e0 Added subscriber view to Maniphest.
Summary:
People want to see all the tasks they have subscribed to in one
view. A new table was added for this to make queries faster.

Test Plan:
Tested that the view was initially empty. After running the
reindex_maniphest.php script, I saw the correct tasks there. Added
myself as a subscriber to one task and made sure the view was
updated. Removed myself as a subscriber from one task and made sure
the view was updated again.

Reviewed By: epriestley
Reviewers: epriestley, jungejason, codeblock
CC: aran, rm, epriestley
Differential Revision: 603
2011-07-07 14:08:52 -07:00
epriestley
ece9d792b2 Build basic infrastructure for an activity feed
Summary: This defines an extremely basic version of an activity feed, like
Facebook's news feed. It doesn't do much of interest yet.
Test Plan: Published some feed stories:
https://secure.phabricator.com/file/view/PHID-FILE-5061aa72105bbdc05b21/
Reviewed By: tuomaspelkonen
Reviewers: jungejason, tuomaspelkonen, aran
Commenters: codeblock, jungejason
CC: aran, epriestley, codeblock, tuomaspelkonen, jungejason
Differential Revision: 593
2011-07-06 16:19:29 -07:00
Ricky Elrod
9454060c29 Add a syntax highlight dropdown, if pygments is enabled.
Summary:
- Add a default list of supported languages to default.conf.php
  and make the initial/default value customizable.
- Store a '' in the database to infer the language from the filename/title.

Test Plan:
Tested in my sandbox with pygments enabled and disabled and various
combinations of filename/extension/dropdown selection.

Reviewers:
epriestley

CC:

Differential Revision: 587
2011-07-04 12:23:43 -04:00
epriestley
de0c89261e Allow Maniphest tasks to be filtered by Project
Summary:
Major things taking place here:

  - A new table for storing <task, project> relationships.
  - Moved all task query logic into a dedicated class.
  - Added a "projects" filter to the UI.

I was originally going to try to drive this off the main search index but the
perf benefits of a custom schema make an overwhelming argument in favor of doing
it this way.

Test Plan:
Filtered tasks by author and owner and zero, one, and more than one project.
Exercised all the group/sort options. Ran the index script over my 100k task
corpus. Edited task-project membership and verified the index updated.

Reviewed By: cadamo
Reviewers: gc3, jungejason, cadamo, tuomaspelkonen, aran
CC: aran, cadamo, epriestley
Differential Revision: 556
2011-06-29 21:56:47 -07:00
epriestley
4469ef8f30 Allow Maniphest to scale to a massive size
Summary:
Maniphest is missing some keys and some query strategy which will make it
cumbersome to manage more than a few tens of thousands of tasks.

Test Plan:
Handily manipulated 100k-scale task groups. Maniphest takes about 250ms to
select and render pages of 1,000 tasks and has no problem paging and filtering
them, etc. We should be good to scale to multiple millions of tasks with these
changes.

Reviewed By: gc3
Reviewers: fratrik, jungejason, aran, tuomaspelkonen, gc3
Commenters: jungejason
CC: anjali, aran, epriestley, gc3, jungejason
Differential Revision: 534
2011-06-28 06:41:05 -07:00
epriestley
e0e6ec9117 Allow affiliations to carry project ownership information; transform profile
images correctly

Summary:
This is sort of doing two things at once:

  - Add an "isOwner" flag to Project Affiliation to lay the groundwork for T237.
  - Rename the "QuickCreate" workflow to "Create" and funnel all creation
through it.
  - Reorganize the image transformation stuff and use it to correctly
crop/resize uploaded images.

Test Plan:
Created and edited projects and affailiations. Uploaded project, user, and
profile photos. Verified existing thumbnailing in Maniphest still works
properly.

Reviewed By: cadamo
Reviewers: cadamo, aran, jungejason, tuomaspelkonen
CC: aran, epriestley, cadamo
Differential Revision: 529
2011-06-28 06:40:41 -07:00
epriestley
46a7676a6a Properly scope some SQL. 2011-06-21 14:46:59 -07:00
epriestley
431c57688e Improve performance of project list view
Summary:
D477 added functionality to the project list view but had a couple of
performance issues that I missed in review, because it took the query count for
the page from around 3 to as many as 300, including up to 100 heavyweight search
index queries.

This fixes the two simple N+1 query problems. This general pattern of data
access often occurs:

  COUNTEREXAMPLE
  $cats = load_cats();
  foreach ($cats as $cat) {
    $cats_hats = load_hats_for_cat($cat);
    // ...
  }

But this issues "N+1" queries, i.e. if you load 100 cats you issue 101 queries.
It is faster to group the queries instead:

  $cats = load_cats();
  $hats = load_all_hats_for_these_cats($cats);
  foreach ($cats as $cat) {
    $cats_hats = $hats[$cat->getID()];
  }

MySQL can execute one query which returns all the results much faster than 100
queries which return one result, especially if the database is not local (i.e.,
over the network).

However, this doesn't save a ton of time. The bigger issue is that I didn't have
the right keys on the relationship tables in the search engine. This adds them,
and reduces the search engine lookup cost from 25-80ms (for
secure.phabricator.com) down to 1-3ms.

I still probably want to get this out of the loop at some point but it's okay
for now and the page loads in a few ms rather than taking more than a second.

Test Plan:
Used "services" tab, "xhprof" and "EXPLAIN" to analyze page performance. I
measured these changes:

  - Query count: 1 + (3 * N projects) -> 3 + (N projects) (e.g., 301 -> 103)
  - Total time spent querying, ignoring search indexes: 40ms (local.aprhont.com)
-> 20ms (local.aphront.com)
  - Cost for search index query: 25-80ms (secure.phabricator.com) -> 1-3ms

Reviewed By: cadamo
Reviewers: cadamo, aran, jungejason, tuomaspelkonen
CC: aran, cadamo, epriestley
Differential Revision: 485
2011-06-21 14:43:15 -07:00
epriestley
4acf457431 Merge remote branch 'cadamo/devel'
Conflicts:
	src/__celerity_resource_map__.php
2011-06-20 16:06:03 -07:00
epriestley
25bbe741f4 Provide a default non-NULL timezone in the PhabricatorUser class
Summary:
Without this, user creation throws an exception when trying to insert NULL into
a non-NULL field.

Test Plan:
Created a new user.

Reviewed By: fratrik
Reviewers: fratrik, toulouse, aran, jungejason, tuomaspelkonen
CC: aran, fratrik
Differential Revision: 480
2011-06-20 13:13:51 -07:00
Cristian Adamo
7851b6573f Project list and profile view modifications
Summary: Added some change on the project's list view, to show information about
active tasks, population, etc. Also modified the "profile view", and added a class "PhabricatorProfileView" to render the profile, both on projects and users.

Test Plan: play around the project directory :)

Reviewers: epriestley ericfrenkiel

CC:

Differential Revision: 477
2011-06-20 16:13:44 -03:00
epriestley
5915d1dbde Fix syntax clowning in patch 046. 2011-06-20 05:59:42 -07:00
epriestley
513178c7ec Deconflict SQL patches. 2011-06-20 05:58:32 -07:00
epriestley
1cd1e1ed18 "arc install-certificate", server-side components
Summary:
Provides a new workflow for making it non-horrible to install certificates.
Basically you run "arc install-certificate" and then copy/paste a short token
off a webpage and it does the ~/.arcrc edits for you.

Test Plan:
Installed certificates, used bad tokens, hit rate limiting.

Reviewed By: aran
Reviewers: aran, jungejason, tuomaspelkonen
CC: aran
Differential Revision: 460
2011-06-20 05:18:02 -07:00
Andrew Toulouse
9b522982fa Add timezone support
Summary:
Allows user-configurable timezones. Adds a preference panel, and migrates to the
new date rendering in easily-modified areas of the code. ***In progress***.

Test Plan:
Check database to make sure the field is being changed when the settings are
changed; check affected views to see how they render times.

Reviewed By: epriestley
Reviewers: epriestley
CC: aran, epriestley, toulouse
Differential Revision: 475
2011-06-18 13:07:43 -07:00
epriestley
aa86cf0ed8 Countdown tweaks
Summary:
A few tweaks to hsb's Countdown implementation:

  - Allow the page to be rendered "chromeless", suitable for display on one of
the dozens of monitors everyone has laying around.
  - Show title of countdown in deletion dialog.
  - When creating a new countdown default to time(), not Dec 31, 1969.
  - Add extra "/" after editing to avoid needless redirect.
  - Tweak some page titles.
  - Show countdown author in list view.
  - Highlight tab in list view.
  - Tweak menu copy.
  - Link countdown title in list view, separate buttons into different columns
so they pick up padding.

Test Plan:
Created, edited and deleted a timer. Viewed a timer and toggled chrome mode.
Viewed timer list.

Reviewed By: hsb
Reviewers: hsb, aran, jungejason, tuomaspelkonen
CC: aran, hsb, epriestley
Differential Revision: 454
2011-06-13 18:53:43 -07:00
Hafsteinn Baldvinsson
a11b5e8bbc Countdown
Summary:
Addon that allows you to create a live countdown page to
some event.

Here is the ticket that this code is based on
https://secure.phabricator.com/T36

Test Plan:
Tested by manually setting dates in the timer.js file and
checking if they made sense.
I'm not sure if it works across different timezones though.

Reviewers: epriestley

CC:

Differential Revision: 436
2011-06-13 02:23:46 +00:00
Ricky Elrod
b9c9f90164 Pastebin
Summary:
This is Paste. It needs some work, but epriestley recommended that I just commit something that works, and expand on that later.

Specifically, it lacks the ability to view a raw paste right now, and to turn off line numbers, making it hard to copy/paste from for now. It works for showing other people code, however.

Test Plan:
Pasted stuff, and was able to view it, and see it in the list on /paste/. Put a file extension in the title, and saw that syntax highlighting worked as expected.

Reviewers:
epriestley

CC:

Differential Revision: 424
2011-06-10 13:41:18 -04:00
epriestley
2e8990e9e0 Store metadata with Differential and Maniphest comments, and store added
reviewers as metadata

Summary:
The "Add reviewer" implementation is super lazy right now since I didn't want to
do a schema change. Man up and add a column. I also plan to store "via"
information here (e.g., via email or via mobile).

NOTE: This schema change may take a while since the comment table is pretty big
in Facebook's install.

This needs a little CSS work but I think it's reasonable for now.

Test Plan:
Made comments on revisions and tasks. Added reviewers to a revision, got linked
names instead of a blob of text.

Reviewed By: aran
Reviewers: tomo, jungejason, tuomaspelkonen, aran
Commenters: tomo
CC: aran, tomo, epriestley
Differential Revision: 394
2011-06-09 10:43:25 -07:00
gc3
22c1b38655 herald: add the ability to execute a rule the first time only
Summary:
- added a new config class for representing the kind of repetition a rule has
(once, every time, first time only)

- added an email action to herald rules for differential to allow someone to get
an email but only the first one

- changed the herald rule ui to allow a user to pick the amount of repetition

Test Plan:
created a test rule and ran it over and over

Reviewed By: epriestley
Reviewers: epriestley, tuomaspelkonen
CC: aran, epriestley, gc3
Revert Plan:
Tags:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Differential Revision: 357
2011-06-09 10:35:37 -07:00
epriestley
8af5bb117d Basic image thumbnailing
Summary:
This is still very rough but provides basic support for generating image
thumbnails. I need to separate stuff out a bit but I'm going to integrate into
Maniphest before I hit the profile stuff so this seems like a reasonable
starting point.

Test Plan:
Generated some image thumbnails in various sizes.

Reviewed By: aran
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran
Differential Revision: 333
2011-05-27 09:33:33 -07:00
epriestley
deb80b7652 Provide an activity log for login and administrative actions
Summary: This isn't complete, but I figured I'd ship it for review while it's still smallish.

Provide an activity log for high-level system actions (logins, admin actions). This basically allows two things to happen:

  - The log itself is useful if there are shenanigans.
  - Password login can check it and start CAPTCHA'ing users after a few failed attempts.

I'm going to change how the admin stuff works a little bit too, since right now you can make someone an agent, grab their certificate, revert them back to a normal user, and then act on their behalf over Conduit. This is a little silly, I'm going to move "agent" to the create workflow instead. I'll also add a confirm/email step to the administrative password reset flow.

Test Plan: Took various administrative and non-administrative actions, they appeared in the logs. Filtered the logs in a bunch of different ways.

Reviewers: jungejason, tuomaspelkonen, aran

CC:

Differential Revision: 302
2011-05-20 19:08:26 -07:00
epriestley
f9f8ef0e6e Admin and disabled flags for users
Summary:
Provide an "isAdmin" flag for users, to designate administrative users.

Restore the account editing interface and allow it to set role flags and reset
passwords.

Provide an "isDisabled" flag for users and shut down all system access for them.

Test Plan:
Created "admin" and "disabled" users. Did administrative things with the admin
user. Tried to do stuff with the disabled user and was rebuffed. Tried to access
administrative interfaces with a normal non-admin user and was denied.

Reviewed By: aran
Reviewers: tuomaspelkonen, jungejason, aran
CC: ccheever, aran
Differential Revision: 278
2011-05-12 11:17:50 -07:00
epriestley
f7e2b03077 Add a "setup" mode which guides new users through application configuration
Summary:
Alters the installation instructions to guide installers into a "setup" mode
which does config file sanity checking.

Test Plan:
Put myself in setup mode, simulated all the failures it detects, took myself out
of setup mode, Phabricator works OK.

Reviewed By: tuomaspelkonen
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, tuomaspelkonen, epriestley
Differential Revision: 230
2011-05-10 15:12:30 -07:00
epriestley
25dee6ecd2 Support email replies in Phabricator
Summary:
Provides support for per-user x per-object unique reply-to email addresses, plus
SMTP integration.

This does not actually make Phabricator use these in outbound email.

Test Plan:
Used test console to validate in-Phabricator routing and handling.

Piped emails into the "mail_handler.php" script to validate mail parsing.

Configured sendmail and sent mail to Phabricator.

Technically I haven't conducted all parts of this test on the same machine since
I lost the will to configure more SMTP servers after configuring phabricator.com

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, epriestley, jungejason
Differential Revision: 226
2011-05-05 14:58:57 -07:00
epriestley
03ebbccbc9 Restore image proxying to Remarkup
Summary:
Previously, Remarkup allowed you to paste in an image URI and get an inline
image. However, it did this by hotlinking the image which isn't so hot in an
open source product.

Restore this feature, but use image proxying instead. The existing image macro
code does most of the work.

There is a mild security risk depending on the network setup so I've left this
default-disabled and made a note about it. It should be safe to enable for
Facebook.

Test Plan:
Pasted in image and non-image links, got reasonable behavior. Verified proxying
appears to work. Verified that file:// shenanigans produce 400.

Reviewed By: tuomaspelkonen
Reviewers: aran, jungejason, tuomaspelkonen
Commenters: cpiro
CC: aran, cpiro, tuomaspelkonen
Differential Revision: 214
2011-05-03 18:49:06 -07:00
epriestley
8370f93048 Make X-Herald-Rules header sticky
Summary:
This isn't terribly elegant but it solves the problem without loss of
generality. We can pursue a more finessed solution later if it seems prudent.

Test Plan:
Created a revision matched by a blanket herald rule, and then commented on it.
Comment email had X-Herald-Rules header in it.

Reviewed By: aran
Reviewers: aran, tuomaspelkonen, jungejason
CC: aran
Differential Revision: 218
2011-05-03 06:06:57 -07:00
epriestley
94df249775 Improve schema upgrade workflow for unprivileged users
Summary:
In a basically reasonable configuration where you connect
with a non-privileged user from the web workflow, upgrade_schema.php
won't have enough privileges. Allow the user to override the normal
auth with -u and -p.

Test Plan:
Tried to do a schema upgrade with an underprivileged user,
got a useful error message instead of garbage.

Reviewed By: Girish
Reviewers: Girish, davidrecordon, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, Girish
Differential Revision: 191
2011-04-30 00:50:48 -07:00
elynde
bd0a4c0d04 Differential Updates View
Summary:
This adds a new view to differential called Updates.

The high-level goal of Updates is to enabled differential to be
effectively used without email notifications. I've tried doing things
like automatically deleting differential emails where I'm in the 'to'
line since they show up on the main diffential page but then there's
always the chance an important diff flies by without me seeing it. Also,
sometimes someone comments on a diff post-commit but differential
doesn't surface those diffs.

I re-created a test db on my devserver using mysqldump to get data on
revs > 230000 so I would have some test data. We need to add a simple
viewtime table but I didn't want to do that in production. Here's the
table:

  CREATE TABLE differential_viewtime (
    viewerPHID varchar(64) not null,
    objectPHID varchar(64) not null,
    viewTime int unsigned not null,
    PRIMARY KEY (viewerPHID, objectPHID)
  );

Issues:
  -Once we turn this on, all diffs will be 'unviewed'. What do you think
about a 'Clear All' button or something?
  -Maybe we should add a pager

This feature would be insanely useful, let me know what you think.

Test Plan:
Loaded Updates in my sandbox

  http://phabricator.dev1577.snc6.facebook.com/differential/filter/updates/

Clicked a diff, then went back, made sure diff disappeared from Updates
list

Reviewed By: tuomaspelkonen
Reviewers: epriestley, jungejason, tuomaspelkonen
Commenters: epriestley
CC: epriestley, elynde, tuomaspelkonen
Differential Revision: 169
2011-04-28 14:40:41 -07:00
epriestley
ca7a0de1cf Prevent a race in Phabricator workers
Summary: See D133. Workers can also be subject to the same race, invert the
row relationship in the same way.

Test Plan: Launched repository master daemons and some taskmasters and used
the Daemon console to veify that they were able to process tasks. Manually
checked the database to make sure data got linked correctly and that new data
was inserted correctly.

Reviewers: jungejason

CC: tuomaspelkonen

Differential Revision: 135
2011-04-14 12:09:56 -07:00
epriestley
ee1e2da8fb Avoid Timeline race condition
Summary: While I should fix the transactional stuff, that patch is going to be
tricky and transactions have some performance implications. This is a simple
fix which prevents the race.

Instead of having the data point at the event ID, have the event point at a
data ID. Insert the data first, then insert the event with the right data
pointer. This is super simple and prevents the race issue.

Test Plan:
  - Ran the schema upgrade script, verified that the database was
    correctly upgraded. Was also prompted to stop daemons.
  - Ran 'repository-launch-master', verified that the discovery daemons were
    able to discover new commits and insert events for them. Verified the
    committask daemon was consuming events and converting them into tasks.
  - Verified new tasks looked correct in the database.
  - Browsed web interface.

Reviewers: jungejason

CC: tuomaspelkonen

Differential Revision: 133
2011-04-14 10:12:10 -07:00
tuomaspelkonen
f7fe75f756 Image macros for Phabricator!
Summary:
Added long waited image macro support for differential and others.

Test Plan:
Tried a couple of different macros and made sure they appear nicely
in the comment preview. Made sure that the normal comments are shown
correctly.

Reviewed By: epriestley
Reviewers: epriestley
CC: jungejason, tuomaspelkonen, epriestley
Differential Revision: 129
2011-04-13 20:08:13 -07:00
epriestley
43449e49c0 Very basic system agent support. 2011-04-12 18:19:25 -07:00
epriestley
158a561dff Add some metamta keys. 2011-04-12 18:19:24 -07:00
epriestley
720bf02570 Missing key for large datasizes. 2011-04-10 17:25:24 -07:00
epriestley
75b11d6d7d Lint and unit star support.
Test plan: quack

Differential Revision: 35
2011-04-10 17:19:01 -07:00
epriestley
e6c5d6c8ae Properly support mailing lists, with actual testing! 2011-04-10 10:16:14 -07:00
epriestley
8218ca15b9 Turns out MySQL tables need keys. Who knew?! 2011-04-09 22:19:10 -07:00
epriestley
22297b71a0 Close the loop on Diffusion commits posting back to Differential. 2011-04-07 21:59:42 -07:00
epriestley
aa0ff43999 xhpastview 2011-04-06 22:43:56 -07:00
epriestley
5dea10e27c Fix field capitalization. 2011-04-05 22:30:10 -07:00
epriestley
11ea93260a Sync up UUIDs and create project configs. 2011-04-05 21:55:04 -07:00
epriestley
23f882a0ee Some owners write workflows. 2011-04-03 22:03:27 -07:00
epriestley
65e1386753 owners 2011-04-03 15:50:51 -07:00
epriestley
7140107ed9 Optimize session query for nontrivial number of user accounts. 2011-04-02 16:39:40 -07:00
epriestley
82fffe466a Add basic detail-parser functionality. 2011-04-01 17:11:55 -07:00
tuomaspelkonen
6b8c2d110a Merge branch 'preferences' 2011-03-31 14:30:32 -07:00
tuomaspelkonen
123fdabb75 Removed "PHID Types" storage object and interface components
Summary:
Removed because code wasn't used or really needed.

Test Plan:
* Tested that "PHID List" and "PHID Lookup" pages work correctly.
* Tested that new PHIDs can be allocated with the predefined set of types

Reviewed By: epriestley
Reviewers: epriestley
CC: epriestley
Differential Revision: 88
2011-03-31 14:01:13 -07:00
tuomaspelkonen
28fe9f4eca User preferences ported from tools
Summary:

Internal tools, e.g., differential and diffusion  have user defined
preferences for monospaced font and the option for showing either the
name of the tool or the glyph of the tool in the title.

These preferences were ported to phabricator. These preferences can be
modified in /preferences/ and they both affect diffusion and differential
at the moment.

Test Plan:

* Created an empty database
* Loaded /preferences/ and modified the monospaced font and clicked save
* Confirmed that the same page was loaded with the message that preferences
  have been saved and that the example text used the user defined font

* in /preferences/ changed the option to show tool names as plain text and
  clicked save
* Confirmed that the same page was loaded with '[Preferences]' in the title
  instead of a glyph

* These same tests were also executed for differential and diffusion

Reviewers: epriestley
CC: jungejason

Differential Revision: 91
2011-03-31 13:44:20 -07:00
epriestley
34e8d902c7 Restore "Shortcuts" feature to Diffusion. 2011-03-31 00:33:44 -07:00
epriestley
e6cf7a9cb0 More Diffusion junk. 2011-03-30 22:08:41 -07:00
epriestley
919421ead7 Derp derp, apparentl neglected to save this file. 2011-03-26 23:59:29 -07:00
epriestley
fa8f168bd0 Allow commits to be marked as 'bad' so they won't be parsed. Useful if you work
at a company where someone deleted the entire repository *cough cough*.
2011-03-26 23:52:09 -07:00
epriestley
75fcf56b67 Rough cut of Herald home controller. 2011-03-22 13:50:08 -07:00
epriestley
e1d68e852a Sorta need this file. 2011-03-20 17:55:31 -07:00
epriestley
01a20c0480 Fix various parsing bugs in Differential. 2011-03-19 14:42:17 -07:00
EC2 Default User
31d2790075 durf durf sql 2011-03-15 20:51:53 +00:00
epriestley
bb144542d3 Diffusion/phd/console improvements. 2011-03-15 13:38:14 -07:00
epriestley
4196bfb3ef Minor fixes. 2011-03-14 07:43:13 -07:00
epriestley
383b3d740c Improve some diffusioney things.
Summary:

Test Plan:

Reviewers:

CC:
2011-03-12 22:51:40 -08:00
epriestley
485b5e5ded Make the Diffusion UI vaguely usable in some cases. 2011-03-12 16:17:34 -08:00
epriestley
48b5d6d025 Diffusion change parser for SVN.
Summary: there is a small possibility this works correctly

Test Plan:

Reviewers:

CC:
2011-03-11 14:13:23 -08:00
epriestley
8bcbeface1 Increase parseriness of some parsery things.
Summary: None of these are parsnips.

Test Plan:

Reviewers:

CC:
2011-03-11 09:34:22 -08:00
epriestley
ec084ca419 Rough cut of Workers
Summary: workers do work, provided I have written them correctly. perhaps it
is so.

Test Plan:

Reviewers:

CC:
2011-03-10 13:48:29 -08:00
epriestley
57495c4287 Rough cut of repository tracking
Summary: Basic scaffolding for repository tracking, plus daemon infrastructure
(Timelines, Cursors) and some fixes (memory usage, mysql_connect() junk).

Test Plan: parsed Javelin git commit history via daemon

Reviewers:

CC:
2011-03-06 22:29:22 -08:00
epriestley
063269a00a Store OAuth tokens and more OAuth account info.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-22 10:27:27 -08:00
epriestley
c3c16d0ac0 Github OAuth
Summary:

Test Plan:

Reviewers:

CC:
2011-02-21 00:23:24 -08:00
epriestley
c244d4bff2 Fully-qualify this SQL patchfile.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 20:10:07 -08:00
epriestley
7fb9a48690 Task -> Project assocation, file uploads
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 20:08:53 -08:00
epriestley
fdd510ba17 Rough cut of projects.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 18:41:23 -08:00