1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-30 10:42:41 +01:00
Commit graph

472 commits

Author SHA1 Message Date
epriestley
8d37576bd8 Move some aprhont/ stuff around a little bit
Summary:
"configuration/default" moved to "default". Rename it to "configuration" and put config stuff back there.

Put writeguard into writeguard/.

Test Plan: tools

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2639
2012-06-01 12:52:18 -07:00
epriestley
75dc602033 Move policy tests back into policy/
Summary: These were in an unusual location, but are better back in policy/

Test Plan: implicit arc unit

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2638
2012-06-01 12:43:25 -07:00
vrana
6cc196a2e5 Move files in Phabricator one level up
Summary:
- `kill_init.php` said "Moving 1000 files" - I hope that this is not some limit in `FileFinder`.
- [src/infrastructure/celerity] `git mv utils.php map.php; git mv api/utils.php api.php`
- Comment `phutil_libraries` in `.arcconfig` and run `arc liberate`.

NOTE: `arc diff` timed out so I'm pushing it without review.

Test Plan:
/D1234
Browsed around, especially in `applications/repository/worker/commitchangeparser` and `applications/` in general.

Auditors: epriestley

Maniphest Tasks: T1103
2012-06-01 12:32:44 -07:00
epriestley
09c8af4de0 Upgrade phabricator to libphutil v2
Summary: Mechanical changes from D2588. No "Class.php" moves yet.

Test Plan: See D2588.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2591
2012-05-30 14:26:29 -07:00
vrana
5e49de7b35 Use loadRelatives() in loadPrimaryEmail()
Summary: This is an example of code simplification with D2557.

Test Plan: Display user list, verify the SQL queries.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2558
2012-05-30 10:43:16 -07:00
epriestley
70fd96037b Consolidate user editing code
Summary:
  - We currently have some bugs in account creation due to nontransactional user/email editing.
    - We save $user, then try to save $email. This may fail for various reasons, commonly because the email isn't unique.
    - This leaves us with a $user with no email.
  - Also, logging of edits is somewhat inconsistent across various edit mechanisms.
  - Move all editing to a `PhabricatorUserEditor` class.
  - Handle some broken-data cases more gracefully.

Test Plan:
  - Created and edited a user with `accountadmin`.
  - Created a user with `add_user.php`
  - Created and edited a user with People editor.
  - Created a user with OAuth.
  - Edited user information via Settings.
  - Tried to create an OAuth user with a duplicate email address, got a proper error.
  - Tried to create a user via People with a duplicate email address, got a proper error.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: tberman, aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2569
2012-05-25 07:30:44 -07:00
mkedia
787c95abd8 Add owners.query method
Summary:
Allow callers to find all packages/paths owned by a given
user/project.

Test Plan:
Used conduit api page with --
user owner: PHID-USER-6ce1c976b86e5f3c34f6 (tried both loadPackagesFromProjects
true/false)

proj owner: PHID-PROJ-r5wnmmaawqsn4tvjmqm4

repo/path: E, /tfb/trunk/www/flib/privacy. Checked both path.getowners
and owners.query

(all of these are defined internally at fb)

Reviewers: epriestley, btrahan, nh

Reviewed By: btrahan

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2482
2012-05-22 10:19:58 -07:00
epriestley
77f546c572 Allow installs to require email verification
Summary:
Allow installs to require users to verify email addresses before they can use Phabricator. If a user logs in without a verified email address, they're given instructions to verify their address.

This isn't too useful on its own since we don't actually have arbitrary email registration, but the next step is to allow installs to restrict email to only some domains (e.g., @mycompany.com).

Test Plan:
  - Verification
    - Set verification requirement to `true`.
    - Tried to use Phabricator with an unverified account, was told to verify.
    - Tried to use Conduit, was given a verification error.
    - Verified account, used Phabricator.
    - Unverified account, reset password, verified implicit verification, used Phabricator.
  - People Admin Interface
    - Viewed as admin. Clicked "Administrate User".
    - Viewed as non-admin
  - Sanity Checks
    - Used Conduit normally from web/CLI with a verified account.
    - Logged in/out.
    - Sent password reset email.
    - Created a new user.
    - Logged in with an unverified user but with the configuration set to off.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran, csilvers

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2520
2012-05-21 12:47:38 -07:00
epriestley
0548dc4c4c Show user status on calendar
Summary:
This is a rough cut, but gets some of the basics at least. Here's what it looks like:

{F11690}

Some things that would be nice for future diffs:

  - Different colors for different event types (tasks? MEETINGS?!)
  - When events span across multiple days, keep them in the same row.
  - Switch which month you're looking at.
  - Show specific users instead of all.
  - etc etc etc

Test Plan: See screenshot.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2514
2012-05-21 10:24:23 -07:00
epriestley
a89cef8e39 Remove PHID database, add Harbormaster database
Summary:
  - We currently write every PHID we generate to a table. This was motivated by two concerns:
    - **Understanding Data**: At Facebook, the data was sometimes kind of a mess. You could look at a random user in the ID tool and see 9000 assocs with random binary data attached to them, pointing at a zillion other objects with no idea how any of it got there. I originally created this table to have a canonical source of truth about PHID basics, at least. In practice, our data model has been really tidy and consistent, and we don't use any of the auxiliary data in this table (or even write it). The handle abstraction is powerful and covers essentially all of the useful data in the app, and we have human-readable types in the keys. So I don't think we have a real need here, and this table isn't serving it if we do.
    - **Uniqueness**: With a unique key, we can be sure they're unique, even if we get astronomically unlucky and get a collision. But every table we use them in has a unique key anyway. So we actually get pretty much nothing here, except maybe some vague guarantee that we won't reallocate a key later if the original object is deleted. But it's hard to imagine any install will ever have a collision, given that the key space is 36^20 per object type.
  - We also currently use PHIDs and Users in tests sometimes. This is silly and can break (see D2461).
  - Drop the PHID database.
  - Introduce a "Harbormaster" database (the eventual CI tool, after Drydock).
  - Add a scratch table to the Harbormaster database for doing unit test meta-tests.
  - Now, PHID generation does no writes, and unit tests are isolated from the application.
  - @csilvers: This should slightly improve the performance of the large query-bound tail in D2457.

Test Plan: Ran unit tests. Ran storage upgrade.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: csilvers, aran, nh, edward

Differential Revision: https://secure.phabricator.com/D2466
2012-05-20 14:46:01 -07:00
vrana
fa36dd513c Integrate user.getcurrentstatus into user.query
Summary: D2492#6

Test Plan:
`user.query` of user which is away.
`user.query` of user which is not away.
`user.whoami` - no information there.

Reviewers: epriestley

Reviewed By: epriestley

CC: btrahan, aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2502
2012-05-19 13:11:14 -07:00
vrana
8dca5cdb5a Add Conduit method to disable user
Summary: There's no method for enabling users somewhat intentionally.

Test Plan: Disable myself (oops, this is probably my last diff ever).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2497
2012-05-18 16:29:36 -07:00
vrana
2476c872ff Add Conduit method for querying user status
Summary:
I want to use this to warn user if he specifies reviewers that are away.

We can also implement a general query method but I think that this usage is the
most useful not only for me but also in general case.

Test Plan:
Call the method for user which is away and which is not away.
Add user status through Conduit.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2492
2012-05-17 23:58:50 -07:00
Jason Ge
67d4f1ef4c Detect package change and send out email
Summary:
For package creation and deletion, send email to all the owners For
package modification, detect important fields such as owners and paths, and then
send out emails to all owners (including deleted owners and current owners)

Also start using transaction for package creation/deletion/modification.

Test Plan:
- tested mail creation and deletion
- tested modification to auditing enabled, primary owners, owners, paths

Reviewers: epriestley, nh, vrana

Reviewed By: epriestley

CC: prithvi, aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2470
2012-05-14 15:58:24 -07:00
epriestley
820a6d407a Improve performance of repository discovery under Mercurial
Summary: Prelminary, I want to test this a bit more when I'm better rested. Provide a "bulk" import mode for Mercurial so we can do initial discovery more quickly.

Test Plan: Imported the Mercurial repository in 2m45s, blocked on MySQL I/O rather than Mercurial I/O.

Reviewers: csilvers, btrahan

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2457
2012-05-11 18:29:14 -07:00
Aurelijus
ba98089426 Branch view improvements.
Summary:
Sorting by last commit date
Branch view limit to 25 branches
All branches table page with pagination on Git

Test Plan:
* Check repository view for expected behavior on branch table view
* Check all branches page & test pagination

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1200

Differential Revision: https://secure.phabricator.com/D2442
2012-05-10 11:47:46 -07:00
epriestley
1bf68e06a5 Improve Diffusion error messages and UI for partially imported repositories
Summary:
  - When you have an un-cloned repository, we currently throw random-looking Git/Hg exception. Instead, throw a useful error.
  - When you have a cloned but undiscovered repository, we show no commits. This is crazy confusing. Instead, show commits as "importing...".
  - Fix some warnings and errors for empty path table cases, etc.

Test Plan:
  - Wiped database.
  - Added Mercurial repo without running daemons. Viewed in Diffusion, got a good exception.
  - Pulled Mercurial repo without discovering it. Got "Importing...".
  - Discovered Mercurial repo without parsing it. Got "Importing..." plus date information.
  - Parsed Mercurial repo, got everything working properly.
  - Added Git repo without running daemons, did all the stuff above, same results.
  - This doesn't improve SVN much but that's a trickier case since we don't actually make SVN calls and rely only on the parse state.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776

Differential Revision: https://secure.phabricator.com/D2439
2012-05-09 17:28:57 -07:00
epriestley
b98847d2b1 Show "show more..." link in "Local Commits" view in Differential
Summary: Javascript! Depends on D2437 to do anything useful.

Test Plan: Clicked "show more", saw more.

Reviewers: csilvers, btrahan

Reviewed By: csilvers

CC: aran

Maniphest Tasks: T1189

Differential Revision: https://secure.phabricator.com/D2438
2012-05-09 15:56:37 -07:00
epriestley
b800df8c1b Simplify daemon management: "phd start"
Summary:
  - Merge CommitTask daemon into PullLocal daemon. This is another artifact of past instability (and order-dependent parsers). We still publish to the timeline, although this was the last consumer. Long term we'll probably delete timeline and move to webhooks, since everyone who has asked about this stuff has been eager to trade away the durability and ordering of the timeline for the ease of use of webhooks. There's also no reason to timeline this anymore since parsing is no longer order-dependent.
  - Add `phd start` to start all the daemons you need. Add `phd restart` to restart all the daemons you need. So cool~
  - Simplify and improve phd and Diffusion daemon documentation.

Test Plan:
  - Ran `phd start`.
  - Ran `phd restart`.
  - Generated/read documentation.
  - Imported some stuff, got clean parses.

Reviewers: btrahan, csilvers

Reviewed By: csilvers

CC: aran, jungejason, nh

Differential Revision: https://secure.phabricator.com/D2433
2012-05-09 10:29:37 -07:00
epriestley
d2b01aead0 Use one daemon to discover commits in all repositories, not one per repository
Summary:
See D2418. This merges the commit discovery daemon into the same single daemon, and applies all the same rules to it.

There are relatively few implementation changes, but a few things did change:

  - I simplified/improved Mercurial importing, by finding full branch tip hashes with "--debug branches" and using "parents --template {node}" so we don't need to do separate "--debug id" calls.
  - Added a new "--not" flag to exclude repositories, since I switched to real arg parsing anyway.
  - I removed a web UI notification that you need to restart the daemons, this is no longer true.
  - I added a web UI notification that no pull daemon is running on the machine.

NOTE: @makinde, this doesn't change anything from your perspective, but it something breaks this is the likely cause.

This implicitly resolves T792, because discovery no longer runs before pulling.

Test Plan:

  - Swapped databases to a fresh install.
  - Ran "pulllocal" in debug mode. Verified it correctly does nothing (fixed a minor issue with min() on empty array).
  - Added an SVN repository. Verified it cloned and discovered correctly.
  - Added a Mercurial repository. Verified it cloned and discovered correctly.
  - Added a Git repository. Verified it cloned and discovered correctly.
  - Ran with arguments to verify behaviors: "--not MTEST --not STEST", "P --no-discovery", "P".

Reviewers: btrahan, csilvers, Makinde

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T792

Differential Revision: https://secure.phabricator.com/D2430
2012-05-08 12:53:41 -07:00
Bob Trahan
679f778235 OAuth -- add support for Disqus
Summary:
also fix some bugs where we weren't properly capturing the expiry value or scope of access tokens.

This code isn't the cleanest as some providers don't confirm what scope you've been granted. In that case, assume the access token is of the minimum scope Phabricator requires. This seems more useful to me as only Phabricator at the moment really easily / consistently lets the user increase / decrease the granted scope so its basically always the correct assumption at the time we make it.

Test Plan: linked and unlinked Phabricator, Github, Disqus and Facebook accounts from Phabricator instaneces

Reviewers: epriestley

Reviewed By: epriestley

CC: zeeg, aran, Koolvin

Maniphest Tasks: T1110

Differential Revision: https://secure.phabricator.com/D2431
2012-05-08 12:08:05 -07:00
epriestley
1c62a35710 Run one daemon to pull all working copies, not one daemon per working copy
Summary:
Allow the pull daemon to take a list of repositories. By default, pull all repositories.

Make some effort to respect pull frequencies, although we'll necessarily suffer a bit if running with only one process.

NOTE: We still launch one discovery daemon per working copy, so this only cuts the daemon count in half.

Test Plan:
  - Ran `phd debug pulllocal`, verified behavior.
  - Ran `pull.php P MTEST SVNTEST --trace`, verified it pulled the repos and ran the right commands.
  - Ran `phd repository-launch-master`, verified the right daemons launched, checked daemon console.
  - Ran `phd repository-launch-readonly`, verified the right daemon launched, checked daemon console.

Reviewers: btrahan, csilvers, davidreuss

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2418
2012-05-07 15:01:10 -07:00
Bob Trahan
c58bd95842 Conduit user.query method
Summary: also includes a PhabricatorPeopleQuery object

Test Plan: queried for various combinations username, realname, id, phid, and email. verified correct results.

Reviewers: epriestley

Reviewed By: epriestley

CC: zeeg, aran, Koolvin

Maniphest Tasks: T1075

Differential Revision: https://secure.phabricator.com/D2416
2012-05-07 13:35:09 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
epriestley
c30cb7669e Simplify transaction handling and restore read/write locks
Summary:
  - We used to have connection-level caching, so we needed getTransactionKey() to make sure there was one transaction state per real connection. We now cache in Lisk and each Connection object is guaranteed to represent a real, unique connection, so we can make this a non-static.
  - I kept the classes separate because it was a little easier, but maybe we should merge them?
  - Also track/implement read/write locking.
  - (The advantage of this over just writing LOCK IN SHARE MODE is that you can use, e.g., some Query class even if you don't have access to the queries it runs.)

Test Plan: Can you come up with a way to write unit tests for this? It seems like testing that it works requires deadlocking MySQL if the test is running in one process.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2398
2012-05-05 11:29:09 -07:00
epriestley
049048765d Add "repository.create" and "repository.query" methods to Conduit
Summary: Primarily for @csilvers who has 92 million repositories or something. This is a touch hacky, but movitated by pragmatism.

Test Plan:
  - Ran "repository.create" to create repositories, "repository.query" to list them.
  - Tested most or maybe all of the error conditions, probably.

Reviewers: btrahan, vrana, csilvers

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2396
2012-05-04 16:16:22 -07:00
vrana
e4f4b6a3fe Add conduit method for deleting user status
Test Plan: All four eventualities.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2383
2012-05-04 13:03:02 -07:00
vrana
87ff461470 Add /F123 shortcut
Summary:
I wanted to point someone on a file uploaded to Phabricator and the normal link is just too long.

I guess that this also improves security. Because pointing someone to the file directly reveals the secret key used in /data/ and it can be served without auth?

We already use `{F123}` so there will be no conflicts in future because we wouldn't want to reuse it for something else.

I promote the link on /file/ - it adds one redirect but I think it's worth it. I also considered making the link from the File ID column but there are already too many links (with some duplicity).

Test Plan:
/file/
/F123 (redirect)
/F9999999999 (404)

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2380
2012-05-03 23:51:40 -07:00
vrana
416e4e7b67 Allowing setting user status
Summary:
I will use it for highlighting users which are not currently available.

Maybe I will also use it in the nagging tool.

I don't plan creating a UI for it as API is currently enough for us.
Maybe I will visualize it at /calendar/ later.

I plan creating `user.deletestatus` method when this one will be done.

Test Plan:
`storage upgrade`
Call Conduit `user.addstatus`.
Verify DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2382
2012-05-03 18:24:30 -07:00
epriestley
9b2ededd48 Document configuration of file upload limits
Summary: I have a patch which makes uploads all fancy and adds progress bars, but document the landscape first since it's quite complicated.

Test Plan: Generated, read docs. Configured `storage.upload-size-limit` to various values.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T875

Differential Revision: https://secure.phabricator.com/D2381
2012-05-03 17:30:17 -07:00
vrana
631718f99e Implement getNthBusinessDay()
Summary: I will need it for nagging tool.

Test Plan:
None yet.
Please suggest me how to create a testing database (I need to insert some data in the table). I guess that it is now possible?
There is also probably some bug in `arc unit` - `setEnvConfig()` is not called before `getEnvConfig()` resulting in fatal error.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2376
2012-05-03 12:26:58 -07:00
vrana
73c82e5a94 Display holidays
Summary:
We will need it for two purposes:

- Status tool.
- Nagging tool - @aran suggested using "3 business days" and I don't want it to fall on New Year's Eve or such.

I don't plan working on any interface for editing this as this kind of data should be always imported.

Test Plan:
`bin/storage upgrade`
`scripts/calendar/import_us_holidays.php`
/calendar/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2375
2012-05-03 09:22:52 -07:00
epriestley
63ce372480 Add "DiffusionRawDiffQuery"
Summary:
  - This is only slightly useful for updating Differential, since DiffQuery (vs RawDiffQuery) already gets you most of what you need. The only thing is that DiffQuery returns the diff for one path only right now(and the SVN version is very "special"). Should be easy to fix in the Git/HG cases at least, though (or maybe just use RawDiffQuery to avoid the SVN mess).
  - Added a "download raw diff" link.

Test Plan: Viewed Diffusion and raw commits for SVN, Mercurial and Git repositories.

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2350
2012-05-02 13:43:45 -07:00
epriestley
5ab14d0879 Provide isolated, read/write storage fixtures for unit tests
Summary:
  - Unit tests can request storage fixtures.
  - We build one fixture across all tests in the process, which can quickstart (takes roughly 1s to build, 200ms to destroy for me). This is a one-time cost for running an arbitrary number of fixture-based tests.
  - We isolate all the connections inside transactions for each test, so individual tests don't affect one another.

Test Plan: Ran unit tests, which cover the important properties of fixtures.

Reviewers: btrahan, vrana, jungejason, edward

Reviewed By: btrahan

CC: aran, davidreuss

Maniphest Tasks: T140

Differential Revision: https://secure.phabricator.com/D2345
2012-05-02 12:42:23 -07:00
epriestley
f04d8ab1a7 Further improve unit/lint rendering
Summary:
I think this improves things, let me know if you have feedback.

Also addresses T840.

Test Plan: See screenshots...

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran, zeeg

Maniphest Tasks: T840

Differential Revision: https://secure.phabricator.com/D2357
2012-05-01 10:15:56 -07:00
Bob Trahan
8988667dcc Make Oauth-registration flows a bit more resilient to failures from the providers
Summary: basically by validating we have good user data when we set the user data.

Test Plan: simulated a failure from a phabricator on phabricator oauth scenario. viewed ui that correctly told me there was an error with the provider and to try again.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1077

Differential Revision: https://secure.phabricator.com/D2337
2012-05-01 11:51:40 +02:00
epriestley
087cc0808a Make SQL patch management DAG-based and provide namespace support
Summary:
This addresses three issues with the current patch management system:

  # Two people developing at the same time often pick the same SQL patch number, and then have to go rename it. The system catches this, but it's silly.
  # Second/third-party developers can't use the same system to manage auxiliary storage they may want to add.
  # There's no way to build mock databases for unit tests that need to do reads.

To resolve these things, you can now name your patches whatever you want and conflicts are just merge conflicts, which are less of a pain to fix than filename conflicts.

Dependencies are now a DAG, with implicit dependencies created on the prior patch if no dependencies are specified. Developers can add new concrete subclasses of `PhabricatorSQLPatchList` to add storage management, and define the dependency branchpoint of their patches so they apply in the correct order (although, generally, they should not depend on the mainline patches, presumably).

The commands `storage upgrade --namespace test1234` and `storage destroy --namespace test1234` will allow unit tests to build and destroy MySQL storage.

A "quickstart" mode allows an upgrade from scratch in ~1200ms. Destruction takes about 200ms. These seem like fairily reasonable costs to actually use in tests. Building from scratch patch-by-patch takes about 6000ms.

Test Plan:
  - Created new databases from scratch with and without quickstart in a separate test namespace. Pointed the webapp at the test namespaces, browsed around, everything looked good.
  - Compared quickstart and no-quickstart dump states, they're identical except for mysqldump timestamps and a few similar things.
  - Upgraded a legacy database to the new storage format.
  - Destroyed / dumped storage.

Reviewers: edward, vrana, btrahan, jungejason

Reviewed By: btrahan

CC: aran, nh

Maniphest Tasks: T140, T345

Differential Revision: https://secure.phabricator.com/D2323
2012-04-30 07:54:00 -07:00
epriestley
bf505ef51c Hard-code and unify "more applications" in Phabricator
Summary:
This is mostly intended to simplify D2323.

We currently allow users to edit and customize the links on the homepage, but as far as I know no one actually does this (no one complained when we redid the homepage earlier this year) and it creates a lot of mess in the database patches and quickstart dump. After D2331, this is the only data we load in the patch files. The patch files are also a mess with respect to this data and have various different versions of it.

Also the current UI is just kind of bad, it stretches stuff across too many screens and is generally ungood. Nuking this lets us nuke a lot of code in general.

(In the long term, I think we'll move toward an "application" model anyway, and this stuff will go away sooner or later.)

I'll add a drop-database patch some time later, just in case anyone does actually use this, so they can get their data out of MySQL.

Test Plan: Looked at home page, clicked "More Stuff", got a single list of other apps/things.

Reviewers: btrahan, vrana, edward, jungejason

Reviewed By: jungejason

CC: aran

Maniphest Tasks: T345

Differential Revision: https://secure.phabricator.com/D2332
2012-04-28 07:17:38 -07:00
vrana
d27a751339 Create overview of Conduit methods
Summary:
Also couple of small changes:

- Add method name to title.
- 404 for /conduit/method/x/.
- Remove utilities from side panel.
- Remove side panel from log.

Test Plan:
/conduit/
/conduit/method/x/
/conduit/method/user.whoami/
/conduit/log/

Reviewers: btrahan, epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2326
2012-04-27 17:27:47 -07:00
epriestley
3ce69b6306 Allow Phabricator to write an access log using PhutilDeferredLog
Summary: Provide a configurable access log.

Test Plan:
Got a sensible-looking log including logged-in, logged-out, conduit, 404, etc:

  [Mon, 23 Apr 2012 20:08:12 -0700]	32599	orbital	-	epriestley	DifferentialCommentPreviewController	-	/differential/comment/preview/42/	http://local.aphront.com:8080/D42	200	65406
  [Mon, 23 Apr 2012 20:08:12 -0700]	32881	orbital	-	epriestley	DifferentialChangesetViewController	-	/differential/changeset/	http://local.aphront.com:8080/D42	200	72669
  [Mon, 23 Apr 2012 20:08:39 -0700]	32882	orbital	127.0.0.1	epriestley	DifferentialRevisionListController	-	/differential/	http://local.aphront.com:8080/D42	200	106444
  [Mon, 23 Apr 2012 20:08:54 -0700]	32867	orbital	127.0.0.1	epriestley	DifferentialRevisionListController	-	/differential/	http://local.aphront.com:8080/differential/	200	112229
  [Mon, 23 Apr 2012 20:09:05 -0700]	32530	orbital	127.0.0.1	epriestley	PhabricatorDirectoryMainController	-	/	http://local.aphront.com:8080/differential/	200	141350
  [Mon, 23 Apr 2012 20:09:10 -0700]	32598	orbital	127.0.0.1	epriestley	PhabricatorDirectoryCategoryViewController	-	/directory/6/	http://local.aphront.com:8080/	200	43474
  [Mon, 23 Apr 2012 20:09:12 -0700]	32880	orbital	127.0.0.1	epriestley	PhabricatorConduitConsoleController	-	/conduit/	http://local.aphront.com:8080/directory/6/	200	139340
  [Mon, 23 Apr 2012 20:09:15 -0700]	32868	orbital	127.0.0.1	epriestley	PhabricatorConduitAPIController	arcanist.projectinfo	/api/arcanist.projectinfo	http://local.aphront.com:8080/conduit/	200	128774
  [Mon, 23 Apr 2012 20:10:04 -0700]	32599	orbital	127.0.0.1	epriestley	Phabricator404Controller	-	/asdbmabdmbsm	-	404	38782
  [Mon, 23 Apr 2012 20:10:04 -0700]	32881	orbital	127.0.0.1	-	CelerityResourceController	-	/res/c9a43002/rsrc/css/aphront/request-failure-view.css	http://local.aphront.com:8080/asdbmabdmbsm	200	25160
  [Mon, 23 Apr 2012 20:10:57 -0700]	32882	orbital	127.0.0.1	epriestley	PhabricatorLogoutController	-	/logout/	http://local.aphront.com:8080/asdbmabdmbsm	200	40810
  [Mon, 23 Apr 2012 20:10:57 -0700]	32867	orbital	127.0.0.1	-	PhabricatorLoginController	-	/login/	http://local.aphront.com:8080/asdbmabdmbsm	200	42526
  [Mon, 23 Apr 2012 20:10:59 -0700]	32919	orbital	127.0.0.1	-	PhabricatorLoginController	-	/login/	http://local.aphront.com:8080/asdbmabdmbsm	200	49052
  [Mon, 23 Apr 2012 20:10:59 -0700]	32880	orbital	127.0.0.1	-	CelerityResourceController	-	/res/c80156c4/rsrc/js/application/core/behavior-dark-console.js	http://local.aphront.com:8080/login/	200	33166
  [Mon, 23 Apr 2012 20:10:59 -0700]	32868	orbital	127.0.0.1	-	CelerityResourceController	-	/res/4965d970/rsrc/css/aphront/dark-console.css	http://local.aphront.com:8080/login/	200	38078
  [Mon, 23 Apr 2012 20:10:59 -0700]	32599	orbital	127.0.0.1	-	CelerityResourceController	-	/res/pkg/8a5de8a3/javelin.pkg.js	http://local.aphront.com:8080/login/	200	40534
  [Mon, 23 Apr 2012 20:10:59 -0700]	32882	orbital	127.0.0.1	-	CelerityResourceController	-	/res/pkg/9c4e265b/core.pkg.css	http://local.aphront.com:8080/login/	200	41262
  [Mon, 23 Apr 2012 20:10:59 -0700]	32881	orbital	127.0.0.1	-	CelerityResourceController	-	/res/pkg/0c96375e/core.pkg.js	http://local.aphront.com:8080/login/	200	43720
  [Mon, 23 Apr 2012 20:10:59 -0700]	32921	orbital	127.0.0.1	-	CelerityResourceController	-	/res/caa86a45/rsrc/js/javelin/core/init.js	http://local.aphront.com:8080/login/	200	47566
  [Mon, 23 Apr 2012 20:10:59 -0700]	32867	orbital	127.0.0.1	-	CelerityResourceController	-	/res/f46289e9/rsrc/js/application/core/behavior-error-log.js	http://local.aphront.com:8080/login/	200	29328
  [Mon, 23 Apr 2012 20:10:59 -0700]	32919	orbital	127.0.0.1	-	CelerityResourceController	-	/res/7e62ff40/rsrc/image/phabricator_logo.png	http://local.aphront.com:8080/login/	200	25583
  [Mon, 23 Apr 2012 20:10:59 -0700]	32880	orbital	127.0.0.1	-	CelerityResourceController	-	/res/8c6200d3/rsrc/image/sprite.png	http://local.aphront.com:8080/login/	200	29829
  [Mon, 23 Apr 2012 20:11:01 -0700]	32868	orbital	127.0.0.1	-	PhabricatorOAuthLoginController	-	/oauth/facebook/login/  http://local.aphront.com:8080/login/	200	855931
  [Mon, 23 Apr 2012 20:11:02 -0700]	32882	orbital	127.0.0.1	epriestley789	PhabricatorLoginValidateController	-	/login/validate/	http://local.aphront.com:8080/login/	200	29793
  [Mon, 23 Apr 2012 20:11:02 -0700]	32881	orbital	127.0.0.1	epriestley789	PhabricatorDirectoryMainController	-	/	http://local.aphront.com:8080/login/	200	91638

Reviewers: jungejason, btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2310
2012-04-25 07:24:08 -07:00
epriestley
7b334f37bd Improve tag support in Diffusion
Summary:
  - When viewing a commit, show its tags.
  - For commits with many tags, show a list of all tags on the tag list interface.
  - Improve some handling of symbolic references.
  - When tags contain content, show it on the browse view reached by clicking the tag name.

Test Plan: Looked at commits with and without tags, clicked "More tags...", clicked tag names.

Reviewers: btrahan, vrana, davidreuss, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1130

Differential Revision: https://secure.phabricator.com/D2290
2012-04-23 18:36:25 -07:00
epriestley
e9bd842227 Allow "blame previous revision" to track file moves and handle edge cases
Summary:
  - Track + message through file moves.
  - Stop + message on file create.
  - Stop + message on first commit.

Test Plan:
  - Tested blaming through a move, through a create, and through the first commit.
  - Verified this doesn't break anything in SVN / Mercurial.

Reviewers: vrana, btrahan, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1091

Differential Revision: https://secure.phabricator.com/D2295
2012-04-23 18:16:38 -07:00
epriestley
20a5c9b261 Use "closed", not "committed", in Differential
Summary: "Committed" is SVN-specific language, and confusing in Git and Mercurial. Use neutral language instead.

Test Plan: Inspection.

Reviewers: btrahan, Makinde, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T909

Differential Revision: https://secure.phabricator.com/D2087
2012-04-23 17:40:57 -07:00
epriestley
bbe2063443 [NO CLUE WHAT I'M DOING] Add an Elasticsearch engine
Summary:
I have no idea what I'm doing, but here's part of an elasticsearch engine. These things work:

  - Indexing stuff (??)
  - Searching for text/type?
  - Reconstructing things??

All the complicated stuff doesn't work. I'm having a hard time figuring out the best way to model things because elasticsearch's documentation is not exactly the most complete or illuminating.

@amckinley, does this look sane-ish so far? Particularly, the /phabricator/<type>/<phid>/ URI scheme and how I've set up the relationships and fields in the documents?

How should I model the relationship and field queries? I want, like, an "equal" query but it seems like I've got "text" or "term" to work with and neither are exact match? And "term" doesn't consider PHIDs to be terms since they have hyphens in them?

I'll keep kind of slogging my way forward here but if you have valuable wisdom to share it would probably get me to a better end state much faster. The whole query construction phase is pretty much black magic to me.

Test Plan: nyancat

Reviewers: amckinley, vrana

Reviewed By: vrana

CC: jungejason, tuomaspelkonen, aran, 20after4, vrana

Differential Revision: https://secure.phabricator.com/D790
2012-04-20 15:33:09 -07:00
epriestley
944049d871 Add a paginated list of all repository tags to Diffusion
Summary: Now supports more than 25 tags!

Test Plan: Set page size to 1, paginated. Verified SVN / Hg don't break/explode.

Reviewers: davidreuss, vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1130

Differential Revision: https://secure.phabricator.com/D2268
2012-04-19 09:39:19 -07:00
epriestley
dec8acd803 Add very basic tag support to Diffusion
Summary: Lists the 25 most recent tags on the "Repository" page.

Test Plan: Looked at a git repository with a tag, saw it. Looked at HG/SVN repos, they didn't break.

Reviewers: davidreuss, 20after4, btrahan, vrana, jungejason

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T1130

Differential Revision: https://secure.phabricator.com/D2255
2012-04-18 08:02:08 -07:00
epriestley
0f06287dc5 Allow PhabricatorEnv to be temporarily made mutable for unit tests
Summary: Introduces a scope-guarded way to override the env config, for unit tests which are sensitive to config values.

Test Plan: Ran unit tests.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2237
2012-04-17 07:52:01 -07:00
epriestley
ded641ae32 Add basic per-object privacy policies
Summary:
Provides a basic start for access policies. Objects expose various capabilities, like CAN_VIEW, CAN_EDIT, etc., and set a policy for each capability. We currently implement three policies, PUBLIC (anyone, including logged-out), USERS (any logged-in) and NOONE (nobody). There's also a way to provide automatic capability grants (e.g., the owner of an object can always see it, even if some capability is set to "NOONE"), but I'm not sure how great the implementation feels and it might change.

Most of the code here is providing a primitive for efficient policy-aware list queries. The problem with doing queries naively is that you have to do crazy amounts of filtering, e.g. to show the user page 6, you need to filter at least 600 objects (and likely more) before you can figure out which ones are 500-600 for them. You can't just do "LIMIT 500, 100" because that might have only 50 results, or no results. Instead, the query looks like "WHERE id > last_visible_id", and then we fetch additional pages as necessary to satisfy the request.

The general idea is that we move all data access to Query classes and have them do object filtering. The ID paging primitive allows efficient paging in most cases, and the executeOne() method provides a concise way to do policy checks for edit/view screens.

We'll probably end up with mostly broader policy UIs or configuration-based policies, but there are at least a few cases for per-object privacy (e.g., marking tasks as "Security", and restricting things to the members of projects) so I figured we'd start with a flexible primitive and the simplify it in the UI where we can.

Test Plan: Unit tests, played around in the UI with various policy settings.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D2210
2012-04-14 10:13:29 -07:00
Bob Trahan
51418900f7 Phame V1 - Phabricator blogging software
Summary:
'cuz we need to be phamous!

V1 feature set

- posts
-- standard thing you'd expect - a title and a remarkup-powered body and...
-- "phame" title - a short string that can be used to reference the story. this gets auto-updated when you mess with the title.
-- configuration - for now, do you want Facebook, Disqus or no comments? this is a per-post thing but feeds from an instance-wide configuration

Please do toss out any must have features or changes.

Test Plan: played around with this bad boy like whoa

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, vrana

Maniphest Tasks: T1111

Differential Revision: https://secure.phabricator.com/D2202
2012-04-12 13:09:04 -07:00
Bob Trahan
1175784d5d PhabricatorSlug
Summary:
This is to be used in Phame so the logic is shared where possible. The change has three main things going on

- broke out functionality from PhrictionDocument that isn't Phriction specific.
- swept up code base to use new PhabricatorSlug class.
- altered the regex ever so slightly per discussion and http://stackoverflow.com/questions/2028022/javascript-how-to-convert-unicode-string-to-ascii

I think maybe we should punt on unicode here for quite a bit -- http://www.456bereastreet.com/archive/201006/be_careful_with_non-ascii_characters_in_urls/ -- but we'll be well-positioned to add it with the code here.

Test Plan: used phriction to create, edit, view documents. used a tool (codemod) for the codebase sweeping

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2195
2012-04-10 14:18:20 -07:00