1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-23 15:22:41 +01:00
Commit graph

172 commits

Author SHA1 Message Date
epriestley
2b0b9a1573 Add a generic multistep Markup cache
Summary:
The immediate issue this addresses is T1366, adding a rendering cache to Phriction. For wiki pages with code blocks especially, rerendering them each time is expensive.

The broader issue is that out markup caches aren't very good right now. They have three major problems:

**Problem 1: the data is stored in the wrong place.** We currently store remarkup caches on objects. This means we're always loading it and passing it around even when we don't need it, can't genericize cache management code (e.g., have one simple script to drop/GC caches), need to update authoritative rows to clear caches, and can't genericize rendering code since each object is different.

To solve this, I created a dedicated cache database that I plan to move all markup caches to use.

**Problem 2: time-variant rules break when cached.** Some rules like `**bold**` are time-invariant and always produce the same output, but some rules like `{Tnnn}` and `@username` are variant and may render differently (because a task was closed or a user is on vacation). Currently, we cache the raw output, so these time-variant rules get locked at whatever values they had when they were first rendered. This is the main reason Phriction doesn't have a cache right now -- I wanted `{Tnnn}` rules to reflect open/closed tasks.

To solve this, I split markup into a "preprocessing" phase (which does all the parsing and evaluates all time-invariant rules) and a "postprocessing" phase (which evaluates time-variant rules only). The preprocessing phase is most of the expense (and, notably, includes syntax highlighting) so this is nearly as good as caching the final output. I did most of the work here in D737 / D738, but we never moved to use it in Phabricator -- we currently just do the two operations serially in all cases.

This diff splits them apart and caches the output of preprocessing only, so we benefit from caching but also get accurate time-variant rendering.

**Problem 3: cache access isn't batched/pipelined optimally.** When we're rendering a list of markup blocks, we should be able to batch datafetching better than we do. D738 helped with this (fetching is batched within a single hunk of markup) and this improves batching on cache access. We could still do better here, but this is at least a step forward.

Also fixes a bug with generating a link in the Phriction history interface ($uri gets clobbered).

I'm using PHP serialization instead of JSON serialization because Remarkup does some stuff with non-ascii characters that might not survive JSON.

Test Plan:
  - Created a Phriction document and verified that previews don't go to cache (no rows appear in the cache table).
  - Verified that published documents come out of cache.
  - Verified that caches generate/regenerate correctly, time-variant rules render properly and old documents hit the right caches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1366

Differential Revision: https://secure.phabricator.com/D2945
2012-07-09 15:20:56 -07:00
epriestley
7cf6313be9 Add a generic object for unit tests
Summary:
A later diff adds unit tests against edges, but we need real objects to connect with edges. Add some trivial objects to the Harbormaster database to compliment the similar HarbormasterScratchTable.
On its own, this does nothing interesting.

Test Plan: Built unit tests on this in a followup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2937
2012-07-09 10:39:14 -07:00
dschleimer
86fa4fd97f [Phabricator] track Mercurial bookmarks for differential diffs
Summary:
This adds all the changes necessary to track the active Mercurial
bookmark for differential diffs.  We render both branch and bookmark
information in the branch field of the Differential revison view, as
seen in
https://secure.phabricator.com/file/data/kzpmu3evfkukxdjyxrfz/PHID-FILE-eqorsqupxvwirqi2s5lo/bookmark_differential.jpg

The Arcanist half of this is https://secure.phabricator.com/D2896

Test Plan:
Mostly D2896.

Additionally, loaded a diff created with a bookmark, as per the link in the summary.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1331

Differential Revision: https://secure.phabricator.com/D2897
2012-06-30 15:41:58 -07:00
vrana
48ebcf0679 Allow user override translation and implement PhutilPerson
Test Plan:
Altered database.
Wrote a custom translation and selected it in preferences.
Verified that the text is custom translated.
Set language back to default.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2757
2012-06-14 18:33:00 -07:00
vrana
892a2d1b61 Make Thread-Topic human readable
Summary:
Some e-mail clients display this header and it needs to be constant.

This is somehow involved but I doubt that there is a simpler solution.

Test Plan:
Applied SQL patch.
Commented on revision, commented on commit, changed package.
Verified that the `Thread-Topic` has constant and human readable value.

Reviewers: epriestley

Reviewed By: epriestley

CC: ola, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2745
2012-06-14 11:36:34 -07:00
Espen Volden
726041584f Made it possible to login using LDAP
Summary: Made it possible to link and unlink LDAP accounts with  Phabricator accounts.

Test Plan:
I've tested this code locally and in production where I work.
I've tried creating an account from scratch by logging in with LDAP and linking and unlinking an LDAP account with an existing account. I've tried to associate the same LDAP account with different Phabricator accounts and it failed as expected.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, auduny, svemir

Maniphest Tasks: T742

Differential Revision: https://secure.phabricator.com/D2722
2012-06-13 08:58:46 -07:00
Keebuhm Park
207f101aee SQL patch for notification
Summary: Added `PhabricatorBuiltinPatchList` entry so that "storage upgrade" will update the database. Renamed and numbered the notification.sql patch.

Test Plan: Drop phabricator_feed.feed_storynotification table if it exists and run bin/storage upgrade to check if the patch is correctly applied.

Reviewers: epriestley, btrahan, allenjohnashton

Reviewed By: epriestley

CC: ddfisher, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2687
2012-06-08 12:42:59 -07:00
John-Ashton Allen
3a6ee79190 Adds base notification application
Summary: First diff in a series of diffs to add notifications to Phabricator. This is the notification application ONLY. This commit does not include the changes to other applications that makes them add notifications. As such, no notifications will be generated beyond the initial database import.

Test Plan: This is part of the notifications architecture which has been running on http://theoryphabricator.com for the past several months.

Reviewers: epriestley, btrahan, ddfisher

Reviewed By: epriestley

CC: allenjohnashton, keebuhm, aran, Korvin, jungejason, nh

Maniphest Tasks: T974

Differential Revision: https://secure.phabricator.com/D2571
2012-06-08 06:32:02 -07:00
epriestley
f6fbe40bd5 Minor, completely remove references to PHID from schema patches so upgrade-from-scratch works. 2012-05-24 13:59:12 -07:00
epriestley
a89cef8e39 Remove PHID database, add Harbormaster database
Summary:
  - We currently write every PHID we generate to a table. This was motivated by two concerns:
    - **Understanding Data**: At Facebook, the data was sometimes kind of a mess. You could look at a random user in the ID tool and see 9000 assocs with random binary data attached to them, pointing at a zillion other objects with no idea how any of it got there. I originally created this table to have a canonical source of truth about PHID basics, at least. In practice, our data model has been really tidy and consistent, and we don't use any of the auxiliary data in this table (or even write it). The handle abstraction is powerful and covers essentially all of the useful data in the app, and we have human-readable types in the keys. So I don't think we have a real need here, and this table isn't serving it if we do.
    - **Uniqueness**: With a unique key, we can be sure they're unique, even if we get astronomically unlucky and get a collision. But every table we use them in has a unique key anyway. So we actually get pretty much nothing here, except maybe some vague guarantee that we won't reallocate a key later if the original object is deleted. But it's hard to imagine any install will ever have a collision, given that the key space is 36^20 per object type.
  - We also currently use PHIDs and Users in tests sometimes. This is silly and can break (see D2461).
  - Drop the PHID database.
  - Introduce a "Harbormaster" database (the eventual CI tool, after Drydock).
  - Add a scratch table to the Harbormaster database for doing unit test meta-tests.
  - Now, PHID generation does no writes, and unit tests are isolated from the application.
  - @csilvers: This should slightly improve the performance of the large query-bound tail in D2457.

Test Plan: Ran unit tests. Ran storage upgrade.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: csilvers, aran, nh, edward

Differential Revision: https://secure.phabricator.com/D2466
2012-05-20 14:46:01 -07:00
epriestley
95017e14e9 Minor, derrrp. 2012-05-07 10:34:07 -07:00
epriestley
58efe52b3a Minor, fix method call fatal. 2012-05-07 10:32:29 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
vrana
416e4e7b67 Allowing setting user status
Summary:
I will use it for highlighting users which are not currently available.

Maybe I will also use it in the nagging tool.

I don't plan creating a UI for it as API is currently enough for us.
Maybe I will visualize it at /calendar/ later.

I plan creating `user.deletestatus` method when this one will be done.

Test Plan:
`storage upgrade`
Call Conduit `user.addstatus`.
Verify DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2382
2012-05-03 18:24:30 -07:00
vrana
73c82e5a94 Display holidays
Summary:
We will need it for two purposes:

- Status tool.
- Nagging tool - @aran suggested using "3 business days" and I don't want it to fall on New Year's Eve or such.

I don't plan working on any interface for editing this as this kind of data should be always imported.

Test Plan:
`bin/storage upgrade`
`scripts/calendar/import_us_holidays.php`
/calendar/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2375
2012-05-03 09:22:52 -07:00
epriestley
68b597ff75 SQL Patch Management: SQL Changes
Summary:
Splits out the SQL changes. These are most of the changes, but primarily mechanical:

  - Moved "initialize.sql" to "0000.legacy.sql" and partially reverted to an older version, such that patches 0000 + 000 + 001 + ... + 137 put us in the right state when applied sequentially.
  - Removed "create database" commands from all SQL. These are handled by separate DB patches now, so we have the data to do operations like "storage databases" (list databases) and "storage destroy" (drop databases).
  - Removed "phabricator_" namespace from all SQL, and replaced with "{$NAMESPACE}_" token so we can namespace databases.
  - Shortened some column lengths so patches apply correctly if originally created as InnoDB; also a few similar tweaks elsewhere.

Test Plan: See D2323 for discussion and test plan.

Reviewers: edward, vrana, btrahan, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T140, T345

Differential Revision: https://secure.phabricator.com/D2329
2012-04-30 07:53:53 -07:00
David Reuss
42b1c73f41 Allow CC's/Auditors added to audits
Test Plan:
Added CC's/Auditors, clicked the form elements, and saw correct
behaviour. Verified that metadata was present in the detail table.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, 20after4, Koolvin

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D2002
2012-04-23 13:50:25 -07:00
vrana
549a86cd96 Add sex
Summary:
We will need it for intl.

I've put it to User instead of UserProfile to be easier accessible.

Test Plan:
Apply SQL patch.
Change sex to Male.
Change sex to Unknown.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2287
2012-04-19 16:05:25 -07:00
vrana
1f2cf78c1b Display committed date in Revision Status field
Summary:
This is slightly more complicated for this reason:

- We don't set `dateCommitted` for normal commits, only for markcommitted.
-- We need to add this date to old revisions now.

Test Plan:
Reparse a revision - commit date was set.
Conduit `markcommitted` - commit date was set.
Run SQL script.
Display closed revision.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2282
2012-04-19 15:05:09 -07:00
vrana
204b6694af Improve performance of empty search
Summary:
Searching for empty query kills us because whole `search_document` table is read in this case.

This diff adds an index just for this query.

Other solution would be to disable searching for empty string. But it can be actually useful (listing newest documents of any type).

Test Plan:
  lang=sql
  EXPLAIN SELECT document.phid, document.documentType, document.documentTitle, document.documentCreated
  FROM `search_document` document
  GROUP BY document.phid
  ORDER BY documentCreated DESC
  LIMIT 0, 21;

Reviewers: epriestley

Reviewed By: epriestley

CC: jungejason, aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2277
2012-04-18 14:51:09 -07:00
vrana
1f1c7a34b7 Improve image macros
Summary:
Couple of small improvements:

- Delete `randomon` macro.
- Make name unique (deleting current conflicts randomly).
- Image macro must be alone on the line.
- Filter by name.

Test Plan:
Run SQL.
/file/macro/
/file/macro/?name=imagemacro
Try to create conflicting name.
Write this comment:

  Test imagemacro.
  imagemacro

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: epriestley, Koolvin

Differential Revision: https://secure.phabricator.com/D2230
2012-04-17 12:16:58 -07:00
Nick Harper
1ea8bd3ab7 Fix patch 131 for db with lots of revisions
Summary:
The old version of this loads all differential revisions at once, but that much
can't all be loaded into memory when there are close to 500,000 revisions. This
diff splits up loading the revisions.

Test Plan: Ran this to run the migration in our install

Reviewers: jungejason, epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2243
2012-04-16 12:35:48 -07:00
Bob Trahan
51418900f7 Phame V1 - Phabricator blogging software
Summary:
'cuz we need to be phamous!

V1 feature set

- posts
-- standard thing you'd expect - a title and a remarkup-powered body and...
-- "phame" title - a short string that can be used to reference the story. this gets auto-updated when you mess with the title.
-- configuration - for now, do you want Facebook, Disqus or no comments? this is a per-post thing but feeds from an instance-wide configuration

Please do toss out any must have features or changes.

Test Plan: played around with this bad boy like whoa

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, vrana

Maniphest Tasks: T1111

Differential Revision: https://secure.phabricator.com/D2202
2012-04-12 13:09:04 -07:00
epriestley
fe9ba6bc67 Improve DifferentialRevisionQuery and add the ability to query by arcanist project
Summary:
  - We currently post-filter by branches, but should do this in SQL. See T799.
  - We currently identify branch-name-matches as being in the working copy even if they belong to a different project (e.g., two different projects with commits on the branch "master"). See T1100.
  - Denormalize branch and project information into DifferentialRevision.
  - Expose project information in the API.

Test Plan: Ran conduit API queries with branches and arc project IDs, got reasonable results.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1100, T799

Differential Revision: https://secure.phabricator.com/D2190
2012-04-10 12:51:34 -07:00
epriestley
488b1cf641 Allow Maniphest queries to be saved
Summary:
There have been a couple of requests for this since bookmarks are "out this year like woah" and "totally uncool dude".

Allow users to save named custom queries and make them the /maniphest/ default if they so desire.

A little messy. :/

Test Plan: Saved, edited, deleted custom queries. Made custom query default; made 'no default' default. Verified default behavior. Issued a modified search from a custom query.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley, 20after4

Maniphest Tasks: T923, T1034

Differential Revision: https://secure.phabricator.com/D1964
2012-04-10 09:46:04 -07:00
vrana
32d2395a45 Unify links to www.phabricator.com and phabricator.com
Test Plan:
  scripts/sql/upgrade_schema.php

Verify links at /directory/2/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Maniphest Tasks: T1096

Differential Revision: https://secure.phabricator.com/D2172
2012-04-09 14:32:03 -07:00
vrana
6aa729b1c9 Support MySQL 5.5
Summary:
`116.utf8.sql` throws this under MySQL 5.5:

> Column length too big for column 'keyBody' (max = 21845); use BLOB or TEXT instead

I guess that's because MySQL 5.5 changed maximum length of UTF-8 character from 3 bytes to 4.

I've updated also `116.utf8.sql` for people with new installs.

Test Plan:
  upgrade_schema.php

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2117
2012-04-06 09:55:58 -07:00
epriestley
877cb136e8 Add an assocations-like "Edges" framework
Summary:
We have a lot of cases where we store object relationships, but it's all kind of messy and custom. Some particular problems:

  - We go to great lengths to enforce order stability in Differential revisions, but the implementation is complex and inelegant.
  - Some relationships are stored on-object, so we can't pull the inverses easily. For example, Maniphest shows child tasks but not parent tasks.
  - I want to add more of these and don't want to continue building custom stuff.
  - UIs like the "attach stuff to other stuff" UI need custom branches for each object type.
  - Stuff like "allow commits to close tasks" is notrivial because of nonstandard metadata storage.

Provide an association-like "edge" framework to fix these problems. This is nearly identical to associations, with a few differences:

  - I put edge metadata in a separate table and don't load it by default, to keep edge rows small and allow large metadata if necessary. The on-edge metadata seemed to get abused a lot at Facebook.
  - I put a 'seq' column on the edges to ensure they have an explicit, stable ordering within a source and type.

This isn't actually used anywhere yet, but my first target is attaching commits to tasks for T904.

Test Plan: Made a mock page that used Editor and Query. Verified adding and removing edges, overwriting edges, writing and loading edge data, sequence number generation.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, 20after4

Differential Revision: https://secure.phabricator.com/D2088
2012-04-04 15:30:21 -07:00
vrana
5885764728 Allow IPv6
Test Plan:
/people/logs/
Search for `2620:0:1cfe:`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, jungejason

Differential Revision: https://secure.phabricator.com/D2089
2012-04-03 11:36:50 -07:00
epriestley
b028920a5e Minor, subpriority default previously reverse ordered eveything
Auditors: btrahan
2012-04-02 12:26:50 -07:00
epriestley
e7853e4801 Allow tasks to be subprioritized by drag-and-drop
Summary:
Like the title says, similar to Facebook Tasks.

Not sure how I really feel about this, but I guess it's kind of OK? I never used
this feature in Facebook Tasks but I think some people like it.

The drag-and-drop to repri across priorities feels okayish.

Because subpriority is a double and we just split the difference when
reprioritizing, you lose ~a bit of precision every time you repri two tasks
against each other and so you can break it by swapping the priorities of two
tasks ~50 times. This case is pretty silly and pathological. We can add some
code to deal with this at some point if necessary.

I think this also fixes the whacky task layout widths once and for all.

(There are a couple of minor UI glitches like headers not vanishing and header
counts not updating that I'm not fixing because I am lazy.)

Test Plan: Dragged and dropped tasks around.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley, mgummelt

Maniphest Tasks: T859

Differential Revision: https://secure.phabricator.com/D1731
2012-04-02 12:12:04 -07:00
epriestley
698ec68327 General Herald refactoring pass
Summary:
**Who can delete global rules?**: I discussed this with @jungejason. The current behavior is that the rule author or any administrator can delete a global rule, but this
isn't consistent with who can edit a rule (anyone) and doesn't really make much sense (it's an artifact of the global/personal split). I proposed that anyone can delete a
rule but we don't actually delete them, and log the deletion. However, when it came time to actually write the code for this I backed off a bit and continued actually
deleting the rules -- I think this does a reasonable job of balancing accountability with complexity. So the new impelmentation is:

  - Personal rules can be deleted only by their owners.
  - Global rules can be deleted by any user.
  - All deletes are logged.
  - Logs are more detailed.
  - All logged actions can be viewed in aggregate.

**Minor Cleanup**

  - Merged `HomeController` and `AllController`.
  - Moved most queries to Query classes.
  - Use AphrontFormSelectControl::renderSelectTag() where appropriate (this is a fairly recent addition).
  - Use an AphrontErrorView to render the dry run notice (this didn't exist when I ported).
  - Reenable some transaction code (this works again now).
  - Removed the ability for admins to change rule authors (this was a little buggy, messy, and doesn't make tons of sense after the personal/global rule split).
  - Rules which depend on other rules now display the right options (all global rules, all your personal rules for personal rules).
  - Fix a bug in AphrontTableView where the "no data" cell would be rendered too wide if some columns are not visible.
  - Allow selectFilter() in AphrontNavFilterView to be called without a 'default' argument.

Test Plan:
  - Browsed, created, edited, deleted personal and gules.
  - Verified generated logs.
  - Did some dry runs.
  - Verified transcript list and transcript details.
  - Created/edited all/any rules; created/edited once/every time rules.
  - Filtered admin views by users.

Reviewers: jungejason, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2040
2012-03-30 10:49:55 -07:00
epriestley
7ad68e63e4 Add "Flags" to allow users to collect the things they love
Summary:
Flags are a personal collection of things you want to take a look at later. You can use several different colors and add notes.

Not really sure if this is actually a good idea or not but it was easy to build.

Planned features:

  - Allow Herald rules to add flags.
  - In the "edit flag" dialog, have a "[x] Subscribe Me" checkbox that CCs you.
  - Support Diffusion.
  - Support Phriction.
  - Always show flags on an object if you have them (in every view)?
  - Edit dialog feels a little heavy?
  - More filtering in /flag/ tool.
  - Add a top-level links somewhere?

Test Plan: Added, edited and removed flags from things. Viewed flags in flag view.

Reviewers: aran, btrahan

Reviewed By: btrahan

CC: aran, epriestley, Koolvin

Maniphest Tasks: T1041

Differential Revision: https://secure.phabricator.com/D2024
2012-03-27 16:22:40 -07:00
epriestley
914f044b62 More Drydock Stuff
Summary:
  - Still really really rough.
  - Adds a full synchronous mode for debugging.
  - Adds some logging.
  - It can now allocate EC2 machines and put webroots on them in a hacky, terrible way.
  - Adds a base query class.

Test Plan: oh hey look a test page? http://ec2-50-18-65-151.us-west-1.compute.amazonaws.com:2011/

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D2026
2012-03-26 20:54:26 -07:00
epriestley
0a4cbdff5e Straighten out Diffusion file integration
Summary:
This is in preparation for getting the "View Options" dropdown working on audits.

  - Use Files to serve raw data so we get all the security benefits of the alternate file domain. Although the difficulty of exploiting this is high (you need commit access to the repo) there's no reason to leave it dangling.
  - Add a "contentHash" to Files so we can lookup files by content rather than adding some weird linker table. We can do other things with this later, potentially.
  - Don't use 'data' URIs since they're crazy and we can just link to the file URI.
  - When showing a binary file or an image, don't give options like "show highlighted text with blame" or "edit in external editor" since they don't make any sense.
  - Use the existing infrastructure to figure out if things are images or binaries instead of an ad-hoc thing in this class.

Test Plan: Looked at text, image and binary files in Diffusion. Verified we reuse existing files if we've already generated them.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1899
2012-03-19 19:52:24 -07:00
epriestley
900190b2fe Add inline comments to Diffusion/Audit
Summary:
  - Add inline comments to Audits, like Differential.
  - Creates new storage for the comments in the Audits database.
  - Creates a new PhabricatorAuditInlineComment class, similar to DifferentialInlineComment.
  - Defines an Interface which Differential and Audit comments conform to.
  - Makes consumers of DifferentialInlineComments consume objects which implement that interface instead.
  - Adds save

NOTE: Some features are still missing! Wanted to cut this off before it got crazy:

  - Inline comments aren't shown in the main comment list.
  - Inline comments aren't shown in the emails.
  - Inline comments aren't previewed.

I'll followup with those but this was getting pretty big.

@vrana, does the SQL change look correct?

Test Plan:
  - Created, edited, deleted, replied to, reloaded and saved inline comments in Diffusion, on the left and right side of diffs.
  - Created, edited, deleted, replied to, reloaded and saved inline comments in Differentila, on the left and right side of primary and diff-versus-diff diffs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1898
2012-03-14 12:56:01 -07:00
vrana
f4be64792e Drop unused column
Summary: https://secure.phabricator.com/D1830?id=3203#inline-2058

Test Plan:
Run script on db containing this column.
Run script on db not containing this column.
Visit /repository/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1854
2012-03-12 12:11:36 -07:00
vrana
4c1e356658 Convert database to UTF-8
Summary: This is the script used for conversion: P319

Test Plan:
Update diff with UTF-8 characters in description.
`sql/upgrade_schema.php`
Verify data in DB and that it looks good on web.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T327

Differential Revision: https://secure.phabricator.com/D1830
2012-03-12 12:11:02 -07:00
vrana
d5bf30bb48 Prepare database for UTF-8
Summary: D1830#8

Test Plan:
`scripts/sql/upgrade_schema.php`
Try adding duplicate SSH Public Key - failed.
Try adding new SSH Public Key - succeeded.

Reviewers: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1852
2012-03-09 18:56:22 -08:00
epriestley
d94129b739 Migrate "PhabricatorOwnersPackageCommitRelationship" to "PhabricatorRepositoryAuditRequest"
Summary:
  - Move table to Repository, since we have no Owners joins in the application anymore but would like to do a Repository join.
  - Rename "packagePHID" to "auditorPHID", since this column may contain package, project, or user PHIDs.

Test Plan:
  - Browsed Owners, Audit, and Differential interfaces to the Audit tool.
  - Made comments and state changes.
  - Ran "reparse.php --herald --owners" on several commits.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley, nh, vrana

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1787
2012-03-05 13:17:29 -08:00
epriestley
1e4e3d1fef Minor, prevent patch 113 from breaking in some environments(?) 2012-03-05 12:34:33 -08:00
vrana
f5f7987013 Revert rP87c60abbd02d, apply D1772
Test Plan:
Apply SQL patch.
Visit /differential/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1781
2012-03-05 11:04:55 -08:00
Bob Trahan
0327a5fc69 OAuthServer polish and random sauce
Summary:
This diff makes the OAuthServer more compliant with the spec by
- making it return well-formatted error codes with error types from the spec.
- making it respect the "state" variable, which is a transparent variable the
client passes and the server passes back
- making it be super, duper compliant with respect to redirect uris
-- if specified in authorization step, check if its valid relative to the client
registered URI and if so save it
-- if specified in authorization step, check if its been specified in the access
step and error if it doesn't match or doesn't exist
-- note we don't make any use of it in the access step which seems strange but
hey, that's what the spec says!
This diff makes the OAuthServer suck less by
- making the "cancel" button do something in the user authorization flow
- making the client list view and client edit view be a bit more usable around
client secrets
- fixing a few bugs I managed to introduce along the way

Test Plan:
- create a test phabricator client, updated my conf, and then linked and
unlinked phabricator to itself
- wrote some tests for PhabricatorOAuthServer -- they pass!
-- these validate the various validate URI checks
- tried a few important authorization calls
--
http://phabricator.dev/oauthserver/auth/?client_id=X&state=test&redirect_uri=http://www.evil.com
--- verified error'd from mismatching redirect uri's
--- verified state parameter in response
--- verified did not redirect to client redirect uri
-- http://phabricator.dev/oauthserver/auth/?client_id=X w/ existing
authorization
--- got redirected to proper client url with error that response_type not
specified
-- http://phabricator.dev/oauthserver/auth/?client_id=X&response_type=code w/
existing authorization
--- got redirected to proper client url with pertinent code!
- tried a few important access calls
-- verified appropriate errors if missing any required parameters
-- verified good access code with appropriate other variables resulted in an
access token
- verified that if redirect_uri set correctly in authorization required for
access and errors if differs at all / only succeeds if exactly the same

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, ajtrichards

Maniphest Tasks: T889, T906, T897

Differential Revision: https://secure.phabricator.com/D1727
2012-03-01 14:46:18 -08:00
epriestley
8a0a00f118 Make PhabricatorRepositoryCommmit schema changes for audit
Summary:
  - Add a proper mailKey field to make these things mailable. Backfill all
existing objects.
  - Denormalize authorPHID to the commit object so we can query by it
efficiently in a future diff. We currently use the search engine to drive
"commits by author" but that's not so good for audit, which needs more
constraints.
  - Add an overall audit status field so we can efficiently query "commits that
needs your attention".
  - Add enough code to convince myself that these fields are basically
reasonable and work correctly.

Test Plan:
  - Ran schema upgrades. Checked database state afterward.
  - Ran "reparse.php --owners --herald" to verify worker changes.
  - Looked at a commit, altered aggregate status via audits / reparse.php,
verified it responded correctly.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran, epriestley, nh

Maniphest Tasks: T904

Differential Revision: https://secure.phabricator.com/D1706
2012-02-28 21:06:34 -08:00
Bob Trahan
3c4070a168 OAuth Server -- add controllers to RUD client authorizations and CRUD clients
Summary:
beyond the title, this diff tweaks the test console to have a bit more
functionality.  also makes a small change to CSS for AphrontFormControlMarkup,
which IMO fixes a display issue on
https://secure.phabricator.com/settings/page/profile/ where the Profile URI is
all up in the air and whatnot

I think this is missing pagination.  I am getting tired of the size though and
will add later.  See T905.

Test Plan:
viewed, updated and deleted client authorizations.  viewed, created,
updated and deleted clients

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T849, T850, T848

Differential Revision: https://secure.phabricator.com/D1683
2012-02-24 14:56:18 -08:00
Bob Trahan
af295e0b26 OAuth Server enhancements -- more complete access token response and groundwork
for scope

Summary:
this patch makes the access token response "complete" relative to spec by
returning when it expires AND that the token_type is in fact 'Bearer'.

This patch also lays the groundwork for scope by fixing the underlying data
model and adding the first scope checks for "offline_access" relative to expires
and the "whoami" method.   Further, conduit is augmented to open up individual
methods for access via OAuth generally to enable "whoami" access.   There's also
a tidy little scope class to keep track of all the various scopes we plan to
have as well as strings for display (T849 - work undone)

Somewhat of a hack but Conduit methods by default have SCOPE_NOT_ACCESSIBLE.  We
then don't even bother with the OAuth stuff within conduit if we're not supposed
to be accessing the method via Conduit.   Felt relatively clean to me in terms
of additional code complexity, etc.

Next up ends up being T848 (scope in OAuth) and T849 (let user's authorize
clients for specific scopes which kinds of needs T850).  There's also a bunch of
work that needs to be done to return the appropriate, well-formatted error
codes.  All in due time...!

Test Plan:
verified that an access_token with no scope doesn't let me see
anything anymore.  :(  verified that access_tokens made awhile ago expire.  :(

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T888, T848

Differential Revision: https://secure.phabricator.com/D1657
2012-02-21 16:33:06 -08:00
Bob Trahan
7a3f33b5c2 OAuth - Phabricator OAuth server and Phabricator client for new Phabricator OAuth Server
Summary:
adds a Phabricator OAuth server, which has three big commands:
 - auth - allows $user to authorize a given client or application.  if $user has already authorized, it hands an authoization code back to $redirect_uri
 - token - given a valid authorization code, this command returns an authorization token
 - whoami - Conduit.whoami, all nice and purdy relative to the oauth server.
Also has a "test" handler, which I used to create some test data.  T850 will
delete this as it adds the ability to create this data in the Phabricator
product.

This diff also adds the corresponding client in Phabricator for the Phabricator
OAuth Server.  (Note that clients are known as "providers" in the Phabricator
codebase but client makes more sense relative to the server nomenclature)

Also, related to make this work well
 - clean up the diagnostics page by variabilizing the provider-specific
information and extending the provider classes as appropriate.
 - augment Conduit.whoami for more full-featured OAuth support, at least where
the Phabricator client is concerned

What's missing here...   See T844, T848, T849, T850, and T852.

Test Plan:
- created a dummy client via the test handler.   setup development.conf to have
have proper variables for this dummy client.  went through authorization and
de-authorization flows
- viewed the diagnostics page for all known oauth providers and saw
provider-specific debugging information

Reviewers: epriestley

CC: aran, epriestley

Maniphest Tasks: T44, T797

Differential Revision: https://secure.phabricator.com/D1595
2012-02-19 14:00:13 -08:00
epriestley
7200040479 Add a basic chatlog
Summary:
This is pretty simple and unpolished, but it's getting pretty big and it seems
like a reasonable starting point.

  - Log chat in various "channels".
  - Conduit record and query methods.
  - IRCBot integration for IRC logging

Major TODO:

  - Web UI is really unpolished and has no search, paging, anchor-linking, etc.
Basically all presentation stuff, though.
  - I think the bot should have a map of channels to log with channel aliases?
  - The "channels" should probably be in a separate table.
  - The "authors" should probably be correlated to Phabricator accounts somehow,
where possible.

Test Plan: Used phabotlocal to log #phabricator.

Reviewers: kdeggelman, btrahan, Koolvin

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T837

Differential Revision: https://secure.phabricator.com/D1625
2012-02-17 10:21:38 -08:00
epriestley
d7bb1f183c Trim "\n" from mimeType field in phabricator_file.file
Summary: See patch comment, which explains the situation.

Test Plan: Ran upgrade, verified mimeType field is \n-free in database.

Reviewers: davidreuss, btrahan

Reviewed By: davidreuss

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1624
2012-02-16 07:25:56 -08:00
epriestley
35c5852d3f Add a safeguard against multiple patches with the same version
Summary:
I accidentally added two "104" patches. This actually works OK for the most part
but is fundamentally bad and wrong.

Merge the patches (installs applied both as "104", so we can't move one to
"105") and add a safeguard.

Test Plan: Ran upgrade_schema.php with two "104" patches, got error'd. Ran
without, got successs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1614
2012-02-14 16:24:02 -08:00