1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-05 05:02:44 +01:00
Commit graph

210 commits

Author SHA1 Message Date
epriestley
ba7723d905 Modernize Macro application
Summary: Adds feed, email, notifications, comments, partial editing, subscriptions, enable/disable, flags and crumbs to Macro.

Test Plan:
{F26839}
{F26840}
{F26841}
{F26842}
{F26843}
{F26844}
{F26845}

Reviewers: vrana, btrahan, chad

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2157, T175, T2104

Differential Revision: https://secure.phabricator.com/D4141
2012-12-11 14:01:03 -08:00
epriestley
7b6fa0db12 Genericize transactions in Pholio
Summary:
Split Pholio's transaction implementation into generic and application-specific parts. Moves us toward generic transactions, with support for:

  - Editing and deleting comments.
  - Setting visibility of individual comments (I'm not a fan of this feature but we'll see).

I want to move everything to a more generic piece of infrastructure but there's very little they can share right now so adding transactions to, e.g., Paste or Macros (T2157) means massive amounts of similar code.

Tons of work left to do here, but I think it basically works. Here's a screenshot:

{F26820}

Test Plan: Made transactions in Pholio.

Reviewers: btrahan, vrana, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2104

Differential Revision: https://secure.phabricator.com/D4136
2012-12-11 13:59:20 -08:00
vrana
4f615ad2a9 Allow excluding paths from package
Summary: Resolves T2149.

Test Plan:
  $ bin/storage upgrade

# /owners/ - saw +
# /owners/package/1/ - saw +
# /owners/edit/1/ - added exclude paths, saw correct e-mail
# /rPabc123 - included paths are still highlighted and excluded not
# /owners/view/search/?path=/included/ - found
# /owners/view/search/?path=/excluded/ - not found
# owners.query - path: /included/
# owners.query - path: /excluded/
# new unit test

  PhabricatorOwnersPackage::loadAffectedPackages(
    $repository,
    array('/excluded/b.php'));

  PhabricatorOwnersPackage::loadAffectedPackages(
    $repository,
    array('/included/a.php'));

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2149

Differential Revision: https://secure.phabricator.com/D4102
2012-12-07 16:33:16 -08:00
epriestley
fc9ad37b26 Add very basic scaffolding for Pholio
Summary:
I'm not going to land this until it's a bit more fleshed out since it would just confuse users, but this is probably more reviewable as a few diffs adding a couple features than one ULTRA-diff adding everything. Implement application basics for Pholio. This does more or less nothing, but adds storage, subscribe, flag, markup, indexing, query basics, PHIDs, handle loads, a couple of realy really basic controllers, etc.
Basic hierarchy is:

  - **Moleskine**: Top-level object like a Differential Revision, like "Ponder Feed Ideas".
  - **Image**: Each Moleskine has one or more images, like the unexpanded / expanded / mobile / empty states of feed.
  - **Transaction**: Comment or edit, like Maniphest. I generally want to move most apps to a transaction model so we can log edits.
  - **PixelComment**: Equivalent of an inline comment.

Test Plan: Created a fake object and viewed it.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran, davidreuss

Maniphest Tasks: T2097

Differential Revision: https://secure.phabricator.com/D3817
2012-11-21 17:22:36 -08:00
epriestley
ee2e85a0bb Fix several migration issues with the Task/Counter patch
Summary:
People hit three issues with D3914:

  - As per T2059, we applied a schema change from a `.php` patch, which currently does not work if you use a different user to make schema changes than for normal use.
    - Since the change in question is idempotent, just move it to a `.sql` patch. We'll follow up in T2059 and fix it properly.
  - Rogue daemons at several installs used old code (expecting autoincrement) to insert into the new table (no autoincrement), thereby creating tasks with ID 0.
    - Rename the table so they'll fail.
    - This also makes the code a little more consistent.
  - Some installs now have tasks with ID 0.
    - Use checks against null rather than against 0 so we can process these tasks.

The major issues this fixes are the schema upgrade failure in T2059, and the infinite loops in T2072 and elsewhere.

This isn't really a fully statisfactory fix. I'll discuss some next steps in T2072.

Test Plan: Created new tasks via MetaMTA/Differential. Ran tasks with `phd debug taskmaster`. Inserted a task 0 and verified it ran and archived correctly.

Reviewers: btrahan, vrana, nh

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2072, T2059

Differential Revision: https://secure.phabricator.com/D3973
2012-11-16 10:19:22 -08:00
vrana
23a046b3cd Allow saving lint errors to database
Summary: This saves lint errors to the path change of current commit. It requires pushed revision. It doesn't save difference from previous commit mentioned in T2038#comment-4 - I don't plan doing it after all, everything would be much more complicated and the amount of data saved with this approach isn't that bad.

Test Plan: Applied patch, ran script, verified DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3899
2012-11-08 15:39:43 -08:00
vrana
6cfc15ad6c Make Lisk counters patch more robust
Summary:
We need to revert this patch and we will need to re-apply it later.
We can't drop the table and delete these rows as we need to run both versions for a temporary period.

Test Plan: Applied it.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3924
2012-11-08 11:36:50 -08:00
Bob Trahan
9966af50dd Delete PhabricatorRemarkupRuleProxyImage
Summary: don't need it now that uploading files is so easy. Plus it made for some buggy jonx if / when there were bad image links coupled with caching. In theory this is a lot less pretty though if folks linked to a bunch of files served elsewhere using images.

Test Plan: http://does-not-exist.com/imaginary.jpg rendered as a link!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2000

Differential Revision: https://secure.phabricator.com/D3908
2012-11-07 14:31:43 -08:00
epriestley
7332599e03 Provide an IDS_COUNTER mechanism for ID assignment
Summary: See D3912 for discussion. InnoDB may reuse autoincrement IDs after restart; provide a way to avoid it.

Test Plan: Unit tests. Scheduled and executed tasks through `drydock lease --type host` and `phd debug taskmaster`.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3914
2012-11-07 13:33:07 -08:00
vrana
c6503f019f Start Ponder questions from 11
Summary: We don't link Q1 - Q4.

Test Plan: Created the table, insterted row, verified that the id is 11.

Reviewers: pieter, epriestley

Reviewed By: pieter

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3891
2012-11-05 13:21:24 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
89b37f0357 Make various Drydock improvements
Summary:
Tightens up a bunch of stuff:

  - In `drydock lease`, pull and print logs so the user can see what's happening.
  - Remove `DrydockAllocator`, which was a dumb class that did nothing. Move the tiny amount of logic it held directly to `DrydockLease`.
  - Move `resourceType` from worker task metadata directly to `DrydockLease`. Other things (like the web UI) can be more informative with this information available.
  - Pass leases to `allocateResource()`. We always allocate in response to a lease activation request, and the lease often has vital information. This also allows us to associate logs with leases correctly.

Test Plan: Ran `drydock lease --type host` and saw it perform a host allocation in EC2.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3870
2012-11-01 16:53:17 -07:00
epriestley
f0fdcf1a51 Undumb the Drydock resource allocator pipeline
Summary:
This was the major goal of D3859/D3855, and to a lesser degree D3854/D3852.

As Drydock is allocating a resource, it may need to allocate other resources first. For example, if it's allocating a working copy, it may need to allocate a host first.

Currently, we have the process basically queue up the allocation (insert a task into the queue) and sleep() until it finishes. This is problematic for a bunch of reasons, but the major one is that if allocation takes more resources (host, port, machine, DNS) than you have daemons, they could all end up sleeping and waiting for some other daemon to do their work. This is really stupid. Even if you only take up some of them, you're spending slots sleeping when you could be doing useful work.

To partially get around this and make the CLI experience less dumb, there's this goofy `synchronous` flag that gets passed around everywhere and pushes the workflow through a pile of special cases. Basically the `synchronous` flag causes us to do everything in-process. But this is dumb too because we'd rather do things in parallel if we can, and we have to have a lot of special case code to make it work at all.

Get rid of all of this. Instead of sleep()ing, try to work on the tasks that need to be worked on. If another daemon grabbed them already that's fine, but in the worst case we just gracefully degrade and do everything in process. So we get the best of both worlds: if we have parallelizable tasks and free daemons, things will execute in parallel. If we have nonparallelizable tasks or no free daemons, things will execute in process.

Test Plan: Ran `drydock_control.php --trace` and saw it perform cascading allocations without sleeping or special casing.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3861
2012-11-01 11:30:42 -07:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
Jakub Vrana
01de3dff81 Allow using StorageFixtureScopeGuard on Windows 2012-10-24 13:59:22 -07:00
Bob Trahan
60466d3bcc Create a status tool by giving /calendar/ some teeth
Summary: you can now add, edit, and delete status events. also added a "description" to status events and surface it in the big calendar view on mouse hover. some refactoring changes as well to make validation logic centralized within the storage class.

Test Plan: added, edited, deleted. yay.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T407

Differential Revision: https://secure.phabricator.com/D3810
2012-10-24 13:22:24 -07:00
epriestley
a50b8e39b1 Make posts 1:1 with blogs and implement policy controls
Summary:
This leaves the UI in a pretty rough state, but implements blog policy controls and queries, and 1:1 relationships between posts and blogs. Needs a bunch more cleanup but seemed like an okayish breaking point in terms of cohesiveness.

Posts have these rules:

  - Drafts are visible only to the author.
  - Published posts are visible to anyone who can see the blog they appear on.
  - Posts are only editable by the author.

...so we don't need any special policy UI or state to accommodate these rules.

Posts may have no blog if they're grandfathered in or you write a post to a blog and then lose the ability to see the blog. This is the messiest edge case -- specifically:

  - You write a post to blog A.
  - You publish the post.
  - I edit the "Visible To:" for blog A and set it to exclude you.

What we do in this case is let you see the post in "My Posts", but you can no longer see the blog and you'll see the post as not being part of a blog. We can maybe give you some UI to let you move it later or something.

Test Plan: Hit all (I think?) of the interfaces without issues. Definitely some UI problems still right now.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3694
2012-10-15 14:50:04 -07:00
epriestley
dbcf2e44e8 Make PhameBlogs respect policies
Summary:
Adds "can view" and "can edit" policies to blogs. Replaces "bloggers" with "can join".

This doesn't fully remove "bloggers" because I didn't want this to get too crazy/huge.

Test Plan: Created, edited, deleted blogs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3693
2012-10-15 14:49:52 -07:00
epriestley
2532cb9613 Add mail keys to Ponder questions
Summary:
We need to go slightly farther to stub reply handler functionality for Ponder in at least some configurations, where we rely on the presence of a unique random key to generate per-object or per-object+user reply addresses.

This should probably be formalized in an interface since it's currently pretty ad-hoc.

Test Plan:
  - Made comments in Ponder under a per-user email configuration.
  - Ran migration, verified mail keys were generated.
  - Ran migration again (with --apply), verified existing questions were skipped.
  - Created a new question, verified mail key generation.

Reviewers: pieter

Reviewed By: pieter

CC: aran

Maniphest Tasks: T1873

Differential Revision: https://secure.phabricator.com/D3665
2012-10-08 20:14:58 -07:00
vrana
cbde56cdce Properly create xhpast database
Summary:
It isn't deleted by `storage destroy`.

This should be a no-op on current storage because we execute `CREATE DATABASE IF NOT EXISTS`.

Test Plan:
  $ bin/storage destroy --dryrun

Reviewers: nh, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3659
2012-10-08 16:09:20 -07:00
Bob Trahan
9e1b643896 Phame - allow blogs to specify custom URIs
Summary: this then enables people to create blog.theircompany.com. And for us, blog.phacility.com...!

Test Plan:
 - created custom URIs of various goodness and verified the error messages were sensical.
 - verified if "false" in configuration then custom uri stuff disappears

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3542
2012-09-30 17:10:27 -07:00
vrana
d119ac672f Remember action in Differential comment draft
Summary:
It happens to me quite often that I leave the window with revision (by closing it or by visiting a link from it).
When I return then the comment draft is there so I clowncopterize it but forget that I wanted to take some other action than Comment.

Test Plan: Selected "Add Reviewers", added some reviewers, closed the window, opened it - the action and reviewers were still there.

Reviewers: btrahan, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3530
2012-09-21 13:05:09 -07:00
Nick Harper
5978bbfc64 Do sampled profiling of requests
Summary:
People have occasionally complained about phabricator being slow. We have
the access log to look at to see when slowness happens, but it doesn't tell
us much about why it happened. Since it's usually a sporadic issue that's
reported, it's hard to reproduce and then profile. This change will allow us
to collect sampled profiles so we can look at them when slowness occurs.

Test Plan:
checking that sampling works correctly:
- set rate to 0; do several page loads; check no new entries in table
- set rate to 1; check that there's a new row in the table for each page load
- set rate to 10; check that some requests write to table and some don't
check new ui for samples:
- load /xhprof/list/all/, see a list with a lot of samples
- load /xhprof/list/sampled/, see only sampled runs
- load /xhprof/list/manual/, see only non-sampled runs
- load /xhprof/list/my-runs/, se only my manual runs

Reviewers: vrana, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3458
2012-09-17 10:53:45 -07:00
epriestley
b39175342d Add paste policy storage
Summary: Add storage to Pastes for view policies.

Test Plan: Set policies on pastes, see next diff.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3474
2012-09-13 10:11:14 -07:00
Pieter Hooimeijer
5883b4f50c adding comments to ponder
Summary: This is pretty spartan, but it does the job.

Test Plan:
Patch, update storage, add some comment
to your favorite question or answer.

Reviewers: nh, vrana, epriestley

Reviewed By: epriestley

CC: aran, Korvin, starruler, syrneus, me.here, victorzarate7

Maniphest Tasks: T1645

Differential Revision: https://secure.phabricator.com/D3471
2012-09-11 12:13:20 -07:00
vrana
f770900983 Save edge type as number
Summary: We use numbers here and I see no reason for strings.

Test Plan:
  $ bin/storage upgrade

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3303
2012-08-16 14:43:03 -07:00
Nick Harper
3908f7db2e Show list of non-exited daemons
Summary: This is arguably a more useful view than listing all daemons.

Test Plan: Looked at list, only saw daemons that haven't exited

Reviewers: vrana, epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3286
2012-08-14 18:01:15 -07:00
epriestley
bd0be1c650 Add View, Edit and Join policies to PhabricatorProject
Summary:
  - In ProjectQuery, always load the viewer's membership in the project because we need it to perform a CAN_VIEW test.
  - Add storage for the view, edit and join policies.
  - A user can always view a project if they are a member.
  - A user can always join a project if they can edit it.
  - Editing a project requires both "view" and "edit" permissions, and edit does not imply view.
  - This has no effect on the application yet.

Test Plan: See next diff.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3219
2012-08-11 07:05:01 -07:00
Pieter Hooimeijer
64472dd7b8 Adding Ponder-related files.
Summary:
Ponder is similar in spirit to the Wiki tool, but uses a Q&A
format and up/downvotes to signal user sentiment. Popular
questions are moved to the top of the feed on a 5-minute
cycle based on age (younger is better) and vote count (higher
is better).

Pre-apologies for noob diff.

Test Plan:
- `./bin/phd list` Should include `PonderHeatDaemon`; phd launch it
  if necessary.

- Navigate to /ponder/ ; observe sanity when adding questions,
  voting on them, and adding answers.

- Confirm that questions and answers are linkable using Q5 / Q5#A5 formatted object links.

- Confirm that searching for Ponder Questions works using built-in
  search.

Feedback on code / schema / whatever organization very welcome.

Reviewers: nh, vrana, epriestley

Reviewed By: epriestley

CC: gmarcotte, aran, Korvin, starruler

Differential Revision: https://secure.phabricator.com/D3136
2012-08-10 10:44:04 -07:00
epriestley
d32926e5f7 Work-in-progress schema for Facts app
Summary: See discussion in D3078 for why I've separated this. Pretty sure it's not quite ready yet -- I want to build a couple of things on it so we have a better idea of what we need (autoincrement ID? <factType, objectA, epoch> primary key? objectB column? valueZ?) and don't need to do a ton of schema patches.

Test Plan: Applied patches, ran D3078.

Reviewers: vrana, btrahan, majak

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1581, T1562

Differential Revision: https://secure.phabricator.com/D3088
2012-08-09 08:40:56 -07:00
epriestley
f9fcaa1f84 Migrate project membership to edges
Summary:
  - Store project members in edges.
  - Migrate existing members to edge storage.
  - Delete PhabricatorProjectAffiliation.
  - I left the actual underlying data around just in case something goes wrong; we can delete it evenutally.

Test Plan:
  - Ran migration.
  - Created a new project.
  - Joined and left a project.
  - Added and removed project members.
  - Manually called PhabricatorOwnersOwner::loadAffiliatedUserPHIDs() to verify its behavior.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3186
2012-08-07 18:02:05 -07:00
Alan Huang
bcb9de4ea1 Add a context field to symbol objects
Summary:
See T1602.

This is just the minimal functional patch; the scripts will continue
working because of the `DEFAULT ''`.

Test Plan:
Can't fully test this until I get more code working, but
nothing broke horribly yet.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3147
2012-08-06 12:20:45 -07:00
Nick Harper
88caa45854 Save daemon state to database
Summary:
To make it easier to monitor daemons, let's store their current state
(running, died, exited, or unknown) to the db. The purpose of this is to
provide more information on the daemon console about the status of daemons,
especially when they are running on multiple machines. This is mostly backend
work, with only a few frontend changes. (It is also dependent on a change
to libphutil.)

These changes will make dead or stuck daemons more obvious, and will allow
more work on the frontend to hide daemons (and logs) that have exited cleanly,
i.e. ones we don't care about any more.

Test Plan:
- run db migration, check in db that all daemons were marked as exited
- start up a daemon, check in db that it is marked as running
- open web interface, check that daemon is listed as running
- after daemon has been running for a little bit, check in db that dateModified
  is being updated (indicating daemon is properly sending heartbeat)
- kill -9 daemon (but don't run bin/phd yet), and check that db still shows it
  as running
- edit daemon db entry to show it as being on a different host, and backdate
  dateModified field by 3 minutes, and check the web ui to show that the status
  is unknown.
- change db entry to have proper host, check in web ui that daemon status is
  displayed as dead. Check db to see that the status was saved.
- run bin/phd stop, and see that the formerly dead daemon is now exited.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3126
2012-08-01 17:06:04 -07:00
epriestley
9be12551a9 Move Task <=> Revision storage to Edges
Summary:
  - Add edges for this relationship.
  - Use edges to store this data.
  - Migrate old data.
  - Fix some warnings with generating feed stories about Aux and Edge transactions.
  - Fix a task-task edge issue with "Create Subtask".

Test Plan:
  - Migrated data, verified reivsions showed up.
  - Attached and detached tasks to revisions and vice versa.
  - Created a new revision with attached tasks.
  - Created a subtask.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3018
2012-07-20 08:59:39 -07:00
Bob Trahan
ae13d33859 Phame - introduce blogs
Summary:
blogs are collections of posts. a blog also has metadata like a name, description and "bloggers" that can edit the metadata of the blog and contribute posts.

changes include the post edit flow where bloggers can now select which blogs to publish to. also made various small tweaks throughout the UI to make things sensical and clean as the concept of blogs is introduced.

there's edges powering this stuff.  bloggers <=> blogs and posts <=> blogs in particular.

Test Plan:
made blogs, deleted blogs, tried to make blogs with no bloggers. all went well.
verified ui to publish only showed up for public posts, published posts to blogs, un-published posts to blogs, re-published posts to blogs, deleted posts and verified they disappeared from blogs.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3003
2012-07-19 09:03:10 -07:00
epriestley
ee709a0543 Use Edges to store dependencies between revisions in Differential
Summary: See D3006. Move this data to the edge store.

Test Plan:
  - Created dependencies, migrated, verified dependencies were preserved.
  - Added new dependencies, they worked.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3007
2012-07-18 20:42:06 -07:00
epriestley
9196a6bd9f Use Edges to store dependencies between tasks in Maniphest
Summary:
  - Use edges to store "X depends on Y" information in Maniphest.
  - Show both "Depends On" and "Dependent Tasks".
  - Migrate all the old edges.

Test Plan:
  - Added some relationships, migrated, verified they were preserved.
  - Added some new valid relationships, verified tasks got updated with sensible transactions and sent reasonable emails.
  - Tried to add a cycle, got an ugly but effective error.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3006
2012-07-18 20:41:42 -07:00
epriestley
5d8b75b4da Use the unified markup cache for Maniphest
Summary:
  - See D2945.
  - Drop `cache` field from ManiphestTransaction.
  - Render task descriptions and transactions through PhabricatorMarkupEngine.
  - Also pull the list of macros more lazily.

Test Plan:
  - Verified transactions and transaction preview work correctly and interact with cache correctly.
  - Verified tasks descriptions and task preview work correctly.
  - Verified we don't hit the imagemacro table when we're rendering everything from cache anymore.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2946
2012-07-11 11:40:10 -07:00
epriestley
2b0b9a1573 Add a generic multistep Markup cache
Summary:
The immediate issue this addresses is T1366, adding a rendering cache to Phriction. For wiki pages with code blocks especially, rerendering them each time is expensive.

The broader issue is that out markup caches aren't very good right now. They have three major problems:

**Problem 1: the data is stored in the wrong place.** We currently store remarkup caches on objects. This means we're always loading it and passing it around even when we don't need it, can't genericize cache management code (e.g., have one simple script to drop/GC caches), need to update authoritative rows to clear caches, and can't genericize rendering code since each object is different.

To solve this, I created a dedicated cache database that I plan to move all markup caches to use.

**Problem 2: time-variant rules break when cached.** Some rules like `**bold**` are time-invariant and always produce the same output, but some rules like `{Tnnn}` and `@username` are variant and may render differently (because a task was closed or a user is on vacation). Currently, we cache the raw output, so these time-variant rules get locked at whatever values they had when they were first rendered. This is the main reason Phriction doesn't have a cache right now -- I wanted `{Tnnn}` rules to reflect open/closed tasks.

To solve this, I split markup into a "preprocessing" phase (which does all the parsing and evaluates all time-invariant rules) and a "postprocessing" phase (which evaluates time-variant rules only). The preprocessing phase is most of the expense (and, notably, includes syntax highlighting) so this is nearly as good as caching the final output. I did most of the work here in D737 / D738, but we never moved to use it in Phabricator -- we currently just do the two operations serially in all cases.

This diff splits them apart and caches the output of preprocessing only, so we benefit from caching but also get accurate time-variant rendering.

**Problem 3: cache access isn't batched/pipelined optimally.** When we're rendering a list of markup blocks, we should be able to batch datafetching better than we do. D738 helped with this (fetching is batched within a single hunk of markup) and this improves batching on cache access. We could still do better here, but this is at least a step forward.

Also fixes a bug with generating a link in the Phriction history interface ($uri gets clobbered).

I'm using PHP serialization instead of JSON serialization because Remarkup does some stuff with non-ascii characters that might not survive JSON.

Test Plan:
  - Created a Phriction document and verified that previews don't go to cache (no rows appear in the cache table).
  - Verified that published documents come out of cache.
  - Verified that caches generate/regenerate correctly, time-variant rules render properly and old documents hit the right caches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1366

Differential Revision: https://secure.phabricator.com/D2945
2012-07-09 15:20:56 -07:00
epriestley
7cf6313be9 Add a generic object for unit tests
Summary:
A later diff adds unit tests against edges, but we need real objects to connect with edges. Add some trivial objects to the Harbormaster database to compliment the similar HarbormasterScratchTable.
On its own, this does nothing interesting.

Test Plan: Built unit tests on this in a followup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2937
2012-07-09 10:39:14 -07:00
dschleimer
86fa4fd97f [Phabricator] track Mercurial bookmarks for differential diffs
Summary:
This adds all the changes necessary to track the active Mercurial
bookmark for differential diffs.  We render both branch and bookmark
information in the branch field of the Differential revison view, as
seen in
https://secure.phabricator.com/file/data/kzpmu3evfkukxdjyxrfz/PHID-FILE-eqorsqupxvwirqi2s5lo/bookmark_differential.jpg

The Arcanist half of this is https://secure.phabricator.com/D2896

Test Plan:
Mostly D2896.

Additionally, loaded a diff created with a bookmark, as per the link in the summary.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1331

Differential Revision: https://secure.phabricator.com/D2897
2012-06-30 15:41:58 -07:00
vrana
48ebcf0679 Allow user override translation and implement PhutilPerson
Test Plan:
Altered database.
Wrote a custom translation and selected it in preferences.
Verified that the text is custom translated.
Set language back to default.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2757
2012-06-14 18:33:00 -07:00
vrana
892a2d1b61 Make Thread-Topic human readable
Summary:
Some e-mail clients display this header and it needs to be constant.

This is somehow involved but I doubt that there is a simpler solution.

Test Plan:
Applied SQL patch.
Commented on revision, commented on commit, changed package.
Verified that the `Thread-Topic` has constant and human readable value.

Reviewers: epriestley

Reviewed By: epriestley

CC: ola, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2745
2012-06-14 11:36:34 -07:00
Espen Volden
726041584f Made it possible to login using LDAP
Summary: Made it possible to link and unlink LDAP accounts with  Phabricator accounts.

Test Plan:
I've tested this code locally and in production where I work.
I've tried creating an account from scratch by logging in with LDAP and linking and unlinking an LDAP account with an existing account. I've tried to associate the same LDAP account with different Phabricator accounts and it failed as expected.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, auduny, svemir

Maniphest Tasks: T742

Differential Revision: https://secure.phabricator.com/D2722
2012-06-13 08:58:46 -07:00
Keebuhm Park
207f101aee SQL patch for notification
Summary: Added `PhabricatorBuiltinPatchList` entry so that "storage upgrade" will update the database. Renamed and numbered the notification.sql patch.

Test Plan: Drop phabricator_feed.feed_storynotification table if it exists and run bin/storage upgrade to check if the patch is correctly applied.

Reviewers: epriestley, btrahan, allenjohnashton

Reviewed By: epriestley

CC: ddfisher, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2687
2012-06-08 12:42:59 -07:00
John-Ashton Allen
3a6ee79190 Adds base notification application
Summary: First diff in a series of diffs to add notifications to Phabricator. This is the notification application ONLY. This commit does not include the changes to other applications that makes them add notifications. As such, no notifications will be generated beyond the initial database import.

Test Plan: This is part of the notifications architecture which has been running on http://theoryphabricator.com for the past several months.

Reviewers: epriestley, btrahan, ddfisher

Reviewed By: epriestley

CC: allenjohnashton, keebuhm, aran, Korvin, jungejason, nh

Maniphest Tasks: T974

Differential Revision: https://secure.phabricator.com/D2571
2012-06-08 06:32:02 -07:00
epriestley
f6fbe40bd5 Minor, completely remove references to PHID from schema patches so upgrade-from-scratch works. 2012-05-24 13:59:12 -07:00
epriestley
a89cef8e39 Remove PHID database, add Harbormaster database
Summary:
  - We currently write every PHID we generate to a table. This was motivated by two concerns:
    - **Understanding Data**: At Facebook, the data was sometimes kind of a mess. You could look at a random user in the ID tool and see 9000 assocs with random binary data attached to them, pointing at a zillion other objects with no idea how any of it got there. I originally created this table to have a canonical source of truth about PHID basics, at least. In practice, our data model has been really tidy and consistent, and we don't use any of the auxiliary data in this table (or even write it). The handle abstraction is powerful and covers essentially all of the useful data in the app, and we have human-readable types in the keys. So I don't think we have a real need here, and this table isn't serving it if we do.
    - **Uniqueness**: With a unique key, we can be sure they're unique, even if we get astronomically unlucky and get a collision. But every table we use them in has a unique key anyway. So we actually get pretty much nothing here, except maybe some vague guarantee that we won't reallocate a key later if the original object is deleted. But it's hard to imagine any install will ever have a collision, given that the key space is 36^20 per object type.
  - We also currently use PHIDs and Users in tests sometimes. This is silly and can break (see D2461).
  - Drop the PHID database.
  - Introduce a "Harbormaster" database (the eventual CI tool, after Drydock).
  - Add a scratch table to the Harbormaster database for doing unit test meta-tests.
  - Now, PHID generation does no writes, and unit tests are isolated from the application.
  - @csilvers: This should slightly improve the performance of the large query-bound tail in D2457.

Test Plan: Ran unit tests. Ran storage upgrade.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: csilvers, aran, nh, edward

Differential Revision: https://secure.phabricator.com/D2466
2012-05-20 14:46:01 -07:00
epriestley
95017e14e9 Minor, derrrp. 2012-05-07 10:34:07 -07:00
epriestley
58efe52b3a Minor, fix method call fatal. 2012-05-07 10:32:29 -07:00