Summary:
Ref T1191. Now that the whole database is covered, we don't need to do as much work to build expected schemata. Doing them database-by-database was helpful in converting, but is just reudndant work now.
Instead of requiring every application to build its Lisk objects, just build all Lisk objects.
I removed `harbormaster.lisk_counter` because it is unused.
It would be nice to autogenerate edge schemata, too, but that's a little trickier.
Test Plan: Database setup issues are all green.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley, hach-que
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10620
Summary:
Ref T1191.
- Adds definitions for missing keys and keys with wrong uniqueness. Generally, I defined these before fixing the key query to actually pull all keys and support uniqueness.
- Moves "key uniqueness" to note severity; this is fixable (probably?) and there are no remaining issues.
- Moves "Missing Key" to note severity; missing keys are fixable and all remaining missing keys are really missing (either missing edge keys, or missing PHID keys):
{F210089}
- Moves "Surplus Key" to note seveirty; surplus keys are fixable all remaining surplus keys are really surplus (duplicate key in Harbormaster, key on unused column in Worker):
{F210090}
Test Plan:
- Vetted missing/surplus/unique messages.
- 146 issues remaining.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10590
Summary: Fixes T6201. This stuff didn't fully get updated for ApplicationTransactions. Get it working again (notably, make inline comment text publish) and clean it up a little bit.
Test Plan:
- Published a Differential feed story into Asana with comment text.
- Pulbished a Diffusion feed story into Asana with comment text.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T6201
Differential Revision: https://secure.phabricator.com/D10584
Summary:
Ref T1191. Notable:
- Allowed objects to remove default columns (some feed tables have no `id`).
- Added a "note" severity and moved all the charset stuff down to that to make progress more clear.
Test Plan:
Trying to make the whole thing blue...
{F205970}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10519
Summary: Provide an implementation for the `getName` method rather than automagically determining the application name.
Test Plan: Saw reasonable application names in the launcher.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D10027
Summary: Ref T5655. Some discussion in D9839. Generally speaking, `Phabricator{$name}Application` is clearer than `PhabricatorApplication{$name}`.
Test Plan:
# Pinned and uninstalled some applications.
# Applied patch and performed migrations.
# Verified that the pinned applications were still pinned and that the uninstalled applications were still uninstalled.
# Performed a sanity check on the database contents.
Reviewers: btrahan, epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: hach-que, epriestley, Korvin
Maniphest Tasks: T5655
Differential Revision: https://secure.phabricator.com/D9982
Summary:
Ref T5245. These were a bad idea.
We no longer need actors for edge edits either, so remove those. Generally, edges have fit into the policy model as pure/low-level infrastructure, and they do not have any policy or capability information in and of themselves.
Test Plan: `grep`
Reviewers: chad, btrahan, joshuaspence
Reviewed By: joshuaspence
Subscribers: epriestley
Maniphest Tasks: T5245
Differential Revision: https://secure.phabricator.com/D9840
Summary:
Fixes T3732. Ref T1205. Ref T3116.
External accounts (like emails used as identities, Facebook accounts, LDAP accounts, etc.) are stored in "ExternalAccount" objects.
Currently, we have a very restrictive `CAN_VIEW` policy for ExternalAccounts, to add an extra layer of protection to make sure users can't use them in unintended ways. For example, it would be bad if a user could link their Phabricator account to a Facebook account without proper authentication. All of the controllers which do sensitive things have checks anyway, but a restrictive CAN_VIEW provided an extra layer of protection. Se T3116 for some discussion.
However, this means that when grey/external users take actions (via email, or via applications like Legalpad) other users can't load the account handles and can't see anything about the actor (they just see "Restricted External Account" or similar).
Balancing these concerns is mostly about not making a huge mess while doing it. This seems like a reasonable approach:
- Add `CAN_EDIT` on these objects.
- Make that very restricted, but open up `CAN_VIEW`.
- Require `CAN_EDIT` any time we're going to do something authentication/identity related.
This is slightly easier to get wrong (forget CAN_EDIT) than other approaches, but pretty simple, and we always have extra checks in place anyway -- this is just a safety net.
I'm not quite sure how we should identify external accounts, so for now we're just rendering "Email User" or similar -- clearly not a bug, but not identifying. We can figure out what to render in the long term elsewhere.
Test Plan:
- Viewed external accounts.
- Linked an external account.
- Refreshed an external account.
- Edited profile picture.
- Viewed sessions panel.
- Published a bunch of stuff to Asana/JIRA.
- Legalpad signature page now shows external accounts.
{F171595}
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T3732, T1205, T3116
Differential Revision: https://secure.phabricator.com/D9767
Summary: Remarkup rules can not safely use arbitrary text in tag attributes,
because it may include tokens which are later replaced. Precedence rules
should prevent this in general. Use flat text assertions and adjust precedence
rules in cases where they may not prevent tokens from appearing in attributes.
Auditors: btrahan
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.
Test Plan: Eyeballed it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin, hach-que
Differential Revision: https://secure.phabricator.com/D9431
Summary:
This does some backend cleanup of the tile stuff, and some general cleanup of other application things:
- Users who haven't customized preferences get a small, specific set of pinned applications: Differential, Maniphest, Diffusion, Audit, Phriction, Projects (and, for administrators, Auth, Config and People).
- Old tile size methods are replaced with `isPinnnedByDefault()`.
- Shortened some short descriptions.
- `shouldAppearInLaunchView()` replaced by less ambiguous `isLaunchable()`.
- Added a marker for third-party / extension applications.
Test Plan: Faked away my preferences and viewed the home page, saw a smaller set of default pins.
Reviewers: chad
Reviewed By: chad
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D9358
Summary:
This probably needs some tweaks, but the idea is to make it easier to browse and access applications without necessarily needing them to be on the homepage.
Open to feedback.
Test Plan:
(This screenshot merges "Organization", "Communication" and "Core" into a single "Core" group. We can't actually do this yet because it wrecks the homepage.)
{F160052}
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T5176
Differential Revision: https://secure.phabricator.com/D9297
Summary: Fixes T4859. See that for details.
Test Plan:
- Verified things still work on my local (domain root) install.
- Added some unit tests.
- Did not verify a non-root install since I don't have one handy, hopefully @salehe can help.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: salehe, epriestley
Maniphest Tasks: T4859
Differential Revision: https://secure.phabricator.com/D8836
Summary:
Currently, users get an error when making any changes to this field if they don't have a linked JIRA account.
Instead:
- We should only raise an error if they're trying to //add// issues, and only on the new issues. It's always fine to remove issues, and existing issues the author can't see are also fine.
- When we can't add things because there's no account (vs because there's a permissions error or they don't exist), raise a more tailored exception.
Test Plan:
- As JIRA and non-JIRA users, made various edits to this field.
- Got appropriate exceptions, including better tailoring.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: mbishopim3, epriestley
Differential Revision: https://secure.phabricator.com/D8676
Summary:
Report from Asana. In some unclear circumstances, we my attempt to resolve duplicate refs which currently ends up hitting a duplicate key error.
Instead, reference the same external object if we happen to be handed duplicate refs.
Test Plan:
Used this script to reproduce the issue. Applied the fix; issue went away:
#!/usr/bin/env php
<?php
require_once 'scripts/__init_script__.php';
$args = new PhutilArgumentParser($argv);
$args->parseStandardArguments();
$ref = id(new DoorkeeperObjectRef())
->setApplicationType(DoorkeeperBridgeAsana::APPTYPE_ASANA)
->setApplicationDomain(DoorkeeperBridgeAsana::APPDOMAIN_ASANA)
->setObjectType(DoorkeeperBridgeAsana::OBJTYPE_TASK)
->setObjectID(7253737283629); // Use a new task ID which we've never pulled.
$refs = array(clone $ref, clone $ref);
$asana_user = id(new PhabricatorPeopleQuery())
->setViewer(PhabricatorUser::getOmnipotentUser())
->withUsernames(array('asana'))
->executeOne();
$resolved_refs = id(new DoorkeeperImportEngine())
->setViewer($asana_user)
->setRefs($refs)
->execute();
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7709
Summary: Request from Asana. Adds an option for adding tasks to projects.
Test Plan: Used `bin/feed republish` to create and update Asana tasks with projects configured. Saw them end up in the right projects.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7655
Summary: Fixes T3687. Instead of rendering "JIRA Issues" in Differential using plain links, render them using Doorkeeper tags so they get the nice "enhance with object name" effect.
Test Plan: {F84886}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3687
Differential Revision: https://secure.phabricator.com/D7648
Summary:
While we mostly have reasonable effective object accessibility when you lock a user out of an application, it's primarily enforced at the controller level. Users can still, e.g., load the handles of objects they can't actually see. Instead, lock the queries to the applications so that you can, e.g., never load a revision if you don't have access to Differential.
This has several parts:
- For PolicyAware queries, provide an application class name method.
- If the query specifies a class name and the user doesn't have permission to use it, fail the entire query unconditionally.
- For handles, simplify query construction and count all the PHIDs as "restricted" so we get a UI full of "restricted" instead of "unknown" handles.
Test Plan:
- Added a unit test to verify I got all the class names right.
- Browsed around, logged in/out as a normal user with public policies on and off.
- Browsed around, logged in/out as a restricted user with public policies on and off. With restrictions, saw all traces of restricted apps removed or restricted.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7367
Summary:
Ref T603. Move toward stamping out all the Project / ProjectProfile query irregularities with respect to policies.
- Fixes a bug with Asana publishing when the remote task is deleted.
- Fixes an issue with Herald commit rules.
Test Plan:
- Viewed projects;
- edited projects;
- added and removed members from projects;
- republished Asana-bridged feed stories about commits.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7251
Summary:
Ref T603. Adds clarifying text which expands on policies and explains exceptions and rules. The goal is to provide an easy way for users to learn about special policy rules, like "task owners can always see a task".
This presentation might be a little aggressive. That's probably OK as we introduce policies, but something a little more tempered might be better down the road.
Test Plan: See screenshot.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7150
Summary:
Ref T3687. JIRA is able to piggyback on a fair amount of Asana infrastructure, but the voicing we use on Asana tasks (which are always about one object) isn't very good for JIRA issues (which may have many linked objects). Specifically, we publish stories like this to Asana:
alincoln accepted this revision.
This is meaningless in JIRA since you have no idea what it's talking about. Instead, publish like this:
alincoln accepted D999: Put a bird on it
Additionally, supplement it with a URI, so the total story text we publish is:
alincoln accepted D999: Put a bird on it
https://phabricator.whitehouse.gov/D999
Signifcantly less useless!
Test Plan: {F57523} {F57524}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3687
Differential Revision: https://secure.phabricator.com/D6907
Summary:
Ref T3687. See some discussion in D6892. The JIRA doorkeeper publisher shares a reasonable amount of code with the Asana publisher. Remedy this:
- Create `DoorkeeperFeedWorker`, where shared functionality lives (mostly related to building story context objects).
- Push responsibility for enabling/disabling a worker into this new layer, via `isEnabled()`. This allows `FeedPublisherWorker` to dynamically find and schedule doorkeeper publishers, so third parties can add additional doorkeeper publishers.
- Some general cleanup/documentation.
Test Plan: Used `bin/feed republish` to republish stories about objects with JIRA and Asana links. Verified that doorkeeper publishers activated properly, made calls, and published events into the remote systems.
Reviewers: btrahan, akopanev22
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3687
Differential Revision: https://secure.phabricator.com/D6906
Summary:
Ref T3687. Publish stories into JIRA.
These need some voicing fixes, which maybe involves straightening out the feed code. For example, they're voiced in-context ("updated this revision") when they should be voiced out-of-context ("updated D123").
Generally, this is similar to the Asana stuff but a lot simpler since we don't need to do any state management.
Test Plan: {F57366}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3687
Differential Revision: https://secure.phabricator.com/D6892
Summary:
Ref T3687. Adds a Doorkeeper bridge for JIRA issues, plus remarkup support. In particular:
- The Asana and JIRA remarkup rules shared most of their implementation, so I refactored what I could into a base class.
- Actual bridge implementation is straightforward and similar to Asana, although probably not similar enough to really justify refactoring.
Test Plan:
- When logged in as a JIRA-connected user, pasted a JIRA issue link and saw it enriched at rendering time.
- Logged in and out with JIRA.
- Tested an Asana link, too (seems I haven't broken anything).
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3687
Differential Revision: https://secure.phabricator.com/D6878
Summary: Ref T2852. Asana is launching some kind of silent follow thing today; I don't know what the API is but it's probably something like this. I'll update this to actually make the right call once the call exists, this is mostly just a placeholder so I don't forget about it.
Test Plan: None yet, this API isn't documented or live and doesn't work yet so it can't be tested.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, moskov
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6740
Summary: Ref T2852. Currently, we publish commits with no audit requests and reviews with no CCs or reviewers into Asana. This creates undesired notifications, so drop events which would publish an object that doesn't exist yet and has no followers or respible users.
Test Plan: Used `bin/feed republish` to publish a story about an object with no related users, saw the publish abort with the new message. Added a CC, published again, got a publish.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6727
Summary: Ref T2852. Asana adds the actor as a follower when they create a task, so subtasks currently have up to two followers (the actor and the reviewer) when they should have only one (the reviewer). Simply removing the actor is an effective remedy for this because unfollowing tasks occurs with sneaky ninja stealth in Asana and doesn't generate notifications or even transaction activity.
Test Plan: Synchronized a revision without this patch, saw two followers on the subtask. Synchronized a revision after this patch, saw the "removeFollowers" fire and only one follower on the subtask, with no record of the removal in notifications or the transaction log.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6700
Summary:
Ref T2852. Two issues:
- Embeds (`T12`, `{T12}`) have some handle issues because handles run afoul of visibility checks under some configs. Make handles unconditionally visible.
- Asana links don't render correctly into text mode. Give them a valid text mode rendering so they don't flip out.
Test Plan: Made comments with `T12` and `http://app.asana.com/...` and published them to Asana.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6696
Summary: Ref T2852. After some discussion, Asana doesn't want "close" stories either.
Test Plan: Used `bin/feed republish` to publish close and non-close stories from Differential and Diffusion. Verified comments were synchronized in the expected cases.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6697
Summary: Ref T2852. Current code works fine, but although we want to drop creation stories, we really only want to drop the story text, not the other effects of the creation story. Also generalize this mechanism so we don't have Asana-specific code in the publishers.
Test Plan: Used `bin/feed republish` to publish creation and non-creation stories. Verified creation story published no text.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6639
Summary: Ref T2852. Pulls the Differential-specific aspects of the Asana sync out of the worker. Next diff will add a publisher for Audit/Diffusion.
Test Plan: Published events, including state changes. Saw them reflected correctly in Asana.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6569
Summary: Ref T2852. When the parent task is actionable (needs revision, accepted) give it an "Upcoming" status; otherwise give it a "Later" status.
Test Plan: Sync'd "Needs Revision" and "Needs Review" tasks and saw them both bucket correctly
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6551
Summary: `FeedStory->renderText()` is garbage and I don't want to fix it in general until after T2222 / T2217. Provide an Asana-specific alternative for higher-quality feed stories (notably, including comment text).
Test Plan: {F51035}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6521
Summary:
Ref T2852. We need to distinguish between an API call which worked but got back nothing (404) and an API call which failed.
In particular, Asana hit a sync issue which was likely the result of treating a 500 (or some other error) as a 404.
Also clean up a couple small things.
Test Plan: Ran syncs against deleted tasks and saw successful syncs of non-tasks, and simulated random failures and saw them get handled correctly.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6470
Summary: We attempt to choose the most reasonable actor when synchronizing to Asana, but Asana is seeing the sync choose a less-reasonable actor. I spotted two places where the order may get disrupted; make sure we retain order. This is somewhat tricky to repro locally (it depends on things like native account order) but I think this is the right fix. If not, I'll add more logging. Ref T2852.
Test Plan: Used `bin/feed republish` to sync Asana events.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6443
Summary:
Ref T2852.
- Respect the existing setting for `"[Differential]"`.
- Show `[Request, X lines]` to make this more similar to the email.
Test Plan:
Sync'd to asana and got a task with the right subject:
{F49950}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6439
Summary:
Ref T2852. It's a little tricky to figure out Asana workspace IDs right now. If the viewer has a linked account, just pull their workspaces and show them which IDs are available.
(In theory, we could use a `<select>`, but it would have more edge cases; this seems like a pretty solid fix.)
Test Plan: {F49938}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6437
Summary: Ref T2852. When a Differential revision is linked to an Asana task, show the related task in Differential.
Test Plan: {F49234}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6387
Summary:
Ref T2852. Asana sync tasks currently have a standard retry/backoff schedule, but the defaults are quite aggressive (retry every 60s forever). Instead, retry at increasing intervals and stop retrying after a few tries.
- Retry at intervals and stop retrying after a few iterations.
- Modernize some interfaces.
- Add better information about retry behaviors to the web UI.
Test Plan: {F49194}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6381
Summary:
Depends on D6329. This fixes `http://www.example.com/D123`, which currently gets the "D123" rendered, after addition of the Asana rule. It also removes a hack for object refernces.
Basically, the "hyperlink" rule needs to happen after rules which specialize hyperlinks (Youtube, Asana) but before rules which apply to general text (like the Differential and Maniphest rules). Allow these rules to specify that they have higher or lower priority.
Test Plan: Asana rules, Differential rules and Diffusion rules now all markup correctly.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6330
Summary: Ref T2852. Setting followers (like CCs) is a separate API call, but we don't need to do anything complicated.
Test Plan: Synchronized revisions and verified the parent task got followers.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6308
Summary:
Ref T2852.
Before trying related users, try using the feed story's actor. This is the most correct voice to act in.
Test Plan: Ran `feed/republish`.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6305
