1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-23 05:50:55 +01:00
Commit graph

736 commits

Author SHA1 Message Date
epriestley
809ae81752 Update Slowvote poll status to use sensible string constants
Summary: Ref T13682. This prepares for modernizing Slowvote and exposing a more usable API.

Test Plan: Ran migrations, opened and closed polls.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682

Differential Revision: https://secure.phabricator.com/D21848
2022-05-27 10:15:01 -07:00
epriestley
03d3d1889d Update Slowvote voting methods to use sensible string constants
Summary: Ref T13682. Use API-friendly string constants instead of opaque integers in Slowvote voting methods.

Test Plan: Created, edited, and voted in polls with various voting methods. Examined database after migrations.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682

Differential Revision: https://secure.phabricator.com/D21846
2022-05-26 10:27:01 -07:00
epriestley
9f075839a2 Update Slowvote to use sensible string constants for response visibility
Summary: Ref T13682. Migrate and update Slowvote to use API-friendly string constants for response visibility, not opaque integers.

Test Plan: Created and edited slowvotes, changing response visibility.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682

Differential Revision: https://secure.phabricator.com/D21844
2022-05-26 09:58:48 -07:00
epriestley
b1533e5468 Migrate "SlowvotePoll" to modern mailkey storage
Summary:
Ref T13682. Ref T13065. Migrate and update "SlowvotePoll" to use more modern mailkey storage (in the mail application, rather than onboard).

Also make other minor modernization changes.

Test Plan: Created a new poll. Ran migration, verified mail key copied properly in database.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682, T13065

Differential Revision: https://secure.phabricator.com/D21842
2022-05-26 09:29:05 -07:00
epriestley
c25a8fabfc Remove all "ObjectHasFile" edge reads and writes
Summary: Ref T13603. Migrate all code which interacts with the "ObjectHasFile" edge to use the "Attachments" table instead.

Test Plan:
  - Edited a paste's view policy, confirmed associated file secret was scrambled.
  - Verified I could still view paste content as a user who could not naturally view the underlying file.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13603

Differential Revision: https://secure.phabricator.com/D21819
2022-05-19 13:21:04 -07:00
epriestley
0329df0fa8 Copy file attachment edges to new storage
Summary: Ref T13603. Copy all existing file attachment data into the new storage table.

Test Plan: Ran migration, checked attachment table for general reasonableness.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13603

Differential Revision: https://secure.phabricator.com/D21818
2022-05-19 13:21:03 -07:00
epriestley
cfa42c5e65 Add database storage for a dedicated file attachment table
Summary: Ref T13603. Prepare to move this relationship out of edge storage into dedicated storage so it is easier to formalize better in the UI.

Test Plan: Ran `bin/storage upgrade`.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13603

Differential Revision: https://secure.phabricator.com/D21813
2022-05-19 13:21:02 -07:00
epriestley
aae23f0204 Give Phame blog posts configurable interact policies, with a default policy of "Same as Blog"
Summary: Ref T13661. This allows posts to have comments disabled (or restricted) on a per-post basis, and makes them inherit the containing blog policy by default.

Test Plan: Locked a post by editing its policy explicitly; locked a post by editing the containing blog policy.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13661

Differential Revision: https://secure.phabricator.com/D21754
2022-04-01 12:52:58 -07:00
epriestley
c25595417f Give Phame blogs mutable interact policies
Summary:
Ref T13661.

I'm fairly sure these policies don't actually do anything (you can't "interact" with a blog) but the primarily support a Phame Post object policy of "Same as Parent Blog", which is the "natural" interact policy for a post.

Most of this is infrastructure support for mutable interact policies: today, only Maniphest has interact mutability and only via indirect effects (locking tasks), not through a directly mutable "Can Interact" policy.

Test Plan:
Ran storage upgrade, edited interact policy of a blog, saw appropriate persistence and transactions.

Created and edited a task to make sure there's no weird fallout from increasing what can be done with interact policies.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13661

Differential Revision: https://secure.phabricator.com/D21751
2022-04-01 12:52:56 -07:00
epriestley
28b12fb3cd Make Phame blog policies non-nullable
Summary: Ref T6203. Ref T13661. These policies are incorrectly nullable, although it's likely that no pathway exists in the application to write NULL into them. Fix the schema.

Test Plan: Ran `bin/storage upgrade`.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13661, T6203

Differential Revision: https://secure.phabricator.com/D21748
2022-04-01 12:52:55 -07:00
epriestley
8bbee92139 Modularize individual Harbormaster build messages
Summary:
Ref T13072. Further modularize build messages by applying each one in a separate transaction type.

This makes it easier to add new types of messages (although I have no particular plans to do this, offhand) and reduces the amount of switch-boilerplate.

This will probably also simplify validating "harbormaster.sendmessage".

Test Plan:
  - Applied all commands.
  - Ran migration, saw transactions render properly

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13072

Differential Revision: https://secure.phabricator.com/D21690
2021-07-21 14:16:56 -07:00
epriestley
b48d4fabaf Merge the "HarbormasterBuildCommand" table into "HarbormasterBuildMessage"
Summary:
Ref T13072. These two similar tables don't make sense to keep separate. Instead, make Build a valid receiver for BuildMessage objects.

These tables are practically the same, so this is straightforward: just copy the rows in and then drop the old table.

(This table was trivial and ephemeral anyway, so I'm not bothering to do the usual "keep it around for a couple years just in case".)

Test Plan:
  - Populated BuildCommand table, ran migration, saw Builds end up in the proper transitional state (e.g., pausing, aborting, restarting) with appropriate queued messages.
  - Queued new messages by clicking UI buttons.
  - Ran BuildWorkers, saw them process messages and mark them as consumed.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13072

Differential Revision: https://secure.phabricator.com/D21684
2021-07-21 14:16:53 -07:00
epriestley
a641ec82a3 Add an "Authority" control to Packages to support "Watcher" packages
Summary: See T13657. An install has "watcher" packages which should not allow owners to "Force Accept" other packages.

Test Plan:
  - Created package A, which I own, on "/", with "Weak" authority.
  - Created package B, which I do not own, on "/src".
  - Created a revision which touches "/src" and added package B as a reviewer.
  - Attempted to accept the revision...
    - Before patch: permitted to "Force Accept" for package B.
    - After patch: not allowed to "Force Accept" for package B.
  - Verified that setting package "A" back to "Strong" authority allows a force-accept for package B.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D21674
2021-06-25 13:48:46 -07:00
epriestley
3267859aee Modernize "mailKey" on Fund initiatives
Summary: Ref T13065. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean upgrade, saw migrated values in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065

Differential Revision: https://secure.phabricator.com/D21634
2021-03-16 15:51:50 -07:00
epriestley
eeb009b4fa Modernize "mailKey" for Calendar Event
Summary: Ref T13065. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report, saw migrated data in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065

Differential Revision: https://secure.phabricator.com/D21633
2021-03-16 15:51:50 -07:00
epriestley
86669ab54f Modernize "mailKey" for Almanac Networks
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report, saw keys migrate in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21632
2021-03-16 15:51:49 -07:00
epriestley
8226b3c880 Modernize "mailKey" on Almanac Namespaces
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report, saw mail keys migrated to mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21631
2021-03-16 15:51:49 -07:00
epriestley
7c0e33c34d Modernize "mailKey" for Almanac Bindings
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report and saw binding mail keys in the mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21630
2021-03-16 15:51:48 -07:00
epriestley
fadb2bd52b Modernize "mailKey" on AlamnacService
Summary: Ref T13641. Ref T13065. Migrate and drop the onboard "mailKey" column for Almanac Services.

Test Plan: Ran storage migration, got clean report, saw migrated value in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21629
2021-03-16 15:51:48 -07:00
epriestley
5d64fb1815 Add a "status" property to Almanac devices
Summary:
Ref T13641. Add a "status" property with most of the relevant support code.

This currently has no impact on use of the device or bindings by Diffusion or Drydock: they ignore the status of devices bound to services.

Test Plan:
  - Created a new device.
  - Changed the status of a device via web and API.
  - Queried for devices via API.
  - Searched for active and disabled devices.
  - Viewed UI in list view, detail view.
  - Used typeahead to add a new binding to an interface on a disabled device, got disabled hint in typeahead UI.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641

Differential Revision: https://secure.phabricator.com/D21627
2021-03-16 15:51:47 -07:00
epriestley
c3e6db6f0b Migrate Almanac Device "mailKey" to modern storage
Summary: Ref T13065. See similar changes attached to that task.

Test Plan: Ran migration, got a clean database state, saw mail keys populate in mail property table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065, T13641

Differential Revision: https://secure.phabricator.com/D21625
2021-03-16 15:51:46 -07:00
epriestley
01ea84029d Update table schema for "AffectedPath" table
Summary:
Ref T13639. Make schema changes:

  - Make repositoryID nullable, for revisions with no repository.
  - Remove "epoch", which has no readers and no clear use.
  - Change the ordering of the key, since "pathID" has more unique values and no queries ever issue without it.

Test Plan:
  - Ran `bin/storage upgrade`, got a clean schema.
  - Reindexed all revisions with an external script.
  - Reviewed index via debug UI, saw appropriate index for non-repositoy revisions.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13639

Differential Revision: https://secure.phabricator.com/D21617
2021-03-15 16:16:10 -07:00
epriestley
32da29b965 Provide more help around GRANT errors, particularly for missing TEMPORARY TABLE permission
Summary:
Fixes T13622. Figuring out what permissions we have seems difficult, so address this a bit more narrowly:

  - Make the "access denied" error message a bit more helpful.
  - Tailor error handling for the "CREATE TEMPORARY TABLE" statement.

Test Plan:
  - Created a new user, granted them "SELECT ON *.*" but not "CREATE TEMPORARY TABLE", ran `bin/storage upgrade --force --apply phabricator:20210215.changeset.02.phid-populate.php`.
  - Before: fairly opaque error.
  - After: fairly useful error.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13622

Differential Revision: https://secure.phabricator.com/D21608
2021-03-11 14:55:21 -08:00
epriestley
2636d84d0c Remove very old Audit status constants and AuditRequest data
Summary:
Ref T13631. See that task for discussion.

  - "NONE": Probably never used?
  - "CC": Obsoleted by subscribers.
  - "AUDIT_NOT_REQUIRED": For Owners packages, obsoleted by edges.
  - "CLOSED": For "Close Audit", obsoleted by "Request Verification".

Test Plan:
  - Grepped for constants, browsed Diffusion.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13631

Differential Revision: https://secure.phabricator.com/D21598
2021-03-10 09:21:54 -08:00
epriestley
b3976acc40 Improve performance of "phabricator:20210215.changeset.02.phid-populate.php"
Summary: Ref T13613. Improve the performance of this migration by using a temporary table and an "UPDATE x JOIN y ..." pattern.

Test Plan:
  - Ran on `secure`, got exit after a few seconds since the migration is idempotent and changesets already had PHIDs.
  - Ran on `secure` with the `continue;` commented out, got valid new PHIDs in 53s (from 153s).
  - Tried a larger page size (16K), didn't see any improvement.
  - From "--trace", client PHID generation seems to be the limiting factor.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13613

Differential Revision: https://secure.phabricator.com/D21570
2021-02-19 07:53:14 -08:00
epriestley
6703fec3e2 When documents are indexed, record the indexer version (versus the object version) and index epoch
Summary:
Ref T13587. D21495 has significant changes to the ngram indexer, which might possibly contain bugs.

Make it easier to reindex a subset of documents (based on the date when the index was built, and/or the software version which generated the index).

This is in addition to the existing versioning, which is focused on object versions.

Test Plan: Ran `bin/search index` with various old and new arguments. Spot-checked the `IndexVersion` table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13587

Differential Revision: https://secure.phabricator.com/D21560
2021-02-16 16:09:31 -08:00
epriestley
ec5476a01f Add a PHID to Changesets
Summary:
Ref T13605. Changesets currently have no PHID, which limits their ability to use standard API infrastructure.

Give them a PHID, since there's no reason they don't have one other than their age.

Test Plan:
  - Ran migrations, saw PHIDs populated.
  - Created new changesets, saw PHIDs.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13605

Differential Revision: https://secure.phabricator.com/D21557
2021-02-15 11:11:12 -08:00
epriestley
201e0d2943 Add support for a "containerPHID" in the worker queue
Summary:
Ref T13591. Worker queue tasks which affect commits currently (mostly) store the commit as an "objectPHID", but do not directly reference the repository the commit belongs to.

This can make certain operations (like "change the priority of all tasks affecting repository Y") more difficult than it needs to be.

Support a "containerPHID", similar to the field of the same name on builds, that can store a parent object like a repository and better support operations against subsets of tasks.

See also D11044 for the genesis of "objectPHID".

This depends on the introduction of storage patch phases (in D21529) so that earlier migrations which queue worker tasks don't try to insert this column before it actually exists.

Test Plan:
  - Ran `bin/storage upgrade`.
  - No callers yet, see further changes for usage.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13591

Differential Revision: https://secure.phabricator.com/D21531
2021-02-02 13:40:09 -08:00
epriestley
32942f6232 Introduce storage patch "phases" to allow index-rebuilding patches to execute after worker queue schema changes
Summary:
Ref T13591. Some storage patches queue worker tasks, currently always to rebuild search indexes.

These patches can not execute in creation order if a later patch modifies the worker task table, since they'll try to perform a modern INSERT against an out-of-date table schema. Such a modification is desirable in the context of T13591, but making it causes these patches to fail.

Patches have an existing "after" mechanism which allows them to have explicit dependencies. This mechanism could be used to resolve this issue, but all patches with a dependency like this would need to be updated every time the queue table changes.

Instead, introduce "phases" to provide broader ordering rules. There are now two phases: "default" and "worker". Patches in the "worker" phase execute after patches in the "default" phase.

Phases may eventually be further separated, but

Test Plan:
  - Ran `bin/storage status`, saw patches annotated with phases.
  - Will apply `containerPHID` changes on top of this.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13591

Differential Revision: https://secure.phabricator.com/D21529
2021-02-02 13:40:09 -08:00
epriestley
6e1b5da112 Fix additional "xprintf()"-class static parameter lint errors
Summary: Ref T13577. After the fix in D21453, lint identifies additional static errors in Phabricator; fix them.

Test Plan: Ran `arc lint`; these messages are essentially all very obscure.

Subscribers: hach-que, yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13577

Differential Revision: https://secure.phabricator.com/D21457
2020-09-08 11:45:48 -07:00
epriestley
d3d41324be Drop old "differential_commit" table
Summary: Ref T13276. Ref T13513. All readers and writers were removed more than a year ago; clean up the last remnants of this table.

Test Plan: Grepped for table references, found none.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13513, T13276

Differential Revision: https://secure.phabricator.com/D21281
2020-05-20 14:30:39 -07:00
epriestley
6d0dbeb77f Use the changeset parse cache to cache suggestion changesets
Summary:
Ref T13513. Syntax highlighting is potentially expensive, and the changeset rendering pipeline can cache it. However, the cache is currently keyed ONLY by Differential changeset ID.

Destroy the existing cache and rebuild it with a more standard cache key so it can be used in a more ad-hoc way by inline suggestion snippets.

Test Plan: Used Darkconsole, saw cache hits and no more inline syntax highlighting for changesets with many inlines.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13513

Differential Revision: https://secure.phabricator.com/D21280
2020-05-20 14:29:27 -07:00
epriestley
5ff0ae7d48 Add generic "attributes" storage to inline comment tables
Summary: Ref T13513. This plans for "currently editing", character range comments, code suggestions, document engine tracking. And absolutely nothing else.

Test Plan:
  - Ran `bin/storage upgrade -f`, got a clean upgrade.
  - Created and submitted some inline comments; nothing exploded.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13513

Differential Revision: https://secure.phabricator.com/D21184
2020-05-04 13:09:55 -07:00
epriestley
8aac55cc57 Make "Highlight As..." sticky across reloads in Diffusion and Differential
Summary:
Ref T13455. Add container-level storage for persistent view state, and persist "Highlight As..." inside it.

The storage generates a "PhabricatorChangesetViewState" configuration object as an output.

When preferences are expressed on a diff and that diff is later attached to a revision, we attempt to copy the preferences.

The internal storage tracks per-changeset settings, but currently always uses "last update wins" to apply the settings in the UI.

Test Plan:
  - Viewed revisions, changed highlighting, reloaded. Saw highlighting stick in revision view and standalone view.
  - Viewed commits, changed highlighting, reloaded. Saw highlighting stick.
  - Created a diff, changed highlighting, turned it into a revision, saw highlighting persist.

Subscribers: jmeador, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13455

Differential Revision: https://secure.phabricator.com/D21137
2020-04-19 08:58:39 -07:00
epriestley
3e676bce9e No-op an ancient Paste edge migration which no longer functions after a database rename
Summary:
Fixes T13510. This migration currently fails because it tries to affect the "paste" database, but when it runs this database will be named "pastebin".

Since the cost of fixing it in place or moving it past the rename migration both seem relatively high (and the cost of throwing it away is plausibly zero) just throw it for now.

Test Plan: Looked at file, saw no more code that can execute.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13510

Differential Revision: https://secure.phabricator.com/D21132
2020-04-17 05:38:31 -07:00
Austin McKinley
ef1340bd32 Add Ferret support to Paste
Summary:
Ref PHI1292. Enable fulltext searchs in paste. Maybe this should only index a snippet instead of the entire content?

Also updates table names in `PhabricatorPasteQuery`.

Test Plan: Created some pastes, indexed them, searched for them.

Reviewers: amckinley

Subscribers: codeblock, Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20650
2020-04-16 14:10:23 -07:00
epriestley
d86506052c Update a very old Phriction migration which incorrectly uses "save()"
Summary:
See <https://discourse.phabricator-community.org/t/storage-upgrade-error/3748>.

It is broadly unsafe for migrations to use "save()". If the object gains new fields later, the query will include "SET newField = X", which will fail against the old schema which is in the process of being upgraded.

Instead, migrations must issue raw SQL against the schema as it is expected to exist at the time the migration executes.

Migrations have followed this rule for a long time, but this ~6 year old migration was overlooked. Update it to issue a raw query to perform the policy update.

Test Plan: This is somewhat flimsy since rebuilding a genuine reproduction case is messy, but used "bin/storage --apply ..." to at least get the new query to execute against modern Phabricator without issues.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D21124
2020-04-15 08:06:30 -07:00
epriestley
84b5ad09e6 Remove all readers and all nontrivial writers for "accountType" and "accountDomain" on "ExternalAccount"
Summary:
Depends on D21018. Ref T13493. Ref T6703. The "ExternalAccount" table has a unique key on `<accountType, accountDomain, accountID>` but this no longer matches our model of reality and changes in this sequence end writes to `accountID`.

Remove this key.

Then, remove all readers of `accountType` and `accountDomain` (and all nontrivial writers) because none of these callsites are well-aligned with plans in T6703.

This change has no user-facing impact today: all the rules about linking/unlinking/etc remain unchanged, because other rules currently prevent creation of more than one provider with a given "accountType".

Test Plan:
- Linked an OAuth1 account (JIRA).
- Linked an OAuth2 account (Asana).
- Used `bin/auth refresh` to cycle OAuth tokens.
- Grepped for affected symbols.
- Published an Asana update.
- Published a JIRA link.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13493, T6703

Differential Revision: https://secure.phabricator.com/D21019
2020-02-22 17:48:46 -08:00
epriestley
faf9f06e0a Migrate all "accountID" values to "ExternalAccountIdentifier" objects
Summary: Depends on D21016. Ref T13493. This copies existing external account "accountID" values into the "ExternalAccountIdentifier" table, preparing for an authority switch.

Test Plan: Ran migration several times, looked at the data that came out of it, saw sensible results. Logged out / in with external accounts.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13493

Differential Revision: https://secure.phabricator.com/D21017
2020-02-22 17:47:37 -08:00
epriestley
70845a2d13 Add an "ExternalAccountIdentifier" table
Summary:
Depends on D21010. Ref T13493. External accounts may have multiple different unique identifiers, most often when v1 of the API makes a questionable choice (and provies a mutable, non-unique, or PII identifier) and v2 of the API uses an immutable, unique, random identifier.

Allow Phabricator to store multiple identifiers per external account.

Test Plan: Storage only, see followup changes.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13493

Differential Revision: https://secure.phabricator.com/D21011
2020-02-22 17:44:13 -08:00
epriestley
89dcf9792a Give "PhabricatorUserEmail" a PHID
Summary:
Ref T13444. To interact meaningfully with "DestructionEngine", objects need a PHID. The "UserEmail" object currently does not have one (or a real "Query").

Provide basic PHID support so "DestructionEngine" can interact with the object more powerfully.

Test Plan:
  - Ran migrations, checked data in database, saw sensible PHIDs assigned.
  - Added a new email address to my account, saw it get a PHID.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20913
2019-11-19 09:41:29 -08:00
epriestley
a2b2c391a1 Distinguish between "Assigned" and "Effective" identity PHIDs more clearly and consistently
Summary:
Ref T13444. You can currently explicitly unassign an identity (useful if the matching algorithm is misfiring). However, this populates the "currentEffectiveUserPHID" with the "unassigned()" token, which mostly makes things more difficult.

When an identity is explicitly unassigned, convert that into an explicit `null` in the effective user PHID.

Then, realign "assigned" / "effective" language a bit. Previously, `withAssigneePHIDs(...)` actualy queried effective users, which was misleading. Finally, bulk up the list view a little bit to make testing slightly easier.

Test Plan:
  - Unassigned an identity, ran migration, saw `currentEffectiveUserPHID` become `NULL` for the identity.
  - Unassigned a fresh identity, saw NULL.
  - Queried for various identities under the modified constraints.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20908
2019-11-19 09:37:44 -08:00
epriestley
df0f5c6cee Make repository identity email address association case-insensitive
Summary:
Ref T13444. Currently, identities for a particular email address are queried with "LIKE" against a binary column, which makes the query case-sensitive.

  - Extract the email address into a separate "sort255" column.
  - Add a key for it.
  - Make the query a standard "IN (%Ls)" query.
  - Deal with weird cases where an email address is 10000 bytes long or full of binary junk.

Test Plan:
  - Ran migration, inspected database for general sanity.
  - Ran query script in T13444, saw it return the same hits for "git@" and "GIT@".

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20907
2019-11-19 09:37:26 -08:00
epriestley
5d8457a07e In the repository URI index, store Phabricator's own URIs as tokens
Summary:
Fixes T13435. If you move Phabricator or copy data from one environment to another, the repository URI index currently still references the old URI, since it writes the URI as a plain string. This may make "arc which" and similar workflows have difficulty identifying repositories.

Instead, store the "phabricator.base-uri" domain and the "diffusion.ssh-host" domain as tokens, so lookups continue to work correctly even after these values change.

Test Plan:
  - Added unit tests to cover the normalization.
  - Ran migration, ran daemons, inspected `repository_uriindex` table, saw a mixture of sensible tokens (for local domains) and static domains (like "github.com").
  - Ran this thing:

```
$ echo '{"remoteURIs": ["ssh://git@local.phacility.com/diffusion/P"]}' | ./bin/conduit call --method repository.query --trace --input -
Reading input from stdin...
>>> [2] (+0) <conduit> repository.query()
>>> [3] (+3) <connect> local_repository
<<< [3] (+3) <connect> 555 us
>>> [4] (+5) <query> SELECT `r`.* FROM `repository` `r` LEFT JOIN `local_repository`.`repository_uriindex` uri ON r.phid = uri.repositoryPHID WHERE (uri.repositoryURI IN ('<base-uri>/diffusion/P')) GROUP BY `r`.phid ORDER BY `r`.`id` DESC LIMIT 101
<<< [4] (+5) <query> 596 us
<<< [2] (+6) <conduit> 6,108 us
{
  "result": [
    {
      "id": "1",
      "name": "Phabricator",
      "phid": "PHID-REPO-2psrynlauicce7d3q7g2",
      "callsign": "P",
      "monogram": "rP",
      "vcs": "git",
      "uri": "http://local.phacility.com/source/phabricator/",
      "remoteURI": "https://github.com/phacility/phabricator.git",
      "description": "asdf",
      "isActive": true,
      "isHosted": false,
      "isImporting": false,
      "encoding": "UTF-8",
      "staging": {
        "supported": true,
        "prefix": "phabricator",
        "uri": null
      }
    }
  ]
}
```

Note the `WHERE` clause in the query normalizes the URI into "<base-uri>", and the lookup succeeds.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13435

Differential Revision: https://secure.phabricator.com/D20872
2019-10-28 14:44:06 -07:00
epriestley
06edcf2709 Fix an issue where ancestors of permanent refs might not be published during import or if a branch is later made permanent
Summary:
Fixes T13284. See that task for substantial discussion. There are currently two cases where we'll skip over commits which we should publish:

  - if a branch is not permanent, then later made permanent; or
  - in some cases, the first time we examine branches in a repository.

In both cases, this error is one-shot and things work correctly going forward. The root cause is conflation between the states "this ref currently permanent" and "this ref was permanent the last time we updated refs".

Separate these pieces of state and cover all these cases. Also introduce a "--rebuild" flag to fix the state of bad commits.

Test Plan:
See T13284 for the three major cases:

  - initial import;
  - push changes to a nonpermanent branch, update, then make it permanent;
  - push chanegs to a nonpermanent branch, update, push more changes, then make it permanent.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13284

Differential Revision: https://secure.phabricator.com/D20829
2019-09-25 08:54:50 -07:00
epriestley
d965d9a669 Index Herald fields, not just actions, when identifying objects related to a particular Herald rule
Summary:
Fixes T13408. Currently, when a package (or other object) appears in a field (rather than an action), it is not indexed.

Instead: index fields too, not just actions.

Test Plan:
  - Wrote a rule like "[ Affected packages include ] ...".
  - Updated the search index.
  - Saw rule appear on "Affected By Herald Rules" on the package detail page.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13408

Differential Revision: https://secure.phabricator.com/D20795
2019-09-09 12:50:43 -07:00
epriestley
9bcd683c08 Update Phortune Merchant UI to bring it in line with Account UI
Summary:
Depends on D20732. Ref T13366. This generally makes the "Merchant" UI look and work like the "Payment Account" UI.

This is mostly simpler since the permissions have largely been sorted out already and there's less going on here and less weirdness around view/edit policies.

Test Plan: Browsed all Merchant functions as a merchant member and non-member.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20733
2019-08-22 21:12:33 -07:00
epriestley
a542024b63 Update Phortune subscriptions for modern infrastructure
Summary:
Depends on D20720. Ref T13366.

  - Use modern policies and policy interfaces.
  - Use new merchant authority cache.
  - Add (some) transactions.
  - Move MFA from pre-upgrade-gate to post-one-shot-check.
  - Simplify the autopay workflow.
  - Use the "reloading arrows" icon for subscriptions more consistently.

Test Plan: As a merchant-authority and account-authority, viewed, edited, and changed autopay for subscriptions.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20721
2019-08-22 21:07:17 -07:00
epriestley
201634848e Make Phortune payment methods transaction-oriented and always support "Add Payment Method"
Summary:
Depends on D20718. Ref T13366. Ref T13367.

  - Phortune payment methods currently do not use transactions; update them.
  - Give them a proper view page with a transaction log.
  - Add an "Add Payment Method" button which always works.
  - Show which subscriptions a payment method is associated with.
  - Get rid of the "Active" status indicator since we now treat "disabled" as "removed", to align with user expectation/intent.
  - Swap out of some of the super weird div-form-button UI into the new "big, clickable" UI for choice dialogs among a small number of options on a single dimension.

Test Plan:
  - As a mechant-authority and account-authority, created payment methods from carts, subscriptions, and accounts. Edited and viewed payment methods.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13367, T13366

Differential Revision: https://secure.phabricator.com/D20719
2019-08-22 21:04:04 -07:00
epriestley
277bce5638 In Phortune, write relationships between payment accounts and merchants they interact with
Summary:
Depends on D20713. Ref T13366. When a payment account establishes a relationship with a merchant by creating a cart or subscription, create an edge to give the merchant access to view the payment account.

Also, migrate all existing subscriptions and carts to write these edges.

This aims at straightening out Phortune permissions, which are currently a bit wonky on a couple of dimensions. See T13366 for detailed discussion.

Test Plan:
  - Created and edited carts/subscriptions, saw edges write.
  - Ran migrations, saw edges write.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13366

Differential Revision: https://secure.phabricator.com/D20715
2019-08-22 21:01:04 -07:00