1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-22 14:52:41 +01:00
Commit graph

1175 commits

Author SHA1 Message Date
Dylan F
2b7c0ec92f Destroy file attachments when file is deleted, or object is deleted
Summary:
Adds file attachment deletion logics:

- PhabricatorFile: delete the attachment if file is deleted
- destruction engine extension: delete attachment if object is deleted
- SQL patch: delete existing leftover attachments from deleted files

To apply the cleanup, as usual, run:

    ./bin/storage upgrade

This cleanup may take some time, proportionally to the size of these tables:

    phabricator_file.file
    phabricator_file.file_attachment

Just as an indication: the storage upgrade, in a Phorge with `file` count 1.3M rows and `file_attachment` consisting in 9K rows, it may delete 170K rows in less than 1 second on average hardware.

Closes T15110

Test Plan:
Apply the patch, run `./bin/storage/upgrade`:

- no "Unknown Object" in any "Referenced Files" curtain of any object.

Have phd daemon running.

Upload file, attach the file to a task, delete the file from the web interface:

- no "Unknown Object" in "Referenced Files" curtain of that task.
- the query `SELECT * FROM file_attachment WHERE filePHID = '<file phid>'` returns empty result

Upload file, attach the file to a task, delete the task from the `./bin/remove destroy` workflow:

- the query `SELECT * FROM file_attachment WHERE objectPHID = '<task phid>'` returns empty result

Reviewers: O1 Blessed Committers, avivey, valerio.bozzolan

Reviewed By: O1 Blessed Committers, avivey, valerio.bozzolan

Subscribers: Ekubischta, speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno

Tags: #files

Maniphest Tasks: T15110

Differential Revision: https://we.phorge.it/D25051
2024-08-10 22:16:53 +02:00
Matthew Bowker
4d4712b58d Remove Chatlog entirely
Summary:
This commit removes ChatLog entirely.  All of the application files are removed, and the migrations used are stubbed out. I stubbed the migrations as that allows for existing installs to make no changes, but new installs will not create the database.

Fixes T15126

Test Plan: Loaded up http://phorge.local/chatlog and confirmed the 404.  Loaded up http://phorge.local/applications/view/PhabricatorChatLogApplication and confirmed the 404.  Created a new database prefix and ran `bin/storage upgrade` against it, confirmed that the chatlog database was not created.  Restored another prefix (an old one) and ran `bin/storage upgrade` and confirmed database was not deleted.

Reviewers: O1 Blessed Committers, avivey

Reviewed By: O1 Blessed Committers, avivey

Subscribers: avivey, tobiaswiese, valerio.bozzolan, Cigaryno

Maniphest Tasks: T15126

Differential Revision: https://we.phorge.it/D25480
2023-11-27 11:38:37 -07:00
Benjamin Kausch
dc10a7e69e Implement ferret engine in typeahead datasource query for repos
Summary:
This broadens the typeahead datasource search for repos. Before this patch a repository named "Alligator Simulator" would not be found with the search string "simu...". This is patched with the ferret engine search and indexing features.

See T15583

Test Plan: Create repositories with titles with 2 or more words. Search for these repos with the global typeahead search. The search term should begin with the second/third/n-th word of the repo title.

Reviewers: O1 Blessed Committers, avivey

Reviewed By: O1 Blessed Committers, avivey

Subscribers: avivey, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno

Maniphest Tasks: T15583

Differential Revision: https://we.phorge.it/D25430
2023-09-04 09:48:41 +02:00
Aviv Eyal
c3f0c0b0f7 Merge Phacility/master into phorge 2022-07-25 11:50:26 -07:00
epriestley
809ae81752 Update Slowvote poll status to use sensible string constants
Summary: Ref T13682. This prepares for modernizing Slowvote and exposing a more usable API.

Test Plan: Ran migrations, opened and closed polls.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682

Differential Revision: https://secure.phabricator.com/D21848
2022-05-27 10:15:01 -07:00
epriestley
03d3d1889d Update Slowvote voting methods to use sensible string constants
Summary: Ref T13682. Use API-friendly string constants instead of opaque integers in Slowvote voting methods.

Test Plan: Created, edited, and voted in polls with various voting methods. Examined database after migrations.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682

Differential Revision: https://secure.phabricator.com/D21846
2022-05-26 10:27:01 -07:00
epriestley
9f075839a2 Update Slowvote to use sensible string constants for response visibility
Summary: Ref T13682. Migrate and update Slowvote to use API-friendly string constants for response visibility, not opaque integers.

Test Plan: Created and edited slowvotes, changing response visibility.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682

Differential Revision: https://secure.phabricator.com/D21844
2022-05-26 09:58:48 -07:00
epriestley
b1533e5468 Migrate "SlowvotePoll" to modern mailkey storage
Summary:
Ref T13682. Ref T13065. Migrate and update "SlowvotePoll" to use more modern mailkey storage (in the mail application, rather than onboard).

Also make other minor modernization changes.

Test Plan: Created a new poll. Ran migration, verified mail key copied properly in database.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13682, T13065

Differential Revision: https://secure.phabricator.com/D21842
2022-05-26 09:29:05 -07:00
epriestley
c25a8fabfc Remove all "ObjectHasFile" edge reads and writes
Summary: Ref T13603. Migrate all code which interacts with the "ObjectHasFile" edge to use the "Attachments" table instead.

Test Plan:
  - Edited a paste's view policy, confirmed associated file secret was scrambled.
  - Verified I could still view paste content as a user who could not naturally view the underlying file.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13603

Differential Revision: https://secure.phabricator.com/D21819
2022-05-19 13:21:04 -07:00
epriestley
0329df0fa8 Copy file attachment edges to new storage
Summary: Ref T13603. Copy all existing file attachment data into the new storage table.

Test Plan: Ran migration, checked attachment table for general reasonableness.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13603

Differential Revision: https://secure.phabricator.com/D21818
2022-05-19 13:21:03 -07:00
epriestley
cfa42c5e65 Add database storage for a dedicated file attachment table
Summary: Ref T13603. Prepare to move this relationship out of edge storage into dedicated storage so it is easier to formalize better in the UI.

Test Plan: Ran `bin/storage upgrade`.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13603

Differential Revision: https://secure.phabricator.com/D21813
2022-05-19 13:21:02 -07:00
epriestley
c415622923 Remove the "Phragment" application
Summary:
Ref T5479. Ref T13658. This was a contributed application from the early days of Phabricator which never had customers or users in the wild. The contributor moved on from the project many years ago.

Any capabilities in this general role would look different today. It also has one or two product name literal strings, so this is as good a time as any to remove it.

This change does not remove storage; I'll issue upgrade guidance and do that separately after some time.

Test Plan: Grepped for "phragment", got no relevant hits.

Subscribers: hach-que, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13658, T5479

Differential Revision: https://secure.phabricator.com/D21793
2022-04-25 16:46:27 -07:00
epriestley
84b0c5a669 Remove the "Releeph" application
Summary:
Ref T9530. Ref T13658. The "Releeph" application was never useful outside of Facebook and any application providing release support would not resemble it much.

It has some product name literal strings, so now is as good a time as any to get rid of it.

This application never left prototype and I'm not aware of any install in the wild that uses it (or has ever used it).

I did not destroy the database itself. I'll issue upgrade guidance and destroy the database in some future release, just in case.

Test Plan: Grepped for "releeph", found no relevant/removable hits.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13658, T9530

Differential Revision: https://secure.phabricator.com/D21792
2022-04-25 16:46:27 -07:00
epriestley
039cbec155 Remove product literal strings in "pht()", part 11
Summary: Ref T13658.

Test Plan: Static checks only, none of these looked easily reachable.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13658

Differential Revision: https://secure.phabricator.com/D21777
2022-04-25 16:46:24 -07:00
epriestley
3e82ab5adb Remove product literal strings in "pht()", part 1
Summary: Ref T13658. I used the linter in D21763 to identify these and `split` them into arbitrary groups of 10 files.

Test Plan:
This test plan is non-exhaustive, because some of these strings are difficult to reach.

  - Looked at "Create Service" in Almanac.
  - Used "bin/auth" to go through a one-time auth workflow (not all related strings can be hit on a single workflow).
  - Started the "Generate Keypair" worfklow in "SSH Public Keys".

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13658

Differential Revision: https://secure.phabricator.com/D21765
2022-04-25 12:22:25 -07:00
epriestley
aae23f0204 Give Phame blog posts configurable interact policies, with a default policy of "Same as Blog"
Summary: Ref T13661. This allows posts to have comments disabled (or restricted) on a per-post basis, and makes them inherit the containing blog policy by default.

Test Plan: Locked a post by editing its policy explicitly; locked a post by editing the containing blog policy.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13661

Differential Revision: https://secure.phabricator.com/D21754
2022-04-01 12:52:58 -07:00
epriestley
c25595417f Give Phame blogs mutable interact policies
Summary:
Ref T13661.

I'm fairly sure these policies don't actually do anything (you can't "interact" with a blog) but the primarily support a Phame Post object policy of "Same as Parent Blog", which is the "natural" interact policy for a post.

Most of this is infrastructure support for mutable interact policies: today, only Maniphest has interact mutability and only via indirect effects (locking tasks), not through a directly mutable "Can Interact" policy.

Test Plan:
Ran storage upgrade, edited interact policy of a blog, saw appropriate persistence and transactions.

Created and edited a task to make sure there's no weird fallout from increasing what can be done with interact policies.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13661

Differential Revision: https://secure.phabricator.com/D21751
2022-04-01 12:52:56 -07:00
epriestley
28b12fb3cd Make Phame blog policies non-nullable
Summary: Ref T6203. Ref T13661. These policies are incorrectly nullable, although it's likely that no pathway exists in the application to write NULL into them. Fix the schema.

Test Plan: Ran `bin/storage upgrade`.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13661, T6203

Differential Revision: https://secure.phabricator.com/D21748
2022-04-01 12:52:55 -07:00
James Brown
a4948ec800 Add conduit endpoints for querying legalpad
Summary:
This diff adds conduit methods for searching for legalpad documents and signatures. This is very helpful for auditing who's actually signed a document. It also fixes the "contributorPHIDs" constraint in the existing search engine.

In order to expose legalpad signatures through Conduit, this adds a `phid` column to the `legalpad_documentsignature` table. It includes a migration (in the style of many previous phid-adding migrations) to actually populate the column.

Test Plan: We run this on my company's internal fork and it seems to work okay. I don't think any other conduit methods anywhere have tests (???), but if you can point me at one I'm glad to write a unit test!

Reviewers: O1 Blessed Committers, speck

Reviewed By: O1 Blessed Committers, speck

Subscribers: 20after4, speck, tobiaswiese

Differential Revision: https://we.phorge.it/D25018
2021-09-04 13:53:13 -04:00
epriestley
8bbee92139 Modularize individual Harbormaster build messages
Summary:
Ref T13072. Further modularize build messages by applying each one in a separate transaction type.

This makes it easier to add new types of messages (although I have no particular plans to do this, offhand) and reduces the amount of switch-boilerplate.

This will probably also simplify validating "harbormaster.sendmessage".

Test Plan:
  - Applied all commands.
  - Ran migration, saw transactions render properly

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13072

Differential Revision: https://secure.phabricator.com/D21690
2021-07-21 14:16:56 -07:00
epriestley
b48d4fabaf Merge the "HarbormasterBuildCommand" table into "HarbormasterBuildMessage"
Summary:
Ref T13072. These two similar tables don't make sense to keep separate. Instead, make Build a valid receiver for BuildMessage objects.

These tables are practically the same, so this is straightforward: just copy the rows in and then drop the old table.

(This table was trivial and ephemeral anyway, so I'm not bothering to do the usual "keep it around for a couple years just in case".)

Test Plan:
  - Populated BuildCommand table, ran migration, saw Builds end up in the proper transitional state (e.g., pausing, aborting, restarting) with appropriate queued messages.
  - Queued new messages by clicking UI buttons.
  - Ran BuildWorkers, saw them process messages and mark them as consumed.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13072

Differential Revision: https://secure.phabricator.com/D21684
2021-07-21 14:16:53 -07:00
epriestley
5521f76fe4 Remove some "Phacility" and "epriestley" references
Summary:
Ref T13658. This just scrubs some of the simple references from the codebase.

Most of what's left is in documentation which won't be relevant for a fork and/or which I need to separately revise (or more-or-less delete) at some point anyway.

I removed the "install RHEL" and "install Ubuntu" scripts outright since I don't have any reasonable way to test them and don't plan to maintain them.

Test Plan: Grepped for "phacility", "epriestley"; ran unit tests.

Reviewers: cspeckmim

Reviewed By: cspeckmim

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13658

Differential Revision: https://secure.phabricator.com/D21678
2021-07-08 10:46:17 -07:00
epriestley
a641ec82a3 Add an "Authority" control to Packages to support "Watcher" packages
Summary: See T13657. An install has "watcher" packages which should not allow owners to "Force Accept" other packages.

Test Plan:
  - Created package A, which I own, on "/", with "Weak" authority.
  - Created package B, which I do not own, on "/src".
  - Created a revision which touches "/src" and added package B as a reviewer.
  - Attempted to accept the revision...
    - Before patch: permitted to "Force Accept" for package B.
    - After patch: not allowed to "Force Accept" for package B.
  - Verified that setting package "A" back to "Strong" authority allows a force-accept for package B.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D21674
2021-06-25 13:48:46 -07:00
epriestley
3267859aee Modernize "mailKey" on Fund initiatives
Summary: Ref T13065. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean upgrade, saw migrated values in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065

Differential Revision: https://secure.phabricator.com/D21634
2021-03-16 15:51:50 -07:00
epriestley
eeb009b4fa Modernize "mailKey" for Calendar Event
Summary: Ref T13065. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report, saw migrated data in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065

Differential Revision: https://secure.phabricator.com/D21633
2021-03-16 15:51:50 -07:00
epriestley
86669ab54f Modernize "mailKey" for Almanac Networks
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report, saw keys migrate in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21632
2021-03-16 15:51:49 -07:00
epriestley
8226b3c880 Modernize "mailKey" on Almanac Namespaces
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report, saw mail keys migrated to mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21631
2021-03-16 15:51:49 -07:00
epriestley
7c0e33c34d Modernize "mailKey" for Almanac Bindings
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.

Test Plan: Ran "bin/storage upgrade", got a clean report and saw binding mail keys in the mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21630
2021-03-16 15:51:48 -07:00
epriestley
fadb2bd52b Modernize "mailKey" on AlamnacService
Summary: Ref T13641. Ref T13065. Migrate and drop the onboard "mailKey" column for Almanac Services.

Test Plan: Ran storage migration, got clean report, saw migrated value in mail properties table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641, T13065

Differential Revision: https://secure.phabricator.com/D21629
2021-03-16 15:51:48 -07:00
epriestley
5d64fb1815 Add a "status" property to Almanac devices
Summary:
Ref T13641. Add a "status" property with most of the relevant support code.

This currently has no impact on use of the device or bindings by Diffusion or Drydock: they ignore the status of devices bound to services.

Test Plan:
  - Created a new device.
  - Changed the status of a device via web and API.
  - Queried for devices via API.
  - Searched for active and disabled devices.
  - Viewed UI in list view, detail view.
  - Used typeahead to add a new binding to an interface on a disabled device, got disabled hint in typeahead UI.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13641

Differential Revision: https://secure.phabricator.com/D21627
2021-03-16 15:51:47 -07:00
epriestley
c3e6db6f0b Migrate Almanac Device "mailKey" to modern storage
Summary: Ref T13065. See similar changes attached to that task.

Test Plan: Ran migration, got a clean database state, saw mail keys populate in mail property table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13065, T13641

Differential Revision: https://secure.phabricator.com/D21625
2021-03-16 15:51:46 -07:00
epriestley
01ea84029d Update table schema for "AffectedPath" table
Summary:
Ref T13639. Make schema changes:

  - Make repositoryID nullable, for revisions with no repository.
  - Remove "epoch", which has no readers and no clear use.
  - Change the ordering of the key, since "pathID" has more unique values and no queries ever issue without it.

Test Plan:
  - Ran `bin/storage upgrade`, got a clean schema.
  - Reindexed all revisions with an external script.
  - Reviewed index via debug UI, saw appropriate index for non-repositoy revisions.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13639

Differential Revision: https://secure.phabricator.com/D21617
2021-03-15 16:16:10 -07:00
epriestley
32da29b965 Provide more help around GRANT errors, particularly for missing TEMPORARY TABLE permission
Summary:
Fixes T13622. Figuring out what permissions we have seems difficult, so address this a bit more narrowly:

  - Make the "access denied" error message a bit more helpful.
  - Tailor error handling for the "CREATE TEMPORARY TABLE" statement.

Test Plan:
  - Created a new user, granted them "SELECT ON *.*" but not "CREATE TEMPORARY TABLE", ran `bin/storage upgrade --force --apply phabricator:20210215.changeset.02.phid-populate.php`.
  - Before: fairly opaque error.
  - After: fairly useful error.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13622

Differential Revision: https://secure.phabricator.com/D21608
2021-03-11 14:55:21 -08:00
epriestley
2636d84d0c Remove very old Audit status constants and AuditRequest data
Summary:
Ref T13631. See that task for discussion.

  - "NONE": Probably never used?
  - "CC": Obsoleted by subscribers.
  - "AUDIT_NOT_REQUIRED": For Owners packages, obsoleted by edges.
  - "CLOSED": For "Close Audit", obsoleted by "Request Verification".

Test Plan:
  - Grepped for constants, browsed Diffusion.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13631

Differential Revision: https://secure.phabricator.com/D21598
2021-03-10 09:21:54 -08:00
epriestley
b3976acc40 Improve performance of "phabricator:20210215.changeset.02.phid-populate.php"
Summary: Ref T13613. Improve the performance of this migration by using a temporary table and an "UPDATE x JOIN y ..." pattern.

Test Plan:
  - Ran on `secure`, got exit after a few seconds since the migration is idempotent and changesets already had PHIDs.
  - Ran on `secure` with the `continue;` commented out, got valid new PHIDs in 53s (from 153s).
  - Tried a larger page size (16K), didn't see any improvement.
  - From "--trace", client PHID generation seems to be the limiting factor.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13613

Differential Revision: https://secure.phabricator.com/D21570
2021-02-19 07:53:14 -08:00
epriestley
6703fec3e2 When documents are indexed, record the indexer version (versus the object version) and index epoch
Summary:
Ref T13587. D21495 has significant changes to the ngram indexer, which might possibly contain bugs.

Make it easier to reindex a subset of documents (based on the date when the index was built, and/or the software version which generated the index).

This is in addition to the existing versioning, which is focused on object versions.

Test Plan: Ran `bin/search index` with various old and new arguments. Spot-checked the `IndexVersion` table.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13587

Differential Revision: https://secure.phabricator.com/D21560
2021-02-16 16:09:31 -08:00
epriestley
ec5476a01f Add a PHID to Changesets
Summary:
Ref T13605. Changesets currently have no PHID, which limits their ability to use standard API infrastructure.

Give them a PHID, since there's no reason they don't have one other than their age.

Test Plan:
  - Ran migrations, saw PHIDs populated.
  - Created new changesets, saw PHIDs.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13605

Differential Revision: https://secure.phabricator.com/D21557
2021-02-15 11:11:12 -08:00
epriestley
201e0d2943 Add support for a "containerPHID" in the worker queue
Summary:
Ref T13591. Worker queue tasks which affect commits currently (mostly) store the commit as an "objectPHID", but do not directly reference the repository the commit belongs to.

This can make certain operations (like "change the priority of all tasks affecting repository Y") more difficult than it needs to be.

Support a "containerPHID", similar to the field of the same name on builds, that can store a parent object like a repository and better support operations against subsets of tasks.

See also D11044 for the genesis of "objectPHID".

This depends on the introduction of storage patch phases (in D21529) so that earlier migrations which queue worker tasks don't try to insert this column before it actually exists.

Test Plan:
  - Ran `bin/storage upgrade`.
  - No callers yet, see further changes for usage.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13591

Differential Revision: https://secure.phabricator.com/D21531
2021-02-02 13:40:09 -08:00
epriestley
32942f6232 Introduce storage patch "phases" to allow index-rebuilding patches to execute after worker queue schema changes
Summary:
Ref T13591. Some storage patches queue worker tasks, currently always to rebuild search indexes.

These patches can not execute in creation order if a later patch modifies the worker task table, since they'll try to perform a modern INSERT against an out-of-date table schema. Such a modification is desirable in the context of T13591, but making it causes these patches to fail.

Patches have an existing "after" mechanism which allows them to have explicit dependencies. This mechanism could be used to resolve this issue, but all patches with a dependency like this would need to be updated every time the queue table changes.

Instead, introduce "phases" to provide broader ordering rules. There are now two phases: "default" and "worker". Patches in the "worker" phase execute after patches in the "default" phase.

Phases may eventually be further separated, but

Test Plan:
  - Ran `bin/storage status`, saw patches annotated with phases.
  - Will apply `containerPHID` changes on top of this.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13591

Differential Revision: https://secure.phabricator.com/D21529
2021-02-02 13:40:09 -08:00
epriestley
6e1b5da112 Fix additional "xprintf()"-class static parameter lint errors
Summary: Ref T13577. After the fix in D21453, lint identifies additional static errors in Phabricator; fix them.

Test Plan: Ran `arc lint`; these messages are essentially all very obscure.

Subscribers: hach-que, yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13577

Differential Revision: https://secure.phabricator.com/D21457
2020-09-08 11:45:48 -07:00
epriestley
d3d41324be Drop old "differential_commit" table
Summary: Ref T13276. Ref T13513. All readers and writers were removed more than a year ago; clean up the last remnants of this table.

Test Plan: Grepped for table references, found none.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13513, T13276

Differential Revision: https://secure.phabricator.com/D21281
2020-05-20 14:30:39 -07:00
epriestley
6d0dbeb77f Use the changeset parse cache to cache suggestion changesets
Summary:
Ref T13513. Syntax highlighting is potentially expensive, and the changeset rendering pipeline can cache it. However, the cache is currently keyed ONLY by Differential changeset ID.

Destroy the existing cache and rebuild it with a more standard cache key so it can be used in a more ad-hoc way by inline suggestion snippets.

Test Plan: Used Darkconsole, saw cache hits and no more inline syntax highlighting for changesets with many inlines.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13513

Differential Revision: https://secure.phabricator.com/D21280
2020-05-20 14:29:27 -07:00
epriestley
5ff0ae7d48 Add generic "attributes" storage to inline comment tables
Summary: Ref T13513. This plans for "currently editing", character range comments, code suggestions, document engine tracking. And absolutely nothing else.

Test Plan:
  - Ran `bin/storage upgrade -f`, got a clean upgrade.
  - Created and submitted some inline comments; nothing exploded.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13513

Differential Revision: https://secure.phabricator.com/D21184
2020-05-04 13:09:55 -07:00
epriestley
8aac55cc57 Make "Highlight As..." sticky across reloads in Diffusion and Differential
Summary:
Ref T13455. Add container-level storage for persistent view state, and persist "Highlight As..." inside it.

The storage generates a "PhabricatorChangesetViewState" configuration object as an output.

When preferences are expressed on a diff and that diff is later attached to a revision, we attempt to copy the preferences.

The internal storage tracks per-changeset settings, but currently always uses "last update wins" to apply the settings in the UI.

Test Plan:
  - Viewed revisions, changed highlighting, reloaded. Saw highlighting stick in revision view and standalone view.
  - Viewed commits, changed highlighting, reloaded. Saw highlighting stick.
  - Created a diff, changed highlighting, turned it into a revision, saw highlighting persist.

Subscribers: jmeador, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13455

Differential Revision: https://secure.phabricator.com/D21137
2020-04-19 08:58:39 -07:00
epriestley
3e676bce9e No-op an ancient Paste edge migration which no longer functions after a database rename
Summary:
Fixes T13510. This migration currently fails because it tries to affect the "paste" database, but when it runs this database will be named "pastebin".

Since the cost of fixing it in place or moving it past the rename migration both seem relatively high (and the cost of throwing it away is plausibly zero) just throw it for now.

Test Plan: Looked at file, saw no more code that can execute.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13510

Differential Revision: https://secure.phabricator.com/D21132
2020-04-17 05:38:31 -07:00
Austin McKinley
ef1340bd32 Add Ferret support to Paste
Summary:
Ref PHI1292. Enable fulltext searchs in paste. Maybe this should only index a snippet instead of the entire content?

Also updates table names in `PhabricatorPasteQuery`.

Test Plan: Created some pastes, indexed them, searched for them.

Reviewers: amckinley

Subscribers: codeblock, Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D20650
2020-04-16 14:10:23 -07:00
epriestley
d86506052c Update a very old Phriction migration which incorrectly uses "save()"
Summary:
See <https://discourse.phabricator-community.org/t/storage-upgrade-error/3748>.

It is broadly unsafe for migrations to use "save()". If the object gains new fields later, the query will include "SET newField = X", which will fail against the old schema which is in the process of being upgraded.

Instead, migrations must issue raw SQL against the schema as it is expected to exist at the time the migration executes.

Migrations have followed this rule for a long time, but this ~6 year old migration was overlooked. Update it to issue a raw query to perform the policy update.

Test Plan: This is somewhat flimsy since rebuilding a genuine reproduction case is messy, but used "bin/storage --apply ..." to at least get the new query to execute against modern Phabricator without issues.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D21124
2020-04-15 08:06:30 -07:00
epriestley
84b5ad09e6 Remove all readers and all nontrivial writers for "accountType" and "accountDomain" on "ExternalAccount"
Summary:
Depends on D21018. Ref T13493. Ref T6703. The "ExternalAccount" table has a unique key on `<accountType, accountDomain, accountID>` but this no longer matches our model of reality and changes in this sequence end writes to `accountID`.

Remove this key.

Then, remove all readers of `accountType` and `accountDomain` (and all nontrivial writers) because none of these callsites are well-aligned with plans in T6703.

This change has no user-facing impact today: all the rules about linking/unlinking/etc remain unchanged, because other rules currently prevent creation of more than one provider with a given "accountType".

Test Plan:
- Linked an OAuth1 account (JIRA).
- Linked an OAuth2 account (Asana).
- Used `bin/auth refresh` to cycle OAuth tokens.
- Grepped for affected symbols.
- Published an Asana update.
- Published a JIRA link.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13493, T6703

Differential Revision: https://secure.phabricator.com/D21019
2020-02-22 17:48:46 -08:00
epriestley
faf9f06e0a Migrate all "accountID" values to "ExternalAccountIdentifier" objects
Summary: Depends on D21016. Ref T13493. This copies existing external account "accountID" values into the "ExternalAccountIdentifier" table, preparing for an authority switch.

Test Plan: Ran migration several times, looked at the data that came out of it, saw sensible results. Logged out / in with external accounts.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13493

Differential Revision: https://secure.phabricator.com/D21017
2020-02-22 17:47:37 -08:00
epriestley
70845a2d13 Add an "ExternalAccountIdentifier" table
Summary:
Depends on D21010. Ref T13493. External accounts may have multiple different unique identifiers, most often when v1 of the API makes a questionable choice (and provies a mutable, non-unique, or PII identifier) and v2 of the API uses an immutable, unique, random identifier.

Allow Phabricator to store multiple identifiers per external account.

Test Plan: Storage only, see followup changes.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13493

Differential Revision: https://secure.phabricator.com/D21011
2020-02-22 17:44:13 -08:00